hive-c0re pulls closures through the nix daemon; hive-priv verifies the swarm signature before applying
This commit is contained in:
atlas
mara
parent
448b703995
commit
ecd4445680
3 changed files with 1257 additions and 584 deletions
|
|
@ -42,11 +42,11 @@ because there is no malformed spec to reject.
|
|||
|
||||
Nix-heavy — hold one of the `buildSlots` permits for the node's duration:
|
||||
|
||||
| Node | Wraps |
|
||||
| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `Prebuild` | download the swarm's closure for the agent into the store while the container keeps serving (its meta preamble is the upstream `MetaSync` node). With no closure to deploy it fails here, before the stop. Skipped when the container is already down; `Swap` downloads it instead |
|
||||
| `Swap` | drop-in rewrite + profile-swap onto the swarm's closure (requires the container stopped); the post-swap bookkeeping tail lives in the sibling `RebuildBookkeeping` node |
|
||||
| `Create` | first-spawn `nixos-container create` proper; assumes the upstream `Provision` node already registered the agent; downloads the closure the swarm built for the agent and creates the container from it (with none to deploy it fails and creates nothing), then marks the closure applied |
|
||||
| Node | Wraps |
|
||||
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `Prebuild` | pull the swarm's closure for the agent into the store through the host's nix daemon while the container keeps serving (its meta preamble is the upstream `MetaSync` node). With no closure to deploy it fails here, before the stop. Skipped when the container is already down; `Swap` downloads it instead |
|
||||
| `Swap` | drop-in rewrite + profile-swap onto the swarm's closure (requires the container stopped); the post-swap bookkeeping tail lives in the sibling `RebuildBookkeeping` node |
|
||||
| `Create` | first-spawn `nixos-container create` proper; assumes the upstream `Provision` node already registered the agent; pulls the closure the swarm built for the agent and creates the container from it (with none to deploy it fails and creates nothing), then marks the closure applied |
|
||||
|
||||
Cheap — no build slot:
|
||||
|
||||
|
|
|
|||
|
|
@ -452,7 +452,8 @@ you change `swarm.attic.domain`, delete the stale `url` leaf and restart
|
|||
replaces the stored one with the key `attic cache info` prints, so a cache
|
||||
that loses its database and gets a new key needs no step: the next run of the
|
||||
unit stores the new key, the controller copies it to the hives within five
|
||||
minutes, and `hive-priv-cache-facts` reads it within ten.
|
||||
minutes, and `hive-priv-cache-facts` reads it within ten (or at once, when a
|
||||
rebuild finds the host without facts).
|
||||
|
||||
`swarm.attic.domain` and `swarm.attic.cache` default to `attic.<swarm domain>`
|
||||
and `swarm` on every host, whether or not it runs the cache, so a builder on any
|
||||
|
|
@ -532,14 +533,17 @@ identity and writes `/run/hyperhive/attic/netrc`, `substituter` and
|
|||
token goes out in a header and in the netrc; a hive that finds another scheme
|
||||
writes nothing.
|
||||
|
||||
`hive-priv` doesn't read those files. The key that must have signed a download and
|
||||
the pull token come from `hive-priv-cache-facts`, a root unit that reads the
|
||||
hive's own `attic/public-key` and `attic/pull-token` from the store under the
|
||||
hive's own store identity every ten minutes and writes them to
|
||||
`/run/hyperhive-priv/attic/` (`0700`, root; each file `0600`). The cache's
|
||||
address is the host's `services.hyperhive.swarm.attic.domain` and `cache`,
|
||||
handed to `hive-priv` in its unit's environment. A file `hive-c0re` writes
|
||||
decides nothing `hive-priv` trusts.
|
||||
`hive-priv` and the nix daemon don't read those files. The key that must have
|
||||
signed a download and the pull token come from `hive-priv-cache-facts`, a root
|
||||
unit that reads the hive's own `attic/public-key` and `attic/pull-token` from
|
||||
the store under the hive's own store identity every ten minutes and writes them
|
||||
to `/run/hyperhive-priv/attic/` (`0700`, root; each file `0600`). It also writes
|
||||
the nix.conf fragment `/run/hyperhive-nix/cache.conf`, which the host's nix.conf
|
||||
includes: the cache's address (the host's `services.hyperhive.swarm.attic.domain`
|
||||
and `cache`) as a trusted substituter, the key as a trusted key and the netrc's
|
||||
path. It restarts `nix-daemon` only when the fragment changed after the daemon
|
||||
started, which matters for CppNix; Lix re-reads nix.conf on every connection. A
|
||||
file `hive-c0re` writes decides nothing the daemon or `hive-priv` trusts.
|
||||
|
||||
The same pass checks each of the hive's agents. It reads the agent's record in
|
||||
`agent-closures`, then fetches the closure's narinfo from
|
||||
|
|
@ -581,17 +585,20 @@ means check attic's fingerprint format; it says nothing about an attacker. A
|
|||
### Rebuilds download the closure
|
||||
|
||||
A hive builds nothing. A rebuild's `Prebuild` node, which runs while the agent is
|
||||
still up, reads the agent's record in `agent-closures` and asks `hive-priv` to
|
||||
realise the record's store path (`RealiseClosure`). `hive-priv` runs
|
||||
`nix-store --realise` with `--max-jobs 0`, so nix can't build the path, and with
|
||||
the host's configured cache address, and the public key and netrc from the
|
||||
root-owned `/run/hyperhive-priv/attic/`, as per-call settings. The settings replace the host's `substituters` and
|
||||
`trusted-public-keys` and set `require-sigs true`: the swarm cache is the only
|
||||
source, and nix stores the path only when the cache's key signed it. The host's
|
||||
nix configuration stays as it was.
|
||||
still up, reads the agent's record in `agent-closures` and has `hive-c0re` pull
|
||||
the record's store path. `hive-c0re` isn't root and needs no root: it runs
|
||||
`nix-store --realise <path> --max-jobs 0 --option substituters <cache address>`
|
||||
through the host's nix daemon. The address is the one in the fragment above, the
|
||||
only substituter the daemon lets an unprivileged client name. The daemon
|
||||
ignores a client's keys, `require-sigs` and netrc, so it stores the path only
|
||||
when a key it trusts signed it. When the fragment is missing, `hive-c0re` first
|
||||
asks `hive-priv` to run `hive-priv-cache-facts` (up to 60 s) instead of waiting for
|
||||
its timer. A host that still has no facts, a signature refusal and a path the
|
||||
cache lacks each fail with their own message.
|
||||
|
||||
`Swap` then does the same lookup and download, a no-op once the path is in the
|
||||
store, and gives `hive-priv` the store path to apply. A stopped agent skips
|
||||
`Swap` then does the same lookup and pull, a no-op once the path is in the
|
||||
store, and gives `hive-priv` the store path to apply. `hive-priv` applies it only
|
||||
when `nix store verify --sigs-needed 1` finds the swarm key's signature on it. A stopped agent skips
|
||||
`Prebuild`, so `Swap` downloads it. A first deploy's `Create` node does the
|
||||
same and creates the container from that path.
|
||||
|
||||
|
|
@ -608,13 +615,13 @@ an unreachable cache costs is one function.
|
|||
|
||||
Mixed versions:
|
||||
|
||||
| hive | controller or record | what happens |
|
||||
| ---- | -------------------- | ------------------------------------------------------------------------------------------ |
|
||||
| old | any | builds locally, as before |
|
||||
| new | old, or no record | refuses the create; a running agent keeps running, and the rebuild logs an error naming it |
|
||||
| new | cache down | the download fails; same as no record, with the download's error in the log |
|
||||
| new | signature mismatch | nix refuses the path; same as no record, and `hive-priv` logs the refusal as well |
|
||||
| new | old `hive-priv` | it refuses the request, which has no `meta_rev`, so the swap or create fails |
|
||||
| hive | controller or record | what happens |
|
||||
| ---- | -------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| old | any | builds locally, as before |
|
||||
| new | old, or no record | refuses the create; a running agent keeps running, and the rebuild logs an error naming it |
|
||||
| new | cache down | the download fails; same as no record, with the download's error in the log |
|
||||
| new | signature mismatch | nix refuses the path; same as no record, and `hive-priv` logs the refusal as well |
|
||||
| new | old `hive-priv` | it refuses the apply of a path it didn't download; `hive-c0re` has it download the path (already in the store) and applies again |
|
||||
|
||||
## Credentials
|
||||
|
||||
|
|
|
|||
|
|
@ -28,35 +28,36 @@ for every operation that genuinely requires root.
|
|||
**Narrow interface** — `PrivRequest` variants map 1:1 to specific
|
||||
known operations; there is no arbitrary command pass-through:
|
||||
|
||||
| Operation | What it runs |
|
||||
| ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `StartContainer` / `StopContainer` | `nixos-container start/stop <name>` |
|
||||
| `KillContainer` | `machinectl kill <machine> --signal=SIGKILL` (`nixos-container` has no kill verb) |
|
||||
| `StartFailureLog` | `journalctl _SYSTEMD_UNIT=container@h-<name>.service --since @<t> -n 30 -o cat --no-pager`, fixed argv, 5 s timeout, output capped at 8 KiB; hive-priv clamps `<t>` to the last hour. hive-c0re appends the lines to the failed start's build log, which only the dashboard serves. |
|
||||
| `CreateContainer` / `UpdateContainer` | `nixos-container create <name> --system-path <toplevel>`; for an update, `nix-env -p <profile> --set <toplevel>` and a reload of the unit. `<toplevel>` must be the store path the last successful `RealiseClosure` returned for that agent (in memory, per agent), so hive-c0re can't name a path never realised for the agent; hive-priv builds nothing. The binding doesn't mean hive-c0re can't choose the closure: it can still ask hive-priv to download any swarm-signed path for any agent and then apply it. The swarm key and the cache bound that to a cross-assignment risk (one agent gets another's closure), not an arbitrary-path one |
|
||||
| `RealiseClosure` | `nix-store --realise <store_path> --max-jobs 0` against the swarm cache, with the host's configured cache address, and the public key and netrc from root-owned `/run/hyperhive-priv/attic/`, as per-call settings that replace the host's substituters and trusted keys and require a signature. The path must be `/nix/store/<32 chars>-<name>` and the agent name valid. It downloads and never builds |
|
||||
| `DestroyContainer` | `nixos-container destroy <name>` |
|
||||
| `ListContainers` | `nixos-container list` |
|
||||
| `ReloadGatewayNginx` | `systemctl reload/start/reset-failed nginx` (host unit; the unit name is hard-coded, not a parameter) |
|
||||
| `WriteNspawnFlags` | write `/etc/nixos-containers/<container>.conf` (bind-mount list, network isolation vars, and `--ephemeral` after the root's salvage). hive-priv first copies a stopped container's root to the agent's `work/`, then renames it aside (it deletes nothing), following no link the container could have planted; for an agent whose closure record sets `private_users` it also adds `--private-users=pick`, mounts every bind `owneridmap`, and once chowns the agent's `state`, `harness`, `work` and `claude` dirs (never `config`, following no link) to the data owner |
|
||||
| `EnsureAgentSocketDir` | `mkdirat` `/run/hive-agent/<name>` `0751 root:root` under an `O_NOFOLLOW` parent fd; leaves an existing directory alone, refuses anything else; for an agent whose closure record sets `private_users` it then chowns the dir to the data owner |
|
||||
| `WriteResourceLimits` | write `CPUQuota=`/`MemoryMax=`/`CPUWeight=`/`IOWeight=` systemd drop-in for agent container, with an `ExecStartPre=` that runs `hive-priv check-rootfs <container>` (no-follow check of the root paths the start script touches; see persistence.md), and a GC root on the build it names |
|
||||
| `RemoveServiceDropin` | remove `container@<name>.service.d/` drop-in on destroy, and the GC root of the hive-priv build it named |
|
||||
| `DaemonReload` | `systemctl daemon-reload` |
|
||||
| `ControlInfraContainer` | `systemctl <action> nginx.service` for the gateway — the `InfraContainer` enum is the allowlist, and serde rejects unknown names at the wire boundary (`hive-c0re`, the forge and the matrix have no variant, so no request can name them) |
|
||||
| `SyncAgentTmpfiles` | unlink `/etc/tmpfiles.d/hyperhive-agents.conf` and return `Ok` |
|
||||
| `SetAgentPaused` | create / remove the `<state>/<name>/harness/paused` marker that parks an agent's turn loop |
|
||||
| `EnsureAgentSubvolume` | `btrfs subvolume create <state>/<name>` for a new agent — no-op when the path exists or the filesystem isn't btrfs |
|
||||
| `UpgradeAgentSubvolume` | migrate an existing plain state dir into a subvolume: create, `cp -a --reflink=auto`, atomic swap. Operator opt-in, and the caller stops the agent first |
|
||||
| `DeleteAgentSubvolume` | `btrfs subvolume delete <state>/<name>` — purge path only, never a plain destroy |
|
||||
| `EnsureBtrfsQuota` | `btrfs quota enable <AGENT_STATE_ROOT>`. Operator opt-in — enabling forces a full rescan |
|
||||
| `ReadSubvolumeUsage` | `btrfs qgroup show -f --raw <state>/<name>` |
|
||||
| `SetSubvolumeQuota` | `btrfs qgroup limit <bytes\|none> <state>/<name>` |
|
||||
| `MeasureAgentDisk` | `du -s -x -b -c -P` over `<state>/<name>/state`, `<state>/<name>/work` and `/var/lib/nixos-containers/h-<name>`, killed after 10 min. hive-priv derives the paths from the name; the request carries none |
|
||||
| `SnapshotAgentSubvolume` | `btrfs subvolume snapshot -r <agent_root> <snapshot_path>` — the frozen point-in-time copy a migration streams from |
|
||||
| `DeleteAgentSnapshot` | `btrfs subvolume delete <snapshot_path>` |
|
||||
| `SendAgentSnapshotToFile` | `btrfs send [-p <parent>] <snapshot>` into a bare filename under `MIGRATE_STAGING_ROOT`; refuses to overwrite an existing export |
|
||||
| `SendAgentSnapshotToFd` | `btrfs send [-p <parent>] <snapshot>` into a **file descriptor passed with the request** (`SCM_RIGHTS`). hive-c0re connects to the peer and hands over the connected socket, so hive-priv never learns an address or protocol. hive-priv requires exactly one descriptor here, and refuses one that arrives alongside any other operation |
|
||||
| Operation | What it runs |
|
||||
| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `StartContainer` / `StopContainer` | `nixos-container start/stop <name>` |
|
||||
| `KillContainer` | `machinectl kill <machine> --signal=SIGKILL` (`nixos-container` has no kill verb) |
|
||||
| `StartFailureLog` | `journalctl _SYSTEMD_UNIT=container@h-<name>.service --since @<t> -n 30 -o cat --no-pager`, fixed argv, 5 s timeout, output capped at 8 KiB; hive-priv clamps `<t>` to the last hour. hive-c0re appends the lines to the failed start's build log, which only the dashboard serves. |
|
||||
| `CreateContainer` / `UpdateContainer` | `nixos-container create <name> --system-path <toplevel>`; for an update, `nix-env -p <profile> --set <toplevel>` and a reload of the unit. `<toplevel>` must already be in the store with a signature by the swarm cache's key: hive-priv runs `nix store verify --no-contents --sigs-needed 1` with `trusted-public-keys` set to the key in root-owned `/run/hyperhive-priv/attic/public-key`, and refuses a path that's absent or that the key didn't sign. hive-priv builds nothing. The check doesn't bind the path to the agent whose pull produced it: hive-c0re could already pull any swarm-signed path and apply it to any agent, so the check bounds the same risk (one agent gets another's closure), not an arbitrary-path one |
|
||||
| `RealiseClosure` | Kept for one release, for an older hive-c0re that still asks hive-priv to download. Downloads nothing: answers `Ok` when the path is in the store and signed by the swarm key (what `CreateContainer` needs), else `not available: <reason>`. The path must be `/nix/store/<32 chars>-<name>` and the agent name valid |
|
||||
| `RefreshCacheFacts` | `systemctl start hive-priv-cache-facts.service`, waited for up to 60 s. The request names no unit and has no field. hive-c0re asks for it when its cache fragment is missing |
|
||||
| `DestroyContainer` | `nixos-container destroy <name>` |
|
||||
| `ListContainers` | `nixos-container list` |
|
||||
| `ReloadGatewayNginx` | `systemctl reload/start/reset-failed nginx` (host unit; the unit name is hard-coded, not a parameter) |
|
||||
| `WriteNspawnFlags` | write `/etc/nixos-containers/<container>.conf` (bind-mount list, network isolation vars, and `--ephemeral` after the root's salvage). hive-priv first copies a stopped container's root to the agent's `work/`, then renames it aside (it deletes nothing), following no link the container could have planted; for an agent whose closure record sets `private_users` it also adds `--private-users=pick`, mounts every bind `owneridmap`, and once chowns the agent's `state`, `harness`, `work` and `claude` dirs (never `config`, following no link) to the data owner |
|
||||
| `EnsureAgentSocketDir` | `mkdirat` `/run/hive-agent/<name>` `0751 root:root` under an `O_NOFOLLOW` parent fd; leaves an existing directory alone, refuses anything else; for an agent whose closure record sets `private_users` it then chowns the dir to the data owner |
|
||||
| `WriteResourceLimits` | write `CPUQuota=`/`MemoryMax=`/`CPUWeight=`/`IOWeight=` systemd drop-in for agent container, with an `ExecStartPre=` that runs `hive-priv check-rootfs <container>` (no-follow check of the root paths the start script touches; see persistence.md), and a GC root on the build it names |
|
||||
| `RemoveServiceDropin` | remove `container@<name>.service.d/` drop-in on destroy, and the GC root of the hive-priv build it named |
|
||||
| `DaemonReload` | `systemctl daemon-reload` |
|
||||
| `ControlInfraContainer` | `systemctl <action> nginx.service` for the gateway — the `InfraContainer` enum is the allowlist, and serde rejects unknown names at the wire boundary (`hive-c0re`, the forge and the matrix have no variant, so no request can name them) |
|
||||
| `SyncAgentTmpfiles` | unlink `/etc/tmpfiles.d/hyperhive-agents.conf` and return `Ok` |
|
||||
| `SetAgentPaused` | create / remove the `<state>/<name>/harness/paused` marker that parks an agent's turn loop |
|
||||
| `EnsureAgentSubvolume` | `btrfs subvolume create <state>/<name>` for a new agent — no-op when the path exists or the filesystem isn't btrfs |
|
||||
| `UpgradeAgentSubvolume` | migrate an existing plain state dir into a subvolume: create, `cp -a --reflink=auto`, atomic swap. Operator opt-in, and the caller stops the agent first |
|
||||
| `DeleteAgentSubvolume` | `btrfs subvolume delete <state>/<name>` — purge path only, never a plain destroy |
|
||||
| `EnsureBtrfsQuota` | `btrfs quota enable <AGENT_STATE_ROOT>`. Operator opt-in — enabling forces a full rescan |
|
||||
| `ReadSubvolumeUsage` | `btrfs qgroup show -f --raw <state>/<name>` |
|
||||
| `SetSubvolumeQuota` | `btrfs qgroup limit <bytes\|none> <state>/<name>` |
|
||||
| `MeasureAgentDisk` | `du -s -x -b -c -P` over `<state>/<name>/state`, `<state>/<name>/work` and `/var/lib/nixos-containers/h-<name>`, killed after 10 min. hive-priv derives the paths from the name; the request carries none |
|
||||
| `SnapshotAgentSubvolume` | `btrfs subvolume snapshot -r <agent_root> <snapshot_path>` — the frozen point-in-time copy a migration streams from |
|
||||
| `DeleteAgentSnapshot` | `btrfs subvolume delete <snapshot_path>` |
|
||||
| `SendAgentSnapshotToFile` | `btrfs send [-p <parent>] <snapshot>` into a bare filename under `MIGRATE_STAGING_ROOT`; refuses to overwrite an existing export |
|
||||
| `SendAgentSnapshotToFd` | `btrfs send [-p <parent>] <snapshot>` into a **file descriptor passed with the request** (`SCM_RIGHTS`). hive-c0re connects to the peer and hands over the connected socket, so hive-priv never learns an address or protocol. hive-priv requires exactly one descriptor here, and refuses one that arrives alongside any other operation |
|
||||
|
||||
**Strings that reach root-loaded config** — `WriteResourceLimits` and
|
||||
`WriteNspawnFlags` copy request strings into a systemd drop-in and an env
|
||||
|
|
@ -68,39 +69,61 @@ refuses, never escapes, any value outside its shape. `memory_max` is
|
|||
bind or credential path is absolute with no whitespace, control byte, `\`,
|
||||
`$`, backtick, `"` or `:`. The size limits of a value stay with systemd.
|
||||
|
||||
**Closures from the swarm cache** — `RealiseClosure` is how a hive's rebuild
|
||||
takes the closure the swarm built instead of building it. Root-owned sources
|
||||
alone decide what root accepts:
|
||||
**Closures from the swarm cache** — `hive-c0re` pulls the closure the swarm
|
||||
built through the host's nix daemon, and `hive-priv` applies it only when the
|
||||
swarm's key signed it. Pulling needs no root: `nix-store --realise` of a store
|
||||
path is a request to the daemon. Root-owned sources alone decide what the
|
||||
daemon accepts:
|
||||
|
||||
- The cache signs. The swarm's cache (attic) holds the signing key and signs
|
||||
each narinfo it serves; the private key never leaves it. The builder pushes
|
||||
with a push token and holds no signing key.
|
||||
- The address is the host's configuration:
|
||||
`services.hyperhive.swarm.attic.domain` and `cache` become
|
||||
`HIVE_PRIV_CACHE_URL` in the `hive-priv` unit. No request and no file chooses
|
||||
it, and `hive-priv` refuses an address that isn't `https`.
|
||||
- The key and the pull token come from the secret store. The root unit
|
||||
`hive-priv-cache-facts` reads the hive's own copies under the hive's own store
|
||||
identity and writes them to `/run/hyperhive-priv/attic/` (directory `0700`,
|
||||
files `0600`, root). `hive-priv` reads nothing from `/run/hyperhive/attic/`,
|
||||
which `hive-c0re` writes for the probe, and it refuses a file in the root
|
||||
directory that isn't a plain file owned by root or that others can read, and
|
||||
the directory under the same rule.
|
||||
- Nix verifies. For that one call `hive-priv` sets `substituters` and
|
||||
`trusted-public-keys` to the swarm cache and its key, and `require-sigs true`.
|
||||
A path without a signature by that key isn't stored; `hive-priv` answers
|
||||
`not available` and logs the refusal at error level. `--max-jobs 0` means no
|
||||
answer, whatever the cache sends, starts a build as root.
|
||||
- A hive realises only paths whose signature checks. The request carries an
|
||||
agent name and a store path, both validated, and no address, key or file; any
|
||||
other field never reaches the verb.
|
||||
`services.hyperhive.swarm.attic.domain` and `cache`. The root unit
|
||||
`hive-priv-cache-facts` lists it as `extra-trusted-substituters` in a nix.conf
|
||||
fragment, `/run/hyperhive-nix/cache.conf`, that the host's nix.conf includes
|
||||
(`!include`, so a host whose unit hasn't run yet still starts).
|
||||
- The key and the pull token come from the secret store. The unit reads the
|
||||
hive's own copies under the hive's own store identity and writes the key to
|
||||
`/run/hyperhive-priv/attic/public-key` and the token to `netrc` there
|
||||
(directory `0700`, files `0600`, root), and the key into the fragment as
|
||||
`extra-trusted-public-keys` and the netrc's path as `netrc-file`. The fragment
|
||||
holds no secret. `hive-c0re` reads its cache address, and nothing else, from
|
||||
it; `hive-priv` reads nothing from `/run/hyperhive/attic/`, which `hive-c0re`
|
||||
writes for the probe.
|
||||
- The daemon enforces. `hive-core` is an unprivileged client: the daemon takes
|
||||
its `--option substituters` only for an address in `trusted-substituters`,
|
||||
ignores its `trusted-public-keys`, `require-sigs` and `netrc-file`, and
|
||||
refuses a path no trusted key signed. `--max-jobs 0` means no answer, whatever
|
||||
the cache sends, starts a build.
|
||||
- `hive-priv` verifies. Before `CreateContainer` or `UpdateContainer` applies a
|
||||
toplevel, it runs `nix store verify --no-contents --sigs-needed 1` with
|
||||
`trusted-public-keys` set to the key in the root file, and refuses a path that
|
||||
is absent or that the key didn't sign. The daemon's trusted keys also include
|
||||
the host's other caches, so this check is the one that counts only the swarm's.
|
||||
- When the fragment is missing, `hive-c0re` asks `hive-priv` for `RefreshCacheFacts`,
|
||||
which starts the facts unit and waits up to 60 s, instead of waiting for the
|
||||
unit's timer. A host the swarm hasn't given facts to answers
|
||||
`this hive has no cache facts yet`.
|
||||
|
||||
What a compromised `hive-c0re` can do with this verb: ask for any store path
|
||||
that has the shape of one and that the swarm cache signed, which fills the
|
||||
store with paths the swarm vouches for. It can't make root accept a path from
|
||||
a cache or under a key of its choosing, because it writes none of the places
|
||||
those come from. The trust anchor is the cache's signing key and the store that
|
||||
hands out its public key, as for every other credential.
|
||||
What a compromised `hive-c0re` can do: pull any store path that the swarm cache
|
||||
signed, which fills the store with paths the swarm vouches for, and then ask
|
||||
`hive-priv` to apply any such path to any agent. Before this change it could
|
||||
already do both through `RealiseClosure` and an apply; the former check bound an
|
||||
apply to the path `hive-priv` itself downloaded for that agent, and not to the
|
||||
agent the swarm built it for. It can't make root accept a path from a cache or
|
||||
under a key of its choosing, because it writes none of the places those come
|
||||
from. The trust anchor is the cache's signing key and the store that hands out
|
||||
its public key, as for every other credential.
|
||||
|
||||
**Mixed versions on one host.** `hive-priv` and `hive-c0re` restart separately.
|
||||
A new `hive-c0re` with an older `hive-priv`: the older one refuses an apply of a
|
||||
path it didn't download, so `hive-c0re` answers that refusal by sending the
|
||||
older `RealiseClosure`, which downloads and pins the path (already in the
|
||||
store), and applies again. A new `hive-priv` with an older `hive-c0re`: it still
|
||||
accepts `RealiseClosure`, answers `Ok` for a path already in the store with the
|
||||
swarm's signature and `not available` for one that isn't, and applies. Both
|
||||
shims go in the release after this one.
|
||||
|
||||
**Container allowlist** — `hive-priv` validates every request against
|
||||
an allowlist before any operation: the allowlist accepts only names
|
||||
|
|
|
|||
Loading…
Reference in a new issue