mara/hyperhive
mara/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

hive-c0re pulls closures through the nix daemon; hive-priv verifies the swarm signature before applying

Browse source
This commit is contained in:
atlas 2026-11-07 13:20:55 +01:00 • committed by mara
commit ecd4445680
1 changed files with 1257 additions and 584 deletions
Download patch file Download diff file Expand all files Collapse all files

59
docs/4-implementation/swarm/builder.md
View file

@ -452,7 +452,8 @@ you change `swarm.attic.domain`, delete the stale `url` leaf and restart
replaces the stored one with the key `attic cache info` prints, so a cache
that loses its database and gets a new key needs no step: the next run of the
unit stores the new key, the controller copies it to the hives within five
minutes, and `hive-priv-cache-facts` reads it within ten.
minutes, and `hive-priv-cache-facts` reads it within ten (or at once, when a
rebuild finds the host without facts).
`swarm.attic.domain` and `swarm.attic.cache` default to `attic.<swarm domain>`
and `swarm` on every host, whether or not it runs the cache, so a builder on any
@ -532,14 +533,17 @@ identity and writes `/run/hyperhive/attic/netrc`, `substituter` and
token goes out in a header and in the netrc; a hive that finds another scheme
writes nothing.
`hive-priv` doesn't read those files. The key that must have signed a download and
the pull token come from `hive-priv-cache-facts`, a root unit that reads the
hive's own `attic/public-key` and `attic/pull-token` from the store under the
hive's own store identity every ten minutes and writes them to
`/run/hyperhive-priv/attic/` (`0700`, root; each file `0600`). The cache's
address is the host's `services.hyperhive.swarm.attic.domain` and `cache`,
handed to `hive-priv` in its unit's environment. A file `hive-c0re` writes
decides nothing `hive-priv` trusts.
`hive-priv` and the nix daemon don't read those files. The key that must have
signed a download and the pull token come from `hive-priv-cache-facts`, a root
unit that reads the hive's own `attic/public-key` and `attic/pull-token` from
the store under the hive's own store identity every ten minutes and writes them
to `/run/hyperhive-priv/attic/` (`0700`, root; each file `0600`). It also writes
the nix.conf fragment `/run/hyperhive-nix/cache.conf`, which the host's nix.conf
includes: the cache's address (the host's `services.hyperhive.swarm.attic.domain`
and `cache`) as a trusted substituter, the key as a trusted key and the netrc's
path. It restarts `nix-daemon` only when the fragment changed after the daemon
started, which matters for CppNix; Lix re-reads nix.conf on every connection. A
file `hive-c0re` writes decides nothing the daemon or `hive-priv` trusts.
The same pass checks each of the hive's agents. It reads the agent's record in
`agent-closures`, then fetches the closure's narinfo from
@ -581,17 +585,20 @@ means check attic's fingerprint format; it says nothing about an attacker. A
### Rebuilds download the closure
A hive builds nothing. A rebuild's `Prebuild` node, which runs while the agent is
still up, reads the agent's record in `agent-closures` and asks `hive-priv` to
realise the record's store path (`RealiseClosure`). `hive-priv` runs
`nix-store --realise` with `--max-jobs 0`, so nix can't build the path, and with
the host's configured cache address, and the public key and netrc from the
root-owned `/run/hyperhive-priv/attic/`, as per-call settings. The settings replace the host's `substituters` and
`trusted-public-keys` and set `require-sigs true`: the swarm cache is the only
source, and nix stores the path only when the cache's key signed it. The host's
nix configuration stays as it was.
still up, reads the agent's record in `agent-closures` and has `hive-c0re` pull
the record's store path. `hive-c0re` isn't root and needs no root: it runs
`nix-store --realise <path> --max-jobs 0 --option substituters <cache address>`
through the host's nix daemon. The address is the one in the fragment above, the
only substituter the daemon lets an unprivileged client name. The daemon
ignores a client's keys, `require-sigs` and netrc, so it stores the path only
when a key it trusts signed it. When the fragment is missing, `hive-c0re` first
asks `hive-priv` to run `hive-priv-cache-facts` (up to 60 s) instead of waiting for
its timer. A host that still has no facts, a signature refusal and a path the
cache lacks each fail with their own message.
`Swap` then does the same lookup and download, a no-op once the path is in the
store, and gives `hive-priv` the store path to apply. A stopped agent skips
`Swap` then does the same lookup and pull, a no-op once the path is in the
store, and gives `hive-priv` the store path to apply. `hive-priv` applies it only
when `nix store verify --sigs-needed 1` finds the swarm key's signature on it. A stopped agent skips
`Prebuild`, so `Swap` downloads it. A first deploy's `Create` node does the
same and creates the container from that path.
@ -608,13 +615,13 @@ an unreachable cache costs is one function.
Mixed versions:
| hive | controller or record | what happens |
| ---- | -------------------- | ------------------------------------------------------------------------------------------ |
| old | any | builds locally, as before |
| new | old, or no record | refuses the create; a running agent keeps running, and the rebuild logs an error naming it |
| new | cache down | the download fails; same as no record, with the download's error in the log |
| new | signature mismatch | nix refuses the path; same as no record, and `hive-priv` logs the refusal as well |
| new | old `hive-priv` | it refuses the request, which has no `meta_rev`, so the swap or create fails |
| hive | controller or record | what happens |
| ---- | -------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| old | any | builds locally, as before |
| new | old, or no record | refuses the create; a running agent keeps running, and the rebuild logs an error naming it |
| new | cache down | the download fails; same as no record, with the download's error in the log |
| new | signature mismatch | nix refuses the path; same as no record, and `hive-priv` logs the refusal as well |
| new | old `hive-priv` | it refuses the apply of a path it didn't download; `hive-c0re` has it download the path (already in the store) and applies again |
## Credentials