mara/hyperhive
mara/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

pull swarm closures through the nix daemon: substituters in the fragment, merged root-only netrc

Browse source
This commit is contained in:
atlas 2026-11-08 10:29:46 +01:00 • committed by mara
commit ef68baa00f
1 changed files with 582 additions and 382 deletions
Download patch file Download diff file Expand all files Collapse all files

44
docs/4-implementation/swarm/builder.md
View file

@ -536,18 +536,24 @@ writes nothing.
`hive-priv` and the nix daemon don't read those files. The key that must have
signed a download and the pull token come from `hive-priv-cache-facts`, a root
unit that reads the hive's own `attic/public-key` and `attic/pull-token` from
the store under the hive's own store identity every ten minutes and writes them
to `/run/hyperhive-priv/attic/` (`0700`, root; each file `0600`). It also writes
the nix.conf fragment `/run/hyperhive-nix/cache.conf`, which the host's nix.conf
includes, with the key as `extra-trusted-public-keys` and nothing else: no
substituter and no netrc, so the daemon never contacts the cache and no other
client of the daemon, such as an agent on `nixBuilds = host`, can fetch from it.
A second copy of the netrc, `/run/hyperhive-nix/netrc` (`root:hive-core 0640`),
is the pull token `hive-c0re` uses. The cache's address is the host's
`services.hyperhive.swarm.attic.domain` and `cache`, handed to `hive-c0re`. The
unit restarts `nix-daemon` only when the fragment changed after the daemon
started, which matters for CppNix; Lix re-reads nix.conf on every connection. A
file `hive-c0re` writes decides nothing the daemon or `hive-priv` trusts.
the store under the hive's own store identity every ten minutes. It writes the
key to `/run/hyperhive-priv/attic/public-key` (`0700` directory, `0600` file)
and the nix.conf fragment `/run/hyperhive-nix/cache.conf`, which the host's
nix.conf includes, with the cache's address as `extra-substituters`, the key as
`extra-trusted-public-keys` and `netrc-file = /run/hyperhive-nix/netrc`. The
netrc (`root:root 0600`) holds the pull token's entry for the cache's host and
then the entries of the host's own netrc, which is `nix.settings.netrc-file`
when the host sets it and nix's default `/etc/nix/netrc` otherwise, because
`netrc-file` takes one path and would replace the host's. The unit merges again
on every run, so an edit to the host's netrc reaches the daemon within ten
minutes. An entry for the cache's own host in the host's netrc gives way to the
pull token's. The unit doesn't see a netrc that the host names only in
`nix.extraOptions`. The cache's address is the host's
`services.hyperhive.swarm.attic.domain` and `cache`. The unit restarts
`nix-daemon` only when the fragment changed after the daemon started, which
matters for CppNix; Lix re-reads nix.conf on every connection and runs no
`nix-daemon.service`, so the unit logs and restarts nothing there. A file
`hive-c0re` writes decides nothing the daemon or `hive-priv` trusts.
The same pass checks each of the hive's agents. It reads the agent's record in
`agent-closures`, then fetches the closure's narinfo from
@ -591,13 +597,15 @@ means check attic's fingerprint format; it says nothing about an attacker. A
A hive builds nothing. A rebuild's `Prebuild` node, which runs while the agent is
still up, reads the agent's record in `agent-closures` and has `hive-c0re` pull
the record's store path. `hive-c0re` isn't root and needs no root: it runs
`nix copy --from <cache address> --option netrc-file /run/hyperhive-nix/netrc <path>`
as `hive-core`. The nix client fetches from the cache with the pull token and
hands the NARs to the host's nix daemon, which stores each path only when a key
it trusts signed it; nothing builds. When the netrc or the key is missing,
`nix-store --realise <path> --max-jobs 0` as `hive-core`, which asks the host's
nix daemon to substitute the path. The daemon takes each path of the closure
from the swarm cache or from its other substituters (`cache.nixos.org`),
whichever has it, which matters because the swarm cache leaves out the paths
`cache.nixos.org` serves. It stores a path only when a key it trusts signed it,
and nothing builds. When the fragment is missing or lacks a setting,
`hive-c0re` first asks `hive-priv` to run `hive-priv-cache-facts` (up to 60 s)
instead of waiting for its timer. A host that still has no facts, a signature refusal and a path the
cache lacks each fail with their own message.
instead of waiting for its timer. A host that still has no facts, a signature
refusal and a path no substituter has each fail with their own message.
`Swap` then does the same lookup and pull, a no-op once the path is in the
store, and gives `hive-priv` the store path to apply. `hive-priv` applies it only