vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity
hyperhive/docs/swarm/builder.md
atlas e873195cf6 swarm-builder: keep only the tail of nix's output while it runs 
nix_build::run read nix's stdout and stderr with read_to_end, so a long
--print-build-logs run held its whole log in memory until nix exited,
though the reply uses only the last 30 lines (4000 bytes) of stderr and
the last non-empty line of stdout. Both streams now go through a Tail
that holds at most 128 KiB and keeps the last 64 KiB. Output under 64 KiB
reaches tail() whole, so those replies are unchanged.
2026-10-17 08:45:42 +02:00

5.3 KiB
Raw Blame History

Swarm builder

The swarm builder checks that an agent config PR builds before an operator merges it. Each time an agent-configs/<agent> PR opens, reopens or gets a new push, the builder builds the PR's head on the agent base, and the result shows as a swarm-builder commit status on the PR and as a build_agent node on the swarm UI's Jobs page.

Where it runs and how it's enabled: services.md.

What happens on a config PR

  1. The forge delivers the pull_request event on the config-pr webhook swarm-controller already registers on agent-configs. An opened, reopened or synchronized action on an open PR queues one build_agent node carrying the agent, the head commit and the PR number. An edit, a label or a close builds nothing. Every build_agent node needs the one builder slot, so builds run one at a time and the rest wait pending. A newer head cancels its PR's pending build; a build already running finishes and writes its status on its own, older commit.

  2. The node sets the status to pending, then sends the build as a request on the queue subject $SWARM.builder.build and waits up to 61 minutes for the reply.

  3. The builder runs nix build of nix/builder/check-flake's check output with every input overridden:

    • hyperhive and nixpkgs: the sources the builder's own host was deployed with, the same two a hive on that host renders into its meta flake (c0re.hyperhiveFlake, c0re.nixpkgsFlake).
    • agent: git+https://<forge>/agent-configs/<agent>?ref=refs/pull/<pr>/head&rev=<head>. A build has one hour.

  4. The reply is the system closure's store path, or the last lines nix printed. The node writes the status on the head commit:

    Outcome Status Description Node
    built success the store path done
    nix failed or timed out failure the last line nix printed failed
    no builder listening, or no reply in time error why failed

    The full tail is in the failed node's error on the Jobs page and in the builder's journal line for the build.

    The builder holds the last 64 KiB of each of nix's output streams while the build runs, and the reply is the last 30 lines of that (at most 4000 bytes), so a long build log never grows its memory.

What the check is, and isn't

  • It composes the config the way a hive's meta flake does (agent-base.extendModules with the config's nixosModules.default), minus what a hive adds: its CA bundle, docs source, claude-code path, ports and OTEL settings, and the follows that point the config's own inputs at the hive's. A green check can therefore still fail on a hive, and the reverse.
  • The agent base is the builder's host's, not the hive's. Hives on another hyperhive or nixpkgs pin build something else.
  • The builder pushes nothing and records nothing. The closure stays in the builder host's store until it's collected, so on a host that also runs the hive, the hive's own build can reuse whatever matches.
  • It doesn't gate anything. A merge deploys whatever the status says.
  • A lost webhook delivery isn't retried: the head isn't built until the next push. The jobs graph is in memory, so a controller restart drops a build in flight and leaves its status pending.

Credentials

What Where Who reads it
store leaf deploy.bao.builderClient{Cert,Key}File nspawn, into the service as a credential
queue client secret secret/swarm/services/swarm-builder/oidc/client the builder, at start and when authelia refuses it
forge token (read-only) secret/swarm/services/builder/forge/token the builder, on every build

The token belongs to the builder forge account, which swarm-controller creates, puts in the hives team of agent-configs (read on every config repo) and mints a read:repository token for, in a five-minute pass. git gets the token through a credential helper reading the child's environment: it's in no argument, URL or lock, so no line nix prints can carry it into a status. The same git fetches a config's own inputs, so the builder registers the helper for the forge's URL only, and the helper answers only an https request for the forge's host: any other host gets nothing. The builder holds no write credential; swarm-controller writes the status with its own forge token. Details per path: credentials.md.

Isolation

The container has its own network namespace on the bridge, as the CI runner does, so a config's evaluation can't reach a host-loopback service. Evaluation is pure and runs in the container. Builds go to the host's nix daemon as an untrusted client: a config can set no daemon option, and its derivations build in the host's sandbox. A fixed-output derivation still has the network, as it does for any build on that host.