Watch

atlas cafa120d77 swarm-builder: scope the forge token to the forge, one build at a time, opt-in

argus's review on #5177: the git credential helper answered every `get`,
so a config PR's own flake inputs could have git send the builder's forge
token to any host. The helper is now configured under
`credential.<forge-url>.helper` and itself answers only an https request
whose host is the forge's; a test runs it as git does and checks that
other hosts, http and non-get operations get nothing.

Builds are bounded: every `build_agent` node needs the one `builder` jobq
resource, so one runs at a time and the rest wait pending, and a newer
head cancels its PR's pending build (a running one finishes and writes on
its own commit). The builder also holds a one-permit semaphore around each
build.

`builder` and `review-classifier`, the forge logins swarm-controller owns,
join nix/reserved-names.nix, with a test tying the list to both constants.

The builder is now opt-in like the CI runner: `allSwarmServices` no longer
turns it on, since it runs unreviewed config PRs through the host's nix
daemon. mara had not answered atlas's question on the PR.

2026-10-14 03:30:46 +02:00

5.1 KiB

Raw Blame History

Swarm builder

The swarm builder checks that an agent config PR builds before an operator merges it. Each time an agent-configs/<agent> PR opens, reopens or gets a new push, the builder builds the PR's head on the agent base, and the result shows as a swarm-builder commit status on the PR and as a build_agent node on the swarm UI's Jobs page.

Where it runs and how it's enabled: services.md.

What happens on a config PR

The forge delivers the pull_request event on the config-pr webhook swarm-controller already registers on agent-configs. An opened, reopened or synchronized action on an open PR queues one build_agent node carrying the agent, the head commit and the PR number. An edit, a label or a close builds nothing. Every build_agent node needs the one builder slot, so builds run one at a time and the rest wait pending. A newer head cancels its PR's pending build; a build already running finishes and writes its status on its own, older commit.
The node sets the status to pending, then sends the build as a request on the queue subject $SWARM.builder.build and waits up to 61 minutes for the reply.
The builder runs nix build of nix/builder/check-flake's check output with every input overridden:
- hyperhive and nixpkgs: the sources the builder's own host was deployed with, the same two a hive on that host renders into its meta flake (c0re.hyperhiveFlake, c0re.nixpkgsFlake).
- agent: git+https://<forge>/agent-configs/<agent>?ref=refs/pull/<pr>/head&rev=<head>. A build has one hour.

The reply is the system closure's store path, or the last lines nix printed. The node writes the status on the head commit:

Outcome	Status	Description	Node
built	`success`	the store path	done
`nix` failed or timed out	`failure`	the last line `nix` printed	failed
no builder listening, or no reply in time	`error`	why	failed

The full tail is in the failed node's error on the Jobs page and in the builder's journal line for the build.

What the check is, and isn't

It composes the config the way a hive's meta flake does (agent-base.extendModules with the config's nixosModules.default), minus what a hive adds: its CA bundle, docs source, claude-code path, ports and OTEL settings, and the follows that point the config's own inputs at the hive's. A green check can therefore still fail on a hive, and the reverse.
The agent base is the builder's host's, not the hive's. Hives on another hyperhive or nixpkgs pin build something else.
The builder pushes nothing and records nothing. The closure stays in the builder host's store until it's collected, so on a host that also runs the hive, the hive's own build can reuse whatever matches.
It doesn't gate anything. A merge deploys whatever the status says.
A lost webhook delivery isn't retried: the head isn't built until the next push. The jobs graph is in memory, so a controller restart drops a build in flight and leaves its status pending.

Credentials

What	Where	Who reads it
store leaf	`deploy.bao.builderClient{Cert,Key}File`	nspawn, into the service as a credential
queue client secret	`secret/swarm/services/swarm-builder/oidc/client`	the builder, at start
forge token (read-only)	`secret/swarm/services/builder/forge/token`	the builder, on every build

The token belongs to the builder forge account, which swarm-controller creates, puts in the hives team of agent-configs (read on every config repo) and mints a read:repository token for, in a five-minute pass. git gets the token through a credential helper reading the child's environment: it's in no argument, URL or lock, so no line nix prints can carry it into a status. The same git fetches a config's own inputs, so the builder registers the helper for the forge's URL only, and the helper answers only an https request for the forge's host: any other host gets nothing. The builder holds no write credential; swarm-controller writes the status with its own forge token. Details per path: credentials.md.

Isolation

The container has its own network namespace on the bridge, as the CI runner does, so a config's evaluation can't reach a host-loopback service. Evaluation is pure and runs in the container. Builds go to the host's nix daemon as an untrusted client: a config can set no daemon option, and its derivations build in the host's sandbox. A fixed-output derivation still has the network, as it does for any build on that host.

5.1 KiB Raw Blame History