Watch

atlas b68b74d687 swarm-builder: build each agent config PR head and post a commit status

A new `swarm-builder` service, in its own container on the host that sets
`deploy.builder.enable` (defaulted on by `allSwarmServices` where the host
has the builder's store leaf, like the review classifier).

When an `agent-configs/<agent>` PR is opened, reopened or synchronized,
swarm-controller queues a `build_agent` jobq node. The node sets a
`swarm-builder` commit status to pending on the PR head, sends
`{agent, rev, pr}` to the builder over NATS request-reply
(`$SWARM.builder.build`), and writes success (store path), failure (last
line nix printed) or error (no builder / no reply) on the head. The node
fails when the build does, so the Jobs page matches the PR.

The builder builds nix/builder/check-flake's `check` with every input
overridden: hyperhive and nixpkgs to the host's own
`c0re.hyperhiveFlake`/`c0re.nixpkgsFlake` (what a hive there renders into
its meta flake), and the agent to the PR head on the forge. Evaluation is
pure and runs in the container; builds go to the host nix daemon as an
untrusted client. The private config repo is fetched with the token of a
new read-only `builder` forge account (in the `hives` team), which the
controller mints into `swarm/services/builder/forge/token`; git gets it
from a credential helper reading the child's environment, so it is in no
argument or URL.

Credentials: a `builder` store leaf minted by glue-bao-tls on the store
host, a `swarm-builder` reader policy (forge token + its own OIDC client
secret), an authelia machine client, and a `--builder-client` grant in the
queue's auth-callout responder. The controller gains a grant on the token
path and the secret publisher a deny on it.

This is the agent-config build check atlas scoped on #5081 (slice 3):
nothing is pushed to a cache, nothing is written to KV, and a red status
gates no deploy. The check composes agent-base + the config without the
values a hive adds (CA, docs, claude-code path, ports, input follows), so
it can disagree with a hive's own build.

2026-10-14 03:30:46 +02:00

4.6 KiB

Raw Blame History

Swarm builder

The swarm builder checks that an agent config PR builds before an operator merges it. Each time an agent-configs/<agent> PR opens, reopens or gets a new push, the builder builds the PR's head on the agent base, and the result shows as a swarm-builder commit status on the PR and as a build_agent node on the swarm UI's Jobs page.

Where it runs and how it's enabled: services.md.

What happens on a config PR

The forge delivers the pull_request event on the config-pr webhook swarm-controller already registers on agent-configs. An opened, reopened or synchronized action on an open PR queues one build_agent node carrying the agent, the head commit and the PR number. An edit, a label or a close builds nothing.
The node sets the status to pending, then sends the build as a request on the queue subject $SWARM.builder.build and waits up to 61 minutes for the reply.
The builder runs nix build of nix/builder/check-flake's check output with every input overridden:
- hyperhive and nixpkgs: the sources the builder's own host was deployed with, the same two a hive on that host renders into its meta flake (c0re.hyperhiveFlake, c0re.nixpkgsFlake).
- agent: git+https://<forge>/agent-configs/<agent>?ref=refs/pull/<pr>/head&rev=<head>. A build has one hour.

The reply is the system closure's store path, or the last lines nix printed. The node writes the status on the head commit:

Outcome	Status	Description	Node
built	`success`	the store path	done
`nix` failed or timed out	`failure`	the last line `nix` printed	failed
no builder listening, or no reply in time	`error`	why	failed

The full tail is in the failed node's error on the Jobs page and in the builder's journal line for the build.

What the check is, and isn't

It composes the config the way a hive's meta flake does (agent-base.extendModules with the config's nixosModules.default), minus what a hive adds: its CA bundle, docs source, claude-code path, ports and OTEL settings, and the follows that point the config's own inputs at the hive's. A green check can therefore still fail on a hive, and the reverse.
The agent base is the builder's host's, not the hive's. Hives on another hyperhive or nixpkgs pin build something else.
The builder pushes nothing and records nothing. The closure stays in the builder host's store until it's collected, so on a host that also runs the hive, the hive's own build can reuse whatever matches.
It doesn't gate anything. A merge deploys whatever the status says.
A lost webhook delivery isn't retried: the head isn't built until the next push. The jobs graph is in memory, so a controller restart drops a build in flight and leaves its status pending.

Credentials

What	Where	Who reads it
store leaf	`deploy.bao.builderClient{Cert,Key}File`	nspawn, into the service as a credential
queue client secret	`secret/swarm/services/swarm-builder/oidc/client`	the builder, at start
forge token (read-only)	`secret/swarm/services/builder/forge/token`	the builder, on every build

The token belongs to the builder forge account, which swarm-controller creates, puts in the hives team of agent-configs (read on every config repo) and mints a read:repository token for, in a five-minute pass. git gets the token through a credential helper reading the child's environment: it's in no argument, URL or lock, so no line nix prints can carry it into a status. The builder holds no write credential; swarm-controller writes the status with its own forge token. Details per path: credentials.md.

Isolation

The container has its own network namespace on the bridge, as the CI runner does, so a config's evaluation can't reach a host-loopback service. Evaluation is pure and runs in the container. Builds go to the host's nix daemon as an untrusted client: a config can set no daemon option, and its derivations build in the host's sandbox. A fixed-output derivation still has the network, as it does for any build on that host.

4.6 KiB Raw Blame History