# Swarm builder The swarm builder checks that an agent config PR builds before an operator merges it. Each time an `agent-configs/` PR opens, reopens or gets a new push, the builder builds the PR's head on the agent base, and the result shows as a `swarm-builder` commit status on the PR and as a `build_agent` node on the swarm UI's Jobs page. Where it runs and how it's enabled: [`services.md`](services.md#builder). ## What happens on a config PR 1. The forge delivers the `pull_request` event on the `config-pr` webhook swarm-controller already registers on `agent-configs`. An `opened`, `reopened` or `synchronized` action on an open PR queues one `build_agent` node carrying the agent, the head commit and the PR number. An edit, a label or a close builds nothing. Every `build_agent` node needs the one `builder` slot, so builds run one at a time and the rest wait pending. A newer head cancels its PR's pending build; a build already running finishes and writes its status on its own, older commit. 2. The node sets the status to `pending`, then sends the build as a request on the queue subject `$SWARM.builder.build` and waits up to 61 minutes for the reply. The builder runs one build at a time and refuses a request that has waited 30 seconds for its turn, such as one behind a build a controller restart orphaned: a build started later could still run when the 61 minutes end. Reading the forge token before a build has 20 seconds, and the rest of the minute covers the reply's trip back. 3. The builder runs `nix build` of `nix/builder/check-flake`'s `check` output with every input overridden: - `hyperhive` and `nixpkgs`: the sources the builder's own host was deployed with, the same two a hive on that host renders into its meta flake (`c0re.hyperhiveFlake`, `c0re.nixpkgsFlake`). - `hyperhive-docs`: the `docs/` source a hive on that host renders into its meta flake (`c0re.hyperhiveDocs`). The check sets `services.hyperhive.agent.docs.source` from it, as the meta flake does, since the `hyperhive` source has no `docs/`. - `agent`: `git+https:///agent-configs/?ref=refs/pull//head&rev=`. A build has one hour. 4. The reply is the system closure's store path, or the last lines `nix` printed. The node writes the status on the head commit: | Outcome | Status | Description | Node | | ----------------------------------------- | --------- | --------------------------- | ------ | | built | `success` | the store path | done | | `nix` failed or timed out | `failure` | the last line `nix` printed | failed | | the builder busy for 30 seconds | `failure` | why | failed | | no builder listening, or no reply in time | `error` | why | failed | The controller records every build that gets a final status, and the status links to that record: see [Build records](#build-records). The full tail is also in the failed node's error on the Jobs page and in the builder's journal line for the build, as the field `F_ERROR_TAIL`: ```console $ swarm-logs query '_time:7d _SYSTEMD_UNIT:="swarm-builder.service" _msg:="agent config build failed" | fields _time, F_AGENT, F_PR, F_REV, F_ERROR_TAIL' --format json ``` The builder holds the last 64 KiB of each of `nix`'s output streams while the build runs, and the reply is the last 30 lines of that (at most 4000 bytes), so a long build log never grows its memory. ## Build records `swarm-controller` records each build when it writes the final status: built, failed, or no build ran or came back. The newest 100 are one JSON array under the key `recent` in the `builder-runs` KV bucket on the swarm queue, which only the controller reads or writes. It creates the bucket on first use and adds each build with a compare-and-set write, so the records survive a controller restart. Recording the 101st drops the oldest. A record the controller can't decode stays in the bucket, out of the list, and counts toward the 100. Each record has: - `id`: `--` - `repo` (`agent-configs/`), `pr` and `head_sha` - `started_unix` and `finished_unix` - `outcome`: `built`, `failed` or `error`, the status's `success`, `failure` or `error` - `out_path` for `built`, `error_tail` (the last 30 lines `nix` printed) for `failed`, and `detail` (why) for `error` `GET /api/builder/runs` lists them, newest first, and `GET /api/builder/runs/` returns one, or 404 when it isn't among the newest 100. Both are behind the swarm UI vhost's `admins` gate, and answer 503 when the controller has no swarm queue or can't open the bucket, 500 when it can't read it. The final status links to `https:///api/builder/runs/`, so selecting **Details** on a red `swarm-builder` check opens the record, with `nix`'s tail, in the browser. A status has no link when the record wasn't stored (the controller logs the failed write) or when the controller doesn't serve the swarm vhost (`SWARM_CONTROLLER_PUBLIC_URL` unset). The `pending` status has no link. ## What the check is, and isn't - It composes the config the way a hive's meta flake does (`agent-base.extendModules` with the config's `nixosModules.default`), minus what a hive adds: its CA bundle, `claude-code` path, ports and OTEL settings, and the `follows` that point the config's own inputs at the hive's. A green check can therefore still fail on a hive, and the reverse. - The agent base is the builder's host's, not the hive's. Hives on another hyperhive or nixpkgs pin build something else. - The builder pushes nothing. The closure stays in the builder host's store until it's collected, so on a host that also runs the hive, the hive's own build can reuse whatever matches. - It doesn't gate anything. A merge deploys whatever the status says. - A lost webhook delivery isn't retried: the head isn't built until the next push. The jobs graph is in memory, so a controller restart drops a build in flight and leaves its status `pending`. ## Archetype evaluation The builder also answers `$SWARM.builder.archetypes`: it evaluates the `meta` of every agent archetype its own hyperhive source declares, with no forge token, and answers with the JSON. An evaluation doesn't wait for a build in progress and has two minutes. What the controller does with the answer: [`archetypes.md`](archetypes.md). ## Credentials | What | Where | Who reads it | | ----------------------- | ------------------------------------------------- | -------------------------------------------------- | | store leaf | `deploy.bao.builderClient{Cert,Key}File` | nspawn, into the service as a credential | | queue client secret | `secret/swarm/services/swarm-builder/oidc/client` | the builder, at start and when authelia refuses it | | forge token (read-only) | `secret/swarm/services/builder/forge/token` | the builder, on every build | The token belongs to the `builder` forge account, which swarm-controller creates, puts in the `hives` team of `agent-configs` (read on every config repo) and mints a `read:repository` token for, in a five-minute pass. `git` gets the token through a credential helper reading the child's environment: it's in no argument, URL or lock, so no line `nix` prints can carry it into a status. The same `git` fetches a config's own inputs, so the builder registers the helper for the forge's URL only, and the helper answers only an `https` request for the forge's host: any other host gets nothing. The builder holds no write credential; swarm-controller writes the status with its own forge token. Details per path: [`credentials.md`](credentials.md). ## Isolation The container has its own network namespace on the bridge, as the CI runner does, so a config's evaluation can't reach a host-loopback service. Evaluation is pure and runs in the container. Builds go to the host's nix daemon as an untrusted client: a config can set no daemon option, and its derivations build in the host's sandbox. A fixed-output derivation still has the network, as it does for any build on that host.