hyperhive/docs/swarm/builder.md

# Swarm builder

The swarm builder checks that an agent config PR builds before an operator
merges it. Each time an `agent-configs/<agent>` PR opens, reopens or gets a new
push, the builder builds the PR's head on the agent base, and the result shows as a
`swarm-builder` commit status on the PR and as a `build_agent` node on the
swarm UI's Jobs page.

Where it runs and how it's enabled: [`services.md`](services.md#builder).

## What happens on a config PR

1. The forge delivers the `pull_request` event on the `config-pr` webhook
   swarm-controller already registers on `agent-configs`. An `opened`,
   `reopened` or `synchronized` action on an open PR queues one `build_agent`
   node carrying the agent, the head commit and the PR number. An edit, a
   label or a close builds nothing. Every `build_agent` node needs the one
   `builder` slot, so builds run one at a time and the rest wait pending. A
   newer head cancels its PR's pending build; a build already running
   finishes and writes its status on its own, older commit.
2. The node sets the status to `pending`, then sends the build as a request on
   the queue subject `$SWARM.builder.build` and waits up to 61 minutes for the
   reply.
3. The builder runs `nix build` of `nix/builder/check-flake`'s `check` output
   with every input overridden:
   - `hyperhive` and `nixpkgs`: the sources the builder's own host was
     deployed with, the same two a hive on that host renders into its meta
     flake (`c0re.hyperhiveFlake`, `c0re.nixpkgsFlake`).
   - `hyperhive-docs`: the `docs/` source a hive on that host renders into its
     meta flake (`c0re.hyperhiveDocs`). The check sets
     `services.hyperhive.agent.docs.source` from it, as the meta flake does,
     since the `hyperhive` source has no `docs/`.
   - `agent`: `git+https://<forge>/agent-configs/<agent>?ref=refs/pull/<pr>/head&rev=<head>`.
     A build has one hour.
4. The reply is the system closure's store path, or the last lines `nix`
   printed. The node writes the status on the head commit:

   | Outcome                                   | Status    | Description                 | Node   |
   | ----------------------------------------- | --------- | --------------------------- | ------ |
   | built                                     | `success` | the store path              | done   |
   | `nix` failed or timed out                 | `failure` | the last line `nix` printed | failed |
   | no builder listening, or no reply in time | `error`   | why                         | failed |

   The full tail is in the failed node's error on the Jobs page and in the
   builder's journal line for the build.

   The builder holds the last 64 KiB of each of `nix`'s output streams while the
   build runs, and the reply is the last 30 lines of that (at most 4000
   bytes), so a long build log never grows its memory.

## What the check is, and isn't

- It composes the config the way a hive's meta flake does
  (`agent-base.extendModules` with the config's `nixosModules.default`), minus
  what a hive adds: its CA bundle, `claude-code` path, ports and
  OTEL settings, and the `follows` that point the config's own inputs at the
  hive's. A green check can therefore still fail on a hive, and the reverse.
- The agent base is the builder's host's, not the hive's. Hives on another
  hyperhive or nixpkgs pin build something else.
- The builder pushes nothing and records nothing. The closure stays in the builder
  host's store until it's collected, so on a host that also runs the hive, the
  hive's own build can reuse whatever matches.
- It doesn't gate anything. A merge deploys whatever the status says.
- A lost webhook delivery isn't retried: the head isn't built until the next
  push. The jobs graph is in memory, so a controller restart drops a build in
  flight and leaves its status `pending`.

## Archetype evaluation

The builder also answers `$SWARM.builder.archetypes`: it evaluates the
`meta` of every agent archetype its own hyperhive source declares, with no
forge token, and answers with the JSON. An evaluation doesn't wait for a
build in progress and has two minutes. What the controller does with the
answer: [`archetypes.md`](archetypes.md).

## Credentials

| What                    | Where                                             | Who reads it                                       |
| ----------------------- | ------------------------------------------------- | -------------------------------------------------- |
| store leaf              | `deploy.bao.builderClient{Cert,Key}File`          | nspawn, into the service as a credential           |
| queue client secret     | `secret/swarm/services/swarm-builder/oidc/client` | the builder, at start and when authelia refuses it |
| forge token (read-only) | `secret/swarm/services/builder/forge/token`       | the builder, on every build                        |

The token belongs to the `builder` forge account, which swarm-controller
creates, puts in the `hives` team of `agent-configs` (read on every config
repo) and mints a `read:repository` token for, in a five-minute pass. `git`
gets the token through a credential helper reading the child's environment:
it's in no argument, URL or lock, so no line `nix` prints can carry it into a
status. The same `git` fetches a config's own inputs, so the builder registers
the helper for the forge's URL only, and the helper answers only an `https`
request for the forge's host: any other host gets nothing. The builder holds no write credential; swarm-controller writes the
status with its own forge token. Details per path:
[`credentials.md`](credentials.md).

## Isolation

The container has its own network namespace on the bridge, as the CI runner
does, so a config's evaluation can't reach a host-loopback service. Evaluation
is pure and runs in the container. Builds go to the host's nix daemon as an
untrusted client: a config can set no daemon option, and its derivations build
in the host's sandbox. A fixed-output derivation still has the network, as it
does for any build on that host.