cafa120d77
argus's review on #5177: the git credential helper answered every `get`, so a config PR's own flake inputs could have git send the builder's forge token to any host. The helper is now configured under `credential.<forge-url>.helper` and itself answers only an https request whose host is the forge's; a test runs it as git does and checks that other hosts, http and non-get operations get nothing. Builds are bounded: every `build_agent` node needs the one `builder` jobq resource, so one runs at a time and the rest wait pending, and a newer head cancels its PR's pending build (a running one finishes and writes on its own commit). The builder also holds a one-permit semaphore around each build. `builder` and `review-classifier`, the forge logins swarm-controller owns, join nix/reserved-names.nix, with a test tying the list to both constants. The builder is now opt-in like the CI runner: `allSwarmServices` no longer turns it on, since it runs unreviewed config PRs through the host's nix daemon. mara had not answered atlas's question on the PR.
86 lines
5.1 KiB
Markdown
86 lines
5.1 KiB
Markdown
# Swarm builder
|
|
|
|
The swarm builder checks that an agent config PR builds before an operator
|
|
merges it. Each time an `agent-configs/<agent>` PR opens, reopens or gets a new
|
|
push, the builder builds the PR's head on the agent base, and the result shows as a
|
|
`swarm-builder` commit status on the PR and as a `build_agent` node on the
|
|
swarm UI's Jobs page.
|
|
|
|
Where it runs and how it's enabled: [`services.md`](services.md#builder).
|
|
|
|
## What happens on a config PR
|
|
|
|
1. The forge delivers the `pull_request` event on the `config-pr` webhook
|
|
swarm-controller already registers on `agent-configs`. An `opened`,
|
|
`reopened` or `synchronized` action on an open PR queues one `build_agent`
|
|
node carrying the agent, the head commit and the PR number. An edit, a
|
|
label or a close builds nothing. Every `build_agent` node needs the one
|
|
`builder` slot, so builds run one at a time and the rest wait pending. A
|
|
newer head cancels its PR's pending build; a build already running
|
|
finishes and writes its status on its own, older commit.
|
|
2. The node sets the status to `pending`, then sends the build as a request on
|
|
the queue subject `$SWARM.builder.build` and waits up to 61 minutes for the
|
|
reply.
|
|
3. The builder runs `nix build` of `nix/builder/check-flake`'s `check` output
|
|
with every input overridden:
|
|
- `hyperhive` and `nixpkgs`: the sources the builder's own host was
|
|
deployed with, the same two a hive on that host renders into its meta
|
|
flake (`c0re.hyperhiveFlake`, `c0re.nixpkgsFlake`).
|
|
- `agent`: `git+https://<forge>/agent-configs/<agent>?ref=refs/pull/<pr>/head&rev=<head>`.
|
|
A build has one hour.
|
|
4. The reply is the system closure's store path, or the last lines `nix`
|
|
printed. The node writes the status on the head commit:
|
|
|
|
| Outcome | Status | Description | Node |
|
|
| ----------------------------------------- | --------- | --------------------------- | ------ |
|
|
| built | `success` | the store path | done |
|
|
| `nix` failed or timed out | `failure` | the last line `nix` printed | failed |
|
|
| no builder listening, or no reply in time | `error` | why | failed |
|
|
|
|
The full tail is in the failed node's error on the Jobs page and in the
|
|
builder's journal line for the build.
|
|
|
|
## What the check is, and isn't
|
|
|
|
- It composes the config the way a hive's meta flake does
|
|
(`agent-base.extendModules` with the config's `nixosModules.default`), minus
|
|
what a hive adds: its CA bundle, docs source, `claude-code` path, ports and
|
|
OTEL settings, and the `follows` that point the config's own inputs at the
|
|
hive's. A green check can therefore still fail on a hive, and the reverse.
|
|
- The agent base is the builder's host's, not the hive's. Hives on another
|
|
hyperhive or nixpkgs pin build something else.
|
|
- The builder pushes nothing and records nothing. The closure stays in the builder
|
|
host's store until it's collected, so on a host that also runs the hive, the
|
|
hive's own build can reuse whatever matches.
|
|
- It doesn't gate anything. A merge deploys whatever the status says.
|
|
- A lost webhook delivery isn't retried: the head isn't built until the next
|
|
push. The jobs graph is in memory, so a controller restart drops a build in
|
|
flight and leaves its status `pending`.
|
|
|
|
## Credentials
|
|
|
|
| What | Where | Who reads it |
|
|
| ----------------------- | ------------------------------------------------- | ---------------------------------------- |
|
|
| store leaf | `deploy.bao.builderClient{Cert,Key}File` | nspawn, into the service as a credential |
|
|
| queue client secret | `secret/swarm/services/swarm-builder/oidc/client` | the builder, at start |
|
|
| forge token (read-only) | `secret/swarm/services/builder/forge/token` | the builder, on every build |
|
|
|
|
The token belongs to the `builder` forge account, which swarm-controller
|
|
creates, puts in the `hives` team of `agent-configs` (read on every config
|
|
repo) and mints a `read:repository` token for, in a five-minute pass. `git`
|
|
gets the token through a credential helper reading the child's environment:
|
|
it's in no argument, URL or lock, so no line `nix` prints can carry it into a
|
|
status. The same `git` fetches a config's own inputs, so the builder registers
|
|
the helper for the forge's URL only, and the helper answers only an `https`
|
|
request for the forge's host: any other host gets nothing. The builder holds no write credential; swarm-controller writes the
|
|
status with its own forge token. Details per path:
|
|
[`credentials.md`](credentials.md).
|
|
|
|
## Isolation
|
|
|
|
The container has its own network namespace on the bridge, as the CI runner
|
|
does, so a config's evaluation can't reach a host-loopback service. Evaluation
|
|
is pure and runs in the container. Builds go to the host's nix daemon as an
|
|
untrusted client: a config can set no daemon option, and its derivations build
|
|
in the host's sandbox. A fixed-output derivation still has the network, as it
|
|
does for any build on that host.
|