Each container now runs the journal forwarder the queue's container runs,
pushing to a receiver of its own on the swarm collector as the service's
existing OIDC client, with the client's secret read inside the container
by the fetch unit. Their store identities are system credentials, so the
fetch unit loads them too. The three containers drop --link-journal=host,
which hid their journals from the forwarder, and the host collector stops
reading them.
`swarm-bao-pki` was a `RemainAfterExit` oneshot. Once active, nothing
re-ran it: a switch restarts it only when its definition changes, and a
`Requires=`/`Wants=` on an active unit is satisfied without running it.
On muede-lpt2 the 10:57Z switch that added the builder line did change
the unit, so switch-to-configuration stopped it in its stop phase. A
start of `container@swarm-bao` 4 s later (before the daemon-reload at
10:57:35Z) pulled it back in through `swarm-bao-certs` with the OLD
definition, which had no builder line: 48 ms, nothing missing. The start
phase then found it active and did nothing. The 20:14Z switch that
enabled the builder left the unit unchanged, so it was never re-run, and
`container@swarm-builder` crash-looped on the absent `builder.pem`.
The unit now goes inactive after each run. Every switch starts
`multi-user.target`, which runs it again post-reload with the current
definition, and every unit that wants or requires it runs it on its own
start. Each line is idempotent on presence, so an existing leaf is never
re-signed.
The three consumers started from `machines.target` rather than
`multi-user.target` also want it, since `After=` alone does not order
against a job in another transaction: the builder's and the review
classifier's containers (nspawn `--load-credential`) and
`swarm-nats-auth-secrets` (pulled in by the queue's container).
Fixes#5284
deploy.builder.enable defaults on under deploy.allSwarmServices (mkDefault),
where the host has the builder's store identity pair, as for the review
classifier. An explicit false still wins.
argus's review on #5177: the git credential helper answered every `get`,
so a config PR's own flake inputs could have git send the builder's forge
token to any host. The helper is now configured under
`credential.<forge-url>.helper` and itself answers only an https request
whose host is the forge's; a test runs it as git does and checks that
other hosts, http and non-get operations get nothing.
Builds are bounded: every `build_agent` node needs the one `builder` jobq
resource, so one runs at a time and the rest wait pending, and a newer
head cancels its PR's pending build (a running one finishes and writes on
its own commit). The builder also holds a one-permit semaphore around each
build.
`builder` and `review-classifier`, the forge logins swarm-controller owns,
join nix/reserved-names.nix, with a test tying the list to both constants.
The builder is now opt-in like the CI runner: `allSwarmServices` no longer
turns it on, since it runs unreviewed config PRs through the host's nix
daemon. mara had not answered atlas's question on the PR.
A new `swarm-builder` service, in its own container on the host that sets
`deploy.builder.enable` (defaulted on by `allSwarmServices` where the host
has the builder's store leaf, like the review classifier).
When an `agent-configs/<agent>` PR is opened, reopened or synchronized,
swarm-controller queues a `build_agent` jobq node. The node sets a
`swarm-builder` commit status to pending on the PR head, sends
`{agent, rev, pr}` to the builder over NATS request-reply
(`$SWARM.builder.build`), and writes success (store path), failure (last
line nix printed) or error (no builder / no reply) on the head. The node
fails when the build does, so the Jobs page matches the PR.
The builder builds nix/builder/check-flake's `check` with every input
overridden: hyperhive and nixpkgs to the host's own
`c0re.hyperhiveFlake`/`c0re.nixpkgsFlake` (what a hive there renders into
its meta flake), and the agent to the PR head on the forge. Evaluation is
pure and runs in the container; builds go to the host nix daemon as an
untrusted client. The private config repo is fetched with the token of a
new read-only `builder` forge account (in the `hives` team), which the
controller mints into `swarm/services/builder/forge/token`; git gets it
from a credential helper reading the child's environment, so it is in no
argument or URL.
Credentials: a `builder` store leaf minted by glue-bao-tls on the store
host, a `swarm-builder` reader policy (forge token + its own OIDC client
secret), an authelia machine client, and a `--builder-client` grant in the
queue's auth-callout responder. The controller gains a grant on the token
path and the secret publisher a deny on it.
This is the agent-config build check atlas scoped on #5081 (slice 3):
nothing is pushed to a cache, nothing is written to KV, and a red status
gates no deploy. The check composes agent-base + the config without the
values a hive adds (CA, docs, claude-code path, ports, input follows), so
it can disagree with a hive's own build.
