Watch
0
0
Fork
You've already forked hyperhive
0
Commit graph hyperhive/docs/swarm/builder.md
Author SHA1 Message Date
iris
0e77d61a9f swarm-ui: a builds page for the swarm builder's runs, and the status links to it
/builds lists the builder's recent builds newest first, one card each with how
it ended, and opens a failed build's nix tail. /builds?run=<id> opens that
build's card and scrolls to it. The swarm-builder commit status now links
there instead of to the raw JSON record.
2026-10-18 01:26:45 +02:00
atlas
d4714072a9 swarm-controller: record builder runs with nix's tail and link the status to them 2026-10-17 23:51:35 +02:00
atlas
bf848ba65f swarm-builder: refuse a build that waited too long for the slot
A request that waits behind another build, e.g. one a controller
restart orphaned, could start a full build after most of the
controller's 61-minute reply deadline was gone. The builder now waits
at most REPLY_DEADLINE - BUILD_DEADLINE - 30 s for the slot and
answers Failed, without building, past that. The 30 s covers the
forge token read, now bounded to 20 s, and the reply's trip back.

Fixes #5437
2026-10-17 23:30:13 +02:00
atlas
e9c471caeb swarm-builder: give the check flake the hive's docs source
The builder's hyperhive input is the docs-stripped flake source, so an
agent config with docs.enable = true failed to evaluate: docs.source
defaults to reference-docs, built from the missing docs/. Pass the host's
c0re.hyperhiveDocs as a hyperhive-docs input and set docs.source from it,
as the hive's meta flake does.

Fixes #5411
2026-10-17 13:39:42 +02:00
atlas
51035eac1d archetypes: prefill the create form from flake-declared agent archetypes
A flake declares agent archetypes in its `hyperhiveArchetypes` output,
`<name> = { module; meta; }`. The hyperhive flake ships `reviewer`
(default permissions without `execution`).

swarm-builder answers `$SWARM.builder.archetypes` with the `meta` of
every archetype its own hyperhive source declares. swarm-controller asks
once the queue connects and every 10 minutes, checks each archetype as
`POST /api/agents` checks its settings, and stores the result in the
`agent-archetypes` KV bucket when it changed. `GET /api/archetypes`
serves it, and the create form's archetype select prefills permissions,
resource caps and inference from it.

The agent created is an ordinary agent with its own config repo; the
archetype's module is not applied yet.

Refs #5386
2026-10-17 12:57:20 +02:00
atlas
e873195cf6 swarm-builder: keep only the tail of nix's output while it runs
nix_build::run read nix's stdout and stderr with read_to_end, so a long
--print-build-logs run held its whole log in memory until nix exited,
though the reply uses only the last 30 lines (4000 bytes) of stderr and
the last non-empty line of stdout. Both streams now go through a Tail
that holds at most 128 KiB and keeps the last 64 KiB. Output under 64 KiB
reaches tail() whole, so those replies are unchanged.
2026-10-17 08:45:42 +02:00
atlas
6d7fc0dd99 docs: builder.md says the client secret is re-read on refusal too
The builder reads its queue client secret at start and again when
authelia refuses it. The credentials table in builder.md and the
swarm-builder README still said it was read only at start.
2026-10-14 03:30:46 +02:00
atlas
cafa120d77 swarm-builder: scope the forge token to the forge, one build at a time, opt-in
argus's review on #5177: the git credential helper answered every `get`,
so a config PR's own flake inputs could have git send the builder's forge
token to any host. The helper is now configured under
`credential.<forge-url>.helper` and itself answers only an https request
whose host is the forge's; a test runs it as git does and checks that
other hosts, http and non-get operations get nothing.

Builds are bounded: every `build_agent` node needs the one `builder` jobq
resource, so one runs at a time and the rest wait pending, and a newer
head cancels its PR's pending build (a running one finishes and writes on
its own commit). The builder also holds a one-permit semaphore around each
build.

`builder` and `review-classifier`, the forge logins swarm-controller owns,
join nix/reserved-names.nix, with a test tying the list to both constants.

The builder is now opt-in like the CI runner: `allSwarmServices` no longer
turns it on, since it runs unreviewed config PRs through the host's nix
daemon. mara had not answered atlas's question on the PR.
2026-10-14 03:30:46 +02:00
atlas
b68b74d687 swarm-builder: build each agent config PR head and post a commit status
A new `swarm-builder` service, in its own container on the host that sets
`deploy.builder.enable` (defaulted on by `allSwarmServices` where the host
has the builder's store leaf, like the review classifier).

When an `agent-configs/<agent>` PR is opened, reopened or synchronized,
swarm-controller queues a `build_agent` jobq node. The node sets a
`swarm-builder` commit status to pending on the PR head, sends
`{agent, rev, pr}` to the builder over NATS request-reply
(`$SWARM.builder.build`), and writes success (store path), failure (last
line nix printed) or error (no builder / no reply) on the head. The node
fails when the build does, so the Jobs page matches the PR.

The builder builds nix/builder/check-flake's `check` with every input
overridden: hyperhive and nixpkgs to the host's own
`c0re.hyperhiveFlake`/`c0re.nixpkgsFlake` (what a hive there renders into
its meta flake), and the agent to the PR head on the forge. Evaluation is
pure and runs in the container; builds go to the host nix daemon as an
untrusted client. The private config repo is fetched with the token of a
new read-only `builder` forge account (in the `hives` team), which the
controller mints into `swarm/services/builder/forge/token`; git gets it
from a credential helper reading the child's environment, so it is in no
argument or URL.

Credentials: a `builder` store leaf minted by glue-bao-tls on the store
host, a `swarm-builder` reader policy (forge token + its own OIDC client
secret), an authelia machine client, and a `--builder-client` grant in the
queue's auth-callout responder. The controller gains a grant on the token
path and the secret publisher a deny on it.

This is the agent-config build check atlas scoped on #5081 (slice 3):
nothing is pushed to a cache, nothing is written to KV, and a red status
gates no deploy. The check composes agent-base + the config without the
values a hive adds (CA, docs, claude-code path, ports, input follows), so
it can disagree with a hive's own build.
2026-10-14 03:30:46 +02:00