A store answers the revision it wrote and what it replaced, and restoring
compares against that revision, so a write that landed after the creation's
own is kept instead of overwritten. A write that changed nothing rolls back
nothing, and a key that held no value is removed rather than set to an empty
one. A test fails the second store after the first and checks the earlier
caps come back.
create_agent puts back the caps and inference settings it replaced if a
later store or the job insert fails, so a refused creation leaves nothing
behind for the name. The shared field components take an id prefix so the
create dialog and an open editor never share a DOM id. The queue check now
follows the hive roster check, the 422 is documented, and the copy says
empty rather than closed and that destroy keeps stored settings.
POST /api/agents takes optional limits and inference objects, checked by
the same rules as the two PUT routes before anything is queued and stored
ahead of the job graph. The create form gets collapsed sections for both,
reusing the fields of the detail-panel editors.
The three issue-report handlers awaited the forge bare, so a forge that
accepts a connection and never answers hung the request. Wrap each in
within_forge_deadline, as pulls.rs does, and answer 503 on overrun.
The minter parses `Request.agent` with `Ident::parse`, as the controller
does, before it makes a scratch directory, runs `claude auth login` or
touches the store, and answers `Failed` otherwise. The name becomes a
directory name and a store path in the one service that writes every
agent's credentials, so it is checked at this service's own boundary.
The doc states that the `operatorsOnly` authelia rule, not the
controller, keeps the routes to operators, and that a stale version record
waits for a relogin. The env-strip comment says it is defence in depth.
Refs #5268
A new swarm service, `swarm-claude-login`, in its own container on the host
that sets `deploy.claude-login.enable` (defaulted on by `allSwarmServices`
where the host has the minter's store leaf, like the review classifier).
- swarm-ui: an agent card showing "login needed" gets a "log in" button.
Its dialog starts the login, shows the URL claude printed, and takes the
pasted code.
- swarm-controller: `POST/DELETE /api/agents/{name}/claude-login` and
`POST /api/agents/{name}/claude-login/code`, behind the swarm UI's
authelia-gated `/api/`, relay the steps to the minter as requests on
`$SWARM.claude-login.request`.
- swarm-claude-login: runs `claude auth login` over piped stdio with HOME
and CLAUDE_CONFIG_DIR in a per-login scratch dir, scrapes the first
https URL, writes the code to stdin, and stores the `.credentials.json`
at `swarm/agents/<agent>/claude-credentials` with a KV v2 write, a new
version over any earlier one. The scratch dir goes on every exit path; a
login waits 15 minutes for its code.
- hive-agent: while parked in `wait_for_login`, reads that leaf every 30 s
under the agent's own cert and installs it only when its KV version is
newer than the one recorded in `<harness>/claude-credentials-version`.
An equal or older version is never put back, because claude rotates the
refresh token and the stored copy is stale after the first refresh.
- swarm-secret-client: `claude_login` paths and `SecretStore::read_versioned`.
- Credentials: a `claude-login` store leaf minted by glue-bao-tls, a
`swarm-claude-login` policy (create/update on
`swarm/agents/+/claude-credentials`, read on its own queue client
secret), an authelia machine client, and a `--claude-login-client`
grant in the queue's auth-callout responder. The unit runs as a static
system user.
The hive's own web-UI login is unchanged.
Refs #5268
Swarm::http() built a reqwest client with no timeout. A token refresh runs
under the exclusive flock on the login cache (kept: authelia rotates the
refresh token, and a second refresher presenting the used one revokes the
login), so an authelia that accepted the refresh and never answered held
that lock forever and every other `swarmctl --swarm` invocation blocked
behind it in flock.
The client now has a 5 s connect timeout and a 30 s total timeout, the
pair swarm-controller's own clients of swarm services use. No caller
streams a response (every gateway call and every login POST reads one
whole body), so a total bound fits all of them. The lock is unchanged; it
is now held at most for one bounded request.
Closes#5308
The wide diff's context drops from 200 to 25 lines, so a block's
enclosing gate stays in view when it opens within 25 lines above the
change (the forge's own diff carries 3). The fallback note, the doc and
the gate fixture (now 20 lines out, not 47) follow.
A failure reading the changed files (a forge error on a tree or blob, a
truncated tree, a non-base64 blob) or joining the diff task made the PR
read fail, so the review posted no verdict. It now falls back to the
forge's 3-line .diff with "reading the files failed: …" as the note.
render wrote each file's whole diff before wide compared it with
MAX_BYTES. It now writes through a writer that refuses once past the cap,
which stops similar's formatter mid-file, so one huge file costs about
MAX_BYTES of output, not its full diff.
The classifier read the forge's PR `.diff`, whose context is fixed at 3
lines. A block moved into a glue module lost its enclosing gate (a
`config = lib.mkIf …` 13 to 191 lines up in the misfired moves #5199,
#5200, #5201), so the glue restating that gate read as a new condition.
The controller now builds the diff itself: it lists the PR's changed files,
reads each at the merge base and at the head through the git tree and blob
API with the token it already holds, and diffs them with 200 lines of
context. It sends the forge's `.diff` instead when the PR changes more than
100 files, the forge names no merge base, or the wide diff is over 256 KiB,
and the posted review then says so with the reason.
Fixes#5227
The granting units (granterPolicyUnit's 17, swarm-bao-granter-role,
swarm-bao-controller-policy, swarm-bao-agent-pki) and swarm-bao-nats-tls
copy store-retry's restart budget by hand, are Type=oneshot, and set no
TimeoutStartSec, so systemd gives each attempt no start timeout. A store
that accepts a connection and never answers leaves the attempt running
forever: Restart=on-failure never fires, units ordered after it wait, and
swarm-bao-nats-tls's daily timer starts a unit that is still activating.
Each now sets TimeoutStartSec: 60 for the granting units and the
granter's role, 2min for controller-policy (it may generate the services
root), 5min for agent-pki and nats-tls (each may wait on a blocking
restart of a server in a container). module-eval-bao-grants and
module-eval-nats-tls assert the bound is finite.
The restart budget itself is unchanged; that is #5305.
Closes#5306
swarm-services-cert, swarm-services-cert-renew and hive-tls-resign are
oneshots, which systemd gives no start timeout, and the root-only
`ca/pem` curl set no connect or total timeout. A store that accepted the
connection and never answered held the attempt forever: no failure, so
no store-retry restart, and a hung renew unit swallowed every later
timer tick.
The curl now gives up after 10s to connect and 30s overall. Each unit
gets a TimeoutStartSec: 20min for the services-leaf pair (the script
waits up to 10min on its lock), 10min for the gateway leaf. The
`--no-block` comment no longer cites a TimeoutStartSec that did not
exist.
module-eval-hive-tls gains a case pinning both.
Closes#5303
The metrics vhost forwarded the store's read API (query, series, labels,
export, vmui, ...) with no credential, so anything that could reach
metrics.<swarm> could read every hive's and agent's metrics. mara ruled on
checked by an auth_request responder, and operators use their authelia
session.
- swarm-metrics-auth (new crate): auth_request responder on the metrics
store's host. Compares the presented bearer, in constant time, with the
one stored at swarm/services/<grafana client id>/metrics/bearer, read
under its own bao leaf and cached for 60s. Static user, the post-#5264
swarm-logs-auth unit shape.
- swarm-bao-grafana-metrics-bearer: on Grafana's host, creates that bearer
in bao if absent (create without update, Grafana's existing leaf) and
writes "Bearer <value>" into the container; the datasource sends it as
its Authorization header via $__file{}.
- The vhost: read locations take auth_request through a map (a bearer goes
to swarm-metrics-auth, anything else to authelia, no login redirect);
vmui takes an authelia session with the login redirect; forceSSL like
the logs vhost. The authelia subrequest location is renamed from
/__metrics_push_authz to /__metrics_authelia since it now serves reads.
- Authelia: the collector keeps every path; everyone else is refused the
collector's push and /metrics paths; group admins is admitted to the
rest; then deny.
- bao: a swarm-metrics-auth cert role and policy (read on the one leaf),
its leaf minted by glue-bao-tls, the CN reserved as a hive name, and
Grafana's grant gains create on the bearer leaf.
Fixes#3596
The retries covered about a minute, and the swarm-bao restart that lost
#5286's review lasted 3m19s. Each try now gets its own forge deadline,
a try with no answer by then is retried like a refused connection, and
the waits are 5s, 15s, 45s, 90s and 150s: six tries over about five
minutes.
The docs also said to request the review again after a post-failed run.
That does nothing while the account's earlier request is still pending,
so they now say to remove the request and then request it again.
A review request's verdict was lost when the post failed: on 2026-10-14
the classifier approved #5286, but its post hit swarm-bao mid-restart
(cert login: connection refused), the run was recorded post-failed, and
nothing on the PR showed it.
post() now tries the post up to four times, waiting 5s, 15s and 45s,
when it fails because the connection to the secret store or the forge
failed or timed out. Any other error fails at once, as before. The
verdict is computed once and reposted as is. All tries share the one
forge deadline the single post had, and the warn log carries the number
of tries.
network.md listed matrix among the shared-netns containers and the
private-netns list named only agents and the CI runner; it now says
every privateNetwork container keeps its own firewall. The authelia
module no longer calls the matrix container shared, and the hive-c0re
comment on having no homeserver fallback no longer names port 8008.
Part of #4815.
tuwunel bound `0.0.0.0:<httpPort>` in the host netns, so the homeserver
port was open on every host address, and the gateway proxied to it on
host loopback.
The container now sets `privateNetwork = true` and hangs off the hive
bridge with a DHCP lease, the same attach the CI runner, builder and
review classifier use (resolvconf on, `networking.nameservers` the bridge
IP). tuwunel listens only on `/run/hive-matrix/tuwunel.sock`, a host
directory (root:nginx 0770) bind-mounted into the container; its primary
group is nginx's gid and the socket is 0660. The gateway's `/_matrix/`
location proxies to the socket.
The static resolv.conf goes: it existed because resolvconf's host
tracking came up empty in the shared netns.
`swarm.matrix.httpPort` and `{swarm,deploy}.matrix.openFirewall` are
removed with a message: there is no host port to choose or open. The
`.well-known/matrix/*` fallback to `<hive>:<httpPort>` is dropped with
them; it only rendered with `gatewayHost = null`, which the homeserver's
SSO assertion already rejects.
Part of #4815.
The swarm lives in the swarm UI; this tab lists the hive's agents. The tab
is 4G3NTS at #agents, its pane and count pill follow, and the module that
renders the containers is agents.js. Comments and docs that named SW4RM or
swarm.js follow.
The build lifecycle used to be its own page with three sub-tabs. They are
now dashboard tabs (rebuild, meta, buildlogs) beside SW4RM and
P3RM1SS10NS: builds.js is a module tabs.js imports, so the dashboard's one
tab strip, /api/state snapshot and event stream (plus the meta_inputs_changed
and meta_update_running kinds) serve them, and the page, its bundle entry,
stylesheet and home tile are gone. The SW4RM banner's "view queue" link
points at the #rebuild tab, and the rebuild tab carries a count pill of the
rebuilds in flight.
An empty tab-count pill drew a "0": its display beat the UA [hidden] rule.
`swarm-bao-pki` was a `RemainAfterExit` oneshot. Once active, nothing
re-ran it: a switch restarts it only when its definition changes, and a
`Requires=`/`Wants=` on an active unit is satisfied without running it.
On muede-lpt2 the 10:57Z switch that added the builder line did change
the unit, so switch-to-configuration stopped it in its stop phase. A
start of `container@swarm-bao` 4 s later (before the daemon-reload at
10:57:35Z) pulled it back in through `swarm-bao-certs` with the OLD
definition, which had no builder line: 48 ms, nothing missing. The start
phase then found it active and did nothing. The 20:14Z switch that
enabled the builder left the unit unchanged, so it was never re-run, and
`container@swarm-builder` crash-looped on the absent `builder.pem`.
The unit now goes inactive after each run. Every switch starts
`multi-user.target`, which runs it again post-reload with the current
definition, and every unit that wants or requires it runs it on its own
start. Each line is idempotent on presence, so an existing leaf is never
re-signed.
The three consumers started from `machines.target` rather than
`multi-user.target` also want it, since `After=` alone does not order
against a job in another transaction: the builder's and the review
classifier's containers (nspawn `--load-credential`) and
`swarm-nats-auth-secrets` (pulled in by the queue's container).
Fixes#5284
builder.nix's bare, servicesHere and natsHere were exact copies of
shared.bareWithController, swarm-services-switch's swarmServicesHere and
nats-authelia's natsOldPath. The last two move into shared.nix; each file
rebinds them under its local name, so no case changes.
RESTORE_TIMEOUT bounded only retry_down's restore. The boot loop in main
(primary and secondaries) and bring_up_secondary_with_retry's backoff
retries called bring_up_account with no bound, so a restore that never
answered held up every account after it, and the retry loop that would
recover it never started. A hung primary also never reached the fatal Err
path, so systemd never restarted the daemon.
The boot loop moves out of main into bring_up_all, which takes the restore
as a closure (the same seam retry_loop already has) so a test can stand in
a restore that never resolves. Every restore, at boot, on a boot retry and
in retry_once, now goes through within_restore_timeout, so RESTORE_TIMEOUT
is applied in one place. A timeout is an Err naming the account: fatal for
the primary, retried for a secondary, as any other restore error.
bring_up_account had no caller left and is removed.
Closes#5266
The hive no longer sets an agent's CPU or memory cap: caps come from the swarm, else the hive-wide defaults. The container load tab was the only hive UI that set them, and its figures are already exported as OTEL metrics, so with the tab gone nothing wrote meta/resource-limits.json.
Removed: the file's reader and writer, the `hivectl agent set-limits` verb with its host-socket request, the `GET /api/container-resources` and `POST /api/resource-limits/{name}` dashboard endpoints, and the `cpu_quota`/`memory_max` fields of the container view that only the tab read. A leftover resource-limits.json is ignored.
The cpu, memory and disk figures are exported as OTEL metrics and graphed in Grafana, and the caps are set in the swarm UI, so the C0NT41N3R L04D tab on the C0R3 page only duplicated both. C0R3 is now the kept-state page alone, without a tab strip.
The socket is wanted by sockets.target and nothing orders the wireguard
unit before it, so the mesh address can be missing at bind time
(EADDRNOTAVAIL) and the receiver never listens. FreeBind lets the bind
succeed regardless, and also covers the interface coming back after a flap.
Also assert the mesh address is not a wildcard, which the comment above the
socket already claimed, and cover the rendered socket in a module-eval case.
Job-level timeout-minutes is parsed but not enforced by this runner; only
the runner's jobTimeout bounds a job. Also fixes the devshell coverage
comment, the shellcheck job comment, an option description citing a script
that is not in the repo, and comment file pointers that resolve to nothing.
A separate $SWARM.control.<agent> subject carries commands, apart from
the text subject. swarm-controller exposes POST /api/agents/{name}/cancel
as a NATS request; the harness answers whether a turn was actually
stopped, through the same cancel path as the dashboard button. The agent
credential gets subscribe on its control subject and allow_responses to
answer; only the controller may publish there.
Refs #3803
The review body's table is now: result | check | certainty / threshold |
reason. The result cell is a symbol for what the row did to the review,
after polarity (✅ counted and passed, ❌ counted and failed, ❔ not
counted or no answer), then the model's verdict. The overall line carries
✅ / ❌. Polarity, threshold and counted are folded into those cells.
The prompt asks for a reason under 350 characters; the enforced cap
stays 500, so an overshoot is truncated less often.
The queue's metrics were scraped over loopback from the exporter, declared
only where the queue and the collector share a host, so a collector on any
other host had no job for it. Its target is now declared on every host as
`https://<swarm.nats.domain>/metrics`, and the queue's host gets a gateway
vhost on that name serving `= /metrics` from the exporter behind the
collector's auth subrequest (`/__nats_metrics_authz`, the forge's shape).
Every other path there is 403; the monitoring endpoint is never proxied.
The rules derived from `publishedScrapeTargets` admit the collector's
client and deny every other subject. The loopback entry, and with it
`glue-otel-scrape-targets.nix`, is gone.
In self-signed mode the vhost serves the queue's own `swarm-nats` leaf
(issued for exactly that name, `server_flag` set), copied 0640
root:nginx by `swarm-nats-gateway-cert`; until the leaf exists that unit
writes a throwaway pair, so nginx never starts on a missing file. The
name stays out of `swarm.serviceDomains`, so a host whose only swarm
service is the queue fronts no services name and its gateway neither
waits for nor needs the swarm-services leaf: no services-issuer pair has
to be placed on it. With ACME or an operator cert the vhost uses the
gateway's own TLS, as every other name does.
The queue's host now enables the gateway (`mkDefault`), as every other
swarm service's host does.
Fixes#5205
deploy.builder.enable defaults on under deploy.allSwarmServices (mkDefault),
where the host has the builder's store identity pair, as for the review
classifier. An explicit false still wins.
The two review tabs, runs and try-out, are one: /review. It is a column of
three sections: try a classification and the classifier settings, each a
folded bar (the settings' bar says what they are now), then the recent runs,
always open. The settings had been pinned under the try-out form; they are now
a section of their own. A new Disclosure primitive is the folding bar, a
native details wearing the Panel's classes, so the folded and open sections
read as the same surface.
/review-runs and /review-tryout are gone, with no redirect.
Three follow-ups to argus's review of #5237:
- A down secondary's retry runs `restore` under RESTORE_TIMEOUT (2 min).
Tries run one per tick on one future, so a restore that never resolved
held back every other down account's retries for good; a try that
outlasts the timeout now counts as a failure like any other.
- `client::build_and_restore` warns about a rejected token only for the
primary, which exits right after. For a secondary it logs at debug: the
retry loop already warns once per outage (`DownAccounts::failed` is news
on the first failure and on a changed reason, and an account that came
up starts afresh), so the extra warn repeated on every try.
- `a_secondary_that_comes_up_late_does_not_wipe_the_todos` runs the late
bring-up in a child process of the test binary with HIVE_AGENT_SOCKET
pointed at the test's listener. In-process the variable was unset, so a
wipe placed back in the bring-up path was a no-op and the test passed.
New tests: a_second_rejection_in_a_row_does_not_warn_again (captured logs, alone
in a child process since tracing caches callsite interest process-wide;
a stub whoami answering M_UNKNOWN_TOKEN, the real build_and_restore),
a_failure_after_the_account_came_up_is_news_again, and
an_account_whose_restore_hangs_does_not_hold_back_the_others (paused time).
Grafana's own metrics were scraped over loopback through the
`grafana-metrics` nginx vhost, declared only where grafana and the
collector share a host, so a collector on any other host had no job for
it. Its target is now declared on every host as
`https://<swarm.grafana.domain>/metrics`, next to the other published
targets, and grafana's vhost serves `= /metrics` behind the collector's
auth subrequest (`/__grafana_metrics_authz`, the forge's shape). The
rules derived from `publishedScrapeTargets` admit the collector's client
there and deny every other subject.
The loopback scrape is gone with the vhost that served it and its
`swarm.grafana.metricsPort` option, so a single host still scrapes
grafana once, now by name. A config that sets that option now fails
evaluation with "option does not exist".
On the published vhost `/metrics` was answered with 403 for everyone.
It now answers the collector's token, and a plain session gets authelia's
`deny`. The design note that ruled the published vhost out for scraping
predates the collector's credential; it is replaced by the location's
own comment and a line on `swarm.grafana.domain`.
Part of #5205
Two comments said authelia's rule requires `group:operators`. The group
is `admins` (`operatorGroup` in swarm-authelia.nix), and the rule is now
an `operatorsOnly` pair.
The apex's `operatorsOnly` pair was guarded on
`deploy.swarm-controller.enable`, which is true only on the host that
runs the controller and with it the swarm UI's vhost. When authelia runs
on another host, that vhost still asks it, but the authelia host
rendered no rule for the apex at all, so every session reached the
swarm UI and its `/api/` through `default_policy = one_factor`.
The pair now renders whenever the swarm domain is set, like the bao UI's
pair. No vhost on a host that doesn't serve the apex asks about it, so
the extra rules there are inert. authelia 4.39.20's `validate-config`
accepts them on an authelia-only host's rendered access_control.
module-eval-swarm-otel-identity gains a case on the existing
`autheliaOnly` fixture (authelia without the controller): the apex has
exactly the allow+deny pair, the rule list is closed, and an account in
no group is refused `/api/agents` while `group:admins` passes.
Fixes#5120
