the agent's own root unit reads swarm/agents/<agent>/backend-env into /run and
creates it from backendEnvironmentFile with -cas=0 when the store holds nothing.
the shared agent policy gains create (not update) on that one leaf. the harness
and the subagent daemon load the old file first and the fetched one after it.
The folded-in secrets.md text sat after the reference's table, so the
reference was half prose. The overview now holds all the prose (where a
secret lives, topologies, the swarm- and hive-level notes, how a reader
reaches the store, the browser UI, adding a secret) and the reference holds
only the tables, each pointing at the section that explains it. No text
dropped; links that asked for the explanation now go to the overview, and
the ones that asked for a list or a row go to the reference.
secrets.md and credentials-reference.md answered the same question from two
pages. secrets.md's text now follows the reference's table and operating
notes as a 'Where each secret lives' section, unchanged apart from where it
sits. Every link and doc path that named secrets.md (docs, nix comments,
crate READMEs) points at credentials-reference.md; anchors are unchanged.
The overview keeps the store invariant, the mTLS exception and the
minter/reader/renewal contract. The table of every credential and the
operating notes (forge token checks, re-creating agents, revoking a queue
credential, store login retries) move unchanged to credentials-reference.md,
and the links that asked where a credential lives point there.
Every agent now connects to the queue and reads logs with its own secret
from the store (#5181), so the per-hive `hive-<h>-agent` OIDC client has
no reader left. Remove it end to end:
- hive-c0re no longer forwards `hive-queue-agent-{secret,client-id}` into
containers, and no longer forwards HIVE_AGENT_OIDC_TOKEN_ENDPOINT; the
agent module that loaded them (nix/agent-modules/queue.nix) is gone, its
options refused with a removal message.
- The host reader (glue-queue-agent-credential.nix, swarm-bao-queue-agent),
its store role and policy unit (swarm-bao-queue-agent-policy), its PKI
leaf and the deploy.bao.queueAgent* / queue.agentCredentialDir options
are gone; the options are refused with a removal message.
- authelia mints no `hive-<h>-agent` client, and swarm-secret-publish no
longer writes swarm/hives/<h>/queue/agent.
- swarm-secret-client drops queue::agent_client_path / queue::Credential.
- HIVE_AGENT_NATS_URL is now gated on the agents' queue address alone.
The log store's authelia check stays for operators. swarm-nats-auth still
refuses a `-agent` client id; that refusal and `agentClientSuffix` are left
for a follow-up.
Refs #4459
Refs #4765
- table cells: cut to what the credential is, where it comes from and what the operator does
- prose sections: cut the matrix-prefix, appservice-token and leaf-delivery sections; trimmed revocation and retry notes to their operator-visible effect
- progressive enhancement section: removed (contributor rules)
- intro: cut to the store, the on-disk certificate and the column meanings
- docs index: drop the "progressive-enhancement rule" clause from the credentials.md entry in docs/README.md
- matrix rows: one sentence saying the name `main` is reserved
A new swarm service, swarm-notify-router, in its own container on the host
that sets deploy.notify-router.enable (defaulted on by allSwarmServices where
the host has the router's store leaf, like the claude login minter).
- swarm-controller registers instance-wide hooks for `issues` +
`issue_assign` and `pull_request` + `pull_request_assign`, updates the
event list of a hook already registered with fewer (leaving its active flag as it is), and keeps each
verified body unchanged in the SWARM_EVENTS stream on
$SWARM.events.forge.<X-Forgejo-Event-Type>, deduped by X-Forgejo-Delivery.
A delivery the stream did not keep answers 503.
- swarm-notify-router drains its durable consumer on that stream, runs its
flows (source -> match -> todo sink, stored as nodes plus edges and checked
to be a tree) and queues each op on $SWARM.router.todo.<agent> in the
work-queue stream ROUTER_TODOS. Built-in flows: assigned, unassigned (same
key, rewritten) and closed (prefix-clears every holder's router todos). A
per-issue snapshot in its notify-router-state bucket holds the last
assignees and the todo holders; only agents in the agent-status bucket get
ops, and an assignee that was not one is left out of the snapshot so the
next assign event retries it. After the first event that must be retried,
the rest of the batch is nakked so redelivery keeps stream order.
- hive-agent creates its own durable consumer todo-<agent> and applies each
op to its todo store as subsystem `router`, then acks; a refused op stops
the rest of its batch the same way. The store's (subsystem, key) dedup
makes a redelivery harmless. New Todos::clear_prefix.
- swarm-nats-auth: a --notify-router-client arm; the controller may publish
$SWARM.events.forge.*; each agent gets three {agent} templates (extended
consumer create whose filter the server checks, MSG.NEXT, ACK).
- Credentials: a notify-router store leaf minted by glue-bao-tls, a
swarm-notify-router bao policy (read on its own queue client secret), an
authelia machine client, the secret publisher entry, a journal forwarder
route. No manual step.
hive-forge-notify is unchanged, so agents also still get forge's own
notifications.
Refs #5571
After a failed or timed-out store read the responder denies for LOOKUP_FAILURE_WINDOW (3 s) without a new lookup, so a burst of requests during an outage costs one lookup instead of one per request.
Three edits besides the code in main.rs belong to this change:
- swarm-metrics-auth/Cargo.toml gains tokio's test-util dev-dependency because the burst test runs on paused time (start_paused) to let a hanging store sleep past LOOKUP_TIMEOUT without a real 5 s wait.
- docs/swarm/services.md gains the sentence that a failed or timed-out read is denied for three seconds without a retry, which documents the window this change adds.
- docs/swarm/credentials.md gains the clause "every three seconds while the store fails" on the Grafana bearer row, which documents the same window in the secrets table.
The user JWT the callout issues carried no exp, so nats-server armed no
expiry timer and a connection kept its grant until the client dropped it:
a deleted or re-minted agent queue secret, a removed OIDC client or an
expired access token never reached a live connection.
Every issued user JWT now expires at the presented OIDC token's exp
(read from the introspection answer) or an hour after issue, whichever
is sooner; the agent queue-secret path has no exp of its own and gets
the hour. Every queue client reconnects through an auth callback that
re-reads or re-mints its credential, so the forced reconnect presents a
current one.
Fixes#5432
swarm-metrics reads the swarm's VictoriaMetrics from inside an agent
container: query, query-range, series, labels and label-values, each one
GET against the store's Prometheus read API through the gateway, printing
the store's JSON. It authenticates the way swarm-logs does, with the
agent's own queue secret read from bao under the agent's leaf, sent as
a swarm-agent bearer.
On the metrics store's host the read API's auth map gains an agent arm
ahead of the Grafana one. It routes to swarm-metrics-agent-auth, a
swarm-logs-auth instance logged in under the existing swarm-metrics-auth
leaf, whose bao grant now includes swarm/agents/+/queue read. The check is
rate-limited per address like the log store's
(deploy.victoriametrics.agentAuth{Rate,Burst,VerdictTtl}).
nix/agent-modules/metrics.nix installs the wrapped binary when the hive
forwards HIVE_AGENT_METRICS_URL (from swarm.victoriametrics.domain) and a
store address, the same gate as swarm-logs. Adds the base:swarm-metrics
skill and the CLI reference with its freshness check.
Fixes#5395
swarm-secret-publish and its path unit now run inside authelia's container,
beside the mint, and read each client secret at the path authelia writes it.
The host unit and host path unit, which read the secrets through the
container's root on the host, are gone, and so is
deploy.authelia.hostClientSecretDir, which only they read. The publisher keeps
its store identity, the secret-publisher leaf. The host hands the leaf and the
store's CA into the container as nspawn system credentials, and the unit loads
them; where this host mints the leaf, the container waits for swarm-bao-pki.
Authelia does not wait for the publisher, which retries every 30 seconds while
the store is down.
The container also runs the journal forwarder the other service containers
run, pushing to a receiver of its own on the swarm collector (serviceProducers
swarm-authelia) as a new machine OIDC client (swarm.authelia.otel.clientId)
that authelia registers for itself and the publisher carries to the store. The
fetch unit reads it back under the publisher's leaf, also handed to it as a
system credential, so the swarm-secret-publisher policy gains read on that one
leaf; its stanza keeps create and update, which the exact path would otherwise
take from the services/* prefix. Without the leaf the forwarder is off.
The container drops --link-journal=host, and the host collector stops reading
it. The services' "nothing at <path>" errors name the publisher in authelia's
container.
A swarm-ci-metrics host unit on a 5-minute timer lists every forge
repo's Actions runs as the new read-only ci-metrics account and pushes
each finished run's duration and queue wait over OTLP, stamped at the
run's stop time. The first pass backfills from run 1. swarm-controller
mints the account's token into bao and keeps a ci-metrics team with
read on the Actions unit in every organisation.
Closes#5376
`forgejo-swarm-controller-account`, in the forge's container, now puts
the swarm-controller's forge token into the swarm secret store itself,
under the forge publisher's leaf. The container already receives that
leaf as nspawn system credentials for its journal forwarder; the mint
loads the same three. The token reaches bao on stdin (`value=-`) and
the forge's API through a header file, never as an argument, and never
touches disk.
The mint's guard is a marker holding the published token's name, at
/var/lib/forgejo/swarm-controller-token-name, written last: after the
put, and after every other `swarm-controller-boot-*` token of the
account is revoked through forgejo's admin API, authenticated as the
new token. So a put that failed, or a marker lost while the store kept
an older token, leaves no live token behind once a mint completes. The
mint logs in to the store and checks that the account is a site admin
before it mints, so a store that is down or an account the admin API
would refuse mints nothing on each retry. Failures retry under
store-retry; forgejo does not wait for the mint. No bao read and no
grant change: the forge publisher keeps create/update on the token.
Removed: the host's `swarm-bao-forge-publisher` service and path unit,
which read the token file through the container's root, and their
entries in the store and policy ordering glue. The forge publisher's
policy unit stays; `swarm-bao-forge-oidc` still uses the leaf.
On deploy, an existing forge host has no marker, so the mint runs once:
it mints a new token, publishes it, revokes the old one, and deletes
the old /var/lib/forgejo/swarm-controller-token. The controller's next
forge call gets a 401, re-reads the store and carries on.
The metrics store's container now runs the journal forwarder the other
service containers run, pushing to a receiver of its own on the swarm
collector (serviceProducers swarm-victoriametrics) as a new machine OIDC
client (swarm.victoriametrics.otel.clientId), registered by a glue file
wherever authelia runs and published to the store by the secret publisher.
The container keeps its loopback-only netns. The collector cannot dial a
unix socket, so swarm-victoriametrics-egress-relay, a socket-activated
systemd-socket-proxyd inside the container, takes its connections on
127.0.0.1:4380 and passes them to /run/swarm-victoriametrics-egress/egress.sock.
The host's nginx listens there and sends /swarm-victoriametrics/ to the swarm
collector and /api/oidc/token to authelia, both by name and verified, and
refuses every other path. nginx gets ReadWritePaths on that directory and
removes a stale socket before it starts.
The container cannot reach the store, so the client secret is fetched on the
host by swarm-victoriametrics-forwarder-oidc under swarm-metrics-auth's leaf,
the shape swarm-bao-forwarder-oidc has, and bound into the container
read-only. The forwarder module gains hostSecretFile for that: set, it renders
no fetch unit. swarm-metrics-auth's policy gains one read stanza on the
forwarder's client path.
The container drops --link-journal=host, and the host collector stops reading
it.
Also lists the forge publisher's and matrix-ctl's leaves in the forwarder
secrets row of docs/swarm/secrets.md, which the hive-forge/hive-matrix change
missed.
Each container now runs the journal forwarder the CI runner's, Grafana's and
the log store's containers run, pushing to a receiver of its own on the swarm
collector (serviceProducers hive-forge and hive-matrix). Each authenticates as
a new machine OIDC client of its own (swarm.forge.otel.clientId,
swarm.matrix.otel.clientId), registered by a glue file wherever authelia runs,
minted by authelia and published to the store by the secret publisher. The
forge's and the homeserver's browser login clients are unchanged. The client
ids are swarm-forge-collector and swarm-matrix-collector, not
hive-<x>-collector, which is the client id a hive named <x>-collector gets.
The fetch unit inside each container reads the client secret with the store
identity the host already holds for that service. The forge publisher's leaf
is handed into the forge container as an nspawn system credential, as the
grafana and log store containers do. matrix-ctl's leaf is already bound into
the homeserver's container, so its forwarder reads it at those paths, as
swarm-matrix-appservice-publish does; with no matrix-ctl identity the
homeserver's forwarder is off. Each of the two policies gains one read stanza
on its forwarder's client path. The host units that use those identities are
unchanged.
The two containers drop --link-journal=host, and the host collector stops
reading them.
Each container now runs the journal forwarder the queue's and the builder's
containers run, pushing to a receiver of its own on the swarm collector. Each
authenticates as a new machine OIDC client of its own
(swarm.<service>.otel.clientId), registered by a glue file wherever authelia
runs, minted by authelia and published to the store by the secret publisher.
Grafana's browser login client is unchanged.
The fetch unit inside each container reads the client secret with the store
identity the host already holds for that service (the runner's, Grafana's,
and the log store responder's), now also handed into the container as an
nspawn system credential. Each of those three policies gains one read stanza
on its forwarder's client path. The host units that use those identities are
unchanged.
The three containers drop --link-journal=host, and the host collector stops
reading them.
credentials.md, services.md and secrets.md each started with text that
belongs elsewhere, ahead of the heading: a cell fragment glued onto the
heading line, and two table rows. Put each back in its table or cell.
secrets.md was already broken on main (the swarm-builder row of the
per-unit table sat above the heading); the other two came in with this
branch.
Grafana's VictoriaLogs datasource now targets https://logs.<swarm> and sends
the same static bearer as the metrics datasource. The log store's vhost routes
a 64-hex Authorization bearer to a second swarm-metrics-auth instance
(swarm-logs-grafana-auth, cert role swarm-logs-auth, which gains a read on the
bearer's store path). swarm-agent. tokens and everything else keep their routes.
Part of #3742 (DoD 3, logs).
swarm-bao-ci-runner still reads the token from bao under the runner's own
ci-runner leaf, but now writes it to /run/swarm-ci-runner on the host
(tmpfs), which the container bind-mounts read-only for the register
unit's LoadCredential. Before, it wrote the token to
/var/lib/nixos-containers/swarm-ci-runner/var/lib/swarm-ci-runner/registration-token
on persistent disk, where a superseded token stayed after rotation and
went into backups of the container root. The container's tmpfiles now
remove that old file.
Rotation is unchanged: the ten-minute timer rewrites the file when the
value differs and refresh_consumer restarts the runner. After a reboot
the token is fetched again; until the store answers, a runner that is
already registered keeps running and only a new registration waits.
Part of #4998 (slice 1).
swarm-bao-grafana-controller-token logs in to the store with Grafana's
leaf, like its three Grafana siblings, and glue-bao-readers-store-order.nix
already orders it after the store's container and PKI unit. The
services-switch case that pins that ordering did not list it, so dropping
it from the glue would have passed. It is now in the roster, and the count
of readers rendered off the store's host goes from 7 to 8: the unit exists
wherever Grafana does.
credentials.md gets the row for its store path, beside the controller's
forge token.
A new swarm service, `swarm-claude-login`, in its own container on the host
that sets `deploy.claude-login.enable` (defaulted on by `allSwarmServices`
where the host has the minter's store leaf, like the review classifier).
- swarm-ui: an agent card showing "login needed" gets a "log in" button.
Its dialog starts the login, shows the URL claude printed, and takes the
pasted code.
- swarm-controller: `POST/DELETE /api/agents/{name}/claude-login` and
`POST /api/agents/{name}/claude-login/code`, behind the swarm UI's
authelia-gated `/api/`, relay the steps to the minter as requests on
`$SWARM.claude-login.request`.
- swarm-claude-login: runs `claude auth login` over piped stdio with HOME
and CLAUDE_CONFIG_DIR in a per-login scratch dir, scrapes the first
https URL, writes the code to stdin, and stores the `.credentials.json`
at `swarm/agents/<agent>/claude-credentials` with a KV v2 write, a new
version over any earlier one. The scratch dir goes on every exit path; a
login waits 15 minutes for its code.
- hive-agent: while parked in `wait_for_login`, reads that leaf every 30 s
under the agent's own cert and installs it only when its KV version is
newer than the one recorded in `<harness>/claude-credentials-version`.
An equal or older version is never put back, because claude rotates the
refresh token and the stored copy is stale after the first refresh.
- swarm-secret-client: `claude_login` paths and `SecretStore::read_versioned`.
- Credentials: a `claude-login` store leaf minted by glue-bao-tls, a
`swarm-claude-login` policy (create/update on
`swarm/agents/+/claude-credentials`, read on its own queue client
secret), an authelia machine client, and a `--claude-login-client`
grant in the queue's auth-callout responder. The unit runs as a static
system user.
The hive's own web-UI login is unchanged.
Refs #5268
The metrics vhost forwarded the store's read API (query, series, labels,
export, vmui, ...) with no credential, so anything that could reach
metrics.<swarm> could read every hive's and agent's metrics. mara ruled on
checked by an auth_request responder, and operators use their authelia
session.
- swarm-metrics-auth (new crate): auth_request responder on the metrics
store's host. Compares the presented bearer, in constant time, with the
one stored at swarm/services/<grafana client id>/metrics/bearer, read
under its own bao leaf and cached for 60s. Static user, the post-#5264
swarm-logs-auth unit shape.
- swarm-bao-grafana-metrics-bearer: on Grafana's host, creates that bearer
in bao if absent (create without update, Grafana's existing leaf) and
writes "Bearer <value>" into the container; the datasource sends it as
its Authorization header via $__file{}.
- The vhost: read locations take auth_request through a map (a bearer goes
to swarm-metrics-auth, anything else to authelia, no login redirect);
vmui takes an authelia session with the login redirect; forceSSL like
the logs vhost. The authelia subrequest location is renamed from
/__metrics_push_authz to /__metrics_authelia since it now serves reads.
- Authelia: the collector keeps every path; everyone else is refused the
collector's push and /metrics paths; group admins is admitted to the
rest; then deny.
- bao: a swarm-metrics-auth cert role and policy (read on the one leaf),
its leaf minted by glue-bao-tls, the CN reserved as a hive name, and
Grafana's grant gains create on the bearer leaf.
Fixes#3596
The daemon's two-minute refresh compared the store's linked account
names with the names it booted with. A secondary that was named but did
not come up - no token at bring-up, a token the homeserver rejected
(M_UNKNOWN_TOKEN), or a homeserver still down after the ~52 s boot
retries - kept its name in the set, so nothing ever noticed it again:
it stayed down until the daemon restarted for some other reason, and
an operator re-linking a rejected account changed no name.
Each such secondary is now tried again on the same two-minute period,
one per tick, after one refresh and then doubling waits capped at
sixteen minutes. One that comes up joins the registry (now behind a
lock) and its sync loop joins the running set; the primary and the
other accounts are not restarted. The store is still only read.
The startup todo wipe moves out of bring_up_account to run once before
any sync loop: run per account, a late secondary would have cleared the
todos the running accounts' sweeps had already deduped.
Part of #5017 (slice M).
The controller read its forge token from the swarm secret store once, when
its client slot filled, and kept sending it until the daemon restarted. A
token the forge re-minted or revoked left every forge-backed path failing
with 401 until someone restarted the controller, and a restart drops the
runtime-only job graph.
Every forge request now goes through `forge::api::Api::call`. On a 401 it
asks a `StoredSecret` (the queue clients' debounced, single-flight re-read
with backoff) for a token to try instead, rebuilds the forgejo client with
it, and sends the request once more. A second 401, or a store still holding
the refused token, surfaces the error as before. Only 401 counts: unlike an
IdP, the forge answers 400 to a malformed request, and a re-read started by
one would put a real 401 that follows inside the backoff.
forgejo-api bakes the token into its own reqwest client's default headers
and takes no client or middleware from outside, so there is no single
request function to hook. Each `self.api.<endpoint>(..)` site becomes
`self.api.call(|api| api.<endpoint>(..).into_future())`; option structs a
site moved into the call are cloned, since the closure can run twice. The
`OnceLock<Arc<Client>>` slot and its `get()` callers are unchanged.
Refs #5017
The swarm settings gain `runtime` (`api-key`, the default, or `acp`) and
an `acp` provider record: an https base URL, the provider's model, and
its prices per million tokens. A PUT now changes only the fields its
body names, so the settings form that sends model, effort and limit
leaves the runtime alone; `null` clears `effort` or `acp`. The body stays
strict, and the stored document keeps #5192's lenient read.
The controller sends the runtime in each run request. On `acp` the
classifier runs one prompt on a new `opencode acp` session per
classification through hive-runtime's AcpRuntime, in a scratch
directory, with a config rendered from the settings. opencode runs
`read` and other tools without asking, so OPENCODE_PERMISSION
{"*":"deny"} takes every tool away from the model, and a deny-all policy
refuses any request on top. The title call, which resends the prompt,
is turned off. The deadline cancels the turn, and a turn that does not
end within 15s of that is dropped, which also covers an agent stuck
before its prompt. The answer is the final assistant text, as ACP has
no result event.
The provider key is read per run from
swarm/services/review-classifier/acp-provider, which the bao UI's
admins login may create and update, and reaches opencode through
AcpRuntime::with_provider_key and the trampoline's allowlist.
Each ACP classification's cost is computed from the configured prices
and the reported tokens, and added to
hyperhive.review_classifier.cost.usage labelled repo, pr, model and
currency.
Part of #5096
A new `swarm-builder` service, in its own container on the host that sets
`deploy.builder.enable` (defaulted on by `allSwarmServices` where the host
has the builder's store leaf, like the review classifier).
When an `agent-configs/<agent>` PR is opened, reopened or synchronized,
swarm-controller queues a `build_agent` jobq node. The node sets a
`swarm-builder` commit status to pending on the PR head, sends
`{agent, rev, pr}` to the builder over NATS request-reply
(`$SWARM.builder.build`), and writes success (store path), failure (last
line nix printed) or error (no builder / no reply) on the head. The node
fails when the build does, so the Jobs page matches the PR.
The builder builds nix/builder/check-flake's `check` with every input
overridden: hyperhive and nixpkgs to the host's own
`c0re.hyperhiveFlake`/`c0re.nixpkgsFlake` (what a hive there renders into
its meta flake), and the agent to the PR head on the forge. Evaluation is
pure and runs in the container; builds go to the host nix daemon as an
untrusted client. The private config repo is fetched with the token of a
new read-only `builder` forge account (in the `hives` team), which the
controller mints into `swarm/services/builder/forge/token`; git gets it
from a credential helper reading the child's environment, so it is in no
argument or URL.
Credentials: a `builder` store leaf minted by glue-bao-tls on the store
host, a `swarm-builder` reader policy (forge token + its own OIDC client
secret), an authelia machine client, and a `--builder-client` grant in the
queue's auth-callout responder. The controller gains a grant on the token
path and the secret publisher a deny on it.
This is the agent-config build check atlas scoped on #5081 (slice 3):
nothing is pushed to a cache, nothing is written to KV, and a red status
gates no deploy. The check composes agent-base + the config without the
values a hive adds (CA, docs, claude-code path, ports, input follows), so
it can disagree with a hive's own build.
An agent's matrix encryption keys lived only in its local crypto store, so
`destroy --purge` or a primary-account token reset lost every encrypted
room's history with no way back (#5022).
swarm-controller now mints a random passphrase (32 bytes from the kernel's
CSPRNG, base64url) for every account listed under each agent's `matrix/`
directory, at `swarm/agents/<agent>/matrix-recovery/<account>`, in a
five-minute pass. The write is a cas=0 create after a read; a path that
holds anything is never replaced, and a create that loses a race counts as
done. The controller's existing `data/swarm/agents/*` grant covers the
write and the agent's own `agents/<agent>/*` read grant covers the read, so
no policy changes.
hive-matrix-daemon reads the passphrase when it brings an account up, after
cross-signing: with no backup on the homeserver it enables recovery with it
(backup + secret storage); with one there it recovers, which imports the
backup key and the self-signing key, and room keys then download through
`BackupDownloadStrategy::OneShot`. No passphrase yet logs at INFO and the
account comes up without backup; any failure logs WARN and never stops the
daemon.
Recovery runs after `ensure_cross_signing`, not before: on a new account
cross-signing must exist locally for `enable` to export it into secret
storage, and on an existing identity `bootstrap_cross_signing_if_needed` is
a no-op, leaving `recover` to import and sign.
Part of #5022 (S1+S2, with the S4 docs).
Any request on logs.<swarm> carrying a well-formed swarm-agent bearer
made swarm-logs-auth do a bao cert login and KV read before it could
refuse, with no cap on the rate.
nginx now applies limit_req (zone swarm_logs_agent_auth, keyed by
client address, 5r/s burst 20 nodelay by default) on the internal
location that forwards to swarm-logs-auth, answering 403 when over,
since auth_request passes through only 401/403 and swarm-logs retries
a 401. The authelia path and the query location carry no limit.
swarm-logs-auth remembers bearers it has granted, keyed by SHA-256 of
the bearer, for --verdict-ttl seconds (deploy.victorialogs.authVerdictTtl,
60 by default), holding at most 4096 and evicting the oldest. Denials
are never remembered. A replaced or deleted queue secret therefore
keeps working at the log store for up to the TTL after its last check.
Closes#5190
hive-c0re (`hive-<name>`), swarm-controller and swarm-review-classifier
read their queue client secret from the store once and held it as
`ClientSecret::Value` for the life of the process. After authelia
re-minted a secret and swarm-secret-publish stored the new one, every
later token mint presented the old value and was refused until the
daemon restarted.
`ClientSecret::Stored` replaces `Value`: a `StoredSecret` holds the
secret in memory with a caller-supplied read callback, so the crate
takes no store dependency. When the token endpoint answers 400 or 401,
`mint_token` (and its blocking sibling, used by the controller's OTLP
exporters) re-reads it and, if the store holds a different secret,
mints once more with it. One re-read runs at a time and is spawned so
it outlives a caller that gives up. A re-read that fails or finds the
refused value delays the next one, 5 s doubling to 5 min, so a secret
that stays wrong costs one store read per delay. Logs name the client
id and never the secret.
swarm-nats-auth's `oidc_secret` was the template; its re-read machinery
now sits on `StoredSecret`, so the tree has one copy. The responder
gains the same re-read backoff.
The controller keeps its `IDENTITY` OnceLock: it now holds a
`QueueConfig` whose secret is re-read, and the lazily built vcs-metrics
exporter reads the identity from it.
agent_renewal.rs's module doc said a re-minted agent queue secret only
reaches the agent at its next start; hive-agent reads the store on every
connection attempt after its first, so the doc now says that.
Part of #5017 (slice Q).
swarm-logs reads `swarm/agents/<agent>/queue` from the swarm secret
store with the agent's own store certificate (the read hive-agent makes
for its queue connection) and presents it as
`Bearer swarm-agent.<agent>.<secret>`, which the gateway's
swarm-logs-auth checks. On a 401 it reads the secret again and retries
once, so a secret re-minted between the read and the check does not
fail the query.
The OIDC mint against the hive's shared `hive-<h>-agent` client is gone
from swarm-logs, along with the unused `HIVE_AGENT_OIDC_CA_FILE` anchor.
logs.nix now gates on the store address instead of the queue's token
endpoint and wraps the binary with the paths queue-identity.nix gives
the harness; the host forwards the query URL wherever it forwards a
store identity.
The shared client is still delivered into containers; removing that is
Part of #4459.
`swarmctl agent create` declares an agent the swarm has no declaration
for as paused, so ruth needs setting `up` from her card after step 4.
Restores the warning to re-create agents before a hive-side change makes
a container require a store identity.
Re-creating an agent mints its store identity the same way, so the
backfill verb and the `POST /api/agents/{name}/identity` route that only
it called go. The `MintAgentIdentity` job node stays: agent creation and
the renewal pass still queue it.
Part of #4765 (c).
The poller read `<state>/github-token` once at start, so after a PAT
rotation it kept polling with the revoked token until the unit
restarted. It now re-reads the file on every tick, as
`hive-forge-notify` does: a different non-empty value rebuilds the
GitHub source, clears the own-login cache, and logs one info line. A
missing or empty file keeps the token in use.
A 401 from the notifications endpoint is now logged at warn, once per
token value; other non-2xx statuses stay at debug.
Part of #5017 (slice G).
Seven host oneshots fetch a secret from the swarm secret store into a file
their consumer loads at start: swarm-bao-grafana-oidc, swarm-bao-otel-oidc,
swarm-bao-matrix-oidc, swarm-bao-ci-runner, swarm-bao-forge-oidc,
swarm-bao-hive-otel-oidc and swarm-bao-forwarder-oidc. Each was
RemainAfterExit with no timer, so it ran at boot or deploy and never again:
a secret re-minted in the store reached its consumer only through an
operator restarting the fetch.
lib/store-refresh-timer.nix gives each one a timer (OnUnitInactiveSec 10min,
RandomizedDelaySec 1min, the agent's own store re-read interval), and each
drops RemainAfterExit so a timer start is not a no-op. Each script already
writes the file only when secret_differs, and refresh_consumer/refresh_unit
restart the consumer only when the file is newer than its last start, so a
run reading an unchanged value restarts nothing.
Two docs said these units do not run on a timer; credentials.md's
ci-runner row said the runner token is not re-pulled.
Part of #5017 (slice T).
Follow-ups from the review of #5142.
A `Done` answer with fewer or more rows than the classifications sent, or
none, now posts the no-verdict comment instead of the classifier's overall
verdict, so a short answer can't approve.
Requests are accepted per repo, PR and head into a set bounded at the two
running plus eight waiting. A request for a key already running or waiting
starts nothing and logs at info; a request with every place taken starts no
run and posts a no-verdict comment saying so. Before, every delivery spawned
a task waiting on the semaphore, so toggling the review request queued one
paid run per toggle.
Table cells get a zero-width space after `#`, `!`, `://` and `www.`, so a
model's reason can't render an issue or PR reference or a bare URL as a link.
The secret publisher's policy denies the classifier's forge token leaf, the
way #5139 denied Grafana's admin password, with a module-eval-bao-grants arm
pinning it. The controller posts reviews with whatever token is there, so a
publisher write picked the account reviews come from.
Closes#5143, part of #5096.
Requesting the forge account `review-classifier` as a reviewer now runs the
repo's rules against the PR and posts the result as that account's review,
on the head commit the run read. A push starts no run; only a fresh review
request does.
- An instance-wide `pull_request_review_request` system hook delivers to
`/webhook/forge/review-request`. A request for any other reviewer, or a
removed request, does nothing.
- The controller reads the PR, loads the rules with the same reader as
`GET /api/review-classifier/rules`, sends one request on the try-out
subject, and posts APPROVED or REQUEST_CHANGES with the table as the body.
- With no clean verdict (PR or rules don't load, the run fails, or any
classification has no answer) it posts a COMMENT saying why, never an
approval. The event is one function, `no_clean_verdict_event`.
- Table cells are flattened, markup-escaped, and every `@` gets a
zero-width space, so a model's reason can't break the table or mention
anyone.
- At most two requested reviews run at once; the rest wait in memory.
- The account is minted the way hive accounts are: a five-minute pass
queues a `MintClassifierForgeToken` node when the token at
`swarm/services/review-classifier/forge/token` is missing or stale. The
token is `write:repository`. The controller's bao policy gains that leaf
(create/read/update), pinned in module-eval bao-grants. The classifier
never sees the token.
The account gets no repo access automatically; a repo admin adds it by
hand. A re-request retires the account's earlier review: Forgejo clears
`official` on a reviewer's earlier reviews when it is requested again
(models/issues/review.go AddReviewRequest, v16.0.5), and approvals and
rejection blocks count official reviews only.
Refs #5137
- review-classifier.md: claude runs spawn through hive-claude directly
with no tools and no session (hive-runtime holds only TurnStop); the
answer's fields, the 500-character reason cap and what a failure does;
how the try-out page reaches the classifier; the settled parts of the
open list (API key path, try-out subject, --tools "" on claude).
The banner stays.
- services.md: a Review classifier section with the two operator steps.
- secrets.md, credentials.md: the classifier's leaf and the key row.
- network.md: the container's row. CLAUDE.md: the crate.
swarm-controller re-issues an agent's store leaf at half its validity, but
the running container kept presenting the copy its hive delivered at
start, so a re-issued leaf reached the agent only on its next restart.
hive-agent-bao-identity now seeds a live copy of the pair in its own
runtime directory from the delivered credentials, and re-runs every 10
minutes: it logs in with the live leaf, reads its own
swarm/agents/<agent>/bao-mtls (a read render_agent already grants), and
when the leaf changed, logs in with the new pair and swaps it in only if
the store accepts it.
Every unit in the container that logs in to the store presents the live
pair: hive-agent (queue-identity.nix), hive-matrix-daemon,
hive-subagent-daemon, hive-agent-forge-token, hive-agent-forge-accounts
and hive-agent-github-token. They keep loading the server CA as a
credential. The long-running daemons are ordered after the refresher so
the live pair exists when they start.
It renews only while the held leaf is still valid: the login that reads
the new leaf is made with the current one.
Refs #5017
