An agent's queue credential could subscribe to every subject in the swarm
account, so any agent could read every other agent's terminal, turn state,
operator input and request replies. Agents are the least trusted clients
on the queue (mara's ruling on #4901), so their grants now carry a
subscribe allow-list:
- an agent's own credential (the harness and its subagent daemon):
`_INBOX.agent.<agent>.>` and `$SWARM.input.<agent>`
- the hive's shared agent client: `_INBOX.client.<client-id>.>`
Hive and swarm-controller grants are unchanged (no `sub` claim, so still
every subject).
The clients connect with the matching inbox prefix, so the replies they
rely on still arrive: the icon write's JetStream ack and the schedule
DIRECT.GET. `connect_with_token` takes the agent name and sets the prefix
itself; the hive-shared fallback uses the new
`connect_with_inbox_prefix`.
The harness no longer subscribes to its input subject when it fell back
to the hive's shared client: that identity names no agent, so it cannot
be granted one agent's input without all of them. It logs that and
skips, the same as the schedule reader on that path.
Refs #4901
- swarm.serviceDomains lists every swarm service name on every host. The
store's issuing role is built from it, so the store now admits a leaf
for grafana, metrics, otel, logs, the bao UI and the apex whichever host
runs the store. Each gateway's own leaf stays narrowed by
localServiceDomains.
- swarm.matrix.apiUrl defaults to http://<swarm.matrix.gatewayHost>
(null when gatewayHost is null) instead of loopback on the matrix host.
- The authelia vhost proxies /bridge/ to swarm-authelia-bridge with the
prefix stripped, and deploy.swarm-controller.authBridgeUrl defaults to
${swarm.authelia.url}/bridge on every host. swarm.authelia.bridgeUrl is
removed.
- swarm-authelia-bridge admits only swarm-controller's OIDC client on
every request (403 for any other active token), configured by
SWARM_AUTHELIA_BRIDGE_CONTROLLER_CLIENT_ID (renamed from
..._OPERATOR_READER_CLIENT_ID, which gated ListOperators alone).
- swarm.peerHives is removed; swarm-wireguard.nix filters this host's own
hive out of swarm.hives itself.
Closes#4933
A start in the past fires the latest missed time once, right away, so the
form says that instead of 'already passed'. A one-shot past its time reads
'past' / 'time passed' (the controller's clock, not a record of firing) and
keeps its pause button. The edit form compares against what it opened with,
so a schedule changing under it is not reverted on save.
/schedules lists the controller's scheduled prompts as cards: name as the
title, the agents it fires for, the message, its cadence and next fire.
Edit turns a card into its own form; pause, resume and remove act in place,
and + schedule opens an empty form. The form sends only what changed, so a
rename never moves the fire times, and warns when the start or the repeat
does. Filters reuse the shared card filter bar.
Adds a TextAreaField to the ui kit, and an upcoming mode on RelativeTime so
a next-fire time reads "in 5m". Needs the controller's /api/schedules
routes.
swarm-controller keeps every scheduled prompt (id, name, agents, body,
anchor, interval, paused) as one document under one key of a new
`schedule-definitions` KV bucket, so each change is a single
compare-and-set and a prompt exists for all of its agents or none. It
writes each agent's key of a new `agent-schedules` bucket as the prompts
naming that agent, without names or other agents, after every change and
on a one-minute reconcile pass. It is the only writer of both, behind
GET/POST /api/schedules and PATCH/DELETE /api/schedules/{id}.
`swarmctl schedule add|list|edit|pause|resume|rm` drives those routes over
the controller socket.
An agent's own queue credential is granted one more subject, the direct
get of its own key in `agent-schedules`. A KV watch is not granted: its
consumer create subject carries no key, so it would cover every agent's.
hive-agent's new schedule ticker reads that key every 10 seconds and fires
due slots as the same `schedule:<id>` todo hive-c0re's worker pushes. Fire
times are anchor + k*interval; the last fired slot per schedule id is kept
in the state db, so a body edit or list refresh never moves or refires a
slot, downtime fires the latest missed slot once, a restart refires
nothing, a one-shot fires once, and paused slots are skipped.
Refs #4914
The subagent list endpoint now returns {name, last_row_unix} per
subagent, the publish time of its newest terminal row, read from the
queue controller-side so agents publish nothing new. The panel sorts
newest first, shows a ticking age beside each name, and dims one quiet
for 15 minutes.
Closes#4928
swarm-authelia-bridge gains ListOperators, the usernames in authelia's
operator group (`operatorGroup = "admins"`, passed in from
swarm-authelia.nix). Unlike the two agent verbs, which admit any active
token, it answers only the client named by
SWARM_AUTHELIA_BRIDGE_OPERATOR_READER_CLIENT_ID (swarm-controller's
queueClientId) and returns 403 to every other active token: authelia
issues machine tokens to every hive and its agents as well. The bridge's
introspection now models `client_id` to make that check, the same way
swarm-nats-auth's does.
swarm-controller runs a grant-only pass at start and every five minutes:
read the group, map each username to the account an SSO login lands on
(tuwunel lowercases `preferred_username`), skip anyone already joined to
`#admins:<server>` (tuwunel's definition of admin) or without a matrix
account, and send `!admin users make-user-admin <user>` there as @swarm
with the swarm appservice token. Nothing is ever revoked: tuwunel has no
command for it (#4799).
The admin-room calls (alias, joined members, profile, send) go into
swarm-matrix-client next to whoami, keeping its no-body-in-errors rule.
Closes#4585
A hive fronting no swarm name fetches the store's pki root presenting its
reader certificate. When deploy.bao.clientCertFile or clientKeyFile is unset,
or either named file is not readable, the unit now exits 1 with a message
naming both options and their paths, instead of exiting 0 with the root
missing from the trust bundle.
The module-eval cases that asserted the quiet skip assert the failure
instead, and a new case covers the named-but-absent files.
The `want_svc=0` root fetch presents `deploy.bao.clientCertFile`/
`clientKeyFile` to a listener that demands a client certificate. On a
hive with neither set, the handshake is refused every time, and
`swarm-services-cert` went from green to failed-and-retrying for 24h.
The branch is now chosen at eval: with no reader identity, the unit
logs that the store root was not staged and names the option, then
exits 0, and no fetch is rendered. A hive with an identity fetches as
before. `hivectl peer-config` already handles the absent file.
Refs #4588 (review c88774, point 2).
A self-signed hive fronting no swarm service name never filled
`$svcroot` (/var/lib/hive-tls/swarm-services-root.pem): the only writer
was `swarm-services-cert`'s issuance, which exits early on
`want_svc=0`. So those hives' trust-bundle.pem lacked the store's `pki`
root, even though every other hive's swarm-service names chain to it.
On `want_svc=0`, `swarm-services-cert` now reads the root from the
store's unauthenticated `GET /v1/pki/ca/pem`, over the hive's existing
BAO_ADDR, `deploy.bao.serverCaFile` and reader identity
(`deploy.bao.clientCertFile`/`clientKeyFile`) at the TLS layer only; no
login. It writes `$svcroot` and restarts `hive-tls-ca` when the root
changed, as the issuing path already does. The daily
`swarm-services-cert-renew` timer runs the same script, so the copy
follows a root rotation.
The gateway no longer `Requires=` `swarm-services-cert` on such a host:
nginx reads nothing that unit writes there, and the fetch retries
against a sealed store.
`hivectl peer-config` prints the root as step 2 of the self-signed
instructions: the swarm's service names are issued under it, not the
swarm root.
No new option. Refs #4588.
The agent view in the swarm UI gets a message box. The text reaches the
agent's harness straight from swarm-controller over the swarm queue and
becomes an inbox turn from the operator's authelia username. hive-c0re is
not in the path.
- gateway: `/api/` forwards authelia's `Remote-User`, replacing any the
client sent.
- swarm-controller: `POST /api/agents/{name}/input` takes that header as
the sender (401 without it) and sends `{from, body}` as a NATS request
on `$SWARM.input.<agent>`. No subscriber is a 503.
- swarm-nats-auth: the controller is granted publish on `$SWARM.input.*`;
no hive or agent grant names the family, so the harness can trust
`from`.
- hive-agent: a task subscribes to the agent's own input subject and
pushes each message into the serve loop's existing reminder channel,
renamed `local_msg_*` since it now carries both.
The harness does not reply to the request: no agent grant covers a reply
subject. The controller therefore answers 202 once the request times out
(2s) and 503 on the server's no-responders status.
Closes#4828
The matrix, forge and github link routes wrote their credential
unconditionally, so linking a name that was already linked replaced the
working account. For matrix that lost the device the agent's crypto store
belongs to (#4838).
Each route now reads the account's store path first and answers 409,
naming the existing account, when something is stored there. Nothing is
written. Replacing an account takes the delete from #4899, then a link.
The matrix route checks before password mode's login, so a refused link
mints no new device at the homeserver.
The check is a read then a write, not an atomic step; two concurrent
links to one name can still both pass it.
Closes#4856
The controller-created term-sub-<agent> stream had max_age only, so a
publishing agent could grow it without bound for 24h. Add a 64 MiB
max_bytes cap with discard: Old (oldest rows drop first, publish never
fails on the cap), and size max_message_size off the queue's live
max_payload rather than a hardcoded guess.
swarm-controller now creates `term-sub-<agent>` for every agent a hive is
declared to run, at start and every minute after, with the config
`swarm_queue_client::subagent_term::open_or_create` spells (subjects
`$SWARM.term.<agent>.sub.>`, max_age 24h). An existing stream is opened as
it is, as the controller does for its other streams and buckets, under the
`$JS.API.STREAM.CREATE.*` grant it already holds.
The agent no longer creates the stream: its token is granted publish on
`$SWARM.term.<agent>.sub.>` and no `$JS.API.STREAM.CREATE|INFO` subject,
and the subagent daemon only publishes. A `CREATE` carries the stream's
config in its payload, which no subject grant narrows, so the agent could
otherwise pick the stream's subjects and limits.
An agent's subagent daemon publishes each subagent's output as terminal
rows on `$SWARM.term.<agent>.sub.<subagent>`, as the agent, into a
per-agent stream it creates itself; swarm-controller lists an agent's
subagents from that stream's subjects and relays one subagent's rows as
SSE; the swarm UI lists them under the agent's terminal preview and
reuses AgentTermPreview, full-screen tab included, with no input.
- swarm-nats.nix: the agent token may also publish
`$SWARM.term.{agent}.sub.>` and `$JS.API.STREAM.CREATE|INFO` on
`term-sub-{agent}`, and nothing else of JetStream. A module-eval arm
pins the agent-token grant as an exact list.
- mcp.nix: hive-subagent-daemon loads the agent's store identity
(`hive-agent-bao-cert/-key/-server-ca`, the ones hive-agent loads)
whenever the agent has a store, not only on the opencode preset. The
agent's own queue secret lives in the store, so this is the credential
the harness connects with.
- hive-subagent-mcp: `swarm_term` reads the agent's queue secret under
that identity, connects with the agent token, opens or creates
`term-sub-<agent>` (max_age 24h), and publishes classified rows from
the sink every subagent line already passes through. The sink only
queues (bounded, drop-and-count); a missing store, refused credential,
failed stream create or failed publish is a log line.
- The stream-json classifier (`stream_enrich`) and the `TermMsg` row
types plus `fit` move from the hive-agent binary into hive-sh4re, so
the subagent daemon publishes the rows AgentTermPreview already
renders. hive-agent keeps its LiveEvent classifier on top.
- swarm-controller: `GET /api/agents/{name}/subagents` and
`GET /api/agents/{name}/subagents/{subagent}/term/stream`.
- docs/swarm: what the UI shows and what the queue carries.
Closes#4827
Addresses argus review comment 90297 on PR #4899:
- swarm-controller/README.md: list the three DELETE routes (including
matrix's ?revoke=true) beside the PUT/GET ones already documented.
- LinkedAccounts.tsx: a delete answering 404 means the account is
already gone, so treat it as the delete's end state — re-fetch and
close the dialog instead of showing an error.
- matrix_account.rs: matrix_logout treats a 401 M_UNKNOWN_TOKEN as the
token already being revoked and proceeds with the delete; every
other logout failure still keeps the account. Adds unit tests and
updates docs/swarm/ui.md to match.
Each row of an agent's linked accounts, except its own `main` matrix
account, gets a delete action. swarm-controller serves DELETE beside each
PUT (matrix-accounts/{account}, forge-accounts/{label}, github-account),
answers 404 for an account the store does not hold, refuses `main`, and
removes every version through `delete_all_versions`.
The matrix confirmation has a revoke checkbox, off by default: the
controller logs the stored token out at its homeserver first, and keeps
the account when that fails or no homeserver is stored.
The controller's policy gains `delete` on each agent's
`metadata/.../matrix/+`, `forge/+` and `github-token`, pinned in
bao-grants.nix.
Refs #4855
- hive-c0re/README.md: drop deleted webhook_secret.rs from the module
list
- docs/swarm/README.md, docs/agent-lifecycle/approvals.md: rewrite
temporal wording (legacy/older-release phrasing) as current
behaviour
- docs/swarm/README.md: note that a lost closed delivery deploys
nothing and how the operator recovers
The hook a hive registered on the agent-configs org points at its
/webhook/config-pr route, which no hive serves any more, so every
config-repo event fails delivery to it. The forge sweep now deletes it.
Only a hook whose URL equals this hive's own exactly is removed. The same
path on another base belongs to another hive and is left alone. A forge
error is logged and boot continues; once the hook is gone the step is a
no-op.
Closes#4850
Rewrites the config-change flow around the forge merge and the
DeployRequest{rev} deploy, drops the MergeConfigPr approval, its deploy
DAG, the hive's `/webhook/` route and the `core` merge allowlist from
the docs, and states that operators join the `operators` team by hand.
Refs #4850
docs: the config-repo `main` merge gate (merge = `core` + team
`operators`, approvals = `operators`), the operator merge in the forge
UI and what it deploys, the hand-added `operators` membership, and that
with no eval-verify a failed rebuild leaves `applied/main` at the merged
commit. The swarm README lists the converged gate and the merge deploy.
`merged()` also requires `pull_request.base.ref == "main"`: the hive
deploys its config repo's `main`, so a merge into another branch would
only cost a forge fetch and a refusal comment (argus, #4894).
Refs #4850
The detail panel makes one request for the agent it shows,
GET /api/hives/{hive}/agents/{agent}/linked-accounts, which returns every
matrix, forge and github account of that agent as names and hosts. The
all-agents route and the table's matrix-column rows are removed, so the
table makes no linked-accounts request. The panel stays keyed by
hive/agent. The bao grant is unchanged.
Refs #4855
GET /api/agents/linked-accounts returns one entry per agent that
/api/agents/status has a row for, as {hive, agent, accounts}, from one
store login. The agents page fetches it once (and again when a link dialog
closes) and hands each table row and the detail panel its agent's slice,
so the page makes no per-agent request. The per-agent route had no caller
left and is removed. The bao grant is unchanged: the same list on each
agent's matrix and forge metadata directories.
Refs #4855
GET /api/hives/{hive}/agents/{agent}/linked-accounts returns one row per
account linked to the agent, as kind, name and host: each matrix account
under swarm/agents/<agent>/matrix (with its homeserver, and the agent's own
`main` marked reserved), each forge label under swarm/agents/<agent>/forge
(with its url), and github when swarm/agents/<agent>/github-token exists
(host github.com, which is not stored). No credential field is in the
response type.
Listing those two directories needs a new controller grant: `list` on
secret/metadata/swarm/agents/+/matrix and .../+/forge only, pinned in
bao-grants.nix as the only metadata stanzas under agents/ beside the queue
revocation. Checked against a dev OpenBao 2.6.3: the grant lists those two
directories and is refused on agents/, agents/<agent>/, and a leaf.
The swarm UI agent detail panel shows all rows under "accounts"; the table
view's matrix column shows the matrix rows. The link badges stay.
Refs #4855
argus (PR #4895): the link table didn't say an entry can exist without
the service running anywhere in the swarm. Every option in the table
defaults to a value, so a reader shouldn't take an entry's presence as
proof the service is up.
The swarm-controller builds the Forge quick link from
services.hyperhive.swarm.forge.domain, replacing hive-forge/default.nix's
per-host entry, so all seven swarm-service links come from swarm-level
options.
Also drops the remaining references to the removed matrix GUI switch:
the HiveUrls / Urls / hive_urls docs, the hivectl.md `open` note and
the gateway.md vhost-map rows, which name `gatewayHost` instead. The
grafana, victoriametrics and victorialogs modules' comments no longer
mention a quick-link they do not define.
Refs #4885
The swarm-controller adds a Bao quick link built from
services.hyperhive.swarm.bao.ui.domain, so the popover links the
store's browser UI whichever host runs bao.
Refs #4885
Removes services.hyperhive.deploy.matrix.gui.enable and its
swarm.matrix.gui.enable alias; both are mkRemovedOptionModule stubs. A
host running the homeserver serves fluffychat at gatewayHost's vhost,
and the hive's /matrix/ redirect follows the same condition.
The swarm-controller builds the Matrix quick link from
swarm.matrix.gatewayHost, replacing hive-matrix.nix's per-host entry.
HIVE_MATRIX_PUBLIC_URL is set on every hive with a gatewayHost, so
`hivectl open matrix` resolves off the homeserver's host too.
Drops HIVE_MATRIX_GUI_ENABLED and the dashboard's matrix_gui_enabled
field; nothing in the frontend reads it.
Refs #4885
The swarm-controller module builds the Authelia, Grafana, Metrics and
Logs quick links from services.hyperhive.swarm.<service>.domain, on the
controller's host, instead of each service module adding its entry only
on the host that runs it. A controller whose swarm runs those services
on other hosts lists them in its /api/links popover.
Forge, matrix and bao links are not moved yet: forge waits on #4891,
matrix and bao on whether their GUI gate becomes swarm-level.
Refs #4885
The sentence claiming a per-agent github token sits outside this page
and never passes through the store contradicted the
swarm/agents/<agent>/github-token row already in the table: the token
is minted by swarm-controller and read by hive-agent-github-token
through bao like every other row.
Refs #4347
An operator links an agent's GitHub personal access token in the swarm UI
(LinkGithubAccountForm, "link github account" on /agents). swarm-controller's
PUT /api/hives/{hive}/agents/{agent}/github-account stores it at
swarm/agents/<agent>/github-token (swarm_secret_client::github), a flat leaf
under the agent's prefix that the agent's existing read grant already covers:
no policy change, and no list grant, since there is one token per agent.
In the agent, hive-agent-github-token (oneshot + 2-minute timer, as the agent
user, under its own store certificate, ordered before hive-github-notify)
reads that path and writes <state>/github-token, 0600 and agent-owned, the
file the gh wrapper, git credential helper and hive-github-notify already
read. It replaces the file by rename only when the bytes changed and never
deletes it: a hive-written github-token stays until a token is linked in the
swarm UI. It is installed only with a store address and
services.hyperhive.agent.github.enable.
Removed: the dashboard's CR3D3NTIALS page (credentials.html/js/css, its
build entries and H0M3 tile; GITHUB was its only tab), hive-c0re's
dashboard/matrix_accounts.rs with GET/POST /api/github-account,
priv_client::write_agent_github_token, the host socket's
SetAgentGithubToken and `hivectl github set-token`, and hive-priv's
WriteAgentGithubToken with write_agent_state_file, its only caller gone.
Docs: integrations/github.md and swarm/ui.md describe the swarm path,
swarm/credentials.md gains the store-path row, and the hive UI docs,
hivectl docs and security.md's hive-priv table drop the removed pieces.
Closes#4347
error-pages.nix paragraph (gateway.md:433) blamed a dead authelia
upstream on an empty user set; the real reason the route earns a
custom page is that a bare 502 there blames the proxy while the
gateway itself is fine. gateway.md:38 dropped 'yet' from the
placeholder-while-empty phrasing. services.md:109 corrected
'seeds an empty users database' to the disabled placeholder subject
swarm-authelia.nix actually seeds (swarm-authelia.nix:873).
Refs #3902
Per mara's review: the hive UI doc covers only what hive-c0re's pages
render. Removed the M4TR1X page section (the hive gateway redirects
/matrix/ to the swarm matrix client, which the swarm UI's quick links
open), the swarm-UI forge/matrix account-linking lines from the
CR3D3NTIALS section, the infra-services hivectl paragraph, and the H0M3
Matrix/Forge absence line. Added a single pointer to docs/swarm/ui.md,
and stated the account-linking and Matrix quick-link facts there.
Refs #3902
The swarm.domain assertion in hive-network.nix fired on every host that
imported the module, so a host that enables nothing failed eval. It now
fires only when one of the hyperhive service switches is on (every
deploy.*.enable that runs something, gateway, gateway.dns, network,
otel, snapshotStore). The requirement itself is unchanged: any host that
runs a hyperhive service still needs swarm.domain.
The core-toggle module-eval suite gains a case: a missing swarm.domain is
refused on a hive and on a swarm-service-only host, and a host enabling
nothing passes every assertion.
Closes#4887
authelia 4.39.20 exits at startup on `users: {}` ("users: non zero value
required"), and the first-boot unit seeded exactly that, so a swarm with
no users crash-looped authelia and answered 502 until `swarmctl user add`
ran.
The first-boot unit now writes one subject, `swarm.placeholder`, when
the users database is absent, empty, or exactly `users: {}`:
- `disabled: true` — authelia returns "user not found" for a disabled
user before any password check (file_user_provider.go,
CheckUserPassword).
- password: an argon2id digest with an all-zero key. It decodes (authelia
rejects a non-digest at startup) and no known password hashes to it.
- the `.` keeps it out of agent names (`[a-z0-9-]`), and `swarmctl user
add` refuses it as already existing. Neither writer removes users, and
both round-trip `disabled`.
A file with any user in it is never touched.
The docs that described the crash-loop (sso.md, gateway.md, setup.md,
the sso-unavailable error page) now describe the placeholder; the
writers' load_store docs and the seed fixtures follow. module-eval
nats-authelia asserts the seed branch.
swarm/README.md opens with the swarm and its control plane; hive identity
and the directory follow as the substrate. Upgrade notes move into a
<details> block, the per-agent queue publishing detail into another, and
the one-paragraph pointer sections collapse into a link list.
Fact fixes, checked against origin/main:
- an empty swarm.hives fails eval (swarm.nix:341-354); it does not mean
"not in a swarm"
- swarm.domain is required with a hive (hive-network.nix:156,188), hiveName
with a hive, store or homeserver (hyperhive.nix:161-166)
- the matrix container trusts the hive's trust-bundle.pem at runtime under
self-signed certs (hive-matrix.nix:1046-1052, lib/hive-ca-trust.nix:76-85)
- singleHostSwarm also defaults the controller, localHostsEntry, the nats
callout keys and the bao bootstrap token path (local-defaults.nix:72-129)
- swarm-controller serves far more than /health: roster, wanted state, job
graph, agent creation and credential mints (main.rs:2874-2899)
- swarmctl user add needs --email for the forge account and refuses an
existing user (setup.md:67-71, swarmctl/src/main.rs:425-430); document
agent mint-identity and mint-forge-token
- agent creation also mints store identity, forge token and matrix
account, and declares the agent paused (main.rs:1822-1920, 247-248)
Refs #3902
Fix the 4 pre-existing vale error-level hits on main (docs/README.md:83,
docs/getting-started/setup.md:11,127, docs/swarm/bao.md:4) that fail CI's
prose-lint-errors job for every docs PR regardless of its own diff.
An operator now links an agent's external forge account (label, base URL,
token) in the swarm UI. swarm-controller stores it at
swarm/agents/<agent>/forge/<label>. There is no index: the store's
listing of the agent's forge/ directory is the set of accounts.
In the agent, hive-agent-forge-accounts (oneshot + 2-minute timer, as
the agent user, under its own store certificate) lists
swarm/agents/<agent>/forge/ with the `list` #4866 grants an agent on its
own metadata subtree, reads each account, and writes
<state>/forge-<label>-token and forge-<label>.json in the names and shape
hive-forge -f already reads. An empty listing (a 404, which `bao kv list
-format=json` answers with `{}` and an empty stderr) is zero accounts; a
denial or an unreachable store fails the unit. It never deletes: files
for labels not listed, including ones the hive wrote, stay as they are.
Removed: the dashboard FORGES tab (credentials.js/html section and its
CSS), hive-c0re's extra_forges.rs and its routes, priv_client's
extra-forge calls, and hive-priv's WriteAgentExtraForgeAccount /
DeleteAgentExtraForgeAccount with their helpers. The GITHUB tab and
WriteAgentGithubToken stay.
Also: persistence.md's matrix avatar note names the exit-75 restart on a
changed account listing, not the dashboard, as what brings a linked
account up.
Refs #4348
render_agent gains a second stanza: list on
secret/metadata/swarm/agents/<agent>/*, next to the existing read on
secret/data/swarm/agents/<agent>/*. An agent can now learn which
credentials it holds by listing its own subtree. Metadata read, writes
and every other principal's paths stay refused.
An agent's policy was only written when it was minted, so existing agents
would never get the new stanza. swarm-controller now rewrites every
agent's policy at start (read_policy::ensure_agent_policies), with the
same 30s / 24h retry as ensure_hive_access. The roster is the store's
hive-agent-* cert-auth roles, listed with the controller's existing
`list` on auth/cert/certs; the writes use its existing grant on
sys/policies/acl/hive-*. Only the policy is written: mint_and_verify
also reissues the certificate, so the pass does not call it.
Refs #4348
The previous commit's comments said a unit in auto-restart keeps its
start job, so anything ordered after it waits for the whole 24h retry
window. That is wrong under the default RestartMode=normal: each failed
attempt passes through `failed`, which ends that start job. `After=`
dependents proceed after one attempt, `Requires=` dependents fail with
`dependency`, and the retries continue as fresh start jobs. The
2026-09-24 journal shows it with the already-2880 swarm-services-cert:
nginx got "Dependency failed" 1ms after the first failure, and
switch-to-configuration exited before the first restart was scheduled.
The comments in lib/store-retry.nix, glue-matrix-bao-token.nix,
glue-queue-agent-credential.nix, swarm-otel.nix and swarm-grafana.nix
now say that, and so does docs/swarm/credentials.md.
Because dependents start after one attempt, a consumer that loads its
credential at start never sees a value a later attempt lands, or a
rotated one. nix/host-modules/lib/refresh-consumer.nix adds
`secret_differs` and `refresh_consumer`, and the four fetch units whose
consumers take a start-time copy call them after the write, only when
the value changed:
- swarm-bao-matrix-token -> tuwunel.service in hive-matrix
- swarm-bao-otel-oidc -> opentelemetry-collector.service in swarm-otel
- swarm-bao-grafana-oidc -> grafana.service in the grafana container
- swarm-bao-forwarder-oidc -> opentelemetry-collector.service in swarm-bao
A running consumer is try-restarted, a failed one is reset and started,
all with --no-block. Inline in the fetch script rather than a
PathChanged path unit because the fetch script is the only writer and
already knows whether the value changed, and it is the same shape as
this PR's nginx hook and swarm-bao-nats-tls's restart of nats.
module-eval-bao-grants gains one case per consumer.
Refs #4662
Every hive is in a swarm and every swarm runs matrix, so every swarm has a
swarm-controller, and since #4810 its hive_sender pass mints each hive's
@hive-<hive>: sender token into the store every five minutes. The two
other minters of that token go:
- swarm-matrix-ctl mint: the systemd.services.swarm-matrix-ctl unit in the
hive-matrix container, Command::Mint and src/mint.rs. The binary, its
appservice render/publish verbs, ctlPackage, ctlActive and the ctl cert
role stay. bao-matrix-reader's checks on the deleted unit are removed;
the leaf-identity and no-token-in-env checks now look at
swarm-matrix-appservice-publish, which runs under the same identity.
- the hive-side mint ladder in hive-c0re's ensure_hive_user
(register/appservice-login/password-login with the local as_token), with
read_appservice_token, paths::matrix_appservice_token and the helpers
only it used. ensure_hive_user now takes the store's token, keeps the
file when the store has none or can't be reached, and fails otherwise.
- hivectl matrix sync-admin: the verb, HostRequest::MatrixSyncAdmin and
handle_matrix_sync_admin. The periodic MatrixSweep (ensure_all) is
unchanged apart from no longer reading the local as_token.
This removes the double-mint race #4810's review flagged: two minters
logging in on one pinned device could leave a dead token in the store
until the next pass.
Closes#4813Closes#4814
