authelia's file backend has no self-service reset (no SMTP notifier),
so the only way a human account got a new password after the old one
was forgotten was hand-editing users.yml as root. `user add` already
hashes a password into the file; this verb does the same for an
existing user instead of refusing on the name.
Mirrors `user add`'s UX exactly: no password flag, authelia generates
and hashes it (never crosses argv), and it's printed once and never
stored. Refuses on an unknown user before ever invoking authelia. Same
publish path as add/update, so the same atomic write and no-restart
(authelia watches the file) behaviour apply.
Split the digest-replacement into users::reset_password so it's
testable without a command line or a running authelia, same pattern
as apply_update.
The harness now reads swarm/agents/<agent>/queue from the store itself,
under the agent's own store certificate, and holds it in memory only.
It reads once before the first connect and again on every reconnect
attempt (async-nats `ConnectOptions::with_auth_callback`), so an agent
whose secret was re-minted reconnects with the new value instead of
being refused until the container restarts.
hive-agent-queue-credential.service, the /run file it wrote, and
HIVE_AGENT_QUEUE_AGENT_SECRET_FILE are gone; queue-identity.nix now
hands hive-agent.service the store address, its certificate paths and
the agent name.
A failed or empty read before the first connect still falls back to
the hive's shared client. Each read is bounded by a 10s timeout, and
retries wait out the existing reconnect backoff (500ms doubling, capped
at 60s).
Closes#4783
post_purge_tombstone discarded fail_pending_for_agent's error with
let _ =. Mirrors #4740's fix for the identical discard in
job_queue/exec.rs's run_destroy_bookkeeping: warn and continue, since
the purge itself has already succeeded by this point.
Refs #4747
du_bytes returned None uniformly for every du failure, and both
du_bytes and measure_agent_disk had no logging at all, so a container
rootfs du couldn't read (permission, I/O error, unparseable output)
silently reported as 0 disk with no signal in the journal.
Distinguish the expected case — the path plain doesn't exist, e.g. a
destroyed-but-kept agent's rootfs or a not-yet-created state dir — from
an actual read failure, and warn only on the latter, with the path plus
whichever of the failure (spawn error, exit status, stderr, unparseable
stdout) applies.
The raw reqwest client hive-forge uses for Forgejo's web-router-only
routes (attachment downloads, Actions artifact/log routes) had no
timeout, so a hung Forgejo response blocked the calling CLI invocation
indefinitely.
Adds a 5s connect_timeout (matching #4737's outbound-HTTP sites) plus
a per-call request timeout: 15s (config_pr_poll's forge budget) for
the small JSON calls (get_api_json, post_json_web), and 10 minutes for
get_bytes_named/get_bytes_raw, which download attachments, Actions
artifact zips and persisted job logs that can be large.
reqwest::blocking has no separate read/idle timeout, so a single
whole-request budget has to cover those downloads; the smaller JSON
budget would cut them off partway through.
Refs #4746, #4737
`script-test-agent-bao-fetch` executes the rendered `ExecStart` of
`hive-agent-forge-token` and `hive-agent-queue-credential` under
`umask 0377`, with a stub `bao` first on the unit's own PATH. It covers
every error branch, the happy path, forge rotation/unchanged, and 0400
files already in place: the redirect failure #4736 fixed, whose live
symptom was `bao.err: Permission denied` reported as a refused
certificate. The TLS-alert fixture is the `unknown certificate
authority` error h-atlas's identity check got from the store.
#4736 claimed this test but never committed it; the two module-eval
suites for these units only evaluate the config.
Closes#4748
The collector's oidc/* authenticators call out to authelia at startup, so a
restart that races authelia's own (a redeploy that touches both, a store
outage) can fail immediately. nixpkgs' upstream opentelemetry-collector
module sets Restart=always with no RestartSec, so systemd's defaults
(100ms RestartSec, 5-in-10s start limit) burn the whole allowance in well
under a second and leave the unit in start-limit-hit, dead until someone
resets it by hand.
Sets RestartSec=5 plus an explicit startLimitBurst/startLimitIntervalSec
window (12/120s) sized so the burst can never trip while authelia comes
back — same values host-modules/otel.nix already uses for the sibling
host-tier collector, which depends on authelia the same way. Pins the
[Unit]-vs-[Service] placement and the window relation in
module-eval-swarm-otel-core, mirroring module-eval-hive-otel's existing
case for the host tier.
prebuild_toplevel awaited nix build with a bare child.wait().await, so
a wedged nix-daemon (unreachable remote builder, stuck build slot)
hung the rebuild job forever with no way for the job queue to recover
short of restarting hive-c0re.
Mirrors #4741's hive-priv fix: the child now leads its own process
group, and after PREBUILD_TIMEOUT (1h, four times CI's observed
cold-cache flake check) the whole group is SIGKILLed and the call
fails with a named timeout error instead of hanging.
Refs #4723, #4741
revoke_queue_credential only ever deletes swarm/agents/<agent>/queue
(agent_queue_path + a literal "queue" suffix), never anything else
under an agent's prefix. secret/metadata/swarm/agents/+/queue matches
that exactly — `+` is bao's single-segment glob, the same form
swarm-nats-auth's read grant already uses for the data-side path.
Also rewords the module-eval test's stale note about a read/list grant
handing a "write-only principal" the version history: the controller
has held read on secret/data/swarm/agents/* since the mint-and-verify
read-before-write change, so it was never write-only on that path.
rustdoc runs with -D rustdoc::broken-intra-doc-links and the type is
imported inside the function body, not at module scope, so the bare
link resolved to nothing and failed the workspace-doc derivation.
Pins the three properties the revocation rests on and cannot check
against a store: that only `Destroyed` revokes (a revocation on
`Offline` or `Paused` would give an agent that stops and never
restarts), that a 404 is absence while a 403 stays a failure, and that
the delete addresses `secret/metadata/` -- the path that takes every
version, which is the string the grant has to match.
docs/swarm/credentials.md gains the revocation section and its table
cell stops describing the deletion as something an operator does by
hand.
A per-agent queue credential is minted at agent creation and nothing has
ever removed it. An agent declared destroyed loses its container and
keeps its credential: a bearer secret recovered from a snapshot or a
stale capture still authenticates as that agent, so the set of usable
credentials only grows.
Delete the path the mint published, on the one transition that ends an
agent's life. It mirrors step 3 of `mint_and_verify` and no other step:
the leaf, the ACL document and the cert role are what a hive uses to
collect an agent's secrets and are re-minted on every run of the mint.
Every version, not the newest. The mint rewrites the path when the
principal it names needs correcting, so KV v2's plain delete would leave
the identical secret readable at ?version=N. That is a separately-ACL'd
path, hence the second stanza in the controller's grant -- `delete` on
metadata discloses nothing, and `update` on the data path already lets
this principal destroy any agent credential's usability.
The destroy is not blocked by a failed revocation: the declaration is
already published and refusing the call would leave an operator with an
agent they cannot tear down. The failure is logged at error instead,
naming the agent, since a silent orphan is the fault being removed.
mara, PR review: "the dynamic tab should be in the top bar, not a new
one below". Moves the .shell-tabs group from its own sticky row under
the header into .shell-nav itself, right after the nav indicator.
Also adds a third re-measure effect for the sliding nav indicator,
keyed on tabs.length: with tabs inline in the same flex row the
indicator measures, closing a background tab (no navigation) can
shrink the row without the hop effect's own re-measure ever firing.
Same reflow-not-navigation reasoning as the existing resize-listener
effect.
argus's review on PR #4784 caught a wrong technical claim: the
comment said the .ui-agent-term-preview-full override rules relied
on source order because they had equal specificity to the
un-modified rules above. They don't — each override selector adds
one more class (the .ui-agent-term-preview-full prefix) than what it
overrides, so they're strictly more specific and win regardless of
file order. Corrected the comment to say so.
Adds a full, non-capped agent terminal reachable from a new expand
trigger on the embedded AgentTermPreview (the detail-panel preview on
AgentsPage stays as-is, just gains the trigger). Opens
/agents/:name/terminal in a new dynamic tab in Shell's header, next to
the static nav row — tabs persist across a reload via useDynamicTabs,
a small localStorage-backed hook built on @hive/shared's existing
settings-storage primitive.
AgentTermPreview gains two new props to support both mounts from one
component: fullHeight (drops the 12em preview cap, fills its page)
and showHeaderBadges (default true — lets a future caller that
already shows turn_state/model/ctx/cost elsewhere suppress this
cluster; AgentsPage doesn't use it, see below).
Deviation from the originally posted plan (issue comment 80596): that
plan proposed AgentsPage's embedded preview pass showHeaderBadges as
false, reasoning the detail panel already duplicates that info.
Checked the actual code before implementing — it doesn't; AgentRow/
AgentTypes.ts carry none of turn_state/model/ctx/cost, and
AgentTermPreview's own floating badges are the only place swarm-ui
shows them. Left the badges visible there instead of shipping a
regression the plan's own stated justification didn't hold up to.
Also fixed a same-tab pub/sub race found by actually rendering a cold
load of /agents/:name/terminal (headless chromium, not just reasoning
about the code): useLocalSetting subscribes inside a useEffect, and
mount effects fire children-before-parents, so a descendant's
mount-time write (AgentTerminalPage registering its own tab) can beat
an ancestor's (Shell's) subscription into existence, leaving Shell's
tab row silently empty on a direct/reload load. Fixed by having
useDynamicTabs re-sync from storage on every location change, not
just on notify() — the fix lives in the new hook itself, not in the
shared settings-storage primitive theme/motion overrides also use.
No Rust ever minted a secret_id; approle was dead attack surface. The
bootstrap step's check-then-enable case becomes check-then-disable: if
approle is mounted, `bao auth disable approle`; otherwise a no-op.
Disabling costs `delete`+`sudo` on `sys/auth/approle`, not
`create`/`update` — verified against `bao auth disable -output-policy`
on a live dev store. The bootstrap policy grant is narrowed to match.
nix/module-eval/bao-grants.nix pins the new shape: the bootstrap policy
may disable approle, and the granter's role unit never enables it.
A stored queue secret with no `minted_at` counted as due, and no secret
minted before the renewal pass has one, so the first pass after deploy
would re-mint every agent's secret. Every reconnect before that agent's
next restart would then be refused.
Such a secret is now stamped instead: `minted_at = now` is written beside
the unchanged `value`, and its 45-day clock starts there. Only a secret
whose recorded mint time is at least 45 days old gets a new value.
The decision is `secret_step` (Keep / Backfill / Remint), and the pass
reports an unstamped secret as `Observed::Unstamped`. Backfill and
re-mint log different lines.
A five-minute pass over every agent some hive's wanted state declares as
anything but destroyed queues, per agent:
- `MintAgentIdentity` (the node agent creation uses) when the stored
certificate at swarm/agents/<agent>/bao-mtls is past half its validity,
read from its own notBefore/notAfter: day 45 of the role's 90;
- the new `RenewAgentQueueCredential` node when the queue secret at
swarm/agents/<agent>/queue is 45 days old or has no mint time. The node
re-decides, writes a fresh value with `minted_at`, reads it back, and logs
the agent and the old age.
When both are due the secret node runs after_any the certificate node,
because mint_and_verify compares the queue secret it read with the one it
reads back. A credential that is not stored is never created here.
`queue::AgentCredential` gains an optional `minted_at` (unix seconds);
agent creation now sets it. Stored objects without it decode unchanged and
count as due, so every existing queue secret is re-minted on the first pass.
Both replacements reach the agent at its next start. The old certificate
stays valid until it expires; the old queue secret does not, so a queue
reconnect before that restart is denied.
Adds x509-cert 0.2 (with der_derive and flagset) to read the validity.
docs/swarm/credentials.md: the renewal column splits into automatic re-mint
and automatic re-pull, filled from the code as it stands.
The bao UI at bao-ui.<swarm> took a raw store token and nothing else.
It now offers an OIDC tab: authelia's `admins` group logs in and lands
on `swarm-operator-viewer`, which is list+read on `secret/metadata/*`
and nothing under `secret/data/` or `sys/`.
- authelia registers an interactive client `swarm-bao-ui`
(glue-bao-ui-oidc-client.nix) with redirect
`https://bao-ui.<swarm>/ui/vault/auth/oidc/oidc/callback`; the secret
publisher carries its secret to
`secret/swarm/services/swarm-bao-ui/oidc/client`.
- `swarm-bao-granter-role` (bootstrap token) enables the `oidc` auth
mount with listing visibility `unauth`, asked before attempted like
cert/approle; `bao-bootstrap-policy.hcl` gains `sys/auth/oidc`.
- The granter's policy gains `auth/oidc/config`, `auth/oidc/role/swarm-*`
and read on that one secret leaf. It still holds no `sys/auth`.
- New granting unit `swarm-bao-operator-viewer-policy` writes the viewer
policy, and once the granter may configure `auth/oidc/config` (checked
through `sys/capabilities-self`), writes the mount's config from the
published secret and the role binding `groups=admins` to the viewer.
Before the bootstrap step re-runs it writes the policy, logs the step
and exits 0.
Route (a) per mara on #4775: enabling the auth method stays a
bootstrap-token step, re-run once on the live store.
module-eval pins the viewer policy's single metadata stanza, that the
granter's policy has no sys/auth path, the oidc enable in the bootstrap
unit, the exit-0 path, the config/role contents, and the client
registration + publish.
openbao gains a second listener, `ui`, on 127.0.0.1:<deploy.bao.uiPort>
(default 8204) with TLS off and no client-certificate requirement, and
`ui = true`. The existing listeners are unchanged. An nginx inside the
store's container, on 127.0.0.1:<deploy.bao.uiProxyPort> (default 8206),
forwards only /ui/ and /v1/ to it, redirects / to /ui/, answers 403 on
sys/unseal, sys/seal, sys/step-down, sys/rekey* and sys/generate-root*,
and 404 on everything else.
The gateway on the store's host serves `swarm.bao.ui.domain` (default
bao-ui.<swarm>) behind the authelia auth_request subrequest, proxying to
that nginx; the name joins serviceDomains and localNames like every
other gateway-published swarm service. authelia gets an access_control
rule restricting that name to group:admins, rendered wherever authelia
runs, since the default policy admits any session.
Trade-off, ruled by the operator on the parent issue: the UI listener
asks for no client certificate, so on that door a bao token alone is the
credential.
Three comments and a doc line claimed every API listener demands a
client certificate; they now except the loopback UI listener. The
module-eval case counting declared listeners excludes `ui` by name, as
it already did `metrics`.
On a self-signed gateway, the UI's name is a swarm service name, so its
host requests the services leaf from the store. `swarm-services-cert`
sits Before= and RequiredBy= the gateway's cert import, which nginx
Requires=. On a host whose only swarm name is the UI, that would hold
nginx, and with it the stream passthrough every reader dials, on a login
to a store that may be sealed. hive-tls drops those two edges exactly
when the UI is the only local swarm name: nginx starts on the existing
hive-leaf fallback, and the script's existing re-import reloads nginx
once the leaf issues. Every other host keeps both edges.
A job-level if: is evaluated by whichever runner picks the job up. The
public forge's copy has only a nixos runner, so a job guarded by
if: vars.PUBLIC_FORGE != 'true' and pinned to runs-on: [hive-ci] never
gets picked up there to evaluate the guard at all - it just sits queued.
runs-on now carries the same PUBLIC_FORGE variable: hive-ci internally,
nixos on the public copy. There the nixos runner picks the job up,
evaluates the existing if:, and skips it immediately. Internally
PUBLIC_FORGE is unset, so runs-on still resolves to hive-ci and nothing
changes.
The controller's OIDC client secret (client `swarm-controller`, used for
the queue connection, the auth-bridge bearer and the OTLP push) came from
an operator-placed file, `deploy.swarm-controller.queue.clientSecretFile`,
handed in by `LoadCredential=`.
Now `swarm-secret-publish`, which already copies authelia's minted OIDC
secrets into the store, also publishes this one, to
`swarm/controller/swarm-controller/oidc/client`. That path sits under
`controller/`, which no hive's policy reads. The controller reads it once
at start with its existing store certificate and holds it in memory, as
`swarm_queue_client::ClientSecret::Value`. If the store is down, it
retries for about a minute and then fails the start, so `Restart=` tries
again.
Policy delta: the controller gets `read` on that leaf, and the publisher
gets `create`/`update` on that leaf.
Removed: the `queue.clientSecretFile` option (both spellings, now removed
options with a message), its singleHostSwarm default, the credential and
placeholder, and the path watcher plus its restart oneshot. A controller
without a store identity is now an eval error, because it has no other
way to get the secret.
Guards internal-only jobs (ci.yml, coverage.yml, flake-update.yml) with
`vars.PUBLIC_CACHE != 'true'` and adds public-cache.yml, guarded to the
inverse, to build the deployed closures and push them to the public
attic cache on a push to main.
`PUBLIC_CACHE` is a repo Actions variable, opt-in only on the public
copy: unset here, it leaves internal CI's `!=` guards true so internal
jobs always run. Job-level `if:` cannot see the `github`/`forgejo`
context at all on this runner (confirmed empirically — a
`github.server_url` comparison always evaluates false at job level,
though the identical comparison resolves correctly inside a step),
so `vars.*`, which is available at job level, is the only usable
opt-in signal here.
Same gap #4611 fixed for logout: the Preact rewrite's StatusChips menu
never picked up new-session as a click path, only /new-session typed
twice into the terminal. postNewSession already existed in
termActions.ts. Wire it into the status menu with the same arm-then-
confirm click pattern logout/cancel-turn already use.
Closes#4612
Each agent card now leads with the agent's icon, loaded as an `<img>`
from `GET /api/agents/<name>/icon`: the same 5em square, background and
fallback as the hive dashboard's container row. An agent with no icon
(the route's 404), or any other failed load, shows the dimmed hyperhive
mark (`/favicon.svg`) instead of a broken image.
Only ever an `<img>`, never inline markup: the body is an agent-authored
SVG, and an image load does not run its script.
The auth callout grants an agent that presents its own queue credential
one more subject, `$KV.agent-icons.<agent>`: its own key in the
agent-icons bucket and no other. The hive's shared agent client is
granted none of the bucket, since every agent on a hive presents it.
hive-agent writes `/etc/hyperhive/icon.svg`, the file its `GET /icon`
serves, to that key once per start, as a JetStream publish straight to
the subject (what `kv::Store::put` sends, minus the bucket lookup), so
the one subject is the whole grant. No icon deletes the key. A failed
write, including one that arrives before the bucket exists, is retried
with backoff until acked. An agent connected with the hive's shared
client publishes nothing.
swarm-controller creates the bucket as soon as its queue connection is
up, instead of on the first icon read, so an agent's write does not
wait for someone to look.
Measured against a local nats-server with a user allowed publish on
`$KV.agent-icons.atlas` only: the write to its own key is stored and
readable, a write to `$KV.agent-icons.argus` is refused (the ack times
out), the DEL marker makes the key read as absent, and a write before
the bucket exists fails with "no responders".
`scripts/check-issue-refs.sh` blocks a `#N`-shaped reference in source.
The quoted ruling reads identically without it, so the tag goes rather
than a lint suppression.
An agent is not fixed to a hive, so its icon cannot be resolved as
hive -> agent. This adds the swarm-level half: an `agent-icons` KV
bucket keyed by the agent name alone — no hive token, so an agent that
moves hives keeps its icon and one that is stopped still has one — and
`GET /api/agents/<name>/icon` on swarm-controller serving it
same-origin, like every other `/api/*` route swarm-ui calls.
404 is the "this agent has no icon" answer, the same contract the
per-agent harness's own `GET /icon` has for an unconfigured agent.
Until the agent-side publisher lands, that is every agent's answer:
the publisher runs inside the container and an agent's NATS grants are
hive-scoped, which cannot authorise a write to a single-token agent
key. The read side needs no grant change — the controller already
holds `$KV.*.>` and `$JS.API.DIRECT.GET.*.>`.
The response carries `Content-Security-Policy: sandbox` and `nosniff`:
the body is an operator-authored SVG served from this daemon's own
origin, and an SVG can carry script.
Hive-side icon serving is untouched.
Refs #4502
Per mara's review on #4769: don't extend stale info to say it's stale,
just remove it. The sentence only described the absence of the removed
topology tree, giving the operator nothing actionable.
C0NTAINERS already states the flat/alphabetical/no-nesting fact;
the section repeated it with zero new operator info. Fix the two
dangling references (README.md reading-path index, swarm.js
comment) rather than leaving them pointing at a removed heading.
The backend dropped the agent hierarchy's parent field, so every
container is a root and buildAgentTree/treePrefixDom could only ever
produce a single-level flat list — the .tree-prefix CSS lane rules
already matched nothing. Replaced with sortedContainerRows, a plain
alphabetical sort, and dropped the now-dead .tree-prefix/.tree-lane/
data-depth CSS and the depth/isLast/ancestorIsLast fields from the row
fingerprint and buildContainerLi's signature. No visible behavior
change - the rendered list was already flat, just via dead machinery.
Docs updated to describe the simpler implementation directly instead
of narrating the removal (kept the heading name since swarm.js still
points a comment at it).
c9ba9062 replaced 5 bargauge panels with one table panel using a
joinByField+organize+sortBy transformations chain and a
vizConfig.group="table" viz spec. That construct had zero precedent
anywhere else in this repo's dashboards and its live render was
explicitly flagged as unverified at merge time (no in-container way
to render Grafana to confirm).
#4658 reports a dashboard crash: "'' not found in: reduce,
filterFieldsByName, ... transpose" - a Grafana transform-registry
lookup failing on an empty transformer id. That registry's full id
list matches the error text exactly, and the table panel is the only
panel in the whole swarm-grafana/dashboards tree using a non-empty
transformations array, so it's the leading suspect: some part of how
the dashboard-v2beta1 schema expects a QueryGroup's transformations
to be shaped likely differs from the classic {id, options} shape used
here, and the file-based provisioner may not be normalizing the
mismatch the same way a schema-v2-aware loader does.
Reverting to the prior, previously-uneventful bargauge panels
(byte-for-byte c9ba9062^'s version of this section) to stop the
active crash while the correct v2beta1 transformation shape gets
verified against a live Grafana render instead of guessed at again.
swarm-bao-granter-role used its token for `bao auth list` with no check,
so an expired, revoked or policy-less token in bootstrapTokenFile exited
2 with a raw 403 and no hint. It now checks whether the store is up when
that call fails: if it is, the token is at fault, and the unit prints the
one-time bootstrap step and exits 4. A missing token file is still a
ConditionPathExists skip, so the two read differently in the journal.
The twelve granterLogin units printed "the store is sealed or
unreachable" on a healthy store, because `bao status` exited 1 there:
the CLI resolves a token helper under $HOME before asking, systemd sets
no HOME for a unit without User=, and the fallback shells out to
`getent`/`sh`, neither of which is on the unit's PATH ("failed to get
token helper: error expanding config path "": exec: "sh": executable
file not found in $PATH"). The check now runs with HOME=/var/empty and
keeps its stderr, so a genuinely unreachable store says why. When the
store is up and the login is refused, the units now name
swarm-bao-granter-role as the unit that writes the missing role.
setup.md's post-step restart used 'swarm-bao-*-policy.service', which
misses swarm-bao-agent-pki. It now names that unit too, and a
module-eval case fails when the restart misses any unit that logs in as
the granter.
Refs #4704
matrixHomeserverUrl re-derived gatewayHost's own default
(chat.${swarmDomain}) instead of reading gatewayHost itself, so a
deployment that pins gatewayHost away from that default (the exact
case hive-matrix.nix's own option doc describes) left the controller
calling a name nothing serves. Read matrix.gatewayHost directly,
mirroring hive-matrix.nix's ctlHomeserverUrl. Add module-eval cases
that pin gatewayHost and that null it out, alongside the existing
unpinned default case.
Closes#4757
When the per-agent secret `queue-identity.nix` fetched is present, the
harness connects with `swarm-agent.<agent>.<secret>` as a static token
and publishes on `$SWARM.term.<agent>` and `$SWARM.agent-state.<agent>`.
When it is absent, or that first connect fails for any reason, a refusal
from a responder that does not verify agent tokens included, it connects
with the hive's shared OIDC client and publishes on the hive-scoped
subjects as before. Which one it took is logged once per connect.
`swarm_queue_client::connect_with_token` is the static-token connect: no
retry on the initial attempt, so the caller sees the refusal and can
fall back. Reconnects share the existing backoff, now a named function.
Closes#4630
The terminal and turn-state relays now subscribe to `$SWARM.term.<agent>`
and `$SWARM.agent-state.<agent>`, the subjects a verified agent token is
granted, as well as the hive-scoped `<prefix>.<hive>.<agent>` an agent
still on its hive's shared credential publishes to. Both at once, so a
swarm-ui terminal keeps working whichever credential an agent connected
with and in whatever order hosts deploy.
An `auth_token` spelled `swarm-agent.<agent>.<secret>` is no longer sent
to introspection. The responder reads `swarm/agents/<agent>/queue` with
an identity of its own, checks that the stored object names the same
agent, compares the secret in constant time, and grants the subjects
`--agent-token-publish-subject` lists with `{agent}` expanded. Every
other outcome denies: a malformed token, no store identity, nothing
stored, a failed or slow lookup, a different secret. A token without
the prefix takes the OIDC path unchanged.
The journal's `auth request` line names such a caller `agent:<agent>`;
the hive-shared credential keeps `hive-<h>-agent`.
The new principal: a `swarm-nats-auth` cert-auth role and policy with
read on `secret/data/swarm/agents/+/queue` alone, a leaf signed by the
store's PKI glue, and `glue-nats-auth-bao-identity.nix` pairing the two.
The copy unit delivers the identity into the queue's container, and an
absent leaf is delivered empty so the responder still starts and only
agent tokens are refused.
The policy and role are written by `swarm-bao-nats-auth-policy`, logged in
as the bao granter: both names fall under its `swarm-*` globs, so the
deploy writes them with no operator step. module-eval counts it among the
granting units, so every generic granting-unit case covers it.
The secret compare uses `subtle`, already in the lock file through the
TLS stack; no workspace crate offered one directly.
`swarm_queue_client::agent_token::format_agent_token` / `parse_agent_token`
are the spelling an agent presents its own queue secret in,
`swarm-agent.<agent>.<secret>`, and the one the auth-callout responder
reads back. The prefix is what separates it from an OIDC access token,
which may itself contain `.`. Parsing distinguishes "not an agent token"
(no prefix) from "a malformed one"; the error names the problem and never
the value. The module is store-free, so the agent formats its token
without linking the secret-store client.
`swarm_secret_client::queue::AgentCredential` loses `hive`: an agent's
identity is not tied to a hive, and nothing reads the field. Objects
already in the store carry it and still decode, since unknown fields are
ignored; a test parses one. The controller stops writing it.
With the credential no longer naming a hive, and the agent's policy
naming none since #4762, nothing in the mint consumes one. `hive` goes
from `mint_and_verify`, from the `MintAgentIdentity` node, and from
`POST /api/agents/{name}/identity`, which now takes no body and no longer
checks a hive against the roster; a caller that still sends one is not
refused, the body is ignored. `swarmctl agent mint-identity` loses
`--hive`, so passing it is now a usage error.
An agent's store identity was signed in swarm-controller's memory by a CA
a controller-host unit generated on disk, and the listener never trusted
that CA. Agent leaves now come from the store itself: a `pki-agents` PKI
mount whose root openbao generates internally, so the agent CA's key
never exists outside the store.
- swarm-bao-agent-pki (new, store host, as the bao granter): enables and
tunes the mount, generates the root once (guarded on an empty issuer
list, no replace branch), upserts the `swarm-agent` role (client
certificates named `hive-agent-*` only, 90 days), caches the CA at
/var/lib/swarm-bao-tls/agent-ca.pem and composes the listener bundle.
- The listener's tls_client_ca_file is a new listener-client-ca.pem
(client-ca.pem, then the agent CA). Host cert-auth roles still pin
client-ca.pem, so an agent leaf satisfies no host role. swarm-bao-certs
composes the same bundle before openbao starts.
- openbao reads tls_client_ca_file only at start, so when the bundle
changed after openbao started, swarm-bao-agent-pki restarts
openbao.service in the container; under `seal = "shamir"` it prints
the step instead. Once swarm-bao-certs has a cached CA, later boots
start openbao with it and do not restart.
- The controller policy gains exactly `update` on
pki-agents/issue/swarm-agent. mint_and_verify now asks that role for
the leaf (the store generates the key), writes the agent's cert-auth
role pinning the issuing CA bao returned, and writes the agent's
policy as render_agent alone: the hive-shared queue credential stanza
is gone.
- deploy.bao.agentPkiRoleName (must start `swarm-`, asserted with the
other pki role names); swarm-controller gets
SWARM_CONTROLLER_AGENT_PKI_MOUNT/_ROLE from the deploy.bao options.
Deleted: swarm-controller-agent-ca and its options (agentCaFile,
agentCaKeyFile), env, LoadCredential entries and assertion;
agent_identity's Authority, rcgen signing and validity window; the
rcgen and time dependencies of swarm-controller (rcgen leaves the
workspace); policy::render_agent_with_queue and its tests. The CN-prefix
assertion policy.rs said was owed is not: agent and host roles pin
different CAs.
Migration is re-creating each agent after deploy; that overwrites the
stale role and policy.
Closes#4756
#4756 moves agent client certificates onto a PKI mount of their own,
`pki-agents`, whose root bao generates internally. The unit that sets
that mount up runs as the bao granter, and the granter's policy is only
written while #4754's one-time bootstrap token is in place. Adding these
grants after an operator has done that step would cost a second token
placement, so they go into the granter's policy here, before it.
Six stanzas: enable and tune the mount, list its issuers, read its CA,
generate its root internally, and write `swarm-*` roles on it. No root
delete or sudo: agent cert-auth roles pin that root by value, so
replacing it must not be something a deploy can do.
Adds deploy.bao.agentPkiMountPath (default `pki-agents`), which the
stanzas are rendered from. The module-eval case pinning the granter's
policy now lists all seventeen stanzas, and the "grants nothing outside"
case also refuses the agent mount's root, issue, sign and a roles/*
glob.
Every unit that writes a bao policy or cert-auth role ran only while the
operator-placed bootstrap token existed, and skipped silently otherwise.
The token lives 24h, so on any real swarm a PR adding or changing a grant
deployed with its unit skipped, and each one needed a manual token refresh
(plus a root `bao policy write` when it added a path).
A `bao-granter` principal now writes them. Its leaf is minted by
swarm-bao-pki on the store host (0600 root, never copied off it), and its
policy covers `swarm-*` policies, `swarm-*` cert-auth roles and
`pki/roles/swarm-*` by glob, plus the mount and services-root paths the
controller's unit already used. All ten granting units
(controller, secret-publisher, matrix-ctl, matrix-token, queue-agent,
grafana-oidc, otel-oidc, forwarder-oidc, services-issuer, nats-tls) log in
with it instead of reading the token. They keep the 2880 x 30s retry, now
require swarm-bao-pki, and when the store refuses the granter they fail
and print the one-time step instead of skipping.
swarm-bao-granter-role is the one unit left on the token. It enables the
auth mounts (moved out of the controller's unit) and writes the granter's
own policy and role. The bootstrap policy is renamed `bao-bootstrap` and
shrinks to those five stanzas; it is shipped at
/etc/hyperhive/bao-bootstrap-policy.hcl. The old name `swarm-bootstrap`
matched the granter's own `swarm-*` glob.
The granter's CN joins certAuthCns, so no hive can be named into its role.
An assertion keeps both pki role names under `swarm-`. With no client CA
the granting units no longer render, and a warning says so.
module-eval pins the granter's policy stanza by stanza, what it cannot
reach, that every call a granting unit makes is granted, and that only
swarm-bao-granter-role reads the token.
Refs #4704
hive-screen-mcp ran `grim` / `wtype` through an unbounded
`Command::output()` and spoke RFB to neatvnc with no deadline, so a
wedged compositor or a VNC server that accepts and never speaks held the
agent's turn forever. Each subprocess now has a 30s limit and is killed
when it hits it; each RFB exchange has a 10s limit. Both come back to the
agent as the tool's text result, like every other failure in this crate.
hive-c0re operator-facing text that described behaviour the code lacks:
- `hivectl matrix reset-password` printed a "next: hivectl matrix
create-user" hint that fails for every target (agents are refused, a
non-agent hits M_USER_IN_USE). The line is gone.
- a failed `nixos-container update` appended the container's journal
tail, read with `journalctl -M`. Since the job DAG, `update` only runs
from the `Swap` node on a stopped container, so the read always came
back empty. The helper is removed; the error still points at the
build log.
- the matrix sweep comment in main.rs said it re-provisions agent token
files; `ensure_all` creates no agent accounts or tokens.
- the knowledge-pull comments named a webhook caller that no longer
exists and claimed a race was "fixed at its source".
- `handle_spawn`'s doc and the `mcp_sockets` module doc named callers of
`register_agent` / a rollback that do not exist.
Refs #4723
mara chose to accept extra queued sweep passes over adding a new
hive-jobq primitive (or a hive-c0re one-off) for "don't queue another
of this kind". Every sweep caller now plain-inserts its node; the
capacity-1 Dep::Resource per sweep kind (MatrixSweep/KnowledgeTree)
still keeps two passes of the same kind from running concurrently, it
just no longer collapses a tick that lands while one is live or queued
into the existing one.
The matrix sweep and the /knowledge pull each had concurrent callers
(#4723 item 4). Two overlapping knowledge pulls fail on .git/index.lock
and the remote-tracking ref lock: 30 of 30 concurrent replays of the
reset/clean/pull sequence in a scratch repo errored, 0 of 10 sequential
ones did. Two overlapping matrix sweeps on a hive with no persisted
Space / chat-room id both miss the by-name lookup and both createRoom
(from reading the code, not reproduced against a homeserver). On every
boot the MatrixSweep DAG node and the main.rs loop's immediate first
call ran at once.
Every sweep now runs as a job node, and each sweep's node holds its own
capacity-1 queue resource (Resource::MatrixSweep,
Resource::KnowledgeTree), the MetaWindow pattern: the scheduler never
starts a second pass of one sweep while the first holds the resource,
and different sweeps still run side by side.
- templates::matrix_sweep / templates::knowledge_pull build the node
with its resource; boot, the periodic loops and the swarm event all
use them.
- JobQueue::insert_unless_live folds a submission into a live node of
the same kind instead of queueing another. Periodic ticks fold into a
queued or running pass. The swarm knowledge event folds into a queued
pull only, and queues one behind a running pull, which may have
fetched before the push.
- The main.rs matrix loop no longer sweeps immediately at startup; the
boot MatrixSweep node is the startup pass, as KnowledgePull already
was for knowledge.
- The executors bound each pass (10 min matrix, 5 min knowledge), since
a hung pass would otherwise hold its resource against every later one,
and own the sweep-health banners, so every pass reports to them.
Replaces the SweepLock version of this branch, per review.
Refs #4723
