A role file is read into every subagent dispatch that names it, and
nothing measured its size. A new watch on the same 15-minute tick as
claude_md_watch records `hyperhive.agent.subagent_role.lines`, one
series per `<role>.md` under the state dir's `subagent_roles/`,
labelled `role` with the file stem. Metric only: no todo, no threshold.
Fixes#4558
Every classification's shared context began with the PR's head sha,
headed each commit with its sha, and carried each file's
`index <hash>..<hash>` line in the diff. A push or rebase rewrites all
of those, so the provider's prompt cache missed from the first line of
the user message even when the commit messages and hunks were
unchanged (#5245: "cache hits show as 0.1x").
The header now names only repo#pr, each commit is headed by its
position, and the diff drops `index` lines before lockfiles collapse.
The `diff --git` paths and every hunk stay. The TryOut's head_sha is
untouched: the controller still posts the review against it, and the
journal line still logs it.
On the ACP runtime the cost sink also adds each classification's tokens
to `hyperhive.review_classifier.token.usage`, typed input / output /
cacheRead / cacheCreation like claude's own `claude_code.token.usage`,
so the hit rate is a number in the metrics store on both runtimes.
No access rule covered the push route on `metrics.<swarm>`, so a request
there fell to `default_policy = one_factor`: any logged-in session cookie
for the swarm could write samples into VictoriaMetrics.
The name now gets two rules with no `resources`: the collector's client
at `one_factor`, then `deny`. Only the push route and `/metrics` on that
vhost send an authelia subrequest (the read API, Grafana's datasource
included, is served with no check), and the collector is the one caller
of both, so a domain-wide pair covers them with no path or query regex
to slip past. It replaces the per-path `/metrics` pair the published
target would otherwise derive for that name. Both rules sit ahead of the
apex and store-UI rules, and authelia matches `domain` exactly, so they
shadow no other name.
Authelia 4.39.20, read from source: a rule with no `domain` wildcard
matches with `strings.EqualFold` (access_control_domain.go), an empty
`resources` or `subject` list matches everything (access_control_rule.go),
and the first matching rule wins (authorizer.go).
Fixes#5218
The store's own /metrics was scraped over loopback, declared under
deploy.victoriametrics.enable, so a collector on another host never had
a job for it. Declare it as a published target on every host instead,
keyed on swarm.victoriametrics.domain, and serve it on the metrics vhost
behind the existing authz subrequest. Authelia's per-target rules (one
collector rule, then deny) now cover it too.
The loopback entry goes, so a single-host swarm scrapes the store once.
The ACP runtime handed the classifier's system prompt to hive-runtime's
system_prompt_file, which prepends it to the first user message, so the
instructions and the PR text sat in the same user turn.
The rendered opencode.json now defines a review-classifier agent whose
prompt is {file:./system.md} and selects it with default_agent. opencode
1.15.10 sends an agent's prompt as the system-role message (in place of
its per-model default prompt), followed only by its environment block;
the first user message is the PR material alone. The API key runtime
keeps --system-prompt.
The agent carries no permission or tools key, and a test pins its exact
shape: opencode merges an agent's own permission after
OPENCODE_PERMISSION, so a permission there would offer tools again
(checked on real opencode: {"*":"allow"} on the agent offered all ten
tools despite OPENCODE_PERMISSION={"*":"deny"}).
Follow-up to #5214.
`swarmctl login` (and any `--swarm` verb) on a controller host failed TLS
with `UnknownIssuer` against the swarm's own gateway: its leaf chains to
the store's `pki` root, which no system store holds, and the only way
through was `--ca-file`/`SWARMCTL_CA_FILE` by hand.
The host's swarmctl wrapper now sets `SSL_CERT_FILE` to the bundle the
swarm-controller daemon already trusts (`/run/swarm-controller-ca/
trust-bundle.pem`: system CAs plus the swarm's root), when the gateway is
self-signed. swarmctl's reqwest/rustls resolves roots the same way the
daemon's does, so this is the path the daemon uses to reach the same
gateway. `--set` rather than a default, since a nix shell exports its own
`SSL_CERT_FILE`.
Pointing `SWARMCTL_CA_FILE` at the bundle instead would not work (#5243):
`--ca-file` adds each certificate as a root and fails the whole client on
one it can't parse.
Hosts without the controller (an operator laptop) still need `--ca-file`.
Refs #5239Fixes#5243
#5215 moved authelia's scrape to `publishedScrapeTargets` and removed
its loopback entry from the authelia module. It was rebased at merge
onto c06aa78b6, which had already moved that loopback entry into
glue-otel-scrape-targets.nix, so the entry survived there. Every host
running authelia and the collector, `singleHostSwarm` included, then
failed swarm-otel.nix's job-name assertion.
The glue now declares only grafana and nats. forge-placement's case
that pinned authelia on loopback now pins its absence, and
swarm-otel-identity gains a case reading the assertion itself on the
single-host fixture, since the option-level case cannot fail a
rebuild the way the assertion does.
Fixes#5247
#5177 added five forge requests under the controller's token that went
straight to the forgejo client: the commit status in
`Client::create_commit_status`, and in `forge/builder_account.rs` the
hives-team add, the user read, the repo-creation lockdown and the team
membership read. Each now goes through `Api::call`, so a 401 re-reads
the token from the store and sends the request once more, like the
other controller forge calls.
The read-back of the builder's newly minted token in
`ensure_builder_forge_token` stays a direct client: it sends the
builder's own token, not the controller's, the same as the read-backs
in `agent_token`, `classifier_account` and `hive_token`.
The controller read its forge token from the swarm secret store once, when
its client slot filled, and kept sending it until the daemon restarted. A
token the forge re-minted or revoked left every forge-backed path failing
with 401 until someone restarted the controller, and a restart drops the
runtime-only job graph.
Every forge request now goes through `forge::api::Api::call`. On a 401 it
asks a `StoredSecret` (the queue clients' debounced, single-flight re-read
with backoff) for a token to try instead, rebuilds the forgejo client with
it, and sends the request once more. A second 401, or a store still holding
the refused token, surfaces the error as before. Only 401 counts: unlike an
IdP, the forge answers 400 to a malformed request, and a re-read started by
one would put a real 401 that follows inside the backoff.
forgejo-api bakes the token into its own reqwest client's default headers
and takes no client or middleware from outside, so there is no single
request function to hook. Each `self.api.<endpoint>(..)` site becomes
`self.api.call(|api| api.<endpoint>(..).into_future())`; option structs a
site moved into the call are cloned, since the closure can run twice. The
`OnceLock<Arc<Client>>` slot and its `get()` callers are unchanged.
Refs #5017
Authelia 4.39.20 matches `resources` against the cleaned path plus
`?<query>` (authorization/types.go:94 NewObject, utils/url.go:11-34
URLPathFullClean), while nginx's `= /metrics` serves the request whatever
its query. `^/metrics$` therefore missed `/metrics?x=1`: both the collector
rule and the trailing `deny` were skipped and a plain session reached
`default_policy = one_factor`. The pattern is now `^<path>([?].*)?$` for
every published target.
Forgejo strips trailing slashes before routing (routers/common/
middleware.go stripSlashesMiddleware) and serves metrics without a token
when none is configured, so `forge.<swarm>/metrics/` reached its metrics
through the ungated `/` location. The forge vhost now answers
`= /metrics/` with 404; nginx merges slashes and decodes `%2F` before
matching, so that also covers `/metrics//` and `/metrics%2F`.
Part of #5205
Authelia's metrics were scraped over loopback, declared only on a host
running both authelia and the collector, so a split deploy collected none.
Its target is now declared on every host as
`https://<swarm.authelia.domain>/metrics`, next to forgejo's, and authelia's
vhost serves `= /metrics` behind the same collector auth subrequest as the
forge's. The loopback entry is gone, so a single host still scrapes
authelia once, now by name.
The forge-only `metricsRule` becomes `metricsRules`, derived from every
`publishedScrapeTargets` entry. Each target gets the collector-scoped
`one_factor` rule followed by an unconditional `deny`. The `deny` is
needed because authelia 4.39 skips a rule whose `subject` does not
match and moves on. With nothing after the collector rule, any other
authenticated session reached `default_policy = one_factor` and was
admitted. That was already true of the forge's `/metrics`, and this
change closes it there too.
Part of #5205
The wg-hive allowedTCPPorts rule gated on deploy.wireguard.enable was a
cross-service toggle read in swarm-nats.nix; it now lives in
glue-nats-wireguard.nix under the same condition and value.
`swarmctl login --swarm X` failed with "unexpected argument", and
`login --help` listed no `--swarm`: the flag was only accepted before the
verb. `--swarm` and `--ca-file` are now global, so they parse on either side
of the subcommand and show in every subcommand's help.
`--swarm` / `SWARMCTL_SWARM` accept a bare domain and prepend `https://`.
A value with a scheme is parsed as before, so `http://` is still refused.
On a swarm-controller host, the swarmctl wrapper sets
`SWARMCTL_LOGIN_DEFAULT_SWARM=https://<swarm.domain>` (`--set-default`).
Only `login` reads it, after `--swarm` and `SWARMCTL_SWARM`. The wrapper
does not set `SWARMCTL_SWARM`: that would send every socket verb (agent
create, schedule, inference, forge) through the gateway and make each need
a login. Other verbs resolve exactly as before.
No CA file is wired. The only CA material the module names is the
controller's assembled trust bundle (system CAs plus the hive anchor).
`--ca-file` hands every certificate in it to rustls-platform-verifier as
an extra root, and that fails the whole client on any one it cannot
parse, so pointing it at the system CAs is a risk nothing here can test.
The controller daemon trusts the same CA through `SSL_CERT_FILE`, which
skips unparsable certificates; using that for swarmctl is left open.
Machines that are not swarm-controller hosts (an operator laptop) get no
default; that case waits on an answer in the issue.
Fixes#5239
Nothing exercised the real record path: the acp tests use a mock sink and
the cost tests only covered amount(). Split the counter construction and
the add out of Otel::record so a test can run them against an in-memory
exporter, and assert the counter name, the amount and the four labels.
Dev-dependency: opentelemetry_sdk's `testing` feature (same crate; adds
tokio to its lockfile entry).
The swarm settings gain `runtime` (`api-key`, the default, or `acp`) and
an `acp` provider record: an https base URL, the provider's model, and
its prices per million tokens. A PUT now changes only the fields its
body names, so the settings form that sends model, effort and limit
leaves the runtime alone; `null` clears `effort` or `acp`. The body stays
strict, and the stored document keeps #5192's lenient read.
The controller sends the runtime in each run request. On `acp` the
classifier runs one prompt on a new `opencode acp` session per
classification through hive-runtime's AcpRuntime, in a scratch
directory, with a config rendered from the settings. opencode runs
`read` and other tools without asking, so OPENCODE_PERMISSION
{"*":"deny"} takes every tool away from the model, and a deny-all policy
refuses any request on top. The title call, which resends the prompt,
is turned off. The deadline cancels the turn, and a turn that does not
end within 15s of that is dropped, which also covers an agent stuck
before its prompt. The answer is the final assistant text, as ACP has
no result event.
The provider key is read per run from
swarm/services/review-classifier/acp-provider, which the bao UI's
admins login may create and update, and reaches opencode through
AcpRuntime::with_provider_key and the trampoline's allowlist.
Each ACP classification's cost is computed from the configured prices
and the reported tokens, and added to
hyperhive.review_classifier.cost.usage labelled repo, pr, model and
currency.
Part of #5096
The hive collector's `settings` was `{ extensions.file_storage = …; … }
// lib.optionalAttrs senderAuth { extensions.oauth2client/swarm = …; }`.
`//` is shallow, so whenever senderAuth held (every valid hive config)
the right-hand `extensions` replaced the left one and `file_storage`
vanished, while `service.extensions` still listed it. `otelcol validate`
then failed the config.yaml build:
Error: service::extensions: references extension "file_storage" which is not configured
Broken by 02831fb5e (PR #5178), which added `extensions.file_storage`
to the left-hand attrset; the `//` block for the auth extension predates
it (cb787997bd). Both extensions now live in one `extensions` attrset.
module-eval-hive-otel gains a case that every name in
`service.extensions` is defined under `extensions`; it evaluates false
on the parent commit and true here.
Fixes#5240
swarm-controller README, the swarmctl inference doc comment (and the
regenerated swarmctl-cli.md) and the hive-agent swarm_inference module doc
still listed only model, effort and subagent models.
docs/swarm/inference.md had a table-row suffix pasted before the H1 by a
bad substitution, and the PUT row had lost it; the text is back in the
row.
A failed read of the agent-inference key already applied no cap (an
unreachable swarm must not block subagent work) and warned, but the
warning did not say which agent. It now names the agent and the error,
a test pins that, and subagent.md says a failed read starts anyway.
The max-subagents field stays a text field with a digit pattern: the
review-settings page's numeric field (concurrent reviews) is a TextField
with pattern="[0-9]+" and validation, and no field in swarm-ui uses
type="number". The new field gets the same pattern.
Nothing capped how many subagents an agent runs at once; the operator
wants to set that per agent from swarm-ui (#5234, shape (b)).
AgentInference gains `max_subagents: Option<u32>`. An absent field
decodes as None, so every stored value still parses and means "no cap",
which is today's behaviour.
swarm-controller accepts 1..=32 on PUT. 0 would refuse every start; 32
is far more concurrent claude sessions than one agent container has
memory for, so anything above it is a typo, not a cap.
The subagent daemon already read the agent's agent-inference key for
subagent_models; it now reads it on every start and continue (not only
those naming a model) and refuses a new run once the cap's worth of
names is tracked, counted under the same lock as the name reservation.
Every tracked slot counts: running, starting, a goal run between turns,
and an interrupted one still exiting. A goal re-prompt never goes
through the reservation, so it is never refused. A failed read applies
no cap, as it falls back for models.
swarmctl's `inference set` does GET, merge, PUT; its mirror type now
carries the field so a set from the CLI keeps a cap set in swarm-ui
instead of dropping it, and `get` prints it.
The builder reads its queue client secret at start and again when
authelia refuses it. The credentials table in builder.md and the
swarm-builder README still said it was read only at start.
argus's review on #5177: the git credential helper answered every `get`,
so a config PR's own flake inputs could have git send the builder's forge
token to any host. The helper is now configured under
`credential.<forge-url>.helper` and itself answers only an https request
whose host is the forge's; a test runs it as git does and checks that
other hosts, http and non-get operations get nothing.
Builds are bounded: every `build_agent` node needs the one `builder` jobq
resource, so one runs at a time and the rest wait pending, and a newer
head cancels its PR's pending build (a running one finishes and writes on
its own commit). The builder also holds a one-permit semaphore around each
build.
`builder` and `review-classifier`, the forge logins swarm-controller owns,
join nix/reserved-names.nix, with a test tying the list to both constants.
The builder is now opt-in like the CI runner: `allSwarmServices` no longer
turns it on, since it runs unreviewed config PRs through the host's nix
daemon. mara had not answered atlas's question on the PR.
A new `swarm-builder` service, in its own container on the host that sets
`deploy.builder.enable` (defaulted on by `allSwarmServices` where the host
has the builder's store leaf, like the review classifier).
When an `agent-configs/<agent>` PR is opened, reopened or synchronized,
swarm-controller queues a `build_agent` jobq node. The node sets a
`swarm-builder` commit status to pending on the PR head, sends
`{agent, rev, pr}` to the builder over NATS request-reply
(`$SWARM.builder.build`), and writes success (store path), failure (last
line nix printed) or error (no builder / no reply) on the head. The node
fails when the build does, so the Jobs page matches the PR.
The builder builds nix/builder/check-flake's `check` with every input
overridden: hyperhive and nixpkgs to the host's own
`c0re.hyperhiveFlake`/`c0re.nixpkgsFlake` (what a hive there renders into
its meta flake), and the agent to the PR head on the forge. Evaluation is
pure and runs in the container; builds go to the host nix daemon as an
untrusted client. The private config repo is fetched with the token of a
new read-only `builder` forge account (in the `hives` team), which the
controller mints into `swarm/services/builder/forge/token`; git gets it
from a credential helper reading the child's environment, so it is in no
argument or URL.
Credentials: a `builder` store leaf minted by glue-bao-tls on the store
host, a `swarm-builder` reader policy (forge token + its own OIDC client
secret), an authelia machine client, and a `--builder-client` grant in the
queue's auth-callout responder. The controller gains a grant on the token
path and the secret publisher a deny on it.
This is the agent-config build check atlas scoped on #5081 (slice 3):
nothing is pushed to a cache, nothing is written to KV, and a red status
gates no deploy. The check composes agent-base + the config without the
values a hive adds (CA, docs, claude-code path, ports, input follows), so
it can disagree with a hive's own build.
`recovery().enable()` refuses only when a key backup exists. It then makes
a new secret-storage key and sets it as the account's default regardless
of the default already there, so an account with secret storage but no
backup (set up in another client, backup since deleted) would have its
default key replaced and its other clients' recovery key locked out.
`recover_or_enable` now asks `secret_storage().is_enabled()` before
`enable` and returns an error instead, which `ensure_recovery` logs at
WARN with the account and the passphrase's store path. The decision is
behind a small trait over the matrix-sdk calls so a fake can pin it: with
no backup and secret storage present, neither `enable` nor `recover` runs.
Review: argus on #5188.
An agent's matrix encryption keys lived only in its local crypto store, so
`destroy --purge` or a primary-account token reset lost every encrypted
room's history with no way back (#5022).
swarm-controller now mints a random passphrase (32 bytes from the kernel's
CSPRNG, base64url) for every account listed under each agent's `matrix/`
directory, at `swarm/agents/<agent>/matrix-recovery/<account>`, in a
five-minute pass. The write is a cas=0 create after a read; a path that
holds anything is never replaced, and a create that loses a race counts as
done. The controller's existing `data/swarm/agents/*` grant covers the
write and the agent's own `agents/<agent>/*` read grant covers the read, so
no policy changes.
hive-matrix-daemon reads the passphrase when it brings an account up, after
cross-signing: with no backup on the homeserver it enables recovery with it
(backup + secret storage); with one there it recovers, which imports the
backup key and the self-signing key, and room keys then download through
`BackupDownloadStrategy::OneShot`. No passphrase yet logs at INFO and the
account comes up without backup; any failure logs WARN and never stops the
daemon.
Recovery runs after `ensure_cross_signing`, not before: on a new account
cross-signing must exist locally for `enable` to export it into secret
storage, and on an existing identity `bootstrap_cross_signing_if_needed` is
a no-op, leaving `recover` to import and sign.
Part of #5022 (S1+S2, with the S4 docs).
All agents on a host reach the gateway from one address and share one
limit_req budget; a host boot is one cache miss per agent at once.
Defaults go from 5r/s burst 20 to 50r/s burst 500. The module-eval case
reads the option values.
Any request on logs.<swarm> carrying a well-formed swarm-agent bearer
made swarm-logs-auth do a bao cert login and KV read before it could
refuse, with no cap on the rate.
nginx now applies limit_req (zone swarm_logs_agent_auth, keyed by
client address, 5r/s burst 20 nodelay by default) on the internal
location that forwards to swarm-logs-auth, answering 403 when over,
since auth_request passes through only 401/403 and swarm-logs retries
a 401. The authelia path and the query location carry no limit.
swarm-logs-auth remembers bearers it has granted, keyed by SHA-256 of
the bearer, for --verdict-ttl seconds (deploy.victorialogs.authVerdictTtl,
60 by default), holding at most 4096 and evicting the oldest. Denials
are never remembered. A replaced or deleted queue secret therefore
keeps working at the log store for up to the TTL after its last check.
Closes#5190
The publisher's `enable` default read `deploy.authelia.enable` from the
publisher's own module (swarm-secret-publisher.nix). It now defaults to
`false`, and glue-secret-publisher-authelia.nix sets it `mkDefault true`
wherever authelia runs, so the effective value is unchanged. The option
loses its `defaultText`, and its description names the glue.
The `deploy.authelia.enable` read in `active` stays: every file the unit
copies is under authelia's `hostClientSecretDir`, so it is a requirement
of the unit, not a co-location toggle.
Part of #3742.
The forgejo entry in `swarm.otel.publishedScrapeTargets` was declared
under `deploy.forgejo.enable`, so it existed only in the forge host's
evaluation. Its two readers evaluate elsewhere: the collector renders
its prometheus jobs from its own host's list, and authelia registers
the collector client's audience from its own host's list. With the
forge, the collector and authelia on different hosts, the collector
had no forgejo job and authelia would have refused its token.
Declare the entry unconditionally in swarm-otel-service.nix, keyed on
`swarm.forge.domain`, which every host evaluates identically. A swarm
with no forge scrapes a name that answers nothing and shows a failing
job, the same trade the collector and grafana OIDC clients already make.
Single-host render is unchanged: the entry already existed there.
Part of #5205
Each store reader (forge's OIDC fetch and token publisher, matrix's OIDC
fetch, the hive collector's fetch, Grafana's OIDC fetch and admin password,
the queue's callout-xkey unit, the CI runner's token fetch, the swarm
collector's OIDC fetch) used to read `deploy.bao.enable` in its own module
to add `after`/`wants` on `container@<store machine>` and
`after`/`requires` on `swarm-bao-pki.service`. Per the #3742 ruling, that
cross-service read moves into glue-bao-readers-store-order.nix, which sets
the same edges when the store and the reader are both on this host and
restates each reader's own gate so no stub unit is rendered.
Rendered `after`/`wants`/`requires` are identical as sets to main on 16
fixtures, store here and elsewhere. New module-eval cases in existing
fixtures pin both arms.
Part of #3742.
The hive collector's secret-wait unit read deploy.authelia.enable to
order itself after authelia's container. That ordering is now set by
glue-hive-otel-authelia.nix, which restates the unit's own gate (otel's
`waits`). The rendered unit is unchanged.
The hive-otel module-eval fixture `hiveOtel` gains authelia, and a case
pins the edge with authelia here and its absence without.
Part of #3742.
Grafana runs with metrics.enabled and no basic_auth_*, so its Prometheus
exposition was served to any anonymous client of grafana.<swarm> through
the published vhost's `location "/"`. Add an exact-match `= /metrics`
location returning 403 there. The loopback `grafana-metrics` vhost and
the collector's scrape of it are unchanged.
Comments on the metricsPort option and the loopback vhost now state that
the published vhost refuses /metrics; the gateway doc says the same.
A module-eval case pins the refusing location and that `/` still proxies.
An answer whose reason ran past 500 characters was rejected whole, so the
verdict was lost and the one error row turned the review into no_verdict.
The reason is cut to the cap, ending in an ellipsis; verdict and certainty
are kept.
Forgejo's markdown renders $...$ and $$...$$ as KaTeX; a reason quoting
nix `${...}` put two $ in one cell and the span between them became a KaTeX
parse error. The cell escaper now backslash-escapes $ like its other markup.
Rule source: the math inline parser triggers on '$' and only starts a span
there (modules/markup/markdown/math/inline_parser.go, Trigger/Parse in
go-gitea/gitea, which Forgejo carries); goldmark's CommonMark backslash
escape consumes '\$' first, so the parser never sees it. Escaping each $
also covers '$$'. Backslashes are already escaped, so '\\$' stays literal.
deploy.swarm-ui.enable and swarm.ui.enable are declared again as
nullOr bool, default null, and do nothing. Setting either adds an eval
warning that swarm-ui now runs wherever swarm-controller runs.
Remove deploy.swarm-ui.enable. Every reader (the swarm-ui vhost, the
authelia operator rule for the swarm apex, hive-network's
anyServiceEnabled) now reads deploy.swarm-controller.enable, and the
controller's webhook public URL is gated on the swarm domain alone,
since the vhost it names exists wherever the daemon does.
Both spellings of the old switch (deploy.swarm-ui.enable and the
already-renamed swarm.ui.enable) become mkRemovedOptionModule entries,
so a host that still sets either fails evaluation with a message
naming the controller toggle instead of silently changing.
Ruling (mara, #3742): "swarm-ui always runs wherever controller runs".
Part of #3742
The responder re-read its secret on any non-2xx introspection answer.
With the re-read backoff added in the previous commit, an IdP outage
(503s, 502s from nginx, 429s) made each re-read find the unchanged
secret, so the delay grew to minutes. A real 401 after the rotation
then landed inside that window, and the responder denied every new
NATS connect until it ran out.
`Answer::Refused` now carries the status. `OidcSecret::introspect`
re-reads only when `stored_secret::is_credential_refusal` says the
status is 400 or 401, the same predicate `ClientSecret::refused` uses on
the client path. Every other non-2xx is a denial with no re-read.
The client-path test for a 429 now runs the mint against a loopback
endpoint answering 429 and 503, instead of calling `refused()` directly.
swarm-queue-client's README no longer names `ClientSecret::Value`.
Part of #5017 (slice Q).
hive-c0re (`hive-<name>`), swarm-controller and swarm-review-classifier
read their queue client secret from the store once and held it as
`ClientSecret::Value` for the life of the process. After authelia
re-minted a secret and swarm-secret-publish stored the new one, every
later token mint presented the old value and was refused until the
daemon restarted.
`ClientSecret::Stored` replaces `Value`: a `StoredSecret` holds the
secret in memory with a caller-supplied read callback, so the crate
takes no store dependency. When the token endpoint answers 400 or 401,
`mint_token` (and its blocking sibling, used by the controller's OTLP
exporters) re-reads it and, if the store holds a different secret,
mints once more with it. One re-read runs at a time and is spawned so
it outlives a caller that gives up. A re-read that fails or finds the
refused value delays the next one, 5 s doubling to 5 min, so a secret
that stays wrong costs one store read per delay. Logs name the client
id and never the secret.
swarm-nats-auth's `oidc_secret` was the template; its re-read machinery
now sits on `StoredSecret`, so the tree has one copy. The responder
gains the same re-read backoff.
The controller keeps its `IDENTITY` OnceLock: it now holds a
`QueueConfig` whose secret is re-read, and the lazily built vcs-metrics
exporter reads the identity from it.
agent_renewal.rs's module doc said a re-minted agent queue secret only
reaches the agent at its next start; hive-agent reads the store on every
connection attempt after its first, so the doc now says that.
Part of #5017 (slice Q).
`swarm-bao-agent-pki` carried its own copy of the mtime-vs-start compare,
with the same `stat -c %.6Y | tr -d .` read: on a comma-locale host the
`-le` errored and, under `seal = "pkcs11"`, openbao was restarted on
every run of the unit.
The compare moves out of `refresh_unit` into `written_after_start`
(0 newer, 1 not newer, 2 unreadable and logged at `err`), and both
`refresh_unit` and the agent-pki unit call it. The agent-pki unit cannot
call `refresh_unit` itself: it restarts with `restart`, not
`try-restart`, runs a fail-closed check of the bundle between the
compare and the restart, and under shamir prints the step instead of
restarting. It acts only on 0, so an unreadable time skips the restart
as `refresh_unit` does.
`script-test-refresh-consumer` covers the three return codes in the
agent-pki call shape; `module-eval-bao-basics` asserts both seal arms
decide on that call and that no `%.6Y` read is left in the unit.
Refs #5230
