hive-ci, hive-forge, hive-matrix, swarm-authelia, swarm-bao, swarm-grafana,
swarm-nats, swarm-otel and swarm-victorialogs now import
./swarm-container.nix and drop their own copies of the stateVersion,
firewall and resolvconf lines. Each binds privateNetwork once in its
top-level let and passes it to both the host attr and the in-container
option, as swarm-victoriametrics already does.
hive-forge (25.11) and swarm-otel (the host's value) keep their own
stateVersion over the module's mkDefault. hive-ci sets privateNetwork =
true and writesOwnResolvConf = false, which leaves its firewall and
resolvconf on, as before. hive-matrix keeps its useHostResolvConf
override and static resolv.conf; its resolvconf mkForce now comes from
the module default.
Every container's system.build.toplevel drvPath and host-side attrs
evaluate identical to the parent commit.
module-eval-swarm-services-switch gains a fixture with all ten service
containers and checks that each one's in-container privateNetwork equals
its host-side value, that the nine on the host netns run no firewall or
resolvconf, that hive-ci keeps both, and that hive-forge keeps its pinned
stateVersion.
Refs #3773
ExecStartPost (not postStart, which can't take a prefix) with a
leading -: a failed enqueue -- an already-running viewer unit,
or systemctl itself failing -- must not mark the granter failed
or trigger its own Restart=on-failure.
swarm-bao-operator-viewer-policy exits 0 while the granter may not
configure auth/oidc, so it never retries on its own. On 2026-09-29 the
operator fixed the granter (swarm-bao-granter-role succeeded at 15:29Z),
but the viewer unit had last run on 2026-09-28 19:11Z on that exit-0
branch. auth/oidc/config and the viewer role stayed unwritten and OIDC
login failed until a manual restart.
The granter unit now restarts the viewer unit from ExecStartPost, which
runs only after its script exits 0. Restart rather than start, because
the viewer unit is RemainAfterExit and a start would be a no-op.
--no-block, because the viewer unit is ordered after the granter and a
blocking restart would deadlock. The link is one-way, so the viewer's
own Restart=on-failure never re-runs the granter.
OnSuccess= would not fire (the granter stays active under
RemainAfterExit), and Wants=/PartOf= either no-op on an active unit or
also propagate a failed restart and every stop.
The viewer's log message no longer tells the operator to restart it.
Refs #4772
revoke_queue_credential only ever deletes swarm/agents/<agent>/queue
(agent_queue_path + a literal "queue" suffix), never anything else
under an agent's prefix. secret/metadata/swarm/agents/+/queue matches
that exactly — `+` is bao's single-segment glob, the same form
swarm-nats-auth's read grant already uses for the data-side path.
Also rewords the module-eval test's stale note about a read/list grant
handing a "write-only principal" the version history: the controller
has held read on secret/data/swarm/agents/* since the mint-and-verify
read-before-write change, so it was never write-only on that path.
A per-agent queue credential is minted at agent creation and nothing has
ever removed it. An agent declared destroyed loses its container and
keeps its credential: a bearer secret recovered from a snapshot or a
stale capture still authenticates as that agent, so the set of usable
credentials only grows.
Delete the path the mint published, on the one transition that ends an
agent's life. It mirrors step 3 of `mint_and_verify` and no other step:
the leaf, the ACL document and the cert role are what a hive uses to
collect an agent's secrets and are re-minted on every run of the mint.
Every version, not the newest. The mint rewrites the path when the
principal it names needs correcting, so KV v2's plain delete would leave
the identical secret readable at ?version=N. That is a separately-ACL'd
path, hence the second stanza in the controller's grant -- `delete` on
metadata discloses nothing, and `update` on the data path already lets
this principal destroy any agent credential's usability.
The destroy is not blocked by a failed revocation: the declaration is
already published and refusing the call would leave an operator with an
agent they cannot tear down. The failure is logged at error instead,
naming the agent, since a silent orphan is the fault being removed.
No Rust ever minted a secret_id; approle was dead attack surface. The
bootstrap step's check-then-enable case becomes check-then-disable: if
approle is mounted, `bao auth disable approle`; otherwise a no-op.
Disabling costs `delete`+`sudo` on `sys/auth/approle`, not
`create`/`update` — verified against `bao auth disable -output-policy`
on a live dev store. The bootstrap policy grant is narrowed to match.
nix/module-eval/bao-grants.nix pins the new shape: the bootstrap policy
may disable approle, and the granter's role unit never enables it.
A five-minute pass over every agent some hive's wanted state declares as
anything but destroyed queues, per agent:
- `MintAgentIdentity` (the node agent creation uses) when the stored
certificate at swarm/agents/<agent>/bao-mtls is past half its validity,
read from its own notBefore/notAfter: day 45 of the role's 90;
- the new `RenewAgentQueueCredential` node when the queue secret at
swarm/agents/<agent>/queue is 45 days old or has no mint time. The node
re-decides, writes a fresh value with `minted_at`, reads it back, and logs
the agent and the old age.
When both are due the secret node runs after_any the certificate node,
because mint_and_verify compares the queue secret it read with the one it
reads back. A credential that is not stored is never created here.
`queue::AgentCredential` gains an optional `minted_at` (unix seconds);
agent creation now sets it. Stored objects without it decode unchanged and
count as due, so every existing queue secret is re-minted on the first pass.
Both replacements reach the agent at its next start. The old certificate
stays valid until it expires; the old queue secret does not, so a queue
reconnect before that restart is denied.
Adds x509-cert 0.2 (with der_derive and flagset) to read the validity.
docs/swarm/credentials.md: the renewal column splits into automatic re-mint
and automatic re-pull, filled from the code as it stands.
The bao UI at bao-ui.<swarm> took a raw store token and nothing else.
It now offers an OIDC tab: authelia's `admins` group logs in and lands
on `swarm-operator-viewer`, which is list+read on `secret/metadata/*`
and nothing under `secret/data/` or `sys/`.
- authelia registers an interactive client `swarm-bao-ui`
(glue-bao-ui-oidc-client.nix) with redirect
`https://bao-ui.<swarm>/ui/vault/auth/oidc/oidc/callback`; the secret
publisher carries its secret to
`secret/swarm/services/swarm-bao-ui/oidc/client`.
- `swarm-bao-granter-role` (bootstrap token) enables the `oidc` auth
mount with listing visibility `unauth`, asked before attempted like
cert/approle; `bao-bootstrap-policy.hcl` gains `sys/auth/oidc`.
- The granter's policy gains `auth/oidc/config`, `auth/oidc/role/swarm-*`
and read on that one secret leaf. It still holds no `sys/auth`.
- New granting unit `swarm-bao-operator-viewer-policy` writes the viewer
policy, and once the granter may configure `auth/oidc/config` (checked
through `sys/capabilities-self`), writes the mount's config from the
published secret and the role binding `groups=admins` to the viewer.
Before the bootstrap step re-runs it writes the policy, logs the step
and exits 0.
Route (a) per mara on #4775: enabling the auth method stays a
bootstrap-token step, re-run once on the live store.
module-eval pins the viewer policy's single metadata stanza, that the
granter's policy has no sys/auth path, the oidc enable in the bootstrap
unit, the exit-0 path, the config/role contents, and the client
registration + publish.
openbao gains a second listener, `ui`, on 127.0.0.1:<deploy.bao.uiPort>
(default 8204) with TLS off and no client-certificate requirement, and
`ui = true`. The existing listeners are unchanged. An nginx inside the
store's container, on 127.0.0.1:<deploy.bao.uiProxyPort> (default 8206),
forwards only /ui/ and /v1/ to it, redirects / to /ui/, answers 403 on
sys/unseal, sys/seal, sys/step-down, sys/rekey* and sys/generate-root*,
and 404 on everything else.
The gateway on the store's host serves `swarm.bao.ui.domain` (default
bao-ui.<swarm>) behind the authelia auth_request subrequest, proxying to
that nginx; the name joins serviceDomains and localNames like every
other gateway-published swarm service. authelia gets an access_control
rule restricting that name to group:admins, rendered wherever authelia
runs, since the default policy admits any session.
Trade-off, ruled by the operator on the parent issue: the UI listener
asks for no client certificate, so on that door a bao token alone is the
credential.
Three comments and a doc line claimed every API listener demands a
client certificate; they now except the loopback UI listener. The
module-eval case counting declared listeners excludes `ui` by name, as
it already did `metrics`.
On a self-signed gateway, the UI's name is a swarm service name, so its
host requests the services leaf from the store. `swarm-services-cert`
sits Before= and RequiredBy= the gateway's cert import, which nginx
Requires=. On a host whose only swarm name is the UI, that would hold
nginx, and with it the stream passthrough every reader dials, on a login
to a store that may be sealed. hive-tls drops those two edges exactly
when the UI is the only local swarm name: nginx starts on the existing
hive-leaf fallback, and the script's existing re-import reloads nginx
once the leaf issues. Every other host keeps both edges.
The controller's OIDC client secret (client `swarm-controller`, used for
the queue connection, the auth-bridge bearer and the OTLP push) came from
an operator-placed file, `deploy.swarm-controller.queue.clientSecretFile`,
handed in by `LoadCredential=`.
Now `swarm-secret-publish`, which already copies authelia's minted OIDC
secrets into the store, also publishes this one, to
`swarm/controller/swarm-controller/oidc/client`. That path sits under
`controller/`, which no hive's policy reads. The controller reads it once
at start with its existing store certificate and holds it in memory, as
`swarm_queue_client::ClientSecret::Value`. If the store is down, it
retries for about a minute and then fails the start, so `Restart=` tries
again.
Policy delta: the controller gets `read` on that leaf, and the publisher
gets `create`/`update` on that leaf.
Removed: the `queue.clientSecretFile` option (both spellings, now removed
options with a message), its singleHostSwarm default, the credential and
placeholder, and the path watcher plus its restart oneshot. A controller
without a store identity is now an eval error, because it has no other
way to get the secret.
swarm-bao-granter-role used its token for `bao auth list` with no check,
so an expired, revoked or policy-less token in bootstrapTokenFile exited
2 with a raw 403 and no hint. It now checks whether the store is up when
that call fails: if it is, the token is at fault, and the unit prints the
one-time bootstrap step and exits 4. A missing token file is still a
ConditionPathExists skip, so the two read differently in the journal.
The twelve granterLogin units printed "the store is sealed or
unreachable" on a healthy store, because `bao status` exited 1 there:
the CLI resolves a token helper under $HOME before asking, systemd sets
no HOME for a unit without User=, and the fallback shells out to
`getent`/`sh`, neither of which is on the unit's PATH ("failed to get
token helper: error expanding config path "": exec: "sh": executable
file not found in $PATH"). The check now runs with HOME=/var/empty and
keeps its stderr, so a genuinely unreachable store says why. When the
store is up and the login is refused, the units now name
swarm-bao-granter-role as the unit that writes the missing role.
setup.md's post-step restart used 'swarm-bao-*-policy.service', which
misses swarm-bao-agent-pki. It now names that unit too, and a
module-eval case fails when the restart misses any unit that logs in as
the granter.
Refs #4704
An `auth_token` spelled `swarm-agent.<agent>.<secret>` is no longer sent
to introspection. The responder reads `swarm/agents/<agent>/queue` with
an identity of its own, checks that the stored object names the same
agent, compares the secret in constant time, and grants the subjects
`--agent-token-publish-subject` lists with `{agent}` expanded. Every
other outcome denies: a malformed token, no store identity, nothing
stored, a failed or slow lookup, a different secret. A token without
the prefix takes the OIDC path unchanged.
The journal's `auth request` line names such a caller `agent:<agent>`;
the hive-shared credential keeps `hive-<h>-agent`.
The new principal: a `swarm-nats-auth` cert-auth role and policy with
read on `secret/data/swarm/agents/+/queue` alone, a leaf signed by the
store's PKI glue, and `glue-nats-auth-bao-identity.nix` pairing the two.
The copy unit delivers the identity into the queue's container, and an
absent leaf is delivered empty so the responder still starts and only
agent tokens are refused.
The policy and role are written by `swarm-bao-nats-auth-policy`, logged in
as the bao granter: both names fall under its `swarm-*` globs, so the
deploy writes them with no operator step. module-eval counts it among the
granting units, so every generic granting-unit case covers it.
The secret compare uses `subtle`, already in the lock file through the
TLS stack; no workspace crate offered one directly.
An agent's store identity was signed in swarm-controller's memory by a CA
a controller-host unit generated on disk, and the listener never trusted
that CA. Agent leaves now come from the store itself: a `pki-agents` PKI
mount whose root openbao generates internally, so the agent CA's key
never exists outside the store.
- swarm-bao-agent-pki (new, store host, as the bao granter): enables and
tunes the mount, generates the root once (guarded on an empty issuer
list, no replace branch), upserts the `swarm-agent` role (client
certificates named `hive-agent-*` only, 90 days), caches the CA at
/var/lib/swarm-bao-tls/agent-ca.pem and composes the listener bundle.
- The listener's tls_client_ca_file is a new listener-client-ca.pem
(client-ca.pem, then the agent CA). Host cert-auth roles still pin
client-ca.pem, so an agent leaf satisfies no host role. swarm-bao-certs
composes the same bundle before openbao starts.
- openbao reads tls_client_ca_file only at start, so when the bundle
changed after openbao started, swarm-bao-agent-pki restarts
openbao.service in the container; under `seal = "shamir"` it prints
the step instead. Once swarm-bao-certs has a cached CA, later boots
start openbao with it and do not restart.
- The controller policy gains exactly `update` on
pki-agents/issue/swarm-agent. mint_and_verify now asks that role for
the leaf (the store generates the key), writes the agent's cert-auth
role pinning the issuing CA bao returned, and writes the agent's
policy as render_agent alone: the hive-shared queue credential stanza
is gone.
- deploy.bao.agentPkiRoleName (must start `swarm-`, asserted with the
other pki role names); swarm-controller gets
SWARM_CONTROLLER_AGENT_PKI_MOUNT/_ROLE from the deploy.bao options.
Deleted: swarm-controller-agent-ca and its options (agentCaFile,
agentCaKeyFile), env, LoadCredential entries and assertion;
agent_identity's Authority, rcgen signing and validity window; the
rcgen and time dependencies of swarm-controller (rcgen leaves the
workspace); policy::render_agent_with_queue and its tests. The CN-prefix
assertion policy.rs said was owed is not: agent and host roles pin
different CAs.
Migration is re-creating each agent after deploy; that overwrites the
stale role and policy.
Closes#4756
#4756 moves agent client certificates onto a PKI mount of their own,
`pki-agents`, whose root bao generates internally. The unit that sets
that mount up runs as the bao granter, and the granter's policy is only
written while #4754's one-time bootstrap token is in place. Adding these
grants after an operator has done that step would cost a second token
placement, so they go into the granter's policy here, before it.
Six stanzas: enable and tune the mount, list its issuers, read its CA,
generate its root internally, and write `swarm-*` roles on it. No root
delete or sudo: agent cert-auth roles pin that root by value, so
replacing it must not be something a deploy can do.
Adds deploy.bao.agentPkiMountPath (default `pki-agents`), which the
stanzas are rendered from. The module-eval case pinning the granter's
policy now lists all seventeen stanzas, and the "grants nothing outside"
case also refuses the agent mount's root, issue, sign and a roles/*
glob.
Every unit that writes a bao policy or cert-auth role ran only while the
operator-placed bootstrap token existed, and skipped silently otherwise.
The token lives 24h, so on any real swarm a PR adding or changing a grant
deployed with its unit skipped, and each one needed a manual token refresh
(plus a root `bao policy write` when it added a path).
A `bao-granter` principal now writes them. Its leaf is minted by
swarm-bao-pki on the store host (0600 root, never copied off it), and its
policy covers `swarm-*` policies, `swarm-*` cert-auth roles and
`pki/roles/swarm-*` by glob, plus the mount and services-root paths the
controller's unit already used. All ten granting units
(controller, secret-publisher, matrix-ctl, matrix-token, queue-agent,
grafana-oidc, otel-oidc, forwarder-oidc, services-issuer, nats-tls) log in
with it instead of reading the token. They keep the 2880 x 30s retry, now
require swarm-bao-pki, and when the store refuses the granter they fail
and print the one-time step instead of skipping.
swarm-bao-granter-role is the one unit left on the token. It enables the
auth mounts (moved out of the controller's unit) and writes the granter's
own policy and role. The bootstrap policy is renamed `bao-bootstrap` and
shrinks to those five stanzas; it is shipped at
/etc/hyperhive/bao-bootstrap-policy.hcl. The old name `swarm-bootstrap`
matched the granter's own `swarm-*` glob.
The granter's CN joins certAuthCns, so no hive can be named into its role.
An assertion keeps both pki role names under `swarm-`. With no client CA
the granting units no longer render, and a warning says so.
module-eval pins the granter's policy stanza by stanza, what it cannot
reach, that every call a granting unit makes is granted, and that only
swarm-bao-granter-role reads the token.
Refs #4704
The pipe contract had a gap: if a producer piped into
atomic_write_secret exited non-zero after writing partial output,
cat still saw a clean EOF and wrote that partial content through to
the live target via mv — pipefail only reported the failure
afterward, once the bad write was already committed. The helper now
takes the value as its 4th argument and writes it itself with printf
(a shell builtin, so the value never touches an external process's
own argv/environ, same as a function argument never does), so there
is no pipe left to fail silently.
Callers that compute the value with a command now capture it into a
variable first (`value=$(cmd)`), which fails under `set -e` before
atomic_write_secret is ever called — swarm-bao.nix's pin.env site is
the one that needed this (`pin_env_value="BAO_HSM_PIN=$(cat ...)"`).
All seven call sites converted; output is byte-identical (same
printf '%s\n' framing, now applied inside the helper instead of by
each caller).
Refs #4723
swarm-bao-token's pin.env write (the BAO_HSM_PIN EnvironmentFile for
openbao's pkcs11 seal) had the same write-then-chmod-on-live-path
shape as the sites already converted: printf > path directly on the
live file, chmod after. Same fix, same helper. Content
("BAO_HSM_PIN=<user-pin>\n") and final mode (0400, root-owned — no
chown, same as before) are unchanged; pin.env stays at the same path,
so openbao's EnvironmentFile= reference needs no change.
Refs #4723
atomic_write_secret's cleanup trap used RETURN, which never fires when
set -e aborts the function mid-body (a failing cat/chmod/chown), so a
secret-bearing temp file was left behind instead of being removed.
The write now runs in a subshell with its own EXIT trap, invoked via a
named handler (so `local rc=$?` is a normal, shellcheck-visible
assignment) that only removes the temp file when the subshell's exit
status is nonzero — the subshell's trap table is private, so a calling
unit's own EXIT trap is untouched. Reproduced the leftover-tmp bug
against the prior commit, confirmed it's gone, and confirmed both the
success path and a caller's own EXIT trap still work as before.
swarm-bao.nix's swarm-bao-forwarder-oidc unit had the identical
write-then-chmod-on-live-path defect as the four sites already fixed
here (fetches an OIDC client secret from swarm-bao, printfs it to the
live host path, chowns/chmods after) and was missed by the original
sweep. Converted it to atomic_write_secret; content and final
owner/mode (root:root, 0400) are unchanged.
Refs #4723
`services.hyperhive.enable` and `services.hyperhive.c0re.enable` are gone.
One switch, `services.hyperhive.deploy.hive-controller.enable` (default
false, as the old toggle was), now gates hive-c0re and hive-priv. Both old
paths are `mkRenamedOptionModule` shims in deploy.nix, so a host config
that still sets either evaluates as before and gets a rename warning.
Every other read of the old toggle is resolved, including the 29 made
through the `hyperhiveCfg`/`hiveCfg` aliases:
- Dropped: each swarm service and its glue keeps only its own deploy
toggle (authelia, bao and its PKI glue, grafana, victorialogs,
victoriametrics, the secret publisher, swarm-ca, the OIDC client rows,
the controller/nats/matrix-ctl/publisher/services-issuer identities),
the forge, and the `domain` deprecation warning.
- To deploy.hive-controller.enable: the queue-agent credential reader and
its assertion, which feed hive-c0re and write under its state dir, plus
their policy-order entry; the network identity assertions; hive-tls's
two writes into hive-c0re's environment.
- hive-tls runs where the gateway runs self-signed
(`gateway.enable && useSelfSigned`), not on every host.
- The matrix appservice-token reader and its assertion stay on
`deploy.matrix.enable` plus their client-identity checks. They read
deploy.matrix's token file and registration script; their deploy.bao
inputs are the client-half options a hive sets to read a store it does
not run, so gating on deploy.bao.enable would drop the tested
remote-reader case.
- The `hiveName` assertion moves from hive-network.nix to hyperhive.nix
and fires wherever the hive, the store or the homeserver runs: each
turns the name into an identifier with no fallback.
On a host with `deploy.allSwarmServices` and no hive, the documented
services-host recipe, authelia, bao, grafana, victorialogs,
victoriametrics, the OIDC client rows and the hive CA now render; before,
the old toggle being off left them out.
Refs #4500
The store's `swarm-services` role issues the services leaf for 720h, and
`swarm-services-cert` only ever ran at boot or rebuild: it is a
`RemainAfterExit` oneshot wanted by `multi-user.target` and no timer
targeted it. A hive not rebuilt within 30 days served an expired leaf.
`swarm-services-cert-renew` runs the same script from a daily timer. It
is a unit of its own because a timer starting the `RemainAfterExit` unit
is a no-op, and restarting that unit instead would propagate through
`hive-gateway-self-signed-cert`'s `Requires=` to nginx, so a sealed store
would take the gateway down over a still-valid leaf. Nothing requires or
orders against the new unit; it has no `Restart=`, so a failure stays in
`systemctl --failed` until the next tick, and the script only moves files
into place after the store has answered.
The re-issue threshold was `checkend 2592000`, the whole 30-day
lifetime, so every run re-issued. It is now half the role's lifetime,
read from a new internal option `deploy.bao.servicesPkiLeafTtlHours`
that the role's `ttl`/`max_ttl` also read. Boot and timer share the
script and so the threshold. The services-root re-check reads the
same option, at the store's own replacement threshold (hours × 3600),
so the hive asks for a new leaf when the store replaces its root. A
`flock` keeps the two runs from interleaving one issuance's key with another's leaf.
`checks.module-eval-hive-tls` pins the timer, that the unit it starts
re-runs the issuance without `RemainAfterExit`, that nothing depends on
it, and that both the leaf and root thresholds move with the option.
Closes#4587
The matrix container renders the swarm registration before tuwunel starts
(`requiredBy` it, no network), tuwunel loads it as a second `.yaml`
credential, and a publish unit hands its token to the store. tuwunel 1.9.1
refuses only a duplicate id or as_token, not overlapping non-exclusive
namespaces, so it sits beside the hive's `hyperhive` registration.
`admin_execute` promotes exactly `@swarm` at boot. The module-eval pin
narrows from "no account" to that one list, compared whole so a second
entry fails; `admin_execute_errors_ignore` is pinned too. matrix-ctl may
write the swarm token's leaf, and the controller may only read it.
Everything is gated on matrix-ctl's store identity: with nobody to publish
the token, the registration would be an admin credential nobody reads.
bao's own in-container collector labelled every log line and metric it
forwards `service.name=swarm-bao` (`processors.resource.attributes`,
keyed off `swarm.bao.machine`), while every panel in the shipped
Grafana bao dashboard queries the literal `service.name="bao"` — no
panel has matched since the scrape moved into that collector.
Rename it in the collector instead of templating the dashboard: a new
`collectorServiceName` binding in swarm-bao.nix, deliberately not
`cfg.machine` (that value names the container/receiver, not bao's
display identity), stamps `service.name="bao"` directly. bao.json is
back to its origin/main shape, unchanged.
Adds a module-eval case to checks.module-eval-grafana that reads the
collector's own evaluated config and asserts its service.name matches
every selector the shipped dashboard uses; verified invert-proof by
setting the value back to cfg.machine and confirming that specific
case (and only it) fails.
--link-journal=host bind-mounts a host directory journald never
writes into for this container (empty, root:nogroup, confirmed on
the live host — #4527). Dropping it falls back to nixpkgs' default
--link-journal=try-guest, the same shape every agent container
already uses, so the in-container collector's journald receiver
(directory = /var/log/journal) now reads a journal that is actually
written.
merge stays true and services.hyperhive.swarm.otel.journaldUnits is
untouched so this deploy changes exactly one thing; comments that
described the old host-linked shape are rewritten to match.
Adds a module-eval assertion (bao-otel-collector.nix) that the
container's extraFlags never re-add --link-journal=host.
Refs #4527, #4499.
`swarm-bao-forwarder-oidc` fetches the store container's collector secret,
one path, and was the last reader still logging in with
`deploy.bao.clientCertFile`: the hive's own leaf, whose policy reads every
agent's credentials, the hive's tree and every service's OIDC secret. The
four-way split gave grafana's and the swarm collector's readers leaves of
their own and left this one behind.
It now holds `forwarder-oidc.pem`, minted by `swarm-bao-pki`, and logs in
under the `swarm-forwarder-oidc` cert-auth role, whose policy reads
`secret/data/swarm/services/<store forwarder client id>/oidc/client` and
nothing else. The role is written by `swarm-bao-forwarder-oidc-policy`
from the bootstrap token, which gains the two grants that unit calls, and
the reader is ordered after it. The subject is reserved as a hive name. A
store host whose pair is null is refused at eval rather than falling back
to the hive's leaf.
The hive's own role and `client.pem` are untouched; nothing is revoked.
The queue listened in plaintext on 4222, reached by bridge IP or loopback,
and nothing in-tree opened it to another hive. It now has a name, serves a
certificate for that name alone, and refuses clients that do not speak TLS.
- `swarm.nats.domain`, default `nats.<swarm.domain>`, a sibling name like
`swarm.bao.domain`. The queue host answers it via `gateway.localNames`;
every other hive resolves it through the operator's DNS, as for bao.
- `pki/roles/swarm-nats` allows that one name (bare domain, no subdomains,
IPs or localhost, server flag). A `swarm-nats` cert-auth role and policy
may only `update` `pki/issue/swarm-nats`, written by
`swarm-bao-nats-tls-policy`. The login leaf is minted by glue-bao-tls and
paired by glue-nats-bao-identity. `deploy.bao.natsCommonName` is reserved
as a hive name.
- `swarm-bao-nats-tls` issues the leaf into a directory bound read-only into
the container, restarts nats when it rotates, and re-runs daily.
It joins glue-bao-readers-policy-order, so it is ordered after its policy
unit (`after` and `wants`, never `requires`) where the store is on the
same host. The policy unit joins the store's journald list.
- nats gets `tls {}`, with the key via `LoadCredential`, and no
`allow_non_tls`. `validateConfig` is now off in every mode, because the
build-time check loads a leaf that only exists at runtime.
- 4222 is also open on `wg-hive` when the host is on the mesh, never
host-wide.
- `statusPublish.natsUrl`, `queue.agentNatsUrl`, the controller's URL under
`singleHostSwarm`, and the auth responder all dial
`tls://<swarm.nats.domain>:<port>`. swarm-queue-client hands its CA file
to the NATS connection too, so hive-c0re and the controller trust the
leaf's root.
- docs/swarm/README.md: the queue URL and the one DNS record a multi-host
swarm needs.
module-eval-nats-tls pins the role, the policy, the served leaf, the
firewall, the ordering, and a scan of every `*_NATS_URL` and the
responder's URL across the host and its containers.
Closes#4626
The units on the path a deploy takes to TLS, the store's grants and the
swarm collector itself were not on the host collector's journald
allowlist, so an ingest outage one of them caused showed in the store
only as every source going quiet at once.
Each module names its own units, per the option's rule:
- hive-tls.nix: hive-tls-ca, swarm-services-cert
- hive-gateway: hive-gateway-self-signed-cert (self-signed mode only)
- swarm-bao.nix: the seven grant units beside
swarm-bao-services-issuer-policy
- swarm-otel.nix: container@<machine>, and
nixos-rebuild-switch-to-configuration, the transient unit nixos-rebuild
runs the activation in and whose syslog lines carry its status
The module-eval arm pins each unit as both listed and defined, since a
listed name that matches nothing is silent.
A failed `bao read` of the services root made the checkend pipeline
non-zero, so the unit deleted a working root and minted a new trust
anchor. A failed `bao list` of the issuers likewise looked like an empty
mount. Both now fail the unit, which retries on its own restart budget;
the root is replaced only when openssl parsed the returned certificate
and -checkend said it expires inside a leaf's window.
Closes#4663
The container's collector has never shipped a line. `journalctl --follow`
— which the journald receiver passes unconditionally — scopes itself to
the current boot unless `--merge` is given too, and `--link-journal=host`
makes /var/log/journal the HOST's journal tree, where this container's
current boot has no entry. journalctl exited 1 with "No journal boot
entry found for the specified boot (+0)" and the receiver respawned it
every ~2s, so nothing was ever read and nothing was ever exported.
`merge = true` is the receiver's key for `--merge` (buildArgs() in
pkg/stanza/operator/input/journald/config_linux.go at tag
receiver/journaldreceiver/v0.151.0, the deployed collector's version),
and --merge is what clears the implicit boot scope in journalctl.c
(systemd v260.4, the version on the host).
The module-eval arm pins both halves: the boot filter is gone AND the
directory is still the host-linked one — either alone is satisfiable by
the broken config.
Three defects in the store-issued path, each of which alone kept nginx
from starting at all. The gateway's cert import `Requires=` this leaf, so
a leaf that is never issued is not a name mismatch — it is an empty
listener, and the swarm's own forge stopped answering on :443.
`swarm-services-cert` declared `Before=hive-tls-ca` for the trust
bundle's sake while also being `After=` the store's container, which is
itself `After=hive-tls-ca`. systemd resolved the cycle the only way it
can, by deleting the job, so the leaf went unissued on every activation.
The edge is gone; the bundle converges the other way round, through the
restart this unit already performed when the root it wrote was new.
The `pki` mount was enabled without `-max-lease-ttl`, so bao clamped the
30-year root to the 768h default and then refused every issue call,
because a leaf of the mount's own default length would outlive the CA
signing it. The mount is tuned on every run, the role pins a 720h leaf,
and a root that can no longer cover one is replaced rather than left to
refuse forever. A hive tests its own copy of that certificate against the
same threshold, so both ends reach a fresh leaf without signalling.
`swarm-bao-pki` mints the services-issuer leaf that opens the mount, but
only `swarm-bao-certs` required it. `RemainAfterExit` plus an
already-active unit means an activation that ADDS a leaf mints nothing —
which is how a host whose config named `services-issuer.pem` came to have
no such file. A target wants it now, like every sibling granting unit.
The `pki` mount had no issuer and no principal could log in to it, so the
swarm's service certificates were still minted by two openssl hops from a
root key on disk. Close both halves and retire the openssl path with them.
The mount now generates its own root, once. The granting unit asks bao
whether an issuer already exists (`bao list pki/issuers`) before calling
`pki/root/generate/internal`, so a rebuild or a reboot re-asserts the role
and the grant without touching the anchor — a root that changed per boot
would invalidate every certificate issued under it and every browser
taught to trust it. The guard asks the store rather than looking for a
marker file on this host's disk: a file is a claim about a mount that may
have been restored from a snapshot or disabled and re-enabled underneath
it.
`swarm-services-issuer` stops being an inert policy. A fourth cert-auth
role attaches it, following the shape the controller, the publisher and
matrix-ctl already use, and glue-bao-tls.nix signs the leaf carrying its
CN — that credential is what opens the mount, so it cannot come out of it.
`swarm-services-cert.service` logs in with that leaf, calls
`pki/issue/swarm-services`, and writes the result to the path
hive-tls.nix already wrote and the gateway already copies from. The
sub-CA layer does not move; it stops existing. The role's
`allowed_domains`, read from the same `swarm.serviceDomains` the SANs
come from, enforces at issue time what the sub-CA encoded in x509
`nameConstraints`, and with the root inside the mount there is nothing
left for an intermediate to be an intermediate of.
Not a flag day: the issuing root is published beside the leaf as
`swarm-services-root.pem` (0644) and joins `trust-bundle.pem`, where the
swarm root still sits. A leaf chaining to the old sub-CA and one issued
by the store both verify against the same bundle, so hives can be
rebuilt in any order. The same file is what an operator hands a browser
— readable without a store login, which matters because every listener
demands a client certificate.
The eval-time warning about uncovered service names is gone rather than
reworded. It fired on "this host does not hold the swarm root key", which
was the reason a hive could end up serving its own leaf on a
swarm-service name. Every hive now asks the store with its own identity,
so that stopped being the thing that decides.
Closes#4586
mint_and_verify reads the queue credential back before writing it, so a
re-run keeps the value a live agent already authenticates with. that read
is read_optional, which maps only a 404 to absence — so with create/update
alone every mint aborted on a 403 at its first store read.
read on the same paths the stanza already grants create and update, and
nothing else: no list, no delete, no patch.
Four units read one path each out of the store, and all four logged in
holding `deploy.bao.clientCertFile` — the hive's own leaf. Bao identifies
a principal by the subject of the certificate it presents, so four
readers behind one certificate were ONE principal, and the only grant
expressible was the union of what the four need: read on
`swarm/agents/*`, `swarm/hives/<hive>/*` and `swarm/services/*`. The unit
fetching Grafana's OIDC client secret could fetch every agent credential
in the swarm; the one fetching this hive's matrix token could fetch
Grafana's. Least privilege was not misconfigured here, it was
unrepresentable.
Each now holds a leaf, a cert-auth role and a policy of its own, and each
policy is the single `secret/data/…` path that unit's own script names —
spelled to the leaf, not to a prefix, the way matrix-ctl's already is.
Following the four exemplars in-tree rather than building a mechanism:
`signLeaf` mints the leaves, `swarm-bao.nix` writes the roles from the
bootstrap token, the consumers name their own pair.
Two of the four are written PER HIVE and two are not, which is the shape
of the paths rather than a preference. A matrix appservice token and a
queue credential live under `swarm/hives/<name>/` and every hive runs a
reader for its own, so one role for all of them would have to be granted
`hives/*` — letting one hive read another's, a reach no hive has today.
An OIDC client secret lives under `swarm/services/<client-id>/` and a
swarm registers each exactly once, so one role each is enough. The
per-hive subjects are `<prefix>-<hive>` and swarm.nix reserves every
composed spelling as a hive name, so a hive cannot be named into another
hive's role.
The shared leaf stays: hive-c0re still passes it into its container, the
`bao` CLI wrapper still defaults to it, and the three
`glue-*-bao-identity.nix` files derive the PKI directory from it.
module-eval-bao-grants gains a negative arm per principal — each pins the
three stanzas the hive's leaf carried and the two wildcards a later
widening would reach for, so a policy that grows fails here rather than
in a store. Plus the consuming side: repointing a unit back at the hive's
leaf would evaluate, deploy and log in, and silently restore the union.
A hive that reads a store on another machine now places one leaf per
principal instead of one shared by four. That cost is the point, and
docs/swarm/secrets.md lists the pairs.
`services.opentelemetry-collector` already defines
`serviceConfig.Restart = "always"` at the normal priority. The forwarder
added in #4537 defined `"on-failure"` beside it, and two definitions at
one priority are a conflict the module system refuses to resolve — so
`containers.swarm-bao` stopped evaluating at all (#4615).
Drop our definition rather than force a value over it, which is what the
sibling collector in swarm-otel.nix already does. The resolved value is
`"always"`, which is the one we want here: a forwarder that exits for any
reason, clean or not, has stopped shipping the store's journal.
`RestartSec` stays — it conflicts with nothing and is what keeps the
restarts during first-boot secret delivery under the default start-rate
limit.
bao's metrics were scraped by the SWARM collector over loopback, via a
`swarm.otel.scrapeTargets.bao` entry gated on `deploy.swarm-otel.enable`
— "does the swarm's collector run on THIS host". It had to be: loopback
only reaches a reader that landed on the same host.
What that rendered everywhere else was nothing at all. Off that host the
metrics listener was not emitted, so the store's metrics reached the
store nowhere, and a host with no entry is indistinguishable from a host
nobody asked to scrape.
Moves the scrape into the collector this container already runs, per
mara on #4537: "move the existing scraper to the local collector". The
container shares the host netns (privateNetwork = false), so the scrape
still dials 127.0.0.1 — the listener keeps its address, its
`metrics_only` narrowing and its loopback-only bind, and the API
listener's `tls_require_and_verify_client_cert` is untouched.
The listener and its `prometheus_retention_time` lose their gate: the
reader ships with the store now, so there is no host where the endpoint
has none. The metrics pipeline reuses the logs pipeline's `resource`
processor and `otlphttp` exporter, so both signals carry the same
`service.name` and leave by the one hop.
Logs are unaffected: `journaldUnits` and --link-journal=host stay until
every sibling swarm container has a collector of its own.
The module-eval absence arm "a store with no collector beside it serves
no metrics" is inverted rather than dropped — the condition it asserted
is the bug. Three cases join it: the job is in swarm-bao AND gone from
swarm-otel (a move, not a copy), the scrape target and listener are both
pinned to loopback, and the metrics pipeline shares its exporter with
the logs one.
Both earlier versions asked the wrong host. `hyperhive.otel.enable` asked
whether this host runs a HIVE collector; `deploy.swarm-otel.enable` asked
whether this host runs the SWARM one. Neither answers the question the
forwarder actually has — "is there a collector to forward to" — and that
question cannot be false: a swarm always runs at least one instance of
every swarm-level service. So the forwarder renders under the condition
already enclosing it, that the store is deployed here, and nothing else.
`scrapeHere` deliberately keeps its `deploy.*` gate one line up. It is a
loopback metrics listener, which genuinely only works where the scraper
is — the two are different tiers, and the name says so.
The module-eval case that pins it is the split topology: the swarm
collector on another host, nothing local naming it, and the forwarder
still enabled and still addressed at `swarm.otel.domain`'s route. Both
removed gates render nothing in that fixture, which the co-located ones
they shipped with could not show.
The forwarder pointed at the hive bridge address and was gated on the
hive's `otel.enable`, so it existed only where a hive collector stood
beside it. It now exports to `swarm.otel.domain` — the gateway-served
name that resolves locally when co-located and over the network
otherwise — on the swarm tier's own producer route, and is gated on
`deploy.swarm-otel.enable` like its sibling `scrapeHere`.
Refs #4526
Every container is supposed to run a collector that passes its logs and
metrics to the next hop. swarm-bao did not: its journal reached the store
only because `--link-journal=host` puts it in the host tree, where the
swarm collector — a different container — reads it through a unit
allowlist. That is the topology being retired, and in this deployment it
delivers nothing: no `_SYSTEMD_UNIT` value in the seven-day store mentions
openbao at all.
So the store's container now runs its own journal forwarder, copied from
an agent container's (nix/agent-modules/otel.nix): the whole journal, no
unit allowlist, pushed to the same first hop every agent on the host
already exports to. A local collector reads the local journal, so there is
nothing for a list of unit names to disagree with.
The `swarm.otel.journaldUnits` entry and `--link-journal=host` both stay.
Every sibling swarm container still rides the shared collector, and they
come out once each of them has a forwarder of its own.
Closes#4526
A swarm runs one homeserver and every hive on it logged in as the same
`@hive:` localpart, holding the same access token out of one swarm-wide
store path. That is one matrix identity for N hives: the homeserver
cannot attribute an action to the hive that took it, and revoking one
hive's standing revokes every hive's.
Three changes, and the third is the one that makes the other two real:
- **The localpart carries the hive's name** (`hive-<hive>`), derived in
one place, `swarm_secret_client::matrix::hive_localpart`.
`hive-matrix.nix` renders the same string as the appservice
registration's `sender_localpart`, so the shared account stops being
created rather than merely stops being used.
- **The store path is templated by hive**, not a constant. The
"a swarm runs one homeserver, so this is a constant rather than a
parameter" rationale went with it; it stopped holding the moment two
hives shared the homeserver it describes.
- **The path moved out from under the grant every hive has.** It sat at
`swarm/services/matrix/sender-token`, inside the
`secret/data/swarm/services/*` read stanza `policy::render` gives every
hive. It now sits under that hive's own stanza,
`secret/data/swarm/hives/<hive>/*`, which interpolates the reader's
name — so a hive reads its own token and is refused another's. The
policy renderer itself is unchanged: narrowing the `services/*` grant
would break the OIDC-secret read it exists for, and moving the
credential is what this needed instead. A policy test walks the
rendered stanzas and asserts none of hive alpha's covers hive beta's
sender token, so a later stanza that widened it fails here.
`swarm-matrix-ctl` takes a new required `MATRIX_MINT_HIVE` and writes
that hive's path; its store grant in `swarm-bao.nix` follows, scoped to
one hive's leaf via the new `deploy.bao.matrixCtlHiveName` (defaulting to
this host's `hiveName`) rather than a `hives/*` wildcard, which would
hand the matrix container every hive's token back.
Migration: no outage at deploy. `ensure_hive_user` short-circuits on the
local token file, so a hive keeps running on what it has; with no such
file it reads the new per-hive path, finds nothing, and falls through to
the existing register-or-appservice-login ladder against its own
localpart — which needs only the per-hive `as_token` on local disk. The
old shared object is read by nothing afterwards. Rooms do not follow the
identity, and that is the one operator step; both ways out are written
into `docs/integrations/matrix.md`.
No admin standing is granted to the per-hive accounts: `admin_execute`
stays empty and the assertion pinning it is untouched.
Renames `swarm-matrix-minter` and reshapes it around subcommands. Minting
is now `swarm-matrix-ctl mint`.
Running rust inside `containers.hive-matrix` is not free: it needs its own
store identity, its own cert role and its own bind mounts, and every one of
those is per-*container*, not per-task. A second single-purpose crate would
have had to duplicate that plumbing to add one action, so the next thing
that has to run in there should be a verb here rather than a new crate.
The old name guaranteed the opposite.
`main.rs` is clap dispatch; the minting logic moves to `mint.rs` unchanged.
A bare invocation is refused: `mint` writes a credential, so "no verb"
defaulting to it would make a typo in the unit mint rather than fail.
The environment prefix moves with it, `MATRIX_MINTER_*` → `MATRIX_MINT_*`.
Scoped to the verb and not to the binary, because a binary-scoped prefix is
one the next verb has to share or widen, and a widened one never narrows
again. A test asserts every variable carries the verb's prefix.
The principal renames too. The cert role, bao policy, granting unit, leaf
filename and `certAuthCns` entry all have to spell one string the same way,
so leaving them as `swarm-matrix-minter` would have rebuilt the naming
split this branch exists to remove. Renaming the nix options alongside is
free here: every one of them is introduced by this PR and has never been
released, so no operator config names them yet.
`ExecStart` now names the verb, which is a contract between a nix string
and a clap enum that fails at deploy time with no local signal. Both ends
assert it: `mint_is_spelled_the_way_the_unit_invokes_it` in the crate, and
a new module-eval arm reading the rendered `ExecStart`.
docs/getting-started/setup.md drops the sender token from its "live on the
host" list: setup does not touch this credential, so a setup guide has no
reason to name it.
The store path and every identifier around it called this an admin
token. It is not one: of ~15 hive-c0re call sites only two need
homeserver admin, and the homeserver no longer promotes the account at
boot, so the name overstated both what the credential is and what it may
do.
Renaming it to the account was not enough either. "The `@hive:` token"
reads as the token of a hive user, and no such user is provisioned —
`@hive:<server_name>` is the appservice registration's own
`sender_localpart`, an account the homeserver creates for itself when it
loads the registration.
So it is the **sender token**: the matrix appservice sender account's
access token, at `swarm/services/matrix/sender-token`. The name says
what it authenticates as rather than what it may do, which is the part
that was wrong.
The path has one constructor, and the bao grant, the grant assertion and
three unit tests pin its literal independently — so a half-finished
rename fails a check rather than leaving the minter and its readers
disagreeing at runtime. `tracing` messages are renamed with the code, so
the journal reads the way the source does.
The host-side file keeps its name (`matrix/access-token`): it carried no
admin framing, and renaming it would orphan the file on every deployed
hive for nothing.
`docs/tools/hivectl-cli.md` is regenerated from the clap tree.
A swarm runs one homeserver and a homeserver has one appservice sender
account, so "mint it once" is a property of the thing being minted
rather than something a lock has to enforce. That is what makes this
account the one to move first: no trigger route, no controller change
and no agent list — a boot-time oneshot beside tuwunel is the whole
mechanism.
`swarm-matrix-minter` runs inside `containers.hive-matrix`, which
already holds the appservice token: the rendered registration is bound
in read-only because that is how tuwunel is handed it. What the
container lacked was an identity of its own, so this adds one — a leaf
from the store's CA with a grant of exactly one path, not the hive's
leaf, which reads every secret in the store.
Both ends of the credential ship here. The minter reads the path it
publishes to before it touches the homeserver, and returning on a
non-empty read IS the "only once"; `hive-c0re`'s `ensure_hive_user`
reads the same path, authenticating with the hive name already in
`HYPERHIVE_HIVE_NAME`. The existing mint-then-`M_USER_IN_USE`-login
ladder stays as the fallback for a store that is empty, unconfigured or
unreachable, which is every swarm deployed before this — so nothing
needs backfilling and nothing breaks if the rest of the sequence never
lands.
The credential is not an admin credential, and is not named like one.
It is the access token of the appservice registration's own
`sender_localpart` — `@hive:<server_name>`, an account the homeserver
creates for itself when it loads the registration. The store path is
`swarm/services/matrix/sender-token`, the host path is
`matrix/access-token`, and the homeserver no longer runs an
`admin_execute` promotion for that account at boot. Everything the hive
provisions with it — the Space, the chat room, their hierarchy and join
rules, the invites — rides on being the creator of those rooms at power
level 100, not on homeserver admin; there is no Synapse admin API here
to need, tuwunel has none.
Two operations do need an admin *sender* and therefore stop working:
`hivectl matrix promote-user` and `hivectl matrix reset-password`, both
`!admin …` messages into `#admins:<server>`, plus the password-reset
recovery path that an agent with a lost password file falls back to.
They are swarm-level operations and are left failing loudly rather than
served by an over-privileged token every other call site would also
carry. The sweep's own admin-rights check and self-repair go with them:
an account that is deliberately not an admin has nothing to check.
`ephemeral = false` stays, and hive root can still read the container's
filesystem. Accepted: what this buys is identity separation — no hive
*process* holds or reads the appservice token — not physical isolation.
Refs #4345
`services.hyperhive.gateway.enable`, `gateway.dns.enable` and
`network.enable` replace the `hyperhive.enable` gate on all three. Each
defaults to false; the modules that need one assert it with `mkDefault
true` from inside the guard their own deployment already carries, and
`swarm-required-services.nix` — the module that owns what the
swarm-services toggle implies — asserts all three explicitly.
hive-c0re asserts all three unconditionally, so an ordinary hive keeps
getting them with no opt-in: it is the host's only knowledge that agent
containers exist.
The resolver moves to its own `hive-gateway/dns.nix` so it can be gated
without reindenting the nginx half of the module.
Reinstates `network.enable`, dropping its `mkRemovedOptionModule` shim.
A config still carrying `network.enable = false` from before the removal
now switches the bridge off instead of failing eval.
Also deletes a duplicate `centralToggleOff` fixture in nix/module-eval.nix.
Two sibling slices added it independently (c5f60fd5, ce3b3d94); the merge
was textually clean and left `main` failing to evaluate at all, so this
file could not be gated without removing one.
First slice of the move off the hand-rolled swarm services sub-CA: the
store gains the three objects the eventual minter needs, and nothing
else.
- a `pki` secrets mount, enabled ask-first off the same `bao secrets
list` snapshot the kv-v2 mount beside it uses, so a rebuild that finds
it mounted does nothing;
- a `swarm-services` role on it, `allowed_domains` read straight out of
`swarm.serviceDomains` — the same swarm-tier list swarm-ca.nix
name-constrains its sub-CA to and hive-tls.nix carries as the leaf's
SANs — with subdomains, localhost, globs and IP SANs all off;
- a `swarm-services-issuer` policy granting `update` on
`pki/issue/swarm-services`.
Inert: the mount has no issuer generated into it, the role therefore
cannot issue, and no login role attaches the policy, so no token in the
swarm carries it. No consumer changes; swarm-ca.nix still mints the
services sub-CA exactly as before. Reversible with
`bao secrets disable pki`.
The cert-auth role that attaches the policy waits for the leaf carrying
its CN, which glue-bao-tls.nix mints — a later step.
Refs #4256
Adds approle to the auth methods enabled during swarm-bao bootstrap,
using the same idempotent check pattern as cert. No role or policy
is configured yet — those are handled in later slices.
Refs #4386
An agent container cannot dial the store's loopback listener: the bridge
to-loopback DROP rule is there precisely to stop that, and the store
authenticates every reader by client certificate, so the usual answer —
a gateway vhost — is the one shape that cannot work. A terminating proxy
strips the certificate and bao sees nginx as the client for every hive.
nginx's stream module does not terminate. `ssl_preread` reads the SNI off
the ClientHello and splices the rest of the connection through byte for
byte, so bao completes the handshake itself and authenticates the client
it actually has. That is the no-vhost rule kept, not bent.
The listener binds the bridge IP rather than every address, because bao
already holds `127.0.0.1:<port>` in the same netns and a wildcard bind
there is EADDRINUSE — nginx would fail to start, taking the gateway with
it. Nothing moves as a result: the name already resolves two ways, so a
host-side reader still goes straight to loopback and an agent goes
through the passthrough, both on one `BAO_ADDR`.
Renders only inside the store's own `deploy.bao.enable` region; a host
that runs no store grows no listener and opens no port.
Per-agent certificates and per-agent policy are separate work.
Refs #4386
Grafana's OIDC client secret only existed where authelia did. One
`ssoLocal` gate — `grafana.enable && authelia.enable` — decided the
client registration, the minted secret's delivery and the whole
`auth.generic_oauth` block, so a swarm whose authelia runs on another
host got Grafana with no SSO wiring at all. The local login form is
disabled unconditionally, so that is no way in.
Split the one gate into the two questions it was conflating:
- `ssoConfigured` — does this SWARM have an identity provider
(`swarm.authelia.url`, which is swarm-wide and whose own description
makes null mean "no SSO configured"). With a delivery route present
this is what emits Grafana's OIDC block.
- `ssoLocal` — is authelia on THIS host, now spelled as the forge and
matrix modules spell it. It decides only which unit delivers the
secret.
Where authelia is elsewhere, `swarm-bao-grafana-oidc.service` reads the
secret from the swarm secret store, shaped after
glue-queue-agent-credential.nix: cert login fails loudly because a retry
fixes every state it fails on, the read degrades quietly because no
retry turns "no value there" into a value, and nothing writes a
stand-in. The producer is the publisher that already runs on authelia's
host, which gains the swarm's service clients beside the per-hive ones
at `swarm/services/<id>/oidc/client` — with the write grant in
swarm-bao.nix and the hive read grant in `policy::render` to match.
Registration moved to glue-grafana-oidc-client.nix. It has to be
declared where authelia's config is rendered, and swarm-grafana.nix's
config block hangs off this host running Grafana.
Two judgement calls stated rather than buried: a hive's read policy now
grants the whole `services` prefix, because a service's path names the
service and nothing swarm-wide records which hive runs it (cost recorded
in docs/trust-boundary/security.md); and the client is registered on any
authelia host, because no swarm-wide "this swarm has a Grafana" fact
exists to gate it on.
Refs #4234
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A sibling unit rather than more script in swarm-bao-controller-policy, because
that unit's name is an operator-facing string: docs/getting-started/setup.md
tells a reader to run `systemctl status swarm-bao-controller-policy`. Widening
it to two principals makes the name wrong; renaming it makes the instruction
wrong.
`after` and not `requires`. The controller's unit creates the KV and cert-auth
mounts this one writes into, so the ordering is real — but a failed oneshot
still counts as finished, so `requires` would neither wait for its success nor
re-run this unit when the sibling's own retry eventually lands. Ordering plus
this unit's `Restart=on-failure` is what converges.
Four module-eval cases, because the unit arrived with every claim about it in
prose and the suite still reporting the same count: the grant is write-only and
reaches the hive prefix alone (pinned as the whole capability list, since an
added capability is what a presence check misses, with negative arms for the
agent prefix, the bare swarm prefix and the policy path); it is ordered after
the unit that creates the mounts; it renders on the host; and the control, that
it does not render inside the store's container.
Refs #3853
