vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

swarm-bao: give each hive-cert consumer its own bao identity

Browse source 
Four units read one path each out of the store, and all four logged in
holding `deploy.bao.clientCertFile` — the hive's own leaf. Bao identifies
a principal by the subject of the certificate it presents, so four
readers behind one certificate were ONE principal, and the only grant
expressible was the union of what the four need: read on
`swarm/agents/*`, `swarm/hives/<hive>/*` and `swarm/services/*`. The unit
fetching Grafana's OIDC client secret could fetch every agent credential
in the swarm; the one fetching this hive's matrix token could fetch
Grafana's. Least privilege was not misconfigured here, it was
unrepresentable.

Each now holds a leaf, a cert-auth role and a policy of its own, and each
policy is the single `secret/data/…` path that unit's own script names —
spelled to the leaf, not to a prefix, the way matrix-ctl's already is.
Following the four exemplars in-tree rather than building a mechanism:
`signLeaf` mints the leaves, `swarm-bao.nix` writes the roles from the
bootstrap token, the consumers name their own pair.

Two of the four are written PER HIVE and two are not, which is the shape
of the paths rather than a preference. A matrix appservice token and a
queue credential live under `swarm/hives/<name>/` and every hive runs a
reader for its own, so one role for all of them would have to be granted
`hives/*` — letting one hive read another's, a reach no hive has today.
An OIDC client secret lives under `swarm/services/<client-id>/` and a
swarm registers each exactly once, so one role each is enough. The
per-hive subjects are `<prefix>-<hive>` and swarm.nix reserves every
composed spelling as a hive name, so a hive cannot be named into another
hive's role.

The shared leaf stays: hive-c0re still passes it into its container, the
`bao` CLI wrapper still defaults to it, and the three
`glue-*-bao-identity.nix` files derive the PKI directory from it.

module-eval-bao-grants gains a negative arm per principal — each pins the
three stanzas the hive's leaf carried and the two wildcards a later
widening would reach for, so a policy that grows fails here rather than
in a store. Plus the consuming side: repointing a unit back at the hive's
leaf would evaluate, deploy and log in, and silently restore the union.

A hive that reads a store on another machine now places one leaf per
principal instead of one shared by four. That cost is the point, and
docs/swarm/secrets.md lists the pairs.
This commit is contained in:
atlas 2026-09-21 23:44:59 +02:00 • committed by mara
commit f1445b4c8b
14 changed files with 859 additions and 43 deletions
Download patch file Download diff file Expand all files Collapse all files

15
docs/getting-started/setup.md
View file

@ -170,16 +170,19 @@ What else you need depends on
restart, with the keys `init` printed.
The store serves TLS, and on a hive that deploys it you need do nothing: a
first-boot unit mints a CA of the store's own plus the two leaves it signs —
the store's server certificate and this host's client certificate — and points
`deploy.bao.serverCertFile`, `.serverKeyFile` and `.clientCaFile` at the store's
half, `.clientCertFile`, `.clientKeyFile` and `.serverCaFile` at the reader's.
first-boot unit mints a CA of the store's own plus the leaves it signs — the
store's server certificate, this host's client certificate, and one per service
principal — and points `deploy.bao.serverCertFile`, `.serverKeyFile` and
`.clientCaFile` at the store's half, `.clientCertFile`, `.clientKeyFile` and
`.serverCaFile` at the reader's, and each principal's own pair at its own leaf.
Those are `mkDefault`s, so naming your own paths wins. Do that when your
certificates come from a real internal CA; the store has no opinion about
which. A hive that does **not** deploy the store names the reader's three
itself: the operator issues that leaf out of band, and it's the one credential
the store can't hand you, being what opens it. ⚠️ Not the gateway's HTTPS certificates and not the hive CA — this is
itself, plus a pair for every principal it runs — see
[per-principal identities](../swarm/secrets.md#per-principal-identities) for the
list. The operator issues those leaves out of band; they're the credentials the
store can't hand you, being what opens it. ⚠️ Not the gateway's HTTPS certificates and not the hive CA — this is
**mTLS between services and the store**, a separate trust domain, because a
store that took its identity from an authority it itself distributes could
never come up before that authority.

68
docs/swarm/secrets.md
View file

@ -88,7 +88,7 @@ where an absent OIDC block is a container with no way in at all, a collector
with nothing to authenticate with still receives every hive's telemetry —
only its own pushes to the stores go out unauthenticated and get refused
there. `swarm-bao-otel-oidc.service` therefore doesn't render at all without
`deploy.bao.clientCertFile` / `clientKeyFile`, the same choice
`deploy.bao.otelOidcClientCertFile` / `otelOidcClientKeyFile`, the same choice
`glue-matrix-bao-token.nix` and `glue-queue-agent-credential.nix` make for
their own optional readers, rather than the hard assertion `swarm-grafana.nix`
uses for its non-optional one. The operator can still deliver the secret by
@ -249,14 +249,14 @@ straight out of authelia's tree when the two happen to share a host is one
round trip cheaper and costs a second delivery unit, a second way for the file
to be wrong, and a gate deciding between them — and the gate is the expensive
part, because whatever it's wrong about is an outage nobody can read. The store
exists so a host holds **one** out-of-band secret, its client certificate, and
reads everything else with it.
exists so the only secrets a host holds out of band are **certificates**, and
everything else is read with them.
Two things follow, and `swarm-grafana.nix` asserts both rather than degrading:
running Grafana requires `swarm.authelia.url` (this module disables its local
login form unconditionally, so SSO isn't a feature of some topologies — it's the
only way in), and it requires this host's `deploy.bao.clientCertFile` /
`clientKeyFile`. Each refusal names the option to set. Both used to be silent:
only way in), and it requires this host's `deploy.bao.grafanaOidcClientCertFile`
/ `grafanaOidcClientKeyFile`. Each refusal names the option to set. Both used to be silent:
a null URL dropped the OIDC block, a missing leaf produced a warning, and
either one left a Grafana with no SSO and no password box — nothing failed, so
nothing said anything. Registration is separate and stays where authelia is
@ -271,7 +271,8 @@ runs the collector, and registration lives separately in
`glue-swarm-otel-oidc-client.nix` for the same reason Grafana's does. Where it
differs is what an absent credential means: the collector still receives
telemetry with none, so `swarm-otel.nix` doesn't assert
`deploy.bao.clientCertFile` / `clientKeyFile` the way Grafana does — it
`deploy.bao.otelOidcClientCertFile` / `otelOidcClientKeyFile` the way Grafana
does — it
doesn't render the reading unit without them, the shape
`glue-matrix-bao-token.nix` and `glue-queue-agent-credential.nix` use for
their own optional reads.
@ -290,10 +291,18 @@ A service's secret is one value for the whole swarm rather than one per hive, so
it lives under the `services` prefix, and a hive's read policy grants that prefix
whole. That's because nothing in a swarm's configuration records which hive runs
a given service — placement is a `deploy.*` fact, per-host by definition — so
there is nothing to scope the grant to. Every hive can therefore read every swarm
service's client secret; that's the cost of a shape where the service's host has
no store identity of its own to present, and it's stated in
`swarm-secret-client`'s `policy` module beside the grant itself. Absence behaves
there is nothing to scope the grant to. A hive's **own** leaf can therefore read
every swarm service's client secret, and that's stated in
`swarm-secret-client`'s `policy` module beside the grant itself.
⚠️ **The readers no longer present it.** Four units used to log in with
`deploy.bao.clientCertFile`, which is the hive's own leaf, and bao identifies a
principal by the subject of the certificate it presents — so four readers behind
one leaf were one principal holding the union of their needs. Each now holds a
leaf of its own, and a policy naming only the path that unit reads. See
[per-principal identities](#per-principal-identities) below.
Absence behaves
as it does for the other readers: a store that says "not here" leaves the
file alone and says so, since there is no such thing as a locally valid OIDC
client secret to stand in.
@ -303,14 +312,43 @@ credential you must already hold to authenticate from the thing it
authenticates you to, so the store's identity can't come from an authority the store
distributes — which excludes the hive CA and the swarm CA both, and has nothing
to do with the gateway's HTTPS certificates either way. `glue-bao-tls.nix`
mints a CA that signs exactly two things, the store's server certificate and a
reader's client certificate, and distributes nothing. A deployment with a real
mints a CA that signs a fixed, short list of leaves — the store's server
certificate, the hive's own reader certificate, and one per service principal —
and distributes nothing. A deployment with a real
internal CA deletes that file and names its own paths in
`deploy.bao.serverCertFile` / `clientCaFile`; the store itself has no opinion.
A hive that reads from a store on **another** machine names the reader's half —
`clientCertFile`, `clientKeyFile`, `serverCaFile` — and places that leaf by hand.
It's the one credential that can't come out of the store, being what opens it;
everything else a hive needs does.
`clientCertFile`, `clientKeyFile`, `serverCaFile` — and places that leaf by hand,
plus one leaf per principal it runs (the options are listed below). Those are the
credentials that can't come out of the store, being what opens it; everything
else a hive needs does.
### Per-principal identities
Bao matches a cert-auth role on the certificate's subject, so a certificate is an
identity and sharing one merges the identities. These four units read one path
each and each holds a leaf, a role and a policy of its own:
| unit | option pair under `deploy.bao.` | reads |
| ------------------------ | -------------------------------------------------------- | -------------------------------------------------- |
| `swarm-bao-matrix-token` | `matrixTokenClientCertFile` / `matrixTokenClientKeyFile` | `swarm/hives/<hive>/matrix/appservice-token` |
| `swarm-bao-queue-agent` | `queueAgentClientCertFile` / `queueAgentClientKeyFile` | `swarm/hives/<hive>/queue/agent` |
| `swarm-bao-grafana-oidc` | `grafanaOidcClientCertFile` / `grafanaOidcClientKeyFile` | `swarm/services/<grafana client id>/oidc/client` |
| `swarm-bao-otel-oidc` | `otelOidcClientCertFile` / `otelOidcClientKeyFile` | `swarm/services/<collector client id>/oidc/client` |
The first two are written **per hive**, because the path they read carries a hive
name and every hive runs its own reader. Their subjects are
`<deploy.bao.matrixTokenCommonNamePrefix>-<hive>` and
`<deploy.bao.queueAgentCommonNamePrefix>-<hive>`; `swarm.nix` reserves both
composed spellings as hive names, so a hive can't be named into another hive's
role. The other two read a path that names a swarm service rather than a hive, so
one role each is enough and their subjects are the flat
`deploy.bao.grafanaOidcCommonName` and `deploy.bao.otelOidcCommonName`.
On a host that mints its own PKI, `glue-bao-tls.nix` signs all four and defaults
all eight options, and there is nothing to do. Elsewhere each leaf is issued from
that CA out of band and named here — one file per principal rather than one file
shared by four, which is the whole of what this buys.
## How a reader reaches the store