f4df4fc4a9
The `pki` mount had no issuer and no principal could log in to it, so the swarm's service certificates were still minted by two openssl hops from a root key on disk. Close both halves and retire the openssl path with them. The mount now generates its own root, once. The granting unit asks bao whether an issuer already exists (`bao list pki/issuers`) before calling `pki/root/generate/internal`, so a rebuild or a reboot re-asserts the role and the grant without touching the anchor — a root that changed per boot would invalidate every certificate issued under it and every browser taught to trust it. The guard asks the store rather than looking for a marker file on this host's disk: a file is a claim about a mount that may have been restored from a snapshot or disabled and re-enabled underneath it. `swarm-services-issuer` stops being an inert policy. A fourth cert-auth role attaches it, following the shape the controller, the publisher and matrix-ctl already use, and glue-bao-tls.nix signs the leaf carrying its CN — that credential is what opens the mount, so it cannot come out of it. `swarm-services-cert.service` logs in with that leaf, calls `pki/issue/swarm-services`, and writes the result to the path hive-tls.nix already wrote and the gateway already copies from. The sub-CA layer does not move; it stops existing. The role's `allowed_domains`, read from the same `swarm.serviceDomains` the SANs come from, enforces at issue time what the sub-CA encoded in x509 `nameConstraints`, and with the root inside the mount there is nothing left for an intermediate to be an intermediate of. Not a flag day: the issuing root is published beside the leaf as `swarm-services-root.pem` (0644) and joins `trust-bundle.pem`, where the swarm root still sits. A leaf chaining to the old sub-CA and one issued by the store both verify against the same bundle, so hives can be rebuilt in any order. The same file is what an operator hands a browser — readable without a store login, which matters because every listener demands a client certificate. The eval-time warning about uncovered service names is gone rather than reworded. It fired on "this host does not hold the swarm root key", which was the reason a hive could end up serving its own leaf on a swarm-service name. Every hive now asks the store with its own identity, so that stopped being the thing that decides. Closes #4586
225 lines
10 KiB
Markdown
225 lines
10 KiB
Markdown
# Swarm CA
|
|
|
|
A hive's internal TLS chains to a **swarm root CA**: the root signs each
|
|
hive's own CA, and that hive CA signs the gateway leaf. A peer that
|
|
trusts the root once validates every hive in the swarm, present and
|
|
future, instead of pinning to each one by hand.
|
|
|
|
That's the whole point of the hierarchy — it turns per-peer trust from
|
|
O(n²) hand-pinning into one anchor per swarm.
|
|
|
|
## Two provisioning modes, one structure
|
|
|
|
What differs is who puts the artifacts on disk:
|
|
|
|
| | swarm root | this hive's CA |
|
|
| ---------------------------------------- | --------------------------------------------- | ---------------------------------------------- |
|
|
| default | operator-provided | operator-provided, else self-signed as before |
|
|
| `autoConfigure = true` (all on one host) | generated by `swarm-ca.service` on first boot | issued by `hive-tls-ca.service` under the root |
|
|
|
|
`services.hyperhive.swarm.ca.autoConfigure` selects between them, and is
|
|
**off by default**: a swarm's services and its hives can live on
|
|
different hosts, and a host can't tell whether it's the one holding
|
|
the root, so setting the swarm CA up is an operator action rather than
|
|
something a host assumes. Turn it on for an all-on-one-host deployment
|
|
and the hierarchy costs no configuration.
|
|
|
|
It defaults from `services.hyperhive.deploy.singleHostSwarm`, the single
|
|
switch that says "this box is the whole deployment."
|
|
|
|
**A hive given neither artifact keeps the self-signed CA it has always
|
|
had.** It serves TLS exactly as before and isn't part of a
|
|
swarm's trust hierarchy — the right outcome for a hive nobody has
|
|
federated yet. Only `autoConfigure` issues a hive sub-CA, because only
|
|
that case can: signing one needs the root's private key.
|
|
|
|
Moving the swarm CA onto its own host is then a matter of moving
|
|
`services.hyperhive.swarm.ca.stateDir` and leaving `autoConfigure` off —
|
|
there is no second code path to switch to.
|
|
|
|
### The swarm's service names come from the secret store
|
|
|
|
The swarm's service names (`swarm.forge.domain`,
|
|
`swarm.matrix.gatewayHost`, `swarm.authelia.domain`) default to
|
|
siblings of the hive domain — `forge.<swarm.domain>`, not
|
|
`forge.<hive domain>`. The hive CA's leaf is a **single-label** wildcard
|
|
over the hive's own domain, so it can't cover them.
|
|
|
|
That leaf is issued by the secret store's `pki` mount instead, from a
|
|
root generated inside it. `swarm-services-cert.service` logs in with
|
|
this host's services-issuer certificate, calls
|
|
`pki/issue/swarm-services`, and writes the result to
|
|
`/var/lib/hive-tls/swarm-services.pem` — the path the gateway already
|
|
copies from. Every hive does this with its own identity, so holding the
|
|
swarm root's private key stopped being what decides whether a hive can
|
|
serve its swarm's names.
|
|
|
|
The role's `allowed_domains` is read from the same
|
|
`services.hyperhive.swarm.serviceDomains` the SANs are built from, so
|
|
asking for a name nobody configured is a refusal from the store naming
|
|
that name — not a certificate quietly issued for it.
|
|
|
|
**The root's public certificate is a file, on every hive:**
|
|
`/var/lib/hive-tls/swarm-services-root.pem` (0644), written beside the
|
|
leaf it signed and folded into `trust-bundle.pem`. On the host running
|
|
the store it is also at
|
|
`/var/lib/swarm-bao-services-pki/services-root.pem`. That is the file to
|
|
hand a browser, and no store login is needed to read it — which matters,
|
|
because every store listener demands a client certificate.
|
|
|
|
**The root is generated once and never again.** The granting unit asks
|
|
the mount whether it already has an issuer (`bao list pki/issuers`)
|
|
before calling `pki/root/generate/internal`, so a rebuild or a reboot
|
|
re-asserts the role and the grant without touching the anchor. A root
|
|
that changed per boot would invalidate every certificate issued under it
|
|
and every browser taught to trust it.
|
|
|
|
## Constraints on the material
|
|
|
|
The root's private key never reaches the nix store: the store is
|
|
world-readable and content-addressed, so a key committed to a flake is a
|
|
key published to everyone who builds it. The swarm distributes only
|
|
certificates.
|
|
|
|
Each hive CA is **name-constrained** (X.509 `nameConstraints`) to that
|
|
hive's own domain, so a hive CA that leaks can only mint names inside
|
|
its own subdomain — enforced by every verifier rather than by
|
|
convention. The constraint excludes both IP families as well, since a
|
|
permitted-DNS-only constraint says nothing about IP SANs.
|
|
|
|
<!-- vale write-good.Passive = NO -->
|
|
|
|
The root is issued with `pathlen:1`: it may sign hive CAs, and those may
|
|
sign leaves, and the chain stops there.
|
|
|
|
<!-- vale write-good.Passive = YES -->
|
|
|
|
## What to hand a peer
|
|
|
|
`hivectl peer-config` prints the `services.hyperhive.swarm.hives."<name>"`
|
|
block a peer operator pastes into their own config. When this hive's
|
|
gateway serves a self-signed leaf under the hierarchy (detected by the
|
|
presence of `<deploy.hive-controller.tls.stateDir>/trust-bundle.pem`), it
|
|
also prints a one-time
|
|
`scp` line installing the **swarm root** —
|
|
`<swarm.ca.stateDir>/root.pem`, not this hive's own CA — on the peer's
|
|
host:
|
|
|
|
```
|
|
scp /var/lib/swarm-ca/root.pem <peer-host>:/var/lib/swarm-ca/root.pem
|
|
```
|
|
|
|
That's the point of the hierarchy: the operator installs the root
|
|
**once per swarm host**, not once per peer, so a hive joining later
|
|
needs no edit on the hives already running. A hive whose cert already chains to a public
|
|
CA has nothing to install — `peer-config` says so instead.
|
|
|
|
Handing a peer this hive's own `ca.pem` would not work even as a
|
|
one-off: once a hive CA is an intermediate under the swarm root, it's
|
|
no longer something a verifier can build a chain _to_ — OpenSSL won't
|
|
terminate a chain at a trusted non-self-signed certificate without
|
|
`-partial_chain`. That's why the tool distributes the root, not a
|
|
per-hive file.
|
|
|
|
## Adopting the hierarchy on an existing hive
|
|
|
|
<!-- vale write-good.Passive = NO -->
|
|
|
|
A hive that predates the swarm root carries a self-signed `ca.pem`, and
|
|
adopting the hierarchy means replacing it. That invalidates an anchor
|
|
consumers already trust, and they refresh on their own schedule — agents
|
|
only pick up new trust when their container restarts, peers only on
|
|
their own rebuild. **Who is allowed to decide that's what splits the
|
|
two cases.**
|
|
|
|
<!-- vale write-good.Passive = YES -->
|
|
|
|
### Where this host owns the root (`autoConfigure`)
|
|
|
|
Adoption happens by itself, once. `hive-tls-ca.service` notices that
|
|
`ca.pem` doesn't chain to the root, keeps the old certificate as
|
|
`ca-previous.pem`, and re-issues under the root; the new CA signs the
|
|
next leaf.
|
|
|
|
It's safe to automate here precisely because this is the
|
|
all-on-one-host shape: every consumer is on this box, so "when will they
|
|
have refreshed" is knowable rather than guessed.
|
|
|
|
The old CA stays in `trust-bundle.pem` afterwards, so adoption is
|
|
**additive to the anchor set before it's subtractive** — a container
|
|
that hasn't restarted yet still validates. Removing `ca-previous.pem`
|
|
is a deliberate later step: how long is long enough is a property of the
|
|
deployment, not something the unit can know.
|
|
|
|
A marker file (`.swarm-ca-adopted`) records that this ran. Its _absence_
|
|
is the trigger, so adoption fires once per hive rather than being
|
|
re-decided on every activation.
|
|
|
|
### Everywhere else
|
|
|
|
No automatic adoption. `hive-tls-ca.service` fails, loudly, naming both
|
|
certificates and giving the two-command recipe:
|
|
|
|
```
|
|
rm <deploy.hive-controller.tls.stateDir>/ca.pem <deploy.hive-controller.tls.stateDir>/ca-key.pem
|
|
systemctl restart hive-tls-ca.service
|
|
```
|
|
|
|
<!-- vale write-good.Passive = NO -->
|
|
|
|
Failing rather than warning is deliberate: a hive whose CA doesn't
|
|
chain to the root it received is misconfigured, and a warning in a
|
|
build log isn't something anyone reads twice.
|
|
|
|
<!-- vale write-good.Passive = YES -->
|
|
|
|
To keep the current CA on purpose — a hive that deliberately stays
|
|
outside the hierarchy, or one mid-migration — `touch` the marker file
|
|
named in the message. That's a decision, and it's recorded as one.
|
|
|
|
A hive with **no** root configured at all isn't affected by any of
|
|
this: it self-signs exactly as it always has.
|
|
|
|
## Distributing the root
|
|
|
|
The root **key** is a runtime file for the reason in
|
|
[Constraints on the material](#constraints-on-the-material).
|
|
|
|
The root **certificate** is a runtime file as a _consequence_ — it lives
|
|
beside the key under `swarm.ca.stateDir` — and that has a cost worth
|
|
naming, because it's subtle and it bites at a distance:
|
|
|
|
> **Nothing whose trust store nix assembles at build time can reference
|
|
> the swarm root.** The derivation reads `security.pki.certificateFiles`
|
|
> inside itself; the root doesn't exist there.
|
|
|
|
Two consumers, and only one of them is fine:
|
|
|
|
- **`hive-tls.nix` covers agents.** It folds the root into this hive's
|
|
`trust-bundle.pem`, hive-c0re receives that path as
|
|
`HIVE_TLS_CA_PATH`, and the meta-flake renderer embeds that one file
|
|
next to every agent's flake. The bundle is the runtime-to-build-time
|
|
bridge.
|
|
- **The Matrix container crosses the same bridge**, via the shared
|
|
`lib/hive-ca-trust.nix` helper that `hive-ci` and `hive-forge` already
|
|
use: the bundle is bind-mounted read-only into the container, and a
|
|
oneshot concatenates it with the system CAs before tuwunel starts.
|
|
|
|
The consumption differs per runtime and is the part worth knowing.
|
|
tuwunel links no openssl, which makes `SSL_CERT_FILE` look inapplicable
|
|
— it's not: its rustls-based TLS stack still resolves trust through the
|
|
platform certificate store on Linux, and that store honors
|
|
`SSL_CERT_FILE`, so the env var takes effect the same way it would for
|
|
an OpenSSL-linked binary.
|
|
|
|
> ⚠️ **Concatenate; never point `SSL_CERT_FILE` at the anchor alone.**
|
|
> `openssl-probe` uses it _instead of_ the default store, so naming
|
|
> just the bundle would drop every public CA and break federation with
|
|
> the wider matrix network — a much bigger outage than the one being
|
|
> fixed. The same caveat applies to `hive-forge` (Go) for the same
|
|
> reason.
|
|
|
|
To put the root on another host, copy the certificate to the same path
|
|
there (`scp <stateDir>/root.pem <host>:<stateDir>/root.pem`). One anchor
|
|
per host, not one per peer: a hive joining later needs no edit on the
|
|
hives already running, which is the whole point of the hierarchy.
|