44164d1a89
Names the two ways out (install the sub-CA, or pin back under the hive domain) and says why it is a warning rather than an assertion — the module can see what it issues, not what the operator installed.
196 lines
9 KiB
Markdown
196 lines
9 KiB
Markdown
# Swarm CA
|
|
|
|
A hive's internal TLS chains to a **swarm root CA**: the root signs each
|
|
hive's own CA, and that hive CA signs the gateway leaf. A peer that
|
|
trusts the root once validates every hive in the swarm, present and
|
|
future, instead of being pinned to each one by hand.
|
|
|
|
That is the whole point of the hierarchy — it turns per-peer trust from
|
|
O(n²) hand-pinning into one anchor per swarm.
|
|
|
|
## Two provisioning modes, one structure
|
|
|
|
What differs is who puts the artifacts on disk:
|
|
|
|
| | swarm root | this hive's CA |
|
|
| --- | --- | --- |
|
|
| default | operator-provided | operator-provided, else self-signed as before |
|
|
| `autoConfigure = true` (all on one host) | generated by `swarm-ca.service` on first boot | issued by `hive-tls-ca.service` under the root |
|
|
|
|
`services.hyperhive.swarm.ca.autoConfigure` selects between them, and is
|
|
**off by default**: a swarm's services and its hives can live on
|
|
different hosts, and a host cannot tell whether it is the one holding
|
|
the root, so setting the swarm CA up is an operator action rather than
|
|
something a host assumes. Turn it on for an all-on-one-host deployment
|
|
and the hierarchy costs no configuration.
|
|
|
|
It defaults from `services.hyperhive.enableAllLocalDefaults`, the single
|
|
switch that says "this box is the whole deployment".
|
|
|
|
**A hive given neither artifact keeps the self-signed CA it has always
|
|
had.** It serves TLS exactly as before and simply isn't part of a
|
|
swarm's trust hierarchy — the right outcome for a hive nobody has
|
|
federated yet. Only `autoConfigure` issues a hive sub-CA, because only
|
|
that case can: signing one needs the root's private key.
|
|
|
|
Moving the swarm CA onto its own host is then a matter of moving
|
|
`services.hyperhive.swarm.ca.stateDir` and leaving `autoConfigure` off —
|
|
there is no second code path to switch to.
|
|
|
|
### The warning about uncovered service names
|
|
|
|
The swarm's service names (`swarm.forge.domain`,
|
|
`swarm.matrix.gatewayHost`, `swarm.authelia.domain`) default to
|
|
siblings of the hive domain — `forge.<swarm.domain>`, not
|
|
`forge.<hive domain>`. The hive CA's leaf is a **single-label** wildcard
|
|
over the hive's own domain, so it cannot cover them; only the
|
|
swarm-services leaf can, and this host signs that one only under
|
|
`autoConfigure`. A hive with neither serves the hive leaf on those names
|
|
and every client sees a name mismatch.
|
|
|
|
So `hive-tls` emits an eval-time **warning** naming the uncovered names.
|
|
It is deliberately not an assertion: this module can see what *it* is
|
|
able to issue, but not an operator-installed sub-CA in
|
|
`swarm.ca.stateDir`, an external ACME setup, or a certificate delivered
|
|
by any other means. If you have arranged one, the warning is expected
|
|
and can be ignored. Otherwise either install the sub-CA, or pin the
|
|
names back under the hive domain — a supported migration, since the
|
|
sub-CA is constrained to the *configured* names and the swarm root
|
|
carries no name constraints at all.
|
|
|
|
## Constraints on the material
|
|
|
|
The root's private key never reaches the nix store: the store is
|
|
world-readable and content-addressed, so a key committed to a flake is a
|
|
key published to everyone who builds it. Only certificates are
|
|
distributed.
|
|
|
|
Each hive CA is **name-constrained** (X.509 `nameConstraints`) to that
|
|
hive's own domain, so a hive CA that leaks can only mint names inside
|
|
its own subdomain — enforced by every verifier rather than by
|
|
convention. The constraint excludes both IP families as well, since a
|
|
permitted-DNS-only constraint says nothing about IP SANs.
|
|
|
|
The root is issued with `pathlen:1`: it may sign hive CAs, and those may
|
|
sign leaves, and the chain stops there.
|
|
|
|
## What to hand a peer
|
|
|
|
`hivectl peer-config` prints the `cp` line. The file is
|
|
`<tls.stateDir>/trust-bundle.pem` — the hive CA plus the swarm root —
|
|
**not `ca.pem`**.
|
|
|
|
The distinction is load-bearing rather than cosmetic: once a hive CA is
|
|
an intermediate, it is no longer something a verifier can build a chain
|
|
*to*. OpenSSL will not terminate a chain at a trusted non-self-signed
|
|
certificate without `-partial_chain`, so handing a peer the bare
|
|
intermediate produces a verification failure that reads like a bad cert
|
|
rather than like a missing anchor. The bundle carries both, so the same
|
|
file works whichever mode issued it.
|
|
|
|
On a hive that predates the swarm root the bundle is just that hive's
|
|
self-signed CA, so the recipe does not change.
|
|
|
|
⚠️ Consumers must read `trust-bundle.pem`, never `ca.pem` directly. A
|
|
consumer that reads `ca.pem` works fine on a hive that has always been
|
|
self-signed and breaks the moment that hive adopts the hierarchy — so
|
|
the failure is invisible on the deployment you are most likely to test
|
|
on.
|
|
|
|
## Adopting the hierarchy on an existing hive
|
|
|
|
A hive that predates the swarm root carries a self-signed `ca.pem`, and
|
|
adopting the hierarchy means replacing it. That invalidates an anchor
|
|
consumers already trust, and they refresh on their own schedule — agents
|
|
only pick up new trust when their container restarts, peers only on
|
|
their own rebuild. **Who is allowed to decide that is what splits the
|
|
two cases.**
|
|
|
|
### Where this host owns the root (`autoConfigure`)
|
|
|
|
Adoption happens by itself, once. `hive-tls-ca.service` notices that
|
|
`ca.pem` does not chain to the root, keeps the old certificate as
|
|
`ca-previous.pem`, and re-issues under the root; the next leaf is signed
|
|
by the new CA.
|
|
|
|
It is safe to automate here precisely because this is the
|
|
all-on-one-host shape: every consumer is on this box, so "when will they
|
|
have refreshed" is knowable rather than guessed.
|
|
|
|
The old CA stays in `trust-bundle.pem` afterwards, so adoption is
|
|
**additive to the anchor set before it is subtractive** — a container
|
|
that has not restarted yet still validates. Removing `ca-previous.pem`
|
|
is a deliberate later step: how long is long enough is a property of the
|
|
deployment, not something the unit can know.
|
|
|
|
A marker file (`.swarm-ca-adopted`) records that this ran. Its *absence*
|
|
is the trigger, so adoption fires once per hive rather than being
|
|
re-decided on every activation.
|
|
|
|
### Everywhere else
|
|
|
|
No automatic adoption. `hive-tls-ca.service` fails, loudly, naming both
|
|
certificates and giving the two-command recipe:
|
|
|
|
```
|
|
rm <tls.stateDir>/ca.pem <tls.stateDir>/ca-key.pem
|
|
systemctl restart hive-tls-ca.service
|
|
```
|
|
|
|
Failing rather than warning is deliberate: a hive whose CA does not
|
|
chain to the root it has been given is misconfigured, and a warning in a
|
|
build log is not something anyone reads twice.
|
|
|
|
To keep the current CA on purpose — a hive that deliberately stays
|
|
outside the hierarchy, or one mid-migration — `touch` the marker file
|
|
named in the message. That is a decision, and it is recorded as one.
|
|
|
|
A hive with **no** root configured at all is not affected by any of
|
|
this: it self-signs exactly as it always has.
|
|
|
|
## Distributing the root
|
|
|
|
The root **key** is a runtime file for an obvious reason: the nix store
|
|
is world-readable and content-addressed, so a key committed to a flake
|
|
is a key *published* to every consumer of that flake.
|
|
|
|
The root **certificate** is a runtime file as a *consequence* — it lives
|
|
beside the key under `swarm.ca.stateDir` — and that has a cost worth
|
|
naming, because it is not obvious and it bites at a distance:
|
|
|
|
> **Nothing whose trust store is assembled at build time can reference
|
|
> the swarm root.** `security.pki.certificateFiles` is read inside the
|
|
> derivation; the root does not exist there.
|
|
|
|
Two consumers, and only one of them is fine:
|
|
|
|
- **Agents are covered.** `hive-tls.nix` folds the root into this hive's
|
|
`trust-bundle.pem`, hive-c0re receives that path as
|
|
`HIVE_TLS_CA_PATH`, and the meta-flake renderer embeds that one file
|
|
next to every agent's flake. The bundle is the runtime-to-build-time
|
|
bridge.
|
|
- **The Matrix container crosses the same bridge**, via the shared
|
|
`lib/hive-ca-trust.nix` helper that `hive-ci` and `hive-forge` already
|
|
use: the bundle is bind-mounted read-only into the container, and a
|
|
oneshot concatenates it with the system CAs before tuwunel starts.
|
|
|
|
The consumption differs per runtime and is the part worth knowing.
|
|
tuwunel links no openssl, which makes `SSL_CERT_FILE` look inapplicable
|
|
— it isn't. Its outbound client is `reqwest` with the `rustls` feature,
|
|
which builds a `rustls_platform_verifier::Verifier`; because tuwunel
|
|
calls `tls_certs_merge` (additive) rather than `tls_certs_only`, the
|
|
platform roots stay alongside its compiled-in webpki set. On Linux that
|
|
verifier resolves through `rustls-native-certs` → `openssl-probe`,
|
|
which reads `SSL_CERT_FILE`.
|
|
|
|
> ⚠️ **Concatenate; never point `SSL_CERT_FILE` at the anchor alone.**
|
|
> `openssl-probe` uses it *instead of* the default store, so naming
|
|
> just the bundle would drop every public CA and break federation with
|
|
> the wider matrix network — a much bigger outage than the one being
|
|
> fixed. The same caveat applies to `hive-forge` (Go) for the same
|
|
> reason.
|
|
|
|
To put the root on another host, copy the certificate to the same path
|
|
there (`scp <stateDir>/root.pem <host>:<stateDir>/root.pem`). One anchor
|
|
per host, not one per peer: a hive joining later needs no edit on the
|
|
hives already running, which is the whole point of the hierarchy.
|