vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

hive-tls: issue the gateway leaf from bao with a per-hive identity

Browse source 
Every hive gets its own store identity (CN
<deploy.bao.hiveLeafCommonNamePrefix>-<hive>) and its own role on the
existing pki mount (<deploy.bao.hiveLeafPkiRolePrefix>-<hive>), whose
allowed_domains is the hive's domain and *.<domain>. The policy grants
update on that one issue path. swarm-bao-hive-leaf-policy writes role,
policy and cert-auth role for every hive in swarm.hives; the store's host
mints its own hive's identity in glue-bao-tls.nix and a new glue file
points deploy.hive-controller.tls.baoHiveIdentity{Cert,Key}File at it.
Hives on other hosts place it by hand. swarm.nix reserves the composed CN.

hive-tls-ca no longer generates a hive CA or signs a leaf: caGenScript,
signLeafScript/signHiveLeaf and the adoption block are gone, ca-key.pem
is deleted where it exists, and the unit only assembles trust-bundle.pem.
The retired ca.pem/ca-previous.pem stay in the bundle so leaves they
signed keep verifying.

hive-tls-resign becomes the bao issuance: at boot, on deploy and from a
daily timer it re-issues when the leaf is missing, past half the role's
lifetime, missing a name, or not chained to the store's pki root. Nothing
orders the gateway after it, so a valid leaf on disk never waits on the
store; a failed renewal leaves the leaf in place. A missing identity
fails the unit naming both options, as does the root fetch on a hive
fronting no swarm name, which now presents this identity instead of the
reader cert.

caValidityDays and leafValidityDays are removed (both spellings), since
nothing reads them.

Refs #4588
This commit is contained in:
atlas 2026-10-04 15:11:48 +02:00 • committed by mara
commit 5526daab97
16 changed files with 859 additions and 628 deletions
Download patch file Download diff file Expand all files Collapse all files

231
docs/swarm/ca.md
View file

@ -1,52 +1,100 @@
# Swarm CA
A hive's internal TLS chains to a **swarm root CA**: the root signs each
hive's own CA, and that hive CA signs the gateway leaf. A peer that
trusts the root once validates every hive in the swarm, present and
future, instead of pinning to each one by hand.
A hive's gateway serves a leaf issued by the secret store's `pki` mount,
through a role that belongs to that hive. A peer that trusts the mount's
root once validates every hive's gateway and every swarm service name,
present and future, instead of pinning to each one by hand.
That's the whole point of the hierarchy — it turns per-peer trust from
O(n²) hand-pinning into one anchor per swarm.
That's the point of one issuer — it turns per-peer trust from O(n²)
hand-pinning into one anchor per swarm.
## Two provisioning modes, one structure
## The gateway leaf comes from the secret store
What differs is who puts the artifacts on disk:
`hive-tls-resign.service` logs in to the store with this hive's own
identity, calls `pki/issue/<deploy.bao.hiveLeafPkiRolePrefix>-<hiveName>`
(`swarm-hive-leaf-<hiveName>` by default), and writes the result to
`<deploy.hive-controller.tls.stateDir>/gateway.pem` and `gateway-key.pem`,
the files the gateway copies from. The leaf's subject is the hive domain,
and its SANs are the hive domain and `*.<hive domain>`.
| | swarm root | this hive's CA |
| ---------------------------------------- | --------------------------------------------- | ---------------------------------------------- |
| default | operator-provided | operator-provided, else self-signed as before |
| `autoConfigure = true` (all on one host) | generated by `swarm-ca.service` on first boot | issued by `hive-tls-ca.service` under the root |
**Each hive's role covers that hive alone.** Its `allowed_domains` is the
hive's domain and `*.<hive domain>`, read from
`services.hyperhive.swarm.hives.<name>.domain` on the store's host, with no
IP SANs and no `localhost`. A request for any other name is a refusal from
the store naming that name. The hive's policy grants `update` on that one
issue path and nothing else. `swarm-bao-hive-leaf-policy.service`, on the
store's host, writes the role, the policy and the cert-auth role for every
hive in `swarm.hives`.
`services.hyperhive.swarm.ca.autoConfigure` selects between them, and is
**off by default**: a swarm's services and its hives can live on
different hosts, and a host can't tell whether it's the one holding
the root, so setting the swarm CA up is an operator action rather than
something a host assumes. Turn it on for an all-on-one-host deployment
and the hierarchy costs no configuration.
**Every hive needs its own store identity.** It's a client certificate
from the store's own client CA, with the subject
`<deploy.bao.hiveLeafCommonNamePrefix>-<hiveName>`
(`swarm-bao-hive-leaf-<hiveName>` by default), and it's named by
`services.hyperhive.deploy.hive-controller.tls.baoHiveIdentityCertFile` and
`baoHiveIdentityKeyFile`:
It defaults from `services.hyperhive.deploy.singleHostSwarm`, the single
switch that says "this box is the whole deployment."
- On the host running the store, `swarm-bao-pki.service` mints it at
`/var/lib/swarm-bao-pki/hive-leaf.pem` and the two options default to it.
- On every other host, copy a leaf with that subject from the store's host
and set both options, before the host's next deploy.
**A hive given neither artifact keeps the self-signed CA it has always
had.** It serves TLS exactly as before and isn't part of a
swarm's trust hierarchy — the right outcome for a hive nobody has
federated yet. Only `autoConfigure` issues a hive sub-CA, because only
that case can: signing one needs the root's private key.
Without it, `hive-tls-resign.service` fails with a message naming both
options, and so does `swarm-services-cert.service` on a hive fronting no
swarm name. The unit checks the identity before the leaf, so it fails even while a
valid leaf keeps the gateway up. The identity is the hive's alone:
it issues this hive's leaf and fetches the store's root, and it's never the
identity a swarm service uses.
Moving the swarm CA onto its own host is then a matter of moving
`services.hyperhive.swarm.ca.stateDir` and leaving `autoConfigure` off —
there is no second code path to switch to.
**When it issues.** The unit runs at boot, on every deploy, and from
`hive-tls-resign.timer` once a day. It asks the store for a new leaf when
the current one is missing, past half the role's 720-hour lifetime, missing
a configured name, or doesn't chain to the store's root
(`swarm-services-root.pem`). That last check is what replaces a leaf the
retired hive CA signed, on the first run that reaches the store.
### The swarm's service names come from the secret store
**The gateway doesn't wait for the store.** Nothing orders the gateway after
`hive-tls-resign.service`, so a hive holding a valid leaf starts its gateway
while the store can't answer — still sealed, or out of reach. A renewal that fails leaves the
current leaf in place, says so in the journal, and retries every 30 seconds
for a day. Only a hive with no leaf yet needs the store reachable and
unsealed: its gateway's certificate import fails until the first leaf
lands, and then the unit starts nginx.
nginx serves the leaf alone, without its issuer: the issuer is the
self-signed root a consumer already holds in its trust bundle.
## The trust bundle
`hive-tls-ca.service` writes `<deploy.hive-controller.tls.stateDir>/trust-bundle.pem`,
the file every consumer anchors on. It holds whichever of these exist:
- `swarm-services-root.pem`, the store's `pki` root, which the gateway leaf
and the swarm-services leaf both chain to.
- `ca.pem` and `ca-previous.pem`, the retired hive CA's public certificates,
so a leaf it signed keeps verifying until it's replaced. Removing them is a
deliberate later step.
- the swarm root, `<swarm.ca.stateDir>/root.pem`, on the host that generates
it.
No hive keeps a CA key. `hive-tls-ca.service` deletes
`<deploy.hive-controller.tls.stateDir>/ca-key.pem` where an earlier
generation left one, and generates no CA.
⚠️ **A consumer reads its trust when it starts.** An agent or service
container that last started before the store's root reached the bundle
fails TLS to a gateway serving a store-issued leaf, until it restarts.
## The swarm's service names come from the secret store
The swarm's service names (`swarm.forge.domain`,
`swarm.matrix.gatewayHost`, `swarm.authelia.domain`) default to
siblings of the hive domain — `forge.<swarm.domain>`, not
`forge.<hive domain>`. The hive CA's leaf is a **single-label** wildcard
over the hive's own domain, so it can't cover them.
`forge.<hive domain>`. The gateway leaf carries the hive's domain and a
**single-label** wildcard over it, and its role allows nothing outside the hive's
domain, so it can't cover them.
The secret store's `pki` mount issues that leaf instead, from a
root generated inside it. `swarm-services-cert.service` logs in with
The secret store's `pki` mount issues them a leaf of their own, through its
`swarm-services` role. `swarm-services-cert.service` logs in with
this host's services-issuer certificate, calls
`pki/issue/swarm-services`, and writes the result to
`/var/lib/hive-tls/swarm-services.pem` — the path the gateway already
@ -65,10 +113,11 @@ gateway can't serve a swarm service that runs behind another.
**The root's public certificate is a file, on every hive:**
`/var/lib/hive-tls/swarm-services-root.pem` (0644), written beside the
leaf it signed and folded into `trust-bundle.pem`. A hive fronting no
swarm name asks for no leaf and reads the root alone, from the store's
`pki/ca/pem`, presenting its store identity
(`deploy.bao.clientCertFile` / `clientKeyFile`) for the handshake. Every hive
must hold one: without it, or with either file absent,
swarm name asks for no services leaf and reads the root alone, from the store's
`pki/ca/pem`, presenting its own store identity
(`deploy.hive-controller.tls.baoHiveIdentityCertFile` /
`baoHiveIdentityKeyFile`) for the handshake. Every hive must hold one:
without it, or with either file absent,
`swarm-services-cert.service` fails with a message naming both options.
On the host running
the store it's also at
@ -98,32 +147,24 @@ list-timers swarm-services-cert-renew` shows the next run.
## Constraints on the material
The root's private key never reaches the nix store: the store is
No CA private key exists on any hive. The `pki` root's key stays inside the
store's mount (`root/generate/internal`), and each hive's narrowing is its
role's `allowed_domains`, enforced by the store at issue time: a hive's
identity that leaks can get certificates for that hive's names and no
others.
The swarm root's private key never reaches the nix store: the store is
world-readable and content-addressed, so a key committed to a flake is a
key published to everyone who builds it. The swarm distributes only
certificates.
Each hive CA is **name-constrained** (X.509 `nameConstraints`) to that
hive's own domain, so a hive CA that leaks can only mint names inside
its own subdomain — enforced by every verifier rather than by
convention. The constraint excludes both IP families as well, since a
permitted-DNS-only constraint says nothing about IP SANs.
<!-- vale write-good.Passive = NO -->
The root is issued with `pathlen:1`: it may sign hive CAs, and those may
sign leaves, and the chain stops there.
<!-- vale write-good.Passive = YES -->
## What to hand a peer
`hivectl peer-config` prints the `services.hyperhive.swarm.hives."<name>"`
block a peer operator pastes into their own config. When this hive's
gateway serves a self-signed leaf under the hierarchy (detected by the
presence of `<deploy.hive-controller.tls.stateDir>/trust-bundle.pem`), it
also prints a one-time
`scp` line installing the **swarm root** —
gateway serves a self-signed leaf (detected by the presence of
`<deploy.hive-controller.tls.stateDir>/trust-bundle.pem`), it also prints a
one-time `scp` line installing the **swarm root** —
`<swarm.ca.stateDir>/root.pem`, not this hive's own CA — on the peer's
host:
@ -132,80 +173,14 @@ scp /var/lib/swarm-ca/root.pem <peer-host>:/var/lib/swarm-ca/root.pem
```
It then prints the secret store's `pki` root itself, read from
`/var/lib/hive-tls/swarm-services-root.pem`. The store issues the swarm's
service names under that root, not the swarm root, so a peer host trusting only
the swarm root can't verify them.
`/var/lib/hive-tls/swarm-services-root.pem`. The store issues every hive's
gateway leaf and the swarm's service names under that root, so a peer host
trusting only the swarm root can't verify them.
That's the point of the hierarchy: the operator installs the root
**once per swarm host**, not once per peer, so a hive joining later
needs no edit on the hives already running. A hive whose cert already chains to a public
CA has nothing to install — `peer-config` says so instead.
Handing a peer this hive's own `ca.pem` would not work even as a
one-off: once a hive CA is an intermediate under the swarm root, it's
no longer something a verifier can build a chain _to_ — OpenSSL won't
terminate a chain at a trusted non-self-signed certificate without
`-partial_chain`. That's why the tool distributes the root, not a
per-hive file.
## Adopting the hierarchy on an existing hive
<!-- vale write-good.Passive = NO -->
A hive that predates the swarm root carries a self-signed `ca.pem`, and
adopting the hierarchy means replacing it. That invalidates an anchor
consumers already trust, and they refresh on their own schedule — agents
only pick up new trust when their container restarts, peers only on
their own rebuild. **Who is allowed to decide that's what splits the
two cases.**
<!-- vale write-good.Passive = YES -->
### Where this host owns the root (`autoConfigure`)
Adoption happens by itself, once. `hive-tls-ca.service` notices that
`ca.pem` doesn't chain to the root, keeps the old certificate as
`ca-previous.pem`, and re-issues under the root; the new CA signs the
next leaf.
It's safe to automate here precisely because this is the
all-on-one-host shape: every consumer is on this box, so "when will they
have refreshed" is knowable rather than guessed.
The old CA stays in `trust-bundle.pem` afterwards, so adoption is
**additive to the anchor set before it's subtractive** — a container
that hasn't restarted yet still validates. Removing `ca-previous.pem`
is a deliberate later step: how long is long enough is a property of the
deployment, not something the unit can know.
A marker file (`.swarm-ca-adopted`) records that this ran. Its _absence_
is the trigger, so adoption fires once per hive rather than being
re-decided on every activation.
### Everywhere else
No automatic adoption. `hive-tls-ca.service` fails, loudly, naming both
certificates and giving the two-command recipe:
```
rm <deploy.hive-controller.tls.stateDir>/ca.pem <deploy.hive-controller.tls.stateDir>/ca-key.pem
systemctl restart hive-tls-ca.service
```
<!-- vale write-good.Passive = NO -->
Failing rather than warning is deliberate: a hive whose CA doesn't
chain to the root it received is misconfigured, and a warning in a
build log isn't something anyone reads twice.
<!-- vale write-good.Passive = YES -->
To keep the current CA on purpose — a hive that deliberately stays
outside the hierarchy, or one mid-migration — `touch` the marker file
named in the message. That's a decision, and it's recorded as one.
A hive with **no** root configured at all isn't affected by any of
this: it self-signs exactly as it always has.
The operator installs the roots **once per swarm host**, not once per peer,
so a hive joining later needs no edit on the hives already running. A hive
whose cert already chains to a public CA has nothing to install —
`peer-config` says so instead.
## Distributing the root