swarm CA: retire the host-held root CA, the store holds the only anchor
The secret store's pki root issues every hive's gateway leaf and the
swarm's service names, so the root swarm-ca.service generated on the
host anchors nothing and its key is a secret outside the store.
- swarm-ca.nix is gone: swarm-ca.service, root.pem and root-key.pem,
and the swarm.ca.{autoConfigure,stateDir,validityDays} options. A
definition of any of them fails evaluation with a removal message
(swarm-ca-removed.nix), and a tmpfiles rule deletes /var/lib/swarm-ca
on every host. singleHostSwarm no longer sets a swarm.* option.
- hive-tls-ca drops the $root slot: trust-bundle.pem holds the store's
pki root and a hive CA's public ca.pem/ca-previous.pem only.
- hivectl peer-config no longer prints the scp line for the swarm root;
the store's pki root is the one step.
- The gateway's cert import no longer stands a hive leaf or a throwaway
in for a missing swarm-services leaf where it Requires= the issuing
unit: it fails, naming swarm-services-cert.service. The stand-in stays
where the gateway does not wait (no swarm name, or the store's UI
only, whose passthrough must start with the store sealed).
Comments and docs that named swarm-ca.nix, swarm.ca.* or the host root
now name the store's pki root.
Refs #4588
This commit is contained in:
atlas
mara
parent
5d6ebda3e8
commit
93f1a76ea7
22 changed files with 305 additions and 395 deletions
|
|
@ -76,11 +76,11 @@ On by default, listening on `httpsPort` (default 443) beside the plain-http `por
|
|||
|
||||
The issuer is the **secret store's `pki` mount**, not a bare self-signed leaf. `hive-tls-resign.service` (from `hive-tls.nix`) asks the store for a gateway **leaf** through this hive's own role, presenting the hive's own store identity (`services.hyperhive.deploy.hive-controller.tls.baoHiveIdentityCertFile` / `baoHiveIdentityKeyFile`), and writes it under `…tls.stateDir` (default `/var/lib/hive-tls`). `hive-gateway-self-signed-cert` then imports the leaf into nginx's state dir (`/var/lib/hive-gateway/tls/{cert,key}.pem`). On a host that runs no hive the import unit installs the swarm-services leaf there instead, or a throwaway self-signed pair until that leaf lands. [`swarm/ca.md`](../swarm/ca.md) has the role, the identity and what a hive without one does.
|
||||
|
||||
⚠️ **Keep the import unit.** It does two jobs nginx needs: it re-modes the key to `0640 root:nginx` (nginx's pre-start `nginx -t` runs as the nginx user and fails on the source's `0600 root:root` key), and it makes sure **every cert path the config names exists** — when the swarm-services leaf is missing it installs the hive leaf in its place, or on a host that runs no hive a throwaway self-signed pair. nginx refuses a config naming a missing cert file, so without that fallback one missing leaf takes down every vhost, not just one.
|
||||
⚠️ **Keep the import unit.** It does two jobs nginx needs: it re-modes the key to `0640 root:nginx` (nginx's pre-start `nginx -t` runs as the nginx user and fails on the source's `0600 root:root` key), and it puts a file at **every cert path the config names**, because nginx refuses a config naming a missing cert file and one missing leaf would take down every vhost, not just one. Where the gateway waits for the swarm-services leaf, a missing one fails the import unit, naming `swarm-services-cert.service`, and nginx stays down until that unit issues it. Where it doesn't — a host fronting no swarm name, or only the store's UI, which starts its passthrough with the store sealed — the hive leaf stands in until the services leaf lands, or on a host that runs no hive a throwaway self-signed pair.
|
||||
|
||||
**Why an issuer, not a bare leaf**: a bare self-signed leaf is its own trust anchor, so every regeneration is a new anchor every consumer must re-trust, and an agent's build sets its trust store once, before any leaf exists. Agents and federation peers trust the store's root once; leaf rotation never breaks them.
|
||||
|
||||
**What consumers trust**: `trust-bundle.pem` in the same state dir. It carries the store's `pki` root, plus a hive CA's public `ca.pem` and the swarm root where they exist ([`swarm/ca.md`](../swarm/ca.md#the-trust-bundle)). nginx serves the leaf alone, since its issuer is a self-signed root. Agents (via `security.pki.certificateFiles`), the CI and forge containers and federating peers all read the bundle.
|
||||
**What consumers trust**: `trust-bundle.pem` in the same state dir. It carries the store's `pki` root, plus a hive CA's public `ca.pem` where one exists ([`swarm/ca.md`](../swarm/ca.md#the-trust-bundle)). nginx serves the leaf alone, since its issuer is a self-signed root. Agents (via `security.pki.certificateFiles`), the CI and forge containers and federating peers all read the bundle.
|
||||
|
||||
**Why on by default**: matrix-dart-sdk (FluffyChat's SDK) fetches `https://<host>/.well-known/matrix/client` and never falls back to plain http, so without TLS the browser client can't bootstrap.
|
||||
|
||||
|
|
|
|||
|
|
@ -83,8 +83,9 @@ the file every consumer anchors on. It holds whichever of these exist:
|
|||
issues under that CA; they stay in the bundle so a leaf it signed verifies
|
||||
until `hive-tls-resign` replaces it. Removing them is a deliberate later
|
||||
step.
|
||||
- the swarm root, `<swarm.ca.stateDir>/root.pem`, on the host that generates
|
||||
it.
|
||||
|
||||
No other anchor goes in: the store's `pki` root is the only root the swarm
|
||||
has.
|
||||
|
||||
No hive generates a CA or signs with a CA key. A
|
||||
`<deploy.hive-controller.tls.stateDir>/ca-key.pem` stays on disk, unread:
|
||||
|
|
@ -163,46 +164,34 @@ role's `allowed_domains`, enforced by the store at issue time: a hive's
|
|||
identity that leaks can get certificates for that hive's names and no
|
||||
others.
|
||||
|
||||
The swarm root's private key never reaches the nix store: the store is
|
||||
world-readable and content-addressed, so a key committed to a flake is a
|
||||
key published to everyone who builds it. The swarm distributes only
|
||||
certificates.
|
||||
No hive holds a root CA or its key, and nothing under `/var/lib/swarm-ca`
|
||||
exists: every host deletes that directory at boot and on every deploy. The
|
||||
swarm distributes only certificates.
|
||||
|
||||
## What to hand a peer
|
||||
|
||||
`hivectl peer-config` prints the `services.hyperhive.swarm.hives."<name>"`
|
||||
block a peer operator pastes into their own config. When this hive's
|
||||
gateway serves a self-signed leaf (detected by the presence of
|
||||
`<deploy.hive-controller.tls.stateDir>/trust-bundle.pem`), it also prints a
|
||||
one-time `scp` line installing the **swarm root** —
|
||||
`<swarm.ca.stateDir>/root.pem`, not this hive's own CA — on the peer's
|
||||
host:
|
||||
|
||||
```
|
||||
scp /var/lib/swarm-ca/root.pem <peer-host>:/var/lib/swarm-ca/root.pem
|
||||
```
|
||||
|
||||
It then prints the secret store's `pki` root itself, read from
|
||||
`<deploy.hive-controller.tls.stateDir>/trust-bundle.pem`), it also prints the
|
||||
secret store's `pki` root itself, read from
|
||||
`/var/lib/hive-tls/swarm-services-root.pem`. The store issues every hive's
|
||||
gateway leaf and the swarm's service names under that root, so a peer host
|
||||
trusting only the swarm root can't verify them.
|
||||
gateway leaf and the swarm's service names under that root, so it's the one
|
||||
anchor a peer host needs.
|
||||
|
||||
The operator installs the roots **once per swarm host**, not once per peer,
|
||||
The operator installs the root **once per swarm host**, not once per peer,
|
||||
so a hive joining later needs no edit on the hives already running. A hive
|
||||
whose cert already chains to a public CA has nothing to install —
|
||||
`peer-config` says so instead.
|
||||
|
||||
## Distributing the root
|
||||
|
||||
The root **key** is a runtime file for the reason in
|
||||
[Constraints on the material](#constraints-on-the-material).
|
||||
|
||||
The root **certificate** is a runtime file as a _consequence_ — it lives
|
||||
beside the key under `swarm.ca.stateDir` — and that has a cost worth
|
||||
The root **certificate** is a runtime file: each hive fetches it from the
|
||||
store at boot, so nothing can name it at build time. That has a cost worth
|
||||
naming, because it's subtle and it bites at a distance:
|
||||
|
||||
> **Nothing whose trust store nix assembles at build time can reference
|
||||
> the swarm root.** The derivation reads `security.pki.certificateFiles`
|
||||
> the store's root.** The derivation reads `security.pki.certificateFiles`
|
||||
> inside itself; the root doesn't exist there.
|
||||
|
||||
Two consumers, and only one of them is fine:
|
||||
|
|
@ -231,7 +220,7 @@ Two consumers, and only one of them is fine:
|
|||
> fixed. The same caveat applies to `hive-forge` (Go) for the same
|
||||
> reason.
|
||||
|
||||
To put the root on another host, copy the certificate to the same path
|
||||
there (`scp <stateDir>/root.pem <host>:<stateDir>/root.pem`). One anchor
|
||||
per host, not one per peer: a hive joining later needs no edit on the
|
||||
hives already running, which is the whole point of the hierarchy.
|
||||
A hive in the swarm fetches the root itself. A host outside it gets the
|
||||
certificate from `hivectl peer-config` ([What to hand a peer](#what-to-hand-a-peer)).
|
||||
One anchor per host, not one per peer: a hive joining later needs no edit
|
||||
on the hives already running.
|
||||
|
|
|
|||
|
|
@ -43,8 +43,6 @@ neither is a renaming of the other.
|
|||
|
||||
| secret | generated by | lives at | hive elsewhere |
|
||||
| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| swarm root CA cert | `swarm-ca.nix` first-boot unit, when the operator sets `autoConfigure` | `/var/lib/swarm-ca/root.pem` | operator copies the **cert** in; it's public |
|
||||
| swarm root CA key | same unit | `/var/lib/swarm-ca/root-key.pem`, `0600` | stays on whichever host holds it — see the constraint below |
|
||||
| swarm-services root CA (cert + key) | `swarm-bao.nix`'s granting unit, inside the store's `pki` mount | the **key** never leaves the mount; the **cert** lands at `/var/lib/swarm-bao-services-pki/services-root.pem` (0644) and at `/var/lib/hive-tls/swarm-services-root.pem` on every hive | nothing to copy — every hive receives the cert with the leaf it asks the store for, or from `pki/ca/pem` where it fronts no swarm name and needs a store identity |
|
||||
| authelia session, JWT and storage-encryption keys | authelia's first-boot unit, in-container | `/var/lib/authelia-swarm/{session,jwt,storage-encryption}.key` | generated in place; nothing outside that container reads them |
|
||||
| authelia OIDC HMAC key | same unit | `/var/lib/authelia-swarm/oidc-hmac.key` | same |
|
||||
|
|
@ -445,17 +443,12 @@ the hive domain rather than a child** — `forge.<swarm>` next to `<hive>.<swarm
|
|||
That role can't issue a certificate for a swarm service; the store refuses it.
|
||||
|
||||
The store's `swarm-services` role issues them instead, from the root inside its
|
||||
`pki` mount. The swarm root (`swarm.ca.*`) is still generated on one host, and
|
||||
two things follow for it:
|
||||
|
||||
- **The root's private key is a runtime file and must never enter the nix
|
||||
store**, so nothing build-time can name it — the system reads
|
||||
`security.pki.certificateFiles`' value at build time, which makes it the
|
||||
wrong tool here. Trust reaches containers through a bind-mounted bundle
|
||||
assembled at boot instead.
|
||||
- **On any topology other than all-local, placing that key is an operations
|
||||
decision**, not something this module tree makes for you. A hive that hosts no
|
||||
swarm services needs only the root's _cert_, to trust what others issue.
|
||||
`pki` mount. That root's private key never leaves the mount, and its
|
||||
certificate is the only anchor the swarm has. Each hive fetches the certificate
|
||||
from the store at boot, so nothing build-time can name it — the system reads
|
||||
`security.pki.certificateFiles`' value at build time, which makes it the wrong
|
||||
tool here. Trust reaches containers through a bind-mounted bundle assembled at
|
||||
boot instead.
|
||||
|
||||
## Adding a secret
|
||||
|
||||
|
|
|
|||
|
|
@ -20,8 +20,7 @@ sections below don't repeat it, so a service that stops deriving is a
|
|||
visible difference rather than one more paragraph saying the same thing.
|
||||
|
||||
`singleHostSwarm` is the all-on-one-box mode above it. It defaults
|
||||
`deploy.allSwarmServices`, the swarm CA (`swarm.ca.autoConfigure`, generated
|
||||
on this host), the swarm controller (`deploy.swarm-controller.enable`), the
|
||||
`deploy.allSwarmServices`, the swarm controller (`deploy.swarm-controller.enable`), the
|
||||
host's `/etc/hosts` entries for the names it serves
|
||||
(`gateway.localHostsEntry`), the queue's auth-callout keys
|
||||
(`deploy.nats.autoGenerateCallout`) and where the secret store's bootstrap
|
||||
|
|
|
|||
Loading…
Reference in a new issue