# Swarm CA A hive's gateway serves a leaf issued by the secret store's `pki` mount, through a role that belongs to that hive. A peer that trusts the mount's root once validates every hive's gateway and every swarm service name, present and future, instead of pinning to each one by hand. That's the point of one issuer — it turns per-peer trust from O(n²) hand-pinning into one anchor per swarm. ## The gateway leaf comes from the secret store `hive-tls-resign.service` logs in to the store with this hive's own identity, calls `pki/issue/-` (`swarm-hive-leaf-` by default), and writes the result to `/gateway.pem` and `gateway-key.pem`, the files the gateway copies from. The leaf's subject is the hive domain, and its SANs are the hive domain and `*.`. **Each hive's role covers that hive alone.** Its `allowed_domains` is the hive's domain and `*.`, read from `services.hyperhive.swarm.hives..domain` on the store's host, with no IP SANs and no `localhost`. A request for any other name is a refusal from the store naming that name. The hive's policy grants `update` on that one issue path and nothing else. `swarm-bao-hive-leaf-policy.service`, on the store's host, writes the role, the policy and the cert-auth role for every hive in `swarm.hives`. **Every hive needs its own store identity.** It's a client certificate from the store's own client CA, with the subject `-` (`swarm-bao-hive-leaf-` by default), and it's named by `services.hyperhive.deploy.hive-controller.tls.baoHiveIdentityCertFile` and `baoHiveIdentityKeyFile`: - On the host running the store, `swarm-bao-pki.service` mints it at `/var/lib/swarm-bao-pki/hive-leaf.pem` and the two options default to it. - On every other host, copy a leaf with that subject from the store's host and set both options, before the host's next deploy. Without it, `hive-tls-resign.service` fails with a message naming both options, and so does `swarm-services-cert.service` on a hive fronting no swarm name. The unit checks the identity before the leaf, so it fails even while a valid leaf keeps the gateway up. The identity is the hive's alone: it issues this hive's leaf and fetches the store's root, and it's never the identity a swarm service uses. **When it issues.** The unit runs at boot, on every deploy, and from `hive-tls-resign.timer` once a day. It asks the store for a new leaf when the current one is missing, past half the role's 720-hour lifetime, missing a configured name, doesn't match the key beside it, or doesn't chain to the store's root (`swarm-services-root.pem`). That last check is what replaces a leaf the retired hive CA signed, on the first run that reaches the store. **The gateway doesn't wait for the store.** Nothing orders the gateway after `hive-tls-resign.service`, so a hive holding a valid leaf starts its gateway while the store can't answer — still sealed, or out of reach. A renewal that fails leaves the current leaf in place, says so in the journal, and retries every 30 seconds for a day. Only a hive with no leaf yet needs the store reachable and unsealed: its gateway's certificate import fails until the first leaf lands, and then the unit starts nginx. nginx serves the leaf alone, without its issuer: the issuer is the self-signed root a consumer already holds in its trust bundle. ## The trust bundle `hive-tls-ca.service` writes `/trust-bundle.pem`, the file every consumer anchors on. It holds whichever of these exist: - `swarm-services-root.pem`, the store's `pki` root, which the gateway leaf and the swarm-services leaf both chain to. - `ca.pem` and `ca-previous.pem`, the retired hive CA's public certificates, so a leaf it signed keeps verifying until it's replaced. Removing them is a deliberate later step. - the swarm root, `/root.pem`, on the host that generates it. No hive generates a CA or signs with a CA key. A `/ca-key.pem` an earlier generation left stays on disk, unread: removing it comes with dropping `ca.pem` from the bundle. ⚠️ **A consumer reads its trust when it starts.** An agent or service container that last started before the store's root reached the bundle fails TLS to a gateway serving a store-issued leaf, until it restarts. ## The swarm's service names come from the secret store The swarm's service names (`swarm.forge.domain`, `swarm.matrix.gatewayHost`, `swarm.authelia.domain`) default to siblings of the hive domain — `forge.`, not `forge.`. The gateway leaf carries the hive's domain and a **single-label** wildcard over it, and its role allows nothing outside the hive's domain, so it can't cover them. The secret store's `pki` mount issues them a leaf of their own, through its `swarm-services` role. `swarm-services-cert.service` logs in with this host's services-issuer certificate, calls `pki/issue/swarm-services`, and writes the result to `/var/lib/hive-tls/swarm-services.pem` — the path the gateway already copies from. Every hive does this with its own identity, so holding the swarm root's private key stopped being what decides whether a hive can serve its swarm's names. `services.hyperhive.swarm.serviceDomains`, every swarm service name and the same list on every host, populates the role's `allowed_domains`, so asking for a name nobody configured is a refusal from the store naming that name — not a certificate quietly issued for it. A host's own SANs are narrower still: it asks only for the names it fronts a vhost for (`swarm.localServiceDomains`), so the key on one gateway can't serve a swarm service that runs behind another. **The root's public certificate is a file, on every hive:** `/var/lib/hive-tls/swarm-services-root.pem` (0644), written beside the leaf it signed and folded into `trust-bundle.pem`. A hive fronting no swarm name asks for no services leaf and reads the root alone, from the store's `pki/ca/pem`, presenting its own store identity (`deploy.hive-controller.tls.baoHiveIdentityCertFile` / `baoHiveIdentityKeyFile`) for the handshake. Every hive must hold one: without it, or with either file absent, `swarm-services-cert.service` fails with a message naming both options. On the host running the store it's also at `/var/lib/swarm-bao-services-pki/services-root.pem`. That's the file to hand a browser, and reading it needs no store login — which matters, because every store listener but the loopback UI one demands a client certificate, and authelia gates that one to the `admins` group, not open to an anonymous browser fetching a trust anchor. **The granting unit generates the root once, and never again.** It asks the mount whether it already has an issuer (`bao list pki/issuers`) before calling `pki/root/generate/internal`, so a rebuild or a reboot re-asserts the role and the grant without touching the anchor. A root that changed per boot would invalidate every certificate issued under it and every browser taught to trust it. **A daily timer renews the leaf.** The store's role issues it for 720 hours. `swarm-services-cert-renew.timer` runs the same script as `swarm-services-cert.service` once a day, and that script asks the store for a new leaf once the current one is past half that window or is missing a configured name. A rotation re-imports the leaf into the gateway and reloads nginx. A store that can't answer — still sealed, or out of reach — fails the unit, which leaves the current leaf in place, and the next day's run tries again: about fifteen tries before the leaf lapses. `systemctl list-timers swarm-services-cert-renew` shows the next run. ## Constraints on the material No CA private key exists on any hive. The `pki` root's key stays inside the store's mount (`root/generate/internal`), and each hive's narrowing is its role's `allowed_domains`, enforced by the store at issue time: a hive's identity that leaks can get certificates for that hive's names and no others. The swarm root's private key never reaches the nix store: the store is world-readable and content-addressed, so a key committed to a flake is a key published to everyone who builds it. The swarm distributes only certificates. ## What to hand a peer `hivectl peer-config` prints the `services.hyperhive.swarm.hives.""` block a peer operator pastes into their own config. When this hive's gateway serves a self-signed leaf (detected by the presence of `/trust-bundle.pem`), it also prints a one-time `scp` line installing the **swarm root** — `/root.pem`, not this hive's own CA — on the peer's host: ``` scp /var/lib/swarm-ca/root.pem :/var/lib/swarm-ca/root.pem ``` It then prints the secret store's `pki` root itself, read from `/var/lib/hive-tls/swarm-services-root.pem`. The store issues every hive's gateway leaf and the swarm's service names under that root, so a peer host trusting only the swarm root can't verify them. The operator installs the roots **once per swarm host**, not once per peer, so a hive joining later needs no edit on the hives already running. A hive whose cert already chains to a public CA has nothing to install — `peer-config` says so instead. ## Distributing the root The root **key** is a runtime file for the reason in [Constraints on the material](#constraints-on-the-material). The root **certificate** is a runtime file as a _consequence_ — it lives beside the key under `swarm.ca.stateDir` — and that has a cost worth naming, because it's subtle and it bites at a distance: > **Nothing whose trust store nix assembles at build time can reference > the swarm root.** The derivation reads `security.pki.certificateFiles` > inside itself; the root doesn't exist there. Two consumers, and only one of them is fine: - **`hive-tls.nix` covers agents.** It folds the root into this hive's `trust-bundle.pem`, hive-c0re receives that path as `HIVE_TLS_CA_PATH`, and the meta-flake renderer embeds that one file next to every agent's flake. The bundle is the runtime-to-build-time bridge. - **The Matrix container crosses the same bridge**, via the shared `lib/hive-ca-trust.nix` helper that `hive-ci` and `hive-forge` already use: the bundle is bind-mounted read-only into the container, and a oneshot concatenates it with the system CAs before tuwunel starts. The consumption differs per runtime and is the part worth knowing. tuwunel links no openssl, which makes `SSL_CERT_FILE` look inapplicable — it's not: its rustls-based TLS stack still resolves trust through the platform certificate store on Linux, and that store honors `SSL_CERT_FILE`, so the env var takes effect the same way it would for an OpenSSL-linked binary. > ⚠️ **Concatenate; never point `SSL_CERT_FILE` at the anchor alone.** > `openssl-probe` uses it _instead of_ the default store, so naming > just the bundle would drop every public CA and break federation with > the wider matrix network — a much bigger outage than the one being > fixed. The same caveat applies to `hive-forge` (Go) for the same > reason. To put the root on another host, copy the certificate to the same path there (`scp /root.pem :/root.pem`). One anchor per host, not one per peer: a hive joining later needs no edit on the hives already running, which is the whole point of the hierarchy.