hyperhive/docs/swarm/secrets.md

Swarm secrets: what exists, and where each one lives

A swarm's credentials are generated in three different places and read in a fourth, so "where does this file go" has a different answer per deployment. This page is that answer, one row per secret.

Two rules run through all of it.

A secret is a path, never a value. Every option that carries a credential takes a file path (*File), because a literal written into a nix expression is rendered into the nix store — which is world-readable and permanent. There is no option anywhere in this tree that accepts a secret inline, and adding one would be a leak rather than a convenience.

The generator and the reader are usually in different containers. They share the host's network namespace, which makes them feel co-located, but their filesystem roots are separate. That is why delivery is a host-side copy rather than a bind mount: nixos-container refuses to start when a bind source is missing, and a secret minted on another container's first boot does not exist yet. Binding it would make one container wait on a file that waits on a container that starts after it.

The three topologies

Every row below is read against one of these.

topology	what it means	who places secrets
all-local	one host runs the swarm's shared services and its own hive	nobody — each secret is generated where it is read, or copied by a host unit
swarm-managed	the swarm's services run on a host with swarmctl	swarmctl writes what it owns; the rest is still generated in place
hive elsewhere	a hive that federates with a swarm it does not host	the operator provides the file and names it in config

Swarm-level — one of each per swarm

secret	generated by	lives at	hive elsewhere
swarm root CA cert	swarm-ca.nix first-boot unit, when autoConfigure is set	/var/lib/swarm-ca/root.pem	operator copies the cert in; it is public
swarm root CA key	same unit	/var/lib/swarm-ca/root-key.pem, 0600	stays on whichever host holds it — see the constraint below
swarm-services sub-CA (cert + key)	swarm-ca.nix, signed by the root	/var/lib/swarm-ca/services-ca{,-key}.pem	issued where the root lives
authelia session, JWT and storage-encryption keys	authelia's first-boot unit, in-container	/var/lib/authelia-swarm/{session,jwt,storage-encryption}.key	generated in place; nothing outside that container reads them
authelia OIDC HMAC key	same unit	/var/lib/authelia-swarm/oidc-hmac.key	same
authelia OIDC issuer key (RSA)	same unit	/var/lib/authelia-swarm/oidc-issuer.key	same — relying parties verify against the public half at /jwks.json
OIDC client secret, plaintext half	authelia crypto hash generate --random	/var/lib/authelia-swarm/oidc-clients/<id>.secret	operator provides the file and names it in the service's sso.clientSecretFile
OIDC client secret, digest half	the same mint	oidc-clients/<id>.digest	authelia's own half; merged at runtime via settingsFiles
authelia subject store	swarmctl	users.json (canonical) → users.yml (rendered)	swarmctl, on the host that runs authelia
wireguard private key	the operator — wg genkey	whatever swarm.wireguard.privateKeyFile names	always operator-provided; nothing generates this for you

The three keys authelia mints for itself are generated in-container precisely because nothing outside that container ever reads them. That is the test worth applying to any secret added here — and the client secret's plaintext half is the one row that fails it, which is the entire reason a delivery step exists.

Hive-level — one of each per hive

secret	generated by	lives at
hive CA cert + key	hive-tls.nix first-boot unit	<tls.stateDir>/ca.pem, ca-key.pem (0600)
hive leaf certs	hive-tls.nix, signed by the hive CA	<tls.stateDir>/<name>.pem
matrix registration token	a host activation script, on first boot	/var/lib/hyperhive/matrix-register-token (0600)
the forge's copy of its OIDC secret	hive-forge-oidc-secret.service copies it from authelia's tree	/var/lib/forgejo-oidc/<id>.secret inside the forge container
the homeserver's copy of its OIDC secret	hive-matrix-oidc-secret.service, same shape	/var/lib/tuwunel-oidc/<id>.secret, handed to tuwunel through LoadCredential

Both delivery units wait for authelia's first boot to mint the secret — a bounded wait, 120s — and then fail loudly rather than skipping. A silent skip produces a service whose login button always fails, which is a symptom several layers from its cause.

The constraint that decides where the root lives

A hive CA carries nameConstraints=permitted;DNS:<hive domain>, and a swarm service name is a sibling of the hive domain rather than a child — forge.<swarm> next to <hive>.<swarm>. So a hive CA cannot issue a certificate for a swarm service. Not by policy: by construction, and openssl enforces it.

Whatever holds the swarm root is therefore what makes swarm-service certificates possible at all. Two things follow:

• The root's private key is a runtime file and must never enter the nix store, so nothing build-time can name it — security.pki.certificateFiles is read when the system is built, and is the wrong tool here. Trust reaches containers through a bind-mounted bundle assembled at boot instead.
• On any topology other than all-local, placing that key is an operations decision, not something this module tree makes for you. A hive that hosts no swarm services needs only the root's cert, to trust what others issue.

Adding a secret

State three things, in the row you add above: who mints it, which container reads it, and what happens when they differ. If they differ, it needs a delivery unit, and the unit copies — it does not bind.