vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs(swarm): one page saying where every secret goes

Browse source 
Per mara on the CA question: outside all-local this is an ops problem,
and what is missing is documentation rather than machinery.

One row per secret, read against three topologies, because the same
credential is generated in place on one and handed over by an operator on
another. sso.md's table is replaced by a pointer -- two tables listing the
same secrets would drift, and its prose about why a secret is generated
in-container is the half worth keeping there.

States the constraint the whole thing rests on: a hive CA is name-
constrained to the hive domain and a swarm service name is a sibling of
it, so a hive CA cannot issue a swarm-service certificate at all. That is
why placing the swarm root is an operations decision.
This commit is contained in:
atlas 2026-08-14 12:04:20 +02:00 • committed by mara
commit c39e94758e
3 changed files with 105 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

6
docs/swarm/README.md
View file

@ -94,6 +94,12 @@ Which secrets the SSO provider generates, which one has a reader in
another container, and the three ways that one gets delivered:
[`sso.md`](sso.md).
## Secrets
Every credential the swarm holds, who mints it, where it must live, and
which of the three topologies makes it the operator's job to place:
[`secrets.md`](secrets.md).
## Swarm UI
The operator-only web surface on the swarm apex, why reaching it needs

89
docs/swarm/secrets.md Normal file
View file

@ -0,0 +1,89 @@
# Swarm secrets: what exists, and where each one lives
A swarm's credentials are generated in three different places and read in a
fourth, so "where does this file go" has a different answer per deployment.
This page is that answer, one row per secret.
Two rules run through all of it.
**A secret is a path, never a value.** Every option that carries a credential
takes a file path (`*File`), because a literal written into a nix expression is
rendered into the nix store — which is world-readable and permanent. There is no
option anywhere in this tree that accepts a secret inline, and adding one would
be a leak rather than a convenience.
**The generator and the reader are usually in different containers.** They share
the host's network namespace, which makes them feel co-located, but their
filesystem roots are separate. That is why delivery is a **host-side copy rather
than a bind mount**: `nixos-container` refuses to start when a bind source is
missing, and a secret minted on another container's first boot does not exist
yet. Binding it would make one container wait on a file that waits on a
container that starts after it.
## The three topologies
Every row below is read against one of these.
| topology | what it means | who places secrets |
|---|---|---|
| **all-local** | one host runs the swarm's shared services and its own hive | nobody — each secret is generated where it is read, or copied by a host unit |
| **swarm-managed** | the swarm's services run on a host with `swarmctl` | `swarmctl` writes what it owns; the rest is still generated in place |
| **hive elsewhere** | a hive that federates with a swarm it does not host | the operator provides the file and names it in config |
## Swarm-level — one of each per swarm
| secret | generated by | lives at | hive elsewhere |
|---|---|---|---|
| swarm root CA cert | `swarm-ca.nix` first-boot unit, when `autoConfigure` is set | `/var/lib/swarm-ca/root.pem` | operator copies the **cert** in; it is public |
| swarm root CA key | same unit | `/var/lib/swarm-ca/root-key.pem`, `0600` | stays on whichever host holds it — see the constraint below |
| swarm-services sub-CA (cert + key) | `swarm-ca.nix`, signed by the root | `/var/lib/swarm-ca/services-ca{,-key}.pem` | issued where the root lives |
| authelia session, JWT and storage-encryption keys | authelia's first-boot unit, in-container | `/var/lib/authelia-swarm/{session,jwt,storage-encryption}.key` | generated in place; nothing outside that container reads them |
| authelia OIDC HMAC key | same unit | `/var/lib/authelia-swarm/oidc-hmac.key` | same |
| authelia OIDC issuer key (RSA) | same unit | `/var/lib/authelia-swarm/oidc-issuer.key` | same — relying parties verify against the **public** half at `/jwks.json` |
| OIDC client secret, plaintext half | `authelia crypto hash generate --random` | `/var/lib/authelia-swarm/oidc-clients/<id>.secret` | operator provides the file and names it in the service's `sso.clientSecretFile` |
| OIDC client secret, digest half | the same mint | `oidc-clients/<id>.digest` | authelia's own half; merged at runtime via `settingsFiles` |
| authelia subject store | `swarmctl` | `users.json` (canonical) → `users.yml` (rendered) | `swarmctl`, on the host that runs authelia |
| wireguard private key | **the operator** — `wg genkey` | whatever `swarm.wireguard.privateKeyFile` names | always operator-provided; nothing generates this for you |
The three keys authelia mints for itself are generated in-container precisely
because nothing outside that container ever reads them. **That is the test worth
applying to any secret added here** — and the client secret's plaintext half is
the one row that fails it, which is the entire reason a delivery step exists.
## Hive-level — one of each per hive
| secret | generated by | lives at |
|---|---|---|
| hive CA cert + key | `hive-tls.nix` first-boot unit | `<tls.stateDir>/ca.pem`, `ca-key.pem` (`0600`) |
| hive leaf certs | `hive-tls.nix`, signed by the hive CA | `<tls.stateDir>/<name>.pem` |
| matrix registration token | a host activation script, on first boot | `/var/lib/hyperhive/matrix-register-token` (`0600`) |
| the forge's copy of its OIDC secret | `hive-forge-oidc-secret.service` copies it from authelia's tree | `/var/lib/forgejo-oidc/<id>.secret` inside the forge container |
| the homeserver's copy of its OIDC secret | `hive-matrix-oidc-secret.service`, same shape | `/var/lib/tuwunel-oidc/<id>.secret`, handed to tuwunel through `LoadCredential` |
Both delivery units wait for authelia's first boot to mint the secret, with a
bound, and **fail loudly** rather than skipping. A silent skip produces a service
whose login button always fails — a symptom several layers from its cause.
## The constraint that decides where the root lives
A hive CA carries `nameConstraints=permitted;DNS:<hive domain>`, and **a swarm
service name is a sibling of the hive domain rather than a child** — `forge.<swarm>`
next to `<hive>.<swarm>`. So a hive CA cannot issue a certificate for a swarm
service. Not by policy: by construction, and openssl enforces it.
Whatever holds the swarm root is therefore what makes swarm-service certificates
possible at all. Two things follow:
- **The root's private key is a runtime file and must never enter the nix
store**, so nothing build-time can name it — `security.pki.certificateFiles` is
read when the system is built, and is the wrong tool here. Trust reaches
containers through a bind-mounted bundle assembled at boot instead.
- **On any topology other than all-local, placing that key is an operations
decision**, not something this module tree makes for you. A hive that hosts no
swarm services needs only the root's *cert*, to trust what others issue.
## Adding a secret
State three things, in the row you add above: **who mints it**, **which
container reads it**, and **what happens when they differ**. If they differ, it
needs a delivery unit, and the unit copies — it does not bind.

19
docs/swarm/sso.md
View file

@ -90,17 +90,16 @@ an attribute edit can invalidate a login by accident.
## What secrets exist, and where each one lives
| secret | generated by | rests in | read by |
|---|---|---|---|
| `jwt.key`, `session.key`, `storage-encryption.key` | authelia's first-boot unit | `/var/lib/authelia-swarm/` | authelia |
| `oidc-hmac.key` | same unit | same directory | authelia |
| `oidc-issuer.key` (RSA) | same unit | same directory | authelia signs with it; clients verify the **public** half at `/jwks.json` |
| `oidc-clients/<id>.digest` | same unit, via `authelia crypto hash generate` | same directory, merged in through `settingsFiles` | authelia |
| `oidc-clients/<id>.secret` | the same mint — this is its plaintext half | same directory | **the relying party, in another container** |
Every secret in the swarm, with its generator and its path, is tabulated
in one place: [`secrets.md`](secrets.md). The rows relevant here are
authelia's own keys (session, JWT, storage-encryption, OIDC HMAC, OIDC
issuer) plus the two halves of each client secret.
Everything above the last row is generated in-container because nothing
outside that container ever reads it. That is the test worth applying to
any secret added here. The last row fails it, and that is the entire
What matters for this page is the shape rather than the paths. Authelia's
own keys are generated **in-container**, because nothing outside that
container ever reads them — that is the test worth applying to any secret
added here. The plaintext half of a client secret is the one that fails
it: its reader lives in a different container, and that is the entire
reason a delivery step exists.
**None of it is ever written into a nix expression.** authelia's