diff --git a/docs/README.md b/docs/README.md index 3483f42a..298a78c3 100644 --- a/docs/README.md +++ b/docs/README.md @@ -76,6 +76,10 @@ declarations. - **Where do agent snapshots go? How does the swarm's `btrfs receive` endpoint authenticate a pushing hive?** → [`networking/snapshot-store.md`](networking/snapshot-store.md). +- **Who mints each credential, who reads it, and how does it rotate — and + where is that shape headed?** → [`swarm/credentials.md`](swarm/credentials.md) + (current state, target state, and the progressive-enhancement rule); + [`swarm/secrets.md`](swarm/secrets.md) for where each file lives today. ## Scheduler, CI, observability diff --git a/docs/swarm/README.md b/docs/swarm/README.md index 3d085e0f..feea18bd 100644 --- a/docs/swarm/README.md +++ b/docs/swarm/README.md @@ -104,6 +104,12 @@ Every credential the swarm holds, who mints it, where it must live, and which of the three topologies makes it the operator's job to place: [`secrets.md`](secrets.md). +Where that shape is **going** — the per-secret minter/reader/renewal +contract, the target of one mTLS identity per host and everything else +through the store, and the test a change has to pass to count as movement +toward it: [`credentials.md`](credentials.md). It supersedes `secrets.md` +when the migration completes. + ## Swarm UI The operator-only web surface on the swarm apex, why reaching it needs diff --git a/docs/swarm/credentials.md b/docs/swarm/credentials.md new file mode 100644 index 00000000..fcae088d --- /dev/null +++ b/docs/swarm/credentials.md @@ -0,0 +1,137 @@ +# Credentials: current state, target state, and the rule in between + +Where [`secrets.md`](secrets.md) is a map of every file a swarm holds, this +page is the **direction of travel**: what the credential shape is today, what +it is meant to become, and the test a change has to pass to count as movement +toward it rather than away. + +It is written to be the thing a discussion points at. **When the migration +below is complete, this page supersedes the old credential docs** — `secrets.md` +gets deleted and this file moves into its place. Until then the two coexist and +neither is wrong: that one says where a file is, this one says whether it should +be a file at all. + +Two conventions carry over unchanged, because the target does not relax them. + +**A secret is a path, never a value.** Every option carrying key material or a +token takes a file path, because a literal in a nix expression lands in the nix +store — world-readable and permanent. That rule survives the migration; what +changes is how many paths there are. + +**Public material is a value.** A certificate or a public nkey is handed to +every client that connects, so the store is a fine place for it. Nothing below +is about those. + +## Current state + +Four credential families live in the swarm's secret store, plus the mTLS +identity that opens it. That fifth row is not a stored secret — it is the +credential that cannot be, being what authenticates the reader to the store. + +| store path | what it is | minter | writer | reader | renewal | persisted outside the store | +| -------------------------------------------- | ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `swarm/agents//matrix/` | one agent's matrix account credential | `swarm-controller`, via a caller-supplied bearer token or its own `m.login.password` (`swarm-controller/src/matrix_account.rs:14-21`) | same unit (`swarm-controller/src/matrix_account.rs:163`) | `hive-c0re`'s credential worker, under the **hive's** identity (`hive-c0re/src/workers/credential.rs:33-64`) | `NONE` | **yes** — `hive-priv` writes `/matrix-token`, `0600`, chowned to the agent (`hive-priv/src/main.rs:535-556`) | +| `swarm/hives//matrix/appservice-token` | the appservice token authenticating one hive's homeserver | `swarm-secret-publisher.nix` itself, `head -c 32 /dev/urandom` on the authelia host (`nix/host-modules/swarm-secret-publisher.nix:280-286`) | same unit (`nix/host-modules/swarm-secret-publisher.nix:288-289`) | `swarm-bao-matrix-token.service` on the hive (`nix/host-modules/glue-matrix-bao-token.nix:159-167`) | `NONE` | **yes, twice** — the publisher's own state copy (`nix/host-modules/swarm-secret-publisher.nix:185`) and the hive's token file, `0600` (`nix/host-modules/glue-matrix-bao-token.nix:186-188`) | +| `swarm/hives//queue/agent` | the OIDC client id + secret agent containers present to the swarm queue | authelia, on first boot, into its own client-secret dir | `swarm-secret-publish.service` (`nix/host-modules/swarm-secret-publisher.nix:222-225`) | `swarm-bao-queue-agent.service` on the hive (`nix/host-modules/glue-queue-agent-credential.nix:187-197`) | `NONE` | **yes** — `secret` `0600` and `client_id` `0644` under the hive's agent-credential dir (`nix/host-modules/glue-queue-agent-credential.nix:224-231`) | +| `swarm/services//oidc/client` | a swarm service's own OIDC client secret, one per service | authelia, on first boot | `swarm-secret-publish.service` (`nix/host-modules/swarm-secret-publisher.nix:245-248`) | whichever host runs that service — `swarm-bao-grafana-oidc.service` (`nix/host-modules/swarm-grafana.nix:565-684`), the collector's twin (`nix/host-modules/swarm-otel.nix:685-767`) | `NONE` | **yes** — Grafana's `0400` host file (`nix/host-modules/swarm-grafana.nix:678-682`), the collector's `0400` file handed in by `LoadCredential` (`nix/host-modules/swarm-otel.nix:763-767`) | +| _(not stored)_ the store's own mTLS leaves | the certificate a host presents to reach the store at all | `swarm-bao-pki.service`, self-signed, 10-year (`nix/host-modules/glue-bao-tls.nix:114-149`) | n/a | every client reads its own leaf off disk via `BAO_CLIENT_CERT` (`swarm-secret-client/src/client.rs:12-14`) | `NONE` — signed only when absent (`nix/host-modules/glue-bao-tls.nix:128-149`) | **always, by construction** — it has no other form | + +Four things in that table are uncomfortable, and they are the reason this page +exists rather than a note in the margin of the old one. + +**Every stored secret is persisted as a plaintext file outside the store.** All +four, without exception. The appservice token is persisted twice: once in the +publisher's own state directory on the authelia host, and again in the token +file on the hive that reads it. + +**Nothing rotates.** Every renewal cell reads `NONE`. There is no rotation path +for any of the four, and none for the mTLS leaves either — those are signed once +if absent and carry ten years of validity, and deleting the CA to reissue +invalidates every client certificate already trusting it. + +**No agent container has a store identity at all.** There is not one +`BAO_ADDR` or `BAO_CLIENT_CERT` anywhere under `nix/agent-modules/`. An agent +never talks to the store; the hive host reads on the agent's behalf and writes +a file into the container. That is the exact inverse of the target below. + +**The appservice token has a second, uncoordinated minter.** `hive-matrix.nix` +keeps a local first-boot mint as a fallback for an unreachable store. Two +minters exist for one logical credential, on different hosts, with no agreement +between them — they can diverge, and the only thing that reconciles them is the +reader overwriting the local value on its next successful start. + +One half of the target is already real: all four families are minted once and +read by an identity that logs in with a certificate, with no ambient trust +anywhere in the path. The cleanest examples are `swarm-bao-queue-agent.service` +and `swarm-bao-grafana-oidc.service`. What each of them then does is write the +fetched bytes to disk. + +Two per-agent credential files — the forge token and the github token +(`hive-priv/src/main.rs:368,413`) — are deliberately absent from the table. +They are operator-supplied and never pass through the store at all, so they are +not credentials this page governs. Their absence is not an oversight. + +## Target state + +⚠️ **This section describes what the swarm is moving toward. None of it is a +description of what exists today.** Read the current-state table above for +that. In particular, the first line below is the one most easily misread as a +statement of fact, so it is stated as a target twice. + +**Every host needs a store mTLS certificate. Everything else is done through +the store.** That is the target. Today only the host running the store itself +auto-mints leaves — for itself, and pre-emptively for a controller and a +publisher that might be co-located. For every other host, `swarm-bao.nix:527-528` +says it outright: this is _"the credential an operator places by hand."_ There +is no automated issuance path for an ordinary hive. The target is one +out-of-band credential per host and zero out-of-band credentials anywhere else; +the current state is one out-of-band credential per host **plus** every file in +the last column of the table above. + +**Agent containers get an mTLS certificate passed in from the hive, and pull +their secrets themselves.** The hive's job becomes handing the agent an +identity, not handing it secrets. Today no agent has an identity of any kind +and every agent secret arrives as a file the host wrote. + +**No store-provided secret is persisted.** A value read from the store lives in +the memory of the process that needs it and nowhere else — not in a state +directory, not in a bind-mounted file, not in a systemd credential. Today every +one of the four is persisted, and one of them twice. + +**Per secret, the docs specify minter, reader, and renewal strategy.** Those +three columns are the contract. A secret whose renewal strategy is `NONE` is +allowed to exist only as an honest record of where the swarm is — it is not a +strategy, and a new secret does not get to write it. + +## Progressive enhancement + +The rule has two halves, and the second is the one that makes the first +survivable. + +**New functionality matches the target shape immediately.** There is no +grandfather clause for code that does not exist yet. + +**Existing functionality moves step by step** — each individual step moving +toward the target shape, never sideways and never away. A step does not have to +finish the migration. It has to not undo it. + +That is checkable, so check it. Apply these to a pull request that touches a +credential: + +| question | a `yes` means | +| ----------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | +| Does it add a minter outside the swarm's existing mint path? | **block** for new functionality. For existing functionality, only if it removes one too. | +| Does it persist a store-provided secret to disk — a state directory, a bind mount, a rendered config? | **block** for new functionality. For existing functionality, only if the change removes a persisted copy elsewhere and nets out fewer. | +| Does it add a credential whose renewal strategy is `NONE`? | **block** for new functionality. State the strategy, even if it is "operator reissues and restarts the reader." | +| Does it read a secret on another principal's behalf and hand it over? | **block** for new functionality. The principal that needs the value authenticates for it. | +| Does it give a host or container an out-of-band credential that is not the store mTLS leaf? | **block**, both. One out-of-band credential per principal is the whole point of the store. | +| Does it change the minter, reader, or renewal of an existing credential? | The current-state table above is **stale until updated in the same change.** | + +For existing functionality the questions are not pass/fail — they are a +direction check. A change that persists one fewer copy, or moves one reader from +a host-written file to its own store read, is a step and lands. A change that +leaves the count the same while moving the file somewhere tidier is not a step; +it is churn with a migration story attached. + +When the last `yes` in the current-state table's final column becomes a `no`, +this page replaces `secrets.md` and the current-state section goes with it.