From 1e5070796d2b3046d10e7d28c7d503c132ea88ed Mon Sep 17 00:00:00 2001 From: atlas Date: Tue, 27 Oct 2026 15:51:13 +0100 Subject: [PATCH] copy the cache's pull facts to each hive and create them in the in-swarm cache the in-swarm cache stores its address, name and public key beside its tokens, create-only. swarm-controller copies the four cache facts to swarm/hives//attic/* every five minutes, with read on the sources and write on the copies. part 1 of 3: nothing reads the copies until the hive reader and probe land. --- docs/swarm/builder.md | 25 ++ docs/swarm/credentials-reference.md | 72 ++-- .../glue-bao-readers-policy-order.nix | 3 +- nix/host-modules/glue-bao-tls.nix | 2 +- nix/host-modules/lib/attic-init.nix | 29 +- nix/host-modules/swarm-attic.nix | 15 +- nix/host-modules/swarm-bao.nix | 48 ++- nix/module-eval/bao-grants.nix | 61 +++- nix/script-tests/attic-init.nix | 4 + nix/script-tests/attic-init.sh | 13 +- swarm-controller/src/attic_copy.rs | 323 ++++++++++++++++++ swarm-controller/src/main.rs | 1 + swarm-controller/src/startup.rs | 4 +- swarm-secret-client/src/attic.rs | 104 ++++++ swarm-secret-client/src/lib.rs | 3 +- 15 files changed, 640 insertions(+), 67 deletions(-) create mode 100644 swarm-controller/src/attic_copy.rs create mode 100644 swarm-secret-client/src/attic.rs diff --git a/docs/swarm/builder.md b/docs/swarm/builder.md index e90780d2..0bbbad4e 100644 --- a/docs/swarm/builder.md +++ b/docs/swarm/builder.md @@ -311,12 +311,21 @@ principal and create what's missing. - `swarm-bao-attic-tokens` creates the cache and mints the push token, which the builder reads, at `secret/swarm/services/attic/push-token`, and a read-only pull token for the hives at `secret/swarm/services/attic/pull-token`. + It also stores the facts a hive pulls with: the cache's address at + `secret/swarm/services/attic/url` (`https://`), its name at + `secret/swarm/services/attic/cache`, and the public key it signs with at + `secret/swarm/services/attic/public-key` (the key `attic cache info` prints). A unit creates each secret only while the store holds none, so a second start mints nothing. Both tokens last ten years and nothing renews them. To replace one, write the new token to its path in the store: nothing mints over a value that exists. A new signing key invalidates both tokens. +The unit creates the address, name and key once and never updates them. After +you change `swarm.attic.domain`, or after the server's database goes and the +cache gets a new key, delete the stale leaf and restart +`swarm-bao-attic-tokens`. + `swarm.attic.domain` and `swarm.attic.cache` default to `attic.` and `swarm` on every host, whether or not it runs the cache, so a builder on any host pushes to it. @@ -345,6 +354,14 @@ the swarm at an existing tenant: or `bao kv put` with `value=-` and the token on stdin, so the token is in no command line. The token needs push rights on that cache. Until it exists, every push fails and builds stay green with `cached` false. + Seed the hives' pull token the same way at + `secret/swarm/services/attic/pull-token`, with read rights on that cache. +3. Seed the three facts hives pull with, each as field `value`: + `secret/swarm/services/attic/url` (`https://` and the host, no trailing + slash), `secret/swarm/services/attic/cache` (the cache's name) and + `secret/swarm/services/attic/public-key` (`:`, from the + cache's page or `attic cache info`). The controller copies the leaves that + exist. The builder checks the instance's certificate against the system roots and pins nothing, so a Let's Encrypt certificate needs no setup. The builder @@ -358,6 +375,14 @@ The swarm then renders no container, vhost or unit of its own cache, and creates no secret in the store. The store's host still mints the cache's leaf and writes its role, which nothing logs in to. +## What hives do with the cache + +Every five minutes `swarm-controller` copies the four `swarm/services/attic/` +leaves (`pull-token`, `url`, `cache`, `public-key`) to +`swarm/hives//attic/` for each hive, because a hive reads only its +own prefix. It writes a copy only when the value differs, skips a leaf the +store doesn't hold, and leaves a copy whose source disappeared. + ## Credentials | What | Where | Who reads it | diff --git a/docs/swarm/credentials-reference.md b/docs/swarm/credentials-reference.md index f73686b7..747fbda0 100644 --- a/docs/swarm/credentials-reference.md +++ b/docs/swarm/credentials-reference.md @@ -14,40 +14,44 @@ the rest of the cell says how. -| store path | minter | reader | automatic re-mint | automatic re-pull | -| ---------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `swarm/agents//matrix/main` | `swarm-controller`, at agent creation and every 5 minutes | the agent | ✅ when missing or rejected by the homeserver | ✅ the daemon restarts within 5 minutes of a rejection | -| `swarm/agents//matrix/` | `swarm-controller`, from a credential you hand it | the agent | must be stated | ✅ retried within 2 to 16 minutes after a rejection | -| `swarm/agents//matrix-recovery/` | `swarm-controller`, a random passphrase per account | the agent | ❌ never replaced: the account's key backup opens only with this passphrase | ❌ read when the account comes up | -| `swarm/controller/swarm-controller/matrix/appservice-token` | `swarm-matrix-ctl appservice sync` in `hive-matrix`, from the registration's `as_token` | `swarm-controller` | ❌ follows the registration below | ✅ read every 5 minutes | -| `swarm/controller/swarm-controller/matrix/appservice-registration` | `swarm-matrix-ctl appservice sync` in `hive-matrix`: seeded from the container's own tokens when the store holds none, a new pair only when the container has none either | `swarm-matrix-ctl` in `hive-matrix`; the controller has no grant to it | ❌ minted once; an operator who replaces it makes `sync` write the new pair into the container and restart the homeserver | ✅ `sync` re-reads it every 10 minutes | -| `swarm/controller/swarm-controller/oidc/client` | authelia, first boot | `swarm-controller` | ❌ minted once; `swarm-secret-publish` republishes a re-mint | ✅ re-read when authelia refuses it | -| `swarm/controller/swarm-controller/forge/token` | the forge, once | `swarm-controller` | ❌ minted once; delete `/var/lib/forgejo/swarm-controller-token-name` in the forge container to mint a new one | ✅ re-read on a 401 | -| `swarm/controller/swarm-controller/grafana/token` | `swarm-bao-grafana-controller-token` on Grafana's host | `swarm-controller` | ✅ every 10 minutes, when Grafana refuses it or none is stored | ✅ re-read on a 401 | -| `swarm/controller/swarm-controller/forge/webhook-secret` | `swarm-controller`, at start, from its old `webhook-secret` file; a new one only when there is no file | `swarm-controller` | ❌ minted once; a rotated secret fails every delivery until the forge hooks are re-registered | ❌ read at start | -| `swarm/agents//bao-mtls` | the store's agent PKI mount (`deploy.bao.agentPkiMountPath`), at agent creation | `hive-c0re`, then the agent | ✅ re-issued every 5 minutes once past half its 7-day validity | ✅ the agent re-reads every 10 minutes; it renews only while its current leaf is valid | -| `swarm/agents//queue` | `swarm-controller`, at agent creation | the agent; the queue, log and metrics auth services | ✅ re-minted at 45 days; skipped for `Destroyed` agents, whose path is deleted | ✅ read on every reconnect; an open connection keeps the old secret for up to an hour; the log and metrics stores honour a replaced secret for up to `deploy.victorialogs.authVerdictTtl` and `deploy.victoriametrics.agentAuthVerdictTtl` (60 s by default) | -| `swarm/agents//forge-token` | `swarm-controller`, at agent creation and every 5 minutes | the agent | ✅ when missing or no longer matching the forge | ✅ every 10 minutes | -| `swarm/agents//forge/