render the matrix appservice registration from the store onto tmpfs
swarm-matrix-ctl appservice render now reads the registration's tokens from the store and renders swarm.yaml under /run/swarm-matrix-appservice, which tuwunel loads through LoadCredential. It mints a pair only when the store answers and holds none; a host that has the old files stores their pair once instead. A store that does not answer fails the unit and keeps tuwunel down; sync starts it once the store answers. Refs #4998 (item 3)
This commit is contained in:
atlas
mara
parent
7813337548
commit
3a4b0ffabf
2 changed files with 594 additions and 378 deletions
|
|
@ -125,11 +125,12 @@ creation doesn't depend on registration being open to anyone who learns
|
|||
a token.
|
||||
|
||||
1. **`swarm-matrix-ctl appservice render`**, inside the `hive-matrix`
|
||||
container and before tuwunel starts, renders the registration to
|
||||
`/var/lib/swarm-matrix-appservice/swarm.yaml` (mode `0600 root:root`)
|
||||
from the `as_token` and spec-required `hs_token` beside it. It mints
|
||||
them only on a container that has none, and never reaches the store, so
|
||||
a store outage can't keep the homeserver down.
|
||||
container and before tuwunel starts, fetches the `as_token` and
|
||||
spec-required `hs_token` from the store and renders the registration to
|
||||
`/run/swarm-matrix-appservice/swarm.yaml` (mode `0600 root:root`) on the
|
||||
container's tmpfs, so no token sits on disk. It mints a pair only when
|
||||
the store answers and holds none. While the store doesn't answer, `render`
|
||||
fails and retries, and tuwunel stays down.
|
||||
2. **systemd `LoadCredential=`** copies the registration into
|
||||
`/run/credentials/tuwunel.service/swarm-appservice.yaml`, owned by
|
||||
tuwunel's dynamic user with mode `0400`, at service start. The source
|
||||
|
|
@ -167,15 +168,17 @@ account with it.
|
|||
|
||||
The swarm secret store holds the registration's tokens, at
|
||||
`swarm/controller/swarm-controller/matrix/appservice-registration`.
|
||||
`swarm-matrix-ctl appservice render` renders the registration tuwunel
|
||||
loads as a credential from the copy of them in the `hive-matrix`
|
||||
container, before tuwunel starts and without the store.
|
||||
`swarm-matrix-ctl appservice render` fetches them and renders the
|
||||
registration tuwunel loads as a credential onto the `hive-matrix`
|
||||
container's tmpfs, before tuwunel starts.
|
||||
`appservice sync` runs beside tuwunel on a timer. The store's tokens win:
|
||||
when they differ from the container's, it writes them there. It
|
||||
when they differ from the rendered copy, it rewrites it. It
|
||||
restarts tuwunel whenever the registration is newer than tuwunel's last
|
||||
start, so the next run retries one that failed after writing. A store that holds none takes the container's
|
||||
tokens with a create-only write, so a running swarm keeps its value, and
|
||||
a container that has none mints a pair. `sync` then writes the `as_token`
|
||||
start, so the next run retries one that failed after writing, and starts
|
||||
one that failed because `render` couldn't reach the store. A store that
|
||||
holds none takes the tokens an earlier version kept in
|
||||
`/var/lib/swarm-matrix-appservice` with a create-only write, so a running
|
||||
swarm keeps its value; the files stay, unused. A host with neither files nor stored tokens mints a pair. `sync` then writes the `as_token`
|
||||
to `swarm/controller/swarm-controller/matrix/appservice-token`. No hive's
|
||||
policy reaches either path: only matrix-ctl (which holds `create` and
|
||||
`read` on the registration, and writes the token) and `swarm-controller`
|
||||
|
|
@ -263,8 +266,8 @@ both are swarm-level operations.
|
|||
<details><summary>Appservice identity</summary>
|
||||
|
||||
`swarm-matrix-ctl appservice render` renders the registration from the
|
||||
container's copy of the appservice tokens before tuwunel starts, so every
|
||||
boot has both halves, with the store up or down.
|
||||
store's copy of the appservice tokens before tuwunel starts, so every
|
||||
boot has both halves. With the store down or sealed, tuwunel waits for it.
|
||||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
- **Access tokens live on the device that minted them.**
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ the rest of the cell says how.
|
|||
| `swarm/agents/<agent>/matrix/<account>` | `swarm-controller`, from a credential you hand it | the agent | must be stated | ✅ retried within 2 to 16 minutes after a rejection |
|
||||
| `swarm/agents/<agent>/matrix-recovery/<account>` | `swarm-controller`, a random passphrase per account | the agent | ❌ never replaced: the account's key backup opens only with this passphrase | ❌ read when the account comes up |
|
||||
| `swarm/controller/swarm-controller/matrix/appservice-token` | `swarm-matrix-ctl appservice sync` in `hive-matrix`, from the registration's `as_token` | `swarm-controller` | ❌ follows the registration below | ✅ read every 5 minutes |
|
||||
| `swarm/controller/swarm-controller/matrix/appservice-registration` | `swarm-matrix-ctl appservice sync` in `hive-matrix`: seeded from the container's own tokens when the store holds none, a new pair only when the container has none either | `swarm-matrix-ctl` in `hive-matrix`; the controller has no grant to it | ❌ minted once; an operator who replaces it makes `sync` write the new pair into the container and restart the homeserver | ✅ `sync` re-reads it every 10 minutes |
|
||||
| `swarm/controller/swarm-controller/matrix/appservice-registration` | `swarm-matrix-ctl appservice render` in `hive-matrix`, when the store answers and holds none; before that, the tokens an earlier version kept in the container, stored unchanged | `swarm-matrix-ctl` in `hive-matrix`; the controller has no grant to it | ❌ minted once; an operator who replaces it makes `sync` render the new pair and restart the homeserver | ✅ `sync` re-reads it every 10 minutes |
|
||||
| `swarm/controller/swarm-controller/oidc/client` | authelia, first boot | `swarm-controller` | ❌ minted once, straight into the store; minted again only when the store holds none | ✅ re-read when authelia refuses it |
|
||||
| `swarm/controller/swarm-controller/forge/token` | the forge, once | `swarm-controller` | ❌ minted once; delete `/var/lib/forgejo/swarm-controller-token-name` in the forge container to mint a new one | ✅ re-read on a 401 |
|
||||
| `swarm/controller/swarm-controller/grafana/token` | `swarm-bao-grafana-controller-token` on Grafana's host | `swarm-controller` | ✅ every 10 minutes, when Grafana refuses it or none is stored | ✅ re-read on a 401 |
|
||||
|
|
@ -113,7 +113,7 @@ pieces fit, and the reasoning behind them, is in [`credentials.md`](credentials.
|
|||
| swarm-services root CA (cert + key) | `swarm-bao.nix`'s granting unit, inside the store's `pki` mount | the **key** never leaves the mount; the **cert** lands at `/var/lib/swarm-bao-services-pki/services-root.pem` (0644) and at `/var/lib/hive-tls/swarm-services-root.pem` on every hive | nothing to copy — every hive receives the cert with the leaf it asks the store for, or from `pki/ca/pem` where it fronts no swarm name and needs a store identity |
|
||||
| authelia session, JWT, storage-encryption and OIDC HMAC keys | `authelia-swarm-secrets` in authelia's container. It stores each in the swarm secret store at `swarm/services/swarm-authelia/keys/<name>` when the store holds none: the value of the key an earlier release generated, or a new one when there is no such file. The files stay on disk. While the store's policy doesn't grant the leaf yet, or the store is down, the unit runs on the files. A container with no secret publisher identity generates the files, as before | `/var/lib/authelia-swarm/{session,jwt,storage-encryption,oidc-hmac}.key` | nothing to place: the publisher's store leaf is the one in the row below. A store that stays unreachable on a start with no key file keeps authelia from starting |
|
||||
| authelia OIDC issuer key (RSA) | same unit, same store path (`oidc-issuer`) | `/var/lib/authelia-swarm/oidc-issuer.key` | same — relying parties verify against the **public** half at `/jwks.json` |
|
||||
| swarm appservice registration (`as_token`, `hs_token`) | `swarm-matrix-ctl appservice sync` in `hive-matrix`. The store's pair wins; a store holding none takes the container's own tokens, so the value doesn't change by moving. Only `appservice render` on a container with neither mints a new pair | `/var/lib/swarm-matrix-appservice/{as-token,hs-token,swarm.yaml}` in `hive-matrix`, the last copy `sync` took, which tuwunel starts from; the store path is `swarm/controller/swarm-controller/matrix/appservice-registration` | nothing to place: matrix-ctl's store leaf is the one the container binds. A store that's down leaves the homeserver running on the files |
|
||||
| swarm appservice registration (`as_token`, `hs_token`) | `swarm-matrix-ctl appservice render` in `hive-matrix`. The store's pair wins; a store holding none takes the tokens an earlier version kept in the container, so the value doesn't change by moving. Only a store that answers, holds none, and a container with no such files gets a new pair | `/run/swarm-matrix-appservice/swarm.yaml` in `hive-matrix` (tmpfs), which tuwunel loads through `LoadCredential`. The earlier `/var/lib/swarm-matrix-appservice/{as-token,hs-token,swarm.yaml}` stay, unused, and bao wins if they differ; the store path is `swarm/controller/swarm-controller/matrix/appservice-registration` | nothing to place: matrix-ctl's store leaf is the one the container binds. A store that's down keeps the homeserver down and mints nothing, and `render` retries every 30 seconds for 24 hours |
|
||||
| OIDC client secret, plaintext half | `authelia crypto hash generate --random` | the swarm secret store, at the path its reader fetches: `swarm/hives/<hive>/oidc/client`, `swarm/controller/swarm-controller/oidc/client` or `swarm/services/<client id>/oidc/client`. The bridge reads its own into `/run/swarm-authelia-bridge/client-secret` before each start. A container with no secret publisher identity keeps it in `/var/lib/authelia-swarm/oidc-clients/<id>.secret`; a file an earlier release left seeds the store once when it holds no value, and stays unused | nothing to place: readers fetch it from the store ([how](credentials.md#hive-level-secrets)) |
|
||||
| OIDC client secret, digest half | the same mint | `oidc-clients/<id>.digest` | authelia's own half; merged at runtime via `settingsFiles` |
|
||||
| the swarm collector's copy of its OIDC secret | `swarm-bao-otel-oidc.service` reads it out of the swarm secret store, **on every host that runs the collector and holds a store identity** | `/run/swarm-otel-oidc/<id>.secret` on the host's tmpfs, bind-mounted read-only into the `swarm-otel` container — never on persistent disk, so a reboot fetches it again | same unit, same path — one route, co-located or not. A collector with no store identity has `clientSecretFile == null`, its already-supported unauthenticated-push degrade — see [swarm-level secrets](credentials.md#swarm-level-secrets) |
|
||||
|
|
|
|||
Loading…
Reference in a new issue