vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

matrix: drop the per-hive appservice and its unused token

Browse source 
Every hive's homeserver loaded a per-hive `hyperhive` appservice
registration, and the swarm minted and delivered an as_token for it
through bao. No client presented that token: swarm-controller creates
every `@hive-<hive>` sender and every agent account with the swarm
appservice token (hive_sender.rs, agent_token.rs), and no Rust code read
the per-hive one.

Removed, with everything that existed only to serve it:
- hive-matrix.nix: the `hyperhive` registration, its activation-time
  render, the host bind mount of /var/lib/hyperhive/matrix-appservice,
  its LoadCredential entry, and the internal options
  `deploy.matrix.appserviceTokenFile` / `appserviceRegistrationScript`.
  tuwunel now loads only the swarm registration.
- glue-matrix-bao-token.nix and its `swarm-bao-matrix-token` unit.
- glue-bao-tls.nix: the matrix-token leaf and its defaults.
- swarm-bao.nix: the `swarm-matrix-token-<hive>` roles and policies
  (`swarm-bao-matrix-token-policy`), and the options
  `deploy.bao.matrixTokenClient{Cert,Key}File` and
  `matrixTokenCommonNamePrefix`, now mkRemovedOptionModule.
- swarm-secret-publisher.nix: the per-hive token mint and its state dir.
- swarm.nix: the reserved `<matrixTokenCommonNamePrefix>-<hive>` CN.
- swarm-secret-client: `matrix::appservice_token_path`.

The swarm registration (swarm-matrix-ctl, swarm-controller) is untouched.
Comments and docs that pointed at the removed glue file now point at
glue-queue-agent-credential.nix, which has the same reader shape.

Refs #4428
This commit is contained in:
atlas 2026-10-04 22:52:22 +02:00 • committed by mara
commit 48067d07e8
31 changed files with 362 additions and 1233 deletions
Download patch file Download diff file Expand all files Collapse all files

97
docs/integrations/matrix.md
View file

@ -105,8 +105,9 @@ delegation (the latter lives in `gateway.md::Discovery flow`).
## Provisioning flow (appservice)
<!-- vale write-good.Passive = NO -->
Registration is closed. **Appservices** create accounts: agents never see
an appservice token, and an agent only ever receives its own `access_token`.
Registration is closed. **The swarm's appservice** creates accounts: agents
never see an appservice token, and an agent only ever receives its own
`access_token`.
<!-- vale write-good.Passive = YES -->
The appservice has no URL (`url: null` in its registration), so the
@ -116,48 +117,42 @@ no secret has to be equal on both sides of the wire, and why account
creation doesn't depend on registration being open to anyone who learns
a token.
1. **System activation** mints a 32-byte random hex appservice token (64
chars) at `/var/lib/hyperhive/matrix-appservice-token` and its
spec-required `hs_token` sibling, mode `0600 root:root`, then renders
the registration to
`/var/lib/hyperhive/matrix-appservice/hyperhive.yaml` (also `0600`).
The script mints the tokens only when missing; the registration is
re-rendered every time, because the token file can be overwritten in
place by the swarm secret store and a registration naming a stale
token authenticates nobody. Runs at activation time, before any
container start, because the directory is bind-mounted and
nixos-container refuses to start when a bind source is missing.
2. **Read-only bind-mount** maps that directory into the tuwunel
container at the same path.
3. **systemd `LoadCredential=`** inside the container copies the
registration into
`/run/credentials/tuwunel.service/hyperhive-appservice.yaml`, owned by
tuwunel's dynamic user with mode `0400`, at service start. The host
1. **`swarm-matrix-ctl appservice render`**, inside the `hive-matrix`
container and before tuwunel starts, mints the registration's
`as_token` and its spec-required `hs_token` when they're missing, and
renders the registration to
`/var/lib/swarm-matrix-appservice/swarm.yaml` (mode `0600 root:root`).
2. **systemd `LoadCredential=`** copies the registration into
`/run/credentials/tuwunel.service/swarm-appservice.yaml`, owned by
tuwunel's dynamic user with mode `0400`, at service start. The source
file stays `root:root 0600` — no `chown :tuwunel` / `chmod 0640` /
GID-pin gymnastics required. Keeps `DynamicUser = true` +
`PrivateUsers = true` intact.
4. tuwunel's `appservice_dir` points at the credentials directory, not at
the bind-mount path. It reads only `.yaml`/`.yml` entries from there,
so the sibling credentials are invisible to it. The `.yaml` suffix on
the credential id is what makes this work.
5. **hive-c0re** doesn't read the appservice token and creates no
account. Its `@hive-<hive>:` token comes from the swarm store (below).
6. **Agents' accounts aren't this hive's.** `swarm-controller` creates each
one with the swarm's own appservice token (next section), stores its
token at `swarm/agents/<agent>/matrix/main`, and the agent's
`hive-matrix-daemon` reads it from there as the agent itself.
3. tuwunel's `appservice_dir` points at the credentials directory. It
reads only `.yaml`/`.yml` entries from there, so the sibling
credentials are invisible to it. The `.yaml` suffix on the credential
id is what makes this work.
4. **hive-c0re** holds no appservice token and creates no account. Its
`@hive-<hive>:` token comes from the swarm store (below).
5. **`swarm-controller` creates every account**: each agent's, with its
token stored at `swarm/agents/<agent>/matrix/main` for the agent's
`hive-matrix-daemon` to read as the agent itself, and each hive's
sender account (below).
A homeserver whose `swarm-matrix-ctl` has no store identity
(`services.hyperhive.deploy.matrix.ctlBaoClientCertFile`) loads no
registration, and nothing creates accounts on it.
### The swarm's appservice, and its admin sender
A second registration sits beside the hive's: id `swarm`, sender `@swarm`,
the same non-exclusive namespace. It's the swarm's identity on the
homeserver, and `swarm-controller` creates every agent's account with it.
tuwunel loads every `.yaml` in `appservice_dir` and refuses only a
duplicate `id` or `as_token`, so the two overlapping namespaces coexist.
The registration's id is `swarm`, its sender `@swarm`, and its user
namespace non-exclusive. It's the swarm's identity on the homeserver:
`swarm-controller` creates every agent's account and every hive's sender
account with it.
`swarm-matrix-ctl appservice render` mints its tokens inside the
`hive-matrix` container, before tuwunel starts, and renders the
registration tuwunel loads as a second credential. `appservice publish`
registration tuwunel loads as a credential. `appservice publish`
then writes its `as_token` to
`swarm/controller/swarm-controller/matrix/appservice-token`. No hive's
policy reaches that path: only matrix-ctl (which writes it) and
@ -199,13 +194,12 @@ lands on.
`services.hyperhive.deploy.swarm-controller.authBridgeUrl` and
`services.hyperhive.deploy.swarm-controller.matrixHomeserverUrl` set.
### The appservice's sender account, and why it isn't an admin
### A hive's sender account, and why it isn't an admin
`@hive-<hive>:<server_name>` is the appservice's own `sender_localpart`,
which the homeserver creates itself when it loads the registration — on a
zero-user database, inside startup, before the HTTP listener accepts
anything. It's an **ordinary account**: nothing promotes it; the
homeserver's `admin_execute` promotes only `@swarm`.
`@hive-<hive>:<server_name>` is the account a hive acts as.
`swarm-controller` creates it with the swarm's appservice token, for every
hive in its directory. It's an **ordinary account**: nothing promotes it;
the homeserver's `admin_execute` promotes only `@swarm`.
**One account per hive.** The localpart carries the hive's name, so a swarm
whose hives share a homeserver gives each of them its own identity: the
@ -220,8 +214,8 @@ appservice token and publishes it to `swarm/hives/<hive>/matrix/sender-token`, a
reads it from there under its own certificate. That path sits inside the
hive's own read grant (`swarm/hives/<hive>/*`), so a hive fetches its own
token and gets a refusal on any other hive's. The name says what it
authenticates as — an account the appservice registration brings into being
— rather than any privilege level, because it carries none.
authenticates as — an account the swarm's appservice creates — rather
than any privilege level, because it carries none.
It needs no promotion for what the hive does with it. Creating the hive
Space and the chat room, writing their hierarchy and join rules, and
@ -251,8 +245,8 @@ required.
<details><summary>Appservice identity</summary>
The activation script mints the appservice token and renders the
registration before the homeserver restarts, so every boot has both
`swarm-matrix-ctl appservice render` mints the appservice tokens and
renders the registration before tuwunel starts, so every boot has both
halves.
<!-- vale write-good.Passive = NO -->
@ -261,11 +255,10 @@ halves.
too.
- **The homeserver's `admin_execute` promotes only `@swarm` at boot**
(above); a hive's sender account is never promoted.
- **The value that matters lives at `matrix/appservice-token`.**
`swarm-secret-publish` on the authelia host mints and `put`s it
there; the hive uses its locally minted token only until the first
successful read. See [`../swarm/secrets.md`](../swarm/secrets.md)
for how that mint stays idempotent across runs.
- **The value that matters lives at
`swarm/controller/swarm-controller/matrix/appservice-token`.**
`appservice publish` writes the container's copy there whenever the
store's differs, and `swarm-controller` reads it from there.
<!-- vale write-good.Passive = YES -->
</details>
@ -277,8 +270,8 @@ Initial rollout settings:
restart. `trusted_servers = []` keeps it effectively closed
until you list peers.
- `allow_registration = false`. tuwunel checks this flag only for
requests that arrive **without** an appservice token, so the appservices
still create accounts and tuwunel refuses everyone else. With no
requests that arrive **without** an appservice token, so the swarm's
appservice still creates accounts and tuwunel refuses everyone else. With no
registration token configured, `allow_registration = true` makes
tuwunel refuse to start unless
`yes_i_am_very_very_sure_…_open_registration_…` is also set.

1
docs/swarm/credentials.md
View file

@ -66,7 +66,6 @@ of the cell says how.
| `swarm/agents/<agent>/forge-token` | `swarm-controller`, at agent creation and in a pass every 5 minutes over every agent with a store identity | the agent container itself, under its own certificate, fetched to `/run/hive-agent-forge-token/token` | ✅ the controller re-mints when the stored token is missing or no longer matches the forge (last eight characters and scopes) | ✅ the agent re-fetches on a 10-minute timer |
| `swarm/agents/<agent>/forge/<label>` | `swarm-controller`, when an operator links an external forge account in the swarm UI | `hive-agent-forge-accounts` in the agent container, under the agent's own certificate, into `<state>/forge-<label>-token` and `forge-<label>.json` | ❌ an operator's token; replaced only by deleting it and linking the label again | ✅ the agent re-fetches on a 2-minute timer |
| `swarm/agents/<agent>/github-token` | `swarm-controller`, when an operator links a GitHub account in the swarm UI | `hive-agent-github-token` in the agent container, under the agent's own certificate, into `<state>/github-token` | ❌ an operator's token; replaced only by deleting it and linking it again | ✅ the agent re-fetches on a 2-minute timer |
| `swarm/hives/<hive>/matrix/appservice-token` | one minter, on the authelia host | the hive process that presents the token to its homeserver, under the hive's own certificate | must be stated | must be stated |
| `swarm/hives/<hive>/matrix/sender-token` | `swarm-controller`, with the swarm's appservice token, for every hive in its directory in a five-minute pass | `swarm-controller` under its own certificate, before it decides whether to mint, and hive-c0re's `stored_sender_token()`, under the hive's own certificate | ✅ the controller's pass re-mints when the stored token is missing, unknown to the homeserver, or someone else's | ✅ hive-c0re's matrix sweep reads the store every run and overwrites its token file when the store's token differs |
| `swarm/hives/<hive>/queue/agent` | authelia | `swarm-bao-queue-agent` on the hive's host, under its own per-hive certificate; no agent's policy reaches it | must be stated | must be stated |
| `swarm/services/<clientId>/oidc/client` | authelia | that service's own reader, under its own certificate; the bao UI's by the store's granter | must be stated | must be stated |

96
docs/swarm/secrets.md
View file

@ -90,8 +90,7 @@ with nothing to authenticate with still receives every hive's telemetry —
only its own pushes to the stores go out unauthenticated and get refused
there. `swarm-bao-otel-oidc.service` therefore doesn't render at all without
`deploy.bao.otelOidcClientCertFile` / `otelOidcClientKeyFile`, the same choice
`glue-matrix-bao-token.nix` and `glue-queue-agent-credential.nix` make for
their own optional readers, rather than the hard assertion `swarm-grafana.nix`
`glue-queue-agent-credential.nix` makes for its own reader, rather than the hard assertion `swarm-grafana.nix`
uses for its non-optional one. The operator can still deliver the secret by
hand, at `services.hyperhive.deploy.swarm-otel.clientSecretFile`, on a
collector this unit never reaches.
@ -159,7 +158,6 @@ when the system builds. The server names the offending file and refuses to run.
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------- |
| hive leaf certs | `hive-tls.nix`, issued by the secret store's `pki` mount | `<deploy.hive-controller.tls.stateDir>/<name>.pem` |
| the hive's own store identity | `glue-bao-tls.nix` on the store's host; placed by hand on every other host ([`ca.md`](ca.md#the-gateway-leaf-comes-from-the-secret-store)) | `deploy.hive-controller.tls.baoHiveIdentityCertFile` / `baoHiveIdentityKeyFile` |
| matrix appservice token | `swarm-secret-publish`, published to the store; a host activation script only as a first-boot fallback | `/var/lib/hyperhive/matrix-appservice-token` (`0600`) |
| the forge's copy of its OIDC secret | `hive-forge-oidc-secret.service` copies it from authelia's tree | `/var/lib/forgejo-oidc/<id>.secret` inside the forge container |
| the homeserver's copy of its OIDC secret | `hive-matrix-oidc-secret.service`, same shape | `/var/lib/tuwunel-oidc/<id>.secret`, handed to tuwunel through `LoadCredential` |
| the agent containers' queue credential | authelia, published to the store by `swarm-secret-publish` | `<deploy.hive-controller.queue.agentCredentialDir>/secret` (`0600`) and `/client_id` (`0644`) |
@ -169,50 +167,7 @@ bounded wait, 120s — and then **fail loudly** rather than skipping. A silent s
produces a service whose login button always fails, which is a symptom many
layers from its cause.
The store's **first reader** is the matrix appservice token, and it's worth
saying why that one: it's an opaque 32-byte value with no second file and no
format. Authelia's OIDC secret needs a `.secret` _and_ a matching `.digest`, so
starting there would have meant debugging "can a reader authenticate and get
bytes back" and "is authelia's file format right" at once, with an
SSO outage as the failure mode.
`glue-matrix-bao-token.nix` fetches it, writes the file `hive-matrix.nix`
already reads, and then runs that module's own renderer to re-stamp the
appservice registration naming the token — so the homeserver never learns the
store exists. The re-render isn't housekeeping: the token is half an
agreement, and a registration carrying the previous value authenticates
nobody. Every failure path — no such key, sealed store, unreachable store,
empty value — leaves the locally minted token in place, so a hive with no store
behaves exactly as it did before.
The store path is `swarm/hives/<hive>/matrix/appservice-token`, and the
**producer is `swarm-secret-publish`** — the same unit that copies authelia's
OIDC secrets in. It mints one token per hive in the swarm's roster and `put`s
it there, so the store is the source of truth and every hive converges on the
value it holds. The hive's own activation mint is still there, but it's a
first-boot fallback now: it fires only when the token file is absent, and the
reader overwrites whatever it produced.
The mint is **idempotent by keeping its own record**, not by asking the store.
The publisher's grant is `create`/`update` under `swarm/hives/*` with no
`read` — write-only on purpose, so a file-copier can't recover every hive's
credentials — which means it can't check whether a hive already has a token.
Instead it keeps the value it minted under its `StateDirectory`,
`/var/lib/swarm-secret-publish/matrix-appservice-token/<hive>` (`0700` dir,
`0600` file), and mints only when that file is missing or empty. The `put`
itself runs every time: re-putting the same bytes changes nothing for any
reader, while a mint whose publish failed must not leave behind a token
this host holds and no hive can reach.
Lose the state directory and the next run mints once more and republishes.
That rotates the token, which readers pick up on their next start — nothing
that already registered breaks, because the token authenticates the
appservice rather than living inside any account it created.
The path was `…/matrix/registration-token` while the homeserver still took a
shared registration secret; nothing reads a value left at the old path.
The **second reader** is the agent containers' queue credential:
The store's **first reader** is the agent containers' queue credential:
`glue-queue-agent-credential.nix` lands it as two files, the client secret and
the client id it authenticates, because that's how a queue client reads them —
the secret by path, the id by value. No local fallback exists, and none is
@ -237,7 +192,7 @@ never reads the bytes at any point: it runs as `hive-core`, which is the reason
a copy wasn't an option either. When the files aren't there the daemon forwards
nothing and says so in its journal, and the container starts without them.
The **third reader** is Grafana's OIDC client secret, and it's the first one
The **second reader** is Grafana's OIDC client secret, and it's the first one
that's a _swarm service's_ own credential rather than a hive's. It has exactly
one delivery route: `swarm-bao-grafana-oidc.service` reads
`swarm/services/<client-id>/oidc/client` out of the store, on **every** host
@ -265,7 +220,7 @@ nothing said anything. Registration is separate and stays where authelia is
so it's declared on the host running authelia, which isn't necessarily the host
running Grafana.
The **fourth reader** is the swarm collector's own OIDC client secret, the
The **third reader** is the swarm collector's own OIDC client secret, the
same shape one level down: `swarm-bao-otel-oidc.service` reads
`swarm/services/<client-id>/oidc/client` out of the store on every host that
runs the collector, and registration lives separately in
@ -275,13 +230,12 @@ telemetry with none, so `swarm-otel.nix` doesn't assert
`deploy.bao.otelOidcClientCertFile` / `otelOidcClientKeyFile` the way Grafana
does — it
doesn't render the reading unit without them, the shape
`glue-matrix-bao-token.nix` and `glue-queue-agent-credential.nix` use for
their own optional reads.
`glue-queue-agent-credential.nix` uses for its own read.
The **fifth reader** is the secret store forwarder's OIDC client secret:
The **fourth reader** is the secret store forwarder's OIDC client secret:
`swarm-bao-forwarder-oidc.service`, on the store's own host, registered by
`glue-swarm-bao-otel-oidc-client.nix` on authelia's. It differs from the
fourth in both directions. It renders unconditionally, because the export it
third in both directions. It renders unconditionally, because the export it
authenticates has no unauthenticated mode to degrade into — and it orders
itself **after** `container@swarm-bao`, because the store it reads runs in the
container it delivers into. Nothing circular sits behind that: the identity it
@ -340,59 +294,55 @@ next controller restart.
### Per-principal identities
Bao matches a cert-auth role on the certificate's subject, so a certificate is an
identity and sharing one merges the identities. These five units read one path
identity and sharing one merges the identities. These four units read one path
each and each holds a leaf, a role and a policy of its own:
| unit | option pair under `deploy.bao.` | reads |
| -------------------------- | ------------------------------------------------------------ | -------------------------------------------------------- |
| `swarm-bao-matrix-token` | `matrixTokenClientCertFile` / `matrixTokenClientKeyFile` | `swarm/hives/<hive>/matrix/appservice-token` |
| `swarm-bao-queue-agent` | `queueAgentClientCertFile` / `queueAgentClientKeyFile` | `swarm/hives/<hive>/queue/agent` |
| `swarm-bao-grafana-oidc` | `grafanaOidcClientCertFile` / `grafanaOidcClientKeyFile` | `swarm/services/<grafana client id>/oidc/client` |
| `swarm-bao-otel-oidc` | `otelOidcClientCertFile` / `otelOidcClientKeyFile` | `swarm/services/<collector client id>/oidc/client` |
| `swarm-bao-forwarder-oidc` | `forwarderOidcClientCertFile` / `forwarderOidcClientKeyFile` | `swarm/services/<store forwarder client id>/oidc/client` |
The first two exist **per hive**, because the path they read carries a hive
name and every hive runs its own reader. Their subjects are
`<deploy.bao.matrixTokenCommonNamePrefix>-<hive>` and
`<deploy.bao.queueAgentCommonNamePrefix>-<hive>`; `swarm.nix` reserves both
composed spellings as hive names, so nobody can name a hive into another hive's
The first exists **per hive**, because the path it reads carries a hive name
and every hive runs its own reader. Its subject is
`<deploy.bao.queueAgentCommonNamePrefix>-<hive>`; `swarm.nix` reserves that
composed spelling as a hive name, so nobody can name a hive into another hive's
role. The other three read a path that names a swarm service rather than a hive,
so one role each is enough and their subjects are the flat
`deploy.bao.grafanaOidcCommonName`, `deploy.bao.otelOidcCommonName` and
`deploy.bao.forwarderOidcCommonName`.
On a host that mints its own PKI, `glue-bao-tls.nix` signs all five and defaults
all ten options, and there is nothing to do. Elsewhere you issue each leaf from
On a host that mints its own PKI, `glue-bao-tls.nix` signs all four and defaults
all eight options, and there is nothing to do. Elsewhere you issue each leaf from
that CA out of band and name it here — one file per principal rather than one file
shared by five, which is the whole of what this buys.
shared by four, which is the whole of what this buys.
On a host without the store, where the fifth never runs, forgetting one of the
other eight isn't a quiet degrade. Three of the four
readers used to render only where their leaf existed, so a hand-configured
remote-store hive that named the hive's own `clientCertFile` and missed a
principal's pair just lost that unit; `swarm-grafana.nix` was alone in refusing
the build. All four now refuse it, each naming its own option pair — see
On a host without the store, where the forwarder never runs, forgetting one of
the other six isn't a quiet degrade. A hand-configured remote-store hive that
names the hive's own `clientCertFile` and misses a principal's pair has the
build refused, by the module of the reader it missed, naming that pair — see
[what a missing leaf costs](#what-a-missing-leaf-costs) below.
### What a missing leaf costs
Each of the four modules asserts its own pair, on the same condition: **this host
Each of the three modules asserts its own pair, on the same condition: **this host
already reads the store** — it holds `deploy.bao.clientCertFile` and
`clientKeyFile` — **and the consumer is on, and this principal's pair is null**.
The refusal names the two options, so an operator acts on the message without
opening the nix.
That condition fires on exactly one shape, the hand-configured remote-store hive
that named seven of the eight. It stays silent everywhere else, and each half of
that named the hive's own pair and missed a principal's. It stays silent everywhere else, and each half of
that matters on its own:
- **A host with no store identity at all** holds no `clientCertFile` either, so
the first clause is false. That's the supported no-store deployment, and also
the one an operator passes through while bringing a hive up — neither should
fail to evaluate.
- **A host that runs the store** gets all eight from `glue-bao-tls.nix` as
- **A host that runs the store** gets every pair from `glue-bao-tls.nix` as
defaults, so the third clause is false. Nothing to set, nothing to refuse.
- **A consumer that's off** — no homeserver, no collector, no Grafana — has no
- **A consumer that's off** — no collector, no Grafana — has no
unit to skip, so the second clause is false. Only the queue-credential reader
has no toggle of its own to check: every hive runs one for its own agents, so
reading the store at all is what asks for it.

10
hive-c0re/src/matrix.rs
View file

@ -58,13 +58,9 @@ const HTTP_TIMEOUT_SECS: u64 = 10;
/// homeserver could not attribute an action to a hive and revoking one hive's
/// standing revoked all of them.
///
/// Also the `sender_localpart` of the hive's appservice registration,
/// which is what creates this account on a homeserver that has never had
/// one: the homeserver creates a registration's sender user itself, at
/// startup, before it accepts a request. See
/// `nix/host-modules/hive-matrix.nix`, where the same string is rendered as
/// `hiveLocalpart`. **The two must match**; nothing wires an override
/// across.
/// `swarm-controller` creates this account with the swarm's appservice token
/// (`matrix_account::hive_sender`) and stores its token where
/// [`stored_sender_token`] reads it; hive-c0re never creates it.
///
/// An ordinary account, with no homeserver-admin standing: what it
/// provisions — the Space, the chat room, the invites — it provisions as

1
nix/host-modules/default.nix
View file

@ -33,7 +33,6 @@
./glue-forge-oidc-client.nix
./glue-grafana-oidc-client.nix
./glue-hive-tls-bao-identity.nix
./glue-matrix-bao-token.nix
./glue-matrix-ctl-bao-identity.nix
./glue-nats-auth-bao-identity.nix
./glue-nats-bao-identity.nix

19
nix/host-modules/deploy.nix
View file

@ -257,23 +257,18 @@ in
# Both spellings of the registration token, removed rather than renamed:
# the homeserver no longer accepts a shared registration secret at all,
# so there is no path to point a definition at. Accounts are created by
# the hive's appservice, whose token lives at a fixed path this module
# deliberately does not let anyone move.
# the swarm's appservice.
(lib.mkRemovedOptionModule [ "services" "hyperhive" "swarm" "matrix" "registrationTokenFile" ] ''
The matrix registration token is gone: hive-c0re creates accounts as the
hive's appservice instead, so there is no shared secret for tuwunel to
check. Remove this definition — nothing needs replacing, the appservice
token is minted and delivered automatically
(/var/lib/hyperhive/matrix-appservice-token, not operator-settable).
The matrix registration token is gone: swarm-controller creates accounts
as the swarm's appservice instead, so there is no shared secret for
tuwunel to check. Remove this definition — nothing needs replacing.
The old token file at /var/lib/hyperhive/matrix-register-token is read by
nothing now and can be deleted. See docs/integrations/matrix.md.
'')
(lib.mkRemovedOptionModule [ "services" "hyperhive" "deploy" "matrix" "registrationTokenFile" ] ''
The matrix registration token is gone: hive-c0re creates accounts as the
hive's appservice instead, so there is no shared secret for tuwunel to
check. Remove this definition — nothing needs replacing, the appservice
token is minted and delivered automatically
(/var/lib/hyperhive/matrix-appservice-token, not operator-settable).
The matrix registration token is gone: swarm-controller creates accounts
as the swarm's appservice instead, so there is no shared secret for
tuwunel to check. Remove this definition — nothing needs replacing.
The old token file at /var/lib/hyperhive/matrix-register-token is read by
nothing now and can be deleted. See docs/integrations/matrix.md.
'')

4
nix/host-modules/glue-bao-readers-policy-order.nix
View file

@ -30,10 +30,6 @@ let
# Reader unit → the gate its own module defines it under.
readersHere = {
# ./glue-matrix-bao-token.nix
swarm-bao-matrix-token =
havePair baoDeploy.matrixTokenClientCertFile baoDeploy.matrixTokenClientKeyFile
&& deployCfg.matrix.enable;
# ./glue-queue-agent-credential.nix
swarm-bao-queue-agent =
deployCfg.hive-controller.enable

52
nix/host-modules/glue-bao-tls.nix
View file

@ -86,26 +86,23 @@ in
clientKeyFile = lib.mkDefault "${pkiDir}/client-key.pem";
serverCaFile = lib.mkDefault "${pkiDir}/ca.pem";
# 🩸 Four readers that used to present `client.pem` above, each now
# pointed at a leaf of its own. The leaf is what bao sees, so this pairing
# is the whole of what turns "one principal with the union of four grants"
# into four principals with one grant each — see ./swarm-bao.nix's
# `perHiveReaders` block for the grants themselves.
# 🩸 Readers that used to present `client.pem` above, each now pointed at
# a leaf of its own. The leaf is what bao sees, so this pairing is the
# whole of what turns "one principal with the union of their grants" into
# one principal per grant — see ./swarm-bao.nix's `perHiveReaders` block
# for the grants themselves.
#
# Defaulted here rather than in four `glue-<consumer>-bao-identity.nix`
# files: those exist where the consumer is a CONTAINER with a
# `deploy.<service>.*` namespace of its own to point at. These four are
# host units reading the store, which is the pairing this file already is.
matrixTokenClientCertFile = lib.mkDefault "${pkiDir}/matrix-token.pem";
matrixTokenClientKeyFile = lib.mkDefault "${pkiDir}/matrix-token-key.pem";
# Defaulted here rather than in `glue-<consumer>-bao-identity.nix` files:
# those exist where the consumer is a CONTAINER with a
# `deploy.<service>.*` namespace of its own to point at. These are host
# units reading the store, which is the pairing this file already is.
queueAgentClientCertFile = lib.mkDefault "${pkiDir}/queue-agent.pem";
queueAgentClientKeyFile = lib.mkDefault "${pkiDir}/queue-agent-key.pem";
grafanaOidcClientCertFile = lib.mkDefault "${pkiDir}/grafana-oidc.pem";
grafanaOidcClientKeyFile = lib.mkDefault "${pkiDir}/grafana-oidc-key.pem";
otelOidcClientCertFile = lib.mkDefault "${pkiDir}/otel-oidc.pem";
otelOidcClientKeyFile = lib.mkDefault "${pkiDir}/otel-oidc-key.pem";
# The fifth, the store forwarder's: the reader the four-way split left
# on `client.pem`.
# The store forwarder's.
forwarderOidcClientCertFile = lib.mkDefault "${pkiDir}/forwarder-oidc.pem";
forwarderOidcClientKeyFile = lib.mkDefault "${pkiDir}/forwarder-oidc-key.pem";
@ -196,26 +193,23 @@ in
[ -s ${pkiDir}/matrix-ctl.pem ] || ${signLeaf} ${pkiDir} matrix-ctl \
${lib.escapeShellArg deployCfg.bao.matrixCtlCommonName} "" clientAuth
# 🩸 The four readers that used to present `client.pem` above. Each
# carries its own subject, which is the entire mechanism: bao matches a
# cert-auth role on the CN, so four units sharing one leaf were one
# principal holding the union of four grants, and the union included
# every agent credential in the swarm.
# 🩸 The readers that used to present `client.pem` above. Each carries
# its own subject, which is the entire mechanism: bao matches a
# cert-auth role on the CN, so units sharing one leaf were one principal
# holding the union of their grants, and the union included every agent
# credential in the swarm.
#
# ⚠️ The first two carry THIS hive's name in the subject, unlike the
# three service leaves above. Their grants name one hive's path, because
# a matrix token and a queue credential live under `swarm/hives/<name>/`
# and every hive runs a reader for its own; ./swarm-bao.nix writes one
# role per hive in the swarm directory to match. The two OIDC readers
# need no such segment — a client is registered once per swarm.
# ⚠️ The first carries THIS hive's name in the subject, unlike the three
# service leaves above. Its grant names one hive's path, because a queue
# credential lives under `swarm/hives/<name>/` and every hive runs a
# reader for its own; ./swarm-bao.nix writes one role per hive in the
# swarm directory to match. The OIDC readers need no such segment — a
# client is registered once per swarm.
#
# Minted whether or not the consumer runs here, for the reason the three
# leaves above give: on a hive that does not run the store this is the
# file an operator copies, and a leaf that only appears where its
# consumer does is one nobody can copy from anywhere.
[ -s ${pkiDir}/matrix-token.pem ] || ${signLeaf} ${pkiDir} matrix-token \
${lib.escapeShellArg "${deployCfg.bao.matrixTokenCommonNamePrefix}-${clientCn}"} "" clientAuth
[ -s ${pkiDir}/queue-agent.pem ] || ${signLeaf} ${pkiDir} queue-agent \
${lib.escapeShellArg "${deployCfg.bao.queueAgentCommonNamePrefix}-${clientCn}"} "" clientAuth
@ -265,8 +259,8 @@ in
${lib.optionalString deployCfg.hive-controller.enable ''
# This hive's gateway-leaf identity, which opens
# `pki/issue/<hiveLeafPkiRolePrefix>-<hive>` and nothing else. Its
# subject carries this hive's name, like the matrix-token and
# queue-agent leaves above, because ./swarm-bao.nix writes one role
# subject carries this hive's name, like the queue-agent leaf
# above, because ./swarm-bao.nix writes one role
# per hive. Only where a hive runs; hives on other hosts are issued
# theirs out of band.
[ -s ${pkiDir}/hive-leaf.pem ] || ${signLeaf} ${pkiDir} hive-leaf \

256
nix/host-modules/glue-matrix-bao-token.nix
View file

@ -1,256 +0,0 @@
# Glue: the matrix appservice token comes from the secret store.
#
# The store's first reader, and deliberately a small one. It fetches an opaque
# 32-byte value and writes it where ./hive-matrix.nix already looks, then asks
# that module's own renderer to re-stamp the appservice registration naming it
# — the homeserver never learns the store exists, and its config is unchanged.
#
# ⚠️ Why this credential first. It has no second file and no format: authelia's
# OIDC secret needs a `.secret` *and* a matching `.digest`, so shipping that
# one first would debug "can a reader authenticate and get bytes back" and
# "did we write authelia's file format right" at the same time, with an SSO
# outage as the failure mode. Here the failure is narrow — new agent accounts
# cannot be provisioned, existing ones are untouched, nothing crash-loops.
#
# ⚠️ The fallback is today's behaviour, not a new one. `hive-matrix.nix`'s
# activation script still mints a token when the file is absent; this unit
# overwrites it with the swarm's copy when the store has one. A store that is
# empty or unreachable leaves a working hive with a local token.
#
# 📌 This runs wherever a client identity is configured, NOT only where the
# store is. `deploy.bao.enable` would have been the co-location assumption
# itself; the reader needs a certificate, not a neighbour. On the store's own
# host ./glue-bao-tls.nix supplies one as a `mkDefault` and nothing changes;
# elsewhere an operator places the leaf and names it, and the same unit works.
{
pkgs,
lib,
config,
...
}:
let
hyperhiveCfg = config.services.hyperhive;
deployCfg = hyperhiveCfg.deploy;
baoCfg = hyperhiveCfg.swarm.bao;
baoDeploy = deployCfg.bao;
# What decides whether this unit exists at all. A reader is defined by holding
# a certificate the store accepts, and that is true on the store's own host
# and on a hive three networks away for exactly the same reason.
#
# 🩸 This principal's OWN leaf, not `clientCertFile` — the hive's, which four
# units used to share. Bao matches a cert-auth role on the CN, so one leaf for
# four readers was ONE principal holding the union of four grants: read on
# `swarm/agents/*` AND `swarm/hives/<hive>/*` AND `swarm/services/*`, when
# this unit reads one appservice token and nothing else. Its own leaf carries
# `<matrixTokenCommonNamePrefix>-<hive>` and its role grants the single path
# below.
haveClientIdentity =
baoDeploy.matrixTokenClientCertFile != null && baoDeploy.matrixTokenClientKeyFile != null;
# Does this host read the store at all — the hive's own leaf, which is the
# one thing a remote-store deployment has always had to place by hand. Only
# used to decide whether a missing per-principal leaf is a mistake or a
# deployment that has no store: a host holding neither is the supported
# no-store shape, and one holding this pair but not the pair above named
# seven of the eight options and stopped.
hiveReaderIdentity = baoDeploy.clientCertFile != null && baoDeploy.clientKeyFile != null;
# Where the token lives in the store. A path, not a convention to guess at:
# whoever writes it and whoever reads it must agree, and the agreement
# belongs in one visible place.
#
# ⚠️ The `hives/<name>` segment is not decoration — it is what the reader's
# own grant covers. The store's read policy grants `swarm/agents/*` and
# `swarm/hives/<this hive>/*` and nothing else, so a path outside those is a
# 403 rather than a miss, however correct it looks. `swarm-secret-client`'s
# `matrix::appservice_token_path` builds the same string from the same
# pieces; this literal is the nix half of that one agreement.
#
# `hiveName` has no fallback here for the reason ./glue-bao-tls.nix gives at
# its own use of it: it is asserted set on every host that runs the
# homeserver.
tokenPath = "secret/swarm/hives/${hyperhiveCfg.hiveName}/matrix/appservice-token";
# A literal, not an option — ./hive-matrix.nix names its container
# `containers.hive-matrix` directly and declares no `machine` to derive it
# from, which the trust-bundle call in that file already says out loud.
# ⚠️ A `swarm.matrix.machine` read parses fine and fails at module-system
# resolution, so this is the kind of mistake only reading the target module
# catches.
matrixMachine = "hive-matrix";
atomicWriteSecret = import ./lib/atomic-write-secret.nix { };
refreshConsumer = import ./lib/refresh-consumer.nix { };
storeRetry = import ./lib/store-retry.nix { };
in
{
config = lib.mkMerge [
# ⚠️ A SEPARATE arm from the unit below, and that separation is the whole
# mechanism: the unit's arm is gated on `haveClientIdentity`, so an
# assertion written inside it could never be reached in the state it
# exists to report.
#
# Shaped after ./swarm-grafana.nix's `haveClientIdentity` assertion — the
# same refusal, named to this principal's own pair. What differs is the
# gate. Grafana asserts wherever Grafana runs, because a Grafana with no
# store identity has no way in at all; a homeserver with no store identity
# is a hive that has no store, which is supported. So this one additionally
# requires `hiveReaderIdentity`: the host demonstrably reads the store, and
# named every option but this pair.
(lib.mkIf (deployCfg.matrix.enable && hiveReaderIdentity) {
assertions = [
{
assertion = haveClientIdentity;
message = ''
This host reads the swarm secret store (services.hyperhive.deploy.bao.clientCertFile
is set) and runs a homeserver, so it needs the matrix appservice
token reader's own client identity: set both
services.hyperhive.deploy.bao.matrixTokenClientCertFile
services.hyperhive.deploy.bao.matrixTokenClientKeyFile
swarm-bao-matrix-token.service fetches this hive's appservice token
out of the store, and without these it is not rendered at all —
leaving the homeserver authenticating hive-c0re against whatever is
already on disk, which is a 401 on every request naming nothing.
⚠️ This reader's OWN leaf, not deploy.bao.clientCertFile. That one is
the hive's, and its grant reads every secret in the store; this role
reads the one appservice-token path. Pointing this option at the
hive's leaf would evaluate, deploy and log in — and undo the split.
On a hive that runs the store, glue-bao-tls.nix supplies both as
defaults and there is nothing to do. Elsewhere the leaf is issued
from that CA out of band and named here — see docs/swarm/secrets.md.
'';
}
];
})
(lib.mkIf (haveClientIdentity && deployCfg.matrix.enable) {
systemd.services.swarm-bao-matrix-token = {
description = "fetch the matrix appservice token from the swarm secret store";
# Every one of these names a unit that exists only where the store runs.
# `Requires=` on an absent unit fails the job outright, so the ordering is
# conditional even though the read is not: off-host there is nothing local
# to wait for, and the timeout below is what bounds the attempt instead.
after = lib.optionals baoDeploy.enable [
"swarm-bao-pki.service"
"container@${baoCfg.machine}.service"
];
wants = lib.optionals baoDeploy.enable [ "container@${baoCfg.machine}.service" ];
requires = lib.optionals baoDeploy.enable [ "swarm-bao-pki.service" ];
before = [ "container@${matrixMachine}.service" ];
wantedBy = [ "container@${matrixMachine}.service" ];
path = [
deployCfg.bao.package
pkgs.coreutils
pkgs.systemd
];
# ./lib/store-retry.nix. This unit is `Before=` the homeserver's
# container, which waits for one attempt and then starts on the token it
# already has; a token a later attempt lands restarts the homeserver
# (below).
inherit (storeRetry) startLimitBurst startLimitIntervalSec;
serviceConfig = storeRetry.serviceConfig // {
Type = "oneshot";
RemainAfterExit = true;
# What actually bounds each attempt below. Stated here rather than
# left to systemd's default, so the number a boot waits on is in
# the file that waits.
TimeoutStartSec = 30;
};
environment = {
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
BAO_CLIENT_CERT = baoDeploy.matrixTokenClientCertFile;
BAO_CLIENT_KEY = baoDeploy.matrixTokenClientKeyFile;
}
# Absent means the system trust store, which is what a deployment with a
# real CA wants and what a self-signed one must not be left with.
// lib.optionalAttrs (baoDeploy.serverCaFile != null) {
BAO_CACERT = baoDeploy.serverCaFile;
};
script = ''
set -euo pipefail
${atomicWriteSecret}
${refreshConsumer}
# A sealed or uninitialised store answers on the port and never
# answers the read, so "the store is up" is not the same as "the
# store can answer". `TimeoutStartSec` above bounds each attempt and
# a timed-out attempt is retried like any other failure; the
# homeserver only `Wants=` this unit, so a failed attempt lets it
# start on the local token rather than failing it.
# `bao`'s own message is the only thing separating a missing value
# from a refused identity from an unreachable host. This unit's
# degraded mode is correct for all three, so it reports which one
# rather than asserting all three in a sentence of ours — a reader
# that cannot say why it read nothing is indistinguishable from a
# broken one.
err="$(mktemp)"
trap 'rm -f "$err"' EXIT
# Cert auth is a login, not a transport setting. The `BAO_CLIENT_*`
# variables above only decide which certificate the TLS handshake
# presents; without a token `bao` asks its token helper instead, and
# that is a `sh` this unit's `path` does not carry. `-token-only`
# answers on stdout and skips the helper on both sides.
#
# Fails LOUDLY, unlike the read below: the three states a login failure
# covers — store not up, sealed, role not written yet — are all things
# a retry fixes, and `Restart=on-failure` above is what retries. Exiting
# 0 here spends the whole boot on a condition that was seconds old.
if ! BAO_TOKEN="$(bao login -method=cert -token-only 2>"$err")"; then
echo "could not log in to swarm-bao with this host's certificate; keeping the token hive-matrix already has." >&2
if [ -s "$err" ]; then
cat "$err" >&2
else
echo "bao failed without writing a diagnostic." >&2
fi
exit 1
fi
export BAO_TOKEN
if ! token="$(bao kv get -field=value ${lib.escapeShellArg tokenPath} 2>"$err")"; then
echo "swarm-bao did not return ${tokenPath}; keeping the token hive-matrix already has." >&2
if [ -s "$err" ]; then
cat "$err" >&2
else
echo "bao failed without writing a diagnostic." >&2
fi
exit 0
fi
if [ -z "$token" ]; then
echo "swarm-bao returned an empty ${tokenPath}; keeping the local token." >&2
exit 0
fi
if secret_differs ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)} "$token"; then
atomic_write_secret 0600 "" ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)} "$token"
fi
# Re-stamp the registration file from the token on disk. The
# token is half an agreement — the registration the homeserver loads
# has to carry the same value — so writing the file and stopping
# would leave the homeserver authenticating hive-c0re against
# whatever activation put there: a 401 on every request, naming
# nothing. Unconditional rather than on-change, because this unit
# has no way to know what the registration currently says.
#
# hive-matrix's own renderer rather than a `printf` here, so the
# registration's shape has one home.
${deployCfg.matrix.appserviceRegistrationScript}
# tuwunel loads the registration through `LoadCredential`, a copy
# taken at start, while hive-c0re reads the token file on every call:
# a changed token splits the two until the homeserver restarts.
refresh_consumer ${lib.escapeShellArg matrixMachine} tuwunel.service ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)}
'';
};
})
];
}

25
nix/host-modules/glue-queue-agent-credential.nix
View file

@ -1,9 +1,8 @@
# Glue: this hive's agent queue credential comes out of the secret store.
#
# The store's second reader, and deliberately the same shape as its first
# (./glue-matrix-bao-token.nix): a cert login that fails LOUDLY because every
# state it fails on is one a retry fixes, then a read that degrades QUIETLY
# because no retry turns "no value there" into a value.
# A cert login that fails LOUDLY because every state it fails on is one a
# retry fixes, then a read that degrades QUIETLY because no retry turns "no
# value there" into a value.
#
# ⚠️ Two files, not one, and that is the consumer's shape rather than a
# preference. `swarm_queue_client::QueueConfig::from_env` takes the secret as
@ -15,15 +14,17 @@
# that is correct rather than degraded. The publisher runs on the authelia
# host and authelia mints on its first boot, so "nothing at that path yet" is
# the ordinary early state of a swarm. Nothing here writes a local stand-in:
# unlike a matrix appservice token there is no such thing as a locally valid
# OIDC client secret, so a placeholder would turn a hive that cannot connect
# there is no such thing as a locally valid OIDC client secret, so a placeholder would turn a hive that cannot connect
# into one that is refused, which reaches the agent as a timeout.
#
# 📌 This runs wherever a client identity is configured, NOT only where the
# store is — the rule ./glue-matrix-bao-token.nix states in full — and
# wherever this host runs a hive (`deploy.hive-controller.enable`): the
# credential is for the agents hive-c0re creates here at runtime, and the unit
# writes under hive-c0re's own state directory.
# store is. `deploy.bao.enable` would have been the co-location assumption
# itself; the reader needs a certificate, not a neighbour. On the store's own
# host ./glue-bao-tls.nix supplies one as a `mkDefault`; elsewhere an operator
# places the leaf and names it, and the same unit works. It also needs this
# host to run a hive (`deploy.hive-controller.enable`): the credential is for
# the agents hive-c0re creates here at runtime, and the unit writes under
# hive-c0re's own state directory.
{
pkgs,
lib,
@ -55,8 +56,8 @@ let
# one thing a remote-store deployment has always had to place by hand. Only
# used to decide whether a missing per-principal leaf is a mistake or a
# deployment that has no store: a host holding neither is the supported
# no-store shape, and one holding this pair but not the pair above named
# seven of the eight options and stopped.
# no-store shape, and one holding this pair but not the pair above stopped
# short of naming this reader's own.
hiveReaderIdentity = baoDeploy.clientCertFile != null && baoDeploy.clientKeyFile != null;
credentialDir = toString deployCfg.hive-controller.queue.agentCredentialDir;

4
nix/host-modules/hive-c0re/default.nix
View file

@ -333,8 +333,8 @@ in
# reachable from an unprivileged daemon without widening either.
# The gate is the identity, never `deploy.bao.enable`: a hive that
# reads a store on another machine holds one of these and runs no
# store. ./glue-matrix-bao-token.nix gates its own reader the same
# way.
# store. ./glue-queue-agent-credential.nix gates its own reader the
# same way.
++ lib.optionals baoClientIdentity [
"bao-client.pem:${baoDeploy.clientCertFile}"
"bao-client-key.pem:${baoDeploy.clientKeyFile}"

2
nix/host-modules/hive-c0re/environment.nix
View file

@ -11,7 +11,7 @@ let
cfg = config.services.hyperhive.c0re;
baoDeploy = config.services.hyperhive.deploy.bao;
baoCfg = config.services.hyperhive.swarm.bao;
# The same test ../glue-matrix-bao-token.nix applies, and for the same
# The same test ../glue-queue-agent-credential.nix applies, and for the same
# reason: a reader is defined by holding a certificate the store accepts,
# not by sharing a host with the store. Deriving this from
# `deploy.bao.enable` would be the co-location assumption itself.

319
nix/host-modules/hive-matrix.nix
View file

@ -50,79 +50,20 @@ let
# visible to the service after sandboxing (ReadWritePaths / ProtectHome),
# typically by placing the file under /etc/tuwunel/" — which this
# container has no writable etc for. `LoadCredential` is the answer
# already in use for the appservice registration below, and for the
# already in use for the swarm appservice registration below, and for the
# same reason: it keeps `DynamicUser=true` + `PrivateUsers=true` intact
# with no host-side chown or GID pinning.
matrixSecretCredential = "/run/credentials/tuwunel.service/oidc_client_secret";
# The hive's appservice registration, whose `url` is null. Null is a legal
# `url` (ruma's `Registration` types it `Option<String>`), and it is the
# whole point — with no URL the homeserver
# never calls out, so there is no HTTP service to run and no daemon to
# operate. tuwunel loads the registration and creates its sender account
# (below). No client presents its `as_token`: `swarm-controller` mints the
# sender's token with the swarm registration's.
appserviceId = "hyperhive";
# The appservice's own user, and the account the hive acts as. An
# ordinary user, not a homeserver admin: everything the hive does with
# it — the Space, the chat room, the invites — it does as the creator of
# those rooms. Loading a registration CREATES its `sender_localpart` user
# when absent (tuwunel `src/service/appservice/mod.rs`), on a zero-user
# database, inside `Services::start()` — before the HTTP listener accepts
# anything, so the account exists on the very first boot of a fresh
# homeserver without hive-c0re racing to register it.
# Where tuwunel reads appservice registrations from: its own credentials
# directory. A registration is a 0600 root-owned file and the homeserver
# runs under `DynamicUser=true`, so `LoadCredential` copies it to a 0400
# dynamic-user-owned path as root, before the sandbox and that user exist.
#
# 🩸 **Derived from the hive name, so it is one account per hive.** It was
# the bare `hive` and a swarm runs one homeserver: every hive on it acted
# as the same `@hive:`, which is one identity with no attribution and no
# way to revoke a single hive. The hives that do not run this container
# register their own `hive-<name>` through the appservice namespace below,
# which already admits it.
#
# ⚠️ Must equal `swarm_secret_client::matrix::hive_localpart`, which
# hive-c0re and swarm-controller both derive from independently with
# nothing wiring an override across — same agreement, and same reason for
# saying so, as the token path below.
#
# `hiveName` has no fallback here for the reason ./glue-matrix-bao-token.nix
# gives at its own use of it: it is asserted set on every host that runs the
# homeserver.
hiveLocalpart = "hive-${toString config.services.hyperhive.hiveName}";
# The `as_token`, and the `hs_token` the spec requires alongside it. Both
# minted by the render script below, mode 0600; the `as_token` is the one
# the swarm secret store overwrites (see
# `glue-matrix-bao-token.nix`). The `hs_token` authenticates the homeserver
# TO the appservice, which with `url = null` is nobody — it exists because
# the registration format requires it.
#
# `appserviceTokenPath` is the single source for the option's `default`,
# its `config`-level `mkDefault` self-definition, and the assertion that
# rejects a moved path. See `appserviceTokenFile`'s own comment below.
appserviceTokenPath = "/var/lib/hyperhive/matrix-appservice-token";
appserviceHsTokenPath = "/var/lib/hyperhive/matrix-appservice-hs-token";
# The registration file, and the directory that holds it. A directory
# rather than a `[global.appservice.<id>]` stanza because a stanza's
# `as_token` would be a nix literal, and a nix literal is a world-readable
# store path — the same rule that sends the OIDC client secret through a
# runtime file. `docs/swarm/secrets.md` has the general form of it.
appserviceDir = "/var/lib/hyperhive/matrix-appservice";
appserviceRegistrationPath = "${appserviceDir}/${appserviceId}.yaml";
# ⚠️ tuwunel does not read the host path directly, for exactly the reason
# given for the two secrets above: the file is 0600 root-owned and the
# homeserver runs under `DynamicUser=true`. `LoadCredential` copies it to a
# 0400 dynamic-user-owned path as root, before the sandbox and that user
# exist.
#
# So `appservice_dir` points at the credentials directory ITSELF. That is
# safe rather than clever: tuwunel's loader takes only `.yaml`/`.yml`
# Safe rather than clever: tuwunel's loader takes only `.yaml`/`.yml`
# entries from the directory and skips every other file, so the sibling
# credentials (the OIDC secret) are invisible to it. A credential id is a
# free-form filename, which is what lets ours end in `.yaml`.
appserviceCredentialId = "${appserviceId}-appservice.yaml";
# free-form filename, which is what lets a registration's end in `.yaml`.
appserviceCredentialDir = "/run/credentials/tuwunel.service";
# ── swarm-matrix-ctl ────────────────────────────────────────────────
@ -147,9 +88,8 @@ let
# derived rather than assumed, because an operator naming paths by hand is
# exactly the deployment the gate above is written for.
#
# Directories rather than the files, for the reason `appserviceDir`'s own
# comment gives: a re-issued leaf is a new inode, and binding the file would
# pin the one the container saw when it started.
# Directories rather than the files: a re-issued leaf is a new inode, and
# binding the file would pin the one the container saw when it started.
ctlPkiDirs = lib.optionals ctlActive (
lib.unique (
map builtins.dirOf (
@ -169,9 +109,9 @@ let
# ── the swarm's own appservice ──────────────────────────────────────
#
# A second registration beside the hive's `hyperhive` one, and the swarm's
# identity on this homeserver: `swarm-controller` creates every agent's
# account with its token. Minted INSIDE this container by
# The homeserver's one appservice registration, and the swarm's identity on
# it: `swarm-controller` creates every agent's account and every hive's
# `@hive-<hive>` sender with its token. Minted INSIDE this container by
# `swarm-matrix-ctl appservice render` and published to the store by
# `appservice publish`, which is why it is gated on matrix-ctl's store
# identity: with nobody to publish it, a registration here would be an
@ -187,8 +127,8 @@ let
swarmAppserviceDir = "/var/lib/swarm-matrix-appservice";
swarmAppserviceCredentialId = "swarm-appservice.yaml";
# Every local user this hive may provision — agents and `@hive-<hive>:`
# itself — which is the whole matrix localpart charset.
# Every local user the swarm appservice may create — agents and every
# hive's `@hive-<hive>:` sender — which is the whole matrix localpart charset.
#
# ⚠️ Anchored deliberately: tuwunel compiles a namespace into a `RegexSet`
# and asks it for a MATCH, not a full match, so an unanchored
@ -201,73 +141,6 @@ let
# create these names, not a monopoly on them.
appserviceUserRegex = "^@[a-z0-9._=/-]+:${lib.escapeRegex effectiveServerName}$";
# Mint the tokens if they are absent, then render the registration from
# whatever they now hold. **One script with two callers** — the activation
# script below and `glue-matrix-bao-token.nix`, which overwrites the
# `as_token` with the swarm store's copy and has to re-render afterwards.
#
# ⚠️ The `as_token` mint here is the FIRST-BOOT FALLBACK, not the normal
# route. `swarm-secret-publisher.nix` mints one per hive swarm-side and
# publishes it to the store; the reader above overwrites this file before
# the homeserver's container starts, so what a hive actually runs on is the
# swarm's value. This keeps a hive that has never reached the store working
# — and it fires only when the file is absent, so it never clobbers one the
# store delivered. (`hs_token` has no swarm half: it is how the homeserver
# authenticates to *us*, so it stays hive-local and is minted here for real.)
# A second copy of the registration's shape would be a second source of
# truth for a format whose mismatch is silent: the homeserver would load a
# registration naming a token nobody holds, and every request hive-c0re
# makes would come back 401 naming nothing.
#
# Re-rendering unconditionally is the point rather than thoroughness: "the
# token file exists" does not mean "the registration carries what is in
# it".
appserviceRegistrationScript = pkgs.writeShellApplication {
name = "hive-matrix-appservice-registration";
runtimeInputs = [ pkgs.coreutils ];
text = ''
# Both the tokens and the rendered registration are secrets; 077
# covers every file this script creates rather than each one
# separately.
umask 077
mkdir -p ${lib.escapeShellArg appserviceDir}
for f in ${lib.escapeShellArg appserviceTokenPath} ${lib.escapeShellArg appserviceHsTokenPath}; do
if [ ! -s "$f" ]; then
head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$f"
echo >> "$f"
echo "hive-matrix: generated appservice token at $f"
fi
chmod 0600 "$f"
done
# Read into shell variables and emitted with `printf`, a shell
# builtin: a token passed as an argument to a real command would land
# in that process's argv, which is world-readable for its lifetime.
asToken="$(cat ${lib.escapeShellArg appserviceTokenPath})"
hsToken="$(cat ${lib.escapeShellArg appserviceHsTokenPath})"
# The quoted heredoc keeps the regex's own `$` and `\` out of the
# shell's hands; the YAML single quotes keep them out of YAML's.
{
cat <<'REGISTRATION'
id: ${appserviceId}
url: null
sender_localpart: ${hiveLocalpart}
rate_limited: false
namespaces:
users:
- exclusive: false
regex: '${appserviceUserRegex}'
aliases: []
rooms: []
REGISTRATION
printf 'as_token: %s\nhs_token: %s\n' "$asToken" "$hsToken"
} > ${lib.escapeShellArg appserviceRegistrationPath}
chmod 0600 ${lib.escapeShellArg appserviceRegistrationPath}
chmod 0700 ${lib.escapeShellArg appserviceDir}
'';
};
# Format-locked by tuwunel, not chosen here: the callback host must point
# directly at the matrix server and the path is fixed at
# `/_matrix/client/unstable/login/sso/callback/<client_id>`. Built once
@ -482,61 +355,6 @@ in
'';
};
appserviceTokenFile = lib.mkOption {
type = lib.types.path;
internal = true;
default = appserviceTokenPath;
description = ''
Host path to a file containing this hive's matrix appservice
token (`as_token`). Minted automatically on first activation
(32-byte random hex, mode 0600) and rendered into the
appservice registration the homeserver loads at boot. Agents
never see it; an agent only ever receives its own
`access_token`.
Not operator-settable — the module's registration renderer reads
this same path as a literal, not through the option,
with nothing wiring an override across, so a moved path desyncs
the two silently. An externally-managed token is delivered by
writing into *this* fixed path instead of moving it — see
`glue-matrix-bao-token.nix`, which fetches from the swarm secret
store, overwrites this file in place, and re-renders the
registration that names it.
Enforced by an `assertions` entry below rather than `readOnly`:
the `config` block gives this option its own `mkDefault`
definition (lowest priority) so a real override still resolves
cleanly instead of crashing eval with nixpkgs' generic
"read-only, set multiple times" message; the assertion then
names the actual problem. (`readOnly` was the first attempt on
this option's predecessor — atlas caught that it only rejects a
*second* definition, so with nothing else defining the option a
lone override sailed through silently, same desync as before
with a lock that wasn't locked. Fixed by defining the value here
instead of leaving it on `default` alone, but a proper
`assertions` message beat re-adding `readOnly` on top once the
option had a real definition either way.)
'';
};
appserviceRegistrationScript = lib.mkOption {
type = lib.types.path;
internal = true;
default = "${appserviceRegistrationScript}/bin/hive-matrix-appservice-registration";
defaultText = lib.literalMD "the module's own registration renderer";
description = ''
The script that mints the appservice tokens when absent and
(re-)renders the registration file from them. An option only so
that `glue-matrix-bao-token.nix` can run the same one after
overwriting the token with the swarm store's copy, rather than
carrying a second copy of the registration's shape.
Not operator-settable, and not a hook: replacing it means
deciding what the homeserver's appservice registration says,
which is this module's job.
'';
};
gui.package = lib.mkOption {
type = lib.types.package;
default = fluffychat-web-fixed;
@ -629,13 +447,6 @@ in
};
config = lib.mkIf deployCfg.matrix.enable {
# The option's own value, defined explicitly rather than left on its
# bare `default` — `mkDefault` so a real override still resolves
# cleanly rather than crashing eval; the `assertions` entry below is
# what actually rejects it, with a message naming the reason. See that
# option's own comment above.
services.hyperhive.deploy.matrix.appserviceTokenFile = lib.mkDefault appserviceTokenPath;
# Matrix's own gateway surface: the sub-domain vhost, the name the
# hive resolver answers for, and the Accept-header map that vhost's
# SPA fallback reads. All three are matrix knowledge and none of
@ -769,25 +580,6 @@ in
httpPort and has no such name.
'';
}
{
# mkDefault above lets an override resolve instead of crashing
# eval — this is the actual rejection, with a message that names
# the real fix instead of nixpkgs' generic conflicting-definition
# text.
assertion = deployCfg.matrix.appserviceTokenFile == appserviceTokenPath;
message = ''
services.hyperhive.deploy.matrix.appserviceTokenFile is fixed at
${appserviceTokenPath} and cannot be moved — the registration
renderer reads this same path as a literal and has no way to learn
an override, so moving it desyncs the two silently instead of
loudly.
Integrating an externally-managed appservice token? Deliver it
by writing into ${appserviceTokenPath} instead of pointing this
option elsewhere — see glue-matrix-bao-token.nix, which does
exactly that from the swarm secret store.
'';
}
];
# One declaration, two readers. The homeserver knows its own callback
@ -829,11 +621,10 @@ in
# permanent stall presenting as "matrix is broken", several layers from
# its cause.
#
# The appservice registration dodges that with an activation script
# that renders the file first. ⚠️ That dodge is NOT available here:
# tuwunel requires the secret file to exist *and be non-empty*, so a
# zero-byte placeholder would satisfy the bind mount and then stop the
# homeserver from starting.
# ⚠️ Rendering a placeholder first is no way around it: tuwunel
# requires the secret file to exist *and be non-empty*, so a zero-byte
# placeholder would satisfy the bind mount and then stop the homeserver
# from starting.
systemd.services.hive-matrix-oidc-secret = lib.mkIf ssoLocal {
description = "deliver the homeserver's OIDC client secret from authelia";
after = [ "container@${autheliaCfg.machine}.service" ];
@ -938,23 +729,6 @@ in
''
);
# Mint the appservice tokens and render the registration, before any
# container start.
#
# An activation script rather than a unit: the directory below is
# bind-mounted into the container and nixos-container refuses to start
# when a bind source is missing, so this has to have run first.
# Activation is what runs before the container both on a switch and on
# every boot. (The registration token this replaced used an activation
# script for the same reason, and additionally to dodge nspawn creating
# an empty file at a missing bind target — which tuwunel then read as
# "no token", refusing every registration until the next restart. A
# missing registration file is not silent in that way: the homeserver
# fails its appservice load loudly.)
system.activationScripts.hive-matrix-appservice = lib.stringAfter [ "var" ] ''
${deployCfg.matrix.appserviceRegistrationScript}
'';
containers.hive-matrix = {
autoStart = true;
ephemeral = false;
@ -962,23 +736,12 @@ in
# --link-journal=try-guest, and EXTRA_NSPAWN_FLAGS expands after it.
extraFlags = [ "--link-journal=host" ];
inherit privateNetwork;
# Read-only bind of the host-managed appservice registration; tuwunel
# reads it via systemd LoadCredential below (not directly).
#
# The directory, not the file inside it: the registration is
# re-rendered rather than edited, and binding the file would pin the
# inode the container saw when it started.
bindMounts = {
${appserviceDir} = {
hostPath = appserviceDir;
isReadOnly = true;
};
}
// caTrust.bindMount
# matrix-ctl's bao client identity, and nothing else of the hive's. See
# `ctlPkiDirs` above for why it is a derived set of directories
# rather than one literal.
// ctlBindMounts;
bindMounts =
caTrust.bindMount
# matrix-ctl's bao client identity, and nothing else of the hive's. See
# `ctlPkiDirs` above for why it is a derived set of directories
# rather than one literal.
// ctlBindMounts;
config =
{ ... }:
{
@ -1061,10 +824,10 @@ in
allow_federation = true;
trusted_servers = deployCfg.matrix.trustedServers;
# Nobody registers themselves here. Accounts are created by
# the hive's appservice, which this flag does not gate —
# the swarm's appservice, which this flag does not gate —
# tuwunel checks it only for requests that arrive WITHOUT an
# appservice token, so hive-c0re provisions exactly as before
# and everyone else is refused outright.
# appservice token, so swarm-controller provisions and
# everyone else is refused outright.
#
# ⚠️ Not a hardening afterthought: `allow_registration = true`
# with no registration token configured makes tuwunel REFUSE
@ -1074,7 +837,7 @@ in
# homeserver, it is a homeserver that does not boot.
allow_registration = false;
# Where the hive's appservice registration is read from — the
# Where the swarm's appservice registration is read from — the
# credentials directory, for the reasons at
# `appserviceCredentialDir`'s own definition. Loaded inside
# `Services::start()`, before the listener accepts anything.
@ -1083,8 +846,8 @@ in
}
# Exactly one account is promoted to homeserver admin at boot:
# the swarm appservice's sender. Its token is read by
# matrix-ctl and swarm-controller only. `@${hiveLocalpart}`
# stays an ordinary user, because every hive reads its
# matrix-ctl and swarm-controller only. Each `@hive-<hive>`
# stays an ordinary user, because its hive reads its
# credential; what it does (the Space, the chat room, the
# invites) rides on being the rooms' creator, not on admin.
#
@ -1161,20 +924,15 @@ in
# host-side chown :tuwunel / GID-pin gymnastics needed.
# See `man systemd.exec` → LoadCredential.
systemd.services.tuwunel.serviceConfig.LoadCredential = [
# The credential id carries a `.yaml` suffix on purpose:
# `appservice_dir` above names this very directory, and tuwunel
# takes only `.yaml`/`.yml` entries from it.
"${appserviceCredentialId}:${appserviceRegistrationPath}"
# Same mechanism, second secret. tuwunel re-reads this file on
# every OAuth exchange, not just at startup, so it has to
# tuwunel re-reads this file on every OAuth exchange, not just at startup, so it has to
# outlive the unit's start — a credentials path does.
"oidc_client_secret:${toString deployCfg.matrix.sso.clientSecretFile}"
]
# The swarm registration, a second `.yaml` in the same directory:
# tuwunel loads every one it finds (`appservice/mod.rs`), and
# refuses only a duplicate `id` or `as_token`, never an
# overlapping namespace. A missing source fails this unit, which
# is why the render below is `requiredBy` it and local only.
# The swarm registration. The credential id carries a `.yaml`
# suffix on purpose: `appservice_dir` above names this very
# directory, and tuwunel takes only `.yaml`/`.yml` entries from
# it. A missing source fails this unit, which is why the render
# below is `requiredBy` it and local only.
++ lib.optional ctlActive "${swarmAppserviceCredentialId}:${swarmAppserviceDir}/swarm.yaml";
# The swarm registration, minted and rendered before the homeserver
@ -1196,9 +954,8 @@ in
environment = {
MATRIX_APPSERVICE_DIR = swarmAppserviceDir;
MATRIX_APPSERVICE_SENDER = swarmSenderLocalpart;
# The hive registration's namespace. Agents' localparts are
# bare `[a-z0-9-]`, so nothing narrower covers them without
# also covering people.
# Agents' localparts are bare `[a-z0-9-]`, so nothing narrower
# covers them without also covering people.
MATRIX_APPSERVICE_USER_REGEX = appserviceUserRegex;
};
};

2
nix/host-modules/hyperhive.nix
View file

@ -156,7 +156,7 @@ in
# Each of these deployments turns `hiveName` into an identifier with no
# fallback: hive-c0re's OIDC client and matrix localpart, the CN of the
# store's `client.pem` (./glue-bao-tls.nix), and the `hives/<name>` path
# the matrix token and queue credential readers fetch. A null there renders
# the queue credential reader fetches. A null there renders
# as an empty string that evaluates and deploys.
config.assertions = [
{

165
nix/host-modules/swarm-bao.nix
View file

@ -755,32 +755,31 @@ let
}
'';
# ── the four principals that used to share the hive's own leaf ─────────────
# ── the principals that used to share the hive's own leaf ──────────────────
#
# 🩸 Each of the four reads exactly ONE path in the store, and until this
# split each did it holding `deploy.bao.clientCertFile` — the hive's own leaf,
# 🩸 Each of these reads exactly ONE path in the store, and until this split
# each did it holding `deploy.bao.clientCertFile` — the hive's own leaf,
# whose policy (`swarm-secret-client`'s `policy::render`) granted read on
# `swarm/agents/*`, `swarm/hives/<hive>/*` AND `swarm/services/*`. Four
# principals presenting one certificate are ONE principal to bao, so the union
# of what the four need was the only grant expressible: the unit that fetches
# Grafana's OIDC client secret could fetch every agent credential in the
# swarm, and the unit that fetches this hive's matrix token could fetch
# Grafana's. Least privilege was not misconfigured, it was unrepresentable.
# `swarm/agents/*`, `swarm/hives/<hive>/*` AND `swarm/services/*`. Principals
# presenting one certificate are ONE principal to bao, so the union of what
# they need was the only grant expressible: the unit that fetches Grafana's
# OIDC client secret could fetch every agent credential in the swarm, and the
# unit that fetches this hive's queue credential could fetch Grafana's. Least
# privilege was not misconfigured, it was unrepresentable.
#
# Each now gets its own leaf, its own cert-auth role and a policy holding the
# one `secret/data/…` path that principal's script actually names — spelled to
# the LEAF rather than to a prefix, for the reason `matrixCtlPolicyText` above
# gives for doing the same.
#
# ⚠️ Two of the four are PER HIVE and two are not, and that asymmetry is the
# shape of the paths rather than a preference. A matrix appservice token and a
# queue credential live under `swarm/hives/<name>/`, and every hive runs a
# ⚠️ The queue-credential reader is PER HIVE and the OIDC readers are not,
# and that asymmetry is the shape of the paths rather than a preference. A
# queue credential lives under `swarm/hives/<name>/`, and every hive runs a
# reader for its own; an OIDC client secret lives under
# `swarm/services/<client-id>/` and a swarm registers each of those exactly
# once. A single role for a per-hive reader would have to name `hives/*` to
# serve every hive — which would let one hive read another's matrix token, a
# reach no hive has today and one this change exists to remove rather than
# create.
# serve every hive — which would let one hive read another's queue
# credential.
readStanza = path: ''
path "${path}" {
capabilities = ["read"]
@ -799,12 +798,6 @@ let
policyText = readStanza (spec.path hiveName);
}) hyperhiveCfg.swarm.hives;
matrixTokenReaders = perHiveReaders {
rolePrefix = "swarm-matrix-token";
cnPrefix = baoDeploy.matrixTokenCommonNamePrefix;
path = h: "${credentialMountPath}/data/swarm/hives/${h}/matrix/appservice-token";
};
queueAgentReaders = perHiveReaders {
rolePrefix = "swarm-queue-agent";
cnPrefix = baoDeploy.queueAgentCommonNamePrefix;
@ -812,7 +805,7 @@ let
};
# Singletons, so a list of one rather than a second shape: the granting unit
# below takes a list either way and the four units stay identical apart from
# below takes a list either way and the units stay identical apart from
# the objects they write.
grafanaOidcReaders = [
{
@ -830,8 +823,7 @@ let
}
];
# The fifth, and the one the four-way split left on the hive's leaf: the
# store's own forwarder. Its path is `forwarderStoreSecretPath` below with
# The store's own forwarder. Its path is `forwarderStoreSecretPath` below with
# KV v2's `data/` inserted.
forwarderOidcReaders = [
{
@ -876,10 +868,10 @@ let
display_name=${lib.escapeShellArg obj.cn}
'';
# One unit per PRINCIPAL, not one unit for the four: these names are
# One unit per PRINCIPAL, not one unit for all of them: these names are
# operator-facing strings, the same reason `swarm-bao-secret-publisher-policy`
# above gives for not being more script inside the controller's unit. A hive
# whose matrix token never arrives is diagnosed by the state of the unit named
# whose queue credential never arrives is diagnosed by the state of the unit named
# after it.
#
# `after` and not `requires`, for the reason the publisher's unit states: the
@ -1224,7 +1216,22 @@ in
and this option named a write grant matrix-ctl no longer holds. Delete
this setting.
'')
];
]
++
map
(
option:
lib.mkRemovedOptionModule [ "services" "hyperhive" "deploy" "bao" option ] ''
The per-hive matrix appservice token and the unit that fetched it are
gone: swarm-controller creates every hive's and every agent's matrix
account with the swarm's appservice token. Delete this setting.
''
)
[
"matrixTokenClientCertFile"
"matrixTokenClientKeyFile"
"matrixTokenCommonNamePrefix"
];
# One service, two namespaces, and the split decides who may set what.
#
@ -1236,7 +1243,8 @@ in
# `serverCaFile` live here too, and they are set by whoever authenticates TO
# the store — which includes a host that runs none of it. A reader is defined
# by holding a certificate the store accepts, never by sharing a host with
# one; ./glue-matrix-bao-token.nix gates on exactly that and not on `enable`.
# one; ./glue-queue-agent-credential.nix gates on exactly that and not on
# `enable`.
# Reading this block as store-runner-only is what makes an off-host reader
# look inexpressible when it is already supported.
#
@ -1629,27 +1637,6 @@ in
'';
};
matrixTokenCommonNamePrefix = lib.mkOption {
type = lib.types.str;
default = "swarm-bao-matrix-token";
example = "swarm-bao-matrix-token.svc";
description = ''
Prefix of the subject the store's per-hive matrix-token roles accept.
The role for hive `h` accepts `<prefix>-h` and nothing else, and grants
read on exactly `swarm/hives/h/matrix/appservice-token`.
A **prefix** rather than one subject because every hive runs its own
reader and the path it reads carries that hive's name. One subject for
all of them would need a `hives/*` grant, which would let one hive read
another hive's appservice token — a reach no hive has today.
⚠️ Reserved as a hive name by ./swarm.nix, per hive, for the reason its
fixed-subject siblings are: cert auth trusts the CA and
`allowed_common_names` is the whole narrowing, so a hive whose own leaf
carried one of these spellings would receive that grant.
'';
};
queueAgentCommonNamePrefix = lib.mkOption {
type = lib.types.str;
default = "swarm-bao-queue-agent";
@ -1660,10 +1647,15 @@ in
exactly `swarm/hives/h/queue/agent` — the client credential that hive's
agent containers authenticate to the swarm queue with.
Per hive for the same reason
{option}`services.hyperhive.deploy.bao.matrixTokenCommonNamePrefix` is,
and more sharply: every hive in the swarm runs this reader, so a single
subject would have to be granted `hives/*`.
A **prefix** rather than one subject because every hive runs its own
reader and the path it reads carries that hive's name. One subject for
all of them would need a `hives/*` grant, which would let one hive read
another hive's queue credential.
⚠️ Reserved as a hive name by ./swarm.nix, per hive, for the reason its
fixed-subject siblings are: cert auth trusts the CA and
`allowed_common_names` is the whole narrowing, so a hive whose own leaf
carried one of these spellings would receive that grant.
'';
};
@ -1678,7 +1670,7 @@ in
for `h`'s domain and `*.` that domain and nothing else.
Per hive for the reason
{option}`services.hyperhive.deploy.bao.matrixTokenCommonNamePrefix` is:
{option}`services.hyperhive.deploy.bao.queueAgentCommonNamePrefix` is:
one subject for every hive could ask for any hive's certificate.
⚠️ Reserved as a hive name by ./swarm.nix, per hive, like its siblings.
@ -1725,7 +1717,7 @@ in
presents. Its grant is one path,
`swarm/services/<collector client id>/oidc/client`, and read only.
A **fourth** identity rather than reuse of
Its **own** identity rather than reuse of
{option}`services.hyperhive.deploy.bao.grafanaOidcCommonName`: the two
read different services' client secrets, and a collector is not
entitled to Grafana's.
@ -1742,46 +1734,13 @@ in
own forwarder's OIDC client secret, presents. Its grant is one path,
`swarm/services/<forwarder client id>/oidc/client`, and read only.
A **fifth** identity rather than reuse of
Its **own** identity rather than reuse of
{option}`services.hyperhive.deploy.bao.otelOidcCommonName`: the two
read different clients' secrets, and the store's forwarder is not
entitled to the swarm collector's.
'';
};
matrixTokenClientCertFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-bao-pki/matrix-token.pem";
description = ''
Certificate the unit that fetches this hive's matrix appservice token
presents to the store. Its subject must be
`<matrixTokenCommonNamePrefix>-<this hive's name>`; cert auth matches on
the CN and the role accepts nothing else.
⚠️ **Not** {option}`services.hyperhive.deploy.bao.clientCertFile`.
Pointing this at the hive's own leaf would evaluate, deploy and log in —
and hand a unit that reads one token a credential that reads every
secret in the store. That is what this option exists to end.
No default here: ./glue-bao-tls.nix points it at the leaf it mints,
wherever this host mints its own PKI. A deployment that reads a store on
another host names it by hand, alongside the leaf itself — one file per
principal placed out of band instead of one shared by four.
'';
};
matrixTokenClientKeyFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-bao-pki/matrix-token-key.pem";
description = ''
Private key for
{option}`services.hyperhive.deploy.bao.matrixTokenClientCertFile`. Both
or neither — a certificate with no key authenticates nothing.
'';
};
queueAgentClientCertFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
@ -1789,10 +1748,17 @@ in
description = ''
Certificate the unit that fetches this hive's queue client credential
presents to the store. Its subject must be
`<queueAgentCommonNamePrefix>-<this hive's name>`.
`<queueAgentCommonNamePrefix>-<this hive's name>`; cert auth matches on
the CN and the role accepts nothing else.
⚠️ Not the hive's own leaf, for the reason
{option}`services.hyperhive.deploy.bao.matrixTokenClientCertFile` gives.
⚠️ **Not** {option}`services.hyperhive.deploy.bao.clientCertFile`.
Pointing this at the hive's own leaf would evaluate, deploy and log in —
and hand a unit that reads one credential a certificate that reads every
secret in the store.
No default here: ./glue-bao-tls.nix points it at the leaf it mints,
wherever this host mints its own PKI. A deployment that reads a store on
another host names it by hand, alongside the leaf itself.
'';
};
@ -1816,7 +1782,7 @@ in
{option}`services.hyperhive.deploy.bao.grafanaOidcCommonName`.
⚠️ Not the hive's own leaf, for the reason
{option}`services.hyperhive.deploy.bao.matrixTokenClientCertFile` gives.
{option}`services.hyperhive.deploy.bao.queueAgentClientCertFile` gives.
'';
};
@ -1840,7 +1806,7 @@ in
{option}`services.hyperhive.deploy.bao.otelOidcCommonName`.
⚠️ Not the hive's own leaf, for the reason
{option}`services.hyperhive.deploy.bao.matrixTokenClientCertFile` gives.
{option}`services.hyperhive.deploy.bao.queueAgentClientCertFile` gives.
'';
};
@ -1864,7 +1830,7 @@ in
{option}`services.hyperhive.deploy.bao.forwarderOidcCommonName`.
⚠️ Not the hive's own leaf, for the reason
{option}`services.hyperhive.deploy.bao.matrixTokenClientCertFile` gives.
{option}`services.hyperhive.deploy.bao.queueAgentClientCertFile` gives.
'';
};
@ -2924,13 +2890,11 @@ in
'';
};
# 🩸 The four principals that used to log in as the hive. See the block
# around `perHiveReaders` for what each grant is and why the first two are
# per hive; the four units are identical apart from the objects they
# 🩸 The principals that used to log in as the hive. See the block around
# `perHiveReaders` for what each grant is and why the first is per hive;
# the units are identical apart from the objects they
# write, so the shape lives in `readerPolicyUnit` and the difference is
# visible here in one line each.
systemd.services.swarm-bao-matrix-token-policy = readerPolicyUnit "write the per-hive matrix-token bao policies and cert-auth roles" matrixTokenReaders;
systemd.services.swarm-bao-queue-agent-policy = readerPolicyUnit "write the per-hive queue-credential bao policies and cert-auth roles" queueAgentReaders;
systemd.services.swarm-bao-grafana-oidc-policy = readerPolicyUnit "write Grafana's OIDC-secret-reader bao policy and cert-auth role" grafanaOidcReaders;
@ -3139,7 +3103,6 @@ in
"swarm-bao-controller-policy.service"
"swarm-bao-secret-publisher-policy.service"
"swarm-bao-matrix-ctl-policy.service"
"swarm-bao-matrix-token-policy.service"
"swarm-bao-queue-agent-policy.service"
"swarm-bao-grafana-oidc-policy.service"
"swarm-bao-otel-oidc-policy.service"

2
nix/host-modules/swarm-grafana.nix
View file

@ -206,7 +206,7 @@ let
# A reader of the store is defined by holding a certificate the store
# accepts, never by standing next to it — the rule
# ./glue-matrix-bao-token.nix states in full.
# ./glue-queue-agent-credential.nix states in full.
#
# Also asserted rather than gating anything, and that is the ruling this
# module's second delivery route was deleted under: the store exists so that

5
nix/host-modules/swarm-otel.nix
View file

@ -241,11 +241,10 @@ let
# A reader of the store is defined by holding a certificate the store
# accepts, never by standing next to it — the rule
# ./glue-matrix-bao-token.nix states in full. Unlike Grafana's identical-
# ./glue-queue-agent-credential.nix states in full. Unlike Grafana's identical-
# looking flag, this one does not gate an assertion on its own: the
# store-reading unit below simply does not render without it, the same choice
# ./glue-matrix-bao-token.nix and ./glue-queue-agent-credential.nix make for
# their own readers, because a collector with no client identity is
# ./glue-queue-agent-credential.nix makes for its own reader, because a collector with no client identity is
# `haveCollectorSecret = false` above, and that is already a supported,
# merely degraded shape rather than a service with no way in at all. What
# IS asserted is narrower and lives in `assertions` below — see

82
nix/host-modules/swarm-secret-publisher.nix
View file

@ -1,7 +1,7 @@
# The unit that puts swarm-level secrets into the swarm's secret store, so
# whoever needs one can read it there: a hive its agents' credential, a
# service or the controller its own, a hive its appservice token. OIDC secrets
# it only copies; the appservice token it MINTS, having no swarm-side producer.
# service or the controller its own. It mints nothing: every value is an OIDC
# client secret authelia minted, and this unit copies it.
#
# ⚠️ "Whoever", including a reader on THIS host. A swarm service's secret goes
# into the store even when the service runs beside authelia, because its
@ -21,13 +21,11 @@
# ⚠️ ITS OWN STORE IDENTITY, not swarm-controller's. That principal may rewrite
# every hive's policy and login role; a unit that writes a handful of secrets
# has no business holding it. ./swarm-bao.nix grants this one `create`/`update`
# under the hive prefix and nothing else — no `read`, which is why the mint
# below keeps its own record of what it published.
# under the hive prefix and nothing else — no `read`.
#
# ⚠️ THE SECRET NEVER REACHES `argv`. `bao` is an external binary, so every
# argument is world-readable in /proc for the life of the call — the value is
# passed as `@<path>` and read by bao itself. Same rule governs the mint: it
# redirects into a file and never holds the token in a shell variable.
# passed as `@<path>` and read by bao itself.
{
pkgs,
lib,
@ -41,7 +39,8 @@ let
cfg = deployCfg.swarm-secret-publisher;
# A reader is defined by holding a certificate the store accepts, never by
# standing next to it — the rule ./glue-matrix-bao-token.nix states in full.
# standing next to it — the rule ./glue-queue-agent-credential.nix states in
# full.
# `deploy.bao.enable` here would be the co-location assumption itself.
haveClientIdentity = cfg.baoClientCertFile != null && cfg.baoClientKeyFile != null;
@ -86,21 +85,6 @@ let
# under its own certificate.
controllerClientId = hyperhiveCfg.swarm.controller.queueClientId;
controllerSecretPath = "secret/swarm/controller/swarm-controller/oidc/client";
# Where this unit keeps the appservice tokens it minted, and the whole reason
# a re-publish is idempotent. The store cannot be that record: ./swarm-bao.nix
# grants this principal `create`/`update` and deliberately no `read`, so "does
# this hive already have a token?" is a question it can only ask locally.
# Without the file every run would mint a fresh value and rotate the swarm's
# token for no reason — and a rotation is not free, because the homeserver
# only reloads the registration naming it when its container restarts.
#
# `StateDirectory=` below is what creates it, so the name there is relative
# and this unit owns the tree. Losing it mints once more and republishes;
# readers pick that up on their next start, and nothing that already
# registered breaks — the token authenticates the appservice, it does not
# live inside any account it created.
matrixTokenDir = "/var/lib/swarm-secret-publish/matrix-appservice-token";
in
{
options.services.hyperhive.deploy.swarm-secret-publisher = {
@ -112,9 +96,7 @@ in
Publish the OIDC client secrets this host mints into the swarm's
secret store, so a hive that does not run authelia can read its
agents' credential and a swarm service or the controller can read its
own — from wherever it runs, this host included. Also mints each hive's matrix
appservice token, which has no other swarm-side producer, and
publishes it the same way.
own — from wherever it runs, this host included.
Defaults to whether this host mints the OIDC secrets, which is the
only half of the question that is a property of *this* host.
@ -128,9 +110,8 @@ in
Turning it off leaves every hive but this one without its agents'
credential, the swarm controller unable to start, the swarm's Grafana
without any login at all, its collector pushing unauthenticated, and
every hive falling back to the appservice token its own first boot
minted — so two hives never agree on one. Each secret has exactly one route and this is the producer's
without any login at all, and its collector pushing unauthenticated.
Each secret has exactly one route and this is the producer's
end of it, so the honest reason to set it false is a deployment
delivering those secrets by some other mechanism it owns.
'';
@ -190,12 +171,6 @@ in
TimeoutStartSec = 60;
Restart = "on-failure";
RestartSec = 30;
# The minted appservice tokens, see `matrixTokenDir` above. Named
# relative to /var/lib because that is the form systemd creates; the
# mode is narrowed from systemd's 0755 default because the directory
# listing is the roster of hives and every file in it is a secret.
StateDirectory = "swarm-secret-publish/matrix-appservice-token";
StateDirectoryMode = "0700";
};
environment = {
BAO_ADDR = "https://${hyperhiveCfg.swarm.bao.domain}:${toString hyperhiveCfg.swarm.bao.port}";
@ -222,7 +197,6 @@ in
published=0
skipped=0
tokens=0
${lib.concatMapStringsSep "\n" (hive: ''
src=${lib.escapeShellArg "${deployCfg.authelia.hostClientSecretDir}/${agentClientId hive}.secret"}
@ -281,43 +255,7 @@ in
skipped=$((skipped + 1))
fi
# The matrix appservice token, per hive. Unlike the two loops above
# there is nothing to copy: authelia never minted this one, and each
# homeserver's own host minted its own — which is exactly why two hives
# never converged on a value the swarm has to agree on. The mint moves
# here because this unit already holds a store identity and already
# writes under the hive prefix; ../host-modules/hive-matrix.nix keeps
# its activation mint as the first-boot fallback for a hive that has
# never reached the store.
#
# 32 bytes of urandom as lowercase hex, the shape that file has
# generated all along. Written with no trailing newline, unlike that
# file: what goes into the store is the value, and
# ./glue-matrix-bao-token.nix adds the newline the hive's token file
# carries.
${lib.concatMapStringsSep "\n" (hive: ''
src=${lib.escapeShellArg "${matrixTokenDir}/${hive}"}
if [ ! -s "$src" ]; then
# The subshell keeps the umask local — the loops above rely on
# bao's own file handling and should not inherit one.
( umask 077; head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$src" )
echo "minted a matrix appservice token for ${hive}" >&2
fi
# `value=@$src` for the same reason as both loops above: bao opens
# the file itself, so the token is never an argument of this process.
#
# Put on every run, not only on the run that minted — a put of the
# value already there changes nothing for any reader, while a mint
# whose publish failed must not be left as a token this host holds
# and no hive can reach. The `-s` test above is what keeps the VALUE
# stable across those runs: rotating on every boot would hand the
# homeserver a registration its running container does not carry.
bao kv put ${lib.escapeShellArg "secret/swarm/hives/${hive}/matrix/appservice-token"} \
value=@"$src"
tokens=$((tokens + 1))
'') hiveNames}
echo "published $published client secret(s), skipped $skipped, and $tokens matrix appservice token(s)"
echo "published $published client secret(s), skipped $skipped"
'';
};
};

1
nix/host-modules/swarm.nix
View file

@ -63,7 +63,6 @@ let
# Every spelling is reserved for every declared hive, including the one the
# CN belongs to — a hive cannot be named after its own reader either.
++ lib.concatMap (hive: [
"${deployCfg.bao.matrixTokenCommonNamePrefix}-${hive}"
"${deployCfg.bao.queueAgentCommonNamePrefix}-${hive}"
"${deployCfg.bao.hiveLeafCommonNamePrefix}-${hive}"
]) hiveNames;

7
nix/module-eval/bao-basics.nix
View file

@ -162,13 +162,6 @@ let
name = "an operator's own certificate path beats the glue's default";
ok = baoExplicitCerts.services.hyperhive.deploy.bao.serverCertFile == "/etc/pki/bao.pem";
}
{
# Absence arm. A store with nothing to serve renders no reader, so the
# unit is a function of the PAIRING rather than of the store — which is
# the property that makes it glue instead of a feature of either side.
name = "a store with no homeserver beside it renders no token reader";
ok = !(baoPkcs11.systemd.services ? swarm-bao-matrix-token);
}
{
# The name a reader dials has to resolve where the store runs; a
# multi-host swarm resolves it upstream instead.

112
nix/module-eval/bao-grants.nix
View file

@ -87,20 +87,19 @@ let
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
deploy.bao.clientCaFile = lib.mkForce null;
};
# The store plus every one of the four readers that used to log in as the
# hive. One fixture rather than four: the claim they are four *separate*
# principals is only testable where all four render at once — that is the
# deployment in which two of them sharing a leaf would be invisible.
# The store plus every reader that used to log in as the hive. One fixture
# rather than one per reader: the claim they are *separate* principals is
# only testable where all of them render at once — that is the deployment in
# which two of them sharing a leaf would be invisible.
baoGrantWithConsumers = hive {
deploy.bao.enable = true;
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
deploy.matrix.enable = true;
deploy.grafana.enable = true;
deploy.swarm-otel.enable = true;
};
# The store with none of the four readers beside it. Grafana, the collector and
# the homeserver are simply off; the queue reader renders on any host holding
# The store with none of the readers beside it. Grafana and the collector are
# simply off; the queue reader renders on any host holding
# its leaf, which ./glue-bao-tls.nix mints here, so that leaf is taken away.
baoGrantNoReaders = hive {
deploy.bao.enable = true;
@ -117,16 +116,13 @@ let
deploy.bao.forwarderOidcClientKeyFile = lib.mkForce null;
};
# All four readers against a store they do not run, each with a leaf placed
# by hand. The deployment in which there is no local policy unit to wait for.
# Every reader against a store it does not run, each with a leaf placed by
# hand. The deployment in which there is no local policy unit to wait for.
baoRemoteReaders = hive {
deploy.matrix.enable = true;
deploy.grafana.enable = true;
deploy.swarm-otel.enable = true;
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
deploy.bao.matrixTokenClientCertFile = "/etc/pki/bao-matrix-token.pem";
deploy.bao.matrixTokenClientKeyFile = "/etc/pki/bao-matrix-token-key.pem";
deploy.bao.queueAgentClientCertFile = "/etc/pki/bao-queue-agent.pem";
deploy.bao.queueAgentClientKeyFile = "/etc/pki/bao-queue-agent-key.pem";
deploy.bao.grafanaOidcClientCertFile = "/etc/pki/bao-grafana-oidc.pem";
@ -135,10 +131,9 @@ let
deploy.bao.otelOidcClientKeyFile = "/etc/pki/bao-otel-oidc-key.pem";
};
# The four readers ./glue-bao-readers-policy-order.nix orders after their
# policy units.
# The readers ./glue-bao-readers-policy-order.nix orders after their policy
# units.
policyReaders = [
"swarm-bao-matrix-token"
"swarm-bao-queue-agent"
"swarm-bao-grafana-oidc"
"swarm-bao-otel-oidc"
@ -166,13 +161,12 @@ let
_: u: (u.environment.BAO_CLIENT_CERT or null) == granterCertFile
) baoGrantWithConsumers.systemd.services;
# The fourteen units that write a `swarm-*` grant, by name, for the discovery
# The thirteen units that write a `swarm-*` grant, by name, for the discovery
# control below.
grantingUnitNames = [
"swarm-bao-controller-policy"
"swarm-bao-secret-publisher-policy"
"swarm-bao-matrix-ctl-policy"
"swarm-bao-matrix-token-policy"
"swarm-bao-queue-agent-policy"
"swarm-bao-grafana-oidc-policy"
"swarm-bao-otel-oidc-policy"
@ -479,11 +473,11 @@ let
&& !(baoGrantHere.containers.swarm-bao.config.systemd.services ? swarm-bao-matrix-ctl-policy);
}
# ── the four readers that used to share the hive's own leaf ──────────────
# ── the readers that used to share the hive's own leaf ───────────────────
#
# 🩸 Until this split all four presented `deploy.bao.clientCertFile`, whose
# policy granted read on `swarm/agents/*`, `swarm/hives/<hive>/*` AND
# `swarm/services/*`. Four principals behind one certificate are one
# 🩸 Until this split all of them presented `deploy.bao.clientCertFile`,
# whose policy granted read on `swarm/agents/*`, `swarm/hives/<hive>/*` AND
# `swarm/services/*`. Principals behind one certificate are one
# principal to bao, so the only expressible grant was the union: the unit
# fetching Grafana's OIDC secret could fetch every agent credential in the
# swarm.
@ -493,30 +487,6 @@ let
# just as well when the other two stanzas are still there beside it. The
# arms pin what each principal must NOT reach, so a later widening fails
# here instead of being noticed in a store.
{
name = "the matrix-token reader's grant is one hive's appservice token and nothing else";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-matrix-token-policy.script;
in
lib.hasInfix "path \"secret/data/swarm/hives/h1/matrix/appservice-token\" {" s
&& lib.hasInfix "capabilities = [\"read\"]" s
# The three stanzas the hive's own leaf carried, none of which this
# principal needs: every agent's credential, every service's OIDC
# client, and the rest of its own hive's tree — including the queue
# credential its sibling reader fetches.
&& !(lib.hasInfix "secret/data/swarm/agents" s)
&& !(lib.hasInfix "secret/data/swarm/services" s)
&& !(lib.hasInfix "secret/data/swarm/hives/h1/*" s)
&& !(lib.hasInfix "secret/data/swarm/hives/h1/queue" s)
# A `hives/*` wildcard would serve every hive from one role and let any
# hive read any other's token — the reach this split exists to remove,
# not to create.
&& !(lib.hasInfix "secret/data/swarm/hives/*" s)
# Nothing may rewrite the policy constraining it, for the reason the
# controller's own `hive-*` narrowing above gives.
&& !(lib.hasInfix "sys/policies/acl" s);
}
{
name = "the queue-credential reader's grant is one hive's queue credential and nothing else";
ok =
@ -616,13 +586,13 @@ let
{
# 🩸 The half that makes the policies above bind: a policy grants only
# through a token that carries it, and a token is minted by a cert-auth
# role matching a CN. Four distinct subjects is the whole mechanism — one
# subject for four readers is one principal however the policies read.
# role matching a CN. Distinct subjects is the whole mechanism — one
# subject for every reader is one principal however the policies read.
#
# The per-hive subjects carry the hive name because their paths do; the
# two service subjects do not, because an OIDC client is registered once
# The per-hive subject carries the hive name because its path does; the
# service subjects do not, because an OIDC client is registered once
# per swarm. Pinned so neither shape is tidied into the other.
name = "each of the five readers logs in under a subject of its own";
name = "each of the four readers logs in under a subject of its own";
ok =
let
subjectOf =
@ -637,8 +607,7 @@ let
# the reason the three service principals above state.
&& !(lib.hasInfix "auth/cert/certs/hive-" s);
in
subjectOf "swarm-bao-matrix-token-policy" "swarm-matrix-token-h1" "swarm-bao-matrix-token-h1"
&& subjectOf "swarm-bao-queue-agent-policy" "swarm-queue-agent-h1" "swarm-bao-queue-agent-h1"
subjectOf "swarm-bao-queue-agent-policy" "swarm-queue-agent-h1" "swarm-bao-queue-agent-h1"
&& subjectOf "swarm-bao-grafana-oidc-policy" "swarm-grafana-oidc" "swarm-bao-grafana-oidc"
&& subjectOf "swarm-bao-otel-oidc-policy" "swarm-otel-oidc" "swarm-bao-otel-oidc"
&& subjectOf "swarm-bao-forwarder-oidc-policy" "swarm-forwarder-oidc" "swarm-bao-forwarder-oidc";
@ -652,7 +621,7 @@ let
#
# Each pair is asserted whole: a certificate with no key authenticates
# nothing, so a half-set pair is a reader that does not render.
name = "each of the five readers presents its own leaf, never the hive's";
name = "each of the four readers presents its own leaf, never the hive's";
ok =
let
b = baoGrantWithConsumers.services.hyperhive.deploy.bao;
@ -661,8 +630,6 @@ let
b.clientKeyFile
];
own = [
b.matrixTokenClientCertFile
b.matrixTokenClientKeyFile
b.queueAgentClientCertFile
b.queueAgentClientKeyFile
b.grafanaOidcClientCertFile
@ -680,7 +647,6 @@ let
lib.all (p: p != null) own
&& !(lib.any (p: lib.elem p hiveLeaf) own)
&& lib.length (lib.unique own) == lib.length own
&& presents "swarm-bao-matrix-token" b.matrixTokenClientCertFile b.matrixTokenClientKeyFile
&& presents "swarm-bao-queue-agent" b.queueAgentClientCertFile b.queueAgentClientKeyFile
&& presents "swarm-bao-grafana-oidc" b.grafanaOidcClientCertFile b.grafanaOidcClientKeyFile
&& presents "swarm-bao-otel-oidc" b.otelOidcClientCertFile b.otelOidcClientKeyFile
@ -689,7 +655,7 @@ let
{
# The minting side of the same claim. A role matching a subject nothing
# signs is a reader that cannot log in, so the leaves and the roles have
# to be asserted against each other — and the two per-hive leaves carry
# to be asserted against each other — and the per-hive leaf carries
# THIS host's hive name, which is what makes one hive's leaf useless
# against another hive's role.
name = "the PKI unit signs a leaf per reader, each under that reader's own subject";
@ -702,8 +668,6 @@ let
# literal spanning both would pin this file's line wrapping rather than
# the pairing it means to.
lib.all (lib.flip lib.hasInfix s) [
"/matrix-token.pem ]"
"swarm-bao-matrix-token-h1 \"\" clientAuth"
"/queue-agent.pem ]"
"swarm-bao-queue-agent-h1 \"\" clientAuth"
"/grafana-oidc.pem ]"
@ -733,11 +697,10 @@ let
# client certificate and the host is the side that has one, so a unit
# rendered inside the store's container would have neither an identity
# nor a route. Plus the ordering that makes the mounts exist first.
name = "the five readers' granting units are ordered after the mounts and rendered on the host";
name = "the four readers' granting units are ordered after the mounts and rendered on the host";
ok =
let
units = [
"swarm-bao-matrix-token-policy"
"swarm-bao-queue-agent-policy"
"swarm-bao-grafana-oidc-policy"
"swarm-bao-otel-oidc-policy"
@ -759,7 +722,7 @@ let
# The forwarder is listed apart from `policyReaders`: it renders wherever
# the store does, so it is never absent on a store host and never present
# on a remote one, and the two cases below would fail on it for that.
name = "each of the five readers is ordered after the unit writing its role";
name = "each of the four readers is ordered after the unit writing its role";
ok =
let
s = baoGrantWithConsumers.systemd.services;
@ -778,7 +741,7 @@ let
# The ordering is set apart from each reader's own definition, so it can
# define a reader by itself: `after` on a unit nothing else declares is a
# unit with no ExecStart. On the store's host without the readers, none
# of the four may exist.
# of them may exist.
name = "a store host without the readers gains no reader unit from their ordering";
ok = lib.all (reader: !(baoGrantNoReaders.systemd.services ? ${reader})) policyReaders;
}
@ -800,24 +763,24 @@ let
}
{
# A store host without the granter's pair writes its grants some other
# way, so none of the fourteen units may exist. Without this arm
# way, so none of the thirteen units may exist. Without this arm
# `lib.mkIf haveGranter` could be dropped from any of them and every other
# case here would still pass.
name = "without the granter's pair none of the fourteen granting units render";
name = "without the granter's pair none of the thirteen granting units render";
ok =
let
s = baoGranterOptOut.systemd.services;
in
lib.all (unit: !(s ? ${unit})) (grantingUnitNames ++ [ "swarm-bao-granter-role" ])
# The control: the same store with the pair renders all fourteen.
# The control: the same store with the pair renders all thirteen.
&& lib.all (unit: baoGrantHere.systemd.services ? ${unit}) grantingUnitNames;
}
{
# 🩸 What replaced the silent skip. With no bootstrap token the fourteen still
# 🩸 What replaced the silent skip. With no bootstrap token the thirteen still
# render, and a refused granter fails them with the step that fixes it.
# A store host that never named a token is told to name one, since the
# unit that sets the granter up renders only where it has.
name = "a store host without a bootstrap token renders the fourteen, each failing loudly with the one-time step";
name = "a store host without a bootstrap token renders the thirteen, each failing loudly with the one-time step";
ok =
let
s = baoGranterNoToken.systemd.services;
@ -1247,8 +1210,8 @@ let
}
{
# `matrix_account::hive_sender` writes every hive's sender token, and
# nothing else under `hives/`: a hive's appservice token sits beside
# it. `+` is one segment, which keeps this to the one leaf; a `*` is a
# nothing else under `hives/`, where each hive's queue credential also
# lives. `+` is one segment, which keeps this to the one leaf; a `*` is a
# glob only at the end of a path. Pinned as the whole stanza, so an
# added capability fails.
name = "the controller writes each hive's sender token and nothing else under hives";
@ -1392,8 +1355,8 @@ let
}
{
# What makes the case above mean something: discovery by the granter's
# certificate reaches all fourteen units, and each yields calls.
name = "the granter-policy check sees all fourteen granting units, and parses calls from each";
# certificate reaches all thirteen units, and each yields calls.
name = "the granter-policy check sees all thirteen granting units, and parses calls from each";
ok =
lib.sort lib.lessThan (lib.attrNames granterUnits) == lib.sort lib.lessThan grantingUnitNames
&& lib.all (u: baoCalls u.script != [ ]) (lib.attrValues granterUnits)
@ -1723,11 +1686,6 @@ let
}
)
[
{
fetch = "swarm-bao-matrix-token";
machine = "hive-matrix";
consumer = "tuwunel.service";
}
{
fetch = "swarm-bao-otel-oidc";
machine = baoGrantWithConsumers.services.hyperhive.swarm.otel.machine;

154
nix/module-eval/bao-matrix-reader.nix
View file

@ -34,16 +34,14 @@ let
# inexpressible — the gate asked whether the store was a neighbour.
#
# 🩸 One leaf PER READER, and this fixture is where that cost is visible: the
# two units below identify themselves to the store separately, so an operator
# placing leaves by hand places one for each rather than one for both. Naming
# only `clientCertFile` here would leave neither unit rendered — which is what
# the arms below would then be asserting about.
# queue-credential unit identifies itself to the store separately from the
# hive, so an operator placing leaves by hand places one for it as well as
# the hive's own. Naming only `clientCertFile` here would leave the unit
# unrendered — which is what the arms below would then be asserting about.
baoRemoteReader = hive {
deploy.matrix.enable = true;
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
deploy.bao.matrixTokenClientCertFile = "/etc/pki/bao-matrix-token.pem";
deploy.bao.matrixTokenClientKeyFile = "/etc/pki/bao-matrix-token-key.pem";
deploy.bao.queueAgentClientCertFile = "/etc/pki/bao-queue-agent.pem";
deploy.bao.queueAgentClientKeyFile = "/etc/pki/bao-queue-agent-key.pem";
};
@ -54,24 +52,19 @@ let
# fixture set its own, as ./lib.nix asks.
matrixNoBaoIdentity = hive { deploy.matrix.enable = true; };
# 🩸 The hand-configured remote reader that named seven of the eight options.
# 🩸 The hand-configured remote reader that named the hive's leaf and stopped.
# `baoRemoteReader` above is the same deployment done right; this one holds
# the hive's own leaf, so it demonstrably reads the store, and is missing both
# per-principal pairs. Before the four-way split this host rendered both units
# off `clientCertFile` alone, so what it has now is a silent regression rather
# than any shape an operator chose — which is what the refusal arms below are
# about. Kept as one fixture rather than two because both refusals fire on it
# and each arm names which.
# the hive's own leaf, so it demonstrably reads the store, and is missing the
# queue reader's pair. Without the refusal it would render no queue reader
# and say nothing — which is what the refusal arms below are about.
baoRemoteReaderMissingLeaves = hive {
deploy.matrix.enable = true;
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
};
# The same omission on a hive that does NOT run a homeserver. Separates the
# matrix refusal's `deploy.matrix.enable` clause from the queue refusal, which
# checks only the hive's own toggle — an arm below reads exactly one refusal
# off it.
# The same omission on a hive that does NOT run a homeserver: the queue
# refusal checks only the hive's own toggle.
remoteReaderNoMatrixMissingLeaves = hive {
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
@ -92,10 +85,10 @@ let
# holding this host's role — all of which a retry fixes. A read that
# answers "nothing there" is not, so only the first is allowed to fail
# the unit.
name = "the matrix token reader retries a failed login and still degrades on an empty read";
name = "the queue credential reader retries a failed login and still degrades on an empty read";
ok =
let
u = baoWithMatrix.systemd.services.swarm-bao-matrix-token;
u = baoWithMatrix.systemd.services.swarm-bao-queue-agent;
# Everything between the login's failure branch and the read's, which
# is where the exit that decides "retry or give up" lives.
afterLogin = lib.last (lib.splitString "bao login" u.script);
@ -109,30 +102,15 @@ let
&& lib.hasInfix "exit 0" (lib.last (lib.splitString "bao kv get" u.script));
}
{
# The reader's own grant covers `swarm/hives/<this hive>/*` and
# `swarm/agents/*`; a path outside those answers 403, not "no such key".
# So the hive segment is what makes the read reachable, and a rename that
# drops it looks correct and fails identically on every boot.
name = "the matrix token path sits inside the prefix the reader is granted";
ok =
let
s = baoWithMatrix.systemd.services.swarm-bao-matrix-token.script;
in
lib.hasInfix "secret/swarm/hives/" s
&& lib.hasInfix "/matrix/appservice-token" s
# The shape it used to have: `matrix` where a principal kind belongs,
# which no grant covers.
&& !(lib.hasInfix "secret/swarm/matrix/" s);
}
{
# The store's second reader, and the gate that decides it exists is the
# certificate rather than anything about agents: containers are created
# The gate that decides this reader exists is the certificate rather
# than anything about agents: containers are created
# at runtime, so there is no static "this hive runs agents" fact to ask.
name = "a hive that names a client identity reads its agent queue credential";
ok = baoRemoteReader.systemd.services ? swarm-bao-queue-agent;
}
{
# Same 403-not-a-miss reason as the matrix arm above, against the path
# The reader's own grant covers `swarm/hives/<this hive>/*`; a path
# outside it answers 403, not "no such key". This is the path
# `swarm_secret_client::queue::agent_client_path` builds from the same
# pieces. The negative arm is the rename this one is exposed to: a
# credential named for the queue rather than for the hive that presents
@ -176,17 +154,6 @@ let
&& !(builtins.elem "hive-c0re.service" (u.requiredBy or [ ]))
&& !(builtins.elem "hive-c0re.service" (u.requires or [ ]));
}
{
# The store's first reader. Its unit belongs to the pairing, not to
# either service: matrix must not learn the store exists, and the store
# must not know who reads it.
name = "a store deployed beside the homeserver fetches its appservice token";
ok = baoWithMatrix.systemd.services ? swarm-bao-matrix-token;
}
{
name = "a hive that names a client identity reads from a store it does not run";
ok = baoRemoteReader.systemd.services ? swarm-bao-matrix-token;
}
{
# `Requires=` on a unit that does not exist fails the job, and nothing
# local mints certificates off-host — so this orders against nothing.
@ -198,7 +165,7 @@ let
let
s = baoRemoteReader.systemd.services;
in
s ? swarm-bao-matrix-token && s.swarm-bao-matrix-token.requires == [ ];
s ? swarm-bao-queue-agent && s.swarm-bao-queue-agent.requires == [ ];
}
{
# Presence control for the case above: the list is conditional, not gone.
@ -207,17 +174,17 @@ let
let
s = baoWithMatrix.systemd.services;
in
s ? swarm-bao-matrix-token && s.swarm-bao-matrix-token.requires == [ "swarm-bao-pki.service" ];
s ? swarm-bao-queue-agent && s.swarm-bao-queue-agent.requires == [ "swarm-bao-pki.service" ];
}
{
# Nothing asserted this script before, which is how it kept a branch that
# named three states and threw away the only thing telling them apart. A
# missing value, a refused identity and an unreachable host all end in the
# same degraded mode here, correctly — what must survive is which one.
name = "the matrix token reader carries the store's own diagnostic into the journal";
name = "the queue credential reader carries the store's own diagnostic into the journal";
ok =
let
s = baoWithMatrix.systemd.services.swarm-bao-matrix-token.script;
s = baoWithMatrix.systemd.services.swarm-bao-queue-agent.script;
in
!(lib.hasInfix "2>/dev/null" s) && lib.hasInfix ''cat "''$err"'' s;
}
@ -295,18 +262,12 @@ let
# it the unit names two paths the container does not have, and fails at
# the TLS handshake naming no cause. Read off the mount table rather than
# the option, so a pairing that stops reaching `bindMounts` still fails.
#
# The second arm is the shape guard: `bindMounts` is one literal plus two
# merges, and a rewrite that dropped the appservice registration would
# take the homeserver's own credential with it.
name = "the matrix container binds matrix-ctl's PKI read-only, without losing the appservice registration";
name = "the matrix container binds matrix-ctl's PKI read-only";
ok =
let
mounts = baoWithMatrix.containers.hive-matrix.bindMounts;
in
mounts ? "/var/lib/swarm-bao-pki"
&& mounts."/var/lib/swarm-bao-pki".isReadOnly
&& mounts ? "/var/lib/hyperhive/matrix-appservice";
mounts ? "/var/lib/swarm-bao-pki" && mounts."/var/lib/swarm-bao-pki".isReadOnly;
}
{
# The cert role ./host-modules/swarm-bao.nix writes, which is the one
@ -366,12 +327,18 @@ let
or false;
}
{
# The swarm registration reaches tuwunel the way the hive's does: a
# `.yaml` credential in the directory `appservice_dir` names.
name = "tuwunel loads the swarm registration as a yaml credential";
ok = lib.elem "swarm-appservice.yaml:/var/lib/swarm-matrix-appservice/swarm.yaml" (
baoWithMatrix.containers.hive-matrix.config.systemd.services.tuwunel.serviceConfig.LoadCredential
);
# The swarm registration reaches tuwunel as a `.yaml` credential in the
# directory `appservice_dir` names, and it is the only registration
# there: no client presents a per-hive one's token, so loading one would
# be a credential nobody reads.
name = "tuwunel loads the swarm registration as a yaml credential, and no other";
ok =
let
creds =
baoWithMatrix.containers.hive-matrix.config.systemd.services.tuwunel.serviceConfig.LoadCredential;
in
lib.elem "swarm-appservice.yaml:/var/lib/swarm-matrix-appservice/swarm.yaml" creds
&& lib.length (lib.filter (c: lib.hasInfix ".yaml:" c) creds) == 1;
}
{
# A missing `LoadCredential` source fails tuwunel, so the render has to
@ -402,38 +369,30 @@ let
&& !(lib.any (lib.hasPrefix "swarm-appservice.yaml") c.systemd.services.tuwunel.serviceConfig.LoadCredential);
}
{
# 🩸 The arm the whole refusal exists for. Both readers are gated on their
# own leaf, so the way this regresses is the build going green and three
# units rendering where four should — which no presence check on a
# rendered unit can see, because the unit that is missing is the evidence.
# Read as a refusal naming each option, so an operator acts on the message
# without opening the nix.
name = "a remote reader missing the per-principal leaves is refused, naming both options";
# 🩸 The arm the whole refusal exists for. The reader is gated on its own
# leaf, so the way this regresses is the build going green and the unit
# not rendering — which no presence check on a rendered unit can see,
# because the unit that is missing is the evidence. Read as a refusal
# naming each option, so an operator acts on the message without opening
# the nix.
name = "a remote reader missing the queue reader's leaf is refused, naming both options";
ok =
refusedOver baoRemoteReaderMissingLeaves "matrixTokenClientCertFile"
&& refusedOver baoRemoteReaderMissingLeaves "matrixTokenClientKeyFile"
&& refusedOver baoRemoteReaderMissingLeaves "queueAgentClientCertFile"
refusedOver baoRemoteReaderMissingLeaves "queueAgentClientCertFile"
&& refusedOver baoRemoteReaderMissingLeaves "queueAgentClientKeyFile";
}
{
# The matrix refusal's own gate, which the queue refusal does not have.
# Without this arm the two are indistinguishable on the fixture above.
name = "the queue refusal needs no homeserver, and the matrix refusal stays quiet without one";
ok =
refusedOver remoteReaderNoMatrixMissingLeaves "queueAgentClientCertFile"
&& !(refusedOver remoteReaderNoMatrixMissingLeaves "matrixTokenClientCertFile");
name = "the queue refusal needs no homeserver";
ok = refusedOver remoteReaderNoMatrixMissingLeaves "queueAgentClientCertFile";
}
{
# ⚠️ The arm that keeps the refusal from being worse than the silence it
# replaced. A hive with NO store identity is the supported no-store
# deployment and also the state an operator passes through bringing a hive
# up — neither may fail to evaluate. Asserted as "no refusal names any of
# the four options", not as "this one fixture is fine", because the way
# this breaks is a gate widened to the principal's leaf alone.
# up — neither may fail to evaluate. Asserted as "no refusal names either
# option", not as "this one fixture is fine", because the way this breaks
# is a gate widened to the principal's leaf alone.
name = "a hive with no store identity at all is refused over none of the per-principal leaves";
ok = lib.all (option: !(refusedOver matrixNoBaoIdentity option)) [
"matrixTokenClientCertFile"
"matrixTokenClientKeyFile"
"queueAgentClientCertFile"
"queueAgentClientKeyFile"
];
@ -441,22 +400,13 @@ let
{
# The other half of the same guard, and the one an operator meets far more
# often: on the store's own host ./host-modules/glue-bao-tls.nix mkDefaults
# all eight, so there is nothing to name and nothing to refuse. Paired with
# every pair, so there is nothing to name and nothing to refuse. Paired with
# the fully-named remote reader, which is the same deployment done by hand.
name = "neither a store host nor a correctly-named remote reader is refused";
ok =
lib.all
(
m:
lib.all (option: !(refusedOver m option)) [
"matrixTokenClientCertFile"
"queueAgentClientCertFile"
]
)
[
baoWithMatrix
baoRemoteReader
];
ok = lib.all (m: !(refusedOver m "queueAgentClientCertFile")) [
baoWithMatrix
baoRemoteReader
];
}
];
in

9
nix/module-eval/matrix-core.nix
View file

@ -59,14 +59,7 @@ let
lib.hasInfix "client_max_body_size 100048576;" c && !(lib.hasInfix "client_max_body_size 50M" c);
}
{
# Absence arm for the one above, and the reason the gate is the identity
# rather than the homeserver: without it, deploying matrix anywhere would
# render a reader that cannot authenticate.
name = "a homeserver with no way to authenticate to the store renders no token reader";
ok = !(matrixNoBaoIdentity.systemd.services ? swarm-bao-matrix-token);
}
{
# Absence arm for the case above. A hive with no client identity gets no
# Absence arm. A hive with no client identity gets no
# store environment at all — the daemon reports a queue it cannot serve
# rather than a handshake it cannot explain.
name = "a hive with no client identity gives hive-c0re no store environment";

4
nix/module-eval/name-guards.nix
View file

@ -90,8 +90,8 @@ let
swarm.hives.grctl.domain = "gr.t.local";
};
# 🩸 A different shape from every fixture above: the matrix-token and
# queue-credential roles are written PER HIVE, so the subject a hive must not
# 🩸 A different shape from every fixture above: the queue-credential roles
# are written PER HIVE, so the subject a hive must not
# be is `<prefix>-<some hive's name>` rather than the prefix itself. Reserving
# only the prefix would leave the composed spelling free, and a hive taking it
# would present a leaf the other hive's role accepts — which is a hive reading

37
nix/module-eval/secret-publisher.nix
View file

@ -243,41 +243,6 @@ let
lib.hasInfix "secret/swarm/hives/h1/queue/agent" s
&& lib.hasInfix "secret/swarm/hives/h2/queue/agent" s;
}
{
# The producer's end of the read `glue-matrix-bao-token.nix` already did.
# Both hives are asserted for the reason the queue case above gives: a
# publisher that knew one name rather than the roster would pass on a
# single-hive fixture and strand the second hive ever added — which is
# the two-hives-never-converge shape this slice exists to close.
name = "the publisher mints an appservice token for every hive and writes it to that hive's matrix path";
ok =
let
s = secretPublisherHere.systemd.services.swarm-secret-publish.script;
in
lib.hasInfix "secret/swarm/hives/h1/matrix/appservice-token" s
&& lib.hasInfix "secret/swarm/hives/h2/matrix/appservice-token" s
&& lib.hasInfix "/dev/urandom" s;
}
{
# What makes a re-publish idempotent. This principal is granted
# `create`/`update` and no `read`, so it cannot ask the store whether a
# hive already has a token — with nowhere to keep one, every run would
# mint a fresh value and rotate the swarm's token. A state directory is
# that somewhere, and nothing else in this unit needs one, so its absence
# means exactly this.
#
# The second arm is the mint's own guard: the state file is only written
# when it is missing or empty. Dropping that test leaves a unit that
# still has a state directory and still rotates on every boot.
name = "the publisher keeps the tokens it minted, and mints only when it holds none";
ok =
let
u = secretPublisherHere.systemd.services.swarm-secret-publish;
in
lib.hasInfix "matrix-appservice-token" (u.serviceConfig.StateDirectory or "")
&& u.serviceConfig.StateDirectoryMode or null == "0700"
&& lib.hasInfix "if [ ! -s \"$src\" ]" u.script;
}
{
# A property of the SET, not of one unit: both of these authenticate by
# certificate, and `BAO_CLIENT_CERT` is transport rather than identity, so
@ -306,7 +271,7 @@ let
&& !(lib.hasInfix "bao kv" (lib.head (lib.splitString "bao login" c)));
in
holdsTokenFirst secretPublisherHere.systemd.services.swarm-secret-publish.script
&& holdsTokenFirst baoWithMatrix.systemd.services.swarm-bao-matrix-token.script
&& holdsTokenFirst baoWithMatrix.systemd.services.swarm-bao-queue-agent.script
# Controls, so a clean verdict above means something. In order: a bare
# read is refused, a read placed before the login is refused, and a
# login that exists only in a comment is refused — that last one is the

4
swarm-matrix-client/src/lib.rs
View file

@ -500,8 +500,8 @@ mod tests {
#[test]
fn the_expected_already_exists_answer_is_recognised_by_its_errcode() {
// Matched on the spec's code rather than on message text, because this
// is the arm a healthy homeserver takes every time: the hive's `@hive-<hive>:`
// account is the appservice registration's own sender, created at startup.
// is the arm a healthy homeserver takes every time: an account minted
// on an earlier pass already exists.
let json = serde_json::json!({ "errcode": "M_USER_IN_USE", "error": "User ID taken" });
assert_eq!(errcode(&json), Some("M_USER_IN_USE"));
}

15
swarm-matrix-ctl/src/appservice.rs
View file

@ -38,19 +38,18 @@ const ENV_DIR: &str = "MATRIX_APPSERVICE_DIR";
/// The registration's `sender_localpart`: the account the homeserver creates
/// for it and the one `admin_execute` promotes.
const ENV_SENDER: &str = "MATRIX_APPSERVICE_SENDER";
/// The user namespace regex, rendered by nix beside the hive registration's.
/// The user namespace regex, rendered by nix.
const ENV_USER_REGEX: &str = "MATRIX_APPSERVICE_USER_REGEX";
/// Role on the store's `cert` auth mount that [`publish`] logs in with.
const ENV_CERT_ROLE: &str = "MATRIX_APPSERVICE_CERT_ROLE";
/// The registration's `id`. Distinct from the hive registration's
/// (`hyperhive`): tuwunel refuses two registrations with one id.
/// The registration's `id`.
const ID: &str = "swarm";
/// File names inside [`ENV_DIR`]. `REGISTRATION` is what tuwunel loads.
const AS_TOKEN: &str = "as-token";
const HS_TOKEN: &str = "hs-token";
const REGISTRATION: &str = "swarm.yaml";
/// Random bytes per token, as the hive registration's renderer uses.
/// Random bytes per token.
const TOKEN_BYTES: usize = 32;
/// Read a required variable.
@ -115,11 +114,11 @@ fn write_secret(path: &Path, contents: &str) -> Result<()> {
std::fs::rename(&tmp, path).with_context(|| format!("renaming onto {}", path.display()))
}
/// The registration, in the shape `hive-matrix.nix` renders the hive's.
/// The registration tuwunel loads.
///
/// `exclusive: false` for that file's reason: an exclusive namespace does not
/// widen what this appservice may do, it refuses everyone else — and the hive
/// registration covers the same names until it is retired.
/// `exclusive: false` for the reason `hive-matrix.nix` gives at
/// `appserviceUserRegex`: an exclusive namespace does not widen what this
/// appservice may do, it refuses everyone else.
fn registration_yaml(sender: &str, regex: &str, as_token: &str, hs_token: &str) -> String {
format!(
"id: {ID}\n\

24
swarm-matrix-ctl/src/registration.rs
View file

@ -1,14 +1,13 @@
//! Reading the `as_token` out of the appservice registration the matrix
//! container already has.
//!
//! No new credential is delivered for this. `nix/host-modules/hive-matrix.nix`
//! binds the registration directory into the container read-only so tuwunel can
//! load it, and the registration **is** the `as_token` — so the file this
//! module opens is one this process could already read, and one the homeserver
//! beside it reads too.
//! No new credential is delivered for this. The registration **is** the
//! `as_token`, and `appservice render` writes it into this container's own
//! state dir — so the file this module opens is one this process could already
//! read, and one the homeserver beside it reads too.
//!
//! Scanned line-by-line rather than parsed as YAML. The file has exactly one
//! renderer (`appserviceRegistrationScript` in that same module, a `printf` of
//! renderer (`registration_yaml` in `appservice.rs`, a `format!` of
//! `as_token: <hex>`), so a parser would be a second, looser reading of a shape
//! this repo writes itself — and it would pull a YAML crate into a binary whose
//! only other input is JSON.
@ -52,13 +51,12 @@ fn extract(text: &str) -> Result<String> {
mod tests {
use super::*;
/// The registration exactly as `appserviceRegistrationScript` renders it —
/// the quoted heredoc, then the `printf` of the two tokens. Reproduced
/// verbatim because that script is the other end of this agreement and
/// lives in a file no Rust test can reach.
const RENDERED: &str = "id: hyperhive\n\
/// The registration in the shape `registration_yaml` renders it, tokens
/// last. Reproduced rather than rendered so this module's tests do not
/// depend on the renderer's.
const RENDERED: &str = "id: swarm\n\
url: null\n\
sender_localpart: hive\n\
sender_localpart: swarm\n\
rate_limited: false\n\
namespaces:\n \
users:\n \
@ -97,7 +95,7 @@ mod tests {
// request carrying one with a 403 that names the account rather than
// the credential — so failing here is the only report an operator can
// act on.
for bad in ["id: hyperhive\n", "as_token:\n", "as_token: \n"] {
for bad in ["id: swarm\n", "as_token:\n", "as_token: \n"] {
assert!(extract(bad).is_err(), "{bad:?} must not yield a token");
}
}

4
swarm-secret-client/README.md
View file

@ -17,8 +17,8 @@ formatted at each call site is an agreement with no owner — it holds right up
until one side is edited alone, and then it fails as a missing key rather than
as a mismatch.
The third end is what settles it. `nix/host-modules/glue-matrix-bao-token.nix`
reads the store with `bao kv get -field=value`. That reader is a shell line in
The third end is what settles it. `nix/agent-modules/forge-token.nix` reads
the store with `bao kv get -field=value`. That reader is a shell line in
a nix module: it cannot be renamed by the same refactor as a Rust struct, and
no Rust test reaches it. So the field name is pinned by a test against the
serialised literal rather than left to the struct definition.

82
swarm-secret-client/src/matrix.rs
View file

@ -35,8 +35,9 @@ pub fn accounts_dir(agent: &str) -> Result<String, Error> {
Ok(format!("{prefix}/matrix"))
}
/// The localpart `hive` acts as on the homeserver, and the `sender_localpart`
/// of that hive's appservice registration.
/// The localpart `hive` acts as on the homeserver: the account
/// `swarm-controller` creates for it with the swarm's appservice token, and
/// the one hive-c0re's sender token belongs to.
///
/// **Derived from the hive's name, which is what makes it one account per
/// hive.** It used to be the bare constant `hive`: one swarm runs one
@ -45,9 +46,8 @@ pub fn accounts_dir(agent: &str) -> Result<String, Error> {
/// one hive without revoking all of them.
///
/// The prefix is kept so the account still reads as a hive's rather than an
/// agent's; `nix/host-modules/hive-matrix.nix` renders the same string into
/// the registration's `sender_localpart`, and **the two must match** — nothing
/// wires an override across.
/// agent's. Both `swarm-controller` and hive-c0re derive it here, so the two
/// cannot disagree on which account a hive is.
///
/// Infallible on purpose: a hive name is `[a-z0-9-]` (`hive_types::Ident`) and
/// the appservice namespace regex admits that charset, so there is no
@ -60,7 +60,7 @@ pub fn hive_localpart(hive: &str) -> String {
/// The path holding `hive`'s matrix sender account's homeserver access token.
///
/// Keyed per **hive**, like [`appservice_token_path`] below. Under
/// Keyed per **hive**, like [`crate::queue::agent_client_path`]. Under
/// [`Kind::Hive`] the grant that reaches it is the hive's **own** stanza,
/// `secret/data/swarm/hives/<this hive>/*`, which interpolates the name: hive
/// `alpha` reads its own token and gets a 403 on `beta`'s.
@ -73,27 +73,6 @@ pub fn sender_token_path(hive: &str) -> Result<String, Error> {
Ok(format!("{prefix}/matrix/sender-token"))
}
/// The path holding `hive`'s matrix appservice token (`as_token`).
///
/// Keyed per **hive**, not per agent, like [`crate::queue::agent_client_path`]
/// and unlike [`account_path`] above: one homeserver admits one hive's
/// accounts, so the identity that creates them is the hive's.
///
/// ⚠️ Renamed from `registration-token` along with what it holds: the
/// homeserver no longer accepts a shared registration secret at all, so a
/// value still stored under the old path would be read by nothing. A hive
/// whose store has only the old path falls back to its locally minted
/// token — see `glue-matrix-bao-token.nix` — so the rename degrades rather
/// than breaks, but the store needs a fresh `put` to take effect again.
///
/// # Errors
/// [`Error::PathSegment`] when `hive` contains anything but `[A-Za-z0-9_-]`,
/// which is what keeps one hive's name from addressing another hive's secret.
pub fn appservice_token_path(hive: &str) -> Result<String, Error> {
let prefix = principal_prefix(Kind::Hive, hive)?;
Ok(format!("{prefix}/matrix/appservice-token"))
}
/// The name segment of the controller's own subtree, and the cert-auth role it
/// logs in under (`swarm-controller`'s `store::CERT_ROLE`).
pub(crate) const CONTROLLER: &str = "swarm-controller";
@ -122,9 +101,9 @@ pub fn swarm_appservice_token_path() -> Result<String, Error> {
/// no other place its homeserver is written down.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct Credential {
/// `glue-matrix-bao-token.nix` reads the store with
/// `bao kv get -field=value`, so this name is load-bearing for a reader
/// this crate does not control. Renaming it silently breaks that unit.
/// Named `value` like [`crate::forge::Credential::value`] and
/// [`crate::queue::Credential::value`], so a nix-side reader spells
/// `bao kv get -field=value` for every kind.
pub value: String,
/// Absent on every object written before this field existed, and KV2 keeps
@ -134,8 +113,8 @@ pub struct Credential {
/// would break every stored credential at once.
///
/// `skip_serializing_if` is doing separate work: without it a token-only
/// credential serialises `"homeserver":null`, and this object is read by
/// `glue-matrix-bao-token.nix` with `bao kv get -field=value`.
/// credential serialises `"homeserver":null` rather than the
/// `{"value":…}` shape the objects written before this field have.
#[serde(skip_serializing_if = "Option::is_none")]
pub homeserver: Option<String>,
}
@ -183,10 +162,8 @@ mod tests {
#[test]
fn the_localpart_is_derived_from_the_hive_name() {
// The literal is the point: `nix/host-modules/hive-matrix.nix` renders
// the same string into the registration's `sender_localpart`, and
// nothing wires an override across — so a change here is a change
// there.
// The literal is the point: an existing hive's account on the
// homeserver carries this localpart, so a change here strands it.
assert_eq!(hive_localpart("pr1ma"), "hive-pr1ma");
assert_ne!(
hive_localpart("pr1ma"),
@ -197,33 +174,6 @@ mod tests {
assert_ne!(hive_localpart("pr1ma"), "hive");
}
#[test]
fn the_appservice_token_lands_under_the_hive_prefix_the_grant_covers() {
// Spelled out for the same reason as above, and with a second job here:
// the read policy grants `secret/data/swarm/hives/<hive>/*`, so this
// string is what makes the path reachable at all.
assert_eq!(
appservice_token_path("pr1ma").expect("a plain name is legal"),
"swarm/hives/pr1ma/matrix/appservice-token"
);
}
#[test]
fn the_appservice_token_is_not_a_top_level_namespace() {
// The path its predecessor used to hold. `Kind` is a closed set and
// `matrix` is not one of its members, so a path with `matrix` as the
// second segment is outside every grant — which is how it came to 403
// on every read.
let p = appservice_token_path("pr1ma").expect("legal");
assert!(!p.starts_with("swarm/matrix/"), "{p}");
}
#[test]
fn a_traversal_in_the_hive_name_is_refused() {
let e = appservice_token_path("../beta").expect_err("a traversal is not");
assert!(matches!(e, Error::PathSegment { kind: "hive", .. }), "{e}");
}
#[test]
fn the_swarm_appservice_token_lands_under_the_controller() {
assert_eq!(
@ -310,14 +260,14 @@ mod tests {
#[test]
fn the_value_field_matches_what_the_nix_reader_asks_for() {
// The literal is the point: `bao kv get -field=value` is the other end
// of this agreement and lives in a file no Rust test can reach, so the
// The literal is the point: a nix-side reader spells the field as
// `bao kv get -field=value` in a file no Rust test can reach, so the
// name is pinned here rather than derived from the struct.
//
// It doubles as the compatibility control for `homeserver`: a
// token-only credential must still serialise to exactly these bytes,
// with no `homeserver` key at all, so adding the field cannot change
// what that unit reads.
// what such a reader gets.
let json = serde_json::to_string(&Credential {
value: "t".to_owned(),
homeserver: None,

6
swarm-secret-client/src/path.rs
View file

@ -10,9 +10,9 @@ use crate::Error;
/// The KV v2 mount every swarm secret lives under.
///
/// A literal because the nix-side reader spells the same one in
/// `glue-matrix-bao-token.nix`; an option nobody sets would be two ways to say
/// one thing.
/// A literal because the nix-side readers spell the same one, as in
/// `glue-queue-agent-credential.nix`; an option nobody sets would be two ways
/// to say one thing.
pub const MOUNT: &str = "secret";
/// The root under [`MOUNT`] that every swarm secret lives beneath.