diff --git a/docs/integrations/matrix.md b/docs/integrations/matrix.md index ccde7da6..849e917d 100644 --- a/docs/integrations/matrix.md +++ b/docs/integrations/matrix.md @@ -90,39 +90,54 @@ by default. Reaching it on 8448 needs either an explicit tuwunel bind to that port OR a reverse-proxy + `.well-known/matrix/server` delegation (the latter lives in `gateway.md::Discovery flow`). -## Provisioning flow (registration token) +## Provisioning flow (appservice) -Token-gated registration: hive-c0re holds the token, agents never -see it. The agent only receives the resulting `access_token`. +Registration is closed. Accounts are created by the hive's own +**appservice**: hive-c0re holds the appservice token, agents never see +it, and an agent only ever receives its own `access_token`. -1. **System activation** writes a 32-byte random hex token (64 - chars) to `services.hyperhive.deploy.matrix.registrationTokenFile` - (`/var/lib/hyperhive/matrix-register-token` by default), mode - `0600 root:root`, before any container start. Idempotent — only - writes when the file is missing or empty; always re-applies 0600 - (normalises any 0640 / world-readable carry-over from - pre-LoadCredential deployments). This runs at activation time - (not first container start) to dodge a race where nspawn creates - an empty file when the bind-mount target is missing and tuwunel - reads `registration_token_file=""`, rejecting every registration - until next restart. -2. **Read-only bind-mount** maps the host file into the tuwunel +The appservice has no URL (`url: null` in its registration), so the +homeserver never calls out to it and there is no service to run. What the +registration buys is an identity the homeserver recognises — which is why +no secret has to be equal on both sides of the wire, and why account +creation doesn't depend on registration being open to anyone who learns +a token. + +1. **System activation** mints a 32-byte random hex appservice token (64 + chars) at `/var/lib/hyperhive/matrix-appservice-token` and its + spec-required `hs_token` sibling, mode `0600 root:root`, then renders + the registration to + `/var/lib/hyperhive/matrix-appservice/hyperhive.yaml` (also `0600`). + The tokens are minted only when missing; the registration is + re-rendered every time, because the token file can be overwritten in + place by the swarm secret store and a registration naming a stale + token authenticates nobody. Runs at activation time, before any + container start, because the directory is bind-mounted and + nixos-container refuses to start when a bind source is missing. +2. **Read-only bind-mount** maps that directory into the tuwunel container at the same path. 3. **systemd `LoadCredential=`** inside the container copies the - bind-mounted file into - `/run/credentials/tuwunel.service/registration_token`, owned by - tuwunel's dynamic user with mode `0400`, at service start. The - host file stays `root:root 0600` — no `chown :tuwunel` / - `chmod 0640` / GID-pin gymnastics required. Keeps - `DynamicUser = true` + `PrivateUsers = true` intact. -4. tuwunel's `registration_token_file` points at the credentials - path, not the original bind-mount path. -5. **hive-c0re** uses the token to register each agent account via - the matrix-spec UIAA registration flow, persists the returned - `access_token` to `/matrix-token`. The agent's - matrix MCP client authenticates with that access_token and - never touches the shared registration token. -6. **hive-c0re restarts `hive-matrix-daemon`** for the agent + registration into + `/run/credentials/tuwunel.service/hyperhive-appservice.yaml`, owned by + tuwunel's dynamic user with mode `0400`, at service start. The host + file stays `root:root 0600` — no `chown :tuwunel` / `chmod 0640` / + GID-pin gymnastics required. Keeps `DynamicUser = true` + + `PrivateUsers = true` intact. +4. tuwunel's `appservice_dir` points at the credentials directory, not at + the bind-mount path. It reads only `.yaml`/`.yml` entries from there, + so the sibling credentials are invisible to it. The `.yaml` suffix on + the credential id is what makes this work. +5. **hive-c0re** reads the appservice token and creates each account with + one `POST /register` typed `m.login.application_service`, persisting + the returned `access_token` to `/matrix-token`. It never + mints the token itself: the value has to be the one the rendered + registration names, and only the nix side writes that. +6. **An account that exists but has lost its token file** is re-tokened + by an appservice `POST /login` — no password and no admin rights + involved. A stored-password login and an admin-room password reset + remain behind that, for accounts created before the appservice existed + or named outside its namespace. +7. **hive-c0re restarts `hive-matrix-daemon`** for the agent immediately after writing the token so the daemon picks up the new credential without waiting for a full container restart. If the restart fails (for example daemon not yet running on first boot) @@ -130,16 +145,70 @@ see it. The agent only receives the resulting `access_token`. (`hive-matrix-daemon.path` watching for `matrix-token` appearance) brings the daemon up on the same boot cycle anyway. +### The admin account, and why it needs no first-user luck + +`@hive:` is the appservice's own `sender_localpart`, which +the homeserver creates itself when it loads the registration — on a +zero-user database, inside startup, before the HTTP listener accepts +anything. Its **admin rights** then come from an explicit +`make_user_admin`, run by tuwunel's `admin_execute` in the same startup +and likewise before the listener — so a fresh hive has a joined, +power-level-100 admin on its first boot. + +This replaces a dependency on being the first account ever registered, +which was fragile in both directions: an appservice-created account is +excluded from that automatic grant by design, and on a homeserver that +already had users the rule never fired at all. + +Promotion can't be bootstrapped over the API, and that's upstream's +design rather than a gap: tuwunel only treats an admin-room message as a +command when its sender is already an admin. `admin_execute` is the one +lever with no sender to check. hive-c0re re-checks the result on every +sweep by reading the admin account's own joined-rooms list; if the rights +are missing it says so, names +`systemctl restart container@hive-matrix` as the fix, and carries on — +agent accounts, the hive Space and the chat room need no admin. + +
Upgrading a hive that used the registration token + +Nothing to do, and nothing to time. The activation script mints the +appservice token and renders the registration before the homeserver +restarts, so the first boot after the switch already has both halves. + +- **Existing accounts keep working.** An access token lives on the + device that minted it; removing the registration token touches no + device, no account and no session. `login_with_password` stays on, so + the password fallback is still there too. +- **Existing token files are honoured.** The per-agent sweep skips any + agent that already has a `matrix-token`, so no account is re-registered + and no session is displaced. +- **The admin account is already admin** on such a hive (it won the + first-user grant when the hive was new), so the startup promotion is a + no-op — upstream's `make_user_admin` short-circuits when the user is + already joined at power level 100. +- **`/var/lib/hyperhive/matrix-register-token` is left on disk**, read by + nothing. Delete it or leave it; neither does any harm. +- **`registrationTokenFile` is a removed option.** A config that still + sets it fails to evaluate with a message naming the appservice — a hive + that never set it (the default) is unaffected. +- **A swarm store holding the old `matrix/registration-token` path** is + no longer read at all; the hive uses its locally minted appservice + token until someone `put`s a value at `matrix/appservice-token`. + +
+ Initial rollout settings: - `allow_federation = true` at the protocol level so swarms can be wired up later by extending `trustedServers` without a homeserver restart. `trusted_servers = []` keeps it effectively closed until you list peers. -- `allow_registration = true` (required for the token flow to - engage). The absent - `yes_i_am_very_very_sure_…_open_registration_…` flag keeps the - server closed to anyone without the token. +- `allow_registration = false`. tuwunel checks this flag only for + requests that arrive **without** an appservice token, so hive-c0re + provisions exactly as before and everyone else is refused. It's not a + hardening afterthought: with no registration token configured, + `allow_registration = true` makes tuwunel refuse to start unless + `yes_i_am_very_very_sure_…_open_registration_…` is also set. - `allow_encryption` — server-side E2EE switch, sourced from `services.hyperhive.swarm.matrix.allowEncryption` (**default `false`**, opt-in). Off by default because on the hive-internal homeserver the operator diff --git a/docs/swarm/secrets.md b/docs/swarm/secrets.md index 24481b57..79c1bedb 100644 --- a/docs/swarm/secrets.md +++ b/docs/swarm/secrets.md @@ -136,7 +136,7 @@ when the system builds. The server names the offending file and refuses to run. | ---------------------------------------- | --------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | | hive CA cert + key | `hive-tls.nix` first-boot unit | `/ca.pem`, `ca-key.pem` (`0600`) | | hive leaf certs | `hive-tls.nix`, signed by the hive CA | `/.pem` | -| matrix registration token | a host activation script, on first boot | `/var/lib/hyperhive/matrix-register-token` (`0600`) | +| matrix appservice token | a host activation script, on first boot | `/var/lib/hyperhive/matrix-appservice-token` (`0600`) | | the forge's copy of its OIDC secret | `hive-forge-oidc-secret.service` copies it from authelia's tree | `/var/lib/forgejo-oidc/.secret` inside the forge container | | the homeserver's copy of its OIDC secret | `hive-matrix-oidc-secret.service`, same shape | `/var/lib/tuwunel-oidc/.secret`, handed to tuwunel through `LoadCredential` | | the agent containers' queue credential | authelia, published to the store by `swarm-secret-publish` | `/secret` (`0600`) and `/client_id` (`0644`) | @@ -146,18 +146,26 @@ bounded wait, 120s — and then **fail loudly** rather than skipping. A silent s produces a service whose login button always fails, which is a symptom many layers from its cause. -The store's **first reader** is the matrix registration token, and it's worth +The store's **first reader** is the matrix appservice token, and it's worth saying why that one: it's an opaque 32-byte value with no second file and no format. Authelia's OIDC secret needs a `.secret` _and_ a matching `.digest`, so starting there would have meant debugging "can a reader authenticate and get bytes back" and "is authelia's file format right" at once, with an SSO outage as the failure mode. -`glue-matrix-bao-token.nix` fetches it and writes the file `hive-matrix.nix` -already reads, so the homeserver never learns the store exists. Every failure -path — no such key, sealed store, unreachable store, empty value — leaves the -locally minted token in place, so a hive with no store behaves exactly as it -did before. +`glue-matrix-bao-token.nix` fetches it, writes the file `hive-matrix.nix` +already reads, and then runs that module's own renderer to re-stamp the +appservice registration naming the token — so the homeserver never learns the +store exists. The re-render isn't housekeeping: the token is half an +agreement, and a registration carrying the previous value authenticates +nobody. Every failure path — no such key, sealed store, unreachable store, +empty value — leaves the locally minted token in place, so a hive with no store +behaves exactly as it did before. + +The store path is `swarm/hives//matrix/appservice-token`. It was +`…/matrix/registration-token` while the homeserver still took a shared +registration secret; a value left at the old path is read by nothing, and the +hive falls back to its local token until someone `put`s the new one. The **second reader** is the agent containers' queue credential: `glue-queue-agent-credential.nix` lands it as two files, the client secret and diff --git a/docs/swarm/sso.md b/docs/swarm/sso.md index cb38d00a..4054d120 100644 --- a/docs/swarm/sso.md +++ b/docs/swarm/sso.md @@ -168,9 +168,9 @@ Two consequences worth stating plainly: - **tuwunel re-reads its secret file on every OAuth exchange**, not only at startup, and its own sandboxing hides most paths from it. It gets the - file through `LoadCredential` for the same reason the registration token - does — that keeps `DynamicUser` and `PrivateUsers` intact, with no - host-side ownership arrangement to maintain. + file through `LoadCredential` for the same reason the appservice + registration does — that keeps `DynamicUser` and `PrivateUsers` intact, + with no host-side ownership arrangement to maintain. - **Matrix SSO lives inside the homeserver.** The client-server API is spoken by non-browser clients holding matrix access tokens — every agent's own daemon — as well as by federation, so the homeserver diff --git a/hive-c0re/src/matrix.rs b/hive-c0re/src/matrix.rs index 5ec53206..b4c09558 100644 --- a/hive-c0re/src/matrix.rs +++ b/hive-c0re/src/matrix.rs @@ -982,9 +982,8 @@ pub async fn ensure_admin_user(client: &reqwest::Client, as_token: &str) -> Resu /// admin is degraded, not broken. Only `hivectl matrix promote-user` / /// `reset-password` need it. /// -/// # Errors -/// Never — the outcome is the return value. `false` means "not admin and -/// could not be made one", already logged with what to do about it. +/// Returns whether the account holds admin rights now. A `false` has +/// already been logged, with what to do about it. async fn ensure_admin_rights( client: &reqwest::Client, admin_token: &str, @@ -1003,7 +1002,9 @@ async fn ensure_admin_rights( return true; } Some(_) => { - tracing::warn!("matrix: hive admin is not in the admin room — attempting to promote it") + tracing::warn!( + "matrix: hive admin is not in the admin room — attempting to promote it" + ); } None => tracing::debug!( "matrix: could not read the hive admin's joined rooms; attempting to promote it" diff --git a/hive-matrix-mcp/src/client.rs b/hive-matrix-mcp/src/client.rs index 867ced5d..f77619fe 100644 --- a/hive-matrix-mcp/src/client.rs +++ b/hive-matrix-mcp/src/client.rs @@ -5,10 +5,10 @@ //! to start sync. //! //! No OAuth dance / cross-signing setup (in contrast to damocles-daemon's -//! ccc.de connection): for the in-hive tuwunel the `registration_token` -//! UIAA flow already minted the token + user/device, hive-c0re just -//! handed us the bearer in a file. matrix-sdk's `restore_session` with -//! a constructed `MatrixSession` skips the login flow entirely. +//! ccc.de connection): for the in-hive tuwunel hive-c0re already minted +//! the token + user/device as the hive's appservice and handed us the +//! bearer in a file. matrix-sdk's `restore_session` with a constructed +//! `MatrixSession` skips the login flow entirely. //! //! E2EE is enabled via `with_encryption_settings(EncryptionSettings::default())`. //! Crypto keys are persisted in the sqlite store under `state_dir` diff --git a/nix/host-modules/deploy.nix b/nix/host-modules/deploy.nix index 0d491143..d89a1e7d 100644 --- a/nix/host-modules/deploy.nix +++ b/nix/host-modules/deploy.nix @@ -247,10 +247,29 @@ in [ "services" "hyperhive" "swarm" "matrix" "maxRequestSize" ] [ "services" "hyperhive" "deploy" "matrix" "maxRequestSize" ] ) - (lib.mkRenamedOptionModule - [ "services" "hyperhive" "swarm" "matrix" "registrationTokenFile" ] - [ "services" "hyperhive" "deploy" "matrix" "registrationTokenFile" ] - ) + # Both spellings of the registration token, removed rather than renamed: + # the homeserver no longer accepts a shared registration secret at all, + # so there is no path to point a definition at. Accounts are created by + # the hive's appservice, whose token lives at a fixed path this module + # deliberately does not let anyone move. + (lib.mkRemovedOptionModule [ "services" "hyperhive" "swarm" "matrix" "registrationTokenFile" ] '' + The matrix registration token is gone: hive-c0re creates accounts as the + hive's appservice instead, so there is no shared secret for tuwunel to + check. Remove this definition — nothing needs replacing, the appservice + token is minted and delivered automatically + (/var/lib/hyperhive/matrix-appservice-token, not operator-settable). + The old token file at /var/lib/hyperhive/matrix-register-token is read by + nothing now and can be deleted. See docs/integrations/matrix.md. + '') + (lib.mkRemovedOptionModule [ "services" "hyperhive" "deploy" "matrix" "registrationTokenFile" ] '' + The matrix registration token is gone: hive-c0re creates accounts as the + hive's appservice instead, so there is no shared secret for tuwunel to + check. Remove this definition — nothing needs replacing, the appservice + token is minted and delivered automatically + (/var/lib/hyperhive/matrix-appservice-token, not operator-settable). + The old token file at /var/lib/hyperhive/matrix-register-token is read by + nothing now and can be deleted. See docs/integrations/matrix.md. + '') (lib.mkRenamedOptionModule [ "services" "hyperhive" "swarm" "matrix" "gui" "enable" ] [ "services" "hyperhive" "deploy" "matrix" "gui" "enable" ] diff --git a/nix/host-modules/glue-matrix-bao-token.nix b/nix/host-modules/glue-matrix-bao-token.nix index 4f2fe486..077c7b86 100644 --- a/nix/host-modules/glue-matrix-bao-token.nix +++ b/nix/host-modules/glue-matrix-bao-token.nix @@ -1,8 +1,9 @@ -# Glue: the matrix registration token comes from the secret store. +# Glue: the matrix appservice token comes from the secret store. # # The store's first reader, and deliberately a small one. It fetches an opaque -# 32-byte value and writes it where ./hive-matrix.nix already looks — the -# homeserver never learns the store exists, and its config is unchanged. +# 32-byte value and writes it where ./hive-matrix.nix already looks, then asks +# that module's own renderer to re-stamp the appservice registration naming it +# — the homeserver never learns the store exists, and its config is unchanged. # # ⚠️ Why this credential first. It has no second file and no format: authelia's # OIDC secret needs a `.secret` *and* a matching `.digest`, so shipping that @@ -47,12 +48,12 @@ let # own grant covers. The store's read policy grants `swarm/agents/*` and # `swarm/hives//*` and nothing else, so a path outside those is a # 403 rather than a miss, however correct it looks. `swarm-secret-client`'s - # `matrix::registration_token_path` builds the same string from the same + # `matrix::appservice_token_path` builds the same string from the same # pieces; this literal is the nix half of that one agreement. # # `hiveName` has no fallback here for the reason ./glue-bao-tls.nix gives at # its own use of it: it is asserted set for every hyperhive host. - tokenPath = "secret/swarm/hives/${hyperhiveCfg.hiveName}/matrix/registration-token"; + tokenPath = "secret/swarm/hives/${hyperhiveCfg.hiveName}/matrix/appservice-token"; # A literal, not an option — ./hive-matrix.nix names its container # `containers.hive-matrix` directly and declares no `machine` to derive it @@ -70,7 +71,7 @@ in services.hyperhive.swarm.otel.journaldUnits = [ "swarm-bao-matrix-token" ]; systemd.services.swarm-bao-matrix-token = { - description = "fetch the matrix registration token from the swarm secret store"; + description = "fetch the matrix appservice token from the swarm secret store"; # Every one of these names a unit that exists only where the store runs. # `Requires=` on an absent unit fails the job outright, so the ordering is # conditional even though the read is not: off-host there is nothing local @@ -176,8 +177,20 @@ in fi umask 077 - printf '%s\n' "$token" > ${lib.escapeShellArg (toString deployCfg.matrix.registrationTokenFile)} - chmod 0600 ${lib.escapeShellArg (toString deployCfg.matrix.registrationTokenFile)} + printf '%s\n' "$token" > ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)} + chmod 0600 ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)} + + # Re-stamp the registration file from the token just written. The + # token is half an agreement — the registration the homeserver loads + # has to carry the same value — so writing the file and stopping + # would leave the homeserver authenticating hive-c0re against + # whatever activation put there: a 401 on every request, naming + # nothing. Unconditional rather than on-change, because this unit + # has no way to know what the registration currently says. + # + # hive-matrix's own renderer rather than a `printf` here, so the + # registration's shape has one home. + ${deployCfg.matrix.appserviceRegistrationScript} ''; }; }; diff --git a/nix/host-modules/glue-queue-agent-credential.nix b/nix/host-modules/glue-queue-agent-credential.nix index 81e3c350..9e57b512 100644 --- a/nix/host-modules/glue-queue-agent-credential.nix +++ b/nix/host-modules/glue-queue-agent-credential.nix @@ -15,7 +15,7 @@ # that is correct rather than degraded. The publisher runs on the authelia # host and authelia mints on its first boot, so "nothing at that path yet" is # the ordinary early state of a swarm. Nothing here writes a local stand-in: -# unlike a matrix registration token there is no such thing as a locally valid +# unlike a matrix appservice token there is no such thing as a locally valid # OIDC client secret, so a placeholder would turn a hive that cannot connect # into one that is refused, which reaches the agent as a timeout. # diff --git a/nix/host-modules/hive-matrix.nix b/nix/host-modules/hive-matrix.nix index 48605f58..71485f0b 100644 --- a/nix/host-modules/hive-matrix.nix +++ b/nix/host-modules/hive-matrix.nix @@ -40,18 +40,12 @@ let # login into an intermittent one. matrixSecretPath = "/var/lib/tuwunel-oidc/${cfg.sso.clientId}.secret"; - # Single source for the registration-token path: the option's `default`, - # its `config`-level `mkDefault` self-definition, and the assertion that - # rejects a moved path all read this same binding rather than repeating - # the string literal. See `registrationTokenFile`'s own comment below. - registrationTokenPath = "/var/lib/hyperhive/matrix-register-token"; - - # ⚠️ tuwunel does NOT read the path above directly, and this indirection + # ⚠️ tuwunel does NOT read the host path directly, and this indirection # is not ceremony. Upstream's own words: "under systemd the path must be # visible to the service after sandboxing (ReadWritePaths / ProtectHome), # typically by placing the file under /etc/tuwunel/" — which this # container has no writable etc for. `LoadCredential` is the answer - # already in use two units below for the registration token, and for the + # already in use for the appservice registration below, and for the # same reason: it keeps `DynamicUser=true` + `PrivateUsers=true` intact # with no host-side chown or GID pinning. matrixSecretCredential = "/run/credentials/tuwunel.service/oidc_client_secret"; @@ -79,11 +73,15 @@ let adminLocalpart = "hive"; # The `as_token`, and the `hs_token` the spec requires alongside it. Both - # minted by the activation script below, mode 0600; the `as_token` is the - # one hive-c0re reads and the one the swarm secret store overwrites (see + # minted by the render script below, mode 0600; the `as_token` is the one + # hive-c0re reads and the one the swarm secret store overwrites (see # `glue-matrix-bao-token.nix`). The `hs_token` authenticates the homeserver # TO the appservice, which with `url = null` is nobody — it exists because # the registration format requires it. + # + # `appserviceTokenPath` is the single source for the option's `default`, + # its `config`-level `mkDefault` self-definition, and the assertion that + # rejects a moved path. See `appserviceTokenFile`'s own comment below. appserviceTokenPath = "/var/lib/hyperhive/matrix-appservice-token"; appserviceHsTokenPath = "/var/lib/hyperhive/matrix-appservice-hs-token"; @@ -124,6 +122,64 @@ let # create these names, not a monopoly on them. appserviceUserRegex = "^@[a-z0-9._=/-]+:${lib.escapeRegex effectiveServerName}$"; + # Mint the tokens if they are absent, then render the registration from + # whatever they now hold. **One script with two callers** — the activation + # script below and `glue-matrix-bao-token.nix`, which overwrites the + # `as_token` with the swarm store's copy and has to re-render afterwards. + # A second copy of the registration's shape would be a second source of + # truth for a format whose mismatch is silent: the homeserver would load a + # registration naming a token nobody holds, and every request hive-c0re + # makes would come back 401 naming nothing. + # + # Re-rendering unconditionally is the point rather than thoroughness: "the + # token file exists" does not mean "the registration carries what is in + # it". + appserviceRegistrationScript = pkgs.writeShellApplication { + name = "hive-matrix-appservice-registration"; + runtimeInputs = [ pkgs.coreutils ]; + text = '' + # Both the tokens and the rendered registration are secrets; 077 + # covers every file this script creates rather than each one + # separately. + umask 077 + mkdir -p ${lib.escapeShellArg appserviceDir} + for f in ${lib.escapeShellArg appserviceTokenPath} ${lib.escapeShellArg appserviceHsTokenPath}; do + if [ ! -s "$f" ]; then + head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$f" + echo >> "$f" + echo "hive-matrix: generated appservice token at $f" + fi + chmod 0600 "$f" + done + + # Read into shell variables and emitted with `printf`, a shell + # builtin: a token passed as an argument to a real command would land + # in that process's argv, which is world-readable for its lifetime. + asToken="$(cat ${lib.escapeShellArg appserviceTokenPath})" + hsToken="$(cat ${lib.escapeShellArg appserviceHsTokenPath})" + + # The quoted heredoc keeps the regex's own `$` and `\` out of the + # shell's hands; the YAML single quotes keep them out of YAML's. + { + cat <<'REGISTRATION' + id: ${appserviceId} + url: null + sender_localpart: ${adminLocalpart} + rate_limited: false + namespaces: + users: + - exclusive: false + regex: '${appserviceUserRegex}' + aliases: [] + rooms: [] + REGISTRATION + printf 'as_token: %s\nhs_token: %s\n' "$asToken" "$hsToken" + } > ${lib.escapeShellArg appserviceRegistrationPath} + chmod 0600 ${lib.escapeShellArg appserviceRegistrationPath} + chmod 0700 ${lib.escapeShellArg appserviceDir} + ''; + }; + # Format-locked by tuwunel, not chosen here: the callback host must point # directly at the matrix server and the path is fixed at # `/_matrix/client/unstable/login/sso/callback/`. Built once @@ -511,40 +567,59 @@ in ''; }; - registrationTokenFile = lib.mkOption { + appserviceTokenFile = lib.mkOption { type = lib.types.path; internal = true; - default = registrationTokenPath; + default = appserviceTokenPath; description = '' - Host path to a file containing the matrix registration token - tuwunel reads to authorise new-account creation. The token is - generated automatically by `hive-c0re` on first boot (32-byte - random hex, mode 0600) and is bind-mounted read-only into the - tuwunel container at the same path. Agents never see this - token — hive-c0re uses it to provision per-agent accounts - and the agent only receives the resulting `access_token`. + Host path to a file containing this hive's matrix appservice + token (`as_token`) — the identity `hive-c0re` creates and logs + into accounts with. Minted automatically on first activation + (32-byte random hex, mode 0600) and rendered into the + appservice registration the homeserver loads at boot. Agents + never see it; an agent only ever receives its own + `access_token`. Not operator-settable — `hive-c0re`'s Rust side derives this - same path independently (`paths::matrix_register_token()`) with - nothing wiring an override across, so a moved path used to - desync the two silently. An externally-managed token is - delivered by writing into *this* fixed path instead of moving it — - see `glue-matrix-bao-token.nix`, which fetches from the swarm - secret store and overwrites this file in place. + same path independently (`paths::matrix_appservice_token()`) + with nothing wiring an override across, so a moved path desyncs + the two silently. An externally-managed token is delivered by + writing into *this* fixed path instead of moving it — see + `glue-matrix-bao-token.nix`, which fetches from the swarm secret + store, overwrites this file in place, and re-renders the + registration that names it. Enforced by an `assertions` entry below rather than `readOnly`: the `config` block gives this option its own `mkDefault` - definition (lowest priority) so any real override — an operator's, - or the pre-rename `swarm.matrix.registrationTokenFile` shim's — - still resolves cleanly instead of crashing eval with nixpkgs' - generic "read-only, set multiple times" message; the assertion - then names the actual problem. (`readOnly` was the first attempt — - atlas caught that it only rejects a *second* definition, - so with nothing else defining the option a lone override sailed - through silently, same desync as before with a lock that wasn't - locked. Fixed by defining the value here instead of leaving it on - `default` alone, but a proper `assertions` message beat re-adding - `readOnly` on top once the option had a real definition either way.) + definition (lowest priority) so a real override still resolves + cleanly instead of crashing eval with nixpkgs' generic + "read-only, set multiple times" message; the assertion then + names the actual problem. (`readOnly` was the first attempt on + this option's predecessor — atlas caught that it only rejects a + *second* definition, so with nothing else defining the option a + lone override sailed through silently, same desync as before + with a lock that wasn't locked. Fixed by defining the value here + instead of leaving it on `default` alone, but a proper + `assertions` message beat re-adding `readOnly` on top once the + option had a real definition either way.) + ''; + }; + + appserviceRegistrationScript = lib.mkOption { + type = lib.types.path; + internal = true; + default = "${appserviceRegistrationScript}/bin/hive-matrix-appservice-registration"; + defaultText = lib.literalMD "the module's own registration renderer"; + description = '' + The script that mints the appservice tokens when absent and + (re-)renders the registration file from them. An option only so + that `glue-matrix-bao-token.nix` can run the same one after + overwriting the token with the swarm store's copy, rather than + carrying a second copy of the registration's shape. + + Not operator-settable, and not a hook: replacing it means + deciding what the homeserver's appservice registration says, + which is this module's job. ''; }; @@ -606,11 +681,11 @@ in config = lib.mkIf deployCfg.matrix.enable { # The option's own value, defined explicitly rather than left on its - # bare `default` — `mkDefault` so a real override (an operator's, or - # the pre-rename shim's) still resolves cleanly rather than crashing - # eval; the `assertions` entry below is what actually rejects it, with - # a message naming the reason. See that option's own comment above. - services.hyperhive.deploy.matrix.registrationTokenFile = lib.mkDefault registrationTokenPath; + # bare `default` — `mkDefault` so a real override still resolves + # cleanly rather than crashing eval; the `assertions` entry below is + # what actually rejects it, with a message naming the reason. See that + # option's own comment above. + services.hyperhive.deploy.matrix.appserviceTokenFile = lib.mkDefault appserviceTokenPath; # Matrix's own gateway surface: the sub-domain vhost, the name the # hive resolver answers for, and the Accept-header map that vhost's @@ -791,19 +866,17 @@ in # mkDefault above lets an override resolve instead of crashing # eval — this is the actual rejection, with a message that names # the real fix instead of nixpkgs' generic conflicting-definition - # text. Covers both the current path and the pre-rename - # `swarm.matrix.registrationTokenFile` shim in one check, since - # both land on the same merged option. - assertion = deployCfg.matrix.registrationTokenFile == registrationTokenPath; + # text. + assertion = deployCfg.matrix.appserviceTokenFile == appserviceTokenPath; message = '' - services.hyperhive.deploy.matrix.registrationTokenFile is fixed at - ${registrationTokenPath} and cannot be moved — hive-c0re's Rust + services.hyperhive.deploy.matrix.appserviceTokenFile is fixed at + ${appserviceTokenPath} and cannot be moved — hive-c0re's Rust side derives this same path independently and has no way to learn an override, so moving it desyncs the two silently instead of loudly. - Integrating an externally-managed registration token? Deliver it - by writing into ${registrationTokenPath} instead of pointing this + Integrating an externally-managed appservice token? Deliver it + by writing into ${appserviceTokenPath} instead of pointing this option elsewhere — see glue-matrix-bao-token.nix, which does exactly that from the swarm secret store. ''; @@ -849,8 +922,8 @@ in # permanent stall presenting as "matrix is broken", several layers from # its cause. # - # The registration token above dodges that with an activation script - # that pre-creates the file. ⚠️ That dodge is NOT available here: + # The appservice registration dodges that with an activation script + # that renders the file first. ⚠️ That dodge is NOT available here: # tuwunel requires the secret file to exist *and be non-empty*, so a # zero-byte placeholder would satisfy the bind mount and then stop the # homeserver from starting. @@ -958,74 +1031,21 @@ in '' ); - system.activationScripts.hive-matrix-register-token = lib.stringAfter [ "var" ] '' - tokenFile=${lib.escapeShellArg (toString deployCfg.matrix.registrationTokenFile)} - if [ ! -s "$tokenFile" ]; then - mkdir -p "$(dirname "$tokenFile")" - head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$tokenFile" - echo >> "$tokenFile" - echo "hive-matrix: generated registration token at $tokenFile" - fi - # Re-apply 0600 (normalises any pre-LoadCredential carry-over). - chmod 0600 "$tokenFile" - ''; - - # The appservice registration: mint the two tokens once, then re-render - # the registration file from them on EVERY activation. + # Mint the appservice tokens and render the registration, before any + # container start. # - # Re-rendering unconditionally is the point, not thoroughness. The token - # file is overwritten in place by `glue-matrix-bao-token.nix` when the - # swarm secret store has a value for this hive, so "the file exists" does - # not mean "the registration carries what is in it" — and a registration - # carrying a stale token is a homeserver that refuses every request - # hive-c0re makes, with a 401 that names nothing. - # - # An activation script rather than a unit, same as the token above: the - # directory is bind-mounted into the container, and nixos-container - # refuses to start when a bind source is missing. Activation runs before - # the container on a switch and on every boot. + # An activation script rather than a unit: the directory below is + # bind-mounted into the container and nixos-container refuses to start + # when a bind source is missing, so this has to have run first. + # Activation is what runs before the container both on a switch and on + # every boot. (The registration token this replaced used an activation + # script for the same reason, and additionally to dodge nspawn creating + # an empty file at a missing bind target — which tuwunel then read as + # "no token", refusing every registration until the next restart. A + # missing registration file is not silent in that way: the homeserver + # fails its appservice load loudly.) system.activationScripts.hive-matrix-appservice = lib.stringAfter [ "var" ] '' - appserviceDir=${lib.escapeShellArg appserviceDir} - regFile=${lib.escapeShellArg appserviceRegistrationPath} - - # Both the tokens and the rendered registration are secrets; 077 covers - # every file this script creates rather than each one separately. - umask 077 - mkdir -p "$appserviceDir" - for f in ${lib.escapeShellArg appserviceTokenPath} ${lib.escapeShellArg appserviceHsTokenPath}; do - if [ ! -s "$f" ]; then - head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$f" - echo >> "$f" - echo "hive-matrix: generated appservice token at $f" - fi - chmod 0600 "$f" - done - - # Read into shell variables and emitted with `printf`, a bash builtin: - # a token passed as an argument to a real command would land in that - # process's argv, which is world-readable for its lifetime. - asToken="$(cat ${lib.escapeShellArg appserviceTokenPath})" - hsToken="$(cat ${lib.escapeShellArg appserviceHsTokenPath})" - - # The quoted heredoc keeps the regex's own `$` and `\` out of the - # shell's hands; the YAML single quotes keep them out of YAML's. - { - cat <<'REGISTRATION' - id: ${appserviceId} - url: null - sender_localpart: ${adminLocalpart} - rate_limited: false - namespaces: - users: - - exclusive: false - regex: '${appserviceUserRegex}' - aliases: [] - rooms: [] - REGISTRATION - printf 'as_token: %s\nhs_token: %s\n' "$asToken" "$hsToken" - } > "$regFile" - chmod 0600 "$regFile" - chmod 0700 "$appserviceDir" + ${deployCfg.matrix.appserviceRegistrationScript} ''; containers.hive-matrix = { @@ -1036,16 +1056,13 @@ in extraFlags = [ "--link-journal=host" ]; # Shared host netns — agents reach tuwunel at localhost:. privateNetwork = false; - # Read-only bind of the host-managed registration token; tuwunel + # Read-only bind of the host-managed appservice registration; tuwunel # reads it via systemd LoadCredential below (not directly). + # + # The directory, not the file inside it: the registration is + # re-rendered rather than edited, and binding the file would pin the + # inode the container saw when it started. bindMounts = { - ${deployCfg.matrix.registrationTokenFile} = { - hostPath = deployCfg.matrix.registrationTokenFile; - isReadOnly = true; - }; - # The directory, not the file inside it: the registration is - # re-rendered on every activation, and binding the file would pin - # the inode the container saw when it started. ${appserviceDir} = { hostPath = appserviceDir; isReadOnly = true; @@ -1141,13 +1158,19 @@ in # trustedServers keeps it effectively closed. allow_federation = true; trusted_servers = deployCfg.matrix.trustedServers; - # Token-gated registration. The absent - # `yes_i_am_very_very_sure_…_open_registration_…` flag - # keeps the server closed to anyone without the token. - allow_registration = true; - # LoadCredential below copies the host file into a - # 0400 dynamic-user-owned path; tuwunel reads from there. - registration_token_file = "/run/credentials/tuwunel.service/registration_token"; + # Nobody registers themselves here. Accounts are created by + # the hive's appservice, which this flag does not gate — + # tuwunel checks it only for requests that arrive WITHOUT an + # appservice token, so hive-c0re provisions exactly as before + # and everyone else is refused outright. + # + # ⚠️ Not a hardening afterthought: `allow_registration = true` + # with no registration token configured makes tuwunel REFUSE + # TO START (it demands + # `yes_i_am_very_very_sure_…_open_registration_…` instead). So + # dropping the token and leaving this true is not a lax + # homeserver, it is a homeserver that does not boot. + allow_registration = false; # Where the hive's appservice registration is read from — the # credentials directory, for the reasons at @@ -1239,11 +1262,9 @@ in # host-side chown :tuwunel / GID-pin gymnastics needed. # See `man systemd.exec` → LoadCredential. systemd.services.tuwunel.serviceConfig.LoadCredential = [ - "registration_token:${toString deployCfg.matrix.registrationTokenFile}" - # Same mechanism, third secret — and the one whose credential id - # carries a `.yaml` suffix on purpose, since `appservice_dir` - # above names this very directory and tuwunel takes only - # `.yaml`/`.yml` entries from it. + # The credential id carries a `.yaml` suffix on purpose: + # `appservice_dir` above names this very directory, and tuwunel + # takes only `.yaml`/`.yml` entries from it. "${appserviceCredentialId}:${appserviceRegistrationPath}" # Same mechanism, second secret. tuwunel re-reads this file on # every OAuth exchange, not just at startup, so it has to diff --git a/nix/module-eval.nix b/nix/module-eval.nix index a2682c6f..490076d2 100644 --- a/nix/module-eval.nix +++ b/nix/module-eval.nix @@ -116,24 +116,45 @@ let ]; }; - # The homeserver's turn to split. All eight host-side options are set through - # their pre-rename paths — including `gui.enable`, whose value is deliberately - # the opposite of its default so the definition has to actually land, and both - # packages, stubbed to a derivation neither option defaults to for the same - # reason. All eight so that dropping any single shim entry fails the eval, not - # just the ones the assertions read. + # The homeserver's turn to split. All seven remaining host-side options are + # set through their pre-rename paths — including `gui.enable`, whose value is + # deliberately the opposite of its default so the definition has to actually + # land, and both packages, stubbed to a derivation neither option defaults to + # for the same reason. All seven so that dropping any single shim entry fails + # the eval, not just the ones the assertions read. + # + # `registrationTokenFile` used to be the eighth: both spellings of it are now + # `mkRemovedOptionModule`, exercised by `matrixRemovedToken` below instead — + # setting it here would make every case that reads this fixture fail on that + # one removed option. matrixOldPath = hive { deploy.matrix.enable = true; swarm.matrix.openFirewall = true; swarm.matrix.trustedServers = [ "matrix.example.invalid" ]; swarm.matrix.maxRequestSize = 31457280; - swarm.matrix.registrationTokenFile = "/etc/matrix/register.token"; swarm.matrix.gui.enable = false; swarm.matrix.package = pkgs.emptyDirectory; swarm.matrix.gui.package = pkgs.emptyDirectory; swarm.matrix.sso.clientSecretFile = "/etc/matrix/oidc.secret"; }; + # Moving the appservice token, which desyncs hive-c0re's independently + # derived path from the homeserver's. Its own fixture because the assertion + # it trips would otherwise fire for every case reading `matrixOldPath`. + matrixMovedAppserviceToken = hive { + deploy.matrix.enable = true; + deploy.matrix.appserviceTokenFile = "/etc/matrix/as.token"; + swarm.matrix.sso.clientSecretFile = "/etc/matrix/oidc.secret"; + }; + + # A config still naming the removed registration token, through the + # pre-rename spelling — the one an old deployment is most likely to carry. + matrixRemovedToken = hive { + deploy.matrix.enable = true; + swarm.matrix.registrationTokenFile = "/etc/matrix/register.token"; + swarm.matrix.sso.clientSecretFile = "/etc/matrix/oidc.secret"; + }; + # The queue's callout identity, fourth split slice. `autoGenerateCallout` is # left FALSE on purpose: that is what makes the seed paths the thing deciding # `responderConfigured`, so the assertion below is about the seeds rather @@ -840,33 +861,41 @@ let } { # Third split, and the one whose readers were hardest to see: the - # registration token is read only through a `let` alias in another + # appservice token is read only through a `let` alias in another # module, so no full path names it anywhere. Both arms read a rendered # effect — the host firewall and the container's bind-mount table — so a # rename that resolves but stops reaching the module still fails. - name = "a config written against the pre-rename matrix paths still opens the port and mounts the token"; + name = "a config written against the pre-rename matrix paths still opens the port and mounts the appservice registration"; ok = let ports = matrixOldPath.networking.firewall.allowedTCPPorts; httpPort = matrixOldPath.services.hyperhive.swarm.matrix.httpPort; in builtins.elem httpPort ports - && matrixOldPath.containers.hive-matrix.bindMounts ? "/etc/matrix/register.token"; + && matrixOldPath.containers.hive-matrix.bindMounts ? "/var/lib/hyperhive/matrix-appservice"; } { - # registrationTokenFile lost its override capability entirely - # (bao-delivered secrets don't need one — glue-matrix-bao-token.nix - # already writes into the fixed path instead of moving it), unlike its - # five siblings in the same rename. `matrixOldPath` above proves the - # override still *resolves* (mkDefault, not a crash) — this proves it - # also gets *rejected*, by a named assertion rather than nixpkgs' generic - # conflicting-definition text. Reads `.assertions` directly (cheap: a - # list of `{assertion; message;}`, not `system.build.toplevel`) rather - # than forcing a real build just to observe a boolean. - name = "overriding registrationTokenFile (even via the pre-rename shim) trips a named assertion, not a silent desync"; + # appserviceTokenFile has no override capability (a store-delivered + # secret doesn't need one — glue-matrix-bao-token.nix writes into the + # fixed path instead of moving it), unlike its siblings in the same + # rename. The fixture proves the override still *resolves* (mkDefault, + # not a crash); this proves it also gets *rejected*, by a named + # assertion rather than nixpkgs' generic conflicting-definition text. + # Reads `.assertions` directly (cheap: a list of `{assertion; + # message;}`, not `system.build.toplevel`) rather than forcing a real + # build just to observe a boolean. + name = "moving appserviceTokenFile trips a named assertion, not a silent desync"; ok = lib.any ( - a: !a.assertion && lib.hasInfix "registrationTokenFile" a.message - ) matrixOldPath.assertions; + a: !a.assertion && lib.hasInfix "appserviceTokenFile" a.message + ) matrixMovedAppserviceToken.assertions; + } + { + # The removal has to SAY so. A config carrying the old option through + # an upgrade is the common case, and nixpkgs' bare "option does not + # exist" would name neither what replaced the token nor where to read + # about it. + name = "a config still setting registrationTokenFile is refused with a message naming the appservice"; + ok = lib.any (a: !a.assertion && lib.hasInfix "appservice" a.message) matrixRemovedToken.assertions; } { # Reads the DELIVERY UNIT, not the options: `responderConfigured` gates @@ -1518,7 +1547,7 @@ let s = baoWithMatrix.systemd.services.swarm-bao-matrix-token.script; in lib.hasInfix "secret/swarm/hives/" s - && lib.hasInfix "/matrix/registration-token" s + && lib.hasInfix "/matrix/appservice-token" s # The shape it used to have: `matrix` where a principal kind belongs, # which no grant covers. && !(lib.hasInfix "secret/swarm/matrix/" s); @@ -2471,7 +2500,7 @@ let # The store's first reader. Its unit belongs to the pairing, not to # either service: matrix must not learn the store exists, and the store # must not know who reads it. - name = "a store deployed beside the homeserver fetches its registration token"; + name = "a store deployed beside the homeserver fetches its appservice token"; ok = baoWithMatrix.systemd.services ? swarm-bao-matrix-token; } { diff --git a/swarm-secret-client/src/matrix.rs b/swarm-secret-client/src/matrix.rs index 69a130c7..9a0cda5b 100644 --- a/swarm-secret-client/src/matrix.rs +++ b/swarm-secret-client/src/matrix.rs @@ -26,18 +26,25 @@ pub fn account_path(agent: &str, account: &str) -> Result { Ok(format!("{prefix}/matrix/{account}")) } -/// The path holding `hive`'s matrix registration token. +/// The path holding `hive`'s matrix appservice token (`as_token`). /// /// Keyed per **hive**, not per agent, like [`crate::queue::agent_client_path`] /// and unlike [`account_path`] above: one homeserver admits one hive's -/// accounts, so the token that creates them is the hive's. +/// accounts, so the identity that creates them is the hive's. +/// +/// ⚠️ Renamed from `registration-token` along with what it holds: the +/// homeserver no longer accepts a shared registration secret at all, so a +/// value still stored under the old path would be read by nothing. A hive +/// whose store has only the old path falls back to its locally minted +/// token — see `glue-matrix-bao-token.nix` — so the rename degrades rather +/// than breaks, but the store needs a fresh `put` to take effect again. /// /// # Errors /// [`Error::PathSegment`] when `hive` contains anything but `[A-Za-z0-9_-]`, /// which is what keeps one hive's name from addressing another hive's secret. -pub fn registration_token_path(hive: &str) -> Result { +pub fn appservice_token_path(hive: &str) -> Result { let prefix = principal_prefix(Kind::Hive, hive)?; - Ok(format!("{prefix}/matrix/registration-token")) + Ok(format!("{prefix}/matrix/appservice-token")) } /// What an account's path holds: the token, plus the homeserver it belongs to. @@ -80,28 +87,29 @@ mod tests { } #[test] - fn the_registration_token_lands_under_the_hive_prefix_the_grant_covers() { + fn the_appservice_token_lands_under_the_hive_prefix_the_grant_covers() { // Spelled out for the same reason as above, and with a second job here: // the read policy grants `secret/data/swarm/hives//*`, so this // string is what makes the path reachable at all. assert_eq!( - registration_token_path("pr1ma").expect("a plain name is legal"), - "swarm/hives/pr1ma/matrix/registration-token" + appservice_token_path("pr1ma").expect("a plain name is legal"), + "swarm/hives/pr1ma/matrix/appservice-token" ); } #[test] - fn the_registration_token_is_not_a_top_level_namespace() { - // The path it used to hold. `Kind` is a closed set and `matrix` is not - // one of its members, so a path with `matrix` as the second segment is - // outside every grant — which is how it came to 403 on every read. - let p = registration_token_path("pr1ma").expect("legal"); + fn the_appservice_token_is_not_a_top_level_namespace() { + // The path its predecessor used to hold. `Kind` is a closed set and + // `matrix` is not one of its members, so a path with `matrix` as the + // second segment is outside every grant — which is how it came to 403 + // on every read. + let p = appservice_token_path("pr1ma").expect("legal"); assert!(!p.starts_with("swarm/matrix/"), "{p}"); } #[test] fn a_traversal_in_the_hive_name_is_refused() { - let e = registration_token_path("../beta").expect_err("a traversal is not"); + let e = appservice_token_path("../beta").expect_err("a traversal is not"); assert!(matches!(e, Error::PathSegment { kind: "hive", .. }), "{e}"); }