diff --git a/docs/integrations/matrix.md b/docs/integrations/matrix.md index 497a1103..ccde7da6 100644 --- a/docs/integrations/matrix.md +++ b/docs/integrations/matrix.md @@ -90,54 +90,39 @@ by default. Reaching it on 8448 needs either an explicit tuwunel bind to that port OR a reverse-proxy + `.well-known/matrix/server` delegation (the latter lives in `gateway.md::Discovery flow`). -## Provisioning flow (appservice) +## Provisioning flow (registration token) -Registration is closed. Accounts are created by the hive's own -**appservice**: hive-c0re holds the appservice token, agents never see -it, and an agent only ever receives its own `access_token`. +Token-gated registration: hive-c0re holds the token, agents never +see it. The agent only receives the resulting `access_token`. -The appservice has no URL (`url: null` in its registration), so the -homeserver never calls out to it and there is no service to run. What the -registration buys is an identity the homeserver recognises — which is why -no secret has to be equal on both sides of the wire, and why account -creation doesn't depend on registration being open to anyone who learns -a token. - -1. **System activation** mints a 32-byte random hex appservice token (64 - chars) at `/var/lib/hyperhive/matrix-appservice-token` and its - spec-required `hs_token` sibling, mode `0600 root:root`, then renders - the registration to - `/var/lib/hyperhive/matrix-appservice/hyperhive.yaml` (also `0600`). - The tokens are minted only when missing; the registration is - re-rendered every time, because the token file can be overwritten in - place by the swarm secret store and a registration naming a stale - token authenticates nobody. Runs at activation time, before any - container start, because the directory is bind-mounted and - nixos-container refuses to start when a bind source is missing. -2. **Read-only bind-mount** maps that directory into the tuwunel +1. **System activation** writes a 32-byte random hex token (64 + chars) to `services.hyperhive.deploy.matrix.registrationTokenFile` + (`/var/lib/hyperhive/matrix-register-token` by default), mode + `0600 root:root`, before any container start. Idempotent — only + writes when the file is missing or empty; always re-applies 0600 + (normalises any 0640 / world-readable carry-over from + pre-LoadCredential deployments). This runs at activation time + (not first container start) to dodge a race where nspawn creates + an empty file when the bind-mount target is missing and tuwunel + reads `registration_token_file=""`, rejecting every registration + until next restart. +2. **Read-only bind-mount** maps the host file into the tuwunel container at the same path. 3. **systemd `LoadCredential=`** inside the container copies the - registration into - `/run/credentials/tuwunel.service/hyperhive-appservice.yaml`, owned by - tuwunel's dynamic user with mode `0400`, at service start. The host - file stays `root:root 0600` — no `chown :tuwunel` / `chmod 0640` / - GID-pin gymnastics required. Keeps `DynamicUser = true` + - `PrivateUsers = true` intact. -4. tuwunel's `appservice_dir` points at the credentials directory, not at - the bind-mount path. It reads only `.yaml`/`.yml` entries from there, - so the sibling credentials are invisible to it. The `.yaml` suffix on - the credential id is what makes this work. -5. **hive-c0re** reads the appservice token and creates each account with - one `POST /register` typed `m.login.application_service`, persisting - the returned `access_token` to `/matrix-token`. It never - mints the token itself: the value has to be the one the rendered - registration names, and only the nix side writes that. -6. **An account that exists but has lost its token file** is re-tokened - by an appservice `POST /login` — no password and no admin rights - involved. A stored-password login and an admin-room password reset - remain behind that, for accounts created before the appservice existed - or named outside its namespace. -7. **hive-c0re restarts `hive-matrix-daemon`** for the agent + bind-mounted file into + `/run/credentials/tuwunel.service/registration_token`, owned by + tuwunel's dynamic user with mode `0400`, at service start. The + host file stays `root:root 0600` — no `chown :tuwunel` / + `chmod 0640` / GID-pin gymnastics required. Keeps + `DynamicUser = true` + `PrivateUsers = true` intact. +4. tuwunel's `registration_token_file` points at the credentials + path, not the original bind-mount path. +5. **hive-c0re** uses the token to register each agent account via + the matrix-spec UIAA registration flow, persists the returned + `access_token` to `/matrix-token`. The agent's + matrix MCP client authenticates with that access_token and + never touches the shared registration token. +6. **hive-c0re restarts `hive-matrix-daemon`** for the agent immediately after writing the token so the daemon picks up the new credential without waiting for a full container restart. If the restart fails (for example daemon not yet running on first boot) @@ -145,74 +130,16 @@ a token. (`hive-matrix-daemon.path` watching for `matrix-token` appearance) brings the daemon up on the same boot cycle anyway. -### The admin account, and why it needs no first-user luck - -`@hive:` is the appservice's own `sender_localpart`, which -the homeserver creates itself when it loads the registration — on a -zero-user database, inside startup, before the HTTP listener accepts -anything. Its **admin rights** then come from an explicit -`make_user_admin`, run by tuwunel's `admin_execute` in the same startup -and likewise before the listener — so a fresh hive has a joined, -power-level-100 admin on its first boot. - -This replaces a dependency on being the first account ever registered, -which was fragile in both directions: an appservice-created account is -excluded from that automatic grant by design, and on a homeserver that -already had users the rule never fired at all. - -Promotion can't be bootstrapped over the API, and that's upstream's -design rather than a gap: tuwunel only treats an admin-room message as a -command when its sender is already an admin. `admin_execute` is the one -lever with no sender to check. hive-c0re re-checks the result on every -sweep by reading the admin account's own joined-rooms list; if the rights -are missing it says so, names -`systemctl restart container@hive-matrix` as the fix, and carries on — -agent accounts, the hive Space and the chat room need no admin. - -
Upgrading a hive that used the registration token - -Nothing to do, and nothing to time. The activation script mints the -appservice token and renders the registration before the homeserver -restarts, so the first boot after the switch already has both halves. - -- **Existing accounts keep working.** An access token lives on the - device that minted it; removing the registration token touches no - device, no account and no session. `login_with_password` stays on, so - the password fallback is still there too. -- **Existing token files are honoured.** The per-agent sweep skips any - agent that already has a `matrix-token`, so no account is re-registered - and no session is displaced. -- **The admin account is already admin** on such a hive (it won the - first-user grant when the hive was new), so the startup promotion is a - no-op — upstream's `make_user_admin` short-circuits when the user is - already joined at power level 100. -- **`/var/lib/hyperhive/matrix-register-token` is left on disk**, read by - nothing. Delete it or leave it; neither does any harm. -- **`registrationTokenFile` is a removed option.** A config that still - sets it fails to evaluate with a message naming the appservice — a hive - that never set it (the default) is unaffected. -- **A swarm store holding the old `matrix/registration-token` path** is - no longer read at all. The value that matters now lives at - `matrix/appservice-token`, and `swarm-secret-publish` on the authelia - host mints and `put`s it there — the hive uses its locally minted - token only until the first successful read. See - [`../swarm/secrets.md`](../swarm/secrets.md) for how that mint stays - idempotent across runs. - -
- Initial rollout settings: - `allow_federation = true` at the protocol level so swarms can be wired up later by extending `trustedServers` without a homeserver restart. `trusted_servers = []` keeps it effectively closed until you list peers. -- `allow_registration = false`. tuwunel checks this flag only for - requests that arrive **without** an appservice token, so hive-c0re - provisions exactly as before and everyone else is refused. It's not a - hardening afterthought: with no registration token configured, - `allow_registration = true` makes tuwunel refuse to start unless - `yes_i_am_very_very_sure_…_open_registration_…` is also set. +- `allow_registration = true` (required for the token flow to + engage). The absent + `yes_i_am_very_very_sure_…_open_registration_…` flag keeps the + server closed to anyone without the token. - `allow_encryption` — server-side E2EE switch, sourced from `services.hyperhive.swarm.matrix.allowEncryption` (**default `false`**, opt-in). Off by default because on the hive-internal homeserver the operator diff --git a/docs/swarm/secrets.md b/docs/swarm/secrets.md index 7c17abe4..24481b57 100644 --- a/docs/swarm/secrets.md +++ b/docs/swarm/secrets.md @@ -132,62 +132,32 @@ when the system builds. The server names the offending file and refuses to run. ## Hive-level — one of each per hive -| secret | generated by | lives at | -| ---------------------------------------- | ------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------- | -| hive CA cert + key | `hive-tls.nix` first-boot unit | `/ca.pem`, `ca-key.pem` (`0600`) | -| hive leaf certs | `hive-tls.nix`, signed by the hive CA | `/.pem` | -| matrix appservice token | `swarm-secret-publish`, published to the store; a host activation script only as a first-boot fallback | `/var/lib/hyperhive/matrix-appservice-token` (`0600`) | -| the forge's copy of its OIDC secret | `hive-forge-oidc-secret.service` copies it from authelia's tree | `/var/lib/forgejo-oidc/.secret` inside the forge container | -| the homeserver's copy of its OIDC secret | `hive-matrix-oidc-secret.service`, same shape | `/var/lib/tuwunel-oidc/.secret`, handed to tuwunel through `LoadCredential` | -| the agent containers' queue credential | authelia, published to the store by `swarm-secret-publish` | `/secret` (`0600`) and `/client_id` (`0644`) | +| secret | generated by | lives at | +| ---------------------------------------- | --------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | +| hive CA cert + key | `hive-tls.nix` first-boot unit | `/ca.pem`, `ca-key.pem` (`0600`) | +| hive leaf certs | `hive-tls.nix`, signed by the hive CA | `/.pem` | +| matrix registration token | a host activation script, on first boot | `/var/lib/hyperhive/matrix-register-token` (`0600`) | +| the forge's copy of its OIDC secret | `hive-forge-oidc-secret.service` copies it from authelia's tree | `/var/lib/forgejo-oidc/.secret` inside the forge container | +| the homeserver's copy of its OIDC secret | `hive-matrix-oidc-secret.service`, same shape | `/var/lib/tuwunel-oidc/.secret`, handed to tuwunel through `LoadCredential` | +| the agent containers' queue credential | authelia, published to the store by `swarm-secret-publish` | `/secret` (`0600`) and `/client_id` (`0644`) | Both delivery units wait for authelia's first boot to mint the secret — a bounded wait, 120s — and then **fail loudly** rather than skipping. A silent skip produces a service whose login button always fails, which is a symptom many layers from its cause. -The store's **first reader** is the matrix appservice token, and it's worth +The store's **first reader** is the matrix registration token, and it's worth saying why that one: it's an opaque 32-byte value with no second file and no format. Authelia's OIDC secret needs a `.secret` _and_ a matching `.digest`, so starting there would have meant debugging "can a reader authenticate and get bytes back" and "is authelia's file format right" at once, with an SSO outage as the failure mode. -`glue-matrix-bao-token.nix` fetches it, writes the file `hive-matrix.nix` -already reads, and then runs that module's own renderer to re-stamp the -appservice registration naming the token — so the homeserver never learns the -store exists. The re-render isn't housekeeping: the token is half an -agreement, and a registration carrying the previous value authenticates -nobody. Every failure path — no such key, sealed store, unreachable store, -empty value — leaves the locally minted token in place, so a hive with no store -behaves exactly as it did before. - -The store path is `swarm/hives//matrix/appservice-token`, and the -**producer is `swarm-secret-publish`** — the same unit that copies authelia's -OIDC secrets in. It mints one token per hive in the swarm's roster and `put`s -it there, so the store is the source of truth and every hive converges on the -value it holds. The hive's own activation mint is still there, but it's a -first-boot fallback now: it fires only when the token file is absent, and the -reader overwrites whatever it produced. - -The mint is **idempotent by keeping its own record**, not by asking the store. -The publisher's grant is `create`/`update` under `swarm/hives/*` with no -`read` — write-only on purpose, so a file-copier can't recover every hive's -credentials — which means it can't check whether a hive already has a token. -Instead it keeps the value it minted under its `StateDirectory`, -`/var/lib/swarm-secret-publish/matrix-appservice-token/` (`0700` dir, -`0600` file), and mints only when that file is missing or empty. The `put` -itself runs every time: re-putting the same bytes changes nothing for any -reader, while a mint whose publish failed must not be left as a token this -host holds and no hive can reach. - -Lose the state directory and the next run mints once more and republishes. -That rotates the token, which readers pick up on their next start — nothing -that already registered breaks, because the token authenticates the -appservice rather than living inside any account it created. - -The path was `…/matrix/registration-token` while the homeserver still took a -shared registration secret; a value left at the old path is read by nothing. +`glue-matrix-bao-token.nix` fetches it and writes the file `hive-matrix.nix` +already reads, so the homeserver never learns the store exists. Every failure +path — no such key, sealed store, unreachable store, empty value — leaves the +locally minted token in place, so a hive with no store behaves exactly as it +did before. The **second reader** is the agent containers' queue credential: `glue-queue-agent-credential.nix` lands it as two files, the client secret and diff --git a/docs/swarm/sso.md b/docs/swarm/sso.md index 4054d120..cb38d00a 100644 --- a/docs/swarm/sso.md +++ b/docs/swarm/sso.md @@ -168,9 +168,9 @@ Two consequences worth stating plainly: - **tuwunel re-reads its secret file on every OAuth exchange**, not only at startup, and its own sandboxing hides most paths from it. It gets the - file through `LoadCredential` for the same reason the appservice - registration does — that keeps `DynamicUser` and `PrivateUsers` intact, - with no host-side ownership arrangement to maintain. + file through `LoadCredential` for the same reason the registration token + does — that keeps `DynamicUser` and `PrivateUsers` intact, with no + host-side ownership arrangement to maintain. - **Matrix SSO lives inside the homeserver.** The client-server API is spoken by non-browser clients holding matrix access tokens — every agent's own daemon — as well as by federation, so the homeserver diff --git a/hive-c0re/src/matrix.rs b/hive-c0re/src/matrix.rs index b4c09558..720c851d 100644 --- a/hive-c0re/src/matrix.rs +++ b/hive-c0re/src/matrix.rs @@ -1,19 +1,12 @@ -//! Optional matrix-tuwunel wiring: the hive's appservice identity (host) + -//! per-agent account creation → `/matrix-token`. No-op +//! Optional matrix-tuwunel wiring: shared registration token (host) + +//! per-agent UIAA registration → `/matrix-token`. No-op //! when the `hive-matrix` container isn't running, so operators who //! haven't flipped `services.hyperhive.deploy.matrix.enable = true` pay //! nothing. //! -//! Accounts are created **as the hive's appservice**, not by presenting a -//! shared registration token in a UIAA flow. The difference that matters -//! here is not the round-trip count: an appservice token is an *identity* -//! the homeserver knows, so the secret never has to be the same on both -//! sides of the wire, and account creation does not depend on registration -//! being open to anyone who learns a token. -//! -//! See `docs/integrations/matrix.md::Provisioning flow (appservice)` for the -//! registration file's shape, how its token reaches both halves, and the -//! host/container bind-mount layout. +//! See `docs/integrations/matrix.md::Provisioning flow (registration token)` for +//! the full UIAA round-trip, token-file shape, and host/container +//! bind-mount layout. use std::path::PathBuf; @@ -56,8 +49,11 @@ fn matrix_base() -> Result<&'static str> { this path should have been gated on matrix::is_present()", ) } -/// HTTP timeout for registration round-trips. Account creation is one -/// POST; even the slow path should finish well inside this budget. +/// Length (bytes) of the random registration token. 32 raw bytes ⇒ +/// 64-char hex string; comfortable for a long-lived shared secret. +const REGISTER_TOKEN_BYTES: usize = 32; +/// HTTP timeout for registration round-trips. UIAA is two POSTs; even +/// the slow path should finish well inside this budget. const HTTP_TIMEOUT_SECS: u64 = 10; /// Length (bytes) of the throwaway per-agent matrix password. Random /// 32-byte hex — agents never log in with the password (they @@ -65,17 +61,10 @@ const HTTP_TIMEOUT_SECS: u64 = 10; /// store it nowhere. const PASSWORD_BYTES: usize = 32; -/// Matrix localpart for the hive system admin account. Not an agent; has -/// no state dir. -/// -/// Also the `sender_localpart` of the hive's appservice registration, -/// which is what creates this account on a homeserver that has never had -/// one: the homeserver creates a registration's sender user itself, at -/// startup, before it accepts a request. The account's *admin rights* -/// then come from the `admin_execute` promotion beside that registration -/// — see `nix/host-modules/hive-matrix.nix`, where this same literal -/// appears as `adminLocalpart`. **The two must match**; nothing wires an -/// override across. +/// Matrix localpart for the hive system admin account. Registered +/// before any agent account in [`ensure_all`] so it becomes the first +/// user on the homeserver — Conduit/tuwunel grants admin rights to the +/// first registered user automatically. Not an agent; has no state dir. pub const HIVE_ADMIN_LOCALPART: &str = "hive"; /// Display name of the hive Space. Plain text, no special characters, so @@ -175,36 +164,30 @@ fn random_hex(n: usize) -> Result { Ok(hex) } -/// Read the hive's appservice token — the `as_token` of the registration -/// the homeserver loaded at boot. Every account this module creates is -/// authorised by it. -/// -/// **Reads, never mints**, unlike the registration token it replaced. -/// That token was the whole agreement, so whichever side wrote it first -/// was right; this one has a second half — the registration file naming -/// it, which only the nix side writes. A token minted here would be a -/// token the homeserver has never heard of, and the failure would surface -/// as every request being refused rather than as a missing file. -/// -/// # Errors -/// When the file is absent or empty. That means the host activation -/// script has not run on this generation yet; callers log it and leave -/// existing accounts alone rather than trying to proceed. -pub fn read_appservice_token() -> Result { - let path = crate::paths::matrix_appservice_token(); - std::fs::read_to_string(&path) - .ok() - .map(|s| s.trim().to_owned()) - .filter(|s| !s.is_empty()) - .with_context(|| { - format!( - "matrix appservice token not found at {} — it is minted by the \ - hive-matrix activation script, which also renders the registration \ - file naming it; deploy the hive-matrix module (or re-run \ - `nixos-rebuild switch`) before provisioning matrix users", - path.display() - ) - }) +/// Ensure the registration token file exists; returns its contents. +/// Generates a fresh 32-byte hex token on first call (mode 0600, +/// root-only), then re-reads on subsequent calls. The hive-matrix +/// nixos module bind-mounts this file read-only into the tuwunel +/// container so the homeserver can authenticate registration requests +/// against the same secret hive-c0re holds. +pub fn ensure_register_token() -> Result { + use std::os::unix::fs::PermissionsExt; + let path = crate::paths::matrix_register_token(); + if let Ok(existing) = std::fs::read_to_string(&path) { + let trimmed = existing.trim().to_owned(); + if !trimmed.is_empty() { + return Ok(trimmed); + } + } + let token = random_hex(REGISTER_TOKEN_BYTES)?; + if let Some(parent) = path.parent() { + std::fs::create_dir_all(parent).ok(); + } + std::fs::write(&path, format!("{token}\n")) + .with_context(|| format!("write registration token to {}", path.display()))?; + let _ = std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600)); + tracing::info!(path = %path.display(), "matrix: generated registration token"); + Ok(token) } /// Build the localpart of a matrix user id for `agent`. Matrix @@ -215,20 +198,19 @@ fn user_localpart(agent: &str) -> &str { agent } -/// Send the registration POST as the appservice and parse the response. -/// Returns `Ok((status, body))` on any completed HTTP round-trip -/// (including the `M_USER_IN_USE` 400 the caller treats as "already -/// exists"); errors only on transport failure. +/// Send a single registration POST and parse the response. Returns +/// `Ok((status, body))` on any successful HTTP round-trip (including +/// the expected 401 from the first UIAA leg); errors only on transport +/// failure. Body is the parsed JSON value — UIAA passes session + +/// flow state via JSON, never via headers. async fn register_post( client: &reqwest::Client, - as_token: &str, body: &serde_json::Value, ) -> Result<(StatusCode, serde_json::Value)> { let base = matrix_base()?; let url = format!("{base}/_matrix/client/v3/register"); let resp = client .post(&url) - .bearer_auth(as_token) .json(body) .send() .await @@ -251,38 +233,24 @@ pub fn random_password() -> Result { random_hex(PASSWORD_BYTES) } -/// Create the matrix account for `agent` as the hive's appservice and -/// return an access token for it. One round-trip: an appservice-typed -/// registration needs no UIAA stage at all, so there is no session to -/// carry and no shared secret to present. -/// -/// The account is created **by** the appservice but is an ordinary user -/// afterwards — it gets its own device and its own access token, and the -/// agent authenticates with that rather than with anything the hive -/// holds. The `as_token` never leaves the host. +/// Run the matrix-spec UIAA flow to register `agent` with the given +/// `password` and return the resulting access token. Two round-trips: +/// first POST elicits the 401 + session id, second POST supplies the +/// registration token in the `auth` block. If the homeserver returns +/// 200 on the first POST (no flow stages required — `allow_registration` +/// with no token), we take the access token directly. /// /// Caller picks the password: agents use [`random_password`] (throwaway /// — they auth by `access_token`), operators on the `hivectl` path /// supply their own so they can log into matrix web clients. -/// -/// # Errors -/// Propagates the homeserver's own body, which is what the -/// `M_USER_IN_USE` callers match on. A `M_EXCLUSIVE` body means the -/// localpart falls outside the appservice's namespace — the registration -/// file's `namespaces.users` regex is the place to look, not this call. async fn register_user( client: &reqwest::Client, agent: &str, - as_token: &str, + register_token: &str, password: &str, ) -> Result { let localpart = user_localpart(agent); - let body = serde_json::json!({ - // What makes this an appservice registration rather than an - // ordinary one. Without it the homeserver treats the request as a - // normal client's and asks for a UIAA flow — even holding the - // as_token. - "type": "m.login.application_service", + let initial = serde_json::json!({ "username": localpart, "password": password, // device_id stays stable across re-runs of ensure_user_for so @@ -291,51 +259,35 @@ async fn register_user( "initial_device_display_name": format!("hyperhive ({agent})"), "inhibit_login": false, }); - let (status, body) = register_post(client, as_token, &body).await?; - if !status.is_success() { - anyhow::bail!("matrix: /register as appservice HTTP {status}, body: {body}"); + let (status, body) = register_post(client, &initial).await?; + // Some servers accept the first POST when allow_registration is + // open. Take the access_token + we're done. + if status.is_success() { + return extract_access_token(&body); } - extract_access_token(&body) -} - -/// Log in as an **existing** account using the hive's appservice token, -/// and return a fresh access token for it. No password involved: the -/// appservice is authorised for every localpart in its namespace, so it -/// can mint a session for one without knowing anything about the account. -/// -/// This is the recovery path that used to need a stored password or an -/// admin-room password reset — an account whose token file was lost is -/// re-tokened from the hive's own identity instead. The device id matches -/// [`register_user`]'s, so a re-login replaces that device's token rather -/// than accumulating devices. -async fn appservice_login(client: &reqwest::Client, as_token: &str, agent: &str) -> Result { - let base = matrix_base()?; - let url = format!("{base}/_matrix/client/v3/login"); - let body = serde_json::json!({ - "type": "m.login.application_service", - "identifier": { - "type": "m.id.user", - "user": user_localpart(agent), - }, + if status != StatusCode::UNAUTHORIZED { + anyhow::bail!("matrix: /register first leg HTTP {status}, body: {body}"); + } + let session = body["session"] + .as_str() + .with_context(|| format!("matrix: missing UIAA session in 401, body: {body}"))?; + let authed = serde_json::json!({ + "username": localpart, + "password": password, "device_id": format!("hyperhive-{agent}"), "initial_device_display_name": format!("hyperhive ({agent})"), + "inhibit_login": false, + "auth": { + "type": "m.login.registration_token", + "token": register_token, + "session": session, + }, }); - let resp = client - .post(&url) - .bearer_auth(as_token) - .json(&body) - .send() - .await - .context("matrix: POST /login as appservice")?; - let status = resp.status(); - let json = resp - .json::() - .await - .context("matrix: parse appservice /login response")?; + let (status, body) = register_post(client, &authed).await?; if !status.is_success() { - anyhow::bail!("matrix: appservice /login HTTP {status} for {agent}, body: {json}"); + anyhow::bail!("matrix: /register auth leg HTTP {status}, body: {body}"); } - extract_access_token(&json) + extract_access_token(&body) } /// Pull `access_token` out of a successful /register response. @@ -685,7 +637,11 @@ async fn admin_room_reset_password( /// /// `client` is shared across the sweep so we build one reqwest /// connection pool for all agents rather than one per call. -pub async fn ensure_user_for(client: &reqwest::Client, name: &str, as_token: &str) -> Result<()> { +pub async fn ensure_user_for( + client: &reqwest::Client, + name: &str, + register_token: &str, +) -> Result<()> { use std::os::unix::fs::PermissionsExt; let agent = hive_types::Ident::parse(name) .map_err(|e| anyhow::anyhow!("invalid agent name {name:?}: {e}"))?; @@ -722,7 +678,7 @@ pub async fn ensure_user_for(client: &reqwest::Client, name: &str, as_token: &st } let password = random_password()?; - let access_token = match register_user(client, name, as_token, &password).await { + let access_token = match register_user(client, name, register_token, &password).await { Ok(token) => { // Successful registration — persist the password so we can // fall back to login if the token file is deleted later. @@ -738,21 +694,8 @@ pub async fn ensure_user_for(client: &reqwest::Client, name: &str, as_token: &st token } Err(reg_err) if reg_err.to_string().contains("M_USER_IN_USE") => { - // Account already exists and its token file was lost. The - // appservice can mint a session for any account in its - // namespace, so this needs no password and no admin rights — - // try it before the password paths below, which exist for - // accounts created before the appservice did (or named - // outside its namespace) and stay as the fallback. - tracing::info!(%name, "matrix: user already exists, logging in as the appservice"); - match appservice_login(client, as_token, name).await { - Ok(token) => return finish_user_provisioning(name, &token).await, - Err(e) => tracing::warn!( - %name, - error = ?e, - "matrix: appservice login failed; falling back to the stored password" - ), - } + // Account already exists — try to re-login with the stored password. + tracing::info!(%name, "matrix: user already exists, attempting login with stored password"); let pw_path = password_path(name); let stored = if let Some(pw) = std::fs::read_to_string(&pw_path) .ok() @@ -791,19 +734,11 @@ pub async fn ensure_user_for(client: &reqwest::Client, name: &str, as_token: &st Err(other) => return Err(other), }; - finish_user_provisioning(name, &access_token).await -} - -/// Persist a freshly-obtained agent access token and kick the agent's -/// matrix daemon. Shared by every way [`ensure_user_for`] can end up -/// holding a token — creation, appservice login, password login — so a -/// new recovery path cannot forget half of it. -async fn finish_user_provisioning(name: &str, access_token: &str) -> Result<()> { // Write the token via hive-priv (root helper): hive-c0re runs as the // unprivileged `hive-core` user and cannot write to agent-owned state // directories directly. hive-priv writes the file 0600 and chowns it // to the agent user so it is readable from inside the container. - crate::priv_client::write_agent_matrix_token(name, access_token, None, None) + crate::priv_client::write_agent_matrix_token(name, &access_token, None, None) .await .with_context(|| format!("matrix: write matrix-token for {name} via hive-priv"))?; tracing::info!(%name, "matrix: provisioned access token"); @@ -832,30 +767,30 @@ async fn finish_user_provisioning(name: &str, access_token: &str) -> Result<()> /// behaviour. /// /// **Not idempotent** (unlike [`crate::forge::provision_user_token`]): the -/// matrix `/register` endpoint returns `M_USER_IN_USE` (HTTP 400) on a -/// second call for the same localpart, appservice-authorised or not. -/// Callers re-running this for a known-existing matrix user should expect -/// a hard error from this fn and route to a password-reset path instead. +/// matrix UIAA `/register` endpoint returns `M_USER_IN_USE` (HTTP 400) +/// on second call for the same localpart. Callers re-running this for +/// a known-existing matrix user should expect a hard error from this +/// fn and route to a password-reset path instead. pub async fn provision_user_token( client: &reqwest::Client, name: &str, - as_token: &str, + register_token: &str, password: &str, ) -> Result { - register_user(client, name, as_token, password).await + register_user(client, name, register_token, password).await } /// Per-agent matrix sync: ensure the agent has a matrix account + token. /// All operations are idempotent; failures are logged as warnings but /// don't abort the caller. -pub async fn sync_agent(client: &reqwest::Client, name: &str, as_token: &str) { - if let Err(e) = ensure_user_for(client, name, as_token).await { +pub async fn sync_agent(client: &reqwest::Client, name: &str, register_token: &str) { + if let Err(e) = ensure_user_for(client, name, register_token).await { tracing::warn!(%name, error = ?e, "matrix: ensure_user failed"); } } /// Standalone per-agent sync that handles its own setup: checks if -/// hive-matrix is present, reads the appservice token, and builds +/// hive-matrix is present, reads the registration token, and builds /// an HTTP client before delegating to [`sync_agent`]. Mirrors the /// setup in [`ensure_all`] so the rebuild path and the startup sweep /// stay equivalent. No-op when the matrix container is absent. @@ -863,10 +798,10 @@ pub async fn sync_agent_standalone(name: &str) { if !is_present() { return; } - let as_token = match read_appservice_token() { + let register_token = match ensure_register_token() { Ok(t) => t, Err(e) => { - tracing::warn!(%name, error = ?e, "matrix: read_appservice_token failed"); + tracing::warn!(%name, error = ?e, "matrix: ensure_register_token failed"); return; } }; @@ -880,26 +815,22 @@ pub async fn sync_agent_standalone(name: &str) { return; } }; - sync_agent(&client, name, &as_token).await; + sync_agent(&client, name, ®ister_token).await; } -/// Ensure the hive system admin matrix user exists, that its token is -/// persisted at [`admin_token_path()`], and that it actually holds admin -/// rights. +/// Ensure the hive system admin matrix user exists and its token is +/// persisted at [`admin_token_path()`]. Must be called BEFORE +/// [`ensure_all`]'s agent loop so this account is the first to register +/// and becomes the homeserver admin automatically (Conduit/tuwunel: +/// first registered user = admin). /// -/// **Nothing here depends on registration order any more.** The account -/// used to have to be the first ever registered, to win tuwunel's -/// automatic first-user grant — a rule that cannot fire for an -/// appservice-created account at all, and one that silently did nothing -/// on a homeserver that already had users. Admin rights now come from an -/// explicit `make_user_admin`: the `admin_execute` entry beside the -/// appservice registration performs it at homeserver startup, and -/// [`ensure_admin_rights`] checks the result and says so when it is -/// missing. -/// -/// Idempotent — skips the account work when the token file already exists -/// and is non-empty. -pub async fn ensure_admin_user(client: &reqwest::Client, as_token: &str) -> Result<()> { +/// Idempotent — skips when the token file already exists and is +/// non-empty. Does NOT promote the account via API (that requires +/// admin rights which this fn bootstraps); on a fresh homeserver the +/// first-registered rule fires automatically; on an existing homeserver +/// the operator must promote the account once via +/// `hivectl matrix promote-user hive` or the conduit admin room. +pub async fn ensure_admin_user(client: &reqwest::Client, register_token: &str) -> Result<()> { use std::os::unix::fs::PermissionsExt; let path = admin_token_path(); if path.exists() @@ -910,7 +841,8 @@ pub async fn ensure_admin_user(client: &reqwest::Client, as_token: &str) -> Resu return Ok(()); } let password = random_password()?; - let access_token = match register_user(client, HIVE_ADMIN_LOCALPART, as_token, &password).await + let access_token = match register_user(client, HIVE_ADMIN_LOCALPART, register_token, &password) + .await { Ok(token) => { let pw_path = password_path(HIVE_ADMIN_LOCALPART); @@ -925,34 +857,20 @@ pub async fn ensure_admin_user(client: &reqwest::Client, as_token: &str) -> Resu token } Err(reg_err) if reg_err.to_string().contains("M_USER_IN_USE") => { - // The expected path, not an edge case: this account is the - // appservice's own `sender_localpart`, so the homeserver - // creates it when it loads the registration — before - // hive-c0re gets a chance to ask. An appservice login needs - // no password, which is just as well since an account the - // homeserver created has none. - tracing::info!("matrix: hive admin user already exists, logging in as the appservice"); - match appservice_login(client, as_token, HIVE_ADMIN_LOCALPART).await { - Ok(token) => token, - Err(e) => { - tracing::warn!(error = ?e, "matrix: appservice login for the hive admin failed; falling back to the stored password"); - let pw_path = password_path(HIVE_ADMIN_LOCALPART); - let stored = std::fs::read_to_string(&pw_path) - .ok() - .map(|s| s.trim().to_owned()) - .filter(|s| !s.is_empty()) - .with_context(|| { - format!( - "matrix: hive admin user exists, appservice login failed, and no \ - password is stored at {} — check that the registration file's \ - namespace covers @{HIVE_ADMIN_LOCALPART} and that the homeserver \ - loaded it", - pw_path.display() - ) - })?; - login_user(client, HIVE_ADMIN_LOCALPART, &stored).await? - } - } + tracing::info!("matrix: hive admin user already exists, re-logging in"); + let pw_path = password_path(HIVE_ADMIN_LOCALPART); + let stored = std::fs::read_to_string(&pw_path) + .ok() + .map(|s| s.trim().to_owned()) + .filter(|s| !s.is_empty()) + .with_context(|| { + format!( + "matrix: hive admin user exists but password missing at {} — \ + manual recovery: reset password via admin API or conduit admin room", + pw_path.display() + ) + })?; + login_user(client, HIVE_ADMIN_LOCALPART, &stored).await? } Err(other) => return Err(other), }; @@ -966,149 +884,9 @@ pub async fn ensure_admin_user(client: &reqwest::Client, as_token: &str) -> Resu Ok(()) } -/// Check that the hive admin account holds admin rights, and repair it -/// through the admin room when it does not. -/// -/// Admin-ness in tuwunel is membership of the admin room, so that is what -/// this reads: the account's own joined-rooms list, which needs no -/// privileges to fetch. When the admin room is in it there is nothing to -/// do and nothing is sent — worth insisting on, because the repair is a -/// message in a room and this runs on every sweep. -/// -/// When it is absent, the repair is attempted anyway (a stale or failed -/// read is cheaper to retry than to reason about) and a failure is -/// reported rather than raised: agent accounts, the hive Space and the -/// chat room all work without an admin, so a hive with an unpromoted -/// admin is degraded, not broken. Only `hivectl matrix promote-user` / -/// `reset-password` need it. -/// -/// Returns whether the account holds admin rights now. A `false` has -/// already been logged, with what to do about it. -async fn ensure_admin_rights( - client: &reqwest::Client, - admin_token: &str, - server_name: &str, -) -> bool { - let room_id = match discover_admin_room_id(client, admin_token, server_name).await { - Ok(id) => id, - Err(e) => { - tracing::warn!(error = ?e, "matrix: cannot resolve #admins — not checking the hive admin's rights"); - return false; - } - }; - match joined_rooms(client, admin_token).await { - Some(rooms) if rooms.iter().any(|r| r == &room_id) => { - tracing::debug!("matrix: hive admin is in the admin room"); - return true; - } - Some(_) => { - tracing::warn!( - "matrix: hive admin is not in the admin room — attempting to promote it" - ); - } - None => tracing::debug!( - "matrix: could not read the hive admin's joined rooms; attempting to promote it" - ), - } - if let Err(e) = - promote_user_to_admin(client, admin_token, HIVE_ADMIN_LOCALPART, server_name).await - { - // The honest message. Promoting through the admin room requires - // being admin already, so the one lever that works on a - // homeserver with no admin at all is the `admin_execute` entry in - // the hive-matrix module — which runs at startup, which means a - // restart is the fix rather than another sweep. - tracing::warn!( - error = ?e, - "matrix: hive admin '@{HIVE_ADMIN_LOCALPART}' holds no admin rights. \ - The homeserver promotes it at startup (admin_execute in hive-matrix.nix), \ - so `systemctl restart container@hive-matrix` grants it. Agent accounts and \ - room provisioning are unaffected; `hivectl matrix promote-user` and \ - `reset-password` need it." - ); - return false; - } - tracing::info!("matrix: promoted the hive admin through the admin room"); - true -} - -/// The rooms `token`'s account has joined, or `None` when the list could -/// not be read. `None` is deliberately not an empty list: "no rooms" and -/// "no answer" lead to different decisions in [`ensure_admin_rights`]. -async fn joined_rooms(client: &reqwest::Client, token: &str) -> Option> { - let base = matrix_http()?; - let url = format!("{base}/_matrix/client/v3/joined_rooms"); - let resp = client.get(&url).bearer_auth(token).send().await.ok()?; - if !resp.status().is_success() { - return None; - } - let body = resp.json::().await.ok()?; - Some( - body["joined_rooms"] - .as_array()? - .iter() - .filter_map(|r| r.as_str().map(ToOwned::to_owned)) - .collect(), - ) -} - -/// Whether an admin-room reply says a `make-user-admin` succeeded. -/// -/// Three spellings, because the reply is prose and prose changes between -/// builds. tuwunel v1.9.0's is `" has been granted admin -/// privileges."` — which the original two patterns here (`done…`, -/// `made…admin`) do not match at all, so a promotion that had already -/// worked was reported as a 15-second timeout. The older spellings are -/// kept: a homeserver is not necessarily the version this was written -/// against. -fn is_make_admin_success(body: &str) -> Option<()> { - let lower = body.to_ascii_lowercase(); - let says_ok = lower.starts_with("done") - || lower.contains("granted admin privileges") - || (lower.contains("made") && lower.contains("admin")); - says_ok.then_some(()) -} - -#[cfg(test)] -mod is_make_admin_success_tests { - use super::is_make_admin_success; - - /// The reply tuwunel v1.9.0 actually sends - /// (`src/admin/user/make_user_admin.rs`). This is the case the - /// pre-existing matcher missed. - #[test] - fn tuwunel_1_9_grant_reply() { - let msg = "@hive:pr1ma.darkest.space has been granted admin privileges."; - assert_eq!(is_make_admin_success(msg), Some(())); - } - - #[test] - fn older_spellings_still_match() { - assert_eq!( - is_make_admin_success("Done: user is now an admin"), - Some(()) - ); - assert_eq!(is_make_admin_success("Made @x:y an admin"), Some(())); - } - - /// The control: an unrelated or failing reply must not read as - /// success, or a failed promotion returns Ok and the warning that - /// would have named it never fires. - #[test] - fn failures_and_noise_do_not_match() { - assert_eq!(is_make_admin_success("Command not recognised."), None); - assert_eq!(is_make_admin_success("User @x:y does not exist"), None); - } -} - /// Promote a user to homeserver admin via the Matrix admin room /// (`#admins:`). Sends `!admin users make-user-admin @:` as -/// @hive, polls for the bot's success reply. -/// -/// ⚠️ Requires the **sender** to be an admin already — tuwunel only -/// treats a message as a command when its sender is in the admin room. -/// So this promotes a *second* user; it cannot bootstrap the first one. -/// That is what the `admin_execute` entry in `hive-matrix.nix` is for. +/// @hive, polls for the bot's "Done:" success response. /// /// Goes through the admin room rather than a direct HTTP call because /// tuwunel implements parts of the Synapse admin API but not user @@ -1129,7 +907,14 @@ pub async fn promote_user_to_admin( server_name, &room_url, &command, - is_make_admin_success, + |body| { + let lower = body.to_ascii_lowercase(); + if lower.starts_with("done") || lower.contains("made") && lower.contains("admin") { + Some(()) + } else { + None + } + }, ) .await .with_context(|| { @@ -1852,14 +1637,10 @@ pub async fn ensure_all() -> bool { return true; } let mut ok = true; - // Loud and non-destructive: with no appservice token this sweep can - // create nothing, so it does nothing. Agents that already hold a - // token keep using it — their accounts and sessions are untouched by - // anything in here. - let as_token = match read_appservice_token() { + let register_token = match ensure_register_token() { Ok(t) => t, Err(e) => { - tracing::warn!(error = ?e, "matrix: no appservice token; skipping the user sweep"); + tracing::warn!(error = ?e, "matrix: ensure_register_token failed"); return false; } }; @@ -1875,12 +1656,10 @@ pub async fn ensure_all() -> bool { return false; } }; - // The hive admin first, because everything below provisions THROUGH - // it (the Space, the chat room and every invite are sent with its - // token). Not, any more, so that it wins a first-registered-user - // grant: it holds admin rights by explicit promotion, checked in - // `provision_space` once the server name is known. - if let Err(e) = ensure_admin_user(&client, &as_token).await { + // Provision hive admin user FIRST so it's the first registered + // account on a fresh homeserver (Conduit/tuwunel makes the first + // registered user admin automatically). + if let Err(e) = ensure_admin_user(&client, ®ister_token).await { tracing::warn!(error = ?e, "matrix: ensure_admin_user failed"); ok = false; } @@ -1893,7 +1672,7 @@ pub async fn ensure_all() -> bool { let Some(name) = c.strip_prefix(crate::lifecycle::AGENT_PREFIX) else { continue; }; - sync_agent(&client, name, &as_token).await; + sync_agent(&client, name, ®ister_token).await; agent_names.push(name.to_owned()); } @@ -1926,11 +1705,6 @@ async fn provision_space(client: &reqwest::Client, agent_names: &[String]) -> bo return false; } }; - // Before the rooms: the one thing in this sweep that is about the - // admin account itself rather than about what it provisions. - if !ensure_admin_rights(client, &admin_token, &server_name).await { - ok = false; - } let room_id = match ensure_hive_space(client, &admin_token).await { Ok(id) => id, Err(e) => { diff --git a/hive-c0re/src/paths.rs b/hive-c0re/src/paths.rs index dcf16ef4..759f7eae 100644 --- a/hive-c0re/src/paths.rs +++ b/hive-c0re/src/paths.rs @@ -253,15 +253,14 @@ pub fn gateway_agents_conf() -> PathBuf { // `nix/host-modules/hive-c0re/default.nix` and `nix/host-modules/hive-ci.nix` — must match. pub const FORGE_CORE_TOKEN: &str = "/var/lib/hyperhive/forge-core-token"; -/// `matrix-appservice-token` — the `as_token` of the hive's appservice -/// registration, which authorises every account this daemon creates. -// nix: minted by the `hive-matrix-appservice` activation script in -// `nix/host-modules/hive-matrix.nix`, which renders it into the registration -// file the homeserver loads — must match. Read-only here on purpose: a token -// minted on this side would not be the one in that file. +/// `matrix-register-token` — shared matrix registration token. +// nix: bind-mounted into the tuwunel/matrix container (hive-matrix.nix) — must match. +// Not operator-option-driven: `registrationTokenFile` is `internal` on the nix +// side, and an `assertions` entry there rejects any attempt to move it, so this +// literal can never diverge from it. #[must_use] -pub fn matrix_appservice_token() -> PathBuf { - state_root().join("matrix-appservice-token") +pub fn matrix_register_token() -> PathBuf { + state_root().join("matrix-register-token") } /// `/run/hyperhive` — the runtime root (host admin socket + per-agent dirs). diff --git a/hive-c0re/src/server.rs b/hive-c0re/src/server.rs index 547540d4..67af8b76 100644 --- a/hive-c0re/src/server.rs +++ b/hive-c0re/src/server.rs @@ -558,8 +558,8 @@ async fn handle_matrix_create_user( password: Option<&str>, ) -> Result { require_matrix_present()?; - let as_token = - crate::matrix::read_appservice_token().context("read matrix appservice token")?; + let register_token = + crate::matrix::ensure_register_token().context("read matrix register token")?; let client = matrix_http_client()?; let mut out = Vec::new(); if agent_exists(name)? { @@ -571,7 +571,7 @@ async fn handle_matrix_create_user( "matrix create-user: a password is for non-agent (operator) accounts only; '{name}' is an agent which authenticates via access_token" ); } - crate::matrix::ensure_user_for(&client, name.as_str(), &as_token) + crate::matrix::ensure_user_for(&client, name.as_str(), ®ister_token) .await .with_context(|| format!("matrix create-user {name}"))?; let path = Coordinator::agent_notes_dir(name).join("matrix-token"); @@ -585,7 +585,7 @@ async fn handle_matrix_create_user( let token = crate::matrix::provision_user_token( &client, name.as_str(), - &as_token, + ®ister_token, &effective_password, ) .await @@ -770,10 +770,10 @@ async fn handle_push_snapshot( async fn handle_matrix_sync_admin() -> Result { require_matrix_present()?; - let as_token = - crate::matrix::read_appservice_token().context("read matrix appservice token")?; + let register_token = + crate::matrix::ensure_register_token().context("read matrix register token")?; let client = matrix_http_client()?; - crate::matrix::ensure_admin_user(&client, &as_token) + crate::matrix::ensure_admin_user(&client, ®ister_token) .await .context("matrix sync-admin")?; let path = crate::matrix::admin_token_path(); diff --git a/hive-matrix-mcp/src/client.rs b/hive-matrix-mcp/src/client.rs index f77619fe..867ced5d 100644 --- a/hive-matrix-mcp/src/client.rs +++ b/hive-matrix-mcp/src/client.rs @@ -5,10 +5,10 @@ //! to start sync. //! //! No OAuth dance / cross-signing setup (in contrast to damocles-daemon's -//! ccc.de connection): for the in-hive tuwunel hive-c0re already minted -//! the token + user/device as the hive's appservice and handed us the -//! bearer in a file. matrix-sdk's `restore_session` with a constructed -//! `MatrixSession` skips the login flow entirely. +//! ccc.de connection): for the in-hive tuwunel the `registration_token` +//! UIAA flow already minted the token + user/device, hive-c0re just +//! handed us the bearer in a file. matrix-sdk's `restore_session` with +//! a constructed `MatrixSession` skips the login flow entirely. //! //! E2EE is enabled via `with_encryption_settings(EncryptionSettings::default())`. //! Crypto keys are persisted in the sqlite store under `state_dir` diff --git a/nix/host-modules/deploy.nix b/nix/host-modules/deploy.nix index d89a1e7d..0d491143 100644 --- a/nix/host-modules/deploy.nix +++ b/nix/host-modules/deploy.nix @@ -247,29 +247,10 @@ in [ "services" "hyperhive" "swarm" "matrix" "maxRequestSize" ] [ "services" "hyperhive" "deploy" "matrix" "maxRequestSize" ] ) - # Both spellings of the registration token, removed rather than renamed: - # the homeserver no longer accepts a shared registration secret at all, - # so there is no path to point a definition at. Accounts are created by - # the hive's appservice, whose token lives at a fixed path this module - # deliberately does not let anyone move. - (lib.mkRemovedOptionModule [ "services" "hyperhive" "swarm" "matrix" "registrationTokenFile" ] '' - The matrix registration token is gone: hive-c0re creates accounts as the - hive's appservice instead, so there is no shared secret for tuwunel to - check. Remove this definition — nothing needs replacing, the appservice - token is minted and delivered automatically - (/var/lib/hyperhive/matrix-appservice-token, not operator-settable). - The old token file at /var/lib/hyperhive/matrix-register-token is read by - nothing now and can be deleted. See docs/integrations/matrix.md. - '') - (lib.mkRemovedOptionModule [ "services" "hyperhive" "deploy" "matrix" "registrationTokenFile" ] '' - The matrix registration token is gone: hive-c0re creates accounts as the - hive's appservice instead, so there is no shared secret for tuwunel to - check. Remove this definition — nothing needs replacing, the appservice - token is minted and delivered automatically - (/var/lib/hyperhive/matrix-appservice-token, not operator-settable). - The old token file at /var/lib/hyperhive/matrix-register-token is read by - nothing now and can be deleted. See docs/integrations/matrix.md. - '') + (lib.mkRenamedOptionModule + [ "services" "hyperhive" "swarm" "matrix" "registrationTokenFile" ] + [ "services" "hyperhive" "deploy" "matrix" "registrationTokenFile" ] + ) (lib.mkRenamedOptionModule [ "services" "hyperhive" "swarm" "matrix" "gui" "enable" ] [ "services" "hyperhive" "deploy" "matrix" "gui" "enable" ] diff --git a/nix/host-modules/glue-matrix-bao-token.nix b/nix/host-modules/glue-matrix-bao-token.nix index 077c7b86..4f2fe486 100644 --- a/nix/host-modules/glue-matrix-bao-token.nix +++ b/nix/host-modules/glue-matrix-bao-token.nix @@ -1,9 +1,8 @@ -# Glue: the matrix appservice token comes from the secret store. +# Glue: the matrix registration token comes from the secret store. # # The store's first reader, and deliberately a small one. It fetches an opaque -# 32-byte value and writes it where ./hive-matrix.nix already looks, then asks -# that module's own renderer to re-stamp the appservice registration naming it -# — the homeserver never learns the store exists, and its config is unchanged. +# 32-byte value and writes it where ./hive-matrix.nix already looks — the +# homeserver never learns the store exists, and its config is unchanged. # # ⚠️ Why this credential first. It has no second file and no format: authelia's # OIDC secret needs a `.secret` *and* a matching `.digest`, so shipping that @@ -48,12 +47,12 @@ let # own grant covers. The store's read policy grants `swarm/agents/*` and # `swarm/hives//*` and nothing else, so a path outside those is a # 403 rather than a miss, however correct it looks. `swarm-secret-client`'s - # `matrix::appservice_token_path` builds the same string from the same + # `matrix::registration_token_path` builds the same string from the same # pieces; this literal is the nix half of that one agreement. # # `hiveName` has no fallback here for the reason ./glue-bao-tls.nix gives at # its own use of it: it is asserted set for every hyperhive host. - tokenPath = "secret/swarm/hives/${hyperhiveCfg.hiveName}/matrix/appservice-token"; + tokenPath = "secret/swarm/hives/${hyperhiveCfg.hiveName}/matrix/registration-token"; # A literal, not an option — ./hive-matrix.nix names its container # `containers.hive-matrix` directly and declares no `machine` to derive it @@ -71,7 +70,7 @@ in services.hyperhive.swarm.otel.journaldUnits = [ "swarm-bao-matrix-token" ]; systemd.services.swarm-bao-matrix-token = { - description = "fetch the matrix appservice token from the swarm secret store"; + description = "fetch the matrix registration token from the swarm secret store"; # Every one of these names a unit that exists only where the store runs. # `Requires=` on an absent unit fails the job outright, so the ordering is # conditional even though the read is not: off-host there is nothing local @@ -177,20 +176,8 @@ in fi umask 077 - printf '%s\n' "$token" > ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)} - chmod 0600 ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)} - - # Re-stamp the registration file from the token just written. The - # token is half an agreement — the registration the homeserver loads - # has to carry the same value — so writing the file and stopping - # would leave the homeserver authenticating hive-c0re against - # whatever activation put there: a 401 on every request, naming - # nothing. Unconditional rather than on-change, because this unit - # has no way to know what the registration currently says. - # - # hive-matrix's own renderer rather than a `printf` here, so the - # registration's shape has one home. - ${deployCfg.matrix.appserviceRegistrationScript} + printf '%s\n' "$token" > ${lib.escapeShellArg (toString deployCfg.matrix.registrationTokenFile)} + chmod 0600 ${lib.escapeShellArg (toString deployCfg.matrix.registrationTokenFile)} ''; }; }; diff --git a/nix/host-modules/glue-queue-agent-credential.nix b/nix/host-modules/glue-queue-agent-credential.nix index 9e57b512..81e3c350 100644 --- a/nix/host-modules/glue-queue-agent-credential.nix +++ b/nix/host-modules/glue-queue-agent-credential.nix @@ -15,7 +15,7 @@ # that is correct rather than degraded. The publisher runs on the authelia # host and authelia mints on its first boot, so "nothing at that path yet" is # the ordinary early state of a swarm. Nothing here writes a local stand-in: -# unlike a matrix appservice token there is no such thing as a locally valid +# unlike a matrix registration token there is no such thing as a locally valid # OIDC client secret, so a placeholder would turn a hive that cannot connect # into one that is refused, which reaches the agent as a timeout. # diff --git a/nix/host-modules/hive-matrix.nix b/nix/host-modules/hive-matrix.nix index 53aa9c15..8d087b0a 100644 --- a/nix/host-modules/hive-matrix.nix +++ b/nix/host-modules/hive-matrix.nix @@ -40,155 +40,22 @@ let # login into an intermittent one. matrixSecretPath = "/var/lib/tuwunel-oidc/${cfg.sso.clientId}.secret"; - # ⚠️ tuwunel does NOT read the host path directly, and this indirection + # Single source for the registration-token path: the option's `default`, + # its `config`-level `mkDefault` self-definition, and the assertion that + # rejects a moved path all read this same binding rather than repeating + # the string literal. See `registrationTokenFile`'s own comment below. + registrationTokenPath = "/var/lib/hyperhive/matrix-register-token"; + + # ⚠️ tuwunel does NOT read the path above directly, and this indirection # is not ceremony. Upstream's own words: "under systemd the path must be # visible to the service after sandboxing (ReadWritePaths / ProtectHome), # typically by placing the file under /etc/tuwunel/" — which this # container has no writable etc for. `LoadCredential` is the answer - # already in use for the appservice registration below, and for the + # already in use two units below for the registration token, and for the # same reason: it keeps `DynamicUser=true` + `PrivateUsers=true` intact # with no host-side chown or GID pinning. matrixSecretCredential = "/run/credentials/tuwunel.service/oidc_client_secret"; - # How this hive creates matrix accounts: an appservice registration whose - # `url` is null. Null is a legal `url` (ruma's `Registration` types it - # `Option`), and it is the whole point — with no URL the homeserver - # never calls out, so there is no HTTP service to run and no daemon to - # operate. What the registration delivers is the `as_token`: hive-c0re's - # standing authority to create, and log in as, the accounts named by the - # namespace below, with no shared registration secret in the picture. - appserviceId = "hyperhive"; - - # The appservice's own user, and deliberately the hive admin account. - # Loading a registration CREATES its `sender_localpart` user when absent - # (tuwunel `src/service/appservice/mod.rs`), on a zero-user database, - # inside `Services::start()` — before the HTTP listener accepts anything. - # That is what lets the `admin_execute` promotion below land on the very - # first boot of a fresh homeserver, instead of depending on hive-c0re - # racing to register the first account and win the auto-admin grant. - # - # ⚠️ Must equal `matrix::HIVE_ADMIN_LOCALPART` in hive-c0re, which derives - # it independently with nothing wiring an override across — same - # agreement, and same reason for saying so, as the token path below. - adminLocalpart = "hive"; - - # The `as_token`, and the `hs_token` the spec requires alongside it. Both - # minted by the render script below, mode 0600; the `as_token` is the one - # hive-c0re reads and the one the swarm secret store overwrites (see - # `glue-matrix-bao-token.nix`). The `hs_token` authenticates the homeserver - # TO the appservice, which with `url = null` is nobody — it exists because - # the registration format requires it. - # - # `appserviceTokenPath` is the single source for the option's `default`, - # its `config`-level `mkDefault` self-definition, and the assertion that - # rejects a moved path. See `appserviceTokenFile`'s own comment below. - appserviceTokenPath = "/var/lib/hyperhive/matrix-appservice-token"; - appserviceHsTokenPath = "/var/lib/hyperhive/matrix-appservice-hs-token"; - - # The registration file, and the directory that holds it. A directory - # rather than a `[global.appservice.]` stanza because a stanza's - # `as_token` would be a nix literal, and a nix literal is a world-readable - # store path — the same rule that sends the OIDC client secret through a - # runtime file. `docs/swarm/secrets.md` has the general form of it. - appserviceDir = "/var/lib/hyperhive/matrix-appservice"; - appserviceRegistrationPath = "${appserviceDir}/${appserviceId}.yaml"; - - # ⚠️ tuwunel does not read the host path directly, for exactly the reason - # given for the two secrets above: the file is 0600 root-owned and the - # homeserver runs under `DynamicUser=true`. `LoadCredential` copies it to a - # 0400 dynamic-user-owned path as root, before the sandbox and that user - # exist. - # - # So `appservice_dir` points at the credentials directory ITSELF. That is - # safe rather than clever: tuwunel's loader takes only `.yaml`/`.yml` - # entries from the directory and skips every other file, so the sibling - # credentials (the OIDC secret) are invisible to it. A credential id is a - # free-form filename, which is what lets ours end in `.yaml`. - appserviceCredentialId = "${appserviceId}-appservice.yaml"; - appserviceCredentialDir = "/run/credentials/tuwunel.service"; - - # Every local user this hive may provision — agents, the hive admin, and - # the operator accounts `hivectl matrix create-user` makes, which is the - # whole matrix localpart charset. - # - # ⚠️ Anchored deliberately: tuwunel compiles a namespace into a `RegexSet` - # and asks it for a MATCH, not a full match, so an unanchored - # `@[a-z0-9]+:this.example` also matches `@x:this.example.evil.test`. - # - # Non-exclusive deliberately: an exclusive namespace does not widen what - # the appservice may do, it narrows what everything ELSE may do — an SSO - # login adopting an existing account, or `!admin users create-user`, would - # start failing with `M_EXCLUSIVE`. The appservice needs the right to - # create these names, not a monopoly on them. - appserviceUserRegex = "^@[a-z0-9._=/-]+:${lib.escapeRegex effectiveServerName}$"; - - # Mint the tokens if they are absent, then render the registration from - # whatever they now hold. **One script with two callers** — the activation - # script below and `glue-matrix-bao-token.nix`, which overwrites the - # `as_token` with the swarm store's copy and has to re-render afterwards. - # - # ⚠️ The `as_token` mint here is the FIRST-BOOT FALLBACK, not the normal - # route. `swarm-secret-publisher.nix` mints one per hive swarm-side and - # publishes it to the store; the reader above overwrites this file before - # the homeserver's container starts, so what a hive actually runs on is the - # swarm's value. This keeps a hive that has never reached the store working - # — and it fires only when the file is absent, so it never clobbers one the - # store delivered. (`hs_token` has no swarm half: it is how the homeserver - # authenticates to *us*, so it stays hive-local and is minted here for real.) - # A second copy of the registration's shape would be a second source of - # truth for a format whose mismatch is silent: the homeserver would load a - # registration naming a token nobody holds, and every request hive-c0re - # makes would come back 401 naming nothing. - # - # Re-rendering unconditionally is the point rather than thoroughness: "the - # token file exists" does not mean "the registration carries what is in - # it". - appserviceRegistrationScript = pkgs.writeShellApplication { - name = "hive-matrix-appservice-registration"; - runtimeInputs = [ pkgs.coreutils ]; - text = '' - # Both the tokens and the rendered registration are secrets; 077 - # covers every file this script creates rather than each one - # separately. - umask 077 - mkdir -p ${lib.escapeShellArg appserviceDir} - for f in ${lib.escapeShellArg appserviceTokenPath} ${lib.escapeShellArg appserviceHsTokenPath}; do - if [ ! -s "$f" ]; then - head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$f" - echo >> "$f" - echo "hive-matrix: generated appservice token at $f" - fi - chmod 0600 "$f" - done - - # Read into shell variables and emitted with `printf`, a shell - # builtin: a token passed as an argument to a real command would land - # in that process's argv, which is world-readable for its lifetime. - asToken="$(cat ${lib.escapeShellArg appserviceTokenPath})" - hsToken="$(cat ${lib.escapeShellArg appserviceHsTokenPath})" - - # The quoted heredoc keeps the regex's own `$` and `\` out of the - # shell's hands; the YAML single quotes keep them out of YAML's. - { - cat <<'REGISTRATION' - id: ${appserviceId} - url: null - sender_localpart: ${adminLocalpart} - rate_limited: false - namespaces: - users: - - exclusive: false - regex: '${appserviceUserRegex}' - aliases: [] - rooms: [] - REGISTRATION - printf 'as_token: %s\nhs_token: %s\n' "$asToken" "$hsToken" - } > ${lib.escapeShellArg appserviceRegistrationPath} - chmod 0600 ${lib.escapeShellArg appserviceRegistrationPath} - chmod 0700 ${lib.escapeShellArg appserviceDir} - ''; - }; - # Format-locked by tuwunel, not chosen here: the callback host must point # directly at the matrix server and the path is fixed at # `/_matrix/client/unstable/login/sso/callback/`. Built once @@ -576,59 +443,40 @@ in ''; }; - appserviceTokenFile = lib.mkOption { + registrationTokenFile = lib.mkOption { type = lib.types.path; internal = true; - default = appserviceTokenPath; + default = registrationTokenPath; description = '' - Host path to a file containing this hive's matrix appservice - token (`as_token`) — the identity `hive-c0re` creates and logs - into accounts with. Minted automatically on first activation - (32-byte random hex, mode 0600) and rendered into the - appservice registration the homeserver loads at boot. Agents - never see it; an agent only ever receives its own - `access_token`. + Host path to a file containing the matrix registration token + tuwunel reads to authorise new-account creation. The token is + generated automatically by `hive-c0re` on first boot (32-byte + random hex, mode 0600) and is bind-mounted read-only into the + tuwunel container at the same path. Agents never see this + token — hive-c0re uses it to provision per-agent accounts + and the agent only receives the resulting `access_token`. Not operator-settable — `hive-c0re`'s Rust side derives this - same path independently (`paths::matrix_appservice_token()`) - with nothing wiring an override across, so a moved path desyncs - the two silently. An externally-managed token is delivered by - writing into *this* fixed path instead of moving it — see - `glue-matrix-bao-token.nix`, which fetches from the swarm secret - store, overwrites this file in place, and re-renders the - registration that names it. + same path independently (`paths::matrix_register_token()`) with + nothing wiring an override across, so a moved path used to + desync the two silently. An externally-managed token is + delivered by writing into *this* fixed path instead of moving it — + see `glue-matrix-bao-token.nix`, which fetches from the swarm + secret store and overwrites this file in place. Enforced by an `assertions` entry below rather than `readOnly`: the `config` block gives this option its own `mkDefault` - definition (lowest priority) so a real override still resolves - cleanly instead of crashing eval with nixpkgs' generic - "read-only, set multiple times" message; the assertion then - names the actual problem. (`readOnly` was the first attempt on - this option's predecessor — atlas caught that it only rejects a - *second* definition, so with nothing else defining the option a - lone override sailed through silently, same desync as before - with a lock that wasn't locked. Fixed by defining the value here - instead of leaving it on `default` alone, but a proper - `assertions` message beat re-adding `readOnly` on top once the - option had a real definition either way.) - ''; - }; - - appserviceRegistrationScript = lib.mkOption { - type = lib.types.path; - internal = true; - default = "${appserviceRegistrationScript}/bin/hive-matrix-appservice-registration"; - defaultText = lib.literalMD "the module's own registration renderer"; - description = '' - The script that mints the appservice tokens when absent and - (re-)renders the registration file from them. An option only so - that `glue-matrix-bao-token.nix` can run the same one after - overwriting the token with the swarm store's copy, rather than - carrying a second copy of the registration's shape. - - Not operator-settable, and not a hook: replacing it means - deciding what the homeserver's appservice registration says, - which is this module's job. + definition (lowest priority) so any real override — an operator's, + or the pre-rename `swarm.matrix.registrationTokenFile` shim's — + still resolves cleanly instead of crashing eval with nixpkgs' + generic "read-only, set multiple times" message; the assertion + then names the actual problem. (`readOnly` was the first attempt — + atlas caught that it only rejects a *second* definition, + so with nothing else defining the option a lone override sailed + through silently, same desync as before with a lock that wasn't + locked. Fixed by defining the value here instead of leaving it on + `default` alone, but a proper `assertions` message beat re-adding + `readOnly` on top once the option had a real definition either way.) ''; }; @@ -690,11 +538,11 @@ in config = lib.mkIf deployCfg.matrix.enable { # The option's own value, defined explicitly rather than left on its - # bare `default` — `mkDefault` so a real override still resolves - # cleanly rather than crashing eval; the `assertions` entry below is - # what actually rejects it, with a message naming the reason. See that - # option's own comment above. - services.hyperhive.deploy.matrix.appserviceTokenFile = lib.mkDefault appserviceTokenPath; + # bare `default` — `mkDefault` so a real override (an operator's, or + # the pre-rename shim's) still resolves cleanly rather than crashing + # eval; the `assertions` entry below is what actually rejects it, with + # a message naming the reason. See that option's own comment above. + services.hyperhive.deploy.matrix.registrationTokenFile = lib.mkDefault registrationTokenPath; # Matrix's own gateway surface: the sub-domain vhost, the name the # hive resolver answers for, and the Accept-header map that vhost's @@ -875,17 +723,19 @@ in # mkDefault above lets an override resolve instead of crashing # eval — this is the actual rejection, with a message that names # the real fix instead of nixpkgs' generic conflicting-definition - # text. - assertion = deployCfg.matrix.appserviceTokenFile == appserviceTokenPath; + # text. Covers both the current path and the pre-rename + # `swarm.matrix.registrationTokenFile` shim in one check, since + # both land on the same merged option. + assertion = deployCfg.matrix.registrationTokenFile == registrationTokenPath; message = '' - services.hyperhive.deploy.matrix.appserviceTokenFile is fixed at - ${appserviceTokenPath} and cannot be moved — hive-c0re's Rust + services.hyperhive.deploy.matrix.registrationTokenFile is fixed at + ${registrationTokenPath} and cannot be moved — hive-c0re's Rust side derives this same path independently and has no way to learn an override, so moving it desyncs the two silently instead of loudly. - Integrating an externally-managed appservice token? Deliver it - by writing into ${appserviceTokenPath} instead of pointing this + Integrating an externally-managed registration token? Deliver it + by writing into ${registrationTokenPath} instead of pointing this option elsewhere — see glue-matrix-bao-token.nix, which does exactly that from the swarm secret store. ''; @@ -931,8 +781,8 @@ in # permanent stall presenting as "matrix is broken", several layers from # its cause. # - # The appservice registration dodges that with an activation script - # that renders the file first. ⚠️ That dodge is NOT available here: + # The registration token above dodges that with an activation script + # that pre-creates the file. ⚠️ That dodge is NOT available here: # tuwunel requires the secret file to exist *and be non-empty*, so a # zero-byte placeholder would satisfy the bind mount and then stop the # homeserver from starting. @@ -1040,21 +890,16 @@ in '' ); - # Mint the appservice tokens and render the registration, before any - # container start. - # - # An activation script rather than a unit: the directory below is - # bind-mounted into the container and nixos-container refuses to start - # when a bind source is missing, so this has to have run first. - # Activation is what runs before the container both on a switch and on - # every boot. (The registration token this replaced used an activation - # script for the same reason, and additionally to dodge nspawn creating - # an empty file at a missing bind target — which tuwunel then read as - # "no token", refusing every registration until the next restart. A - # missing registration file is not silent in that way: the homeserver - # fails its appservice load loudly.) - system.activationScripts.hive-matrix-appservice = lib.stringAfter [ "var" ] '' - ${deployCfg.matrix.appserviceRegistrationScript} + system.activationScripts.hive-matrix-register-token = lib.stringAfter [ "var" ] '' + tokenFile=${lib.escapeShellArg (toString deployCfg.matrix.registrationTokenFile)} + if [ ! -s "$tokenFile" ]; then + mkdir -p "$(dirname "$tokenFile")" + head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$tokenFile" + echo >> "$tokenFile" + echo "hive-matrix: generated registration token at $tokenFile" + fi + # Re-apply 0600 (normalises any pre-LoadCredential carry-over). + chmod 0600 "$tokenFile" ''; containers.hive-matrix = { @@ -1065,15 +910,11 @@ in extraFlags = [ "--link-journal=host" ]; # Shared host netns — agents reach tuwunel at localhost:. privateNetwork = false; - # Read-only bind of the host-managed appservice registration; tuwunel + # Read-only bind of the host-managed registration token; tuwunel # reads it via systemd LoadCredential below (not directly). - # - # The directory, not the file inside it: the registration is - # re-rendered rather than edited, and binding the file would pin the - # inode the container saw when it started. bindMounts = { - ${appserviceDir} = { - hostPath = appserviceDir; + ${deployCfg.matrix.registrationTokenFile} = { + hostPath = deployCfg.matrix.registrationTokenFile; isReadOnly = true; }; } @@ -1167,50 +1008,13 @@ in # trustedServers keeps it effectively closed. allow_federation = true; trusted_servers = deployCfg.matrix.trustedServers; - # Nobody registers themselves here. Accounts are created by - # the hive's appservice, which this flag does not gate — - # tuwunel checks it only for requests that arrive WITHOUT an - # appservice token, so hive-c0re provisions exactly as before - # and everyone else is refused outright. - # - # ⚠️ Not a hardening afterthought: `allow_registration = true` - # with no registration token configured makes tuwunel REFUSE - # TO START (it demands - # `yes_i_am_very_very_sure_…_open_registration_…` instead). So - # dropping the token and leaving this true is not a lax - # homeserver, it is a homeserver that does not boot. - allow_registration = false; - - # Where the hive's appservice registration is read from — the - # credentials directory, for the reasons at - # `appserviceCredentialDir`'s own definition. Loaded inside - # `Services::start()`, before the listener accepts anything. - appservice_dir = appserviceCredentialDir; - - # The zero-user bootstrap, and the only thing here that needs - # no account to already exist. These run after startup and - # BEFORE the HTTP listener, with no sender and no permission - # check — which is what makes them the one lever that can - # promote the hive admin on a homeserver where nobody is admin - # yet. The appservice registration above creates - # `@${adminLocalpart}` as its sender user moments earlier; - # this joins it to the admin room at power level 100. - # - # Idempotent by upstream's own guard: `make_user_admin` - # short-circuits when the user is already joined at 100, so a - # hive that has had an admin for months emits nothing. - admin_execute = [ - "users make-user-admin @${adminLocalpart}:${effectiveServerName}" - ]; - - # ⚠️ Load-bearing, not tidiness. An `admin_execute` command - # that fails aborts startup outright when this is false — so - # the one boot where the promotion cannot work (a homeserver - # that has no `@${adminLocalpart}` and no appservice user yet, - # e.g. a registration file that arrived late) would take the - # homeserver down with it rather than converging on the next - # start. - admin_execute_errors_ignore = true; + # Token-gated registration. The absent + # `yes_i_am_very_very_sure_…_open_registration_…` flag + # keeps the server closed to anyone without the token. + allow_registration = true; + # LoadCredential below copies the host file into a + # 0400 dynamic-user-owned path; tuwunel reads from there. + registration_token_file = "/run/credentials/tuwunel.service/registration_token"; # Server-side E2EE is opt-in (default off); the agent matrix # client always supports decryption regardless. allow_encryption = cfg.allowEncryption; @@ -1271,10 +1075,7 @@ in # host-side chown :tuwunel / GID-pin gymnastics needed. # See `man systemd.exec` → LoadCredential. systemd.services.tuwunel.serviceConfig.LoadCredential = [ - # The credential id carries a `.yaml` suffix on purpose: - # `appservice_dir` above names this very directory, and tuwunel - # takes only `.yaml`/`.yml` entries from it. - "${appserviceCredentialId}:${appserviceRegistrationPath}" + "registration_token:${toString deployCfg.matrix.registrationTokenFile}" # Same mechanism, second secret. tuwunel re-reads this file on # every OAuth exchange, not just at startup, so it has to # outlive the unit's start — a credentials path does. diff --git a/nix/host-modules/swarm-secret-publisher.nix b/nix/host-modules/swarm-secret-publisher.nix index 6f1a51e6..b373bf58 100644 --- a/nix/host-modules/swarm-secret-publisher.nix +++ b/nix/host-modules/swarm-secret-publisher.nix @@ -1,7 +1,6 @@ -# The unit that puts swarm-level secrets into the swarm's secret store, so -# whoever needs one can read it there: a hive its agents' credential, a swarm -# service its own, a hive its matrix appservice token. The OIDC secrets it only -# copies; the appservice token it MINTS, having had no swarm-side producer. +# The unit that copies authelia's minted OIDC client secrets into the swarm's +# secret store, so whoever needs one can read it there: a hive its agents' +# credential, a swarm service its own. # # ⚠️ "Whoever", including a reader on THIS host. A swarm service's secret goes # into the store even when the service runs beside authelia, because its @@ -15,19 +14,19 @@ # why: the plaintext's other reader lives in a **different container**, and # containers that share this host's network namespace still have separate # filesystem roots, so "the host is the only place both trees are addressable". -# A delivery step therefore runs on the minting host, not the store's or the -# reading hive's. +# A delivery step therefore runs on the host that mints — not on the store's +# host, and not on the reading hive's. # # ⚠️ ITS OWN STORE IDENTITY, not swarm-controller's. That principal may rewrite -# every hive's policy and login role; a unit that writes a handful of secrets -# has no business holding it. ./swarm-bao.nix grants this one `create`/`update` -# under the hive prefix and nothing else — no `read`, which is why the mint -# below keeps its own record of what it published. +# every hive's policy and login role; a unit whose entire job is copying one +# file has no business holding it. ./swarm-bao.nix grants this one +# `create`/`update` under the hive prefix and nothing else. # # ⚠️ THE SECRET NEVER REACHES `argv`. `bao` is an external binary, so every # argument is world-readable in /proc for the life of the call — the value is -# passed as `@` and read by bao itself. Same rule governs the mint: it -# redirects into a file and never holds the token in a shell variable. +# passed as `@` and read by bao itself. (`/knowledge/secret-hygiene.md`: +# "a path keeps the secret out of the store, and reading it into a shell +# variable puts it straight into argv".) { pkgs, lib, @@ -72,21 +71,6 @@ let hyperhiveCfg.swarm.grafana.oidc.clientId hyperhiveCfg.swarm.otel.clientId ]; - - # Where this unit keeps the appservice tokens it minted, and the whole reason - # a re-publish is idempotent. The store cannot be that record: ./swarm-bao.nix - # grants this principal `create`/`update` and deliberately no `read`, so "does - # this hive already have a token?" is a question it can only ask locally. - # Without the file every run would mint a fresh value and rotate the swarm's - # token for no reason — and a rotation is not free, because the homeserver - # only reloads the registration naming it when its container restarts. - # - # `StateDirectory=` below is what creates it, so the name there is relative - # and this unit owns the tree. Losing it mints once more and republishes; - # readers pick that up on their next start, and nothing that already - # registered breaks — the token authenticates the appservice, it does not - # live inside any account it created. - matrixTokenDir = "/var/lib/swarm-secret-publish/matrix-appservice-token"; in { options.services.hyperhive.deploy.swarm-secret-publisher = { @@ -98,12 +82,10 @@ in Publish the OIDC client secrets this host mints into the swarm's secret store, so a hive that does not run authelia can read its agents' credential and a swarm service can read its own — from - wherever it runs, this host included. Also mints each hive's matrix - appservice token, which has no other swarm-side producer, and - publishes it the same way. + wherever it runs, this host included. - Defaults to whether this host mints the OIDC secrets, which is the - only half of the question that is a property of *this* host. + Defaults to whether this host mints them, which is the only half of + the question that is a property of *this* host. ⚠️ Deliberately **not** `deploy.bao.enable`. That asks whether the store stands here, and a publisher beside a remote store, holding a @@ -113,12 +95,11 @@ in identity's job (see `baoClientCertFile`), not this option's. Turning it off leaves every hive but this one without its agents' - credential, the swarm's Grafana without any login at all, its - collector pushing unauthenticated, and every hive falling back to the - appservice token its own first boot minted — so two hives never agree - on one. Each secret has exactly one route and this is the producer's - end of it, so the honest reason to set it false is a deployment - delivering those secrets by some other mechanism it owns. + credential, the swarm's Grafana without any login at all, and its + collector pushing unauthenticated — each secret has exactly one route + and this is the producer's end of it. So the honest reason to set it + false is a deployment delivering those secrets by some other + mechanism it owns. ''; }; @@ -162,7 +143,7 @@ in }; systemd.services.swarm-secret-publish = { - description = "publish the swarm's minted secrets to the swarm secret store"; + description = "publish minted OIDC client secrets to the swarm secret store"; wantedBy = [ "multi-user.target" ]; after = [ "network.target" ]; path = [ @@ -178,12 +159,6 @@ in TimeoutStartSec = 60; Restart = "on-failure"; RestartSec = 30; - # The minted appservice tokens, see `matrixTokenDir` above. Named - # relative to /var/lib because that is the form systemd creates; the - # mode is narrowed from systemd's 0755 default because the directory - # listing is the roster of hives and every file in it is a secret. - StateDirectory = "swarm-secret-publish/matrix-appservice-token"; - StateDirectoryMode = "0700"; }; environment = { BAO_ADDR = "https://${hyperhiveCfg.swarm.bao.domain}:${toString hyperhiveCfg.swarm.bao.port}"; @@ -210,7 +185,6 @@ in published=0 skipped=0 - tokens=0 ${lib.concatMapStringsSep "\n" (hive: '' src=${lib.escapeShellArg "${deployCfg.authelia.hostClientSecretDir}/${agentClientId hive}.secret"} @@ -254,43 +228,7 @@ in fi '') serviceClientIds} - # The matrix appservice token, per hive. Unlike the two loops above - # there is nothing to copy: authelia never minted this one, and each - # homeserver's own host minted its own — which is exactly why two hives - # never converged on a value the swarm has to agree on. The mint moves - # here because this unit already holds a store identity and already - # writes under the hive prefix; ../host-modules/hive-matrix.nix keeps - # its activation mint as the first-boot fallback for a hive that has - # never reached the store. - # - # 32 bytes of urandom as lowercase hex, the shape that file has - # generated all along. Written with no trailing newline, unlike that - # file: what goes into the store is the value, and - # ./glue-matrix-bao-token.nix adds the newline the hive's token file - # carries. - ${lib.concatMapStringsSep "\n" (hive: '' - src=${lib.escapeShellArg "${matrixTokenDir}/${hive}"} - if [ ! -s "$src" ]; then - # The subshell keeps the umask local — the loops above rely on - # bao's own file handling and should not inherit one. - ( umask 077; head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$src" ) - echo "minted a matrix appservice token for ${hive}" >&2 - fi - # `value=@$src` for the same reason as both loops above: bao opens - # the file itself, so the token is never an argument of this process. - # - # Put on every run, not only on the run that minted — a put of the - # value already there changes nothing for any reader, while a mint - # whose publish failed must not be left as a token this host holds - # and no hive can reach. The `-s` test above is what keeps the VALUE - # stable across those runs: rotating on every boot would hand the - # homeserver a registration its running container does not carry. - bao kv put ${lib.escapeShellArg "secret/swarm/hives/${hive}/matrix/appservice-token"} \ - value=@"$src" - tokens=$((tokens + 1)) - '') hiveNames} - - echo "published $published client secret(s), skipped $skipped, and $tokens matrix appservice token(s)" + echo "published $published client secret(s), skipped $skipped" ''; }; }; diff --git a/nix/module-eval.nix b/nix/module-eval.nix index 7c819347..a2682c6f 100644 --- a/nix/module-eval.nix +++ b/nix/module-eval.nix @@ -116,45 +116,24 @@ let ]; }; - # The homeserver's turn to split. All seven remaining host-side options are - # set through their pre-rename paths — including `gui.enable`, whose value is - # deliberately the opposite of its default so the definition has to actually - # land, and both packages, stubbed to a derivation neither option defaults to - # for the same reason. All seven so that dropping any single shim entry fails - # the eval, not just the ones the assertions read. - # - # `registrationTokenFile` used to be the eighth: both spellings of it are now - # `mkRemovedOptionModule`, exercised by `matrixRemovedToken` below instead — - # setting it here would make every case that reads this fixture fail on that - # one removed option. + # The homeserver's turn to split. All eight host-side options are set through + # their pre-rename paths — including `gui.enable`, whose value is deliberately + # the opposite of its default so the definition has to actually land, and both + # packages, stubbed to a derivation neither option defaults to for the same + # reason. All eight so that dropping any single shim entry fails the eval, not + # just the ones the assertions read. matrixOldPath = hive { deploy.matrix.enable = true; swarm.matrix.openFirewall = true; swarm.matrix.trustedServers = [ "matrix.example.invalid" ]; swarm.matrix.maxRequestSize = 31457280; + swarm.matrix.registrationTokenFile = "/etc/matrix/register.token"; swarm.matrix.gui.enable = false; swarm.matrix.package = pkgs.emptyDirectory; swarm.matrix.gui.package = pkgs.emptyDirectory; swarm.matrix.sso.clientSecretFile = "/etc/matrix/oidc.secret"; }; - # Moving the appservice token, which desyncs hive-c0re's independently - # derived path from the homeserver's. Its own fixture because the assertion - # it trips would otherwise fire for every case reading `matrixOldPath`. - matrixMovedAppserviceToken = hive { - deploy.matrix.enable = true; - deploy.matrix.appserviceTokenFile = "/etc/matrix/as.token"; - swarm.matrix.sso.clientSecretFile = "/etc/matrix/oidc.secret"; - }; - - # A config still naming the removed registration token, through the - # pre-rename spelling — the one an old deployment is most likely to carry. - matrixRemovedToken = hive { - deploy.matrix.enable = true; - swarm.matrix.registrationTokenFile = "/etc/matrix/register.token"; - swarm.matrix.sso.clientSecretFile = "/etc/matrix/oidc.secret"; - }; - # The queue's callout identity, fourth split slice. `autoGenerateCallout` is # left FALSE on purpose: that is what makes the seed paths the thing deciding # `responderConfigured`, so the assertion below is about the seeds rather @@ -861,41 +840,33 @@ let } { # Third split, and the one whose readers were hardest to see: the - # appservice token is read only through a `let` alias in another + # registration token is read only through a `let` alias in another # module, so no full path names it anywhere. Both arms read a rendered # effect — the host firewall and the container's bind-mount table — so a # rename that resolves but stops reaching the module still fails. - name = "a config written against the pre-rename matrix paths still opens the port and mounts the appservice registration"; + name = "a config written against the pre-rename matrix paths still opens the port and mounts the token"; ok = let ports = matrixOldPath.networking.firewall.allowedTCPPorts; httpPort = matrixOldPath.services.hyperhive.swarm.matrix.httpPort; in builtins.elem httpPort ports - && matrixOldPath.containers.hive-matrix.bindMounts ? "/var/lib/hyperhive/matrix-appservice"; + && matrixOldPath.containers.hive-matrix.bindMounts ? "/etc/matrix/register.token"; } { - # appserviceTokenFile has no override capability (a store-delivered - # secret doesn't need one — glue-matrix-bao-token.nix writes into the - # fixed path instead of moving it), unlike its siblings in the same - # rename. The fixture proves the override still *resolves* (mkDefault, - # not a crash); this proves it also gets *rejected*, by a named - # assertion rather than nixpkgs' generic conflicting-definition text. - # Reads `.assertions` directly (cheap: a list of `{assertion; - # message;}`, not `system.build.toplevel`) rather than forcing a real - # build just to observe a boolean. - name = "moving appserviceTokenFile trips a named assertion, not a silent desync"; + # registrationTokenFile lost its override capability entirely + # (bao-delivered secrets don't need one — glue-matrix-bao-token.nix + # already writes into the fixed path instead of moving it), unlike its + # five siblings in the same rename. `matrixOldPath` above proves the + # override still *resolves* (mkDefault, not a crash) — this proves it + # also gets *rejected*, by a named assertion rather than nixpkgs' generic + # conflicting-definition text. Reads `.assertions` directly (cheap: a + # list of `{assertion; message;}`, not `system.build.toplevel`) rather + # than forcing a real build just to observe a boolean. + name = "overriding registrationTokenFile (even via the pre-rename shim) trips a named assertion, not a silent desync"; ok = lib.any ( - a: !a.assertion && lib.hasInfix "appserviceTokenFile" a.message - ) matrixMovedAppserviceToken.assertions; - } - { - # The removal has to SAY so. A config carrying the old option through - # an upgrade is the common case, and nixpkgs' bare "option does not - # exist" would name neither what replaced the token nor where to read - # about it. - name = "a config still setting registrationTokenFile is refused with a message naming the appservice"; - ok = lib.any (a: !a.assertion && lib.hasInfix "appservice" a.message) matrixRemovedToken.assertions; + a: !a.assertion && lib.hasInfix "registrationTokenFile" a.message + ) matrixOldPath.assertions; } { # Reads the DELIVERY UNIT, not the options: `responderConfigured` gates @@ -1477,41 +1448,6 @@ let lib.hasInfix "secret/swarm/hives/h1/queue/agent" s && lib.hasInfix "secret/swarm/hives/h2/queue/agent" s; } - { - # The producer's end of the read `glue-matrix-bao-token.nix` already did. - # Both hives are asserted for the reason the queue case above gives: a - # publisher that knew one name rather than the roster would pass on a - # single-hive fixture and strand the second hive ever added — which is - # the two-hives-never-converge shape this slice exists to close. - name = "the publisher mints an appservice token for every hive and writes it to that hive's matrix path"; - ok = - let - s = secretPublisherHere.systemd.services.swarm-secret-publish.script; - in - lib.hasInfix "secret/swarm/hives/h1/matrix/appservice-token" s - && lib.hasInfix "secret/swarm/hives/h2/matrix/appservice-token" s - && lib.hasInfix "/dev/urandom" s; - } - { - # What makes a re-publish idempotent. This principal is granted - # `create`/`update` and no `read`, so it cannot ask the store whether a - # hive already has a token — with nowhere to keep one, every run would - # mint a fresh value and rotate the swarm's token. A state directory is - # that somewhere, and nothing else in this unit needs one, so its absence - # means exactly this. - # - # The second arm is the mint's own guard: the state file is only written - # when it is missing or empty. Dropping that test leaves a unit that - # still has a state directory and still rotates on every boot. - name = "the publisher keeps the tokens it minted, and mints only when it holds none"; - ok = - let - u = secretPublisherHere.systemd.services.swarm-secret-publish; - in - lib.hasInfix "matrix-appservice-token" (u.serviceConfig.StateDirectory or "") - && u.serviceConfig.StateDirectoryMode or null == "0700" - && lib.hasInfix "if [ ! -s \"$src\" ]" u.script; - } { # A property of the SET, not of one unit: both of these authenticate by # certificate, and `BAO_CLIENT_CERT` is transport rather than identity, so @@ -1582,7 +1518,7 @@ let s = baoWithMatrix.systemd.services.swarm-bao-matrix-token.script; in lib.hasInfix "secret/swarm/hives/" s - && lib.hasInfix "/matrix/appservice-token" s + && lib.hasInfix "/matrix/registration-token" s # The shape it used to have: `matrix` where a principal kind belongs, # which no grant covers. && !(lib.hasInfix "secret/swarm/matrix/" s); @@ -2535,7 +2471,7 @@ let # The store's first reader. Its unit belongs to the pairing, not to # either service: matrix must not learn the store exists, and the store # must not know who reads it. - name = "a store deployed beside the homeserver fetches its appservice token"; + name = "a store deployed beside the homeserver fetches its registration token"; ok = baoWithMatrix.systemd.services ? swarm-bao-matrix-token; } { diff --git a/swarm-secret-client/src/matrix.rs b/swarm-secret-client/src/matrix.rs index 9a0cda5b..69a130c7 100644 --- a/swarm-secret-client/src/matrix.rs +++ b/swarm-secret-client/src/matrix.rs @@ -26,25 +26,18 @@ pub fn account_path(agent: &str, account: &str) -> Result { Ok(format!("{prefix}/matrix/{account}")) } -/// The path holding `hive`'s matrix appservice token (`as_token`). +/// The path holding `hive`'s matrix registration token. /// /// Keyed per **hive**, not per agent, like [`crate::queue::agent_client_path`] /// and unlike [`account_path`] above: one homeserver admits one hive's -/// accounts, so the identity that creates them is the hive's. -/// -/// ⚠️ Renamed from `registration-token` along with what it holds: the -/// homeserver no longer accepts a shared registration secret at all, so a -/// value still stored under the old path would be read by nothing. A hive -/// whose store has only the old path falls back to its locally minted -/// token — see `glue-matrix-bao-token.nix` — so the rename degrades rather -/// than breaks, but the store needs a fresh `put` to take effect again. +/// accounts, so the token that creates them is the hive's. /// /// # Errors /// [`Error::PathSegment`] when `hive` contains anything but `[A-Za-z0-9_-]`, /// which is what keeps one hive's name from addressing another hive's secret. -pub fn appservice_token_path(hive: &str) -> Result { +pub fn registration_token_path(hive: &str) -> Result { let prefix = principal_prefix(Kind::Hive, hive)?; - Ok(format!("{prefix}/matrix/appservice-token")) + Ok(format!("{prefix}/matrix/registration-token")) } /// What an account's path holds: the token, plus the homeserver it belongs to. @@ -87,29 +80,28 @@ mod tests { } #[test] - fn the_appservice_token_lands_under_the_hive_prefix_the_grant_covers() { + fn the_registration_token_lands_under_the_hive_prefix_the_grant_covers() { // Spelled out for the same reason as above, and with a second job here: // the read policy grants `secret/data/swarm/hives//*`, so this // string is what makes the path reachable at all. assert_eq!( - appservice_token_path("pr1ma").expect("a plain name is legal"), - "swarm/hives/pr1ma/matrix/appservice-token" + registration_token_path("pr1ma").expect("a plain name is legal"), + "swarm/hives/pr1ma/matrix/registration-token" ); } #[test] - fn the_appservice_token_is_not_a_top_level_namespace() { - // The path its predecessor used to hold. `Kind` is a closed set and - // `matrix` is not one of its members, so a path with `matrix` as the - // second segment is outside every grant — which is how it came to 403 - // on every read. - let p = appservice_token_path("pr1ma").expect("legal"); + fn the_registration_token_is_not_a_top_level_namespace() { + // The path it used to hold. `Kind` is a closed set and `matrix` is not + // one of its members, so a path with `matrix` as the second segment is + // outside every grant — which is how it came to 403 on every read. + let p = registration_token_path("pr1ma").expect("legal"); assert!(!p.starts_with("swarm/matrix/"), "{p}"); } #[test] fn a_traversal_in_the_hive_name_is_refused() { - let e = appservice_token_path("../beta").expect_err("a traversal is not"); + let e = registration_token_path("../beta").expect_err("a traversal is not"); assert!(matches!(e, Error::PathSegment { kind: "hive", .. }), "{e}"); }