vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

Compare commits

base: vinzenz:5ce0a357b4a527888281b3ac2994d39613e64718
Branches Tags
vinzenz:main
..
compare: vinzenz:199afa41c8eb6f9a6ad9072b095be98b367a9035
Branches Tags
vinzenz:main
Author SHA1 Message Date
atlas
199afa41c8 matrix: publish the appservice token from the swarm, not just read it 
`glue-matrix-bao-token.nix` has read
`secret/swarm/hives/<hive>/matrix/appservice-token` since it landed, but
nothing ever wrote that path. The store was empty in every deployment, so
every read degraded to "keep what activation minted" and each hive stayed
the origin of a value the swarm has to agree on — two hives never
converged.

`swarm-secret-publish` is now the producer. It already holds a store
identity, already writes under the hive prefix, and already runs per
hive in the roster, so the mint is a third loop beside the two OIDC
copies rather than a second shape of this unit.

Idempotence comes from a record of its own, not from the store: this
principal is granted `create`/`update` with no `read`, so it cannot ask
whether a hive already has a token. It keeps what it minted under
`StateDirectory=` (0700 dir, 0600 file) and mints only when that file is
missing or empty; the `put` runs every time, because re-putting the same
bytes changes nothing for a reader while a mint whose publish failed must
not be left as a token this host holds and no hive can reach.

The token never becomes a nix literal and never reaches argv: the mint
redirects into a file, and the publish hands bao `value=@<path>` so bao
opens it itself — the same handling the OIDC loops use.

`hive-matrix.nix`'s activation mint stays as the genuine first-boot
fallback. It already fires only when the token file is absent, so it
cannot clobber a value the store delivered; `hs_token` has no swarm half
and is still minted there for real.

Refs #4402
 2026-09-15 20:57:49 +02:00
atlas
7ee7080b21 matrix: remove the registration token 
Nothing reads it any more: hive-c0re creates accounts as the hive's
appservice, so the mint, the host file, the bind mount, the
`LoadCredential` entry and tuwunel's `registration_token_file` all go.

⚠️ `allow_registration` has to go to `false` in the same change, and not
as hardening. tuwunel refuses to START when registration is allowed with
no token configured — it demands
`yes_i_am_very_very_sure_…_open_registration_…` instead — so dropping the
token and leaving the flag true is not a lax homeserver, it is one that
does not boot. The flag is checked only for requests arriving without an
appservice token, so hive-c0re provisions exactly as before and everyone
else is refused outright.

The swarm secret store keeps its role, repointed at the credential that
replaced the token (`swarm/hives/<hive>/matrix/appservice-token`). Its
unit now also re-runs hive-matrix's own registration renderer after
writing the file: the token is half an agreement, and a registration
still naming the previous value authenticates nobody. The renderer is
shared through an internal option rather than copied, so the
registration's shape has one home.

Both spellings of `registrationTokenFile` become
`mkRemovedOptionModule` with a message naming what replaced them. A hive
that never set the option — the default — is unaffected; one that pinned
it fails to evaluate with instructions instead of a silent no-op.

An upgraded hive needs no intervention: the activation script has both
halves in place before the homeserver restarts, existing agents keep the
tokens their devices already hold, and the old token file is left on
disk read by nothing. docs/integrations/matrix.md spells the path out.

Refs #4402
 2026-09-15 19:58:10 +02:00
atlas
43cd8607ba matrix: create accounts as the appservice, and promote the admin explicitly 
Account creation stops presenting a shared registration token in a UIAA
flow and starts acting as the hive's appservice: one POST, typed
`m.login.application_service`, authorised by the `as_token` the
registration file names. The account that comes out is an ordinary user
with its own device and its own access token — nothing about what an agent
holds changes.

Three things get better than "one fewer round-trip":

- An account whose token file was lost is re-tokened by an appservice
  login, which needs neither its password nor admin rights. That was
  previously a stored-password login, and failing that an admin-room
  password reset. Both are kept behind it, for accounts created before
  this existed or named outside the appservice's namespace.
- The hive admin no longer has to be the first account ever registered.
  It could not be, in fact: tuwunel excludes appservice-created users from
  the automatic first-user grant, and on a homeserver that already had
  users the rule never fired anyway. Rights now come from an explicit
  `make_user_admin` — performed by `admin_execute` at homeserver startup,
  and verified here each sweep by reading the account's own joined-rooms
  list. Absent rights are reported with the one command that grants them,
  and are not fatal: agent accounts, the Space and the chat room all work
  without them.
- hive-c0re reads the appservice token and never mints it. The old token
  was the whole agreement, so whoever wrote it first was right; this one
  is also named by a registration file that only the nix side writes, and
  a token minted here would be one the homeserver has never heard of.

Also fixes the `make-user-admin` reply matcher, which recognised neither
spelling tuwunel v1.9.0 uses ("<user> has been granted admin
privileges.") — a promotion that had already taken effect was reported as
a 15-second timeout.

Refs #4402
 2026-09-15 19:29:13 +02:00
atlas
5809077924 matrix: deliver an appservice registration to the homeserver 
The hive creates matrix accounts with a shared registration token today,
which means the secret that authorises account creation has to reach both
hive-c0re and tuwunel and stay identical in both. An appservice
registration replaces that with an identity: one token that says "this is
the hive's provisioner", carried in an ordinary credential file.

`url = null`, so nothing is served and no daemon is introduced — with no
URL the homeserver never calls out, and the registration exists purely to
give the `as_token` meaning.

Delivered through `appservice_dir` rather than a `[global.appservice.<id>]`
stanza, because a stanza's `as_token` would be a nix literal and a nix
literal is a world-readable store path. The file is minted and rendered by
a host activation script, bind-mounted into the container, and handed to
the homeserver by `LoadCredential` — the same two steps the registration
token and the OIDC client secret already take, and for the same reason
(0600 root on the host, `DynamicUser=true` in the container).

`sender_localpart` is the hive admin account on purpose: loading a
registration creates its sender user on a zero-user database inside
`Services::start()`, and the `admin_execute` promotion runs after that and
still before the HTTP listener accepts anything. So a fresh homeserver has
a joined, power-level-100 admin on its first boot without anyone having
won the first-registered-user grant. `admin_execute_errors_ignore` is set
because a failing startup command otherwise aborts startup outright.

Nothing reads the registration yet — hive-c0re still provisions through
the registration token, which is untouched here.

Refs #4402
 2026-09-15 19:21:15 +02:00
14 changed files with 1047 additions and 352 deletions
Download patch file Download diff file Expand all files Collapse all files

139
docs/integrations/matrix.md
View file

@ -90,39 +90,54 @@ by default. Reaching it on 8448 needs either an explicit tuwunel
bind to that port OR a reverse-proxy + `.well-known/matrix/server`
delegation (the latter lives in `gateway.md::Discovery flow`).
## Provisioning flow (registration token)
## Provisioning flow (appservice)
Token-gated registration: hive-c0re holds the token, agents never
see it. The agent only receives the resulting `access_token`.
Registration is closed. Accounts are created by the hive's own
**appservice**: hive-c0re holds the appservice token, agents never see
it, and an agent only ever receives its own `access_token`.
1. **System activation** writes a 32-byte random hex token (64
chars) to `services.hyperhive.deploy.matrix.registrationTokenFile`
(`/var/lib/hyperhive/matrix-register-token` by default), mode
`0600 root:root`, before any container start. Idempotent — only
writes when the file is missing or empty; always re-applies 0600
(normalises any 0640 / world-readable carry-over from
pre-LoadCredential deployments). This runs at activation time
(not first container start) to dodge a race where nspawn creates
an empty file when the bind-mount target is missing and tuwunel
reads `registration_token_file=""`, rejecting every registration
until next restart.
2. **Read-only bind-mount** maps the host file into the tuwunel
The appservice has no URL (`url: null` in its registration), so the
homeserver never calls out to it and there is no service to run. What the
registration buys is an identity the homeserver recognises — which is why
no secret has to be equal on both sides of the wire, and why account
creation doesn't depend on registration being open to anyone who learns
a token.
1. **System activation** mints a 32-byte random hex appservice token (64
chars) at `/var/lib/hyperhive/matrix-appservice-token` and its
spec-required `hs_token` sibling, mode `0600 root:root`, then renders
the registration to
`/var/lib/hyperhive/matrix-appservice/hyperhive.yaml` (also `0600`).
The tokens are minted only when missing; the registration is
re-rendered every time, because the token file can be overwritten in
place by the swarm secret store and a registration naming a stale
token authenticates nobody. Runs at activation time, before any
container start, because the directory is bind-mounted and
nixos-container refuses to start when a bind source is missing.
2. **Read-only bind-mount** maps that directory into the tuwunel
container at the same path.
3. **systemd `LoadCredential=`** inside the container copies the
bind-mounted file into
`/run/credentials/tuwunel.service/registration_token`, owned by
tuwunel's dynamic user with mode `0400`, at service start. The
host file stays `root:root 0600` — no `chown :tuwunel` /
`chmod 0640` / GID-pin gymnastics required. Keeps
`DynamicUser = true` + `PrivateUsers = true` intact.
4. tuwunel's `registration_token_file` points at the credentials
path, not the original bind-mount path.
5. **hive-c0re** uses the token to register each agent account via
the matrix-spec UIAA registration flow, persists the returned
`access_token` to `<agent-state>/matrix-token`. The agent's
matrix MCP client authenticates with that access_token and
never touches the shared registration token.
6. **hive-c0re restarts `hive-matrix-daemon`** for the agent
registration into
`/run/credentials/tuwunel.service/hyperhive-appservice.yaml`, owned by
tuwunel's dynamic user with mode `0400`, at service start. The host
file stays `root:root 0600` — no `chown :tuwunel` / `chmod 0640` /
GID-pin gymnastics required. Keeps `DynamicUser = true` +
`PrivateUsers = true` intact.
4. tuwunel's `appservice_dir` points at the credentials directory, not at
the bind-mount path. It reads only `.yaml`/`.yml` entries from there,
so the sibling credentials are invisible to it. The `.yaml` suffix on
the credential id is what makes this work.
5. **hive-c0re** reads the appservice token and creates each account with
one `POST /register` typed `m.login.application_service`, persisting
the returned `access_token` to `<agent-state>/matrix-token`. It never
mints the token itself: the value has to be the one the rendered
registration names, and only the nix side writes that.
6. **An account that exists but has lost its token file** is re-tokened
by an appservice `POST /login` — no password and no admin rights
involved. A stored-password login and an admin-room password reset
remain behind that, for accounts created before the appservice existed
or named outside its namespace.
7. **hive-c0re restarts `hive-matrix-daemon`** for the agent
immediately after writing the token so the daemon picks up the
new credential without waiting for a full container restart. If
the restart fails (for example daemon not yet running on first boot)
@ -130,16 +145,74 @@ see it. The agent only receives the resulting `access_token`.
(`hive-matrix-daemon.path` watching for `matrix-token` appearance)
brings the daemon up on the same boot cycle anyway.
### The admin account, and why it needs no first-user luck
`@hive:<server_name>` is the appservice's own `sender_localpart`, which
the homeserver creates itself when it loads the registration — on a
zero-user database, inside startup, before the HTTP listener accepts
anything. Its **admin rights** then come from an explicit
`make_user_admin`, run by tuwunel's `admin_execute` in the same startup
and likewise before the listener — so a fresh hive has a joined,
power-level-100 admin on its first boot.
This replaces a dependency on being the first account ever registered,
which was fragile in both directions: an appservice-created account is
excluded from that automatic grant by design, and on a homeserver that
already had users the rule never fired at all.
Promotion can't be bootstrapped over the API, and that's upstream's
design rather than a gap: tuwunel only treats an admin-room message as a
command when its sender is already an admin. `admin_execute` is the one
lever with no sender to check. hive-c0re re-checks the result on every
sweep by reading the admin account's own joined-rooms list; if the rights
are missing it says so, names
`systemctl restart container@hive-matrix` as the fix, and carries on —
agent accounts, the hive Space and the chat room need no admin.
<details><summary>Upgrading a hive that used the registration token</summary>
Nothing to do, and nothing to time. The activation script mints the
appservice token and renders the registration before the homeserver
restarts, so the first boot after the switch already has both halves.
- **Existing accounts keep working.** An access token lives on the
device that minted it; removing the registration token touches no
device, no account and no session. `login_with_password` stays on, so
the password fallback is still there too.
- **Existing token files are honoured.** The per-agent sweep skips any
agent that already has a `matrix-token`, so no account is re-registered
and no session is displaced.
- **The admin account is already admin** on such a hive (it won the
first-user grant when the hive was new), so the startup promotion is a
no-op — upstream's `make_user_admin` short-circuits when the user is
already joined at power level 100.
- **`/var/lib/hyperhive/matrix-register-token` is left on disk**, read by
nothing. Delete it or leave it; neither does any harm.
- **`registrationTokenFile` is a removed option.** A config that still
sets it fails to evaluate with a message naming the appservice — a hive
that never set it (the default) is unaffected.
- **A swarm store holding the old `matrix/registration-token` path** is
no longer read at all. The value that matters now lives at
`matrix/appservice-token`, and `swarm-secret-publish` on the authelia
host mints and `put`s it there — the hive uses its locally minted
token only until the first successful read. See
[`../swarm/secrets.md`](../swarm/secrets.md) for how that mint stays
idempotent across runs.
</details>
Initial rollout settings:
- `allow_federation = true` at the protocol level so swarms can be
wired up later by extending `trustedServers` without a homeserver
restart. `trusted_servers = []` keeps it effectively closed
until you list peers.
- `allow_registration = true` (required for the token flow to
engage). The absent
`yes_i_am_very_very_sure_…_open_registration_…` flag keeps the
server closed to anyone without the token.
- `allow_registration = false`. tuwunel checks this flag only for
requests that arrive **without** an appservice token, so hive-c0re
provisions exactly as before and everyone else is refused. It's not a
hardening afterthought: with no registration token configured,
`allow_registration = true` makes tuwunel refuse to start unless
`yes_i_am_very_very_sure_…_open_registration_…` is also set.
- `allow_encryption` — server-side E2EE switch, sourced from
`services.hyperhive.swarm.matrix.allowEncryption` (**default `false`**, opt-in).
Off by default because on the hive-internal homeserver the operator

58
docs/swarm/secrets.md
View file

@ -132,32 +132,62 @@ when the system builds. The server names the offending file and refuses to run.
## Hive-level — one of each per hive
| secret | generated by | lives at |
| ---------------------------------------- | --------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| hive CA cert + key | `hive-tls.nix` first-boot unit | `<deploy.hive-controller.tls.stateDir>/ca.pem`, `ca-key.pem` (`0600`) |
| hive leaf certs | `hive-tls.nix`, signed by the hive CA | `<deploy.hive-controller.tls.stateDir>/<name>.pem` |
| matrix registration token | a host activation script, on first boot | `/var/lib/hyperhive/matrix-register-token` (`0600`) |
| the forge's copy of its OIDC secret | `hive-forge-oidc-secret.service` copies it from authelia's tree | `/var/lib/forgejo-oidc/<id>.secret` inside the forge container |
| the homeserver's copy of its OIDC secret | `hive-matrix-oidc-secret.service`, same shape | `/var/lib/tuwunel-oidc/<id>.secret`, handed to tuwunel through `LoadCredential` |
| the agent containers' queue credential | authelia, published to the store by `swarm-secret-publish` | `<deploy.hive-controller.queue.agentCredentialDir>/secret` (`0600`) and `/client_id` (`0644`) |
| secret | generated by | lives at |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------- |
| hive CA cert + key | `hive-tls.nix` first-boot unit | `<deploy.hive-controller.tls.stateDir>/ca.pem`, `ca-key.pem` (`0600`) |
| hive leaf certs | `hive-tls.nix`, signed by the hive CA | `<deploy.hive-controller.tls.stateDir>/<name>.pem` |
| matrix appservice token | `swarm-secret-publish`, published to the store; a host activation script only as a first-boot fallback | `/var/lib/hyperhive/matrix-appservice-token` (`0600`) |
| the forge's copy of its OIDC secret | `hive-forge-oidc-secret.service` copies it from authelia's tree | `/var/lib/forgejo-oidc/<id>.secret` inside the forge container |
| the homeserver's copy of its OIDC secret | `hive-matrix-oidc-secret.service`, same shape | `/var/lib/tuwunel-oidc/<id>.secret`, handed to tuwunel through `LoadCredential` |
| the agent containers' queue credential | authelia, published to the store by `swarm-secret-publish` | `<deploy.hive-controller.queue.agentCredentialDir>/secret` (`0600`) and `/client_id` (`0644`) |
Both delivery units wait for authelia's first boot to mint the secret — a
bounded wait, 120s — and then **fail loudly** rather than skipping. A silent skip
produces a service whose login button always fails, which is a symptom many
layers from its cause.
The store's **first reader** is the matrix registration token, and it's worth
The store's **first reader** is the matrix appservice token, and it's worth
saying why that one: it's an opaque 32-byte value with no second file and no
format. Authelia's OIDC secret needs a `.secret` _and_ a matching `.digest`, so
starting there would have meant debugging "can a reader authenticate and get
bytes back" and "is authelia's file format right" at once, with an
SSO outage as the failure mode.
`glue-matrix-bao-token.nix` fetches it and writes the file `hive-matrix.nix`
already reads, so the homeserver never learns the store exists. Every failure
path — no such key, sealed store, unreachable store, empty value — leaves the
locally minted token in place, so a hive with no store behaves exactly as it
did before.
`glue-matrix-bao-token.nix` fetches it, writes the file `hive-matrix.nix`
already reads, and then runs that module's own renderer to re-stamp the
appservice registration naming the token — so the homeserver never learns the
store exists. The re-render isn't housekeeping: the token is half an
agreement, and a registration carrying the previous value authenticates
nobody. Every failure path — no such key, sealed store, unreachable store,
empty value — leaves the locally minted token in place, so a hive with no store
behaves exactly as it did before.
The store path is `swarm/hives/<hive>/matrix/appservice-token`, and the
**producer is `swarm-secret-publish`** — the same unit that copies authelia's
OIDC secrets in. It mints one token per hive in the swarm's roster and `put`s
it there, so the store is the source of truth and every hive converges on the
value it holds. The hive's own activation mint is still there, but it's a
first-boot fallback now: it fires only when the token file is absent, and the
reader overwrites whatever it produced.
The mint is **idempotent by keeping its own record**, not by asking the store.
The publisher's grant is `create`/`update` under `swarm/hives/*` with no
`read` — write-only on purpose, so a file-copier can't recover every hive's
credentials — which means it can't check whether a hive already has a token.
Instead it keeps the value it minted under its `StateDirectory`,
`/var/lib/swarm-secret-publish/matrix-appservice-token/<hive>` (`0700` dir,
`0600` file), and mints only when that file is missing or empty. The `put`
itself runs every time: re-putting the same bytes changes nothing for any
reader, while a mint whose publish failed must not be left as a token this
host holds and no hive can reach.
Lose the state directory and the next run mints once more and republishes.
That rotates the token, which readers pick up on their next start — nothing
that already registered breaks, because the token authenticates the
appservice rather than living inside any account it created.
The path was `…/matrix/registration-token` while the homeserver still took a
shared registration secret; a value left at the old path is read by nothing.
The **second reader** is the agent containers' queue credential:
`glue-queue-agent-credential.nix` lands it as two files, the client secret and

6
docs/swarm/sso.md
View file

@ -168,9 +168,9 @@ Two consequences worth stating plainly:
- **tuwunel re-reads its secret file on every OAuth exchange**, not only
at startup, and its own sandboxing hides most paths from it. It gets the
file through `LoadCredential` for the same reason the registration token
does — that keeps `DynamicUser` and `PrivateUsers` intact, with no
host-side ownership arrangement to maintain.
file through `LoadCredential` for the same reason the appservice
registration does — that keeps `DynamicUser` and `PrivateUsers` intact,
with no host-side ownership arrangement to maintain.
- **Matrix SSO lives inside the homeserver.** The client-server API is
spoken by non-browser clients holding matrix access tokens — every
agent's own daemon — as well as by federation, so the homeserver

510
hive-c0re/src/matrix.rs
View file

@ -1,12 +1,19 @@
//! Optional matrix-tuwunel wiring: shared registration token (host) +
//! per-agent UIAA registration → `<agent-state>/matrix-token`. No-op
//! Optional matrix-tuwunel wiring: the hive's appservice identity (host) +
//! per-agent account creation → `<agent-state>/matrix-token`. No-op
//! when the `hive-matrix` container isn't running, so operators who
//! haven't flipped `services.hyperhive.deploy.matrix.enable = true` pay
//! nothing.
//!
//! See `docs/integrations/matrix.md::Provisioning flow (registration token)` for
//! the full UIAA round-trip, token-file shape, and host/container
//! bind-mount layout.
//! Accounts are created **as the hive's appservice**, not by presenting a
//! shared registration token in a UIAA flow. The difference that matters
//! here is not the round-trip count: an appservice token is an *identity*
//! the homeserver knows, so the secret never has to be the same on both
//! sides of the wire, and account creation does not depend on registration
//! being open to anyone who learns a token.
//!
//! See `docs/integrations/matrix.md::Provisioning flow (appservice)` for the
//! registration file's shape, how its token reaches both halves, and the
//! host/container bind-mount layout.
use std::path::PathBuf;
@ -49,11 +56,8 @@ fn matrix_base() -> Result<&'static str> {
this path should have been gated on matrix::is_present()",
)
}
/// Length (bytes) of the random registration token. 32 raw bytes ⇒
/// 64-char hex string; comfortable for a long-lived shared secret.
const REGISTER_TOKEN_BYTES: usize = 32;
/// HTTP timeout for registration round-trips. UIAA is two POSTs; even
/// the slow path should finish well inside this budget.
/// HTTP timeout for registration round-trips. Account creation is one
/// POST; even the slow path should finish well inside this budget.
const HTTP_TIMEOUT_SECS: u64 = 10;
/// Length (bytes) of the throwaway per-agent matrix password. Random
/// 32-byte hex — agents never log in with the password (they
@ -61,10 +65,17 @@ const HTTP_TIMEOUT_SECS: u64 = 10;
/// store it nowhere.
const PASSWORD_BYTES: usize = 32;
/// Matrix localpart for the hive system admin account. Registered
/// before any agent account in [`ensure_all`] so it becomes the first
/// user on the homeserver — Conduit/tuwunel grants admin rights to the
/// first registered user automatically. Not an agent; has no state dir.
/// Matrix localpart for the hive system admin account. Not an agent; has
/// no state dir.
///
/// Also the `sender_localpart` of the hive's appservice registration,
/// which is what creates this account on a homeserver that has never had
/// one: the homeserver creates a registration's sender user itself, at
/// startup, before it accepts a request. The account's *admin rights*
/// then come from the `admin_execute` promotion beside that registration
/// — see `nix/host-modules/hive-matrix.nix`, where this same literal
/// appears as `adminLocalpart`. **The two must match**; nothing wires an
/// override across.
pub const HIVE_ADMIN_LOCALPART: &str = "hive";
/// Display name of the hive Space. Plain text, no special characters, so
@ -164,30 +175,36 @@ fn random_hex(n: usize) -> Result<String> {
Ok(hex)
}
/// Ensure the registration token file exists; returns its contents.
/// Generates a fresh 32-byte hex token on first call (mode 0600,
/// root-only), then re-reads on subsequent calls. The hive-matrix
/// nixos module bind-mounts this file read-only into the tuwunel
/// container so the homeserver can authenticate registration requests
/// against the same secret hive-c0re holds.
pub fn ensure_register_token() -> Result<String> {
use std::os::unix::fs::PermissionsExt;
let path = crate::paths::matrix_register_token();
if let Ok(existing) = std::fs::read_to_string(&path) {
let trimmed = existing.trim().to_owned();
if !trimmed.is_empty() {
return Ok(trimmed);
}
}
let token = random_hex(REGISTER_TOKEN_BYTES)?;
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent).ok();
}
std::fs::write(&path, format!("{token}\n"))
.with_context(|| format!("write registration token to {}", path.display()))?;
let _ = std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600));
tracing::info!(path = %path.display(), "matrix: generated registration token");
Ok(token)
/// Read the hive's appservice token — the `as_token` of the registration
/// the homeserver loaded at boot. Every account this module creates is
/// authorised by it.
///
/// **Reads, never mints**, unlike the registration token it replaced.
/// That token was the whole agreement, so whichever side wrote it first
/// was right; this one has a second half — the registration file naming
/// it, which only the nix side writes. A token minted here would be a
/// token the homeserver has never heard of, and the failure would surface
/// as every request being refused rather than as a missing file.
///
/// # Errors
/// When the file is absent or empty. That means the host activation
/// script has not run on this generation yet; callers log it and leave
/// existing accounts alone rather than trying to proceed.
pub fn read_appservice_token() -> Result<String> {
let path = crate::paths::matrix_appservice_token();
std::fs::read_to_string(&path)
.ok()
.map(|s| s.trim().to_owned())
.filter(|s| !s.is_empty())
.with_context(|| {
format!(
"matrix appservice token not found at {} — it is minted by the \
hive-matrix activation script, which also renders the registration \
file naming it; deploy the hive-matrix module (or re-run \
`nixos-rebuild switch`) before provisioning matrix users",
path.display()
)
})
}
/// Build the localpart of a matrix user id for `agent`. Matrix
@ -198,19 +215,20 @@ fn user_localpart(agent: &str) -> &str {
agent
}
/// Send a single registration POST and parse the response. Returns
/// `Ok((status, body))` on any successful HTTP round-trip (including
/// the expected 401 from the first UIAA leg); errors only on transport
/// failure. Body is the parsed JSON value — UIAA passes session +
/// flow state via JSON, never via headers.
/// Send the registration POST as the appservice and parse the response.
/// Returns `Ok((status, body))` on any completed HTTP round-trip
/// (including the `M_USER_IN_USE` 400 the caller treats as "already
/// exists"); errors only on transport failure.
async fn register_post(
client: &reqwest::Client,
as_token: &str,
body: &serde_json::Value,
) -> Result<(StatusCode, serde_json::Value)> {
let base = matrix_base()?;
let url = format!("{base}/_matrix/client/v3/register");
let resp = client
.post(&url)
.bearer_auth(as_token)
.json(body)
.send()
.await
@ -233,24 +251,38 @@ pub fn random_password() -> Result<String> {
random_hex(PASSWORD_BYTES)
}
/// Run the matrix-spec UIAA flow to register `agent` with the given
/// `password` and return the resulting access token. Two round-trips:
/// first POST elicits the 401 + session id, second POST supplies the
/// registration token in the `auth` block. If the homeserver returns
/// 200 on the first POST (no flow stages required — `allow_registration`
/// with no token), we take the access token directly.
/// Create the matrix account for `agent` as the hive's appservice and
/// return an access token for it. One round-trip: an appservice-typed
/// registration needs no UIAA stage at all, so there is no session to
/// carry and no shared secret to present.
///
/// The account is created **by** the appservice but is an ordinary user
/// afterwards — it gets its own device and its own access token, and the
/// agent authenticates with that rather than with anything the hive
/// holds. The `as_token` never leaves the host.
///
/// Caller picks the password: agents use [`random_password`] (throwaway
/// — they auth by `access_token`), operators on the `hivectl` path
/// supply their own so they can log into matrix web clients.
///
/// # Errors
/// Propagates the homeserver's own body, which is what the
/// `M_USER_IN_USE` callers match on. A `M_EXCLUSIVE` body means the
/// localpart falls outside the appservice's namespace — the registration
/// file's `namespaces.users` regex is the place to look, not this call.
async fn register_user(
client: &reqwest::Client,
agent: &str,
register_token: &str,
as_token: &str,
password: &str,
) -> Result<String> {
let localpart = user_localpart(agent);
let initial = serde_json::json!({
let body = serde_json::json!({
// What makes this an appservice registration rather than an
// ordinary one. Without it the homeserver treats the request as a
// normal client's and asks for a UIAA flow — even holding the
// as_token.
"type": "m.login.application_service",
"username": localpart,
"password": password,
// device_id stays stable across re-runs of ensure_user_for so
@ -259,37 +291,53 @@ async fn register_user(
"initial_device_display_name": format!("hyperhive ({agent})"),
"inhibit_login": false,
});
let (status, body) = register_post(client, &initial).await?;
// Some servers accept the first POST when allow_registration is
// open. Take the access_token + we're done.
if status.is_success() {
return extract_access_token(&body);
}
if status != StatusCode::UNAUTHORIZED {
anyhow::bail!("matrix: /register first leg HTTP {status}, body: {body}");
}
let session = body["session"]
.as_str()
.with_context(|| format!("matrix: missing UIAA session in 401, body: {body}"))?;
let authed = serde_json::json!({
"username": localpart,
"password": password,
"device_id": format!("hyperhive-{agent}"),
"initial_device_display_name": format!("hyperhive ({agent})"),
"inhibit_login": false,
"auth": {
"type": "m.login.registration_token",
"token": register_token,
"session": session,
},
});
let (status, body) = register_post(client, &authed).await?;
let (status, body) = register_post(client, as_token, &body).await?;
if !status.is_success() {
anyhow::bail!("matrix: /register auth leg HTTP {status}, body: {body}");
anyhow::bail!("matrix: /register as appservice HTTP {status}, body: {body}");
}
extract_access_token(&body)
}
/// Log in as an **existing** account using the hive's appservice token,
/// and return a fresh access token for it. No password involved: the
/// appservice is authorised for every localpart in its namespace, so it
/// can mint a session for one without knowing anything about the account.
///
/// This is the recovery path that used to need a stored password or an
/// admin-room password reset — an account whose token file was lost is
/// re-tokened from the hive's own identity instead. The device id matches
/// [`register_user`]'s, so a re-login replaces that device's token rather
/// than accumulating devices.
async fn appservice_login(client: &reqwest::Client, as_token: &str, agent: &str) -> Result<String> {
let base = matrix_base()?;
let url = format!("{base}/_matrix/client/v3/login");
let body = serde_json::json!({
"type": "m.login.application_service",
"identifier": {
"type": "m.id.user",
"user": user_localpart(agent),
},
"device_id": format!("hyperhive-{agent}"),
"initial_device_display_name": format!("hyperhive ({agent})"),
});
let resp = client
.post(&url)
.bearer_auth(as_token)
.json(&body)
.send()
.await
.context("matrix: POST /login as appservice")?;
let status = resp.status();
let json = resp
.json::<serde_json::Value>()
.await
.context("matrix: parse appservice /login response")?;
if !status.is_success() {
anyhow::bail!("matrix: appservice /login HTTP {status} for {agent}, body: {json}");
}
extract_access_token(&json)
}
/// Pull `access_token` out of a successful /register response.
fn extract_access_token(body: &serde_json::Value) -> Result<String> {
body["access_token"]
@ -637,11 +685,7 @@ async fn admin_room_reset_password(
///
/// `client` is shared across the sweep so we build one reqwest
/// connection pool for all agents rather than one per call.
pub async fn ensure_user_for(
client: &reqwest::Client,
name: &str,
register_token: &str,
) -> Result<()> {
pub async fn ensure_user_for(client: &reqwest::Client, name: &str, as_token: &str) -> Result<()> {
use std::os::unix::fs::PermissionsExt;
let agent = hive_types::Ident::parse(name)
.map_err(|e| anyhow::anyhow!("invalid agent name {name:?}: {e}"))?;
@ -678,7 +722,7 @@ pub async fn ensure_user_for(
}
let password = random_password()?;
let access_token = match register_user(client, name, register_token, &password).await {
let access_token = match register_user(client, name, as_token, &password).await {
Ok(token) => {
// Successful registration — persist the password so we can
// fall back to login if the token file is deleted later.
@ -694,8 +738,21 @@ pub async fn ensure_user_for(
token
}
Err(reg_err) if reg_err.to_string().contains("M_USER_IN_USE") => {
// Account already exists — try to re-login with the stored password.
tracing::info!(%name, "matrix: user already exists, attempting login with stored password");
// Account already exists and its token file was lost. The
// appservice can mint a session for any account in its
// namespace, so this needs no password and no admin rights —
// try it before the password paths below, which exist for
// accounts created before the appservice did (or named
// outside its namespace) and stay as the fallback.
tracing::info!(%name, "matrix: user already exists, logging in as the appservice");
match appservice_login(client, as_token, name).await {
Ok(token) => return finish_user_provisioning(name, &token).await,
Err(e) => tracing::warn!(
%name,
error = ?e,
"matrix: appservice login failed; falling back to the stored password"
),
}
let pw_path = password_path(name);
let stored = if let Some(pw) = std::fs::read_to_string(&pw_path)
.ok()
@ -734,11 +791,19 @@ pub async fn ensure_user_for(
Err(other) => return Err(other),
};
finish_user_provisioning(name, &access_token).await
}
/// Persist a freshly-obtained agent access token and kick the agent's
/// matrix daemon. Shared by every way [`ensure_user_for`] can end up
/// holding a token — creation, appservice login, password login — so a
/// new recovery path cannot forget half of it.
async fn finish_user_provisioning(name: &str, access_token: &str) -> Result<()> {
// Write the token via hive-priv (root helper): hive-c0re runs as the
// unprivileged `hive-core` user and cannot write to agent-owned state
// directories directly. hive-priv writes the file 0600 and chowns it
// to the agent user so it is readable from inside the container.
crate::priv_client::write_agent_matrix_token(name, &access_token, None, None)
crate::priv_client::write_agent_matrix_token(name, access_token, None, None)
.await
.with_context(|| format!("matrix: write matrix-token for {name} via hive-priv"))?;
tracing::info!(%name, "matrix: provisioned access token");
@ -767,30 +832,30 @@ pub async fn ensure_user_for(
/// behaviour.
///
/// **Not idempotent** (unlike [`crate::forge::provision_user_token`]): the
/// matrix UIAA `/register` endpoint returns `M_USER_IN_USE` (HTTP 400)
/// on second call for the same localpart. Callers re-running this for
/// a known-existing matrix user should expect a hard error from this
/// fn and route to a password-reset path instead.
/// matrix `/register` endpoint returns `M_USER_IN_USE` (HTTP 400) on a
/// second call for the same localpart, appservice-authorised or not.
/// Callers re-running this for a known-existing matrix user should expect
/// a hard error from this fn and route to a password-reset path instead.
pub async fn provision_user_token(
client: &reqwest::Client,
name: &str,
register_token: &str,
as_token: &str,
password: &str,
) -> Result<String> {
register_user(client, name, register_token, password).await
register_user(client, name, as_token, password).await
}
/// Per-agent matrix sync: ensure the agent has a matrix account + token.
/// All operations are idempotent; failures are logged as warnings but
/// don't abort the caller.
pub async fn sync_agent(client: &reqwest::Client, name: &str, register_token: &str) {
if let Err(e) = ensure_user_for(client, name, register_token).await {
pub async fn sync_agent(client: &reqwest::Client, name: &str, as_token: &str) {
if let Err(e) = ensure_user_for(client, name, as_token).await {
tracing::warn!(%name, error = ?e, "matrix: ensure_user failed");
}
}
/// Standalone per-agent sync that handles its own setup: checks if
/// hive-matrix is present, reads the registration token, and builds
/// hive-matrix is present, reads the appservice token, and builds
/// an HTTP client before delegating to [`sync_agent`]. Mirrors the
/// setup in [`ensure_all`] so the rebuild path and the startup sweep
/// stay equivalent. No-op when the matrix container is absent.
@ -798,10 +863,10 @@ pub async fn sync_agent_standalone(name: &str) {
if !is_present() {
return;
}
let register_token = match ensure_register_token() {
let as_token = match read_appservice_token() {
Ok(t) => t,
Err(e) => {
tracing::warn!(%name, error = ?e, "matrix: ensure_register_token failed");
tracing::warn!(%name, error = ?e, "matrix: read_appservice_token failed");
return;
}
};
@ -815,22 +880,26 @@ pub async fn sync_agent_standalone(name: &str) {
return;
}
};
sync_agent(&client, name, &register_token).await;
sync_agent(&client, name, &as_token).await;
}
/// Ensure the hive system admin matrix user exists and its token is
/// persisted at [`admin_token_path()`]. Must be called BEFORE
/// [`ensure_all`]'s agent loop so this account is the first to register
/// and becomes the homeserver admin automatically (Conduit/tuwunel:
/// first registered user = admin).
/// Ensure the hive system admin matrix user exists, that its token is
/// persisted at [`admin_token_path()`], and that it actually holds admin
/// rights.
///
/// Idempotent — skips when the token file already exists and is
/// non-empty. Does NOT promote the account via API (that requires
/// admin rights which this fn bootstraps); on a fresh homeserver the
/// first-registered rule fires automatically; on an existing homeserver
/// the operator must promote the account once via
/// `hivectl matrix promote-user hive` or the conduit admin room.
pub async fn ensure_admin_user(client: &reqwest::Client, register_token: &str) -> Result<()> {
/// **Nothing here depends on registration order any more.** The account
/// used to have to be the first ever registered, to win tuwunel's
/// automatic first-user grant — a rule that cannot fire for an
/// appservice-created account at all, and one that silently did nothing
/// on a homeserver that already had users. Admin rights now come from an
/// explicit `make_user_admin`: the `admin_execute` entry beside the
/// appservice registration performs it at homeserver startup, and
/// [`ensure_admin_rights`] checks the result and says so when it is
/// missing.
///
/// Idempotent — skips the account work when the token file already exists
/// and is non-empty.
pub async fn ensure_admin_user(client: &reqwest::Client, as_token: &str) -> Result<()> {
use std::os::unix::fs::PermissionsExt;
let path = admin_token_path();
if path.exists()
@ -841,8 +910,7 @@ pub async fn ensure_admin_user(client: &reqwest::Client, register_token: &str) -
return Ok(());
}
let password = random_password()?;
let access_token = match register_user(client, HIVE_ADMIN_LOCALPART, register_token, &password)
.await
let access_token = match register_user(client, HIVE_ADMIN_LOCALPART, as_token, &password).await
{
Ok(token) => {
let pw_path = password_path(HIVE_ADMIN_LOCALPART);
@ -857,20 +925,34 @@ pub async fn ensure_admin_user(client: &reqwest::Client, register_token: &str) -
token
}
Err(reg_err) if reg_err.to_string().contains("M_USER_IN_USE") => {
tracing::info!("matrix: hive admin user already exists, re-logging in");
let pw_path = password_path(HIVE_ADMIN_LOCALPART);
let stored = std::fs::read_to_string(&pw_path)
.ok()
.map(|s| s.trim().to_owned())
.filter(|s| !s.is_empty())
.with_context(|| {
format!(
"matrix: hive admin user exists but password missing at {} — \
manual recovery: reset password via admin API or conduit admin room",
pw_path.display()
)
})?;
login_user(client, HIVE_ADMIN_LOCALPART, &stored).await?
// The expected path, not an edge case: this account is the
// appservice's own `sender_localpart`, so the homeserver
// creates it when it loads the registration — before
// hive-c0re gets a chance to ask. An appservice login needs
// no password, which is just as well since an account the
// homeserver created has none.
tracing::info!("matrix: hive admin user already exists, logging in as the appservice");
match appservice_login(client, as_token, HIVE_ADMIN_LOCALPART).await {
Ok(token) => token,
Err(e) => {
tracing::warn!(error = ?e, "matrix: appservice login for the hive admin failed; falling back to the stored password");
let pw_path = password_path(HIVE_ADMIN_LOCALPART);
let stored = std::fs::read_to_string(&pw_path)
.ok()
.map(|s| s.trim().to_owned())
.filter(|s| !s.is_empty())
.with_context(|| {
format!(
"matrix: hive admin user exists, appservice login failed, and no \
password is stored at {} — check that the registration file's \
namespace covers @{HIVE_ADMIN_LOCALPART} and that the homeserver \
loaded it",
pw_path.display()
)
})?;
login_user(client, HIVE_ADMIN_LOCALPART, &stored).await?
}
}
}
Err(other) => return Err(other),
};
@ -884,9 +966,149 @@ pub async fn ensure_admin_user(client: &reqwest::Client, register_token: &str) -
Ok(())
}
/// Check that the hive admin account holds admin rights, and repair it
/// through the admin room when it does not.
///
/// Admin-ness in tuwunel is membership of the admin room, so that is what
/// this reads: the account's own joined-rooms list, which needs no
/// privileges to fetch. When the admin room is in it there is nothing to
/// do and nothing is sent — worth insisting on, because the repair is a
/// message in a room and this runs on every sweep.
///
/// When it is absent, the repair is attempted anyway (a stale or failed
/// read is cheaper to retry than to reason about) and a failure is
/// reported rather than raised: agent accounts, the hive Space and the
/// chat room all work without an admin, so a hive with an unpromoted
/// admin is degraded, not broken. Only `hivectl matrix promote-user` /
/// `reset-password` need it.
///
/// Returns whether the account holds admin rights now. A `false` has
/// already been logged, with what to do about it.
async fn ensure_admin_rights(
client: &reqwest::Client,
admin_token: &str,
server_name: &str,
) -> bool {
let room_id = match discover_admin_room_id(client, admin_token, server_name).await {
Ok(id) => id,
Err(e) => {
tracing::warn!(error = ?e, "matrix: cannot resolve #admins — not checking the hive admin's rights");
return false;
}
};
match joined_rooms(client, admin_token).await {
Some(rooms) if rooms.iter().any(|r| r == &room_id) => {
tracing::debug!("matrix: hive admin is in the admin room");
return true;
}
Some(_) => {
tracing::warn!(
"matrix: hive admin is not in the admin room — attempting to promote it"
);
}
None => tracing::debug!(
"matrix: could not read the hive admin's joined rooms; attempting to promote it"
),
}
if let Err(e) =
promote_user_to_admin(client, admin_token, HIVE_ADMIN_LOCALPART, server_name).await
{
// The honest message. Promoting through the admin room requires
// being admin already, so the one lever that works on a
// homeserver with no admin at all is the `admin_execute` entry in
// the hive-matrix module — which runs at startup, which means a
// restart is the fix rather than another sweep.
tracing::warn!(
error = ?e,
"matrix: hive admin '@{HIVE_ADMIN_LOCALPART}' holds no admin rights. \
The homeserver promotes it at startup (admin_execute in hive-matrix.nix), \
so `systemctl restart container@hive-matrix` grants it. Agent accounts and \
room provisioning are unaffected; `hivectl matrix promote-user` and \
`reset-password` need it."
);
return false;
}
tracing::info!("matrix: promoted the hive admin through the admin room");
true
}
/// The rooms `token`'s account has joined, or `None` when the list could
/// not be read. `None` is deliberately not an empty list: "no rooms" and
/// "no answer" lead to different decisions in [`ensure_admin_rights`].
async fn joined_rooms(client: &reqwest::Client, token: &str) -> Option<Vec<String>> {
let base = matrix_http()?;
let url = format!("{base}/_matrix/client/v3/joined_rooms");
let resp = client.get(&url).bearer_auth(token).send().await.ok()?;
if !resp.status().is_success() {
return None;
}
let body = resp.json::<serde_json::Value>().await.ok()?;
Some(
body["joined_rooms"]
.as_array()?
.iter()
.filter_map(|r| r.as_str().map(ToOwned::to_owned))
.collect(),
)
}
/// Whether an admin-room reply says a `make-user-admin` succeeded.
///
/// Three spellings, because the reply is prose and prose changes between
/// builds. tuwunel v1.9.0's is `"<user id> has been granted admin
/// privileges."` — which the original two patterns here (`done…`,
/// `made…admin`) do not match at all, so a promotion that had already
/// worked was reported as a 15-second timeout. The older spellings are
/// kept: a homeserver is not necessarily the version this was written
/// against.
fn is_make_admin_success(body: &str) -> Option<()> {
let lower = body.to_ascii_lowercase();
let says_ok = lower.starts_with("done")
|| lower.contains("granted admin privileges")
|| (lower.contains("made") && lower.contains("admin"));
says_ok.then_some(())
}
#[cfg(test)]
mod is_make_admin_success_tests {
use super::is_make_admin_success;
/// The reply tuwunel v1.9.0 actually sends
/// (`src/admin/user/make_user_admin.rs`). This is the case the
/// pre-existing matcher missed.
#[test]
fn tuwunel_1_9_grant_reply() {
let msg = "@hive:pr1ma.darkest.space has been granted admin privileges.";
assert_eq!(is_make_admin_success(msg), Some(()));
}
#[test]
fn older_spellings_still_match() {
assert_eq!(
is_make_admin_success("Done: user is now an admin"),
Some(())
);
assert_eq!(is_make_admin_success("Made @x:y an admin"), Some(()));
}
/// The control: an unrelated or failing reply must not read as
/// success, or a failed promotion returns Ok and the warning that
/// would have named it never fires.
#[test]
fn failures_and_noise_do_not_match() {
assert_eq!(is_make_admin_success("Command not recognised."), None);
assert_eq!(is_make_admin_success("User @x:y does not exist"), None);
}
}
/// Promote a user to homeserver admin via the Matrix admin room
/// (`#admins:<server>`). Sends `!admin users make-user-admin @<localpart>:<server>` as
/// @hive, polls for the bot's "Done:" success response.
/// @hive, polls for the bot's success reply.
///
/// ⚠️ Requires the **sender** to be an admin already — tuwunel only
/// treats a message as a command when its sender is in the admin room.
/// So this promotes a *second* user; it cannot bootstrap the first one.
/// That is what the `admin_execute` entry in `hive-matrix.nix` is for.
///
/// Goes through the admin room rather than a direct HTTP call because
/// tuwunel implements parts of the Synapse admin API but not user
@ -907,14 +1129,7 @@ pub async fn promote_user_to_admin(
server_name,
&room_url,
&command,
|body| {
let lower = body.to_ascii_lowercase();
if lower.starts_with("done") || lower.contains("made") && lower.contains("admin") {
Some(())
} else {
None
}
},
is_make_admin_success,
)
.await
.with_context(|| {
@ -1637,10 +1852,14 @@ pub async fn ensure_all() -> bool {
return true;
}
let mut ok = true;
let register_token = match ensure_register_token() {
// Loud and non-destructive: with no appservice token this sweep can
// create nothing, so it does nothing. Agents that already hold a
// token keep using it — their accounts and sessions are untouched by
// anything in here.
let as_token = match read_appservice_token() {
Ok(t) => t,
Err(e) => {
tracing::warn!(error = ?e, "matrix: ensure_register_token failed");
tracing::warn!(error = ?e, "matrix: no appservice token; skipping the user sweep");
return false;
}
};
@ -1656,10 +1875,12 @@ pub async fn ensure_all() -> bool {
return false;
}
};
// Provision hive admin user FIRST so it's the first registered
// account on a fresh homeserver (Conduit/tuwunel makes the first
// registered user admin automatically).
if let Err(e) = ensure_admin_user(&client, &register_token).await {
// The hive admin first, because everything below provisions THROUGH
// it (the Space, the chat room and every invite are sent with its
// token). Not, any more, so that it wins a first-registered-user
// grant: it holds admin rights by explicit promotion, checked in
// `provision_space` once the server name is known.
if let Err(e) = ensure_admin_user(&client, &as_token).await {
tracing::warn!(error = ?e, "matrix: ensure_admin_user failed");
ok = false;
}
@ -1672,7 +1893,7 @@ pub async fn ensure_all() -> bool {
let Some(name) = c.strip_prefix(crate::lifecycle::AGENT_PREFIX) else {
continue;
};
sync_agent(&client, name, &register_token).await;
sync_agent(&client, name, &as_token).await;
agent_names.push(name.to_owned());
}
@ -1705,6 +1926,11 @@ async fn provision_space(client: &reqwest::Client, agent_names: &[String]) -> bo
return false;
}
};
// Before the rooms: the one thing in this sweep that is about the
// admin account itself rather than about what it provisions.
if !ensure_admin_rights(client, &admin_token, &server_name).await {
ok = false;
}
let room_id = match ensure_hive_space(client, &admin_token).await {
Ok(id) => id,
Err(e) => {

15
hive-c0re/src/paths.rs
View file

@ -253,14 +253,15 @@ pub fn gateway_agents_conf() -> PathBuf {
// `nix/host-modules/hive-c0re/default.nix` and `nix/host-modules/hive-ci.nix` — must match.
pub const FORGE_CORE_TOKEN: &str = "/var/lib/hyperhive/forge-core-token";
/// `matrix-register-token` — shared matrix registration token.
// nix: bind-mounted into the tuwunel/matrix container (hive-matrix.nix) — must match.
// Not operator-option-driven: `registrationTokenFile` is `internal` on the nix
// side, and an `assertions` entry there rejects any attempt to move it, so this
// literal can never diverge from it.
/// `matrix-appservice-token` — the `as_token` of the hive's appservice
/// registration, which authorises every account this daemon creates.
// nix: minted by the `hive-matrix-appservice` activation script in
// `nix/host-modules/hive-matrix.nix`, which renders it into the registration
// file the homeserver loads — must match. Read-only here on purpose: a token
// minted on this side would not be the one in that file.
#[must_use]
pub fn matrix_register_token() -> PathBuf {
state_root().join("matrix-register-token")
pub fn matrix_appservice_token() -> PathBuf {
state_root().join("matrix-appservice-token")
}
/// `/run/hyperhive` — the runtime root (host admin socket + per-agent dirs).

14
hive-c0re/src/server.rs
View file

@ -558,8 +558,8 @@ async fn handle_matrix_create_user(
password: Option<&str>,
) -> Result<HostResponse> {
require_matrix_present()?;
let register_token =
crate::matrix::ensure_register_token().context("read matrix register token")?;
let as_token =
crate::matrix::read_appservice_token().context("read matrix appservice token")?;
let client = matrix_http_client()?;
let mut out = Vec::new();
if agent_exists(name)? {
@ -571,7 +571,7 @@ async fn handle_matrix_create_user(
"matrix create-user: a password is for non-agent (operator) accounts only; '{name}' is an agent which authenticates via access_token"
);
}
crate::matrix::ensure_user_for(&client, name.as_str(), &register_token)
crate::matrix::ensure_user_for(&client, name.as_str(), &as_token)
.await
.with_context(|| format!("matrix create-user {name}"))?;
let path = Coordinator::agent_notes_dir(name).join("matrix-token");
@ -585,7 +585,7 @@ async fn handle_matrix_create_user(
let token = crate::matrix::provision_user_token(
&client,
name.as_str(),
&register_token,
&as_token,
&effective_password,
)
.await
@ -770,10 +770,10 @@ async fn handle_push_snapshot(
async fn handle_matrix_sync_admin() -> Result<HostResponse> {
require_matrix_present()?;
let register_token =
crate::matrix::ensure_register_token().context("read matrix register token")?;
let as_token =
crate::matrix::read_appservice_token().context("read matrix appservice token")?;
let client = matrix_http_client()?;
crate::matrix::ensure_admin_user(&client, &register_token)
crate::matrix::ensure_admin_user(&client, &as_token)
.await
.context("matrix sync-admin")?;
let path = crate::matrix::admin_token_path();

8
hive-matrix-mcp/src/client.rs
View file

@ -5,10 +5,10 @@
//! to start sync.
//!
//! No OAuth dance / cross-signing setup (in contrast to damocles-daemon's
//! ccc.de connection): for the in-hive tuwunel the `registration_token`
//! UIAA flow already minted the token + user/device, hive-c0re just
//! handed us the bearer in a file. matrix-sdk's `restore_session` with
//! a constructed `MatrixSession` skips the login flow entirely.
//! ccc.de connection): for the in-hive tuwunel hive-c0re already minted
//! the token + user/device as the hive's appservice and handed us the
//! bearer in a file. matrix-sdk's `restore_session` with a constructed
//! `MatrixSession` skips the login flow entirely.
//!
//! E2EE is enabled via `with_encryption_settings(EncryptionSettings::default())`.
//! Crypto keys are persisted in the sqlite store under `state_dir`

27
nix/host-modules/deploy.nix
View file

@ -247,10 +247,29 @@ in
[ "services" "hyperhive" "swarm" "matrix" "maxRequestSize" ]
[ "services" "hyperhive" "deploy" "matrix" "maxRequestSize" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "matrix" "registrationTokenFile" ]
[ "services" "hyperhive" "deploy" "matrix" "registrationTokenFile" ]
)
# Both spellings of the registration token, removed rather than renamed:
# the homeserver no longer accepts a shared registration secret at all,
# so there is no path to point a definition at. Accounts are created by
# the hive's appservice, whose token lives at a fixed path this module
# deliberately does not let anyone move.
(lib.mkRemovedOptionModule [ "services" "hyperhive" "swarm" "matrix" "registrationTokenFile" ] ''
The matrix registration token is gone: hive-c0re creates accounts as the
hive's appservice instead, so there is no shared secret for tuwunel to
check. Remove this definition — nothing needs replacing, the appservice
token is minted and delivered automatically
(/var/lib/hyperhive/matrix-appservice-token, not operator-settable).
The old token file at /var/lib/hyperhive/matrix-register-token is read by
nothing now and can be deleted. See docs/integrations/matrix.md.
'')
(lib.mkRemovedOptionModule [ "services" "hyperhive" "deploy" "matrix" "registrationTokenFile" ] ''
The matrix registration token is gone: hive-c0re creates accounts as the
hive's appservice instead, so there is no shared secret for tuwunel to
check. Remove this definition — nothing needs replacing, the appservice
token is minted and delivered automatically
(/var/lib/hyperhive/matrix-appservice-token, not operator-settable).
The old token file at /var/lib/hyperhive/matrix-register-token is read by
nothing now and can be deleted. See docs/integrations/matrix.md.
'')
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "matrix" "gui" "enable" ]
[ "services" "hyperhive" "deploy" "matrix" "gui" "enable" ]

29
nix/host-modules/glue-matrix-bao-token.nix
View file

@ -1,8 +1,9 @@
# Glue: the matrix registration token comes from the secret store.
# Glue: the matrix appservice token comes from the secret store.
#
# The store's first reader, and deliberately a small one. It fetches an opaque
# 32-byte value and writes it where ./hive-matrix.nix already looks — the
# homeserver never learns the store exists, and its config is unchanged.
# 32-byte value and writes it where ./hive-matrix.nix already looks, then asks
# that module's own renderer to re-stamp the appservice registration naming it
# — the homeserver never learns the store exists, and its config is unchanged.
#
# ⚠️ Why this credential first. It has no second file and no format: authelia's
# OIDC secret needs a `.secret` *and* a matching `.digest`, so shipping that
@ -47,12 +48,12 @@ let
# own grant covers. The store's read policy grants `swarm/agents/*` and
# `swarm/hives/<this hive>/*` and nothing else, so a path outside those is a
# 403 rather than a miss, however correct it looks. `swarm-secret-client`'s
# `matrix::registration_token_path` builds the same string from the same
# `matrix::appservice_token_path` builds the same string from the same
# pieces; this literal is the nix half of that one agreement.
#
# `hiveName` has no fallback here for the reason ./glue-bao-tls.nix gives at
# its own use of it: it is asserted set for every hyperhive host.
tokenPath = "secret/swarm/hives/${hyperhiveCfg.hiveName}/matrix/registration-token";
tokenPath = "secret/swarm/hives/${hyperhiveCfg.hiveName}/matrix/appservice-token";
# A literal, not an option — ./hive-matrix.nix names its container
# `containers.hive-matrix` directly and declares no `machine` to derive it
@ -70,7 +71,7 @@ in
services.hyperhive.swarm.otel.journaldUnits = [ "swarm-bao-matrix-token" ];
systemd.services.swarm-bao-matrix-token = {
description = "fetch the matrix registration token from the swarm secret store";
description = "fetch the matrix appservice token from the swarm secret store";
# Every one of these names a unit that exists only where the store runs.
# `Requires=` on an absent unit fails the job outright, so the ordering is
# conditional even though the read is not: off-host there is nothing local
@ -176,8 +177,20 @@ in
fi
umask 077
printf '%s\n' "$token" > ${lib.escapeShellArg (toString deployCfg.matrix.registrationTokenFile)}
chmod 0600 ${lib.escapeShellArg (toString deployCfg.matrix.registrationTokenFile)}
printf '%s\n' "$token" > ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)}
chmod 0600 ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)}
# Re-stamp the registration file from the token just written. The
# token is half an agreement — the registration the homeserver loads
# has to carry the same value — so writing the file and stopping
# would leave the homeserver authenticating hive-c0re against
# whatever activation put there: a 401 on every request, naming
# nothing. Unconditional rather than on-change, because this unit
# has no way to know what the registration currently says.
#
# hive-matrix's own renderer rather than a `printf` here, so the
# registration's shape has one home.
${deployCfg.matrix.appserviceRegistrationScript}
'';
};
};

2
nix/host-modules/glue-queue-agent-credential.nix
View file

@ -15,7 +15,7 @@
# that is correct rather than degraded. The publisher runs on the authelia
# host and authelia mints on its first boot, so "nothing at that path yet" is
# the ordinary early state of a swarm. Nothing here writes a local stand-in:
# unlike a matrix registration token there is no such thing as a locally valid
# unlike a matrix appservice token there is no such thing as a locally valid
# OIDC client secret, so a placeholder would turn a hive that cannot connect
# into one that is refused, which reaches the agent as a timeout.
#

339
nix/host-modules/hive-matrix.nix
View file

@ -40,22 +40,155 @@ let
# login into an intermittent one.
matrixSecretPath = "/var/lib/tuwunel-oidc/${cfg.sso.clientId}.secret";
# Single source for the registration-token path: the option's `default`,
# its `config`-level `mkDefault` self-definition, and the assertion that
# rejects a moved path all read this same binding rather than repeating
# the string literal. See `registrationTokenFile`'s own comment below.
registrationTokenPath = "/var/lib/hyperhive/matrix-register-token";
# ⚠️ tuwunel does NOT read the path above directly, and this indirection
# ⚠️ tuwunel does NOT read the host path directly, and this indirection
# is not ceremony. Upstream's own words: "under systemd the path must be
# visible to the service after sandboxing (ReadWritePaths / ProtectHome),
# typically by placing the file under /etc/tuwunel/" — which this
# container has no writable etc for. `LoadCredential` is the answer
# already in use two units below for the registration token, and for the
# already in use for the appservice registration below, and for the
# same reason: it keeps `DynamicUser=true` + `PrivateUsers=true` intact
# with no host-side chown or GID pinning.
matrixSecretCredential = "/run/credentials/tuwunel.service/oidc_client_secret";
# How this hive creates matrix accounts: an appservice registration whose
# `url` is null. Null is a legal `url` (ruma's `Registration` types it
# `Option<String>`), and it is the whole point — with no URL the homeserver
# never calls out, so there is no HTTP service to run and no daemon to
# operate. What the registration delivers is the `as_token`: hive-c0re's
# standing authority to create, and log in as, the accounts named by the
# namespace below, with no shared registration secret in the picture.
appserviceId = "hyperhive";
# The appservice's own user, and deliberately the hive admin account.
# Loading a registration CREATES its `sender_localpart` user when absent
# (tuwunel `src/service/appservice/mod.rs`), on a zero-user database,
# inside `Services::start()` — before the HTTP listener accepts anything.
# That is what lets the `admin_execute` promotion below land on the very
# first boot of a fresh homeserver, instead of depending on hive-c0re
# racing to register the first account and win the auto-admin grant.
#
# ⚠️ Must equal `matrix::HIVE_ADMIN_LOCALPART` in hive-c0re, which derives
# it independently with nothing wiring an override across — same
# agreement, and same reason for saying so, as the token path below.
adminLocalpart = "hive";
# The `as_token`, and the `hs_token` the spec requires alongside it. Both
# minted by the render script below, mode 0600; the `as_token` is the one
# hive-c0re reads and the one the swarm secret store overwrites (see
# `glue-matrix-bao-token.nix`). The `hs_token` authenticates the homeserver
# TO the appservice, which with `url = null` is nobody — it exists because
# the registration format requires it.
#
# `appserviceTokenPath` is the single source for the option's `default`,
# its `config`-level `mkDefault` self-definition, and the assertion that
# rejects a moved path. See `appserviceTokenFile`'s own comment below.
appserviceTokenPath = "/var/lib/hyperhive/matrix-appservice-token";
appserviceHsTokenPath = "/var/lib/hyperhive/matrix-appservice-hs-token";
# The registration file, and the directory that holds it. A directory
# rather than a `[global.appservice.<id>]` stanza because a stanza's
# `as_token` would be a nix literal, and a nix literal is a world-readable
# store path — the same rule that sends the OIDC client secret through a
# runtime file. `docs/swarm/secrets.md` has the general form of it.
appserviceDir = "/var/lib/hyperhive/matrix-appservice";
appserviceRegistrationPath = "${appserviceDir}/${appserviceId}.yaml";
# ⚠️ tuwunel does not read the host path directly, for exactly the reason
# given for the two secrets above: the file is 0600 root-owned and the
# homeserver runs under `DynamicUser=true`. `LoadCredential` copies it to a
# 0400 dynamic-user-owned path as root, before the sandbox and that user
# exist.
#
# So `appservice_dir` points at the credentials directory ITSELF. That is
# safe rather than clever: tuwunel's loader takes only `.yaml`/`.yml`
# entries from the directory and skips every other file, so the sibling
# credentials (the OIDC secret) are invisible to it. A credential id is a
# free-form filename, which is what lets ours end in `.yaml`.
appserviceCredentialId = "${appserviceId}-appservice.yaml";
appserviceCredentialDir = "/run/credentials/tuwunel.service";
# Every local user this hive may provision — agents, the hive admin, and
# the operator accounts `hivectl matrix create-user` makes, which is the
# whole matrix localpart charset.
#
# ⚠️ Anchored deliberately: tuwunel compiles a namespace into a `RegexSet`
# and asks it for a MATCH, not a full match, so an unanchored
# `@[a-z0-9]+:this.example` also matches `@x:this.example.evil.test`.
#
# Non-exclusive deliberately: an exclusive namespace does not widen what
# the appservice may do, it narrows what everything ELSE may do — an SSO
# login adopting an existing account, or `!admin users create-user`, would
# start failing with `M_EXCLUSIVE`. The appservice needs the right to
# create these names, not a monopoly on them.
appserviceUserRegex = "^@[a-z0-9._=/-]+:${lib.escapeRegex effectiveServerName}$";
# Mint the tokens if they are absent, then render the registration from
# whatever they now hold. **One script with two callers** — the activation
# script below and `glue-matrix-bao-token.nix`, which overwrites the
# `as_token` with the swarm store's copy and has to re-render afterwards.
#
# ⚠️ The `as_token` mint here is the FIRST-BOOT FALLBACK, not the normal
# route. `swarm-secret-publisher.nix` mints one per hive swarm-side and
# publishes it to the store; the reader above overwrites this file before
# the homeserver's container starts, so what a hive actually runs on is the
# swarm's value. This keeps a hive that has never reached the store working
# — and it fires only when the file is absent, so it never clobbers one the
# store delivered. (`hs_token` has no swarm half: it is how the homeserver
# authenticates to *us*, so it stays hive-local and is minted here for real.)
# A second copy of the registration's shape would be a second source of
# truth for a format whose mismatch is silent: the homeserver would load a
# registration naming a token nobody holds, and every request hive-c0re
# makes would come back 401 naming nothing.
#
# Re-rendering unconditionally is the point rather than thoroughness: "the
# token file exists" does not mean "the registration carries what is in
# it".
appserviceRegistrationScript = pkgs.writeShellApplication {
name = "hive-matrix-appservice-registration";
runtimeInputs = [ pkgs.coreutils ];
text = ''
# Both the tokens and the rendered registration are secrets; 077
# covers every file this script creates rather than each one
# separately.
umask 077
mkdir -p ${lib.escapeShellArg appserviceDir}
for f in ${lib.escapeShellArg appserviceTokenPath} ${lib.escapeShellArg appserviceHsTokenPath}; do
if [ ! -s "$f" ]; then
head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$f"
echo >> "$f"
echo "hive-matrix: generated appservice token at $f"
fi
chmod 0600 "$f"
done
# Read into shell variables and emitted with `printf`, a shell
# builtin: a token passed as an argument to a real command would land
# in that process's argv, which is world-readable for its lifetime.
asToken="$(cat ${lib.escapeShellArg appserviceTokenPath})"
hsToken="$(cat ${lib.escapeShellArg appserviceHsTokenPath})"
# The quoted heredoc keeps the regex's own `$` and `\` out of the
# shell's hands; the YAML single quotes keep them out of YAML's.
{
cat <<'REGISTRATION'
id: ${appserviceId}
url: null
sender_localpart: ${adminLocalpart}
rate_limited: false
namespaces:
users:
- exclusive: false
regex: '${appserviceUserRegex}'
aliases: []
rooms: []
REGISTRATION
printf 'as_token: %s\nhs_token: %s\n' "$asToken" "$hsToken"
} > ${lib.escapeShellArg appserviceRegistrationPath}
chmod 0600 ${lib.escapeShellArg appserviceRegistrationPath}
chmod 0700 ${lib.escapeShellArg appserviceDir}
'';
};
# Format-locked by tuwunel, not chosen here: the callback host must point
# directly at the matrix server and the path is fixed at
# `/_matrix/client/unstable/login/sso/callback/<client_id>`. Built once
@ -443,40 +576,59 @@ in
'';
};
registrationTokenFile = lib.mkOption {
appserviceTokenFile = lib.mkOption {
type = lib.types.path;
internal = true;
default = registrationTokenPath;
default = appserviceTokenPath;
description = ''
Host path to a file containing the matrix registration token
tuwunel reads to authorise new-account creation. The token is
generated automatically by `hive-c0re` on first boot (32-byte
random hex, mode 0600) and is bind-mounted read-only into the
tuwunel container at the same path. Agents never see this
token — hive-c0re uses it to provision per-agent accounts
and the agent only receives the resulting `access_token`.
Host path to a file containing this hive's matrix appservice
token (`as_token`) — the identity `hive-c0re` creates and logs
into accounts with. Minted automatically on first activation
(32-byte random hex, mode 0600) and rendered into the
appservice registration the homeserver loads at boot. Agents
never see it; an agent only ever receives its own
`access_token`.
Not operator-settable — `hive-c0re`'s Rust side derives this
same path independently (`paths::matrix_register_token()`) with
nothing wiring an override across, so a moved path used to
desync the two silently. An externally-managed token is
delivered by writing into *this* fixed path instead of moving it —
see `glue-matrix-bao-token.nix`, which fetches from the swarm
secret store and overwrites this file in place.
same path independently (`paths::matrix_appservice_token()`)
with nothing wiring an override across, so a moved path desyncs
the two silently. An externally-managed token is delivered by
writing into *this* fixed path instead of moving it — see
`glue-matrix-bao-token.nix`, which fetches from the swarm secret
store, overwrites this file in place, and re-renders the
registration that names it.
Enforced by an `assertions` entry below rather than `readOnly`:
the `config` block gives this option its own `mkDefault`
definition (lowest priority) so any real override — an operator's,
or the pre-rename `swarm.matrix.registrationTokenFile` shim's —
still resolves cleanly instead of crashing eval with nixpkgs'
generic "read-only, set multiple times" message; the assertion
then names the actual problem. (`readOnly` was the first attempt —
atlas caught that it only rejects a *second* definition,
so with nothing else defining the option a lone override sailed
through silently, same desync as before with a lock that wasn't
locked. Fixed by defining the value here instead of leaving it on
`default` alone, but a proper `assertions` message beat re-adding
`readOnly` on top once the option had a real definition either way.)
definition (lowest priority) so a real override still resolves
cleanly instead of crashing eval with nixpkgs' generic
"read-only, set multiple times" message; the assertion then
names the actual problem. (`readOnly` was the first attempt on
this option's predecessor — atlas caught that it only rejects a
*second* definition, so with nothing else defining the option a
lone override sailed through silently, same desync as before
with a lock that wasn't locked. Fixed by defining the value here
instead of leaving it on `default` alone, but a proper
`assertions` message beat re-adding `readOnly` on top once the
option had a real definition either way.)
'';
};
appserviceRegistrationScript = lib.mkOption {
type = lib.types.path;
internal = true;
default = "${appserviceRegistrationScript}/bin/hive-matrix-appservice-registration";
defaultText = lib.literalMD "the module's own registration renderer";
description = ''
The script that mints the appservice tokens when absent and
(re-)renders the registration file from them. An option only so
that `glue-matrix-bao-token.nix` can run the same one after
overwriting the token with the swarm store's copy, rather than
carrying a second copy of the registration's shape.
Not operator-settable, and not a hook: replacing it means
deciding what the homeserver's appservice registration says,
which is this module's job.
'';
};
@ -538,11 +690,11 @@ in
config = lib.mkIf deployCfg.matrix.enable {
# The option's own value, defined explicitly rather than left on its
# bare `default` — `mkDefault` so a real override (an operator's, or
# the pre-rename shim's) still resolves cleanly rather than crashing
# eval; the `assertions` entry below is what actually rejects it, with
# a message naming the reason. See that option's own comment above.
services.hyperhive.deploy.matrix.registrationTokenFile = lib.mkDefault registrationTokenPath;
# bare `default` — `mkDefault` so a real override still resolves
# cleanly rather than crashing eval; the `assertions` entry below is
# what actually rejects it, with a message naming the reason. See that
# option's own comment above.
services.hyperhive.deploy.matrix.appserviceTokenFile = lib.mkDefault appserviceTokenPath;
# Matrix's own gateway surface: the sub-domain vhost, the name the
# hive resolver answers for, and the Accept-header map that vhost's
@ -723,19 +875,17 @@ in
# mkDefault above lets an override resolve instead of crashing
# eval — this is the actual rejection, with a message that names
# the real fix instead of nixpkgs' generic conflicting-definition
# text. Covers both the current path and the pre-rename
# `swarm.matrix.registrationTokenFile` shim in one check, since
# both land on the same merged option.
assertion = deployCfg.matrix.registrationTokenFile == registrationTokenPath;
# text.
assertion = deployCfg.matrix.appserviceTokenFile == appserviceTokenPath;
message = ''
services.hyperhive.deploy.matrix.registrationTokenFile is fixed at
${registrationTokenPath} and cannot be moved — hive-c0re's Rust
services.hyperhive.deploy.matrix.appserviceTokenFile is fixed at
${appserviceTokenPath} and cannot be moved — hive-c0re's Rust
side derives this same path independently and has no way to learn
an override, so moving it desyncs the two silently instead of
loudly.
Integrating an externally-managed registration token? Deliver it
by writing into ${registrationTokenPath} instead of pointing this
Integrating an externally-managed appservice token? Deliver it
by writing into ${appserviceTokenPath} instead of pointing this
option elsewhere — see glue-matrix-bao-token.nix, which does
exactly that from the swarm secret store.
'';
@ -781,8 +931,8 @@ in
# permanent stall presenting as "matrix is broken", several layers from
# its cause.
#
# The registration token above dodges that with an activation script
# that pre-creates the file. ⚠️ That dodge is NOT available here:
# The appservice registration dodges that with an activation script
# that renders the file first. ⚠️ That dodge is NOT available here:
# tuwunel requires the secret file to exist *and be non-empty*, so a
# zero-byte placeholder would satisfy the bind mount and then stop the
# homeserver from starting.
@ -890,16 +1040,21 @@ in
''
);
system.activationScripts.hive-matrix-register-token = lib.stringAfter [ "var" ] ''
tokenFile=${lib.escapeShellArg (toString deployCfg.matrix.registrationTokenFile)}
if [ ! -s "$tokenFile" ]; then
mkdir -p "$(dirname "$tokenFile")"
head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$tokenFile"
echo >> "$tokenFile"
echo "hive-matrix: generated registration token at $tokenFile"
fi
# Re-apply 0600 (normalises any pre-LoadCredential carry-over).
chmod 0600 "$tokenFile"
# Mint the appservice tokens and render the registration, before any
# container start.
#
# An activation script rather than a unit: the directory below is
# bind-mounted into the container and nixos-container refuses to start
# when a bind source is missing, so this has to have run first.
# Activation is what runs before the container both on a switch and on
# every boot. (The registration token this replaced used an activation
# script for the same reason, and additionally to dodge nspawn creating
# an empty file at a missing bind target — which tuwunel then read as
# "no token", refusing every registration until the next restart. A
# missing registration file is not silent in that way: the homeserver
# fails its appservice load loudly.)
system.activationScripts.hive-matrix-appservice = lib.stringAfter [ "var" ] ''
${deployCfg.matrix.appserviceRegistrationScript}
'';
containers.hive-matrix = {
@ -910,11 +1065,15 @@ in
extraFlags = [ "--link-journal=host" ];
# Shared host netns — agents reach tuwunel at localhost:<port>.
privateNetwork = false;
# Read-only bind of the host-managed registration token; tuwunel
# Read-only bind of the host-managed appservice registration; tuwunel
# reads it via systemd LoadCredential below (not directly).
#
# The directory, not the file inside it: the registration is
# re-rendered rather than edited, and binding the file would pin the
# inode the container saw when it started.
bindMounts = {
${deployCfg.matrix.registrationTokenFile} = {
hostPath = deployCfg.matrix.registrationTokenFile;
${appserviceDir} = {
hostPath = appserviceDir;
isReadOnly = true;
};
}
@ -1008,13 +1167,50 @@ in
# trustedServers keeps it effectively closed.
allow_federation = true;
trusted_servers = deployCfg.matrix.trustedServers;
# Token-gated registration. The absent
# `yes_i_am_very_very_sure_…_open_registration_…` flag
# keeps the server closed to anyone without the token.
allow_registration = true;
# LoadCredential below copies the host file into a
# 0400 dynamic-user-owned path; tuwunel reads from there.
registration_token_file = "/run/credentials/tuwunel.service/registration_token";
# Nobody registers themselves here. Accounts are created by
# the hive's appservice, which this flag does not gate —
# tuwunel checks it only for requests that arrive WITHOUT an
# appservice token, so hive-c0re provisions exactly as before
# and everyone else is refused outright.
#
# ⚠️ Not a hardening afterthought: `allow_registration = true`
# with no registration token configured makes tuwunel REFUSE
# TO START (it demands
# `yes_i_am_very_very_sure_…_open_registration_…` instead). So
# dropping the token and leaving this true is not a lax
# homeserver, it is a homeserver that does not boot.
allow_registration = false;
# Where the hive's appservice registration is read from — the
# credentials directory, for the reasons at
# `appserviceCredentialDir`'s own definition. Loaded inside
# `Services::start()`, before the listener accepts anything.
appservice_dir = appserviceCredentialDir;
# The zero-user bootstrap, and the only thing here that needs
# no account to already exist. These run after startup and
# BEFORE the HTTP listener, with no sender and no permission
# check — which is what makes them the one lever that can
# promote the hive admin on a homeserver where nobody is admin
# yet. The appservice registration above creates
# `@${adminLocalpart}` as its sender user moments earlier;
# this joins it to the admin room at power level 100.
#
# Idempotent by upstream's own guard: `make_user_admin`
# short-circuits when the user is already joined at 100, so a
# hive that has had an admin for months emits nothing.
admin_execute = [
"users make-user-admin @${adminLocalpart}:${effectiveServerName}"
];
# ⚠️ Load-bearing, not tidiness. An `admin_execute` command
# that fails aborts startup outright when this is false — so
# the one boot where the promotion cannot work (a homeserver
# that has no `@${adminLocalpart}` and no appservice user yet,
# e.g. a registration file that arrived late) would take the
# homeserver down with it rather than converging on the next
# start.
admin_execute_errors_ignore = true;
# Server-side E2EE is opt-in (default off); the agent matrix
# client always supports decryption regardless.
allow_encryption = cfg.allowEncryption;
@ -1075,7 +1271,10 @@ in
# host-side chown :tuwunel / GID-pin gymnastics needed.
# See `man systemd.exec` → LoadCredential.
systemd.services.tuwunel.serviceConfig.LoadCredential = [
"registration_token:${toString deployCfg.matrix.registrationTokenFile}"
# The credential id carries a `.yaml` suffix on purpose:
# `appservice_dir` above names this very directory, and tuwunel
# takes only `.yaml`/`.yml` entries from it.
"${appserviceCredentialId}:${appserviceRegistrationPath}"
# Same mechanism, second secret. tuwunel re-reads this file on
# every OAuth exchange, not just at startup, so it has to
# outlive the unit's start — a credentials path does.

104
nix/host-modules/swarm-secret-publisher.nix
View file

@ -1,6 +1,7 @@
# The unit that copies authelia's minted OIDC client secrets into the swarm's
# secret store, so whoever needs one can read it there: a hive its agents'
# credential, a swarm service its own.
# The unit that puts swarm-level secrets into the swarm's secret store, so
# whoever needs one can read it there: a hive its agents' credential, a swarm
# service its own, a hive its matrix appservice token. The OIDC secrets it only
# copies; the appservice token it MINTS, having had no swarm-side producer.
#
# ⚠️ "Whoever", including a reader on THIS host. A swarm service's secret goes
# into the store even when the service runs beside authelia, because its
@ -14,19 +15,19 @@
# why: the plaintext's other reader lives in a **different container**, and
# containers that share this host's network namespace still have separate
# filesystem roots, so "the host is the only place both trees are addressable".
# A delivery step therefore runs on the host that mints — not on the store's
# host, and not on the reading hive's.
# A delivery step therefore runs on the minting host, not the store's or the
# reading hive's.
#
# ⚠️ ITS OWN STORE IDENTITY, not swarm-controller's. That principal may rewrite
# every hive's policy and login role; a unit whose entire job is copying one
# file has no business holding it. ./swarm-bao.nix grants this one
# `create`/`update` under the hive prefix and nothing else.
# every hive's policy and login role; a unit that writes a handful of secrets
# has no business holding it. ./swarm-bao.nix grants this one `create`/`update`
# under the hive prefix and nothing else — no `read`, which is why the mint
# below keeps its own record of what it published.
#
# ⚠️ THE SECRET NEVER REACHES `argv`. `bao` is an external binary, so every
# argument is world-readable in /proc for the life of the call — the value is
# passed as `@<path>` and read by bao itself. (`/knowledge/secret-hygiene.md`:
# "a path keeps the secret out of the store, and reading it into a shell
# variable puts it straight into argv".)
# passed as `@<path>` and read by bao itself. Same rule governs the mint: it
# redirects into a file and never holds the token in a shell variable.
{
pkgs,
lib,
@ -71,6 +72,21 @@ let
hyperhiveCfg.swarm.grafana.oidc.clientId
hyperhiveCfg.swarm.otel.clientId
];
# Where this unit keeps the appservice tokens it minted, and the whole reason
# a re-publish is idempotent. The store cannot be that record: ./swarm-bao.nix
# grants this principal `create`/`update` and deliberately no `read`, so "does
# this hive already have a token?" is a question it can only ask locally.
# Without the file every run would mint a fresh value and rotate the swarm's
# token for no reason — and a rotation is not free, because the homeserver
# only reloads the registration naming it when its container restarts.
#
# `StateDirectory=` below is what creates it, so the name there is relative
# and this unit owns the tree. Losing it mints once more and republishes;
# readers pick that up on their next start, and nothing that already
# registered breaks — the token authenticates the appservice, it does not
# live inside any account it created.
matrixTokenDir = "/var/lib/swarm-secret-publish/matrix-appservice-token";
in
{
options.services.hyperhive.deploy.swarm-secret-publisher = {
@ -82,10 +98,12 @@ in
Publish the OIDC client secrets this host mints into the swarm's
secret store, so a hive that does not run authelia can read its
agents' credential and a swarm service can read its own — from
wherever it runs, this host included.
wherever it runs, this host included. Also mints each hive's matrix
appservice token, which has no other swarm-side producer, and
publishes it the same way.
Defaults to whether this host mints them, which is the only half of
the question that is a property of *this* host.
Defaults to whether this host mints the OIDC secrets, which is the
only half of the question that is a property of *this* host.
⚠️ Deliberately **not** `deploy.bao.enable`. That asks whether the
store stands here, and a publisher beside a remote store, holding a
@ -95,11 +113,12 @@ in
identity's job (see `baoClientCertFile`), not this option's.
Turning it off leaves every hive but this one without its agents'
credential, the swarm's Grafana without any login at all, and its
collector pushing unauthenticated — each secret has exactly one route
and this is the producer's end of it. So the honest reason to set it
false is a deployment delivering those secrets by some other
mechanism it owns.
credential, the swarm's Grafana without any login at all, its
collector pushing unauthenticated, and every hive falling back to the
appservice token its own first boot minted — so two hives never agree
on one. Each secret has exactly one route and this is the producer's
end of it, so the honest reason to set it false is a deployment
delivering those secrets by some other mechanism it owns.
'';
};
@ -143,7 +162,7 @@ in
};
systemd.services.swarm-secret-publish = {
description = "publish minted OIDC client secrets to the swarm secret store";
description = "publish the swarm's minted secrets to the swarm secret store";
wantedBy = [ "multi-user.target" ];
after = [ "network.target" ];
path = [
@ -159,6 +178,12 @@ in
TimeoutStartSec = 60;
Restart = "on-failure";
RestartSec = 30;
# The minted appservice tokens, see `matrixTokenDir` above. Named
# relative to /var/lib because that is the form systemd creates; the
# mode is narrowed from systemd's 0755 default because the directory
# listing is the roster of hives and every file in it is a secret.
StateDirectory = "swarm-secret-publish/matrix-appservice-token";
StateDirectoryMode = "0700";
};
environment = {
BAO_ADDR = "https://${hyperhiveCfg.swarm.bao.domain}:${toString hyperhiveCfg.swarm.bao.port}";
@ -185,6 +210,7 @@ in
published=0
skipped=0
tokens=0
${lib.concatMapStringsSep "\n" (hive: ''
src=${lib.escapeShellArg "${deployCfg.authelia.hostClientSecretDir}/${agentClientId hive}.secret"}
@ -228,7 +254,43 @@ in
fi
'') serviceClientIds}
echo "published $published client secret(s), skipped $skipped"
# The matrix appservice token, per hive. Unlike the two loops above
# there is nothing to copy: authelia never minted this one, and each
# homeserver's own host minted its own — which is exactly why two hives
# never converged on a value the swarm has to agree on. The mint moves
# here because this unit already holds a store identity and already
# writes under the hive prefix; ../host-modules/hive-matrix.nix keeps
# its activation mint as the first-boot fallback for a hive that has
# never reached the store.
#
# 32 bytes of urandom as lowercase hex, the shape that file has
# generated all along. Written with no trailing newline, unlike that
# file: what goes into the store is the value, and
# ./glue-matrix-bao-token.nix adds the newline the hive's token file
# carries.
${lib.concatMapStringsSep "\n" (hive: ''
src=${lib.escapeShellArg "${matrixTokenDir}/${hive}"}
if [ ! -s "$src" ]; then
# The subshell keeps the umask local — the loops above rely on
# bao's own file handling and should not inherit one.
( umask 077; head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$src" )
echo "minted a matrix appservice token for ${hive}" >&2
fi
# `value=@$src` for the same reason as both loops above: bao opens
# the file itself, so the token is never an argument of this process.
#
# Put on every run, not only on the run that minted — a put of the
# value already there changes nothing for any reader, while a mint
# whose publish failed must not be left as a token this host holds
# and no hive can reach. The `-s` test above is what keeps the VALUE
# stable across those runs: rotating on every boot would hand the
# homeserver a registration its running container does not carry.
bao kv put ${lib.escapeShellArg "secret/swarm/hives/${hive}/matrix/appservice-token"} \
value=@"$src"
tokens=$((tokens + 1))
'') hiveNames}
echo "published $published client secret(s), skipped $skipped, and $tokens matrix appservice token(s)"
'';
};
};

112
nix/module-eval.nix
View file

@ -116,24 +116,45 @@ let
];
};
# The homeserver's turn to split. All eight host-side options are set through
# their pre-rename paths — including `gui.enable`, whose value is deliberately
# the opposite of its default so the definition has to actually land, and both
# packages, stubbed to a derivation neither option defaults to for the same
# reason. All eight so that dropping any single shim entry fails the eval, not
# just the ones the assertions read.
# The homeserver's turn to split. All seven remaining host-side options are
# set through their pre-rename paths — including `gui.enable`, whose value is
# deliberately the opposite of its default so the definition has to actually
# land, and both packages, stubbed to a derivation neither option defaults to
# for the same reason. All seven so that dropping any single shim entry fails
# the eval, not just the ones the assertions read.
#
# `registrationTokenFile` used to be the eighth: both spellings of it are now
# `mkRemovedOptionModule`, exercised by `matrixRemovedToken` below instead —
# setting it here would make every case that reads this fixture fail on that
# one removed option.
matrixOldPath = hive {
deploy.matrix.enable = true;
swarm.matrix.openFirewall = true;
swarm.matrix.trustedServers = [ "matrix.example.invalid" ];
swarm.matrix.maxRequestSize = 31457280;
swarm.matrix.registrationTokenFile = "/etc/matrix/register.token";
swarm.matrix.gui.enable = false;
swarm.matrix.package = pkgs.emptyDirectory;
swarm.matrix.gui.package = pkgs.emptyDirectory;
swarm.matrix.sso.clientSecretFile = "/etc/matrix/oidc.secret";
};
# Moving the appservice token, which desyncs hive-c0re's independently
# derived path from the homeserver's. Its own fixture because the assertion
# it trips would otherwise fire for every case reading `matrixOldPath`.
matrixMovedAppserviceToken = hive {
deploy.matrix.enable = true;
deploy.matrix.appserviceTokenFile = "/etc/matrix/as.token";
swarm.matrix.sso.clientSecretFile = "/etc/matrix/oidc.secret";
};
# A config still naming the removed registration token, through the
# pre-rename spelling — the one an old deployment is most likely to carry.
matrixRemovedToken = hive {
deploy.matrix.enable = true;
swarm.matrix.registrationTokenFile = "/etc/matrix/register.token";
swarm.matrix.sso.clientSecretFile = "/etc/matrix/oidc.secret";
};
# The queue's callout identity, fourth split slice. `autoGenerateCallout` is
# left FALSE on purpose: that is what makes the seed paths the thing deciding
# `responderConfigured`, so the assertion below is about the seeds rather
@ -840,33 +861,41 @@ let
}
{
# Third split, and the one whose readers were hardest to see: the
# registration token is read only through a `let` alias in another
# appservice token is read only through a `let` alias in another
# module, so no full path names it anywhere. Both arms read a rendered
# effect — the host firewall and the container's bind-mount table — so a
# rename that resolves but stops reaching the module still fails.
name = "a config written against the pre-rename matrix paths still opens the port and mounts the token";
name = "a config written against the pre-rename matrix paths still opens the port and mounts the appservice registration";
ok =
let
ports = matrixOldPath.networking.firewall.allowedTCPPorts;
httpPort = matrixOldPath.services.hyperhive.swarm.matrix.httpPort;
in
builtins.elem httpPort ports
&& matrixOldPath.containers.hive-matrix.bindMounts ? "/etc/matrix/register.token";
&& matrixOldPath.containers.hive-matrix.bindMounts ? "/var/lib/hyperhive/matrix-appservice";
}
{
# registrationTokenFile lost its override capability entirely
# (bao-delivered secrets don't need one — glue-matrix-bao-token.nix
# already writes into the fixed path instead of moving it), unlike its
# five siblings in the same rename. `matrixOldPath` above proves the
# override still *resolves* (mkDefault, not a crash) — this proves it
# also gets *rejected*, by a named assertion rather than nixpkgs' generic
# conflicting-definition text. Reads `.assertions` directly (cheap: a
# list of `{assertion; message;}`, not `system.build.toplevel`) rather
# than forcing a real build just to observe a boolean.
name = "overriding registrationTokenFile (even via the pre-rename shim) trips a named assertion, not a silent desync";
# appserviceTokenFile has no override capability (a store-delivered
# secret doesn't need one — glue-matrix-bao-token.nix writes into the
# fixed path instead of moving it), unlike its siblings in the same
# rename. The fixture proves the override still *resolves* (mkDefault,
# not a crash); this proves it also gets *rejected*, by a named
# assertion rather than nixpkgs' generic conflicting-definition text.
# Reads `.assertions` directly (cheap: a list of `{assertion;
# message;}`, not `system.build.toplevel`) rather than forcing a real
# build just to observe a boolean.
name = "moving appserviceTokenFile trips a named assertion, not a silent desync";
ok = lib.any (
a: !a.assertion && lib.hasInfix "registrationTokenFile" a.message
) matrixOldPath.assertions;
a: !a.assertion && lib.hasInfix "appserviceTokenFile" a.message
) matrixMovedAppserviceToken.assertions;
}
{
# The removal has to SAY so. A config carrying the old option through
# an upgrade is the common case, and nixpkgs' bare "option does not
# exist" would name neither what replaced the token nor where to read
# about it.
name = "a config still setting registrationTokenFile is refused with a message naming the appservice";
ok = lib.any (a: !a.assertion && lib.hasInfix "appservice" a.message) matrixRemovedToken.assertions;
}
{
# Reads the DELIVERY UNIT, not the options: `responderConfigured` gates
@ -1448,6 +1477,41 @@ let
lib.hasInfix "secret/swarm/hives/h1/queue/agent" s
&& lib.hasInfix "secret/swarm/hives/h2/queue/agent" s;
}
{
# The producer's end of the read `glue-matrix-bao-token.nix` already did.
# Both hives are asserted for the reason the queue case above gives: a
# publisher that knew one name rather than the roster would pass on a
# single-hive fixture and strand the second hive ever added — which is
# the two-hives-never-converge shape this slice exists to close.
name = "the publisher mints an appservice token for every hive and writes it to that hive's matrix path";
ok =
let
s = secretPublisherHere.systemd.services.swarm-secret-publish.script;
in
lib.hasInfix "secret/swarm/hives/h1/matrix/appservice-token" s
&& lib.hasInfix "secret/swarm/hives/h2/matrix/appservice-token" s
&& lib.hasInfix "/dev/urandom" s;
}
{
# What makes a re-publish idempotent. This principal is granted
# `create`/`update` and no `read`, so it cannot ask the store whether a
# hive already has a token — with nowhere to keep one, every run would
# mint a fresh value and rotate the swarm's token. A state directory is
# that somewhere, and nothing else in this unit needs one, so its absence
# means exactly this.
#
# The second arm is the mint's own guard: the state file is only written
# when it is missing or empty. Dropping that test leaves a unit that
# still has a state directory and still rotates on every boot.
name = "the publisher keeps the tokens it minted, and mints only when it holds none";
ok =
let
u = secretPublisherHere.systemd.services.swarm-secret-publish;
in
lib.hasInfix "matrix-appservice-token" (u.serviceConfig.StateDirectory or "")
&& u.serviceConfig.StateDirectoryMode or null == "0700"
&& lib.hasInfix "if [ ! -s \"$src\" ]" u.script;
}
{
# A property of the SET, not of one unit: both of these authenticate by
# certificate, and `BAO_CLIENT_CERT` is transport rather than identity, so
@ -1518,7 +1582,7 @@ let
s = baoWithMatrix.systemd.services.swarm-bao-matrix-token.script;
in
lib.hasInfix "secret/swarm/hives/" s
&& lib.hasInfix "/matrix/registration-token" s
&& lib.hasInfix "/matrix/appservice-token" s
# The shape it used to have: `matrix` where a principal kind belongs,
# which no grant covers.
&& !(lib.hasInfix "secret/swarm/matrix/" s);
@ -2471,7 +2535,7 @@ let
# The store's first reader. Its unit belongs to the pairing, not to
# either service: matrix must not learn the store exists, and the store
# must not know who reads it.
name = "a store deployed beside the homeserver fetches its registration token";
name = "a store deployed beside the homeserver fetches its appservice token";
ok = baoWithMatrix.systemd.services ? swarm-bao-matrix-token;
}
{

34
swarm-secret-client/src/matrix.rs
View file

@ -26,18 +26,25 @@ pub fn account_path(agent: &str, account: &str) -> Result<String, Error> {
Ok(format!("{prefix}/matrix/{account}"))
}
/// The path holding `hive`'s matrix registration token.
/// The path holding `hive`'s matrix appservice token (`as_token`).
///
/// Keyed per **hive**, not per agent, like [`crate::queue::agent_client_path`]
/// and unlike [`account_path`] above: one homeserver admits one hive's
/// accounts, so the token that creates them is the hive's.
/// accounts, so the identity that creates them is the hive's.
///
/// ⚠️ Renamed from `registration-token` along with what it holds: the
/// homeserver no longer accepts a shared registration secret at all, so a
/// value still stored under the old path would be read by nothing. A hive
/// whose store has only the old path falls back to its locally minted
/// token — see `glue-matrix-bao-token.nix` — so the rename degrades rather
/// than breaks, but the store needs a fresh `put` to take effect again.
///
/// # Errors
/// [`Error::PathSegment`] when `hive` contains anything but `[A-Za-z0-9_-]`,
/// which is what keeps one hive's name from addressing another hive's secret.
pub fn registration_token_path(hive: &str) -> Result<String, Error> {
pub fn appservice_token_path(hive: &str) -> Result<String, Error> {
let prefix = principal_prefix(Kind::Hive, hive)?;
Ok(format!("{prefix}/matrix/registration-token"))
Ok(format!("{prefix}/matrix/appservice-token"))
}
/// What an account's path holds: the token, plus the homeserver it belongs to.
@ -80,28 +87,29 @@ mod tests {
}
#[test]
fn the_registration_token_lands_under_the_hive_prefix_the_grant_covers() {
fn the_appservice_token_lands_under_the_hive_prefix_the_grant_covers() {
// Spelled out for the same reason as above, and with a second job here:
// the read policy grants `secret/data/swarm/hives/<hive>/*`, so this
// string is what makes the path reachable at all.
assert_eq!(
registration_token_path("pr1ma").expect("a plain name is legal"),
"swarm/hives/pr1ma/matrix/registration-token"
appservice_token_path("pr1ma").expect("a plain name is legal"),
"swarm/hives/pr1ma/matrix/appservice-token"
);
}
#[test]
fn the_registration_token_is_not_a_top_level_namespace() {
// The path it used to hold. `Kind` is a closed set and `matrix` is not
// one of its members, so a path with `matrix` as the second segment is
// outside every grant — which is how it came to 403 on every read.
let p = registration_token_path("pr1ma").expect("legal");
fn the_appservice_token_is_not_a_top_level_namespace() {
// The path its predecessor used to hold. `Kind` is a closed set and
// `matrix` is not one of its members, so a path with `matrix` as the
// second segment is outside every grant — which is how it came to 403
// on every read.
let p = appservice_token_path("pr1ma").expect("legal");
assert!(!p.starts_with("swarm/matrix/"), "{p}");
}
#[test]
fn a_traversal_in_the_hive_name_is_refused() {
let e = registration_token_path("../beta").expect_err("a traversal is not");
let e = appservice_token_path("../beta").expect_err("a traversal is not");
assert!(matches!(e, Error::PathSegment { kind: "hive", .. }), "{e}");
}