vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

matrix: remove the registration token

Browse source 
Nothing reads it any more: hive-c0re creates accounts as the hive's
appservice, so the mint, the host file, the bind mount, the
`LoadCredential` entry and tuwunel's `registration_token_file` all go.

⚠️ `allow_registration` has to go to `false` in the same change, and not
as hardening. tuwunel refuses to START when registration is allowed with
no token configured — it demands
`yes_i_am_very_very_sure_…_open_registration_…` instead — so dropping the
token and leaving the flag true is not a lax homeserver, it is one that
does not boot. The flag is checked only for requests arriving without an
appservice token, so hive-c0re provisions exactly as before and everyone
else is refused outright.

The swarm secret store keeps its role, repointed at the credential that
replaced the token (`swarm/hives/<hive>/matrix/appservice-token`). Its
unit now also re-runs hive-matrix's own registration renderer after
writing the file: the token is half an agreement, and a registration
still naming the previous value authenticates nobody. The renderer is
shared through an internal option rather than copied, so the
registration's shape has one home.

Both spellings of `registrationTokenFile` become
`mkRemovedOptionModule` with a message naming what replaced them. A hive
that never set the option — the default — is unaffected; one that pinned
it fails to evaluate with instructions instead of a silent no-op.

An upgraded hive needs no intervention: the activation script has both
halves in place before the homeserver restarts, existing agents keep the
tokens their devices already hold, and the old token file is left on
disk read by nothing. docs/integrations/matrix.md spells the path out.

Refs #4402
This commit is contained in:
atlas 2026-09-15 19:54:50 +02:00
commit 7ee7080b21
11 changed files with 410 additions and 242 deletions
Download patch file Download diff file Expand all files Collapse all files

135
docs/integrations/matrix.md
View file

@ -90,39 +90,54 @@ by default. Reaching it on 8448 needs either an explicit tuwunel
bind to that port OR a reverse-proxy + `.well-known/matrix/server`
delegation (the latter lives in `gateway.md::Discovery flow`).
## Provisioning flow (registration token)
## Provisioning flow (appservice)
Token-gated registration: hive-c0re holds the token, agents never
see it. The agent only receives the resulting `access_token`.
Registration is closed. Accounts are created by the hive's own
**appservice**: hive-c0re holds the appservice token, agents never see
it, and an agent only ever receives its own `access_token`.
1. **System activation** writes a 32-byte random hex token (64
chars) to `services.hyperhive.deploy.matrix.registrationTokenFile`
(`/var/lib/hyperhive/matrix-register-token` by default), mode
`0600 root:root`, before any container start. Idempotent — only
writes when the file is missing or empty; always re-applies 0600
(normalises any 0640 / world-readable carry-over from
pre-LoadCredential deployments). This runs at activation time
(not first container start) to dodge a race where nspawn creates
an empty file when the bind-mount target is missing and tuwunel
reads `registration_token_file=""`, rejecting every registration
until next restart.
2. **Read-only bind-mount** maps the host file into the tuwunel
The appservice has no URL (`url: null` in its registration), so the
homeserver never calls out to it and there is no service to run. What the
registration buys is an identity the homeserver recognises — which is why
no secret has to be equal on both sides of the wire, and why account
creation doesn't depend on registration being open to anyone who learns
a token.
1. **System activation** mints a 32-byte random hex appservice token (64
chars) at `/var/lib/hyperhive/matrix-appservice-token` and its
spec-required `hs_token` sibling, mode `0600 root:root`, then renders
the registration to
`/var/lib/hyperhive/matrix-appservice/hyperhive.yaml` (also `0600`).
The tokens are minted only when missing; the registration is
re-rendered every time, because the token file can be overwritten in
place by the swarm secret store and a registration naming a stale
token authenticates nobody. Runs at activation time, before any
container start, because the directory is bind-mounted and
nixos-container refuses to start when a bind source is missing.
2. **Read-only bind-mount** maps that directory into the tuwunel
container at the same path.
3. **systemd `LoadCredential=`** inside the container copies the
bind-mounted file into
`/run/credentials/tuwunel.service/registration_token`, owned by
tuwunel's dynamic user with mode `0400`, at service start. The
host file stays `root:root 0600` — no `chown :tuwunel` /
`chmod 0640` / GID-pin gymnastics required. Keeps
`DynamicUser = true` + `PrivateUsers = true` intact.
4. tuwunel's `registration_token_file` points at the credentials
path, not the original bind-mount path.
5. **hive-c0re** uses the token to register each agent account via
the matrix-spec UIAA registration flow, persists the returned
`access_token` to `<agent-state>/matrix-token`. The agent's
matrix MCP client authenticates with that access_token and
never touches the shared registration token.
6. **hive-c0re restarts `hive-matrix-daemon`** for the agent
registration into
`/run/credentials/tuwunel.service/hyperhive-appservice.yaml`, owned by
tuwunel's dynamic user with mode `0400`, at service start. The host
file stays `root:root 0600` — no `chown :tuwunel` / `chmod 0640` /
GID-pin gymnastics required. Keeps `DynamicUser = true` +
`PrivateUsers = true` intact.
4. tuwunel's `appservice_dir` points at the credentials directory, not at
the bind-mount path. It reads only `.yaml`/`.yml` entries from there,
so the sibling credentials are invisible to it. The `.yaml` suffix on
the credential id is what makes this work.
5. **hive-c0re** reads the appservice token and creates each account with
one `POST /register` typed `m.login.application_service`, persisting
the returned `access_token` to `<agent-state>/matrix-token`. It never
mints the token itself: the value has to be the one the rendered
registration names, and only the nix side writes that.
6. **An account that exists but has lost its token file** is re-tokened
by an appservice `POST /login` — no password and no admin rights
involved. A stored-password login and an admin-room password reset
remain behind that, for accounts created before the appservice existed
or named outside its namespace.
7. **hive-c0re restarts `hive-matrix-daemon`** for the agent
immediately after writing the token so the daemon picks up the
new credential without waiting for a full container restart. If
the restart fails (for example daemon not yet running on first boot)
@ -130,16 +145,70 @@ see it. The agent only receives the resulting `access_token`.
(`hive-matrix-daemon.path` watching for `matrix-token` appearance)
brings the daemon up on the same boot cycle anyway.
### The admin account, and why it needs no first-user luck
`@hive:<server_name>` is the appservice's own `sender_localpart`, which
the homeserver creates itself when it loads the registration — on a
zero-user database, inside startup, before the HTTP listener accepts
anything. Its **admin rights** then come from an explicit
`make_user_admin`, run by tuwunel's `admin_execute` in the same startup
and likewise before the listener — so a fresh hive has a joined,
power-level-100 admin on its first boot.
This replaces a dependency on being the first account ever registered,
which was fragile in both directions: an appservice-created account is
excluded from that automatic grant by design, and on a homeserver that
already had users the rule never fired at all.
Promotion can't be bootstrapped over the API, and that's upstream's
design rather than a gap: tuwunel only treats an admin-room message as a
command when its sender is already an admin. `admin_execute` is the one
lever with no sender to check. hive-c0re re-checks the result on every
sweep by reading the admin account's own joined-rooms list; if the rights
are missing it says so, names
`systemctl restart container@hive-matrix` as the fix, and carries on —
agent accounts, the hive Space and the chat room need no admin.
<details><summary>Upgrading a hive that used the registration token</summary>
Nothing to do, and nothing to time. The activation script mints the
appservice token and renders the registration before the homeserver
restarts, so the first boot after the switch already has both halves.
- **Existing accounts keep working.** An access token lives on the
device that minted it; removing the registration token touches no
device, no account and no session. `login_with_password` stays on, so
the password fallback is still there too.
- **Existing token files are honoured.** The per-agent sweep skips any
agent that already has a `matrix-token`, so no account is re-registered
and no session is displaced.
- **The admin account is already admin** on such a hive (it won the
first-user grant when the hive was new), so the startup promotion is a
no-op — upstream's `make_user_admin` short-circuits when the user is
already joined at power level 100.
- **`/var/lib/hyperhive/matrix-register-token` is left on disk**, read by
nothing. Delete it or leave it; neither does any harm.
- **`registrationTokenFile` is a removed option.** A config that still
sets it fails to evaluate with a message naming the appservice — a hive
that never set it (the default) is unaffected.
- **A swarm store holding the old `matrix/registration-token` path** is
no longer read at all; the hive uses its locally minted appservice
token until someone `put`s a value at `matrix/appservice-token`.
</details>
Initial rollout settings:
- `allow_federation = true` at the protocol level so swarms can be
wired up later by extending `trustedServers` without a homeserver
restart. `trusted_servers = []` keeps it effectively closed
until you list peers.
- `allow_registration = true` (required for the token flow to
engage). The absent
`yes_i_am_very_very_sure_…_open_registration_…` flag keeps the
server closed to anyone without the token.
- `allow_registration = false`. tuwunel checks this flag only for
requests that arrive **without** an appservice token, so hive-c0re
provisions exactly as before and everyone else is refused. It's not a
hardening afterthought: with no registration token configured,
`allow_registration = true` makes tuwunel refuse to start unless
`yes_i_am_very_very_sure_…_open_registration_…` is also set.
- `allow_encryption` — server-side E2EE switch, sourced from
`services.hyperhive.swarm.matrix.allowEncryption` (**default `false`**, opt-in).
Off by default because on the hive-internal homeserver the operator

22
docs/swarm/secrets.md
View file

@ -136,7 +136,7 @@ when the system builds. The server names the offending file and refuses to run.
| ---------------------------------------- | --------------------------------------------------------------- | --------------------------------------------------------------------------------------------- |
| hive CA cert + key | `hive-tls.nix` first-boot unit | `<deploy.hive-controller.tls.stateDir>/ca.pem`, `ca-key.pem` (`0600`) |
| hive leaf certs | `hive-tls.nix`, signed by the hive CA | `<deploy.hive-controller.tls.stateDir>/<name>.pem` |
| matrix registration token | a host activation script, on first boot | `/var/lib/hyperhive/matrix-register-token` (`0600`) |
| matrix appservice token | a host activation script, on first boot | `/var/lib/hyperhive/matrix-appservice-token` (`0600`) |
| the forge's copy of its OIDC secret | `hive-forge-oidc-secret.service` copies it from authelia's tree | `/var/lib/forgejo-oidc/<id>.secret` inside the forge container |
| the homeserver's copy of its OIDC secret | `hive-matrix-oidc-secret.service`, same shape | `/var/lib/tuwunel-oidc/<id>.secret`, handed to tuwunel through `LoadCredential` |
| the agent containers' queue credential | authelia, published to the store by `swarm-secret-publish` | `<deploy.hive-controller.queue.agentCredentialDir>/secret` (`0600`) and `/client_id` (`0644`) |
@ -146,18 +146,26 @@ bounded wait, 120s — and then **fail loudly** rather than skipping. A silent s
produces a service whose login button always fails, which is a symptom many
layers from its cause.
The store's **first reader** is the matrix registration token, and it's worth
The store's **first reader** is the matrix appservice token, and it's worth
saying why that one: it's an opaque 32-byte value with no second file and no
format. Authelia's OIDC secret needs a `.secret` _and_ a matching `.digest`, so
starting there would have meant debugging "can a reader authenticate and get
bytes back" and "is authelia's file format right" at once, with an
SSO outage as the failure mode.
`glue-matrix-bao-token.nix` fetches it and writes the file `hive-matrix.nix`
already reads, so the homeserver never learns the store exists. Every failure
path — no such key, sealed store, unreachable store, empty value — leaves the
locally minted token in place, so a hive with no store behaves exactly as it
did before.
`glue-matrix-bao-token.nix` fetches it, writes the file `hive-matrix.nix`
already reads, and then runs that module's own renderer to re-stamp the
appservice registration naming the token — so the homeserver never learns the
store exists. The re-render isn't housekeeping: the token is half an
agreement, and a registration carrying the previous value authenticates
nobody. Every failure path — no such key, sealed store, unreachable store,
empty value — leaves the locally minted token in place, so a hive with no store
behaves exactly as it did before.
The store path is `swarm/hives/<hive>/matrix/appservice-token`. It was
`…/matrix/registration-token` while the homeserver still took a shared
registration secret; a value left at the old path is read by nothing, and the
hive falls back to its local token until someone `put`s the new one.
The **second reader** is the agent containers' queue credential:
`glue-queue-agent-credential.nix` lands it as two files, the client secret and

6
docs/swarm/sso.md
View file

@ -168,9 +168,9 @@ Two consequences worth stating plainly:
- **tuwunel re-reads its secret file on every OAuth exchange**, not only
at startup, and its own sandboxing hides most paths from it. It gets the
file through `LoadCredential` for the same reason the registration token
does — that keeps `DynamicUser` and `PrivateUsers` intact, with no
host-side ownership arrangement to maintain.
file through `LoadCredential` for the same reason the appservice
registration does — that keeps `DynamicUser` and `PrivateUsers` intact,
with no host-side ownership arrangement to maintain.
- **Matrix SSO lives inside the homeserver.** The client-server API is
spoken by non-browser clients holding matrix access tokens — every
agent's own daemon — as well as by federation, so the homeserver

9
hive-c0re/src/matrix.rs
View file

@ -982,9 +982,8 @@ pub async fn ensure_admin_user(client: &reqwest::Client, as_token: &str) -> Resu
/// admin is degraded, not broken. Only `hivectl matrix promote-user` /
/// `reset-password` need it.
///
/// # Errors
/// Never — the outcome is the return value. `false` means "not admin and
/// could not be made one", already logged with what to do about it.
/// Returns whether the account holds admin rights now. A `false` has
/// already been logged, with what to do about it.
async fn ensure_admin_rights(
client: &reqwest::Client,
admin_token: &str,
@ -1003,7 +1002,9 @@ async fn ensure_admin_rights(
return true;
}
Some(_) => {
tracing::warn!("matrix: hive admin is not in the admin room — attempting to promote it")
tracing::warn!(
"matrix: hive admin is not in the admin room — attempting to promote it"
);
}
None => tracing::debug!(
"matrix: could not read the hive admin's joined rooms; attempting to promote it"

8
hive-matrix-mcp/src/client.rs
View file

@ -5,10 +5,10 @@
//! to start sync.
//!
//! No OAuth dance / cross-signing setup (in contrast to damocles-daemon's
//! ccc.de connection): for the in-hive tuwunel the `registration_token`
//! UIAA flow already minted the token + user/device, hive-c0re just
//! handed us the bearer in a file. matrix-sdk's `restore_session` with
//! a constructed `MatrixSession` skips the login flow entirely.
//! ccc.de connection): for the in-hive tuwunel hive-c0re already minted
//! the token + user/device as the hive's appservice and handed us the
//! bearer in a file. matrix-sdk's `restore_session` with a constructed
//! `MatrixSession` skips the login flow entirely.
//!
//! E2EE is enabled via `with_encryption_settings(EncryptionSettings::default())`.
//! Crypto keys are persisted in the sqlite store under `state_dir`

27
nix/host-modules/deploy.nix
View file

@ -247,10 +247,29 @@ in
[ "services" "hyperhive" "swarm" "matrix" "maxRequestSize" ]
[ "services" "hyperhive" "deploy" "matrix" "maxRequestSize" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "matrix" "registrationTokenFile" ]
[ "services" "hyperhive" "deploy" "matrix" "registrationTokenFile" ]
)
# Both spellings of the registration token, removed rather than renamed:
# the homeserver no longer accepts a shared registration secret at all,
# so there is no path to point a definition at. Accounts are created by
# the hive's appservice, whose token lives at a fixed path this module
# deliberately does not let anyone move.
(lib.mkRemovedOptionModule [ "services" "hyperhive" "swarm" "matrix" "registrationTokenFile" ] ''
The matrix registration token is gone: hive-c0re creates accounts as the
hive's appservice instead, so there is no shared secret for tuwunel to
check. Remove this definition — nothing needs replacing, the appservice
token is minted and delivered automatically
(/var/lib/hyperhive/matrix-appservice-token, not operator-settable).
The old token file at /var/lib/hyperhive/matrix-register-token is read by
nothing now and can be deleted. See docs/integrations/matrix.md.
'')
(lib.mkRemovedOptionModule [ "services" "hyperhive" "deploy" "matrix" "registrationTokenFile" ] ''
The matrix registration token is gone: hive-c0re creates accounts as the
hive's appservice instead, so there is no shared secret for tuwunel to
check. Remove this definition — nothing needs replacing, the appservice
token is minted and delivered automatically
(/var/lib/hyperhive/matrix-appservice-token, not operator-settable).
The old token file at /var/lib/hyperhive/matrix-register-token is read by
nothing now and can be deleted. See docs/integrations/matrix.md.
'')
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "matrix" "gui" "enable" ]
[ "services" "hyperhive" "deploy" "matrix" "gui" "enable" ]

29
nix/host-modules/glue-matrix-bao-token.nix
View file

@ -1,8 +1,9 @@
# Glue: the matrix registration token comes from the secret store.
# Glue: the matrix appservice token comes from the secret store.
#
# The store's first reader, and deliberately a small one. It fetches an opaque
# 32-byte value and writes it where ./hive-matrix.nix already looks — the
# homeserver never learns the store exists, and its config is unchanged.
# 32-byte value and writes it where ./hive-matrix.nix already looks, then asks
# that module's own renderer to re-stamp the appservice registration naming it
# — the homeserver never learns the store exists, and its config is unchanged.
#
# ⚠️ Why this credential first. It has no second file and no format: authelia's
# OIDC secret needs a `.secret` *and* a matching `.digest`, so shipping that
@ -47,12 +48,12 @@ let
# own grant covers. The store's read policy grants `swarm/agents/*` and
# `swarm/hives/<this hive>/*` and nothing else, so a path outside those is a
# 403 rather than a miss, however correct it looks. `swarm-secret-client`'s
# `matrix::registration_token_path` builds the same string from the same
# `matrix::appservice_token_path` builds the same string from the same
# pieces; this literal is the nix half of that one agreement.
#
# `hiveName` has no fallback here for the reason ./glue-bao-tls.nix gives at
# its own use of it: it is asserted set for every hyperhive host.
tokenPath = "secret/swarm/hives/${hyperhiveCfg.hiveName}/matrix/registration-token";
tokenPath = "secret/swarm/hives/${hyperhiveCfg.hiveName}/matrix/appservice-token";
# A literal, not an option — ./hive-matrix.nix names its container
# `containers.hive-matrix` directly and declares no `machine` to derive it
@ -70,7 +71,7 @@ in
services.hyperhive.swarm.otel.journaldUnits = [ "swarm-bao-matrix-token" ];
systemd.services.swarm-bao-matrix-token = {
description = "fetch the matrix registration token from the swarm secret store";
description = "fetch the matrix appservice token from the swarm secret store";
# Every one of these names a unit that exists only where the store runs.
# `Requires=` on an absent unit fails the job outright, so the ordering is
# conditional even though the read is not: off-host there is nothing local
@ -176,8 +177,20 @@ in
fi
umask 077
printf '%s\n' "$token" > ${lib.escapeShellArg (toString deployCfg.matrix.registrationTokenFile)}
chmod 0600 ${lib.escapeShellArg (toString deployCfg.matrix.registrationTokenFile)}
printf '%s\n' "$token" > ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)}
chmod 0600 ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)}
# Re-stamp the registration file from the token just written. The
# token is half an agreement — the registration the homeserver loads
# has to carry the same value — so writing the file and stopping
# would leave the homeserver authenticating hive-c0re against
# whatever activation put there: a 401 on every request, naming
# nothing. Unconditional rather than on-change, because this unit
# has no way to know what the registration currently says.
#
# hive-matrix's own renderer rather than a `printf` here, so the
# registration's shape has one home.
${deployCfg.matrix.appserviceRegistrationScript}
'';
};
};

2
nix/host-modules/glue-queue-agent-credential.nix
View file

@ -15,7 +15,7 @@
# that is correct rather than degraded. The publisher runs on the authelia
# host and authelia mints on its first boot, so "nothing at that path yet" is
# the ordinary early state of a swarm. Nothing here writes a local stand-in:
# unlike a matrix registration token there is no such thing as a locally valid
# unlike a matrix appservice token there is no such thing as a locally valid
# OIDC client secret, so a placeholder would turn a hive that cannot connect
# into one that is refused, which reaches the agent as a timeout.
#

295
nix/host-modules/hive-matrix.nix
View file

@ -40,18 +40,12 @@ let
# login into an intermittent one.
matrixSecretPath = "/var/lib/tuwunel-oidc/${cfg.sso.clientId}.secret";
# Single source for the registration-token path: the option's `default`,
# its `config`-level `mkDefault` self-definition, and the assertion that
# rejects a moved path all read this same binding rather than repeating
# the string literal. See `registrationTokenFile`'s own comment below.
registrationTokenPath = "/var/lib/hyperhive/matrix-register-token";
# ⚠️ tuwunel does NOT read the path above directly, and this indirection
# ⚠️ tuwunel does NOT read the host path directly, and this indirection
# is not ceremony. Upstream's own words: "under systemd the path must be
# visible to the service after sandboxing (ReadWritePaths / ProtectHome),
# typically by placing the file under /etc/tuwunel/" — which this
# container has no writable etc for. `LoadCredential` is the answer
# already in use two units below for the registration token, and for the
# already in use for the appservice registration below, and for the
# same reason: it keeps `DynamicUser=true` + `PrivateUsers=true` intact
# with no host-side chown or GID pinning.
matrixSecretCredential = "/run/credentials/tuwunel.service/oidc_client_secret";
@ -79,11 +73,15 @@ let
adminLocalpart = "hive";
# The `as_token`, and the `hs_token` the spec requires alongside it. Both
# minted by the activation script below, mode 0600; the `as_token` is the
# one hive-c0re reads and the one the swarm secret store overwrites (see
# minted by the render script below, mode 0600; the `as_token` is the one
# hive-c0re reads and the one the swarm secret store overwrites (see
# `glue-matrix-bao-token.nix`). The `hs_token` authenticates the homeserver
# TO the appservice, which with `url = null` is nobody — it exists because
# the registration format requires it.
#
# `appserviceTokenPath` is the single source for the option's `default`,
# its `config`-level `mkDefault` self-definition, and the assertion that
# rejects a moved path. See `appserviceTokenFile`'s own comment below.
appserviceTokenPath = "/var/lib/hyperhive/matrix-appservice-token";
appserviceHsTokenPath = "/var/lib/hyperhive/matrix-appservice-hs-token";
@ -124,6 +122,64 @@ let
# create these names, not a monopoly on them.
appserviceUserRegex = "^@[a-z0-9._=/-]+:${lib.escapeRegex effectiveServerName}$";
# Mint the tokens if they are absent, then render the registration from
# whatever they now hold. **One script with two callers** — the activation
# script below and `glue-matrix-bao-token.nix`, which overwrites the
# `as_token` with the swarm store's copy and has to re-render afterwards.
# A second copy of the registration's shape would be a second source of
# truth for a format whose mismatch is silent: the homeserver would load a
# registration naming a token nobody holds, and every request hive-c0re
# makes would come back 401 naming nothing.
#
# Re-rendering unconditionally is the point rather than thoroughness: "the
# token file exists" does not mean "the registration carries what is in
# it".
appserviceRegistrationScript = pkgs.writeShellApplication {
name = "hive-matrix-appservice-registration";
runtimeInputs = [ pkgs.coreutils ];
text = ''
# Both the tokens and the rendered registration are secrets; 077
# covers every file this script creates rather than each one
# separately.
umask 077
mkdir -p ${lib.escapeShellArg appserviceDir}
for f in ${lib.escapeShellArg appserviceTokenPath} ${lib.escapeShellArg appserviceHsTokenPath}; do
if [ ! -s "$f" ]; then
head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$f"
echo >> "$f"
echo "hive-matrix: generated appservice token at $f"
fi
chmod 0600 "$f"
done
# Read into shell variables and emitted with `printf`, a shell
# builtin: a token passed as an argument to a real command would land
# in that process's argv, which is world-readable for its lifetime.
asToken="$(cat ${lib.escapeShellArg appserviceTokenPath})"
hsToken="$(cat ${lib.escapeShellArg appserviceHsTokenPath})"
# The quoted heredoc keeps the regex's own `$` and `\` out of the
# shell's hands; the YAML single quotes keep them out of YAML's.
{
cat <<'REGISTRATION'
id: ${appserviceId}
url: null
sender_localpart: ${adminLocalpart}
rate_limited: false
namespaces:
users:
- exclusive: false
regex: '${appserviceUserRegex}'
aliases: []
rooms: []
REGISTRATION
printf 'as_token: %s\nhs_token: %s\n' "$asToken" "$hsToken"
} > ${lib.escapeShellArg appserviceRegistrationPath}
chmod 0600 ${lib.escapeShellArg appserviceRegistrationPath}
chmod 0700 ${lib.escapeShellArg appserviceDir}
'';
};
# Format-locked by tuwunel, not chosen here: the callback host must point
# directly at the matrix server and the path is fixed at
# `/_matrix/client/unstable/login/sso/callback/<client_id>`. Built once
@ -511,40 +567,59 @@ in
'';
};
registrationTokenFile = lib.mkOption {
appserviceTokenFile = lib.mkOption {
type = lib.types.path;
internal = true;
default = registrationTokenPath;
default = appserviceTokenPath;
description = ''
Host path to a file containing the matrix registration token
tuwunel reads to authorise new-account creation. The token is
generated automatically by `hive-c0re` on first boot (32-byte
random hex, mode 0600) and is bind-mounted read-only into the
tuwunel container at the same path. Agents never see this
token — hive-c0re uses it to provision per-agent accounts
and the agent only receives the resulting `access_token`.
Host path to a file containing this hive's matrix appservice
token (`as_token`) — the identity `hive-c0re` creates and logs
into accounts with. Minted automatically on first activation
(32-byte random hex, mode 0600) and rendered into the
appservice registration the homeserver loads at boot. Agents
never see it; an agent only ever receives its own
`access_token`.
Not operator-settable — `hive-c0re`'s Rust side derives this
same path independently (`paths::matrix_register_token()`) with
nothing wiring an override across, so a moved path used to
desync the two silently. An externally-managed token is
delivered by writing into *this* fixed path instead of moving it —
see `glue-matrix-bao-token.nix`, which fetches from the swarm
secret store and overwrites this file in place.
same path independently (`paths::matrix_appservice_token()`)
with nothing wiring an override across, so a moved path desyncs
the two silently. An externally-managed token is delivered by
writing into *this* fixed path instead of moving it — see
`glue-matrix-bao-token.nix`, which fetches from the swarm secret
store, overwrites this file in place, and re-renders the
registration that names it.
Enforced by an `assertions` entry below rather than `readOnly`:
the `config` block gives this option its own `mkDefault`
definition (lowest priority) so any real override — an operator's,
or the pre-rename `swarm.matrix.registrationTokenFile` shim's —
still resolves cleanly instead of crashing eval with nixpkgs'
generic "read-only, set multiple times" message; the assertion
then names the actual problem. (`readOnly` was the first attempt —
atlas caught that it only rejects a *second* definition,
so with nothing else defining the option a lone override sailed
through silently, same desync as before with a lock that wasn't
locked. Fixed by defining the value here instead of leaving it on
`default` alone, but a proper `assertions` message beat re-adding
`readOnly` on top once the option had a real definition either way.)
definition (lowest priority) so a real override still resolves
cleanly instead of crashing eval with nixpkgs' generic
"read-only, set multiple times" message; the assertion then
names the actual problem. (`readOnly` was the first attempt on
this option's predecessor — atlas caught that it only rejects a
*second* definition, so with nothing else defining the option a
lone override sailed through silently, same desync as before
with a lock that wasn't locked. Fixed by defining the value here
instead of leaving it on `default` alone, but a proper
`assertions` message beat re-adding `readOnly` on top once the
option had a real definition either way.)
'';
};
appserviceRegistrationScript = lib.mkOption {
type = lib.types.path;
internal = true;
default = "${appserviceRegistrationScript}/bin/hive-matrix-appservice-registration";
defaultText = lib.literalMD "the module's own registration renderer";
description = ''
The script that mints the appservice tokens when absent and
(re-)renders the registration file from them. An option only so
that `glue-matrix-bao-token.nix` can run the same one after
overwriting the token with the swarm store's copy, rather than
carrying a second copy of the registration's shape.
Not operator-settable, and not a hook: replacing it means
deciding what the homeserver's appservice registration says,
which is this module's job.
'';
};
@ -606,11 +681,11 @@ in
config = lib.mkIf deployCfg.matrix.enable {
# The option's own value, defined explicitly rather than left on its
# bare `default` — `mkDefault` so a real override (an operator's, or
# the pre-rename shim's) still resolves cleanly rather than crashing
# eval; the `assertions` entry below is what actually rejects it, with
# a message naming the reason. See that option's own comment above.
services.hyperhive.deploy.matrix.registrationTokenFile = lib.mkDefault registrationTokenPath;
# bare `default` — `mkDefault` so a real override still resolves
# cleanly rather than crashing eval; the `assertions` entry below is
# what actually rejects it, with a message naming the reason. See that
# option's own comment above.
services.hyperhive.deploy.matrix.appserviceTokenFile = lib.mkDefault appserviceTokenPath;
# Matrix's own gateway surface: the sub-domain vhost, the name the
# hive resolver answers for, and the Accept-header map that vhost's
@ -791,19 +866,17 @@ in
# mkDefault above lets an override resolve instead of crashing
# eval — this is the actual rejection, with a message that names
# the real fix instead of nixpkgs' generic conflicting-definition
# text. Covers both the current path and the pre-rename
# `swarm.matrix.registrationTokenFile` shim in one check, since
# both land on the same merged option.
assertion = deployCfg.matrix.registrationTokenFile == registrationTokenPath;
# text.
assertion = deployCfg.matrix.appserviceTokenFile == appserviceTokenPath;
message = ''
services.hyperhive.deploy.matrix.registrationTokenFile is fixed at
${registrationTokenPath} and cannot be moved — hive-c0re's Rust
services.hyperhive.deploy.matrix.appserviceTokenFile is fixed at
${appserviceTokenPath} and cannot be moved — hive-c0re's Rust
side derives this same path independently and has no way to learn
an override, so moving it desyncs the two silently instead of
loudly.
Integrating an externally-managed registration token? Deliver it
by writing into ${registrationTokenPath} instead of pointing this
Integrating an externally-managed appservice token? Deliver it
by writing into ${appserviceTokenPath} instead of pointing this
option elsewhere — see glue-matrix-bao-token.nix, which does
exactly that from the swarm secret store.
'';
@ -849,8 +922,8 @@ in
# permanent stall presenting as "matrix is broken", several layers from
# its cause.
#
# The registration token above dodges that with an activation script
# that pre-creates the file. ⚠️ That dodge is NOT available here:
# The appservice registration dodges that with an activation script
# that renders the file first. ⚠️ That dodge is NOT available here:
# tuwunel requires the secret file to exist *and be non-empty*, so a
# zero-byte placeholder would satisfy the bind mount and then stop the
# homeserver from starting.
@ -958,74 +1031,21 @@ in
''
);
system.activationScripts.hive-matrix-register-token = lib.stringAfter [ "var" ] ''
tokenFile=${lib.escapeShellArg (toString deployCfg.matrix.registrationTokenFile)}
if [ ! -s "$tokenFile" ]; then
mkdir -p "$(dirname "$tokenFile")"
head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$tokenFile"
echo >> "$tokenFile"
echo "hive-matrix: generated registration token at $tokenFile"
fi
# Re-apply 0600 (normalises any pre-LoadCredential carry-over).
chmod 0600 "$tokenFile"
'';
# The appservice registration: mint the two tokens once, then re-render
# the registration file from them on EVERY activation.
# Mint the appservice tokens and render the registration, before any
# container start.
#
# Re-rendering unconditionally is the point, not thoroughness. The token
# file is overwritten in place by `glue-matrix-bao-token.nix` when the
# swarm secret store has a value for this hive, so "the file exists" does
# not mean "the registration carries what is in it" — and a registration
# carrying a stale token is a homeserver that refuses every request
# hive-c0re makes, with a 401 that names nothing.
#
# An activation script rather than a unit, same as the token above: the
# directory is bind-mounted into the container, and nixos-container
# refuses to start when a bind source is missing. Activation runs before
# the container on a switch and on every boot.
# An activation script rather than a unit: the directory below is
# bind-mounted into the container and nixos-container refuses to start
# when a bind source is missing, so this has to have run first.
# Activation is what runs before the container both on a switch and on
# every boot. (The registration token this replaced used an activation
# script for the same reason, and additionally to dodge nspawn creating
# an empty file at a missing bind target — which tuwunel then read as
# "no token", refusing every registration until the next restart. A
# missing registration file is not silent in that way: the homeserver
# fails its appservice load loudly.)
system.activationScripts.hive-matrix-appservice = lib.stringAfter [ "var" ] ''
appserviceDir=${lib.escapeShellArg appserviceDir}
regFile=${lib.escapeShellArg appserviceRegistrationPath}
# Both the tokens and the rendered registration are secrets; 077 covers
# every file this script creates rather than each one separately.
umask 077
mkdir -p "$appserviceDir"
for f in ${lib.escapeShellArg appserviceTokenPath} ${lib.escapeShellArg appserviceHsTokenPath}; do
if [ ! -s "$f" ]; then
head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$f"
echo >> "$f"
echo "hive-matrix: generated appservice token at $f"
fi
chmod 0600 "$f"
done
# Read into shell variables and emitted with `printf`, a bash builtin:
# a token passed as an argument to a real command would land in that
# process's argv, which is world-readable for its lifetime.
asToken="$(cat ${lib.escapeShellArg appserviceTokenPath})"
hsToken="$(cat ${lib.escapeShellArg appserviceHsTokenPath})"
# The quoted heredoc keeps the regex's own `$` and `\` out of the
# shell's hands; the YAML single quotes keep them out of YAML's.
{
cat <<'REGISTRATION'
id: ${appserviceId}
url: null
sender_localpart: ${adminLocalpart}
rate_limited: false
namespaces:
users:
- exclusive: false
regex: '${appserviceUserRegex}'
aliases: []
rooms: []
REGISTRATION
printf 'as_token: %s\nhs_token: %s\n' "$asToken" "$hsToken"
} > "$regFile"
chmod 0600 "$regFile"
chmod 0700 "$appserviceDir"
${deployCfg.matrix.appserviceRegistrationScript}
'';
containers.hive-matrix = {
@ -1036,16 +1056,13 @@ in
extraFlags = [ "--link-journal=host" ];
# Shared host netns — agents reach tuwunel at localhost:<port>.
privateNetwork = false;
# Read-only bind of the host-managed registration token; tuwunel
# Read-only bind of the host-managed appservice registration; tuwunel
# reads it via systemd LoadCredential below (not directly).
#
# The directory, not the file inside it: the registration is
# re-rendered rather than edited, and binding the file would pin the
# inode the container saw when it started.
bindMounts = {
${deployCfg.matrix.registrationTokenFile} = {
hostPath = deployCfg.matrix.registrationTokenFile;
isReadOnly = true;
};
# The directory, not the file inside it: the registration is
# re-rendered on every activation, and binding the file would pin
# the inode the container saw when it started.
${appserviceDir} = {
hostPath = appserviceDir;
isReadOnly = true;
@ -1141,13 +1158,19 @@ in
# trustedServers keeps it effectively closed.
allow_federation = true;
trusted_servers = deployCfg.matrix.trustedServers;
# Token-gated registration. The absent
# `yes_i_am_very_very_sure_…_open_registration_…` flag
# keeps the server closed to anyone without the token.
allow_registration = true;
# LoadCredential below copies the host file into a
# 0400 dynamic-user-owned path; tuwunel reads from there.
registration_token_file = "/run/credentials/tuwunel.service/registration_token";
# Nobody registers themselves here. Accounts are created by
# the hive's appservice, which this flag does not gate —
# tuwunel checks it only for requests that arrive WITHOUT an
# appservice token, so hive-c0re provisions exactly as before
# and everyone else is refused outright.
#
# ⚠️ Not a hardening afterthought: `allow_registration = true`
# with no registration token configured makes tuwunel REFUSE
# TO START (it demands
# `yes_i_am_very_very_sure_…_open_registration_…` instead). So
# dropping the token and leaving this true is not a lax
# homeserver, it is a homeserver that does not boot.
allow_registration = false;
# Where the hive's appservice registration is read from — the
# credentials directory, for the reasons at
@ -1239,11 +1262,9 @@ in
# host-side chown :tuwunel / GID-pin gymnastics needed.
# See `man systemd.exec` → LoadCredential.
systemd.services.tuwunel.serviceConfig.LoadCredential = [
"registration_token:${toString deployCfg.matrix.registrationTokenFile}"
# Same mechanism, third secret — and the one whose credential id
# carries a `.yaml` suffix on purpose, since `appservice_dir`
# above names this very directory and tuwunel takes only
# `.yaml`/`.yml` entries from it.
# The credential id carries a `.yaml` suffix on purpose:
# `appservice_dir` above names this very directory, and tuwunel
# takes only `.yaml`/`.yml` entries from it.
"${appserviceCredentialId}:${appserviceRegistrationPath}"
# Same mechanism, second secret. tuwunel re-reads this file on
# every OAuth exchange, not just at startup, so it has to

77
nix/module-eval.nix
View file

@ -116,24 +116,45 @@ let
];
};
# The homeserver's turn to split. All eight host-side options are set through
# their pre-rename paths — including `gui.enable`, whose value is deliberately
# the opposite of its default so the definition has to actually land, and both
# packages, stubbed to a derivation neither option defaults to for the same
# reason. All eight so that dropping any single shim entry fails the eval, not
# just the ones the assertions read.
# The homeserver's turn to split. All seven remaining host-side options are
# set through their pre-rename paths — including `gui.enable`, whose value is
# deliberately the opposite of its default so the definition has to actually
# land, and both packages, stubbed to a derivation neither option defaults to
# for the same reason. All seven so that dropping any single shim entry fails
# the eval, not just the ones the assertions read.
#
# `registrationTokenFile` used to be the eighth: both spellings of it are now
# `mkRemovedOptionModule`, exercised by `matrixRemovedToken` below instead —
# setting it here would make every case that reads this fixture fail on that
# one removed option.
matrixOldPath = hive {
deploy.matrix.enable = true;
swarm.matrix.openFirewall = true;
swarm.matrix.trustedServers = [ "matrix.example.invalid" ];
swarm.matrix.maxRequestSize = 31457280;
swarm.matrix.registrationTokenFile = "/etc/matrix/register.token";
swarm.matrix.gui.enable = false;
swarm.matrix.package = pkgs.emptyDirectory;
swarm.matrix.gui.package = pkgs.emptyDirectory;
swarm.matrix.sso.clientSecretFile = "/etc/matrix/oidc.secret";
};
# Moving the appservice token, which desyncs hive-c0re's independently
# derived path from the homeserver's. Its own fixture because the assertion
# it trips would otherwise fire for every case reading `matrixOldPath`.
matrixMovedAppserviceToken = hive {
deploy.matrix.enable = true;
deploy.matrix.appserviceTokenFile = "/etc/matrix/as.token";
swarm.matrix.sso.clientSecretFile = "/etc/matrix/oidc.secret";
};
# A config still naming the removed registration token, through the
# pre-rename spelling — the one an old deployment is most likely to carry.
matrixRemovedToken = hive {
deploy.matrix.enable = true;
swarm.matrix.registrationTokenFile = "/etc/matrix/register.token";
swarm.matrix.sso.clientSecretFile = "/etc/matrix/oidc.secret";
};
# The queue's callout identity, fourth split slice. `autoGenerateCallout` is
# left FALSE on purpose: that is what makes the seed paths the thing deciding
# `responderConfigured`, so the assertion below is about the seeds rather
@ -840,33 +861,41 @@ let
}
{
# Third split, and the one whose readers were hardest to see: the
# registration token is read only through a `let` alias in another
# appservice token is read only through a `let` alias in another
# module, so no full path names it anywhere. Both arms read a rendered
# effect — the host firewall and the container's bind-mount table — so a
# rename that resolves but stops reaching the module still fails.
name = "a config written against the pre-rename matrix paths still opens the port and mounts the token";
name = "a config written against the pre-rename matrix paths still opens the port and mounts the appservice registration";
ok =
let
ports = matrixOldPath.networking.firewall.allowedTCPPorts;
httpPort = matrixOldPath.services.hyperhive.swarm.matrix.httpPort;
in
builtins.elem httpPort ports
&& matrixOldPath.containers.hive-matrix.bindMounts ? "/etc/matrix/register.token";
&& matrixOldPath.containers.hive-matrix.bindMounts ? "/var/lib/hyperhive/matrix-appservice";
}
{
# registrationTokenFile lost its override capability entirely
# (bao-delivered secrets don't need one — glue-matrix-bao-token.nix
# already writes into the fixed path instead of moving it), unlike its
# five siblings in the same rename. `matrixOldPath` above proves the
# override still *resolves* (mkDefault, not a crash) — this proves it
# also gets *rejected*, by a named assertion rather than nixpkgs' generic
# conflicting-definition text. Reads `.assertions` directly (cheap: a
# list of `{assertion; message;}`, not `system.build.toplevel`) rather
# than forcing a real build just to observe a boolean.
name = "overriding registrationTokenFile (even via the pre-rename shim) trips a named assertion, not a silent desync";
# appserviceTokenFile has no override capability (a store-delivered
# secret doesn't need one — glue-matrix-bao-token.nix writes into the
# fixed path instead of moving it), unlike its siblings in the same
# rename. The fixture proves the override still *resolves* (mkDefault,
# not a crash); this proves it also gets *rejected*, by a named
# assertion rather than nixpkgs' generic conflicting-definition text.
# Reads `.assertions` directly (cheap: a list of `{assertion;
# message;}`, not `system.build.toplevel`) rather than forcing a real
# build just to observe a boolean.
name = "moving appserviceTokenFile trips a named assertion, not a silent desync";
ok = lib.any (
a: !a.assertion && lib.hasInfix "registrationTokenFile" a.message
) matrixOldPath.assertions;
a: !a.assertion && lib.hasInfix "appserviceTokenFile" a.message
) matrixMovedAppserviceToken.assertions;
}
{
# The removal has to SAY so. A config carrying the old option through
# an upgrade is the common case, and nixpkgs' bare "option does not
# exist" would name neither what replaced the token nor where to read
# about it.
name = "a config still setting registrationTokenFile is refused with a message naming the appservice";
ok = lib.any (a: !a.assertion && lib.hasInfix "appservice" a.message) matrixRemovedToken.assertions;
}
{
# Reads the DELIVERY UNIT, not the options: `responderConfigured` gates
@ -1518,7 +1547,7 @@ let
s = baoWithMatrix.systemd.services.swarm-bao-matrix-token.script;
in
lib.hasInfix "secret/swarm/hives/" s
&& lib.hasInfix "/matrix/registration-token" s
&& lib.hasInfix "/matrix/appservice-token" s
# The shape it used to have: `matrix` where a principal kind belongs,
# which no grant covers.
&& !(lib.hasInfix "secret/swarm/matrix/" s);
@ -2471,7 +2500,7 @@ let
# The store's first reader. Its unit belongs to the pairing, not to
# either service: matrix must not learn the store exists, and the store
# must not know who reads it.
name = "a store deployed beside the homeserver fetches its registration token";
name = "a store deployed beside the homeserver fetches its appservice token";
ok = baoWithMatrix.systemd.services ? swarm-bao-matrix-token;
}
{

34
swarm-secret-client/src/matrix.rs
View file

@ -26,18 +26,25 @@ pub fn account_path(agent: &str, account: &str) -> Result<String, Error> {
Ok(format!("{prefix}/matrix/{account}"))
}
/// The path holding `hive`'s matrix registration token.
/// The path holding `hive`'s matrix appservice token (`as_token`).
///
/// Keyed per **hive**, not per agent, like [`crate::queue::agent_client_path`]
/// and unlike [`account_path`] above: one homeserver admits one hive's
/// accounts, so the token that creates them is the hive's.
/// accounts, so the identity that creates them is the hive's.
///
/// ⚠️ Renamed from `registration-token` along with what it holds: the
/// homeserver no longer accepts a shared registration secret at all, so a
/// value still stored under the old path would be read by nothing. A hive
/// whose store has only the old path falls back to its locally minted
/// token — see `glue-matrix-bao-token.nix` — so the rename degrades rather
/// than breaks, but the store needs a fresh `put` to take effect again.
///
/// # Errors
/// [`Error::PathSegment`] when `hive` contains anything but `[A-Za-z0-9_-]`,
/// which is what keeps one hive's name from addressing another hive's secret.
pub fn registration_token_path(hive: &str) -> Result<String, Error> {
pub fn appservice_token_path(hive: &str) -> Result<String, Error> {
let prefix = principal_prefix(Kind::Hive, hive)?;
Ok(format!("{prefix}/matrix/registration-token"))
Ok(format!("{prefix}/matrix/appservice-token"))
}
/// What an account's path holds: the token, plus the homeserver it belongs to.
@ -80,28 +87,29 @@ mod tests {
}
#[test]
fn the_registration_token_lands_under_the_hive_prefix_the_grant_covers() {
fn the_appservice_token_lands_under_the_hive_prefix_the_grant_covers() {
// Spelled out for the same reason as above, and with a second job here:
// the read policy grants `secret/data/swarm/hives/<hive>/*`, so this
// string is what makes the path reachable at all.
assert_eq!(
registration_token_path("pr1ma").expect("a plain name is legal"),
"swarm/hives/pr1ma/matrix/registration-token"
appservice_token_path("pr1ma").expect("a plain name is legal"),
"swarm/hives/pr1ma/matrix/appservice-token"
);
}
#[test]
fn the_registration_token_is_not_a_top_level_namespace() {
// The path it used to hold. `Kind` is a closed set and `matrix` is not
// one of its members, so a path with `matrix` as the second segment is
// outside every grant — which is how it came to 403 on every read.
let p = registration_token_path("pr1ma").expect("legal");
fn the_appservice_token_is_not_a_top_level_namespace() {
// The path its predecessor used to hold. `Kind` is a closed set and
// `matrix` is not one of its members, so a path with `matrix` as the
// second segment is outside every grant — which is how it came to 403
// on every read.
let p = appservice_token_path("pr1ma").expect("legal");
assert!(!p.starts_with("swarm/matrix/"), "{p}");
}
#[test]
fn a_traversal_in_the_hive_name_is_refused() {
let e = registration_token_path("../beta").expect_err("a traversal is not");
let e = appservice_token_path("../beta").expect_err("a traversal is not");
assert!(matches!(e, Error::PathSegment { kind: "hive", .. }), "{e}");
}