matrix: remove the registration token

Nothing reads it any more: hive-c0re creates accounts as the hive's
appservice, so the mint, the host file, the bind mount, the
`LoadCredential` entry and tuwunel's `registration_token_file` all go.

⚠️ `allow_registration` has to go to `false` in the same change, and not
as hardening. tuwunel refuses to START when registration is allowed with
no token configured — it demands
`yes_i_am_very_very_sure_…_open_registration_…` instead — so dropping the
token and leaving the flag true is not a lax homeserver, it is one that
does not boot. The flag is checked only for requests arriving without an
appservice token, so hive-c0re provisions exactly as before and everyone
else is refused outright.

The swarm secret store keeps its role, repointed at the credential that
replaced the token (`swarm/hives/<hive>/matrix/appservice-token`). Its
unit now also re-runs hive-matrix's own registration renderer after
writing the file: the token is half an agreement, and a registration
still naming the previous value authenticates nobody. The renderer is
shared through an internal option rather than copied, so the
registration's shape has one home.

Both spellings of `registrationTokenFile` become
`mkRemovedOptionModule` with a message naming what replaced them. A hive
that never set the option — the default — is unaffected; one that pinned
it fails to evaluate with instructions instead of a silent no-op.

An upgraded hive needs no intervention: the activation script has both
halves in place before the homeserver restarts, existing agents keep the
tokens their devices already hold, and the old token file is left on
disk read by nothing. docs/integrations/matrix.md spells the path out.

Refs #4402
This commit is contained in:
atlas 2026-09-15 19:54:50 +02:00
commit 7ee7080b21
11 changed files with 410 additions and 242 deletions

View file

@ -247,10 +247,29 @@ in
[ "services" "hyperhive" "swarm" "matrix" "maxRequestSize" ]
[ "services" "hyperhive" "deploy" "matrix" "maxRequestSize" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "matrix" "registrationTokenFile" ]
[ "services" "hyperhive" "deploy" "matrix" "registrationTokenFile" ]
)
# Both spellings of the registration token, removed rather than renamed:
# the homeserver no longer accepts a shared registration secret at all,
# so there is no path to point a definition at. Accounts are created by
# the hive's appservice, whose token lives at a fixed path this module
# deliberately does not let anyone move.
(lib.mkRemovedOptionModule [ "services" "hyperhive" "swarm" "matrix" "registrationTokenFile" ] ''
The matrix registration token is gone: hive-c0re creates accounts as the
hive's appservice instead, so there is no shared secret for tuwunel to
check. Remove this definition nothing needs replacing, the appservice
token is minted and delivered automatically
(/var/lib/hyperhive/matrix-appservice-token, not operator-settable).
The old token file at /var/lib/hyperhive/matrix-register-token is read by
nothing now and can be deleted. See docs/integrations/matrix.md.
'')
(lib.mkRemovedOptionModule [ "services" "hyperhive" "deploy" "matrix" "registrationTokenFile" ] ''
The matrix registration token is gone: hive-c0re creates accounts as the
hive's appservice instead, so there is no shared secret for tuwunel to
check. Remove this definition nothing needs replacing, the appservice
token is minted and delivered automatically
(/var/lib/hyperhive/matrix-appservice-token, not operator-settable).
The old token file at /var/lib/hyperhive/matrix-register-token is read by
nothing now and can be deleted. See docs/integrations/matrix.md.
'')
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "matrix" "gui" "enable" ]
[ "services" "hyperhive" "deploy" "matrix" "gui" "enable" ]

View file

@ -1,8 +1,9 @@
# Glue: the matrix registration token comes from the secret store.
# Glue: the matrix appservice token comes from the secret store.
#
# The store's first reader, and deliberately a small one. It fetches an opaque
# 32-byte value and writes it where ./hive-matrix.nix already looks — the
# homeserver never learns the store exists, and its config is unchanged.
# 32-byte value and writes it where ./hive-matrix.nix already looks, then asks
# that module's own renderer to re-stamp the appservice registration naming it
# — the homeserver never learns the store exists, and its config is unchanged.
#
# ⚠️ Why this credential first. It has no second file and no format: authelia's
# OIDC secret needs a `.secret` *and* a matching `.digest`, so shipping that
@ -47,12 +48,12 @@ let
# own grant covers. The store's read policy grants `swarm/agents/*` and
# `swarm/hives/<this hive>/*` and nothing else, so a path outside those is a
# 403 rather than a miss, however correct it looks. `swarm-secret-client`'s
# `matrix::registration_token_path` builds the same string from the same
# `matrix::appservice_token_path` builds the same string from the same
# pieces; this literal is the nix half of that one agreement.
#
# `hiveName` has no fallback here for the reason ./glue-bao-tls.nix gives at
# its own use of it: it is asserted set for every hyperhive host.
tokenPath = "secret/swarm/hives/${hyperhiveCfg.hiveName}/matrix/registration-token";
tokenPath = "secret/swarm/hives/${hyperhiveCfg.hiveName}/matrix/appservice-token";
# A literal, not an option — ./hive-matrix.nix names its container
# `containers.hive-matrix` directly and declares no `machine` to derive it
@ -70,7 +71,7 @@ in
services.hyperhive.swarm.otel.journaldUnits = [ "swarm-bao-matrix-token" ];
systemd.services.swarm-bao-matrix-token = {
description = "fetch the matrix registration token from the swarm secret store";
description = "fetch the matrix appservice token from the swarm secret store";
# Every one of these names a unit that exists only where the store runs.
# `Requires=` on an absent unit fails the job outright, so the ordering is
# conditional even though the read is not: off-host there is nothing local
@ -176,8 +177,20 @@ in
fi
umask 077
printf '%s\n' "$token" > ${lib.escapeShellArg (toString deployCfg.matrix.registrationTokenFile)}
chmod 0600 ${lib.escapeShellArg (toString deployCfg.matrix.registrationTokenFile)}
printf '%s\n' "$token" > ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)}
chmod 0600 ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)}
# Re-stamp the registration file from the token just written. The
# token is half an agreement — the registration the homeserver loads
# has to carry the same value — so writing the file and stopping
# would leave the homeserver authenticating hive-c0re against
# whatever activation put there: a 401 on every request, naming
# nothing. Unconditional rather than on-change, because this unit
# has no way to know what the registration currently says.
#
# hive-matrix's own renderer rather than a `printf` here, so the
# registration's shape has one home.
${deployCfg.matrix.appserviceRegistrationScript}
'';
};
};

View file

@ -15,7 +15,7 @@
# that is correct rather than degraded. The publisher runs on the authelia
# host and authelia mints on its first boot, so "nothing at that path yet" is
# the ordinary early state of a swarm. Nothing here writes a local stand-in:
# unlike a matrix registration token there is no such thing as a locally valid
# unlike a matrix appservice token there is no such thing as a locally valid
# OIDC client secret, so a placeholder would turn a hive that cannot connect
# into one that is refused, which reaches the agent as a timeout.
#

View file

@ -40,18 +40,12 @@ let
# login into an intermittent one.
matrixSecretPath = "/var/lib/tuwunel-oidc/${cfg.sso.clientId}.secret";
# Single source for the registration-token path: the option's `default`,
# its `config`-level `mkDefault` self-definition, and the assertion that
# rejects a moved path all read this same binding rather than repeating
# the string literal. See `registrationTokenFile`'s own comment below.
registrationTokenPath = "/var/lib/hyperhive/matrix-register-token";
# ⚠️ tuwunel does NOT read the path above directly, and this indirection
# ⚠️ tuwunel does NOT read the host path directly, and this indirection
# is not ceremony. Upstream's own words: "under systemd the path must be
# visible to the service after sandboxing (ReadWritePaths / ProtectHome),
# typically by placing the file under /etc/tuwunel/" — which this
# container has no writable etc for. `LoadCredential` is the answer
# already in use two units below for the registration token, and for the
# already in use for the appservice registration below, and for the
# same reason: it keeps `DynamicUser=true` + `PrivateUsers=true` intact
# with no host-side chown or GID pinning.
matrixSecretCredential = "/run/credentials/tuwunel.service/oidc_client_secret";
@ -79,11 +73,15 @@ let
adminLocalpart = "hive";
# The `as_token`, and the `hs_token` the spec requires alongside it. Both
# minted by the activation script below, mode 0600; the `as_token` is the
# one hive-c0re reads and the one the swarm secret store overwrites (see
# minted by the render script below, mode 0600; the `as_token` is the one
# hive-c0re reads and the one the swarm secret store overwrites (see
# `glue-matrix-bao-token.nix`). The `hs_token` authenticates the homeserver
# TO the appservice, which with `url = null` is nobody — it exists because
# the registration format requires it.
#
# `appserviceTokenPath` is the single source for the option's `default`,
# its `config`-level `mkDefault` self-definition, and the assertion that
# rejects a moved path. See `appserviceTokenFile`'s own comment below.
appserviceTokenPath = "/var/lib/hyperhive/matrix-appservice-token";
appserviceHsTokenPath = "/var/lib/hyperhive/matrix-appservice-hs-token";
@ -124,6 +122,64 @@ let
# create these names, not a monopoly on them.
appserviceUserRegex = "^@[a-z0-9._=/-]+:${lib.escapeRegex effectiveServerName}$";
# Mint the tokens if they are absent, then render the registration from
# whatever they now hold. **One script with two callers** — the activation
# script below and `glue-matrix-bao-token.nix`, which overwrites the
# `as_token` with the swarm store's copy and has to re-render afterwards.
# A second copy of the registration's shape would be a second source of
# truth for a format whose mismatch is silent: the homeserver would load a
# registration naming a token nobody holds, and every request hive-c0re
# makes would come back 401 naming nothing.
#
# Re-rendering unconditionally is the point rather than thoroughness: "the
# token file exists" does not mean "the registration carries what is in
# it".
appserviceRegistrationScript = pkgs.writeShellApplication {
name = "hive-matrix-appservice-registration";
runtimeInputs = [ pkgs.coreutils ];
text = ''
# Both the tokens and the rendered registration are secrets; 077
# covers every file this script creates rather than each one
# separately.
umask 077
mkdir -p ${lib.escapeShellArg appserviceDir}
for f in ${lib.escapeShellArg appserviceTokenPath} ${lib.escapeShellArg appserviceHsTokenPath}; do
if [ ! -s "$f" ]; then
head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$f"
echo >> "$f"
echo "hive-matrix: generated appservice token at $f"
fi
chmod 0600 "$f"
done
# Read into shell variables and emitted with `printf`, a shell
# builtin: a token passed as an argument to a real command would land
# in that process's argv, which is world-readable for its lifetime.
asToken="$(cat ${lib.escapeShellArg appserviceTokenPath})"
hsToken="$(cat ${lib.escapeShellArg appserviceHsTokenPath})"
# The quoted heredoc keeps the regex's own `$` and `\` out of the
# shell's hands; the YAML single quotes keep them out of YAML's.
{
cat <<'REGISTRATION'
id: ${appserviceId}
url: null
sender_localpart: ${adminLocalpart}
rate_limited: false
namespaces:
users:
- exclusive: false
regex: '${appserviceUserRegex}'
aliases: []
rooms: []
REGISTRATION
printf 'as_token: %s\nhs_token: %s\n' "$asToken" "$hsToken"
} > ${lib.escapeShellArg appserviceRegistrationPath}
chmod 0600 ${lib.escapeShellArg appserviceRegistrationPath}
chmod 0700 ${lib.escapeShellArg appserviceDir}
'';
};
# Format-locked by tuwunel, not chosen here: the callback host must point
# directly at the matrix server and the path is fixed at
# `/_matrix/client/unstable/login/sso/callback/<client_id>`. Built once
@ -511,40 +567,59 @@ in
'';
};
registrationTokenFile = lib.mkOption {
appserviceTokenFile = lib.mkOption {
type = lib.types.path;
internal = true;
default = registrationTokenPath;
default = appserviceTokenPath;
description = ''
Host path to a file containing the matrix registration token
tuwunel reads to authorise new-account creation. The token is
generated automatically by `hive-c0re` on first boot (32-byte
random hex, mode 0600) and is bind-mounted read-only into the
tuwunel container at the same path. Agents never see this
token hive-c0re uses it to provision per-agent accounts
and the agent only receives the resulting `access_token`.
Host path to a file containing this hive's matrix appservice
token (`as_token`) the identity `hive-c0re` creates and logs
into accounts with. Minted automatically on first activation
(32-byte random hex, mode 0600) and rendered into the
appservice registration the homeserver loads at boot. Agents
never see it; an agent only ever receives its own
`access_token`.
Not operator-settable `hive-c0re`'s Rust side derives this
same path independently (`paths::matrix_register_token()`) with
nothing wiring an override across, so a moved path used to
desync the two silently. An externally-managed token is
delivered by writing into *this* fixed path instead of moving it
see `glue-matrix-bao-token.nix`, which fetches from the swarm
secret store and overwrites this file in place.
same path independently (`paths::matrix_appservice_token()`)
with nothing wiring an override across, so a moved path desyncs
the two silently. An externally-managed token is delivered by
writing into *this* fixed path instead of moving it see
`glue-matrix-bao-token.nix`, which fetches from the swarm secret
store, overwrites this file in place, and re-renders the
registration that names it.
Enforced by an `assertions` entry below rather than `readOnly`:
the `config` block gives this option its own `mkDefault`
definition (lowest priority) so any real override an operator's,
or the pre-rename `swarm.matrix.registrationTokenFile` shim's
still resolves cleanly instead of crashing eval with nixpkgs'
generic "read-only, set multiple times" message; the assertion
then names the actual problem. (`readOnly` was the first attempt
atlas caught that it only rejects a *second* definition,
so with nothing else defining the option a lone override sailed
through silently, same desync as before with a lock that wasn't
locked. Fixed by defining the value here instead of leaving it on
`default` alone, but a proper `assertions` message beat re-adding
`readOnly` on top once the option had a real definition either way.)
definition (lowest priority) so a real override still resolves
cleanly instead of crashing eval with nixpkgs' generic
"read-only, set multiple times" message; the assertion then
names the actual problem. (`readOnly` was the first attempt on
this option's predecessor atlas caught that it only rejects a
*second* definition, so with nothing else defining the option a
lone override sailed through silently, same desync as before
with a lock that wasn't locked. Fixed by defining the value here
instead of leaving it on `default` alone, but a proper
`assertions` message beat re-adding `readOnly` on top once the
option had a real definition either way.)
'';
};
appserviceRegistrationScript = lib.mkOption {
type = lib.types.path;
internal = true;
default = "${appserviceRegistrationScript}/bin/hive-matrix-appservice-registration";
defaultText = lib.literalMD "the module's own registration renderer";
description = ''
The script that mints the appservice tokens when absent and
(re-)renders the registration file from them. An option only so
that `glue-matrix-bao-token.nix` can run the same one after
overwriting the token with the swarm store's copy, rather than
carrying a second copy of the registration's shape.
Not operator-settable, and not a hook: replacing it means
deciding what the homeserver's appservice registration says,
which is this module's job.
'';
};
@ -606,11 +681,11 @@ in
config = lib.mkIf deployCfg.matrix.enable {
# The option's own value, defined explicitly rather than left on its
# bare `default` — `mkDefault` so a real override (an operator's, or
# the pre-rename shim's) still resolves cleanly rather than crashing
# eval; the `assertions` entry below is what actually rejects it, with
# a message naming the reason. See that option's own comment above.
services.hyperhive.deploy.matrix.registrationTokenFile = lib.mkDefault registrationTokenPath;
# bare `default` — `mkDefault` so a real override still resolves
# cleanly rather than crashing eval; the `assertions` entry below is
# what actually rejects it, with a message naming the reason. See that
# option's own comment above.
services.hyperhive.deploy.matrix.appserviceTokenFile = lib.mkDefault appserviceTokenPath;
# Matrix's own gateway surface: the sub-domain vhost, the name the
# hive resolver answers for, and the Accept-header map that vhost's
@ -791,19 +866,17 @@ in
# mkDefault above lets an override resolve instead of crashing
# eval — this is the actual rejection, with a message that names
# the real fix instead of nixpkgs' generic conflicting-definition
# text. Covers both the current path and the pre-rename
# `swarm.matrix.registrationTokenFile` shim in one check, since
# both land on the same merged option.
assertion = deployCfg.matrix.registrationTokenFile == registrationTokenPath;
# text.
assertion = deployCfg.matrix.appserviceTokenFile == appserviceTokenPath;
message = ''
services.hyperhive.deploy.matrix.registrationTokenFile is fixed at
${registrationTokenPath} and cannot be moved hive-c0re's Rust
services.hyperhive.deploy.matrix.appserviceTokenFile is fixed at
${appserviceTokenPath} and cannot be moved hive-c0re's Rust
side derives this same path independently and has no way to learn
an override, so moving it desyncs the two silently instead of
loudly.
Integrating an externally-managed registration token? Deliver it
by writing into ${registrationTokenPath} instead of pointing this
Integrating an externally-managed appservice token? Deliver it
by writing into ${appserviceTokenPath} instead of pointing this
option elsewhere see glue-matrix-bao-token.nix, which does
exactly that from the swarm secret store.
'';
@ -849,8 +922,8 @@ in
# permanent stall presenting as "matrix is broken", several layers from
# its cause.
#
# The registration token above dodges that with an activation script
# that pre-creates the file. ⚠️ That dodge is NOT available here:
# The appservice registration dodges that with an activation script
# that renders the file first. ⚠️ That dodge is NOT available here:
# tuwunel requires the secret file to exist *and be non-empty*, so a
# zero-byte placeholder would satisfy the bind mount and then stop the
# homeserver from starting.
@ -958,74 +1031,21 @@ in
''
);
system.activationScripts.hive-matrix-register-token = lib.stringAfter [ "var" ] ''
tokenFile=${lib.escapeShellArg (toString deployCfg.matrix.registrationTokenFile)}
if [ ! -s "$tokenFile" ]; then
mkdir -p "$(dirname "$tokenFile")"
head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$tokenFile"
echo >> "$tokenFile"
echo "hive-matrix: generated registration token at $tokenFile"
fi
# Re-apply 0600 (normalises any pre-LoadCredential carry-over).
chmod 0600 "$tokenFile"
'';
# The appservice registration: mint the two tokens once, then re-render
# the registration file from them on EVERY activation.
# Mint the appservice tokens and render the registration, before any
# container start.
#
# Re-rendering unconditionally is the point, not thoroughness. The token
# file is overwritten in place by `glue-matrix-bao-token.nix` when the
# swarm secret store has a value for this hive, so "the file exists" does
# not mean "the registration carries what is in it" — and a registration
# carrying a stale token is a homeserver that refuses every request
# hive-c0re makes, with a 401 that names nothing.
#
# An activation script rather than a unit, same as the token above: the
# directory is bind-mounted into the container, and nixos-container
# refuses to start when a bind source is missing. Activation runs before
# the container on a switch and on every boot.
# An activation script rather than a unit: the directory below is
# bind-mounted into the container and nixos-container refuses to start
# when a bind source is missing, so this has to have run first.
# Activation is what runs before the container both on a switch and on
# every boot. (The registration token this replaced used an activation
# script for the same reason, and additionally to dodge nspawn creating
# an empty file at a missing bind target — which tuwunel then read as
# "no token", refusing every registration until the next restart. A
# missing registration file is not silent in that way: the homeserver
# fails its appservice load loudly.)
system.activationScripts.hive-matrix-appservice = lib.stringAfter [ "var" ] ''
appserviceDir=${lib.escapeShellArg appserviceDir}
regFile=${lib.escapeShellArg appserviceRegistrationPath}
# Both the tokens and the rendered registration are secrets; 077 covers
# every file this script creates rather than each one separately.
umask 077
mkdir -p "$appserviceDir"
for f in ${lib.escapeShellArg appserviceTokenPath} ${lib.escapeShellArg appserviceHsTokenPath}; do
if [ ! -s "$f" ]; then
head -c 32 /dev/urandom | od -An -tx1 | tr -d ' \n' > "$f"
echo >> "$f"
echo "hive-matrix: generated appservice token at $f"
fi
chmod 0600 "$f"
done
# Read into shell variables and emitted with `printf`, a bash builtin:
# a token passed as an argument to a real command would land in that
# process's argv, which is world-readable for its lifetime.
asToken="$(cat ${lib.escapeShellArg appserviceTokenPath})"
hsToken="$(cat ${lib.escapeShellArg appserviceHsTokenPath})"
# The quoted heredoc keeps the regex's own `$` and `\` out of the
# shell's hands; the YAML single quotes keep them out of YAML's.
{
cat <<'REGISTRATION'
id: ${appserviceId}
url: null
sender_localpart: ${adminLocalpart}
rate_limited: false
namespaces:
users:
- exclusive: false
regex: '${appserviceUserRegex}'
aliases: []
rooms: []
REGISTRATION
printf 'as_token: %s\nhs_token: %s\n' "$asToken" "$hsToken"
} > "$regFile"
chmod 0600 "$regFile"
chmod 0700 "$appserviceDir"
${deployCfg.matrix.appserviceRegistrationScript}
'';
containers.hive-matrix = {
@ -1036,16 +1056,13 @@ in
extraFlags = [ "--link-journal=host" ];
# Shared host netns — agents reach tuwunel at localhost:<port>.
privateNetwork = false;
# Read-only bind of the host-managed registration token; tuwunel
# Read-only bind of the host-managed appservice registration; tuwunel
# reads it via systemd LoadCredential below (not directly).
#
# The directory, not the file inside it: the registration is
# re-rendered rather than edited, and binding the file would pin the
# inode the container saw when it started.
bindMounts = {
${deployCfg.matrix.registrationTokenFile} = {
hostPath = deployCfg.matrix.registrationTokenFile;
isReadOnly = true;
};
# The directory, not the file inside it: the registration is
# re-rendered on every activation, and binding the file would pin
# the inode the container saw when it started.
${appserviceDir} = {
hostPath = appserviceDir;
isReadOnly = true;
@ -1141,13 +1158,19 @@ in
# trustedServers keeps it effectively closed.
allow_federation = true;
trusted_servers = deployCfg.matrix.trustedServers;
# Token-gated registration. The absent
# `yes_i_am_very_very_sure_…_open_registration_…` flag
# keeps the server closed to anyone without the token.
allow_registration = true;
# LoadCredential below copies the host file into a
# 0400 dynamic-user-owned path; tuwunel reads from there.
registration_token_file = "/run/credentials/tuwunel.service/registration_token";
# Nobody registers themselves here. Accounts are created by
# the hive's appservice, which this flag does not gate —
# tuwunel checks it only for requests that arrive WITHOUT an
# appservice token, so hive-c0re provisions exactly as before
# and everyone else is refused outright.
#
# ⚠️ Not a hardening afterthought: `allow_registration = true`
# with no registration token configured makes tuwunel REFUSE
# TO START (it demands
# `yes_i_am_very_very_sure_…_open_registration_…` instead). So
# dropping the token and leaving this true is not a lax
# homeserver, it is a homeserver that does not boot.
allow_registration = false;
# Where the hive's appservice registration is read from — the
# credentials directory, for the reasons at
@ -1239,11 +1262,9 @@ in
# host-side chown :tuwunel / GID-pin gymnastics needed.
# See `man systemd.exec` → LoadCredential.
systemd.services.tuwunel.serviceConfig.LoadCredential = [
"registration_token:${toString deployCfg.matrix.registrationTokenFile}"
# Same mechanism, third secret — and the one whose credential id
# carries a `.yaml` suffix on purpose, since `appservice_dir`
# above names this very directory and tuwunel takes only
# `.yaml`/`.yml` entries from it.
# The credential id carries a `.yaml` suffix on purpose:
# `appservice_dir` above names this very directory, and tuwunel
# takes only `.yaml`/`.yml` entries from it.
"${appserviceCredentialId}:${appserviceRegistrationPath}"
# Same mechanism, second secret. tuwunel re-reads this file on
# every OAuth exchange, not just at startup, so it has to