vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

matrix: mint the appservice sender token in the matrix container

Browse source 
A swarm runs one homeserver and a homeserver has one appservice sender
account, so "mint it once" is a property of the thing being minted
rather than something a lock has to enforce. That is what makes this
account the one to move first: no trigger route, no controller change
and no agent list — a boot-time oneshot beside tuwunel is the whole
mechanism.

`swarm-matrix-minter` runs inside `containers.hive-matrix`, which
already holds the appservice token: the rendered registration is bound
in read-only because that is how tuwunel is handed it. What the
container lacked was an identity of its own, so this adds one — a leaf
from the store's CA with a grant of exactly one path, not the hive's
leaf, which reads every secret in the store.

Both ends of the credential ship here. The minter reads the path it
publishes to before it touches the homeserver, and returning on a
non-empty read IS the "only once"; `hive-c0re`'s `ensure_hive_user`
reads the same path, authenticating with the hive name already in
`HYPERHIVE_HIVE_NAME`. The existing mint-then-`M_USER_IN_USE`-login
ladder stays as the fallback for a store that is empty, unconfigured or
unreachable, which is every swarm deployed before this — so nothing
needs backfilling and nothing breaks if the rest of the sequence never
lands.

The credential is not an admin credential, and is not named like one.
It is the access token of the appservice registration's own
`sender_localpart` — `@hive:<server_name>`, an account the homeserver
creates for itself when it loads the registration. The store path is
`swarm/services/matrix/sender-token`, the host path is
`matrix/access-token`, and the homeserver no longer runs an
`admin_execute` promotion for that account at boot. Everything the hive
provisions with it — the Space, the chat room, their hierarchy and join
rules, the invites — rides on being the creator of those rooms at power
level 100, not on homeserver admin; there is no Synapse admin API here
to need, tuwunel has none.

Two operations do need an admin *sender* and therefore stop working:
`hivectl matrix promote-user` and `hivectl matrix reset-password`, both
`!admin …` messages into `#admins:<server>`, plus the password-reset
recovery path that an agent with a lost password file falls back to.
They are swarm-level operations and are left failing loudly rather than
served by an over-privileged token every other call site would also
carry. The sweep's own admin-rights check and self-repair go with them:
an account that is deliberately not an admin has nothing to check.

`ephemeral = false` stays, and hive root can still read the container's
filesystem. Accepted: what this buys is identity separation — no hive
*process* holds or reads the appservice token — not physical isolation.

Refs #4345
This commit is contained in:
atlas 2026-09-20 13:42:17 +02:00 • committed by mara
commit f778122f5a
28 changed files with 1566 additions and 320 deletions
Download patch file Download diff file Expand all files Collapse all files

227
nix/host-modules/hive-matrix.nix
View file

@ -26,6 +26,11 @@ let
# `null/.well-known/…`.
autheliaCfg = config.services.hyperhive.swarm.authelia;
deployCfg = config.services.hyperhive.deploy;
# Where the swarm's secret store answers. Swarm-tier, identical on every
# host, which is what lets the container address it without knowing whether
# it stands here.
baoCfg = config.services.hyperhive.swarm.bao;
autheliaUrl = autheliaCfg.url;
# The all-local case: this host runs BOTH the homeserver and the swarm's
@ -59,18 +64,19 @@ let
# namespace below, with no shared registration secret in the picture.
appserviceId = "hyperhive";
# The appservice's own user, and deliberately the hive admin account.
# Loading a registration CREATES its `sender_localpart` user when absent
# (tuwunel `src/service/appservice/mod.rs`), on a zero-user database,
# inside `Services::start()` — before the HTTP listener accepts anything.
# That is what lets the `admin_execute` promotion below land on the very
# first boot of a fresh homeserver, instead of depending on hive-c0re
# racing to register the first account and win the auto-admin grant.
# The appservice's own user, and the account the hive acts as. An
# ordinary user, not a homeserver admin: everything the hive does with
# it — the Space, the chat room, the invites — it does as the creator of
# those rooms. Loading a registration CREATES its `sender_localpart` user
# when absent (tuwunel `src/service/appservice/mod.rs`), on a zero-user
# database, inside `Services::start()` — before the HTTP listener accepts
# anything, so the account exists on the very first boot of a fresh
# homeserver without hive-c0re racing to register it.
#
# ⚠️ Must equal `matrix::HIVE_ADMIN_LOCALPART` in hive-c0re, which derives
# ⚠️ Must equal `matrix::HIVE_LOCALPART` in hive-c0re, which derives
# it independently with nothing wiring an override across — same
# agreement, and same reason for saying so, as the token path below.
adminLocalpart = "hive";
hiveLocalpart = "hive";
# The `as_token`, and the `hs_token` the spec requires alongside it. Both
# minted by the render script below, mode 0600; the `as_token` is the one
@ -107,7 +113,62 @@ let
appserviceCredentialId = "${appserviceId}-appservice.yaml";
appserviceCredentialDir = "/run/credentials/tuwunel.service";
# Every local user this hive may provision — agents, the hive admin, and
# ── swarm-matrix-minter ────────────────────────────────────────────────
#
# The oneshot that publishes the `@hive:` account's access token to
# the swarm's secret store. It runs INSIDE the container, beside tuwunel,
# because the appservice token that authorises the mint is already in here —
# `appserviceDir` below is bound read-only precisely so the homeserver can
# load it — and minting anywhere else would create a second holder of that
# secret, which is the thing this whole arrangement exists to stop.
#
# Gated on the identity, not on `deploy.bao.enable`: a swarm's ONE homeserver
# is the host least likely to also be the host running the store, so
# "co-located with bao" would leave the intended deployment silently minting
# nothing. Same rule ./swarm-secret-publisher.nix's `haveClientIdentity`
# states, for a sharper reason.
minterActive =
deployCfg.matrix.minterBaoClientCertFile != null && deployCfg.matrix.minterBaoClientKeyFile != null;
# The role on the store's `cert` auth mount, and so the single source of the
# string both ends must agree on: ./swarm-bao.nix writes the role under
# `matrixMinterPolicyName` and this hands it to the binary in the
# environment, so the binary itself spells no role at all.
minterCertRole = "swarm-matrix-minter";
# Every host directory the minter's bao identity is spread across. Normally
# one — ./glue-bao-tls.nix puts all three files in ./glue's PKI dir — but
# derived rather than assumed, because an operator naming paths by hand is
# exactly the deployment the gate above is written for.
#
# Directories rather than the files, for the reason `appserviceDir`'s own
# comment gives: a re-issued leaf is a new inode, and binding the file would
# pin the one the container saw when it started.
minterPkiDirs = lib.optionals minterActive (
lib.unique (
map builtins.dirOf (
[
deployCfg.matrix.minterBaoClientCertFile
deployCfg.matrix.minterBaoClientKeyFile
]
++ lib.optional (deployCfg.bao.serverCaFile != null) deployCfg.bao.serverCaFile
)
)
);
minterBindMounts = lib.genAttrs minterPkiDirs (dir: {
hostPath = dir;
isReadOnly = true;
});
# Where a reader of the published credential is told the token is good for.
# Empty when this hive serves no vhost: `matrix::Credential.homeserver` is an
# `Option`, and the minter reads an empty variable as absent rather than as
# the string "null" — which is what a hive with no gateway host actually
# knows about itself.
minterHomeserverUrl = if cfg.gatewayHost == null then "" else "https://${toString cfg.gatewayHost}";
# Every local user this hive may provision — agents, `@hive:` itself, and
# the operator accounts `hivectl matrix create-user` makes, which is the
# whole matrix localpart charset.
#
@ -173,7 +234,7 @@ let
cat <<'REGISTRATION'
id: ${appserviceId}
url: null
sender_localpart: ${adminLocalpart}
sender_localpart: ${hiveLocalpart}
rate_limited: false
namespaces:
users:
@ -686,6 +747,56 @@ in
match authelia's register.
'';
};
minterPackage = lib.mkOption {
type = lib.types.package;
defaultText = lib.literalExpression "hyperhive.packages.\${system}.swarm-matrix-minter";
description = ''
The `swarm-matrix-minter` build run inside the matrix container.
⚠️ Named `minterPackage`, not folded into `package` above: that one is
the homeserver, and this is a hyperhive binary that happens to run
beside it. Same split, and same reason, as
{option}`services.hyperhive.deploy.nats.authPackage`.
'';
};
minterBaoClientCertFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
description = ''
Client certificate the matrix container's minter presents to the
swarm's secret store. Its subject must be
{option}`services.hyperhive.deploy.bao.matrixMinterCommonName` — cert
auth matches on the CN, and the role accepts nothing else.
⚠️ **Not the hive's own leaf**, and that is the whole deliverable of
giving this container an identity: the hive's certificate reads every
secret in the store, while this one may write a single path. Pointing
this at `deploy.bao.clientCertFile` would evaluate, deploy and work —
and give away the separation in one line.
No default: a module that guessed would be holding the CA opinion
./swarm-bao.nix deliberately does not hold.
./glue-matrix-minter-bao-identity.nix points it at the leaf
./glue-bao-tls.nix mints, where this host mints one.
The file and its key are bind-mounted into the container read-only.
Co-located in the hive's filesystem and therefore readable by hive
**root** — accepted: the boundary this buys is identity (no hive
*process* holds the appservice token), not physical isolation.
'';
};
minterBaoClientKeyFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
description = ''
Private key for
{option}`services.hyperhive.deploy.matrix.minterBaoClientCertFile`.
Both or neither — the minter unit does not exist unless each is set.
'';
};
};
config = lib.mkIf deployCfg.matrix.enable {
@ -1082,7 +1193,11 @@ in
isReadOnly = true;
};
}
// caTrust.bindMount;
// caTrust.bindMount
# The minter's bao client identity, and nothing else of the hive's. See
# `minterPkiDirs` above for why it is a derived set of directories
# rather than one literal.
// minterBindMounts;
config =
{ ... }:
{
@ -1192,30 +1307,21 @@ in
# `Services::start()`, before the listener accepts anything.
appservice_dir = appserviceCredentialDir;
# The zero-user bootstrap, and the only thing here that needs
# no account to already exist. These run after startup and
# BEFORE the HTTP listener, with no sender and no permission
# check — which is what makes them the one lever that can
# promote the hive admin on a homeserver where nobody is admin
# yet. The appservice registration above creates
# `@${adminLocalpart}` as its sender user moments earlier;
# this joins it to the admin room at power level 100.
# No `admin_execute` promotion for `@${hiveLocalpart}`. The
# hive's account is an ordinary user: it creates the hive
# Space and chat room and invites agents into them, all of
# which ride on being the rooms' own creator at power level
# 100, and none of which is a homeserver-admin capability.
# Granting it server admin at boot would hand a credential
# that every hive reads far more than the work needs.
#
# Idempotent by upstream's own guard: `make_user_admin`
# short-circuits when the user is already joined at 100, so a
# hive that has had an admin for months emits nothing.
admin_execute = [
"users make-user-admin @${adminLocalpart}:${effectiveServerName}"
];
# ⚠️ Load-bearing, not tidiness. An `admin_execute` command
# that fails aborts startup outright when this is false — so
# the one boot where the promotion cannot work (a homeserver
# that has no `@${adminLocalpart}` and no appservice user yet,
# e.g. a registration file that arrived late) would take the
# homeserver down with it rather than converging on the next
# start.
admin_execute_errors_ignore = true;
# The two operations that do need an admin sender —
# `!admin users make-user-admin` and
# `!admin users reset-password`, both messages into
# `#admins:${effectiveServerName}` — therefore have no
# working sender here. They are swarm-level operations and
# are being rehomed as such; until then they fail, loudly,
# rather than being served by an over-privileged token.
# Server-side E2EE is opt-in (default off); the agent matrix
# client always supports decryption regardless.
allow_encryption = cfg.allowEncryption;
@ -1286,6 +1392,57 @@ in
"oidc_client_secret:${toString deployCfg.matrix.sso.clientSecretFile}"
];
# Publish the `@hive:` account's access token to the swarm
# store, once, under an identity that belongs to this container and
# not to the hive. See `minterActive` above for why it runs here.
#
# A `oneshot` with no timer and no retry loop of its own: the whole
# of "and only once" is the binary's first act, a read of the path it
# would write. `Restart=on-failure` covers a store that is sealed or
# a homeserver still starting; `RemainAfterExit` is deliberately NOT
# set, because the unit having succeeded is not the idempotency
# record — the store is, and it outlives this machine.
systemd.services.swarm-matrix-minter = lib.mkIf minterActive {
description = "publish the @hive: matrix credential to the swarm secret store";
# Ordered after the homeserver because both of the ladder's arms
# are client-server API calls. `wants`, not `requires`: a run that
# finds the credential already published never touches tuwunel at
# all, so a homeserver that is slow to come up should delay this,
# not cancel it.
after = [ "tuwunel.service" ];
wants = [ "tuwunel.service" ];
wantedBy = [ "multi-user.target" ];
serviceConfig = {
Type = "oneshot";
ExecStart = "${deployCfg.matrix.minterPackage}/bin/swarm-matrix-minter";
Restart = "on-failure";
RestartSec = 30;
# Bounded here rather than left to systemd's default, for the
# reason ./swarm-secret-publisher.nix states: a sealed store
# answers on the port and never answers the read.
TimeoutStartSec = 60;
SyslogIdentifier = "swarm-matrix-minter";
};
environment = {
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
BAO_CLIENT_CERT = deployCfg.matrix.minterBaoClientCertFile;
BAO_CLIENT_KEY = deployCfg.matrix.minterBaoClientKeyFile;
MATRIX_MINTER_CERT_ROLE = minterCertRole;
# Loopback: this container shares the host netns, so the
# homeserver it must talk to is the one in this very unit's
# netns and needs no name, no vhost and no TLS.
MATRIX_MINTER_API_URL = "http://127.0.0.1:${toString cfg.httpPort}";
# The bind-mounted registration, which IS the as_token. A path,
# never a value.
MATRIX_MINTER_REGISTRATION = appserviceRegistrationPath;
MATRIX_MINTER_LOCALPART = hiveLocalpart;
MATRIX_MINTER_HOMESERVER = minterHomeserverUrl;
}
// lib.optionalAttrs (deployCfg.bao.serverCaFile != null) {
BAO_CACERT = deployCfg.bao.serverCaFile;
};
};
environment.systemPackages = [ deployCfg.matrix.package ];
};
};