vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

matrix: swarm-controller is the only minter

Browse source 
Every hive is in a swarm and every swarm runs matrix, so every swarm has a
swarm-controller, and since #4810 its hive_sender pass mints each hive's
@hive-<hive>: sender token into the store every five minutes. The two
other minters of that token go:

- swarm-matrix-ctl mint: the systemd.services.swarm-matrix-ctl unit in the
  hive-matrix container, Command::Mint and src/mint.rs. The binary, its
  appservice render/publish verbs, ctlPackage, ctlActive and the ctl cert
  role stay. bao-matrix-reader's checks on the deleted unit are removed;
  the leaf-identity and no-token-in-env checks now look at
  swarm-matrix-appservice-publish, which runs under the same identity.
- the hive-side mint ladder in hive-c0re's ensure_hive_user
  (register/appservice-login/password-login with the local as_token), with
  read_appservice_token, paths::matrix_appservice_token and the helpers
  only it used. ensure_hive_user now takes the store's token, keeps the
  file when the store has none or can't be reached, and fails otherwise.
- hivectl matrix sync-admin: the verb, HostRequest::MatrixSyncAdmin and
  handle_matrix_sync_admin. The periodic MatrixSweep (ensure_all) is
  unchanged apart from no longer reading the local as_token.

This removes the double-mint race #4810's review flagged: two minters
logging in on one pinned device could leave a dead token in the store
until the next pass.

Closes #4813
Closes #4814
This commit is contained in:
atlas 2026-09-29 23:49:57 +02:00 • committed by mara
commit ddb7d7196d
22 changed files with 187 additions and 1162 deletions
Download patch file Download diff file Expand all files Collapse all files

12
nix/module-eval/bao-grants.nix
View file

@ -451,14 +451,10 @@ let
&& !(lib.hasInfix "sys/policies/acl" s);
}
{
# 🩸 `read` is load-bearing here, and the publisher — the one sibling
# that still has no `read` — shows what its absence costs. matrix-ctl's
# first act is to read this path back and stop if something is there —
# that read IS "and only once", so without the capability every container
# restart would mint a second access token and invalidate the hive's.
# (The controller holds `read` for the same idempotency reason, on the
# agent prefix.)
name = "matrix-ctl may read back the one path it writes";
# 🩸 `read` is load-bearing here, unlike on the publisher: matrix-ctl's
# `appservice publish` reads the swarm appservice token back and writes
# only when the store's copy differs.
name = "matrix-ctl may read back what it writes";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-matrix-ctl-policy.script;

85
nix/module-eval/bao-matrix-reader.nix
View file

@ -292,7 +292,8 @@ let
name = "matrix-ctl presents its own leaf, never the hive's store-wide one";
ok =
let
env = baoWithMatrix.containers.hive-matrix.config.systemd.services.swarm-matrix-ctl.environment;
env =
baoWithMatrix.containers.hive-matrix.config.systemd.services.swarm-matrix-appservice-publish.environment;
hiveLeaf = baoWithMatrix.services.hyperhive.deploy.bao.clientCertFile;
in
env.BAO_CLIENT_CERT == "/var/lib/swarm-bao-pki/matrix-ctl.pem" && env.BAO_CLIENT_CERT != hiveLeaf;
@ -316,81 +317,37 @@ let
&& mounts ? "/var/lib/hyperhive/matrix-appservice";
}
{
# What the unit is for, read as the two agreements it cannot get wrong:
# the cert role ./host-modules/swarm-bao.nix writes, and a homeserver
# address that is loopback because the container shares the host netns. A
# vhost here would be a request out through the gateway and back.
name = "matrix-ctl is handed the store role and the loopback homeserver";
# The cert role ./host-modules/swarm-bao.nix writes, which is the one
# agreement the unit cannot get wrong, and the verb: a bare invocation
# exits non-zero with clap's usage, a deploy-time failure with no local
# signal.
name = "the publish unit is handed the store role and invokes its verb";
ok =
let
m = baoWithMatrix;
u = m.containers.hive-matrix.config.systemd.services.swarm-matrix-ctl;
port = m.services.hyperhive.swarm.matrix.httpPort;
u = baoWithMatrix.containers.hive-matrix.config.systemd.services.swarm-matrix-appservice-publish;
in
u.environment.MATRIX_MINT_CERT_ROLE == "swarm-matrix-ctl"
&& u.environment.MATRIX_MINT_API_URL == "http://127.0.0.1:${toString port}"
&& u.environment.MATRIX_MINT_REGISTRATION == "/var/lib/hyperhive/matrix-appservice/hyperhive.yaml"
&& u.serviceConfig.Type == "oneshot";
u.environment.MATRIX_APPSERVICE_CERT_ROLE == "swarm-matrix-ctl"
&& lib.hasSuffix "/bin/swarm-matrix-ctl appservice publish" u.serviceConfig.ExecStart;
}
{
# 🩸 The per-hive minting identity, read off the rendered unit rather than
# off the option: the binary builds its store path out of
# `MATRIX_MINT_HIVE` and logs in as `MATRIX_MINT_LOCALPART`, so a unit
# that passed the old bare `hive` would publish one identity for the
# whole swarm again and nothing in the Rust tests could see it. Both
# spellings are pinned, and the localpart is pinned as *derived from* the
# hive name rather than as a literal, which is the agreement
# `swarm_secret_client::matrix::hive_localpart` owns.
name = "matrix-ctl is told which hive it mints for, and acts as that hive's account";
# 🩸 A secret is a path, never a value. Every variable the unit is given
# names a file or an address; the token itself is read out of the state
# dir at runtime, so nothing here can be a token and an environment block
# is world-readable through `systemctl show`.
name = "the publish unit's environment carries paths and addresses, never a token";
ok =
let
m = baoWithMatrix;
u = m.containers.hive-matrix.config.systemd.services.swarm-matrix-ctl;
hive = m.services.hyperhive.hiveName;
in
u.environment.MATRIX_MINT_HIVE == hive
&& u.environment.MATRIX_MINT_LOCALPART == "hive-${hive}"
&& u.environment.MATRIX_MINT_LOCALPART != "hive";
}
{
# 🩸 The crate is a `*ctl` with subcommands, so the unit has to name a
# VERB. This is the one end of that contract nix owns: the binary's own
# test pins how `mint` is spelled, but only a rendered `ExecStart` can
# say the unit actually passes it. A bare invocation exits non-zero with
# clap's usage — which is a deploy-time failure with no local signal, and
# exactly what the next verb added here is most likely to disturb.
name = "the unit invokes a verb rather than the bare binary";
ok =
let
exec =
baoWithMatrix.containers.hive-matrix.config.systemd.services.swarm-matrix-ctl.serviceConfig.ExecStart;
in
lib.hasSuffix "/bin/swarm-matrix-ctl mint" exec;
}
{
# 🩸 A secret is a path, never a value — checked on the one unit in this
# tree whose whole job is an `as_token`. Every variable it is given names
# a file or an address; the token itself is read out of the bind-mounted
# registration at runtime, so nothing here can be a token and an
# environment block is world-readable through `systemctl show`.
name = "matrix-ctl's environment carries paths and addresses, never a token";
ok =
let
env = baoWithMatrix.containers.hive-matrix.config.systemd.services.swarm-matrix-ctl.environment;
env =
baoWithMatrix.containers.hive-matrix.config.systemd.services.swarm-matrix-appservice-publish.environment;
in
!(lib.any (v: lib.hasInfix "as_token" v || lib.hasInfix "syt_" v) (lib.attrValues env));
}
{
# The absence arm, and the deployment it protects: a homeserver on a hive
# with no store identity at all. Without it the unit would exist naming
# `null` as its certificate, which nixos renders as the literal string.
name = "a matrix container with no store identity runs no matrix-ctl and binds no PKI";
ok =
let
units = matrixNoBaoIdentity.containers.hive-matrix.config.systemd.services;
in
!(units ? swarm-matrix-ctl)
&& !(matrixNoBaoIdentity.containers.hive-matrix.bindMounts ? "/var/lib/swarm-bao-pki");
# with no store identity at all. Without it the mount would name `null`
# as its source, which nixos renders as the literal string.
name = "a matrix container with no store identity binds no PKI";
ok = !(matrixNoBaoIdentity.containers.hive-matrix.bindMounts ? "/var/lib/swarm-bao-pki");
}
{
# 🩸 The privilege arm: exactly one account is a homeserver admin, the