vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

matrix: swarm-controller is the only minter

Browse source 
Every hive is in a swarm and every swarm runs matrix, so every swarm has a
swarm-controller, and since #4810 its hive_sender pass mints each hive's
@hive-<hive>: sender token into the store every five minutes. The two
other minters of that token go:

- swarm-matrix-ctl mint: the systemd.services.swarm-matrix-ctl unit in the
  hive-matrix container, Command::Mint and src/mint.rs. The binary, its
  appservice render/publish verbs, ctlPackage, ctlActive and the ctl cert
  role stay. bao-matrix-reader's checks on the deleted unit are removed;
  the leaf-identity and no-token-in-env checks now look at
  swarm-matrix-appservice-publish, which runs under the same identity.
- the hive-side mint ladder in hive-c0re's ensure_hive_user
  (register/appservice-login/password-login with the local as_token), with
  read_appservice_token, paths::matrix_appservice_token and the helpers
  only it used. ensure_hive_user now takes the store's token, keeps the
  file when the store has none or can't be reached, and fails otherwise.
- hivectl matrix sync-admin: the verb, HostRequest::MatrixSyncAdmin and
  handle_matrix_sync_admin. The periodic MatrixSweep (ensure_all) is
  unchanged apart from no longer reading the local as_token.

This removes the double-mint race #4810's review flagged: two minters
logging in on one pinned device could leave a dead token in the store
until the next pass.

Closes #4813
Closes #4814
This commit is contained in:
atlas 2026-09-29 23:49:57 +02:00 • committed by mara
commit ddb7d7196d
22 changed files with 187 additions and 1162 deletions
Download patch file Download diff file Expand all files Collapse all files

115
nix/host-modules/hive-matrix.nix
View file

@ -81,7 +81,7 @@ let
# which already admits it.
#
# ⚠️ Must equal `swarm_secret_client::matrix::hive_localpart`, which
# hive-c0re and swarm-matrix-ctl both derive from independently with
# hive-c0re and swarm-controller both derive from independently with
# nothing wiring an override across — same agreement, and same reason for
# saying so, as the token path below.
#
@ -92,7 +92,7 @@ let
# The `as_token`, and the `hs_token` the spec requires alongside it. Both
# minted by the render script below, mode 0600; the `as_token` is the one
# hive-c0re reads and the one the swarm secret store overwrites (see
# the swarm secret store overwrites (see
# `glue-matrix-bao-token.nix`). The `hs_token` authenticates the homeserver
# TO the appservice, which with `url = null` is nobody — it exists because
# the registration format requires it.
@ -127,18 +127,12 @@ let
# ── swarm-matrix-ctl ────────────────────────────────────────────────
#
# The oneshot that publishes the appservice sender account's access token to
# the swarm's secret store. It runs INSIDE the container, beside tuwunel,
# because the appservice token that authorises the mint is already in here —
# `appserviceDir` below is bound read-only precisely so the homeserver can
# load it — and minting anywhere else would create a second holder of that
# secret, which is the thing this whole arrangement exists to stop.
#
# Gated on the identity, not on `deploy.bao.enable`: a swarm's ONE homeserver
# is the host least likely to also be the host running the store, so
# "co-located with bao" would leave the intended deployment silently minting
# nothing. Same rule ./swarm-secret-publisher.nix's `haveClientIdentity`
# states, for a sharper reason.
# The container's own store identity, which the swarm appservice units below
# run under. Gated on the identity, not on `deploy.bao.enable`: a swarm's ONE
# homeserver is the host least likely to also be the host running the store,
# so "co-located with bao" would leave the intended deployment silently
# publishing nothing. Same rule ./swarm-secret-publisher.nix's
# `haveClientIdentity` states, for a sharper reason.
ctlActive =
deployCfg.matrix.ctlBaoClientCertFile != null && deployCfg.matrix.ctlBaoClientKeyFile != null;
@ -179,9 +173,9 @@ let
# identity on this homeserver: `swarm-controller` creates every agent's
# account with its token. Minted INSIDE this container by
# `swarm-matrix-ctl appservice render` and published to the store by
# `appservice publish`, which is why it is gated on the same identity as
# the sender mint: with nobody to publish it, a registration here would be
# an admin credential nobody reads.
# `appservice publish`, which is why it is gated on matrix-ctl's store
# identity: with nobody to publish it, a registration here would be an
# admin credential nobody reads.
#
# Its sender is promoted to homeserver admin at boot (`admin_execute`
# below), so its token goes only to a store path no hive's policy reaches.
@ -193,13 +187,6 @@ let
swarmAppserviceDir = "/var/lib/swarm-matrix-appservice";
swarmAppserviceCredentialId = "swarm-appservice.yaml";
# Where a reader of the published credential is told the token is good for.
# Empty when this hive serves no vhost: `matrix::Credential.homeserver` is an
# `Option`, and matrix-ctl reads an empty variable as absent rather than as
# the string "null" — which is what a hive with no gateway host actually
# knows about itself.
ctlHomeserverUrl = if cfg.gatewayHost == null then "" else "https://${toString cfg.gatewayHost}";
# Every local user this hive may provision — agents and `@hive-<hive>:`
# itself — which is the whole matrix localpart charset.
#
@ -677,15 +664,14 @@ in
default = appserviceTokenPath;
description = ''
Host path to a file containing this hive's matrix appservice
token (`as_token`) — the identity `hive-c0re` creates and logs
into accounts with. Minted automatically on first activation
token (`as_token`). Minted automatically on first activation
(32-byte random hex, mode 0600) and rendered into the
appservice registration the homeserver loads at boot. Agents
never see it; an agent only ever receives its own
`access_token`.
Not operator-settable — `hive-c0re`'s Rust side derives this
same path independently (`paths::matrix_appservice_token()`)
Not operator-settable — the module's registration renderer reads
this same path as a literal, not through the option,
with nothing wiring an override across, so a moved path desyncs
the two silently. An externally-managed token is delivered by
writing into *this* fixed path instead of moving it — see
@ -1015,8 +1001,8 @@ in
assertion = deployCfg.matrix.appserviceTokenFile == appserviceTokenPath;
message = ''
services.hyperhive.deploy.matrix.appserviceTokenFile is fixed at
${appserviceTokenPath} and cannot be moved — hive-c0re's Rust
side derives this same path independently and has no way to learn
${appserviceTokenPath} and cannot be moved — the registration
renderer reads this same path as a literal and has no way to learn
an override, so moving it desyncs the two silently instead of
loudly.
@ -1415,66 +1401,6 @@ in
# is why the render below is `requiredBy` it and local only.
++ lib.optional ctlActive "${swarmAppserviceCredentialId}:${swarmAppserviceDir}/swarm.yaml";
# Publish the appservice sender account's access token to the swarm
# store, once, under an identity that belongs to this container and
# not to the hive. See `ctlActive` above for why it runs here.
#
# A `oneshot` with no timer and no retry loop of its own: the whole
# of "and only once" is the binary's first act, a read of the path it
# would write. `Restart=on-failure` covers a store that is sealed or
# a homeserver still starting; `RemainAfterExit` is deliberately NOT
# set, because the unit having succeeded is not the idempotency
# record — the store is, and it outlives this machine.
systemd.services.swarm-matrix-ctl = lib.mkIf ctlActive {
description = "publish the matrix sender token to the swarm secret store";
# Ordered after the homeserver because both of the ladder's arms
# are client-server API calls. `wants`, not `requires`: a run that
# finds the credential already published never touches tuwunel at
# all, so a homeserver that is slow to come up should delay this,
# not cancel it.
after = [ "tuwunel.service" ];
wants = [ "tuwunel.service" ];
wantedBy = [ "multi-user.target" ];
serviceConfig = {
Type = "oneshot";
# The verb is part of the contract: `swarm-matrix-ctl` is a
# subcommand binary and refuses a bare invocation, so dropping
# `mint` here fails the unit rather than doing something else.
ExecStart = "${deployCfg.matrix.ctlPackage}/bin/swarm-matrix-ctl mint";
Restart = "on-failure";
RestartSec = 30;
# Bounded here rather than left to systemd's default, for the
# reason ./swarm-secret-publisher.nix states: a sealed store
# answers on the port and never answers the read.
TimeoutStartSec = 60;
SyslogIdentifier = "swarm-matrix-ctl";
};
environment = {
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
BAO_CLIENT_CERT = deployCfg.matrix.ctlBaoClientCertFile;
BAO_CLIENT_KEY = deployCfg.matrix.ctlBaoClientKeyFile;
MATRIX_MINT_CERT_ROLE = ctlCertRole;
# Loopback: this container shares the host netns, so the
# homeserver it must talk to is the one in this very unit's
# netns and needs no name, no vhost and no TLS.
MATRIX_MINT_API_URL = "http://127.0.0.1:${toString cfg.httpPort}";
# The bind-mounted registration, which IS the as_token. A path,
# never a value.
MATRIX_MINT_REGISTRATION = appserviceRegistrationPath;
MATRIX_MINT_LOCALPART = hiveLocalpart;
# The hive segment of the store path the token is published
# under, and so the thing that keeps this hive's token out of
# every other hive's reach: the grant that reaches it is the
# hive's own `swarm/hives/<name>/*` stanza. ./swarm-bao.nix
# spells the same name into matrix-ctl's write grant.
MATRIX_MINT_HIVE = toString config.services.hyperhive.hiveName;
MATRIX_MINT_HOMESERVER = ctlHomeserverUrl;
}
// lib.optionalAttrs (deployCfg.bao.serverCaFile != null) {
BAO_CACERT = deployCfg.bao.serverCaFile;
};
};
# The swarm registration, minted and rendered before the homeserver
# loads it. No network and no store: this is on tuwunel's start
# path, and a store outage must not keep the homeserver down.
@ -1502,9 +1428,12 @@ in
};
# Hand the swarm registration's token to swarm-controller, through
# the store. Same identity and retry shape as `swarm-matrix-ctl`
# above; the write lands at a path only matrix-ctl and the
# controller are granted (./swarm-bao.nix).
# the store, under matrix-ctl's identity; the write lands at a path
# only matrix-ctl and the controller are granted (./swarm-bao.nix).
# `Restart=on-failure` covers a sealed store or one still starting;
# the start timeout is bounded for the reason
# ./swarm-secret-publisher.nix states: a sealed store answers on the
# port and never answers the read.
systemd.services.swarm-matrix-appservice-publish = lib.mkIf ctlActive {
description = "publish the swarm's appservice token to the swarm secret store";
after = [ "swarm-matrix-appservice-render.service" ];