vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

matrix: swarm-controller is the only minter

Browse source 
Every hive is in a swarm and every swarm runs matrix, so every swarm has a
swarm-controller, and since #4810 its hive_sender pass mints each hive's
@hive-<hive>: sender token into the store every five minutes. The two
other minters of that token go:

- swarm-matrix-ctl mint: the systemd.services.swarm-matrix-ctl unit in the
  hive-matrix container, Command::Mint and src/mint.rs. The binary, its
  appservice render/publish verbs, ctlPackage, ctlActive and the ctl cert
  role stay. bao-matrix-reader's checks on the deleted unit are removed;
  the leaf-identity and no-token-in-env checks now look at
  swarm-matrix-appservice-publish, which runs under the same identity.
- the hive-side mint ladder in hive-c0re's ensure_hive_user
  (register/appservice-login/password-login with the local as_token), with
  read_appservice_token, paths::matrix_appservice_token and the helpers
  only it used. ensure_hive_user now takes the store's token, keeps the
  file when the store has none or can't be reached, and fails otherwise.
- hivectl matrix sync-admin: the verb, HostRequest::MatrixSyncAdmin and
  handle_matrix_sync_admin. The periodic MatrixSweep (ensure_all) is
  unchanged apart from no longer reading the local as_token.

This removes the double-mint race #4810's review flagged: two minters
logging in on one pinned device could leave a dead token in the store
until the next pass.

Closes #4813
Closes #4814
This commit is contained in:
atlas 2026-09-29 23:49:57 +02:00 • committed by mara
commit ddb7d7196d
22 changed files with 187 additions and 1162 deletions
Download patch file Download diff file Expand all files Collapse all files

37
swarm-matrix-ctl/src/main.rs
View file

@ -7,20 +7,14 @@
//! identity plumbing to add one action, so the next thing that has to run in
//! here is a verb below, not a new crate.
//!
//! [`mint`] publishes a hive's appservice sender token to the swarm's secret
//! store, once. [`appservice`] mints the **swarm's** own appservice
//! registration and publishes its token for `swarm-controller`.
//!
//! It lives in the container because the appservice `as_token` that authorises
//! the mint is *already* there — the registration tuwunel loads is bind-mounted
//! in — so no second holder of that secret is created.
//! [`appservice`] mints the **swarm's** own appservice registration and
//! publishes its token for `swarm-controller`.
//!
//! 🩸 **A secret is a path, never a value.** The only identifier any verb here
//! logs is the store path; see `swarm_matrix_client`'s module doc for the same rule
//! applied to error messages.
mod appservice;
mod mint;
mod registration;
use anyhow::Result;
@ -38,13 +32,6 @@ struct Cli {
#[derive(Debug, Subcommand)]
enum Command {
/// Publish the appservice sender account's access token to the swarm
/// secret store, once.
///
/// Configured entirely by the `MATRIX_MINT_*` environment the unit sets —
/// no flags, because a systemd `Environment=` block is what a nix module
/// can render and a command line full of paths is not.
Mint,
/// The swarm's own appservice registration, whose sender is the
/// homeserver's admin account. Configured by `MATRIX_APPSERVICE_*`.
#[command(subcommand)]
@ -71,7 +58,6 @@ async fn main() -> Result<()> {
.init();
match Cli::parse().command {
Command::Mint => mint::run().await,
Command::Appservice(Appservice::Render) => appservice::render(),
Command::Appservice(Appservice::Publish) => appservice::publish().await,
}
@ -87,17 +73,8 @@ mod tests {
Cli::command().debug_assert();
}
/// The unit's `ExecStart` names a verb, so a rename of it is a deploy-time
/// failure with no local signal. This is that signal.
#[test]
fn mint_is_spelled_the_way_the_unit_invokes_it() {
let cli = Cli::try_parse_from(["swarm-matrix-ctl", "mint"]).expect("`mint` is a verb");
assert!(matches!(cli.command, Command::Mint));
}
/// The control: without it the case above passes on a parser that accepts
/// anything.
/// The two units name these verbs, same reason as the test above.
/// The two units name these verbs in `ExecStart`, so a rename of one is a
/// deploy-time failure with no local signal. This is that signal.
#[test]
fn the_appservice_verbs_are_spelled_the_way_the_units_invoke_them() {
let cli =
@ -122,9 +99,9 @@ mod tests {
.expect_err("only declared verbs are accepted");
}
/// A bare invocation must not silently do something. `mint` writes a
/// credential, so "no verb" defaulting to it would make a typo in the unit
/// mint rather than fail.
/// A bare invocation must not silently do something. `appservice publish`
/// writes a credential, so "no verb" defaulting to it would make a typo in
/// the unit write rather than fail.
#[test]
fn no_verb_at_all_is_refused() {
Cli::try_parse_from(["swarm-matrix-ctl"]).expect_err("a verb is required");

303
swarm-matrix-ctl/src/mint.rs
View file

@ -1,303 +0,0 @@
//! `swarm-matrix-ctl mint` — publish the appservice sender account's
//! homeserver access token to the swarm's secret store, once.
//!
//! A oneshot inside `containers.hive-matrix`, not a daemon and not part of the
//! swarm controller. It mints for **one hive** — the hive this container runs
//! on, named by [`ENV_HIVE`] — and publishes to that hive's own path, so a
//! swarm whose hives share a homeserver gets one account and one token per
//! hive rather than one shared between all of them. "Only once" is therefore
//! once per hive, and it is still a property of what is being minted rather
//! than of a lock: nothing else writes that path.
//!
//! The store, not the homeserver, is the idempotency key — see
//! [`already_published`]. On the hive side `hive-c0re`'s
//! `matrix::ensure_hive_user` reads exactly the path written here, which is how
//! a hive that holds no `as_token` still gets its matrix account.
use anyhow::{Context, Result};
use swarm_secret_client::{
SecretStore,
client::{DEFAULT_CERT_MOUNT, Settings},
matrix,
};
use swarm_matrix_client as homeserver;
use crate::registration;
/// Role on the store's `cert` auth mount to log in with. Its policy is what
/// allows the write below; the certificate the `BAO_*` variables name has to
/// carry the CN that role accepts.
const ENV_CERT_ROLE: &str = "MATRIX_MINT_CERT_ROLE";
/// Client-server API base of the homeserver beside us — loopback, since the
/// container shares the host netns.
const ENV_API_URL: &str = "MATRIX_MINT_API_URL";
/// The bind-mounted appservice registration, which is where the `as_token`
/// comes from. A path, never a value.
const ENV_REGISTRATION: &str = "MATRIX_MINT_REGISTRATION";
/// Localpart of this hive's sender account. The registration's own
/// `sender_localpart`, rendered by `hive-matrix.nix` from the hive name — the
/// same string `swarm_secret_client::matrix::hive_localpart` builds, which is
/// what `hive-c0re` derives its own copy with.
const ENV_LOCALPART: &str = "MATRIX_MINT_LOCALPART";
/// Name of the hive this container belongs to, and so the segment of the store
/// path the token is published under. It is what keeps one hive's token out of
/// another hive's reach — see `swarm_secret_client::matrix::sender_token_path`.
const ENV_HIVE: &str = "MATRIX_MINT_HIVE";
/// Public base URL of the homeserver, stored beside the token so a reader can
/// reconstruct where it is good for. Optional: a swarm with no gateway vhost
/// has no such URL, and `matrix::Credential` types the field to say so.
const ENV_HOMESERVER: &str = "MATRIX_MINT_HOMESERVER";
/// Everything the unit tells this verb, checked before anything is opened.
///
/// Separate from the work for the reason `swarm_secret_client::client::Settings`
/// is: every arm is a misconfiguration an operator reads an error about, and
/// none of them needs a reachable homeserver or store to happen.
#[derive(Debug, PartialEq, Eq)]
struct Config {
cert_role: String,
api_url: String,
registration: String,
localpart: String,
hive: String,
homeserver: Option<String>,
}
impl Config {
/// Read the `MATRIX_MINT_*` variables from the process environment.
///
/// # Errors
/// Naming the first variable that is unset or empty.
fn from_env() -> Result<Self> {
Self::from_lookup(|k| std::env::var(k).ok())
}
/// [`Config::from_env`] against an arbitrary lookup.
///
/// # Errors
/// Naming the first variable that is unset or empty.
fn from_lookup(get: impl Fn(&str) -> Option<String>) -> Result<Self> {
let required = |var: &'static str| -> Result<String> {
get(var)
.filter(|v| !v.is_empty())
.with_context(|| format!("{var} is unset or empty"))
};
Ok(Self {
cert_role: required(ENV_CERT_ROLE)?,
api_url: required(ENV_API_URL)?,
registration: required(ENV_REGISTRATION)?,
localpart: required(ENV_LOCALPART)?,
hive: required(ENV_HIVE)?,
// Empty is absent: systemd renders an unset nix option as
// `Environment=VAR=`, so that is the shape this arrives in.
homeserver: get(ENV_HOMESERVER).filter(|v| !v.is_empty()),
})
}
}
/// Is the credential already in the store?
///
/// **This read is the "and only once".** The homeserver is not asked — a
/// re-run of the container, or of this unit, costs one store read and stops.
/// It is also the read-back of what a previous run wrote, so the path published
/// and the path consulted cannot drift apart: they are one function call.
///
/// A failure to read is reported and treated as absent rather than raised. The
/// two cases that reach it are a path that has never been written (the first
/// run, which must go on to mint) and a token whose policy does not cover the
/// path — and the second fails again, loudly and with the store's own message,
/// at the write below.
async fn already_published(store: &SecretStore, path: &str) -> bool {
match store.read::<matrix::Credential>(path).await {
Ok(credential) => !credential.value.trim().is_empty(),
Err(e) => {
tracing::info!(%path, error = %e, "nothing readable in the store yet");
false
}
}
}
/// Run the verb.
///
/// # Errors
/// If the environment is incomplete, the store refuses the login or the write,
/// the registration cannot be read, or the homeserver refuses both the
/// registration and the appservice login.
pub async fn run() -> Result<()> {
let config = Config::from_env()?;
// Explicitly, rather than through `SecretStore::from_env`: a missing or
// misspelled `BAO_*` variable is the most likely thing to be wrong with a
// freshly deployed unit, and this reports it before the homeserver is
// touched at all.
let settings = Settings::from_env().context("reading the store's BAO_* environment")?;
let store = SecretStore::connect(&settings, &config.cert_role, DEFAULT_CERT_MOUNT)
.await
.with_context(|| {
format!(
"logging in to the swarm secret store as cert role {}",
config.cert_role
)
})?;
let path = matrix::sender_token_path(&config.hive)
.with_context(|| format!("building the store path for hive {}", config.hive))?;
if already_published(&store, &path).await {
tracing::info!(%path, "the sender token is already published; not minting");
return Ok(());
}
let as_token = registration::as_token(&config.registration)?;
let http = homeserver::client()?;
let token =
match homeserver::register(&http, &config.api_url, &config.localpart, &as_token).await? {
homeserver::Registered::Token(token) => token,
homeserver::Registered::AlreadyExists => {
// The expected arm, not an edge case: this account is the
// appservice's own `sender_localpart`, so the homeserver creates it
// when it loads the registration — before anything gets to ask.
tracing::info!("the sender account exists; logging in as the appservice instead");
homeserver::appservice_login(&http, &config.api_url, &config.localpart, &as_token)
.await?
}
};
store
.write(
&path,
&matrix::Credential {
value: token,
homeserver: config.homeserver,
},
)
.await
.with_context(|| format!("writing the sender token to {path}"))?;
tracing::info!(%path, "published the sender token");
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
/// A lookup standing in for a fully-configured unit's environment.
fn full(k: &str) -> Option<String> {
match k {
ENV_CERT_ROLE => Some("swarm-matrix-ctl".to_owned()),
ENV_API_URL => Some("http://127.0.0.1:8008".to_owned()),
ENV_REGISTRATION => {
Some("/var/lib/hyperhive/matrix-appservice/hyperhive.yaml".to_owned())
}
ENV_LOCALPART => Some("hive-pr1ma".to_owned()),
ENV_HIVE => Some("pr1ma".to_owned()),
_ => None,
}
}
#[test]
fn a_complete_environment_is_accepted() {
// The control: without it every assertion below could be passing
// because `from_lookup` rejects everything.
let c = Config::from_lookup(full).expect("every required variable is set");
assert_eq!(c.localpart, "hive-pr1ma");
assert_eq!(c.hive, "pr1ma");
assert_eq!(c.homeserver, None, "an absent public URL is not an error");
}
#[test]
fn each_required_variable_is_named_when_it_is_the_missing_one() {
for var in [
ENV_CERT_ROLE,
ENV_API_URL,
ENV_REGISTRATION,
ENV_LOCALPART,
ENV_HIVE,
] {
let e = Config::from_lookup(|k| if k == var { None } else { full(k) })
.expect_err("one required variable is absent");
assert!(
format!("{e}").contains(var),
"dropping {var} should name {var}, got {e}"
);
}
}
#[test]
fn an_empty_variable_is_as_absent_as_an_unset_one() {
// systemd writes `Environment=VAR=` for an unset nix option, so empty
// is the shape these actually arrive in.
let e = Config::from_lookup(|k| {
if k == ENV_CERT_ROLE {
Some(String::new())
} else {
full(k)
}
})
.expect_err("an empty role is not a role");
assert!(format!("{e}").contains(ENV_CERT_ROLE), "{e}");
let c = Config::from_lookup(|k| {
if k == ENV_HOMESERVER {
Some(String::new())
} else {
full(k)
}
})
.expect("an empty public URL is optional, not fatal");
assert_eq!(c.homeserver, None);
}
/// The environment prefix is a contract with the nix unit, and the crate
/// rename that produced it moved every one of these. A verb-scoped prefix
/// is the point: the next verb brings its own, instead of widening a
/// binary-scoped one nobody can then narrow.
#[test]
fn every_variable_is_scoped_to_the_verb() {
for var in [
ENV_CERT_ROLE,
ENV_API_URL,
ENV_REGISTRATION,
ENV_LOCALPART,
ENV_HIVE,
ENV_HOMESERVER,
] {
assert!(
var.starts_with("MATRIX_MINT_"),
"{var} is not scoped to the mint verb"
);
}
}
#[test]
fn the_published_path_is_the_one_the_hive_reads() {
// Both ends of this slice's loop resolve the same function, so there is
// no second spelling to drift — this pins that the loop exists at all,
// and names the literal so a move of the path is a deliberate edit on
// both sides rather than a silent 404 on the reading one.
assert_eq!(
matrix::sender_token_path("pr1ma").expect("a plain name is legal"),
"swarm/hives/pr1ma/matrix/sender-token"
);
}
#[test]
fn two_hives_are_published_to_two_paths() {
// What the hive segment is FOR: this binary runs beside a homeserver
// several hives share, so a path without the hive name in it would
// have each run overwrite the last and leave every hive holding one
// identity — which is the shape this change exists to end.
let a = matrix::sender_token_path("alpha").expect("legal");
let b = matrix::sender_token_path("beta").expect("legal");
assert_ne!(a, b);
}
#[test]
fn the_localpart_the_unit_hands_over_is_the_one_derived_from_the_hive() {
// The nix unit renders both variables independently; this pins that
// the pair it is expected to render agrees with the shared derivation,
// so a unit still passing the old bare `hive` fails here rather than
// silently logging in as another hive's account.
let c = Config::from_lookup(full).expect("every required variable is set");
assert_eq!(c.localpart, matrix::hive_localpart(&c.hive));
}
}