vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

matrix: back up agents' room keys with a controller-minted recovery passphrase

Browse source 
An agent's matrix encryption keys lived only in its local crypto store, so
`destroy --purge` or a primary-account token reset lost every encrypted
room's history with no way back (#5022).

swarm-controller now mints a random passphrase (32 bytes from the kernel's
CSPRNG, base64url) for every account listed under each agent's `matrix/`
directory, at `swarm/agents/<agent>/matrix-recovery/<account>`, in a
five-minute pass. The write is a cas=0 create after a read; a path that
holds anything is never replaced, and a create that loses a race counts as
done. The controller's existing `data/swarm/agents/*` grant covers the
write and the agent's own `agents/<agent>/*` read grant covers the read, so
no policy changes.

hive-matrix-daemon reads the passphrase when it brings an account up, after
cross-signing: with no backup on the homeserver it enables recovery with it
(backup + secret storage); with one there it recovers, which imports the
backup key and the self-signing key, and room keys then download through
`BackupDownloadStrategy::OneShot`. No passphrase yet logs at INFO and the
account comes up without backup; any failure logs WARN and never stops the
daemon.

Recovery runs after `ensure_cross_signing`, not before: on a new account
cross-signing must exist locally for `enable` to export it into secret
storage, and on an existing identity `bootstrap_cross_signing_if_needed` is
a no-op, leaving `recover` to import and sign.

Part of #5022 (S1+S2, with the S4 docs).
This commit is contained in:
atlas 2026-10-13 01:19:38 +02:00
commit 45568f6e60
11 changed files with 615 additions and 23 deletions
Download patch file Download diff file Expand all files Collapse all files

285
swarm-controller/src/matrix_account/recovery.rs Normal file
View file

@ -0,0 +1,285 @@
//! A recovery passphrase for every matrix account of every agent, stored at
//! `swarm_secret_client::matrix::recovery_path`: what the agent's matrix daemon
//! turns on server-side key backup with, and recovers its keys with once its
//! local crypto store is gone.
//!
//! Minted here because the agent's store policy is read-only. [`spawn`]'s pass
//! runs at start and every [`RECONCILE_INTERVAL`] over every agent that holds a
//! store identity — the same roster as [`super::agent_token`] — and every
//! account listed under that agent's `matrix/` directory.
//!
//! ⚠️ **Never replaced.** A backup can only be opened with the passphrase it
//! was enabled with, so a new one locks the agent out of everything backed up
//! under the old. The write is a `cas=0` create, and a path that holds anything
//! is left alone.
use anyhow::{Context, Result};
use serde::de::IgnoredAny;
use swarm_secret_client::{Error, SecretStore, client::DEFAULT_CERT_MOUNT, matrix, policy};
/// How often [`spawn`] re-checks every account.
const RECONCILE_INTERVAL: std::time::Duration = std::time::Duration::from_mins(5);
/// Bytes drawn from the kernel's CSPRNG per passphrase.
const PASSPHRASE_BYTES: usize = 32;
/// What [`ensure`] did at one path.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Outcome {
/// The path already held a passphrase.
Present,
/// This pass wrote one.
Minted,
/// Another writer's create landed between this pass's read and its own.
LostRace,
}
/// The two store calls [`ensure`] makes, so a test can stand in for the store.
pub(crate) trait PassphraseStore {
/// Whether anything is stored at `path`, as [`SecretStore::read_optional`]
/// answers it. The value is decoded as [`IgnoredAny`] and never held.
fn exists(&self, path: &str) -> impl Future<Output = Result<bool, Error>> + Send;
/// As [`SecretStore::create`].
fn create(
&self,
path: &str,
value: &matrix::Recovery,
) -> impl Future<Output = Result<(), Error>> + Send;
}
impl PassphraseStore for SecretStore {
async fn exists(&self, path: &str) -> Result<bool, Error> {
Ok(self.read_optional::<IgnoredAny>(path).await?.is_some())
}
async fn create(&self, path: &str, value: &matrix::Recovery) -> Result<(), Error> {
SecretStore::create(self, path, value).await
}
}
/// A passphrase: [`PASSPHRASE_BYTES`] from the kernel's CSPRNG, base64url
/// without padding.
///
/// # Errors
/// When the kernel will not supply randomness.
fn generate() -> Result<matrix::Recovery> {
let mut bytes = [0u8; PASSPHRASE_BYTES];
getrandom::fill(&mut bytes)
.context("drawing a recovery passphrase from the kernel's CSPRNG")?;
Ok(matrix::Recovery {
value: base64::Engine::encode(&base64::engine::general_purpose::URL_SAFE_NO_PAD, bytes),
})
}
/// Make sure `path` holds a passphrase, writing one only if it holds nothing.
///
/// A create that fails is read back: if the path now holds something, a
/// concurrent writer won and that is the passphrase.
///
/// # Errors
/// When the store refuses the read, or the create fails and the path is still
/// empty.
pub(crate) async fn ensure(store: &impl PassphraseStore, path: &str) -> Result<Outcome> {
if store
.exists(path)
.await
.with_context(|| format!("reading {path}"))?
{
return Ok(Outcome::Present);
}
let Err(e) = store.create(path, &generate()?).await else {
return Ok(Outcome::Minted);
};
if store
.exists(path)
.await
.with_context(|| format!("reading {path} back after a failed create"))?
{
return Ok(Outcome::LostRace);
}
Err(anyhow::Error::new(e).context(format!("creating {path}")))
}
/// The account names listed under `agent`'s `matrix/` directory. A key ending
/// in `/` is a directory below it, not an account.
async fn accounts(store: &SecretStore, agent: &str) -> Result<Vec<String>> {
let dir = matrix::accounts_dir(agent)?;
let keys = store
.list(&dir)
.await
.with_context(|| format!("listing {dir}"))?;
Ok(keys.into_iter().filter(|k| !k.ends_with('/')).collect())
}
/// One pass over every agent's accounts. Returns how many paths were checked
/// and how many were minted. A failure for one agent or account is logged and
/// the pass carries on.
async fn pass() -> Result<(usize, usize)> {
let store = crate::store::connect()
.await
.context("logging in to the swarm secret store")?;
let roles = store
.list_cert_roles(DEFAULT_CERT_MOUNT)
.await
.context("listing the store's cert-auth roles")?;
let (mut checked, mut minted) = (0, 0);
for agent in policy::agents_from_role_names(&roles) {
let names = match accounts(&store, &agent).await {
Ok(names) => names,
Err(e) => {
tracing::warn!(agent, error = %format!("{e:#}"), "matrix recovery passphrases: listing failed");
continue;
}
};
for account in names {
checked += 1;
let path = match matrix::recovery_path(&agent, &account) {
Ok(path) => path,
Err(e) => {
tracing::warn!(agent, account, error = %e, "matrix recovery passphrase: unusable account name");
continue;
}
};
match ensure(&store, &path).await {
Ok(Outcome::Present) => {}
Ok(Outcome::Minted) => {
minted += 1;
tracing::info!(agent, account, %path, "matrix recovery passphrase minted");
}
Ok(Outcome::LostRace) => {
tracing::info!(agent, account, %path, "matrix recovery passphrase: another writer created it first");
}
Err(e) => tracing::warn!(
agent,
account,
error = %format!("{e:#}"),
"matrix recovery passphrase: mint failed"
),
}
}
}
Ok((checked, minted))
}
/// Run [`pass`] now and every [`RECONCILE_INTERVAL`] after. A pass that fails
/// is logged and retried on the next tick; it never stops the daemon.
pub fn spawn() {
tokio::spawn(async move {
let mut ticker = tokio::time::interval(RECONCILE_INTERVAL);
loop {
ticker.tick().await;
match pass().await {
Ok((checked, minted)) => {
tracing::debug!(checked, minted, "matrix recovery passphrases: pass done");
}
Err(e) => tracing::warn!(
error = %format!("{e:#}"),
retry_in_s = RECONCILE_INTERVAL.as_secs(),
"matrix recovery passphrases: pass failed; retrying next tick"
),
}
}
});
}
#[cfg(test)]
mod tests {
use std::collections::HashMap;
use std::sync::Mutex;
use super::*;
/// An in-memory store. `race` makes the next create find the path taken by
/// someone else, the way a concurrent `cas=0` loses.
#[derive(Default)]
struct Fake {
held: Mutex<HashMap<String, String>>,
race: Option<String>,
refuse_reads: bool,
}
/// A store error with no store behind it.
fn refused() -> Error {
Error::MissingEnv("BAO_ADDR")
}
impl PassphraseStore for Fake {
async fn exists(&self, path: &str) -> Result<bool, Error> {
if self.refuse_reads {
return Err(refused());
}
Ok(self.held.lock().unwrap().contains_key(path))
}
async fn create(&self, path: &str, value: &matrix::Recovery) -> Result<(), Error> {
let mut held = self.held.lock().unwrap();
if let Some(winner) = &self.race {
held.insert(path.to_owned(), winner.clone());
return Err(refused());
}
if held.contains_key(path) {
return Err(refused());
}
held.insert(path.to_owned(), value.value.clone());
Ok(())
}
}
const PATH: &str = "swarm/agents/atlas/matrix-recovery/main";
#[tokio::test]
async fn an_empty_path_gets_a_passphrase() {
let store = Fake::default();
assert_eq!(ensure(&store, PATH).await.unwrap(), Outcome::Minted);
assert!(store.held.lock().unwrap().contains_key(PATH));
}
#[tokio::test]
async fn a_held_passphrase_is_never_replaced() {
let store = Fake::default();
store
.held
.lock()
.unwrap()
.insert(PATH.to_owned(), "first".to_owned());
assert_eq!(ensure(&store, PATH).await.unwrap(), Outcome::Present);
assert_eq!(store.held.lock().unwrap()[PATH], "first");
}
#[tokio::test]
async fn a_create_another_writer_beat_counts_as_done() {
let store = Fake {
race: Some("winner".to_owned()),
..Fake::default()
};
assert_eq!(ensure(&store, PATH).await.unwrap(), Outcome::LostRace);
assert_eq!(store.held.lock().unwrap()[PATH], "winner");
}
#[tokio::test]
async fn a_store_that_cannot_be_read_gets_no_write() {
// Reading "absent" into a refusal would overwrite a live passphrase
// whenever the store was merely unreachable.
let store = Fake {
refuse_reads: true,
..Fake::default()
};
assert!(ensure(&store, PATH).await.is_err());
assert!(store.held.lock().unwrap().is_empty());
}
#[test]
fn a_passphrase_is_32_random_bytes_in_a_plain_alphabet() {
let a = generate().unwrap().value;
let b = generate().unwrap().value;
// 32 bytes is 43 base64 characters without padding.
assert_eq!(a.len(), 43, "{a}");
assert!(
a.chars()
.all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_'),
"{a}"
);
assert_ne!(a, b);
}
}