vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(#1886): trust a peer hive's root CA hive-wide for self-signed federation

Browse source 
Add swarm.peers.<domain>.caCert (path to a peer hive's root CA PEM),
trusted everywhere the hive's own internal CA is — so a self-signed
peer hive can federate (matrix) and any in-hive consumer validates its
certs.

Mechanism (reuses the existing hive-CA embedding): the meta-flake
renderer embeds a LIST of CA files next to each agent's flake —
hive-ca.pem (the hive's own self-signed CA, when active) plus each peer
caCert as peer-ca-<N>.pem — and emits them all in
security.pki.certificateFiles, so every agent trusts them at build
time. The matrix container trusts the same peer CAs for federation TLS.
Nothing is installed in the host trust store; the certs live in the nix
store (no mutable host file).

- meta.rs: embedded_ca_files() = hive CA + peer CAs (from new
  HIVE_PEER_CA_PATHS env); ca_embed_state() tracks the list (content +
  add/remove); sync_agents materialises + stages the list; render emits
  the multi-entry certificateFiles. Tests cover hive-only / hive+peers
  / peers-only / none.
- hive-c0re.nix: HIVE_PEER_CA_PATHS service env (colon-joined caCerts);
  caCert / certFingerprint option docs updated to the hive-wide scope.
- hive-matrix.nix + docs/swarm.md: scope + comment updates.

certFingerprint stays the c0re-only leaf-pin path.
This commit is contained in:
atlas 2026-06-22 14:29:24 +02:00 • committed by mara
commit edad6f863c
4 changed files with 288 additions and 70 deletions
Download patch file Download diff file Expand all files Collapse all files

237
hive-c0re/src/meta.rs
View file

@ -67,12 +67,12 @@ pub async fn sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<()> {
let on_disk = std::fs::read_to_string(&flake_path).unwrap_or_default();
let initial = !dir.join(".git").exists();
// Hive CA embedding (self-signed TLS): keep `./hive-ca.pem` at the meta
// root in lockstep with the host CA so the build-time `certificateFiles`
// reference render_flake emits always resolves. `ca_desired` is empty
// when self-signed TLS isn't active (cert / ACME mode).
let ca_path = dir.join(HIVE_CA_FILE);
let (ca_desired, ca_changed) = hive_ca_state(&dir);
// Embedded-CA list (self-signed hive CA + peer CAs): keep the
// `./hive-ca.pem` / `./peer-ca-<N>.pem` files at the meta root in
// lockstep with their host sources so the build-time `certificateFiles`
// list render_flake emits always resolves. Empty when neither a
// self-signed hive CA nor any peer CA is configured.
let (ca_files, ca_changed) = ca_embed_state(&dir);
// Skip only when both the flake AND the embedded CA are unchanged — a
// CA rotation with an otherwise-identical flake must still re-commit.
@ -98,15 +98,10 @@ pub async fn sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<()> {
std::fs::write(&flake_path, &new_flake)
.with_context(|| format!("write {}", flake_path.display()))?;
// Materialise (or drop) the embedded hive CA next to flake.nix. When
// self-signed TLS is off, `ca_desired` is empty and we remove any stale
// cert so the flake (which no longer references it) stays buildable.
if ca_desired.is_empty() {
let _ = std::fs::remove_file(&ca_path);
} else if ca_changed {
std::fs::write(&ca_path, &ca_desired)
.with_context(|| format!("write {}", ca_path.display()))?;
}
// Materialise the embedded CA list next to flake.nix + drop any stale
// CA file; `ca_touched` is every filename written or removed, staged
// for commit below. Public CA certs only; no private key is embedded.
let ca_touched = materialise_ca_files(&dir, &ca_files)?;
// Reconcile topology.json against the live agent set — adds
// entries for newly-spawned agents (default: manager as parent,
@ -148,11 +143,13 @@ pub async fn sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<()> {
// contain '/flake.nix'". Lock then commit once with both
// flake.nix and flake.lock — single commit per change.
git(&dir, &["add", "flake.nix"]).await?;
// Stage the embedded hive CA — added/updated when self-signed TLS is on,
// or its deletion when it was just removed. `git add <path>` stages a
// deletion when the path is tracked and now gone; best-effort so the
// never-tracked-and-absent case (pathspec mismatch) is a harmless no-op.
let _ = git(&dir, &["add", "--", HIVE_CA_FILE]).await;
// Stage every embedded CA file we wrote or removed (hive CA + peer
// CAs). `git add <path>` stages a deletion when the path is tracked
// and now gone; best-effort so the never-tracked-and-absent case
// (pathspec mismatch) is a harmless no-op.
for name in &ca_touched {
let _ = git(&dir, &["add", "--", name]).await;
}
// Stage topology.json on every sync (regenerated by reconcile
// above when the agent set changed). git add is a no-op when the
// file content is unchanged.
@ -200,6 +197,7 @@ pub async fn sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<()> {
"flake.nix" => Some("flake"),
"flake.lock" => Some("lock"),
"hive-ca.pem" => Some("hive-ca"),
f if f.starts_with("peer-ca-") && has_pem_ext(f) => Some("peer-ca"),
"topology.json" => Some("topology"),
"capabilities.json" => Some("capabilities"),
"tool-groups.json" => Some("tool-groups"),
@ -576,10 +574,10 @@ fn forwarded_env_vars() -> Vec<(&'static str, String)> {
.collect()
}
/// Filename the hive CA cert is embedded under at the meta-flake root.
/// `sync_agents` writes it and `render_flake` references `./hive-ca.pem`
/// in `security.pki.certificateFiles` so every agent trusts it at build
/// time.
/// Filename the hive's own self-signed CA cert is embedded under at the
/// meta-flake root. One entry of the embedded-CA list `render_flake`
/// references in `security.pki.certificateFiles` (see `embedded_ca_files`);
/// peer CAs sit alongside it as `peer-ca-<N>.pem`.
const HIVE_CA_FILE: &str = "hive-ca.pem";
/// Host path of the hive CA *certificate*, when self-signed TLS is active.
@ -597,17 +595,113 @@ fn hive_ca_source() -> Option<String> {
Some(path)
}
/// Embedded-CA state for the meta repo: `(desired_contents, changed)`.
/// `desired_contents` is the host hive CA cert (empty when self-signed TLS
/// is inactive); `changed` is true when it differs from what's already
/// embedded at `<dir>/hive-ca.pem`, so a CA rotation re-commits even when
/// the flake itself is byte-identical.
fn hive_ca_state(dir: &std::path::Path) -> (String, bool) {
let on_disk = std::fs::read_to_string(dir.join(HIVE_CA_FILE)).unwrap_or_default();
let desired = hive_ca_source()
.and_then(|p| std::fs::read_to_string(p).ok())
.unwrap_or_default();
let changed = desired != on_disk;
/// Host paths of peer-hive root CA certificates, from `HIVE_PEER_CA_PATHS`
/// (colon-separated; set by hive-c0re.nix from `swarm.peers.<d>.caCert`).
/// Each is embedded alongside the hive CA so a peer's CA is trusted
/// everywhere the hive's own internal CA is — i.e. by every agent. Empty
/// segments and paths that don't resolve to a file are dropped, so we
/// never reference a `certificateFiles` entry we couldn't embed.
fn peer_ca_sources() -> Vec<String> {
let Ok(raw) = std::env::var("HIVE_PEER_CA_PATHS") else {
return Vec::new();
};
raw.split(':')
.map(str::trim)
.filter(|p| !p.is_empty() && std::path::Path::new(p).is_file())
.map(ToOwned::to_owned)
.collect()
}
/// The ordered set of CA certs embedded next to the meta flake, as
/// `(filename, host_source_path)`. The self-signed hive CA (when active)
/// is `hive-ca.pem`; each peer CA is `peer-ca-<N>.pem` in declaration
/// order. `render_flake` emits exactly these filenames into
/// `security.pki.certificateFiles` and `sync_agents` materialises them,
/// so the rendered reference and the embedded files always agree.
fn embedded_ca_files() -> Vec<(String, String)> {
let mut out = Vec::new();
if let Some(p) = hive_ca_source() {
out.push((HIVE_CA_FILE.to_owned(), p));
}
for (i, p) in peer_ca_sources().into_iter().enumerate() {
out.push((format!("peer-ca-{i}.pem"), p));
}
out
}
/// Write each desired embedded CA file next to `flake.nix` and remove
/// any stale one (a hive CA turned off, or a peer dropped from config),
/// so the flake never references a file we didn't write. Returns every
/// filename written or removed, for the caller to stage. The public CA
/// certs only; no private key is ever embedded.
fn materialise_ca_files(dir: &Path, ca_files: &[(String, String)]) -> Result<Vec<String>> {
let desired: std::collections::HashSet<&str> =
ca_files.iter().map(|(n, _)| n.as_str()).collect();
let mut touched: Vec<String> = Vec::new();
if let Ok(entries) = std::fs::read_dir(dir) {
for e in entries.flatten() {
let fname = e.file_name();
let Some(name) = fname.to_str() else { continue };
if is_embedded_ca_name(name) && !desired.contains(name) {
let _ = std::fs::remove_file(dir.join(name));
touched.push(name.to_owned());
}
}
}
for (name, content) in ca_files {
let path = dir.join(name);
std::fs::write(&path, content).with_context(|| format!("write {}", path.display()))?;
touched.push(name.clone());
}
Ok(touched)
}
/// True for a filename `embedded_ca_files` can produce — the hive CA or
/// a `peer-ca-<N>.pem`. Lets `sync_agents` find stale CA files to clean
/// up (a CA dropped from config) without touching unrelated meta files.
fn is_embedded_ca_name(name: &str) -> bool {
name == HIVE_CA_FILE || (name.starts_with("peer-ca-") && has_pem_ext(name))
}
/// True when `name` ends in a `.pem` extension (case-insensitive). Split
/// out so the embedded-CA filename checks share one spelling and dodge
/// clippy's case-sensitive-extension lint.
fn has_pem_ext(name: &str) -> bool {
Path::new(name)
.extension()
.is_some_and(|ext| ext.eq_ignore_ascii_case("pem"))
}
/// Embedded-CA state for the meta repo: `(desired_files, changed)`.
/// `desired_files` is `(filename, contents)` for every CA that should sit
/// next to flake.nix (the hive CA + each peer CA). `changed` is true when
/// the on-disk set differs in any way — a file's contents changed, a new
/// CA appeared, or a previously-embedded CA (`hive-ca.pem` /
/// `peer-ca-*.pem`) is no longer wanted (stale, to be removed). Drives
/// both the re-commit decision and the materialise/cleanup in
/// `sync_agents`, so a CA rotation or a peer-set change re-commits even
/// when the flake itself is byte-identical.
fn ca_embed_state(dir: &std::path::Path) -> (Vec<(String, String)>, bool) {
let desired: Vec<(String, String)> = embedded_ca_files()
.into_iter()
.filter_map(|(name, path)| std::fs::read_to_string(&path).ok().map(|c| (name, c)))
.collect();
let desired_names: std::collections::HashSet<&str> =
desired.iter().map(|(n, _)| n.as_str()).collect();
let mut changed = desired.iter().any(|(name, content)| {
std::fs::read_to_string(dir.join(name)).unwrap_or_default() != *content
});
// A previously-embedded CA file no longer wanted → stale (removal is
// a change even when every desired file already matches on disk).
if !changed && let Ok(entries) = std::fs::read_dir(dir) {
changed = entries.flatten().any(|e| {
e.file_name()
.to_str()
.is_some_and(|name| is_embedded_ca_name(name) && !desired_names.contains(name))
});
}
(desired, changed)
}
@ -748,16 +842,26 @@ where
{
"#,
);
// Self-signed TLS trust: embed the hive CA so every agent validates the
// gateway's self-signed leaf at build time. `security.pki.certificateFiles`
// is build-time, so the CA travels with the flake source — `sync_agents`
// writes `./hive-ca.pem` next to flake.nix and stages it. Only the public
// CA cert is embedded; the private key never leaves the host. Emitted only
// when hive-tls.nix signalled a CA (HIVE_TLS_CA_PATH) and the cert exists,
// matching the write condition in `sync_agents` so we never reference a
// file we didn't embed.
if hive_ca_source().is_some() {
out.push_str(" security.pki.certificateFiles = [ ./hive-ca.pem ];\n");
// CA trust: embed every hive-trusted CA so each agent validates them at
// build time. The list is the hive's own self-signed CA (when active)
// plus every peer-hive root CA (`swarm.peers.<d>.caCert`) — a peer CA is
// trusted everywhere the hive's own internal CA is. `certificateFiles` is
// build-time, so the certs travel with the flake source: `sync_agents`
// writes `./hive-ca.pem` + `./peer-ca-<N>.pem` next to flake.nix and
// stages them. Only public CA certs are embedded; no private key ever
// leaves the host. The filename list matches `sync_agents` exactly (both
// derive it from `embedded_ca_files`), so we never reference a file we
// didn't embed; emitted only when the list is non-empty.
let ca_refs: Vec<String> = embedded_ca_files()
.into_iter()
.map(|(name, _)| format!("./{name}"))
.collect();
if !ca_refs.is_empty() {
let _ = writeln!(
out,
" security.pki.certificateFiles = [ {} ];",
ca_refs.join(" ")
);
}
out.push_str(
r#" # The harness service inside the container runs as a
@ -1228,23 +1332,64 @@ mod tests {
)
};
// Two peer-hive CA temp files for the list cases.
let peer0 = std::env::temp_dir().join(format!("peer-ca0-test-{}.pem", std::process::id()));
let peer1 = std::env::temp_dir().join(format!("peer-ca1-test-{}.pem", std::process::id()));
std::fs::write(
&peer0,
"-----BEGIN CERTIFICATE-----\np0\n-----END CERTIFICATE-----\n",
)
.expect("write peer CA 0");
std::fs::write(
&peer1,
"-----BEGIN CERTIFICATE-----\np1\n-----END CERTIFICATE-----\n",
)
.expect("write peer CA 1");
let peer_paths = format!("{}:{}", peer0.display(), peer1.display());
// All env mutations are serialised within this one test (no other
// test asserts on these vars), restored before returning.
unsafe {
std::env::remove_var("HIVE_PEER_CA_PATHS");
std::env::set_var("HIVE_TLS_CA_PATH", &ca_file);
}
let with_ca = render();
// Hive CA + peer CAs: the list carries all three, hive CA first.
unsafe {
std::env::set_var("HIVE_PEER_CA_PATHS", &peer_paths);
}
let with_peers = render();
// Peers only (this hive on ACME, federating with self-signed peers).
unsafe {
std::env::remove_var("HIVE_TLS_CA_PATH");
}
let peers_only = render();
unsafe {
std::env::remove_var("HIVE_PEER_CA_PATHS");
}
let without_ca = render();
let _ = std::fs::remove_file(&ca_file);
let _ = std::fs::remove_file(&peer0);
let _ = std::fs::remove_file(&peer1);
assert!(
with_ca.contains("security.pki.certificateFiles = [ ./hive-ca.pem ]"),
"CA cert must be wired into certificateFiles when signalled:\n{with_ca}"
);
assert!(
with_peers.contains(
"security.pki.certificateFiles = [ ./hive-ca.pem ./peer-ca-0.pem ./peer-ca-1.pem ]"
),
"hive CA + peer CAs must all appear in the certificateFiles list:\n{with_peers}"
);
assert!(
peers_only
.contains("security.pki.certificateFiles = [ ./peer-ca-0.pem ./peer-ca-1.pem ]"),
"peer CAs must be trusted even when this hive has no self-signed CA:\n{peers_only}"
);
assert!(
!without_ca.contains("security.pki.certificateFiles"),
"no certificateFiles reference without the HIVE_TLS_CA_PATH signal:\n{without_ca}"
"no certificateFiles reference without any CA signal:\n{without_ca}"
);
}
}