vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

refactor(nix): swarm.peers becomes swarm.hives, a directory of every hive

Browse source 
One attrset describing every hive in the swarm including this one,
identical on every host, with hiveName selecting which entry is us.
"My peers" is derived (swarm.peerHives) rather than declared.

Every field in the old per-host peer list was intrinsic to the hive it
described, never to the pair -- so the list was a directory each host
kept its own copy of. Beyond the deduplication it removes a bug class:
two hosts could hold different endpoints for the same third hive with
nothing to detect the disagreement.

Drops the per-hive caCert. Trust inside a swarm derives from the swarm
root, which every hive chains to. What that genuinely removes is
trusting a hive whose root this swarm does not own -- a cross-swarm
problem that wants a mechanism of its own, not a field that happened to
work.

The matrix container's certificateFiles block goes with it and could
NOT be migrated: that list is read at build time and the swarm root is
a runtime file (its key must never enter the store), so there is no
build-time name to put there. caCert being a nix path was precisely
what made it the build-time distribution channel. Agents are unaffected
-- hive-tls folds the root into the hive trust bundle and the meta
renderer embeds that one file. Tracked separately.

Migration is an assertion plus warnings, not a rename: hives is peers
union {self}, and the set gains a member no existing config has written
down. A rename migrates a name and a default can re-root a meaning;
neither can conjure a new member. The warning explains, the self-entry
assertion stops the build.
This commit is contained in:
atlas 2026-08-05 20:18:22 +02:00
commit 433b294099
19 changed files with 484 additions and 293 deletions
Download patch file Download diff file Expand all files Collapse all files

6
docs/gateway.md
View file

@ -188,7 +188,7 @@ nginx inside the gateway container obtains and auto-renews certs via the ACME HT
Mutual exclusion: `tls.certDir` set together with `tls.acme.enable = true` fails an assertion — pick one external TLS source (or neither, for the self-signed default).
**Swarm peers**: CA-signed certs are trusted by default — remote hives need no `certFingerprint` in `swarm.peers`.
**Swarm peers**: CA-signed certs are trusted by default — this hive's entry in `swarm.hives` needs no `certFingerprint`.
### Self-signed TLS (default)
@ -232,10 +232,10 @@ security.acme.certs."example.com".group = "nginx";
or make the key world-readable (`0644`) if your threat model allows it. nginx errors out at startup on a key it can't read — the error is explicit in the journal, not a silent failure.
**Peer hive config**: when using a CA-signed cert, peer hives can declare this hive without `certFingerprint` in `swarm.peers` — the standard CA bundle validates:
**Swarm directory entry**: when using a CA-signed cert, this hive's entry needs no `certFingerprint` — the standard CA bundle validates:
```nix
services.hyperhive.swarm.peers."example.com" = { }; # no certFingerprint needed
services.hyperhive.swarm.hives.example = { domain = "example.com"; }; # no certFingerprint needed
```
### Fronting with an external TLS terminator

4
docs/snapshot-store.md
View file

@ -33,8 +33,8 @@ services.hyperhive.swarm.wireguard = {
};
```
The store host is a swarm member like any other: peers declare it, and
it declares them, through `services.hyperhive.swarm.peers`. See
The store host is a swarm member like any other: it gets an entry in
`services.hyperhive.swarm.hives`, the same directory every host holds. See
[swarm/](swarm/README.md) for the mesh itself.
Note that the mesh is gated on `swarm.wireguard.enable`, **not** on

128
docs/swarm/README.md
View file

@ -12,8 +12,9 @@ the additional config needed when the swarm spans multiple hosts.
containers.
- **swarm** — one or more hives whose operators have declared them
as peers. Agents can be qualified as `agent@hive-domain`.
- **peer hive** — a remote hive declared under
`services.hyperhive.swarm.peers` on the local host.
- **peer hive** — any hive in `services.hyperhive.swarm.hives` other
than this one. Peers are *derived*, not declared: the directory lists
every hive including yourself, and `hiveName` says which one you are.
## Hive identity config
@ -71,41 +72,43 @@ One authelia, one matrix, one forge per swarm — which host runs them,
and what a hive that runs none of them configures instead:
[`services.md`](services.md).
## Declaring peer hives
## The swarm's hive directory
```nix
services.hyperhive.swarm.peers = {
"lab.example.com" = { }; # CA-trusted (Let's Encrypt etc.)
"edge.corp" = { certFingerprint = "sha256:…"; }; # self-signed TLS, c0re peer checks only
"mesh.internal" = { caCert = ./mesh-ca.pem; }; # self-signed, trusted for matrix federation
services.hyperhive.swarm.hives = {
pr1ma = { domain = "pr1ma.example.com"; }; # this host, per hiveName
lab = { domain = "lab.example.com"; }; # CA-trusted (Let's Encrypt etc.)
edge = { domain = "edge.corp"; certFingerprint = "sha256:…"; }; # self-signed leaf, pinned
};
```
The attrset key is the peer's DNS domain. Two independent, optional
trust knobs — pick by what you need to trust:
One attrset describing **every** hive in the swarm, **including this
one**, keyed by that hive's `hiveName`. It is meant to be *identical on
every host* — write it once, share it, and each host reads it correctly
because `services.hyperhive.hiveName` says which entry is itself.
- **`certFingerprint`** (`"sha256:…"`) — pin the peer's TLS _leaf_
fingerprint. Scopes **only** to hive-c0re's own peer HTTPS checks
(the P33RS dashboard links + agent peer discovery below). It is
**not** consulted by matrix federation — tuwunel validates a peer's
federation certificate against the system CA bundle independently
(see _Matrix federation_ below), so a fingerprint pin does nothing
for a self-signed matrix cert.
- **`caCert`** (path to the peer's trust bundle / root CA PEM, i.e. what
its `hivectl peer-config` told you to copy) — embeds that CA (at
build time, into the nix store — no runtime file on the host) and
trusts it **everywhere the hive's own internal CA is**: it rides
alongside `hive-ca.pem` in every agent's
`security.pki.certificateFiles` (via the meta-flake renderer) **and**
in the matrix container's trust bundle, so tuwunel validates the
peer's _federation_ TLS when it chains to that CA. Trust stays
**inside the hive** (agents + the matrix container), never the host
system trust store. **This is the knob that unblocks federation with
a self-signed peer hive** — use it instead of `certFingerprint` when
you control the peer's CA. (It does not affect hive-c0re's own peer
HTTPS checks — those stay on `certFingerprint` / the system bundle.)
- **Both omitted** — the stock system CA bundle validates the peer
(correct for Let's Encrypt / any publicly-trusted peer).
Empty (the default) means this host isn't in a swarm. Once non-empty it
**must** contain an entry for `hiveName`; eval fails naming the missing
hive. That assertion is load-bearing rather than pedantic — "my peers"
is derived as *everything that isn't me*, so a directory that doesn't
contain you derives every hive as a peer and you peer with yourself.
`domain` is required per entry and deliberately undefaulted: it is
conventionally `<name>.<swarm.domain>`, but a wrong domain that
evaluates cleanly points at a real machine that isn't the one you meant.
**`certFingerprint`** (`"sha256:…"`, optional) pins that hive's TLS
_leaf_. Scopes **only** to hive-c0re's own peer HTTPS checks (the P33RS
dashboard links + agent peer discovery below); matrix federation never
consults it. Omit it for any hive under the swarm root CA or a public
CA — which is the normal case.
> **There is no per-hive CA field.** Trust inside a swarm comes from the
> swarm root ([`ca.md`](ca.md)): every hive chains to it, so one anchor
> replaces the O(n²) pinning. What that genuinely drops is trusting a
> hive whose root this swarm does *not* own — another swarm's, or one
> keeping its own CA. That is a cross-swarm problem and wants a
> mechanism designed for it, not a field that happened to work.
### Fingerprint format
@ -163,27 +166,33 @@ environment and forwarded to agent containers.
`.well-known/matrix/server` delegation, which the gateway serves).
Federation validates the peer's TLS certificate against the matrix
**container's** trust bundle — independently of `certFingerprint`,
which it never consults. A self-signed gateway certificate therefore
won't federate unless the peer's root CA is trusted: set `caCert`
above (embeds the peer CA into the matrix container's trust bundle),
or give the peers CA-issued certs (ACME). See `docs/matrix.md` for
federation firewall + TLS requirements.
which it never consults.
## Bilateral setup
⚠️ **That container currently trusts no swarm-internal CA**, so a
self-signed gateway certificate does not federate. The swarm root
can't simply be listed there: `security.pki.certificateFiles` is
read when the system is *built*, and the root is a runtime file (its
key must never enter the store), so there is no build-time name for
it. Bridging that needs a runtime mechanism and is tracked as its own
issue. Until then, federation needs CA-issued certs (ACME). See
`docs/matrix.md` for federation firewall + TLS requirements.
Each hive must declare the other. If hive A lists hive B as a peer,
B must also list A for agents on B to see A in their peer list:
## One directory, not a bilateral declaration
Both hives hold the **same** `hives` attrset; neither declares the
other. What differs between the two hosts is only `hiveName`:
```
# hive A (pr1ma.example.com)
services.hyperhive.swarm.peers."edge.corp" = { };
# hive B (edge.corp)
services.hyperhive.swarm.peers."pr1ma.example.com" = { certFingerprint = "sha256:…"; };
# hive A # hive B
hiveName = "pr1ma"; hiveName = "edge";
swarm.hives = { … }; swarm.hives = { … }; # byte-identical
```
Mixed trust is fine: A trusts B via CA bundle (no fingerprint), B
pins A's self-signed cert.
That is the point of the shape, and it removes a class of bug rather
than saving typing: a per-host peer list let two hosts hold *different*
facts about the same third hive — a stale endpoint, a rotated
fingerprint — with nothing to detect the disagreement. One entry per
hive makes it unrepresentable.
## WireGuard inter-hive mesh (optional)
@ -213,11 +222,22 @@ services.hyperhive = {
listenPort = 51820; # optional, default 51820
};
swarm.peers."edge.corp" = {
certFingerprint = "sha256:…"; # TLS trust (unchanged)
wireguardPublicKey = "base64key="; # peer's wg pubkey
wireguardEndpoint = "203.0.113.42:51820"; # peer's public IP:port
wireguardAddress = "10.100.0.2/32"; # peer's mesh IP
# The same `hives` attrset both hosts hold — mesh fields included,
# since "where this hive can be dialled" is a fact about that hive.
swarm.hives = {
pr1ma = {
domain = "pr1ma.example.com";
wireguardPublicKey = "base64keyA=";
wireguardEndpoint = "198.51.100.1:51820";
wireguardAddress = "10.100.0.1/32";
};
edge = {
domain = "edge.corp";
certFingerprint = "sha256:…"; # TLS trust (unchanged)
wireguardPublicKey = "base64keyB=";
wireguardEndpoint = "203.0.113.42:51820";
wireguardAddress = "10.100.0.2/32";
};
};
};
@ -229,11 +249,7 @@ services.hyperhive = {
address = "10.100.0.2/24";
};
swarm.peers."pr1ma.example.com" = {
wireguardPublicKey = "base64key="; # hive A's wg pubkey
wireguardEndpoint = "198.51.100.1:51820";
wireguardAddress = "10.100.0.1/32";
};
swarm.hives = { /* … identical to hive A's … */ };
};
```

32
docs/swarm/ca.md
View file

@ -126,3 +126,35 @@ named in the message. That is a decision, and it is recorded as one.
A hive with **no** root configured at all is not affected by any of
this: it self-signs exactly as it always has.
## Distributing the root
The root **key** is a runtime file for an obvious reason: the nix store
is world-readable and content-addressed, so a key committed to a flake
is a key *published* to every consumer of that flake.
The root **certificate** is a runtime file as a *consequence* — it lives
beside the key under `swarm.ca.stateDir` — and that has a cost worth
naming, because it is not obvious and it bites at a distance:
> **Nothing whose trust store is assembled at build time can reference
> the swarm root.** `security.pki.certificateFiles` is read inside the
> derivation; the root does not exist there.
Two consumers, and only one of them is fine:
- **Agents are covered.** `hive-tls.nix` folds the root into this hive's
`trust-bundle.pem`, hive-c0re receives that path as
`HIVE_TLS_CA_PATH`, and the meta-flake renderer embeds that one file
next to every agent's flake. The bundle is the runtime-to-build-time
bridge.
- **The Matrix container is not.** It has no equivalent bridge, so it
trusts no swarm-internal CA and federation with a self-signed peer
does not validate. Giving it the root needs a runtime mechanism —
bind-mount plus a bundle assembled at unit start, appending to the
system bundle rather than replacing it — which is tracked separately.
To put the root on another host, copy the certificate to the same path
there (`scp <stateDir>/root.pem <host>:<stateDir>/root.pem`). One anchor
per host, not one per peer: a hive joining later needs no edit on the
hives already running, which is the whole point of the hierarchy.

2
docs/tools/hivectl-cli.md
View file

@ -761,7 +761,7 @@ Print the nix to add a peer hive to the mesh
###### **Arguments:**
* `<DOMAIN>` — Peer hive's DNS domain (the `swarm.peers` attrset key)
* `<DOMAIN>` — Peer hive's DNS domain (its `swarm.hives` entry's `domain`)
###### **Options:**

17
docs/web-ui/dashboard.md
View file

@ -607,10 +607,10 @@ prices. Models not covered fall back to hive-c0re's built-in estimate.
## P33RS tab
Peer hives in this swarm. The tab is hidden when the
`state.peer_hives` array from `/api/state` is empty (i.e. no
`services.hyperhive.swarm.peers` are configured). When at least
one peer is present the `hidden` attribute is removed and the tab
becomes active.
`state.peer_hives` array from `/api/state` is empty — i.e. when
`services.hyperhive.swarm.hives` holds no hive other than this one.
When at least one peer is present the `hidden` attribute is removed
and the tab becomes active.
**P33R H1V3S** — each peer renders as a card row: a hexagon icon
(`⬡`), the peer's DNS domain as the primary name, and the peer
@ -619,15 +619,16 @@ URL opens the peer hive's dashboard in a new tab.
### Backend wiring
The host daemon reads `services.hyperhive.swarm.peers` from the
nix config (an attrset keyed by peer domain), serialises each
entry as `{ name, url }` into `state.peer_hives: Vec<PeerHiveView>`,
The host daemon reads `services.hyperhive.swarm.peerHives` from the
nix config (the `swarm.hives` directory minus this hive), serialises
each entry as `{ name, url }` into `state.peer_hives: Vec<PeerHiveView>`,
and includes the field in the `/api/state` snapshot. `tabs.js`
reads `state.peer_hives` on every `refreshState` call and calls
`swarm.js::renderPeerHives(peers)`, which rebuilds the `#peers-section`
div from scratch.
The `name` field is the peer's DNS domain (the attrset key); `url`
The `name` field is the peer's DNS domain (its entry's `domain`, not
the attrset key — the key is the hive's name); `url`
is `https://{domain}/`. Both are derived from the env var
`HYPERHIVE_PEERS` (a JSON array of `{ domain, cert_fingerprint }`
objects) that the nix module writes into the c0re container

11
hive-c0re/src/dashboard/state_snapshot.rs
View file

@ -122,7 +122,8 @@ pub(super) struct StateSnapshot {
swarm_name: Option<String>,
/// Peer hives in the same swarm. Parsed from `HYPERHIVE_PEERS`
/// (JSON array of `{domain,cert_fingerprint}` objects, emitted by
/// the c0re NixOS module from `services.hyperhive.swarm.peers`).
/// the c0re NixOS module from `services.hyperhive.swarm.peerHives`
/// — the swarm's `hives` directory minus this hive).
/// Empty on single-hive deploys. Feeds the P33RS dashboard tab.
peer_hives: Vec<PeerHiveView>,
/// Server-level warnings for the dashboard's top-of-page banner
@ -162,9 +163,9 @@ async fn infra_container_views() -> Vec<InfraContainerView> {
/// One peer hive for the P33RS dashboard tab. Derived from
/// `HYPERHIVE_PEERS` env; `url` is the peer's HTTPS dashboard root.
/// `cert_fingerprint` is `Some("sha256:<hex64>")` when the peer uses a
/// self-signed cert and the operator pinned its fingerprint in
/// `services.hyperhive.swarm.peers`.
/// `cert_fingerprint` is `Some("sha256:<hex64>")` when the operator
/// pinned the peer's leaf in `services.hyperhive.swarm.hives`, which a
/// hive under the swarm root CA does not need.
#[derive(Serialize)]
struct PeerHiveView {
name: String,
@ -450,7 +451,7 @@ pub(super) async fn api_state(
/// Parse `HYPERHIVE_PEERS` env var into dashboard-ready `PeerHiveView`
/// entries. The env var is a JSON array of `{domain, cert_fingerprint}`
/// objects emitted by the c0re NixOS module from
/// `services.hyperhive.swarm.peers`. Each entry becomes
/// `services.hyperhive.swarm.peerHives`. Each entry becomes
/// `{ name: domain, url: "https://domain/" }` for the P33RS tab.
/// Returns empty vec when unset (single-hive deploy).
fn parse_peer_hives() -> Vec<PeerHiveView> {

152
hive-c0re/src/meta.rs
View file

@ -108,11 +108,10 @@ pub async fn sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<()> {
let on_disk = std::fs::read_to_string(&flake_path).unwrap_or_default();
let initial = !dir.join(".git").exists();
// Embedded-CA list (self-signed hive CA + peer CAs): keep the
// `./hive-ca.pem` / `./peer-ca-<N>.pem` files at the meta root in
// lockstep with their host sources so the build-time `certificateFiles`
// list render_flake emits always resolves. Empty when neither a
// self-signed hive CA nor any peer CA is configured.
// Embedded-CA list: keep the `./hive-ca.pem` file at the meta root in
// lockstep with its host source so the build-time `certificateFiles`
// list render_flake emits always resolves. Empty when no self-signed
// hive CA is configured.
let (ca_files, ca_changed) = ca_embed_state(&dir);
// Skip only when both the flake AND the embedded CA are unchanged — a
@ -217,6 +216,7 @@ pub async fn sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<()> {
"flake.nix" => Some("flake"),
"flake.lock" => Some("lock"),
"hive-ca.pem" => Some("hive-ca"),
// Only ever a REMOVAL now; see `is_embedded_ca_name`.
f if f.starts_with("peer-ca-") && has_pem_ext(f) => Some("peer-ca"),
"topology.json" => Some("topology"),
"capabilities.json" => Some("capabilities"),
@ -809,10 +809,10 @@ fn forwarded_env_vars() -> Vec<(&'static str, String)> {
.collect()
}
/// Filename the hive's own self-signed CA cert is embedded under at the
/// meta-flake root. One entry of the embedded-CA list `render_flake`
/// references in `security.pki.certificateFiles` (see `embedded_ca_files`);
/// peer CAs sit alongside it as `peer-ca-<N>.pem`.
/// Filename the hive's own trust anchors are embedded under at the
/// meta-flake root — the only entry of the embedded-CA list
/// `render_flake` references in `security.pki.certificateFiles` (see
/// `embedded_ca_files`).
const HIVE_CA_FILE: &str = "hive-ca.pem";
/// Host path of the hive's TLS trust anchors, when self-signed TLS is
@ -833,23 +833,6 @@ fn hive_ca_source() -> Option<String> {
Some(path)
}
/// Host paths of peer-hive root CA certificates, from `HIVE_PEER_CA_PATHS`
/// (colon-separated; set by hive-c0re.nix from `swarm.peers.<d>.caCert`).
/// Each is embedded alongside the hive CA so a peer's CA is trusted
/// everywhere the hive's own internal CA is — i.e. by every agent. Empty
/// segments and paths that don't resolve to a file are dropped, so we
/// never reference a `certificateFiles` entry we couldn't embed.
fn peer_ca_sources() -> Vec<String> {
let Ok(raw) = std::env::var("HIVE_PEER_CA_PATHS") else {
return Vec::new();
};
raw.split(':')
.map(str::trim)
.filter(|p| !p.is_empty() && std::path::Path::new(p).is_file())
.map(ToOwned::to_owned)
.collect()
}
/// Hive-wide OTEL config injected into every agent's build, read off
/// hive-c0re's own unit env (set from `services.hyperhive.otel.*` in
/// `nix/modules/hive-c0re.nix`). A present, non-empty
@ -906,9 +889,16 @@ fn otel_config() -> Option<OtelConfig> {
}
/// The ordered set of CA certs embedded next to the meta flake, as
/// `(filename, host_source_path)`. The self-signed hive CA (when active)
/// is `hive-ca.pem`; each peer CA is `peer-ca-<N>.pem` in declaration
/// order. `render_flake` emits exactly these filenames into
/// `(filename, host_source_path)` — now just the hive's own trust
/// anchors as `hive-ca.pem`, when self-signed TLS is active.
///
/// That single file already carries the swarm root (`hive-tls.nix`
/// writes the hive CA *and* the root it is issued under into the trust
/// bundle), so every hive under the swarm root validates from it. Which
/// is why the per-peer CAs this used to append are gone: they said the
/// same thing once per peer.
///
/// `render_flake` emits exactly these filenames into
/// `security.pki.certificateFiles` and `sync_agents` materialises them,
/// so the rendered reference and the embedded files always agree.
fn embedded_ca_files() -> Vec<(String, String)> {
@ -916,9 +906,6 @@ fn embedded_ca_files() -> Vec<(String, String)> {
if let Some(p) = hive_ca_source() {
out.push((HIVE_CA_FILE.to_owned(), p));
}
for (i, p) in peer_ca_sources().into_iter().enumerate() {
out.push((format!("peer-ca-{i}.pem"), p));
}
out
}
@ -949,9 +936,16 @@ fn materialise_ca_files(dir: &Path, ca_files: &[(String, String)]) -> Result<Vec
Ok(touched)
}
/// True for a filename `embedded_ca_files` can produce — the hive CA or
/// a `peer-ca-<N>.pem`. Lets `sync_agents` find stale CA files to clean
/// up (a CA dropped from config) without touching unrelated meta files.
/// True for a filename the embedded-CA machinery owns. Lets
/// `sync_agents` find stale CA files to clean up (a CA dropped from
/// config) without touching unrelated meta files.
///
/// ⚠️ Still matches `peer-ca-<N>.pem`, which `embedded_ca_files` no
/// longer produces — deliberately. Cleanup is driven by
/// "recognised but not desired", so this arm is exactly what removes the
/// peer CAs a hive embedded before per-peer pinning was replaced by the
/// swarm root. Drop it and those files are orphaned at every meta root
/// forever, referenced by nothing and cleaned by nobody.
fn is_embedded_ca_name(name: &str) -> bool {
name == HIVE_CA_FILE || (name.starts_with("peer-ca-") && has_pem_ext(name))
}
@ -967,13 +961,12 @@ fn has_pem_ext(name: &str) -> bool {
/// Embedded-CA state for the meta repo: `(desired_files, changed)`.
/// `desired_files` is `(filename, contents)` for every CA that should sit
/// next to flake.nix (the hive CA + each peer CA). `changed` is true when
/// the on-disk set differs in any way — a file's contents changed, a new
/// CA appeared, or a previously-embedded CA (`hive-ca.pem` /
/// `peer-ca-*.pem`) is no longer wanted (stale, to be removed). Drives
/// both the re-commit decision and the materialise/cleanup in
/// `sync_agents`, so a CA rotation or a peer-set change re-commits even
/// when the flake itself is byte-identical.
/// next to flake.nix. `changed` is true when the on-disk set differs in
/// any way — the file's contents changed, a CA appeared, or a
/// previously-embedded CA (`hive-ca.pem` / a legacy `peer-ca-*.pem`) is
/// no longer wanted (stale, to be removed). Drives both the re-commit
/// decision and the materialise/cleanup in `sync_agents`, so a CA
/// rotation re-commits even when the flake itself is byte-identical.
fn ca_embed_state(dir: &std::path::Path) -> (Vec<(String, String)>, bool) {
let desired: Vec<(String, String)> = embedded_ca_files()
.into_iter()
@ -1170,16 +1163,16 @@ where
if let Some(path) = claude_code_path {
let _ = writeln!(out, " hyperhive.claudeCodePath = \"{path}\";");
}
// CA trust: embed every hive-trusted CA so each agent validates them at
// build time. The list is the hive's own self-signed CA (when active)
// plus every peer-hive root CA (`swarm.peers.<d>.caCert`) — a peer CA is
// trusted everywhere the hive's own internal CA is. `certificateFiles` is
// build-time, so the certs travel with the flake source: `sync_agents`
// writes `./hive-ca.pem` + `./peer-ca-<N>.pem` next to flake.nix and
// stages them. Only public CA certs are embedded; no private key ever
// leaves the host. The filename list matches `sync_agents` exactly (both
// derive it from `embedded_ca_files`), so we never reference a file we
// didn't embed; emitted only when the list is non-empty.
// CA trust: embed the hive's trust anchors so each agent validates
// them at build time — the hive's own self-signed CA when active,
// together with the swarm root it is issued under (one file; see
// `hive_ca_source`). `certificateFiles` is build-time, so the certs
// travel with the flake source: `sync_agents` writes `./hive-ca.pem`
// next to flake.nix and stages it. Only public CA certs are embedded;
// no private key ever leaves the host. The filename list matches
// `sync_agents` exactly (both derive it from `embedded_ca_files`), so
// we never reference a file we didn't embed; emitted only when the
// list is non-empty.
let ca_refs: Vec<String> = embedded_ca_files()
.into_iter()
.map(|(name, _)| format!("./{name}"))
@ -2169,20 +2162,16 @@ mod tests {
)
};
// Two peer-hive CA temp files for the list cases.
let peer0 = std::env::temp_dir().join(format!("peer-ca0-test-{}.pem", std::process::id()));
let peer1 = std::env::temp_dir().join(format!("peer-ca1-test-{}.pem", std::process::id()));
// A leftover peer-CA file + the env var that used to name it. Both
// must now be inert: the swarm root rides in the hive's own trust
// bundle, so nothing per-peer is embedded any more.
let stale_peer =
std::env::temp_dir().join(format!("peer-ca0-test-{}.pem", std::process::id()));
std::fs::write(
&peer0,
&stale_peer,
"-----BEGIN CERTIFICATE-----\np0\n-----END CERTIFICATE-----\n",
)
.expect("write peer CA 0");
std::fs::write(
&peer1,
"-----BEGIN CERTIFICATE-----\np1\n-----END CERTIFICATE-----\n",
)
.expect("write peer CA 1");
let peer_paths = format!("{}:{}", peer0.display(), peer1.display());
.expect("write stale peer CA");
// All env mutations are serialised within this one test (no other
// test asserts on these vars), restored before returning.
@ -2191,38 +2180,47 @@ mod tests {
std::env::set_var("HIVE_TLS_CA_PATH", &ca_file);
}
let with_ca = render();
// Hive CA + peer CAs: the list carries all three, hive CA first.
unsafe {
std::env::set_var("HIVE_PEER_CA_PATHS", &peer_paths);
std::env::set_var("HIVE_PEER_CA_PATHS", stale_peer.display().to_string());
}
let with_peers = render();
// Peers only (this hive on ACME, federating with self-signed peers).
let with_stale_peer_env = render();
// The stale var alone, with no hive CA: must produce nothing.
unsafe {
std::env::remove_var("HIVE_TLS_CA_PATH");
}
let peers_only = render();
let stale_peer_env_only = render();
unsafe {
std::env::remove_var("HIVE_PEER_CA_PATHS");
}
let without_ca = render();
let _ = std::fs::remove_file(&ca_file);
let _ = std::fs::remove_file(&peer0);
let _ = std::fs::remove_file(&peer1);
let _ = std::fs::remove_file(&stale_peer);
assert!(
with_ca.contains("security.pki.certificateFiles = [ ./hive-ca.pem ]"),
"CA cert must be wired into certificateFiles when signalled:\n{with_ca}"
);
// The regression guard, scoped to the CA list rather than the whole
// render. Comparing the two flakes wholesale looks stronger and is
// actually FLAKY: `cargo test` runs these in parallel threads and
// sibling tests mutate process env (OTEL, forge URLs) between the
// two `render()` calls, so a whole-output equality assertion fails
// on changes that have nothing to do with this test.
assert!(
with_peers.contains(
"security.pki.certificateFiles = [ ./hive-ca.pem ./peer-ca-0.pem ./peer-ca-1.pem ]"
),
"hive CA + peer CAs must all appear in the certificateFiles list:\n{with_peers}"
with_stale_peer_env.contains("security.pki.certificateFiles = [ ./hive-ca.pem ]"),
"HIVE_PEER_CA_PATHS must not change the CA list — per-peer CA \
embedding was replaced by the swarm root inside the hive's own \
trust bundle:\n{with_stale_peer_env}"
);
assert!(
peers_only
.contains("security.pki.certificateFiles = [ ./peer-ca-0.pem ./peer-ca-1.pem ]"),
"peer CAs must be trusted even when this hive has no self-signed CA:\n{peers_only}"
!with_stale_peer_env.contains("peer-ca-"),
"no peer-ca-<N>.pem may be embedded, however HIVE_PEER_CA_PATHS \
is set:\n{with_stale_peer_env}"
);
assert!(
!stale_peer_env_only.contains("security.pki.certificateFiles"),
"a stale HIVE_PEER_CA_PATHS must not resurrect a certificateFiles \
reference on its own:\n{stale_peer_env_only}"
);
assert!(
!without_ca.contains("security.pki.certificateFiles"),

2
hive-c0re/src/server.rs
View file

@ -1030,7 +1030,7 @@ fn is_broad_scope(scope: &LifecycleScope) -> bool {
/// convention (gateway terminates TLS, so https).
fn hive_urls() -> hive_host_sock::HiveUrls {
// Treat an empty env value as unset everywhere — an empty domain would
// otherwise render `swarm.peers."" = …` (invalid nix) and `https:///`.
// otherwise render `swarm.hives."" = …` (invalid nix) and `https:///`.
let env = |k: &str| std::env::var(k).ok().filter(|v| !v.is_empty());
let domain = env("HYPERHIVE_HIVE_DOMAIN");
hive_host_sock::HiveUrls {

2
hivectl/src/cli.rs
View file

@ -425,7 +425,7 @@ pub enum WgCmd {
},
/// Print the nix to add a peer hive to the mesh.
Peer {
/// Peer hive's DNS domain (the `swarm.peers` attrset key).
/// Peer hive's DNS domain (its `swarm.hives` entry's `domain`)
domain: String,
/// Peer's WireGuard public key (from its `hivectl wg init`).
#[arg(long)]

65
hivectl/src/wg.rs
View file

@ -16,9 +16,10 @@ const WG_INTERFACE: &str = "wg-hive";
/// Host path of this hive's TLS trust bundle (matches the
/// `services.hyperhive.tls.stateDir` default in hive-tls.nix). Its
/// existence means the gateway serves a self-signed, hive-CA-signed leaf,
/// so a federating peer needs this via `swarm.peers.<d>.caCert`. Absent
/// = ACME / operator cert (trusted by the default CA bundle, no `caCert`).
/// existence means the gateway serves a self-signed, hive-CA-signed
/// leaf, so a federating peer needs an anchor for it — which is now the
/// swarm root, not a per-hive CA. Absent = ACME / operator cert, trusted
/// by the default CA bundle with nothing to distribute.
///
/// The bundle rather than `ca.pem`: the hive CA is an intermediate under
/// the swarm root, so `ca.pem` alone is not a chain a peer can validate
@ -28,6 +29,26 @@ const WG_INTERFACE: &str = "wg-hive";
/// path with no mode to branch on.
const HIVE_TLS_TRUST_BUNDLE_PATH: &str = "/var/lib/hive-tls/trust-bundle.pem";
/// Host path of the swarm root CA cert (matches the
/// `services.hyperhive.swarm.ca.stateDir` default in swarm-ca.nix). The
/// anchor a whole swarm shares: install it once per host and every
/// present *and future* hive under it validates, which is what replaced
/// the per-hive CA pinning.
const SWARM_CA_ROOT_PATH: &str = "/var/lib/swarm-ca/root.pem";
/// Attrset key for a hive in `services.hyperhive.swarm.hives`, derived
/// from its domain's first DNS label.
///
/// A hive occupies `<hiveName>.<swarm.domain>`, so the first label *is*
/// the hive name in any deployment that hasn't overridden `domain` by
/// hand. Where it has, a wrong key fails loudly rather than quietly: on
/// that hive's own host the `hives.<hiveName>` assertion fires, because
/// the directory is supposed to be the same attrset everywhere. The
/// snippets below say so rather than presenting the guess as fact.
fn hive_key(domain: &str) -> &str {
domain.split('.').next().unwrap_or(domain)
}
/// Best-effort query for this hive's domain from the running daemon
/// (`HostRequest::Urls`, which reads `HYPERHIVE_HIVE_DOMAIN` from c0re's
/// service env). `None` when the daemon is unreachable or the domain is
@ -143,25 +164,29 @@ fn wg_pubkey(privkey: &[u8]) -> Result<String> {
/// Pure output (no fallible work), so it returns `()`; the dispatch arm
/// wraps it in `Ok` to match the sibling verbs.
pub(crate) fn wg_peer(domain: &str, pubkey: &str, address: &str, endpoint: Option<&str>) {
let key = hive_key(domain);
println!("Add to this hive's NixOS config:");
println!(" services.hyperhive.swarm.peers.\"{domain}\" = {{");
println!(" # `hives` describes the whole swarm and is meant to be the same");
println!(" # attrset on every host — add this entry to all of them.");
println!(" services.hyperhive.swarm.hives.\"{key}\" = {{");
println!(" domain = \"{domain}\";");
println!(" wireguardPublicKey = \"{pubkey}\";");
println!(" wireguardAddress = \"{address}\";");
if let Some(ep) = endpoint {
println!(" wireguardEndpoint = \"{ep}\";");
}
println!(" }};");
println!(" # the key must be that hive's services.hyperhive.hiveName");
}
/// `peer-config` — print the `swarm.peers."<domain>"` block a peer
/// operator pastes to federate with THIS hive, plus a `cp` line for the
/// CA when this hive is self-signed. Reads local state only (the TLS CA
/// cert presence + the wg key); prints, never mutates.
/// `peer-config` — print the `swarm.hives."<name>"` block a peer
/// operator pastes to federate with THIS hive, plus the swarm-root
/// install step when this hive serves a self-signed chain. Reads local
/// state only (the TLS trust bundle's presence + the wg key); prints,
/// never mutates.
pub(crate) fn peer_config(domain: &str, wg_address: Option<&str>, wg_endpoint: Option<&str>) {
let self_signed = Path::new(HIVE_TLS_TRUST_BUNDLE_PATH).exists();
// CA filename derived from the first DNS label so multiple peers'
// certs don't collide in the operator's config dir.
let ca_file = format!("{}-ca.pem", domain.split('.').next().unwrap_or("peer"));
let key = hive_key(domain);
// WireGuard public key, when this hive has a mesh key. Best-effort:
// a missing key or absent `wg` binary just omits the mesh lines.
let wg_pub = std::fs::read(WG_KEY_PATH)
@ -169,17 +194,19 @@ pub(crate) fn peer_config(domain: &str, wg_address: Option<&str>, wg_endpoint: O
.and_then(|k| wg_pubkey(&k).ok());
if self_signed {
println!("# 1. copy this hive's CA cert next to the peer's config:");
println!("cp {HIVE_TLS_TRUST_BUNDLE_PATH} ./{ca_file}");
// One anchor for the whole swarm, installed once per host — not
// a file per peer. That is the point of the hierarchy: a hive
// joining later needs no edit on the hives already running.
println!("# 1. install the SWARM ROOT on the peer host (once, not per hive):");
println!("scp {SWARM_CA_ROOT_PATH} <peer-host>:{SWARM_CA_ROOT_PATH}");
println!("# (skip if that host already has the swarm root)");
println!();
println!("# 2. paste into the peer hive's NixOS config:");
} else {
println!("# paste into the peer hive's NixOS config:");
}
println!("services.hyperhive.swarm.peers.\"{domain}\" = {{");
if self_signed {
println!(" caCert = ./{ca_file};");
}
println!("services.hyperhive.swarm.hives.\"{key}\" = {{");
println!(" domain = \"{domain}\";");
if let Some(pk) = &wg_pub {
println!(" wireguardPublicKey = \"{pk}\";");
}
@ -190,9 +217,11 @@ pub(crate) fn peer_config(domain: &str, wg_address: Option<&str>, wg_endpoint: O
println!(" wireguardEndpoint = \"{ep}\";");
}
println!("}};");
println!("# the key must be this hive's services.hyperhive.hiveName, and the");
println!("# same entry belongs in every hive's config — `hives` is the swarm.");
if !self_signed {
println!(
"# (this hive's cert chains to a public CA — no `caCert` needed; \
"# (this hive's cert chains to a public CA — nothing to install; \
it's trusted by the default bundle.)"
);
}

1
nix/host-modules/default.nix
View file

@ -28,6 +28,7 @@
./swarm-snapshot-store.nix
./swarm-wireguard.nix
./swarm.nix
./swarm-peers-removed.nix
./swarm-required-services.nix
];
}

36
nix/host-modules/hive-c0re/environment.nix
View file

@ -187,41 +187,25 @@ in
in
"${s.address}:${toString s.port}";
}
// lib.optionalAttrs (config.services.hyperhive.swarm.peers != { }) {
// lib.optionalAttrs (config.services.hyperhive.swarm.peerHives != { }) {
# Peer hives serialised as a JSON array of {domain, cert_fingerprint,
# wireguard_address?} objects. Consumed by hive-agent::identity::peers()
# + the dashboard's peer_hives StateSnapshot field (P33RS tab). Domain
# is the attrset key; cert_fingerprint is null for CA-trusted peers;
# wireguard_address is omitted when not part of the mesh.
# + the dashboard's peer_hives StateSnapshot field (P33RS tab).
# `cert_fingerprint` is null for CA-trusted hives; `wireguard_address`
# is omitted when not part of the mesh.
#
# Reads `peerHives` — `swarm.hives` minus this hive — so the "not me"
# filter is the one derived in ../swarm.nix rather than a fifth copy.
HYPERHIVE_PEERS = builtins.toJSON (
lib.mapAttrsToList (
domain: p:
_name: p:
{
inherit domain;
inherit (p) domain;
cert_fingerprint = p.certFingerprint;
}
// lib.optionalAttrs (p.wireguardAddress != null) {
wireguard_address = p.wireguardAddress;
}
) config.services.hyperhive.swarm.peers
) config.services.hyperhive.swarm.peerHives
);
}
//
lib.optionalAttrs
(lib.any (p: p.caCert != null) (lib.attrValues config.services.hyperhive.swarm.peers))
{
# Peer-hive root CA file paths (colon-joined), one per peer that
# declares `swarm.peers.<domain>.caCert`. hive-c0re's meta-flake
# renderer (meta.rs) embeds each next to every agent's flake and
# adds it to `security.pki.certificateFiles`, so a peer CA is
# trusted everywhere the hive's own internal CA (`hive-ca.pem`)
# is — i.e. by every agent. The matrix container trusts the same
# CAs separately for federation TLS. The `caCert` files are
# copied into the nix store at build, so these are store paths —
# nothing mutable lives on the host.
HIVE_PEER_CA_PATHS = lib.concatStringsSep ":" (
lib.filter (c: c != null) (
lib.mapAttrsToList (_domain: p: p.caCert) config.services.hyperhive.swarm.peers
)
);
}

8
nix/host-modules/hive-gateway/options.nix
View file

@ -147,8 +147,8 @@ in
config.security.acme.certs."example.com".directory;
```
When using an external CA cert, peer hives can declare this
hive in `services.hyperhive.swarm.peers` without
When using an external CA cert, other hives can declare this
one in `services.hyperhive.swarm.hives` without
`certFingerprint` — the standard CA bundle validates.
Mutual exclusion with `tls.acme.enable` — set one or the other,
@ -207,8 +207,8 @@ in
};
```
After enabling, peer hives can omit `certFingerprint` in
`swarm.peers` — Let's Encrypt certs are CA-trusted
After enabling, this hive's entry in `swarm.hives` can omit
`certFingerprint` — Let's Encrypt certs are CA-trusted
by default.
'';
};

28
nix/host-modules/hive-matrix.nix
View file

@ -395,20 +395,20 @@ in
# all filtering; never run one in here.
networking.firewall.enable = false;
# Peer-hive root CAs (`swarm.peers.<domain>.caCert`) added to THIS
# container's trust bundle so tuwunel validates *federation* TLS
# from a self-signed peer hive (it checks the peer's federation
# cert against its trust bundle). Peer CAs are trusted everywhere
# the hive's own internal CA is — agents get them via the
# meta-flake renderer (`HIVE_PEER_CA_PATHS` → each agent's
# `security.pki.certificateFiles`); this block is the matrix
# container's copy, since the host `security.pki` store doesn't
# cross the container boundary. They are never installed in the
# HOST trust store. Null entries (CA-bundle / fingerprint-pinned
# peers) drop out.
security.pki.certificateFiles = lib.filter (c: c != null) (
lib.mapAttrsToList (_domain: p: p.caCert) config.services.hyperhive.swarm.peers
);
# ⚠️ This container trusts no swarm-internal CA. It used to take
# per-hive root CAs, hand-pinned as nix paths, so tuwunel could
# validate *federation* TLS from a self-signed peer hive; that
# field is gone, and the swarm root that replaces it cannot be
# substituted here. `security.pki.certificateFiles` is read when
# the system is BUILT, and the swarm root is a runtime file
# (`swarm.ca.stateDir`) precisely because its key must never
# reach the store — so there is nothing build-time to name.
#
# Giving the container the swarm root therefore needs a runtime
# mechanism (bind-mount + a bundle assembled at start), which is
# a different shape than this line and is tracked as its own
# issue. Federation with a peer whose cert chains to the swarm
# root does not validate until then.
# tuwunel hard-fails to boot if `/etc/resolv.conf` has no
# `nameserver` line (`Failed to configure DNS resolver ... no

29
nix/host-modules/swarm-ca.nix
View file

@ -1,17 +1,19 @@
# The swarm root CA: the anchor a whole swarm shares, and the issuer of
# each hive's own CA (which is where it gets used — see ./hive-tls.nix).
#
# Why a hierarchy at all: cross-hive trust is hand-pinned today
# (`swarm.peers.<d>.caCert`), so every hive must name every peer — O(n²)
# configuration that a new hive can only join by editing all the others.
# One root makes it O(1): trust the root once and every present *and
# future* peer validates.
# Why a hierarchy at all: cross-hive trust used to be hand-pinned per
# peer, so every hive had to name every other one — O(n²) configuration
# that a new hive could only join by editing all the others. One root
# makes it O(1): trust the root once and every present *and future* peer
# validates. The per-peer CA field is gone precisely because this
# replaced it.
#
# Two provisioning modes, ONE structure — what differs is who puts the
# artifacts on disk, never what the artifacts are:
#
# - autoconfigured (the default while the hive declares no peers): the
# unit below generates the root here on first boot.
# - autoconfigured (`services.hyperhive.enableAllLocalDefaults`, or
# `autoConfigure` set directly): the unit below generates the root
# here on first boot.
# - operator-provided (multi-host): the operator installs the root cert
# — and, on a host that does not hold the root key, the hive CA too —
# into the state dirs, and this unit does nothing. Splitting a
@ -21,10 +23,11 @@
#
# The root KEY is the reason this is a runtime file and not a nix option:
# the store is world-readable and content-addressed, so a key committed
# to a flake is a key *published* to every consumer of that flake. The
# root CERT has the opposite property, and is already distributed at
# build time via `swarm.peers.<d>.caCert`. That asymmetry is what makes
# the layering work at all.
# to a flake is a key *published* to every consumer of that flake.
#
# ⚠️ The root CERT is therefore a runtime file too, which costs something:
# nothing whose trust store is built at build time can name it. See
# docs/swarm/ca.md § "Distributing the root".
{
lib,
config,
@ -83,8 +86,8 @@ in
default = "/var/lib/swarm-ca";
description = ''
Host directory holding the swarm root CA: `root.pem` (the
anchor, safe to distribute — this is what a peer hive is
pointed at via `swarm.peers.<d>.caCert`) and `root-key.pem`
anchor, safe to distribute — copy it to this same path on every
other host in the swarm) and `root-key.pem`
(0600, the one file that must never reach the nix store or
another host). The directory itself is 0700: nothing reads
out of it but the hive CA issuance in `hive-tls.nix`.

70
nix/host-modules/swarm-peers-removed.nix Normal file
View file

@ -0,0 +1,70 @@
# Migration shim for `services.hyperhive.swarm.peers`, replaced by the
# `swarm.hives` directory in ./swarm.nix.
#
# ⚠️ This could not be a `mkRenamedOptionModule`. `hives` is not `peers`
# under a new name, it is **`peers` ∪ {self}**: the set gains a member no
# existing config has ever written down, because a host's own identity
# lived in *other* options entirely. A rename migrates a name and a
# default can re-root a meaning; neither can conjure a new member.
#
# The whole module is self-contained and deletable — one file to remove
# when the deprecation window closes, with nothing else referring to it.
#
# Deliberately a warning rather than a hard failure, with the loudness
# coming from elsewhere: a config that set only `peers` leaves `hives`
# empty, so the self-entry assertion in ./swarm.nix fails the build
# anyway. The warning is what explains it; the assertion is what stops
# it.
{
lib,
config,
...
}:
let
peers = config.services.hyperhive.swarm.peers;
withCaCert = lib.attrNames (lib.filterAttrs (_: p: p ? caCert && p.caCert != null) peers);
in
{
options.services.hyperhive.swarm.peers = lib.mkOption {
type = lib.types.attrsOf lib.types.anything;
default = { };
visible = false;
internal = true;
description = ''
Removed — use `services.hyperhive.swarm.hives` instead, which
describes every hive in the swarm including this one. Kept only so
an existing definition produces a warning that says where to move
it, rather than an "option does not exist" error that says nothing.
'';
};
config.warnings =
lib.optional (peers != { }) ''
services.hyperhive.swarm.peers is removed and ignored. Move these
entries to services.hyperhive.swarm.hives, keyed by hive NAME
rather than domain, and add an entry for this hive itself
(services.hyperhive.hiveName) — `hives` is the swarm's full
directory, identical on every host.
services.hyperhive.swarm.hives.<name> = {
domain = "<the old attrset key>";
# certFingerprint / wireguard* carry over unchanged
};
Still set: ${lib.concatStringsSep ", " (lib.attrNames peers)}
''
++ lib.optional (withCaCert != [ ]) ''
services.hyperhive.swarm.peers.<hive>.caCert is removed and
ignored, and has no replacement in services.hyperhive.swarm.hives.
Trust inside a swarm now derives from the swarm root CA
(services.hyperhive.swarm.ca — see docs/swarm/ca.md): every hive
under it chains to it, so a per-hive CA is dead weight. What this
genuinely drops is trusting a hive whose root this swarm does NOT
own — another swarm's, or one keeping its own CA. certFingerprint
does not cover that: it pins a leaf for hive-c0re's own HTTPS
checks and does not reach Matrix federation.
Still set on: ${lib.concatStringsSep ", " withCaCert}
'';
}

7
nix/host-modules/swarm-wireguard.nix
View file

@ -108,9 +108,12 @@
networking.wireguard.interfaces =
let
wgCfg = config.services.hyperhive.swarm.wireguard;
# `peerHives` is `swarm.hives` minus this hive (../swarm.nix) —
# a mesh that included our own entry would configure a tunnel to
# ourselves.
meshPeers = lib.filterAttrs (
_: p: p.wireguardPublicKey != null && p.wireguardAddress != null
) config.services.hyperhive.swarm.peers;
) config.services.hyperhive.swarm.peerHives;
in
{
wg-hive = {
@ -118,7 +121,7 @@
listenPort = wgCfg.listenPort;
privateKeyFile = wgCfg.privateKeyFile;
peers = lib.mapAttrsToList (
_domain: p:
_name: p:
{
publicKey = p.wireguardPublicKey;
allowedIPs = [ p.wireguardAddress ];

175
nix/host-modules/swarm.nix
View file

@ -1,68 +1,73 @@
# Swarm peering: who the peer hives are. Serialised into hive-c0re's
# environment (HYPERHIVE_PEERS / HIVE_PEER_CA_PATHS — see ./hive-c0re)
# and consumed by identity.rs + the dashboard's P33RS tab.
# The swarm's directory: one entry per hive, **including this one**,
# identical on every host in the swarm. `services.hyperhive.hiveName`
# says which entry is us, and `peerHives` below derives the rest.
#
# Declaration only — this module has no `config` block. The mesh that
# uses the `wireguard*` fields below lives in ./swarm-wireguard.nix,
# because bringing up an interface is host networking rather than swarm
# Why a directory rather than a per-host peer list: every field here is
# intrinsic to the hive it describes — none of them says anything about
# the *pair*. A list where every field is intrinsic is a directory each
# host was keeping its own copy of, which is O(n²) duplication that
# deduplicates without loss. It is also a correctness gain: two hosts
# could hold different endpoints for the same third hive and nothing
# detected it. One entry per hive makes that unrepresentable.
#
# Consumed by hive-c0re's environment (HYPERHIVE_PEERS — see
# ./hive-c0re), identity.rs + the dashboard's P33RS tab, and the mesh in
# ./swarm-wireguard.nix. The mesh lives there rather than here because
# bringing up an interface is host networking rather than swarm
# bookkeeping, and a host that runs no hive still needs it.
{
lib,
config,
...
}:
let
cfg = config.services.hyperhive;
swarmCfg = cfg.swarm;
in
{
# Peer hives in the same swarm. Each entry declares a remote hive
# reachable from this host.
options.services.hyperhive.swarm.peers = lib.mkOption {
options.services.hyperhive.swarm.hives = lib.mkOption {
type = lib.types.attrsOf (
lib.types.submodule {
options = {
domain = lib.mkOption {
type = lib.types.str;
example = "lab.example.com";
description = ''
Public DNS domain this hive occupies — used for dashboard
links, peer HTTPS checks and Matrix federation discovery.
Deliberately has no default. It is conventionally
`<name>.<swarm.domain>`, but defaulting to that would let
a typo'd swarm domain produce a name that resolves
somewhere real; an eval failure naming the hive is the
better outcome.
'';
};
certFingerprint = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "sha256:b1946ac92492d2347c6235b4d2611184a3f5b6cae6c19d6e3c2f0a8e7d4c9f12";
description = ''
Expected TLS certificate fingerprint for this peer's HTTPS
endpoint. Null = trust the system CA bundle (for Let's
Encrypt peers). Set to pin a self-signed cert.
Expected TLS certificate fingerprint for this hive's HTTPS
endpoint. Null = trust the CA bundle — which for a hive
inside the swarm CA hierarchy is the normal case, since
every hive under the swarm root already chains to it.
Set it to pin a leaf that no CA in the bundle vouches for.
Format: the literal `sha256:` followed by exactly 64
hex digits (case-insensitive, no colon separators) — the
SHA-256 digest of the peer's DER-encoded leaf certificate.
SHA-256 digest of the DER-encoded leaf certificate.
Generate with `openssl x509 -noout -fingerprint -sha256`,
then strip the colons and prepend `sha256:`. A malformed
value is ignored with a warning rather than weakening
trust. See docs/swarm/README.md for the full recipe.
Scopes only to hive-c0re's own peer HTTPS checks — it does
NOT help Matrix federation (tuwunel validates against its
container trust bundle). For a self-signed peer whose root
CA you want trusted hive-wide (every agent + Matrix
federation), set `caCert` below.
'';
};
caCert = lib.mkOption {
type = lib.types.nullOr lib.types.path;
default = null;
example = "./peers/edge-ca.pem";
description = ''
Path to this peer hive's root CA certificate (PEM). When
set, the CA is embedded (at build time, into the nix store
— no runtime file on the host) and trusted **everywhere the
hive's own internal CA is**: it rides alongside `hive-ca.pem`
in each agent's `security.pki.certificateFiles` (via the
meta-flake renderer), and is added to the Matrix homeserver
container's trust bundle so tuwunel validates *federation*
TLS from a self-signed peer hive whose cert chains to it.
This is the CA-trust path that `certFingerprint`
(leaf-pinning, c0re-only) can't cover, and is what unblocks
Matrix federation with a self-signed peer hive. Trust stays
inside the hive (agents + the Matrix container), never the
host system trust store. Mutually complementary with
`certFingerprint`; set `caCert` for the federation case. See
docs/swarm/README.md.
NOT help Matrix federation, which validates against the
container's trust bundle. There is no per-hive CA field to
cover that case any more: the swarm root is the trust path
(see ./swarm-ca.nix).
'';
};
@ -71,9 +76,9 @@
default = null;
example = "base64pubkey=";
description = ''
WireGuard public key for this peer host. Required when
WireGuard public key for this hive's host. Required when
`services.hyperhive.swarm.wireguard.enable = true` and
you want this peer reachable over the mesh. Null = TLS-
you want this hive reachable over the mesh. Null = TLS-
only peering (public internet, no mesh tunnel).
'';
};
@ -83,11 +88,13 @@
default = null;
example = "203.0.113.1:51820";
description = ''
WireGuard endpoint for this peer in `host:port` form.
Required when the peer host is behind a firewall and
this host needs to initiate the tunnel. Null = this host
waits for the peer to connect (peer-initiates; peer must
have an endpoint pointing back at this host).
WireGuard endpoint for this hive in `host:port` form.
Null = this hive has no reachable endpoint, so the tunnel
is initiated from the other side.
Reads like a fact about the relationship and is not: it
says whether *this* hive can be dialled, which every other
hive in the swarm needs the same answer to.
'';
};
@ -96,36 +103,82 @@
default = null;
example = "10.100.0.2/32";
description = ''
IP address (with prefix) of the peer host on the
WireGuard mesh. Used as the `allowedIPs` for the peer's
IP address (with prefix) of this hive's host on the
WireGuard mesh. Used as the `allowedIPs` for its
WireGuard config entry and injected into `HYPERHIVE_PEERS`
so hive-c0re can route intra-swarm traffic to the mesh
address rather than the public domain. Required to include
the peer in the WireGuard mesh (peers missing this field
are silently excluded from `wg-hive`).
a hive in the mesh (entries missing this field are
silently excluded from `wg-hive`).
'';
};
};
}
);
default = { };
example = {
"lab.example.com" = {
certFingerprint = "sha256:b1946ac92492d2347c6235b4d2611184a3f5b6cae6c19d6e3c2f0a8e7d4c9f12";
pr1ma = {
domain = "pr1ma.example.com";
wireguardAddress = "10.100.0.1/32";
wireguardEndpoint = "203.0.113.1:51820";
};
edge = {
domain = "edge.example.com";
wireguardAddress = "10.100.0.2/32";
};
"edge.corp" = { };
};
description = ''
Peer hives in the same swarm. The attrset key is the peer's DNS
domain — used for dashboard links and Matrix federation discovery.
Null `certFingerprint` trusts the system CA bundle; set it to pin
a self-signed TLS cert. Add `wireguardPublicKey` + `wireguardAddress`
(and optionally `wireguardEndpoint`) to include the peer in the
WireGuard mesh when `swarm.wireguard.enable = true`.
Every hive in this swarm, keyed by `hiveName` — **including this
host's own hive**. The same attrset is meant to be identical on
every host in the swarm, so it can be written once and shared;
`services.hyperhive.hiveName` is what makes a given host read it
as "me and four others" rather than "five peers".
Empty (the default) means this host is not part of a swarm. Once
non-empty it must contain an entry for `hiveName`, which is
asserted — a hive that lists everyone but itself would otherwise
derive its own peer set as *everything* and peer with itself.
'';
};
options.services.hyperhive.swarm.peerHives = lib.mkOption {
type = lib.types.attrsOf (lib.types.attrsOf lib.types.unspecified);
readOnly = true;
internal = true;
description = ''
Read-only: `hives` minus this host's own entry. Derived once here
rather than in each consumer, because "everything that isn't me"
is a filter four different modules were re-implementing and only
one of them has to be wrong for a hive to peer with itself.
'';
};
config = {
services.hyperhive.swarm.peerHives = lib.filterAttrs (name: _: name != cfg.hiveName) swarmCfg.hives;
assertions = [
{
# Guarded on `hiveName != null` so the required-hiveName
# assertion in ./hyperhive.nix is what fires for that case —
# two assertions naming the same missing value is noise.
assertion = swarmCfg.hives == { } || cfg.hiveName == null || swarmCfg.hives ? ${cfg.hiveName};
message = ''
services.hyperhive.swarm.hives has no entry for this hive
(services.hyperhive.hiveName = "${toString cfg.hiveName}").
`hives` describes every hive in the swarm including this one,
so that every host can share one identical attrset. Add:
services.hyperhive.swarm.hives."${toString cfg.hiveName}" = {
domain = "${toString cfg.domain}";
};
Declared hives: ${lib.concatStringsSep ", " (lib.attrNames swarmCfg.hives)}
'';
}
];
};
# `enableRequiredServices` is declared in ./swarm-required-services.nix
# together with the per-service `enable`s it asserts — it is a
# deployment-shape switch rather than swarm bookkeeping, so it lives