refactor(nix): swarm.peers becomes swarm.hives, a directory of every hive

One attrset describing every hive in the swarm including this one,
identical on every host, with hiveName selecting which entry is us.
"My peers" is derived (swarm.peerHives) rather than declared.

Every field in the old per-host peer list was intrinsic to the hive it
described, never to the pair -- so the list was a directory each host
kept its own copy of. Beyond the deduplication it removes a bug class:
two hosts could hold different endpoints for the same third hive with
nothing to detect the disagreement.

Drops the per-hive caCert. Trust inside a swarm derives from the swarm
root, which every hive chains to. What that genuinely removes is
trusting a hive whose root this swarm does not own -- a cross-swarm
problem that wants a mechanism of its own, not a field that happened to
work.

The matrix container's certificateFiles block goes with it and could
NOT be migrated: that list is read at build time and the swarm root is
a runtime file (its key must never enter the store), so there is no
build-time name to put there. caCert being a nix path was precisely
what made it the build-time distribution channel. Agents are unaffected
-- hive-tls folds the root into the hive trust bundle and the meta
renderer embeds that one file. Tracked separately.

Migration is an assertion plus warnings, not a rename: hives is peers
union {self}, and the set gains a member no existing config has written
down. A rename migrates a name and a default can re-root a meaning;
neither can conjure a new member. The warning explains, the self-entry
assertion stops the build.
This commit is contained in:
atlas 2026-08-05 20:18:22 +02:00
commit 433b294099
19 changed files with 484 additions and 293 deletions

View file

@ -1,17 +1,19 @@
# The swarm root CA: the anchor a whole swarm shares, and the issuer of
# each hive's own CA (which is where it gets used — see ./hive-tls.nix).
#
# Why a hierarchy at all: cross-hive trust is hand-pinned today
# (`swarm.peers.<d>.caCert`), so every hive must name every peer — O(n²)
# configuration that a new hive can only join by editing all the others.
# One root makes it O(1): trust the root once and every present *and
# future* peer validates.
# Why a hierarchy at all: cross-hive trust used to be hand-pinned per
# peer, so every hive had to name every other one — O(n²) configuration
# that a new hive could only join by editing all the others. One root
# makes it O(1): trust the root once and every present *and future* peer
# validates. The per-peer CA field is gone precisely because this
# replaced it.
#
# Two provisioning modes, ONE structure — what differs is who puts the
# artifacts on disk, never what the artifacts are:
#
# - autoconfigured (the default while the hive declares no peers): the
# unit below generates the root here on first boot.
# - autoconfigured (`services.hyperhive.enableAllLocalDefaults`, or
# `autoConfigure` set directly): the unit below generates the root
# here on first boot.
# - operator-provided (multi-host): the operator installs the root cert
# — and, on a host that does not hold the root key, the hive CA too —
# into the state dirs, and this unit does nothing. Splitting a
@ -21,10 +23,11 @@
#
# The root KEY is the reason this is a runtime file and not a nix option:
# the store is world-readable and content-addressed, so a key committed
# to a flake is a key *published* to every consumer of that flake. The
# root CERT has the opposite property, and is already distributed at
# build time via `swarm.peers.<d>.caCert`. That asymmetry is what makes
# the layering work at all.
# to a flake is a key *published* to every consumer of that flake.
#
# ⚠️ The root CERT is therefore a runtime file too, which costs something:
# nothing whose trust store is built at build time can name it. See
# docs/swarm/ca.md § "Distributing the root".
{
lib,
config,
@ -83,8 +86,8 @@ in
default = "/var/lib/swarm-ca";
description = ''
Host directory holding the swarm root CA: `root.pem` (the
anchor, safe to distribute this is what a peer hive is
pointed at via `swarm.peers.<d>.caCert`) and `root-key.pem`
anchor, safe to distribute copy it to this same path on every
other host in the swarm) and `root-key.pem`
(0600, the one file that must never reach the nix store or
another host). The directory itself is 0700: nothing reads
out of it but the hive CA issuance in `hive-tls.nix`.