vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

refactor(nix): swarm.peers becomes swarm.hives, a directory of every hive

Browse source 
One attrset describing every hive in the swarm including this one,
identical on every host, with hiveName selecting which entry is us.
"My peers" is derived (swarm.peerHives) rather than declared.

Every field in the old per-host peer list was intrinsic to the hive it
described, never to the pair -- so the list was a directory each host
kept its own copy of. Beyond the deduplication it removes a bug class:
two hosts could hold different endpoints for the same third hive with
nothing to detect the disagreement.

Drops the per-hive caCert. Trust inside a swarm derives from the swarm
root, which every hive chains to. What that genuinely removes is
trusting a hive whose root this swarm does not own -- a cross-swarm
problem that wants a mechanism of its own, not a field that happened to
work.

The matrix container's certificateFiles block goes with it and could
NOT be migrated: that list is read at build time and the swarm root is
a runtime file (its key must never enter the store), so there is no
build-time name to put there. caCert being a nix path was precisely
what made it the build-time distribution channel. Agents are unaffected
-- hive-tls folds the root into the hive trust bundle and the meta
renderer embeds that one file. Tracked separately.

Migration is an assertion plus warnings, not a rename: hives is peers
union {self}, and the set gains a member no existing config has written
down. A rename migrates a name and a default can re-root a meaning;
neither can conjure a new member. The warning explains, the self-entry
assertion stops the build.
This commit is contained in:
atlas 2026-08-05 20:18:22 +02:00
commit 433b294099
19 changed files with 484 additions and 293 deletions
Download patch file Download diff file Expand all files Collapse all files

175
nix/host-modules/swarm.nix
View file

@ -1,68 +1,73 @@
# Swarm peering: who the peer hives are. Serialised into hive-c0re's
# environment (HYPERHIVE_PEERS / HIVE_PEER_CA_PATHS — see ./hive-c0re)
# and consumed by identity.rs + the dashboard's P33RS tab.
# The swarm's directory: one entry per hive, **including this one**,
# identical on every host in the swarm. `services.hyperhive.hiveName`
# says which entry is us, and `peerHives` below derives the rest.
#
# Declaration only — this module has no `config` block. The mesh that
# uses the `wireguard*` fields below lives in ./swarm-wireguard.nix,
# because bringing up an interface is host networking rather than swarm
# Why a directory rather than a per-host peer list: every field here is
# intrinsic to the hive it describes — none of them says anything about
# the *pair*. A list where every field is intrinsic is a directory each
# host was keeping its own copy of, which is O(n²) duplication that
# deduplicates without loss. It is also a correctness gain: two hosts
# could hold different endpoints for the same third hive and nothing
# detected it. One entry per hive makes that unrepresentable.
#
# Consumed by hive-c0re's environment (HYPERHIVE_PEERS — see
# ./hive-c0re), identity.rs + the dashboard's P33RS tab, and the mesh in
# ./swarm-wireguard.nix. The mesh lives there rather than here because
# bringing up an interface is host networking rather than swarm
# bookkeeping, and a host that runs no hive still needs it.
{
lib,
config,
...
}:
let
cfg = config.services.hyperhive;
swarmCfg = cfg.swarm;
in
{
# Peer hives in the same swarm. Each entry declares a remote hive
# reachable from this host.
options.services.hyperhive.swarm.peers = lib.mkOption {
options.services.hyperhive.swarm.hives = lib.mkOption {
type = lib.types.attrsOf (
lib.types.submodule {
options = {
domain = lib.mkOption {
type = lib.types.str;
example = "lab.example.com";
description = ''
Public DNS domain this hive occupies — used for dashboard
links, peer HTTPS checks and Matrix federation discovery.
Deliberately has no default. It is conventionally
`<name>.<swarm.domain>`, but defaulting to that would let
a typo'd swarm domain produce a name that resolves
somewhere real; an eval failure naming the hive is the
better outcome.
'';
};
certFingerprint = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "sha256:b1946ac92492d2347c6235b4d2611184a3f5b6cae6c19d6e3c2f0a8e7d4c9f12";
description = ''
Expected TLS certificate fingerprint for this peer's HTTPS
endpoint. Null = trust the system CA bundle (for Let's
Encrypt peers). Set to pin a self-signed cert.
Expected TLS certificate fingerprint for this hive's HTTPS
endpoint. Null = trust the CA bundle — which for a hive
inside the swarm CA hierarchy is the normal case, since
every hive under the swarm root already chains to it.
Set it to pin a leaf that no CA in the bundle vouches for.
Format: the literal `sha256:` followed by exactly 64
hex digits (case-insensitive, no colon separators) — the
SHA-256 digest of the peer's DER-encoded leaf certificate.
SHA-256 digest of the DER-encoded leaf certificate.
Generate with `openssl x509 -noout -fingerprint -sha256`,
then strip the colons and prepend `sha256:`. A malformed
value is ignored with a warning rather than weakening
trust. See docs/swarm/README.md for the full recipe.
Scopes only to hive-c0re's own peer HTTPS checks — it does
NOT help Matrix federation (tuwunel validates against its
container trust bundle). For a self-signed peer whose root
CA you want trusted hive-wide (every agent + Matrix
federation), set `caCert` below.
'';
};
caCert = lib.mkOption {
type = lib.types.nullOr lib.types.path;
default = null;
example = "./peers/edge-ca.pem";
description = ''
Path to this peer hive's root CA certificate (PEM). When
set, the CA is embedded (at build time, into the nix store
— no runtime file on the host) and trusted **everywhere the
hive's own internal CA is**: it rides alongside `hive-ca.pem`
in each agent's `security.pki.certificateFiles` (via the
meta-flake renderer), and is added to the Matrix homeserver
container's trust bundle so tuwunel validates *federation*
TLS from a self-signed peer hive whose cert chains to it.
This is the CA-trust path that `certFingerprint`
(leaf-pinning, c0re-only) can't cover, and is what unblocks
Matrix federation with a self-signed peer hive. Trust stays
inside the hive (agents + the Matrix container), never the
host system trust store. Mutually complementary with
`certFingerprint`; set `caCert` for the federation case. See
docs/swarm/README.md.
NOT help Matrix federation, which validates against the
container's trust bundle. There is no per-hive CA field to
cover that case any more: the swarm root is the trust path
(see ./swarm-ca.nix).
'';
};
@ -71,9 +76,9 @@
default = null;
example = "base64pubkey=";
description = ''
WireGuard public key for this peer host. Required when
WireGuard public key for this hive's host. Required when
`services.hyperhive.swarm.wireguard.enable = true` and
you want this peer reachable over the mesh. Null = TLS-
you want this hive reachable over the mesh. Null = TLS-
only peering (public internet, no mesh tunnel).
'';
};
@ -83,11 +88,13 @@
default = null;
example = "203.0.113.1:51820";
description = ''
WireGuard endpoint for this peer in `host:port` form.
Required when the peer host is behind a firewall and
this host needs to initiate the tunnel. Null = this host
waits for the peer to connect (peer-initiates; peer must
have an endpoint pointing back at this host).
WireGuard endpoint for this hive in `host:port` form.
Null = this hive has no reachable endpoint, so the tunnel
is initiated from the other side.
Reads like a fact about the relationship and is not: it
says whether *this* hive can be dialled, which every other
hive in the swarm needs the same answer to.
'';
};
@ -96,36 +103,82 @@
default = null;
example = "10.100.0.2/32";
description = ''
IP address (with prefix) of the peer host on the
WireGuard mesh. Used as the `allowedIPs` for the peer's
IP address (with prefix) of this hive's host on the
WireGuard mesh. Used as the `allowedIPs` for its
WireGuard config entry and injected into `HYPERHIVE_PEERS`
so hive-c0re can route intra-swarm traffic to the mesh
address rather than the public domain. Required to include
the peer in the WireGuard mesh (peers missing this field
are silently excluded from `wg-hive`).
a hive in the mesh (entries missing this field are
silently excluded from `wg-hive`).
'';
};
};
}
);
default = { };
example = {
"lab.example.com" = {
certFingerprint = "sha256:b1946ac92492d2347c6235b4d2611184a3f5b6cae6c19d6e3c2f0a8e7d4c9f12";
pr1ma = {
domain = "pr1ma.example.com";
wireguardAddress = "10.100.0.1/32";
wireguardEndpoint = "203.0.113.1:51820";
};
edge = {
domain = "edge.example.com";
wireguardAddress = "10.100.0.2/32";
};
"edge.corp" = { };
};
description = ''
Peer hives in the same swarm. The attrset key is the peer's DNS
domain — used for dashboard links and Matrix federation discovery.
Null `certFingerprint` trusts the system CA bundle; set it to pin
a self-signed TLS cert. Add `wireguardPublicKey` + `wireguardAddress`
(and optionally `wireguardEndpoint`) to include the peer in the
WireGuard mesh when `swarm.wireguard.enable = true`.
Every hive in this swarm, keyed by `hiveName` — **including this
host's own hive**. The same attrset is meant to be identical on
every host in the swarm, so it can be written once and shared;
`services.hyperhive.hiveName` is what makes a given host read it
as "me and four others" rather than "five peers".
Empty (the default) means this host is not part of a swarm. Once
non-empty it must contain an entry for `hiveName`, which is
asserted — a hive that lists everyone but itself would otherwise
derive its own peer set as *everything* and peer with itself.
'';
};
options.services.hyperhive.swarm.peerHives = lib.mkOption {
type = lib.types.attrsOf (lib.types.attrsOf lib.types.unspecified);
readOnly = true;
internal = true;
description = ''
Read-only: `hives` minus this host's own entry. Derived once here
rather than in each consumer, because "everything that isn't me"
is a filter four different modules were re-implementing and only
one of them has to be wrong for a hive to peer with itself.
'';
};
config = {
services.hyperhive.swarm.peerHives = lib.filterAttrs (name: _: name != cfg.hiveName) swarmCfg.hives;
assertions = [
{
# Guarded on `hiveName != null` so the required-hiveName
# assertion in ./hyperhive.nix is what fires for that case —
# two assertions naming the same missing value is noise.
assertion = swarmCfg.hives == { } || cfg.hiveName == null || swarmCfg.hives ? ${cfg.hiveName};
message = ''
services.hyperhive.swarm.hives has no entry for this hive
(services.hyperhive.hiveName = "${toString cfg.hiveName}").
`hives` describes every hive in the swarm including this one,
so that every host can share one identical attrset. Add:
services.hyperhive.swarm.hives."${toString cfg.hiveName}" = {
domain = "${toString cfg.domain}";
};
Declared hives: ${lib.concatStringsSep ", " (lib.attrNames swarmCfg.hives)}
'';
}
];
};
# `enableRequiredServices` is declared in ./swarm-required-services.nix
# together with the per-service `enable`s it asserts — it is a
# deployment-shape switch rather than swarm bookkeeping, so it lives