Compare commits

...
Author SHA1 Message Date
atlas
6a6266cd5e refactor(#2862): keep the option at services.hyperhive.snapshotStore
Reverting the namespace move from the previous commit — mara's reason
is better than mine was.

I grouped it with swarm.peers and swarm.wireguard because the module
serves the swarm tier. But those two describe THE SWARM: who is in it,
how it is meshed. snapshotStore describes THIS HOST'S ROLE. On a
standalone store box the operator enables one service, and nesting it
under `swarm` implies they are configuring a swarm when they are not.

The swarm- prefix on the file and units stands: the name says which
tier the component serves, the option path says what you are turning
on. Those are different questions and they are allowed different
answers.
2026-07-31 19:03:24 +02:00
atlas
70bcdb5463 refactor(#2862): swarm- prefix for the snapshot store
mara, in preparation for the swarm tier: the store is a swarm-level
role, not a hive one, so hive- was misleading about which tier it
belongs to. Module, units, syslog identifier, log lines and docs all
move to swarm-snapshot-store.

Also moved the option under services.hyperhive.swarm.snapshotStore, to
sit with swarm.peers and swarm.wireguard rather than dangling off the
top level. That is a judgement call beyond the literal rename — flagged
on the PR, and cheap precisely now: the option has never shipped, so
there is no deployment to migrate, whereas doing it after a release
would be a breaking change for no new benefit.
2026-07-31 19:03:24 +02:00
atlas
57459cb6d8 refactor(#2862): split the wireguard mesh out of swarm.nix
mara asked, and the file had already stopped being one thing: after
the gate moved off c0re.enable, swarm.nix held two concerns with
different audiences and different gates.

swarm.nix now declares WHO the peers are — data hive-c0re serialises
into HYPERHIVE_PEERS and the dashboard renders. Declaration only, no
config block.

swarm-wireguard.nix owns the mesh: assertions, the wg-hive interface,
the firewall port. That is plain host networking, and a machine which
runs no hive at all — the snapshot store — still needs it. Under the
old layout a reader could not tell which half of swarm.nix applied to
a non-hive host.

The two stay coupled by data, not by structure: the per-peer
wireguard* fields stay on the peer submodule, because that is where a
peer is described, and the mesh module reads them.

No behaviour change — same options, same gate, same rendered config.
2026-07-31 19:03:24 +02:00
atlas
c051cd9717 docs(#2862): document the snapshot store, drop the dedicated option
mara: the option was the wrong shape for the concern. "this host runs
nothing else" is a deployment expectation, not something a module
should assert about its own host — and asserting it made co-location
look like a config toggle rather than what it is.

Replaced with docs/snapshot-store.md, which the module had no docs
page at all before: enabling it, why the mesh is the authentication
(cryptokey routing already binds source address to pubkey, so certs
would authenticate the same fact twice and add an expiry), why the
destination is keyed per agent (a per-hive prefix splits an agent's
chain the first time it migrates), what the sender may and may not
choose, why the firewall rule is interface-scoped, what a snapshot
does and does not contain, and what the pull side still needs.

The dedicated-host expectation is stated there as an operational
assumption with its own failure mode — true on day one, quietly false
the day someone notices the box has spare disk — rather than as an
assertion someone flips to false to make the build proceed.

Linked from CLAUDE.md's reading paths.
2026-07-31 19:03:24 +02:00
atlas
4989579270 fix(#2862): open the receiver's port on the mesh interface
argus caught it: binding the socket to the mesh address does not open
the port. NixOS's firewall is default-deny and filters in netfilter,
before a packet reaches a bound socket — the bind chooses which
address accepts connections, not whether packets arrive. As shipped
the receiver was unreachable.

swarm.nix already shows the pattern for exactly this situation: it
opens the mesh's UDP port explicitly right after bringing the
interface up.

Interface-scoped to wg-hive rather than host-wide, so the option's
"reachable exactly by mesh peers" claim is actually true. A global
allowedTCPPorts would open the port on every interface including the
public NIC, leaving only the socket's bind address between the
internet and a root btrfs receive.
2026-07-31 19:03:24 +02:00
atlas
bdf8fdabd7 feat(#2862): swarm snapshot store, the btrfs receive endpoint
P1 of the storage backend: hives push agent snapshots over the
WireGuard mesh that swarm.nix already brings up. No controller
dependency — a btrfs subvolume tree, a socket-activated receiver, and
the existing mesh.

The mesh is the authentication. Cryptokey routing already binds a
peer's source address to its public key (allowedIPs = [
peer.wireguardAddress ]), so the store adds no key material and no
certs; anything else would authenticate the same fact twice.

Destination is keyed per AGENT, not per hive: after a migration the
same agent's next incremental send arrives from a different hive, and
a per-hive prefix would split its snapshot chain and break the
incremental parent lookup — the exact case this store exists to serve.

The sender unavoidably contributes the agent name (a btrfs stream
carries no such notion, and the subvolume name inside it is the
sender's). So the receiver owns the destination root and VALIDATES the
sender-supplied leaf against a whitelist charset — no slash, no dot,
so neither traversal nor an absolute path can survive it.

ListenStream binds this host's mesh address, never a wildcard, and
that is asserted rather than commented: bound to 0.0.0.0 the socket
would be an unauthenticated remote write into agent state.

swarm.nix: the mesh config moves off the c0re.enable gate onto
swarm.wireguard.enable. The mesh is host networking, not a c0re
feature — a swarm host that runs no hive (this store) previously got
no wg-hive interface at all. Nothing in that block was c0re-specific;
the peer data c0re consumes is rendered in hive-c0re and stays gated
there.

Confinement is deliberately not in the module: it is a property of the
deployment (a dedicated VM, or a container in the all-local case). The
systemd hardening is defence in depth only — btrfs receive needs
CAP_SYS_ADMIN, which can mount() its way out of the namespace those
directives set up. The `dedicated` option turns "this host runs
nothing else" into an assertion the build checks instead of an
assumption the deployer remembers.
2026-07-31 19:03:24 +02:00
6 changed files with 565 additions and 125 deletions

View file

@ -165,6 +165,10 @@ read them à la carte.
- **"How do I connect two hives into a swarm? How do I declare peer
hives and configure TLS trust?"** →
[`docs/swarm.md`](docs/swarm.md).
- **"Where do agent snapshots go? How does the swarm's `btrfs receive`
endpoint authenticate a pushing hive, and what does a snapshot
actually contain?"** →
[`docs/snapshot-store.md`](docs/snapshot-store.md).
- **"How does the rebuild queue work? What are queue kinds and sources?"** →
[`docs/coordinator.md`](docs/coordinator.md).
- **"How does the CI runner work? What's the auto-registration flow?"** →

186
docs/snapshot-store.md Normal file
View file

@ -0,0 +1,186 @@
# Snapshot store
The swarm's `btrfs receive` endpoint. Hives push agent snapshots to it
over the WireGuard mesh; a destination hive later pulls one back to
complete a migration.
Two things it is not, both worth stating because both are easy to
assume:
- **It is not the swarm controller**, and does not depend on one. It is
a NixOS host role: a btrfs subvolume tree, a socket-activated
receiver, and the `wg-hive` interface the swarm module already brings
up. That is why it can be deployed before any controller exists.
- **It is not a backup product.** It happens to hold the data a backup
would hold, and it should be operated accordingly (see
[Operating it](#operating-it)) --- but nothing in it does scheduling,
verification, or restore orchestration.
## Enabling it
```nix
services.hyperhive.snapshotStore = {
enable = true;
path = "/var/lib/hyperhive-snapshots"; # must be on btrfs
port = 51821;
};
# The mesh is a hard requirement, and is asserted:
services.hyperhive.swarm.wireguard = {
enable = true;
address = "10.100.0.9/24";
privateKeyFile = "/etc/wireguard/hive.key";
};
```
The store host is a swarm member like any other: peers declare it, and
it declares them, through `services.hyperhive.swarm.peers`. See
[swarm.md](swarm.md) for the mesh itself.
Note that the mesh is gated on `swarm.wireguard.enable`, **not** on
`c0re.enable` --- a store host runs no hive and would otherwise get no
`wg-hive` interface at all.
## The mesh is the authentication
There are no certificates here, and no key material of its own. That is
deliberate rather than an omission.
WireGuard's cryptokey routing already binds a peer's source address to
its public key: the swarm module configures each peer with
`allowedIPs = [ peer.wireguardAddress ]`, so a packet arriving from
that address provably came from the holder of that private key. A
packet that reaches the receiver has therefore already been
authenticated by the kernel.
Layering TLS client certs on top would authenticate *the same fact* a
second time, and add a credential with an expiry --- a migration that
fails because a renewal quietly didn't happen, discovered on the day
you need to move an agent.
## One subvolume per agent, not per hive
The destination is keyed by **agent**.
This is not cosmetic. After a migration, an agent's next incremental
send arrives from a *different* hive than the previous one. Keying by
hive would split that agent's snapshot chain across two directories,
and `btrfs send -p` would fail to find its parent --- breaking exactly
the case the store exists to serve.
## What the sender can and cannot choose
A `btrfs send` stream carries no notion of *which agent* it belongs to,
and the subvolume name inside it is chosen by the sender. So the
protocol is one `agent <name>` header line, then the raw stream.
The rule that matters:
> **The receiver owns the destination root. The sender-supplied name is
> validated, never used as a path.**
Validation is a whitelist --- `[A-Za-z0-9_-]+` and nothing else. No
slash and no dot means neither directory traversal nor an absolute path
can survive it. It is deliberately a whitelist and not a list of
forbidden characters: a blocklist only ever excludes the attacks
somebody already thought of.
## Reachability
The receiver is socket-activated, and the socket binds **this host's
mesh address**, never a wildcard. Both the mesh being enabled and the
address being set are assertions, not documentation --- bound to
`0.0.0.0` this socket is an unauthenticated remote write into agent
state.
Binding is not sufficient on its own. NixOS's firewall is default-deny
and filters in netfilter, *before* a packet reaches a bound socket, so
the port is opened explicitly --- and scoped to the mesh interface:
```nix
networking.firewall.interfaces.wg-hive.allowedTCPPorts = [ cfg.port ];
```
A host-wide `allowedTCPPorts` would open the port on every interface
including a public NIC, leaving only the socket's bind address between
the internet and a root `btrfs receive`.
## Operating it
### Confinement is the deployment's job
`btrfs receive` needs `CAP_SYS_ADMIN`, so the receiver runs as root.
The unit sets `ProtectSystem=strict`, `ProtectHome`, `PrivateTmp` and a
narrow `ReadWritePaths` --- but those are **defence in depth, not a
boundary**: a process holding `CAP_SYS_ADMIN` can call `mount(2)` and
undo the namespace they set up.
The boundary is the machine. The intended deployments are:
- **a swarm**: the store is its own small VM. The machine is the
boundary, which is stronger than anything the unit could assert about
itself.
- **all-in-one / local**: the store runs as a container on the c0re
host.
The second is worth keeping deliberately, and not only for
convenience: it means the confined path is exercised by every local
deployment. The usual failure mode for an isolated variant is that
nobody runs it day to day, so it rots and is discovered broken in
production.
⚠️ **The assumption to keep true over time:** the store host runs
nothing else. That is true on day one and quietly false the day someone
notices the box has spare disk. Nothing in the config objects when it
stops being true.
### It holds every agent's state from every hive
Which makes it the highest-value target in the swarm by a wide margin,
and means it should get the treatment a backup host gets --- restricted
access, and a decision (rather than an omission) on encryption at rest.
The trap is the label: this box holds backup-grade data while not being
called a backup, so it can end up with backup-grade *exposure* and
non-backup-grade *controls*. Nobody puts a migration staging area on
the access-review list.
### What a snapshot contains
The snapshot covers an agent's **state subvolume**, which is the parent
of `state/`, `claude/` and `harness/`. Consequences:
- The Claude session (`claude/`) travels, so a restored agent keeps its
live `--continue` session rather than needing to log in again.
- `harness/` travels too, including `harness/bash-tasks/`. Task output
is part of an agent's working continuity, so this is wanted --- but it
means anything that has ever leaked into a task's captured output is
in the retained snapshots as well.
It does **not** cover the agent's applied config (`/applied/<name>/`) or
its topology entry, both of which live outside the subvolume. A restore
therefore yields an agent's memory without its definition; closing that
gap is tracked separately.
### Retention
Retention lives on the *sending* side (last-N by count, swept
periodically), not here. Count rather than age is deliberate: a count
is bounded by construction, whereas an age policy silently scales disk
usage with how hot a hive runs.
Per-agent or per-hive `btrfs qgroup` quotas are not configured yet.
Without them one runaway hive can fill the store and take out every
other hive's snapshots.
## Not built yet
**The pull side.** Push is safe with minimal authorisation because a
hive can only ever write to a chain it owns. Pull is the direction that
needs a policy: unrestricted, any compromised hive could read every
agent's state from every other hive. It needs a notion of which hive
currently owns which agent, and that ownership record lands with the
swarm controller work.
With a single hive the question is trivial --- the only peer owns
everything it sends --- which is why the receive half ships first.

View file

@ -21,6 +21,8 @@
./hive-priv.nix
./hive-tls.nix
./otel.nix
./swarm-snapshot-store.nix
./swarm-wireguard.nix
./swarm.nix
];
}

View file

@ -0,0 +1,224 @@
# swarm-snapshot-store — the swarm's `btrfs receive` endpoint. Hives push
# agent snapshots here over the existing WireGuard mesh; a destination
# hive later pulls one back to complete a migration. Only the receive
# half exists today — the pull side needs an authorisation model for
# "which hive may fetch which agent's state", which lands with the
# swarm controller.
#
# This is NOT the swarm controller and does not depend on it: a btrfs
# subvolume tree, a socket-activated receiver, and the `wg-hive`
# interface `swarm.nix` already brings up. Deliberately no WireGuard
# config of its own --- the mesh's cryptokey routing
# (`allowedIPs = [ peer.wireguardAddress ]`) already binds a peer's
# source address to its public key, so the mesh IS the authentication
# and adding certs here would authenticate the same fact twice.
#
# Confinement is a property of the DEPLOYMENT, not of this unit: in a
# real swarm the store is its own small VM (the machine is the
# boundary); in the all-local case it's a container on the c0re host.
# The module hardcodes neither. docs/snapshot-store.md covers what the
# deployment is expected to provide.
{
pkgs,
lib,
config,
...
}:
let
cfg = config.services.hyperhive.snapshotStore;
wgCfg = config.services.hyperhive.swarm.wireguard;
# `swarm.wireguard.address` carries a prefix ("10.100.0.1/24") because
# it feeds `networking.wireguard.interfaces.wg-hive.ips`. A listen
# address must be the bare IP, so strip it.
meshAddress = lib.head (lib.splitString "/" wgCfg.address);
# The receiver. Socket-activated with Accept=yes, so stdin IS the
# accepted connection and systemd hands us the peer address in
# $REMOTE_ADDR --- which, on this interface, is a cryptographically
# authenticated statement about which hive is talking (see the
# cryptokey-routing note above).
#
# PROTOCOL: one `agent <name>\n` header line, then the raw `btrfs
# send` stream. The header exists because a btrfs stream does not
# carry the sending hive's notion of *which agent* it is --- the
# subvolume name inside the stream is chosen by the sender.
#
# ⚠️ The security rule, stated precisely, because the absolute form
# ("the sender never names its destination") is not achievable with
# btrfs send/receive: the RECEIVER owns the destination ROOT, and any
# sender-supplied component is VALIDATED, never used as a path. The
# name must match [A-Za-z0-9_-]+ exactly --- no slash, no dot, so no
# traversal and no absolute path can survive it. The root is ours;
# the leaf is checked against a whitelist charset before it is joined.
receiveScript = pkgs.writeShellScript "swarm-snapshot-receive" ''
set -euo pipefail
# Read exactly the header line, leaving the byte stream untouched
# for btrfs receive. `read` stops at the newline and does not
# buffer ahead, which is why the header is a line and not a
# fixed-width record.
if ! read -r keyword agent; then
echo "swarm-snapshot-store: peer ''${REMOTE_ADDR:-?} closed before sending a header" >&2
exit 1
fi
if [ "$keyword" != "agent" ]; then
echo "swarm-snapshot-store: peer ''${REMOTE_ADDR:-?} sent a bad header keyword" >&2
exit 1
fi
# Validate rather than trust. Anything outside this charset is
# rejected outright --- this is the check that makes the joined
# path below safe, so it must stay a whitelist, never a blocklist
# of bad characters.
case "$agent" in
"" | *[!A-Za-z0-9_-]*)
echo "swarm-snapshot-store: peer ''${REMOTE_ADDR:-?} sent an invalid agent name" >&2
exit 1
;;
esac
dest="${cfg.path}/$agent"
# One subvolume tree per AGENT, not per hive: after a migration the
# same agent's next incremental send arrives from a DIFFERENT hive,
# and a per-hive prefix would split its snapshot chain in two and
# break the incremental parent lookup --- exactly the case this
# store exists to serve.
mkdir -p "$dest"
echo "swarm-snapshot-store: receiving agent=$agent from ''${REMOTE_ADDR:-?}" >&2
exec ${pkgs.btrfs-progs}/bin/btrfs receive "$dest"
'';
in
{
options.services.hyperhive.snapshotStore = {
enable = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
Run the swarm snapshot store on this host: a `btrfs receive`
endpoint that hives push agent snapshots to over the WireGuard
mesh. Off by default --- it is a distinct deployment role, not
part of a hive.
Requires `services.hyperhive.swarm.wireguard.enable`: the mesh
is both the transport and the authentication, so there is no
meaningful configuration without it.
'';
};
path = lib.mkOption {
type = lib.types.path;
default = "/var/lib/hyperhive-snapshots";
description = ''
Root of the snapshot tree. Must be on a btrfs filesystem ---
`btrfs receive` fails otherwise. One subvolume directory per
agent is created beneath it, so an agent's incremental chain
stays contiguous across a migration between hives.
'';
};
port = lib.mkOption {
type = lib.types.port;
default = 51821;
description = ''
TCP port the receiver listens on. Bound to this host's
WireGuard mesh address only --- never a wildcard --- so it is
reachable exactly by mesh peers and by nothing else.
'';
};
};
config = lib.mkIf cfg.enable {
assertions = [
{
assertion = wgCfg.enable;
message = ''
services.hyperhive.snapshotStore.enable requires
services.hyperhive.swarm.wireguard.enable --- the mesh is the
store's transport AND its authentication (cryptokey routing
binds a peer's source address to its public key). Without it
there is nothing to bind the listener to and no way to tell
which hive is pushing.
'';
}
{
assertion = wgCfg.address != "";
message = ''
services.hyperhive.snapshotStore.enable requires
services.hyperhive.swarm.wireguard.address to be set --- the
receiver binds to this host's mesh address, and refuses to
fall back to a wildcard.
'';
}
];
# The store root must exist before the first connection arrives ---
# the receiver runs on demand and should not be the thing that
# creates its own tree lazily.
systemd.tmpfiles.rules = [ "d ${cfg.path} 0700 root root -" ];
# Open the receiver's port, scoped to the mesh interface.
#
# ⚠️ Binding the socket to the mesh address is NOT sufficient on its
# own: NixOS's firewall is default-deny and filters in netfilter,
# before a packet ever reaches a bound socket. The bind chooses
# WHICH address accepts connections; it does not open the port. The
# mesh's own UDP port is opened the same explicit way in swarm.nix.
#
# Interface-scoped rather than host-wide so the reachability
# property stays exactly what the option docs claim --- mesh peers
# and nobody else. A global `allowedTCPPorts` would open the port on
# every interface, including whatever public NIC the box has, and
# only the socket's bind address would still be standing between
# the internet and a root `btrfs receive`.
networking.firewall.interfaces.wg-hive.allowedTCPPorts = [ cfg.port ];
# Socket-activated on purpose: no long-running root daemon, and the
# unit exists only while a transfer does.
#
# ⚠️ ListenStream is the mesh address, never 0.0.0.0. Bound to a
# wildcard this socket would be an unauthenticated remote write
# into agent state, so the binding IS the access control and is
# asserted above rather than left to a comment.
#
# Accept=yes gives one service instance per connection and sets
# $REMOTE_ADDR for the handler --- which is how the receiver knows
# which peer it is talking to.
systemd.sockets.swarm-snapshot-store = {
description = "hyperhive swarm snapshot store receiver socket";
wantedBy = [ "sockets.target" ];
socketConfig = {
ListenStream = "${meshAddress}:${toString cfg.port}";
Accept = "yes";
};
};
# `btrfs receive` needs CAP_SYS_ADMIN, so this runs as root by
# nature. The hardening below is defence in depth and NOT a
# boundary: a process holding CAP_SYS_ADMIN can call mount(2) and
# undo the namespace these directives set up. The real boundary is
# the deployment --- see docs/snapshot-store.md.
systemd.services."swarm-snapshot-store@" = {
description = "hyperhive swarm snapshot store receiver";
after = [ "swarm-snapshot-store.socket" ];
requires = [ "swarm-snapshot-store.socket" ];
serviceConfig = {
ExecStart = receiveScript;
SyslogIdentifier = "swarm-snapshot-store";
# StandardInput=socket wires the accepted connection to stdin,
# which is what the handler reads the header + stream from.
StandardInput = "socket";
StandardError = "journal";
User = "root";
PrivateTmp = true;
ProtectHome = true;
ProtectSystem = "strict";
ReadWritePaths = [ cfg.path ];
};
};
};
}

View file

@ -0,0 +1,142 @@
# The WireGuard inter-hive mesh for the local host. Split out of
# ./swarm.nix because the two are different concerns with different
# audiences: that file declares WHO the peers are (data hive-c0re
# serialises into HYPERHIVE_PEERS and the dashboard renders), while
# this one is plain host networking that a machine which runs no hive
# at all --- the snapshot store, for one --- still needs.
#
# The two stay coupled by data, not by structure: the per-peer
# `wireguard*` fields live on the peer submodule in ./swarm.nix, since
# that is where a peer is described, and this module reads them.
{
lib,
config,
...
}:
{
# WireGuard mesh config for the local host.
# When enabled, a `wg-hive` interface connects to all peers that have
# `wireguardPublicKey` declared. Peers reachable over the mesh are
# preferred for inter-hive traffic (no public TLS round-trip needed);
# peers without a public key still work via normal HTTPS.
options.services.hyperhive.swarm.wireguard = {
enable = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
Enable the WireGuard inter-hive mesh. When true, a `wg-hive`
interface is brought up connecting to all swarm peers that
declare a `wireguardPublicKey`. Requires
`privateKeyFile` to be set.
'';
};
privateKeyFile = lib.mkOption {
type = lib.types.nullOr lib.types.path;
default = null;
example = "/etc/wireguard/hive.key";
description = ''
Path to the host's WireGuard private key file. The file must
be readable by root and should have mode 0400. Generate with
`wg genkey > /etc/wireguard/hive.key`. Required when
`swarm.wireguard.enable = true`.
'';
};
address = lib.mkOption {
type = lib.types.str;
default = "";
example = "10.100.0.1/24";
description = ''
IP address (with prefix) of this host on the WireGuard mesh.
Use a /24 (or broader) prefix so the routing table covers all
peer /32 routes. Example: `"10.100.0.1/24"` for a 256-host mesh.
'';
};
listenPort = lib.mkOption {
type = lib.types.port;
default = 51820;
description = ''
UDP port the local WireGuard interface listens on. Must be
reachable from peer hosts when they initiate the tunnel.
Default: 51820 (standard WireGuard port).
'';
};
persistentKeepalive = lib.mkOption {
type = lib.types.nullOr lib.types.int;
default = 25;
example = 25;
description = ''
Seconds between keepalive packets sent to each peer. Useful
when this host (or a peer) is behind NAT keeps the UDP hole
open. Set to null to disable. Default: 25 seconds.
'';
};
};
# Gated on the mesh itself, NOT on the c0re daemon. The mesh is host
# networking, not a c0re feature: a swarm host that runs no hive —
# the snapshot store, for one — still has to join the mesh, and under
# the old `c0re.enable` gate it silently got no `wg-hive` interface
# at all. Nothing below is c0re-specific; the peer data
# c0re consumes (HYPERHIVE_PEERS / HIVE_PEER_CA_PATHS) is rendered in
# ./hive-c0re and stays gated there.
config = lib.mkIf config.services.hyperhive.swarm.wireguard.enable {
assertions = [
{
assertion = config.services.hyperhive.swarm.wireguard.privateKeyFile != null;
message = ''
services.hyperhive.swarm.wireguard.enable requires
services.hyperhive.swarm.wireguard.privateKeyFile to be set.
Generate a key: wg genkey > /etc/wireguard/hive.key
'';
}
{
assertion = config.services.hyperhive.swarm.wireguard.address != "";
message = ''
services.hyperhive.swarm.wireguard.enable requires
services.hyperhive.swarm.wireguard.address to be set
(e.g. "10.100.0.1/24").
'';
}
];
# WireGuard inter-hive mesh. Brings up a `wg-hive` interface and
# connects to each peer that has `wireguardPublicKey` set.
networking.wireguard.interfaces =
let
wgCfg = config.services.hyperhive.swarm.wireguard;
meshPeers = lib.filterAttrs (
_: p: p.wireguardPublicKey != null && p.wireguardAddress != null
) config.services.hyperhive.swarm.peers;
in
{
wg-hive = {
ips = [ wgCfg.address ];
listenPort = wgCfg.listenPort;
privateKeyFile = wgCfg.privateKeyFile;
peers = lib.mapAttrsToList (
_domain: p:
{
publicKey = p.wireguardPublicKey;
allowedIPs = [ p.wireguardAddress ];
}
// lib.optionalAttrs (p.wireguardEndpoint != null) {
endpoint = p.wireguardEndpoint;
}
// lib.optionalAttrs (wgCfg.persistentKeepalive != null) {
persistentKeepalive = wgCfg.persistentKeepalive;
}
) meshPeers;
};
};
# Open the WireGuard UDP port on the host firewall (host-level
# networking — not inside containers).
networking.firewall.allowedUDPPorts = [
config.services.hyperhive.swarm.wireguard.listenPort
];
};
}

View file

@ -1,8 +1,11 @@
# Swarm peering: the peer-hive declarations and the optional
# WireGuard inter-hive mesh. The peers are serialised into hive-c0re's
# Swarm peering: who the peer hives are. Serialised into hive-c0re's
# environment (HYPERHIVE_PEERS / HIVE_PEER_CA_PATHS — see ./hive-c0re)
# and consumed by identity.rs + the dashboard's P33RS tab; the mesh
# config below is host-level networking.
# and consumed by identity.rs + the dashboard's P33RS tab.
#
# Declaration only — this module has no `config` block. The mesh that
# uses the `wireguard*` fields below lives in ./swarm-wireguard.nix,
# because bringing up an interface is host networking rather than swarm
# bookkeeping, and a host that runs no hive still needs it.
{
lib,
config,
@ -122,125 +125,4 @@
'';
};
# WireGuard mesh config for the local host.
# When enabled, a `wg-hive` interface connects to all peers that have
# `wireguardPublicKey` declared. Peers reachable over the mesh are
# preferred for inter-hive traffic (no public TLS round-trip needed);
# peers without a public key still work via normal HTTPS.
options.services.hyperhive.swarm.wireguard = {
enable = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
Enable the WireGuard inter-hive mesh. When true, a `wg-hive`
interface is brought up connecting to all swarm peers that
declare a `wireguardPublicKey`. Requires
`privateKeyFile` to be set.
'';
};
privateKeyFile = lib.mkOption {
type = lib.types.nullOr lib.types.path;
default = null;
example = "/etc/wireguard/hive.key";
description = ''
Path to the host's WireGuard private key file. The file must
be readable by root and should have mode 0400. Generate with
`wg genkey > /etc/wireguard/hive.key`. Required when
`swarm.wireguard.enable = true`.
'';
};
address = lib.mkOption {
type = lib.types.str;
default = "";
example = "10.100.0.1/24";
description = ''
IP address (with prefix) of this host on the WireGuard mesh.
Use a /24 (or broader) prefix so the routing table covers all
peer /32 routes. Example: `"10.100.0.1/24"` for a 256-host mesh.
'';
};
listenPort = lib.mkOption {
type = lib.types.port;
default = 51820;
description = ''
UDP port the local WireGuard interface listens on. Must be
reachable from peer hosts when they initiate the tunnel.
Default: 51820 (standard WireGuard port).
'';
};
persistentKeepalive = lib.mkOption {
type = lib.types.nullOr lib.types.int;
default = 25;
example = 25;
description = ''
Seconds between keepalive packets sent to each peer. Useful
when this host (or a peer) is behind NAT keeps the UDP hole
open. Set to null to disable. Default: 25 seconds.
'';
};
};
# Gated on the c0re daemon being enabled — the mesh is part of the
# coordinator host's networking.
config = lib.mkIf config.services.hyperhive.c0re.enable {
assertions = lib.optionals config.services.hyperhive.swarm.wireguard.enable [
{
assertion = config.services.hyperhive.swarm.wireguard.privateKeyFile != null;
message = ''
services.hyperhive.swarm.wireguard.enable requires
services.hyperhive.swarm.wireguard.privateKeyFile to be set.
Generate a key: wg genkey > /etc/wireguard/hive.key
'';
}
{
assertion = config.services.hyperhive.swarm.wireguard.address != "";
message = ''
services.hyperhive.swarm.wireguard.enable requires
services.hyperhive.swarm.wireguard.address to be set
(e.g. "10.100.0.1/24").
'';
}
];
# WireGuard inter-hive mesh. Brings up a `wg-hive` interface and
# connects to each peer that has `wireguardPublicKey` set.
networking.wireguard.interfaces = lib.mkIf config.services.hyperhive.swarm.wireguard.enable (
let
wgCfg = config.services.hyperhive.swarm.wireguard;
meshPeers = lib.filterAttrs (
_: p: p.wireguardPublicKey != null && p.wireguardAddress != null
) config.services.hyperhive.swarm.peers;
in
{
wg-hive = {
ips = [ wgCfg.address ];
listenPort = wgCfg.listenPort;
privateKeyFile = wgCfg.privateKeyFile;
peers = lib.mapAttrsToList (
_domain: p:
{
publicKey = p.wireguardPublicKey;
allowedIPs = [ p.wireguardAddress ];
}
// lib.optionalAttrs (p.wireguardEndpoint != null) {
endpoint = p.wireguardEndpoint;
}
// lib.optionalAttrs (wgCfg.persistentKeepalive != null) {
persistentKeepalive = wgCfg.persistentKeepalive;
}
) meshPeers;
};
}
);
# Open the WireGuard UDP port on the host firewall when the mesh is
# on (host-level networking — not inside containers).
networking.firewall.allowedUDPPorts = lib.mkIf config.services.hyperhive.swarm.wireguard.enable [
config.services.hyperhive.swarm.wireguard.listenPort
];
};
}