vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

deploy: move the wireguard mesh out of the namespace hives read

Browse source 
`swarm.*` is what a hive needs to be a *client* of the swarm; the mesh is
none of it. A peer needs this host's `wireguardEndpoint` -- the roster entry
in swarm.nix, which stays -- and nothing about the interface this host
brings up. The module already said so: "plain host networking that a machine
which runs no hive at all still needs."

All five options move, so the namespace relocates rather than splitting.
`listenPort` is the one that reads the other way: it is what this host
*binds*, while the port a peer *dials* lives inside `wireguardEndpoint`.

Declared in swarm-wireguard.nix under the `deploy.*` path, following
swarm-victorialogs.nix; deploy.nix carries only the renames, per its own
"a single file to delete when the deprecation window closes". Deliberately
NOT added to deploy.nix's own options block: every entry there is a swarm
service this host deploys, and the mesh is host networking.

hivectl/src/wg.rs generates the config snippet an operator pastes, so it
moves too -- otherwise the tool's own output trips the deprecation warning.

module-eval gains a case that configures a host through the OLD path and
asserts the rendered wg-hive interface, because the new path evaluates
fine without the shim: dropping it reads as a clean tree.
This commit is contained in:
atlas 2026-09-07 00:54:45 +02:00 • committed by mara
commit 368f5d82aa
10 changed files with 85 additions and 27 deletions
Download patch file Download diff file Expand all files Collapse all files

4
docs/networking/snapshot-store.md
View file

@ -26,7 +26,7 @@ services.hyperhive.snapshotStore = {
};
# The mesh is a hard requirement, and is asserted:
services.hyperhive.swarm.wireguard = {
services.hyperhive.deploy.wireguard = {
enable = true;
address = "10.100.0.9/24";
privateKeyFile = "/etc/wireguard/hive.key";
@ -37,7 +37,7 @@ The store host is a swarm member like any other: it gets an entry in
`services.hyperhive.swarm.hives`, the same directory every host holds. See
[swarm/](../swarm/README.md) for the mesh itself.
Note that the mesh is gated on `swarm.wireguard.enable`, **not** on
Note that the mesh is gated on `deploy.wireguard.enable`, **not** on
`c0re.enable` --- a store host runs no hive and would otherwise get no
`wg-hive` interface at all.

6
docs/swarm/README.md
View file

@ -168,7 +168,7 @@ evaluates cleanly points at a real machine that isn't the one you meant.
issue. Until then, federation needs CA-issued certs (ACME). See
`docs/integrations/matrix.md` for federation firewall + TLS requirements.
3. **WireGuard mesh** (optional) — `swarm.wireguard.enable` reads each
3. **WireGuard mesh** (optional) — `deploy.wireguard.enable` reads each
entry's `wireguardPublicKey`/`wireguardEndpoint`/`wireguardAddress`
to configure `wg-hive`. See "WireGuard inter-hive mesh" below.
@ -210,7 +210,7 @@ wg pubkey < /etc/wireguard/hive.key # → share this with peer operators
```nix
# hive A (pr1ma.example.com, mesh IP 10.100.0.1)
services.hyperhive = {
swarm.wireguard = {
deploy.wireguard = {
enable = true;
privateKeyFile = "/etc/wireguard/hive.key";
address = "10.100.0.1/24";
@ -237,7 +237,7 @@ services.hyperhive = {
# hive B (edge.corp, mesh IP 10.100.0.2)
services.hyperhive = {
swarm.wireguard = {
deploy.wireguard = {
enable = true;
privateKeyFile = "/etc/wireguard/hive.key";
address = "10.100.0.2/24";

2
docs/swarm/secrets.md
View file

@ -53,7 +53,7 @@ neither is a renaming of the other.
| OIDC client secret, digest half | the same mint | `oidc-clients/<id>.digest` | authelia's own half; merged at runtime via `settingsFiles` |
| the swarm collector's copy of its OIDC secret | `swarm-otel-oidc-secret.service` copies it from authelia's tree, **when authelia runs on this host** | `/var/lib/swarm-otel-oidc/<id>.secret` inside the `swarm-otel` container | operator provides the file and names it in `swarm.otel.clientSecretFile` — the collector need not share a host with authelia |
| authelia subject store | `swarmctl` and `swarm-authelia-bridge` | `users.yml` — one file, read and written by both | `swarmctl`, on the host that runs authelia |
| wireguard private key | **the operator** — `wg genkey` | whatever `swarm.wireguard.privateKeyFile` names | always operator-provided; nothing generates this for you |
| wireguard private key | **the operator** — `wg genkey` | whatever `deploy.wireguard.privateKeyFile` names | always operator-provided; nothing generates this for you |
| queue auth-callout nkeys (user seed + account seed) | `swarm-nats-callout-keys` first-boot unit, when `nats.autoGenerateCallout` is set | `/var/lib/swarm-nats-callout/{callout-user,issuer}.seed`, `0600` | operator mints both with `nk` and names them in `nats.calloutUserSeedFile` / `nats.calloutIssuerSeedFile` |
| the secret store's own contents | openbao, on first `bao operator init` — **an operator action, not a unit** | inside the `swarm-bao` container, at its own `/var/lib/openbao`, kept across rebuilds by `ephemeral = false`. ⚠️ **Not a host path**: `nixos-container destroy swarm-bao` takes the raft data with it, so back up the container's tree, not `/var/lib/`. Only the store's TLS material (`/var/lib/swarm-bao-tls`) and its PKCS11 token (`/var/lib/swarm-bao-token`) are host-level | n/a — there is one store; a hive elsewhere is a _client_ of it and holds none of this |
| the secret store's unseal material | the HSM/TPM under `deploy.bao.seal = "pkcs11"`; openbao itself under `"shamir"` | in the token; or held by whoever ran `bao operator init`, which is what `"shamir"` means and why it is stated rather than inferred | n/a — only the host running the store seals anything |

6
hivectl/src/wg.rs
View file

@ -9,7 +9,7 @@ use anyhow::{Context as _, Result, bail};
use crate::util::query_hive_urls;
/// Host path of this hive's WireGuard private key (matches the
/// `privateKeyFile` example in the swarm.wireguard nix options).
/// `privateKeyFile` example in the deploy.wireguard nix options).
const WG_KEY_PATH: &str = "/etc/wireguard/hive.key";
/// The mesh interface name hive-c0re's nix module brings up.
const WG_INTERFACE: &str = "wg-hive";
@ -116,7 +116,7 @@ pub(crate) async fn wg_init(socket: &Path, address: Option<&str>) -> Result<()>
println!("\nPublic key (share this with peer hives — they pass it to `hivectl wg peer`):");
println!(" {pubkey}");
println!("\nAdd to this hive's NixOS config:");
println!(" services.hyperhive.swarm.wireguard = {{");
println!(" services.hyperhive.deploy.wireguard = {{");
println!(" enable = true;");
println!(" privateKeyFile = \"{WG_KEY_PATH}\";");
println!(" address = \"{addr}\";");
@ -238,7 +238,7 @@ pub(crate) fn wg_status() -> Result<()> {
if !status.success() {
bail!(
"`wg show {WG_INTERFACE}` failed — is the mesh enabled + up? \
(services.hyperhive.swarm.wireguard.enable = true, then deploy)"
(services.hyperhive.deploy.wireguard.enable = true, then deploy)"
);
}
Ok(())

28
nix/host-modules/deploy.nix
View file

@ -148,6 +148,34 @@ in
[ "services" "hyperhive" "tls" "leafValidityDays" ]
[ "services" "hyperhive" "deploy" "hive-controller" "tls" "leafValidityDays" ]
)
# The WireGuard mesh, whole. Unlike every rename above this one moves a
# namespace rather than a toggle: nothing under it is a fact another hive
# reads. A peer needs this host's `wireguardEndpoint` — the roster entry
# in ./swarm.nix, which stays — and nothing about the interface this host
# brings up. `listenPort` moves for the same reason and is easy to read
# the other way: it is what this host *binds*, while the port a peer
# *dials* is the one inside `wireguardEndpoint`.
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "wireguard" "enable" ]
[ "services" "hyperhive" "deploy" "wireguard" "enable" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "wireguard" "privateKeyFile" ]
[ "services" "hyperhive" "deploy" "wireguard" "privateKeyFile" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "wireguard" "address" ]
[ "services" "hyperhive" "deploy" "wireguard" "address" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "wireguard" "listenPort" ]
[ "services" "hyperhive" "deploy" "wireguard" "listenPort" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "wireguard" "persistentKeepalive" ]
[ "services" "hyperhive" "deploy" "wireguard" "persistentKeepalive" ]
)
];
# ⚠️ `deploy.forgejo` is declared in ./hive-ci.nix, not here, and it is the

10
nix/host-modules/swarm-snapshot-store.nix
View file

@ -26,9 +26,9 @@
}:
let
cfg = config.services.hyperhive.snapshotStore;
wgCfg = config.services.hyperhive.swarm.wireguard;
wgCfg = config.services.hyperhive.deploy.wireguard;
# `swarm.wireguard.address` carries a prefix ("10.100.0.1/24") because
# `deploy.wireguard.address` carries a prefix ("10.100.0.1/24") because
# it feeds `networking.wireguard.interfaces.wg-hive.ips`. A listen
# address must be the bare IP, so strip it.
meshAddress = lib.head (lib.splitString "/" wgCfg.address);
@ -103,7 +103,7 @@ in
mesh. Off by default --- it is a distinct deployment role, not
part of a hive.
Requires `services.hyperhive.swarm.wireguard.enable`: the mesh
Requires `services.hyperhive.deploy.wireguard.enable`: the mesh
is both the transport and the authentication, so there is no
meaningful configuration without it.
'';
@ -138,7 +138,7 @@ in
assertion = wgCfg.enable;
message = ''
services.hyperhive.snapshotStore.enable requires
services.hyperhive.swarm.wireguard.enable --- the mesh is the
services.hyperhive.deploy.wireguard.enable --- the mesh is the
store's transport AND its authentication (cryptokey routing
binds a peer's source address to its public key). Without it
there is nothing to bind the listener to and no way to tell
@ -149,7 +149,7 @@ in
assertion = wgCfg.address != "";
message = ''
services.hyperhive.snapshotStore.enable requires
services.hyperhive.swarm.wireguard.address to be set --- the
services.hyperhive.deploy.wireguard.address to be set --- the
receiver binds to this host's mesh address, and refuses to
fall back to a wildcard.
'';

30
nix/host-modules/swarm-wireguard.nix
View file

@ -8,6 +8,14 @@
# The two stay coupled by data, not by structure: the per-peer
# `wireguard*` fields live on the peer submodule in ./swarm.nix, since
# that is where a peer is described, and this module reads them.
#
# Everything declared here is `deploy.*`, not `swarm.*`: by ./deploy.nix's
# rule, `swarm.*` is what a hive needs to be a *client* of the swarm, and
# none of this is. A peer needs this host's `wireguardEndpoint` (the roster
# entry in ./swarm.nix); what interface this host brings up, on which
# address, with which key, is nobody else's business. `listenPort` included
# — it is what this host binds, while the port a peer dials is the one
# inside `wireguardEndpoint`.
{
lib,
config,
@ -19,7 +27,7 @@
# `wireguardPublicKey` declared. Peers reachable over the mesh are
# preferred for inter-hive traffic (no public TLS round-trip needed);
# peers without a public key still work via normal HTTPS.
options.services.hyperhive.swarm.wireguard = {
options.services.hyperhive.deploy.wireguard = {
enable = lib.mkOption {
type = lib.types.bool;
default = false;
@ -39,7 +47,7 @@
Path to the host's WireGuard private key file. The file must
be readable by root and should have mode 0400. Generate with
`wg genkey > /etc/wireguard/hive.key`. Required when
`swarm.wireguard.enable = true`.
`deploy.wireguard.enable = true`.
'';
};
@ -83,21 +91,21 @@
# at all. Nothing below is c0re-specific; the peer data c0re consumes
# (HIVE_PEER_CA_PATHS) is rendered in ./hive-c0re and stays gated
# there.
config = lib.mkIf config.services.hyperhive.swarm.wireguard.enable {
config = lib.mkIf config.services.hyperhive.deploy.wireguard.enable {
assertions = [
{
assertion = config.services.hyperhive.swarm.wireguard.privateKeyFile != null;
assertion = config.services.hyperhive.deploy.wireguard.privateKeyFile != null;
message = ''
services.hyperhive.swarm.wireguard.enable requires
services.hyperhive.swarm.wireguard.privateKeyFile to be set.
services.hyperhive.deploy.wireguard.enable requires
services.hyperhive.deploy.wireguard.privateKeyFile to be set.
Generate a key: wg genkey > /etc/wireguard/hive.key
'';
}
{
assertion = config.services.hyperhive.swarm.wireguard.address != "";
assertion = config.services.hyperhive.deploy.wireguard.address != "";
message = ''
services.hyperhive.swarm.wireguard.enable requires
services.hyperhive.swarm.wireguard.address to be set
services.hyperhive.deploy.wireguard.enable requires
services.hyperhive.deploy.wireguard.address to be set
(e.g. "10.100.0.1/24").
'';
}
@ -107,7 +115,7 @@
# connects to each peer that has `wireguardPublicKey` set.
networking.wireguard.interfaces =
let
wgCfg = config.services.hyperhive.swarm.wireguard;
wgCfg = config.services.hyperhive.deploy.wireguard;
# `peerHives` is `swarm.hives` minus this hive (../swarm.nix) —
# a mesh that included our own entry would configure a tunnel to
# ourselves.
@ -139,7 +147,7 @@
# Open the WireGuard UDP port on the host firewall (host-level
# networking — not inside containers).
networking.firewall.allowedUDPPorts = [
config.services.hyperhive.swarm.wireguard.listenPort
config.services.hyperhive.deploy.wireguard.listenPort
];
};
}

2
nix/host-modules/swarm.nix
View file

@ -144,7 +144,7 @@ in
example = "base64pubkey=";
description = ''
WireGuard public key for this hive's host. Required when
`services.hyperhive.swarm.wireguard.enable = true` and
`services.hyperhive.deploy.wireguard.enable = true` and
you want this hive reachable over the mesh. Null = TLS-
only peering (public internet, no mesh tunnel).
'';

22
nix/module-eval.nix
View file

@ -62,6 +62,16 @@ let
bare = hive { };
withCi = hive { deploy.forgejo.ci.enable = true; };
# A host configured against the pre-rename option path. `mkRenamedOptionModule`
# is the only thing carrying it, and nothing else in this suite would notice
# if it were dropped: the new path evaluates fine on its own, so a missing
# shim reads as a clean tree and breaks every existing operator config.
wireguardOldPath = hive {
swarm.wireguard.enable = true;
swarm.wireguard.address = "10.100.0.1/24";
swarm.wireguard.privateKeyFile = "/etc/wireguard/hive.key";
};
baoPkcs11 = hive {
deploy.bao.enable = true;
deploy.bao.seal = "pkcs11";
@ -183,6 +193,18 @@ let
name = "the all-local mode turns the swarm controller on";
ok = allLocal.services.hyperhive.deploy.swarm-controller.enable;
}
{
# The mesh moved namespace wholesale, so an existing config sets paths
# that no longer exist. Reading the *rendered interface* rather than the
# option: a rename that resolved but stopped reaching the module would
# satisfy an option-level check and still bring up no tunnel.
name = "a config written against the pre-rename wireguard path still configures the interface";
ok =
let
wg = wireguardOldPath.networking.wireguard.interfaces.wg-hive;
in
wg.ips == [ "10.100.0.1/24" ] && wg.privateKeyFile == "/etc/wireguard/hive.key";
}
{
# The gateway's per-name issuer choice. If this ever collapses to a
# constant, every swarm-service vhost serves a certificate its CA

2
nix/packages/default.nix
View file

@ -93,7 +93,7 @@ let
# Completions come from the binary's own `completions <shell>` verb
# (the single source of truth, so they never drift from the actual
# verbs). The wireguard-tools wrap makes `hivectl wg` subcommands
# work before `swarm.wireguard.enable` is set (wg init is the very
# work before `deploy.wireguard.enable` is set (wg init is the very
# first setup step). Completion generation runs before wrapProgram
# since wrapProgram renames the real binary.
hivectlPkg =