vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

deploy: move the wireguard mesh out of the namespace hives read

Browse source 
`swarm.*` is what a hive needs to be a *client* of the swarm; the mesh is
none of it. A peer needs this host's `wireguardEndpoint` -- the roster entry
in swarm.nix, which stays -- and nothing about the interface this host
brings up. The module already said so: "plain host networking that a machine
which runs no hive at all still needs."

All five options move, so the namespace relocates rather than splitting.
`listenPort` is the one that reads the other way: it is what this host
*binds*, while the port a peer *dials* lives inside `wireguardEndpoint`.

Declared in swarm-wireguard.nix under the `deploy.*` path, following
swarm-victorialogs.nix; deploy.nix carries only the renames, per its own
"a single file to delete when the deprecation window closes". Deliberately
NOT added to deploy.nix's own options block: every entry there is a swarm
service this host deploys, and the mesh is host networking.

hivectl/src/wg.rs generates the config snippet an operator pastes, so it
moves too -- otherwise the tool's own output trips the deprecation warning.

module-eval gains a case that configures a host through the OLD path and
asserts the rendered wg-hive interface, because the new path evaluates
fine without the shim: dropping it reads as a clean tree.
This commit is contained in:
atlas 2026-09-07 00:54:45 +02:00 • committed by mara
commit 368f5d82aa
10 changed files with 85 additions and 27 deletions
Download patch file Download diff file Expand all files Collapse all files

4
docs/networking/snapshot-store.md
View file

@ -26,7 +26,7 @@ services.hyperhive.snapshotStore = {
}; };
# The mesh is a hard requirement, and is asserted: # The mesh is a hard requirement, and is asserted:
services.hyperhive.swarm.wireguard = { services.hyperhive.deploy.wireguard = {
enable = true; enable = true;
address = "10.100.0.9/24"; address = "10.100.0.9/24";
privateKeyFile = "/etc/wireguard/hive.key"; privateKeyFile = "/etc/wireguard/hive.key";
@ -37,7 +37,7 @@ The store host is a swarm member like any other: it gets an entry in
`services.hyperhive.swarm.hives`, the same directory every host holds. See `services.hyperhive.swarm.hives`, the same directory every host holds. See
[swarm/](../swarm/README.md) for the mesh itself. [swarm/](../swarm/README.md) for the mesh itself.
Note that the mesh is gated on `swarm.wireguard.enable`, **not** on Note that the mesh is gated on `deploy.wireguard.enable`, **not** on
`c0re.enable` --- a store host runs no hive and would otherwise get no `c0re.enable` --- a store host runs no hive and would otherwise get no
`wg-hive` interface at all. `wg-hive` interface at all.

6
docs/swarm/README.md
View file

@ -168,7 +168,7 @@ evaluates cleanly points at a real machine that isn't the one you meant.
issue. Until then, federation needs CA-issued certs (ACME). See issue. Until then, federation needs CA-issued certs (ACME). See
`docs/integrations/matrix.md` for federation firewall + TLS requirements. `docs/integrations/matrix.md` for federation firewall + TLS requirements.
3. **WireGuard mesh** (optional) — `swarm.wireguard.enable` reads each 3. **WireGuard mesh** (optional) — `deploy.wireguard.enable` reads each
entry's `wireguardPublicKey`/`wireguardEndpoint`/`wireguardAddress` entry's `wireguardPublicKey`/`wireguardEndpoint`/`wireguardAddress`
to configure `wg-hive`. See "WireGuard inter-hive mesh" below. to configure `wg-hive`. See "WireGuard inter-hive mesh" below.
@ -210,7 +210,7 @@ wg pubkey < /etc/wireguard/hive.key # → share this with peer operators
```nix ```nix
# hive A (pr1ma.example.com, mesh IP 10.100.0.1) # hive A (pr1ma.example.com, mesh IP 10.100.0.1)
services.hyperhive = { services.hyperhive = {
swarm.wireguard = { deploy.wireguard = {
enable = true; enable = true;
privateKeyFile = "/etc/wireguard/hive.key"; privateKeyFile = "/etc/wireguard/hive.key";
address = "10.100.0.1/24"; address = "10.100.0.1/24";
@ -237,7 +237,7 @@ services.hyperhive = {
# hive B (edge.corp, mesh IP 10.100.0.2) # hive B (edge.corp, mesh IP 10.100.0.2)
services.hyperhive = { services.hyperhive = {
swarm.wireguard = { deploy.wireguard = {
enable = true; enable = true;
privateKeyFile = "/etc/wireguard/hive.key"; privateKeyFile = "/etc/wireguard/hive.key";
address = "10.100.0.2/24"; address = "10.100.0.2/24";

2
docs/swarm/secrets.md
View file

@ -53,7 +53,7 @@ neither is a renaming of the other.
| OIDC client secret, digest half | the same mint | `oidc-clients/<id>.digest` | authelia's own half; merged at runtime via `settingsFiles` | | OIDC client secret, digest half | the same mint | `oidc-clients/<id>.digest` | authelia's own half; merged at runtime via `settingsFiles` |
| the swarm collector's copy of its OIDC secret | `swarm-otel-oidc-secret.service` copies it from authelia's tree, **when authelia runs on this host** | `/var/lib/swarm-otel-oidc/<id>.secret` inside the `swarm-otel` container | operator provides the file and names it in `swarm.otel.clientSecretFile` — the collector need not share a host with authelia | | the swarm collector's copy of its OIDC secret | `swarm-otel-oidc-secret.service` copies it from authelia's tree, **when authelia runs on this host** | `/var/lib/swarm-otel-oidc/<id>.secret` inside the `swarm-otel` container | operator provides the file and names it in `swarm.otel.clientSecretFile` — the collector need not share a host with authelia |
| authelia subject store | `swarmctl` and `swarm-authelia-bridge` | `users.yml` — one file, read and written by both | `swarmctl`, on the host that runs authelia | | authelia subject store | `swarmctl` and `swarm-authelia-bridge` | `users.yml` — one file, read and written by both | `swarmctl`, on the host that runs authelia |
| wireguard private key | **the operator** — `wg genkey` | whatever `swarm.wireguard.privateKeyFile` names | always operator-provided; nothing generates this for you | | wireguard private key | **the operator** — `wg genkey` | whatever `deploy.wireguard.privateKeyFile` names | always operator-provided; nothing generates this for you |
| queue auth-callout nkeys (user seed + account seed) | `swarm-nats-callout-keys` first-boot unit, when `nats.autoGenerateCallout` is set | `/var/lib/swarm-nats-callout/{callout-user,issuer}.seed`, `0600` | operator mints both with `nk` and names them in `nats.calloutUserSeedFile` / `nats.calloutIssuerSeedFile` | | queue auth-callout nkeys (user seed + account seed) | `swarm-nats-callout-keys` first-boot unit, when `nats.autoGenerateCallout` is set | `/var/lib/swarm-nats-callout/{callout-user,issuer}.seed`, `0600` | operator mints both with `nk` and names them in `nats.calloutUserSeedFile` / `nats.calloutIssuerSeedFile` |
| the secret store's own contents | openbao, on first `bao operator init` — **an operator action, not a unit** | inside the `swarm-bao` container, at its own `/var/lib/openbao`, kept across rebuilds by `ephemeral = false`. ⚠️ **Not a host path**: `nixos-container destroy swarm-bao` takes the raft data with it, so back up the container's tree, not `/var/lib/`. Only the store's TLS material (`/var/lib/swarm-bao-tls`) and its PKCS11 token (`/var/lib/swarm-bao-token`) are host-level | n/a — there is one store; a hive elsewhere is a _client_ of it and holds none of this | | the secret store's own contents | openbao, on first `bao operator init` — **an operator action, not a unit** | inside the `swarm-bao` container, at its own `/var/lib/openbao`, kept across rebuilds by `ephemeral = false`. ⚠️ **Not a host path**: `nixos-container destroy swarm-bao` takes the raft data with it, so back up the container's tree, not `/var/lib/`. Only the store's TLS material (`/var/lib/swarm-bao-tls`) and its PKCS11 token (`/var/lib/swarm-bao-token`) are host-level | n/a — there is one store; a hive elsewhere is a _client_ of it and holds none of this |
| the secret store's unseal material | the HSM/TPM under `deploy.bao.seal = "pkcs11"`; openbao itself under `"shamir"` | in the token; or held by whoever ran `bao operator init`, which is what `"shamir"` means and why it is stated rather than inferred | n/a — only the host running the store seals anything | | the secret store's unseal material | the HSM/TPM under `deploy.bao.seal = "pkcs11"`; openbao itself under `"shamir"` | in the token; or held by whoever ran `bao operator init`, which is what `"shamir"` means and why it is stated rather than inferred | n/a — only the host running the store seals anything |

6
hivectl/src/wg.rs
View file

@ -9,7 +9,7 @@ use anyhow::{Context as _, Result, bail};
use crate::util::query_hive_urls; use crate::util::query_hive_urls;
/// Host path of this hive's WireGuard private key (matches the /// Host path of this hive's WireGuard private key (matches the
/// `privateKeyFile` example in the swarm.wireguard nix options). /// `privateKeyFile` example in the deploy.wireguard nix options).
const WG_KEY_PATH: &str = "/etc/wireguard/hive.key"; const WG_KEY_PATH: &str = "/etc/wireguard/hive.key";
/// The mesh interface name hive-c0re's nix module brings up. /// The mesh interface name hive-c0re's nix module brings up.
const WG_INTERFACE: &str = "wg-hive"; const WG_INTERFACE: &str = "wg-hive";
@ -116,7 +116,7 @@ pub(crate) async fn wg_init(socket: &Path, address: Option<&str>) -> Result<()>
println!("\nPublic key (share this with peer hives — they pass it to `hivectl wg peer`):"); println!("\nPublic key (share this with peer hives — they pass it to `hivectl wg peer`):");
println!(" {pubkey}"); println!(" {pubkey}");
println!("\nAdd to this hive's NixOS config:"); println!("\nAdd to this hive's NixOS config:");
println!(" services.hyperhive.swarm.wireguard = {{"); println!(" services.hyperhive.deploy.wireguard = {{");
println!(" enable = true;"); println!(" enable = true;");
println!(" privateKeyFile = \"{WG_KEY_PATH}\";"); println!(" privateKeyFile = \"{WG_KEY_PATH}\";");
println!(" address = \"{addr}\";"); println!(" address = \"{addr}\";");
@ -238,7 +238,7 @@ pub(crate) fn wg_status() -> Result<()> {
if !status.success() { if !status.success() {
bail!( bail!(
"`wg show {WG_INTERFACE}` failed — is the mesh enabled + up? \ "`wg show {WG_INTERFACE}` failed — is the mesh enabled + up? \
(services.hyperhive.swarm.wireguard.enable = true, then deploy)" (services.hyperhive.deploy.wireguard.enable = true, then deploy)"
); );
} }
Ok(()) Ok(())

28
nix/host-modules/deploy.nix
View file

@ -148,6 +148,34 @@ in
[ "services" "hyperhive" "tls" "leafValidityDays" ] [ "services" "hyperhive" "tls" "leafValidityDays" ]
[ "services" "hyperhive" "deploy" "hive-controller" "tls" "leafValidityDays" ] [ "services" "hyperhive" "deploy" "hive-controller" "tls" "leafValidityDays" ]
) )
# The WireGuard mesh, whole. Unlike every rename above this one moves a
# namespace rather than a toggle: nothing under it is a fact another hive
# reads. A peer needs this host's `wireguardEndpoint` — the roster entry
# in ./swarm.nix, which stays — and nothing about the interface this host
# brings up. `listenPort` moves for the same reason and is easy to read
# the other way: it is what this host *binds*, while the port a peer
# *dials* is the one inside `wireguardEndpoint`.
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "wireguard" "enable" ]
[ "services" "hyperhive" "deploy" "wireguard" "enable" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "wireguard" "privateKeyFile" ]
[ "services" "hyperhive" "deploy" "wireguard" "privateKeyFile" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "wireguard" "address" ]
[ "services" "hyperhive" "deploy" "wireguard" "address" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "wireguard" "listenPort" ]
[ "services" "hyperhive" "deploy" "wireguard" "listenPort" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "wireguard" "persistentKeepalive" ]
[ "services" "hyperhive" "deploy" "wireguard" "persistentKeepalive" ]
)
]; ];
# ⚠️ `deploy.forgejo` is declared in ./hive-ci.nix, not here, and it is the # ⚠️ `deploy.forgejo` is declared in ./hive-ci.nix, not here, and it is the

10
nix/host-modules/swarm-snapshot-store.nix
View file

@ -26,9 +26,9 @@
}: }:
let let
cfg = config.services.hyperhive.snapshotStore; cfg = config.services.hyperhive.snapshotStore;
wgCfg = config.services.hyperhive.swarm.wireguard; wgCfg = config.services.hyperhive.deploy.wireguard;
# `swarm.wireguard.address` carries a prefix ("10.100.0.1/24") because # `deploy.wireguard.address` carries a prefix ("10.100.0.1/24") because
# it feeds `networking.wireguard.interfaces.wg-hive.ips`. A listen # it feeds `networking.wireguard.interfaces.wg-hive.ips`. A listen
# address must be the bare IP, so strip it. # address must be the bare IP, so strip it.
meshAddress = lib.head (lib.splitString "/" wgCfg.address); meshAddress = lib.head (lib.splitString "/" wgCfg.address);
@ -103,7 +103,7 @@ in
mesh. Off by default --- it is a distinct deployment role, not mesh. Off by default --- it is a distinct deployment role, not
part of a hive. part of a hive.
Requires `services.hyperhive.swarm.wireguard.enable`: the mesh Requires `services.hyperhive.deploy.wireguard.enable`: the mesh
is both the transport and the authentication, so there is no is both the transport and the authentication, so there is no
meaningful configuration without it. meaningful configuration without it.
''; '';
@ -138,7 +138,7 @@ in
assertion = wgCfg.enable; assertion = wgCfg.enable;
message = '' message = ''
services.hyperhive.snapshotStore.enable requires services.hyperhive.snapshotStore.enable requires
services.hyperhive.swarm.wireguard.enable --- the mesh is the services.hyperhive.deploy.wireguard.enable --- the mesh is the
store's transport AND its authentication (cryptokey routing store's transport AND its authentication (cryptokey routing
binds a peer's source address to its public key). Without it binds a peer's source address to its public key). Without it
there is nothing to bind the listener to and no way to tell there is nothing to bind the listener to and no way to tell
@ -149,7 +149,7 @@ in
assertion = wgCfg.address != ""; assertion = wgCfg.address != "";
message = '' message = ''
services.hyperhive.snapshotStore.enable requires services.hyperhive.snapshotStore.enable requires
services.hyperhive.swarm.wireguard.address to be set --- the services.hyperhive.deploy.wireguard.address to be set --- the
receiver binds to this host's mesh address, and refuses to receiver binds to this host's mesh address, and refuses to
fall back to a wildcard. fall back to a wildcard.
''; '';

30
nix/host-modules/swarm-wireguard.nix
View file

@ -8,6 +8,14 @@
# The two stay coupled by data, not by structure: the per-peer # The two stay coupled by data, not by structure: the per-peer
# `wireguard*` fields live on the peer submodule in ./swarm.nix, since # `wireguard*` fields live on the peer submodule in ./swarm.nix, since
# that is where a peer is described, and this module reads them. # that is where a peer is described, and this module reads them.
#
# Everything declared here is `deploy.*`, not `swarm.*`: by ./deploy.nix's
# rule, `swarm.*` is what a hive needs to be a *client* of the swarm, and
# none of this is. A peer needs this host's `wireguardEndpoint` (the roster
# entry in ./swarm.nix); what interface this host brings up, on which
# address, with which key, is nobody else's business. `listenPort` included
# — it is what this host binds, while the port a peer dials is the one
# inside `wireguardEndpoint`.
{ {
lib, lib,
config, config,
@ -19,7 +27,7 @@
# `wireguardPublicKey` declared. Peers reachable over the mesh are # `wireguardPublicKey` declared. Peers reachable over the mesh are
# preferred for inter-hive traffic (no public TLS round-trip needed); # preferred for inter-hive traffic (no public TLS round-trip needed);
# peers without a public key still work via normal HTTPS. # peers without a public key still work via normal HTTPS.
options.services.hyperhive.swarm.wireguard = { options.services.hyperhive.deploy.wireguard = {
enable = lib.mkOption { enable = lib.mkOption {
type = lib.types.bool; type = lib.types.bool;
default = false; default = false;
@ -39,7 +47,7 @@
Path to the host's WireGuard private key file. The file must Path to the host's WireGuard private key file. The file must
be readable by root and should have mode 0400. Generate with be readable by root and should have mode 0400. Generate with
`wg genkey > /etc/wireguard/hive.key`. Required when `wg genkey > /etc/wireguard/hive.key`. Required when
`swarm.wireguard.enable = true`. `deploy.wireguard.enable = true`.
''; '';
}; };
@ -83,21 +91,21 @@
# at all. Nothing below is c0re-specific; the peer data c0re consumes # at all. Nothing below is c0re-specific; the peer data c0re consumes
# (HIVE_PEER_CA_PATHS) is rendered in ./hive-c0re and stays gated # (HIVE_PEER_CA_PATHS) is rendered in ./hive-c0re and stays gated
# there. # there.
config = lib.mkIf config.services.hyperhive.swarm.wireguard.enable { config = lib.mkIf config.services.hyperhive.deploy.wireguard.enable {
assertions = [ assertions = [
{ {
assertion = config.services.hyperhive.swarm.wireguard.privateKeyFile != null; assertion = config.services.hyperhive.deploy.wireguard.privateKeyFile != null;
message = '' message = ''
services.hyperhive.swarm.wireguard.enable requires services.hyperhive.deploy.wireguard.enable requires
services.hyperhive.swarm.wireguard.privateKeyFile to be set. services.hyperhive.deploy.wireguard.privateKeyFile to be set.
Generate a key: wg genkey > /etc/wireguard/hive.key Generate a key: wg genkey > /etc/wireguard/hive.key
''; '';
} }
{ {
assertion = config.services.hyperhive.swarm.wireguard.address != ""; assertion = config.services.hyperhive.deploy.wireguard.address != "";
message = '' message = ''
services.hyperhive.swarm.wireguard.enable requires services.hyperhive.deploy.wireguard.enable requires
services.hyperhive.swarm.wireguard.address to be set services.hyperhive.deploy.wireguard.address to be set
(e.g. "10.100.0.1/24"). (e.g. "10.100.0.1/24").
''; '';
} }
@ -107,7 +115,7 @@
# connects to each peer that has `wireguardPublicKey` set. # connects to each peer that has `wireguardPublicKey` set.
networking.wireguard.interfaces = networking.wireguard.interfaces =
let let
wgCfg = config.services.hyperhive.swarm.wireguard; wgCfg = config.services.hyperhive.deploy.wireguard;
# `peerHives` is `swarm.hives` minus this hive (../swarm.nix) — # `peerHives` is `swarm.hives` minus this hive (../swarm.nix) —
# a mesh that included our own entry would configure a tunnel to # a mesh that included our own entry would configure a tunnel to
# ourselves. # ourselves.
@ -139,7 +147,7 @@
# Open the WireGuard UDP port on the host firewall (host-level # Open the WireGuard UDP port on the host firewall (host-level
# networking — not inside containers). # networking — not inside containers).
networking.firewall.allowedUDPPorts = [ networking.firewall.allowedUDPPorts = [
config.services.hyperhive.swarm.wireguard.listenPort config.services.hyperhive.deploy.wireguard.listenPort
]; ];
}; };
} }

2
nix/host-modules/swarm.nix
View file

@ -144,7 +144,7 @@ in
example = "base64pubkey="; example = "base64pubkey=";
description = '' description = ''
WireGuard public key for this hive's host. Required when WireGuard public key for this hive's host. Required when
`services.hyperhive.swarm.wireguard.enable = true` and `services.hyperhive.deploy.wireguard.enable = true` and
you want this hive reachable over the mesh. Null = TLS- you want this hive reachable over the mesh. Null = TLS-
only peering (public internet, no mesh tunnel). only peering (public internet, no mesh tunnel).
''; '';

22
nix/module-eval.nix
View file

@ -62,6 +62,16 @@ let
bare = hive { }; bare = hive { };
withCi = hive { deploy.forgejo.ci.enable = true; }; withCi = hive { deploy.forgejo.ci.enable = true; };
# A host configured against the pre-rename option path. `mkRenamedOptionModule`
# is the only thing carrying it, and nothing else in this suite would notice
# if it were dropped: the new path evaluates fine on its own, so a missing
# shim reads as a clean tree and breaks every existing operator config.
wireguardOldPath = hive {
swarm.wireguard.enable = true;
swarm.wireguard.address = "10.100.0.1/24";
swarm.wireguard.privateKeyFile = "/etc/wireguard/hive.key";
};
baoPkcs11 = hive { baoPkcs11 = hive {
deploy.bao.enable = true; deploy.bao.enable = true;
deploy.bao.seal = "pkcs11"; deploy.bao.seal = "pkcs11";
@ -183,6 +193,18 @@ let
name = "the all-local mode turns the swarm controller on"; name = "the all-local mode turns the swarm controller on";
ok = allLocal.services.hyperhive.deploy.swarm-controller.enable; ok = allLocal.services.hyperhive.deploy.swarm-controller.enable;
} }
{
# The mesh moved namespace wholesale, so an existing config sets paths
# that no longer exist. Reading the *rendered interface* rather than the
# option: a rename that resolved but stopped reaching the module would
# satisfy an option-level check and still bring up no tunnel.
name = "a config written against the pre-rename wireguard path still configures the interface";
ok =
let
wg = wireguardOldPath.networking.wireguard.interfaces.wg-hive;
in
wg.ips == [ "10.100.0.1/24" ] && wg.privateKeyFile == "/etc/wireguard/hive.key";
}
{ {
# The gateway's per-name issuer choice. If this ever collapses to a # The gateway's per-name issuer choice. If this ever collapses to a
# constant, every swarm-service vhost serves a certificate its CA # constant, every swarm-service vhost serves a certificate its CA

2
nix/packages/default.nix
View file

@ -93,7 +93,7 @@ let
# Completions come from the binary's own `completions <shell>` verb # Completions come from the binary's own `completions <shell>` verb
# (the single source of truth, so they never drift from the actual # (the single source of truth, so they never drift from the actual
# verbs). The wireguard-tools wrap makes `hivectl wg` subcommands # verbs). The wireguard-tools wrap makes `hivectl wg` subcommands
# work before `swarm.wireguard.enable` is set (wg init is the very # work before `deploy.wireguard.enable` is set (wg init is the very
# first setup step). Completion generation runs before wrapProgram # first setup step). Completion generation runs before wrapProgram
# since wrapProgram renames the real binary. # since wrapProgram renames the real binary.
hivectlPkg = hivectlPkg =