Add opt-in WireGuard mesh support to services.hyperhive.swarm: - swarm.peers.<domain>.wireguardPublicKey — peer's wg public key - swarm.peers.<domain>.wireguardEndpoint — peer's UDP endpoint (optional) - swarm.peers.<domain>.wireguardAddress — peer's mesh IP with prefix - swarm.wireguard.enable — bring up wg-hive interface - swarm.wireguard.privateKeyFile — path to host's wg private key - swarm.wireguard.address — this host's mesh IP/prefix - swarm.wireguard.listenPort — UDP listen port (default 51820) - swarm.wireguard.persistentKeepalive — keepalive seconds (default 25) When enabled, generates networking.wireguard.interfaces.wg-hive with one peer entry per mesh-enabled swarm.peers entry. Opens listenPort UDP on the host firewall. Adds wireguard_address to HYPERHIVE_PEERS JSON so hive-c0re can use mesh IPs for intra-swarm routing. Assertions guard against enable=true without privateKeyFile or address. Also refactors networking.firewall.allowedTCPPortRanges from the nested attrset form (which conflicted with the new allowedUDPPorts line) to the per-attribute form. docs/swarm.md: adds WireGuard setup section with key generation commands, two-hive config example, NAT/keepalive notes.
176 lines
6.1 KiB
Markdown
176 lines
6.1 KiB
Markdown
# Multi-hive swarms
|
|
|
|
A **swarm** is a collection of agents that share an identity and
|
|
coordinate across one or more hives. A single hyperhive instance
|
|
running on one host is already a swarm (one hive). This doc covers
|
|
the additional config needed when the swarm spans multiple hosts.
|
|
|
|
## Terminology
|
|
|
|
- **hive** — a single hyperhive installation on one host. Has its
|
|
own `services.hyperhive.domain` DNS name and its own set of agent
|
|
containers.
|
|
- **swarm** — one or more hives whose operators have declared them
|
|
as peers. Agents can be qualified as `agent@hive-domain`.
|
|
- **peer hive** — a remote hive declared under
|
|
`services.hyperhive.swarm.peers` on the local host.
|
|
|
|
## Hive identity config
|
|
|
|
```nix
|
|
services.hyperhive = {
|
|
domain = "pr1ma.example.com"; # machine-addressable DNS domain
|
|
hiveName = "pr1ma"; # human display name (optional)
|
|
swarmName = "constellat1on"; # shared swarm display name (optional)
|
|
};
|
|
```
|
|
|
|
`domain` is required when matrix federation is on (`matrix.enable`);
|
|
it drives `HYPERHIVE_HIVE_DOMAIN` in every container so agents can
|
|
form qualified labels (`iris@pr1ma.example.com`). `hiveName` and
|
|
`swarmName` are purely display — they surface in the dashboard chrome
|
|
header and per-agent system prompts. Federated hives at different
|
|
domains can share a `swarmName`.
|
|
|
|
See `docs/conventions.md` § Hive identity for the env-var chain
|
|
and `qualify()` / `qualified_label()` semantics.
|
|
|
|
## Declaring peer hives
|
|
|
|
```nix
|
|
services.hyperhive.swarm.peers = {
|
|
"lab.example.com" = { }; # CA-trusted (Let's Encrypt etc.)
|
|
"edge.corp" = { certFingerprint = "sha256:…"; }; # self-signed TLS
|
|
};
|
|
```
|
|
|
|
The attrset key is the peer's DNS domain. `certFingerprint` is
|
|
optional:
|
|
|
|
- **Omitted / null** — the system CA bundle validates the peer's TLS
|
|
cert. Correct for peers with Let's Encrypt or any standard CA cert.
|
|
- **Set** (`"sha256:…"`) — pin a specific cert fingerprint. Use this
|
|
for peers whose self-signed TLS cert doesn't chain to a CA your
|
|
host trusts.
|
|
|
|
The nix module serialises the attrset to a `HYPERHIVE_PEERS` JSON
|
|
array (`[{ domain, cert_fingerprint }]`) injected into the c0re
|
|
environment and forwarded to agent containers.
|
|
|
|
## What the config does at runtime
|
|
|
|
1. **Dashboard P33RS tab** — `parse_peer_hives()` in `dashboard.rs`
|
|
reads `HYPERHIVE_PEERS` and includes
|
|
`peer_hives: Vec<{ name, url }>` in `/api/state`. The dashboard
|
|
shows a P33RS tab (hidden when the list is empty) with a card per
|
|
peer linking to `https://{domain}/`. See
|
|
`docs/web-ui/dashboard.md` § P33RS tab.
|
|
|
|
2. **Agent identity** — the same `HYPERHIVE_PEERS` env var is
|
|
forwarded to agent containers by `meta.rs`; agent code can call
|
|
`identity::peers()` to discover peer hives and address them with
|
|
qualified names (`agent@domain`).
|
|
|
|
3. **Matrix federation** — when `matrix.enable` is on, tuwunel
|
|
federates with the peer's matrix server at
|
|
`matrix.{peer-domain}:8448`. No extra config needed; federation
|
|
works as soon as the domains are reachable and TLS validates.
|
|
See `docs/matrix.md` for federation firewall requirements.
|
|
|
|
## Bilateral setup
|
|
|
|
Each hive must declare the other. If hive A lists hive B as a peer,
|
|
B must also list A for agents on B to see A in their peer list:
|
|
|
|
```
|
|
# hive A (pr1ma.example.com)
|
|
services.hyperhive.swarm.peers."edge.corp" = { };
|
|
|
|
# hive B (edge.corp)
|
|
services.hyperhive.swarm.peers."pr1ma.example.com" = { certFingerprint = "sha256:…"; };
|
|
```
|
|
|
|
Mixed trust is fine: A trusts B via CA bundle (no fingerprint), B
|
|
pins A's self-signed cert.
|
|
|
|
## WireGuard inter-hive mesh (optional)
|
|
|
|
The peer config above uses public HTTPS for all inter-hive traffic.
|
|
For private deployments — or to reduce latency and TLS overhead on
|
|
intra-swarm traffic — hive-c0re can configure a host-to-host
|
|
WireGuard mesh.
|
|
|
|
### Generating keys
|
|
|
|
On each hive host:
|
|
|
|
```bash
|
|
wg genkey | install -m 0400 /dev/stdin /etc/wireguard/hive.key
|
|
wg pubkey < /etc/wireguard/hive.key # → share this with peer operators
|
|
```
|
|
|
|
### Config example (two hives)
|
|
|
|
```nix
|
|
# hive A (pr1ma.example.com, mesh IP 10.100.0.1)
|
|
services.hyperhive = {
|
|
swarm.wireguard = {
|
|
enable = true;
|
|
privateKeyFile = "/etc/wireguard/hive.key";
|
|
address = "10.100.0.1/24";
|
|
listenPort = 51820; # optional, default 51820
|
|
};
|
|
|
|
swarm.peers."edge.corp" = {
|
|
certFingerprint = "sha256:…"; # TLS trust (unchanged)
|
|
wireguardPublicKey = "base64key="; # peer's wg pubkey
|
|
wireguardEndpoint = "203.0.113.42:51820"; # peer's public IP:port
|
|
wireguardAddress = "10.100.0.2/32"; # peer's mesh IP
|
|
};
|
|
};
|
|
|
|
# hive B (edge.corp, mesh IP 10.100.0.2)
|
|
services.hyperhive = {
|
|
swarm.wireguard = {
|
|
enable = true;
|
|
privateKeyFile = "/etc/wireguard/hive.key";
|
|
address = "10.100.0.2/24";
|
|
};
|
|
|
|
swarm.peers."pr1ma.example.com" = {
|
|
wireguardPublicKey = "base64key="; # hive A's wg pubkey
|
|
wireguardEndpoint = "198.51.100.1:51820";
|
|
wireguardAddress = "10.100.0.1/32";
|
|
};
|
|
};
|
|
```
|
|
|
|
### What the mesh does
|
|
|
|
- `networking.wireguard.interfaces.wg-hive` is configured on the host
|
|
(not inside agent containers; containers reach peers via the host's
|
|
routing table).
|
|
- UDP port 51820 (or `listenPort`) is opened on the host firewall.
|
|
- `HYPERHIVE_PEERS` gains a `wireguard_address` field for each mesh
|
|
peer so hive-c0re can reach intra-swarm services without a public
|
|
DNS round-trip.
|
|
- `persistentKeepalive = 25` is set by default; override or null to
|
|
disable (not needed when both sides have public IPs and no NAT).
|
|
|
|
### NAT / one-sided endpoints
|
|
|
|
If one host is behind NAT and can't accept incoming connections, only
|
|
that host needs a null `wireguardEndpoint` on the peer config — the
|
|
other side initiates. With keepalive on, the NAT hole stays open.
|
|
|
|
If both hosts are behind NAT, a STUN relay or a third host (exit node)
|
|
is required. Out of scope for v0.
|
|
|
|
## Cross-references
|
|
|
|
- `docs/conventions.md` § Hive identity — env vars, qualified labels
|
|
- `docs/matrix.md` — matrix federation, TLS cert auto-generation,
|
|
firewall posture
|
|
- `docs/web-ui/dashboard.md` § P33RS tab — dashboard surface
|
|
- `docs/gateway.md` — nginx vhosts and the `.well-known/matrix/`
|
|
auto-discovery scheme
|