From d97659c06afa980fa8dd5d2d1091084e4aad445c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?m=C3=BCde?= Date: Mon, 13 Jul 2026 23:58:54 +0200 Subject: [PATCH 1/2] docs: add the network topology map to network.md --- docs/network.md | 63 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) diff --git a/docs/network.md b/docs/network.md index b3c2a7ad..4fbc8dc9 100644 --- a/docs/network.md +++ b/docs/network.md @@ -10,6 +10,69 @@ whenever hyperhive is enabled. Configured via > `services.hyperhive.network.upstreamDns` options were removed; a > config that still sets one fails eval with a removal message. +## Network map + +One picture of the whole hive. There are two planes: **infra +containers share the host netns** and bind host ports directly; +**compute containers (agents + CI) each get a private netns** behind +the bridge. The unix-socket control plane rides the VFS and is +untouched by any of it. + +``` + internet + │ uplink NIC — NAT MASQUERADE for the + │ bridge subnet (10.42.0.0/24 default) +┌──────────────────────────┴─────────────────────────────────────────┐ +│ host netns — the host itself plus gateway / forge / matrix │ +│ │ +│ nginx :80/:443 [hive-gateway] │ +│ dnsmasq 10.42.0.1:53 (DNS) + :67 (DHCP) [hive-gateway] │ +│ forgejo :3000 http, :2222 git-ssh [hive-forge] │ +│ tuwunel :8008 client API [hive-matrix] │ +│ hive-c0re dashboard 127.0.0.1:7000 (host service) │ +│ wg-hive :51820/udp — swarm mesh, when enabled (host iface) │ +│ │ +│ hive-br0 10.42.0.1/24 │ +│ ┌──────────┼──────────────┐ │ +└──────────────┼──────────┼──────────────┼───────────────────────────┘ + vb-h- vb-h- vb-hive-ci veth pairs + │ │ │ + ┌────┴────┐ ┌───┴─────┐ ┌──────┴──┐ one private netns + │ agent a │ │ agent b │ │ hive-ci │ each; eth0 leases + │ eth0 │ │ eth0 │ │ eth0 │ from the DHCP pool + └─────────┘ └─────────┘ └─────────┘ +``` + +| container | netns | IPv4 | listens / reached via | +| -------------- | ----------------------- | -------------------- | -------------------------------------------------------------------------------------------- | +| `hive-gateway` | host (shared) | host addresses | nginx `:80`/`:443` (every vhost); dnsmasq `bridgeIp:53` + DHCP `:67` on the bridge | +| `hive-forge` | host (shared) | host addresses | forgejo `:3000` http, `:2222` git-ssh; fronted by the `forge.` vhost | +| `hive-matrix` | host (shared) | host addresses | tuwunel `:8008` (+ optional federation port); fronted by the matrix vhost | +| `hive-ci` | private, veth on bridge | DHCP pool | outbound only (runner → forge); no inbound surface | +| `h-` | private, veth on bridge | DHCP pool | web UI via UDS `/run/hive-agent/` → nginx sub-path; in-container UI port hashed 8100–8999 | + +The flows, end to end: + +- **DHCP** — agent `dhcpcd` broadcasts on `eth0` → veth → bridge → + host firewall (udp 67 hole) → dnsmasq pool → lease + router option. +- **DNS** — agents query `bridgeIp:53`; hive zones are answered + authoritatively with the bridge IP, everything else forwards to the + host's resolvers (see *Resolver behaviour* below). +- **HTTP** — `forge.` / matrix / dashboard names all resolve + to the bridge IP, land on nginx `:80`/`:443`, and proxy to forgejo + `:3000`, tuwunel `:8008`, hive-c0re `127.0.0.1:7000`, or a per-agent + UI unix socket. +- **Internet egress** — agent default route points at the bridge IP; + the host forwards + masquerades out its uplink. +- **Swarm** — peer hives connect over the `wg-hive` WireGuard mesh + and reach each other's gateway/forge across it + ([`docs/swarm.md`](swarm.md)). +- **Control plane (no network)** — per-agent broker socket + `/run/hive/mcp.sock`, privileged helper `/run/hive/priv.sock`, + operator admin `/run/hyperhive/host.sock`, and the per-agent UI + sockets under `/run/hive-agent/` are unix domain sockets + bind-mounted through the VFS; private netns does not affect them. + ## Container shape (where dnsmasq lives) Co-located in the existing `hive-gateway` container — single From 0d416df1e8b92fb0100d8fc9ec200aaf85ddc747 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?m=C3=BCde?= Date: Tue, 14 Jul 2026 00:08:33 +0200 Subject: [PATCH 2/2] fix: shared-netns infra containers must not run their own firewall --- docs/network.md | 11 +++++++++++ nix/host-modules/hive-forge/default.nix | 8 ++++++++ nix/host-modules/hive-gateway/default.nix | 9 +++++++++ nix/host-modules/hive-matrix.nix | 7 +++++++ 4 files changed, 35 insertions(+) diff --git a/docs/network.md b/docs/network.md index 4fbc8dc9..40128d26 100644 --- a/docs/network.md +++ b/docs/network.md @@ -145,6 +145,17 @@ agent containers. container, shared host netns) for the forge sub-domain, per-agent UI proxies, and any other HTTP services. +The **host** firewall is the only firewall. The shared-netns infra +containers (gateway, forge, matrix) set +`networking.firewall.enable = false`: a NixOS firewall inside a +shared-netns container runs against the *host* ruleset — at container +boot its `firewall-start` flushes the `nixos-fw` chains, rebuilds them +from the container's (empty) port list, and deletes the host's +`nixos-nat-*` chains without recreating them, silently wiping the +bridge holes above plus the agents' NAT. Private-netns containers +(agents, hive-ci) may keep their own firewall — it is scoped to their +namespace. + ### Reaching host services (`exposeHostPorts`) By default agents can only reach the host on 80/443 (+53 DNS), so a diff --git a/nix/host-modules/hive-forge/default.nix b/nix/host-modules/hive-forge/default.nix index 0bdab6f6..bdef1cc1 100644 --- a/nix/host-modules/hive-forge/default.nix +++ b/nix/host-modules/hive-forge/default.nix @@ -327,6 +327,14 @@ in in { system.stateVersion = "25.11"; + + # Shared host netns: this container's own firewall.service + # would rewrite the HOST ruleset (flush nixos-fw, drop the + # host's nixos-nat-* chains) at every boot — killing the + # bridge DHCP/DNS holes and agent NAT. The host firewall owns + # all filtering; never run one in here. + networking.firewall.enable = false; + services.forgejo = { enable = true; package = cfg.package; diff --git a/nix/host-modules/hive-gateway/default.nix b/nix/host-modules/hive-gateway/default.nix index 24555a1f..e8a6b55b 100644 --- a/nix/host-modules/hive-gateway/default.nix +++ b/nix/host-modules/hive-gateway/default.nix @@ -163,6 +163,15 @@ in { system.stateVersion = "26.05"; + # This container shares the host netns, so its own + # firewall.service would run against the HOST ruleset: flush + # the nixos-fw chains, rebuild them from this container's + # (empty) port list, and delete the host's nixos-nat-* chains + # — wiping the bridge DHCP/DNS holes and the agents' NAT on + # every container boot. The host firewall owns all filtering; + # never run one in here. + networking.firewall.enable = false; + # Keep the host-copied /etc/resolv.conf intact. nixos-container # copies the host's file in at every container start, but # resolvconf's host-tracking mode then regenerates it — to an diff --git a/nix/host-modules/hive-matrix.nix b/nix/host-modules/hive-matrix.nix index 1dc5d65d..98fed0db 100644 --- a/nix/host-modules/hive-matrix.nix +++ b/nix/host-modules/hive-matrix.nix @@ -341,6 +341,13 @@ in { system.stateVersion = "26.05"; + # Shared host netns: this container's own firewall.service + # would rewrite the HOST ruleset (flush nixos-fw, drop the + # host's nixos-nat-* chains) at every boot — killing the + # bridge DHCP/DNS holes and agent NAT. The host firewall owns + # all filtering; never run one in here. + networking.firewall.enable = false; + # Peer-hive root CAs (`swarm.peers..caCert`) added to THIS # container's trust bundle so tuwunel validates *federation* TLS # from a self-signed peer hive (it checks the peer's federation