diff --git a/docs/network.md b/docs/network.md index 40128d26..b3c2a7ad 100644 --- a/docs/network.md +++ b/docs/network.md @@ -10,69 +10,6 @@ whenever hyperhive is enabled. Configured via > `services.hyperhive.network.upstreamDns` options were removed; a > config that still sets one fails eval with a removal message. -## Network map - -One picture of the whole hive. There are two planes: **infra -containers share the host netns** and bind host ports directly; -**compute containers (agents + CI) each get a private netns** behind -the bridge. The unix-socket control plane rides the VFS and is -untouched by any of it. - -``` - internet - │ uplink NIC — NAT MASQUERADE for the - │ bridge subnet (10.42.0.0/24 default) -┌──────────────────────────┴─────────────────────────────────────────┐ -│ host netns — the host itself plus gateway / forge / matrix │ -│ │ -│ nginx :80/:443 [hive-gateway] │ -│ dnsmasq 10.42.0.1:53 (DNS) + :67 (DHCP) [hive-gateway] │ -│ forgejo :3000 http, :2222 git-ssh [hive-forge] │ -│ tuwunel :8008 client API [hive-matrix] │ -│ hive-c0re dashboard 127.0.0.1:7000 (host service) │ -│ wg-hive :51820/udp — swarm mesh, when enabled (host iface) │ -│ │ -│ hive-br0 10.42.0.1/24 │ -│ ┌──────────┼──────────────┐ │ -└──────────────┼──────────┼──────────────┼───────────────────────────┘ - vb-h- vb-h- vb-hive-ci veth pairs - │ │ │ - ┌────┴────┐ ┌───┴─────┐ ┌──────┴──┐ one private netns - │ agent a │ │ agent b │ │ hive-ci │ each; eth0 leases - │ eth0 │ │ eth0 │ │ eth0 │ from the DHCP pool - └─────────┘ └─────────┘ └─────────┘ -``` - -| container | netns | IPv4 | listens / reached via | -| -------------- | ----------------------- | -------------------- | -------------------------------------------------------------------------------------------- | -| `hive-gateway` | host (shared) | host addresses | nginx `:80`/`:443` (every vhost); dnsmasq `bridgeIp:53` + DHCP `:67` on the bridge | -| `hive-forge` | host (shared) | host addresses | forgejo `:3000` http, `:2222` git-ssh; fronted by the `forge.` vhost | -| `hive-matrix` | host (shared) | host addresses | tuwunel `:8008` (+ optional federation port); fronted by the matrix vhost | -| `hive-ci` | private, veth on bridge | DHCP pool | outbound only (runner → forge); no inbound surface | -| `h-` | private, veth on bridge | DHCP pool | web UI via UDS `/run/hive-agent/` → nginx sub-path; in-container UI port hashed 8100–8999 | - -The flows, end to end: - -- **DHCP** — agent `dhcpcd` broadcasts on `eth0` → veth → bridge → - host firewall (udp 67 hole) → dnsmasq pool → lease + router option. -- **DNS** — agents query `bridgeIp:53`; hive zones are answered - authoritatively with the bridge IP, everything else forwards to the - host's resolvers (see *Resolver behaviour* below). -- **HTTP** — `forge.` / matrix / dashboard names all resolve - to the bridge IP, land on nginx `:80`/`:443`, and proxy to forgejo - `:3000`, tuwunel `:8008`, hive-c0re `127.0.0.1:7000`, or a per-agent - UI unix socket. -- **Internet egress** — agent default route points at the bridge IP; - the host forwards + masquerades out its uplink. -- **Swarm** — peer hives connect over the `wg-hive` WireGuard mesh - and reach each other's gateway/forge across it - ([`docs/swarm.md`](swarm.md)). -- **Control plane (no network)** — per-agent broker socket - `/run/hive/mcp.sock`, privileged helper `/run/hive/priv.sock`, - operator admin `/run/hyperhive/host.sock`, and the per-agent UI - sockets under `/run/hive-agent/` are unix domain sockets - bind-mounted through the VFS; private netns does not affect them. - ## Container shape (where dnsmasq lives) Co-located in the existing `hive-gateway` container — single @@ -145,17 +82,6 @@ agent containers. container, shared host netns) for the forge sub-domain, per-agent UI proxies, and any other HTTP services. -The **host** firewall is the only firewall. The shared-netns infra -containers (gateway, forge, matrix) set -`networking.firewall.enable = false`: a NixOS firewall inside a -shared-netns container runs against the *host* ruleset — at container -boot its `firewall-start` flushes the `nixos-fw` chains, rebuilds them -from the container's (empty) port list, and deletes the host's -`nixos-nat-*` chains without recreating them, silently wiping the -bridge holes above plus the agents' NAT. Private-netns containers -(agents, hive-ci) may keep their own firewall — it is scoped to their -namespace. - ### Reaching host services (`exposeHostPorts`) By default agents can only reach the host on 80/443 (+53 DNS), so a diff --git a/nix/host-modules/hive-forge/default.nix b/nix/host-modules/hive-forge/default.nix index bdef1cc1..0bdab6f6 100644 --- a/nix/host-modules/hive-forge/default.nix +++ b/nix/host-modules/hive-forge/default.nix @@ -327,14 +327,6 @@ in in { system.stateVersion = "25.11"; - - # Shared host netns: this container's own firewall.service - # would rewrite the HOST ruleset (flush nixos-fw, drop the - # host's nixos-nat-* chains) at every boot — killing the - # bridge DHCP/DNS holes and agent NAT. The host firewall owns - # all filtering; never run one in here. - networking.firewall.enable = false; - services.forgejo = { enable = true; package = cfg.package; diff --git a/nix/host-modules/hive-gateway/default.nix b/nix/host-modules/hive-gateway/default.nix index e8a6b55b..24555a1f 100644 --- a/nix/host-modules/hive-gateway/default.nix +++ b/nix/host-modules/hive-gateway/default.nix @@ -163,15 +163,6 @@ in { system.stateVersion = "26.05"; - # This container shares the host netns, so its own - # firewall.service would run against the HOST ruleset: flush - # the nixos-fw chains, rebuild them from this container's - # (empty) port list, and delete the host's nixos-nat-* chains - # — wiping the bridge DHCP/DNS holes and the agents' NAT on - # every container boot. The host firewall owns all filtering; - # never run one in here. - networking.firewall.enable = false; - # Keep the host-copied /etc/resolv.conf intact. nixos-container # copies the host's file in at every container start, but # resolvconf's host-tracking mode then regenerates it — to an diff --git a/nix/host-modules/hive-matrix.nix b/nix/host-modules/hive-matrix.nix index 98fed0db..1dc5d65d 100644 --- a/nix/host-modules/hive-matrix.nix +++ b/nix/host-modules/hive-matrix.nix @@ -341,13 +341,6 @@ in { system.stateVersion = "26.05"; - # Shared host netns: this container's own firewall.service - # would rewrite the HOST ruleset (flush nixos-fw, drop the - # host's nixos-nat-* chains) at every boot — killing the - # bridge DHCP/DNS holes and agent NAT. The host firewall owns - # all filtering; never run one in here. - networking.firewall.enable = false; - # Peer-hive root CAs (`swarm.peers..caCert`) added to THIS # container's trust bundle so tuwunel validates *federation* TLS # from a self-signed peer hive (it checks the peer's federation