network: fix stale assertion messages; move prose to docs/network.md

This commit is contained in:
atlas 2026-06-03 09:39:53 +02:00 committed by mara
commit b89c5f5334
2 changed files with 61 additions and 63 deletions

View file

@ -95,6 +95,52 @@ agent containers.
interface only. Other interfaces stay closed. The hive resolver
isn't an external-facing service.
## Container isolation
`services.hyperhive.network.isolateContainers` (default `false`) flips
agent containers from shared host netns to private netns. Set only after
`enable = true` is stable in production — an assertion blocks the reverse.
### What the nix side does when `isolateContainers = true`
| effect | mechanism |
|---|---|
| IP forwarding | `boot.kernel.sysctl."net.ipv4.ip_forward" = 1` |
| Internet NAT | `networking.nat { enable = true; internalInterfaces = [ bridgeName ]; }` — MASQUERADE on packets leaving via any external NIC |
| Loopback DROP | `networking.firewall.extraInputRules` — drops bridge-subnet → `127.0.0.0/8` traffic; defence-in-depth against routing table leaks |
| c0re signal | `HIVE_NETWORK_ISOLATION=1`, `HIVE_NETWORK_BRIDGE`, `HIVE_NETWORK_SUBNET` in `systemd.services.hive-c0re.environment` |
`HIVE_NETWORK_SUBNET` is the host-side bridge IP + prefix (e.g.
`10.42.0.1/24`), **not** the canonical network address. The Rust side
must normalise (bitwise-AND with mask) before subnet membership checks or
address arithmetic.
### What the Rust side does
`hive-c0re` reads `HIVE_NETWORK_ISOLATION` and, when set, passes
`PRIVATE_NETWORK=1`, `LOCAL_ADDRESS=<deterministic-ip>`, and
`HOST_BRIDGE=<bridgeName>` via `lifecycle::set_nspawn_flags` when
creating or updating containers. Each agent gets a deterministic IP
derived from its name so the address is reproducible across destroy/recreate.
### Prerequisites before flipping on
- All agents must have `hyperhive.web.useUnixSocket = true`. Agents that
still bind TCP on `0.0.0.0:<port>` will be reachable at their bridge IP
from other agents on the same subnet — defeating the isolation goal. The
gateway routes via unix sockets so gateway reach is unaffected.
### Migration behaviour
Containers are destroyed and re-created when the flag flips. Agent state
under `/agents/<name>/state/` is bind-mounted and survives; the container
rootfs is recreated cleanly from the nix store.
### Follow-up
Issue #1119 tracks defaulting `isolateContainers` to `true` once the
bridge is stable in production.
## Cross-references
- `docs/gateway.md` — vhost map + the gateway container's other duties