network: fix stale assertion messages; move prose to docs/network.md
This commit is contained in:
parent
3db51deace
commit
b89c5f5334
2 changed files with 61 additions and 63 deletions
|
|
@ -95,6 +95,52 @@ agent containers.
|
|||
interface only. Other interfaces stay closed. The hive resolver
|
||||
isn't an external-facing service.
|
||||
|
||||
## Container isolation
|
||||
|
||||
`services.hyperhive.network.isolateContainers` (default `false`) flips
|
||||
agent containers from shared host netns to private netns. Set only after
|
||||
`enable = true` is stable in production — an assertion blocks the reverse.
|
||||
|
||||
### What the nix side does when `isolateContainers = true`
|
||||
|
||||
| effect | mechanism |
|
||||
|---|---|
|
||||
| IP forwarding | `boot.kernel.sysctl."net.ipv4.ip_forward" = 1` |
|
||||
| Internet NAT | `networking.nat { enable = true; internalInterfaces = [ bridgeName ]; }` — MASQUERADE on packets leaving via any external NIC |
|
||||
| Loopback DROP | `networking.firewall.extraInputRules` — drops bridge-subnet → `127.0.0.0/8` traffic; defence-in-depth against routing table leaks |
|
||||
| c0re signal | `HIVE_NETWORK_ISOLATION=1`, `HIVE_NETWORK_BRIDGE`, `HIVE_NETWORK_SUBNET` in `systemd.services.hive-c0re.environment` |
|
||||
|
||||
`HIVE_NETWORK_SUBNET` is the host-side bridge IP + prefix (e.g.
|
||||
`10.42.0.1/24`), **not** the canonical network address. The Rust side
|
||||
must normalise (bitwise-AND with mask) before subnet membership checks or
|
||||
address arithmetic.
|
||||
|
||||
### What the Rust side does
|
||||
|
||||
`hive-c0re` reads `HIVE_NETWORK_ISOLATION` and, when set, passes
|
||||
`PRIVATE_NETWORK=1`, `LOCAL_ADDRESS=<deterministic-ip>`, and
|
||||
`HOST_BRIDGE=<bridgeName>` via `lifecycle::set_nspawn_flags` when
|
||||
creating or updating containers. Each agent gets a deterministic IP
|
||||
derived from its name so the address is reproducible across destroy/recreate.
|
||||
|
||||
### Prerequisites before flipping on
|
||||
|
||||
- All agents must have `hyperhive.web.useUnixSocket = true`. Agents that
|
||||
still bind TCP on `0.0.0.0:<port>` will be reachable at their bridge IP
|
||||
from other agents on the same subnet — defeating the isolation goal. The
|
||||
gateway routes via unix sockets so gateway reach is unaffected.
|
||||
|
||||
### Migration behaviour
|
||||
|
||||
Containers are destroyed and re-created when the flag flips. Agent state
|
||||
under `/agents/<name>/state/` is bind-mounted and survives; the container
|
||||
rootfs is recreated cleanly from the nix store.
|
||||
|
||||
### Follow-up
|
||||
|
||||
Issue #1119 tracks defaulting `isolateContainers` to `true` once the
|
||||
bridge is stable in production.
|
||||
|
||||
## Cross-references
|
||||
|
||||
- `docs/gateway.md` — vhost map + the gateway container's other duties
|
||||
|
|
|
|||
Loading…
Reference in a new issue