docs: restructure into topic subdirectories, collapse duplicated index
Per mara's go-ahead on hyperhive#3902 ("getting started is good, but
terminal rendering does not go in there i think"):
Moved 21 top-level docs/*.md files into 7 new topic subdirectories
(existing web-ui/, turn-loop/, swarm/, tools/, crates/ untouched):
getting-started/ setup.md
agent-lifecycle/ agent-hierarchy.md, approvals.md, persistence.md
trust-boundary/ boundary.md, security.md
integrations/ forge.md, matrix.md, github.md, knowledge.md
networking/ gateway.md, network.md, snapshot-store.md
scheduler/ jobq.md, coordinator.md, ci.md, observability.md
process/ conventions.md, gotchas.md, pr-review-gate.md
web-ui/ terminal-rendering.md (moved into the EXISTING dir,
per mara's correction to the original getting-started
guess -- it's UI implementation detail, not onboarding)
The physical layout now matches docs/README.md's own topical headers,
which already amounted to this taxonomy -- see the scoping comment on
the issue for the two findings that motivated this (a genuine
duplication between CLAUDE.md's old "Reading paths" list and
docs/README.md's grouped one, since drifted out of sync with each
other; and the flat layout not matching the grouping we already had).
Fixed every cross-reference this moved across the whole repo (~120
files: docs/ internal links at every depth, Rust doc comments, nix
module option docs, crate READMEs) -- verified two ways: a grep sweep
confirming zero remaining references to any old path, and a script
that resolves every markdown link in docs/**/*.md + CLAUDE.md +
README.md against the filesystem and reports anything that doesn't
exist (zero broken links).
Collapsed CLAUDE.md's "Reading paths" section (the duplicate) down to
a pointer at docs/README.md, now the single index. Rewrote
docs/README.md itself to use the new subdirectory paths and added the
one doc it was missing that CLAUDE.md's old copy had (pr-review-gate.md).
Classified all 22 docs/*.md files first via a haiku subagent (mara's
suggestion) on two axes -- proposed grouping and operator-vs-
implementation focus -- before finalizing the taxonomy; spot-checked
the report and found internal inconsistencies (its classification
table disagreed with its own summary section for a few files), so this
taxonomy is my original proposal + the one correction mara gave
directly, not a blind application of the subagent's table. The
operator-focus data it gathered is still useful for a follow-up
content pass (docs skewing 'mixed' rather than pure operator-facing),
not addressed in this PR -- structure only.
nix fmt clean, both pre-push lints clean.
This commit is contained in:
iris
mara
parent
e4a22b4190
commit
07b62612b0
124 changed files with 301 additions and 377 deletions
277
docs/network.md
277
docs/network.md
|
|
@ -1,277 +0,0 @@
|
|||
# hive-network
|
||||
|
||||
Host-side bridge + per-agent private-netns isolation — always on
|
||||
whenever hyperhive is enabled. Configured via
|
||||
`services.hyperhive.network.*`.
|
||||
|
||||
> Isolation is the only mode — there is no shared-netns fallback. The
|
||||
> former `services.hyperhive.network.enable`,
|
||||
> `services.hyperhive.network.isolateContainers` and
|
||||
> `services.hyperhive.network.upstreamDns` options were removed; a
|
||||
> config that still sets one fails eval with a removal message.
|
||||
|
||||
## Network map
|
||||
|
||||
One picture of the whole hive. There are two planes: **infra
|
||||
containers share the host netns** and bind host ports directly;
|
||||
**compute containers (agents + CI) each get a private netns** behind
|
||||
the bridge. The unix-socket control plane rides the VFS and is
|
||||
untouched by any of it.
|
||||
|
||||
```
|
||||
internet
|
||||
│ uplink NIC — NAT MASQUERADE for the
|
||||
│ bridge subnet (10.42.0.0/24 default)
|
||||
┌──────────────────────────┴─────────────────────────────────────────┐
|
||||
│ host netns — the host itself plus gateway / forge / matrix │
|
||||
│ │
|
||||
│ nginx :80/:443 [hive-gateway] │
|
||||
│ dnsmasq 10.42.0.1:53 (DNS) + :67 (DHCP) [hive-gateway] │
|
||||
│ forgejo :3000 http, :2222 git-ssh [hive-forge] │
|
||||
│ tuwunel :8008 client API [hive-matrix] │
|
||||
│ hive-c0re dashboard 127.0.0.1:7000 (host service) │
|
||||
│ wg-hive :51820/udp — swarm mesh, when enabled (host iface) │
|
||||
│ │
|
||||
│ hive-br0 10.42.0.1/24 │
|
||||
│ ┌──────────┼──────────────┐ │
|
||||
└──────────────┼──────────┼──────────────┼───────────────────────────┘
|
||||
vb-h-<a> vb-h-<b> vb-hive-ci veth pairs
|
||||
│ │ │
|
||||
┌────┴────┐ ┌───┴─────┐ ┌──────┴──┐ one private netns
|
||||
│ agent a │ │ agent b │ │ hive-ci │ each; eth0 leases
|
||||
│ eth0 │ │ eth0 │ │ eth0 │ from the DHCP pool
|
||||
└─────────┘ └─────────┘ └─────────┘
|
||||
```
|
||||
|
||||
| container | netns | IPv4 | listens / reached via |
|
||||
| -------------- | ----------------------- | -------------------- | -------------------------------------------------------------------------------------------- |
|
||||
| `hive-gateway` | host (shared) | host addresses | nginx `:80`/`:443` (every vhost); dnsmasq `bridgeIp:53` + DHCP `:67` on the bridge |
|
||||
| `hive-forge` | host (shared) | host addresses | forgejo `:3000` http, `:2222` git-ssh; fronted by the `forge.<swarm-domain>` vhost |
|
||||
| `hive-matrix` | host (shared) | host addresses | tuwunel `:8008` (+ optional federation port); fronted by the matrix vhost |
|
||||
| `hive-ci` | private, veth on bridge | DHCP pool | outbound only (runner → forge); no inbound surface |
|
||||
| `h-<agent>` | private, veth on bridge | DHCP pool | web UI via UDS `/run/hive-agent/<name>` → nginx sub-path; in-container UI port hashed 8100–8999 |
|
||||
|
||||
The flows, end to end:
|
||||
|
||||
- **DHCP** — agent `dhcpcd` broadcasts on `eth0` → veth → bridge →
|
||||
host firewall (udp 67 hole) → dnsmasq pool → lease + router option.
|
||||
- **DNS** — agents and the service containers query `bridgeIp:53`; hive
|
||||
zones are answered authoritatively with the bridge IP, everything else
|
||||
forwards to the host's resolvers (see *Resolver behaviour* below). Each
|
||||
container points its own `resolv.conf` there, and one that instead
|
||||
inherits the host's resolves no swarm name at all — those records exist
|
||||
only on the bridge.
|
||||
- **HTTP** — `forge.` and `chat.` (under `swarm.domain`) plus the hive's
|
||||
own dashboard name resolve to the bridge IP, land on nginx
|
||||
`:80`/`:443`, and proxy to forgejo
|
||||
`:3000`, tuwunel `:8008`, hive-c0re `127.0.0.1:7000`, or a per-agent
|
||||
UI unix socket.
|
||||
- **Internet egress** — agent default route points at the bridge IP;
|
||||
the host forwards + masquerades out its uplink.
|
||||
- **Swarm** — peer hives connect over the `wg-hive` WireGuard mesh
|
||||
and reach each other's gateway/forge across it
|
||||
([`docs/swarm/`](swarm/README.md)).
|
||||
- **Control plane (no network)** — per-agent broker socket
|
||||
`/run/hive/mcp.sock`, privileged helper `/run/hive/priv.sock`,
|
||||
operator admin `/run/hyperhive/host.sock`, and the per-agent UI
|
||||
sockets under `/run/hive-agent/` are unix domain sockets
|
||||
bind-mounted through the VFS; private netns does not affect them.
|
||||
|
||||
## Container shape (where dnsmasq lives)
|
||||
|
||||
Co-located in the existing `hive-gateway` container — single
|
||||
front-door for both DNS and HTTP, saves a sibling container, single
|
||||
systemd-unit / state surface to monitor. The gateway shares host
|
||||
netns (`privateNetwork = false`) so dnsmasq's `bind-interfaces`
|
||||
listener on `bridgeIp` is on the host's bridge interface.
|
||||
|
||||
## Configuration
|
||||
|
||||
```nix
|
||||
{
|
||||
services.hyperhive = {
|
||||
enable = true;
|
||||
hiveName = "pr1ma";
|
||||
swarm.domain = "darkest.space";
|
||||
swarm.hives.pr1ma = { }; # -> domain = pr1ma.darkest.space
|
||||
# network.bridgeIp = "10.42.0.1"; # default
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
Requires `services.hyperhive.domain` to be set — the dnsmasq resolver
|
||||
is authoritative for `<hive-domain>` and its sub-domains. You do not
|
||||
write it: it is read from this hive's entry in the swarm directory
|
||||
(`docs/swarm/README.md` § Hive identity config).
|
||||
|
||||
## Bridge addressing
|
||||
|
||||
Default subnet is `10.42.0.0/24`, host-side gateway at `10.42.0.1`.
|
||||
RFC 1918 space, unlikely to clash with operator's existing setup;
|
||||
override `bridgeIp` + `bridgePrefixLength` if a different range is
|
||||
already in use. `/24` gives 254 usable per-agent addresses — enough
|
||||
for any single-host hive; bigger swarms or tighter addressing
|
||||
schemes pick their own.
|
||||
|
||||
## Resolver behaviour
|
||||
|
||||
dnsmasq is **authoritative** for the hive's own zone (`<hive-domain>`)
|
||||
plus whatever swarm-service names this host contributes via
|
||||
`gateway.localNames` — `forge.<swarm-domain>` and `chat.<swarm-domain>`
|
||||
(matrix) when this host runs those services, and `auth.<swarm-domain>`
|
||||
when it runs authelia — answering each with the bridge IP (where nginx
|
||||
is reachable). Note forge and matrix are swarm-domain names, not
|
||||
sub-domains of `<hive-domain>`: a swarm runs one forge and one
|
||||
homeserver, so their names belong to the swarm rather than to whichever
|
||||
hive happens to host them. Everything else is forwarded to the host's
|
||||
own resolvers: dnsmasq runs on the host and reads the host's
|
||||
`/etc/resolv.conf` directly. Containers don't need to know the
|
||||
upstream — they query the bridge IP and dnsmasq does the right thing
|
||||
per-name.
|
||||
|
||||
There is deliberately no fallback `server=`: dnsmasq queries all known
|
||||
upstreams in parallel, so a hardcoded public resolver would take a share
|
||||
of normal traffic, not just cover the gap.
|
||||
|
||||
dnsmasq runs on the host and reads the host's `/etc/resolv.conf`
|
||||
directly, so a network change (new router, new lease, laptop moving
|
||||
networks) reaches it the moment openresolv rewrites the file. There is
|
||||
nothing to synchronise and no unit watching for it.
|
||||
|
||||
`bind-interfaces` + `interface = [ bridgeName "lo" ]` means the
|
||||
listener only accepts queries from the bridge interface (plus lo for
|
||||
container health-checks). External hosts can't reach it — no
|
||||
DNS-amplification surface even when the operator opens port 80 for
|
||||
gateway HTTP.
|
||||
|
||||
`resolveLocalQueries = false` keeps dnsmasq out of the host's own
|
||||
resolution stack — the host's resolver (systemd-resolved, plain
|
||||
glibc nss, dnscrypt-proxy, etc.) keeps doing whatever the operator
|
||||
configured. The hive resolver is purely for inbound queries from
|
||||
agent containers.
|
||||
|
||||
## Firewall posture
|
||||
|
||||
`networking.firewall.interfaces.<bridge>.allowedUDPPorts = [ 53 67 ]`
|
||||
`networking.firewall.interfaces.<bridge>.allowedTCPPorts = [ 53 80 443 ]`
|
||||
|
||||
- Port 53 opens the resolver on the bridge interface only. Other
|
||||
interfaces stay closed. The hive resolver isn't an external-facing
|
||||
service.
|
||||
- Port 67 (UDP) admits DHCP requests to the dnsmasq pool. dnsmasq
|
||||
receives DHCP via a regular UDP socket (it does not use a
|
||||
netfilter-bypassing raw socket), so the hole is mandatory — without
|
||||
it containers never get a lease and fall back to 169.254.x.x.
|
||||
- Ports 80 and 443 let isolated agents reach nginx (gateway
|
||||
container, shared host netns) for the forge sub-domain, per-agent
|
||||
UI proxies, and any other HTTP services.
|
||||
|
||||
The **host** firewall is the only firewall. The shared-netns infra
|
||||
containers (gateway, forge, matrix) set
|
||||
`networking.firewall.enable = false`: a NixOS firewall inside a
|
||||
shared-netns container runs against the *host* ruleset — at container
|
||||
boot its `firewall-start` flushes the `nixos-fw` chains, rebuilds them
|
||||
from the container's (empty) port list, and deletes the host's
|
||||
`nixos-nat-*` chains without recreating them, silently wiping the
|
||||
bridge holes above plus the agents' NAT. Private-netns containers
|
||||
(agents, hive-ci) may keep their own firewall — it is scoped to their
|
||||
namespace.
|
||||
|
||||
### Reaching host services (`exposeHostPorts`)
|
||||
|
||||
By default agents can only reach the host on 80/443 (+53 DNS), so a
|
||||
host-side service on another port — e.g. a dev OTLP collector you want
|
||||
agents to reach directly — is unreachable. (hyperhive's own telemetry
|
||||
needs none of this: `otel.enable` opens its collector's port itself, and
|
||||
`otel.endpoint` is the *upstream*, which no agent ever dials. See
|
||||
`docs/observability.md`.)
|
||||
|
||||
`services.hyperhive.network.exposeHostPorts = [ 4318 ];` opens each
|
||||
listed TCP port `P` on the bridge-interface `allowedTCPPorts`, so an
|
||||
agent can connect to `<bridgeIp>:P` (point the collector endpoint at
|
||||
`http://<bridgeIp>:4318`, default `http://10.42.0.1:4318`).
|
||||
|
||||
This is **firewall-only**: the host service must bind an address
|
||||
reachable from the bridge — `0.0.0.0` or the bridge IP — not loopback
|
||||
only. The bridge→`127.0.0.0/8` DROP rule (below) is unchanged, so a
|
||||
service bound to `127.0.0.1` only stays unreachable; rebind it to
|
||||
`0.0.0.0`.
|
||||
|
||||
The port is reachable by **every** agent on the bridge subnet (like
|
||||
DNS/gateway), so only expose services safe for any agent to reach.
|
||||
|
||||
## Container isolation
|
||||
|
||||
Each agent container runs in a private network namespace with a dedicated
|
||||
veth pair attached to the bridge. The following table summarises what
|
||||
the nix side sets up unconditionally:
|
||||
|
||||
| effect | mechanism |
|
||||
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| IP forwarding | `boot.kernel.sysctl."net.ipv4.ip_forward" = 1` |
|
||||
| Internet NAT | `networking.nat { enable = true; internalInterfaces = [ bridgeName ]; }` — MASQUERADE on packets leaving via any external NIC |
|
||||
| Loopback DROP | `networking.firewall.extraInputRules` — drops bridge-subnet → `127.0.0.0/8` traffic; defence-in-depth against routing table leaks |
|
||||
| Gateway access | `networking.firewall.interfaces.<bridge>.allowedTCPPorts = [ 80 443 ]` — lets isolated agents (private netns, veth on bridge) reach nginx on the host |
|
||||
| c0re signal | `HIVE_NETWORK_BRIDGE`, `HIVE_NETWORK_SUBNET` in `systemd.services.hive-c0re.environment` — both **required**; `hive-c0re` refuses to start without them |
|
||||
|
||||
`HIVE_NETWORK_SUBNET` is the host-side bridge IP + prefix (e.g.
|
||||
`10.42.0.1/24`), **not** the canonical network address. The Rust side
|
||||
must normalise (bitwise-AND with mask) before subnet membership checks or
|
||||
address arithmetic.
|
||||
|
||||
### What the Rust side does
|
||||
|
||||
`hive-c0re` reads `HIVE_NETWORK_BRIDGE` + `HIVE_NETWORK_SUBNET` and passes
|
||||
`PRIVATE_NETWORK=1`, `LOCAL_ADDRESS=` (empty), `HOST_ADDRESS=<bridge-ip>`,
|
||||
and `HOST_BRIDGE=<bridgeName>` via `lifecycle::set_nspawn_flags` when
|
||||
creating or updating containers. Both variables are validated **once at
|
||||
daemon startup**, not per container: they are process-global, so a
|
||||
missing or malformed value is a misconfigured daemon rather than one bad
|
||||
container, and failing at boot gives a single diagnostic instead of one
|
||||
per agent. There is no non-isolated mode to fall back to. `LOCAL_ADDRESS` is left empty so the
|
||||
container's dhcpcd acquires an address from the bridge dnsmasq pool
|
||||
(`networking.useDHCP = true` in `nix/agent-modules/network.nix`). This applies uniformly
|
||||
to all containers — agents and service containers alike.
|
||||
|
||||
`HOST_ADDRESS` is the bridge gateway IP (the address part of
|
||||
`HIVE_NETWORK_SUBNET`, via `lifecycle::bridge_gateway_ip` — taken verbatim
|
||||
so a non-`.1` operator override still resolves to wherever the bridge
|
||||
actually lives). It is **load-bearing**: nixos-container's container-side
|
||||
network setup only installs a default route (`ip route add default via
|
||||
$HOST_ADDRESS`) when `HOST_ADDRESS` is non-empty. In bridge mode the
|
||||
host-side address/route setup is skipped, so writing it only affects the
|
||||
container's default route — without it the container comes up with an IP
|
||||
but no path off the bridge subnet (no internet, no `api.anthropic.com`).
|
||||
|
||||
### How the isolated container gets its resolver
|
||||
|
||||
nixos-container copies the **host's** `/etc/resolv.conf` into the container
|
||||
at every start. The host resolver (e.g. `127.0.0.53` from systemd-resolved,
|
||||
or a LAN router) is unreachable from a private netns and isn't
|
||||
authoritative for the hive's own zones, so it is replaced with the
|
||||
bridge dnsmasq at boot. Because the copy happens on every start, a
|
||||
declarative `environment.etc."resolv.conf"` would be clobbered — so the
|
||||
wiring is runtime:
|
||||
|
||||
- `hive-priv` drops a marker file (`/etc/hyperhive-bridge-dns`, carrying the
|
||||
gateway IP) into each container's `/etc`.
|
||||
- the `hyperhive-isolated-dns` oneshot (`nix/agent-modules/network.nix`), gated on that
|
||||
marker, rewrites `/etc/resolv.conf` to `nameserver <gateway-ip>` at boot.
|
||||
It is ordered `before` the harness (`hive-ag3nt`), the matrix daemon, and
|
||||
`tea-login` so the resolver is correct before the first DNS lookup.
|
||||
|
||||
**Why isolation is safe**: hive-c0re's control-plane sockets are unix
|
||||
domain sockets bind-mounted into containers, not network listeners — see
|
||||
the *Control plane (no network)* bullet under [Network
|
||||
map](#network-map) above. `PRIVATE_NETWORK=1` has no effect on a path
|
||||
that never touches the network stack.
|
||||
|
||||
The nix side also enables IP forwarding + NAT (agents reach the internet
|
||||
through the host) and drops bridge-subnet → loopback traffic (defence-in-depth
|
||||
against a compromised agent reaching the c0re dashboard HTTP at
|
||||
`127.0.0.1`). Agents have no legitimate reason to reach the dashboard over
|
||||
loopback — the hive-c0re admin socket is a UDS, not TCP.
|
||||
|
||||
## Cross-references
|
||||
|
||||
- `docs/gateway.md` — vhost map + the gateway's other duties
|
||||
Loading…
Reference in a new issue