Watch

atlas 07852cabc1 feat(3088): move the gateway's nginx + dnsmasq onto the host

The gateway's nginx + dnsmasq no longer run in their own nspawn container.
`nix/host-modules/hive-gateway/default.nix` loses the
`containers.hive-gateway` wrapper and everything that existed only to punch
holes in it: `privateNetwork = false`, `CAP_NET_ADMIN`, five bind mounts,
its own `stateVersion`, `networking.firewall.enable = false`,
`networking.resolvconf.enable = false`, and the `hive-gateway-resolv`
path+service pair. 465 -> 303 lines.

The container never bought isolation here. It shared the host netns by
necessity — nginx binds the host's :80/:443, dnsmasq answers on the bridge —
so each of those settings was undoing a boundary the gateway could not
afford in the first place.

Four things made it more than a deletion, none of them visible in the nix
diff:

- The self-signed cert service also imports the hive CA leaf, so removing it
  with the container would have left nginx naming a missing cert file, which
  it refuses to load at all.
- The nginx reload is a hive-priv verb. It still needs root, but no longer
  for the reason its doc gave, and `--machine=` was both transport and
  scope — so the unit name is now hard-coded in the helper as the
  containment.
- The lifecycle verb named a container that stops existing.
- `journalctl -M hive-gateway` had no machine to enter.

Per the operator's ruling, the operator verb keeps working and agents lose
it. `InfraContainer` answered three questions that used to share an answer;
it now splits into `name()` (identity), `target()` (Container vs HostUnit),
`service_unit()` (the systemd unit), and `agent_restartable()`, which the
MCP restart path checks before the capability so the refusal cannot read as
"ask for infra_admin". `SIBLING_CONTAINERS` drops the gateway — it gates the
requests that name a container as a string — while `FromStr` still accepts
it, because that answers what a name is, not who may act on it. The
dashboard's gateway journal reads host journald filtered to `nginx.service`.

Prose was corrected where it only named a location, and re-argued where the
container was doing security work: a `0666` per-agent socket was safe
because only the gateway container had the directory bind-mounted. There is
no mount now, so the directory permissions are the whole of the access
control — the constraint holds, its mechanism doesn't.

Gate: nix fmt / clippy --all-targets -D warnings / cargo test all clean (710
tests); hivectl-cli.md regenerated from the clap tree. The nix eval was run
in both TLS shapes at this commit: every delta in the rendered
virtualHosts is one of the three intended path moves, dnsmasq settings are
byte-identical, and the absence probe flips true -> false with bindMounts
emptied.

2026-08-11 18:01:03 +02:00

8.3 KiB

Raw Blame History

The operator/agent boundary

Design rationale for hyperhive's two-principal trust model. The implementation work — container network isolation, the unifying gateway, core-daemon privsep — is tracked as area:ops issues on the forge.

The operator/agent boundary is now technically enforced, not just a convention. Containers run in private netns (network isolation is always on), the gateway proxies all operator-facing traffic, and hive-c0re runs as the unprivileged hive-core user. All three area:ops pillars — network isolation, the gateway, and privsep — are complete and active.

Two principals, two paths

Operator — reaches every UI (the dashboard + every per-agent page) through the gateway, on one origin. Operator-authority actions (approve / deny, answer-as-operator, lifecycle POSTs) are served by the core daemon and only reachable via the gateway.
Agent — speaks only for itself, only over its per-agent unix socket. The socket's identity is the agent (see docs/conventions.md, "identity = socket"). An agent must not be able to reach the core daemon's HTTP surface, another agent's socket, or another agent's web UI.

Design rule

Operator-authority actions never get a per-agent-socket entry point. They live on the core backend.

Worked example — answering an operator-targeted question is a POST /answer-question/{id} on the core dashboard, never an AgentRequest variant. If it were a per-agent-socket request, an agent could curl its own socket and spoof an operator answer. The per-agent web UI POSTs cross-origin to the core for these (see the inline-answer feature — the loose-ends section on each agent page).

Why network isolation is the load-bearing step

Without network isolation, containers share the host network namespace and can reach localhost:<core-port>, the dashboard, and every other agent's web port — the operator/agent split is on the honour system and every boundary claim above is aspirational. Network isolation is what makes the boundary real; the gateway and privsep are ergonomics and defence-in-depth layered on top.

Network isolation is now complete and always on: every agent container runs in a private netns behind the hive bridge. The shared-netns mode was removed. See docs/network.md.

Concretely, the core daemon's dashboard /api carries no application-layer authentication — operator-authority routes are served unauthenticated at the HTTP layer. Their protection is entirely (a) the gateway, which fronts all operator traffic and is where operator auth lives, and (b) network isolation, which keeps agents — and hive-ci's untrusted PR builds — off host-loopback so nothing can reach 127.0.0.1:<dashboard_port> directly. This is deliberate given the load-bearing role of network isolation above, but it is a standing invariant: the /api must never be bound to a non-loopback address or exposed outside the gateway, and every new operator-authority route inherits that assumption. hive-ci is treated like an agent for this purpose — it runs untrusted PR code and is netns-isolated for the same reason.

The area:ops issues followed this sequencing:

Gateway — pure ergonomics win, unblocks same-origin (lets the cross-origin CORS shim on /answer-question/{id} go away), no behavioural risk. An nginx nixos-container now sits in front of all surfaces; per-agent UIs are proxied under /agent/<name>/.
Network isolation — the load-bearing step that turns the honour-system split into an enforced boundary. Complete — always-on, unconditional; the shared-netns mode was removed.
Privsep — defence in depth on the core process; hive-c0re runs as the unprivileged hive-core user and delegates root operations to hive-priv, a narrow socket-activated helper. See docs/security.md for the privilege boundary table.

hive-priv socket activation

hive-priv is always socket-activated by the hive-priv.socket systemd unit. The unit binds /run/hive/priv.sock with SocketGroup=hive-core and mode 0660 and passes the ready listener to the helper as fd 3 (LISTEN_FDS). The helper requires this and bails if it isn't socket-activated — there is intentionally no self-bind fallback.

Dropping the old fallback removed a dev/prod divergence: when hive-priv bound the socket itself it created the file owned by root's primary group rather than hive-core, so a hive-core client couldn't connect the way the socket unit's SocketGroup grant intends. Requiring socket activation everywhere means dev and prod take the exact same path and the group grant always holds.

the per-agent socket dir

/run/hive-agent/<name>/ is shared by three principals that share no group, which is why its mode is what it is:

principal	reaches	needs
the agent's harness	binds + unlinks `agent.sock`, `web.sock`	owner, `rwx`
`hive-c0re`	dials `agent.sock` (todo wakes)	traverse
the gateway's nginx	dials `web.sock`	traverse

The last two land in "other", so the dir is 0751, owned by the agent's container uid/gid — o=--x is traverse without listing, and both sockets are 0666, which is all a dialer needs.

Ownership is declared, not repaired. The tmpfiles.d entry written by SyncAgentTmpfiles names the uid/gid directly. Do not add a chown alongside it: d re-applies on every boot and every agent spawn/destroy, so ownership set afterwards is reverted the next time any agent changes — which is exactly how this dir spent a long time at 0777 root root while a privileged chown appeared to be fixing it.

The mode is load-bearing, not cosmetic. Write permission on a directory is what confers the right to unlink its entries, whoever owns them, and the sticky bit is the only thing that would restrain that (it is not set here). A world-writable socket dir therefore lets anything able to reach the path delete an agent's socket and bind its own — and nginx reaches all of /run/hive-agent (as a plain host path since the gateway moved out of its container; it used to be bind-mounted in, which was the same reach through a longer route). Dropping o=w removes that permission rather than qualifying it.

⚠️ The gateway leaving its container is a deliberate trade, recorded here so it is not mistaken for an oversight. nginx and dnsmasq run on the host next to hive-c0re (see docs/gateway.md). What was given up is a mount/pid namespace — not a network one: that container ran with privateNetwork = false and shared the host's netns, so nginx was already binding host ports and already reaching localhost upstreams. The boundary bought no network isolation while costing a resolv.conf sync, a reload that had to cross the machine bus, and three bind mounts. 🔑 It did cost one real thing, and the replacement is explicit: the privileged reload verb used to be scoped by --machine=hive-gateway, which could only ever reach into that one container. With no namespace to bound it, the unit name is hard-coded in hive-priv instead — see PrivRequest::ReloadGatewayNginx. A caller cannot name the unit, so the verb cannot be steered at another service.

⚠️ Contrast /shared, which is sticky world-writable (1777): it has many legitimate writers, so sticky is the best available answer there. This dir has exactly one writer, so it needs no world write at all.

host admin socket access (`hivectl`)

hivectl drives the whole hive — spawn / kill / destroy / rebuild / deploy — over the host admin socket /run/hyperhive/host.sock, socket-activated by the hive-c0re.socket unit. That socket is the full-control surface, so who can connect to it is a real trust boundary.

By default the socket is 0660 group-owned by hive-admin, an empty group — so it is effectively root-only until an operator is explicitly granted access. Grant sudoless hivectl by listing login users in services.hyperhive.c0re.adminUsers; each is added to hive-admin, and members connect without sudo. The runtime dir /run/hyperhive is 0751 (traverse-only, no listing) so the group can reach the socket path; the socket's own 0660 hive-admin mode gates the connection, and the per-agent subdirs under it keep their own restrictive perms. Keep adminUsers to trusted operators — membership is equivalent to root over the hive.

8.3 KiB Raw Blame History