nix/hive-gateway: UDS upstream for /agent/<name>/ (#784 phase 2 step 3)

Switch per-agent gateway upstreams from TCP loopback to unix-domain
socket when the agent has opted in via `hyperhive.web.useUnixSocket`
(#822). Coexists with the TCP path during rollout.

Changes:

- New `agentSocketsFile` option (default
  `/var/lib/hyperhive/agent-sockets.json`) — c0re writes the map
  there via `hive_c0re::agent_sockets::write` (#809).
- `agentSocketsTable = lib.importJSON ...` (graceful empty when
  file missing).
- `agentUpstreamFor name port` picks `http://unix:<path>:/` when the
  socket has a JSON entry AND the file exists at eval time; else
  `http://127.0.0.1:<port>/`. Path-exists gate guards against
  c0re's blanket-emit shape during the canary window (agents in
  `agent-sockets.json` who haven't actually flipped have no
  bound socket on disk → fall back to TCP). Damocles will ship a
  `.bound` marker filter on the c0re side (#784 step 2d
  follow-up); once that's in, the path-exists check is redundant
  but harmless. Step 4 drops it entirely along with the TCP
  fallback.
- `containers.hive-gateway.bindMounts."/run/hive-agent"` —
  read-only, unconditional. Inert when no agents have opted in.
  Required so nginx inside the gateway container can `connect(2)`
  to the per-agent sockets damocles's #813 bind-mounts into agent
  containers at the same paths.

Docs:

- `docs/gateway.md::Per-agent UDS upstream (#784)` — full rollout
  flow, subdir-bind rationale (damocles #813), eval-time gate
  explainer, step 4 drop plan.

`nix flake check` clean; `nix fmt` clean.

Canary plan: once #822 (`useUnixSocket` option) lands + this PR
merges, manager flips atlas's agent.nix to `useUnixSocket = true`
via the config-update flow. End-to-end validation against atlas
before broader rollout.
This commit is contained in:
atlas 2026-05-31 16:00:45 +02:00 committed by mara
commit 3a29aee001
2 changed files with 276 additions and 153 deletions

View file

@ -165,3 +165,57 @@ Scope is intentionally narrow per mara on #755: "only for routes
already special cased in the nginx config". Other gateway routes
(forge / matrix / fluffychat) get nginx defaults — extending the
custom-error pattern there is a separate follow-up.
## Per-agent UDS upstream (#784)
Per-agent `/agent/<name>/` upstreams default to TCP loopback
(`http://127.0.0.1:<port>/`) but each agent can opt in to unix-
domain socket upstream by flipping `hyperhive.web.useUnixSocket =
true` in its `agent.nix`. Rollout flow:
1. **Harness** binds a `UnixListener` at
`/run/hive-agent/<name>/web.sock` when `HIVE_WEB_SOCKET` is set
(PR #800). The env var is set by `harness-base.nix` from the
`useUnixSocket` option (#822).
2. **hive-c0re** writes a sibling `agent-sockets.json` next to
`agent-ports.json` (PR #809) and bind-mounts the per-agent
subdir `/run/hive-agent/<name>/` into each sub-agent container
via `set_nspawn_flags` (PR #813). Path-shape lives in
`hive_c0re::agent_sockets::socket_path_for(name)` — one canonical
derivation, no triangulation across the c0re / harness / gateway
boundaries.
3. **Gateway** reads both `agentPortsFile` + `agentSocketsFile` at
deploy time. Per agent: a socket entry beats the TCP port. The
gateway container bind-mounts `/run/hive-agent/` read-only so
nginx inside can `connect(2)` to the per-agent sockets.
Mixed state during rollout: agents flip per-agent. Agents that
haven't opted in keep the TCP path; agents that have flipped use
the UDS path. The two coexist on the same gateway with zero
per-agent special-casing in the nginx config (`agentUpstreamFor`
resolves the right shape from the JSON maps).
**Eval-time gate during the rollout window**: `agentUpstreamFor`
checks `builtins.pathExists` on the socket path before picking the
UDS upstream. c0re's `agent_sockets::write` emits an entry for every
sub-agent regardless of whether they've actually flipped, so the
gateway has no other signal that a given agent is or isn't actually
binding the socket. The path-exists check works because a flipped
agent's harness binds the socket on container start, and the
gateway-container rebuild (which re-runs nix eval) happens on every
topology change — so a freshly-flipped agent flips through TCP →
UDS over one rebuild cycle. Once c0re's `.bound` marker filter
ships (#784 step 2d follow-up), `agent-sockets.json` only contains
agents that have actually bound, and the path-exists check is
redundant but harmless. Step 4 drops it.
**Why per-agent subdir** (not a flat `/run/hive-agent/<name>.sock`):
the harness's `bind_unix` helper unlinks any stale socket before
calling `bind(2)`, and a file bind-mount loses its host-side anchor
on unlink. Dir bind-mount keeps the same dir inode visible on both
sides, so the new `web.sock` shows up on the host the moment the
harness binds it (damocles #813 design note).
**Step 4 plan**: once every agent has flipped + soaked, the
`agentPortsFile` fallback drops + the harness's TCP bind goes away
entirely. Tracked at #784 step 4.