fix(#970,#973): retry gateway reload on failure; always enable HIVE_WEB_SOCKET

This commit is contained in:
damocles 2026-06-01 18:18:45 +02:00
commit b83edc40c6
5 changed files with 88 additions and 82 deletions

View file

@ -71,13 +71,14 @@ SSH for forge stays direct on `cfg.sshPort` — separate listener protocol, not
## Per-agent unix-socket upstream
Sub-agent `/agent/<name>/` upstreams flip from TCP loopback to a
unix-domain socket as each agent opts in. The mechanism:
All agents bind their web UI on a unix-domain socket at
`/run/hive-agent/<name>/web.sock` — the `HIVE_WEB_SOCKET` env var is
now set unconditionally for every agent. The mechanism:
1. **Agent side** (`hyperhive.web.useUnixSocket = true` in
`agent.nix`). Sets `HIVE_WEB_SOCKET=/run/hive-agent/<name>/web.sock`
on the harness service env; `web_ui::serve` binds a `UnixListener`
at that path instead of TCP.
1. **Agent side**. `HIVE_WEB_SOCKET=/run/hive-agent/<name>/web.sock`
is set on every harness service env; `web_ui::serve` binds a
`UnixListener` at that path. The deprecated
`hyperhive.web.useUnixSocket` option is now a no-op.
2. **Host side**. `hive-c0re` bind-mounts the per-agent subdir
(`/run/hive-agent/<name>/`) into the agent's container. Dir
bind, not file bind — file bind-mounts don't survive the
@ -87,32 +88,28 @@ unix-domain socket as each agent opts in. The mechanism:
`<dir>/hyperhive-socket-bound` next to the socket. c0re's
`agent_sockets::write` filters its JSON map by marker presence —
only agents whose harness has actually bound the socket appear there.
Without this filter, the gateway would `proxy_pass` to a non-existent
socket for every sub-agent that hasn't opted in yet. (Legacy name
`.bound` also accepted during the transition window.)
(Legacy name `.bound` also accepted during the transition window.)
4. **Gateway side**. `gateway_nginx::write` generates
`/var/lib/hyperhive/gateway/agents.conf` — a plain nginx include
file with one `location /agent/<name>/` block per agent. UDS
upstream (`http://unix:/run/hive-agent/<name>/web.sock:/`) when
`hyperhive-socket-bound` marker present; TCP loopback otherwise.
`hyperhive-socket-bound` marker present; TCP loopback for agents
that haven't yet been rebuilt under the new config.
The gateway container bind-mounts `/var/lib/hyperhive/gateway/` at
`/run/hive-state/`; nginx includes `/run/hive-state/agents.conf`.
After each write, c0re triggers `nginx -s reload` inside the
gateway container from the HOST via
`systemd-run --machine=hive-gateway nginx -s reload`. This is
`systemd-run --machine=hive-gateway --wait nginx -s reload`. This is
intentionally host-side: `IN_MOVED_TO` from an atomic rename does
not propagate across the nspawn mount-namespace boundary, so a
path unit inside the container would never fire (#889).
path unit inside the container would never fire.
c0re regenerates `agents.conf` (and triggers a reload) on two
triggers: every topology change (new/removed agents) and every 10s
marker poll tick (`agent_sockets::spawn_poll`). `write()` is
idempotent — skips the rename+reload when content is unchanged.
Transition: agents that haven't flipped `useUnixSocket = true` get a
TCP loopback upstream in `agents.conf` (deterministic port from
`agent_web_port(name)`). A future cleanup will drop the TCP fallback
once every agent's flipped.
idempotent — skips the rename when content is unchanged. Failed reloads
are retried automatically on subsequent poll ticks via
`gateway_nginx::reload_if_pending`.
## Agent port map (`agent-ports.json`)