07852cabc1
The gateway's nginx + dnsmasq no longer run in their own nspawn container. `nix/host-modules/hive-gateway/default.nix` loses the `containers.hive-gateway` wrapper and everything that existed only to punch holes in it: `privateNetwork = false`, `CAP_NET_ADMIN`, five bind mounts, its own `stateVersion`, `networking.firewall.enable = false`, `networking.resolvconf.enable = false`, and the `hive-gateway-resolv` path+service pair. 465 -> 303 lines. The container never bought isolation here. It shared the host netns by necessity — nginx binds the host's :80/:443, dnsmasq answers on the bridge — so each of those settings was undoing a boundary the gateway could not afford in the first place. Four things made it more than a deletion, none of them visible in the nix diff: - The self-signed cert service also imports the hive CA leaf, so removing it with the container would have left nginx naming a missing cert file, which it refuses to load at all. - The nginx reload is a hive-priv verb. It still needs root, but no longer for the reason its doc gave, and `--machine=` was both transport and scope — so the unit name is now hard-coded in the helper as the containment. - The lifecycle verb named a container that stops existing. - `journalctl -M hive-gateway` had no machine to enter. Per the operator's ruling, the operator verb keeps working and agents lose it. `InfraContainer` answered three questions that used to share an answer; it now splits into `name()` (identity), `target()` (Container vs HostUnit), `service_unit()` (the systemd unit), and `agent_restartable()`, which the MCP restart path checks before the capability so the refusal cannot read as "ask for infra_admin". `SIBLING_CONTAINERS` drops the gateway — it gates the requests that name a container as a string — while `FromStr` still accepts it, because that answers what a name is, not who may act on it. The dashboard's gateway journal reads host journald filtered to `nginx.service`. Prose was corrected where it only named a location, and re-argued where the container was doing security work: a `0666` per-agent socket was safe because only the gateway container had the directory bind-mounted. There is no mount now, so the directory permissions are the whole of the access control — the constraint holds, its mechanism doesn't. Gate: nix fmt / clippy --all-targets -D warnings / cargo test all clean (710 tests); hivectl-cli.md regenerated from the clap tree. The nix eval was run in both TLS shapes at this commit: every delta in the rendered virtualHosts is one of the three intended path moves, dnsmasq settings are byte-identical, and the absence probe flips true -> false with bindMounts emptied.
165 lines
8.3 KiB
Markdown
165 lines
8.3 KiB
Markdown
# The operator/agent boundary
|
|
|
|
Design rationale for hyperhive's two-principal trust model. The
|
|
_implementation_ work — container network isolation, the unifying
|
|
gateway, core-daemon privsep — is tracked as `area:ops` issues on
|
|
the forge.
|
|
|
|
The operator/agent boundary is now technically enforced, not just a
|
|
convention. Containers run in private netns (network isolation is
|
|
always on), the gateway proxies all operator-facing traffic, and
|
|
`hive-c0re` runs as the unprivileged `hive-core` user. All three
|
|
`area:ops` pillars — network isolation, the gateway, and privsep —
|
|
are complete and active.
|
|
|
|
## Two principals, two paths
|
|
|
|
- **Operator** — reaches every UI (the dashboard + every
|
|
per-agent page) through the gateway, on one origin.
|
|
Operator-authority actions (approve / deny, answer-as-operator,
|
|
lifecycle POSTs) are served by the core daemon and only
|
|
reachable via the gateway.
|
|
- **Agent** — speaks only for itself, only over its per-agent
|
|
unix socket. The socket's identity _is_ the agent (see
|
|
`docs/conventions.md`, "identity = socket"). An agent must not
|
|
be able to reach the core daemon's HTTP surface, another
|
|
agent's socket, or another agent's web UI.
|
|
|
|
## Design rule
|
|
|
|
**Operator-authority actions never get a per-agent-socket entry
|
|
point.** They live on the core backend.
|
|
|
|
Worked example — answering an operator-targeted question is a
|
|
`POST /answer-question/{id}` on the core dashboard, _never_ an
|
|
`AgentRequest` variant. If it were a per-agent-socket request, an
|
|
agent could `curl` its own socket and spoof an operator answer.
|
|
The per-agent web UI POSTs cross-origin to the core for these
|
|
(see the inline-answer feature — the loose-ends section on each
|
|
agent page).
|
|
|
|
## Why network isolation is the load-bearing step
|
|
|
|
Without network isolation, containers share the host network namespace
|
|
and can reach `localhost:<core-port>`, the dashboard, and every other
|
|
agent's web port — the operator/agent split is on the honour system and
|
|
every boundary claim above is aspirational. Network isolation is what
|
|
makes the boundary _real_; the gateway and privsep are ergonomics and
|
|
defence-in-depth layered on top.
|
|
|
|
Network isolation is now complete and always on: every agent container
|
|
runs in a private netns behind the hive bridge. The shared-netns mode
|
|
was removed. See `docs/network.md`.
|
|
|
|
Concretely, the core daemon's dashboard `/api` carries **no
|
|
application-layer authentication** — operator-authority routes are served
|
|
unauthenticated at the HTTP layer. Their protection is entirely (a) the
|
|
gateway, which fronts all operator traffic and is where operator auth lives,
|
|
and (b) network isolation, which keeps agents — and `hive-ci`'s untrusted PR
|
|
builds — off host-loopback so nothing can reach `127.0.0.1:<dashboard_port>`
|
|
directly. This is deliberate given the load-bearing role of network isolation
|
|
above, but it is a standing invariant: the `/api` must never be bound to a
|
|
non-loopback address or exposed outside the gateway, and every new
|
|
operator-authority route inherits that assumption. `hive-ci` is treated like an
|
|
agent for this purpose — it runs untrusted PR code and is netns-isolated for
|
|
the same reason.
|
|
|
|
The `area:ops` issues followed this sequencing:
|
|
|
|
1. **Gateway** — pure ergonomics win, unblocks same-origin (lets the
|
|
cross-origin CORS shim on `/answer-question/{id}` go away), no
|
|
behavioural risk. An nginx nixos-container now sits in front of all
|
|
surfaces; per-agent UIs are proxied under `/agent/<name>/`.
|
|
2. **Network isolation** — the load-bearing step that turns the
|
|
honour-system split into an enforced boundary. **Complete** —
|
|
always-on, unconditional; the shared-netns mode was removed.
|
|
3. **Privsep** — defence in depth on the core process; `hive-c0re`
|
|
runs as the unprivileged `hive-core` user and delegates root
|
|
operations to `hive-priv`, a narrow socket-activated helper. See
|
|
[`docs/security.md`](security.md) for the privilege boundary table.
|
|
|
|
### hive-priv socket activation
|
|
|
|
`hive-priv` is **always** socket-activated by the `hive-priv.socket`
|
|
systemd unit. The unit binds `/run/hive/priv.sock` with
|
|
`SocketGroup=hive-core` and mode `0660` and passes the ready listener
|
|
to the helper as fd 3 (`LISTEN_FDS`). The helper requires this and
|
|
bails if it isn't socket-activated — there is intentionally no
|
|
self-bind fallback.
|
|
|
|
Dropping the old fallback removed a dev/prod divergence: when
|
|
`hive-priv` bound the socket itself it created the file owned by
|
|
root's primary group rather than `hive-core`, so a `hive-core` client
|
|
couldn't connect the way the socket unit's `SocketGroup` grant
|
|
intends. Requiring socket activation everywhere means dev and prod
|
|
take the exact same path and the group grant always holds.
|
|
|
|
### the per-agent socket dir
|
|
|
|
`/run/hive-agent/<name>/` is shared by **three principals that share no
|
|
group**, which is why its mode is what it is:
|
|
|
|
| principal | reaches | needs |
|
|
|---|---|---|
|
|
| the agent's harness | binds + unlinks `agent.sock`, `web.sock` | owner, `rwx` |
|
|
| `hive-c0re` | dials `agent.sock` (todo wakes) | traverse |
|
|
| the gateway's nginx | dials `web.sock` | traverse |
|
|
|
|
The last two land in "other", so the dir is **`0751`, owned by the
|
|
agent's container uid/gid** — `o=--x` is traverse without listing, and
|
|
both sockets are `0666`, which is all a dialer needs.
|
|
|
|
**Ownership is declared, not repaired.** The tmpfiles.d entry written by
|
|
`SyncAgentTmpfiles` names the uid/gid directly. Do not add a chown
|
|
alongside it: `d` re-applies on every boot *and* every agent
|
|
spawn/destroy, so ownership set afterwards is reverted the next time any
|
|
agent changes — which is exactly how this dir spent a long time at
|
|
`0777 root root` while a privileged chown appeared to be fixing it.
|
|
|
|
The mode is load-bearing, not cosmetic. Write permission on a
|
|
*directory* is what confers the right to unlink its entries, whoever owns
|
|
them, and the sticky bit is the only thing that would restrain that (it
|
|
is not set here). A world-writable socket dir therefore lets anything
|
|
able to reach the path delete an agent's socket and bind its own — and
|
|
nginx reaches all of `/run/hive-agent` (as a plain host path since the
|
|
gateway moved out of its container; it used to be bind-mounted in, which
|
|
was the same reach through a longer route). Dropping `o=w` removes that
|
|
permission rather than qualifying it.
|
|
|
|
⚠️ **The gateway leaving its container is a deliberate trade, recorded
|
|
here so it is not mistaken for an oversight.** nginx and dnsmasq run on
|
|
the host next to `hive-c0re` (see `docs/gateway.md`). What was given up
|
|
is a *mount/pid* namespace — **not** a network one: that container ran
|
|
with `privateNetwork = false` and shared the host's netns, so nginx was
|
|
already binding host ports and already reaching `localhost` upstreams.
|
|
The boundary bought no network isolation while costing a resolv.conf
|
|
sync, a reload that had to cross the machine bus, and three bind mounts.
|
|
🔑 It did cost one real thing, and the replacement is explicit: the
|
|
privileged reload verb used to be scoped by `--machine=hive-gateway`,
|
|
which could only ever reach into that one container. With no namespace
|
|
to bound it, the unit name is hard-coded in `hive-priv` instead — see
|
|
`PrivRequest::ReloadGatewayNginx`. **A caller cannot name the unit, so
|
|
the verb cannot be steered at another service.**
|
|
|
|
⚠️ Contrast `/shared`, which *is* sticky world-writable (`1777`): it has
|
|
many legitimate writers, so sticky is the best available answer there.
|
|
This dir has exactly one writer, so it needs no world write at all.
|
|
|
|
### host admin socket access (`hivectl`)
|
|
|
|
`hivectl` drives the whole hive — spawn / kill / destroy / rebuild /
|
|
deploy — over the **host admin socket** `/run/hyperhive/host.sock`,
|
|
socket-activated by the `hive-c0re.socket` unit. That socket *is* the
|
|
full-control surface, so who can connect to it is a real trust
|
|
boundary.
|
|
|
|
By default the socket is `0660` group-owned by **`hive-admin`**, an
|
|
empty group — so it is effectively **root-only** until an operator is
|
|
explicitly granted access. Grant sudoless `hivectl` by listing login
|
|
users in `services.hyperhive.c0re.adminUsers`; each is added to
|
|
`hive-admin`, and members connect without `sudo`. The runtime dir
|
|
`/run/hyperhive` is `0751` (traverse-only, no listing) so the group can
|
|
reach the socket path; the socket's own `0660 hive-admin` mode gates
|
|
the connection, and the per-agent subdirs under it keep their own
|
|
restrictive perms. Keep `adminUsers` to trusted operators — membership
|
|
is equivalent to root over the hive.
|