feat(3088): move the gateway's nginx + dnsmasq onto the host

The gateway's nginx + dnsmasq no longer run in their own nspawn container.
`nix/host-modules/hive-gateway/default.nix` loses the
`containers.hive-gateway` wrapper and everything that existed only to punch
holes in it: `privateNetwork = false`, `CAP_NET_ADMIN`, five bind mounts,
its own `stateVersion`, `networking.firewall.enable = false`,
`networking.resolvconf.enable = false`, and the `hive-gateway-resolv`
path+service pair. 465 -> 303 lines.

The container never bought isolation here. It shared the host netns by
necessity — nginx binds the host's :80/:443, dnsmasq answers on the bridge —
so each of those settings was undoing a boundary the gateway could not
afford in the first place.

Four things made it more than a deletion, none of them visible in the nix
diff:

- The self-signed cert service also imports the hive CA leaf, so removing it
  with the container would have left nginx naming a missing cert file, which
  it refuses to load at all.
- The nginx reload is a hive-priv verb. It still needs root, but no longer
  for the reason its doc gave, and `--machine=` was both transport and
  scope — so the unit name is now hard-coded in the helper as the
  containment.
- The lifecycle verb named a container that stops existing.
- `journalctl -M hive-gateway` had no machine to enter.

Per the operator's ruling, the operator verb keeps working and agents lose
it. `InfraContainer` answered three questions that used to share an answer;
it now splits into `name()` (identity), `target()` (Container vs HostUnit),
`service_unit()` (the systemd unit), and `agent_restartable()`, which the
MCP restart path checks before the capability so the refusal cannot read as
"ask for infra_admin". `SIBLING_CONTAINERS` drops the gateway — it gates the
requests that name a container as a string — while `FromStr` still accepts
it, because that answers what a name is, not who may act on it. The
dashboard's gateway journal reads host journald filtered to `nginx.service`.

Prose was corrected where it only named a location, and re-argued where the
container was doing security work: a `0666` per-agent socket was safe
because only the gateway container had the directory bind-mounted. There is
no mount now, so the directory permissions are the whole of the access
control — the constraint holds, its mechanism doesn't.

Gate: nix fmt / clippy --all-targets -D warnings / cargo test all clean (710
tests); hivectl-cli.md regenerated from the clap tree. The nix eval was run
in both TLS shapes at this commit: every delta in the rendered
virtualHosts is one of the three intended path moves, dnsmasq settings are
byte-identical, and the absence probe flips true -> false with bindMounts
emptied.
This commit is contained in:
atlas 2026-08-11 18:00:27 +02:00
commit 07852cabc1
34 changed files with 704 additions and 618 deletions

View file

@ -121,8 +121,25 @@ The mode is load-bearing, not cosmetic. Write permission on a
them, and the sticky bit is the only thing that would restrain that (it them, and the sticky bit is the only thing that would restrain that (it
is not set here). A world-writable socket dir therefore lets anything is not set here). A world-writable socket dir therefore lets anything
able to reach the path delete an agent's socket and bind its own — and able to reach the path delete an agent's socket and bind its own — and
the gateway container has all of `/run/hive-agent` bind-mounted in. nginx reaches all of `/run/hive-agent` (as a plain host path since the
Dropping `o=w` removes that permission rather than qualifying it. gateway moved out of its container; it used to be bind-mounted in, which
was the same reach through a longer route). Dropping `o=w` removes that
permission rather than qualifying it.
⚠️ **The gateway leaving its container is a deliberate trade, recorded
here so it is not mistaken for an oversight.** nginx and dnsmasq run on
the host next to `hive-c0re` (see `docs/gateway.md`). What was given up
is a *mount/pid* namespace — **not** a network one: that container ran
with `privateNetwork = false` and shared the host's netns, so nginx was
already binding host ports and already reaching `localhost` upstreams.
The boundary bought no network isolation while costing a resolv.conf
sync, a reload that had to cross the machine bus, and three bind mounts.
🔑 It did cost one real thing, and the replacement is explicit: the
privileged reload verb used to be scoped by `--machine=hive-gateway`,
which could only ever reach into that one container. With no namespace
to bound it, the unit name is hard-coded in `hive-priv` instead — see
`PrivRequest::ReloadGatewayNginx`. **A caller cannot name the unit, so
the verb cannot be steered at another service.**
⚠️ Contrast `/shared`, which *is* sticky world-writable (`1777`): it has ⚠️ Contrast `/shared`, which *is* sticky world-writable (`1777`): it has
many legitimate writers, so sticky is the best available answer there. many legitimate writers, so sticky is the best available answer there.

View file

@ -381,7 +381,7 @@ that allows the underlying resource access.
| `manage_root_agent` | may lifecycle-manage the root/manager agent via `kill`/`start`/`restart` | | `manage_root_agent` | may lifecycle-manage the root/manager agent via `kill`/`start`/`restart` |
| `read_host_journal` | `get_host_journal` MCP tool is registered + `GET /journal-host` requests are served | | `read_host_journal` | `get_host_journal` MCP tool is registered + `GET /journal-host` requests are served |
| `query_agent_state` | may call `get_loose_ends` / `CountPendingReminders` targeting non-child agents | | `query_agent_state` | may call `get_loose_ends` / `CountPendingReminders` targeting non-child agents |
| `infra_admin` | may call `restart(name)` on hive infrastructure containers (`hive-ci`, `hive-gateway`, `hive-forge`); each restart is logged to the dashboard AUDIT trail | | `infra_admin` | may call `restart(name)` on hive infrastructure containers (`hive-ci`, `hive-forge`, `hive-matrix`**not** `hive-gateway`, which is the host's nginx and is operator-only); each restart is logged to the dashboard AUDIT trail |
**Config storage** — per-agent capabilities live in **Config storage** — per-agent capabilities live in
`/var/lib/hyperhive/meta/capabilities.json` alongside `tool-groups.json`. `/var/lib/hyperhive/meta/capabilities.json` alongside `tool-groups.json`.

View file

@ -500,9 +500,9 @@ Two things to know about the weights:
override in `meta/resource-limits.json`, so every agent carries the override in `meta/resource-limits.json`, so every agent carries the
same value and the weight does *not* rank agents against each other. same value and the weight does *not* rank agents against each other.
What `80` buys is that agents yield to everything **not** on this What `80` buys is that agents yield to everything **not** on this
drop-in path: host services and the infra containers (`hive-ci`, drop-in path: host services (nginx and dnsmasq among them) and the
`hive-forge`, `hive-gateway`, `hive-matrix`), which stay at the infra containers (`hive-ci`, `hive-forge`, `hive-matrix`), which stay
kernel default of `100`. at the kernel default of `100`.
- `IOWeight=` is only honoured when the backing device runs the BFQ - `IOWeight=` is only honoured when the backing device runs the BFQ
scheduler or has blk-iocost QoS enabled. On a host using scheduler or has blk-iocost QoS enabled. On a host using
`none`/`mq-deadline`/`kyber` without iocost, systemd writes the value `none`/`mq-deadline`/`kyber` without iocost, systemd writes the value

View file

@ -1,6 +1,6 @@
# hive-gateway # hive-gateway
Single nginx in front of every hyperhive web surface. Container `hive-gateway`, shared host netns, system-config (not meta-flake managed). Configured via `services.hyperhive.gateway.*` + per-subsystem opt-in flags in `services.hyperhive.{forge,matrix,...}`. Single nginx in front of every hyperhive web surface. Runs on the **host**, next to hive-c0re; system-config (not meta-flake managed). Configured via `services.hyperhive.gateway.*` + per-subsystem opt-in flags in `services.hyperhive.{forge,matrix,...}`. (It lived in a `hive-gateway` container until #3088 — one that shared the host netns anyway, so the boundary gave no network isolation while costing a resolv.conf sync, a machine-bus reload and three bind mounts.)
## Vhost map ## Vhost map
@ -112,18 +112,22 @@ now set unconditionally for every agent. The mechanism:
a UDS upstream (`http://unix:/run/hive-agent/<name>/web.sock:/`); a UDS upstream (`http://unix:/run/hive-agent/<name>/web.sock:/`);
if the socket is not yet bound, nginx returns 502 caught by the if the socket is not yet bound, nginx returns 502 caught by the
`error_page 502 503 504 = /__hive_agent_unreachable` directive. `error_page 502 503 504 = /__hive_agent_unreachable` directive.
The gateway container bind-mounts `/var/lib/hyperhive/gateway/` at nginx includes `/var/lib/hyperhive/gateway/agents.conf` — the same
`/run/hive-state/`; nginx includes `/run/hive-state/agents.conf`. path c0re writes, since both run on the host.
After each write, c0re triggers the appropriate nginx action inside After each write, c0re triggers the appropriate nginx action via
the gateway container via `hive-priv` (which runs as root and has `hive-priv` (which is root; hive-c0re runs as the unprivileged
`--machine=hive-gateway` transport rights that hive-c0re lacks). `hive-core` user and cannot act on a system unit).
`hive-priv` queries `ActiveState` and dispatches: `hive-priv` queries `ActiveState` and dispatches:
- active → `systemctl reload nginx` (SIGHUP, zero-downtime) - active → `systemctl reload nginx` (SIGHUP, zero-downtime)
- failed → `systemctl reset-failed nginx` + `systemctl start nginx` - failed → `systemctl reset-failed nginx` + `systemctl start nginx`
- otherwise → `systemctl start nginx` - otherwise → `systemctl start nginx`
This is intentionally host-side: `IN_MOVED_TO` from an atomic rename This is an explicit trigger rather than a path unit watching the
does not propagate across the nspawn mount-namespace boundary, so a file. It used to be *impossible* to do it any other way — `IN_MOVED_TO`
path unit inside the container would never fire. from the atomic rename did not cross the nspawn mount-namespace
boundary, so an in-container path unit never fired. With nginx on the
host a path unit would now work, and it is still not wanted: the write
and the reload belong in one causal chain c0re can retry and report on
(`RELOAD_PENDING`), not two units racing on an inotify event.
c0re regenerates `agents.conf` (and triggers a reload) on two c0re regenerates `agents.conf` (and triggers a reload) on two
triggers: every topology change (new/removed agents) and every 10s triggers: every topology change (new/removed agents) and every 10s
@ -160,8 +164,8 @@ selected by which (if any) external TLS source is set:
| mode | config | cert source | `.well-known` scheme | | mode | config | cert source | `.well-known` scheme |
|---|---|---|---| |---|---|---|---|
| self-signed (default) | neither `tls.certDir` nor `tls.acme` set | host hive-CA signs a gateway leaf (RSA-4096) | `https` | | self-signed (default) | neither `tls.certDir` nor `tls.acme` set | host hive-CA signs a gateway leaf (RSA-4096) | `https` |
| ACME (Let's Encrypt) | `tls.acme.enable = true` | nginx inside container via HTTP-01 | `https` | | ACME (Let's Encrypt) | `tls.acme.enable = true` | nginx via HTTP-01 | `https` |
| operator cert | `tls.certDir` set | bind-mounted from host | `https` | | operator cert | `tls.certDir` set | read from the operator's dir | `https` |
The `gateway.selfSignedTls` option is **deprecated and ignored** — self-signed The `gateway.selfSignedTls` option is **deprecated and ignored** — self-signed
is now derived from the absence of `tls.certDir` / `tls.acme`. Setting it to is now derived from the absence of `tls.certDir` / `tls.acme`. Setting it to
@ -182,7 +186,7 @@ services.hyperhive.gateway = {
}; };
``` ```
nginx inside the gateway container obtains and auto-renews certs via the ACME HTTP-01 challenge on `port` (default 80). The gateway container shares the host network namespace (`privateNetwork = false`) so outbound ACME requests work without any extra routing. Certs are stored inside the container's persistent state dir (`/var/lib/acme/` inside `hive-gateway`; survives restarts because `ephemeral = false`). nginx obtains and auto-renews certs via the ACME HTTP-01 challenge on `port` (default 80). Certs land in `/var/lib/acme/` on the host, managed by nixpkgs's `security.acme` in the ordinary way.
**Requirements**: `services.hyperhive.domain` must be publicly DNS-resolvable to this host, and `openFirewall = true` so Let's Encrypt can reach `/.well-known/acme-challenge/`. Each active vhost (main domain, `forge.<swarm-domain>`, `chat.<swarm-domain>`) gets its own cert via separate ACME challenges — the swarm services default to names under `services.hyperhive.swarm.domain`, so **every one of those names must resolve to this host too**, not just the hive's own. **Requirements**: `services.hyperhive.domain` must be publicly DNS-resolvable to this host, and `openFirewall = true` so Let's Encrypt can reach `/.well-known/acme-challenge/`. Each active vhost (main domain, `forge.<swarm-domain>`, `chat.<swarm-domain>`) gets its own cert via separate ACME challenges — the swarm services default to names under `services.hyperhive.swarm.domain`, so **every one of those names must resolve to this host too**, not just the hive's own.
@ -194,9 +198,11 @@ Mutual exclusion: `tls.certDir` set together with `tls.acme.enable = true` fails
On by default, and listens on `httpsPort` (default 443) on every vhost beside the plain-http `port` (default 80). On by default, and listens on `httpsPort` (default 443) on every vhost beside the plain-http `port` (default 80).
The issuer is a **host-held hive CA**, not a bare self-signed leaf. A host service (`hive-tls-ca.service`, from the `hive-tls` module) generates a long-lived CA (`services.hyperhive.tls.caValidityDays`, default ~20y) under `services.hyperhive.tls.stateDir` (default `/var/lib/hive-tls`), then signs a gateway **leaf** (`leafValidityDays`, default 30d) with it. The leaf dir is bind-mounted read-only into the gateway container at `/run/hive-ca`; an in-container import unit copies the leaf into nginx's state dir (`/var/lib/hive-gateway/tls/{cert,key}.pem`) with the owner/mode nginx needs. The issuer is a **host-held hive CA**, not a bare self-signed leaf. A host service (`hive-tls-ca.service`, from the `hive-tls` module) generates a long-lived CA (`services.hyperhive.tls.caValidityDays`, default ~20y) under `services.hyperhive.tls.stateDir` (default `/var/lib/hive-tls`), then signs a gateway **leaf** (`leafValidityDays`, default 30d) with it. `hive-gateway-self-signed-cert` then imports the leaf into nginx's state dir (`/var/lib/hive-gateway/tls/{cert,key}.pem`).
**Why a CA, not a bare leaf**: a bare self-signed leaf is its own trust anchor, so every regeneration is a new anchor every consumer must re-trust — and a runtime-generated, in-container leaf can't be wired into an agent's build-time trust store at all. With a stable CA, agents and federation peers trust it *once*; leaf rotation never re-breaks them. ⚠️ **That import unit is not a leftover of the old container — do not collapse it into pointing nginx at the CA dir.** It does two jobs. It re-modes the leaf (`hive-tls-ca` writes the key `0600 root:root`; nginx's pre-start `nginx -t` runs as the *nginx user*, so a `0600` key fails the config test and blocks the unit), and it guarantees **every cert path the nginx config names exists** — which is what the swarm-services fallback below is for. Removing it re-creates the #3097 outage.
**Why a CA, not a bare leaf**: a bare self-signed leaf is its own trust anchor, so every regeneration is a new anchor every consumer must re-trust — and a runtime-generated leaf can't be wired into an agent's build-time trust store at all. With a stable CA, agents and federation peers trust it *once*; leaf rotation never re-breaks them.
**What consumers trust**: `trust-bundle.pem` in the same state dir, not `ca.pem`. The hive CA is itself issued under the swarm root ([`swarm/ca.md`](swarm/ca.md) has the hierarchy), and an intermediate is not a chain a verifier can terminate at — so the bundle carries the hive CA plus whatever it is rooted at. nginx is handed the leaf with the hive CA appended for the same reason. Everything that trusts the hive's TLS reads the bundle: agents (via `security.pki.certificateFiles`), the CI and forge containers, and a federating peer. **What consumers trust**: `trust-bundle.pem` in the same state dir, not `ca.pem`. The hive CA is itself issued under the swarm root ([`swarm/ca.md`](swarm/ca.md) has the hierarchy), and an intermediate is not a chain a verifier can terminate at — so the bundle carries the hive CA plus whatever it is rooted at. nginx is handed the leaf with the hive CA appended for the same reason. Everything that trusts the hive's TLS reads the bundle: agents (via `security.pki.certificateFiles`), the CI and forge containers, and a federating peer.
@ -220,11 +226,11 @@ services.hyperhive.gateway = {
}; };
``` ```
The directory is bind-mounted read-only into the gateway container at `/run/hive-tls/`. nginx uses `cert.pem` + `key.pem` (override `tls.certName`/`tls.keyName` for different filenames). Both modes listen on `httpsPort` (default 443) and emit `https://` in `.well-known` responses. nginx reads the directory directly and uses `cert.pem` + `key.pem` (override `tls.certName`/`tls.keyName` for different filenames). Both modes listen on `httpsPort` (default 443) and emit `https://` in `.well-known` responses.
`tls.certDir` and `tls.acme.enable` set together is an assertion error. `tls.certDir` and `tls.acme.enable` set together is an assertion error.
**Key file permissions**: nixpkgs's `security.acme` outputs private keys as `0640 root:acme` by default. nginx inside the gateway container runs as the `nginx` user and cannot read a key with that ownership. Fix with: **Key file permissions**: nixpkgs's `security.acme` outputs private keys as `0640 root:acme` by default. nginx runs as the `nginx` user and cannot read a key with that ownership. Fix with:
```nix ```nix
security.acme.certs."example.com".group = "nginx"; security.acme.certs."example.com".group = "nginx";
@ -424,11 +430,12 @@ JS/CSS assets skip `try_files` entirely and get the immutable cache
headers. Nix store paths are content-addressed — the hash changes on headers. Nix store paths are content-addressed — the hash changes on
any content change — so `max-age=31536000` is safe. any content change — so `max-age=31536000` is safe.
**Why nix store is reachable from the gateway container**: nspawn **Why the nix store path resolves**: `HIVE_AGENT_FRONTEND_DIR` is a nix
containers bind-mount `/nix/store` read-only by default. The store path baked in at hive-c0re build time, and c0re (writing
`HIVE_AGENT_FRONTEND_DIR` path is a nix store path baked in at `agents.conf`) and nginx (serving files from it) are on the same machine,
hive-c0re build time — the same path is visible to both c0re (writing so they see the same store. This used to need explaining — nspawn
`agents.conf`) and the gateway nginx (serving files from it). bind-mounts `/nix/store` read-only into a container, which is what made
the baked-in path work from inside the gateway.
**Graceful degradation**: if `HIVE_AGENT_FRONTEND_DIR` is empty or **Graceful degradation**: if `HIVE_AGENT_FRONTEND_DIR` is empty or
unset (e.g. a build that predates `cfg.frontend`), each agent gets the unset (e.g. a build that predates `cfg.frontend`), each agent gets the
@ -493,9 +500,7 @@ services.hyperhive.gateway.auth = {
The credential store lives at the fixed path The credential store lives at the fixed path
`/var/lib/hyperhive/gateway/gateway.htpasswd` on the host. A tmpfiles `/var/lib/hyperhive/gateway/gateway.htpasswd` on the host. A tmpfiles
rule pre-creates the file on first boot; no manual path configuration rule pre-creates the file on first boot; no manual path configuration
is required. The file is exposed inside the gateway container at is required. nginx reads it at that path directly.
`/run/hive-state/gateway.htpasswd` via the existing gateway state
bind-mount.
Manage users with `hivectl gateway`. `hivectl` sends the request over the Manage users with `hivectl gateway`. `hivectl` sends the request over the
host admin socket and the `hive-c0re` daemon performs the write at its host admin socket and the `hive-c0re` daemon performs the write at its

View file

@ -115,49 +115,46 @@ schemes pick their own.
dnsmasq is **authoritative** for the hive's own zones — answers dnsmasq is **authoritative** for the hive's own zones — answers
`<hive-domain>`, `forge.<hive-domain>`, `matrix.<hive-domain>` `<hive-domain>`, `forge.<hive-domain>`, `matrix.<hive-domain>`
queries with the bridge IP (where nginx is reachable). Everything queries with the bridge IP (where nginx is reachable). Everything
else is forwarded to the host's own resolvers: dnsmasq reads the else is forwarded to the host's own resolvers: dnsmasq runs on the host
gateway container's `/etc/resolv.conf`, the host copy nixos-container and reads the host's `/etc/resolv.conf` directly. Containers don't need
makes at each container start. Containers don't need to know the to know the upstream — they query the bridge IP and dnsmasq does the
upstream — they query the bridge IP and dnsmasq does the right thing right thing per-name.
per-name.
That copy is one-shot — systemd-nspawn(1) is explicit that nothing There is deliberately no fallback `server=`: dnsmasq queries all known
propagates into it after early init, because resolv.conf is normally upstreams in parallel, so a hardcoded public resolver would take a share
updated by rename rather than in place. Left alone, a host network of normal traffic, not just cover the gap.
change (new router, new lease, laptop moving networks) would strand
dnsmasq on a resolver that no longer answers, and every non-hive
lookup from every agent would hang until someone restarted the
gateway. The host-side **`hive-gateway-resolv`** path unit closes
that: it watches `/etc/resolv.conf`, `machinectl copy-to`s it into
the container, and reloads dnsmasq (`SIGHUP` — re-read upstreams +
flush cache, nothing dropped). The watch is armed before
`network-pre.target` so the boot's first DHCP write is caught as well,
and the sync also runs once per gateway start to pick up a resolver
change that happened while the container was down. A host file with no
`nameserver` line is
skipped rather than pushed, so a mid-rewrite snapshot can't blank the
hive's DNS. There is deliberately no fallback `server=`: dnsmasq
queries all known upstreams in parallel, so a hardcoded public
resolver would take a share of normal traffic, not just cover the gap.
Two alternatives that look simpler and aren't: ### History: the resolv.conf sync, and why it is gone
- **A path unit inside the container.** The host replaces Until the gateway moved onto the host, dnsmasq ran in the `hive-gateway`
`/etc/resolv.conf` by rename, and that `IN_MOVED_TO` doesn't cross container and read *that* container's `/etc/resolv.conf` — a one-shot
the nspawn mount namespace — the same reason hive-c0re reloads nginx copy nixos-container made at start. systemd-nspawn(1) is explicit that
from the host side after each `agents.conf` write. nothing propagates into it after early init, because resolv.conf is
- **Bind-mounting the host's `/etc/resolv.conf` into the container.** normally updated by rename rather than in place. So a host network change
openresolv writes a temp file and renames over the target, so the (new router, new lease, laptop moving networks) stranded dnsmasq on a
bind mount would pin the *first* inode for the container's whole resolver that no longer answered, and every non-hive lookup from every
lifetime — strictly worse than the copy, which at least a restart agent hung until someone restarted the gateway.
clears. (Reachability is not the problem here: the gateway runs with
`privateNetwork = false`, so it shares the host's netns and can reach
anything the host can.)
`machinectl copy-to` is used rather than writing to the container's A host-side `hive-gateway-resolv` path unit closed that gap: watch
rootfs from the host, so the push goes through the container's own `/etc/resolv.conf`, `machinectl copy-to` it into the container, reload
mount namespace and stays correct if `/etc` is ever assembled dnsmasq. Roughly eighty lines of watcher, marker file, is-active guard
differently (e.g. `system.etc.overlay`). and mid-rewrite-snapshot check — **all of it bridging two copies of one
file.** With one machine there is one file, and the whole unit is
deleted.
🔑 Worth keeping as a shape, not just a story: **the sync was not
complexity anyone chose. It was the cost of a boundary that bought
nothing here** — the gateway already ran with `privateNetwork = false`,
sharing the host's netns, so the container never provided network
isolation in the first place. When a workaround is that elaborate, the
question to ask is what the boundary is *for*.
(Two alternatives were considered at the time and both were worse than
the copy: a path unit *inside* the container never fired, because the
host replaces the file by rename and `IN_MOVED_TO` does not cross the
nspawn mount namespace; and bind-mounting the host's `/etc/resolv.conf`
would have pinned the *first* inode for the container's whole lifetime,
since openresolv writes a temp file and renames over the target.)
`bind-interfaces` + `interface = [ bridgeName "lo" ]` means the `bind-interfaces` + `interface = [ bridgeName "lo" ]` means the
listener only accepts queries from the bridge interface (plus lo for listener only accepts queries from the bridge interface (plus lo for
@ -289,4 +286,4 @@ loopback — the hive-c0re admin socket is a UDS, not TCP.
## Cross-references ## Cross-references
- `docs/gateway.md` — vhost map + the gateway container's other duties - `docs/gateway.md` — vhost map + the gateway's other duties

View file

@ -187,8 +187,20 @@ bind-mount compatibility with user namespace UID mapping and is tracked as a TOD
`hive-c0re` runs as the unprivileged system user `hive-core` `hive-c0re` runs as the unprivileged system user `hive-core`
(`/var/lib/hyperhive` owned by `hive-core:hive-core`). It cannot (`/var/lib/hyperhive` owned by `hive-core:hive-core`). It cannot
directly invoke `nixos-container`, `journalctl -M`, or `systemctl directly invoke `nixos-container`, `journalctl -M`, or act on a system
-M hive-gateway` — those require root. `hive-priv` fills this gap. unit (`systemctl reload nginx`) — those require root. `hive-priv` fills
this gap.
⚠️ **Note what that costs when a helper verb loses its namespace.**
`ReloadGatewayNginx` used to run `systemctl -M hive-gateway …`, and the
`--machine=` flag was doing two jobs: it was the *transport* into the
container **and** the *scope* — the verb could not reach anything
outside that one machine. With nginx on the host the transport is
unnecessary and the scope went with it, so the containment is now the
hard-coded unit name in `hive-priv`: a caller cannot name the unit, so
the verb cannot be steered at another service. **When a privileged verb
stops needing a namespace, check whether the namespace was also what
bounded it.**
### hive-priv ### hive-priv
@ -207,7 +219,7 @@ known operations; there is no arbitrary command pass-through:
| `DestroyContainer` | `nixos-container destroy <name>` | | `DestroyContainer` | `nixos-container destroy <name>` |
| `ListContainers` | `nixos-container list` | | `ListContainers` | `nixos-container list` |
| `ReadContainerJournal` | `journalctl -M <container> -n <n> [filters...]` | | `ReadContainerJournal` | `journalctl -M <container> -n <n> [filters...]` |
| `ReloadGatewayNginx` | `systemctl -M hive-gateway reload/start/reset-failed nginx` | | `ReloadGatewayNginx` | `systemctl reload/start/reset-failed nginx` (host unit; the unit name is hard-coded, not a parameter) |
| `WriteNspawnFlags` | write `/etc/nixos-containers/<container>.conf` (bind-mount list + network isolation vars) | | `WriteNspawnFlags` | write `/etc/nixos-containers/<container>.conf` (bind-mount list + network isolation vars) |
| `WriteResourceLimits` | write `CPUQuota=`/`MemoryMax=`/`CPUWeight=`/`IOWeight=` systemd drop-in for agent container | | `WriteResourceLimits` | write `CPUQuota=`/`MemoryMax=`/`CPUWeight=`/`IOWeight=` systemd drop-in for agent container |
| `RemoveServiceDropin` | remove `container@<name>.service.d/` drop-in on destroy | | `RemoveServiceDropin` | remove `container@<name>.service.d/` drop-in on destroy |

View file

@ -808,7 +808,7 @@ Bare `hivectl stop` stops everything; scope flags narrow it to specific sub-agen
* `--agent <NAME>` — A specific sub-agent by name. Repeatable: `--agent a --agent b` * `--agent <NAME>` — A specific sub-agent by name. Repeatable: `--agent a --agent b`
* `--ci` — The CI runner container (`hive-ci`) * `--ci` — The CI runner container (`hive-ci`)
* `--forge` — The forge container (`hive-forge`) * `--forge` — The forge container (`hive-forge`)
* `--gateway` — The gateway container (`hive-gateway`) * `--gateway` — The gateway (`hive-gateway`) — nginx on the host, not a container
* `--matrix` — The matrix container (`hive-matrix`) * `--matrix` — The matrix container (`hive-matrix`)
* `--graceful` — Gracefully quiesce each agent before stopping, instead of a hard stop. Each agent gets a graceful-stop DAG on the job queue: the harness is signalled, runs one stop-checkpoint turn to flush durable `/state`, drains, then the container is stopped (bounded by a 3-min timeout that falls back to a hard stop). All drains overlap. Applies to agents only * `--graceful` — Gracefully quiesce each agent before stopping, instead of a hard stop. Each agent gets a graceful-stop DAG on the job queue: the harness is signalled, runs one stop-checkpoint turn to flush durable `/state`, drains, then the container is stopped (bounded by a 3-min timeout that falls back to a hard stop). All drains overlap. Applies to agents only
* `--no-wait` — Return immediately after the stop DAGs are queued instead of waiting for them with live per-node progress * `--no-wait` — Return immediately after the stop DAGs are queued instead of waiting for them with live per-node progress
@ -829,7 +829,7 @@ Bare `hivectl start` restores the agents stopped by the last broad-scope `stop`
* `--agent <NAME>` — A specific sub-agent by name. Repeatable: `--agent a --agent b` * `--agent <NAME>` — A specific sub-agent by name. Repeatable: `--agent a --agent b`
* `--ci` — The CI runner container (`hive-ci`) * `--ci` — The CI runner container (`hive-ci`)
* `--forge` — The forge container (`hive-forge`) * `--forge` — The forge container (`hive-forge`)
* `--gateway` — The gateway container (`hive-gateway`) * `--gateway` — The gateway (`hive-gateway`) — nginx on the host, not a container
* `--matrix` — The matrix container (`hive-matrix`) * `--matrix` — The matrix container (`hive-matrix`)
* `--no-wait` — Return immediately after the start DAGs are queued instead of waiting for them with live per-node progress * `--no-wait` — Return immediately after the start DAGs are queued instead of waiting for them with live per-node progress
@ -849,7 +849,7 @@ Bare `hivectl restart` restarts everything; scope flags narrow it.
* `--agent <NAME>` — A specific sub-agent by name. Repeatable: `--agent a --agent b` * `--agent <NAME>` — A specific sub-agent by name. Repeatable: `--agent a --agent b`
* `--ci` — The CI runner container (`hive-ci`) * `--ci` — The CI runner container (`hive-ci`)
* `--forge` — The forge container (`hive-forge`) * `--forge` — The forge container (`hive-forge`)
* `--gateway` — The gateway container (`hive-gateway`) * `--gateway` — The gateway (`hive-gateway`) — nginx on the host, not a container
* `--matrix` — The matrix container (`hive-matrix`) * `--matrix` — The matrix container (`hive-matrix`)
* `--graceful` — Gracefully quiesce each agent on the stop half (see `stop --graceful`). Applies to agents only * `--graceful` — Gracefully quiesce each agent on the stop half (see `stop --graceful`). Applies to agents only

View file

@ -79,8 +79,10 @@ lifecycle events, or another container's boot log.
- `unit` — filter to a systemd unit (e.g. `hive-c0re.service`). - `unit` — filter to a systemd unit (e.g. `hive-c0re.service`).
- `container` — nspawn machine name verbatim. Agent containers use - `container` — nspawn machine name verbatim. Agent containers use
the `h-<name>` prefix (e.g. `h-iris`); infrastructure containers the `h-<name>` prefix (e.g. `h-iris`); infrastructure containers
use their full name (e.g. `hive-ci`, `hive-forge`, `hive-matrix`, use their full name (e.g. `hive-ci`, `hive-forge`, `hive-matrix`).
`hive-gateway`). Omit for the host journal. Omit for the host journal. The gateway has no machine — its nginx
runs on the host, so read it with `unit: nginx.service` and no
`container`.
- `lines` — how many lines to return (default 30, max 100). - `lines` — how many lines to return (default 30, max 100).
- `priority` — minimum syslog level (`emerg``debug`). - `priority` — minimum syslog level (`emerg``debug`).
- `grep` — regex matched against log message fields (`journalctl --grep`). - `grep` — regex matched against log message fields (`journalctl --grep`).

View file

@ -180,10 +180,12 @@ omitted — agents share the host netns, so there is no per-container net
counter (per-agent network needs the netns-isolation roadmap in counter (per-agent network needs the netns-isolation roadmap in
`docs/network.md`). `docs/network.md`).
**1NFR4** — start / stop / restart the four hive infrastructure **1NFR4** — start / stop / restart the four hive infrastructure services
containers (`hive-ci`, `hive-forge`, `hive-gateway`, `hive-matrix`) (`hive-ci`, `hive-forge`, `hive-gateway`, `hive-matrix`) directly from the
directly from the dashboard, without needing an `infra_admin` agent's dashboard, without needing an `infra_admin` agent's `restart` MCP tool.
`restart` MCP tool. One row per container: name, a `badge-ok`/`badge-fail` Three are containers; `hive-gateway` is the host's `nginx.service`, and is
the one an agent may **not** restart — this panel is the way it gets
bounced. One row per service: name, a `badge-ok`/`badge-fail`
running/stopped dot, and `↺ R3ST4RT` + `■ ST0P` (running) or `▶ ST4RT` running/stopped dot, and `↺ R3ST4RT` + `■ ST0P` (running) or `▶ ST4RT`
(stopped) buttons, same themed-confirm pattern as the K3PT ST4T3 (stopped) buttons, same themed-confirm pattern as the K3PT ST4T3
tombstone actions. Backed by tombstone actions. Backed by
@ -460,7 +462,7 @@ The current capabilities are:
| `manage_root_agent` | allows the `set_status` / lifecycle tools on the root agent | | `manage_root_agent` | allows the `set_status` / lifecycle tools on the root agent |
| `read_host_journal` | unlocks `get_host_journal` to read journald from inside a container | | `read_host_journal` | unlocks `get_host_journal` to read journald from inside a container |
| `query_agent_state` | allows `get_loose_ends(agent: "<name>")` calls targeting other agents | | `query_agent_state` | allows `get_loose_ends(agent: "<name>")` calls targeting other agents |
| `infra_admin` | allows `restart` on hive infrastructure containers (`hive-ci`, `hive-gateway`, `hive-forge`); each restart is logged to the AUDIT trail | | `infra_admin` | allows `restart` on hive infrastructure containers (`hive-ci`, `hive-forge`, `hive-matrix`; the gateway is operator-only); each restart is logged to the AUDIT trail |
Each row is one agent. Columns are the capability names returned by Each row is one agent. Columns are the capability names returned by
`GET /api/capabilities` as `caps: Vec<String>`. Checking or unchecking `GET /api/capabilities` as `caps: Vec<String>`. Checking or unchecking
@ -1340,9 +1342,11 @@ below — some endpoints aren't in it yet.
rootfs every ~5 min, `-x` excluding the shared read-only nix store. rootfs every ~5 min, `-x` excluding the shared read-only nix store.
`null` until the first sample lands. `null` until the first sample lands.
- `POST /api/infra-container/{name}/{action}` — start / stop / restart a - `POST /api/infra-container/{name}/{action}` — start / stop / restart a
hive infra container (C0R3 1NFR4 panel). `name` parses into the hive infra service (C0R3 1NFR4 panel). `name` parses into the
`InfraContainer` allowlist (`hive-ci`/`hive-forge`/`hive-gateway`/ `InfraContainer` allowlist (`hive-ci`/`hive-forge`/`hive-gateway`/
`hive-matrix`, 400 on unknown), `action``start|stop|restart`. Calls `hive-matrix`, 400 on unknown), and the variant decides the unit —
`container@<name>.service`, or `nginx.service` for the gateway.
`action``start|stop|restart`. Calls
the same `priv_client::control_infra_container` helper the the same `priv_client::control_infra_container` helper the
`infra_admin` agent path uses; records an `audit_log` entry `infra_admin` agent path uses; records an `audit_log` entry
(`start_infra`/`stop_infra`/`restart_infra`, actor `"operator"`) either (`start_infra`/`stop_infra`/`restart_infra`, actor `"operator"`) either

View file

@ -326,7 +326,8 @@ pub struct GetHostJournalArgs {
/// nspawn machine name verbatim (e.g. `h-iris`). Omit for host journal. /// nspawn machine name verbatim (e.g. `h-iris`). Omit for host journal.
/// Agent containers use the `h-` prefix (e.g. `h-iris`); infrastructure /// Agent containers use the `h-` prefix (e.g. `h-iris`); infrastructure
/// containers use their full name (e.g. `hive-ci`, `hive-forge`, /// containers use their full name (e.g. `hive-ci`, `hive-forge`,
/// `hive-matrix`, `hive-gateway`). /// `hive-matrix`). The gateway has no machine — its nginx runs on the
/// host, so read it with `unit: nginx.service` and no `container`.
#[serde(default)] #[serde(default)]
pub container: Option<String>, pub container: Option<String>,
/// Number of lines to return (default 30, max 100). /// Number of lines to return (default 30, max 100).

View file

@ -681,8 +681,9 @@ impl AgentServer {
Only succeeds if `name` is a direct child of this agent in the topology \ Only succeeds if `name` is a direct child of this agent in the topology \
tree the server enforces this. No approval required. \ tree the server enforces this. No approval required. \
Agents holding the `infra_admin` capability may also pass a hive \ Agents holding the `infra_admin` capability may also pass a hive \
infrastructure container name (`hive-ci`, `hive-gateway`, `hive-forge`) \ infrastructure container name (`hive-ci`, `hive-forge`, `hive-matrix`) \
to restart it directly via the privileged helper." to restart it directly via the privileged helper. The gateway is \
not restartable by an agent ask the operator."
)] )]
async fn restart(&self, Parameters(args): Parameters<RestartArgs>) -> String { async fn restart(&self, Parameters(args): Parameters<RestartArgs>) -> String {
let log = format!("{args:?}"); let log = format!("{args:?}");
@ -790,7 +791,8 @@ impl AgentServer {
`container`: nspawn machine name verbatim (e.g. `h-iris`). Omit for host journal. \ `container`: nspawn machine name verbatim (e.g. `h-iris`). Omit for host journal. \
Agent containers use the `h-` prefix (e.g. `h-iris`, `h-atlas`); \ Agent containers use the `h-` prefix (e.g. `h-iris`, `h-atlas`); \
infrastructure containers use their full name (e.g. `hive-ci`, `hive-forge`, \ infrastructure containers use their full name (e.g. `hive-ci`, `hive-forge`, \
`hive-matrix`, `hive-gateway`). \ `hive-matrix`). The gateway is not a container: its nginx logs are in \
the host journal, so omit `container` and pass `unit: nginx.service`. \
`lines`: how many lines (default 30, max 100). \ `lines`: how many lines (default 30, max 100). \
`priority`: minimum syslog level enum. \ `priority`: minimum syslog level enum. \
`grep`: regex matched against log message fields (journalctl --grep). \ `grep`: regex matched against log message fields (journalctl --grep). \

View file

@ -60,8 +60,8 @@ fn allowed_capability_tools() -> Vec<String> {
match t.as_str() { match t.as_str() {
"read_host_journal" => tools.push("get_host_journal".to_owned()), "read_host_journal" => tools.push("get_host_journal".to_owned()),
// infra_admin lets an agent restart hive infrastructure // infra_admin lets an agent restart hive infrastructure
// containers (hive-ci / hive-gateway / hive-forge) through the // containers (hive-ci / hive-forge / hive-matrix — not the
// existing `restart` tool. Unlock it here so agents that hold // gateway) through the existing `restart` tool. Unlock it here so agents that hold
// the capability without the full `lifecycle` group can still // the capability without the full `lifecycle` group can still
// call it; c0re re-checks the capability server-side and only // call it; c0re re-checks the capability server-side and only
// honours infra-container names via this path. // honours infra-container names via this path.

View file

@ -169,11 +169,12 @@ pub async fn serve(
/// Best-effort unlinks any stale socket left from a crashed previous /// Best-effort unlinks any stale socket left from a crashed previous
/// harness (clean exit removes it, but `bind(2)` refuses to overwrite /// harness (clean exit removes it, but `bind(2)` refuses to overwrite
/// an existing file) and `mkdir -p`s the parent for first-boot. Mode /// an existing file) and `mkdir -p`s the parent for first-boot. Mode
/// `0o666` — world-accessible so the gateway container's nginx process /// `0o666` — world-accessible so the gateway's nginx process can
/// can `connect(2)` without sharing a group with the agent user. /// `connect(2)` without sharing a group with the agent user. What bounds
/// The per-agent subdir (`/run/hive-agent/<name>/`) is only accessible /// that is the per-agent subdir (`/run/hive-agent/<name>/`): the socket
/// to containers that have it bind-mounted, so world-accessible sockets /// mode grants everyone, the directory decides who gets to ask. It used
/// are not a material risk. /// to be bind-mounted into the one container that needed it; nginx is a
/// host unit now, so the directory's own permissions are the whole story.
/// ///
/// Marker-gating + the gateway-side consumer: see /// Marker-gating + the gateway-side consumer: see
/// [`docs/gateway.md::Per-agent unix-socket upstream`](../../../docs/gateway.md). /// [`docs/gateway.md::Per-agent unix-socket upstream`](../../../docs/gateway.md).

View file

@ -1,9 +1,13 @@
//! Dashboard endpoint for operator-driven infra-container lifecycle //! Dashboard endpoint for operator-driven infra lifecycle (start / stop /
//! (start / stop / restart on `hive-ci`, `hive-forge`, `hive-gateway`, //! restart on `hive-ci`, `hive-forge`, `hive-gateway`, `hive-matrix`).
//! `hive-matrix`). Parallels the `infra_admin`-gated agent path in //! Parallels the `infra_admin`-gated agent path in
//! `socket_server/lifecycle_handlers.rs::handle_restart_infra`, but this one //! `socket_server/lifecycle_handlers.rs::handle_restart_infra`, but this one
//! is reached from the dashboard — already fully operator-authenticated — //! is reached from the dashboard — already fully operator-authenticated —
//! so no capability check is needed here, just the same audit trail. //! so no capability check is needed here, just the same audit trail.
//!
//! The two surfaces no longer cover the same set: the gateway is the
//! operator's to restart and not an agent's, since nginx on the host fronts
//! every hive service. This endpoint keeps all four.
use axum::{ use axum::{
extract::{Path as AxumPath, State}, extract::{Path as AxumPath, State},
@ -27,7 +31,7 @@ use super::{AppState, error_response};
post, post,
path = "/api/infra-container/{name}/{action}", path = "/api/infra-container/{name}/{action}",
params( params(
("name" = String, Path, description = "infra container name (hive-ci/hive-forge/hive-gateway/hive-matrix)"), ("name" = String, Path, description = "infra service name (hive-ci/hive-forge/hive-gateway/hive-matrix)"),
("action" = String, Path, description = "start | stop | restart"), ("action" = String, Path, description = "start | stop | restart"),
), ),
responses( responses(
@ -53,8 +57,8 @@ pub(super) async fn post_infra_container(
)); ));
} }
}; };
let unit = container.unit_name(); let target = container.name();
tracing::info!(%unit, %action, "dashboard: infra container action"); tracing::info!(%target, %action, "dashboard: infra container action");
let result = crate::priv_client::control_infra_container(container, infra_action).await; let result = crate::priv_client::control_infra_container(container, infra_action).await;
let outcome = if result.is_ok() { let outcome = if result.is_ok() {
crate::audit_log::AuditOutcome::Ok crate::audit_log::AuditOutcome::Ok
@ -66,12 +70,12 @@ pub(super) async fn post_infra_container(
state state
.coord .coord
.audit_log .audit_log
.record("operator", action_label, unit, outcome, detail.as_deref()) .record("operator", action_label, target, outcome, detail.as_deref())
{ {
state.coord.emit_audit_entry(entry); state.coord.emit_audit_entry(entry);
} }
match result { match result {
Ok(()) => (StatusCode::OK, "ok").into_response(), Ok(()) => (StatusCode::OK, "ok").into_response(),
Err(e) => error_response(&format!("{unit}: {e:#}")), Err(e) => error_response(&format!("{target}: {e:#}")),
} }
} }

View file

@ -1,13 +1,15 @@
//! Journal-read endpoints for the dashboard. //! Journal-read endpoints for the dashboard.
//! //!
//! `GET /api/journal/{name}` reads a managed agent container's journal, OR //! `GET /api/journal/{name}` reads a managed agent container's journal, OR
//! one of the four hive infra containers (`hive-ci`, `hive-forge`, //! one of the four hive infra services (`hive-ci`, `hive-forge`,
//! `hive-gateway`, `hive-matrix` — [`hive_priv_sock::InfraContainer`] is the //! `hive-gateway`, `hive-matrix` — [`hive_priv_sock::InfraContainer`] is the
//! allowlist), via the root helper (`journalctl -M`, delegated to hive-priv //! allowlist). A container's journal is a `journalctl -M` read, delegated
//! since hive-c0re is unprivileged). `GET /api/journal-host` reads //! to the root helper since entering a machine needs privileges hive-c0re
//! host-side journald, both gated by an allow-list of known units so //! doesn't have; `hive-gateway` is nginx on the host, so it reads host
//! arbitrary unit names can't be probed. Operator-only by virtue of the //! journald filtered to that unit and needs no helper at all.
//! dashboard binding host-only. //! `GET /api/journal-host` reads host-side journald, both gated by an
//! allow-list of known units so arbitrary unit names can't be probed.
//! Operator-only by virtue of the dashboard binding host-only.
use axum::{ use axum::{
extract::Path as AxumPath, extract::Path as AxumPath,
@ -40,11 +42,12 @@ pub(super) struct JournalQuery {
/// container namespace and needs root — is delegated to hive-priv. /// container namespace and needs root — is delegated to hive-priv.
/// ///
/// `name` is either a managed agent name (`iris`, optionally already /// `name` is either a managed agent name (`iris`, optionally already
/// carrying the `h-` prefix) or one of the four infra container names /// carrying the `h-` prefix) or one of the four infra names (`hive-ci` /
/// (`hive-ci` / `hive-forge` / `hive-gateway` / `hive-matrix` — see /// `hive-forge` / `hive-gateway` / `hive-matrix` — see
/// [`hive_priv_sock::InfraContainer`]). Infra containers don't run the /// [`hive_priv_sock::InfraContainer`]). Infra targets don't run the
/// per-agent hive daemons, so `unit` is ignored for them — always the /// per-agent hive daemons, so `unit` is ignored for them — the whole
/// full machine journal. /// machine journal, or for the gateway the host journal filtered to its
/// own unit.
#[utoipa::path( #[utoipa::path(
get, get,
path = "/api/journal/{name}", path = "/api/journal/{name}",
@ -66,7 +69,17 @@ pub(super) async fn get_journal(
let lines = q.lines.unwrap_or(500).min(5000); let lines = q.lines.unwrap_or(500).min(5000);
if let Ok(infra) = name.parse::<hive_priv_sock::InfraContainer>() { if let Ok(infra) = name.parse::<hive_priv_sock::InfraContainer>() {
return read_journal_response(infra.unit_name(), None, lines).await; return match infra.target() {
hive_priv_sock::InfraTarget::Container(machine) => {
read_journal_response(machine, None, lines).await
}
// No machine to enter — the gateway's nginx is a host unit, so
// this is a plain host-journal read filtered to it. `-M` is
// what needed root here, not journalctl itself.
hive_priv_sock::InfraTarget::HostUnit(unit) => {
read_host_journal_response(Some(unit), lines).await
}
};
} }
// Defense-in-depth format check so weird chars never reach the // Defense-in-depth format check so weird chars never reach the
@ -180,21 +193,43 @@ pub(super) async fn get_journal_host(
axum::extract::Query(q): axum::extract::Query<JournalHostQuery>, axum::extract::Query(q): axum::extract::Query<JournalHostQuery>,
) -> Result<Response, ProblemDetails> { ) -> Result<Response, ProblemDetails> {
let lines = q.lines.unwrap_or(500).min(5000); let lines = q.lines.unwrap_or(500).min(5000);
let allowed = ["hive-c0re.service", "hive-priv.service"]; // `nginx.service` is the gateway: its logs used to live in the
// hive-gateway container's journal and are host-side now.
let allowed = ["hive-c0re.service", "hive-priv.service", "nginx.service"];
let unit = match q.unit.as_deref().filter(|s| !s.is_empty()) {
Some(u) => {
let unit = if u.ends_with(".service") {
u.to_owned()
} else {
format!("{u}.service")
};
if !allowed.contains(&unit.as_str()) {
return Err(ProblemDetails::from_status_code(StatusCode::BAD_REQUEST)
.with_detail(format!("journal-host: unknown unit {unit:?}")));
}
Some(unit)
}
None => None,
};
read_host_journal_response(unit.as_deref(), lines).await
}
/// `journalctl [-u <unit>]` on the host + response formatting. No `-M`, so
/// no root and no priv-client hop — hive-c0re reads host journald directly.
///
/// ⚠️ `unit` is trusted by the time it gets here: [`get_journal_host`]
/// allow-lists an operator-supplied one, and [`get_journal`] passes a unit
/// that came from the [`hive_priv_sock::InfraContainer`] enum. Don't hand
/// this a raw query parameter.
async fn read_host_journal_response(
unit: Option<&str>,
lines: u32,
) -> Result<Response, ProblemDetails> {
let mut cmd = tokio::process::Command::new("journalctl"); let mut cmd = tokio::process::Command::new("journalctl");
cmd.args(["--no-pager", "--output=short-iso", "--lines"]) cmd.args(["--no-pager", "--output=short-iso", "--lines"])
.arg(lines.to_string()); .arg(lines.to_string());
if let Some(u) = q.unit.as_deref().filter(|s| !s.is_empty()) { if let Some(u) = unit {
let unit = if u.ends_with(".service") { cmd.args(["-u", u]);
u.to_owned()
} else {
format!("{u}.service")
};
if !allowed.contains(&unit.as_str()) {
return Err(ProblemDetails::from_status_code(StatusCode::BAD_REQUEST)
.with_detail(format!("journal-host: unknown unit {unit:?}")));
}
cmd.args(["-u", &unit]);
} }
match cmd.output().await { match cmd.output().await {
Ok(out) => { Ok(out) => {

View file

@ -154,7 +154,7 @@ async fn infra_container_views() -> Vec<InfraContainerView> {
let mut infra_containers = Vec::with_capacity(hive_priv_sock::InfraContainer::ALL.len()); let mut infra_containers = Vec::with_capacity(hive_priv_sock::InfraContainer::ALL.len());
for container in hive_priv_sock::InfraContainer::ALL { for container in hive_priv_sock::InfraContainer::ALL {
infra_containers.push(InfraContainerView { infra_containers.push(InfraContainerView {
name: container.unit_name(), name: container.name(),
running: crate::lifecycle::infra_is_running(container).await, running: crate::lifecycle::infra_is_running(container).await,
}); });
} }

View file

@ -1,8 +1,7 @@
//! Runtime nginx include-file generator for the gateway's per-agent //! Runtime nginx include-file generator for the gateway's per-agent
//! `/agent/<name>/` location blocks. Writes //! `/agent/<name>/` location blocks. Writes
//! `/var/lib/hyperhive/gateway/agents.conf` on every topology change. //! `/var/lib/hyperhive/gateway/agents.conf` on every topology change.
//! UDS upstream selection, reload trigger (`systemd-run //! UDS upstream selection, the reload trigger, and idempotency:
//! --machine=hive-gateway`), and idempotency:
//! `docs/gateway.md::Per-agent unix-socket upstream`. //! `docs/gateway.md::Per-agent unix-socket upstream`.
use anyhow::{Context, Result}; use anyhow::{Context, Result};
@ -19,13 +18,13 @@ use crate::agent_sockets;
/// Set when `write` publishes a new agents.conf; cleared when /// Set when `write` publishes a new agents.conf; cleared when
/// `reload_gateway_nginx` submits the reload command successfully. /// `reload_gateway_nginx` submits the reload command successfully.
/// Lets `spawn_poll` retry the reload on subsequent ticks when the /// Lets `spawn_poll` retry the reload on subsequent ticks when the
/// previous attempt failed (e.g. gateway container temporarily down, /// previous attempt failed (e.g. gateway nginx temporarily down, priv
/// systemd-run not found) without re-writing the already-correct file. /// helper unreachable) without re-writing the already-correct file.
static RELOAD_PENDING: AtomicBool = AtomicBool::new(false); static RELOAD_PENDING: AtomicBool = AtomicBool::new(false);
/// Unix timestamp (seconds) of the last failed reload attempt. /// Unix timestamp (seconds) of the last failed reload attempt.
/// `reload_if_pending` backs off to once per `RELOAD_RETRY_SECS` after a /// `reload_if_pending` backs off to once per `RELOAD_RETRY_SECS` after a
/// failure so a permanently broken gateway (bad config, container down) /// failure so a permanently broken gateway (bad config, nginx down)
/// doesn't hammer `systemctl` on every 10-second `spawn_poll` tick. /// doesn't hammer `systemctl` on every 10-second `spawn_poll` tick.
static LAST_FAILED_RELOAD: AtomicU64 = AtomicU64::new(0); static LAST_FAILED_RELOAD: AtomicU64 = AtomicU64::new(0);
@ -79,7 +78,7 @@ fn render(names: &[String], frontend_dir: Option<&str>) -> String {
let mut out = String::from( let mut out = String::from(
"# Generated by hive-c0re \u{2014} do not edit.\ "# Generated by hive-c0re \u{2014} do not edit.\
\n# Refreshed on every topology change + when agents bind/drop their unix sockets.\ \n# Refreshed on every topology change + when agents bind/drop their unix sockets.\
\n# Reload triggered by hive-c0re via systemd-run --machine=hive-gateway.\n", \n# Reload triggered by hive-c0re via hive-priv (systemctl reload nginx).\n",
); );
for name in names { for name in names {
// Two upstream forms because named locations (split mode's // Two upstream forms because named locations (split mode's
@ -165,15 +164,19 @@ fn render(names: &[String], frontend_dir: Option<&str>) -> String {
/// body matches what's already on disk (idempotent; avoids spurious /// body matches what's already on disk (idempotent; avoids spurious
/// gateway reloads on a quiet tick). /// gateway reloads on a quiet tick).
/// ///
/// After a successful write, triggers the appropriate nginx action inside /// After a successful write, triggers the appropriate nginx action via
/// the gateway container via `hive-priv` (which has the /// `hive-priv` (hive-c0re runs unprivileged and cannot act on a system
/// `--machine=hive-gateway` transport rights hive-c0re lacks): /// unit): reload when nginx is active, reset-failed+start when in a
/// reload when nginx is active, reset-failed+start when in a failed /// failed state, plain start otherwise. Writer and nginx are now on the
/// state, plain start otherwise. This is intentionally host-side rather /// same machine, so this is a plain unit action rather than the old
/// than relying on a systemd path unit inside the container watching the /// `systemd-run --machine=hive-gateway` hop across the container
/// bind-mounted file: `IN_MOVED_TO` (fired by the atomic rename) does /// boundary. It stays an explicit trigger rather than a systemd path
/// not reliably propagate across the nspawn mount-namespace boundary, so /// unit watching the file. A path unit would now *work* — `IN_MOVED_TO`
/// the path-unit approach was silently broken (see `docs/gateway.md`). /// (fired by the atomic rename) failed to propagate across the nspawn
/// mount-namespace boundary, and that boundary is gone — but it is still
/// not wanted: the write already knows it changed something, and a
/// watcher turns one causal edge into a race with the writer's own
/// rename (see `docs/gateway.md`).
/// ///
/// The priv call is best-effort — a failed sync is logged but not fatal. /// The priv call is best-effort — a failed sync is logged but not fatal.
/// `reload_if_pending` retries on the next `spawn_poll` tick so a /// `reload_if_pending` retries on the next `spawn_poll` tick so a
@ -212,7 +215,7 @@ pub async fn write(names: &[String]) -> Result<()> {
/// Retry a pending nginx reload if a previous attempt failed. /// Retry a pending nginx reload if a previous attempt failed.
/// Called by `spawn_poll` on each tick so a transient failure /// Called by `spawn_poll` on each tick so a transient failure
/// (gateway container temporarily down, systemd-run error) is /// (gateway nginx temporarily down, priv-helper error) is
/// recovered automatically without requiring a new file write. /// recovered automatically without requiring a new file write.
/// ///
/// Backs off to one retry per `RELOAD_RETRY_SECS` after a failure so a /// Backs off to one retry per `RELOAD_RETRY_SECS` after a failure so a
@ -240,8 +243,8 @@ pub async fn reload_if_pending() {
/// Synchronise the gateway nginx unit with the current agents.conf via /// Synchronise the gateway nginx unit with the current agents.conf via
/// `hive-priv` (privileged helper). The state-aware logic (active → /// `hive-priv` (privileged helper). The state-aware logic (active →
/// reload; failed → reset-failed + start; inactive/unknown → start) /// reload; failed → reset-failed + start; inactive/unknown → start)
/// runs inside hive-priv where it has the `--machine=hive-gateway` /// runs inside hive-priv, which is root; hive-c0re runs as the
/// transport rights that hive-c0re (unprivileged) lacks. /// unprivileged `hive-core` user and cannot act on a system unit.
/// ///
/// `RELOAD_PENDING` is cleared only after a successful operation so /// `RELOAD_PENDING` is cleared only after a successful operation so
/// `reload_if_pending` keeps retrying on failure. /// `reload_if_pending` keeps retrying on failure.

View file

@ -604,14 +604,14 @@ pub async fn is_running(name: &str) -> bool {
.is_ok_and(|s| s.success()) .is_ok_and(|s| s.success())
} }
/// True when a hive infrastructure container's systemd unit is active. /// True when a hive infrastructure service's systemd unit is active.
/// Sibling of [`is_running`] for sub-agents, but infra container/unit names /// Sibling of [`is_running`] for sub-agents, but infra names (`hive-ci`, …)
/// (`hive-ci`, …) already have no `h-` prefix to strip, so this queries /// have no `h-` prefix to strip and are not all containers, so the unit
/// `container@<unit_name>.service` directly rather than going through /// comes from the variant itself rather than from [`container_name`]. Used
/// [`container_name`]. Used by the dashboard C0R3 page's 1NFR4 sub-tab to /// by the dashboard C0R3 page's 1NFR4 sub-tab to show each one's live
/// show each infra container's live status dot. /// status dot.
pub async fn infra_is_running(container: hive_priv_sock::InfraContainer) -> bool { pub async fn infra_is_running(container: hive_priv_sock::InfraContainer) -> bool {
let unit = format!("container@{}.service", container.unit_name()); let unit = container.service_unit();
Command::new("systemctl") Command::new("systemctl")
.args(["is-active", "--quiet", &unit]) .args(["is-active", "--quiet", &unit])
.status() .status()

View file

@ -168,9 +168,9 @@ pub async fn sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<()> {
} }
// Refresh /var/lib/hyperhive/gateway/agents.conf — the nginx include // Refresh /var/lib/hyperhive/gateway/agents.conf — the nginx include
// file the gateway container bind-mounts and nginx reads at runtime. // file the gateway reads at runtime. c0re then triggers a reload (or
// c0re triggers a reload (or start) inside hive-gateway via hive-priv // start) of the host's nginx via hive-priv, since c0re is unprivileged.
// after writing the file. Same best-effort + non-fatal shape. // Same best-effort + non-fatal shape.
if let Err(e) = crate::gateway_nginx::write(&agent_names).await { if let Err(e) = crate::gateway_nginx::write(&agent_names).await {
tracing::warn!(error = ?e, "gateway_nginx::write failed (non-fatal)"); tracing::warn!(error = ?e, "gateway_nginx::write failed (non-fatal)");
} }

View file

@ -219,11 +219,12 @@ pub fn shared_root() -> PathBuf {
pub const KNOWLEDGE_DIR: &str = "/var/lib/hyperhive/knowledge"; pub const KNOWLEDGE_DIR: &str = "/var/lib/hyperhive/knowledge";
/// `gateway/` — generated nginx include fragments for the gateway vhost. /// `gateway/` — generated nginx include fragments for the gateway vhost.
/// The gateway container bind-mounts *this subdir only* (not the whole /// nginx runs on the host and reads this path directly; it used to be
/// state root) at `/run/hive-state/`, so nginx can read `agents.conf` /// bind-mounted into a gateway container at `/run/hive-state/`, exposing
/// without the rest of `/var/lib/hyperhive/` (forge/matrix tokens, etc.) /// this subdir *only* so the rest of `/var/lib/hyperhive/` (forge/matrix
/// being exposed to the gateway container. /// tokens, etc.) stayed out of reach. On the host that narrowing is the
// nix: bind-mounted into the gateway container (hive-gateway.nix) — must match. /// unit's sandbox, not a mount — nginx is not confined by this path.
// nix: named by the gateway's nginx config (hive-gateway/vhosts.nix) — must match.
#[must_use] #[must_use]
pub fn gateway_dir() -> PathBuf { pub fn gateway_dir() -> PathBuf {
state_root().join("gateway") state_root().join("gateway")

View file

@ -477,20 +477,21 @@ pub async fn register_ci_runner(token: &str) -> Result<()> {
.await?) .await?)
} }
/// Restart a hive infrastructure container on the host (thin wrapper over /// Restart a hive infrastructure service on the host (thin wrapper over
/// [`control_infra_container`] with `action = Restart`). hive-priv /// [`control_infra_container`] with `action = Restart`). Callers must
/// re-validates `container` against its root-side allowlist; callers must /// already have checked that the requesting agent holds the `infra_admin`
/// already have checked the requesting agent holds the `infra_admin` /// capability *and* that the target is
/// capability. /// [`agent_restartable`](InfraContainer::agent_restartable).
pub async fn restart_infra_container(container: InfraContainer) -> Result<()> { pub async fn restart_infra_container(container: InfraContainer) -> Result<()> {
control_infra_container(container, InfraAction::Restart).await control_infra_container(container, InfraAction::Restart).await
} }
/// Start / stop / restart a hive infrastructure container (`hive-ci`, /// Start / stop / restart a hive infrastructure service (`hive-ci`,
/// `hive-gateway`, `hive-forge`, `hive-matrix`) on the host via `systemctl /// `hive-gateway`, `hive-forge`, `hive-matrix`) on the host via `systemctl
/// <action> container@<container>.service`. The [`InfraContainer`] enum is /// <action> <unit>`, where the unit is derived root-side from the variant
/// the allowlist — hive-priv needs no name re-validation. Used by the /// (`container@<name>.service`, or `nginx.service` for the gateway). The
/// hive-wide `hivectl stop` / `hivectl start` flow. /// [`InfraContainer`] enum is the allowlist — hive-priv needs no name
/// re-validation. Used by the hive-wide `hivectl stop` / `start` flow.
pub async fn control_infra_container(container: InfraContainer, action: InfraAction) -> Result<()> { pub async fn control_infra_container(container: InfraContainer, action: InfraAction) -> Result<()> {
ok(call(&PrivRequest::ControlInfraContainer { container, action }).await?) ok(call(&PrivRequest::ControlInfraContainer { container, action }).await?)
} }

View file

@ -863,7 +863,7 @@ async fn handle_stop(
await_dags(coord, &queued, std::time::Duration::from_mins(2)).await; await_dags(coord, &queued, std::time::Duration::from_mins(2)).await;
} }
for &container in infra { for &container in infra {
let name = container.unit_name(); let name = container.name();
match crate::priv_client::control_infra_container(container, InfraAction::Stop).await { match crate::priv_client::control_infra_container(container, InfraAction::Stop).await {
Ok(()) => ok_items.push(name.to_owned()), Ok(()) => ok_items.push(name.to_owned()),
Err(e) => { Err(e) => {
@ -919,7 +919,7 @@ async fn handle_start(
let mut errors: Vec<String> = Vec::new(); let mut errors: Vec<String> = Vec::new();
for &container in infra { for &container in infra {
let name = container.unit_name(); let name = container.name();
match crate::priv_client::control_infra_container(container, InfraAction::Start).await { match crate::priv_client::control_infra_container(container, InfraAction::Start).await {
Ok(()) => ok_items.push(name.to_owned()), Ok(()) => ok_items.push(name.to_owned()),
Err(e) => { Err(e) => {
@ -990,7 +990,7 @@ async fn handle_restart_scoped(
} }
for &container in &infra { for &container in &infra {
let name = container.unit_name(); let name = container.name();
let res = async { let res = async {
crate::priv_client::control_infra_container(container, InfraAction::Stop).await?; crate::priv_client::control_infra_container(container, InfraAction::Stop).await?;
crate::priv_client::control_infra_container(container, InfraAction::Start).await crate::priv_client::control_infra_container(container, InfraAction::Start).await

View file

@ -31,12 +31,13 @@ pub(super) async fn handle_start(coord: &Arc<Coordinator>, agent: &str, name: &s
/// orthogonal: it is gated on the `infra_admin` capability and audited, so it /// orthogonal: it is gated on the `infra_admin` capability and audited, so it
/// stays ahead of the topology guard. /// stays ahead of the topology guard.
pub(super) async fn handle_restart(coord: &Arc<Coordinator>, agent: &str, name: &str) -> Response { pub(super) async fn handle_restart(coord: &Arc<Coordinator>, agent: &str, name: &str) -> Response {
// Infra-container restart: an agent holding the `infra_admin` // Infra restart: an agent holding the `infra_admin` capability can
// capability can restart a hive infrastructure container (hive-ci / // restart a hive infrastructure service (hive-ci / hive-forge /
// hive-gateway / hive-forge / hive-matrix) by passing its name to the // hive-matrix) by passing its name to the same restart tool. The
// same restart tool. The `InfraContainer` enum parse both recognises // `InfraContainer` enum parse both recognises these (never agent
// these (never agent children, so disjoint from the child path below) // children, so disjoint from the child path below) and yields the typed
// and yields the typed value the restart path needs. // value the restart path needs. It recognises `hive-gateway` too, which
// is then refused — a name the agent surface knows but may not act on.
if let Ok(container) = name.parse::<hive_priv_sock::InfraContainer>() { if let Ok(container) = name.parse::<hive_priv_sock::InfraContainer>() {
return handle_restart_infra(coord, agent, container).await; return handle_restart_infra(coord, agent, container).await;
} }
@ -60,7 +61,7 @@ async fn handle_restart_infra(
agent: &str, agent: &str,
container: hive_priv_sock::InfraContainer, container: hive_priv_sock::InfraContainer,
) -> Response { ) -> Response {
let name = container.unit_name(); let name = container.name();
// Record the attempt in the operator-visible privileged-action audit // Record the attempt in the operator-visible privileged-action audit
// trail, then emit a live `AuditEntryAdded` so the dashboard audit view // trail, then emit a live `AuditEntryAdded` so the dashboard audit view
// appends it off `/dashboard/stream`. Best-effort: `record` returns the // appends it off `/dashboard/stream`. Best-effort: `record` returns the
@ -75,6 +76,22 @@ async fn handle_restart_infra(
coord.emit_audit_entry(entry); coord.emit_audit_entry(entry);
} }
}; };
// Some targets are off-limits to agents regardless of capability — the
// gateway, because nginx fronts every hive service from the host and an
// agent bouncing it takes out the forge, the dashboard and matrix at
// once, including the route its own fix would have to travel. Checked
// before the capability so the refusal doesn't read as "ask for
// infra_admin"; no capability grants this.
if !container.agent_restartable() {
tracing::warn!(%agent, %name, "agent: infra restart denied (not agent-restartable)");
audit(
crate::audit_log::AuditOutcome::Err,
Some("denied: target is not agent-restartable"),
);
return Response::Err {
message: format!("`{name}` cannot be restarted by an agent; ask the operator"),
};
}
if !crate::capabilities::has_cap(agent, hive_sh4re::permissions::Capability::InfraAdmin) { if !crate::capabilities::has_cap(agent, hive_sh4re::permissions::Capability::InfraAdmin) {
tracing::warn!(%agent, %name, "agent: infra restart denied (no infra_admin capability)"); tracing::warn!(%agent, %name, "agent: infra restart denied (no infra_admin capability)");
audit( audit(

View file

@ -14,10 +14,11 @@ use std::path::{Path, PathBuf};
use anyhow::{Context, Result}; use anyhow::{Context, Result};
/// Host-side parent directory holding per-agent socket subdirs. The /// Host-side parent directory holding per-agent socket subdirs. The
/// gateway container bind-mounts this whole tree (read-only) so it /// gateway's nginx runs on the host and reads this whole tree, so it
/// can `proxy_pass` to any agent. Each agent's container bind-mounts /// can `proxy_pass` to any agent — it used to get there through a
/// only its own `<name>/` subdir — agents can only access their own /// read-only bind-mount of the same tree. Each agent's container
/// sockets. The literal lives in `hive-host-sock` (shared with /// bind-mounts only its own `<name>/` subdir, which is still what stops
/// one agent reaching another's socket. The literal lives in `hive-host-sock` (shared with
/// `hivectl`); re-exported here under the name this module's consumers /// `hivectl`); re-exported here under the name this module's consumers
/// have always used. /// have always used.
pub use hive_host_sock::AGENT_SOCKET_DIR; pub use hive_host_sock::AGENT_SOCKET_DIR;
@ -166,7 +167,7 @@ pub fn write(names: &[String]) -> Result<()> {
/// ///
/// Also calls `gateway_nginx::reload_if_pending` on every tick to /// Also calls `gateway_nginx::reload_if_pending` on every tick to
/// retry a gateway nginx reload that may have failed on the previous /// retry a gateway nginx reload that may have failed on the previous
/// tick (e.g. gateway container temporarily down). This recovers /// tick (e.g. gateway nginx temporarily down). This recovers
/// gateway routing without needing a manual gateway restart. /// gateway routing without needing a manual gateway restart.
/// ///
/// Mirrors the spawn-loop shape used by `crash_watch`, /// Mirrors the spawn-loop shape used by `crash_watch`,

View file

@ -50,7 +50,7 @@ pub fn agent_state_dir(name: &Ident) -> PathBuf {
/// `gateway/gateway.htpasswd` — nginx basic-auth credential store for the /// `gateway/gateway.htpasswd` — nginx basic-auth credential store for the
/// operator dashboard vhost. `hivectl`'s `--htpasswd-file` clap default. /// operator dashboard vhost. `hivectl`'s `--htpasswd-file` clap default.
// nix: read by the gateway container's nginx (hive-gateway.nix) — must match. // nix: read by the gateway's nginx on the host (hive-gateway/) — must match.
pub const GATEWAY_HTPASSWD: &str = "/var/lib/hyperhive/gateway/gateway.htpasswd"; pub const GATEWAY_HTPASSWD: &str = "/var/lib/hyperhive/gateway/gateway.htpasswd";
/// `/run/hive-agent` — per-agent runtime socket dir root (web UI unix /// `/run/hive-agent` — per-agent runtime socket dir root (web UI unix
@ -452,7 +452,8 @@ pub struct QuotaRow {
/// (the bare `hivectl stop` / `start`); set individual fields to restrict /// (the bare `hivectl stop` / `start`); set individual fields to restrict
/// (e.g. only `agents` → just the sub-agent containers). `agents` covers /// (e.g. only `agents` → just the sub-agent containers). `agents` covers
/// every managed sub-agent container; the rest are the named infra /// every managed sub-agent container; the rest are the named infra
/// containers (`hive-ci`, `hive-forge`, `hive-gateway`, `hive-matrix`). /// services (`hive-ci`, `hive-forge`, `hive-gateway`, `hive-matrix`) —
/// containers, bar the gateway, which is the host's nginx unit.
// //
// A flat bag of independent flag toggles — one bool per selectable // A flat bag of independent flag toggles — one bool per selectable
// container class — is exactly the right shape here: each maps 1:1 to a // container class — is exactly the right shape here: each maps 1:1 to a

View file

@ -33,17 +33,24 @@ pub const MANAGER_NAME: &str = "ruth";
pub const AGENT_PREFIX: &str = "h-"; pub const AGENT_PREFIX: &str = "h-";
/// Sibling service containers managed by hive-c0re. This doubles as the /// Sibling service containers managed by hive-c0re. This doubles as the
/// authoritative allowlist for infra lifecycle ops /// authoritative allowlist for the requests that name a container as a
/// ([`PrivRequest::ControlInfraContainer`]): any of these four may be /// string (bind-mount edits, journal reads): only these — or a valid agent
/// started / stopped / restarted (by the hive-wide `hivectl stop`/`start` /// name — are accepted. `hive-c0re` is deliberately absent; so is the
/// flow or an `infra_admin` agent's `restart`). `hive-c0re` is deliberately /// gateway, whose nginx is a plain host unit rather than a container.
/// absent — stopping it would sever the very socket the request arrived on.
/// hive-priv re-validates against this list root-side, so it's authoritative /// hive-priv re-validates against this list root-side, so it's authoritative
/// regardless of what the caller sends. /// regardless of what the caller sends.
pub const SIBLING_CONTAINERS: &[&str] = &["hive-forge", "hive-matrix", "hive-gateway", "hive-ci"]; ///
/// ⚠️ This is the *container-name* allowlist, not the lifecycle one:
/// [`InfraContainer`] is what gates
/// [`PrivRequest::ControlInfraContainer`], and it has one variant more than
/// this list (the gateway). Keep the distinction — a name that belongs to
/// no container has no business reaching a `-M` / `nixos-container` call.
pub const SIBLING_CONTAINERS: &[&str] = &["hive-forge", "hive-matrix", "hive-ci"];
/// Lifecycle verb for [`PrivRequest::ControlInfraContainer`]. Maps directly /// Lifecycle verb for [`PrivRequest::ControlInfraContainer`]. Maps directly
/// to `systemctl <verb> container@<container>.service`. /// to `systemctl <verb> <unit>`, where the unit comes from
/// [`InfraContainer::service_unit`] — usually `container@<name>.service`,
/// but not always (see [`InfraTarget`]).
#[derive(Debug, Clone, Copy, Serialize, Deserialize)] #[derive(Debug, Clone, Copy, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")] #[serde(rename_all = "snake_case")]
pub enum InfraAction { pub enum InfraAction {
@ -71,8 +78,9 @@ impl InfraAction {
/// by a runtime check. The c0re↔hive-priv wire form uses serde's default /// by a runtime check. The c0re↔hive-priv wire form uses serde's default
/// variant naming (`"Ci"`, `"Forge"`, …); it's an internal protocol (both /// variant naming (`"Ci"`, `"Forge"`, …); it's an internal protocol (both
/// ends rebuild together) so it needn't match the container name. /// ends rebuild together) so it needn't match the container name.
/// [`unit_name`](Self::unit_name) is the separate systemd / container name /// [`name`](Self::name) is the separate stable identity string
/// (`hive-ci`). /// (`hive-ci`), and [`service_unit`](Self::service_unit) the systemd unit
/// it actually resolves to.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
pub enum InfraContainer { pub enum InfraContainer {
Ci, Ci,
@ -81,9 +89,27 @@ pub enum InfraContainer {
Matrix, Matrix,
} }
/// What an [`InfraContainer`] resolves to on the host — i.e. the thing a
/// lifecycle verb actually acts on.
///
/// The gateway is why this exists: its nginx + dnsmasq were lifted out of
/// an nspawn container and onto the host, so "restart the gateway" is a
/// plain host unit now. The operator verb is unchanged; only its target
/// moved. Everything else is still a container.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum InfraTarget {
/// An nspawn container, controlled via `container@<name>.service` and
/// readable with `journalctl -M <name>`.
Container(&'static str),
/// A plain host unit. There is no machine to enter: no `-M` journal,
/// no `nixos-container` verb.
HostUnit(&'static str),
}
impl InfraContainer { impl InfraContainer {
/// Every controllable infra container. The source of truth that /// Every controllable infra target. A superset of
/// [`SIBLING_CONTAINERS`] is kept consistent with (see the test). /// [`SIBLING_CONTAINERS`] — not all of these are containers (see the
/// test).
pub const ALL: [InfraContainer; 4] = [ pub const ALL: [InfraContainer; 4] = [
InfraContainer::Ci, InfraContainer::Ci,
InfraContainer::Forge, InfraContainer::Forge,
@ -91,11 +117,14 @@ impl InfraContainer {
InfraContainer::Matrix, InfraContainer::Matrix,
]; ];
/// The container / systemd-unit name, e.g. `hive-ci` → /// Stable identity string, e.g. `hive-ci`. This is what the operator
/// `container@hive-ci.service`. (Distinct from the serde wire form, /// types, what the dashboard displays, and what [`FromStr`](std::str::FromStr)
/// which is the default variant name `"Ci"`.) /// parses — it stays `hive-gateway` even though the gateway is no
/// longer a container, because it names the *service*, not its
/// implementation. (Distinct from the serde wire form, which is the
/// default variant name `"Ci"`.)
#[must_use] #[must_use]
pub fn unit_name(self) -> &'static str { pub fn name(self) -> &'static str {
match self { match self {
InfraContainer::Ci => "hive-ci", InfraContainer::Ci => "hive-ci",
InfraContainer::Forge => "hive-forge", InfraContainer::Forge => "hive-forge",
@ -103,16 +132,48 @@ impl InfraContainer {
InfraContainer::Matrix => "hive-matrix", InfraContainer::Matrix => "hive-matrix",
} }
} }
/// Where this target lives on the host.
#[must_use]
pub fn target(self) -> InfraTarget {
match self {
InfraContainer::Gateway => InfraTarget::HostUnit("nginx.service"),
other => InfraTarget::Container(other.name()),
}
}
/// The systemd unit a lifecycle verb acts on.
#[must_use]
pub fn service_unit(self) -> String {
match self.target() {
InfraTarget::Container(name) => format!("container@{name}.service"),
InfraTarget::HostUnit(unit) => unit.to_owned(),
}
}
/// Whether an agent holding `infra_admin` may restart this target.
///
/// The gateway is excluded by operator ruling: nginx now fronts every
/// hive service from the host, so an agent restarting it can take the
/// forge, dashboard and matrix down with it — including the path its
/// own PR would have to travel to fix it. The operator surface
/// (`hivectl`, dashboard) is unaffected.
#[must_use]
pub fn agent_restartable(self) -> bool {
!matches!(self, InfraContainer::Gateway)
}
} }
impl std::str::FromStr for InfraContainer { impl std::str::FromStr for InfraContainer {
type Err = (); type Err = ();
/// Parse a container name (`hive-ci`, …) into a variant. Used to decide /// Parse an infra name (`hive-ci`, …) into a variant. Recognition
/// whether an MCP `restart(<name>)` target is a controllable infra /// only — it says the name denotes a hive service, *not* that the
/// container. `Err(())` for anything that isn't one. /// caller may act on it. The agent restart path additionally checks
/// [`agent_restartable`](InfraContainer::agent_restartable).
/// `Err(())` for anything that isn't one.
fn from_str(s: &str) -> Result<Self, ()> { fn from_str(s: &str) -> Result<Self, ()> {
Self::ALL.into_iter().find(|c| c.unit_name() == s).ok_or(()) Self::ALL.into_iter().find(|c| c.name() == s).ok_or(())
} }
} }
@ -364,15 +425,25 @@ pub enum PrivRequest {
/// Run `systemctl daemon-reload`. /// Run `systemctl daemon-reload`.
DaemonReload, DaemonReload,
/// Synchronise the nginx unit inside the `hive-gateway` container. /// Synchronise the host's nginx unit after an `agents.conf` write.
/// ///
/// hive-priv queries `ActiveState` and dispatches: /// hive-priv queries `ActiveState` and dispatches:
/// - `active` → `systemctl reload nginx` (SIGHUP, zero-downtime) /// - `active` → `systemctl reload nginx` (SIGHUP, zero-downtime)
/// - `failed` → `systemctl reset-failed nginx` + `systemctl start nginx` /// - `failed` → `systemctl reset-failed nginx` + `systemctl start nginx`
/// - otherwise → `systemctl start nginx` /// - otherwise → `systemctl start nginx`
/// ///
/// Requires root: `--machine=hive-gateway` enters the container /// Requires root because hive-c0re runs as the unprivileged
/// namespace via the machine bus (forbidden for unprivileged users). /// `hive-core` user and cannot act on a system unit. It used to be
/// root for a *different* reason — `--machine=hive-gateway` entering
/// the container's namespace over the machine bus — and that reason
/// died with the container: nginx is a host unit now. The
/// requirement survived the move; its justification did not.
///
/// ⚠️ The unit name is **not** a parameter and must stay that way.
/// `--machine=` was doing double duty — transport *and* scope — so
/// dropping it removed the containment along with the namespace hop.
/// Hard-coding `nginx` is what replaces it: a caller cannot name the
/// unit, so this verb cannot be steered at any other service.
ReloadGatewayNginx, ReloadGatewayNginx,
// --- Forge admin CLI --- // --- Forge admin CLI ---
@ -824,13 +895,14 @@ pub enum PrivEvent {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::{InfraContainer, SIBLING_CONTAINERS}; use super::{InfraContainer, InfraTarget, SIBLING_CONTAINERS};
#[test] #[test]
fn infra_control_allowlist_excludes_c0re_includes_matrix() { fn infra_control_allowlist_excludes_c0re_includes_matrix() {
// SIBLING_CONTAINERS is the authoritative allowlist for infra // SIBLING_CONTAINERS is the authoritative allowlist for the
// lifecycle ops. hive-c0re must NEVER be in it — stopping the daemon // requests that name a container as a string. hive-c0re must NEVER
// would sever the socket the request arrived on. // be in it — stopping the daemon would sever the socket the request
// arrived on.
assert!(!SIBLING_CONTAINERS.contains(&"hive-c0re")); assert!(!SIBLING_CONTAINERS.contains(&"hive-c0re"));
// hive-matrix IS controllable (operator can stop/start/restart it). // hive-matrix IS controllable (operator can stop/start/restart it).
assert!(SIBLING_CONTAINERS.contains(&"hive-matrix")); assert!(SIBLING_CONTAINERS.contains(&"hive-matrix"));
@ -838,26 +910,62 @@ mod tests {
#[test] #[test]
fn infra_container_enum_matches_sibling_containers() { fn infra_container_enum_matches_sibling_containers() {
// The InfraContainer enum (the control-path allowlist) and the // The two lists are the same set *minus the things that aren't
// SIBLING_CONTAINERS slice (the general container-name validator) // containers*: every `Container` variant must appear in
// must list exactly the same four containers — they're separate // SIBLING_CONTAINERS and vice versa, so a `-M` journal read or a
// surfaces for the same set, so keep them in lockstep. // `nixos-container` verb can never be pointed at a name that has no
let mut from_enum: Vec<&str> = InfraContainer::ALL.iter().map(|c| c.unit_name()).collect(); // machine behind it.
let mut from_enum: Vec<&str> = InfraContainer::ALL
.iter()
.filter_map(|c| match c.target() {
InfraTarget::Container(name) => Some(name),
InfraTarget::HostUnit(_) => None,
})
.collect();
from_enum.sort_unstable(); from_enum.sort_unstable();
let mut from_slice: Vec<&str> = SIBLING_CONTAINERS.to_vec(); let mut from_slice: Vec<&str> = SIBLING_CONTAINERS.to_vec();
from_slice.sort_unstable(); from_slice.sort_unstable();
assert_eq!(from_enum, from_slice); assert_eq!(from_enum, from_slice);
// The gateway is the one that is not: a host unit, and absent from
// the container-name allowlist.
assert_eq!(
InfraContainer::Gateway.target(),
InfraTarget::HostUnit("nginx.service")
);
assert!(!SIBLING_CONTAINERS.contains(&"hive-gateway"));
// hive-c0re has no variant — unrepresentable, can't be controlled. // hive-c0re has no variant — unrepresentable, can't be controlled.
assert!("hive-c0re".parse::<InfraContainer>().is_err()); assert!("hive-c0re".parse::<InfraContainer>().is_err());
} }
#[test] #[test]
fn infra_container_name_round_trips() { fn infra_container_name_round_trips() {
// `unit_name` is the single source of truth for the wire form (the // `name` is the single source of truth for the operator-facing form
// serde impls + FromStr all key off it), so a name→variant→name // (FromStr keys off it), so a name→variant→name round-trip proves
// round-trip proves the mapping is consistent in both directions. // the mapping is consistent in both directions. It holds for the
// gateway too: the name is still recognised, it's the *permission*
// that differs (see below), not the parse.
for c in InfraContainer::ALL { for c in InfraContainer::ALL {
assert_eq!(c.unit_name().parse::<InfraContainer>(), Ok(c)); assert_eq!(c.name().parse::<InfraContainer>(), Ok(c));
}
}
#[test]
fn service_unit_wraps_containers_but_not_host_units() {
assert_eq!(
InfraContainer::Ci.service_unit(),
"container@hive-ci.service"
);
assert_eq!(InfraContainer::Gateway.service_unit(), "nginx.service");
}
#[test]
fn only_the_gateway_is_off_limits_to_agents() {
// Recognising a name and being allowed to restart it are separate
// questions — the gateway parses fine and is still refused.
assert!("hive-gateway".parse::<InfraContainer>().is_ok());
assert!(!InfraContainer::Gateway.agent_restartable());
for c in InfraContainer::ALL {
assert_eq!(c.agent_restartable(), c != InfraContainer::Gateway, "{c:?}");
} }
} }
} }

View file

@ -888,18 +888,23 @@ async fn register_ci_runner(token: &str) -> Result<(String, String)> {
} }
/// `ControlInfraContainer` — start/stop/restart a hive infrastructure /// `ControlInfraContainer` — start/stop/restart a hive infrastructure
/// container via `systemctl <verb> container@<container>.service`. The /// service via `systemctl <verb> <unit>`. The [`InfraContainer`] enum is
/// [`InfraContainer`] enum is the allowlist: serde already rejected any /// the allowlist: serde already rejected any unknown / unsafe name
/// unknown / unsafe name (hive-c0re has no variant, so a stop can't sever /// (hive-c0re has no variant, so a stop can't sever the daemon socket) at
/// the daemon socket) at deserialisation, so no root-side `.contains()` /// deserialisation, so no root-side `.contains()` check is needed here.
/// check is needed here. Serves both the hive-wide `hivectl stop`/`start` /// Serves both the hive-wide `hivectl stop`/`start` flow and an
/// flow and an `infra_admin` agent's `restart` (action = Restart). /// `infra_admin` agent's `restart` (action = Restart).
///
/// ⚠️ The unit is derived from the variant, never sent by the caller —
/// which is what keeps this from being a general `systemctl` pass-through.
/// It is not always `container@<name>.service`: the gateway resolves to the
/// host's `nginx.service`.
async fn control_infra_container( async fn control_infra_container(
container: InfraContainer, container: InfraContainer,
action: InfraAction, action: InfraAction,
) -> Result<(String, String)> { ) -> Result<(String, String)> {
let verb = action.systemctl_verb(); let verb = action.systemctl_verb();
let unit = format!("container@{}.service", container.unit_name()); let unit = container.service_unit();
let out = Command::new("systemctl") let out = Command::new("systemctl")
.args([verb, &unit]) .args([verb, &unit])
.output() .output()
@ -2173,34 +2178,33 @@ async fn read_container_journal(container: &str, query: &JournalQuery) -> Result
Ok((stdout, stderr)) Ok((stdout, stderr))
} }
/// Synchronise the nginx unit inside the `hive-gateway` container. /// Synchronise the host's nginx unit after an `agents.conf` write.
/// ///
/// Queries `ActiveState` via `systemctl --machine=hive-gateway` (requires /// Queries `ActiveState` and dispatches:
/// root — machine-bus transport enters the container namespace), then
/// dispatches:
/// - `active` → `systemctl reload nginx` (SIGHUP, zero-downtime) /// - `active` → `systemctl reload nginx` (SIGHUP, zero-downtime)
/// - `failed` → `systemctl reset-failed nginx` + `systemctl start nginx` /// - `failed` → `systemctl reset-failed nginx` + `systemctl start nginx`
/// - otherwise → `systemctl start nginx` /// - otherwise → `systemctl start nginx`
/// ///
/// ⚠️ `nginx` is hard-coded on purpose — see `PrivRequest::ReloadGatewayNginx`.
/// The unit name is the scope of this verb, and it used to be enforced by
/// `--machine=hive-gateway` (which could only reach into that container).
/// With nginx on the host there is no namespace to bound it, so the
/// literal is the only thing standing between "reload the gateway" and
/// "reload anything".
///
/// Returns `(String::new(), String::new())` on success so it fits the /// Returns `(String::new(), String::new())` on success so it fits the
/// `exec` return type directly. /// `exec` return type directly.
async fn sync_gateway_nginx() -> Result<(String, String)> { async fn sync_gateway_nginx() -> Result<(String, String)> {
let state_out = Command::new("systemctl") let state_out = Command::new("systemctl")
.args([ .args(["show", "--property=ActiveState", "--value", "nginx"])
"--machine=hive-gateway",
"show",
"--property=ActiveState",
"--value",
"nginx",
])
.output() .output()
.await .await
.context("query nginx ActiveState in hive-gateway")?; .context("query gateway nginx ActiveState")?;
if !state_out.status.success() { if !state_out.status.success() {
tracing::warn!( tracing::warn!(
exit_code = ?state_out.status.code(), exit_code = ?state_out.status.code(),
stderr = %String::from_utf8_lossy(&state_out.stderr).trim(), stderr = %String::from_utf8_lossy(&state_out.stderr).trim(),
"systemctl show ActiveState exited non-zero — gateway container may be down" "systemctl show ActiveState exited non-zero — gateway nginx may be down"
); );
} }
let state = String::from_utf8_lossy(&state_out.stdout).trim().to_owned(); let state = String::from_utf8_lossy(&state_out.stdout).trim().to_owned();
@ -2209,10 +2213,10 @@ async fn sync_gateway_nginx() -> Result<(String, String)> {
match state.as_str() { match state.as_str() {
"active" => { "active" => {
let out = Command::new("systemctl") let out = Command::new("systemctl")
.args(["--machine=hive-gateway", "reload", "nginx"]) .args(["reload", "nginx"])
.output() .output()
.await .await
.context("reload nginx in hive-gateway")?; .context("reload gateway nginx")?;
if !out.status.success() { if !out.status.success() {
bail!( bail!(
"gateway nginx reload failed ({}): {}", "gateway nginx reload failed ({}): {}",
@ -2224,14 +2228,14 @@ async fn sync_gateway_nginx() -> Result<(String, String)> {
"failed" => { "failed" => {
// Clear start-limit so the next start can proceed. // Clear start-limit so the next start can proceed.
let _ = Command::new("systemctl") let _ = Command::new("systemctl")
.args(["--machine=hive-gateway", "reset-failed", "nginx"]) .args(["reset-failed", "nginx"])
.status() .status()
.await; .await;
let out = Command::new("systemctl") let out = Command::new("systemctl")
.args(["--machine=hive-gateway", "start", "nginx"]) .args(["start", "nginx"])
.output() .output()
.await .await
.context("start nginx after reset-failed in hive-gateway")?; .context("start gateway nginx after reset-failed")?;
if !out.status.success() { if !out.status.success() {
bail!( bail!(
"gateway nginx start (after reset-failed) failed ({}): {}", "gateway nginx start (after reset-failed) failed ({}): {}",
@ -2243,10 +2247,10 @@ async fn sync_gateway_nginx() -> Result<(String, String)> {
_ => { _ => {
// inactive, activating, deactivating, unknown — just start. // inactive, activating, deactivating, unknown — just start.
let out = Command::new("systemctl") let out = Command::new("systemctl")
.args(["--machine=hive-gateway", "start", "nginx"]) .args(["start", "nginx"])
.output() .output()
.await .await
.context("start nginx in hive-gateway")?; .context("start gateway nginx")?;
if !out.status.success() { if !out.status.success() {
bail!( bail!(
"gateway nginx start failed (state={state:?}) ({}): {}", "gateway nginx start failed (state={state:?}) ({}): {}",

View file

@ -224,11 +224,17 @@ pub enum Capability {
/// manager socket for swarm-wide scans. /// manager socket for swarm-wide scans.
QueryAgentState, QueryAgentState,
/// Agent can restart hive infrastructure containers (hive-ci, /// Agent can restart hive infrastructure containers (hive-ci,
/// hive-gateway, hive-forge) via the `restart` MCP tool. hive-c0re /// hive-forge, hive-matrix) via the `restart` MCP tool. hive-c0re
/// checks this capability before routing the restart through /// checks this capability before routing the restart through
/// hive-priv; the concrete service allowlist lives root-side in /// hive-priv; the concrete service allowlist lives root-side in
/// hive-priv. Deliberately generic ("infra admin") so future /// hive-priv. Deliberately generic ("infra admin") so future
/// privileged infra ops can hang off the same grant. /// privileged infra ops can hang off the same grant.
///
/// ⚠️ The gateway is **not** in reach of this capability, by operator
/// ruling — it is the host's nginx and fronts the forge, dashboard and
/// matrix, so an agent restarting it can cut the path its own fix
/// travels. That refusal is a property of the target, not of the
/// grant: no capability re-opens it.
InfraAdmin, InfraAdmin,
} }
@ -265,7 +271,7 @@ impl Capability {
"query non-child agents' loose ends and reminder state via get_loose_ends" "query non-child agents' loose ends and reminder state via get_loose_ends"
} }
Self::InfraAdmin => { Self::InfraAdmin => {
"restart hive infrastructure containers (hive-ci, hive-gateway, hive-forge) via the restart tool" "restart hive infrastructure containers (hive-ci, hive-forge, hive-matrix; not the gateway) via the restart tool"
} }
} }
} }

View file

@ -225,7 +225,7 @@ pub struct ScopeArgs {
/// The forge container (`hive-forge`). /// The forge container (`hive-forge`).
#[arg(long)] #[arg(long)]
forge: bool, forge: bool,
/// The gateway container (`hive-gateway`). /// The gateway (`hive-gateway`) — nginx on the host, not a container.
#[arg(long)] #[arg(long)]
gateway: bool, gateway: bool,
/// The matrix container (`hive-matrix`). /// The matrix container (`hive-matrix`).

View file

@ -1,7 +1,10 @@
# Single nginx in front of every hyperhive web surface — dashboard, # Single nginx in front of every hyperhive web surface — dashboard,
# per-agent UIs (sub-path), forge + matrix (sub-domain), .well-known # per-agent UIs (sub-path), forge + matrix (sub-domain), .well-known
# delegations — plus the hive-internal dnsmasq resolver, co-located in # delegations — plus the hive-internal dnsmasq resolver. Both run on the
# the same `hive-gateway` container (shared host netns, state-free). # HOST, next to hive-c0re. They used to live in a `hive-gateway`
# container that shared the host netns anyway, so the boundary bought no
# network isolation and cost a resolv.conf sync, a reload that had to
# cross the machine bus, and four bind mounts.
# Full vhost map + discovery flow + design rationale in # Full vhost map + discovery flow + design rationale in
# `docs/gateway.md`. Layout: ./options.nix (option declarations), # `docs/gateway.md`. Layout: ./options.nix (option declarations),
# ./vhosts.nix (the nginx virtual-host tree), ./error-pages.nix # ./vhosts.nix (the nginx virtual-host tree), ./error-pages.nix
@ -42,6 +45,43 @@ let
# so the gateway always terminates TLS. # so the gateway always terminates TLS.
# `cfg.useSelfSigned` (options.nix) is the derived single source of truth. # `cfg.useSelfSigned` (options.nix) is the derived single source of truth.
useSelfSigned = cfg.useSelfSigned; useSelfSigned = cfg.useSelfSigned;
# nginx's own state dir. Kept at the historical `/var/lib/hive-gateway`
# path rather than renamed with the move: it holds the imported leaf
# across reboots, and renaming it would strand every existing hive's
# certs for no gain.
tlsDir = "/var/lib/hive-gateway/tls";
# TLS cert + key paths.
# - self-signed (default): the hive-CA-signed leaf, imported into the
# state dir by `hive-gateway-self-signed-cert` below.
# - tls.certDir set: the operator's own cert dir, read directly.
tlsCert =
if cfg.tls.certDir != null then "${cfg.tls.certDir}/${cfg.tls.certName}" else "${tlsDir}/cert.pem";
tlsKey =
if cfg.tls.certDir != null then "${cfg.tls.certDir}/${cfg.tls.keyName}" else "${tlsDir}/key.pem";
# The swarm-services pair, used only by the vhosts whose names this
# hive's CA cannot sign. Self-signed mode only: with an operator cert
# or ACME the operator owns every name and there is no second issuer.
svcCert = "${tlsDir}/swarm-services.pem";
svcKey = "${tlsDir}/swarm-services-key.pem";
nginxTree = import ./vhosts.nix {
inherit
lib
cfg
forgeCfg
matrixCfg
hyperhiveDomain
dashboardDist
swaggerUiTheme
tlsCert
tlsKey
svcCert
svcKey
swarmServiceDomains
;
errorPages = import ./error-pages.nix { inherit pkgs; };
};
in in
{ {
imports = [ ./options.nix ]; imports = [ ./options.nix ];
@ -65,10 +105,12 @@ in
} }
]; ];
# Ensure bind-mount sources exist at host boot before the gateway # Ensure the gateway state dirs exist at host boot, before anything
# container's first start. nspawn would auto-create missing dirs; # reads or writes them. They used to double as bind-mount sources
# tmpfiles rules make the intent explicit and cover the fresh-boot # for the container (nspawn would auto-create a missing one); the
# window before c0re has run. # rules stay because they still cover the fresh-boot window before
# c0re has run, and they pin owner + mode rather than leaving it to
# whoever creates the path first.
# #
# /run/hive-agent — per-agent UDS socket dir, written by c0re's # /run/hive-agent — per-agent UDS socket dir, written by c0re's
# set_nspawn_flags when agents start. Owned by `hive-core` (the # set_nspawn_flags when agents start. Owned by `hive-core` (the
@ -95,349 +137,170 @@ in
"f /var/lib/hyperhive/gateway/gateway.htpasswd 0644 root root - -" "f /var/lib/hyperhive/gateway/gateway.htpasswd 0644 root root - -"
]; ];
# Keep the gateway's `/etc/resolv.conf` in step with the host's. # ⚠️ REMOVED WITH THE CONTAINER, and each one was a workaround for the
# boundary rather than a thing nginx or dnsmasq needed:
# #
# nixos-container does `cp --remove-destination /etc/resolv.conf # - `privateNetwork = false` — the container already shared the host
# "$root/etc/resolv.conf"` in its start script, and nspawn's # netns, which is why nginx bound host ports and `localhost`
# `--resolv-conf=auto` copies rather than binds for a writable, # upstreams reached hive-c0re. On the host that is simply true.
# host-netns container like this one. Both are one-shot: systemd-nspawn(1) # - `additionalCapabilities = [ "CAP_NET_ADMIN" ]` — dnsmasq refuses
# says outright that "no further propagation of configuration is # to start with a `dhcp-range` unless it holds NET_ADMIN, and
# generally done after the one-time early initialization (this is # nspawn's bounding set dropped it for a host-netns container.
# because the file is usually updated through copying and renaming)". # Host root has it.
# - `networking.firewall.enable = false` — a container sharing the
# host netns would run *its* firewall.service against the HOST
# ruleset, flushing nixos-fw and deleting the nixos-nat-* chains on
# every boot. With one machine there is one firewall (below).
# - `networking.resolvconf.enable = false` + the `hive-gateway-resolv`
# path/service pair — the container's /etc/resolv.conf was a
# one-shot copy frozen at start, so a host network change left
# dnsmasq forwarding to a resolver that was gone. The whole
# watch-copy-reload machine existed to bridge two files. There is
# now one.
# - three bind mounts — /run/hive-agent, /run/hive-state, and either
# /run/hive-tls (operator cert) or /run/hive-ca (self-signed);
# those last two are mutually exclusive mkIfs, so it was never
# four. All plain host paths now.
# #
# So the gateway's copy is frozen at container start. dnsmasq has no # See `docs/network.md::Resolver behaviour` for the resolver history.
# explicit upstream (see ./dnsmasq.nix) and follows that file, which # ACME (Let's Encrypt) integration. nginx vhosts set
# means a host network change — new router, new DHCP lease, laptop # `enableACME = true` via the vhost builder; this provides the
# moving between networks — leaves dnsmasq forwarding to a resolver # shared ACME config (acceptTerms + email).
# that is gone, and every non-hive lookup from every agent hangs. The security.acme = lib.mkIf cfg.tls.acme.enable {
# agents' own resolvers point at the static bridge IP and never go acceptTerms = true;
# stale, which is exactly why the symptom presents as "the gateway defaults.email = cfg.tls.acme.email;
# needs a kick".
#
# Hence: watch on the host, push into the container, reload dnsmasq.
# `reload` is `kill -HUP $MAINPID` (the upstream dnsmasq unit's
# ExecReload), so dnsmasq re-reads its upstream list and drops its
# cache without severing anything — nginx never notices. Why this has
# to run host-side, and why a file bind-mount is worse than the copy:
# `docs/network.md::Resolver behaviour`.
systemd.paths.hive-gateway-resolv = {
description = "Watch the host's /etc/resolv.conf for the hive-gateway container";
wantedBy = [ "multi-user.target" ];
# Arm the watch before anything configures the network, so the very
# first DHCP-driven resolv.conf write of the boot is caught too —
# that's the "gateway came up while DHCP was still settling" case.
# Inert if nothing pulls network-pre.target into the transaction.
before = [ "network-pre.target" ];
pathConfig = {
# PathChanged also watches the parent directory, so openresolv's
# atomic rename-over lands as IN_MOVED_TO on /etc and triggers —
# a watch on the inode alone would die with the replaced file.
PathChanged = "/etc/resolv.conf";
Unit = "hive-gateway-resolv.service";
};
}; };
systemd.services.hive-gateway-resolv = { # Import the hive-CA leaf into nginx's state dir before nginx starts.
description = "Sync the host's resolvers into hive-gateway and reload dnsmasq"; #
# Also run once per gateway start, to catch a host resolver change # 🚨 THIS LOOKS LIKE A LEFTOVER OF THE CONTAINER AND IS NOT — do not
# that happened while the container was down. Deliberately NOT # "simplify" it into pointing nginx at the CA dir. It does TWO jobs:
# ordered after network-online.target: pulling that target in on #
# every resolv.conf change risks blocking the sync behind a # (1) It re-modes the leaf. `hive-tls-ca` writes the key 0600
# wait-online timeout on hosts where nothing else reaches it. The # root:root; nginx's pre-start `nginx -t` runs as the nginx
# boot race is closed by arming the path unit early instead. # *user*, so a 0600 key fails the config test with
wantedBy = [ "container@hive-gateway.service" ]; # `BIO_new_file() … Permission denied` and blocks the unit —
after = [ "container@hive-gateway.service" ]; # hence the 0640 root:nginx copy below. That is a file-mode fact,
path = [ # not a namespace one, and it did not go away with the container.
pkgs.systemd # (2) It guarantees that **every cert path the nginx config names
pkgs.coreutils # exists** — which is what the swarm-services fallback at the
pkgs.gnugrep # bottom of the script is for. nginx refuses to load a config
]; # naming a missing cert file, so a leaf that never issues takes
# the whole gateway down rather than one vhost; that has already
# happened once and it took the forge, dashboard and matrix with
# it. Removing this unit re-creates it exactly.
#
# nginx
# `Requires=` this via `requiredBy`, so it refuses to start until
# the copy succeeds. ALWAYS runs (no ConditionPathExists) and is
# idempotent — necessary to reconcile broken state from prior
# failed boots (a 0700 dir from a stale UMask, a truncated copy
# from an interrupted oneshot, etc.). The leaf covers the bare
# hive domain plus `forge.`, `matrix.` and `*.${hyperhiveDomain}`
# so all sub-domains validate under the same cert + the hive CA.
# See `docs/gateway.md` ("Self-signed TLS").
systemd.services.hive-gateway-self-signed-cert = lib.mkIf useSelfSigned {
description = "Import host-generated TLS leaf for hive-gateway";
wantedBy = [ "multi-user.target" ];
before = [ "nginx.service" ];
requiredBy = [ "nginx.service" ];
serviceConfig = { serviceConfig = {
Type = "oneshot"; Type = "oneshot";
SyslogIdentifier = "hive-gateway-resolv"; RemainAfterExit = true;
# Pin the journal identity (else it's the `script` store-path wrapper).
SyslogIdentifier = "hive-gateway-self-signed-cert";
}; };
path = [ pkgs.coreutils ];
script = '' script = ''
set -euo pipefail set -eu
src=/etc/resolv.conf mkdir -p ${tlsDir}
# Marker lives on tmpfs, so the first sync after every host boot # 0755 on BOTH the cert dir and its parent so the nginx
# always goes through rather than trusting a stale comparison. # user can traverse the full path. The parent
marker=/run/hive-gateway/resolv.synced # `/var/lib/hive-gateway` lands at 0700 by default (systemd
# StateDirectory / mkdir umask depending on which service
# created it first), which on its own blocks traversal.
# Re-applied every boot in case a prior run left a tighter
# mode behind.
chmod 0755 ${builtins.dirOf tlsDir}
chmod 0755 ${tlsDir}
# Copy the host leaf in. `install` writes atomically with the
# target mode; run as root (container root == host root,
# privateUsers=false) so the 0600 root:root host key is
# readable. Key ends up root:nginx 0640 so nginx-pre-start
# (which runs `nginx -t` as the nginx user, not root) can
# read it — a 0600 root:root key passes the master load but
# fails the pre-start config test with `BIO_new_file() …
# Permission denied`, blocking the unit. Cert is world-read.
install -m 0644 ${config.services.hyperhive.tls.stateDir}/gateway.pem ${tlsCert}
install -m 0640 -g nginx ${config.services.hyperhive.tls.stateDir}/gateway-key.pem ${tlsKey}
# No nameserver line means either a mid-rewrite snapshot or a host # The swarm-services leaf, when this host issues one. It is
# with no DNS at all. In both cases the gateway's existing copy is # a separate pair rather than more SANs on the one above
# the best information available — keep it and wait for the next # because no hive CA can sign these names — each is
# event, instead of pushing a file that resolves nothing. # constrained to its own hive's domain and the service
if ! grep -q '^[[:space:]]*nameserver[[:space:]]' "$src"; then # names are siblings of it.
echo "host resolv.conf has no nameserver line keeping the gateway's current copy" #
exit 0 # Absent is a normal state, not a failure: the leaf exists
# only where the swarm CA is autoconfigured, and issuance
# can also fail on a host that wants one.
#
# ⚠️ When it is absent the HIVE leaf goes to this path
# anyway, and that fallback is load-bearing rather than
# tidy. nginx refuses to load a config naming a cert file
# that does not exist — `cannot load certificate … no such
# file` fails the pre-start test, so the vhost does not
# degrade, the ENTIRE gateway dies and takes the forge, the
# dashboard and matrix with it. Serving the hive leaf on a
# swarm-service name is a name mismatch: browsers warn,
# strict clients refuse, everything else keeps working, and
# the operator gets a bad cert instead of no hive.
#
# Measured, not theorised: this exact path took pr1ma's
# gateway down when the services sub-CA failed to issue.
if [ -s ${config.services.hyperhive.tls.stateDir}/swarm-services.pem ]; then
install -m 0644 ${config.services.hyperhive.tls.stateDir}/swarm-services.pem ${svcCert}
install -m 0640 -g nginx ${config.services.hyperhive.tls.stateDir}/swarm-services-key.pem ${svcKey}
else
echo "no swarm-services leaf serving the hive leaf on those names (mismatch, not an outage)" >&2
install -m 0644 ${config.services.hyperhive.tls.stateDir}/gateway.pem ${svcCert}
install -m 0640 -g nginx ${config.services.hyperhive.tls.stateDir}/gateway-key.pem ${svcKey}
fi fi
if [ -e "$marker" ] && cmp -s "$src" "$marker"; then
echo "host resolvers unchanged since last sync nothing to do"
exit 0
fi
# A stopped gateway needs no push: its next start copies the
# current host file itself.
if ! systemctl is-active --quiet container@hive-gateway.service; then
echo "hive-gateway not running its next start copies the current file itself"
exit 0
fi
# `machinectl copy-to` writes through the container's own mount
# namespace, so this stays correct regardless of how the container
# assembles /etc (e.g. if system.etc.overlay is ever turned on) —
# unlike poking at the rootfs path from the host side.
machinectl copy-to hive-gateway "$src" /etc/resolv.conf --force
# Not fatal: dnsmasq polls resolv.conf for mtime changes on its own,
# so a failed reload costs a second of staleness, not correctness.
if ! systemctl -M hive-gateway reload dnsmasq.service; then
echo "dnsmasq reload failed (not up yet?) copy is in place, its own poll will pick it up"
fi
mkdir -p "$(dirname "$marker")"
install -m 0644 "$src" "$marker"
echo "synced host resolvers into hive-gateway and reloaded dnsmasq"
''; '';
}; };
containers.hive-gateway = { # nginx reload is triggered from the HOST side by hive-c0re
autoStart = true; # after each agents.conf write, through hive-priv (c0re is
ephemeral = false; # unprivileged and cannot act on a system unit).
# Share host netns — nginx then binds host-level ports directly, #
# `localhost` upstream resolution reaches hive-c0re without any # It stays an explicit trigger rather than a systemd path unit
# port-forward dance, and the firewall config below is the only # watching the file. That used to be impossible — an IN_MOVED_TO
# layer that matters. # from the atomic rename did not cross the nspawn mount-namespace
privateNetwork = false; # boundary — and with one machine it would now work. It is still
# dnsmasq refuses to start once a dhcp-range is configured unless it # not wanted: the write and the reload belong in one causal chain
# holds CAP_NET_ADMIN (DNS-only mode doesn't need it). Private-network # c0re can retry and report on (see RELOAD_PENDING), not two
# containers retain NET_ADMIN implicitly, but this container shares the # independent units racing on an inotify event.
# host netns (above), so nspawn's default bounding set drops it — grant
# it explicitly. Note this is NET_ADMIN over the *host* netns; the
# gateway container is trusted infra (it already terminates TLS and
# fronts every vhost), so no new trust boundary is crossed.
additionalCapabilities = [ "CAP_NET_ADMIN" ];
# Bind-mount the per-agent socket dir so nginx inside the gateway
# container can `connect(2)` to the UDS upstreams.
# Read-only (we just connect; harness writes the socket inside
# the agent's own container). Host-side dir is pre-created by a
# tmpfiles rule so nspawn always finds a source at boot.
bindMounts."/run/hive-agent" = {
hostPath = "/run/hive-agent";
isReadOnly = true;
};
# Bind-mount ONLY the gateway-specific subdir of the hyperhive
# state dir. Scoped to /var/lib/hyperhive/gateway/ rather than
# the whole parent so the gateway container can't read forge
# tokens or other files that may live at the parent level.
# c0re writes agents.conf under this subdir and triggers an nginx
# reload from the host via systemd-run after each write.
# Pre-created by a tmpfiles rule.
bindMounts."/run/hive-state" = {
hostPath = "/var/lib/hyperhive/gateway";
isReadOnly = true;
};
# Operator-provided TLS cert dir (e.g. Let's Encrypt / ACME).
# Only mounted when `tls.certDir` is set; when it is, the self-signed
# floor is off (so the `/run/hive-ca` mount below is absent). nginx
# reads cert + key from `/run/hive-tls/<certName>` and `<keyName>`.
bindMounts."/run/hive-tls" = lib.mkIf (cfg.tls.certDir != null) {
hostPath = cfg.tls.certDir;
isReadOnly = true;
};
# Self-signed mode: the host `hive-tls-ca` service generates a hive
# CA + a leaf signed by it under `services.hyperhive.tls.stateDir`.
# Bind-mount that dir read-only so the in-container import service
# (below) can copy the leaf into nginx's state dir with the right
# owner/mode. Source files: `gateway.pem` + `gateway-key.pem`.
bindMounts."/run/hive-ca" = lib.mkIf useSelfSigned {
hostPath = config.services.hyperhive.tls.stateDir;
isReadOnly = true;
};
config =
{ pkgs, ... }:
let
tlsDir = "/var/lib/hive-gateway/tls";
# TLS cert + key paths inside the container.
# - self-signed (default): imported hive-CA-signed leaf in the
# persistent state dir.
# - tls.certDir set: operator-provided cert bind-mounted at /run/hive-tls.
tlsCert =
if cfg.tls.certDir != null then "/run/hive-tls/${cfg.tls.certName}" else "${tlsDir}/cert.pem";
tlsKey =
if cfg.tls.certDir != null then "/run/hive-tls/${cfg.tls.keyName}" else "${tlsDir}/key.pem";
# The swarm-services pair, used only by the vhosts whose names
# this hive's CA cannot sign. Self-signed mode only: with an
# operator cert or ACME the operator owns every name and there
# is no second issuer in the picture.
svcCert = "${tlsDir}/swarm-services.pem";
svcKey = "${tlsDir}/swarm-services-key.pem";
nginxTree = import ./vhosts.nix {
inherit
lib
cfg
forgeCfg
matrixCfg
hyperhiveDomain
dashboardDist
swaggerUiTheme
tlsCert
tlsKey
svcCert
svcKey
swarmServiceDomains
;
errorPages = import ./error-pages.nix { inherit pkgs; };
};
in
{
system.stateVersion = "26.05";
# This container shares the host netns, so its own services.nginx = {
# firewall.service would run against the HOST ruleset: flush enable = true;
# the nixos-fw chains, rebuild them from this container's recommendedProxySettings = true;
# (empty) port list, and delete the host's nixos-nat-* chains recommendedTlsSettings = true;
# — wiping the bridge DHCP/DNS holes and the agents' NAT on recommendedGzipSettings = true;
# every container boot. The host firewall owns all filtering; recommendedOptimisation = true;
# never run one in here. inherit (nginxTree) appendHttpConfig virtualHosts;
networking.firewall.enable = false; };
# Keep the host-copied /etc/resolv.conf intact. nixos-container # dnsmasq moves with nginx rather than staying behind: it was only in
# copies the host's file in at every container start, but # the container because nginx was, and leaving it there would keep the
# resolvconf's host-tracking mode then regenerates it — to an # whole resolv.conf sync machine alive for a resolver that no longer
# empty file, since the host file doesn't cross the boundary # needs it. Host-side it reads the one /etc/resolv.conf directly.
# after start (the same failure the matrix container hit). services.dnsmasq = import ./dnsmasq.nix {
# With resolvconf off, nothing in here touches the copy: nginx's inherit
# own lookups (ACME) and dnsmasq's follow-the-host upstream lib
# default (see ./dnsmasq.nix) both read the host's resolvers. networkCfg
# Keeping it stale-free is the host's job — see the forgeCfg
# `hive-gateway-resolv` path unit above. matrixCfg
networking.resolvconf.enable = false; hyperhiveDomain
;
# ACME (Let's Encrypt) integration. nginx vhosts set
# `enableACME = true` via the vhost builder; this provides the
# shared ACME config (acceptTerms + email). The gateway
# container has shared host netns so outbound ACME requests
# work without extra routing config. Certs are stored in the
# container's persistent state (`ephemeral = false`).
security.acme = lib.mkIf cfg.tls.acme.enable {
acceptTerms = true;
defaults.email = cfg.tls.acme.email;
};
# Import the host-generated leaf cert before nginx starts.
# The hive CA + gateway leaf are generated on the HOST by
# `hive-tls-ca` (see `hive-tls.nix`) and bind-mounted read-only
# at `/run/hive-ca`; this service copies the leaf into nginx's
# state dir with the owner/mode nginx needs, rather than reading
# the bind-mount directly (the host key is 0600 root:root and a
# cross-namespace bind-mount can't be relaxed in place). nginx
# `Requires=` this via `requiredBy`, so it refuses to start until
# the copy succeeds. ALWAYS runs (no ConditionPathExists) and is
# idempotent — necessary to reconcile broken state from prior
# failed boots (a 0700 dir from a stale UMask, a truncated copy
# from an interrupted oneshot, etc.). The leaf covers the bare
# hive domain plus `forge.`, `matrix.` and `*.${hyperhiveDomain}`
# so all sub-domains validate under the same cert + the hive CA.
# See `docs/gateway.md` ("Self-signed TLS").
systemd.services.hive-gateway-self-signed-cert = lib.mkIf useSelfSigned {
description = "Import host-generated TLS leaf for hive-gateway";
wantedBy = [ "multi-user.target" ];
before = [ "nginx.service" ];
requiredBy = [ "nginx.service" ];
serviceConfig = {
Type = "oneshot";
RemainAfterExit = true;
# Pin the journal identity (else it's the `script` store-path wrapper).
SyslogIdentifier = "hive-gateway-self-signed-cert";
};
path = [ pkgs.coreutils ];
script = ''
set -eu
mkdir -p ${tlsDir}
# 0755 on BOTH the cert dir and its parent so the nginx
# user can traverse the full path. The parent
# `/var/lib/hive-gateway` lands at 0700 by default (systemd
# StateDirectory / mkdir umask depending on which service
# created it first), which on its own blocks traversal.
# Re-applied every boot in case a prior run left a tighter
# mode behind.
chmod 0755 ${builtins.dirOf tlsDir}
chmod 0755 ${tlsDir}
# Copy the host leaf in. `install` writes atomically with the
# target mode; run as root (container root == host root,
# privateUsers=false) so the 0600 root:root host key is
# readable. Key ends up root:nginx 0640 so nginx-pre-start
# (which runs `nginx -t` as the nginx user, not root) can
# read it — a 0600 root:root key passes the master load but
# fails the pre-start config test with `BIO_new_file() …
# Permission denied`, blocking the unit. Cert is world-read.
install -m 0644 /run/hive-ca/gateway.pem ${tlsCert}
install -m 0640 -g nginx /run/hive-ca/gateway-key.pem ${tlsKey}
# The swarm-services leaf, when this host issues one. It is
# a separate pair rather than more SANs on the one above
# because no hive CA can sign these names — each is
# constrained to its own hive's domain and the service
# names are siblings of it.
#
# Absent is a normal state, not a failure: the leaf exists
# only where the swarm CA is autoconfigured, and issuance
# can also fail on a host that wants one.
#
# ⚠️ When it is absent the HIVE leaf goes to this path
# anyway, and that fallback is load-bearing rather than
# tidy. nginx refuses to load a config naming a cert file
# that does not exist — `cannot load certificate … no such
# file` fails the pre-start test, so the vhost does not
# degrade, the ENTIRE gateway dies and takes the forge, the
# dashboard and matrix with it. Serving the hive leaf on a
# swarm-service name is a name mismatch: browsers warn,
# strict clients refuse, everything else keeps working, and
# the operator gets a bad cert instead of no hive.
#
# Measured, not theorised: this exact path took pr1ma's
# gateway down when the services sub-CA failed to issue.
if [ -s /run/hive-ca/swarm-services.pem ]; then
install -m 0644 /run/hive-ca/swarm-services.pem ${svcCert}
install -m 0640 -g nginx /run/hive-ca/swarm-services-key.pem ${svcKey}
else
echo "no swarm-services leaf serving the hive leaf on those names (mismatch, not an outage)" >&2
install -m 0644 /run/hive-ca/gateway.pem ${svcCert}
install -m 0640 -g nginx /run/hive-ca/gateway-key.pem ${svcKey}
fi
'';
};
# nginx reload is triggered from the HOST side by hive-c0re
# via `systemctl -M hive-gateway reload nginx` after each
# agents.conf write — letting systemd resolve the nginx binary
# path avoids exit-203 EXEC failures. A path unit watching the
# bind-mounted file inside the container does not work: an
# IN_MOVED_TO from an atomic rename on the host does not
# propagate across the nspawn mount-namespace boundary. The
# host-side trigger is the correct approach.
services.nginx = {
enable = true;
recommendedProxySettings = true;
recommendedTlsSettings = true;
recommendedGzipSettings = true;
recommendedOptimisation = true;
inherit (nginxTree) appendHttpConfig virtualHosts;
};
services.dnsmasq = import ./dnsmasq.nix {
inherit
lib
networkCfg
forgeCfg
matrixCfg
hyperhiveDomain
;
};
};
}; };
networking.firewall = lib.mkIf cfg.openFirewall { networking.firewall = lib.mkIf cfg.openFirewall {

View file

@ -134,9 +134,9 @@ in
and uses this cert, overriding the self-signed default the and uses this cert, overriding the self-signed default the
auto-generated hive-CA-signed leaf is skipped entirely. auto-generated hive-CA-signed leaf is skipped entirely.
The directory is bind-mounted read-only into the gateway nginx reads `<certDir>/<tls.certName>` and
container at `/run/hive-tls/`. nginx reads `<certDir>/<tls.keyName>` directly it runs on the host, so
`<certDir>/<tls.certName>` and `<certDir>/<tls.keyName>`. the directory needs no bind mount and no copy.
Default filenames (`cert.pem` / `key.pem`) match the output Default filenames (`cert.pem` / `key.pem`) match the output
layout of nixpkgs's `security.acme` module. layout of nixpkgs's `security.acme` module.
@ -232,9 +232,7 @@ in
enabled, every request to the gateway's main vhost requires a enabled, every request to the gateway's main vhost requires a
valid username and password. nginx's built-in `auth_basic` valid username and password. nginx's built-in `auth_basic`
module validates credentials against module validates credentials against
`/var/lib/hyperhive/gateway/gateway.htpasswd` on the host `/var/lib/hyperhive/gateway/gateway.htpasswd`. Off by default.
(exposed as `/run/hive-state/gateway.htpasswd` inside the
container via the existing gateway state bind-mount). Off by default.
Manage users with `hivectl gateway create-user`, `delete-user`, Manage users with `hivectl gateway create-user`, `delete-user`,
and `list-users` see `hivectl gateway --help` for usage. and `list-users` see `hivectl gateway --help` for usage.

View file

@ -241,7 +241,7 @@ let
# `/agent/` catch-all 404 + the two internal error-page targets it # `/agent/` catch-all 404 + the two internal error-page targets it
# points at. Per-agent `location /agent/<name>/` blocks live in the # points at. Per-agent `location /agent/<name>/` blocks live in the
# runtime-generated `/run/hive-state/agents.conf` (included via # runtime-generated `/var/lib/hyperhive/gateway/agents.conf` (included via
# `extraConfig` on the vhost); nginx longest-prefix-match makes a # `extraConfig` on the vhost); nginx longest-prefix-match makes a
# real `/agent/<name>/` beat this catch-all. `internal` keeps the # real `/agent/<name>/` beat this catch-all. `internal` keeps the
# error pages reachable only through nginx's error handling. # error pages reachable only through nginx's error handling.
@ -275,7 +275,7 @@ let
# secret (`X-Hub-Signature-256`) protects those endpoints instead. # secret (`X-Hub-Signature-256`) protects those endpoints instead.
dashboardAuth = lib.optionalString cfg.auth.enable '' dashboardAuth = lib.optionalString cfg.auth.enable ''
auth_basic "${cfg.auth.realm}"; auth_basic "${cfg.auth.realm}";
auth_basic_user_file /run/hive-state/gateway.htpasswd; auth_basic_user_file /var/lib/hyperhive/gateway/gateway.htpasswd;
# `=401` keeps the status 401 so the login dialog shows; the # `=401` keeps the status 401 so the login dialog shows; the
# internal page explains `hivectl gateway create-user`. # internal page explains `hivectl gateway create-user`.
error_page 401 =401 /__hive_auth_unauthorized; error_page 401 =401 /__hive_auth_unauthorized;
@ -397,15 +397,14 @@ in
}; };
# Per-agent location blocks, generated at runtime by # Per-agent location blocks, generated at runtime by
# hive-c0re and written to /var/lib/hyperhive/gateway/agents.conf # hive-c0re and written to /var/lib/hyperhive/gateway/agents.conf
# on the host. The bind-mount at /run/hive-state/ exposes # on the host — the same machine nginx runs on. nginx parses
# that file here. nginx parses `include` at config-load # `include` at config-load time so a reload (triggered by c0re
# time so a reload (triggered by c0re via systemd-run
# after each agents.conf write) picks up new or removed # after each agents.conf write) picks up new or removed
# agents without a nixos-rebuild. nginx's longest-prefix- # agents without a nixos-rebuild. nginx's longest-prefix-
# match rule ensures `/agent/<name>/` from this file beats # match rule ensures `/agent/<name>/` from this file beats
# the `/agent/` catch-all above. # the `/agent/` catch-all above.
extraConfig = securityHeaders + '' extraConfig = securityHeaders + ''
include /run/hive-state/agents.conf; include /var/lib/hyperhive/gateway/agents.conf;
''; '';
}; };
} }

View file

@ -35,17 +35,18 @@ The socket is `0666`. It has to be: nginx runs as a different user and
sockets, and rests on the same argument — *"the bind source dir is per-agent on sockets, and rests on the same argument — *"the bind source dir is per-agent on
host so blast radius is unchanged."* host so blast radius is unchanged."*
What keeps that safe is that the directory holds **one** socket and is What keeps that safe is that the directory holds **one** socket. So:
bind-mounted into **one** container. So:
> **Never point `socketPath` at a directory that carries anything else.** > **Never point `socketPath` at a directory that carries anything else.**
> `/run/hyperhive` above all — it holds `host.sock`, the host **admin** socket. > `/run/hyperhive` above all — it holds `host.sock`, the host **admin** socket.
> Mounting that directory to reach this socket would hand the gateway container > Pointing nginx at that directory to reach this socket would put the admin
> the admin socket along with it. > socket within its reach too.
Changing `socketPath` therefore means re-checking the gateway bind-mount, not This got *less* forgiving when nginx moved onto the host: the gateway used to
just the daemon. A unit test pins the default path so a tidying edit fails reach a unix upstream through a bind-mount, so the mount list was a second
instead of reviewing cleanly. bound on what it could touch. There is no mount now — the directory is the
whole of the access control. A unit test pins the default path so a tidying
edit fails instead of reviewing cleanly.
`RuntimeDirectoryPreserve=yes` and the daemon's stale-socket unlink on start are `RuntimeDirectoryPreserve=yes` and the daemon's stale-socket unlink on start are
a **pair**: preserving the directory without the unlink means `bind` fails with a **pair**: preserving the directory without the unlink means `bind` fails with

View file

@ -27,10 +27,12 @@ use axum::{Router, routing::get};
/// `ExecStart`, so the default names a directory the unit just produced. /// `ExecStart`, so the default names a directory the unit just produced.
/// ///
/// The directory is its own — deliberately not shared with hive-c0re's /// The directory is its own — deliberately not shared with hive-c0re's
/// `/run/hyperhive`. nginx reaches a unix upstream by having the socket's /// `/run/hyperhive`. The socket is `0666`, so its directory is the only
/// *directory* bind-mounted into the gateway container, so co-locating /// access control it has; co-locating it with c0re's admin socket would
/// this socket with c0re's admin socket would hand the gateway the admin /// put both within reach of whatever can reach either. That used to be
/// socket along with it. /// enforced by which *directory* was bind-mounted into the gateway
/// container; with nginx on the host the mount is gone and the directory
/// is all that is left, so the rule matters more, not less.
const DEFAULT_SOCKET: &str = "/run/swarm-controller/controller.sock"; const DEFAULT_SOCKET: &str = "/run/swarm-controller/controller.sock";
fn socket_path() -> PathBuf { fn socket_path() -> PathBuf {
@ -96,8 +98,9 @@ mod tests {
/// The socket must not share a directory with anything else, because /// The socket must not share a directory with anything else, because
/// the socket is `0666` and the directory is therefore the only access /// the socket is `0666` and the directory is therefore the only access
/// control it has. `/run/hyperhive` in particular holds hive-c0re's /// control it has. `/run/hyperhive` in particular holds hive-c0re's
/// **admin** socket, and nginx reaches a unix upstream by mounting the /// **admin** socket. nginx used to reach a unix upstream by mounting
/// socket's whole directory into the gateway container. /// the socket's whole directory into the gateway container; it runs on
/// the host now, so nothing narrows its reach but the directory itself.
/// ///
/// A test rather than a comment: the failure this guards against is a /// A test rather than a comment: the failure this guards against is a
/// one-word edit that looks tidier and reads fine in review. /// one-word edit that looks tidier and reads fine in review.
@ -111,7 +114,7 @@ mod tests {
Path::new("/run/swarm-controller"), Path::new("/run/swarm-controller"),
"the socket's directory is its access control — moving it under a shared \ "the socket's directory is its access control — moving it under a shared \
directory (notably /run/hyperhive, which holds the host admin socket) \ directory (notably /run/hyperhive, which holds the host admin socket) \
exposes everything else in that directory to the gateway container" exposes everything else in that directory to the gateway's nginx"
); );
} }
} }