Compare commits
34 changed files with 617 additions and 634 deletions
|
|
@ -121,18 +121,8 @@ The mode is load-bearing, not cosmetic. Write permission on a
|
||||||
them, and the sticky bit is the only thing that would restrain that (it
|
them, and the sticky bit is the only thing that would restrain that (it
|
||||||
is not set here). A world-writable socket dir therefore lets anything
|
is not set here). A world-writable socket dir therefore lets anything
|
||||||
able to reach the path delete an agent's socket and bind its own — and
|
able to reach the path delete an agent's socket and bind its own — and
|
||||||
nginx reaches all of `/run/hive-agent` as a plain host path. Dropping
|
the gateway container has all of `/run/hive-agent` bind-mounted in.
|
||||||
`o=w` removes that permission rather than qualifying it.
|
Dropping `o=w` removes that permission rather than qualifying it.
|
||||||
|
|
||||||
⚠️ **The gateway's nginx and dnsmasq are host services, next to
|
|
||||||
`hive-c0re`** (see `docs/gateway.md`) — there is no namespace between
|
|
||||||
them and the rest of the host. That costs no network isolation: nginx
|
|
||||||
binds the host's `:80`/`:443` and reaches `localhost` upstreams, which a
|
|
||||||
netns would have to be opened up for anyway.
|
|
||||||
🔑 It does mean nothing *implicitly* scopes the privileged reload verb,
|
|
||||||
so the scope is explicit: the unit name is hard-coded in `hive-priv` —
|
|
||||||
see `PrivRequest::ReloadGatewayNginx`. **A caller cannot name the unit,
|
|
||||||
so the verb cannot be steered at another service.**
|
|
||||||
|
|
||||||
⚠️ Contrast `/shared`, which *is* sticky world-writable (`1777`): it has
|
⚠️ Contrast `/shared`, which *is* sticky world-writable (`1777`): it has
|
||||||
many legitimate writers, so sticky is the best available answer there.
|
many legitimate writers, so sticky is the best available answer there.
|
||||||
|
|
|
||||||
|
|
@ -381,7 +381,7 @@ that allows the underlying resource access.
|
||||||
| `manage_root_agent` | may lifecycle-manage the root/manager agent via `kill`/`start`/`restart` |
|
| `manage_root_agent` | may lifecycle-manage the root/manager agent via `kill`/`start`/`restart` |
|
||||||
| `read_host_journal` | `get_host_journal` MCP tool is registered + `GET /journal-host` requests are served |
|
| `read_host_journal` | `get_host_journal` MCP tool is registered + `GET /journal-host` requests are served |
|
||||||
| `query_agent_state` | may call `get_loose_ends` / `CountPendingReminders` targeting non-child agents |
|
| `query_agent_state` | may call `get_loose_ends` / `CountPendingReminders` targeting non-child agents |
|
||||||
| `infra_admin` | may call `restart(name)` on hive infrastructure containers (`hive-ci`, `hive-forge`, `hive-matrix` — **not** `hive-gateway`, which is the host's nginx and is operator-only); each restart is logged to the dashboard AUDIT trail |
|
| `infra_admin` | may call `restart(name)` on hive infrastructure containers (`hive-ci`, `hive-gateway`, `hive-forge`); each restart is logged to the dashboard AUDIT trail |
|
||||||
|
|
||||||
**Config storage** — per-agent capabilities live in
|
**Config storage** — per-agent capabilities live in
|
||||||
`/var/lib/hyperhive/meta/capabilities.json` alongside `tool-groups.json`.
|
`/var/lib/hyperhive/meta/capabilities.json` alongside `tool-groups.json`.
|
||||||
|
|
|
||||||
|
|
@ -500,9 +500,9 @@ Two things to know about the weights:
|
||||||
override in `meta/resource-limits.json`, so every agent carries the
|
override in `meta/resource-limits.json`, so every agent carries the
|
||||||
same value and the weight does *not* rank agents against each other.
|
same value and the weight does *not* rank agents against each other.
|
||||||
What `80` buys is that agents yield to everything **not** on this
|
What `80` buys is that agents yield to everything **not** on this
|
||||||
drop-in path: host services (nginx and dnsmasq among them) and the
|
drop-in path: host services and the infra containers (`hive-ci`,
|
||||||
infra containers (`hive-ci`, `hive-forge`, `hive-matrix`), which stay
|
`hive-forge`, `hive-gateway`, `hive-matrix`), which stay at the
|
||||||
at the kernel default of `100`.
|
kernel default of `100`.
|
||||||
- `IOWeight=` is only honoured when the backing device runs the BFQ
|
- `IOWeight=` is only honoured when the backing device runs the BFQ
|
||||||
scheduler or has blk-iocost QoS enabled. On a host using
|
scheduler or has blk-iocost QoS enabled. On a host using
|
||||||
`none`/`mq-deadline`/`kyber` without iocost, systemd writes the value
|
`none`/`mq-deadline`/`kyber` without iocost, systemd writes the value
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
# hive-gateway
|
# hive-gateway
|
||||||
|
|
||||||
Single nginx in front of every hyperhive web surface. Runs on the **host**, next to hive-c0re; system-config (not meta-flake managed). Configured via `services.hyperhive.gateway.*` + per-subsystem opt-in flags in `services.hyperhive.{forge,matrix,...}`. (It lived in a `hive-gateway` container until #3088 — one that shared the host netns anyway, so the boundary gave no network isolation while costing a resolv.conf sync, a machine-bus reload and three bind mounts.)
|
Single nginx in front of every hyperhive web surface. Container `hive-gateway`, shared host netns, system-config (not meta-flake managed). Configured via `services.hyperhive.gateway.*` + per-subsystem opt-in flags in `services.hyperhive.{forge,matrix,...}`.
|
||||||
|
|
||||||
## Vhost map
|
## Vhost map
|
||||||
|
|
||||||
|
|
@ -112,19 +112,18 @@ now set unconditionally for every agent. The mechanism:
|
||||||
a UDS upstream (`http://unix:/run/hive-agent/<name>/web.sock:/`);
|
a UDS upstream (`http://unix:/run/hive-agent/<name>/web.sock:/`);
|
||||||
if the socket is not yet bound, nginx returns 502 caught by the
|
if the socket is not yet bound, nginx returns 502 caught by the
|
||||||
`error_page 502 503 504 = /__hive_agent_unreachable` directive.
|
`error_page 502 503 504 = /__hive_agent_unreachable` directive.
|
||||||
nginx includes `/var/lib/hyperhive/gateway/agents.conf` — the same
|
The gateway container bind-mounts `/var/lib/hyperhive/gateway/` at
|
||||||
path c0re writes, since both run on the host.
|
`/run/hive-state/`; nginx includes `/run/hive-state/agents.conf`.
|
||||||
After each write, c0re triggers the appropriate nginx action via
|
After each write, c0re triggers the appropriate nginx action inside
|
||||||
`hive-priv` (which is root; hive-c0re runs as the unprivileged
|
the gateway container via `hive-priv` (which runs as root and has
|
||||||
`hive-core` user and cannot act on a system unit).
|
`--machine=hive-gateway` transport rights that hive-c0re lacks).
|
||||||
`hive-priv` queries `ActiveState` and dispatches:
|
`hive-priv` queries `ActiveState` and dispatches:
|
||||||
- active → `systemctl reload nginx` (SIGHUP, zero-downtime)
|
- active → `systemctl reload nginx` (SIGHUP, zero-downtime)
|
||||||
- failed → `systemctl reset-failed nginx` + `systemctl start nginx`
|
- failed → `systemctl reset-failed nginx` + `systemctl start nginx`
|
||||||
- otherwise → `systemctl start nginx`
|
- otherwise → `systemctl start nginx`
|
||||||
This is an explicit trigger rather than a path unit watching the
|
This is intentionally host-side: `IN_MOVED_TO` from an atomic rename
|
||||||
file: the write and the reload belong in one causal chain c0re can
|
does not propagate across the nspawn mount-namespace boundary, so a
|
||||||
retry and report on (`RELOAD_PENDING`), not two units racing on an
|
path unit inside the container would never fire.
|
||||||
inotify event.
|
|
||||||
|
|
||||||
c0re regenerates `agents.conf` (and triggers a reload) on two
|
c0re regenerates `agents.conf` (and triggers a reload) on two
|
||||||
triggers: every topology change (new/removed agents) and every 10s
|
triggers: every topology change (new/removed agents) and every 10s
|
||||||
|
|
@ -161,8 +160,8 @@ selected by which (if any) external TLS source is set:
|
||||||
| mode | config | cert source | `.well-known` scheme |
|
| mode | config | cert source | `.well-known` scheme |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| self-signed (default) | neither `tls.certDir` nor `tls.acme` set | host hive-CA signs a gateway leaf (RSA-4096) | `https` |
|
| self-signed (default) | neither `tls.certDir` nor `tls.acme` set | host hive-CA signs a gateway leaf (RSA-4096) | `https` |
|
||||||
| ACME (Let's Encrypt) | `tls.acme.enable = true` | nginx via HTTP-01 | `https` |
|
| ACME (Let's Encrypt) | `tls.acme.enable = true` | nginx inside container via HTTP-01 | `https` |
|
||||||
| operator cert | `tls.certDir` set | read from the operator's dir | `https` |
|
| operator cert | `tls.certDir` set | bind-mounted from host | `https` |
|
||||||
|
|
||||||
The `gateway.selfSignedTls` option is **deprecated and ignored** — self-signed
|
The `gateway.selfSignedTls` option is **deprecated and ignored** — self-signed
|
||||||
is now derived from the absence of `tls.certDir` / `tls.acme`. Setting it to
|
is now derived from the absence of `tls.certDir` / `tls.acme`. Setting it to
|
||||||
|
|
@ -183,7 +182,7 @@ services.hyperhive.gateway = {
|
||||||
};
|
};
|
||||||
```
|
```
|
||||||
|
|
||||||
nginx obtains and auto-renews certs via the ACME HTTP-01 challenge on `port` (default 80). Certs land in `/var/lib/acme/` on the host, managed by nixpkgs's `security.acme` in the ordinary way.
|
nginx inside the gateway container obtains and auto-renews certs via the ACME HTTP-01 challenge on `port` (default 80). The gateway container shares the host network namespace (`privateNetwork = false`) so outbound ACME requests work without any extra routing. Certs are stored inside the container's persistent state dir (`/var/lib/acme/` inside `hive-gateway`; survives restarts because `ephemeral = false`).
|
||||||
|
|
||||||
**Requirements**: `services.hyperhive.domain` must be publicly DNS-resolvable to this host, and `openFirewall = true` so Let's Encrypt can reach `/.well-known/acme-challenge/`. Each active vhost (main domain, `forge.<swarm-domain>`, `chat.<swarm-domain>`) gets its own cert via separate ACME challenges — the swarm services default to names under `services.hyperhive.swarm.domain`, so **every one of those names must resolve to this host too**, not just the hive's own.
|
**Requirements**: `services.hyperhive.domain` must be publicly DNS-resolvable to this host, and `openFirewall = true` so Let's Encrypt can reach `/.well-known/acme-challenge/`. Each active vhost (main domain, `forge.<swarm-domain>`, `chat.<swarm-domain>`) gets its own cert via separate ACME challenges — the swarm services default to names under `services.hyperhive.swarm.domain`, so **every one of those names must resolve to this host too**, not just the hive's own.
|
||||||
|
|
||||||
|
|
@ -195,11 +194,9 @@ Mutual exclusion: `tls.certDir` set together with `tls.acme.enable = true` fails
|
||||||
|
|
||||||
On by default, and listens on `httpsPort` (default 443) on every vhost beside the plain-http `port` (default 80).
|
On by default, and listens on `httpsPort` (default 443) on every vhost beside the plain-http `port` (default 80).
|
||||||
|
|
||||||
The issuer is a **host-held hive CA**, not a bare self-signed leaf. A host service (`hive-tls-ca.service`, from the `hive-tls` module) generates a long-lived CA (`services.hyperhive.tls.caValidityDays`, default ~20y) under `services.hyperhive.tls.stateDir` (default `/var/lib/hive-tls`), then signs a gateway **leaf** (`leafValidityDays`, default 30d) with it. `hive-gateway-self-signed-cert` then imports the leaf into nginx's state dir (`/var/lib/hive-gateway/tls/{cert,key}.pem`).
|
The issuer is a **host-held hive CA**, not a bare self-signed leaf. A host service (`hive-tls-ca.service`, from the `hive-tls` module) generates a long-lived CA (`services.hyperhive.tls.caValidityDays`, default ~20y) under `services.hyperhive.tls.stateDir` (default `/var/lib/hive-tls`), then signs a gateway **leaf** (`leafValidityDays`, default 30d) with it. The leaf dir is bind-mounted read-only into the gateway container at `/run/hive-ca`; an in-container import unit copies the leaf into nginx's state dir (`/var/lib/hive-gateway/tls/{cert,key}.pem`) with the owner/mode nginx needs.
|
||||||
|
|
||||||
⚠️ **Do not collapse that import unit into pointing nginx at the CA dir.** It does two jobs. It re-modes the leaf (`hive-tls-ca` writes the key `0600 root:root`; nginx's pre-start `nginx -t` runs as the *nginx user*, so a `0600` key fails the config test and blocks the unit), and it guarantees **every cert path the nginx config names exists** — which is what the swarm-services fallback below is for. Removing it re-creates the #3097 outage.
|
**Why a CA, not a bare leaf**: a bare self-signed leaf is its own trust anchor, so every regeneration is a new anchor every consumer must re-trust — and a runtime-generated, in-container leaf can't be wired into an agent's build-time trust store at all. With a stable CA, agents and federation peers trust it *once*; leaf rotation never re-breaks them.
|
||||||
|
|
||||||
**Why a CA, not a bare leaf**: a bare self-signed leaf is its own trust anchor, so every regeneration is a new anchor every consumer must re-trust — and a runtime-generated leaf can't be wired into an agent's build-time trust store at all. With a stable CA, agents and federation peers trust it *once*; leaf rotation never re-breaks them.
|
|
||||||
|
|
||||||
**What consumers trust**: `trust-bundle.pem` in the same state dir, not `ca.pem`. The hive CA is itself issued under the swarm root ([`swarm/ca.md`](swarm/ca.md) has the hierarchy), and an intermediate is not a chain a verifier can terminate at — so the bundle carries the hive CA plus whatever it is rooted at. nginx is handed the leaf with the hive CA appended for the same reason. Everything that trusts the hive's TLS reads the bundle: agents (via `security.pki.certificateFiles`), the CI and forge containers, and a federating peer.
|
**What consumers trust**: `trust-bundle.pem` in the same state dir, not `ca.pem`. The hive CA is itself issued under the swarm root ([`swarm/ca.md`](swarm/ca.md) has the hierarchy), and an intermediate is not a chain a verifier can terminate at — so the bundle carries the hive CA plus whatever it is rooted at. nginx is handed the leaf with the hive CA appended for the same reason. Everything that trusts the hive's TLS reads the bundle: agents (via `security.pki.certificateFiles`), the CI and forge containers, and a federating peer.
|
||||||
|
|
||||||
|
|
@ -223,11 +220,11 @@ services.hyperhive.gateway = {
|
||||||
};
|
};
|
||||||
```
|
```
|
||||||
|
|
||||||
nginx reads the directory directly and uses `cert.pem` + `key.pem` (override `tls.certName`/`tls.keyName` for different filenames). Both modes listen on `httpsPort` (default 443) and emit `https://` in `.well-known` responses.
|
The directory is bind-mounted read-only into the gateway container at `/run/hive-tls/`. nginx uses `cert.pem` + `key.pem` (override `tls.certName`/`tls.keyName` for different filenames). Both modes listen on `httpsPort` (default 443) and emit `https://` in `.well-known` responses.
|
||||||
|
|
||||||
`tls.certDir` and `tls.acme.enable` set together is an assertion error.
|
`tls.certDir` and `tls.acme.enable` set together is an assertion error.
|
||||||
|
|
||||||
**Key file permissions**: nixpkgs's `security.acme` outputs private keys as `0640 root:acme` by default. nginx runs as the `nginx` user and cannot read a key with that ownership. Fix with:
|
**Key file permissions**: nixpkgs's `security.acme` outputs private keys as `0640 root:acme` by default. nginx inside the gateway container runs as the `nginx` user and cannot read a key with that ownership. Fix with:
|
||||||
|
|
||||||
```nix
|
```nix
|
||||||
security.acme.certs."example.com".group = "nginx";
|
security.acme.certs."example.com".group = "nginx";
|
||||||
|
|
@ -427,10 +424,11 @@ JS/CSS assets skip `try_files` entirely and get the immutable cache
|
||||||
headers. Nix store paths are content-addressed — the hash changes on
|
headers. Nix store paths are content-addressed — the hash changes on
|
||||||
any content change — so `max-age=31536000` is safe.
|
any content change — so `max-age=31536000` is safe.
|
||||||
|
|
||||||
**Why the nix store path resolves**: `HIVE_AGENT_FRONTEND_DIR` is a nix
|
**Why nix store is reachable from the gateway container**: nspawn
|
||||||
store path baked in at hive-c0re build time, and c0re (writing
|
containers bind-mount `/nix/store` read-only by default. The
|
||||||
`agents.conf`) and nginx (serving files from it) are on the same machine,
|
`HIVE_AGENT_FRONTEND_DIR` path is a nix store path baked in at
|
||||||
so they see the same store.
|
hive-c0re build time — the same path is visible to both c0re (writing
|
||||||
|
`agents.conf`) and the gateway nginx (serving files from it).
|
||||||
|
|
||||||
**Graceful degradation**: if `HIVE_AGENT_FRONTEND_DIR` is empty or
|
**Graceful degradation**: if `HIVE_AGENT_FRONTEND_DIR` is empty or
|
||||||
unset (e.g. a build that predates `cfg.frontend`), each agent gets the
|
unset (e.g. a build that predates `cfg.frontend`), each agent gets the
|
||||||
|
|
@ -495,7 +493,9 @@ services.hyperhive.gateway.auth = {
|
||||||
The credential store lives at the fixed path
|
The credential store lives at the fixed path
|
||||||
`/var/lib/hyperhive/gateway/gateway.htpasswd` on the host. A tmpfiles
|
`/var/lib/hyperhive/gateway/gateway.htpasswd` on the host. A tmpfiles
|
||||||
rule pre-creates the file on first boot; no manual path configuration
|
rule pre-creates the file on first boot; no manual path configuration
|
||||||
is required. nginx reads it at that path directly.
|
is required. The file is exposed inside the gateway container at
|
||||||
|
`/run/hive-state/gateway.htpasswd` via the existing gateway state
|
||||||
|
bind-mount.
|
||||||
|
|
||||||
Manage users with `hivectl gateway`. `hivectl` sends the request over the
|
Manage users with `hivectl gateway`. `hivectl` sends the request over the
|
||||||
host admin socket and the `hive-c0re` daemon performs the write at its
|
host admin socket and the `hive-c0re` daemon performs the write at its
|
||||||
|
|
|
||||||
|
|
@ -115,19 +115,49 @@ schemes pick their own.
|
||||||
dnsmasq is **authoritative** for the hive's own zones — answers
|
dnsmasq is **authoritative** for the hive's own zones — answers
|
||||||
`<hive-domain>`, `forge.<hive-domain>`, `matrix.<hive-domain>`
|
`<hive-domain>`, `forge.<hive-domain>`, `matrix.<hive-domain>`
|
||||||
queries with the bridge IP (where nginx is reachable). Everything
|
queries with the bridge IP (where nginx is reachable). Everything
|
||||||
else is forwarded to the host's own resolvers: dnsmasq runs on the host
|
else is forwarded to the host's own resolvers: dnsmasq reads the
|
||||||
and reads the host's `/etc/resolv.conf` directly. Containers don't need
|
gateway container's `/etc/resolv.conf`, the host copy nixos-container
|
||||||
to know the upstream — they query the bridge IP and dnsmasq does the
|
makes at each container start. Containers don't need to know the
|
||||||
right thing per-name.
|
upstream — they query the bridge IP and dnsmasq does the right thing
|
||||||
|
per-name.
|
||||||
|
|
||||||
There is deliberately no fallback `server=`: dnsmasq queries all known
|
That copy is one-shot — systemd-nspawn(1) is explicit that nothing
|
||||||
upstreams in parallel, so a hardcoded public resolver would take a share
|
propagates into it after early init, because resolv.conf is normally
|
||||||
of normal traffic, not just cover the gap.
|
updated by rename rather than in place. Left alone, a host network
|
||||||
|
change (new router, new lease, laptop moving networks) would strand
|
||||||
|
dnsmasq on a resolver that no longer answers, and every non-hive
|
||||||
|
lookup from every agent would hang until someone restarted the
|
||||||
|
gateway. The host-side **`hive-gateway-resolv`** path unit closes
|
||||||
|
that: it watches `/etc/resolv.conf`, `machinectl copy-to`s it into
|
||||||
|
the container, and reloads dnsmasq (`SIGHUP` — re-read upstreams +
|
||||||
|
flush cache, nothing dropped). The watch is armed before
|
||||||
|
`network-pre.target` so the boot's first DHCP write is caught as well,
|
||||||
|
and the sync also runs once per gateway start to pick up a resolver
|
||||||
|
change that happened while the container was down. A host file with no
|
||||||
|
`nameserver` line is
|
||||||
|
skipped rather than pushed, so a mid-rewrite snapshot can't blank the
|
||||||
|
hive's DNS. There is deliberately no fallback `server=`: dnsmasq
|
||||||
|
queries all known upstreams in parallel, so a hardcoded public
|
||||||
|
resolver would take a share of normal traffic, not just cover the gap.
|
||||||
|
|
||||||
dnsmasq runs on the host and reads the host's `/etc/resolv.conf`
|
Two alternatives that look simpler and aren't:
|
||||||
directly, so a network change (new router, new lease, laptop moving
|
|
||||||
networks) reaches it the moment openresolv rewrites the file. There is
|
- **A path unit inside the container.** The host replaces
|
||||||
nothing to synchronise and no unit watching for it.
|
`/etc/resolv.conf` by rename, and that `IN_MOVED_TO` doesn't cross
|
||||||
|
the nspawn mount namespace — the same reason hive-c0re reloads nginx
|
||||||
|
from the host side after each `agents.conf` write.
|
||||||
|
- **Bind-mounting the host's `/etc/resolv.conf` into the container.**
|
||||||
|
openresolv writes a temp file and renames over the target, so the
|
||||||
|
bind mount would pin the *first* inode for the container's whole
|
||||||
|
lifetime — strictly worse than the copy, which at least a restart
|
||||||
|
clears. (Reachability is not the problem here: the gateway runs with
|
||||||
|
`privateNetwork = false`, so it shares the host's netns and can reach
|
||||||
|
anything the host can.)
|
||||||
|
|
||||||
|
`machinectl copy-to` is used rather than writing to the container's
|
||||||
|
rootfs from the host, so the push goes through the container's own
|
||||||
|
mount namespace and stays correct if `/etc` is ever assembled
|
||||||
|
differently (e.g. `system.etc.overlay`).
|
||||||
|
|
||||||
`bind-interfaces` + `interface = [ bridgeName "lo" ]` means the
|
`bind-interfaces` + `interface = [ bridgeName "lo" ]` means the
|
||||||
listener only accepts queries from the bridge interface (plus lo for
|
listener only accepts queries from the bridge interface (plus lo for
|
||||||
|
|
@ -259,4 +289,4 @@ loopback — the hive-c0re admin socket is a UDS, not TCP.
|
||||||
|
|
||||||
## Cross-references
|
## Cross-references
|
||||||
|
|
||||||
- `docs/gateway.md` — vhost map + the gateway's other duties
|
- `docs/gateway.md` — vhost map + the gateway container's other duties
|
||||||
|
|
|
||||||
|
|
@ -187,16 +187,8 @@ bind-mount compatibility with user namespace UID mapping and is tracked as a TOD
|
||||||
|
|
||||||
`hive-c0re` runs as the unprivileged system user `hive-core`
|
`hive-c0re` runs as the unprivileged system user `hive-core`
|
||||||
(`/var/lib/hyperhive` owned by `hive-core:hive-core`). It cannot
|
(`/var/lib/hyperhive` owned by `hive-core:hive-core`). It cannot
|
||||||
directly invoke `nixos-container`, `journalctl -M`, or act on a system
|
directly invoke `nixos-container`, `journalctl -M`, or `systemctl
|
||||||
unit (`systemctl reload nginx`) — those require root. `hive-priv` fills
|
-M hive-gateway` — those require root. `hive-priv` fills this gap.
|
||||||
this gap.
|
|
||||||
|
|
||||||
⚠️ **`ReloadGatewayNginx` acts on a host unit, so nothing implicitly
|
|
||||||
scopes it.** Its containment is the unit name hard-coded in `hive-priv`:
|
|
||||||
a caller cannot name the unit, so the verb cannot be steered at another
|
|
||||||
service. **A privileged verb needs something bounding what it can act
|
|
||||||
on; when that isn't a namespace, it has to be a constant the caller
|
|
||||||
can't supply.**
|
|
||||||
|
|
||||||
### hive-priv
|
### hive-priv
|
||||||
|
|
||||||
|
|
@ -215,7 +207,7 @@ known operations; there is no arbitrary command pass-through:
|
||||||
| `DestroyContainer` | `nixos-container destroy <name>` |
|
| `DestroyContainer` | `nixos-container destroy <name>` |
|
||||||
| `ListContainers` | `nixos-container list` |
|
| `ListContainers` | `nixos-container list` |
|
||||||
| `ReadContainerJournal` | `journalctl -M <container> -n <n> [filters...]` |
|
| `ReadContainerJournal` | `journalctl -M <container> -n <n> [filters...]` |
|
||||||
| `ReloadGatewayNginx` | `systemctl reload/start/reset-failed nginx` (host unit; the unit name is hard-coded, not a parameter) |
|
| `ReloadGatewayNginx` | `systemctl -M hive-gateway reload/start/reset-failed nginx` |
|
||||||
| `WriteNspawnFlags` | write `/etc/nixos-containers/<container>.conf` (bind-mount list + network isolation vars) |
|
| `WriteNspawnFlags` | write `/etc/nixos-containers/<container>.conf` (bind-mount list + network isolation vars) |
|
||||||
| `WriteResourceLimits` | write `CPUQuota=`/`MemoryMax=`/`CPUWeight=`/`IOWeight=` systemd drop-in for agent container |
|
| `WriteResourceLimits` | write `CPUQuota=`/`MemoryMax=`/`CPUWeight=`/`IOWeight=` systemd drop-in for agent container |
|
||||||
| `RemoveServiceDropin` | remove `container@<name>.service.d/` drop-in on destroy |
|
| `RemoveServiceDropin` | remove `container@<name>.service.d/` drop-in on destroy |
|
||||||
|
|
|
||||||
|
|
@ -808,7 +808,7 @@ Bare `hivectl stop` stops everything; scope flags narrow it to specific sub-agen
|
||||||
* `--agent <NAME>` — A specific sub-agent by name. Repeatable: `--agent a --agent b`
|
* `--agent <NAME>` — A specific sub-agent by name. Repeatable: `--agent a --agent b`
|
||||||
* `--ci` — The CI runner container (`hive-ci`)
|
* `--ci` — The CI runner container (`hive-ci`)
|
||||||
* `--forge` — The forge container (`hive-forge`)
|
* `--forge` — The forge container (`hive-forge`)
|
||||||
* `--gateway` — The gateway (`hive-gateway`) — nginx on the host, not a container
|
* `--gateway` — The gateway container (`hive-gateway`)
|
||||||
* `--matrix` — The matrix container (`hive-matrix`)
|
* `--matrix` — The matrix container (`hive-matrix`)
|
||||||
* `--graceful` — Gracefully quiesce each agent before stopping, instead of a hard stop. Each agent gets a graceful-stop DAG on the job queue: the harness is signalled, runs one stop-checkpoint turn to flush durable `/state`, drains, then the container is stopped (bounded by a 3-min timeout that falls back to a hard stop). All drains overlap. Applies to agents only
|
* `--graceful` — Gracefully quiesce each agent before stopping, instead of a hard stop. Each agent gets a graceful-stop DAG on the job queue: the harness is signalled, runs one stop-checkpoint turn to flush durable `/state`, drains, then the container is stopped (bounded by a 3-min timeout that falls back to a hard stop). All drains overlap. Applies to agents only
|
||||||
* `--no-wait` — Return immediately after the stop DAGs are queued instead of waiting for them with live per-node progress
|
* `--no-wait` — Return immediately after the stop DAGs are queued instead of waiting for them with live per-node progress
|
||||||
|
|
@ -829,7 +829,7 @@ Bare `hivectl start` restores the agents stopped by the last broad-scope `stop`
|
||||||
* `--agent <NAME>` — A specific sub-agent by name. Repeatable: `--agent a --agent b`
|
* `--agent <NAME>` — A specific sub-agent by name. Repeatable: `--agent a --agent b`
|
||||||
* `--ci` — The CI runner container (`hive-ci`)
|
* `--ci` — The CI runner container (`hive-ci`)
|
||||||
* `--forge` — The forge container (`hive-forge`)
|
* `--forge` — The forge container (`hive-forge`)
|
||||||
* `--gateway` — The gateway (`hive-gateway`) — nginx on the host, not a container
|
* `--gateway` — The gateway container (`hive-gateway`)
|
||||||
* `--matrix` — The matrix container (`hive-matrix`)
|
* `--matrix` — The matrix container (`hive-matrix`)
|
||||||
* `--no-wait` — Return immediately after the start DAGs are queued instead of waiting for them with live per-node progress
|
* `--no-wait` — Return immediately after the start DAGs are queued instead of waiting for them with live per-node progress
|
||||||
|
|
||||||
|
|
@ -849,7 +849,7 @@ Bare `hivectl restart` restarts everything; scope flags narrow it.
|
||||||
* `--agent <NAME>` — A specific sub-agent by name. Repeatable: `--agent a --agent b`
|
* `--agent <NAME>` — A specific sub-agent by name. Repeatable: `--agent a --agent b`
|
||||||
* `--ci` — The CI runner container (`hive-ci`)
|
* `--ci` — The CI runner container (`hive-ci`)
|
||||||
* `--forge` — The forge container (`hive-forge`)
|
* `--forge` — The forge container (`hive-forge`)
|
||||||
* `--gateway` — The gateway (`hive-gateway`) — nginx on the host, not a container
|
* `--gateway` — The gateway container (`hive-gateway`)
|
||||||
* `--matrix` — The matrix container (`hive-matrix`)
|
* `--matrix` — The matrix container (`hive-matrix`)
|
||||||
* `--graceful` — Gracefully quiesce each agent on the stop half (see `stop --graceful`). Applies to agents only
|
* `--graceful` — Gracefully quiesce each agent on the stop half (see `stop --graceful`). Applies to agents only
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -79,10 +79,8 @@ lifecycle events, or another container's boot log.
|
||||||
- `unit` — filter to a systemd unit (e.g. `hive-c0re.service`).
|
- `unit` — filter to a systemd unit (e.g. `hive-c0re.service`).
|
||||||
- `container` — nspawn machine name verbatim. Agent containers use
|
- `container` — nspawn machine name verbatim. Agent containers use
|
||||||
the `h-<name>` prefix (e.g. `h-iris`); infrastructure containers
|
the `h-<name>` prefix (e.g. `h-iris`); infrastructure containers
|
||||||
use their full name (e.g. `hive-ci`, `hive-forge`, `hive-matrix`).
|
use their full name (e.g. `hive-ci`, `hive-forge`, `hive-matrix`,
|
||||||
Omit for the host journal. The gateway has no machine — its nginx
|
`hive-gateway`). Omit for the host journal.
|
||||||
runs on the host, so read it with `unit: nginx.service` and no
|
|
||||||
`container`.
|
|
||||||
- `lines` — how many lines to return (default 30, max 100).
|
- `lines` — how many lines to return (default 30, max 100).
|
||||||
- `priority` — minimum syslog level (`emerg` … `debug`).
|
- `priority` — minimum syslog level (`emerg` … `debug`).
|
||||||
- `grep` — regex matched against log message fields (`journalctl --grep`).
|
- `grep` — regex matched against log message fields (`journalctl --grep`).
|
||||||
|
|
|
||||||
|
|
@ -180,12 +180,10 @@ omitted — agents share the host netns, so there is no per-container net
|
||||||
counter (per-agent network needs the netns-isolation roadmap in
|
counter (per-agent network needs the netns-isolation roadmap in
|
||||||
`docs/network.md`).
|
`docs/network.md`).
|
||||||
|
|
||||||
**1NFR4** — start / stop / restart the four hive infrastructure services
|
**1NFR4** — start / stop / restart the four hive infrastructure
|
||||||
(`hive-ci`, `hive-forge`, `hive-gateway`, `hive-matrix`) directly from the
|
containers (`hive-ci`, `hive-forge`, `hive-gateway`, `hive-matrix`)
|
||||||
dashboard, without needing an `infra_admin` agent's `restart` MCP tool.
|
directly from the dashboard, without needing an `infra_admin` agent's
|
||||||
Three are containers; `hive-gateway` is the host's `nginx.service`, and is
|
`restart` MCP tool. One row per container: name, a `badge-ok`/`badge-fail`
|
||||||
the one an agent may **not** restart — this panel is the way it gets
|
|
||||||
bounced. One row per service: name, a `badge-ok`/`badge-fail`
|
|
||||||
running/stopped dot, and `↺ R3ST4RT` + `■ ST0P` (running) or `▶ ST4RT`
|
running/stopped dot, and `↺ R3ST4RT` + `■ ST0P` (running) or `▶ ST4RT`
|
||||||
(stopped) buttons, same themed-confirm pattern as the K3PT ST4T3
|
(stopped) buttons, same themed-confirm pattern as the K3PT ST4T3
|
||||||
tombstone actions. Backed by
|
tombstone actions. Backed by
|
||||||
|
|
@ -462,7 +460,7 @@ The current capabilities are:
|
||||||
| `manage_root_agent` | allows the `set_status` / lifecycle tools on the root agent |
|
| `manage_root_agent` | allows the `set_status` / lifecycle tools on the root agent |
|
||||||
| `read_host_journal` | unlocks `get_host_journal` to read journald from inside a container |
|
| `read_host_journal` | unlocks `get_host_journal` to read journald from inside a container |
|
||||||
| `query_agent_state` | allows `get_loose_ends(agent: "<name>")` calls targeting other agents |
|
| `query_agent_state` | allows `get_loose_ends(agent: "<name>")` calls targeting other agents |
|
||||||
| `infra_admin` | allows `restart` on hive infrastructure containers (`hive-ci`, `hive-forge`, `hive-matrix`; the gateway is operator-only); each restart is logged to the AUDIT trail |
|
| `infra_admin` | allows `restart` on hive infrastructure containers (`hive-ci`, `hive-gateway`, `hive-forge`); each restart is logged to the AUDIT trail |
|
||||||
|
|
||||||
Each row is one agent. Columns are the capability names returned by
|
Each row is one agent. Columns are the capability names returned by
|
||||||
`GET /api/capabilities` as `caps: Vec<String>`. Checking or unchecking
|
`GET /api/capabilities` as `caps: Vec<String>`. Checking or unchecking
|
||||||
|
|
@ -1342,11 +1340,9 @@ below — some endpoints aren't in it yet.
|
||||||
rootfs every ~5 min, `-x` excluding the shared read-only nix store.
|
rootfs every ~5 min, `-x` excluding the shared read-only nix store.
|
||||||
`null` until the first sample lands.
|
`null` until the first sample lands.
|
||||||
- `POST /api/infra-container/{name}/{action}` — start / stop / restart a
|
- `POST /api/infra-container/{name}/{action}` — start / stop / restart a
|
||||||
hive infra service (C0R3 › 1NFR4 panel). `name` parses into the
|
hive infra container (C0R3 › 1NFR4 panel). `name` parses into the
|
||||||
`InfraContainer` allowlist (`hive-ci`/`hive-forge`/`hive-gateway`/
|
`InfraContainer` allowlist (`hive-ci`/`hive-forge`/`hive-gateway`/
|
||||||
`hive-matrix`, 400 on unknown), and the variant decides the unit —
|
`hive-matrix`, 400 on unknown), `action` ∈ `start|stop|restart`. Calls
|
||||||
`container@<name>.service`, or `nginx.service` for the gateway.
|
|
||||||
`action` ∈ `start|stop|restart`. Calls
|
|
||||||
the same `priv_client::control_infra_container` helper the
|
the same `priv_client::control_infra_container` helper the
|
||||||
`infra_admin` agent path uses; records an `audit_log` entry
|
`infra_admin` agent path uses; records an `audit_log` entry
|
||||||
(`start_infra`/`stop_infra`/`restart_infra`, actor `"operator"`) either
|
(`start_infra`/`stop_infra`/`restart_infra`, actor `"operator"`) either
|
||||||
|
|
|
||||||
|
|
@ -326,8 +326,7 @@ pub struct GetHostJournalArgs {
|
||||||
/// nspawn machine name verbatim (e.g. `h-iris`). Omit for host journal.
|
/// nspawn machine name verbatim (e.g. `h-iris`). Omit for host journal.
|
||||||
/// Agent containers use the `h-` prefix (e.g. `h-iris`); infrastructure
|
/// Agent containers use the `h-` prefix (e.g. `h-iris`); infrastructure
|
||||||
/// containers use their full name (e.g. `hive-ci`, `hive-forge`,
|
/// containers use their full name (e.g. `hive-ci`, `hive-forge`,
|
||||||
/// `hive-matrix`). The gateway has no machine — its nginx runs on the
|
/// `hive-matrix`, `hive-gateway`).
|
||||||
/// host, so read it with `unit: nginx.service` and no `container`.
|
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
pub container: Option<String>,
|
pub container: Option<String>,
|
||||||
/// Number of lines to return (default 30, max 100).
|
/// Number of lines to return (default 30, max 100).
|
||||||
|
|
|
||||||
|
|
@ -681,9 +681,8 @@ impl AgentServer {
|
||||||
Only succeeds if `name` is a direct child of this agent in the topology \
|
Only succeeds if `name` is a direct child of this agent in the topology \
|
||||||
tree — the server enforces this. No approval required. \
|
tree — the server enforces this. No approval required. \
|
||||||
Agents holding the `infra_admin` capability may also pass a hive \
|
Agents holding the `infra_admin` capability may also pass a hive \
|
||||||
infrastructure container name (`hive-ci`, `hive-forge`, `hive-matrix`) \
|
infrastructure container name (`hive-ci`, `hive-gateway`, `hive-forge`) \
|
||||||
to restart it directly via the privileged helper. The gateway is \
|
to restart it directly via the privileged helper."
|
||||||
not restartable by an agent — ask the operator."
|
|
||||||
)]
|
)]
|
||||||
async fn restart(&self, Parameters(args): Parameters<RestartArgs>) -> String {
|
async fn restart(&self, Parameters(args): Parameters<RestartArgs>) -> String {
|
||||||
let log = format!("{args:?}");
|
let log = format!("{args:?}");
|
||||||
|
|
@ -791,8 +790,7 @@ impl AgentServer {
|
||||||
`container`: nspawn machine name verbatim (e.g. `h-iris`). Omit for host journal. \
|
`container`: nspawn machine name verbatim (e.g. `h-iris`). Omit for host journal. \
|
||||||
Agent containers use the `h-` prefix (e.g. `h-iris`, `h-atlas`); \
|
Agent containers use the `h-` prefix (e.g. `h-iris`, `h-atlas`); \
|
||||||
infrastructure containers use their full name (e.g. `hive-ci`, `hive-forge`, \
|
infrastructure containers use their full name (e.g. `hive-ci`, `hive-forge`, \
|
||||||
`hive-matrix`). The gateway is not a container: its nginx logs are in \
|
`hive-matrix`, `hive-gateway`). \
|
||||||
the host journal, so omit `container` and pass `unit: nginx.service`. \
|
|
||||||
`lines`: how many lines (default 30, max 100). \
|
`lines`: how many lines (default 30, max 100). \
|
||||||
`priority`: minimum syslog level enum. \
|
`priority`: minimum syslog level enum. \
|
||||||
`grep`: regex matched against log message fields (journalctl --grep). \
|
`grep`: regex matched against log message fields (journalctl --grep). \
|
||||||
|
|
|
||||||
|
|
@ -60,8 +60,8 @@ fn allowed_capability_tools() -> Vec<String> {
|
||||||
match t.as_str() {
|
match t.as_str() {
|
||||||
"read_host_journal" => tools.push("get_host_journal".to_owned()),
|
"read_host_journal" => tools.push("get_host_journal".to_owned()),
|
||||||
// infra_admin lets an agent restart hive infrastructure
|
// infra_admin lets an agent restart hive infrastructure
|
||||||
// containers (hive-ci / hive-forge / hive-matrix — not the
|
// containers (hive-ci / hive-gateway / hive-forge) through the
|
||||||
// gateway) through the existing `restart` tool. Unlock it here so agents that hold
|
// existing `restart` tool. Unlock it here so agents that hold
|
||||||
// the capability without the full `lifecycle` group can still
|
// the capability without the full `lifecycle` group can still
|
||||||
// call it; c0re re-checks the capability server-side and only
|
// call it; c0re re-checks the capability server-side and only
|
||||||
// honours infra-container names via this path.
|
// honours infra-container names via this path.
|
||||||
|
|
|
||||||
|
|
@ -169,11 +169,11 @@ pub async fn serve(
|
||||||
/// Best-effort unlinks any stale socket left from a crashed previous
|
/// Best-effort unlinks any stale socket left from a crashed previous
|
||||||
/// harness (clean exit removes it, but `bind(2)` refuses to overwrite
|
/// harness (clean exit removes it, but `bind(2)` refuses to overwrite
|
||||||
/// an existing file) and `mkdir -p`s the parent for first-boot. Mode
|
/// an existing file) and `mkdir -p`s the parent for first-boot. Mode
|
||||||
/// `0o666` — world-accessible so the gateway's nginx process can
|
/// `0o666` — world-accessible so the gateway container's nginx process
|
||||||
/// `connect(2)` without sharing a group with the agent user. What bounds
|
/// can `connect(2)` without sharing a group with the agent user.
|
||||||
/// that is the per-agent subdir (`/run/hive-agent/<name>/`): the socket
|
/// The per-agent subdir (`/run/hive-agent/<name>/`) is only accessible
|
||||||
/// mode grants everyone, so the directory's permissions decide who gets
|
/// to containers that have it bind-mounted, so world-accessible sockets
|
||||||
/// to ask.
|
/// are not a material risk.
|
||||||
///
|
///
|
||||||
/// Marker-gating + the gateway-side consumer: see
|
/// Marker-gating + the gateway-side consumer: see
|
||||||
/// [`docs/gateway.md::Per-agent unix-socket upstream`](../../../docs/gateway.md).
|
/// [`docs/gateway.md::Per-agent unix-socket upstream`](../../../docs/gateway.md).
|
||||||
|
|
|
||||||
|
|
@ -1,13 +1,9 @@
|
||||||
//! Dashboard endpoint for operator-driven infra lifecycle (start / stop /
|
//! Dashboard endpoint for operator-driven infra-container lifecycle
|
||||||
//! restart on `hive-ci`, `hive-forge`, `hive-gateway`, `hive-matrix`).
|
//! (start / stop / restart on `hive-ci`, `hive-forge`, `hive-gateway`,
|
||||||
//! Parallels the `infra_admin`-gated agent path in
|
//! `hive-matrix`). Parallels the `infra_admin`-gated agent path in
|
||||||
//! `socket_server/lifecycle_handlers.rs::handle_restart_infra`, but this one
|
//! `socket_server/lifecycle_handlers.rs::handle_restart_infra`, but this one
|
||||||
//! is reached from the dashboard — already fully operator-authenticated —
|
//! is reached from the dashboard — already fully operator-authenticated —
|
||||||
//! so no capability check is needed here, just the same audit trail.
|
//! so no capability check is needed here, just the same audit trail.
|
||||||
//!
|
|
||||||
//! The two surfaces cover different sets: this endpoint takes all four,
|
|
||||||
//! while the agent path refuses the gateway — nginx on the host fronts
|
|
||||||
//! every hive service, so bouncing it is the operator's call.
|
|
||||||
|
|
||||||
use axum::{
|
use axum::{
|
||||||
extract::{Path as AxumPath, State},
|
extract::{Path as AxumPath, State},
|
||||||
|
|
@ -31,7 +27,7 @@ use super::{AppState, error_response};
|
||||||
post,
|
post,
|
||||||
path = "/api/infra-container/{name}/{action}",
|
path = "/api/infra-container/{name}/{action}",
|
||||||
params(
|
params(
|
||||||
("name" = String, Path, description = "infra service name (hive-ci/hive-forge/hive-gateway/hive-matrix)"),
|
("name" = String, Path, description = "infra container name (hive-ci/hive-forge/hive-gateway/hive-matrix)"),
|
||||||
("action" = String, Path, description = "start | stop | restart"),
|
("action" = String, Path, description = "start | stop | restart"),
|
||||||
),
|
),
|
||||||
responses(
|
responses(
|
||||||
|
|
@ -57,8 +53,8 @@ pub(super) async fn post_infra_container(
|
||||||
));
|
));
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
let target = container.name();
|
let unit = container.unit_name();
|
||||||
tracing::info!(%target, %action, "dashboard: infra container action");
|
tracing::info!(%unit, %action, "dashboard: infra container action");
|
||||||
let result = crate::priv_client::control_infra_container(container, infra_action).await;
|
let result = crate::priv_client::control_infra_container(container, infra_action).await;
|
||||||
let outcome = if result.is_ok() {
|
let outcome = if result.is_ok() {
|
||||||
crate::audit_log::AuditOutcome::Ok
|
crate::audit_log::AuditOutcome::Ok
|
||||||
|
|
@ -70,12 +66,12 @@ pub(super) async fn post_infra_container(
|
||||||
state
|
state
|
||||||
.coord
|
.coord
|
||||||
.audit_log
|
.audit_log
|
||||||
.record("operator", action_label, target, outcome, detail.as_deref())
|
.record("operator", action_label, unit, outcome, detail.as_deref())
|
||||||
{
|
{
|
||||||
state.coord.emit_audit_entry(entry);
|
state.coord.emit_audit_entry(entry);
|
||||||
}
|
}
|
||||||
match result {
|
match result {
|
||||||
Ok(()) => (StatusCode::OK, "ok").into_response(),
|
Ok(()) => (StatusCode::OK, "ok").into_response(),
|
||||||
Err(e) => error_response(&format!("{target}: {e:#}")),
|
Err(e) => error_response(&format!("{unit}: {e:#}")),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -1,15 +1,13 @@
|
||||||
//! Journal-read endpoints for the dashboard.
|
//! Journal-read endpoints for the dashboard.
|
||||||
//!
|
//!
|
||||||
//! `GET /api/journal/{name}` reads a managed agent container's journal, OR
|
//! `GET /api/journal/{name}` reads a managed agent container's journal, OR
|
||||||
//! one of the four hive infra services (`hive-ci`, `hive-forge`,
|
//! one of the four hive infra containers (`hive-ci`, `hive-forge`,
|
||||||
//! `hive-gateway`, `hive-matrix` — [`hive_priv_sock::InfraContainer`] is the
|
//! `hive-gateway`, `hive-matrix` — [`hive_priv_sock::InfraContainer`] is the
|
||||||
//! allowlist). A container's journal is a `journalctl -M` read, delegated
|
//! allowlist), via the root helper (`journalctl -M`, delegated to hive-priv
|
||||||
//! to the root helper since entering a machine needs privileges hive-c0re
|
//! since hive-c0re is unprivileged). `GET /api/journal-host` reads
|
||||||
//! doesn't have; `hive-gateway` is nginx on the host, so it reads host
|
//! host-side journald, both gated by an allow-list of known units so
|
||||||
//! journald filtered to that unit and needs no helper at all.
|
//! arbitrary unit names can't be probed. Operator-only by virtue of the
|
||||||
//! `GET /api/journal-host` reads host-side journald, both gated by an
|
//! dashboard binding host-only.
|
||||||
//! allow-list of known units so arbitrary unit names can't be probed.
|
|
||||||
//! Operator-only by virtue of the dashboard binding host-only.
|
|
||||||
|
|
||||||
use axum::{
|
use axum::{
|
||||||
extract::Path as AxumPath,
|
extract::Path as AxumPath,
|
||||||
|
|
@ -42,12 +40,11 @@ pub(super) struct JournalQuery {
|
||||||
/// container namespace and needs root — is delegated to hive-priv.
|
/// container namespace and needs root — is delegated to hive-priv.
|
||||||
///
|
///
|
||||||
/// `name` is either a managed agent name (`iris`, optionally already
|
/// `name` is either a managed agent name (`iris`, optionally already
|
||||||
/// carrying the `h-` prefix) or one of the four infra names (`hive-ci` /
|
/// carrying the `h-` prefix) or one of the four infra container names
|
||||||
/// `hive-forge` / `hive-gateway` / `hive-matrix` — see
|
/// (`hive-ci` / `hive-forge` / `hive-gateway` / `hive-matrix` — see
|
||||||
/// [`hive_priv_sock::InfraContainer`]). Infra targets don't run the
|
/// [`hive_priv_sock::InfraContainer`]). Infra containers don't run the
|
||||||
/// per-agent hive daemons, so `unit` is ignored for them — the whole
|
/// per-agent hive daemons, so `unit` is ignored for them — always the
|
||||||
/// machine journal, or for the gateway the host journal filtered to its
|
/// full machine journal.
|
||||||
/// own unit.
|
|
||||||
#[utoipa::path(
|
#[utoipa::path(
|
||||||
get,
|
get,
|
||||||
path = "/api/journal/{name}",
|
path = "/api/journal/{name}",
|
||||||
|
|
@ -69,17 +66,7 @@ pub(super) async fn get_journal(
|
||||||
let lines = q.lines.unwrap_or(500).min(5000);
|
let lines = q.lines.unwrap_or(500).min(5000);
|
||||||
|
|
||||||
if let Ok(infra) = name.parse::<hive_priv_sock::InfraContainer>() {
|
if let Ok(infra) = name.parse::<hive_priv_sock::InfraContainer>() {
|
||||||
return match infra.target() {
|
return read_journal_response(infra.unit_name(), None, lines).await;
|
||||||
hive_priv_sock::InfraTarget::Container(machine) => {
|
|
||||||
read_journal_response(machine, None, lines).await
|
|
||||||
}
|
|
||||||
// No machine to enter — the gateway's nginx is a host unit, so
|
|
||||||
// this is a plain host-journal read filtered to it. `-M` is
|
|
||||||
// what needed root here, not journalctl itself.
|
|
||||||
hive_priv_sock::InfraTarget::HostUnit(unit) => {
|
|
||||||
read_host_journal_response(Some(unit), lines).await
|
|
||||||
}
|
|
||||||
};
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Defense-in-depth format check so weird chars never reach the
|
// Defense-in-depth format check so weird chars never reach the
|
||||||
|
|
@ -193,42 +180,21 @@ pub(super) async fn get_journal_host(
|
||||||
axum::extract::Query(q): axum::extract::Query<JournalHostQuery>,
|
axum::extract::Query(q): axum::extract::Query<JournalHostQuery>,
|
||||||
) -> Result<Response, ProblemDetails> {
|
) -> Result<Response, ProblemDetails> {
|
||||||
let lines = q.lines.unwrap_or(500).min(5000);
|
let lines = q.lines.unwrap_or(500).min(5000);
|
||||||
// `nginx.service` is the gateway — its logs are host-side.
|
let allowed = ["hive-c0re.service", "hive-priv.service"];
|
||||||
let allowed = ["hive-c0re.service", "hive-priv.service", "nginx.service"];
|
|
||||||
let unit = match q.unit.as_deref().filter(|s| !s.is_empty()) {
|
|
||||||
Some(u) => {
|
|
||||||
let unit = if u.ends_with(".service") {
|
|
||||||
u.to_owned()
|
|
||||||
} else {
|
|
||||||
format!("{u}.service")
|
|
||||||
};
|
|
||||||
if !allowed.contains(&unit.as_str()) {
|
|
||||||
return Err(ProblemDetails::from_status_code(StatusCode::BAD_REQUEST)
|
|
||||||
.with_detail(format!("journal-host: unknown unit {unit:?}")));
|
|
||||||
}
|
|
||||||
Some(unit)
|
|
||||||
}
|
|
||||||
None => None,
|
|
||||||
};
|
|
||||||
read_host_journal_response(unit.as_deref(), lines).await
|
|
||||||
}
|
|
||||||
|
|
||||||
/// `journalctl [-u <unit>]` on the host + response formatting. No `-M`, so
|
|
||||||
/// no root and no priv-client hop — hive-c0re reads host journald directly.
|
|
||||||
///
|
|
||||||
/// ⚠️ `unit` is trusted by the time it gets here: [`get_journal_host`]
|
|
||||||
/// allow-lists an operator-supplied one, and [`get_journal`] passes a unit
|
|
||||||
/// that came from the [`hive_priv_sock::InfraContainer`] enum. Don't hand
|
|
||||||
/// this a raw query parameter.
|
|
||||||
async fn read_host_journal_response(
|
|
||||||
unit: Option<&str>,
|
|
||||||
lines: u32,
|
|
||||||
) -> Result<Response, ProblemDetails> {
|
|
||||||
let mut cmd = tokio::process::Command::new("journalctl");
|
let mut cmd = tokio::process::Command::new("journalctl");
|
||||||
cmd.args(["--no-pager", "--output=short-iso", "--lines"])
|
cmd.args(["--no-pager", "--output=short-iso", "--lines"])
|
||||||
.arg(lines.to_string());
|
.arg(lines.to_string());
|
||||||
if let Some(u) = unit {
|
if let Some(u) = q.unit.as_deref().filter(|s| !s.is_empty()) {
|
||||||
cmd.args(["-u", u]);
|
let unit = if u.ends_with(".service") {
|
||||||
|
u.to_owned()
|
||||||
|
} else {
|
||||||
|
format!("{u}.service")
|
||||||
|
};
|
||||||
|
if !allowed.contains(&unit.as_str()) {
|
||||||
|
return Err(ProblemDetails::from_status_code(StatusCode::BAD_REQUEST)
|
||||||
|
.with_detail(format!("journal-host: unknown unit {unit:?}")));
|
||||||
|
}
|
||||||
|
cmd.args(["-u", &unit]);
|
||||||
}
|
}
|
||||||
match cmd.output().await {
|
match cmd.output().await {
|
||||||
Ok(out) => {
|
Ok(out) => {
|
||||||
|
|
|
||||||
|
|
@ -154,7 +154,7 @@ async fn infra_container_views() -> Vec<InfraContainerView> {
|
||||||
let mut infra_containers = Vec::with_capacity(hive_priv_sock::InfraContainer::ALL.len());
|
let mut infra_containers = Vec::with_capacity(hive_priv_sock::InfraContainer::ALL.len());
|
||||||
for container in hive_priv_sock::InfraContainer::ALL {
|
for container in hive_priv_sock::InfraContainer::ALL {
|
||||||
infra_containers.push(InfraContainerView {
|
infra_containers.push(InfraContainerView {
|
||||||
name: container.name(),
|
name: container.unit_name(),
|
||||||
running: crate::lifecycle::infra_is_running(container).await,
|
running: crate::lifecycle::infra_is_running(container).await,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,8 @@
|
||||||
//! Runtime nginx include-file generator for the gateway's per-agent
|
//! Runtime nginx include-file generator for the gateway's per-agent
|
||||||
//! `/agent/<name>/` location blocks. Writes
|
//! `/agent/<name>/` location blocks. Writes
|
||||||
//! `/var/lib/hyperhive/gateway/agents.conf` on every topology change.
|
//! `/var/lib/hyperhive/gateway/agents.conf` on every topology change.
|
||||||
//! UDS upstream selection, the reload trigger, and idempotency:
|
//! UDS upstream selection, reload trigger (`systemd-run
|
||||||
|
//! --machine=hive-gateway`), and idempotency:
|
||||||
//! `docs/gateway.md::Per-agent unix-socket upstream`.
|
//! `docs/gateway.md::Per-agent unix-socket upstream`.
|
||||||
|
|
||||||
use anyhow::{Context, Result};
|
use anyhow::{Context, Result};
|
||||||
|
|
@ -18,13 +19,13 @@ use crate::agent_sockets;
|
||||||
/// Set when `write` publishes a new agents.conf; cleared when
|
/// Set when `write` publishes a new agents.conf; cleared when
|
||||||
/// `reload_gateway_nginx` submits the reload command successfully.
|
/// `reload_gateway_nginx` submits the reload command successfully.
|
||||||
/// Lets `spawn_poll` retry the reload on subsequent ticks when the
|
/// Lets `spawn_poll` retry the reload on subsequent ticks when the
|
||||||
/// previous attempt failed (e.g. gateway nginx temporarily down, priv
|
/// previous attempt failed (e.g. gateway container temporarily down,
|
||||||
/// helper unreachable) without re-writing the already-correct file.
|
/// systemd-run not found) without re-writing the already-correct file.
|
||||||
static RELOAD_PENDING: AtomicBool = AtomicBool::new(false);
|
static RELOAD_PENDING: AtomicBool = AtomicBool::new(false);
|
||||||
|
|
||||||
/// Unix timestamp (seconds) of the last failed reload attempt.
|
/// Unix timestamp (seconds) of the last failed reload attempt.
|
||||||
/// `reload_if_pending` backs off to once per `RELOAD_RETRY_SECS` after a
|
/// `reload_if_pending` backs off to once per `RELOAD_RETRY_SECS` after a
|
||||||
/// failure so a permanently broken gateway (bad config, nginx down)
|
/// failure so a permanently broken gateway (bad config, container down)
|
||||||
/// doesn't hammer `systemctl` on every 10-second `spawn_poll` tick.
|
/// doesn't hammer `systemctl` on every 10-second `spawn_poll` tick.
|
||||||
static LAST_FAILED_RELOAD: AtomicU64 = AtomicU64::new(0);
|
static LAST_FAILED_RELOAD: AtomicU64 = AtomicU64::new(0);
|
||||||
|
|
||||||
|
|
@ -78,7 +79,7 @@ fn render(names: &[String], frontend_dir: Option<&str>) -> String {
|
||||||
let mut out = String::from(
|
let mut out = String::from(
|
||||||
"# Generated by hive-c0re \u{2014} do not edit.\
|
"# Generated by hive-c0re \u{2014} do not edit.\
|
||||||
\n# Refreshed on every topology change + when agents bind/drop their unix sockets.\
|
\n# Refreshed on every topology change + when agents bind/drop their unix sockets.\
|
||||||
\n# Reload triggered by hive-c0re via hive-priv (systemctl reload nginx).\n",
|
\n# Reload triggered by hive-c0re via systemd-run --machine=hive-gateway.\n",
|
||||||
);
|
);
|
||||||
for name in names {
|
for name in names {
|
||||||
// Two upstream forms because named locations (split mode's
|
// Two upstream forms because named locations (split mode's
|
||||||
|
|
@ -164,15 +165,15 @@ fn render(names: &[String], frontend_dir: Option<&str>) -> String {
|
||||||
/// body matches what's already on disk (idempotent; avoids spurious
|
/// body matches what's already on disk (idempotent; avoids spurious
|
||||||
/// gateway reloads on a quiet tick).
|
/// gateway reloads on a quiet tick).
|
||||||
///
|
///
|
||||||
/// After a successful write, triggers the appropriate nginx action via
|
/// After a successful write, triggers the appropriate nginx action inside
|
||||||
/// `hive-priv` (hive-c0re runs unprivileged and cannot act on a system
|
/// the gateway container via `hive-priv` (which has the
|
||||||
/// unit): reload when nginx is active, reset-failed+start when in a
|
/// `--machine=hive-gateway` transport rights hive-c0re lacks):
|
||||||
/// failed state, plain start otherwise. Writer and nginx are on the same
|
/// reload when nginx is active, reset-failed+start when in a failed
|
||||||
/// machine, so it is a plain unit action with no machine-bus hop.
|
/// state, plain start otherwise. This is intentionally host-side rather
|
||||||
/// It stays an explicit trigger rather than a systemd path
|
/// than relying on a systemd path unit inside the container watching the
|
||||||
/// unit watching the file: the write already knows it changed something,
|
/// bind-mounted file: `IN_MOVED_TO` (fired by the atomic rename) does
|
||||||
/// and a watcher would turn one causal edge into a race with the writer's
|
/// not reliably propagate across the nspawn mount-namespace boundary, so
|
||||||
/// own rename (see `docs/gateway.md`).
|
/// the path-unit approach was silently broken (see `docs/gateway.md`).
|
||||||
///
|
///
|
||||||
/// The priv call is best-effort — a failed sync is logged but not fatal.
|
/// The priv call is best-effort — a failed sync is logged but not fatal.
|
||||||
/// `reload_if_pending` retries on the next `spawn_poll` tick so a
|
/// `reload_if_pending` retries on the next `spawn_poll` tick so a
|
||||||
|
|
@ -211,7 +212,7 @@ pub async fn write(names: &[String]) -> Result<()> {
|
||||||
|
|
||||||
/// Retry a pending nginx reload if a previous attempt failed.
|
/// Retry a pending nginx reload if a previous attempt failed.
|
||||||
/// Called by `spawn_poll` on each tick so a transient failure
|
/// Called by `spawn_poll` on each tick so a transient failure
|
||||||
/// (gateway nginx temporarily down, priv-helper error) is
|
/// (gateway container temporarily down, systemd-run error) is
|
||||||
/// recovered automatically without requiring a new file write.
|
/// recovered automatically without requiring a new file write.
|
||||||
///
|
///
|
||||||
/// Backs off to one retry per `RELOAD_RETRY_SECS` after a failure so a
|
/// Backs off to one retry per `RELOAD_RETRY_SECS` after a failure so a
|
||||||
|
|
@ -239,8 +240,8 @@ pub async fn reload_if_pending() {
|
||||||
/// Synchronise the gateway nginx unit with the current agents.conf via
|
/// Synchronise the gateway nginx unit with the current agents.conf via
|
||||||
/// `hive-priv` (privileged helper). The state-aware logic (active →
|
/// `hive-priv` (privileged helper). The state-aware logic (active →
|
||||||
/// reload; failed → reset-failed + start; inactive/unknown → start)
|
/// reload; failed → reset-failed + start; inactive/unknown → start)
|
||||||
/// runs inside hive-priv, which is root; hive-c0re runs as the
|
/// runs inside hive-priv where it has the `--machine=hive-gateway`
|
||||||
/// unprivileged `hive-core` user and cannot act on a system unit.
|
/// transport rights that hive-c0re (unprivileged) lacks.
|
||||||
///
|
///
|
||||||
/// `RELOAD_PENDING` is cleared only after a successful operation so
|
/// `RELOAD_PENDING` is cleared only after a successful operation so
|
||||||
/// `reload_if_pending` keeps retrying on failure.
|
/// `reload_if_pending` keeps retrying on failure.
|
||||||
|
|
|
||||||
|
|
@ -604,14 +604,14 @@ pub async fn is_running(name: &str) -> bool {
|
||||||
.is_ok_and(|s| s.success())
|
.is_ok_and(|s| s.success())
|
||||||
}
|
}
|
||||||
|
|
||||||
/// True when a hive infrastructure service's systemd unit is active.
|
/// True when a hive infrastructure container's systemd unit is active.
|
||||||
/// Sibling of [`is_running`] for sub-agents, but infra names (`hive-ci`, …)
|
/// Sibling of [`is_running`] for sub-agents, but infra container/unit names
|
||||||
/// have no `h-` prefix to strip and are not all containers, so the unit
|
/// (`hive-ci`, …) already have no `h-` prefix to strip, so this queries
|
||||||
/// comes from the variant itself rather than from [`container_name`]. Used
|
/// `container@<unit_name>.service` directly rather than going through
|
||||||
/// by the dashboard C0R3 page's 1NFR4 sub-tab to show each one's live
|
/// [`container_name`]. Used by the dashboard C0R3 page's 1NFR4 sub-tab to
|
||||||
/// status dot.
|
/// show each infra container's live status dot.
|
||||||
pub async fn infra_is_running(container: hive_priv_sock::InfraContainer) -> bool {
|
pub async fn infra_is_running(container: hive_priv_sock::InfraContainer) -> bool {
|
||||||
let unit = container.service_unit();
|
let unit = format!("container@{}.service", container.unit_name());
|
||||||
Command::new("systemctl")
|
Command::new("systemctl")
|
||||||
.args(["is-active", "--quiet", &unit])
|
.args(["is-active", "--quiet", &unit])
|
||||||
.status()
|
.status()
|
||||||
|
|
|
||||||
|
|
@ -168,9 +168,9 @@ pub async fn sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<()> {
|
||||||
}
|
}
|
||||||
|
|
||||||
// Refresh /var/lib/hyperhive/gateway/agents.conf — the nginx include
|
// Refresh /var/lib/hyperhive/gateway/agents.conf — the nginx include
|
||||||
// file the gateway reads at runtime. c0re then triggers a reload (or
|
// file the gateway container bind-mounts and nginx reads at runtime.
|
||||||
// start) of the host's nginx via hive-priv, since c0re is unprivileged.
|
// c0re triggers a reload (or start) inside hive-gateway via hive-priv
|
||||||
// Same best-effort + non-fatal shape.
|
// after writing the file. Same best-effort + non-fatal shape.
|
||||||
if let Err(e) = crate::gateway_nginx::write(&agent_names).await {
|
if let Err(e) = crate::gateway_nginx::write(&agent_names).await {
|
||||||
tracing::warn!(error = ?e, "gateway_nginx::write failed (non-fatal)");
|
tracing::warn!(error = ?e, "gateway_nginx::write failed (non-fatal)");
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -219,12 +219,11 @@ pub fn shared_root() -> PathBuf {
|
||||||
pub const KNOWLEDGE_DIR: &str = "/var/lib/hyperhive/knowledge";
|
pub const KNOWLEDGE_DIR: &str = "/var/lib/hyperhive/knowledge";
|
||||||
|
|
||||||
/// `gateway/` — generated nginx include fragments for the gateway vhost.
|
/// `gateway/` — generated nginx include fragments for the gateway vhost.
|
||||||
/// nginx runs on the host and reads this path directly. ⚠️ Nothing about
|
/// The gateway container bind-mounts *this subdir only* (not the whole
|
||||||
/// this path confines it: what keeps nginx away from the rest of
|
/// state root) at `/run/hive-state/`, so nginx can read `agents.conf`
|
||||||
/// `/var/lib/hyperhive/` (forge/matrix tokens, etc.) is the unit's own
|
/// without the rest of `/var/lib/hyperhive/` (forge/matrix tokens, etc.)
|
||||||
/// sandbox, so widening that sandbox widens what a gateway compromise
|
/// being exposed to the gateway container.
|
||||||
/// reaches.
|
// nix: bind-mounted into the gateway container (hive-gateway.nix) — must match.
|
||||||
// nix: named by the gateway's nginx config (hive-gateway/vhosts.nix) — must match.
|
|
||||||
#[must_use]
|
#[must_use]
|
||||||
pub fn gateway_dir() -> PathBuf {
|
pub fn gateway_dir() -> PathBuf {
|
||||||
state_root().join("gateway")
|
state_root().join("gateway")
|
||||||
|
|
|
||||||
|
|
@ -477,21 +477,20 @@ pub async fn register_ci_runner(token: &str) -> Result<()> {
|
||||||
.await?)
|
.await?)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Restart a hive infrastructure service on the host (thin wrapper over
|
/// Restart a hive infrastructure container on the host (thin wrapper over
|
||||||
/// [`control_infra_container`] with `action = Restart`). Callers must
|
/// [`control_infra_container`] with `action = Restart`). hive-priv
|
||||||
/// already have checked that the requesting agent holds the `infra_admin`
|
/// re-validates `container` against its root-side allowlist; callers must
|
||||||
/// capability *and* that the target is
|
/// already have checked the requesting agent holds the `infra_admin`
|
||||||
/// [`agent_restartable`](InfraContainer::agent_restartable).
|
/// capability.
|
||||||
pub async fn restart_infra_container(container: InfraContainer) -> Result<()> {
|
pub async fn restart_infra_container(container: InfraContainer) -> Result<()> {
|
||||||
control_infra_container(container, InfraAction::Restart).await
|
control_infra_container(container, InfraAction::Restart).await
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Start / stop / restart a hive infrastructure service (`hive-ci`,
|
/// Start / stop / restart a hive infrastructure container (`hive-ci`,
|
||||||
/// `hive-gateway`, `hive-forge`, `hive-matrix`) on the host via `systemctl
|
/// `hive-gateway`, `hive-forge`, `hive-matrix`) on the host via `systemctl
|
||||||
/// <action> <unit>`, where the unit is derived root-side from the variant
|
/// <action> container@<container>.service`. The [`InfraContainer`] enum is
|
||||||
/// (`container@<name>.service`, or `nginx.service` for the gateway). The
|
/// the allowlist — hive-priv needs no name re-validation. Used by the
|
||||||
/// [`InfraContainer`] enum is the allowlist — hive-priv needs no name
|
/// hive-wide `hivectl stop` / `hivectl start` flow.
|
||||||
/// re-validation. Used by the hive-wide `hivectl stop` / `start` flow.
|
|
||||||
pub async fn control_infra_container(container: InfraContainer, action: InfraAction) -> Result<()> {
|
pub async fn control_infra_container(container: InfraContainer, action: InfraAction) -> Result<()> {
|
||||||
ok(call(&PrivRequest::ControlInfraContainer { container, action }).await?)
|
ok(call(&PrivRequest::ControlInfraContainer { container, action }).await?)
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -863,7 +863,7 @@ async fn handle_stop(
|
||||||
await_dags(coord, &queued, std::time::Duration::from_mins(2)).await;
|
await_dags(coord, &queued, std::time::Duration::from_mins(2)).await;
|
||||||
}
|
}
|
||||||
for &container in infra {
|
for &container in infra {
|
||||||
let name = container.name();
|
let name = container.unit_name();
|
||||||
match crate::priv_client::control_infra_container(container, InfraAction::Stop).await {
|
match crate::priv_client::control_infra_container(container, InfraAction::Stop).await {
|
||||||
Ok(()) => ok_items.push(name.to_owned()),
|
Ok(()) => ok_items.push(name.to_owned()),
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
|
|
@ -919,7 +919,7 @@ async fn handle_start(
|
||||||
let mut errors: Vec<String> = Vec::new();
|
let mut errors: Vec<String> = Vec::new();
|
||||||
|
|
||||||
for &container in infra {
|
for &container in infra {
|
||||||
let name = container.name();
|
let name = container.unit_name();
|
||||||
match crate::priv_client::control_infra_container(container, InfraAction::Start).await {
|
match crate::priv_client::control_infra_container(container, InfraAction::Start).await {
|
||||||
Ok(()) => ok_items.push(name.to_owned()),
|
Ok(()) => ok_items.push(name.to_owned()),
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
|
|
@ -990,7 +990,7 @@ async fn handle_restart_scoped(
|
||||||
}
|
}
|
||||||
|
|
||||||
for &container in &infra {
|
for &container in &infra {
|
||||||
let name = container.name();
|
let name = container.unit_name();
|
||||||
let res = async {
|
let res = async {
|
||||||
crate::priv_client::control_infra_container(container, InfraAction::Stop).await?;
|
crate::priv_client::control_infra_container(container, InfraAction::Stop).await?;
|
||||||
crate::priv_client::control_infra_container(container, InfraAction::Start).await
|
crate::priv_client::control_infra_container(container, InfraAction::Start).await
|
||||||
|
|
|
||||||
|
|
@ -31,13 +31,12 @@ pub(super) async fn handle_start(coord: &Arc<Coordinator>, agent: &str, name: &s
|
||||||
/// orthogonal: it is gated on the `infra_admin` capability and audited, so it
|
/// orthogonal: it is gated on the `infra_admin` capability and audited, so it
|
||||||
/// stays ahead of the topology guard.
|
/// stays ahead of the topology guard.
|
||||||
pub(super) async fn handle_restart(coord: &Arc<Coordinator>, agent: &str, name: &str) -> Response {
|
pub(super) async fn handle_restart(coord: &Arc<Coordinator>, agent: &str, name: &str) -> Response {
|
||||||
// Infra restart: an agent holding the `infra_admin` capability can
|
// Infra-container restart: an agent holding the `infra_admin`
|
||||||
// restart a hive infrastructure service (hive-ci / hive-forge /
|
// capability can restart a hive infrastructure container (hive-ci /
|
||||||
// hive-matrix) by passing its name to the same restart tool. The
|
// hive-gateway / hive-forge / hive-matrix) by passing its name to the
|
||||||
// `InfraContainer` enum parse both recognises these (never agent
|
// same restart tool. The `InfraContainer` enum parse both recognises
|
||||||
// children, so disjoint from the child path below) and yields the typed
|
// these (never agent children, so disjoint from the child path below)
|
||||||
// value the restart path needs. It recognises `hive-gateway` too, which
|
// and yields the typed value the restart path needs.
|
||||||
// is then refused — a name the agent surface knows but may not act on.
|
|
||||||
if let Ok(container) = name.parse::<hive_priv_sock::InfraContainer>() {
|
if let Ok(container) = name.parse::<hive_priv_sock::InfraContainer>() {
|
||||||
return handle_restart_infra(coord, agent, container).await;
|
return handle_restart_infra(coord, agent, container).await;
|
||||||
}
|
}
|
||||||
|
|
@ -61,7 +60,7 @@ async fn handle_restart_infra(
|
||||||
agent: &str,
|
agent: &str,
|
||||||
container: hive_priv_sock::InfraContainer,
|
container: hive_priv_sock::InfraContainer,
|
||||||
) -> Response {
|
) -> Response {
|
||||||
let name = container.name();
|
let name = container.unit_name();
|
||||||
// Record the attempt in the operator-visible privileged-action audit
|
// Record the attempt in the operator-visible privileged-action audit
|
||||||
// trail, then emit a live `AuditEntryAdded` so the dashboard audit view
|
// trail, then emit a live `AuditEntryAdded` so the dashboard audit view
|
||||||
// appends it off `/dashboard/stream`. Best-effort: `record` returns the
|
// appends it off `/dashboard/stream`. Best-effort: `record` returns the
|
||||||
|
|
@ -76,22 +75,6 @@ async fn handle_restart_infra(
|
||||||
coord.emit_audit_entry(entry);
|
coord.emit_audit_entry(entry);
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
// Some targets are off-limits to agents regardless of capability — the
|
|
||||||
// gateway, because nginx fronts every hive service from the host and an
|
|
||||||
// agent bouncing it takes out the forge, the dashboard and matrix at
|
|
||||||
// once, including the route its own fix would have to travel. Checked
|
|
||||||
// before the capability so the refusal doesn't read as "ask for
|
|
||||||
// infra_admin"; no capability grants this.
|
|
||||||
if !container.agent_restartable() {
|
|
||||||
tracing::warn!(%agent, %name, "agent: infra restart denied (not agent-restartable)");
|
|
||||||
audit(
|
|
||||||
crate::audit_log::AuditOutcome::Err,
|
|
||||||
Some("denied: target is not agent-restartable"),
|
|
||||||
);
|
|
||||||
return Response::Err {
|
|
||||||
message: format!("`{name}` cannot be restarted by an agent; ask the operator"),
|
|
||||||
};
|
|
||||||
}
|
|
||||||
if !crate::capabilities::has_cap(agent, hive_sh4re::permissions::Capability::InfraAdmin) {
|
if !crate::capabilities::has_cap(agent, hive_sh4re::permissions::Capability::InfraAdmin) {
|
||||||
tracing::warn!(%agent, %name, "agent: infra restart denied (no infra_admin capability)");
|
tracing::warn!(%agent, %name, "agent: infra restart denied (no infra_admin capability)");
|
||||||
audit(
|
audit(
|
||||||
|
|
|
||||||
|
|
@ -14,10 +14,10 @@ use std::path::{Path, PathBuf};
|
||||||
use anyhow::{Context, Result};
|
use anyhow::{Context, Result};
|
||||||
|
|
||||||
/// Host-side parent directory holding per-agent socket subdirs. The
|
/// Host-side parent directory holding per-agent socket subdirs. The
|
||||||
/// gateway's nginx runs on the host and reads this whole tree, so it can
|
/// gateway container bind-mounts this whole tree (read-only) so it
|
||||||
/// `proxy_pass` to any agent. Each agent's container bind-mounts only its
|
/// can `proxy_pass` to any agent. Each agent's container bind-mounts
|
||||||
/// own `<name>/` subdir — that mount is what stops one agent reaching
|
/// only its own `<name>/` subdir — agents can only access their own
|
||||||
/// another's socket. The literal lives in `hive-host-sock` (shared with
|
/// sockets. The literal lives in `hive-host-sock` (shared with
|
||||||
/// `hivectl`); re-exported here under the name this module's consumers
|
/// `hivectl`); re-exported here under the name this module's consumers
|
||||||
/// have always used.
|
/// have always used.
|
||||||
pub use hive_host_sock::AGENT_SOCKET_DIR;
|
pub use hive_host_sock::AGENT_SOCKET_DIR;
|
||||||
|
|
@ -166,7 +166,7 @@ pub fn write(names: &[String]) -> Result<()> {
|
||||||
///
|
///
|
||||||
/// Also calls `gateway_nginx::reload_if_pending` on every tick to
|
/// Also calls `gateway_nginx::reload_if_pending` on every tick to
|
||||||
/// retry a gateway nginx reload that may have failed on the previous
|
/// retry a gateway nginx reload that may have failed on the previous
|
||||||
/// tick (e.g. gateway nginx temporarily down). This recovers
|
/// tick (e.g. gateway container temporarily down). This recovers
|
||||||
/// gateway routing without needing a manual gateway restart.
|
/// gateway routing without needing a manual gateway restart.
|
||||||
///
|
///
|
||||||
/// Mirrors the spawn-loop shape used by `crash_watch`,
|
/// Mirrors the spawn-loop shape used by `crash_watch`,
|
||||||
|
|
|
||||||
|
|
@ -50,7 +50,7 @@ pub fn agent_state_dir(name: &Ident) -> PathBuf {
|
||||||
|
|
||||||
/// `gateway/gateway.htpasswd` — nginx basic-auth credential store for the
|
/// `gateway/gateway.htpasswd` — nginx basic-auth credential store for the
|
||||||
/// operator dashboard vhost. `hivectl`'s `--htpasswd-file` clap default.
|
/// operator dashboard vhost. `hivectl`'s `--htpasswd-file` clap default.
|
||||||
// nix: read by the gateway's nginx on the host (hive-gateway/) — must match.
|
// nix: read by the gateway container's nginx (hive-gateway.nix) — must match.
|
||||||
pub const GATEWAY_HTPASSWD: &str = "/var/lib/hyperhive/gateway/gateway.htpasswd";
|
pub const GATEWAY_HTPASSWD: &str = "/var/lib/hyperhive/gateway/gateway.htpasswd";
|
||||||
|
|
||||||
/// `/run/hive-agent` — per-agent runtime socket dir root (web UI unix
|
/// `/run/hive-agent` — per-agent runtime socket dir root (web UI unix
|
||||||
|
|
@ -452,8 +452,7 @@ pub struct QuotaRow {
|
||||||
/// (the bare `hivectl stop` / `start`); set individual fields to restrict
|
/// (the bare `hivectl stop` / `start`); set individual fields to restrict
|
||||||
/// (e.g. only `agents` → just the sub-agent containers). `agents` covers
|
/// (e.g. only `agents` → just the sub-agent containers). `agents` covers
|
||||||
/// every managed sub-agent container; the rest are the named infra
|
/// every managed sub-agent container; the rest are the named infra
|
||||||
/// services (`hive-ci`, `hive-forge`, `hive-gateway`, `hive-matrix`) —
|
/// containers (`hive-ci`, `hive-forge`, `hive-gateway`, `hive-matrix`).
|
||||||
/// containers, bar the gateway, which is the host's nginx unit.
|
|
||||||
//
|
//
|
||||||
// A flat bag of independent flag toggles — one bool per selectable
|
// A flat bag of independent flag toggles — one bool per selectable
|
||||||
// container class — is exactly the right shape here: each maps 1:1 to a
|
// container class — is exactly the right shape here: each maps 1:1 to a
|
||||||
|
|
|
||||||
|
|
@ -33,24 +33,17 @@ pub const MANAGER_NAME: &str = "ruth";
|
||||||
pub const AGENT_PREFIX: &str = "h-";
|
pub const AGENT_PREFIX: &str = "h-";
|
||||||
|
|
||||||
/// Sibling service containers managed by hive-c0re. This doubles as the
|
/// Sibling service containers managed by hive-c0re. This doubles as the
|
||||||
/// authoritative allowlist for the requests that name a container as a
|
/// authoritative allowlist for infra lifecycle ops
|
||||||
/// string (bind-mount edits, journal reads): only these — or a valid agent
|
/// ([`PrivRequest::ControlInfraContainer`]): any of these four may be
|
||||||
/// name — are accepted. `hive-c0re` is deliberately absent; so is the
|
/// started / stopped / restarted (by the hive-wide `hivectl stop`/`start`
|
||||||
/// gateway, whose nginx is a plain host unit rather than a container.
|
/// flow or an `infra_admin` agent's `restart`). `hive-c0re` is deliberately
|
||||||
|
/// absent — stopping it would sever the very socket the request arrived on.
|
||||||
/// hive-priv re-validates against this list root-side, so it's authoritative
|
/// hive-priv re-validates against this list root-side, so it's authoritative
|
||||||
/// regardless of what the caller sends.
|
/// regardless of what the caller sends.
|
||||||
///
|
pub const SIBLING_CONTAINERS: &[&str] = &["hive-forge", "hive-matrix", "hive-gateway", "hive-ci"];
|
||||||
/// ⚠️ This is the *container-name* allowlist, not the lifecycle one:
|
|
||||||
/// [`InfraContainer`] is what gates
|
|
||||||
/// [`PrivRequest::ControlInfraContainer`], and it has one variant more than
|
|
||||||
/// this list (the gateway). Keep the distinction — a name that belongs to
|
|
||||||
/// no container has no business reaching a `-M` / `nixos-container` call.
|
|
||||||
pub const SIBLING_CONTAINERS: &[&str] = &["hive-forge", "hive-matrix", "hive-ci"];
|
|
||||||
|
|
||||||
/// Lifecycle verb for [`PrivRequest::ControlInfraContainer`]. Maps directly
|
/// Lifecycle verb for [`PrivRequest::ControlInfraContainer`]. Maps directly
|
||||||
/// to `systemctl <verb> <unit>`, where the unit comes from
|
/// to `systemctl <verb> container@<container>.service`.
|
||||||
/// [`InfraContainer::service_unit`] — usually `container@<name>.service`,
|
|
||||||
/// but not always (see [`InfraTarget`]).
|
|
||||||
#[derive(Debug, Clone, Copy, Serialize, Deserialize)]
|
#[derive(Debug, Clone, Copy, Serialize, Deserialize)]
|
||||||
#[serde(rename_all = "snake_case")]
|
#[serde(rename_all = "snake_case")]
|
||||||
pub enum InfraAction {
|
pub enum InfraAction {
|
||||||
|
|
@ -78,9 +71,8 @@ impl InfraAction {
|
||||||
/// by a runtime check. The c0re↔hive-priv wire form uses serde's default
|
/// by a runtime check. The c0re↔hive-priv wire form uses serde's default
|
||||||
/// variant naming (`"Ci"`, `"Forge"`, …); it's an internal protocol (both
|
/// variant naming (`"Ci"`, `"Forge"`, …); it's an internal protocol (both
|
||||||
/// ends rebuild together) so it needn't match the container name.
|
/// ends rebuild together) so it needn't match the container name.
|
||||||
/// [`name`](Self::name) is the separate stable identity string
|
/// [`unit_name`](Self::unit_name) is the separate systemd / container name
|
||||||
/// (`hive-ci`), and [`service_unit`](Self::service_unit) the systemd unit
|
/// (`hive-ci`).
|
||||||
/// it actually resolves to.
|
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||||
pub enum InfraContainer {
|
pub enum InfraContainer {
|
||||||
Ci,
|
Ci,
|
||||||
|
|
@ -89,26 +81,9 @@ pub enum InfraContainer {
|
||||||
Matrix,
|
Matrix,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// What an [`InfraContainer`] resolves to on the host — i.e. the thing a
|
|
||||||
/// lifecycle verb actually acts on.
|
|
||||||
///
|
|
||||||
/// The gateway is why this exists: its nginx + dnsmasq are host
|
|
||||||
/// services, so "restart the gateway" means a plain host unit. Every
|
|
||||||
/// other variant is a container.
|
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
||||||
pub enum InfraTarget {
|
|
||||||
/// An nspawn container, controlled via `container@<name>.service` and
|
|
||||||
/// readable with `journalctl -M <name>`.
|
|
||||||
Container(&'static str),
|
|
||||||
/// A plain host unit. There is no machine to enter: no `-M` journal,
|
|
||||||
/// no `nixos-container` verb.
|
|
||||||
HostUnit(&'static str),
|
|
||||||
}
|
|
||||||
|
|
||||||
impl InfraContainer {
|
impl InfraContainer {
|
||||||
/// Every controllable infra target. A superset of
|
/// Every controllable infra container. The source of truth that
|
||||||
/// [`SIBLING_CONTAINERS`] — not all of these are containers (see the
|
/// [`SIBLING_CONTAINERS`] is kept consistent with (see the test).
|
||||||
/// test).
|
|
||||||
pub const ALL: [InfraContainer; 4] = [
|
pub const ALL: [InfraContainer; 4] = [
|
||||||
InfraContainer::Ci,
|
InfraContainer::Ci,
|
||||||
InfraContainer::Forge,
|
InfraContainer::Forge,
|
||||||
|
|
@ -116,14 +91,11 @@ impl InfraContainer {
|
||||||
InfraContainer::Matrix,
|
InfraContainer::Matrix,
|
||||||
];
|
];
|
||||||
|
|
||||||
/// Stable identity string, e.g. `hive-ci`. This is what the operator
|
/// The container / systemd-unit name, e.g. `hive-ci` →
|
||||||
/// types, what the dashboard displays, and what [`FromStr`](std::str::FromStr)
|
/// `container@hive-ci.service`. (Distinct from the serde wire form,
|
||||||
/// parses — it stays `hive-gateway` even though the gateway is no
|
/// which is the default variant name `"Ci"`.)
|
||||||
/// longer a container, because it names the *service*, not its
|
|
||||||
/// implementation. (Distinct from the serde wire form, which is the
|
|
||||||
/// default variant name `"Ci"`.)
|
|
||||||
#[must_use]
|
#[must_use]
|
||||||
pub fn name(self) -> &'static str {
|
pub fn unit_name(self) -> &'static str {
|
||||||
match self {
|
match self {
|
||||||
InfraContainer::Ci => "hive-ci",
|
InfraContainer::Ci => "hive-ci",
|
||||||
InfraContainer::Forge => "hive-forge",
|
InfraContainer::Forge => "hive-forge",
|
||||||
|
|
@ -131,48 +103,16 @@ impl InfraContainer {
|
||||||
InfraContainer::Matrix => "hive-matrix",
|
InfraContainer::Matrix => "hive-matrix",
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Where this target lives on the host.
|
|
||||||
#[must_use]
|
|
||||||
pub fn target(self) -> InfraTarget {
|
|
||||||
match self {
|
|
||||||
InfraContainer::Gateway => InfraTarget::HostUnit("nginx.service"),
|
|
||||||
other => InfraTarget::Container(other.name()),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The systemd unit a lifecycle verb acts on.
|
|
||||||
#[must_use]
|
|
||||||
pub fn service_unit(self) -> String {
|
|
||||||
match self.target() {
|
|
||||||
InfraTarget::Container(name) => format!("container@{name}.service"),
|
|
||||||
InfraTarget::HostUnit(unit) => unit.to_owned(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Whether an agent holding `infra_admin` may restart this target.
|
|
||||||
///
|
|
||||||
/// The gateway is excluded by operator ruling: nginx now fronts every
|
|
||||||
/// hive service from the host, so an agent restarting it can take the
|
|
||||||
/// forge, dashboard and matrix down with it — including the path its
|
|
||||||
/// own PR would have to travel to fix it. The operator surface
|
|
||||||
/// (`hivectl`, dashboard) is unaffected.
|
|
||||||
#[must_use]
|
|
||||||
pub fn agent_restartable(self) -> bool {
|
|
||||||
!matches!(self, InfraContainer::Gateway)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
impl std::str::FromStr for InfraContainer {
|
impl std::str::FromStr for InfraContainer {
|
||||||
type Err = ();
|
type Err = ();
|
||||||
|
|
||||||
/// Parse an infra name (`hive-ci`, …) into a variant. Recognition
|
/// Parse a container name (`hive-ci`, …) into a variant. Used to decide
|
||||||
/// only — it says the name denotes a hive service, *not* that the
|
/// whether an MCP `restart(<name>)` target is a controllable infra
|
||||||
/// caller may act on it. The agent restart path additionally checks
|
/// container. `Err(())` for anything that isn't one.
|
||||||
/// [`agent_restartable`](InfraContainer::agent_restartable).
|
|
||||||
/// `Err(())` for anything that isn't one.
|
|
||||||
fn from_str(s: &str) -> Result<Self, ()> {
|
fn from_str(s: &str) -> Result<Self, ()> {
|
||||||
Self::ALL.into_iter().find(|c| c.name() == s).ok_or(())
|
Self::ALL.into_iter().find(|c| c.unit_name() == s).ok_or(())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -424,21 +364,15 @@ pub enum PrivRequest {
|
||||||
/// Run `systemctl daemon-reload`.
|
/// Run `systemctl daemon-reload`.
|
||||||
DaemonReload,
|
DaemonReload,
|
||||||
|
|
||||||
/// Synchronise the host's nginx unit after an `agents.conf` write.
|
/// Synchronise the nginx unit inside the `hive-gateway` container.
|
||||||
///
|
///
|
||||||
/// hive-priv queries `ActiveState` and dispatches:
|
/// hive-priv queries `ActiveState` and dispatches:
|
||||||
/// - `active` → `systemctl reload nginx` (SIGHUP, zero-downtime)
|
/// - `active` → `systemctl reload nginx` (SIGHUP, zero-downtime)
|
||||||
/// - `failed` → `systemctl reset-failed nginx` + `systemctl start nginx`
|
/// - `failed` → `systemctl reset-failed nginx` + `systemctl start nginx`
|
||||||
/// - otherwise → `systemctl start nginx`
|
/// - otherwise → `systemctl start nginx`
|
||||||
///
|
///
|
||||||
/// Requires root because hive-c0re runs as the unprivileged
|
/// Requires root: `--machine=hive-gateway` enters the container
|
||||||
/// `hive-core` user and cannot act on a system unit.
|
/// namespace via the machine bus (forbidden for unprivileged users).
|
||||||
///
|
|
||||||
/// ⚠️ The unit name is **not** a parameter and must stay that way.
|
|
||||||
/// nginx is a host unit, so nothing else narrows what this verb can
|
|
||||||
/// touch: hard-coding `nginx` is the entire containment. A caller
|
|
||||||
/// cannot name the unit, so this verb cannot be steered at any other
|
|
||||||
/// service.
|
|
||||||
ReloadGatewayNginx,
|
ReloadGatewayNginx,
|
||||||
|
|
||||||
// --- Forge admin CLI ---
|
// --- Forge admin CLI ---
|
||||||
|
|
@ -890,14 +824,13 @@ pub enum PrivEvent {
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::{InfraContainer, InfraTarget, SIBLING_CONTAINERS};
|
use super::{InfraContainer, SIBLING_CONTAINERS};
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn infra_control_allowlist_excludes_c0re_includes_matrix() {
|
fn infra_control_allowlist_excludes_c0re_includes_matrix() {
|
||||||
// SIBLING_CONTAINERS is the authoritative allowlist for the
|
// SIBLING_CONTAINERS is the authoritative allowlist for infra
|
||||||
// requests that name a container as a string. hive-c0re must NEVER
|
// lifecycle ops. hive-c0re must NEVER be in it — stopping the daemon
|
||||||
// be in it — stopping the daemon would sever the socket the request
|
// would sever the socket the request arrived on.
|
||||||
// arrived on.
|
|
||||||
assert!(!SIBLING_CONTAINERS.contains(&"hive-c0re"));
|
assert!(!SIBLING_CONTAINERS.contains(&"hive-c0re"));
|
||||||
// hive-matrix IS controllable (operator can stop/start/restart it).
|
// hive-matrix IS controllable (operator can stop/start/restart it).
|
||||||
assert!(SIBLING_CONTAINERS.contains(&"hive-matrix"));
|
assert!(SIBLING_CONTAINERS.contains(&"hive-matrix"));
|
||||||
|
|
@ -905,62 +838,26 @@ mod tests {
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn infra_container_enum_matches_sibling_containers() {
|
fn infra_container_enum_matches_sibling_containers() {
|
||||||
// The two lists are the same set *minus the things that aren't
|
// The InfraContainer enum (the control-path allowlist) and the
|
||||||
// containers*: every `Container` variant must appear in
|
// SIBLING_CONTAINERS slice (the general container-name validator)
|
||||||
// SIBLING_CONTAINERS and vice versa, so a `-M` journal read or a
|
// must list exactly the same four containers — they're separate
|
||||||
// `nixos-container` verb can never be pointed at a name that has no
|
// surfaces for the same set, so keep them in lockstep.
|
||||||
// machine behind it.
|
let mut from_enum: Vec<&str> = InfraContainer::ALL.iter().map(|c| c.unit_name()).collect();
|
||||||
let mut from_enum: Vec<&str> = InfraContainer::ALL
|
|
||||||
.iter()
|
|
||||||
.filter_map(|c| match c.target() {
|
|
||||||
InfraTarget::Container(name) => Some(name),
|
|
||||||
InfraTarget::HostUnit(_) => None,
|
|
||||||
})
|
|
||||||
.collect();
|
|
||||||
from_enum.sort_unstable();
|
from_enum.sort_unstable();
|
||||||
let mut from_slice: Vec<&str> = SIBLING_CONTAINERS.to_vec();
|
let mut from_slice: Vec<&str> = SIBLING_CONTAINERS.to_vec();
|
||||||
from_slice.sort_unstable();
|
from_slice.sort_unstable();
|
||||||
assert_eq!(from_enum, from_slice);
|
assert_eq!(from_enum, from_slice);
|
||||||
// The gateway is the one that is not: a host unit, and absent from
|
|
||||||
// the container-name allowlist.
|
|
||||||
assert_eq!(
|
|
||||||
InfraContainer::Gateway.target(),
|
|
||||||
InfraTarget::HostUnit("nginx.service")
|
|
||||||
);
|
|
||||||
assert!(!SIBLING_CONTAINERS.contains(&"hive-gateway"));
|
|
||||||
// hive-c0re has no variant — unrepresentable, can't be controlled.
|
// hive-c0re has no variant — unrepresentable, can't be controlled.
|
||||||
assert!("hive-c0re".parse::<InfraContainer>().is_err());
|
assert!("hive-c0re".parse::<InfraContainer>().is_err());
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn infra_container_name_round_trips() {
|
fn infra_container_name_round_trips() {
|
||||||
// `name` is the single source of truth for the operator-facing form
|
// `unit_name` is the single source of truth for the wire form (the
|
||||||
// (FromStr keys off it), so a name→variant→name round-trip proves
|
// serde impls + FromStr all key off it), so a name→variant→name
|
||||||
// the mapping is consistent in both directions. It holds for the
|
// round-trip proves the mapping is consistent in both directions.
|
||||||
// gateway too: the name is still recognised, it's the *permission*
|
|
||||||
// that differs (see below), not the parse.
|
|
||||||
for c in InfraContainer::ALL {
|
for c in InfraContainer::ALL {
|
||||||
assert_eq!(c.name().parse::<InfraContainer>(), Ok(c));
|
assert_eq!(c.unit_name().parse::<InfraContainer>(), Ok(c));
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn service_unit_wraps_containers_but_not_host_units() {
|
|
||||||
assert_eq!(
|
|
||||||
InfraContainer::Ci.service_unit(),
|
|
||||||
"container@hive-ci.service"
|
|
||||||
);
|
|
||||||
assert_eq!(InfraContainer::Gateway.service_unit(), "nginx.service");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn only_the_gateway_is_off_limits_to_agents() {
|
|
||||||
// Recognising a name and being allowed to restart it are separate
|
|
||||||
// questions — the gateway parses fine and is still refused.
|
|
||||||
assert!("hive-gateway".parse::<InfraContainer>().is_ok());
|
|
||||||
assert!(!InfraContainer::Gateway.agent_restartable());
|
|
||||||
for c in InfraContainer::ALL {
|
|
||||||
assert_eq!(c.agent_restartable(), c != InfraContainer::Gateway, "{c:?}");
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -888,23 +888,18 @@ async fn register_ci_runner(token: &str) -> Result<(String, String)> {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// `ControlInfraContainer` — start/stop/restart a hive infrastructure
|
/// `ControlInfraContainer` — start/stop/restart a hive infrastructure
|
||||||
/// service via `systemctl <verb> <unit>`. The [`InfraContainer`] enum is
|
/// container via `systemctl <verb> container@<container>.service`. The
|
||||||
/// the allowlist: serde already rejected any unknown / unsafe name
|
/// [`InfraContainer`] enum is the allowlist: serde already rejected any
|
||||||
/// (hive-c0re has no variant, so a stop can't sever the daemon socket) at
|
/// unknown / unsafe name (hive-c0re has no variant, so a stop can't sever
|
||||||
/// deserialisation, so no root-side `.contains()` check is needed here.
|
/// the daemon socket) at deserialisation, so no root-side `.contains()`
|
||||||
/// Serves both the hive-wide `hivectl stop`/`start` flow and an
|
/// check is needed here. Serves both the hive-wide `hivectl stop`/`start`
|
||||||
/// `infra_admin` agent's `restart` (action = Restart).
|
/// flow and an `infra_admin` agent's `restart` (action = Restart).
|
||||||
///
|
|
||||||
/// ⚠️ The unit is derived from the variant, never sent by the caller —
|
|
||||||
/// which is what keeps this from being a general `systemctl` pass-through.
|
|
||||||
/// It is not always `container@<name>.service`: the gateway resolves to the
|
|
||||||
/// host's `nginx.service`.
|
|
||||||
async fn control_infra_container(
|
async fn control_infra_container(
|
||||||
container: InfraContainer,
|
container: InfraContainer,
|
||||||
action: InfraAction,
|
action: InfraAction,
|
||||||
) -> Result<(String, String)> {
|
) -> Result<(String, String)> {
|
||||||
let verb = action.systemctl_verb();
|
let verb = action.systemctl_verb();
|
||||||
let unit = container.service_unit();
|
let unit = format!("container@{}.service", container.unit_name());
|
||||||
let out = Command::new("systemctl")
|
let out = Command::new("systemctl")
|
||||||
.args([verb, &unit])
|
.args([verb, &unit])
|
||||||
.output()
|
.output()
|
||||||
|
|
@ -2178,31 +2173,34 @@ async fn read_container_journal(container: &str, query: &JournalQuery) -> Result
|
||||||
Ok((stdout, stderr))
|
Ok((stdout, stderr))
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Synchronise the host's nginx unit after an `agents.conf` write.
|
/// Synchronise the nginx unit inside the `hive-gateway` container.
|
||||||
///
|
///
|
||||||
/// Queries `ActiveState` and dispatches:
|
/// Queries `ActiveState` via `systemctl --machine=hive-gateway` (requires
|
||||||
|
/// root — machine-bus transport enters the container namespace), then
|
||||||
|
/// dispatches:
|
||||||
/// - `active` → `systemctl reload nginx` (SIGHUP, zero-downtime)
|
/// - `active` → `systemctl reload nginx` (SIGHUP, zero-downtime)
|
||||||
/// - `failed` → `systemctl reset-failed nginx` + `systemctl start nginx`
|
/// - `failed` → `systemctl reset-failed nginx` + `systemctl start nginx`
|
||||||
/// - otherwise → `systemctl start nginx`
|
/// - otherwise → `systemctl start nginx`
|
||||||
///
|
///
|
||||||
/// ⚠️ `nginx` is hard-coded on purpose — see `PrivRequest::ReloadGatewayNginx`.
|
|
||||||
/// The unit name is the scope of this verb: nginx is a host unit, so no
|
|
||||||
/// namespace bounds it and the literal is the only thing standing between
|
|
||||||
/// "reload the gateway" and "reload anything".
|
|
||||||
///
|
|
||||||
/// Returns `(String::new(), String::new())` on success so it fits the
|
/// Returns `(String::new(), String::new())` on success so it fits the
|
||||||
/// `exec` return type directly.
|
/// `exec` return type directly.
|
||||||
async fn sync_gateway_nginx() -> Result<(String, String)> {
|
async fn sync_gateway_nginx() -> Result<(String, String)> {
|
||||||
let state_out = Command::new("systemctl")
|
let state_out = Command::new("systemctl")
|
||||||
.args(["show", "--property=ActiveState", "--value", "nginx"])
|
.args([
|
||||||
|
"--machine=hive-gateway",
|
||||||
|
"show",
|
||||||
|
"--property=ActiveState",
|
||||||
|
"--value",
|
||||||
|
"nginx",
|
||||||
|
])
|
||||||
.output()
|
.output()
|
||||||
.await
|
.await
|
||||||
.context("query gateway nginx ActiveState")?;
|
.context("query nginx ActiveState in hive-gateway")?;
|
||||||
if !state_out.status.success() {
|
if !state_out.status.success() {
|
||||||
tracing::warn!(
|
tracing::warn!(
|
||||||
exit_code = ?state_out.status.code(),
|
exit_code = ?state_out.status.code(),
|
||||||
stderr = %String::from_utf8_lossy(&state_out.stderr).trim(),
|
stderr = %String::from_utf8_lossy(&state_out.stderr).trim(),
|
||||||
"systemctl show ActiveState exited non-zero — gateway nginx may be down"
|
"systemctl show ActiveState exited non-zero — gateway container may be down"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
let state = String::from_utf8_lossy(&state_out.stdout).trim().to_owned();
|
let state = String::from_utf8_lossy(&state_out.stdout).trim().to_owned();
|
||||||
|
|
@ -2211,10 +2209,10 @@ async fn sync_gateway_nginx() -> Result<(String, String)> {
|
||||||
match state.as_str() {
|
match state.as_str() {
|
||||||
"active" => {
|
"active" => {
|
||||||
let out = Command::new("systemctl")
|
let out = Command::new("systemctl")
|
||||||
.args(["reload", "nginx"])
|
.args(["--machine=hive-gateway", "reload", "nginx"])
|
||||||
.output()
|
.output()
|
||||||
.await
|
.await
|
||||||
.context("reload gateway nginx")?;
|
.context("reload nginx in hive-gateway")?;
|
||||||
if !out.status.success() {
|
if !out.status.success() {
|
||||||
bail!(
|
bail!(
|
||||||
"gateway nginx reload failed ({}): {}",
|
"gateway nginx reload failed ({}): {}",
|
||||||
|
|
@ -2226,14 +2224,14 @@ async fn sync_gateway_nginx() -> Result<(String, String)> {
|
||||||
"failed" => {
|
"failed" => {
|
||||||
// Clear start-limit so the next start can proceed.
|
// Clear start-limit so the next start can proceed.
|
||||||
let _ = Command::new("systemctl")
|
let _ = Command::new("systemctl")
|
||||||
.args(["reset-failed", "nginx"])
|
.args(["--machine=hive-gateway", "reset-failed", "nginx"])
|
||||||
.status()
|
.status()
|
||||||
.await;
|
.await;
|
||||||
let out = Command::new("systemctl")
|
let out = Command::new("systemctl")
|
||||||
.args(["start", "nginx"])
|
.args(["--machine=hive-gateway", "start", "nginx"])
|
||||||
.output()
|
.output()
|
||||||
.await
|
.await
|
||||||
.context("start gateway nginx after reset-failed")?;
|
.context("start nginx after reset-failed in hive-gateway")?;
|
||||||
if !out.status.success() {
|
if !out.status.success() {
|
||||||
bail!(
|
bail!(
|
||||||
"gateway nginx start (after reset-failed) failed ({}): {}",
|
"gateway nginx start (after reset-failed) failed ({}): {}",
|
||||||
|
|
@ -2245,10 +2243,10 @@ async fn sync_gateway_nginx() -> Result<(String, String)> {
|
||||||
_ => {
|
_ => {
|
||||||
// inactive, activating, deactivating, unknown — just start.
|
// inactive, activating, deactivating, unknown — just start.
|
||||||
let out = Command::new("systemctl")
|
let out = Command::new("systemctl")
|
||||||
.args(["start", "nginx"])
|
.args(["--machine=hive-gateway", "start", "nginx"])
|
||||||
.output()
|
.output()
|
||||||
.await
|
.await
|
||||||
.context("start gateway nginx")?;
|
.context("start nginx in hive-gateway")?;
|
||||||
if !out.status.success() {
|
if !out.status.success() {
|
||||||
bail!(
|
bail!(
|
||||||
"gateway nginx start failed (state={state:?}) ({}): {}",
|
"gateway nginx start failed (state={state:?}) ({}): {}",
|
||||||
|
|
|
||||||
|
|
@ -224,17 +224,11 @@ pub enum Capability {
|
||||||
/// manager socket for swarm-wide scans.
|
/// manager socket for swarm-wide scans.
|
||||||
QueryAgentState,
|
QueryAgentState,
|
||||||
/// Agent can restart hive infrastructure containers (hive-ci,
|
/// Agent can restart hive infrastructure containers (hive-ci,
|
||||||
/// hive-forge, hive-matrix) via the `restart` MCP tool. hive-c0re
|
/// hive-gateway, hive-forge) via the `restart` MCP tool. hive-c0re
|
||||||
/// checks this capability before routing the restart through
|
/// checks this capability before routing the restart through
|
||||||
/// hive-priv; the concrete service allowlist lives root-side in
|
/// hive-priv; the concrete service allowlist lives root-side in
|
||||||
/// hive-priv. Deliberately generic ("infra admin") so future
|
/// hive-priv. Deliberately generic ("infra admin") so future
|
||||||
/// privileged infra ops can hang off the same grant.
|
/// privileged infra ops can hang off the same grant.
|
||||||
///
|
|
||||||
/// ⚠️ The gateway is **not** in reach of this capability, by operator
|
|
||||||
/// ruling — it is the host's nginx and fronts the forge, dashboard and
|
|
||||||
/// matrix, so an agent restarting it can cut the path its own fix
|
|
||||||
/// travels. That refusal is a property of the target, not of the
|
|
||||||
/// grant: no capability re-opens it.
|
|
||||||
InfraAdmin,
|
InfraAdmin,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -271,7 +265,7 @@ impl Capability {
|
||||||
"query non-child agents' loose ends and reminder state via get_loose_ends"
|
"query non-child agents' loose ends and reminder state via get_loose_ends"
|
||||||
}
|
}
|
||||||
Self::InfraAdmin => {
|
Self::InfraAdmin => {
|
||||||
"restart hive infrastructure containers (hive-ci, hive-forge, hive-matrix; not the gateway) via the restart tool"
|
"restart hive infrastructure containers (hive-ci, hive-gateway, hive-forge) via the restart tool"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -225,7 +225,7 @@ pub struct ScopeArgs {
|
||||||
/// The forge container (`hive-forge`).
|
/// The forge container (`hive-forge`).
|
||||||
#[arg(long)]
|
#[arg(long)]
|
||||||
forge: bool,
|
forge: bool,
|
||||||
/// The gateway (`hive-gateway`) — nginx on the host, not a container.
|
/// The gateway container (`hive-gateway`).
|
||||||
#[arg(long)]
|
#[arg(long)]
|
||||||
gateway: bool,
|
gateway: bool,
|
||||||
/// The matrix container (`hive-matrix`).
|
/// The matrix container (`hive-matrix`).
|
||||||
|
|
|
||||||
|
|
@ -1,9 +1,7 @@
|
||||||
# Single nginx in front of every hyperhive web surface — dashboard,
|
# Single nginx in front of every hyperhive web surface — dashboard,
|
||||||
# per-agent UIs (sub-path), forge + matrix (sub-domain), .well-known
|
# per-agent UIs (sub-path), forge + matrix (sub-domain), .well-known
|
||||||
# delegations — plus the hive-internal dnsmasq resolver. Both run on the
|
# delegations — plus the hive-internal dnsmasq resolver, co-located in
|
||||||
# HOST, next to hive-c0re: nginx binds the host's :80/:443 and dnsmasq
|
# the same `hive-gateway` container (shared host netns, state-free).
|
||||||
# answers on the hive bridge, so neither can be confined to a network
|
|
||||||
# namespace of its own.
|
|
||||||
# Full vhost map + discovery flow + design rationale in
|
# Full vhost map + discovery flow + design rationale in
|
||||||
# `docs/gateway.md`. Layout: ./options.nix (option declarations),
|
# `docs/gateway.md`. Layout: ./options.nix (option declarations),
|
||||||
# ./vhosts.nix (the nginx virtual-host tree), ./error-pages.nix
|
# ./vhosts.nix (the nginx virtual-host tree), ./error-pages.nix
|
||||||
|
|
@ -44,43 +42,6 @@ let
|
||||||
# so the gateway always terminates TLS.
|
# so the gateway always terminates TLS.
|
||||||
# `cfg.useSelfSigned` (options.nix) is the derived single source of truth.
|
# `cfg.useSelfSigned` (options.nix) is the derived single source of truth.
|
||||||
useSelfSigned = cfg.useSelfSigned;
|
useSelfSigned = cfg.useSelfSigned;
|
||||||
|
|
||||||
# nginx's own state dir. Kept at the historical `/var/lib/hive-gateway`
|
|
||||||
# path rather than renamed with the move: it holds the imported leaf
|
|
||||||
# across reboots, and renaming it would strand every existing hive's
|
|
||||||
# certs for no gain.
|
|
||||||
tlsDir = "/var/lib/hive-gateway/tls";
|
|
||||||
# TLS cert + key paths.
|
|
||||||
# - self-signed (default): the hive-CA-signed leaf, imported into the
|
|
||||||
# state dir by `hive-gateway-self-signed-cert` below.
|
|
||||||
# - tls.certDir set: the operator's own cert dir, read directly.
|
|
||||||
tlsCert =
|
|
||||||
if cfg.tls.certDir != null then "${cfg.tls.certDir}/${cfg.tls.certName}" else "${tlsDir}/cert.pem";
|
|
||||||
tlsKey =
|
|
||||||
if cfg.tls.certDir != null then "${cfg.tls.certDir}/${cfg.tls.keyName}" else "${tlsDir}/key.pem";
|
|
||||||
# The swarm-services pair, used only by the vhosts whose names this
|
|
||||||
# hive's CA cannot sign. Self-signed mode only: with an operator cert
|
|
||||||
# or ACME the operator owns every name and there is no second issuer.
|
|
||||||
svcCert = "${tlsDir}/swarm-services.pem";
|
|
||||||
svcKey = "${tlsDir}/swarm-services-key.pem";
|
|
||||||
|
|
||||||
nginxTree = import ./vhosts.nix {
|
|
||||||
inherit
|
|
||||||
lib
|
|
||||||
cfg
|
|
||||||
forgeCfg
|
|
||||||
matrixCfg
|
|
||||||
hyperhiveDomain
|
|
||||||
dashboardDist
|
|
||||||
swaggerUiTheme
|
|
||||||
tlsCert
|
|
||||||
tlsKey
|
|
||||||
svcCert
|
|
||||||
svcKey
|
|
||||||
swarmServiceDomains
|
|
||||||
;
|
|
||||||
errorPages = import ./error-pages.nix { inherit pkgs; };
|
|
||||||
};
|
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
imports = [ ./options.nix ];
|
imports = [ ./options.nix ];
|
||||||
|
|
@ -104,10 +65,10 @@ in
|
||||||
}
|
}
|
||||||
];
|
];
|
||||||
|
|
||||||
# Ensure the gateway state dirs exist at host boot, before anything
|
# Ensure bind-mount sources exist at host boot before the gateway
|
||||||
# reads or writes them: these rules cover the fresh-boot window
|
# container's first start. nspawn would auto-create missing dirs;
|
||||||
# before c0re has run, and pin owner + mode rather than leaving it
|
# tmpfiles rules make the intent explicit and cover the fresh-boot
|
||||||
# to whoever creates the path first.
|
# window before c0re has run.
|
||||||
#
|
#
|
||||||
# /run/hive-agent — per-agent UDS socket dir, written by c0re's
|
# /run/hive-agent — per-agent UDS socket dir, written by c0re's
|
||||||
# set_nspawn_flags when agents start. Owned by `hive-core` (the
|
# set_nspawn_flags when agents start. Owned by `hive-core` (the
|
||||||
|
|
@ -134,166 +95,349 @@ in
|
||||||
"f /var/lib/hyperhive/gateway/gateway.htpasswd 0644 root root - -"
|
"f /var/lib/hyperhive/gateway/gateway.htpasswd 0644 root root - -"
|
||||||
];
|
];
|
||||||
|
|
||||||
# ⚠️ REMOVED WITH THE CONTAINER, and each one was a workaround for the
|
# Keep the gateway's `/etc/resolv.conf` in step with the host's.
|
||||||
# boundary rather than a thing nginx or dnsmasq needed:
|
|
||||||
#
|
#
|
||||||
# - `privateNetwork = false` — the container already shared the host
|
# nixos-container does `cp --remove-destination /etc/resolv.conf
|
||||||
# netns, which is why nginx bound host ports and `localhost`
|
# "$root/etc/resolv.conf"` in its start script, and nspawn's
|
||||||
# upstreams reached hive-c0re. On the host that is simply true.
|
# `--resolv-conf=auto` copies rather than binds for a writable,
|
||||||
# - `additionalCapabilities = [ "CAP_NET_ADMIN" ]` — dnsmasq refuses
|
# host-netns container like this one. Both are one-shot: systemd-nspawn(1)
|
||||||
# to start with a `dhcp-range` unless it holds NET_ADMIN, and
|
# says outright that "no further propagation of configuration is
|
||||||
# nspawn's bounding set dropped it for a host-netns container.
|
# generally done after the one-time early initialization (this is
|
||||||
# Host root has it.
|
# because the file is usually updated through copying and renaming)".
|
||||||
# - `networking.firewall.enable = false` — a container sharing the
|
|
||||||
# host netns would run *its* firewall.service against the HOST
|
|
||||||
# ruleset, flushing nixos-fw and deleting the nixos-nat-* chains on
|
|
||||||
# every boot. With one machine there is one firewall (below).
|
|
||||||
# - `networking.resolvconf.enable = false` + the `hive-gateway-resolv`
|
|
||||||
# path/service pair — the container's /etc/resolv.conf was a
|
|
||||||
# one-shot copy frozen at start, so a host network change left
|
|
||||||
# dnsmasq forwarding to a resolver that was gone. The whole
|
|
||||||
# watch-copy-reload machine existed to bridge two files. There is
|
|
||||||
# now one.
|
|
||||||
# - three bind mounts — /run/hive-agent, /run/hive-state, and either
|
|
||||||
# /run/hive-tls (operator cert) or /run/hive-ca (self-signed);
|
|
||||||
# those last two are mutually exclusive mkIfs, so it was never
|
|
||||||
# four. All plain host paths now.
|
|
||||||
#
|
#
|
||||||
# See `docs/network.md::Resolver behaviour` for the resolver history.
|
# So the gateway's copy is frozen at container start. dnsmasq has no
|
||||||
# ACME (Let's Encrypt) integration. nginx vhosts set
|
# explicit upstream (see ./dnsmasq.nix) and follows that file, which
|
||||||
# `enableACME = true` via the vhost builder; this provides the
|
# means a host network change — new router, new DHCP lease, laptop
|
||||||
# shared ACME config (acceptTerms + email).
|
# moving between networks — leaves dnsmasq forwarding to a resolver
|
||||||
security.acme = lib.mkIf cfg.tls.acme.enable {
|
# that is gone, and every non-hive lookup from every agent hangs. The
|
||||||
acceptTerms = true;
|
# agents' own resolvers point at the static bridge IP and never go
|
||||||
defaults.email = cfg.tls.acme.email;
|
# stale, which is exactly why the symptom presents as "the gateway
|
||||||
|
# needs a kick".
|
||||||
|
#
|
||||||
|
# Hence: watch on the host, push into the container, reload dnsmasq.
|
||||||
|
# `reload` is `kill -HUP $MAINPID` (the upstream dnsmasq unit's
|
||||||
|
# ExecReload), so dnsmasq re-reads its upstream list and drops its
|
||||||
|
# cache without severing anything — nginx never notices. Why this has
|
||||||
|
# to run host-side, and why a file bind-mount is worse than the copy:
|
||||||
|
# `docs/network.md::Resolver behaviour`.
|
||||||
|
systemd.paths.hive-gateway-resolv = {
|
||||||
|
description = "Watch the host's /etc/resolv.conf for the hive-gateway container";
|
||||||
|
wantedBy = [ "multi-user.target" ];
|
||||||
|
# Arm the watch before anything configures the network, so the very
|
||||||
|
# first DHCP-driven resolv.conf write of the boot is caught too —
|
||||||
|
# that's the "gateway came up while DHCP was still settling" case.
|
||||||
|
# Inert if nothing pulls network-pre.target into the transaction.
|
||||||
|
before = [ "network-pre.target" ];
|
||||||
|
pathConfig = {
|
||||||
|
# PathChanged also watches the parent directory, so openresolv's
|
||||||
|
# atomic rename-over lands as IN_MOVED_TO on /etc and triggers —
|
||||||
|
# a watch on the inode alone would die with the replaced file.
|
||||||
|
PathChanged = "/etc/resolv.conf";
|
||||||
|
Unit = "hive-gateway-resolv.service";
|
||||||
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
# Import the hive-CA leaf into nginx's state dir before nginx starts.
|
systemd.services.hive-gateway-resolv = {
|
||||||
#
|
description = "Sync the host's resolvers into hive-gateway and reload dnsmasq";
|
||||||
# 🚨 THIS LOOKS LIKE A LEFTOVER OF THE CONTAINER AND IS NOT — do not
|
# Also run once per gateway start, to catch a host resolver change
|
||||||
# "simplify" it into pointing nginx at the CA dir. It does TWO jobs:
|
# that happened while the container was down. Deliberately NOT
|
||||||
#
|
# ordered after network-online.target: pulling that target in on
|
||||||
# (1) It re-modes the leaf. `hive-tls-ca` writes the key 0600
|
# every resolv.conf change risks blocking the sync behind a
|
||||||
# root:root; nginx's pre-start `nginx -t` runs as the nginx
|
# wait-online timeout on hosts where nothing else reaches it. The
|
||||||
# *user*, so a 0600 key fails the config test with
|
# boot race is closed by arming the path unit early instead.
|
||||||
# `BIO_new_file() … Permission denied` and blocks the unit —
|
wantedBy = [ "container@hive-gateway.service" ];
|
||||||
# hence the 0640 root:nginx copy below. That is a file-mode fact,
|
after = [ "container@hive-gateway.service" ];
|
||||||
# not a namespace one, and it did not go away with the container.
|
path = [
|
||||||
# (2) It guarantees that **every cert path the nginx config names
|
pkgs.systemd
|
||||||
# exists** — which is what the swarm-services fallback at the
|
pkgs.coreutils
|
||||||
# bottom of the script is for. nginx refuses to load a config
|
pkgs.gnugrep
|
||||||
# naming a missing cert file, so a leaf that never issues takes
|
];
|
||||||
# the whole gateway down rather than one vhost; that has already
|
|
||||||
# happened once and it took the forge, dashboard and matrix with
|
|
||||||
# it. Removing this unit re-creates it exactly.
|
|
||||||
#
|
|
||||||
# nginx
|
|
||||||
# `Requires=` this via `requiredBy`, so it refuses to start until
|
|
||||||
# the copy succeeds. ALWAYS runs (no ConditionPathExists) and is
|
|
||||||
# idempotent — necessary to reconcile broken state from prior
|
|
||||||
# failed boots (a 0700 dir from a stale UMask, a truncated copy
|
|
||||||
# from an interrupted oneshot, etc.). The leaf covers the bare
|
|
||||||
# hive domain plus `forge.`, `matrix.` and `*.${hyperhiveDomain}`
|
|
||||||
# so all sub-domains validate under the same cert + the hive CA.
|
|
||||||
# See `docs/gateway.md` ("Self-signed TLS").
|
|
||||||
systemd.services.hive-gateway-self-signed-cert = lib.mkIf useSelfSigned {
|
|
||||||
description = "Import host-generated TLS leaf for hive-gateway";
|
|
||||||
wantedBy = [ "multi-user.target" ];
|
|
||||||
before = [ "nginx.service" ];
|
|
||||||
requiredBy = [ "nginx.service" ];
|
|
||||||
serviceConfig = {
|
serviceConfig = {
|
||||||
Type = "oneshot";
|
Type = "oneshot";
|
||||||
RemainAfterExit = true;
|
SyslogIdentifier = "hive-gateway-resolv";
|
||||||
# Pin the journal identity (else it's the `script` store-path wrapper).
|
|
||||||
SyslogIdentifier = "hive-gateway-self-signed-cert";
|
|
||||||
};
|
};
|
||||||
path = [ pkgs.coreutils ];
|
|
||||||
script = ''
|
script = ''
|
||||||
set -eu
|
set -euo pipefail
|
||||||
mkdir -p ${tlsDir}
|
src=/etc/resolv.conf
|
||||||
# 0755 on BOTH the cert dir and its parent so the nginx
|
# Marker lives on tmpfs, so the first sync after every host boot
|
||||||
# user can traverse the full path. The parent
|
# always goes through rather than trusting a stale comparison.
|
||||||
# `/var/lib/hive-gateway` lands at 0700 by default (systemd
|
marker=/run/hive-gateway/resolv.synced
|
||||||
# StateDirectory / mkdir umask depending on which service
|
|
||||||
# created it first), which on its own blocks traversal.
|
|
||||||
# Re-applied every boot in case a prior run left a tighter
|
|
||||||
# mode behind.
|
|
||||||
chmod 0755 ${builtins.dirOf tlsDir}
|
|
||||||
chmod 0755 ${tlsDir}
|
|
||||||
# Copy the host leaf in. `install` writes atomically with the
|
|
||||||
# target mode; run as root (container root == host root,
|
|
||||||
# privateUsers=false) so the 0600 root:root host key is
|
|
||||||
# readable. Key ends up root:nginx 0640 so nginx-pre-start
|
|
||||||
# (which runs `nginx -t` as the nginx user, not root) can
|
|
||||||
# read it — a 0600 root:root key passes the master load but
|
|
||||||
# fails the pre-start config test with `BIO_new_file() …
|
|
||||||
# Permission denied`, blocking the unit. Cert is world-read.
|
|
||||||
install -m 0644 ${config.services.hyperhive.tls.stateDir}/gateway.pem ${tlsCert}
|
|
||||||
install -m 0640 -g nginx ${config.services.hyperhive.tls.stateDir}/gateway-key.pem ${tlsKey}
|
|
||||||
|
|
||||||
# The swarm-services leaf, when this host issues one. It is
|
# No nameserver line means either a mid-rewrite snapshot or a host
|
||||||
# a separate pair rather than more SANs on the one above
|
# with no DNS at all. In both cases the gateway's existing copy is
|
||||||
# because no hive CA can sign these names — each is
|
# the best information available — keep it and wait for the next
|
||||||
# constrained to its own hive's domain and the service
|
# event, instead of pushing a file that resolves nothing.
|
||||||
# names are siblings of it.
|
if ! grep -q '^[[:space:]]*nameserver[[:space:]]' "$src"; then
|
||||||
#
|
echo "host resolv.conf has no nameserver line — keeping the gateway's current copy"
|
||||||
# Absent is a normal state, not a failure: the leaf exists
|
exit 0
|
||||||
# only where the swarm CA is autoconfigured, and issuance
|
|
||||||
# can also fail on a host that wants one.
|
|
||||||
#
|
|
||||||
# ⚠️ When it is absent the HIVE leaf goes to this path
|
|
||||||
# anyway, and that fallback is load-bearing rather than
|
|
||||||
# tidy. nginx refuses to load a config naming a cert file
|
|
||||||
# that does not exist — `cannot load certificate … no such
|
|
||||||
# file` fails the pre-start test, so the vhost does not
|
|
||||||
# degrade, the ENTIRE gateway dies and takes the forge, the
|
|
||||||
# dashboard and matrix with it. Serving the hive leaf on a
|
|
||||||
# swarm-service name is a name mismatch: browsers warn,
|
|
||||||
# strict clients refuse, everything else keeps working, and
|
|
||||||
# the operator gets a bad cert instead of no hive.
|
|
||||||
#
|
|
||||||
# Measured, not theorised: this exact path took pr1ma's
|
|
||||||
# gateway down when the services sub-CA failed to issue.
|
|
||||||
if [ -s ${config.services.hyperhive.tls.stateDir}/swarm-services.pem ]; then
|
|
||||||
install -m 0644 ${config.services.hyperhive.tls.stateDir}/swarm-services.pem ${svcCert}
|
|
||||||
install -m 0640 -g nginx ${config.services.hyperhive.tls.stateDir}/swarm-services-key.pem ${svcKey}
|
|
||||||
else
|
|
||||||
echo "no swarm-services leaf — serving the hive leaf on those names (mismatch, not an outage)" >&2
|
|
||||||
install -m 0644 ${config.services.hyperhive.tls.stateDir}/gateway.pem ${svcCert}
|
|
||||||
install -m 0640 -g nginx ${config.services.hyperhive.tls.stateDir}/gateway-key.pem ${svcKey}
|
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
if [ -e "$marker" ] && cmp -s "$src" "$marker"; then
|
||||||
|
echo "host resolvers unchanged since last sync — nothing to do"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# A stopped gateway needs no push: its next start copies the
|
||||||
|
# current host file itself.
|
||||||
|
if ! systemctl is-active --quiet container@hive-gateway.service; then
|
||||||
|
echo "hive-gateway not running — its next start copies the current file itself"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# `machinectl copy-to` writes through the container's own mount
|
||||||
|
# namespace, so this stays correct regardless of how the container
|
||||||
|
# assembles /etc (e.g. if system.etc.overlay is ever turned on) —
|
||||||
|
# unlike poking at the rootfs path from the host side.
|
||||||
|
machinectl copy-to hive-gateway "$src" /etc/resolv.conf --force
|
||||||
|
|
||||||
|
# Not fatal: dnsmasq polls resolv.conf for mtime changes on its own,
|
||||||
|
# so a failed reload costs a second of staleness, not correctness.
|
||||||
|
if ! systemctl -M hive-gateway reload dnsmasq.service; then
|
||||||
|
echo "dnsmasq reload failed (not up yet?) — copy is in place, its own poll will pick it up"
|
||||||
|
fi
|
||||||
|
|
||||||
|
mkdir -p "$(dirname "$marker")"
|
||||||
|
install -m 0644 "$src" "$marker"
|
||||||
|
echo "synced host resolvers into hive-gateway and reloaded dnsmasq"
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|
||||||
# nginx reload is triggered from the HOST side by hive-c0re
|
containers.hive-gateway = {
|
||||||
# after each agents.conf write, through hive-priv (c0re is
|
autoStart = true;
|
||||||
# unprivileged and cannot act on a system unit).
|
ephemeral = false;
|
||||||
#
|
# Share host netns — nginx then binds host-level ports directly,
|
||||||
# It stays an explicit trigger rather than a systemd path unit
|
# `localhost` upstream resolution reaches hive-c0re without any
|
||||||
# watching the file: the write and the reload belong in one causal
|
# port-forward dance, and the firewall config below is the only
|
||||||
# chain c0re can retry and report on (see RELOAD_PENDING), not two
|
# layer that matters.
|
||||||
# independent units racing on an inotify event.
|
privateNetwork = false;
|
||||||
|
# dnsmasq refuses to start once a dhcp-range is configured unless it
|
||||||
|
# holds CAP_NET_ADMIN (DNS-only mode doesn't need it). Private-network
|
||||||
|
# containers retain NET_ADMIN implicitly, but this container shares the
|
||||||
|
# host netns (above), so nspawn's default bounding set drops it — grant
|
||||||
|
# it explicitly. Note this is NET_ADMIN over the *host* netns; the
|
||||||
|
# gateway container is trusted infra (it already terminates TLS and
|
||||||
|
# fronts every vhost), so no new trust boundary is crossed.
|
||||||
|
additionalCapabilities = [ "CAP_NET_ADMIN" ];
|
||||||
|
# Bind-mount the per-agent socket dir so nginx inside the gateway
|
||||||
|
# container can `connect(2)` to the UDS upstreams.
|
||||||
|
# Read-only (we just connect; harness writes the socket inside
|
||||||
|
# the agent's own container). Host-side dir is pre-created by a
|
||||||
|
# tmpfiles rule so nspawn always finds a source at boot.
|
||||||
|
bindMounts."/run/hive-agent" = {
|
||||||
|
hostPath = "/run/hive-agent";
|
||||||
|
isReadOnly = true;
|
||||||
|
};
|
||||||
|
# Bind-mount ONLY the gateway-specific subdir of the hyperhive
|
||||||
|
# state dir. Scoped to /var/lib/hyperhive/gateway/ rather than
|
||||||
|
# the whole parent so the gateway container can't read forge
|
||||||
|
# tokens or other files that may live at the parent level.
|
||||||
|
# c0re writes agents.conf under this subdir and triggers an nginx
|
||||||
|
# reload from the host via systemd-run after each write.
|
||||||
|
# Pre-created by a tmpfiles rule.
|
||||||
|
bindMounts."/run/hive-state" = {
|
||||||
|
hostPath = "/var/lib/hyperhive/gateway";
|
||||||
|
isReadOnly = true;
|
||||||
|
};
|
||||||
|
# Operator-provided TLS cert dir (e.g. Let's Encrypt / ACME).
|
||||||
|
# Only mounted when `tls.certDir` is set; when it is, the self-signed
|
||||||
|
# floor is off (so the `/run/hive-ca` mount below is absent). nginx
|
||||||
|
# reads cert + key from `/run/hive-tls/<certName>` and `<keyName>`.
|
||||||
|
bindMounts."/run/hive-tls" = lib.mkIf (cfg.tls.certDir != null) {
|
||||||
|
hostPath = cfg.tls.certDir;
|
||||||
|
isReadOnly = true;
|
||||||
|
};
|
||||||
|
# Self-signed mode: the host `hive-tls-ca` service generates a hive
|
||||||
|
# CA + a leaf signed by it under `services.hyperhive.tls.stateDir`.
|
||||||
|
# Bind-mount that dir read-only so the in-container import service
|
||||||
|
# (below) can copy the leaf into nginx's state dir with the right
|
||||||
|
# owner/mode. Source files: `gateway.pem` + `gateway-key.pem`.
|
||||||
|
bindMounts."/run/hive-ca" = lib.mkIf useSelfSigned {
|
||||||
|
hostPath = config.services.hyperhive.tls.stateDir;
|
||||||
|
isReadOnly = true;
|
||||||
|
};
|
||||||
|
config =
|
||||||
|
{ pkgs, ... }:
|
||||||
|
let
|
||||||
|
tlsDir = "/var/lib/hive-gateway/tls";
|
||||||
|
# TLS cert + key paths inside the container.
|
||||||
|
# - self-signed (default): imported hive-CA-signed leaf in the
|
||||||
|
# persistent state dir.
|
||||||
|
# - tls.certDir set: operator-provided cert bind-mounted at /run/hive-tls.
|
||||||
|
tlsCert =
|
||||||
|
if cfg.tls.certDir != null then "/run/hive-tls/${cfg.tls.certName}" else "${tlsDir}/cert.pem";
|
||||||
|
tlsKey =
|
||||||
|
if cfg.tls.certDir != null then "/run/hive-tls/${cfg.tls.keyName}" else "${tlsDir}/key.pem";
|
||||||
|
# The swarm-services pair, used only by the vhosts whose names
|
||||||
|
# this hive's CA cannot sign. Self-signed mode only: with an
|
||||||
|
# operator cert or ACME the operator owns every name and there
|
||||||
|
# is no second issuer in the picture.
|
||||||
|
svcCert = "${tlsDir}/swarm-services.pem";
|
||||||
|
svcKey = "${tlsDir}/swarm-services-key.pem";
|
||||||
|
nginxTree = import ./vhosts.nix {
|
||||||
|
inherit
|
||||||
|
lib
|
||||||
|
cfg
|
||||||
|
forgeCfg
|
||||||
|
matrixCfg
|
||||||
|
hyperhiveDomain
|
||||||
|
dashboardDist
|
||||||
|
swaggerUiTheme
|
||||||
|
tlsCert
|
||||||
|
tlsKey
|
||||||
|
svcCert
|
||||||
|
svcKey
|
||||||
|
swarmServiceDomains
|
||||||
|
;
|
||||||
|
errorPages = import ./error-pages.nix { inherit pkgs; };
|
||||||
|
};
|
||||||
|
in
|
||||||
|
{
|
||||||
|
system.stateVersion = "26.05";
|
||||||
|
|
||||||
services.nginx = {
|
# This container shares the host netns, so its own
|
||||||
enable = true;
|
# firewall.service would run against the HOST ruleset: flush
|
||||||
recommendedProxySettings = true;
|
# the nixos-fw chains, rebuild them from this container's
|
||||||
recommendedTlsSettings = true;
|
# (empty) port list, and delete the host's nixos-nat-* chains
|
||||||
recommendedGzipSettings = true;
|
# — wiping the bridge DHCP/DNS holes and the agents' NAT on
|
||||||
recommendedOptimisation = true;
|
# every container boot. The host firewall owns all filtering;
|
||||||
inherit (nginxTree) appendHttpConfig virtualHosts;
|
# never run one in here.
|
||||||
};
|
networking.firewall.enable = false;
|
||||||
|
|
||||||
# dnsmasq is a host service alongside nginx, so it reads the host's
|
# Keep the host-copied /etc/resolv.conf intact. nixos-container
|
||||||
# /etc/resolv.conf directly and picks up network changes as they
|
# copies the host's file in at every container start, but
|
||||||
# happen — no copy to keep in sync.
|
# resolvconf's host-tracking mode then regenerates it — to an
|
||||||
services.dnsmasq = import ./dnsmasq.nix {
|
# empty file, since the host file doesn't cross the boundary
|
||||||
inherit
|
# after start (the same failure the matrix container hit).
|
||||||
lib
|
# With resolvconf off, nothing in here touches the copy: nginx's
|
||||||
networkCfg
|
# own lookups (ACME) and dnsmasq's follow-the-host upstream
|
||||||
forgeCfg
|
# default (see ./dnsmasq.nix) both read the host's resolvers.
|
||||||
matrixCfg
|
# Keeping it stale-free is the host's job — see the
|
||||||
hyperhiveDomain
|
# `hive-gateway-resolv` path unit above.
|
||||||
;
|
networking.resolvconf.enable = false;
|
||||||
|
|
||||||
|
# ACME (Let's Encrypt) integration. nginx vhosts set
|
||||||
|
# `enableACME = true` via the vhost builder; this provides the
|
||||||
|
# shared ACME config (acceptTerms + email). The gateway
|
||||||
|
# container has shared host netns so outbound ACME requests
|
||||||
|
# work without extra routing config. Certs are stored in the
|
||||||
|
# container's persistent state (`ephemeral = false`).
|
||||||
|
security.acme = lib.mkIf cfg.tls.acme.enable {
|
||||||
|
acceptTerms = true;
|
||||||
|
defaults.email = cfg.tls.acme.email;
|
||||||
|
};
|
||||||
|
|
||||||
|
# Import the host-generated leaf cert before nginx starts.
|
||||||
|
# The hive CA + gateway leaf are generated on the HOST by
|
||||||
|
# `hive-tls-ca` (see `hive-tls.nix`) and bind-mounted read-only
|
||||||
|
# at `/run/hive-ca`; this service copies the leaf into nginx's
|
||||||
|
# state dir with the owner/mode nginx needs, rather than reading
|
||||||
|
# the bind-mount directly (the host key is 0600 root:root and a
|
||||||
|
# cross-namespace bind-mount can't be relaxed in place). nginx
|
||||||
|
# `Requires=` this via `requiredBy`, so it refuses to start until
|
||||||
|
# the copy succeeds. ALWAYS runs (no ConditionPathExists) and is
|
||||||
|
# idempotent — necessary to reconcile broken state from prior
|
||||||
|
# failed boots (a 0700 dir from a stale UMask, a truncated copy
|
||||||
|
# from an interrupted oneshot, etc.). The leaf covers the bare
|
||||||
|
# hive domain plus `forge.`, `matrix.` and `*.${hyperhiveDomain}`
|
||||||
|
# so all sub-domains validate under the same cert + the hive CA.
|
||||||
|
# See `docs/gateway.md` ("Self-signed TLS").
|
||||||
|
systemd.services.hive-gateway-self-signed-cert = lib.mkIf useSelfSigned {
|
||||||
|
description = "Import host-generated TLS leaf for hive-gateway";
|
||||||
|
wantedBy = [ "multi-user.target" ];
|
||||||
|
before = [ "nginx.service" ];
|
||||||
|
requiredBy = [ "nginx.service" ];
|
||||||
|
serviceConfig = {
|
||||||
|
Type = "oneshot";
|
||||||
|
RemainAfterExit = true;
|
||||||
|
# Pin the journal identity (else it's the `script` store-path wrapper).
|
||||||
|
SyslogIdentifier = "hive-gateway-self-signed-cert";
|
||||||
|
};
|
||||||
|
path = [ pkgs.coreutils ];
|
||||||
|
script = ''
|
||||||
|
set -eu
|
||||||
|
mkdir -p ${tlsDir}
|
||||||
|
# 0755 on BOTH the cert dir and its parent so the nginx
|
||||||
|
# user can traverse the full path. The parent
|
||||||
|
# `/var/lib/hive-gateway` lands at 0700 by default (systemd
|
||||||
|
# StateDirectory / mkdir umask depending on which service
|
||||||
|
# created it first), which on its own blocks traversal.
|
||||||
|
# Re-applied every boot in case a prior run left a tighter
|
||||||
|
# mode behind.
|
||||||
|
chmod 0755 ${builtins.dirOf tlsDir}
|
||||||
|
chmod 0755 ${tlsDir}
|
||||||
|
# Copy the host leaf in. `install` writes atomically with the
|
||||||
|
# target mode; run as root (container root == host root,
|
||||||
|
# privateUsers=false) so the 0600 root:root host key is
|
||||||
|
# readable. Key ends up root:nginx 0640 so nginx-pre-start
|
||||||
|
# (which runs `nginx -t` as the nginx user, not root) can
|
||||||
|
# read it — a 0600 root:root key passes the master load but
|
||||||
|
# fails the pre-start config test with `BIO_new_file() …
|
||||||
|
# Permission denied`, blocking the unit. Cert is world-read.
|
||||||
|
install -m 0644 /run/hive-ca/gateway.pem ${tlsCert}
|
||||||
|
install -m 0640 -g nginx /run/hive-ca/gateway-key.pem ${tlsKey}
|
||||||
|
|
||||||
|
# The swarm-services leaf, when this host issues one. It is
|
||||||
|
# a separate pair rather than more SANs on the one above
|
||||||
|
# because no hive CA can sign these names — each is
|
||||||
|
# constrained to its own hive's domain and the service
|
||||||
|
# names are siblings of it.
|
||||||
|
#
|
||||||
|
# Absent is a normal state, not a failure: the leaf exists
|
||||||
|
# only where the swarm CA is autoconfigured, and issuance
|
||||||
|
# can also fail on a host that wants one.
|
||||||
|
#
|
||||||
|
# ⚠️ When it is absent the HIVE leaf goes to this path
|
||||||
|
# anyway, and that fallback is load-bearing rather than
|
||||||
|
# tidy. nginx refuses to load a config naming a cert file
|
||||||
|
# that does not exist — `cannot load certificate … no such
|
||||||
|
# file` fails the pre-start test, so the vhost does not
|
||||||
|
# degrade, the ENTIRE gateway dies and takes the forge, the
|
||||||
|
# dashboard and matrix with it. Serving the hive leaf on a
|
||||||
|
# swarm-service name is a name mismatch: browsers warn,
|
||||||
|
# strict clients refuse, everything else keeps working, and
|
||||||
|
# the operator gets a bad cert instead of no hive.
|
||||||
|
#
|
||||||
|
# Measured, not theorised: this exact path took pr1ma's
|
||||||
|
# gateway down when the services sub-CA failed to issue.
|
||||||
|
if [ -s /run/hive-ca/swarm-services.pem ]; then
|
||||||
|
install -m 0644 /run/hive-ca/swarm-services.pem ${svcCert}
|
||||||
|
install -m 0640 -g nginx /run/hive-ca/swarm-services-key.pem ${svcKey}
|
||||||
|
else
|
||||||
|
echo "no swarm-services leaf — serving the hive leaf on those names (mismatch, not an outage)" >&2
|
||||||
|
install -m 0644 /run/hive-ca/gateway.pem ${svcCert}
|
||||||
|
install -m 0640 -g nginx /run/hive-ca/gateway-key.pem ${svcKey}
|
||||||
|
fi
|
||||||
|
'';
|
||||||
|
};
|
||||||
|
|
||||||
|
# nginx reload is triggered from the HOST side by hive-c0re
|
||||||
|
# via `systemctl -M hive-gateway reload nginx` after each
|
||||||
|
# agents.conf write — letting systemd resolve the nginx binary
|
||||||
|
# path avoids exit-203 EXEC failures. A path unit watching the
|
||||||
|
# bind-mounted file inside the container does not work: an
|
||||||
|
# IN_MOVED_TO from an atomic rename on the host does not
|
||||||
|
# propagate across the nspawn mount-namespace boundary. The
|
||||||
|
# host-side trigger is the correct approach.
|
||||||
|
|
||||||
|
services.nginx = {
|
||||||
|
enable = true;
|
||||||
|
recommendedProxySettings = true;
|
||||||
|
recommendedTlsSettings = true;
|
||||||
|
recommendedGzipSettings = true;
|
||||||
|
recommendedOptimisation = true;
|
||||||
|
inherit (nginxTree) appendHttpConfig virtualHosts;
|
||||||
|
};
|
||||||
|
|
||||||
|
services.dnsmasq = import ./dnsmasq.nix {
|
||||||
|
inherit
|
||||||
|
lib
|
||||||
|
networkCfg
|
||||||
|
forgeCfg
|
||||||
|
matrixCfg
|
||||||
|
hyperhiveDomain
|
||||||
|
;
|
||||||
|
};
|
||||||
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
networking.firewall = lib.mkIf cfg.openFirewall {
|
networking.firewall = lib.mkIf cfg.openFirewall {
|
||||||
|
|
|
||||||
|
|
@ -134,9 +134,9 @@ in
|
||||||
and uses this cert, overriding the self-signed default — the
|
and uses this cert, overriding the self-signed default — the
|
||||||
auto-generated hive-CA-signed leaf is skipped entirely.
|
auto-generated hive-CA-signed leaf is skipped entirely.
|
||||||
|
|
||||||
nginx reads `<certDir>/<tls.certName>` and
|
The directory is bind-mounted read-only into the gateway
|
||||||
`<certDir>/<tls.keyName>` directly — it runs on the host, so
|
container at `/run/hive-tls/`. nginx reads
|
||||||
the directory needs no bind mount and no copy.
|
`<certDir>/<tls.certName>` and `<certDir>/<tls.keyName>`.
|
||||||
Default filenames (`cert.pem` / `key.pem`) match the output
|
Default filenames (`cert.pem` / `key.pem`) match the output
|
||||||
layout of nixpkgs's `security.acme` module.
|
layout of nixpkgs's `security.acme` module.
|
||||||
|
|
||||||
|
|
@ -232,7 +232,9 @@ in
|
||||||
enabled, every request to the gateway's main vhost requires a
|
enabled, every request to the gateway's main vhost requires a
|
||||||
valid username and password. nginx's built-in `auth_basic`
|
valid username and password. nginx's built-in `auth_basic`
|
||||||
module validates credentials against
|
module validates credentials against
|
||||||
`/var/lib/hyperhive/gateway/gateway.htpasswd`. Off by default.
|
`/var/lib/hyperhive/gateway/gateway.htpasswd` on the host
|
||||||
|
(exposed as `/run/hive-state/gateway.htpasswd` inside the
|
||||||
|
container via the existing gateway state bind-mount). Off by default.
|
||||||
|
|
||||||
Manage users with `hivectl gateway create-user`, `delete-user`,
|
Manage users with `hivectl gateway create-user`, `delete-user`,
|
||||||
and `list-users` — see `hivectl gateway --help` for usage.
|
and `list-users` — see `hivectl gateway --help` for usage.
|
||||||
|
|
|
||||||
|
|
@ -241,7 +241,7 @@ let
|
||||||
|
|
||||||
# `/agent/` catch-all 404 + the two internal error-page targets it
|
# `/agent/` catch-all 404 + the two internal error-page targets it
|
||||||
# points at. Per-agent `location /agent/<name>/` blocks live in the
|
# points at. Per-agent `location /agent/<name>/` blocks live in the
|
||||||
# runtime-generated `/var/lib/hyperhive/gateway/agents.conf` (included via
|
# runtime-generated `/run/hive-state/agents.conf` (included via
|
||||||
# `extraConfig` on the vhost); nginx longest-prefix-match makes a
|
# `extraConfig` on the vhost); nginx longest-prefix-match makes a
|
||||||
# real `/agent/<name>/` beat this catch-all. `internal` keeps the
|
# real `/agent/<name>/` beat this catch-all. `internal` keeps the
|
||||||
# error pages reachable only through nginx's error handling.
|
# error pages reachable only through nginx's error handling.
|
||||||
|
|
@ -275,7 +275,7 @@ let
|
||||||
# secret (`X-Hub-Signature-256`) protects those endpoints instead.
|
# secret (`X-Hub-Signature-256`) protects those endpoints instead.
|
||||||
dashboardAuth = lib.optionalString cfg.auth.enable ''
|
dashboardAuth = lib.optionalString cfg.auth.enable ''
|
||||||
auth_basic "${cfg.auth.realm}";
|
auth_basic "${cfg.auth.realm}";
|
||||||
auth_basic_user_file /var/lib/hyperhive/gateway/gateway.htpasswd;
|
auth_basic_user_file /run/hive-state/gateway.htpasswd;
|
||||||
# `=401` keeps the status 401 so the login dialog shows; the
|
# `=401` keeps the status 401 so the login dialog shows; the
|
||||||
# internal page explains `hivectl gateway create-user`.
|
# internal page explains `hivectl gateway create-user`.
|
||||||
error_page 401 =401 /__hive_auth_unauthorized;
|
error_page 401 =401 /__hive_auth_unauthorized;
|
||||||
|
|
@ -397,14 +397,15 @@ in
|
||||||
};
|
};
|
||||||
# Per-agent location blocks, generated at runtime by
|
# Per-agent location blocks, generated at runtime by
|
||||||
# hive-c0re and written to /var/lib/hyperhive/gateway/agents.conf
|
# hive-c0re and written to /var/lib/hyperhive/gateway/agents.conf
|
||||||
# on the host — the same machine nginx runs on. nginx parses
|
# on the host. The bind-mount at /run/hive-state/ exposes
|
||||||
# `include` at config-load time so a reload (triggered by c0re
|
# that file here. nginx parses `include` at config-load
|
||||||
|
# time so a reload (triggered by c0re via systemd-run
|
||||||
# after each agents.conf write) picks up new or removed
|
# after each agents.conf write) picks up new or removed
|
||||||
# agents without a nixos-rebuild. nginx's longest-prefix-
|
# agents without a nixos-rebuild. nginx's longest-prefix-
|
||||||
# match rule ensures `/agent/<name>/` from this file beats
|
# match rule ensures `/agent/<name>/` from this file beats
|
||||||
# the `/agent/` catch-all above.
|
# the `/agent/` catch-all above.
|
||||||
extraConfig = securityHeaders + ''
|
extraConfig = securityHeaders + ''
|
||||||
include /var/lib/hyperhive/gateway/agents.conf;
|
include /run/hive-state/agents.conf;
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -35,16 +35,17 @@ The socket is `0666`. It has to be: nginx runs as a different user and
|
||||||
sockets, and rests on the same argument — *"the bind source dir is per-agent on
|
sockets, and rests on the same argument — *"the bind source dir is per-agent on
|
||||||
host so blast radius is unchanged."*
|
host so blast radius is unchanged."*
|
||||||
|
|
||||||
What keeps that safe is that the directory holds **one** socket. So:
|
What keeps that safe is that the directory holds **one** socket and is
|
||||||
|
bind-mounted into **one** container. So:
|
||||||
|
|
||||||
> **Never point `socketPath` at a directory that carries anything else.**
|
> **Never point `socketPath` at a directory that carries anything else.**
|
||||||
> `/run/hyperhive` above all — it holds `host.sock`, the host **admin** socket.
|
> `/run/hyperhive` above all — it holds `host.sock`, the host **admin** socket.
|
||||||
> Pointing nginx at that directory to reach this socket would put the admin
|
> Mounting that directory to reach this socket would hand the gateway container
|
||||||
> socket within its reach too.
|
> the admin socket along with it.
|
||||||
|
|
||||||
nginx is a host service, so nothing narrows what it can reach except the
|
Changing `socketPath` therefore means re-checking the gateway bind-mount, not
|
||||||
directory itself — that is the whole of the access control. A unit test pins
|
just the daemon. A unit test pins the default path so a tidying edit fails
|
||||||
the default path so a tidying edit fails instead of reviewing cleanly.
|
instead of reviewing cleanly.
|
||||||
|
|
||||||
`RuntimeDirectoryPreserve=yes` and the daemon's stale-socket unlink on start are
|
`RuntimeDirectoryPreserve=yes` and the daemon's stale-socket unlink on start are
|
||||||
a **pair**: preserving the directory without the unlink means `bind` fails with
|
a **pair**: preserving the directory without the unlink means `bind` fails with
|
||||||
|
|
|
||||||
|
|
@ -27,10 +27,10 @@ use axum::{Router, routing::get};
|
||||||
/// `ExecStart`, so the default names a directory the unit just produced.
|
/// `ExecStart`, so the default names a directory the unit just produced.
|
||||||
///
|
///
|
||||||
/// The directory is its own — deliberately not shared with hive-c0re's
|
/// The directory is its own — deliberately not shared with hive-c0re's
|
||||||
/// `/run/hyperhive`. The socket is `0666`, so its directory is the only
|
/// `/run/hyperhive`. nginx reaches a unix upstream by having the socket's
|
||||||
/// access control it has; co-locating it with c0re's admin socket would
|
/// *directory* bind-mounted into the gateway container, so co-locating
|
||||||
/// put both within reach of whatever can reach either. nginx runs on the
|
/// this socket with c0re's admin socket would hand the gateway the admin
|
||||||
/// host, so nothing narrows its reach for you.
|
/// socket along with it.
|
||||||
const DEFAULT_SOCKET: &str = "/run/swarm-controller/controller.sock";
|
const DEFAULT_SOCKET: &str = "/run/swarm-controller/controller.sock";
|
||||||
|
|
||||||
fn socket_path() -> PathBuf {
|
fn socket_path() -> PathBuf {
|
||||||
|
|
@ -96,8 +96,8 @@ mod tests {
|
||||||
/// The socket must not share a directory with anything else, because
|
/// The socket must not share a directory with anything else, because
|
||||||
/// the socket is `0666` and the directory is therefore the only access
|
/// the socket is `0666` and the directory is therefore the only access
|
||||||
/// control it has. `/run/hyperhive` in particular holds hive-c0re's
|
/// control it has. `/run/hyperhive` in particular holds hive-c0re's
|
||||||
/// **admin** socket, and nginx — a host service — is bounded by
|
/// **admin** socket, and nginx reaches a unix upstream by mounting the
|
||||||
/// nothing but the directory itself.
|
/// socket's whole directory into the gateway container.
|
||||||
///
|
///
|
||||||
/// A test rather than a comment: the failure this guards against is a
|
/// A test rather than a comment: the failure this guards against is a
|
||||||
/// one-word edit that looks tidier and reads fine in review.
|
/// one-word edit that looks tidier and reads fine in review.
|
||||||
|
|
@ -111,7 +111,7 @@ mod tests {
|
||||||
Path::new("/run/swarm-controller"),
|
Path::new("/run/swarm-controller"),
|
||||||
"the socket's directory is its access control — moving it under a shared \
|
"the socket's directory is its access control — moving it under a shared \
|
||||||
directory (notably /run/hyperhive, which holds the host admin socket) \
|
directory (notably /run/hyperhive, which holds the host admin socket) \
|
||||||
exposes everything else in that directory to the gateway's nginx"
|
exposes everything else in that directory to the gateway container"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue