feat(3088): move the gateway's nginx + dnsmasq onto the host
The gateway's nginx + dnsmasq no longer run in their own nspawn container. `nix/host-modules/hive-gateway/default.nix` loses the `containers.hive-gateway` wrapper and everything that existed only to punch holes in it: `privateNetwork = false`, `CAP_NET_ADMIN`, five bind mounts, its own `stateVersion`, `networking.firewall.enable = false`, `networking.resolvconf.enable = false`, and the `hive-gateway-resolv` path+service pair. 465 -> 303 lines. The container never bought isolation here. It shared the host netns by necessity — nginx binds the host's :80/:443, dnsmasq answers on the bridge — so each of those settings was undoing a boundary the gateway could not afford in the first place. Four things made it more than a deletion, none of them visible in the nix diff: - The self-signed cert service also imports the hive CA leaf, so removing it with the container would have left nginx naming a missing cert file, which it refuses to load at all. - The nginx reload is a hive-priv verb. It still needs root, but no longer for the reason its doc gave, and `--machine=` was both transport and scope — so the unit name is now hard-coded in the helper as the containment. - The lifecycle verb named a container that stops existing. - `journalctl -M hive-gateway` had no machine to enter. Per the operator's ruling, the operator verb keeps working and agents lose it. `InfraContainer` answered three questions that used to share an answer; it now splits into `name()` (identity), `target()` (Container vs HostUnit), `service_unit()` (the systemd unit), and `agent_restartable()`, which the MCP restart path checks before the capability so the refusal cannot read as "ask for infra_admin". `SIBLING_CONTAINERS` drops the gateway — it gates the requests that name a container as a string — while `FromStr` still accepts it, because that answers what a name is, not who may act on it. The dashboard's gateway journal reads host journald filtered to `nginx.service`. Prose was corrected where it only named a location, and re-argued where the container was doing security work: a `0666` per-agent socket was safe because only the gateway container had the directory bind-mounted. There is no mount now, so the directory permissions are the whole of the access control — the constraint holds, its mechanism doesn't. Gate: nix fmt / clippy --all-targets -D warnings / cargo test all clean (710 tests); hivectl-cli.md regenerated from the clap tree. The nix eval was run in both TLS shapes at this commit: every delta in the rendered virtualHosts is one of the three intended path moves, dnsmasq settings are byte-identical, and the absence probe flips true -> false with bindMounts emptied.
This commit is contained in:
parent
cae2cf8df6
commit
07852cabc1
34 changed files with 704 additions and 618 deletions
|
|
@ -1,8 +1,7 @@
|
|||
//! Runtime nginx include-file generator for the gateway's per-agent
|
||||
//! `/agent/<name>/` location blocks. Writes
|
||||
//! `/var/lib/hyperhive/gateway/agents.conf` on every topology change.
|
||||
//! UDS upstream selection, reload trigger (`systemd-run
|
||||
//! --machine=hive-gateway`), and idempotency:
|
||||
//! UDS upstream selection, the reload trigger, and idempotency:
|
||||
//! `docs/gateway.md::Per-agent unix-socket upstream`.
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
|
|
@ -19,13 +18,13 @@ use crate::agent_sockets;
|
|||
/// Set when `write` publishes a new agents.conf; cleared when
|
||||
/// `reload_gateway_nginx` submits the reload command successfully.
|
||||
/// Lets `spawn_poll` retry the reload on subsequent ticks when the
|
||||
/// previous attempt failed (e.g. gateway container temporarily down,
|
||||
/// systemd-run not found) without re-writing the already-correct file.
|
||||
/// previous attempt failed (e.g. gateway nginx temporarily down, priv
|
||||
/// helper unreachable) without re-writing the already-correct file.
|
||||
static RELOAD_PENDING: AtomicBool = AtomicBool::new(false);
|
||||
|
||||
/// Unix timestamp (seconds) of the last failed reload attempt.
|
||||
/// `reload_if_pending` backs off to once per `RELOAD_RETRY_SECS` after a
|
||||
/// failure so a permanently broken gateway (bad config, container down)
|
||||
/// failure so a permanently broken gateway (bad config, nginx down)
|
||||
/// doesn't hammer `systemctl` on every 10-second `spawn_poll` tick.
|
||||
static LAST_FAILED_RELOAD: AtomicU64 = AtomicU64::new(0);
|
||||
|
||||
|
|
@ -79,7 +78,7 @@ fn render(names: &[String], frontend_dir: Option<&str>) -> String {
|
|||
let mut out = String::from(
|
||||
"# Generated by hive-c0re \u{2014} do not edit.\
|
||||
\n# Refreshed on every topology change + when agents bind/drop their unix sockets.\
|
||||
\n# Reload triggered by hive-c0re via systemd-run --machine=hive-gateway.\n",
|
||||
\n# Reload triggered by hive-c0re via hive-priv (systemctl reload nginx).\n",
|
||||
);
|
||||
for name in names {
|
||||
// Two upstream forms because named locations (split mode's
|
||||
|
|
@ -165,15 +164,19 @@ fn render(names: &[String], frontend_dir: Option<&str>) -> String {
|
|||
/// body matches what's already on disk (idempotent; avoids spurious
|
||||
/// gateway reloads on a quiet tick).
|
||||
///
|
||||
/// After a successful write, triggers the appropriate nginx action inside
|
||||
/// the gateway container via `hive-priv` (which has the
|
||||
/// `--machine=hive-gateway` transport rights hive-c0re lacks):
|
||||
/// reload when nginx is active, reset-failed+start when in a failed
|
||||
/// state, plain start otherwise. This is intentionally host-side rather
|
||||
/// than relying on a systemd path unit inside the container watching the
|
||||
/// bind-mounted file: `IN_MOVED_TO` (fired by the atomic rename) does
|
||||
/// not reliably propagate across the nspawn mount-namespace boundary, so
|
||||
/// the path-unit approach was silently broken (see `docs/gateway.md`).
|
||||
/// After a successful write, triggers the appropriate nginx action via
|
||||
/// `hive-priv` (hive-c0re runs unprivileged and cannot act on a system
|
||||
/// unit): reload when nginx is active, reset-failed+start when in a
|
||||
/// failed state, plain start otherwise. Writer and nginx are now on the
|
||||
/// same machine, so this is a plain unit action rather than the old
|
||||
/// `systemd-run --machine=hive-gateway` hop across the container
|
||||
/// boundary. It stays an explicit trigger rather than a systemd path
|
||||
/// unit watching the file. A path unit would now *work* — `IN_MOVED_TO`
|
||||
/// (fired by the atomic rename) failed to propagate across the nspawn
|
||||
/// mount-namespace boundary, and that boundary is gone — but it is still
|
||||
/// not wanted: the write already knows it changed something, and a
|
||||
/// watcher turns one causal edge into a race with the writer's own
|
||||
/// rename (see `docs/gateway.md`).
|
||||
///
|
||||
/// The priv call is best-effort — a failed sync is logged but not fatal.
|
||||
/// `reload_if_pending` retries on the next `spawn_poll` tick so a
|
||||
|
|
@ -212,7 +215,7 @@ pub async fn write(names: &[String]) -> Result<()> {
|
|||
|
||||
/// Retry a pending nginx reload if a previous attempt failed.
|
||||
/// Called by `spawn_poll` on each tick so a transient failure
|
||||
/// (gateway container temporarily down, systemd-run error) is
|
||||
/// (gateway nginx temporarily down, priv-helper error) is
|
||||
/// recovered automatically without requiring a new file write.
|
||||
///
|
||||
/// Backs off to one retry per `RELOAD_RETRY_SECS` after a failure so a
|
||||
|
|
@ -240,8 +243,8 @@ pub async fn reload_if_pending() {
|
|||
/// Synchronise the gateway nginx unit with the current agents.conf via
|
||||
/// `hive-priv` (privileged helper). The state-aware logic (active →
|
||||
/// reload; failed → reset-failed + start; inactive/unknown → start)
|
||||
/// runs inside hive-priv where it has the `--machine=hive-gateway`
|
||||
/// transport rights that hive-c0re (unprivileged) lacks.
|
||||
/// runs inside hive-priv, which is root; hive-c0re runs as the
|
||||
/// unprivileged `hive-core` user and cannot act on a system unit.
|
||||
///
|
||||
/// `RELOAD_PENDING` is cleared only after a successful operation so
|
||||
/// `reload_if_pending` keeps retrying on failure.
|
||||
|
|
|
|||
Loading…
Reference in a new issue