fix(3179): the gateway's config files get their own state dir

`agents.conf` and `gateway.htpasswd` move from /var/lib/hyperhive/gateway
to /var/lib/hive-gateway/conf, alongside the `tls/` the gateway already
kept there.

nginx reads both as an unprivileged user. Under c0re's state dir it could
only reach them by traversing a directory systemd re-declares `0750
hive-core` on every c0re start — so nginx was given `SupplementaryGroups
= [ "hive-core" ]`, which also handed it read access to everything else
group-readable in that tree. The tokens are individually 0600, but the
broker sqlite carries no explicit mode: every message between every agent
was readable by the process whose job is parsing untrusted network input.

Moving the files removes the need and the exposure together. The group is
gone, and its absence is now commented as load-bearing so it doesn't come
back as a fix for a symptom it would recreate.

Also drops this module's `/var/lib/hyperhive` tmpfiles rule. It declared
`0755 root root` and could never win against `StateDirectoryMode`, and a
losing declaration still reads as a guarantee — that is what sent the
first diagnosis of the outage looking for who had changed the mode.

Ordering is unchanged and still the thing that makes a fresh boot work:
tmpfiles runs before services and seeds both files empty-but-valid, nginx
names them (an `include` of a missing file is fatal, not empty), and
content arrives when c0re writes and reloads — which it does on every
topology change, so a boot against the empty seed resolves itself.

Folds in the mode fix: `write` now sets 0644 on the tmp file before the
rename, because a rename carries the source's mode and discards the
destination's, and the tmpfiles rule that declares 0644 is
create-if-absent so it never re-applies.
This commit is contained in:
atlas 2026-08-12 00:05:53 +02:00 committed by mara
commit 0e1a975f9f
9 changed files with 98 additions and 50 deletions

View file

@ -1,11 +1,12 @@
//! Runtime nginx include-file generator for the gateway's per-agent
//! `/agent/<name>/` location blocks. Writes
//! `/var/lib/hyperhive/gateway/agents.conf` on every topology change.
//! `/var/lib/hive-gateway/conf/agents.conf` on every topology change.
//! UDS upstream selection, the reload trigger, and idempotency:
//! `docs/gateway.md::Per-agent unix-socket upstream`.
use anyhow::{Context, Result};
use std::fmt::Write as _;
use std::os::unix::fs::PermissionsExt as _;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::{Mutex, OnceLock};
use std::time::{SystemTime, UNIX_EPOCH};
@ -192,6 +193,17 @@ pub async fn write(names: &[String]) -> Result<()> {
}
let tmp = path.with_extension("conf.tmp");
std::fs::write(&tmp, &body).with_context(|| format!("write tmp {}", tmp.display()))?;
// Mode set on the TMP file, before the rename, because a rename
// carries the source's mode and owner and discards the destination's.
// The tmpfiles rule declaring 0644 is `f` (create-if-absent), so it
// never re-applies: from the first republish on, the published mode is
// whatever this process's umask happened to produce. That is fine
// today and becomes a dead gateway the day anything tightens c0re's
// umask, since nginx reads this file as a different, unprivileged
// user. Making it explicit means the mode is a property of the
// publisher rather than of an unrelated unit's settings.
std::fs::set_permissions(&tmp, std::fs::Permissions::from_mode(0o644))
.with_context(|| format!("chmod tmp {}", tmp.display()))?;
std::fs::rename(&tmp, &path).with_context(|| {
format!(
"rename {} -> {} (atomic publish)",

View file

@ -167,7 +167,7 @@ pub async fn sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<()> {
tracing::warn!(error = ?e, "agent_sockets::write failed (non-fatal)");
}
// Refresh /var/lib/hyperhive/gateway/agents.conf — the nginx include
// Refresh /var/lib/hive-gateway/conf/agents.conf — the nginx include
// file the gateway reads at runtime. c0re then triggers a reload (or
// start) of the host's nginx via hive-priv, since c0re is unprivileged.
// Same best-effort + non-fatal shape.

View file

@ -218,16 +218,24 @@ pub fn shared_root() -> PathBuf {
// nix: bind-mounted read-only into agent containers as `/knowledge` (the harness nix modules) — must match.
pub const KNOWLEDGE_DIR: &str = "/var/lib/hyperhive/knowledge";
/// `gateway/` — generated nginx include fragments for the gateway vhost.
/// nginx runs on the host and reads this path directly. ⚠️ Nothing about
/// this path confines it: what keeps nginx away from the rest of
/// `/var/lib/hyperhive/` (forge/matrix tokens, etc.) is the unit's own
/// sandbox, so widening that sandbox widens what a gateway compromise
/// reaches.
// nix: named by the gateway's nginx config (hive-gateway/vhosts.nix) — must match.
/// `/var/lib/hive-gateway/conf` — generated nginx include fragments,
/// deliberately **outside** `STATE_ROOT`.
///
/// nginx runs on the host as an unprivileged user and reads these
/// directly. Keeping them here rather than under `/var/lib/hyperhive`
/// means nginx needs no access to c0re's state dir at all — no shared
/// parent to traverse, so no group membership handing it everything else
/// that lives there (the broker db above all). The separation IS the
/// confinement; nothing else is doing that job.
///
/// Sibling of the gateway's `tls/`, which already lived here.
// nix: named by the gateway's nginx config (hive-gateway/vhosts.nix) and created by
// its tmpfiles rules (hive-gateway/default.nix) — must match.
pub const GATEWAY_CONF_DIR: &str = "/var/lib/hive-gateway/conf";
#[must_use]
pub fn gateway_dir() -> PathBuf {
state_root().join("gateway")
PathBuf::from(GATEWAY_CONF_DIR)
}
/// `gateway/agents.conf` — per-agent nginx `location` blocks (UDS upstreams).