docs(gateway): describe what is, not what changed
Per review: docs represent current state. Every "used to" / "no longer" clause this branch introduced is gone — including the History section in network.md, which was a whole subsection about a sync mechanism that doesn't exist. Where the removed clause was carrying a real constraint, the constraint stays and is stated in the present tense instead of as a delta: nothing narrows what the gateway's nginx can reach except the directory permissions in front of a socket, and nothing bounds `ReloadGatewayNginx` except the hard-coded unit name. Those read as rules now rather than as the story of how they came to be rules.
This commit is contained in:
parent
07852cabc1
commit
98d895cf9e
15 changed files with 77 additions and 146 deletions
|
|
@ -121,25 +121,18 @@ The mode is load-bearing, not cosmetic. Write permission on a
|
||||||
them, and the sticky bit is the only thing that would restrain that (it
|
them, and the sticky bit is the only thing that would restrain that (it
|
||||||
is not set here). A world-writable socket dir therefore lets anything
|
is not set here). A world-writable socket dir therefore lets anything
|
||||||
able to reach the path delete an agent's socket and bind its own — and
|
able to reach the path delete an agent's socket and bind its own — and
|
||||||
nginx reaches all of `/run/hive-agent` (as a plain host path since the
|
nginx reaches all of `/run/hive-agent` as a plain host path. Dropping
|
||||||
gateway moved out of its container; it used to be bind-mounted in, which
|
`o=w` removes that permission rather than qualifying it.
|
||||||
was the same reach through a longer route). Dropping `o=w` removes that
|
|
||||||
permission rather than qualifying it.
|
|
||||||
|
|
||||||
⚠️ **The gateway leaving its container is a deliberate trade, recorded
|
⚠️ **The gateway's nginx and dnsmasq are host services, next to
|
||||||
here so it is not mistaken for an oversight.** nginx and dnsmasq run on
|
`hive-c0re`** (see `docs/gateway.md`) — there is no namespace between
|
||||||
the host next to `hive-c0re` (see `docs/gateway.md`). What was given up
|
them and the rest of the host. That costs no network isolation: nginx
|
||||||
is a *mount/pid* namespace — **not** a network one: that container ran
|
binds the host's `:80`/`:443` and reaches `localhost` upstreams, which a
|
||||||
with `privateNetwork = false` and shared the host's netns, so nginx was
|
netns would have to be opened up for anyway.
|
||||||
already binding host ports and already reaching `localhost` upstreams.
|
🔑 It does mean nothing *implicitly* scopes the privileged reload verb,
|
||||||
The boundary bought no network isolation while costing a resolv.conf
|
so the scope is explicit: the unit name is hard-coded in `hive-priv` —
|
||||||
sync, a reload that had to cross the machine bus, and three bind mounts.
|
see `PrivRequest::ReloadGatewayNginx`. **A caller cannot name the unit,
|
||||||
🔑 It did cost one real thing, and the replacement is explicit: the
|
so the verb cannot be steered at another service.**
|
||||||
privileged reload verb used to be scoped by `--machine=hive-gateway`,
|
|
||||||
which could only ever reach into that one container. With no namespace
|
|
||||||
to bound it, the unit name is hard-coded in `hive-priv` instead — see
|
|
||||||
`PrivRequest::ReloadGatewayNginx`. **A caller cannot name the unit, so
|
|
||||||
the verb cannot be steered at another service.**
|
|
||||||
|
|
||||||
⚠️ Contrast `/shared`, which *is* sticky world-writable (`1777`): it has
|
⚠️ Contrast `/shared`, which *is* sticky world-writable (`1777`): it has
|
||||||
many legitimate writers, so sticky is the best available answer there.
|
many legitimate writers, so sticky is the best available answer there.
|
||||||
|
|
|
||||||
|
|
@ -122,12 +122,9 @@ now set unconditionally for every agent. The mechanism:
|
||||||
- failed → `systemctl reset-failed nginx` + `systemctl start nginx`
|
- failed → `systemctl reset-failed nginx` + `systemctl start nginx`
|
||||||
- otherwise → `systemctl start nginx`
|
- otherwise → `systemctl start nginx`
|
||||||
This is an explicit trigger rather than a path unit watching the
|
This is an explicit trigger rather than a path unit watching the
|
||||||
file. It used to be *impossible* to do it any other way — `IN_MOVED_TO`
|
file: the write and the reload belong in one causal chain c0re can
|
||||||
from the atomic rename did not cross the nspawn mount-namespace
|
retry and report on (`RELOAD_PENDING`), not two units racing on an
|
||||||
boundary, so an in-container path unit never fired. With nginx on the
|
inotify event.
|
||||||
host a path unit would now work, and it is still not wanted: the write
|
|
||||||
and the reload belong in one causal chain c0re can retry and report on
|
|
||||||
(`RELOAD_PENDING`), not two units racing on an inotify event.
|
|
||||||
|
|
||||||
c0re regenerates `agents.conf` (and triggers a reload) on two
|
c0re regenerates `agents.conf` (and triggers a reload) on two
|
||||||
triggers: every topology change (new/removed agents) and every 10s
|
triggers: every topology change (new/removed agents) and every 10s
|
||||||
|
|
@ -200,7 +197,7 @@ On by default, and listens on `httpsPort` (default 443) on every vhost beside th
|
||||||
|
|
||||||
The issuer is a **host-held hive CA**, not a bare self-signed leaf. A host service (`hive-tls-ca.service`, from the `hive-tls` module) generates a long-lived CA (`services.hyperhive.tls.caValidityDays`, default ~20y) under `services.hyperhive.tls.stateDir` (default `/var/lib/hive-tls`), then signs a gateway **leaf** (`leafValidityDays`, default 30d) with it. `hive-gateway-self-signed-cert` then imports the leaf into nginx's state dir (`/var/lib/hive-gateway/tls/{cert,key}.pem`).
|
The issuer is a **host-held hive CA**, not a bare self-signed leaf. A host service (`hive-tls-ca.service`, from the `hive-tls` module) generates a long-lived CA (`services.hyperhive.tls.caValidityDays`, default ~20y) under `services.hyperhive.tls.stateDir` (default `/var/lib/hive-tls`), then signs a gateway **leaf** (`leafValidityDays`, default 30d) with it. `hive-gateway-self-signed-cert` then imports the leaf into nginx's state dir (`/var/lib/hive-gateway/tls/{cert,key}.pem`).
|
||||||
|
|
||||||
⚠️ **That import unit is not a leftover of the old container — do not collapse it into pointing nginx at the CA dir.** It does two jobs. It re-modes the leaf (`hive-tls-ca` writes the key `0600 root:root`; nginx's pre-start `nginx -t` runs as the *nginx user*, so a `0600` key fails the config test and blocks the unit), and it guarantees **every cert path the nginx config names exists** — which is what the swarm-services fallback below is for. Removing it re-creates the #3097 outage.
|
⚠️ **Do not collapse that import unit into pointing nginx at the CA dir.** It does two jobs. It re-modes the leaf (`hive-tls-ca` writes the key `0600 root:root`; nginx's pre-start `nginx -t` runs as the *nginx user*, so a `0600` key fails the config test and blocks the unit), and it guarantees **every cert path the nginx config names exists** — which is what the swarm-services fallback below is for. Removing it re-creates the #3097 outage.
|
||||||
|
|
||||||
**Why a CA, not a bare leaf**: a bare self-signed leaf is its own trust anchor, so every regeneration is a new anchor every consumer must re-trust — and a runtime-generated leaf can't be wired into an agent's build-time trust store at all. With a stable CA, agents and federation peers trust it *once*; leaf rotation never re-breaks them.
|
**Why a CA, not a bare leaf**: a bare self-signed leaf is its own trust anchor, so every regeneration is a new anchor every consumer must re-trust — and a runtime-generated leaf can't be wired into an agent's build-time trust store at all. With a stable CA, agents and federation peers trust it *once*; leaf rotation never re-breaks them.
|
||||||
|
|
||||||
|
|
@ -433,9 +430,7 @@ any content change — so `max-age=31536000` is safe.
|
||||||
**Why the nix store path resolves**: `HIVE_AGENT_FRONTEND_DIR` is a nix
|
**Why the nix store path resolves**: `HIVE_AGENT_FRONTEND_DIR` is a nix
|
||||||
store path baked in at hive-c0re build time, and c0re (writing
|
store path baked in at hive-c0re build time, and c0re (writing
|
||||||
`agents.conf`) and nginx (serving files from it) are on the same machine,
|
`agents.conf`) and nginx (serving files from it) are on the same machine,
|
||||||
so they see the same store. This used to need explaining — nspawn
|
so they see the same store.
|
||||||
bind-mounts `/nix/store` read-only into a container, which is what made
|
|
||||||
the baked-in path work from inside the gateway.
|
|
||||||
|
|
||||||
**Graceful degradation**: if `HIVE_AGENT_FRONTEND_DIR` is empty or
|
**Graceful degradation**: if `HIVE_AGENT_FRONTEND_DIR` is empty or
|
||||||
unset (e.g. a build that predates `cfg.frontend`), each agent gets the
|
unset (e.g. a build that predates `cfg.frontend`), each agent gets the
|
||||||
|
|
|
||||||
|
|
@ -124,37 +124,10 @@ There is deliberately no fallback `server=`: dnsmasq queries all known
|
||||||
upstreams in parallel, so a hardcoded public resolver would take a share
|
upstreams in parallel, so a hardcoded public resolver would take a share
|
||||||
of normal traffic, not just cover the gap.
|
of normal traffic, not just cover the gap.
|
||||||
|
|
||||||
### History: the resolv.conf sync, and why it is gone
|
dnsmasq runs on the host and reads the host's `/etc/resolv.conf`
|
||||||
|
directly, so a network change (new router, new lease, laptop moving
|
||||||
Until the gateway moved onto the host, dnsmasq ran in the `hive-gateway`
|
networks) reaches it the moment openresolv rewrites the file. There is
|
||||||
container and read *that* container's `/etc/resolv.conf` — a one-shot
|
nothing to synchronise and no unit watching for it.
|
||||||
copy nixos-container made at start. systemd-nspawn(1) is explicit that
|
|
||||||
nothing propagates into it after early init, because resolv.conf is
|
|
||||||
normally updated by rename rather than in place. So a host network change
|
|
||||||
(new router, new lease, laptop moving networks) stranded dnsmasq on a
|
|
||||||
resolver that no longer answered, and every non-hive lookup from every
|
|
||||||
agent hung until someone restarted the gateway.
|
|
||||||
|
|
||||||
A host-side `hive-gateway-resolv` path unit closed that gap: watch
|
|
||||||
`/etc/resolv.conf`, `machinectl copy-to` it into the container, reload
|
|
||||||
dnsmasq. Roughly eighty lines of watcher, marker file, is-active guard
|
|
||||||
and mid-rewrite-snapshot check — **all of it bridging two copies of one
|
|
||||||
file.** With one machine there is one file, and the whole unit is
|
|
||||||
deleted.
|
|
||||||
|
|
||||||
🔑 Worth keeping as a shape, not just a story: **the sync was not
|
|
||||||
complexity anyone chose. It was the cost of a boundary that bought
|
|
||||||
nothing here** — the gateway already ran with `privateNetwork = false`,
|
|
||||||
sharing the host's netns, so the container never provided network
|
|
||||||
isolation in the first place. When a workaround is that elaborate, the
|
|
||||||
question to ask is what the boundary is *for*.
|
|
||||||
|
|
||||||
(Two alternatives were considered at the time and both were worse than
|
|
||||||
the copy: a path unit *inside* the container never fired, because the
|
|
||||||
host replaces the file by rename and `IN_MOVED_TO` does not cross the
|
|
||||||
nspawn mount namespace; and bind-mounting the host's `/etc/resolv.conf`
|
|
||||||
would have pinned the *first* inode for the container's whole lifetime,
|
|
||||||
since openresolv writes a temp file and renames over the target.)
|
|
||||||
|
|
||||||
`bind-interfaces` + `interface = [ bridgeName "lo" ]` means the
|
`bind-interfaces` + `interface = [ bridgeName "lo" ]` means the
|
||||||
listener only accepts queries from the bridge interface (plus lo for
|
listener only accepts queries from the bridge interface (plus lo for
|
||||||
|
|
|
||||||
|
|
@ -191,16 +191,12 @@ directly invoke `nixos-container`, `journalctl -M`, or act on a system
|
||||||
unit (`systemctl reload nginx`) — those require root. `hive-priv` fills
|
unit (`systemctl reload nginx`) — those require root. `hive-priv` fills
|
||||||
this gap.
|
this gap.
|
||||||
|
|
||||||
⚠️ **Note what that costs when a helper verb loses its namespace.**
|
⚠️ **`ReloadGatewayNginx` acts on a host unit, so nothing implicitly
|
||||||
`ReloadGatewayNginx` used to run `systemctl -M hive-gateway …`, and the
|
scopes it.** Its containment is the unit name hard-coded in `hive-priv`:
|
||||||
`--machine=` flag was doing two jobs: it was the *transport* into the
|
a caller cannot name the unit, so the verb cannot be steered at another
|
||||||
container **and** the *scope* — the verb could not reach anything
|
service. **A privileged verb needs something bounding what it can act
|
||||||
outside that one machine. With nginx on the host the transport is
|
on; when that isn't a namespace, it has to be a constant the caller
|
||||||
unnecessary and the scope went with it, so the containment is now the
|
can't supply.**
|
||||||
hard-coded unit name in `hive-priv`: a caller cannot name the unit, so
|
|
||||||
the verb cannot be steered at another service. **When a privileged verb
|
|
||||||
stops needing a namespace, check whether the namespace was also what
|
|
||||||
bounded it.**
|
|
||||||
|
|
||||||
### hive-priv
|
### hive-priv
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -172,9 +172,8 @@ pub async fn serve(
|
||||||
/// `0o666` — world-accessible so the gateway's nginx process can
|
/// `0o666` — world-accessible so the gateway's nginx process can
|
||||||
/// `connect(2)` without sharing a group with the agent user. What bounds
|
/// `connect(2)` without sharing a group with the agent user. What bounds
|
||||||
/// that is the per-agent subdir (`/run/hive-agent/<name>/`): the socket
|
/// that is the per-agent subdir (`/run/hive-agent/<name>/`): the socket
|
||||||
/// mode grants everyone, the directory decides who gets to ask. It used
|
/// mode grants everyone, so the directory's permissions decide who gets
|
||||||
/// to be bind-mounted into the one container that needed it; nginx is a
|
/// to ask.
|
||||||
/// host unit now, so the directory's own permissions are the whole story.
|
|
||||||
///
|
///
|
||||||
/// Marker-gating + the gateway-side consumer: see
|
/// Marker-gating + the gateway-side consumer: see
|
||||||
/// [`docs/gateway.md::Per-agent unix-socket upstream`](../../../docs/gateway.md).
|
/// [`docs/gateway.md::Per-agent unix-socket upstream`](../../../docs/gateway.md).
|
||||||
|
|
|
||||||
|
|
@ -5,9 +5,9 @@
|
||||||
//! is reached from the dashboard — already fully operator-authenticated —
|
//! is reached from the dashboard — already fully operator-authenticated —
|
||||||
//! so no capability check is needed here, just the same audit trail.
|
//! so no capability check is needed here, just the same audit trail.
|
||||||
//!
|
//!
|
||||||
//! The two surfaces no longer cover the same set: the gateway is the
|
//! The two surfaces cover different sets: this endpoint takes all four,
|
||||||
//! operator's to restart and not an agent's, since nginx on the host fronts
|
//! while the agent path refuses the gateway — nginx on the host fronts
|
||||||
//! every hive service. This endpoint keeps all four.
|
//! every hive service, so bouncing it is the operator's call.
|
||||||
|
|
||||||
use axum::{
|
use axum::{
|
||||||
extract::{Path as AxumPath, State},
|
extract::{Path as AxumPath, State},
|
||||||
|
|
|
||||||
|
|
@ -193,8 +193,7 @@ pub(super) async fn get_journal_host(
|
||||||
axum::extract::Query(q): axum::extract::Query<JournalHostQuery>,
|
axum::extract::Query(q): axum::extract::Query<JournalHostQuery>,
|
||||||
) -> Result<Response, ProblemDetails> {
|
) -> Result<Response, ProblemDetails> {
|
||||||
let lines = q.lines.unwrap_or(500).min(5000);
|
let lines = q.lines.unwrap_or(500).min(5000);
|
||||||
// `nginx.service` is the gateway: its logs used to live in the
|
// `nginx.service` is the gateway — its logs are host-side.
|
||||||
// hive-gateway container's journal and are host-side now.
|
|
||||||
let allowed = ["hive-c0re.service", "hive-priv.service", "nginx.service"];
|
let allowed = ["hive-c0re.service", "hive-priv.service", "nginx.service"];
|
||||||
let unit = match q.unit.as_deref().filter(|s| !s.is_empty()) {
|
let unit = match q.unit.as_deref().filter(|s| !s.is_empty()) {
|
||||||
Some(u) => {
|
Some(u) => {
|
||||||
|
|
|
||||||
|
|
@ -167,16 +167,12 @@ fn render(names: &[String], frontend_dir: Option<&str>) -> String {
|
||||||
/// After a successful write, triggers the appropriate nginx action via
|
/// After a successful write, triggers the appropriate nginx action via
|
||||||
/// `hive-priv` (hive-c0re runs unprivileged and cannot act on a system
|
/// `hive-priv` (hive-c0re runs unprivileged and cannot act on a system
|
||||||
/// unit): reload when nginx is active, reset-failed+start when in a
|
/// unit): reload when nginx is active, reset-failed+start when in a
|
||||||
/// failed state, plain start otherwise. Writer and nginx are now on the
|
/// failed state, plain start otherwise. Writer and nginx are on the same
|
||||||
/// same machine, so this is a plain unit action rather than the old
|
/// machine, so it is a plain unit action with no machine-bus hop.
|
||||||
/// `systemd-run --machine=hive-gateway` hop across the container
|
/// It stays an explicit trigger rather than a systemd path
|
||||||
/// boundary. It stays an explicit trigger rather than a systemd path
|
/// unit watching the file: the write already knows it changed something,
|
||||||
/// unit watching the file. A path unit would now *work* — `IN_MOVED_TO`
|
/// and a watcher would turn one causal edge into a race with the writer's
|
||||||
/// (fired by the atomic rename) failed to propagate across the nspawn
|
/// own rename (see `docs/gateway.md`).
|
||||||
/// mount-namespace boundary, and that boundary is gone — but it is still
|
|
||||||
/// not wanted: the write already knows it changed something, and a
|
|
||||||
/// watcher turns one causal edge into a race with the writer's own
|
|
||||||
/// rename (see `docs/gateway.md`).
|
|
||||||
///
|
///
|
||||||
/// The priv call is best-effort — a failed sync is logged but not fatal.
|
/// The priv call is best-effort — a failed sync is logged but not fatal.
|
||||||
/// `reload_if_pending` retries on the next `spawn_poll` tick so a
|
/// `reload_if_pending` retries on the next `spawn_poll` tick so a
|
||||||
|
|
|
||||||
|
|
@ -219,11 +219,11 @@ pub fn shared_root() -> PathBuf {
|
||||||
pub const KNOWLEDGE_DIR: &str = "/var/lib/hyperhive/knowledge";
|
pub const KNOWLEDGE_DIR: &str = "/var/lib/hyperhive/knowledge";
|
||||||
|
|
||||||
/// `gateway/` — generated nginx include fragments for the gateway vhost.
|
/// `gateway/` — generated nginx include fragments for the gateway vhost.
|
||||||
/// nginx runs on the host and reads this path directly; it used to be
|
/// nginx runs on the host and reads this path directly. ⚠️ Nothing about
|
||||||
/// bind-mounted into a gateway container at `/run/hive-state/`, exposing
|
/// this path confines it: what keeps nginx away from the rest of
|
||||||
/// this subdir *only* so the rest of `/var/lib/hyperhive/` (forge/matrix
|
/// `/var/lib/hyperhive/` (forge/matrix tokens, etc.) is the unit's own
|
||||||
/// tokens, etc.) stayed out of reach. On the host that narrowing is the
|
/// sandbox, so widening that sandbox widens what a gateway compromise
|
||||||
/// unit's sandbox, not a mount — nginx is not confined by this path.
|
/// reaches.
|
||||||
// nix: named by the gateway's nginx config (hive-gateway/vhosts.nix) — must match.
|
// nix: named by the gateway's nginx config (hive-gateway/vhosts.nix) — must match.
|
||||||
#[must_use]
|
#[must_use]
|
||||||
pub fn gateway_dir() -> PathBuf {
|
pub fn gateway_dir() -> PathBuf {
|
||||||
|
|
|
||||||
|
|
@ -14,11 +14,10 @@ use std::path::{Path, PathBuf};
|
||||||
use anyhow::{Context, Result};
|
use anyhow::{Context, Result};
|
||||||
|
|
||||||
/// Host-side parent directory holding per-agent socket subdirs. The
|
/// Host-side parent directory holding per-agent socket subdirs. The
|
||||||
/// gateway's nginx runs on the host and reads this whole tree, so it
|
/// gateway's nginx runs on the host and reads this whole tree, so it can
|
||||||
/// can `proxy_pass` to any agent — it used to get there through a
|
/// `proxy_pass` to any agent. Each agent's container bind-mounts only its
|
||||||
/// read-only bind-mount of the same tree. Each agent's container
|
/// own `<name>/` subdir — that mount is what stops one agent reaching
|
||||||
/// bind-mounts only its own `<name>/` subdir, which is still what stops
|
/// another's socket. The literal lives in `hive-host-sock` (shared with
|
||||||
/// one agent reaching another's socket. The literal lives in `hive-host-sock` (shared with
|
|
||||||
/// `hivectl`); re-exported here under the name this module's consumers
|
/// `hivectl`); re-exported here under the name this module's consumers
|
||||||
/// have always used.
|
/// have always used.
|
||||||
pub use hive_host_sock::AGENT_SOCKET_DIR;
|
pub use hive_host_sock::AGENT_SOCKET_DIR;
|
||||||
|
|
|
||||||
|
|
@ -92,10 +92,9 @@ pub enum InfraContainer {
|
||||||
/// What an [`InfraContainer`] resolves to on the host — i.e. the thing a
|
/// What an [`InfraContainer`] resolves to on the host — i.e. the thing a
|
||||||
/// lifecycle verb actually acts on.
|
/// lifecycle verb actually acts on.
|
||||||
///
|
///
|
||||||
/// The gateway is why this exists: its nginx + dnsmasq were lifted out of
|
/// The gateway is why this exists: its nginx + dnsmasq are host
|
||||||
/// an nspawn container and onto the host, so "restart the gateway" is a
|
/// services, so "restart the gateway" means a plain host unit. Every
|
||||||
/// plain host unit now. The operator verb is unchanged; only its target
|
/// other variant is a container.
|
||||||
/// moved. Everything else is still a container.
|
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
pub enum InfraTarget {
|
pub enum InfraTarget {
|
||||||
/// An nspawn container, controlled via `container@<name>.service` and
|
/// An nspawn container, controlled via `container@<name>.service` and
|
||||||
|
|
@ -433,17 +432,13 @@ pub enum PrivRequest {
|
||||||
/// - otherwise → `systemctl start nginx`
|
/// - otherwise → `systemctl start nginx`
|
||||||
///
|
///
|
||||||
/// Requires root because hive-c0re runs as the unprivileged
|
/// Requires root because hive-c0re runs as the unprivileged
|
||||||
/// `hive-core` user and cannot act on a system unit. It used to be
|
/// `hive-core` user and cannot act on a system unit.
|
||||||
/// root for a *different* reason — `--machine=hive-gateway` entering
|
|
||||||
/// the container's namespace over the machine bus — and that reason
|
|
||||||
/// died with the container: nginx is a host unit now. The
|
|
||||||
/// requirement survived the move; its justification did not.
|
|
||||||
///
|
///
|
||||||
/// ⚠️ The unit name is **not** a parameter and must stay that way.
|
/// ⚠️ The unit name is **not** a parameter and must stay that way.
|
||||||
/// `--machine=` was doing double duty — transport *and* scope — so
|
/// nginx is a host unit, so nothing else narrows what this verb can
|
||||||
/// dropping it removed the containment along with the namespace hop.
|
/// touch: hard-coding `nginx` is the entire containment. A caller
|
||||||
/// Hard-coding `nginx` is what replaces it: a caller cannot name the
|
/// cannot name the unit, so this verb cannot be steered at any other
|
||||||
/// unit, so this verb cannot be steered at any other service.
|
/// service.
|
||||||
ReloadGatewayNginx,
|
ReloadGatewayNginx,
|
||||||
|
|
||||||
// --- Forge admin CLI ---
|
// --- Forge admin CLI ---
|
||||||
|
|
|
||||||
|
|
@ -2186,11 +2186,9 @@ async fn read_container_journal(container: &str, query: &JournalQuery) -> Result
|
||||||
/// - otherwise → `systemctl start nginx`
|
/// - otherwise → `systemctl start nginx`
|
||||||
///
|
///
|
||||||
/// ⚠️ `nginx` is hard-coded on purpose — see `PrivRequest::ReloadGatewayNginx`.
|
/// ⚠️ `nginx` is hard-coded on purpose — see `PrivRequest::ReloadGatewayNginx`.
|
||||||
/// The unit name is the scope of this verb, and it used to be enforced by
|
/// The unit name is the scope of this verb: nginx is a host unit, so no
|
||||||
/// `--machine=hive-gateway` (which could only reach into that container).
|
/// namespace bounds it and the literal is the only thing standing between
|
||||||
/// With nginx on the host there is no namespace to bound it, so the
|
/// "reload the gateway" and "reload anything".
|
||||||
/// literal is the only thing standing between "reload the gateway" and
|
|
||||||
/// "reload anything".
|
|
||||||
///
|
///
|
||||||
/// Returns `(String::new(), String::new())` on success so it fits the
|
/// Returns `(String::new(), String::new())` on success so it fits the
|
||||||
/// `exec` return type directly.
|
/// `exec` return type directly.
|
||||||
|
|
|
||||||
|
|
@ -1,10 +1,9 @@
|
||||||
# Single nginx in front of every hyperhive web surface — dashboard,
|
# Single nginx in front of every hyperhive web surface — dashboard,
|
||||||
# per-agent UIs (sub-path), forge + matrix (sub-domain), .well-known
|
# per-agent UIs (sub-path), forge + matrix (sub-domain), .well-known
|
||||||
# delegations — plus the hive-internal dnsmasq resolver. Both run on the
|
# delegations — plus the hive-internal dnsmasq resolver. Both run on the
|
||||||
# HOST, next to hive-c0re. They used to live in a `hive-gateway`
|
# HOST, next to hive-c0re: nginx binds the host's :80/:443 and dnsmasq
|
||||||
# container that shared the host netns anyway, so the boundary bought no
|
# answers on the hive bridge, so neither can be confined to a network
|
||||||
# network isolation and cost a resolv.conf sync, a reload that had to
|
# namespace of its own.
|
||||||
# cross the machine bus, and four bind mounts.
|
|
||||||
# Full vhost map + discovery flow + design rationale in
|
# Full vhost map + discovery flow + design rationale in
|
||||||
# `docs/gateway.md`. Layout: ./options.nix (option declarations),
|
# `docs/gateway.md`. Layout: ./options.nix (option declarations),
|
||||||
# ./vhosts.nix (the nginx virtual-host tree), ./error-pages.nix
|
# ./vhosts.nix (the nginx virtual-host tree), ./error-pages.nix
|
||||||
|
|
@ -106,11 +105,9 @@ in
|
||||||
];
|
];
|
||||||
|
|
||||||
# Ensure the gateway state dirs exist at host boot, before anything
|
# Ensure the gateway state dirs exist at host boot, before anything
|
||||||
# reads or writes them. They used to double as bind-mount sources
|
# reads or writes them: these rules cover the fresh-boot window
|
||||||
# for the container (nspawn would auto-create a missing one); the
|
# before c0re has run, and pin owner + mode rather than leaving it
|
||||||
# rules stay because they still cover the fresh-boot window before
|
# to whoever creates the path first.
|
||||||
# c0re has run, and they pin owner + mode rather than leaving it to
|
|
||||||
# whoever creates the path first.
|
|
||||||
#
|
#
|
||||||
# /run/hive-agent — per-agent UDS socket dir, written by c0re's
|
# /run/hive-agent — per-agent UDS socket dir, written by c0re's
|
||||||
# set_nspawn_flags when agents start. Owned by `hive-core` (the
|
# set_nspawn_flags when agents start. Owned by `hive-core` (the
|
||||||
|
|
@ -273,11 +270,8 @@ in
|
||||||
# unprivileged and cannot act on a system unit).
|
# unprivileged and cannot act on a system unit).
|
||||||
#
|
#
|
||||||
# It stays an explicit trigger rather than a systemd path unit
|
# It stays an explicit trigger rather than a systemd path unit
|
||||||
# watching the file. That used to be impossible — an IN_MOVED_TO
|
# watching the file: the write and the reload belong in one causal
|
||||||
# from the atomic rename did not cross the nspawn mount-namespace
|
# chain c0re can retry and report on (see RELOAD_PENDING), not two
|
||||||
# boundary — and with one machine it would now work. It is still
|
|
||||||
# not wanted: the write and the reload belong in one causal chain
|
|
||||||
# c0re can retry and report on (see RELOAD_PENDING), not two
|
|
||||||
# independent units racing on an inotify event.
|
# independent units racing on an inotify event.
|
||||||
|
|
||||||
services.nginx = {
|
services.nginx = {
|
||||||
|
|
@ -289,10 +283,9 @@ in
|
||||||
inherit (nginxTree) appendHttpConfig virtualHosts;
|
inherit (nginxTree) appendHttpConfig virtualHosts;
|
||||||
};
|
};
|
||||||
|
|
||||||
# dnsmasq moves with nginx rather than staying behind: it was only in
|
# dnsmasq is a host service alongside nginx, so it reads the host's
|
||||||
# the container because nginx was, and leaving it there would keep the
|
# /etc/resolv.conf directly and picks up network changes as they
|
||||||
# whole resolv.conf sync machine alive for a resolver that no longer
|
# happen — no copy to keep in sync.
|
||||||
# needs it. Host-side it reads the one /etc/resolv.conf directly.
|
|
||||||
services.dnsmasq = import ./dnsmasq.nix {
|
services.dnsmasq = import ./dnsmasq.nix {
|
||||||
inherit
|
inherit
|
||||||
lib
|
lib
|
||||||
|
|
|
||||||
|
|
@ -42,11 +42,9 @@ What keeps that safe is that the directory holds **one** socket. So:
|
||||||
> Pointing nginx at that directory to reach this socket would put the admin
|
> Pointing nginx at that directory to reach this socket would put the admin
|
||||||
> socket within its reach too.
|
> socket within its reach too.
|
||||||
|
|
||||||
This got *less* forgiving when nginx moved onto the host: the gateway used to
|
nginx is a host service, so nothing narrows what it can reach except the
|
||||||
reach a unix upstream through a bind-mount, so the mount list was a second
|
directory itself — that is the whole of the access control. A unit test pins
|
||||||
bound on what it could touch. There is no mount now — the directory is the
|
the default path so a tidying edit fails instead of reviewing cleanly.
|
||||||
whole of the access control. A unit test pins the default path so a tidying
|
|
||||||
edit fails instead of reviewing cleanly.
|
|
||||||
|
|
||||||
`RuntimeDirectoryPreserve=yes` and the daemon's stale-socket unlink on start are
|
`RuntimeDirectoryPreserve=yes` and the daemon's stale-socket unlink on start are
|
||||||
a **pair**: preserving the directory without the unlink means `bind` fails with
|
a **pair**: preserving the directory without the unlink means `bind` fails with
|
||||||
|
|
|
||||||
|
|
@ -29,10 +29,8 @@ use axum::{Router, routing::get};
|
||||||
/// The directory is its own — deliberately not shared with hive-c0re's
|
/// The directory is its own — deliberately not shared with hive-c0re's
|
||||||
/// `/run/hyperhive`. The socket is `0666`, so its directory is the only
|
/// `/run/hyperhive`. The socket is `0666`, so its directory is the only
|
||||||
/// access control it has; co-locating it with c0re's admin socket would
|
/// access control it has; co-locating it with c0re's admin socket would
|
||||||
/// put both within reach of whatever can reach either. That used to be
|
/// put both within reach of whatever can reach either. nginx runs on the
|
||||||
/// enforced by which *directory* was bind-mounted into the gateway
|
/// host, so nothing narrows its reach for you.
|
||||||
/// container; with nginx on the host the mount is gone and the directory
|
|
||||||
/// is all that is left, so the rule matters more, not less.
|
|
||||||
const DEFAULT_SOCKET: &str = "/run/swarm-controller/controller.sock";
|
const DEFAULT_SOCKET: &str = "/run/swarm-controller/controller.sock";
|
||||||
|
|
||||||
fn socket_path() -> PathBuf {
|
fn socket_path() -> PathBuf {
|
||||||
|
|
@ -98,9 +96,8 @@ mod tests {
|
||||||
/// The socket must not share a directory with anything else, because
|
/// The socket must not share a directory with anything else, because
|
||||||
/// the socket is `0666` and the directory is therefore the only access
|
/// the socket is `0666` and the directory is therefore the only access
|
||||||
/// control it has. `/run/hyperhive` in particular holds hive-c0re's
|
/// control it has. `/run/hyperhive` in particular holds hive-c0re's
|
||||||
/// **admin** socket. nginx used to reach a unix upstream by mounting
|
/// **admin** socket, and nginx — a host service — is bounded by
|
||||||
/// the socket's whole directory into the gateway container; it runs on
|
/// nothing but the directory itself.
|
||||||
/// the host now, so nothing narrows its reach but the directory itself.
|
|
||||||
///
|
///
|
||||||
/// A test rather than a comment: the failure this guards against is a
|
/// A test rather than a comment: the failure this guards against is a
|
||||||
/// one-word edit that looks tidier and reads fine in review.
|
/// one-word edit that looks tidier and reads fine in review.
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue