6ac637c7e3
No behaviour change: a pure move plus pub(crate) visibility. PrivRequest, hive-priv-sock and every validation rule are untouched; the 58 tests move next to the code they test, unchanged. Why (#5476): the 28 root ops shared one 4.3k-line file, so no op family could be reviewed on its own and every new root op landed there by default. main.rs keeps the socket loop and the dispatch table. container (lifecycle), container_config (nspawn flags, systemd drop-ins, daemon-reload), storage (btrfs subvolumes, quotas, snapshots, du), files (pause marker, socket dir, legacy tmpfiles) and infra (gateway nginx, forge admin, infra containers) each carry their own validation; shared holds the agent-name checks, the credential-name charset and StagedFile, plus the tests' scratch helpers (one counter crate-wide, so test dirs stay unique). hive-priv/README.md is in the diff because it named src/main.rs as the home of container_flake_action and now describes the module layout.
101 lines
5.6 KiB
Markdown
101 lines
5.6 KiB
Markdown
# hive-priv
|
|
|
|
The minimal **root privileged-helper** for hive-c0re. It runs as root and
|
|
exposes a narrow unix socket at `/run/hive/priv.sock` that accepts `PrivRequest`
|
|
JSON lines and performs only the handful of operations that genuinely require
|
|
root — bind-mount edits, `nsenter` into a container, btrfs subvolume ops. All
|
|
coordination logic (broker, HTTP, scheduling) stays in the _unprivileged_
|
|
`hive-c0re` process, which delegates here.
|
|
|
|
## Why it exists
|
|
|
|
Privsep. `hive-c0re` runs as the unprivileged `hive-core` user so a bug or a
|
|
prompt-injection in the large daemon can't directly wield root. The few root
|
|
operations it needs are funnelled through this small, auditable helper instead.
|
|
See `docs/trust-boundary/boundary.md` and `docs/trust-boundary/security.md` for the privilege boundary.
|
|
|
|
## Security model
|
|
|
|
- **Strict allowlist.** hive-priv checks every request against a container-name
|
|
allowlist before any filesystem or process operation, and accepts only names
|
|
matching the hive convention (`h-*`, the manager container included).
|
|
- **No pass-through.** Every `PrivRequest` variant maps to a single known
|
|
operation; there is no arbitrary-command escape hatch.
|
|
- **Socket-activated, always.** systemd binds `/run/hive/priv.sock`
|
|
(`SocketGroup=hive-core`, `0660`) and passes the listener as fd 3
|
|
(`LISTEN_FDS`); the helper requires this and has no self-bind fallback, so dev
|
|
and prod take the identical path and the group grant always holds.
|
|
|
|
The wire contract (`PrivRequest` / response types) lives in the separate
|
|
`hive-priv-sock` crate so this root binary depends on just the protocol shapes,
|
|
not the whole daemon-shared crate.
|
|
|
|
## Layout
|
|
|
|
`src/main.rs` holds the socket loop and the dispatch table, one arm per
|
|
`PrivRequest` variant. Each op family, with its validation, has its own module:
|
|
`container` (lifecycle), `container_config` (nspawn flags, systemd drop-ins),
|
|
`storage` (btrfs subvolumes, quotas, snapshots, disk measurement), `files`
|
|
(pause marker, socket dir, legacy tmpfiles) and `infra` (gateway nginx, forge
|
|
admin, infra containers). Helpers more than one family calls live in `shared`.
|
|
Each module's `//!` header lists its ops.
|
|
|
|
## Implementation notes
|
|
|
|
### Container toplevel builds (create/update)
|
|
|
|
`container_flake_action` (in `src/container.rs`) builds
|
|
`nixosConfigurations.<name>.config.system.build.toplevel` itself
|
|
(`nix_build_toplevel`) and passes the resolved store path to
|
|
`nixos-container create`/`update` via `--system-path`, for both verbs.
|
|
|
|
**Why not let `nixos-container` build it (the old `create`-only
|
|
behaviour, `update` used its own `--flake` path)**: `nixos-container`'s
|
|
own `buildFlake()` — invoked whenever `--system-path` isn't passed —
|
|
builds to a _hardcoded relative path_. `$systemPath` is only ever
|
|
assigned from the CLI flag or from `buildFlake()`'s own result, so with
|
|
no flag it stays `undef` and `"$systemPath.tmp"` interpolates to the
|
|
bare string `.tmp` in whatever the caller's cwd happens to be.
|
|
`buildFlake()` itself takes no lock at all: `create` wraps its _whole
|
|
action_ in an exclusive `flock` before calling it, but `update` used to
|
|
call it with no lock whatsoever — so `create`'s lock never protected
|
|
against a concurrent `update` clobbering the same `.tmp`. hive-priv never
|
|
sets a per-call cwd, so with `services.hyperhive.c0re.buildSlots > 1`,
|
|
two concurrent calls (any mix of `create`/`update`) could share that one
|
|
`.tmp`: one's `readlink(".tmp")` resolving to the _other's_ build
|
|
output, handing an agent's container the wrong agent's closure — the
|
|
"agent container gets closure of other agent" bug.
|
|
|
|
Building the toplevel here and passing the resolved store path via
|
|
`--system-path` for _every_ call means `buildFlake()` never runs at all,
|
|
for either verb — no shared `.tmp` left to race on, no locking invariant
|
|
of a script we don't own to keep track of. `--no-link` avoids a
|
|
competing out-link race of our own; we only need the store path, not a
|
|
GC root (it's safe from collection for as long as it takes
|
|
`nixos-container` to register it against the container's own profile,
|
|
same window every other `--print-out-paths` consumer already relies on).
|
|
|
|
**Forwards stderr live, captures stdout silently — deliberately not
|
|
symmetric.** This build is the multi-minute phase of a `create`/
|
|
`update`, and it used to run _inside_ `nixos-container`'s own `--flake`
|
|
invocation, which streams every line to the caller in real time.
|
|
Buffering it instead (`Command::output()`, as this function first
|
|
shipped) regressed that: nothing on the wire — dashboard or
|
|
`journalctl -f` alike — until the whole build finishes, then everything
|
|
at once. So both pipes are drained concurrently (needed to avoid
|
|
deadlocking if either pipe fills while the other is being read), but
|
|
only stderr — where nix's own progress goes — is forwarded live, same
|
|
shape as `container_run_streaming`. stdout is different:
|
|
`--print-out-paths` writes _only_ the final store path there, once, at
|
|
the end — forwarding it the same way would risk interleaving a progress
|
|
line into the value this function hands back as `--system-path`, trading
|
|
a closure-mixup bug for a corrupted-argument one. So stdout lines are
|
|
accumulated silently and only consulted after the exit status is known
|
|
to be success — and even then, exactly one non-empty, trimmed line is
|
|
required (`nix build --print-out-paths` prints one line _per output_,
|
|
not one line total; `config.system.build.toplevel` is single-output
|
|
today, but a bare whole-buffer `.trim()` would silently hand a
|
|
multi-line string to `--system-path` the day that ever changes, and a
|
|
bare untrimmed/unfiltered `.lines()` turns a lone `"\n"` into a bogus
|
|
empty-string "path" — a wrong line count, or a blank/whitespace one,
|
|
`bail!`s instead).
|