hyperhive/docs/4-implementation/trust-boundary/nix-builds.md

# Nix builds and credential isolation

The mechanism and threat model for `nix build` inside an agent container, from
the [security model](../../2-handbook/trust-boundary/security.md).

## Background

nixos-containers' start script bind-mounts the host's `nix-daemon` socket
directory (`/nix/var/nix/daemon-socket`) into every container, hyperhive's
agents included; hyperhive adds no bind for it. The host daemon may
have `sandbox-fallback = false` (strict NixOS defaults), which causes `nix build`
inside nspawn containers to fail — containers lack kernel user namespaces, so nix
can't set up its build sandbox. the agent modules set `sandbox-fallback = true`
so that builds fall back to unsandboxed execution rather than failing outright.

## Per-agent nix builds mode

`services.hyperhive.agent.nixBuilds` picks where an agent's nix builds run. It
has two values, and defaults to `host`:

- **`host`**: builds go to the host daemon through the socket above. The daemon
  forks them in its own cgroup, so they run **outside the agent container's
  CPU and memory caps**, which sit on `container-<name>.service`.
- **`none`**: the container has no route to the host daemon. hive-priv adds
  `--tmpfs=/nix/var/nix/daemon-socket` to the container's nspawn flags. nspawn
  mounts its custom mounts sorted by destination, then by kind (a bind before
  a tmpfs), so the empty tmpfs lands on top of the start script's bind
  whatever order the flags are in, and the container finds no `socket` in the
  directory. `NIX_REMOTE=daemon` stays, so a build fails with `cannot connect
to socket at '/nix/var/nix/daemon-socket/socket'`, which names the cause. The
  agent's `/nix/store` and its read-only nix database stay bound, so store
  paths already there stay readable; only a realisation of a new path, or any
  `nix` command that needs the daemon, fails.
  The agent's own NixOS config turns its `nix-daemon` socket and service off in
  this mode: the tmpfs is writable, so the unit would otherwise listen on it.
  Under `host` the directory is the host's read-only bind, which the socket
  unit's `ConditionPathIsReadWrite` rejects, so that unit never starts.

The mode reaches the container as `privateUsers` does: the swarm builder reads
the option from the evaluated config into the agent's closure record
(`nix_builds`), hive-c0re reads the record when it writes the container's
flags and sends the mode to hive-priv, and the flags apply at the container's
next start.

- An agent whose module doesn't declare the option reads `host`. When the builder
  can't read the option (its eval fails or times out), the record carries no
  mode, and so does a record or builder from before the field. hive-c0re then
  sends no mode, and hive-priv keeps what the container's last write chose.
- A mode this build doesn't know (a record written by a newer swarm) reads as
  `none`, never `host`: an agent that asked for a mode its hive can't give gets
  no builds, not builds outside its limits.
- A record hive-c0re can't read, or none, sends no mode either, and hive-priv
  keeps what the container's last write chose.
- A hive-c0re or hive-priv from before the option ignores it, so the agent keeps
  host builds until you upgrade both.

`none` cuts the daemon only. It leaves the agent's egress alone.

## Threat model

Unsandboxed nix builds run as `nixbld` users (non-root; `nrBuildUsers` is at least 32, so UIDs 30001 and up).
Without sandbox isolation, a build derivation's builder script has read access to
any file in the container that the nixbld user can read.

The blast radius also has a **network** dimension. `swarm-ci-runner` runs its
unsandboxed builds of untrusted PR code in its own private netns behind the
bridge: a build reaches the forge by name and can't reach host-loopback
services — including the core dashboard at `127.0.0.1:<dashboard_port>`, which
has no application-layer auth of its own (see [`docs/4-implementation/scheduler/ci.md`](../../4-implementation/scheduler/ci.md)). The `0600`
token policy bounds file reads; network isolation bounds network reach.

**What's NOT exposed**:

- `/home/<name>/.claude/` — `.credentials.json`, `history.jsonl` and
  `settings.json` are `0600`; `projects/` and `sessions/` are `0700`. All owned
  by the per-agent user `<name>`, so nixbld users can't read any of them.
  ⚠️ The **directory itself is `0755`**, on purpose: `hive-core` is a different
  user and needs read+execute to list it so `claude_has_session` can detect a
  valid session (`ensure_claude_dir`, `hive-c0re/src/lifecycle/setup.rs`).
  The per-file mode is therefore the whole protection here — anything added to
  this directory at a default mode is world-readable, which isn't
  hypothetical: `plugins/` and `.last-cleanup` already are.
- `$HYPERHIVE_STATE_DIR/forge-token` (= `/agents/<name>/state/forge-token`)
  — the token hive-c0re wrote at mode `0600`, chowned to the per-agent
  uid:gid, before swarm-controller took over minting. Nothing writes it any
  more; it stays as the fallback for an agent without a store identity.
  nixbld users can't read it. The swarm-minted token lives under
  `/run/hive-agent-forge-token/` (`0700` directory, `0400` file), outside
  the state dir.
- `/run/hive-agent-proxy-credential/proxy.env` — the agent's egress-proxy
  credential inside its proxy URL (`0700` directory, `0600` file, tmpfs). Units
  load it as an environment file, so the credential is also in the environment of
  every process they start. It authenticates this agent to the hive's squid and
  nothing else.

**Policy**: all credential files written to agent state directories MUST be mode
`0600` or stricter. Don't create world-readable secret files in agent state dirs.

## Long-term fix

The proper fix is to enable user namespaces inside nspawn containers
(`--private-users=inherit` in `EXTRA_NSPAWN_FLAGS`) so nix can set up its real
sandbox and `sandbox-fallback` becomes a true last resort. This requires verifying
bind-mount compatibility with user namespace UID mapping first.