1261b525d6
A swarm runs one homeserver and every hive on it logged in as the same `@hive:` localpart, holding the same access token out of one swarm-wide store path. That is one matrix identity for N hives: the homeserver cannot attribute an action to the hive that took it, and revoking one hive's standing revokes every hive's. Three changes, and the third is the one that makes the other two real: - **The localpart carries the hive's name** (`hive-<hive>`), derived in one place, `swarm_secret_client::matrix::hive_localpart`. `hive-matrix.nix` renders the same string as the appservice registration's `sender_localpart`, so the shared account stops being created rather than merely stops being used. - **The store path is templated by hive**, not a constant. The "a swarm runs one homeserver, so this is a constant rather than a parameter" rationale went with it; it stopped holding the moment two hives shared the homeserver it describes. - **The path moved out from under the grant every hive has.** It sat at `swarm/services/matrix/sender-token`, inside the `secret/data/swarm/services/*` read stanza `policy::render` gives every hive. It now sits under that hive's own stanza, `secret/data/swarm/hives/<hive>/*`, which interpolates the reader's name — so a hive reads its own token and is refused another's. The policy renderer itself is unchanged: narrowing the `services/*` grant would break the OIDC-secret read it exists for, and moving the credential is what this needed instead. A policy test walks the rendered stanzas and asserts none of hive alpha's covers hive beta's sender token, so a later stanza that widened it fails here. `swarm-matrix-ctl` takes a new required `MATRIX_MINT_HIVE` and writes that hive's path; its store grant in `swarm-bao.nix` follows, scoped to one hive's leaf via the new `deploy.bao.matrixCtlHiveName` (defaulting to this host's `hiveName`) rather than a `hives/*` wildcard, which would hand the matrix container every hive's token back. Migration: no outage at deploy. `ensure_hive_user` short-circuits on the local token file, so a hive keeps running on what it has; with no such file it reads the new per-hive path, finds nothing, and falls through to the existing register-or-appservice-login ladder against its own localpart — which needs only the per-hive `as_token` on local disk. The old shared object is read by nothing afterwards. Rooms do not follow the identity, and that is the one operator step; both ways out are written into `docs/integrations/matrix.md`. No admin standing is granted to the per-hive accounts: `admin_execute` stays empty and the assertion pinning it is untouched.
332 lines
28 KiB
Markdown
332 lines
28 KiB
Markdown
# Security model
|
|
|
|
## Agent trust model
|
|
|
|
The sections below document specific mechanisms (the state-file endpoint,
|
|
nixbld isolation, privilege separation). This section frames the model they
|
|
serve: **what hyperhive defends, what it deliberately doesn't, and where the
|
|
operator is accepting risk.** It's the reference for "is it safe to give an
|
|
agent capability X?."
|
|
|
|
### The trust boundary is the container, not credential storage
|
|
|
|
<!-- vale write-good.Passive = NO -->
|
|
|
|
An agent is **trusted code running inside its own nspawn container**. The
|
|
boundary that matters is the container: a sub-agent can't see the host
|
|
netns, another agent's container, or another agent's state dir. Within its
|
|
own container the agent is privileged — it has **passwordless `sudo` by
|
|
default**. Isolating credentials _from the agent itself_ is therefore **not a
|
|
goal**: an agent can read its own tokens, its own `/home/<name>/.claude`, and
|
|
run arbitrary commands as root inside its container. (The narrow exception is
|
|
_cross-tenant_ leakage — for example the unsandboxed-nix-build `0600` token policy
|
|
below stops a build's nixbld user reading the agent's own forge token, and the
|
|
state-file endpoint stops one agent proxying another's files. Those harden the
|
|
boundary; they don't sandbox the agent from itself.)
|
|
|
|
<!-- vale write-good.Passive = YES -->
|
|
|
|
<!-- vale write-good.Passive = NO -->
|
|
|
|
The corollary: **don't reason about security as "can the agent be stopped from
|
|
touching its credentials." Reason about it as "what's the blast radius if this
|
|
agent does the worst possible thing with everything it can reach."**
|
|
|
|
<!-- vale write-good.Passive = YES -->
|
|
|
|
### Scoped tokens bound the blast radius
|
|
|
|
Each agent gets its own scoped credentials, never shared:
|
|
|
|
- **forge token** → that agent's Forgejo account only (its own repos +
|
|
collaborator grants; can't act as another agent or as `core`).
|
|
- **matrix token** → that agent's matrix account only.
|
|
|
|
<!-- vale write-good.Passive = NO -->
|
|
|
|
_Its own_ account's scope bounds a compromised/confused agent's reach on the
|
|
forge or matrix, not the swarm's. This is the main thing standing
|
|
between "one agent does something dumb" and "the whole hive is affected."
|
|
|
|
<!-- vale write-good.Passive = YES -->
|
|
|
|
**Identity vs. secret (matrix).** The scoping is on the _secret_, not the
|
|
_identity_: an agent's matrix **token** is private to its own account, but its
|
|
matrix **identities** — the public handles (`name`, `user_id` `@user:server`,
|
|
`homeserver`) — are intentionally readable by any agent via `GetAgentMeta`, so
|
|
peers can find and address one another on a shared matrix instance. Only the
|
|
public handle crosses that boundary; the token never does.
|
|
|
|
### The swarm secret store isn't a boundary between hives
|
|
|
|
A hive reads its agents' credentials out of the swarm's secret store with its
|
|
own certificate. **Every hive's policy grants read on every agent's
|
|
credentials**, not only on the agents it hosts — so a compromised hive can read
|
|
the matrix token of an agent running on a different hive.
|
|
|
|
That's deliberate and it's the interim state, not the intent. An agent's
|
|
credential path doesn't name the hive hosting it (agents move between hives),
|
|
so a per-hive grant has to be an enumeration the controller re-emits whenever
|
|
the roster changes — and an enumeration that can drift or land out of order
|
|
advertises a boundary it doesn't actually hold. A wide grant that says what it
|
|
is beats a narrow one that only looks narrow.
|
|
|
|
What still holds: the grant is **read-only** (a hive can't write an agent's
|
|
credential, so it can't hand itself an agent's identity), and it reaches three
|
|
prefixes and nothing else in the store — every agent's credentials, the reader's
|
|
_own_ entry under the hive namespace, which names the hive asking and so widens
|
|
nothing between them, and every **swarm service's** OIDC client secret.
|
|
|
|
That third prefix has the same shape of reason as the first, and the same honest
|
|
cost. A swarm service (Grafana and the swarm collector are the two there today)
|
|
registers **one** client for the whole swarm, so its credential's path names
|
|
the service and never the host — and which hive runs a given service is a
|
|
`deploy.*` fact, per-host by definition, so nothing swarm-wide exists to scope
|
|
the grant to. The host running such a service has no store identity of its own
|
|
either; it reads with the certificate of the hive it is. **Any hive can
|
|
therefore read any swarm service's client secret**, which is worth what it
|
|
buys: every swarm service gets its credential the same way from anywhere,
|
|
instead of only where the identity provider happens to sit. Giving such a
|
|
service its own store identity is what would remove this rather than re-scope
|
|
it.
|
|
|
|
**A tracked follow-up** narrows this, with the two candidate directions: scope
|
|
the grant per hive (and pay for the re-emission), or give each agent container
|
|
its own store identity so credentials never pass through a hive at all.
|
|
|
|
### Per-hive credentials live under the hive kind, not beside a service
|
|
|
|
`swarm_secret_client::policy` scopes the **hive** stanza narrowly (unlike the
|
|
agent and service stanzas above) because a hive's own path names it, so
|
|
scoping to the reader's own name costs nothing and drifts nowhere. That only
|
|
holds if a credential that must be one-per-hive is actually stored under the
|
|
hive kind — the matrix sender token was originally published to
|
|
`swarm/services/matrix/sender-token`, under the **service** kind, and so was
|
|
readable by every hive though it belonged to only one. Making it per-hive meant
|
|
moving its path under the hive kind, not narrowing the service stanza's grant.
|
|
**A credential that must be one-per-hive goes under `Kind::Hive`**; putting it
|
|
under `Kind::Service` and expecting the grant to scope it — that's the mistake
|
|
this note exists to stop.
|
|
|
|
### Threat model: prompt injection → confused deputy
|
|
|
|
The realistic adversary **never needs to breach the container**. They supply
|
|
**untrusted input the agent reads and acts on**: a poisoned issue or PR
|
|
comment, a cloned repo's README/CI, a scraped webpage, a crafted matrix
|
|
message. The agent is the trusted, capable party; the _input_ is the
|
|
untrusted part. A successful injection turns the agent into a **confused
|
|
deputy** — it uses its legitimate capabilities (push, comment, deploy, run
|
|
shell) on the attacker's behalf.
|
|
|
|
Mitigations are therefore about **bounding capability and inserting human
|
|
checkpoints**, not about sandboxing the agent from its own tools:
|
|
|
|
- **Operator merges, not the agent** — an agent may _push_ branches, but a
|
|
**human (the operator) merges the PR**, keeping a person in the loop on the
|
|
highest-value action. On the **internal forge this is technically enforced,
|
|
not just convention**: agents can't create repos (`max_repo_creation = 0`),
|
|
so every repo is `core`-created with branch protection **on by default** —
|
|
merges restricted to the operators team + a required operators-team approval
|
|
(`apply_operator_branch_protection` / the config-repo equivalent) — and an
|
|
agent (a write collaborator, not a repo admin) can neither change those
|
|
settings nor merge its own PR. It's **not** set up for external VCS (GitHub
|
|
etc.), though — there, operator-merge is process + accepted risk, not a
|
|
technical control.
|
|
- **Approvals** — config changes, schedule additions, and other
|
|
blast-radius-y operations route through the operator approval queue
|
|
(see [`approvals.md`](../agent-lifecycle/approvals.md)).
|
|
|
|
### Capability = accepted risk
|
|
|
|
Every capability granted to an agent is a risk the operator is **explicitly
|
|
accepting**. The rule of thumb:
|
|
|
|
> **Don't give an agent access to something you can't afford to lose.**
|
|
|
|
If an agent can deploy to prod, you are accepting the risk of a dropped
|
|
production database (via injection or plain error). If that's unacceptable,
|
|
the answer is _don't grant the capability_ — not "grant it and hope the
|
|
sandbox holds," because there is no sandbox between an agent and the tools you
|
|
handed it.
|
|
|
|
### No autosandboxing of external tokens
|
|
|
|
hyperhive provisions and scopes its **own** per-agent forge + matrix tokens.
|
|
It does **not** automatically sandbox or scope **external** credentials
|
|
(GitHub PATs, cloud keys, third-party API tokens). The scope of an external
|
|
token is **operator-accepted risk**: if you drop a broadly scoped GitHub token
|
|
into an agent's config, that agent has exactly that reach, with no hyperhive
|
|
layer narrowing it. Scope external tokens tightly at the source (the external
|
|
provider) before handing them over.
|
|
|
|
## State-file endpoint security model
|
|
|
|
`GET /api/state-file?path=<p>` serves files from agent state dirs and
|
|
the shared space to authenticated dashboard users (browser, operator).
|
|
It accepts two allow-listed root prefixes and rejects all other paths
|
|
before touching the filesystem:
|
|
|
|
- `/var/lib/hyperhive/agents/<n>/state/` — per-agent durable notes
|
|
(canonical host form or the in-container view `/agents/<n>/state/`)
|
|
- `/var/lib/hyperhive/shared/` — shared docs (`/shared/` in-container)
|
|
|
|
`/state/...` without an agent prefix is explicitly _not_ accepted — it's
|
|
ambiguous from the host's perspective.
|
|
|
|
Defense-in-depth layers (in order):
|
|
|
|
1. **Allow-list prefix check** — rejects without touching the filesystem
|
|
if the path doesn't match either root.
|
|
2. **No symlinks below the matched root** — each path component is
|
|
checked with `symlink_metadata` before canonicalize. A sub-agent
|
|
that plants `ln -s /other/secret /agents/me/state/peek` can't proxy
|
|
another agent's file through this endpoint (canonicalize would
|
|
happily resolve the symlink to a still-within-allow-list path).
|
|
3. **Canonicalize as belt-and-braces** — resolves `..`/`.` traversal
|
|
and rejects if the result escapes the roots.
|
|
4. **`state/` subdir constraint** — under `AGENTS_ROOT`, the second
|
|
path component must be `state/`. Applied, proposed git repos and
|
|
config dirs are off-limits.
|
|
5. **World-readable check** — file must have `mode & 0o004` set.
|
|
A `0600` file inside `state/` would otherwise be accessible to any
|
|
operator with dashboard access.
|
|
|
|
`scan_validated_paths` (broker-message ingest, linkifier) uses the same
|
|
`resolve_state_path` helper so security rules stay in sync — the
|
|
dashboard renders anchors only for tokens that passed the same checks the
|
|
read endpoint enforces.
|
|
|
|
The same invariant holds wherever an agent-supplied name reaches a filesystem
|
|
path: the agent socket's `GetAgentMeta` takes `name` as a serde-validated
|
|
`hive_types::Ident` (or falls back to `Ident::parse` for the "self" case)
|
|
before building `agent_notes_dir(name)`, so a `..` component can't traverse.
|
|
|
|
## Nix builds and credential isolation
|
|
|
|
### Background
|
|
|
|
Agent containers bind-mount the host's `nix-daemon` socket. The host daemon may
|
|
have `sandbox-fallback = false` (strict NixOS defaults), which causes `nix build`
|
|
inside nspawn containers to fail — containers lack kernel user namespaces, so nix
|
|
can't set up its build sandbox. the agent modules set `sandbox-fallback = true`
|
|
so that builds fall back to unsandboxed execution rather than failing outright.
|
|
|
|
### Threat model
|
|
|
|
Unsandboxed nix builds run as `nixbld` users (non-root, typically UIDs 30001-30010).
|
|
Without sandbox isolation, a build derivation's builder script has read access to
|
|
any file in the container that the nixbld user can read.
|
|
|
|
The blast radius also has a **network** dimension. hive-ci runs its unsandboxed
|
|
builds of untrusted PR code in its own private netns behind the hive bridge: a
|
|
build reaches the forge only through the gateway and can't reach host-loopback
|
|
services — including the core dashboard at `127.0.0.1:<dashboard_port>`, which
|
|
has no application-layer auth of its own (see [`docs/scheduler/ci.md`](../scheduler/ci.md)). The `0600`
|
|
token policy bounds file reads; network isolation bounds network reach.
|
|
|
|
**What's NOT exposed**:
|
|
|
|
- `/home/<name>/.claude/` — `.credentials.json`, `history.jsonl` and
|
|
`settings.json` are `0600`; `projects/` and `sessions/` are `0700`. All owned
|
|
by the per-agent user `<name>`, so nixbld users can't read any of them.
|
|
⚠️ The **directory itself is `0755`**, on purpose: `hive-core` is a different
|
|
user and needs read+execute to list it so `claude_has_session` can detect a
|
|
valid session (`ensure_claude_dir`, `hive-c0re/src/lifecycle/setup.rs`).
|
|
The per-file mode is therefore the whole protection here — anything added to
|
|
this directory at a default mode is world-readable, which isn't
|
|
hypothetical: `plugins/` and `.last-cleanup` already are.
|
|
- `$HYPERHIVE_STATE_DIR/forge-token` (= `/agents/<name>/state/forge-token`)
|
|
— written at mode `0600` and chowned to the per-agent uid:gid (see
|
|
`hive-c0re/src/forge/mod.rs`'s module doc for exactly where). nixbld users
|
|
can't read it.
|
|
|
|
**Policy**: all credential files written to agent state directories MUST be mode
|
|
`0600` or stricter. Don't create world-readable secret files in agent state dirs.
|
|
|
|
### Long-term fix
|
|
|
|
The proper fix is to enable user namespaces inside nspawn containers
|
|
(`--private-users=inherit` in `EXTRA_NSPAWN_FLAGS`) so nix can set up its real
|
|
sandbox and `sandbox-fallback` becomes a true last resort. This requires verifying
|
|
bind-mount compatibility with user namespace UID mapping first.
|
|
|
|
## hive-c0re privilege separation
|
|
|
|
### Background
|
|
|
|
`hive-c0re` runs as the unprivileged system user `hive-core`
|
|
(`/var/lib/hyperhive` owned by `hive-core:hive-core`). It can't
|
|
directly invoke `nixos-container`, `journalctl -M`, or act on a system
|
|
unit (`systemctl reload nginx`) — those require root. `hive-priv` fills
|
|
this gap.
|
|
|
|
⚠️ **`ReloadGatewayNginx` acts on a host unit, so nothing implicitly
|
|
scopes it.** Its containment is the unit name hard-coded in `hive-priv`:
|
|
a caller can't name the unit, so it can't steer the verb at another
|
|
service. **A privileged verb needs something bounding what it can act
|
|
on; when that isn't a namespace, it has to be a constant the caller
|
|
can't supply.**
|
|
|
|
### hive-priv
|
|
|
|
`hive-priv` is a minimal privileged helper that runs as root, socket-activated
|
|
at `/run/hive/priv.sock` (mode `0660`, group `hive-core` — only the
|
|
`hive-core` user can connect). `hive-c0re` calls it via `priv_client`
|
|
for every operation that genuinely requires root.
|
|
|
|
**Narrow interface** — `PrivRequest` variants map 1:1 to specific
|
|
known operations; there is no arbitrary command pass-through:
|
|
|
|
| Operation | What it runs |
|
|
| -------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `StartContainer` / `StopContainer` | `nixos-container start/stop <name>` |
|
|
| `KillContainer` | `machinectl kill <machine> --signal=SIGKILL` (`nixos-container` has no kill verb) |
|
|
| `CreateContainer` / `UpdateContainer` | `nixos-container create/update <name> --flake <ref>` |
|
|
| `DestroyContainer` | `nixos-container destroy <name>` |
|
|
| `ListContainers` | `nixos-container list` |
|
|
| `ReadContainerJournal` | `journalctl -M <container> -n <n> [filters...]` |
|
|
| `ReloadGatewayNginx` | `systemctl reload/start/reset-failed nginx` (host unit; the unit name is hard-coded, not a parameter) |
|
|
| `WriteNspawnFlags` | write `/etc/nixos-containers/<container>.conf` (bind-mount list + network isolation vars) |
|
|
| `WriteResourceLimits` | write `CPUQuota=`/`MemoryMax=`/`CPUWeight=`/`IOWeight=` systemd drop-in for agent container |
|
|
| `RemoveServiceDropin` | remove `container@<name>.service.d/` drop-in on destroy |
|
|
| `DaemonReload` | `systemctl daemon-reload` |
|
|
| `RunForgeAdmin` | `nixos-container run hive-forge -- runuser -u forgejo -- forgejo admin <args>` |
|
|
| `WriteAgentForgeToken` / `WriteAgentMatrixToken` | write `0600` credential file into agent state dir |
|
|
| `RestartMatrixDaemon` | `systemctl --machine=h-<name> restart hive-matrix-daemon.service` |
|
|
| `ControlInfraContainer` | `systemctl <action> container@<container>.service` — the `InfraContainer` enum is the allowlist, and serde rejects unknown names at the wire boundary (`hive-c0re` has no variant, so no request can name it) |
|
|
| `SyncAgentTmpfiles` | write `/etc/tmpfiles.d/hyperhive-agents.conf` for the agent set, then `systemd-tmpfiles --create` |
|
|
| `SetAgentPaused` | create / remove the `<state>/<name>/harness/paused` marker that parks an agent's turn loop |
|
|
| `WriteAgentGithubToken` | write `0600` `github-token` into agent state dir (same semantics as the forge/matrix token writes) |
|
|
| `WriteAgentExtraForgeAccount` / `DeleteAgentExtraForgeAccount` | write / remove `forge-<label>-token` + a `forge-<label>.json` base-URL sidecar, both `0600`. hive-priv validates `label` as a plain identifier before it reaches the filename — an unchecked one traverses out of the state dir |
|
|
| `RegisterCiRunner` | write `/run/hive-ci/runner-token` (host path, root-owned) then `systemctl --machine=hive-ci restart gitea-runner-hive.service`. Only the registration token crosses; the forge admin token never enters the container |
|
|
| `EnsureAgentSubvolume` | `btrfs subvolume create <state>/<name>` for a new agent — no-op when the path exists or the filesystem isn't btrfs |
|
|
| `UpgradeAgentSubvolume` | migrate an existing plain state dir into a subvolume: create, `cp -a --reflink=auto`, atomic swap. Operator opt-in, and the caller stops the agent first |
|
|
| `DeleteAgentSubvolume` | `btrfs subvolume delete <state>/<name>` — purge path only, never a plain destroy |
|
|
| `EnsureBtrfsQuota` | `btrfs quota enable <AGENT_STATE_ROOT>`. Operator opt-in — enabling forces a full rescan |
|
|
| `ReadSubvolumeUsage` | `btrfs qgroup show -f --raw <state>/<name>` |
|
|
| `SetSubvolumeQuota` | `btrfs qgroup limit <bytes\|none> <state>/<name>` |
|
|
| `SnapshotAgentSubvolume` | `btrfs subvolume snapshot -r <agent_root> <snapshot_path>` — the frozen point-in-time copy a migration streams from |
|
|
| `DeleteAgentSnapshot` | `btrfs subvolume delete <snapshot_path>` |
|
|
| `SendAgentSnapshotToFile` | `btrfs send [-p <parent>] <snapshot>` into a bare filename under `MIGRATE_STAGING_ROOT`; refuses to overwrite an existing export |
|
|
| `SendAgentSnapshotToFd` | `btrfs send [-p <parent>] <snapshot>` into a **file descriptor passed with the request** (`SCM_RIGHTS`). hive-c0re connects to the peer and hands over the connected socket, so hive-priv never learns an address or protocol. hive-priv requires exactly one descriptor here, and refuses one that arrives alongside any other operation |
|
|
|
|
**Container allowlist** — `hive-priv` validates every request against
|
|
an allowlist before any operation: the allowlist accepts only names
|
|
matching the agent-name convention (char-validated) or the known
|
|
sibling service containers (`hive-forge`, `hive-matrix`, `hive-ci`),
|
|
and rejects arbitrary container names. `hive-gateway` is a host unit,
|
|
not a container, so it's not in this list — see `ReloadGatewayNginx`
|
|
above for how the hard-coded unit name scopes its access instead.
|
|
|
|
**Socket-activated** — systemd starts `hive-priv` on the first
|
|
incoming connection (`LISTEN_FDS=1`); it's not running between calls.
|
|
The `ProtectSystem=strict` + `ReadWritePaths` sandbox limits filesystem
|
|
writes to only the paths `hive-priv` legitimately needs.
|
|
|
|
### Privilege boundary summary
|
|
|
|
| Component | Runs as | Privilege needed for |
|
|
| ---------------------------- | -------------- | ------------------------------------------------------------ |
|
|
| `hive-c0re` | `hive-core` | broker, HTTP dashboard, scheduling, approvals |
|
|
| `hive-priv` | `root` | container lifecycle, journal reads, bind mounts, cred writes |
|
|
| `hive-ag3nt` (per-container) | per-agent user | turn execution, MCP serving |
|