docs: restructure into topic subdirectories, collapse duplicated index
Per mara's go-ahead on hyperhive#3902 ("getting started is good, but
terminal rendering does not go in there i think"):
Moved 21 top-level docs/*.md files into 7 new topic subdirectories
(existing web-ui/, turn-loop/, swarm/, tools/, crates/ untouched):
getting-started/ setup.md
agent-lifecycle/ agent-hierarchy.md, approvals.md, persistence.md
trust-boundary/ boundary.md, security.md
integrations/ forge.md, matrix.md, github.md, knowledge.md
networking/ gateway.md, network.md, snapshot-store.md
scheduler/ jobq.md, coordinator.md, ci.md, observability.md
process/ conventions.md, gotchas.md, pr-review-gate.md
web-ui/ terminal-rendering.md (moved into the EXISTING dir,
per mara's correction to the original getting-started
guess -- it's UI implementation detail, not onboarding)
The physical layout now matches docs/README.md's own topical headers,
which already amounted to this taxonomy -- see the scoping comment on
the issue for the two findings that motivated this (a genuine
duplication between CLAUDE.md's old "Reading paths" list and
docs/README.md's grouped one, since drifted out of sync with each
other; and the flat layout not matching the grouping we already had).
Fixed every cross-reference this moved across the whole repo (~120
files: docs/ internal links at every depth, Rust doc comments, nix
module option docs, crate READMEs) -- verified two ways: a grep sweep
confirming zero remaining references to any old path, and a script
that resolves every markdown link in docs/**/*.md + CLAUDE.md +
README.md against the filesystem and reports anything that doesn't
exist (zero broken links).
Collapsed CLAUDE.md's "Reading paths" section (the duplicate) down to
a pointer at docs/README.md, now the single index. Rewrote
docs/README.md itself to use the new subdirectory paths and added the
one doc it was missing that CLAUDE.md's old copy had (pr-review-gate.md).
Classified all 22 docs/*.md files first via a haiku subagent (mara's
suggestion) on two axes -- proposed grouping and operator-vs-
implementation focus -- before finalizing the taxonomy; spot-checked
the report and found internal inconsistencies (its classification
table disagreed with its own summary section for a few files), so this
taxonomy is my original proposal + the one correction mara gave
directly, not a blind application of the subagent's table. The
operator-focus data it gathered is still useful for a follow-up
content pass (docs skewing 'mixed' rather than pure operator-facing),
not addressed in this PR -- structure only.
nix fmt clean, both pre-push lints clean.
This commit is contained in:
parent
e4a22b4190
commit
07b62612b0
124 changed files with 301 additions and 377 deletions
563
docs/process/conventions.md
Normal file
563
docs/process/conventions.md
Normal file
|
|
@ -0,0 +1,563 @@
|
|||
# Conventions
|
||||
|
||||
Code-style and process expectations across the workspace. Most of these
|
||||
exist because something already went wrong without them.
|
||||
|
||||
## Naming
|
||||
|
||||
- Containers are length-bounded by `nixos-container` (≤ 11 chars).
|
||||
- Sub-agents are `h-<name>` with `<name>` ≤ 9 chars.
|
||||
- One agent is the bootstrap/root container, with a fixed name (`ruth` today).
|
||||
- `MAX_AGENT_NAME` in `hive-c0re/src/lifecycle/mod.rs` enforces the cap.
|
||||
- Per-agent web UI port = `WEB_PORT_BASE + FNV1a(name) % WEB_PORT_RANGE`
|
||||
(8100..8999) for every agent; dashboard
|
||||
`cfg.dashboardPort` (default 7000).
|
||||
|
||||
## Hive identity (label + domain + display names)
|
||||
|
||||
Four env vars cover the identity surface, read by
|
||||
`hive-agent/src/identity.rs`:
|
||||
|
||||
- `HIVE_LABEL` — short, hive-local agent label (`iris`,
|
||||
`damocles`). `label()` returns it; falls back to empty string if
|
||||
the env var is missing so downstream callers can decide how to
|
||||
surface "unknown agent" rather than getting a panic from this
|
||||
module.
|
||||
- `HYPERHIVE_HIVE_DOMAIN` — the hive's canonical DNS domain (e.g.
|
||||
`darkest.space`), set by `nix/host-modules/hive-c0re/environment.nix`
|
||||
from `services.hyperhive.domain`. When configured, `qualified_label()`
|
||||
returns `${label}@${domain}` (e.g. `iris@darkest.space`); when
|
||||
unset (single-hive deployments, dev/test) it degrades to just
|
||||
the short label so existing callers see no change. The
|
||||
qualified form surfaces in the per-agent web UI title, the
|
||||
system-prompt template, and `/api/state.qualified_label`.
|
||||
- `HYPERHIVE_HIVE_NAME` — human display name of this hive
|
||||
(`pr1ma`). Read by `hive_name()`; `None` when unset.
|
||||
- `HYPERHIVE_SWARM_NAME` — human display name of the wider swarm
|
||||
this hive belongs to (`constellat1on`). Read by `swarm_name()`;
|
||||
federated hives at different DNS domains can share a swarm
|
||||
name.
|
||||
|
||||
`hive_name` + `swarm_name` are **distinct** from
|
||||
`HYPERHIVE_HIVE_DOMAIN`: the domain may carry the hive name as
|
||||
its leftmost label by convention, but the convention isn't
|
||||
machine-readable, and federated hives at different DNS domains
|
||||
can share a swarm name. Humans want both: the address
|
||||
(`@darkest.space`) AND the prose name (`pr1ma`). Matrix MXIDs
|
||||
still use the domain-based convention untouched.
|
||||
|
||||
`qualify(label)` is the same shape as `qualified_label()` but
|
||||
applies to an arbitrary label the caller already has (e.g. a peer
|
||||
name from the broker); it's the right surface when rendering a
|
||||
peer's name when the caller knows it's hive-local.
|
||||
|
||||
## Identity = socket
|
||||
|
||||
There are no auth tokens on the per-agent unix sockets. The socket
|
||||
*path* identifies the principal; perms come from "who has the
|
||||
bind-mount." A sub-agent only sees its own `/run/hive/mcp.sock`;
|
||||
hive-c0re owns the host admin socket.
|
||||
|
||||
### Wake injection
|
||||
|
||||
`AgentRequest::Wake { from, body }` (and the manager-flavour mirror)
|
||||
is the wake-event-injection surface. Recipient is implicit — the
|
||||
agent the socket belongs to — and `from` is caller-chosen so the
|
||||
wake prompt can label the source verbatim (`"matrix: new message in
|
||||
#general"`, `"forge: PR #42 opened"`, etc.). Typical caller: an
|
||||
in-container background task (the matrix daemon, a scraper, the
|
||||
forge-notify webhook subscriber) that needs to signal "external work
|
||||
has arrived" without going through the broker as a peer agent.
|
||||
|
||||
Identity = socket means anything that can connect to
|
||||
`/run/hive/mcp.sock` is implicitly trusted to inject wakes. That's
|
||||
fine: the bind-mount only exposes the socket inside the agent's own
|
||||
container, so the trust boundary is the container's process
|
||||
namespace, not the wire surface.
|
||||
|
||||
## Recipient sentinels
|
||||
|
||||
A few recipient names are reserved by the broker and have special
|
||||
meaning that ordinary agent labels can never collide with — agent
|
||||
name validation rejects any character outside `[a-z0-9_-]`, so the
|
||||
angle-bracket and asterisk shapes below are structurally safe.
|
||||
|
||||
- `*` — broadcast: deliver to every running agent except the sender
|
||||
(`socket_server::handle_send` fans out via `Coordinator::broadcast_send`).
|
||||
- `operator` — the human at the dashboard. Messages accumulate in the
|
||||
inbox view; no agent ever `recv`'s them.
|
||||
- `<parent>` — the sender's parent per `topology.json`. Rewritten at
|
||||
send time by `topology::resolve_recipient`: looks up
|
||||
`parent_of(sender)` and falls back to `operator` when the sender is
|
||||
a root agent (or absent from topology entirely). Lets agents address
|
||||
their parent without learning the label, so runtime reparenting
|
||||
propagates with zero agent-side restart.
|
||||
- `<children>` — fan-out to every direct descendant of the sender per
|
||||
`topology.json`. Resolved in `socket_server::handle_send` via
|
||||
`topology::children_of(sender)`: one message is delivered to each
|
||||
child, bypassing the allow-list check (structural fan-out targets are
|
||||
never user-listed peers). No-op for leaf agents (returns `Ok` when the
|
||||
child set is empty). Lets a sub-manager nudge its subtree without
|
||||
enumerating labels.
|
||||
|
||||
When a `<children>` or `<parent>` send resolves to real recipients, the
|
||||
broker stores the *resolved* label(s) as the message recipient(s) — the
|
||||
dashboard and recv side see the real routes. The sentinels are purely
|
||||
send-time addressing conveniences.
|
||||
|
||||
## Wire protocol
|
||||
|
||||
JSON line-delimited over unix sockets in both directions (host admin
|
||||
/ manager / agent). SSE streams (`/dashboard/stream` on hive-c0re,
|
||||
`/events/stream` on the per-agent web UIs) are `text/event-stream`;
|
||||
each frame carries a `seq` field for the snapshot-dedupe dance
|
||||
(see `docs/web-ui.md`). Request/response types live in `hive-sh4re`
|
||||
— change them in one place. The dashboard event vocabulary lives
|
||||
in `hive-c0re::dashboard_events::DashboardEvent`.
|
||||
|
||||
### Broker delivery + ack cycle
|
||||
|
||||
`AgentRequest::Recv` is the only path that delivers messages to an
|
||||
agent. Always returns a list (`Messages { messages }`) — empty when
|
||||
nothing's pending, single-pop when `max = None` (default 1, the
|
||||
single-message behaviour), batched up to `max` when caller asks for
|
||||
more (server-side cap is 5; values above clamp silently). The wire
|
||||
request still carries an optional `wait_seconds` (long-poll the first
|
||||
message, once one arrives — or one is already pending — the call
|
||||
drains up to `max` in total): the harness's own turn-driving loop
|
||||
uses it internally (`hive-agent`'s `recv_next`, 180s). The
|
||||
agent-facing MCP `recv` tool does not expose this parameter — it
|
||||
always passes `wait_seconds: None`, an immediate peek.
|
||||
|
||||
Per-row bookkeeping inside the broker:
|
||||
|
||||
- `delivered_at = NOW` set on every popped row.
|
||||
- Each recipient has an in-memory `unacked_ids` list of every row
|
||||
delivered since the last `AckTurn`.
|
||||
- `redelivered = true` on a row if `RequeueInflight` resurfaced it
|
||||
(the harness prepends a "may already be handled" hint when this
|
||||
flag is set so the per-message warning is visible).
|
||||
|
||||
`AgentRequest::AckTurn` closes out the in-memory list — the harness
|
||||
fires it after `TurnOutcome::Ok`, marking every message popped since
|
||||
the last ack as fully handled. Claude doesn't see this surface; it's
|
||||
strictly a harness↔broker pairing. On `TurnOutcome::Failed` the
|
||||
harness intentionally skips the ack so the unacked rows stay
|
||||
in-flight in the DB and get picked up by the next requeue sweep.
|
||||
|
||||
`AgentRequest::RequeueInflight` is the recovery pair: fired by the
|
||||
harness exactly once at boot, before the serve loop starts. Catches
|
||||
the crashed-mid-turn / OOM-killed / container-restarted cases where
|
||||
a previous harness session popped messages but never drove them to
|
||||
a clean turn-end. Resets `delivered_at` back to NULL on every
|
||||
unacked row (so the next `Recv` pops them again), and remembers
|
||||
each id in a per-recipient in-memory set so the next `Recv` can tag
|
||||
the row with `redelivered: true`. Idempotent + cheap when there's
|
||||
nothing in flight, so the at-boot fire is unconditional.
|
||||
|
||||
`AgentRequest::AckUntil { up_to }` is the agent-facing bulk-triage
|
||||
escape hatch (`mcp__hyperhive__ack_until`). Unlike `AckTurn` it IS
|
||||
visible to claude: each recv row and wake prompt carries a
|
||||
`[msg #<id>]` marker (the broker row id; transient pings show no
|
||||
marker — their sentinel id 0 has nothing to ack), and
|
||||
`ack_until(up_to: n)` marks every one of the agent's rows with
|
||||
`id <= n` handled in a single UPDATE — pending and delivered alike.
|
||||
This bounds the redelivered-flood cost after a restart: instead of
|
||||
popping dozens of already-handled messages one turn at a time, the
|
||||
agent notes the highest id it has seen and acks up to it.
|
||||
Recipient-scoped (an agent can only ack its own rows); also drains
|
||||
the in-memory `unacked_ids` / `requeued_ids` bookkeeping below the
|
||||
cutoff so a later `AckTurn` doesn't double-update and a stale
|
||||
redelivery tag can't outlive its row. The operator-side sibling is
|
||||
the dashboard's "mark all read" (unbounded, per-agent).
|
||||
|
||||
### Loose-ends wire shape
|
||||
|
||||
`LooseEnd` is the per-row response shape for `GetLooseEnds` (both
|
||||
the agent-flavour and manager-flavour requests). Tagged enum so
|
||||
new thread kinds (forge PRs, long-running approvals from a
|
||||
privileged bot, etc.) can land later without breaking existing
|
||||
handlers. Each row carries enough context that the caller renders
|
||||
it directly as a bulleted list, no follow-up fetch needed.
|
||||
|
||||
Per-flavour scoping is uniform across the two cancellable variants:
|
||||
|
||||
- **agent-flavour** `GetLooseEnds` only surfaces rows the calling
|
||||
agent has standing in. `Approval` rows only appear when the
|
||||
calling agent is the manager (sub-agents don't submit
|
||||
approvals). `Reminder` rows are scoped to `owner == self`.
|
||||
- **manager-flavour** `GetLooseEnds` lists every pending row in
|
||||
the swarm — full audit view.
|
||||
|
||||
Per-variant fields:
|
||||
|
||||
- `Approval { id, agent, commit_ref, description?, age_seconds }`
|
||||
— `agent` is the affected agent (target of the spawn / config
|
||||
commit), not the asker. `description` is the manager's free-text
|
||||
blurb shown on the dashboard card. `commit_ref` is the
|
||||
kind-specific payload (see `docs/approvals.md::Approval kinds
|
||||
(wire shapes)`).
|
||||
- `Reminder { id, owner, message, due_at, age_seconds }` —
|
||||
`due_at` is the absolute time the scheduler is targeting (RFC
|
||||
3339 on the wire, see *Timestamps on the wire* below); clients
|
||||
compute time-until-fire against it.
|
||||
- `PendingMessages { count }` — undelivered inbox messages the
|
||||
agent still owes itself a `recv` for. Informational + not
|
||||
cancellable (drain with `recv`); only emitted when `count > 0`,
|
||||
and surfaced first in the agent-flavour list as the most
|
||||
actionable signal. Counted host-side from the broker
|
||||
(`count_pending`), so it reflects what's genuinely still queued —
|
||||
the wake-message that drove the current turn is already delivered
|
||||
and not counted.
|
||||
- `UnreadMatrix { rooms, summary }` — unread matrix notifications.
|
||||
Informational + not cancellable (clear with `mark_read`). Unlike
|
||||
the others this is injected by the in-container harness, not
|
||||
hive-c0re, because the matrix daemon lives inside the agent.
|
||||
|
||||
`age_seconds` saturates at zero on any clock anomaly (back-step,
|
||||
unsynchronised wall clock, etc.) so the bulleted list never
|
||||
shows nonsense ages.
|
||||
|
||||
`CancelLooseEnd { kind, id }` is the matching write surface. The
|
||||
`kind` enum (`Reminder` / `Approval`) selects which underlying store
|
||||
the dispatcher reaches into. `Reminder` cancels from either surface
|
||||
subject to an ownership check (the scheduling agent).
|
||||
`Approval` is manager-only — sub-agents don't submit approvals
|
||||
so they have nothing of their own to withdraw; their wire
|
||||
surface returns a clear error if they try. Cancelling an approval
|
||||
transitions the row to `ApprovalStatus::Cancelled` and fires
|
||||
`ApprovalResolved { status: "cancelled" }` so the dashboard pulls
|
||||
the card out of the pending pane.
|
||||
|
||||
### Agent metadata
|
||||
|
||||
`AgentRequest::GetAgentMeta { name }` returns identity + status for
|
||||
an agent. Self-introspection when `name = None` (replaces the older
|
||||
`Whoami` request); target query when `name = Some`.
|
||||
|
||||
Response is `AgentMeta { name, running, hyperhive_rev,
|
||||
status_text, status_set_at, hive_name, swarm_name, matrix_accounts }`:
|
||||
|
||||
- `hyperhive_rev`: `None` only when the configured flake URL has
|
||||
no canonical path. Otherwise carries the rev the target is
|
||||
currently pinned at.
|
||||
- `running`: whether the target's container is currently up. When
|
||||
`false`, the host clears `status_text` / `status_set_at` —
|
||||
on-disk values from before the stop are stale snapshots and
|
||||
shouldn't be shown as live status. Defaults to `true` on the
|
||||
wire (older harnesses never serialised it, and the host only
|
||||
knew how to ask about live containers — keeps backwards-compat
|
||||
with pre-running-field payloads).
|
||||
- `status_text` / `status_set_at`: last value written via
|
||||
`SetStatus`, plus its unix timestamp. Both `None` when the
|
||||
target has never set a status, when the agent name is unknown,
|
||||
or when `running = false` (see above).
|
||||
- `hive_name` / `swarm_name`: display names read from
|
||||
`HYPERHIVE_HIVE_NAME` / `HYPERHIVE_SWARM_NAME` env (sourced from
|
||||
`services.hyperhive.hiveName` / `services.hyperhive.swarm.name`).
|
||||
Both `None` when the options aren't configured.
|
||||
- `matrix_accounts`: one `MatrixIdentity` per configured + live matrix
|
||||
account the agent can act as. Empty for agents with no matrix
|
||||
provisioning.
|
||||
|
||||
### Timestamps on the wire
|
||||
|
||||
Timestamp fields that cross a JSON boundary (dashboard API + SSE,
|
||||
the wire structs in hive-sh4re) serialize as **RFC 3339 UTC strings**
|
||||
(`2026-07-02T18:30:00Z`) via `hive_sh4re::wire_time` — Rust keeps the
|
||||
fields as `i64` unix seconds internally, only the JSON representation
|
||||
changes, and deserialization leniently accepts both the string form
|
||||
and the legacy bare integer (rolling-deploy skew, persisted blobs).
|
||||
**Input-direction** fields agents compute as epoch (`first_fire_at_unix`,
|
||||
schedule-edit `next_fire_at_unix`, `Wakeup::At`) stay integers. The
|
||||
`*_unix` field *names* are kept for now — renaming is the wire-types
|
||||
refactor's concern. The dashboard frontend parses via
|
||||
`util.js::epochSec` wherever it needs arithmetic and feeds the string
|
||||
straight to `new Date(s)` for display.
|
||||
|
||||
### HTTP error bodies
|
||||
|
||||
Every HTTP API in this repo answers failures with **RFC 9457
|
||||
`application/problem+json`** (`{ type, title, status, detail }`), with the
|
||||
human-readable cause in `detail`. An endpoint of ours returning a bare string
|
||||
or a bespoke error shape is a **bug to file against the backend**, not
|
||||
something for the caller to work around.
|
||||
|
||||
Use the `problem_details` crate (`features = ["axum"]`), which the daemons
|
||||
already depend on: type a handler `Result<_, ProblemDetails>` and hand
|
||||
`ProblemDetails::from_status_code(...).with_detail(...)` to `Err`.
|
||||
|
||||
The reason is the consumer, not tidiness. The UIs show errors through one
|
||||
shared component with a copy button, so a caller has to know **which part of
|
||||
the body is the message**. A bare string forces it to treat the whole payload
|
||||
as prose, which is the difference between offering "copy the cause" and
|
||||
dumping a response — and the cause is frequently the entire diagnosis (a
|
||||
JetStream permission refusal, a TLS chain failure) rather than a summary.
|
||||
|
||||
Not in scope: the `hivectl` host-admin and in-agent unix sockets. Those are a
|
||||
JSON-line protocol with their own result types; RFC 9457 is an HTTP format.
|
||||
|
||||
## Tool groups
|
||||
|
||||
The MCP tool surface an agent receives is derived from a set of named
|
||||
`ToolGroup` values (`hive_sh4re::permissions::ToolGroup`), not from a hardcoded
|
||||
binary flavor.
|
||||
|
||||
| Group | Tools |
|
||||
|---|---|
|
||||
| `messaging` | `send`, `recv`, `ack_until` |
|
||||
| `meta` | `get_agent_meta` (`set_status` is always-on, see below) |
|
||||
| `inbox` | `get_loose_ends`, `cancel_loose_end`, `remind` |
|
||||
| `execution` | vestigial — `mcp__bash__run` / `mcp__bash__status` are always available unconditionally via `extraMcpServers`; this group's entries expand to non-existent `mcp__hyperhive__run` / `mcp__hyperhive__status` and have no effect. See `docs/tools/bash.md`. |
|
||||
| `lifecycle` | `kill`, `start`, `restart`, `update`, `list_containers` *(privileged)* |
|
||||
| `approvals` | `request_init_config`, `request_update_meta_inputs` *(privileged)* |
|
||||
| `scheduling` | `request_schedule_prompt`, `fire_schedule_now`, `cancel_schedule`, `edit_schedule`, `list_schedules` *(privileged)* |
|
||||
| `diagnostics` | `get_logs` *(privileged)* |
|
||||
| `forge` | `create_repo` — create git repos through hive-c0re (operator-gated merge) |
|
||||
| `web_tools` | none (gates the Claude built-ins `WebFetch`/`WebSearch`, not an MCP tool) |
|
||||
|
||||
**Always-on tools** — `set_status`, `compact`, and `mark_todos_done` are
|
||||
exposed to every agent regardless of which groups it holds
|
||||
(`ToolGroup::ALWAYS_ON_TOOLS`). The operator dashboard depends on every agent
|
||||
being able to report its status chip, and the server-side `SetStatus` handler
|
||||
has no tool-group check (only length validation), so gating it would only
|
||||
desync the `--allowedTools` list from what the host actually accepts.
|
||||
Revoking `meta` therefore drops `get_agent_meta` but never `set_status`.
|
||||
`mark_todos_done` is here because todos are pushed to an agent independent of
|
||||
whether it holds `inbox` — an agent without that group still needs a way to
|
||||
clear them.
|
||||
|
||||
**Config storage** — per-agent tool groups live in
|
||||
`/var/lib/hyperhive/meta/tool-groups.json` (hive-c0re-owned, committed to the
|
||||
meta repo alongside `topology.json`). Format: `{ "alice": ["messaging", "meta",
|
||||
"inbox", "lifecycle"], "bob": ["messaging", "meta", "inbox"] }`. An absent entry
|
||||
means "use role default". Tool permissions are intentionally NOT configurable
|
||||
from `agent.nix` — that file goes through the manager's approval flow, so
|
||||
letting it declare its own groups would let the manager grant itself any tool by
|
||||
submitting a config commit, bypassing the operator gate.
|
||||
|
||||
**Setting groups** — the operator sets groups via the dashboard or
|
||||
`hive-c0re::tool_groups::set_groups(name, groups)`. After a change
|
||||
`meta::sync_agents` commits the updated file; the next agent rebuild picks up
|
||||
the new `HIVE_TOOL_GROUPS` env var. Agents with no entry get no var.
|
||||
|
||||
**Runtime resolution** — at session start the harness reads `HIVE_TOOL_GROUPS`
|
||||
(a comma-separated list of snake_case group names injected by the meta renderer
|
||||
from `tool-groups.json`). Unrecognised tokens are logged and skipped. Falls back
|
||||
to `ToolGroup::AGENT_DEFAULT` (`messaging`, `meta`, `inbox`, `execution`) when
|
||||
the var is absent or empty.
|
||||
|
||||
**Updating the surface** — when a new `#[tool]` fn is added to `AgentServer`
|
||||
in `hive-agent-mcp/src/mcp/mod.rs`, add its name to the matching `ToolGroup::tools()`
|
||||
slice in `hive-sh4re/src/permissions.rs`. That's the single source of truth;
|
||||
`mcp_config::allowed_mcp_tools` (in `hive-agent/src/mcp_config.rs`) reads it at
|
||||
session start.
|
||||
|
||||
## Capabilities
|
||||
|
||||
Capabilities gate system-level access that goes beyond the MCP tool surface —
|
||||
things an agent can *access*, not just *call*. Parallel to tool groups but
|
||||
orthogonal: an agent can have a tool group that registers a tool AND a capability
|
||||
that allows the underlying resource access.
|
||||
|
||||
| Capability | Effect |
|
||||
|---|---|
|
||||
| `manage_root_agent` | may lifecycle-manage the root/manager agent via `kill`/`start`/`restart` |
|
||||
| `read_host_journal` | `get_host_journal` MCP tool is registered + `GET /journal-host` requests are served |
|
||||
| `query_agent_state` | may call `get_loose_ends` / `CountPendingReminders` targeting non-child agents |
|
||||
|
||||
**Config storage** — per-agent capabilities live in
|
||||
`/var/lib/hyperhive/meta/capabilities.json` alongside `tool-groups.json`.
|
||||
Format: `{ "atlas": ["read_host_journal"], "ruth": ["manage_root_agent"] }`.
|
||||
An absent entry means "no extra capabilities". `render_flake` in `meta.rs`
|
||||
reads this file and injects `HIVE_CAPABILITIES` (comma-separated
|
||||
`snake_case` names) into each agent's systemd service env; absent entries emit
|
||||
no env var so agents without capabilities don't trigger a spurious rebuild.
|
||||
|
||||
**Setting capabilities** — the operator sets capabilities via the
|
||||
C4P4B1L1T13S section in the dashboard's P3RM1SS10NS tab.
|
||||
`hive-c0re::capabilities::set_caps(name, caps)` is the write path.
|
||||
After a change `meta::sync_agents` commits the updated file; the next agent
|
||||
rebuild picks up the new `HIVE_CAPABILITIES` env var.
|
||||
|
||||
**Runtime resolution** — at session start the harness reads `HIVE_CAPABILITIES`
|
||||
and resolves each token to a `Capability` variant. Unrecognised tokens are
|
||||
logged and skipped. An absent or empty var means no extra capabilities.
|
||||
|
||||
**Capability NOT configurable from `agent.nix`** — same reasoning as tool
|
||||
groups: an agent that could grant its own capabilities via a config commit would
|
||||
bypass the operator approval gate.
|
||||
|
||||
**Adding a new capability** — add a variant to `Capability` in
|
||||
`hive-sh4re/src/permissions.rs` + an arm to `as_str`. Add it to `Capability::ALL`
|
||||
(the source of truth for the permissions UI columns). Implement the access
|
||||
check in the relevant handler (`hive-c0re/src/socket_server/mod.rs`,
|
||||
`hive-c0re/src/socket_server/lifecycle_handlers.rs`, `coordinator.rs`, or a
|
||||
handler under `hive-c0re/src/dashboard/`).
|
||||
|
||||
## Async forms
|
||||
|
||||
Dashboard + per-agent mutating forms carry `data-async`; the shared
|
||||
`bindAsyncForms` `submit` listener (`frontend/packages/shared/src/forms.js`,
|
||||
imported as `@hive/shared/forms.js` and wired up from `tabs.js` on the
|
||||
dashboard and `app.js` on the per-agent UI) intercepts, shows a spinner,
|
||||
POSTs `application/x-www-form-urlencoded` (axum's `Form` extractor
|
||||
rejects multipart), calls `refreshState()` on success. New mutating
|
||||
forms should add `data-async` and optionally `data-confirm` (for a
|
||||
JS-side `confirm()` prompt) or `data-prompt="…"` (for a
|
||||
`window.prompt()` whose answer goes into a hidden input named by
|
||||
`data-prompt-field`, default `note`).
|
||||
|
||||
`refreshState` defers automatically when `document.activeElement`
|
||||
sits inside a managed section so the operator's typing isn't lost;
|
||||
collapsible `<details data-restore-key=…>` survive the re-render
|
||||
via `snapshotOpenDetails` / `restoreOpenDetails`.
|
||||
|
||||
## `rebuild` is the reconcile verb
|
||||
|
||||
`job_queue::templates::rebuild` builds the DAG that reconciles a
|
||||
container to its wanted state: `write_dropins` (the nspawn-conf
|
||||
rewrite — `PRIVATE_NETWORK=1`, `HOST_ADDRESS` = the bridge gateway IP,
|
||||
sets `EXTRA_NSPAWN_FLAGS` — plus the systemd resource-limits drop-in)
|
||||
is folded into the `Swap` node, then `nixos-container update` + stop +
|
||||
start runs across the `StopForUpdate → Swap → RebuildBookkeeping`
|
||||
brace and the tail `Reconcile` node. `flake.nix` itself is no longer
|
||||
regenerated host-side on rebuild — it's tracked in the agent's
|
||||
proposed/applied repos and rides along on every fetch (see
|
||||
`docs/approvals.md::Two repos per agent`).
|
||||
|
||||
Anything that changes per-container state on the host should be
|
||||
re-applied here so a manual `↻ R3BU1LD` from the dashboard is
|
||||
sufficient to recover.
|
||||
|
||||
## Actions are factored
|
||||
|
||||
`approve` / `deny` / `destroy` (and the lifecycle helper) live in
|
||||
`actions.rs` / `hive-c0re/src/dashboard/`. The admin socket and the dashboard
|
||||
POST handlers both call into them so the two surfaces never drift.
|
||||
|
||||
## Commit messages
|
||||
|
||||
Short, lowercase, no `Co-Authored-By` trailer. Imperative mood, no
|
||||
period. Body explains *why* if non-obvious; otherwise the subject
|
||||
alone is fine. Wrap at ~72 cols.
|
||||
|
||||
## Commit before test
|
||||
|
||||
Stage and commit when work *looks* ready, then run validation
|
||||
(`cargo check`, `nix flake check`, real deploy). Failures get a
|
||||
follow-up commit rather than an amend. The commit history is the
|
||||
work log; rewriting it loses signal.
|
||||
|
||||
## Building & local checks
|
||||
|
||||
Build through the **flake devshell**, not a bare toolchain — agent
|
||||
containers ship no global rust. `nix develop -c <cmd>` runs one
|
||||
command inside the project-pinned env (cargo/clippy/rustfmt plus the
|
||||
C compiler + `libsqlite3`/`ring` link deps); outside it a bare
|
||||
`cargo build` fails with `failed to find tool "cc"` / `cannot find
|
||||
-lsqlite3`. One command per invocation — agents run each task as a
|
||||
fresh non-interactive process, so there's no persistent shell to
|
||||
reuse.
|
||||
|
||||
```sh
|
||||
nix develop -c cargo clippy --all-targets -- -D warnings
|
||||
nix develop -c cargo test
|
||||
nix fmt # treefmt — authoritative, NOT bare cargo fmt
|
||||
```
|
||||
|
||||
`nix fmt` (treefmt) is the formatter CI gates on; bare `cargo fmt`
|
||||
misses the non-rust files treefmt also covers, so always run `nix
|
||||
fmt` before pushing.
|
||||
|
||||
**Clippy discipline — never add `#[allow(clippy::…)]`.** All lints are
|
||||
CI-fatal at `-D warnings` (pedantic included); every warning that fires
|
||||
must be *fixed*, not silenced. Common patterns:
|
||||
|
||||
- `too_many_lines` — extract a helper function or a sub-struct
|
||||
(the `TurnAccum` extraction in `stats.rs` is a worked example).
|
||||
- `doc_markdown` (brand name without backticks in a doc comment) — add
|
||||
backticks: `` `DOMPurify` `` instead of `DOMPurify`.
|
||||
- `must_use` / `unused_results` — actually handle or explicitly discard
|
||||
the return value (`let _ = …` is fine when intentional).
|
||||
|
||||
If a lint seems wrong for a specific call site, file an issue and ask
|
||||
mara — don't add `#[allow]` speculatively. The gate is intentional.
|
||||
|
||||
**The devshell checks are not the full `nix flake check`.** Clippy /
|
||||
fmt / `cargo test` cover most gates, but `nix flake check` runs extra
|
||||
check derivations they don't:
|
||||
|
||||
- **`hivectl-docs`** regenerates `docs/tools/hivectl-cli.md` from
|
||||
hivectl's clap tree and **fails if the committed copy is stale**.
|
||||
So **after any change to a hivectl verb or flag, regenerate it**:
|
||||
```sh
|
||||
nix develop -c cargo run --bin hivectl -- markdown-docs > docs/tools/hivectl-cli.md
|
||||
```
|
||||
clippy / fmt / `cargo test` all pass *without* this — only the
|
||||
flake check catches the drift, and `ci-log` often can't show you
|
||||
why (it 500s on a fast failure), so you're left guessing "builder
|
||||
flake" when it's a stale doc.
|
||||
- **`swarmctl-docs`** is the same check for `swarmctl` /
|
||||
`docs/tools/swarmctl-cli.md`:
|
||||
```sh
|
||||
nix develop -c cargo run --bin swarmctl -- markdown-docs > docs/tools/swarmctl-cli.md
|
||||
```
|
||||
- there's also a flake `cargo-test` check and NixOS module
|
||||
evaluation in the set.
|
||||
|
||||
When local clippy/fmt/test pass but CI's `nix flake check` fails,
|
||||
**don't assume a transient builder problem** — reproduce the real
|
||||
gate locally: `nix flake check` (shares the build farm, use
|
||||
sparingly) or build just the suspect check, e.g. `nix build
|
||||
.#checks.x86_64-linux.hivectl-docs`.
|
||||
|
||||
## Best-effort oneshot services
|
||||
|
||||
The harness ships a family of one-shot systemd services that
|
||||
configure agent-side surfaces from values hive-c0re writes into
|
||||
the state dir at provisioning time:
|
||||
|
||||
- `tea-login` — writes `~/.config/tea/config.yml` from the
|
||||
`forge-token` written by `hive-c0re::forge::ensure_user_for`,
|
||||
so `tea repos create` / `tea pulls create` work without
|
||||
interactive prompts.
|
||||
- `forge-avatar-sync` — uploads `hyperhive.icon` SVG to the
|
||||
agent's Forgejo profile, so the icon shows up on commits / PRs /
|
||||
issue comments.
|
||||
|
||||
(The matrix profile avatar is **not** a oneshot — `hive-matrix-daemon`
|
||||
sets it over its live authenticated Client; see
|
||||
`docs/persistence.md::matrix avatar`.)
|
||||
|
||||
Shape contract — every one of these:
|
||||
|
||||
1. **Always `exit 0`**, even on internal failure. A non-zero exit
|
||||
would mark the unit `failed`, which in turn aborts
|
||||
`nixos-container update` and blocks rebuilds. The agent's
|
||||
capability surface is not allowed to gate the container
|
||||
build.
|
||||
2. **No `set -e`** in the script body. Subshell failures must not
|
||||
propagate. Use `... || true` on every external call that can
|
||||
fail (forge unreachable, missing icon, parse error, etc.)
|
||||
3. **Skip silently when prerequisites are missing**: no token
|
||||
file, no icon, no reachable upstream → `echo` a short skip
|
||||
line + `exit 0`. The next boot tries again.
|
||||
4. **Wired to `multi-user.target`** so they run on every boot
|
||||
(lets a rotated token / new icon take effect without
|
||||
`systemctl restart` gymnastics).
|
||||
5. **Re-runnable**: a second invocation produces the same final
|
||||
state (idempotent uploads, idempotent config rewrites). Used
|
||||
by the `.path` watchers that re-fire on token appearance (see
|
||||
`docs/persistence.md::Matrix per-agent daemon`).
|
||||
|
||||
The artefact lives under the agent user's home where applicable
|
||||
(`~/.config/tea/config.yml`) and is chown'd to that user, but the
|
||||
service itself stays root-owned so the bootstrap ordering doesn't
|
||||
need a user-existence check before each fire.
|
||||
|
||||
This pattern keeps the rebuild path resilient: any failure inside
|
||||
these services degrades the corresponding surface (no tea config,
|
||||
no avatar) but never blocks the container from coming up. The
|
||||
operator notices through `journalctl -u <unit>` rather than a
|
||||
broken switch-to-configuration.
|
||||
522
docs/process/gotchas.md
Normal file
522
docs/process/gotchas.md
Normal file
|
|
@ -0,0 +1,522 @@
|
|||
# Gotchas
|
||||
|
||||
NixOS + nspawn quirks and lessons we hit the hard way. If something
|
||||
here looks unmotivated in the code, there's usually a story underneath.
|
||||
Grouped by area — jump to the section that matches what you're
|
||||
touching.
|
||||
|
||||
## NixOS / nspawn containers
|
||||
|
||||
### `nixos-container` doesn't expose `--bind` on the CLI
|
||||
|
||||
The CLI doesn't accept `--bind`. Path is via `EXTRA_NSPAWN_FLAGS` in
|
||||
`/etc/nixos-containers/<NAME>.conf` — the start script
|
||||
(`/nix/store/.../container_-start`) expands it unquoted into the
|
||||
`systemd-nspawn` invocation. `lifecycle::host_config::set_nspawn_flags()`
|
||||
rewrites this line.
|
||||
|
||||
### `/run/systemd/nspawn/*.nspawn` overrides are ignored
|
||||
|
||||
`nixos-container`'s start script builds the nspawn command line
|
||||
directly. Dropping a `.nspawn` file under `/run/systemd/nspawn/`
|
||||
looks like the obvious extension point and does nothing. Use
|
||||
`EXTRA_NSPAWN_FLAGS` (above).
|
||||
|
||||
### `boot.isNspawnContainer = true`
|
||||
|
||||
Not `boot.isContainer = true`. Renamed in nixos-25.11+.
|
||||
|
||||
### `nixos-container create` auto-assigns `HOST_ADDRESS` / `LOCAL_ADDRESS`
|
||||
|
||||
…in the `.conf`. The start script's `if HOST_ADDRESS set →
|
||||
--network-veth` branch then forces a private netns — silently fatal
|
||||
for our web UIs (the bind is invisible from the host). Every agent
|
||||
container runs isolated: `hive-priv`'s `write_nspawn_flags` writes
|
||||
`PRIVATE_NETWORK=1` plus a veth pair onto the host bridge, `HOST_ADDRESS`
|
||||
set to the bridge gateway IP (so `nixos-container`'s in-container init
|
||||
installs a default route before the DHCP lease arrives), rest left for
|
||||
DHCP.
|
||||
|
||||
### systemd service PATH ≠ host PATH
|
||||
|
||||
The hive-c0re service sets `path = [ pkgs.git "/run/current-system/sw" ]`.
|
||||
In-container harness services do the same so anything an agent adds
|
||||
to its own `agent.nix` (`environment.systemPackages`) is visible to
|
||||
the `mcp__bash__run` MCP tool (and any other in-container process) without
|
||||
editing the service definition.
|
||||
`environment.HYPERHIVE_GIT` bakes git's absolute path in (read by
|
||||
`lifecycle::git_command()`) for the host.
|
||||
|
||||
### `systemd.services.*.path` appends `/bin` to every entry
|
||||
|
||||
NixOS's `systemd.services.<unit>.path` list feeds every entry through
|
||||
`lib.makeBinPath`, which **appends `/bin` unconditionally**. That's
|
||||
the right thing for Nix packages (their `outPath` is the store root,
|
||||
not the `bin/` subdir), but it bites when you pass a string that
|
||||
already ends with `/bin`:
|
||||
|
||||
```nix
|
||||
# ❌ /run/wrappers/bin → /run/wrappers/bin/bin (does not exist)
|
||||
path = [ "/run/wrappers/bin" "/run/current-system/sw" ];
|
||||
|
||||
# ✅ /run/wrappers → /run/wrappers/bin (the real wrappers dir)
|
||||
path = [ "/run/wrappers" "/run/current-system/sw" ];
|
||||
```
|
||||
|
||||
The bug is silent: `nix eval` succeeds, the unit starts, but PATH
|
||||
contains a non-existent directory. The first symptom is usually
|
||||
`sudo: must be owned by uid 0 and have the setuid bit set` because
|
||||
the setuid sudo wrapper lives at `/run/wrappers/bin/sudo` and
|
||||
the path entry resolves to `/run/wrappers/bin/bin` instead.
|
||||
|
||||
### `RuntimeDirectoryPreserve = "yes"`
|
||||
|
||||
…keeps `/run/hyperhive/` (and the per-agent sub-dirs) across
|
||||
hive-c0re restarts. Without it, every restart wipes bind sources and
|
||||
existing containers can't be started.
|
||||
|
||||
### `RestrictAddressFamilies` fails as "Address family not supported by protocol"
|
||||
|
||||
A unit whose `RestrictAddressFamilies` omits a family gets `EAFNOSUPPORT`
|
||||
(errno 97) back from `socket()`. Clients surface that as *"tcp open error:
|
||||
Address family not supported by protocol"* — the message names the
|
||||
**protocol** and never the **sandbox**, so it reads like a dead network, a
|
||||
missing route, or an IPv6 problem.
|
||||
|
||||
⇒ On that error, read the unit before you touch the network.
|
||||
|
||||
Two things to get right when a daemon needs outbound TCP:
|
||||
|
||||
- list `AF_INET` **and** `AF_INET6` — omitting one leaves a client that
|
||||
works until DNS hands back the other family;
|
||||
- list `AF_NETLINK` too. glibc's `getaddrinfo` opens a netlink socket to
|
||||
enumerate local addresses before it returns any, so name resolution
|
||||
fails without it even when `AF_INET` is allowed.
|
||||
|
||||
**The directive is a claim about what the program does, and nothing
|
||||
re-checks it when the program changes.** A unit that only served a unix
|
||||
socket when it was written is correct at `[ "AF_UNIX" ]` and silently wrong
|
||||
the day someone adds an HTTP client. Check the unit in the same commit as
|
||||
the client — and when narrowing it, prefer a test that derives the required
|
||||
families from the code (which fails on the *next* client too) over one that
|
||||
asserts today's list.
|
||||
|
||||
### `register_agent` is idempotent
|
||||
|
||||
Drops any prior socket task before rebinding. Required so a
|
||||
hive-c0re restart followed by `rebuild alice` recreates the agent's
|
||||
socket without needing a clean reinstall.
|
||||
|
||||
## Claude Code packaging & credentials
|
||||
|
||||
### `claude-code` is unfree
|
||||
|
||||
`claude-code` comes from the flake's main `nixpkgs` (nixos-26.05).
|
||||
It's unfree, so the agent modules set `config.allowUnfreePredicate`
|
||||
at the container level to whitelist `claude-code` specifically —
|
||||
scoped, only this one package. This is needed because each per-agent
|
||||
`nixosConfiguration` evaluates its own nixpkgs instance and the
|
||||
operator's host-level `allowUnfree` does **not** propagate in.
|
||||
Operators don't need to set anything on their side.
|
||||
|
||||
That same isolation is why an agent can't pick a claude out of a
|
||||
*different* nixpkgs by itself: a container only ever sees the one
|
||||
nixpkgs the meta flake injects, so an `agent.nix` naming the host's
|
||||
`nixpkgs-unstable` has nothing to name. A release channel can trail
|
||||
unstable by weeks on this package, which is what
|
||||
`services.hyperhive.c0re.claudeCodePackage` is for — set it host-side
|
||||
and every agent runs that build.
|
||||
|
||||
What crosses is the **store path**, not the derivation. Containers
|
||||
share the host's `/nix/store`, so the binary is already reachable
|
||||
inside them with its whole closure; hive-c0re writes the path into each
|
||||
agent's flake as a string literal and the agent module symlinks
|
||||
`bin/claude` onto PATH. Two things rule out the obvious alternatives: a
|
||||
`path:/nix/store/<pkg>` flake input is re-copied into the store as a
|
||||
reference-less `-source` (so the runtime closure never arrives), and
|
||||
`lib.types.package` fed a bare path runs `builtins.storePath`, which
|
||||
pure evaluation rejects. `hyperhive.docs.source` gets away with being
|
||||
an input only because a docs tree has no runtime dependencies.
|
||||
|
||||
The `storePath` trap is worth spelling out, because it is not confined
|
||||
to options the operator writes: **any** option of type `package` fed a
|
||||
store-path *string* coerces through `lib.toDerivation`, i.e.
|
||||
`builtins.storePath`. `environment.systemPackages` and
|
||||
`systemd.services.<name>.path` both do it (the latter takes plain
|
||||
strings like `/run/wrappers` happily, but anything under
|
||||
`builtins.storeDir` is treated as a package). So a path handed to the
|
||||
container as text has to be wrapped in a real derivation — a symlink
|
||||
farm built from the interpolated string — before it can go anywhere a
|
||||
package is expected.
|
||||
|
||||
The catch is that a path written into a generated flake is text, not a
|
||||
reference — the container's closure does not keep the binary alive.
|
||||
The **host** does: the package is interpolated into
|
||||
`/etc/hyperhive/serve.json`, so it lands in the host's system closure
|
||||
and is gc-rooted by the running generation. `builtins.toJSON` preserves
|
||||
string context, which is the load-bearing detail; discard the context
|
||||
anywhere on that path and `nix-collect-garbage` will eventually take
|
||||
the hive's `claude` out from under it. The price of the root is that an
|
||||
old `claude-code` can't be reclaimed until every agent has rebuilt past
|
||||
it and the old generations are gone.
|
||||
|
||||
### Claude credentials are per-agent
|
||||
|
||||
`/var/lib/hyperhive/agents/<name>/claude/` bind-mounts to
|
||||
`/home/<name>/.claude` (RW). Sharing one dir across agents is NOT viable —
|
||||
OAuth refresh tokens rotate, so any sibling refresh invalidates all
|
||||
the others. Login flow runs from the per-agent web UI; creds persist
|
||||
across `destroy`/recreate (`--purge` wipes them).
|
||||
|
||||
### Persistent notes dir per agent
|
||||
|
||||
`/var/lib/hyperhive/agents/<name>/state/` bind-mounts to
|
||||
`/agents/<name>/state` (RW; uniform for all agents).
|
||||
The harness exposes the same path
|
||||
via `$HYPERHIVE_STATE_DIR`. System prompts tell agents to keep
|
||||
durable knowledge here (`notes.md`, anything else) — the harness's own
|
||||
internal files (`hyperhive-events.sqlite`, `hyperhive-turn-stats.sqlite`,
|
||||
`hyperhive-model`) live in the separate `harness` dir instead, so they
|
||||
don't clutter what claude sees as "my notes dir" (see
|
||||
[`docs/agent-lifecycle/persistence.md`](../agent-lifecycle/persistence.md)). Survives `destroy`/recreate
|
||||
alongside the claude dir.
|
||||
|
||||
## Networking & ports
|
||||
|
||||
### Web UI ports collide on hash
|
||||
|
||||
Sub-agent web UI ports are deterministic FNV-1a of the agent name
|
||||
modulo 900 (range 8100..8999). With ~30 agents the birthday-paradox
|
||||
collision rate gets meaningful; at 2–3 agents you can still get
|
||||
unlucky. Operator resolves a collision by renaming the offending
|
||||
agent (different hash → different port) and rebuilding. No state
|
||||
file, no probing, no port-allocation drift — the value is
|
||||
reproducible from just the name. Every agent hashes into
|
||||
8100..8999 via the same FNV-1a; dashboard
|
||||
at `cfg.dashboardPort` (default 7000).
|
||||
|
||||
### Restart races on TCP bind
|
||||
|
||||
Both the dashboard and per-agent web UI use `tokio::net::TcpSocket`
|
||||
with `SO_REUSEADDR` plus a retry-on-`AddrInUse` loop (12 tries,
|
||||
exponential backoff capped at 2s, ~22s total). REUSEADDR handles
|
||||
the `TIME_WAIT` case from a clean previous exit; retry covers the
|
||||
genuine "previous process is still alive during a systemd restart
|
||||
overlap" case. REUSEADDR does **not** allow two simultaneous
|
||||
`LISTEN` sockets on the same port (that would be `SO_REUSEPORT`,
|
||||
which we don't use) — exclusivity is preserved.
|
||||
|
||||
## Approvals
|
||||
|
||||
### Orphan approvals
|
||||
|
||||
If state dirs are wiped out from under a pending approval (test
|
||||
scripts, manual `rm -rf`), the dashboard's next render marks them
|
||||
`failed` with note `"agent state dir missing"` so they fall out of
|
||||
`pending`. They stay in sqlite for audit.
|
||||
|
||||
## Gateway / SPA serving
|
||||
|
||||
### SPA fallback: use `Accept` header map, not `try_files ... /index.html`
|
||||
|
||||
The naive nginx pattern for an SPA (`try_files $uri $uri/
|
||||
/index.html`) silently swallows asset 404s — a missing JS file
|
||||
returns `index.html` with a 200, so the JS runtime never loads and the
|
||||
page renders blank with no visible error. Extension allowlists (tried
|
||||
as an alternative) have the same maintenance problem: any new file
|
||||
extension the SPA ships breaks silently.
|
||||
|
||||
The pattern that works (`nix/host-modules/hive-matrix.nix`, serving
|
||||
fluffychat at the matrix gateway vhost's root) keys the fallback on the
|
||||
HTTP `Accept` header:
|
||||
|
||||
```nginx
|
||||
# Outside the server block (appendHttpConfig):
|
||||
map $http_accept $matrix_spa_target {
|
||||
default "/__matrix_spa_no_html_fallback";
|
||||
"~*text/html" "/index.html";
|
||||
}
|
||||
|
||||
# Inside the location:
|
||||
try_files $uri $uri/ $matrix_spa_target =404;
|
||||
```
|
||||
|
||||
Top-frame navigations always send `Accept: text/html,...` (chrome /
|
||||
firefox / safari are consistent). Asset fetches (`image/*`,
|
||||
`application/javascript`, `*/*`) don't carry `text/html`, so they
|
||||
fall through to the trailing `=404`. No extension list to maintain;
|
||||
no named-location indirection needed.
|
||||
|
||||
## Build & dev workflow
|
||||
|
||||
### Nix store `cp -r` preserves read-only bits
|
||||
|
||||
Copying a nix store path with `cp -r src/. $out/` inside a
|
||||
`pkgs.runCommand` derivation preserves the read-only permissions of
|
||||
store files. Any subsequent write into the copied tree (adding new
|
||||
files in subdirectories) fails with `EPERM`. Fix: pass
|
||||
`--no-preserve=mode,ownership` so the output tree is writable.
|
||||
|
||||
### `nix build flake#name` does not walk into `nixosConfigurations`
|
||||
|
||||
`nix build` resolves the fragment (`#name`) against the flake's
|
||||
**top-level output attrs** — not against `nixosConfigurations`
|
||||
specifically. `nixos-container` and `nixos-rebuild` use their own
|
||||
internal convention that routes an agent name to
|
||||
`nixosConfigurations.<name>.config.system.build.toplevel`, but
|
||||
`nix build` has no such convention.
|
||||
|
||||
```
|
||||
# ❌ silently builds the wrong thing (or errors if attr doesn't exist)
|
||||
nix build /var/lib/hyperhive/meta#argus.config.system.build.toplevel
|
||||
|
||||
# ✅ explicit path nix build actually resolves
|
||||
nix build /var/lib/hyperhive/meta#nixosConfigurations.argus.config.system.build.toplevel
|
||||
```
|
||||
|
||||
`lifecycle::prebuild_toplevel` hit this once by constructing the attr
|
||||
path as `{flake_ref}.config…` — which produced `meta#argus.config…`
|
||||
instead of `meta#nixosConfigurations.argus.config…`. The fix:
|
||||
`split_once('#')` to separate flake path from name, then template
|
||||
`{path}#nixosConfigurations.{name}.config.system.build.toplevel`.
|
||||
|
||||
### Containerized nix-daemon needs `sandbox-fallback = true`
|
||||
|
||||
Agent containers bind-mount the host's nix-daemon socket. nspawn
|
||||
containers don't get user-namespaces by default, so `nix build`
|
||||
invocations _inside_ the container can't set up the build sandbox
|
||||
and fail outright if the host daemon's
|
||||
`nix.settings.sandbox-fallback` is `false` (nixpkgs default).
|
||||
`nix/agent-modules/default.nix` does `lib.mkForce true` so builds
|
||||
fall back to unsandboxed local builds rather than failing. Security
|
||||
implications: `docs/security.md`.
|
||||
|
||||
### Linking workspace binaries locally needs `nix develop`
|
||||
|
||||
The Rust workspace links `libsqlite3-sys` (rusqlite) against the
|
||||
system `libsqlite3`. Agent containers carry no system libsqlite3 on
|
||||
the linker path, so a plain `cargo build` of any binary dies with
|
||||
`cannot find -lsqlite3` (deps and `ring` compile fine — only the
|
||||
final link fails). `cargo check` / `cargo clippy` still work in the
|
||||
ambient shell since they never link.
|
||||
|
||||
Build + run binaries through the dev shell, which carries `sqlite`
|
||||
on `NIX_LDFLAGS`:
|
||||
|
||||
```bash
|
||||
nix develop -c cargo build -p hive-c0re --bin hivectl
|
||||
nix develop -c cargo run -p hive-c0re --bin hivectl -- <args>
|
||||
```
|
||||
|
||||
This is also how you regenerate committed generated docs locally —
|
||||
e.g. `docs/tools/hivectl-cli.md` via the `hivectl markdown-docs`
|
||||
subcommand (its `hivectl-docs` flake check otherwise only fails in
|
||||
CI on drift).
|
||||
|
||||
### Split asset derivations away from the rust workspace
|
||||
|
||||
`nix/packages/assets.nix` builds the branding SVG/PNG family + claude
|
||||
system-prompt template + claude-settings JSON as its own derivation,
|
||||
separate from the hive-ag3nt / hive-c0re crates. Reason: when the
|
||||
rust build's `src` was the whole repo tree, any tweak to
|
||||
`branding/agent-configs.svg` or `hive-ag3nt/prompts/system.md`
|
||||
invalidated the cargo cache and forced a full rebuild. crane (and
|
||||
naersk before it) couldn't see "these inputs are unused by rust" on
|
||||
its own — the split breaks the coupling at the derivation boundary.
|
||||
The agent-configs PNG is rendered from the SVG via `rsvg-convert` at
|
||||
build time; librsvg dependency lives here, not in the rust
|
||||
derivation's `nativeBuildInputs`.
|
||||
|
||||
### `nix fmt` fails in a git worktree with "object not found"
|
||||
|
||||
`nix fmt` (and any `nix` command that fetches a `git+file://` flake
|
||||
URL) uses libgit2 internally to compute `revCount` — the number of
|
||||
commits reachable from HEAD. This walk fails with:
|
||||
|
||||
```
|
||||
error: getting Git object '<hash>': object not found (libgit2 error code = 9)
|
||||
```
|
||||
|
||||
when a commit that was reachable at some earlier evaluation is now gone
|
||||
(GC'd, rebased away, or pruned). The failure is persistent: clearing
|
||||
`~/.cache/nix/{eval-cache-v6,gitv3,fetcher-cache-v4.sqlite}` does not
|
||||
help because the missing object is a structural gap in the git object
|
||||
graph itself, not in nix's caches.
|
||||
|
||||
**Workaround: use a plain clone, not a git worktree.**
|
||||
|
||||
```bash
|
||||
git clone http://<forge>/hyperhive/hyperhive.git ~/hh-work
|
||||
cd ~/hh-work && nix fmt
|
||||
```
|
||||
|
||||
The root cause is specific to worktrees: a worktree shares the object
|
||||
store with its parent repo. If the parent repo's history was rewritten
|
||||
(rebase, force-push, `git gc --prune`) while the worktree was checked
|
||||
out at a branch tip that references the pruned commits via its reflog or
|
||||
history, libgit2's rev-walk encounters the gap. A plain clone has its
|
||||
own self-consistent object store and is immune to the issue.
|
||||
|
||||
## Tooling
|
||||
|
||||
### `hive-forge`: prefer over raw curl pipelines
|
||||
|
||||
Full CLI reference: [`docs/tools/forge.md`](../tools/forge.md).
|
||||
Never use raw `curl` for forge access.
|
||||
|
||||
## GUI (weston/VNC)
|
||||
|
||||
### Weston VNC compositor (per-agent `hyperhive.gui.enable`)
|
||||
|
||||
`nix/agent-modules/weston-vnc.nix` adds an optional Weston Wayland
|
||||
compositor with the VNC backend, surfaced as
|
||||
`hyperhive.gui.enable = true` per-agent. The harness's
|
||||
`/screen/ws` WebSocket relay (`docs/web-ui/agent.md::Per-agent endpoints`)
|
||||
connects to the compositor at `127.0.0.1:<vnc_port>`.
|
||||
|
||||
- **Port allocation**: a **fixed** port (`hyperhive.gui.vncPort`,
|
||||
default 5900). No per-agent hashing: network isolation is
|
||||
unconditional (each agent has its own netns — see
|
||||
`docs/network.md#container-isolation`), so the VNC port is
|
||||
container-local and can't collide across agents. The harness learns
|
||||
the port from the `HIVE_GUI_VNC_PORT` env var (set on the harness
|
||||
service when `gui.enable`) — no marker file, no runtime hash. (Unlike
|
||||
the agent **web-UI** port, which is still an FNV-1a hash because those
|
||||
listen on the shared host stack — see `Web UI ports collide on hash`.)
|
||||
- **Non-root, shared user session**: weston runs as the agent's own
|
||||
user (`hyperhive.user.name`, the same user hive-ag3nt runs as), not
|
||||
root, so the GUI and the agent share one session. The runtime dir is a
|
||||
fixed `/run/gui` (systemd `RuntimeDirectory=gui`, `0700`,
|
||||
`RuntimeDirectoryPreserve=yes` so it survives weston restarts for the
|
||||
wayland client sharing the `/run/gui/wayland-0` socket). Wayland
|
||||
clients in the agent's config (e.g. bitburner electron) must run as the
|
||||
same user with `XDG_RUNTIME_DIR=/run/gui`.
|
||||
- **One shared D-Bus session bus (`gui-dbus.service`)**: a single
|
||||
persistent `dbus-daemon --session` bound at `/run/gui/bus`, run as the
|
||||
agent user, ordered `before weston.service` (it shares the same
|
||||
`RuntimeDirectory=gui`, creating the dir first). Chromium/electron via
|
||||
ozone refuse to map an `xdg_toplevel` without a reachable session bus
|
||||
("Failed to connect to the bus" → binds `xdg_wm_base` then destroys it
|
||||
= invisible window even though CDP works). The fix is **not** to wrap
|
||||
each client in its own `dbus-run-session` (a private throwaway bus per
|
||||
process — that's a _separate_ session, defeating the one-session
|
||||
model); it's this one shared bus, whose address is exported as
|
||||
`DBUS_SESSION_BUS_ADDRESS=unix:path=/run/gui/bus` via
|
||||
`systemd.globalEnvironment` so weston, the harness and every GUI client
|
||||
inherit it.
|
||||
- **Fixed Wayland socket name (`--socket=wayland-0`)**: weston is
|
||||
launched with `--socket=wayland-0` so the socket path is
|
||||
deterministic. `nix/agent-modules/weston-vnc.nix` exports `WAYLAND_DISPLAY=wayland-0`
|
||||
and `XDG_RUNTIME_DIR=/run/gui` as global system environment
|
||||
variables (gated on `hyperhive.gui.enable`) so every systemd service
|
||||
in the container inherits them. Without this, services starting
|
||||
Wayland clients could not find the compositor — libwayland falls
|
||||
back to a headless display or errors out, the app "works" on a
|
||||
second invisible display, and the VNC session shows a blank weston
|
||||
desktop (#540 double-screen).
|
||||
- **VNC bind address**: weston's VNC backend has no CLI
|
||||
bind-address flag (unlike the RDP backend's `--address`), so the
|
||||
listener binds `0.0.0.0`. The harness relay only connects via
|
||||
`127.0.0.1`; the host firewall blocks the per-agent VNC port range
|
||||
from external access. A future weston.ini `[vnc] address=` will
|
||||
let us restrict the bind directly once upstream supports it.
|
||||
- **PAM service name**: literal `weston-remote-access` — that's the
|
||||
string libweston passes to `pam_start()` in `libweston/auth.c`.
|
||||
Using `weston` falls back to the system default PAM stack and
|
||||
rejects auth. The service is configured to `pam_permit.so` for
|
||||
all three module types (auth / account / session) so the
|
||||
browser's empty Apple-DH credentials (type 30) always pass —
|
||||
neatvnc ≥ 0.9 calls the PAM auth callback regardless of
|
||||
`weston.ini` `auth-method=none`, so the permit fallback is what
|
||||
actually lets the empty-cred client through.
|
||||
- **`Type = "simple"` (not `notify`)**: `switch-to-configuration`
|
||||
must never block on weston signalling readiness. A misconfigured
|
||||
weston degrades to a `Restart=on-failure` loop visible in
|
||||
`journalctl`, it does not abort the `nixos-container update`.
|
||||
Same reasoning as the `tea-login` unit in `nix/agent-modules/forge.nix`.
|
||||
- **`[core] idle-time=0`**: disables weston's 300-second idle
|
||||
timeout. Without it the VNC desktop fades to black and
|
||||
desktop-shell shows its click-to-unlock screen — useless for an
|
||||
agent desktop viewed over `/screen`. `idle-time=0` updates the
|
||||
idle timer with a 0ms delay, which
|
||||
`wl_event_source_timer_update` treats as "disarm", so the
|
||||
compositor never goes idle and never locks.
|
||||
|
||||
## Nix docs pipeline
|
||||
|
||||
### Nix options reference (`nix/docs/default.nix`)
|
||||
|
||||
`pkgs.nixosOptionsDoc` over two evaluated module trees:
|
||||
`hostEval` (a stub NixOS system loading the `nix/host-modules/` aggregator with every
|
||||
hyperhive subsystem `mkForce false` so heavy build inputs stay out of
|
||||
the eval) and `agentEval` (evaluates `agent.nix` fresh for the
|
||||
per-agent options tree).
|
||||
|
||||
Five output trees consumed by `flake.nix`, all **markdown**:
|
||||
|
||||
- `docs-host` — operator-facing host module options
|
||||
(`services.hyperhive.*` minus the `swarm`/`deploy` subtrees below)
|
||||
- `docs-swarm` — swarm-wide facts, identical on every host in the swarm
|
||||
(`services.hyperhive.swarm.*`)
|
||||
- `docs-deploy` — this host's own deployment decisions, necessarily
|
||||
different per host (`services.hyperhive.deploy.*` — added by
|
||||
`nix/host-modules/deploy.nix`, split out of `swarm.*` for exactly this
|
||||
reason: see that file's own comment)
|
||||
- `docs-agent` — per-agent harness options (`hyperhive.*`
|
||||
declared in `nix/agent-modules/`)
|
||||
- `docs` — bundle of `index.md` + `host.md` + `swarm.md` +
|
||||
`deploy.md` + `agent.md`
|
||||
|
||||
Pipeline:
|
||||
|
||||
- CommonMark from `nixosOptionsDoc.optionsCommonMark` is the only
|
||||
output — the source of truth, emitted as `.md`.
|
||||
- **HTML + CSS is rendered downstream by the website repo**
|
||||
(`nix/options.nix` there), which consumes this bundle's `host.md` /
|
||||
`agent.md`, renders them with `cmark-gfm`, and shares one
|
||||
stylesheet (`docs.css`) across `/options/` and the prose `/docs/`
|
||||
tree. Keeping rendering in the website means the theme has a single
|
||||
home and the colours are shared.
|
||||
- `transformOptions` strips the nix-store prefix from option
|
||||
declaration paths and rewrites them as forge URLs, so the
|
||||
rendered docs link back to the source.
|
||||
|
||||
Host options live entirely under `services.hyperhive.*`. The
|
||||
`pickSubtrees` filter is rooted at `["services" "hyperhive"]` so the
|
||||
options tree picks up everything under that root — picking against
|
||||
stray roots produces an empty tree and renders the host page as
|
||||
template chrome with no `<h2>` headers. `docs-host` then drops the
|
||||
`swarm`/`deploy` subtrees from that picked tree (`removeAttrs`, not a
|
||||
second `pickSubtrees` root — the one exclusion this file needs doesn't
|
||||
earn a general-purpose helper) since `docs-swarm`/`docs-deploy` each
|
||||
pick their own root instead.
|
||||
|
||||
#### Docs drv stability: `nixSrc`
|
||||
|
||||
Naively, the docs evaluation depends on `self` (the flake's store path),
|
||||
so every commit — even Rust-only or frontend-only changes — produces new
|
||||
docs drv hashes. The remote builder must rebuild docs from scratch for
|
||||
every PR branch, and if its store is full the build fails with a cached
|
||||
failure that blocks CI for the whole branch.
|
||||
|
||||
The fix (`nix/docs/default.nix`):
|
||||
|
||||
1. **`nixSrc`** — `builtins.path` on the `nix/` directory, wrapped in
|
||||
`builtins.unsafeDiscardStringContext` to strip `self`'s store-path
|
||||
context. The resulting store path is content-addressed from the nix/
|
||||
file contents only. Docs drvs only change when a `.nix` file changes.
|
||||
|
||||
2. The package options the modules consume (`hyperhive.packages.*`,
|
||||
`services.hyperhive.c0re.*`) carry no in-module defaults and every
|
||||
default that references them has a `defaultText`, so the doc walk
|
||||
never forces a package — no stubs needed, and the Rust/frontend
|
||||
build closure stays out of the eval.
|
||||
|
||||
3. Both `hostEval` and `agentEval` are evaluated from `nixSrc` paths
|
||||
(not `self`), so the docs drv dependency chain ends at `nixSrc`.
|
||||
|
||||
Why `builtins.unsafeDiscardStringContext`? The path string
|
||||
`toString self + "/nix"` carries `self`'s string context, which would
|
||||
make `builtins.path` include `self` as a build dependency even after
|
||||
content-addressing the directory. Discarding the context makes the
|
||||
resulting `nixSrc` truly independent of `self`'s store path.
|
||||
65
docs/process/pr-review-gate.md
Normal file
65
docs/process/pr-review-gate.md
Normal file
|
|
@ -0,0 +1,65 @@
|
|||
# The PR review gate
|
||||
|
||||
What a review verdict means, why a reviewer shouldn't wait on CI to
|
||||
submit one, and what "armed to auto-merge" actually signals about the
|
||||
human review that already happened.
|
||||
|
||||
## The gate has (up to) three parts, and they're per-repo settings
|
||||
|
||||
Whether a PR can merge, and what counts toward "can", is configured
|
||||
per repo in its branch-protection settings — not a fact true of every
|
||||
hive or every repo. The pieces a repo *can* require:
|
||||
|
||||
- **CI is green** — the repo's required status checks pass on the
|
||||
PR's current head commit, if the repo requires any.
|
||||
- **Requested reviews are `APPROVED`** — and whether a review is
|
||||
invalidated by a later commit ("stale") is itself a setting
|
||||
(Forgejo's "dismiss stale approvals" branch-protection option), not
|
||||
universal behavior.
|
||||
- **Someone with write access has armed the PR to merge** — a manual
|
||||
merge once the required conditions hold, or Forgejo's auto-merge
|
||||
(merges automatically the moment the other required conditions are
|
||||
met).
|
||||
|
||||
Where these are required, they're independent of each other. A
|
||||
reviewer only ever owns the review-approval piece — CI resolves (or
|
||||
doesn't) on its own regardless of what a review says, and merge-arming
|
||||
is someone else's call.
|
||||
|
||||
## Reviewers: submit the verdict, don't gate it on CI
|
||||
|
||||
Submit `hive-forge pr-reviews <pr> --approve` or `--request-changes`
|
||||
as soon as you've finished checking the diff — don't hold it back
|
||||
waiting for CI to go green first. Mention CI's current state in the
|
||||
review body if it's relevant (e.g. "approving; `nix flake check` is
|
||||
still running"), but don't gate the formal verdict on it: CI isn't a
|
||||
signal a reviewer waits on, it's a separate condition that resolves
|
||||
independently.
|
||||
|
||||
## What arming auto-merge actually means
|
||||
|
||||
Auto-merge isn't "no human ever looked at this." Whoever arms it has
|
||||
already judged the PR sound at a coarse level — the signal it sends is
|
||||
roughly *"apart from maybe minor tweaks a reviewer can still catch,
|
||||
I think this is fine."* That's the human-in-the-loop step, and it
|
||||
already happened. No large changes are expected to surface after
|
||||
that point — a reviewer's job past that point is to flag it if one
|
||||
does, not to assume none ever will.
|
||||
|
||||
The practical consequence for a reviewer: on a repo where auto-merge
|
||||
may already be armed before your review lands, a plain `APPROVED` can
|
||||
be the last step before the merge actually happens, with no further
|
||||
review pass after yours. That's a reason to actually finish checking
|
||||
before approving — not a reason to hesitate over every small thing.
|
||||
Genuine, substantive doubt (a claim you haven't verified, a real
|
||||
correctness question) is worth a `request-changes` or a clarifying
|
||||
comment before approving; a stray style nit isn't the same category.
|
||||
|
||||
## Re-review after a repo requires it
|
||||
|
||||
Where a repo dismisses stale approvals on a new commit, a review you
|
||||
already gave stops counting the moment a follow-up commit lands — even
|
||||
one whose message reads as trivial ("just a wording fix", "just
|
||||
trimming comments"). Re-diff and re-verify before submitting a fresh
|
||||
verdict; don't take a small-sounding commit message as an accurate
|
||||
description of the diff.
|
||||
Loading…
Reference in a new issue