7a826f9ee2
Six places in the tree hand-rolled the same connect / write one JSON line / read one JSON line back. Two of them — the harness serve loop's client and the MCP server's — were byte-identical apart from a six-line wrapper, ~145 lines of literal copy-paste. The other four each reimplemented a subset, and the subsets had drifted: some named the socket path in their errors and some did not, one classified transient against fatal failures and the rest retried nothing at all, two drained the response and two decoded it. That duplication was defended when the daemons were split out, on the grounds that a daemon's socket etiquette should stay visible in the crate that depends on it. The etiquette genuinely does differ. The code does not, and five copies is where "each daemon documents its own etiquette" stops paying for itself. `hive-sock-client` now owns the transport once, generic over the request and response types so it is protocol-agnostic: the host-served control socket and the harness's in-agent socket both use it with their own wire-type crates. The two real differences become values instead of forks. Retry is `Retry::RideOutRestart` (2/4/8/16/30s, sized to ride out a service restart) for callers with no natural retry of their own, or `Retry::None` for callers already inside a poll loop where the poll interval is the retry — and the reason each caller picked one is a comment at the call site rather than a reimplementation. The response is either decoded (`request`) or half-closed and drained (`notify`, where the drain exists so the server's write-back doesn't land on a closed socket). Whether a failure propagates or is logged and swallowed stays at the call site, because that is the caller's choice and not a property of the transport. Errors always name the socket path now, everywhere. That detail is load-bearing: a permission problem on a socket that reads as "is the daemon running?" sends the operator to fix the wrong thing. The transient-against-fatal enum is gone rather than moved. Serialising happens before the retry loop and deserialising after it, so only connect, I/O and short-read failures can reach the loop at all — a deterministic failure is now unretryable by construction instead of by classification. It is deliberately a new crate and not part of `hive-agent-sock`. The `*-sock` crates are pure wire types by convention — `hive-agent-sock` depends on serde and nothing else — and the two largest copies talk to the host socket, whose types live in a different crate entirely. A transport in either wire-type crate would drag tokio into it and point the wrong way besides. No wire-format change: same JSON line in, same line out.
194 lines
10 KiB
Markdown
194 lines
10 KiB
Markdown
# hyperhive — claude entry point
|
|
|
|
Hey claude. This is your starting page. The detailed docs live in
|
|
[`docs/`](docs/) and are written for humans + you both — read them
|
|
when you need depth on a subsystem. This file is the index.
|
|
|
|
- High-level project intro: **[README.md](README.md)**.
|
|
- Open work + backlog: the **[forge issue
|
|
tracker](http://localhost:3000/hyperhive/hyperhive/issues)**.
|
|
- Operator/agent trust-boundary design:
|
|
**[docs/boundary.md](docs/boundary.md)** (`area:ops` issues
|
|
for the deployment/gateway/privsep work).
|
|
- Agent trust model (trust boundary, prompt-injection threat model,
|
|
capability = accepted risk), credential isolation + sandbox threat model:
|
|
**[docs/security.md](docs/security.md)**.
|
|
|
|
## Repo map
|
|
|
|
One line per crate / top-level dir. **Each module's authoritative,
|
|
always-current description lives in its own `//!` doc-comment** —
|
|
`grep`/read the module when you need detail. This index is kept
|
|
deliberately lean: it auto-loads into every turn's context, and a
|
|
hand-maintained per-file tree drifts out of sync with the code.
|
|
|
|
### Rust workspace (`Cargo.toml` members)
|
|
|
|
- **`hive-c0re/`** — host daemon (runs as the unprivileged `hive-core`
|
|
user). `src/main.rs` is the `hive-c0re` binary — **daemon-only**
|
|
(`serve` + the periodic vacuum/sweep loops); the operator CLI lives in
|
|
the separate `hivectl` crate, which talks to the daemon over the host
|
|
admin socket. Owns the sqlite broker, approval + question + reminder +
|
|
schedule queues, the meta flake, lifecycle (`nixos-container`
|
|
shellouts), gateway / forge / matrix provisioning, per-container stats,
|
|
and the axum operator dashboard (`dashboard.rs`). Largest crate.
|
|
- **`hivectl/`** — standalone operator CLI (`hivectl` binary). Talks to
|
|
the `hive-c0re` daemon over the host admin socket (`hive-host-sock`
|
|
wire types) — does NOT link `hive-c0re`. Verbs: `agents <spawn|kill|
|
|
destroy|rebuild|restart|list|set-parent|…>`, `approvals <pending|
|
|
approve|deny>`, `forge`/`matrix`/`github`/`gateway` provisioning,
|
|
`choom`, `stop`/`start`, `wg`/`peer-config`.
|
|
- **`hive-agent/`**, **`hive-agent-mcp/`** —
|
|
in-container harness, two sibling crates for every agent (not a
|
|
single `hive-ag3nt/` dir — that's the runtime/binary-family nickname,
|
|
not a directory).
|
|
- **`hive-agent/`** — the serve-loop binary: turn-loop *policy* layer
|
|
(`turn.rs`) over the `hive-claude` driver, per-agent web UI (`web_ui/`
|
|
module dir), event + turn-stats sqlite sinks, login flow, system-prompt
|
|
renderer.
|
|
- **`hive-agent-mcp/`** — the embedded MCP server (long-lived
|
|
streamable-http listener, `hive-mcp-http` systemd unit) + its claude
|
|
launch-config layer (tool-group/capability → `--allowedTools`,
|
|
`--mcp-config` render).
|
|
- **`hive-claude/`** — reusable, app-agnostic driver for headless
|
|
`claude --print`: spawns the CLI, streams + classifies stream-json,
|
|
parses per-turn `Telemetry`, and drives a durable self-compacting
|
|
`InfiniteSession` (name + `SessionStore` + `CompactionPolicy`). Uses
|
|
`thiserror` (it's a library); the `hive-*` binaries consume it with
|
|
`anyhow`. See `hive-claude/README.md`.
|
|
- **`hive-priv/`** — minimal root privileged-helper, socket-activated at
|
|
`/run/hive/priv.sock`; performs the few root operations (bind-mount
|
|
edits, nsenter) the unprivileged `hive-c0re` delegates to it. See
|
|
`docs/boundary.md`.
|
|
- **`hive-forge/`** — `hive-forge` Forgejo CLI wrapper; one module per
|
|
verb under `src/verbs/`.
|
|
- **`hive-forge-notify/`** — per-agent Forgejo notification poller daemon
|
|
(`hive-forge-notify`); turns unread notification threads into todos on
|
|
the harness's in-agent socket. Was a task inside the `hive-agent` serve
|
|
loop; own process since it needs nothing else from the harness.
|
|
- **`hive-matrix-mcp/`** — per-agent matrix-sdk daemon
|
|
(`hive-matrix-daemon`); serves its MCP tools (`send_message`,
|
|
`read_room`, …) directly over streamable-http (no stdio bridge),
|
|
same shape as `hive-bash-mcp`.
|
|
- **`hive-bash-mcp/`** — per-agent bash-task runner daemon
|
|
(`hive-bash-daemon`); serves its MCP tools (`run`/`status`/`kill`)
|
|
directly over streamable-http (no stdio bridge), writes task files
|
|
under `/harness/bash-tasks/`, and records the favorite-tools
|
|
`bash_commands` stat into turn-stats.sqlite.
|
|
- **`hive-sh4re/`** — shared wire types (Agent / Manager request +
|
|
response, `Message`, `Approval`, `HelperEvent`) used across the unix
|
|
sockets. Host-admin-socket and hive-priv-socket wire types have been
|
|
split out into their own crates (below) so `hivectl` and `hive-priv`
|
|
don't need to pull in the rest of `hive-sh4re`.
|
|
- **`hive-host-sock/`** — wire types for the host admin socket
|
|
(`/run/hyperhive/host.sock`), the protocol `hivectl` speaks to
|
|
`hive-c0re`. Split out of `hive-sh4re` so a standalone `hivectl` only
|
|
depends on this protocol crate, not the whole daemon crate.
|
|
- **`hive-priv-sock/`** — wire types for the `hive-priv` privileged-helper
|
|
socket (`/run/hive/priv.sock`), shared by `hive-priv` (server) and
|
|
`hive-c0re` (client). Also split out of `hive-sh4re`.
|
|
- **`hive-sock-client/`** — the shared JSON-line-over-unix-socket client
|
|
every daemon uses to talk to a hyperhive socket. Generic over the
|
|
request/response types, so the host-served control socket and the
|
|
harness's in-agent socket both use it with their own wire-type crates.
|
|
Retry is a policy value (`Retry::None` for callers already inside a poll
|
|
loop, `Retry::RideOutRestart` for callers with no natural retry), and
|
|
the response is either decoded (`request`) or drained (`notify`).
|
|
Deliberately separate from the `*-sock` crates — those stay
|
|
dependency-free wire types.
|
|
- **`hive-metric/`** — small CLI to push a single labeled metric to the
|
|
OTEL collector via the OpenTelemetry Rust SDK / OTLP HTTP exporter.
|
|
|
|
### Other top-level dirs
|
|
|
|
- **`frontend/`** — npm workspaces → static dashboard + per-agent UI
|
|
dist, built hermetically by `nix/packages/frontend.nix`. Packages:
|
|
`shared` (terminal pane + Catppuccin palette), `dashboard` (the
|
|
operator SPA), `agent` (the default per-container UI).
|
|
- **`nix/`** — `host-modules/` (the host stack: hyperhive core options,
|
|
`hive-{c0re,priv,forge,gateway,matrix,network,tls,ci}`, otel, swarm),
|
|
`agent-modules/` (the per-agent harness feature modules),
|
|
`templates/{agent,ruth}.nix` (container entry points), `packages/`
|
|
(flake package outputs), `docs/` (the options-doc derivation), plus
|
|
`sources.nix` / `rust.nix` / `checks.nix` / `devshell.nix` /
|
|
`treefmt.nix` behind the thin `flake.nix`.
|
|
- **`docs/`** — subsystem reference docs (see *Reading paths* below).
|
|
- **`branding/`**, **`scripts/`** — static assets + helper scripts.
|
|
|
|
## Reading paths
|
|
|
|
Pick the doc that matches your task. None depend on the others —
|
|
read them à la carte.
|
|
|
|
- **"How do I bring a fresh hive online (first-run hivectl
|
|
bootstrap)?"** → [`docs/setup.md`](docs/setup.md).
|
|
- **"What does the dashboard look like?"** →
|
|
[`docs/web-ui.md`](docs/web-ui.md) (index; sub-pages:
|
|
[`shape`](docs/web-ui/shape.md),
|
|
[`dashboard`](docs/web-ui/dashboard.md),
|
|
[`agent`](docs/web-ui/agent.md)).
|
|
- **"How does the per-agent terminal classify + colour
|
|
events?"** → [`docs/terminal-rendering.md`](docs/terminal-rendering.md).
|
|
- **"How does claude get its prompt and what tools does it have?"** →
|
|
[`docs/turn-loop.md`](docs/turn-loop.md) (index: the loop, binary shape,
|
|
turn outcomes; sub-pages:
|
|
[`claude-invocation`](docs/turn-loop/claude-invocation.md),
|
|
[`config`](docs/turn-loop/config.md), [`mcp`](docs/turn-loop/mcp.md)).
|
|
- **"How do config changes flow from manager to operator to
|
|
container?"** → [`docs/approvals.md`](docs/approvals.md).
|
|
- **"What state survives destroy / purge / restart?"** →
|
|
[`docs/persistence.md`](docs/persistence.md).
|
|
- **"Naming, commit style, wire protocol, the `data-async`
|
|
pattern."** → [`docs/conventions.md`](docs/conventions.md).
|
|
- **"Why does the nspawn flag look like that?"** →
|
|
[`docs/gotchas.md`](docs/gotchas.md).
|
|
- **"What nginx vhosts does the gateway serve? How does matrix
|
|
discovery work?"** → [`docs/gateway.md`](docs/gateway.md).
|
|
- **"How do per-agent forge accounts work? What does forge_notify
|
|
poll + how does it format wake messages?"** →
|
|
[`docs/forge.md`](docs/forge.md).
|
|
- **"What verbs does `hive-forge` support? How do I post a comment,
|
|
upload an attachment, manage subscriptions?"** →
|
|
[`docs/tools/forge.md`](docs/tools/forge.md).
|
|
- **"What does `hivectl` do? How do I provision a forge/matrix account,
|
|
manage gateway users, restart containers, or drop into an agent shell?"** →
|
|
[`docs/tools/hivectl.md`](docs/tools/hivectl.md).
|
|
- **"How does the matrix-tuwunel container work? What about
|
|
fluffychat-web and per-agent matrix accounts?"** →
|
|
[`docs/matrix.md`](docs/matrix.md).
|
|
- **"How do I give an agent a GitHub account (`gh` + `git push`)?
|
|
How is the PAT injected?"** → [`docs/github.md`](docs/github.md).
|
|
- **"How does DNS resolution work in agent containers? What's the
|
|
bridge network for?"** → [`docs/network.md`](docs/network.md).
|
|
- **"How do I connect two hives into a swarm? How do I declare peer
|
|
hives and configure TLS trust?"** →
|
|
[`docs/swarm.md`](docs/swarm.md).
|
|
- **"How does the rebuild queue work? What are queue kinds and sources?"** →
|
|
[`docs/coordinator.md`](docs/coordinator.md).
|
|
- **"How does the CI runner work? What's the auto-registration flow?"** →
|
|
[`docs/ci.md`](docs/ci.md).
|
|
- **"What is `/knowledge`? How does the hive-wide knowledge repo sync,
|
|
and how do I contribute a document?"** →
|
|
[`docs/knowledge.md`](docs/knowledge.md).
|
|
- **"How do I export Claude Code metrics (tokens, cost, tool calls) to
|
|
a Prometheus/Grafana collector? What OTEL options are available?"** →
|
|
[`docs/observability.md`](docs/observability.md).
|
|
|
|
## Conventions & process
|
|
|
|
The docs below own the details — this section just points at them.
|
|
|
|
- **Commit style, naming, identity, reconcile verb:** →
|
|
[`docs/conventions.md`](docs/conventions.md).
|
|
- **Never add `#[allow(clippy::…)]`** — fix the lint instead (extract a
|
|
helper, add backticks, etc.). Details + worked examples: →
|
|
[`docs/conventions.md`](docs/conventions.md#building--local-checks).
|
|
- **NixOS / nspawn quirks** (bind mounts, conf flags, etc.): →
|
|
[`docs/gotchas.md`](docs/gotchas.md).
|
|
- **Turn loop, sentinels (rate-limit, auth-failed), context
|
|
window:** → [`docs/turn-loop.md`](docs/turn-loop.md).
|
|
- **Two-step spawn, approval flow, flake.lock validation:** →
|
|
[`docs/approvals.md`](docs/approvals.md).
|
|
- **Pre-push lint hook** (catches tracker-tag and comment-block failures
|
|
before CI does — install once per clone):
|
|
`ln -sf ../../scripts/pre-push .git/hooks/pre-push`
|