diff --git a/CLAUDE.md b/CLAUDE.md index 6d5d3eb5..b990bfb8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -33,8 +33,9 @@ hand-maintained per-file tree drifts out of sync with the code. the separate `hivectl` crate, which talks to the daemon over the host admin socket. Owns the sqlite broker, approval + reminder + schedule queues, the meta flake, lifecycle (`nixos-container` - shellouts), gateway / forge / matrix provisioning, per-container stats, - and the axum operator dashboard (`dashboard.rs`). Largest crate. + shellouts), gateway / forge / matrix wiring (agent forge and matrix + accounts are `swarm-controller`'s), per-container stats, and the axum + operator dashboard (`dashboard/`). Largest crate. - **`hivectl/`** — standalone operator CLI (`hivectl` binary). Talks to the `hive-c0re` daemon over the host admin socket (`hive-host-sock` wire types) — does NOT link `hive-c0re`. Full, always-current verb @@ -47,11 +48,10 @@ hand-maintained per-file tree drifts out of sync with the code. - **`hive-agent/`** — the serve-loop binary: turn-loop *policy* layer (`turn.rs`) over `hive-runtime`, per-agent web UI (`web_ui/` module dir), event + turn-stats sqlite sinks, login flow, system-prompt - renderer. + renderer, and the claude launch-config layer (`mcp_config.rs`: + tool-group/capability → `--allowedTools`, `--mcp-config` render). - **`hive-agent-mcp/`** — the embedded MCP server (long-lived - streamable-http listener, `hive-mcp-http` systemd unit) + its claude - launch-config layer (tool-group/capability → `--allowedTools`, - `--mcp-config` render). + streamable-http listener, `hive-mcp-http` systemd unit). - **`hive-runtime/`** — the runtime an agent's turns run on: one `Runtime` interface (`run`/`compact`/`archive`/`canceller`/`choices`) with a `claude` backend (a pass-through to `hive-claude`) and an `acp` backend (any Agent Client @@ -89,7 +89,7 @@ hand-maintained per-file tree drifts out of sync with the code. journal, so both layers together store every record twice. Deliberately single-purpose — do not grow it into a utility crate. - **`hive-screen-mcp/`** — stdio MCP bridge for GUI agents - (`hyperhive.gui.enable`): `screenshot` via `grim`, `type_text` / + (`services.hyperhive.agent.gui.enable`): `screenshot` via `grim`, `type_text` / `key_press` via `wtype` (Wayland virtual-keyboard protocol), and `mouse_move` / `mouse_click` as RFB pointer events to the local neatvnc server. All userspace, no daemon of its own. @@ -152,7 +152,7 @@ hand-maintained per-file tree drifts out of sync with the code. `hive-core-agent-sock` above: **this socket never leaves the container** and `hive-c0re` is not in the path at all — no broker round-trip, no long-poll, no marker files. Also carries `extra_mcp`: the - `hyperhive.extraMcpServers` spec + parsing, shared by `hive-agent` and + `services.hyperhive.agent.extraMcpServers` spec + parsing, shared by `hive-agent` and `hive-subagent-mcp` — not a socket protocol, just the natural home since both already depend on this crate. - **`hive-types/`** — zero-dependency (bar serde) leaf crate holding the diff --git a/docs/turn-loop/README.md b/docs/turn-loop/README.md index 15338e7e..60823ca5 100644 --- a/docs/turn-loop/README.md +++ b/docs/turn-loop/README.md @@ -1,7 +1,20 @@ # Turn loop + MCP -How the harness wakes up, what it asks claude to do, and what tools -claude has access to in return. +How the harness wakes up, what it asks the agent's runtime to do, and what +tools the agent has access to in return. + +## Runtimes + +Every turn runs through `hive-runtime`, on the runtime +`services.hyperhive.agent.runtime` selects: + +- **`claude`** (default) — one `claude --print` process per turn. + → [claude-invocation.md](claude-invocation.md) +- **`acp`** — any Agent Client Protocol agent, run as one long-lived child + of the harness. → [`hive-runtime/README.md`](../../hive-runtime/README.md) + +Both report the turn in claude's `stream-json` shape, so the loop below is +the same for either. ## The loop @@ -22,21 +35,22 @@ agents) runs: 2. Pop one message. Peek the remaining inbox depth with `Status`. 3. Emit `LiveEvent::TurnStart { from, body, unread }` onto the SSE bus. -4. Spawn claude (one process per turn) and pipe the wake prompt - over stdin. -5. Stream stdout (JSON lines) into the bus as +4. Hand the wake prompt to the runtime: on `claude`, spawn + `claude --print` and pipe it over stdin; on `acp`, send it as a + prompt on the agent's session. +5. Stream the turn's `stream-json` lines into the bus as `LiveEvent::Stream(value)`. Pump stderr as `Note`. -6. Wait for claude to exit and classify the turn's outcome from the - stream + exit — success, compaction, rate-limit, auth-failure, or - hard failure. The outcome drives the post-turn action (see +6. Wait for the turn to end and classify its outcome from the + stream + result — success, compaction, rate-limit, auth-failure, stall, + or hard failure. The outcome drives the post-turn action (see [Turn outcomes](#turn-outcomes)); the session handles compaction internally (see [Compaction](claude-invocation.md#compaction)). This page describes - rate-limit and auth-failure detection [below](#failure-detection-and-login). + rate-limit and auth-failure detection [below](#failure-detection-and-login-claude). 7. Emit `LiveEvent::TurnEnd { ok, note }`. Sleep `poll_ms` to avoid tight loops on transient failures. -### Failure detection and login +### Failure detection and login (claude) - **Rate limit** — a `429` / `rate_limit` marker on stderr, or a parsed `{"type":"error"}` rate-limit event on stdout (conversation-text @@ -65,13 +79,16 @@ agents) runs: ## Harness binary shape -Two sibling crates, both role-agnostic (there is one role: agent — -the privilege boundary lives server-side at the broker socket -(`/run/hive/mcp.sock`), which refuses privileged `Request` variants -regardless of who sends them): +Two sibling binaries over one runtime library, all role-agnostic (there +is one role: agent — the privilege boundary lives server-side at the +broker socket (`/run/hive/mcp.sock`), which refuses privileged `Request` +variants regardless of who sends them): - `hive-agent` — long-running harness loop (the inbox poll + - claude-pump + ack/requeue cycle described above). + runtime turn + ack/requeue cycle described above). +- `hive-runtime` — the library every turn runs through, one `Runtime` + impl per runtime (`ClaudeRuntime`, `AcpRuntime`); `hive-subagent-mcp` + drives its nested sessions through it too. - `hive-agent-mcp` — MCP server for the built-in `hyperhive` surface. Run with `--http ` as a persistent streamable-HTTP daemon (the `hive-mcp-http` systemd unit, on `services.hyperhive.agent.mcp.httpPort`, default @@ -130,8 +147,8 @@ else a `TurnError`) drives the post-claude branch: | `Err(Failed(err))` | route `[system] \`\` claude turn failed:\n` to `operator` via `send_to_operator` | `ApiStall` catches an Anthropic API stall — a multi-retry connection storm where -the stream goes silent for minutes. The idle watchdog lives in `hive-claude`'s -driver (`Config::idle_timeout`, enforced around `child.wait()`): the timer +the stream goes silent for minutes. On `claude` the idle watchdog lives in +`hive-claude`'s driver (`Config::idle_timeout`, enforced around `child.wait()`): the timer resets on every stdout line, so a large but still-streaming turn is never cut — only complete output silence for the window trips it. The harness sets the window from `HIVE_TURN_IDLE_SECS` (`0` disables) and maps the driver's diff --git a/docs/turn-loop/claude-invocation.md b/docs/turn-loop/claude-invocation.md index edd6d0d1..07deba2b 100644 --- a/docs/turn-loop/claude-invocation.md +++ b/docs/turn-loop/claude-invocation.md @@ -1,20 +1,26 @@ # The claude invocation +How a turn runs on the `claude` runtime, the default +(`services.hyperhive.agent.runtime`). An `acp` agent's turns: +→ [`hive-runtime/README.md`](../../hive-runtime/README.md). + ``` claude --print --verbose --output-format stream-json --model \ --effort --resume # or --name <title> on first use \ - --system-prompt-file /run/hive/claude-system-prompt.md \ - --mcp-config /run/hive/claude-mcp-config.json --strict-mcp-config \ + --system-prompt-file /run/hive-config/claude-system-prompt.md \ + --mcp-config /run/hive-config/claude-mcp-config.json --strict-mcp-config \ --tools <builtins> --allowedTools <builtins+mcp> # wake prompt piped over stdin ``` -**Crate split.** The generic subprocess mechanics — spawning -`claude --print`, streaming + classifying stream-json, session -lookup/archive, and the durable-session compaction loop — live in the -reusable **`hive-claude`** crate (`hive_claude::{Claude, InfiniteSession, -Attach, CompactionPolicy, PercentPolicy, Telemetry, Sink, SessionStore}`; -see `hive-claude/README.md`). `hive-agent`'s `turn` module is the +**Crate split.** `hive-agent` runs every turn through `hive-runtime`, +whose `ClaudeRuntime` is a pass-through to the external **`hive-claude`** +crate (a crates.io dependency, not in this repo). `hive-claude` holds the +generic subprocess mechanics — spawning `claude --print`, streaming + +classifying stream-json, session lookup/archive, and the durable-session +compaction loop (`hive_claude::{Claude, InfiniteSession, Attach, +CompactionPolicy, PercentPolicy, Telemetry, Sink, SessionStore}`). +`hive-agent`'s `turn` module is the hyperhive **policy layer** on top: it builds the per-turn config from the bus, bridges the output stream onto the event bus (`BusSink`), and owns the compaction / autoreset / retry decisions in `drive_turn`. The @@ -278,5 +284,6 @@ identity, the reset/autoreset/retry state machine, and the telemetry-to-bus bridge — lives in `hive-agent`'s `turn` module; see its `//!` doc comment (`hive-agent/src/turn.rs`) for the exact call shape. The actual claude spawn, stream classification, and the -reactive/proactive compaction loop are in the `hive-claude` crate. +reactive/proactive compaction loop are in the `hive-claude` crate, reached +through `hive-runtime`'s `ClaudeRuntime`. Login-wait lives in `hive-agent`'s `login` module. diff --git a/hive-agent/README.md b/hive-agent/README.md index 2fadfb8f..54c38be0 100644 --- a/hive-agent/README.md +++ b/hive-agent/README.md @@ -1,8 +1,9 @@ # hive-agent The in-container harness serve-loop binary — one instance per agent. -Long-polls the broker inbox and drives one `claude --print` turn per -inbox message, over the `hive-claude` driver. There is one role here +Long-polls the broker inbox and drives one turn per inbox message +through `hive-runtime`, on the agent's runtime (`claude --print` by +default, or an ACP agent). It has one role (agent); the `Surface` trait + `AgentSurface` zero-sized type tag keep the turn loop generic and testable for future roles without a parallel copy of the loop. @@ -11,20 +12,21 @@ parallel copy of the loop. You don't call into this crate from elsewhere — it's the top-level binary systemd starts per agent container. Look here when you need to -understand or change: what happens between "a message lands in the -inbox" and "claude produces a reply", how login/auth is bootstrapped, -how the per-agent web UI is served, or how turn/event stats get +understand or change: what happens between a message landing in the +inbox and the agent's reply, how the harness bootstraps login/auth, +how it serves the per-agent web UI, or how turn/event stats get recorded. Architecture detail lives in [`docs/turn-loop/`](../docs/turn-loop/README.md); this README is just the map of the module tree. ## Shape +- **`main.rs`** — the serve loop and the `Surface` trait; broker calls + (inbox poll, ack, send) go through `hive-sock-client` with + `hive-core-agent-sock` wire types. - **`turn.rs`** — the turn-loop policy layer: renders the system - prompt + MCP config, invokes `hive-claude`, classifies the outcome, - and feeds the event/turn-stats sinks. -- **`client.rs`** — broker client (inbox poll, ack, send) speaking the - `hive-sh4re` wire protocol. + prompt + MCP config, runs the turn through `hive-runtime`, classifies + the outcome, and feeds the event/turn-stats sinks. - **`login.rs` / `login_session.rs`** — first-run and session-resume auth flow for the `claude` CLI. - **`mcp_config.rs`** — renders the per-turn `--mcp-config` / @@ -37,8 +39,6 @@ map of the module tree. - **`events.rs` / `turn_stats.rs` / `stats.rs`** — append-only event sink and per-turn telemetry recording (context usage, cost, tool favorites) under `harness/`. -- **`forge_notify.rs`** — subscribes to forge notifications and wakes - the harness on new activity. - **`prompt.rs`** — system-prompt renderer (persona + tool docs + environment facts). - **`web_ui/`** — the per-agent dashboard (terminal pane, status, @@ -46,6 +46,8 @@ map of the module tree. - **`paths.rs`** — canonical path resolution for state/harness dirs and the harness-local sqlite files. -Sibling: **`hive-agent-mcp`** (the MCP server this loop points claude -at every turn). Both are described together in -[`docs/turn-loop/::Harness binary shape`](../docs/turn-loop/README.md). +Siblings: **`hive-agent-mcp`** (the MCP server this loop points the +agent at every turn) and **`hive-runtime`** (the library a turn runs +through). +[`docs/turn-loop/::Harness binary shape`](../docs/turn-loop/README.md#harness-binary-shape) +covers all three together.