docs: give turn-loop/ a README.md landing page
Part of hyperhive#1898 (b): every docs subdir should have a top-level README.md link, achieved by moving/renaming where an existing file already fits the role. docs/turn-loop.md already served as the hub + index for the three sub-pages under turn-loop/ (claude-invocation.md, config.md, mcp.md), so it moves wholesale rather than leaving a redundant top-level pointer stub. Fixes every inbound/relative link across the repo (top-level README.md, CLAUDE.md, docs/persistence.md, docs/tools/scheduling.md, the sub-pages own back-link, hive-agent README + doc comments, hive-agent/Cargo.toml, .prettierignore per-file exemption entry) - grepped the whole tree for both turn-loop.md and turn-loop/ to find every reference rather than trusting a partial list. nix fmt clean, cargo check -p hive-agent clean.
This commit is contained in:
iris
parent
c6af5a2abf
commit
04b274753b
11 changed files with 21 additions and 21 deletions
|
|
@ -16,6 +16,6 @@ docs/persistence.md
|
|||
docs/terminal-rendering.md
|
||||
docs/tools/forge.md
|
||||
docs/tools/matrix.md
|
||||
docs/turn-loop.md
|
||||
docs/turn-loop/README.md
|
||||
docs/web-ui/agent.md
|
||||
docs/web-ui/dashboard.md
|
||||
|
|
|
|||
|
|
@ -179,7 +179,7 @@ read them à la carte.
|
|||
- **"How does the per-agent terminal classify + colour
|
||||
events?"** → [`docs/terminal-rendering.md`](docs/terminal-rendering.md).
|
||||
- **"How does claude get its prompt and what tools does it have?"** →
|
||||
[`docs/turn-loop.md`](docs/turn-loop.md) (index: the loop, binary shape,
|
||||
[`docs/turn-loop/`](docs/turn-loop/README.md) (index: the loop, binary shape,
|
||||
turn outcomes; sub-pages:
|
||||
[`claude-invocation`](docs/turn-loop/claude-invocation.md),
|
||||
[`config`](docs/turn-loop/config.md), [`mcp`](docs/turn-loop/mcp.md)).
|
||||
|
|
@ -239,7 +239,7 @@ The docs below own the details — this section just points at them.
|
|||
- **NixOS / nspawn quirks** (bind mounts, conf flags, etc.): →
|
||||
[`docs/gotchas.md`](docs/gotchas.md).
|
||||
- **Turn loop, sentinels (rate-limit, auth-failed), context
|
||||
window:** → [`docs/turn-loop.md`](docs/turn-loop.md).
|
||||
window:** → [`docs/turn-loop/`](docs/turn-loop/README.md).
|
||||
- **Two-step spawn, approval flow, flake.lock validation:** →
|
||||
[`docs/approvals.md`](docs/approvals.md).
|
||||
- **Pre-push lint hook** (catches tracker-tag and comment-block failures
|
||||
|
|
|
|||
|
|
@ -46,7 +46,7 @@ Depth lives in [`docs/`](docs/) — pick the one matching your task:
|
|||
| reading path | doc |
|
||||
| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| dashboard layout + endpoints | [`docs/web-ui.md`](docs/web-ui.md) ([shape](docs/web-ui/shape.md) · [dashboard](docs/web-ui/dashboard.md) · [agent](docs/web-ui/agent.md)) |
|
||||
| claude turn loop + MCP tools | [`docs/turn-loop.md`](docs/turn-loop.md) |
|
||||
| claude turn loop + MCP tools | [`docs/turn-loop/`](docs/turn-loop/README.md) |
|
||||
| config-edit + approval state machine | [`docs/approvals.md`](docs/approvals.md) |
|
||||
| what survives destroy / purge / restart | [`docs/persistence.md`](docs/persistence.md) |
|
||||
| naming, wire protocol, commit style | [`docs/conventions.md`](docs/conventions.md) |
|
||||
|
|
|
|||
|
|
@ -239,7 +239,7 @@ Survives destroy/recreate, gone on `--purge`.
|
|||
Empty marker file. Its presence parks the agent's turn loop: the
|
||||
harness keeps serving its web UI and MCP daemons but drives no turns,
|
||||
and inbox messages queue unacked until it's removed (see
|
||||
[turn loop](turn-loop.md#the-loop)).
|
||||
[turn loop](turn-loop/README.md#the-loop)).
|
||||
|
||||
Unusually, it's read and written from **both** sides of the harness
|
||||
bind-mount, and that's the whole design: the harness stats it
|
||||
|
|
|
|||
|
|
@ -89,6 +89,6 @@ lifecycle events, or another container's boot log.
|
|||
## See also
|
||||
|
||||
- `remind` (no-approval self-wake path) — documented in
|
||||
[`docs/turn-loop.md`](../turn-loop.md).
|
||||
[`docs/turn-loop/`](../turn-loop/README.md).
|
||||
- [`docs/approvals.md`](../approvals.md) — approval flow for
|
||||
`request_schedule_prompt`.
|
||||
|
|
|
|||
171
docs/turn-loop/README.md
Normal file
171
docs/turn-loop/README.md
Normal file
|
|
@ -0,0 +1,171 @@
|
|||
# Turn loop + MCP
|
||||
|
||||
How the harness wakes up, what it asks claude to do, and what tools
|
||||
claude has access to in return.
|
||||
|
||||
## The loop
|
||||
|
||||
Each agent harness (`hive-agent` — one serve-loop binary for all
|
||||
agents) runs:
|
||||
|
||||
0. Check the pause marker (`<harness>/paused`). While it exists the
|
||||
loop does nothing but re-stat it every 5 s — no broker poll, no
|
||||
claude process. Because step 1 is never reached, messages stay
|
||||
queued and unacked, so a resume drains the backlog instead of
|
||||
losing it; reminders and todo wakes buffer in their channels. Set
|
||||
it with `hivectl agent <name> pause` or the dashboard toggle; see
|
||||
[persistence](../persistence.md#-harnesspaused-per-agent).
|
||||
1. Long-poll `Recv` on its socket. The host-side broker
|
||||
(`broker.rs::recv_blocking_batch`) returns immediately if there's
|
||||
a pending message, otherwise waits up to 30 s for a broker `Sent`
|
||||
event for this recipient.
|
||||
2. Pop one message. Peek the remaining inbox depth with `Status`.
|
||||
3. Emit `LiveEvent::TurnStart { from, body, unread }` onto the SSE
|
||||
bus.
|
||||
4. Spawn claude (one process per turn) and pipe the wake prompt
|
||||
over stdin.
|
||||
5. Stream stdout (JSON lines) into the bus as
|
||||
`LiveEvent::Stream(value)`. Pump stderr as `Note`.
|
||||
6. Wait for claude to exit and classify the turn's outcome from the
|
||||
stream + exit — success, compaction, rate-limit, auth-failure, or
|
||||
hard failure. The outcome drives the post-turn action (see
|
||||
[Turn outcomes](#turn-outcomes)); compaction is handled inside the
|
||||
session (see
|
||||
[Compaction](claude-invocation.md#compaction)). Rate-limit
|
||||
and auth-failure detection is described [below](#failure-detection-and-login).
|
||||
7. Emit `LiveEvent::TurnEnd { ok, note }`. Sleep `poll_ms` to avoid
|
||||
tight loops on transient failures.
|
||||
|
||||
### Failure detection and login
|
||||
|
||||
- **Rate limit** — a `429` / `rate_limit` marker on stderr, or a parsed
|
||||
`{"type":"error"}` rate-limit event on stdout (conversation-text
|
||||
mentions don't count), sets the `rate_limited` sentinel, parks for
|
||||
`HIVE_RATE_LIMIT_SLEEP_SECS` (default 300), then retries. The UI shows
|
||||
a `⊘ rate limited` badge while parked.
|
||||
- **Auth failure (401)** — `drive_turn` retries once (transient
|
||||
token-refresh races clear on retry); a second `AuthFailed` writes
|
||||
`{state_dir}/hyperhive-needs-login`, requeues the message, and parks in
|
||||
`wait_for_login` — the same path as a cold boot with no session. The
|
||||
operator re-auths via the per-agent web UI; the queued message then
|
||||
drives the next turn.
|
||||
- **Login detection** — both boot (`login::has_session`, Online vs
|
||||
NeedsLogin) and `wait_for_login`'s resume check key off the credential
|
||||
files in `login::CRED_FILE_NAMES` (the set `/logout` deletes).
|
||||
`wait_for_login` resumes only when that set changes (a new file or a
|
||||
newer mtime), so stale credentials on disk at the 401 don't trigger an
|
||||
instant false-resume, and leftover session-history files don't read as a
|
||||
live session after a logout + container recreate.
|
||||
|
||||
## Harness binary shape
|
||||
|
||||
Two sibling binaries out of the one `hive-ag3nt` crate, all
|
||||
role-agnostic. (The earlier split into `hive-ag3nt` + `hive-m1nd`
|
||||
was collapsed because the privilege boundary lives server-side at
|
||||
the broker socket (`/run/hive/mcp.sock`): `ManagerRequest` calls are
|
||||
refused by the standard agent socket regardless of who sends them.)
|
||||
|
||||
- `hive-agent` — long-running harness loop (the inbox poll +
|
||||
claude-pump + ack/requeue cycle described above).
|
||||
- `hive-agent-mcp` — MCP server for the built-in `hyperhive` surface.
|
||||
Run with `--http <addr>` as a persistent streamable-HTTP daemon (the
|
||||
`hive-mcp-http` systemd unit, on `hyperhive.mcp.httpPort`, default
|
||||
8790); claude connects to its URL via `--mcp-config`. HTTP is the sole
|
||||
transport — no per-turn stdio child (eliminates the re-registration
|
||||
race).
|
||||
|
||||
### `Surface` trait + zero-sized type tags
|
||||
|
||||
`AgentRequest` / `AgentResponse` (= `ManagerRequest` / `ManagerResponse` —
|
||||
type aliases) are the wire types. There is one role: agent.
|
||||
`bin/hive-agent.rs` factors the turn loop through a `Surface` trait
|
||||
with one zero-sized impl (`AgentSurface`) wrapping:
|
||||
|
||||
- One async method per wire op: `ack_turn`, `requeue_inflight`,
|
||||
`inbox_unread`, `post_turn_counts`, `send_to_parent`, `recv_next`.
|
||||
|
||||
`main()` calls `serve_main::<AgentSurface>` for all roles. The turn
|
||||
loop (`serve_loop` / `handle_turn`) has no per-role branches.
|
||||
|
||||
### Boot wiring
|
||||
|
||||
`serve_main` reads `HIVE_PORT` (default `DEFAULT_WEB_PORT`) +
|
||||
`HIVE_LABEL` (default `"hive"` for standalone runs; the meta
|
||||
flake sets it unconditionally for any container-deployed agent;
|
||||
see `docs/conventions.md::Hive identity` for the env stack),
|
||||
opens turn-stats sqlite, prepares the on-boot files (see
|
||||
[claude-invocation](claude-invocation.md#on-boot-files)),
|
||||
installs claude plugins, spawns `web_ui::serve` + `vacuum::run`,
|
||||
and either drops into `serve_loop` directly (`Online`) or parks on
|
||||
the login flow first (`NeedsLogin`). (The forge notification poller
|
||||
used to be spawned here too; it is its own process now —
|
||||
`hive-forge-notify`, see [`forge.md`](../forge.md).)
|
||||
|
||||
`spawn_todo_socket` opens the todos store and, alongside
|
||||
`todo_server::run` (the socket the out-of-process producers dial),
|
||||
spawns `disk_watch::run` — an *in-process* todo producer, so it shares
|
||||
the store + wake `Notify` directly rather than dialling its own socket.
|
||||
It raises a keyed `disk` todo when the filesystem backing the agent's
|
||||
state gets tight, naming the agent's own biggest directories; the
|
||||
summary is bucketed and carries no raw byte counts, so an unchanged
|
||||
situation re-upserts as `changed == false` and never re-wakes.
|
||||
|
||||
Plugin install failures are not fatal: each entry comes back as a
|
||||
human-readable failure string that gets routed via
|
||||
`Surface::send_to_parent` to the agent's topology parent (the
|
||||
broker resolves `<parent>` per `topology::parent_of`; root agents
|
||||
and the manager fall through to operator).
|
||||
|
||||
### Turn outcomes
|
||||
|
||||
`turn::TurnOutcome` (`Result<bool, TurnError>` — `Ok(compacted)` on success,
|
||||
else a `TurnError`) drives the post-claude branch:
|
||||
|
||||
| Outcome | Action |
|
||||
| --- | --- |
|
||||
| `Ok(_)` (`false` normal / `true` compacted) | `ack_turn` |
|
||||
| `Err(PromptTooLong)` | `drive_turn` archived the session (the lib already compacted + retried and it still overflowed); requeue inflight so the message redelivers into a fresh session that fits — no status park |
|
||||
| `Err(RateLimited)` | sleep `HIVE_RATE_LIMIT_SLEEP_SECS` (default 300), requeue inflight, status back to `online` |
|
||||
| `Err(AuthFailed)` | emit `needs_login_idle` sentinel, requeue inflight, park in `wait_for_login` |
|
||||
| `Err(SessionNotFound)` | resume + create self-heal both missed ("shouldn't happen"); requeue inflight so the next turn creates fresh — no status park, message not dropped |
|
||||
| `Err(ApiStall)` | idle watchdog killed claude after `HIVE_TURN_IDLE_SECS` (default 600) of output silence; sleep `HIVE_STALL_SLEEP_SECS` (default 60), requeue inflight, status back to `online` |
|
||||
| `Err(Failed(err))` | route `[system] \`<qualified-label>\` claude turn failed:\n<err>` to `<parent>` via `send_to_parent` |
|
||||
|
||||
`ApiStall` catches an Anthropic API stall — a multi-retry connection storm where
|
||||
the stream goes silent for minutes. The idle watchdog lives in `hive-claude`'s
|
||||
driver (`Config::idle_timeout`, enforced around `child.wait()`): the timer
|
||||
resets on every stdout line, so a large but still-streaming turn is never cut —
|
||||
only complete output silence for the window trips it. The harness sets the
|
||||
window from `HIVE_TURN_IDLE_SECS` (`0` disables) and maps the driver's
|
||||
`Error::IdleTimeout` onto `TurnError::ApiStall`.
|
||||
|
||||
After the outcome handler, the stats sink records a row. `handle_turn`
|
||||
reports the result to `serve_loop` via `TurnControl { auth_failed }` —
|
||||
on auth failure the loop parks in `wait_for_login`; otherwise it loops
|
||||
straight back to the idle wait (step 1). There is no same-turn
|
||||
self-continue mechanism: every multi-step continuation rides an
|
||||
external wake instead — a new inbox message, a `remind`, or an
|
||||
in-container todo wake (bash-task completion, forge notification,
|
||||
matrix activity). Ending the turn and letting one of
|
||||
those drive the next one is strictly better than parking in-process:
|
||||
it checkpoints the session and observes wakes that only reach the
|
||||
harness between turns.
|
||||
|
||||
## Sub-pages
|
||||
|
||||
The rest lives alongside this page, in three topic files:
|
||||
|
||||
- **[claude-invocation.md](claude-invocation.md)** — how the harness
|
||||
spawns `claude --print` each turn, the two-pronged compaction (reactive +
|
||||
proactive), and the on-boot files it materialises (`--mcp-config`,
|
||||
`--system-prompt-file`).
|
||||
- **[config.md](config.md)** — the optional per-agent knobs the meta
|
||||
flake wires in (reference docs, icon, passwordless sudo, dashboard links,
|
||||
custom static files, connectivity overrides, claude plugins, cargo message
|
||||
filtering).
|
||||
- **[mcp.md](mcp.md)** — the MCP tool surface claude sees: core tools,
|
||||
privileged tool groups, self-wake, authoritative state, the tool envelope,
|
||||
and the built-in tool whitelist.
|
||||
|
||||
Per-subsystem impl detail lives in each module's `//!` doc-comment; these pages
|
||||
describe present-state behaviour + wiring, not line-level mechanics.
|
||||
|
|
@ -116,7 +116,7 @@ window with two triggers baked into its `run`:
|
|||
*still* overflows, `run` surfaces `Error::PromptTooLong`; `drive_turn`
|
||||
then archives the session (session lifecycle stays hive-side) and the
|
||||
serve loop requeues the message so it redelivers into a fresh session
|
||||
(see [Turn outcomes](../turn-loop.md#turn-outcomes) — the wake prompt itself is tiny, so
|
||||
(see [Turn outcomes](README.md#turn-outcomes) — the wake prompt itself is tiny, so
|
||||
the overflow was the accumulated context the archive clears).
|
||||
- **Proactive** — a turn finishes cleanly but the last inference's context
|
||||
size crossed the policy watermark. While the session is still healthy it
|
||||
|
|
|
|||
|
|
@ -41,4 +41,4 @@ tempfile = "3"
|
|||
# The sibling MCP server is its own bin crate now (`hive-agent-mcp`).
|
||||
# Privilege boundary is enforced server-side at the socket (tool
|
||||
# groups / manager surface).
|
||||
# See `docs/turn-loop.md::Harness binary shape`.
|
||||
# See `docs/turn-loop/README.md::Harness binary shape`.
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ understand or change: what happens between "a message lands in the
|
|||
inbox" and "claude produces a reply", how login/auth is bootstrapped,
|
||||
how the per-agent web UI is served, or how turn/event stats get
|
||||
recorded. Architecture detail lives in
|
||||
[`docs/turn-loop.md`](../docs/turn-loop.md); this README is just the
|
||||
[`docs/turn-loop/`](../docs/turn-loop/README.md); this README is just the
|
||||
map of the module tree.
|
||||
|
||||
## Shape
|
||||
|
|
@ -48,4 +48,4 @@ map of the module tree.
|
|||
|
||||
Sibling: **`hive-agent-mcp`** (the MCP server this loop points claude
|
||||
at every turn). Both are described together in
|
||||
[`docs/turn-loop.md::Harness binary shape`](../docs/turn-loop.md).
|
||||
[`docs/turn-loop/::Harness binary shape`](../docs/turn-loop/README.md).
|
||||
|
|
|
|||
|
|
@ -118,7 +118,7 @@ impl LoginState {
|
|||
/// infinite-401 loop a bare-existence check would produce when stale
|
||||
/// credentials are already on disk. Mtime-snapshot resumption rationale
|
||||
/// and `DirSnapshot` two-axis design: see
|
||||
/// [`docs/turn-loop.md::The loop`](../../docs/turn-loop.md).
|
||||
/// [`docs/turn-loop/::The loop`](../../docs/turn-loop/README.md).
|
||||
///
|
||||
/// # Panics
|
||||
///
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@
|
|||
//! generic and testable. Sibling: `hive-agent-mcp` (the MCP server this
|
||||
//! loop points claude at).
|
||||
//! Architecture lives in
|
||||
//! [`docs/turn-loop.md::Harness binary shape`](../../../docs/turn-loop.md).
|
||||
//! [`docs/turn-loop/::Harness binary shape`](../../../docs/turn-loop/README.md).
|
||||
//!
|
||||
//! Single bin crate: the module tree below (formerly this crate's `lib.rs`,
|
||||
//! before lib + bin were collapsed into one) plus the serve loop.
|
||||
|
|
@ -510,7 +510,7 @@ fn spawn_todo_socket(
|
|||
/// Boot — wires up the web UI, login state, stats, plugins, forge
|
||||
/// notifier, and either drops into `serve_loop` directly (`Online`) or
|
||||
/// parks on the login flow first (`NeedsLogin`). See
|
||||
/// `docs/turn-loop.md::Boot wiring`.
|
||||
/// `docs/turn-loop/README.md::Boot wiring`.
|
||||
async fn serve_main<S: Surface>(socket: &Path, poll_ms: u64) -> Result<()> {
|
||||
let port = std::env::var("HIVE_PORT")
|
||||
.ok()
|
||||
|
|
|
|||
Loading…
Reference in a new issue