78021ce982
Apply contraction fixes across ~40 doc files (setup, integrations, lifecycle, networking, scheduler, swarm, tools, trust-boundary, UI, etc.). Skipped 14 hits: - 10 where words appear in ALL CAPS for deliberate emphasis (is NOT, do NOT, etc.) - 4 where text could not be safely located due to markdown formatting or column position Applied via systematic scan with checks for fenced code blocks, inline code spans, and intentional caps. Preserves sentence-initial capitalization throughout.
170 lines
9 KiB
Markdown
170 lines
9 KiB
Markdown
# Turn loop + MCP
|
|
|
|
How the harness wakes up, what it asks claude to do, and what tools
|
|
claude has access to in return.
|
|
|
|
## The loop
|
|
|
|
Each agent harness (`hive-agent` — one serve-loop binary for all
|
|
agents) runs:
|
|
|
|
0. Check the pause marker (`<harness>/paused`). While it exists the
|
|
loop does nothing but re-stat it every 5 s — no broker poll, no
|
|
claude process. Because step 1 is never reached, messages stay
|
|
queued and unacked, so a resume drains the backlog instead of
|
|
losing it; reminders and todo wakes buffer in their channels. Set
|
|
it with `hivectl agent <name> pause` or the dashboard toggle; see
|
|
[persistence](../agent-lifecycle/persistence.md#-harnesspaused-per-agent).
|
|
1. Long-poll `Recv` on its socket. The host-side broker
|
|
(`broker.rs::recv_blocking_batch`) returns immediately if there's
|
|
a pending message, otherwise waits up to 30 s for a broker `Sent`
|
|
event for this recipient.
|
|
2. Pop one message. Peek the remaining inbox depth with `Status`.
|
|
3. Emit `LiveEvent::TurnStart { from, body, unread }` onto the SSE
|
|
bus.
|
|
4. Spawn claude (one process per turn) and pipe the wake prompt
|
|
over stdin.
|
|
5. Stream stdout (JSON lines) into the bus as
|
|
`LiveEvent::Stream(value)`. Pump stderr as `Note`.
|
|
6. Wait for claude to exit and classify the turn's outcome from the
|
|
stream + exit — success, compaction, rate-limit, auth-failure, or
|
|
hard failure. The outcome drives the post-turn action (see
|
|
[Turn outcomes](#turn-outcomes)); compaction is handled inside the
|
|
session (see
|
|
[Compaction](claude-invocation.md#compaction)). Rate-limit
|
|
and auth-failure detection is described [below](#failure-detection-and-login).
|
|
7. Emit `LiveEvent::TurnEnd { ok, note }`. Sleep `poll_ms` to avoid
|
|
tight loops on transient failures.
|
|
|
|
### Failure detection and login
|
|
|
|
- **Rate limit** — a `429` / `rate_limit` marker on stderr, or a parsed
|
|
`{"type":"error"}` rate-limit event on stdout (conversation-text
|
|
mentions don't count), sets the `rate_limited` sentinel, parks for
|
|
`HIVE_RATE_LIMIT_SLEEP_SECS` (default 300), then retries. The UI shows
|
|
a `⊘ rate limited` badge while parked.
|
|
- **Auth failure (401)** — `drive_turn` retries once (transient
|
|
token-refresh races clear on retry); a second `AuthFailed` writes
|
|
`{state_dir}/hyperhive-needs-login`, requeues the message, and parks in
|
|
`wait_for_login` — the same path as a cold boot with no session. The
|
|
operator re-auths via the per-agent web UI; the queued message then
|
|
drives the next turn.
|
|
- **Login detection** — both boot (`login::has_session`, Online vs
|
|
NeedsLogin) and `wait_for_login`'s resume check key off the credential
|
|
files in `login::CRED_FILE_NAMES` (the set `/logout` deletes).
|
|
`wait_for_login` takes a `since: SystemTime` baseline (the instant of
|
|
the 401 that parked it, or `login::NO_PRIOR_FAILURE` at cold boot) and
|
|
resumes only once a credential file's mtime postdates it — so stale
|
|
credentials already on disk at the 401 don't trigger an instant
|
|
false-resume, and a login that lands *before* `wait_for_login` even
|
|
starts polling still resumes correctly (baselining on a fixed instant
|
|
rather than an entry-time directory snapshot is what closes that race).
|
|
Leftover session-history files still don't read as a live session after
|
|
a logout + container recreate (`has_session`/`is_cred_file` scope to
|
|
`CRED_FILE_NAMES` either way).
|
|
|
|
## Harness binary shape
|
|
|
|
Two sibling crates, both role-agnostic (there is one role: agent —
|
|
the privilege boundary lives server-side at the broker socket
|
|
(`/run/hive/mcp.sock`), which refuses privileged `Request` variants
|
|
regardless of who sends them):
|
|
|
|
- `hive-agent` — long-running harness loop (the inbox poll +
|
|
claude-pump + ack/requeue cycle described above).
|
|
- `hive-agent-mcp` — MCP server for the built-in `hyperhive` surface.
|
|
Run with `--http <addr>` as a persistent streamable-HTTP daemon (the
|
|
`hive-mcp-http` systemd unit, on `hyperhive.mcp.httpPort`, default
|
|
8790); claude connects to its URL via `--mcp-config`. HTTP is the sole
|
|
transport — no per-turn stdio child (eliminates the re-registration
|
|
race).
|
|
|
|
`hive-agent`'s wire types (`hive_core_agent_sock::{Request, Response}` —
|
|
one unified enum shared by the agent and manager sockets) and its turn
|
|
loop are factored through a small `Surface` trait with one zero-sized
|
|
impl, so the loop itself has no per-role branches. See
|
|
`hive-agent/src/main.rs`'s module
|
|
doc for the trait shape.
|
|
|
|
### Boot wiring
|
|
|
|
`serve_main` reads `HIVE_PORT` (default `DEFAULT_WEB_PORT`) +
|
|
`HIVE_LABEL` (default `"hive"` for standalone runs; the meta
|
|
flake sets it unconditionally for any container-deployed agent;
|
|
see `docs/process/conventions.md::Hive identity` for the env stack),
|
|
opens turn-stats sqlite, prepares the on-boot files (see
|
|
[claude-invocation](claude-invocation.md#on-boot-files)),
|
|
installs claude plugins, spawns `web_ui::serve` + `vacuum::run`,
|
|
and either drops into `serve_loop` directly (`Online`) or parks on
|
|
the login flow first (`NeedsLogin`). Forge notifications are polled by
|
|
their own process, not this loop — see `hive-forge-notify` in
|
|
[`forge.md`](../integrations/forge.md).
|
|
|
|
Boot also opens the todos store and the socket in-container producers
|
|
dial. Matrix / bash / forge-notify daemons and the in-process
|
|
`disk_watch` todo producer (low state-disk space) are the built-in
|
|
producers, but the socket accepts any `subsystem` marker — a
|
|
user-configured MCP server can push its own todos the same way. See
|
|
[`docs/agent-lifecycle/persistence.md`](../agent-lifecycle/persistence.md#state-dirs-per-agent) for
|
|
what each built-in todo producer watches and how the store + `get_loose_ends`
|
|
merge work.
|
|
|
|
Plugin install failures aren't fatal: each entry comes back as a
|
|
human-readable failure string that gets routed via
|
|
`Surface::send_to_parent` to the agent's topology parent (the
|
|
broker resolves `<parent>` per `topology::resolve_recipient`; root
|
|
agents and the manager fall through to operator).
|
|
|
|
### Turn outcomes
|
|
|
|
`turn::TurnOutcome` (`Result<bool, TurnError>` — `Ok(compacted)` on success,
|
|
else a `TurnError`) drives the post-claude branch:
|
|
|
|
| Outcome | Action |
|
|
| --- | --- |
|
|
| `Ok(_)` (`false` normal / `true` compacted) | `ack_turn` |
|
|
| `Err(PromptTooLong)` | `drive_turn` archived the session (the lib already compacted + retried and it still overflowed); requeue inflight so the message redelivers into a fresh session that fits — no status park |
|
|
| `Err(RateLimited)` | sleep `HIVE_RATE_LIMIT_SLEEP_SECS` (default 300), requeue inflight, status back to `online` |
|
|
| `Err(AuthFailed)` | emit `needs_login_idle` sentinel, requeue inflight, park in `wait_for_login` |
|
|
| `Err(SessionNotFound)` | resume + create self-heal both missed ("shouldn't happen"); requeue inflight so the next turn creates fresh — no status park, message not dropped |
|
|
| `Err(ApiStall)` | idle watchdog killed claude after `HIVE_TURN_IDLE_SECS` (default 600) of output silence; sleep `HIVE_STALL_SLEEP_SECS` (default 60), requeue inflight, status back to `online` |
|
|
| `Err(Failed(err))` | route `[system] \`<qualified-label>\` claude turn failed:\n<err>` to `<parent>` via `send_to_parent` |
|
|
|
|
`ApiStall` catches an Anthropic API stall — a multi-retry connection storm where
|
|
the stream goes silent for minutes. The idle watchdog lives in `hive-claude`'s
|
|
driver (`Config::idle_timeout`, enforced around `child.wait()`): the timer
|
|
resets on every stdout line, so a large but still-streaming turn is never cut —
|
|
only complete output silence for the window trips it. The harness sets the
|
|
window from `HIVE_TURN_IDLE_SECS` (`0` disables) and maps the driver's
|
|
`Error::IdleTimeout` onto `TurnError::ApiStall`.
|
|
|
|
After the outcome handler, the stats sink records a row. `handle_turn`
|
|
reports the result to `serve_loop` via `TurnControl { auth_failed }` —
|
|
on auth failure the loop parks in `wait_for_login`; otherwise it loops
|
|
straight back to the idle wait (step 1). There is no same-turn
|
|
self-continue mechanism: every multi-step continuation rides an
|
|
external wake instead — a new inbox message, a `remind`, or an
|
|
in-container todo wake (bash-task completion, forge notification,
|
|
matrix activity). Ending the turn and letting one of
|
|
those drive the next one is strictly better than parking in-process:
|
|
it checkpoints the session and observes wakes that only reach the
|
|
harness between turns.
|
|
|
|
## Sub-pages
|
|
|
|
The rest lives alongside this page, in three topic files:
|
|
|
|
- **[claude-invocation.md](claude-invocation.md)** — how the harness
|
|
spawns `claude --print` each turn, the two-pronged compaction (reactive +
|
|
proactive), and the on-boot files it materialises (`--mcp-config`,
|
|
`--system-prompt-file`).
|
|
- **[config.md](config.md)** — the optional per-agent knobs the meta
|
|
flake wires in (reference docs, icon, passwordless sudo, dashboard links,
|
|
custom static files, connectivity overrides, claude plugins, cargo message
|
|
filtering).
|
|
- **[mcp.md](mcp.md)** — the MCP tool surface claude sees: core tools,
|
|
privileged tool groups, self-wake, authoritative state, the tool envelope,
|
|
and the built-in tool allowlist.
|
|
|
|
Per-subsystem impl detail lives in each module's `//!` doc-comment; these pages
|
|
describe present-state behaviour + wiring, not line-level mechanics.
|