vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(#2109): harness-side idle watchdog to bail on anthropic api stall storms

Browse source
This commit is contained in:
damocles 2026-07-07 17:17:45 +02:00 • committed by mara
commit 5027068e31
7 changed files with 157 additions and 8 deletions
Download patch file Download diff file Expand all files Collapse all files

9
docs/turn-loop.md
View file

@ -114,8 +114,17 @@ else a `TurnError`) drives the post-claude branch:
| `Err(RateLimited)` | sleep `HIVE_RATE_LIMIT_SLEEP_SECS` (default 300), requeue inflight, status back to `online` |
| `Err(AuthFailed)` | emit `needs_login_idle` sentinel, requeue inflight, park in `wait_for_login` |
| `Err(SessionNotFound)` | resume + create self-heal both missed ("shouldn't happen"); requeue inflight so the next turn creates fresh — no status park, message not dropped |
| `Err(ApiStall)` | idle watchdog killed claude after `HIVE_TURN_IDLE_SECS` (default 600) of output silence; sleep `HIVE_STALL_SLEEP_SECS` (default 60), requeue inflight, status back to `online` |
| `Err(Failed(err))` | route `[system] \`<qualified-label>\` claude turn failed:\n<err>` to `<parent>` via `send_to_parent` |
`ApiStall` catches an Anthropic API stall — a multi-retry connection storm where
the stream goes silent for minutes. The idle watchdog lives in `hive-claude`'s
driver (`Config::idle_timeout`, enforced around `child.wait()`): the timer
resets on every stdout line, so a large but still-streaming turn is never cut —
only complete output silence for the window trips it. The harness sets the
window from `HIVE_TURN_IDLE_SECS` (`0` disables) and maps the driver's
`Error::IdleTimeout` onto `TurnError::ApiStall`.
After the outcome handler, the stats sink records a row and the
`hyperhive-continue` sentinel (dropped by the `request_next_turn`
MCP tool) is consumed if present. `handle_turn` reports the result