From 69604407a903cc4c5b32be6aa3439c46640c1ccc Mon Sep 17 00:00:00 2001 From: iris Date: Wed, 20 May 2026 16:41:43 +0200 Subject: [PATCH 1/3] docs: update for recent commits (rate limiting, reply threading, screen, auto-reset, two-step spawn, ctx chip) --- docs/approvals.md | 13 +++++++++++++ docs/persistence.md | 16 +++++++++++++++- docs/turn-loop.md | 46 ++++++++++++++++++++++++++++++++++++++------- docs/web-ui.md | 43 +++++++++++++++++++++++++++++++----------- 4 files changed, 99 insertions(+), 19 deletions(-) diff --git a/docs/approvals.md b/docs/approvals.md index 8bba144..31e4355 100644 --- a/docs/approvals.md +++ b/docs/approvals.md @@ -47,6 +47,15 @@ step — the operator just sees the name. On approve, hive-c0re creates the container in a background task while the dashboard shows a spinner. +`InitConfig` approvals are the first step in a two-step spawn +flow. On approve, hive-c0re seeds the proposed config repo with +a default `agent.nix` template and sends the manager +`HelperEvent::ConfigReady { agent }`. The manager then reviews, +edits, and commits the template before calling `request_spawn` +to proceed to a Spawn approval. This gives the manager (and +operator) an explicit review gate on the initial configuration +before any container is created. + ## Meta flake The hive-c0re-owned repo at `/var/lib/hyperhive/meta/` @@ -341,6 +350,10 @@ regular claude turn so the manager can react. Variants the operator. - `LoggedIn { agent }` — sub-agent just completed login. Manager often greets the agent on this event. +- `ConfigReady { agent }` — a new agent's proposed config repo was + just seeded (post-`InitConfig` approval). The manager can now + edit `/agents//config/agent.nix`, commit the changes, + and submit `request_spawn` to create the container. - `NeedsUpdate { agent }` — sub-agent's recorded flake rev is stale. Manager calls `update(name)` to rebuild — idempotent, no approval required. diff --git a/docs/persistence.md b/docs/persistence.md index 59d3ece..8acd29f 100644 --- a/docs/persistence.md +++ b/docs/persistence.md @@ -9,7 +9,9 @@ Where state lives, what survives what, and how it's bounded. Three tables, all in one file: - `messages` — every inter-agent / operator-bound message. - `sender / recipient / body / sent_at / delivered_at`. + `sender / recipient / body / sent_at / delivered_at / acked_at / + in_reply_to`. `in_reply_to` links a reply to its parent row id; + the dashboard and per-agent inbox render these as threaded rows. - `approvals` — the queue. `agent / kind (apply_commit | spawn) / commit_ref / requested_at / status / resolved_at / note`. - `operator_questions` — `ask` / `answer` queue (despite the @@ -72,6 +74,18 @@ No host-side vacuum yet — tracked as forge issue [#10](http://localhost:3000/hyperhive/hyperhive/issues/10) (target retention ~90 days, age-only sweep like events_vacuum). +### `/state/hyperhive-rate-limited` (per agent) + +Sentinel file written by `Bus::emit_status("rate_limited")` when the +harness detects a 429 / rate-limit response from the Claude API, and +removed when the retry sleep expires (any subsequent status emit +clears it). The file's presence is checked by hive-c0re's +`container_view::is_rate_limited` on each `build_all` sweep (~10s) to +populate `ContainerView.rate_limited` for the dashboard. Survives a +harness restart (the Bus reads it back at boot and restores the flag), +so the badge remains accurate if hive-c0re restarts while the harness +is mid-sleep. + ### `/state/hyperhive-model` (per agent) Single-line text file holding the claude model name currently diff --git a/docs/turn-loop.md b/docs/turn-loop.md index 5c9c411..8ad00e8 100644 --- a/docs/turn-loop.md +++ b/docs/turn-loop.md @@ -20,7 +20,12 @@ Each agent harness (`hive-ag3nt serve` or `hive-m1nd serve`) runs: `LiveEvent::Stream(value)`. Pump stderr as `Note`. 6. Wait for claude to exit. Compaction is two-pronged — *reactive* on `Prompt is too long` and *proactive* on a context watermark - (see [Compaction](#compaction) below). + (see [Compaction](#compaction) below). **Rate-limit detection**: + if stdout contains `429` or `rate_limit` markers, the harness + sets the `rate_limited` sentinel (`Bus::emit_status("rate_limited")`), + sleeps `HIVE_RATE_LIMIT_SLEEP_SECS` (default 300), then retries. + The dashboard and per-agent page show a `⊘ rate limited` badge + while the harness is parked. 7. Emit `LiveEvent::TurnEnd { ok, note }`. Sleep `poll_ms` to avoid tight loops on transient failures. @@ -78,6 +83,20 @@ best-effort — a failed checkpoint turn or `/compact` is surfaced as a `Note` but never fails the turn that already succeeded. The operator can also force a compaction any time via `/api/compact`. +- **Auto session-reset** — a third path that fires when both + conditions hold: context is ≥ a watermark (`HIVE_AUTO_RESET_WATERMARK_TOKENS`, + default `100_000`) AND the time since the last turn exceeds the + assumed prompt-cache TTL (`HIVE_CACHE_TTL_SECS`, default `300`). + Claude's prompt cache lives ~5 minutes; if the cache is already + cold, resuming with `--continue` pays the full re-upload cost of + the current context with no benefit over starting fresh. So: + `drive_turn` injects one `AUTO_RESET_CHECKPOINT_PROMPT` notes turn + ("flush state to files, cache is cold") then arms + `Bus::take_skip_continue()` for the real turn — the next turn runs + without `--continue`, starting a fresh session. Unlike proactive + compaction the session is dropped entirely, not compacted. Set + `HIVE_AUTO_RESET_WATERMARK_TOKENS=0` to disable. + The child runs with `cwd = /state` (when the bind exists; falls back to the parent's cwd in dev), so any relative path in a tool call (`Read foo.md`, `Bash ls`, `Write notes.md`) lands in the @@ -126,9 +145,11 @@ it as a stdio child via `--mcp-config`. The hyperhive socket name is ### Sub-agent tools -- `send(to, body)` — message a peer (logical agent name), another - agent, or the operator (recipient `operator`, surfaces in the - dashboard inbox). +- `send(to, body, in_reply_to?)` — message a peer (logical agent + name), another agent, or the operator (recipient `operator`, + surfaces in the dashboard inbox). Optional `in_reply_to: i64` + links this message to a prior message id for thread rendering + in the dashboard message flow and the per-agent inbox. - `recv(wait_seconds?, max?)` — drain inbox messages. Without `wait_seconds` (or with `0`) returns immediately, a cheap "anything pending?" peek. Positive value parks the turn up @@ -161,7 +182,9 @@ it as a stdio child via `--mcp-config`. The hyperhive socket name is this agent's own inbox at a future time (sender shows as `reminder`). Large payloads spill to `/agents//state/reminders/` with the inbox message a - short pointer. + short pointer. Each agent's pending-reminder count is capped + (default 50, override via `HIVE_REMIND_MAX_PENDING_PER_AGENT`); + scheduling a new one fails if the cap is already hit. - `whoami()` — `{ name, role, pronouns, hyperhive_rev }` for self-identification without scraping the system prompt. @@ -209,8 +232,17 @@ meta's. ### Manager tools (in addition to send/recv) -- `request_spawn(name)` — queue a Spawn approval for a brand-new - sub-agent (≤9 char name). Operator approves on the dashboard. +- `request_init_config(name, description?)` — first step of a + two-step spawn. Queues an `InitConfig` approval (≤9 char name); + on operator approve, hive-c0re seeds the proposed config repo + with a default `agent.nix` template and sends the manager a + `HelperEvent::ConfigReady { agent }`. The manager then edits + `agent.nix`, commits the changes, and calls `request_spawn`. + Fails if a proposed repo for this name already exists. +- `request_spawn(name)` — second step of a two-step spawn. Queues + a Spawn approval; requires the proposed config repo to exist + (from a prior approved `request_init_config`). Operator approves + on the dashboard to create the container. - `kill(name)` — graceful stop. No approval required. - `start(name)` — start a stopped sub-agent. No approval. - `restart(name)` — stop + start. No approval. diff --git a/docs/web-ui.md b/docs/web-ui.md index b1901fd..7e6b815 100644 --- a/docs/web-ui.md +++ b/docs/web-ui.md @@ -150,10 +150,14 @@ the previous process's socket release resolves itself. Two-line layout (`assets/app.js::renderContainers`): -- Line 1: agent name (link → new tab), m1nd/ag3nt chip, `needs - login` / `needs update` warning badges, in-flight `◐ +- Line 1: agent name (link → new tab), m1nd/ag3nt chip, status + badges — `⊘ rate limited` (red, while the harness is parked + after a 429), `needs login`, `needs update` — in-flight `◐ pending-state…` pill (replaces buttons during start / stop / - restart / rebuild / destroy), container name + port. + restart / rebuild / destroy), container name + port, and a + `ctx · Nk` chip showing the agent's last-turn context size + (from `ContainerView.ctx_tokens`, read from the turn-stats + sqlite on each `build_all` sweep; absent until the first turn). - Line 2: action buttons — `↻ R3BU1LD` always, `DESTR0Y` + `PURG3` on sub-agents, `↺ R3ST4RT` + (sub-agents) `■ ST0P` when running, `▶ ST4RT` when stopped. Buttons dim + disable while a transient @@ -296,9 +300,12 @@ Wire vocabulary on `/dashboard/stream` (kind tag is in the JSON payload): - `sent` / `delivered` — broker traffic, mirrored from the - intra-process channel by a forwarder task. Used by the - message-flow terminal renderer and the operator-inbox - derived state. + intra-process channel by a forwarder task. Both carry `id: i64` + (the broker row id) and `in_reply_to: Option` for thread + rendering. The dashboard message-flow terminal renders reply + rows with a `↳ reply` tag that scroll-highlights the parent + row on click. Used by the message-flow terminal renderer and + the operator-inbox derived state. - `approval_added` (id, agent, approval_kind, sha_short, diff, description) / `approval_resolved` (id, agent, approval_kind, sha_short, status, resolved_at, note, description) — pending @@ -355,8 +362,10 @@ Layout, top to bottom: badge + last-turn timing + cancel-turn button + new-session button. Every chip carries a `title=...` tooltip with the detailed breakdown. - - Alive badge: `● alive` (green) / `◌ needs login` (amber) / - `◌ logging in` / `○ offline` / `… connecting`. Driven by + - Alive badge: `● alive` (green) / `⊘ rate limited` (red, while + the harness is parked after a 429 — clears automatically when + the sleep expires) / `◌ needs login` (amber) / `◌ logging in` / + `○ offline` / `… connecting`. Driven by `LiveEvent::StatusChanged`; replaces the old "harness alive — turn loop running" paragraph so the state row carries every reachability signal. @@ -401,7 +410,9 @@ Layout, top to bottom: updates from SSE; no periodic refresh timer runs. - Inbox `
` block (collapsed): `inbox · N` — last 30 messages addressed to this agent, fetched via - `AgentRequest::Recent { limit: 30 }`. (Separate from + `AgentRequest::Recent { limit: 30 }`. Reply messages (those + with a non-null `in_reply_to`) are indented and prefixed with + `↳ reply ·` in amber. (Separate from `AgentRequest::Recv { wait_seconds, max }` which the harness uses internally to long-poll the broker.) - Loose-ends `
` block: `loose ends · N` — questions, @@ -506,12 +517,22 @@ shaped). - `POST /api/new-session` — arm a one-shot for the next turn to drop `--continue`. Emits a `LiveEvent::Note`. - `GET /events/history` — replay buffer for the terminal. +- `GET /screen` — VNC viewer page (minimal RFB-over-WebSocket + renderer). Only accessible when `hyperhive.gui.enable = true` + in the agent's `agent.nix`; the harness shows a 🖥 screen link + in the state row when `gui_vnc_port` is present. +- `GET /screen/ws` — raw RFB byte relay: proxies WebSocket + frames to the weston VNC server at `127.0.0.1:`. + Transparent to any RFB variant. VNC port comes from + `/etc/hyperhive/gui.json` (written by the weston startup + script in `weston-vnc.nix`). Bus events (new vocabulary on `/events/stream`): -- `status_changed { status }` — `online` / +- `status_changed { status }` — `online` / `rate_limited` / `needs_login_idle` / `needs_login_in_progress`. Drives the - alive-badge. + alive-badge. `rate_limited` is set when the harness detects a + 429 response and cleared when the retry sleep expires. - `model_changed { model }` — drives the model chip. - `token_usage_changed { ctx: TokenUsage, cost: TokenUsage }` — drives the ctx + cost badges. Emitted from From bac7dd6cde819bff4cda71cedf0edccaf8a8ac05 Mon Sep 17 00:00:00 2001 From: iris Date: Wed, 20 May 2026 16:46:28 +0200 Subject: [PATCH 2/3] docs: update prompts for two-step spawn + in_reply_to; trim CLAUDE.md to pointer index --- CLAUDE.md | 471 ++-------------------------------- hive-ag3nt/prompts/agent.md | 4 +- hive-ag3nt/prompts/manager.md | 6 +- 3 files changed, 28 insertions(+), 453 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 029a065..b88c08b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -2,8 +2,7 @@ Hey claude. This is your starting page. The detailed docs live in [`docs/`](docs/) and are written for humans + you both — read them -when you need depth on a subsystem. This file is the index + -scratchpad. +when you need depth on a subsystem. This file is the index. - High-level project intro: **[README.md](README.md)**. - Open work + backlog: the **[forge issue @@ -25,8 +24,7 @@ hive-c0re/ host daemon + CLI (one binary, subcommand-dispatched) src/agent_server.rs per-sub-agent socket listener (long-poll Recv) src/broker.rs sqlite Message store + intra-process broadcast channel (`MessageEvent`) for `recv_blocking_batch` + - the dashboard forwarder; hourly vacuum of - acked>30d + the dashboard forwarder; hourly vacuum of acked>30d src/dashboard_events.rs unified wire-facing event channel feeding `/dashboard/stream`. Carries broker `Sent` / `Delivered` (mirrored by the forwarder task @@ -111,7 +109,8 @@ hive-ag3nt/ in-container harness crate; produces TWO binaries src/lib.rs re-exports + DEFAULT_SOCKET, DEFAULT_WEB_PORT src/client.rs generic JSON-line request/response over unix socket src/web_ui.rs per-container axum HTTP page (incl /api/cancel, - /api/compact, /api/model, /events/history) + /api/compact, /api/model, /events/history, + /screen, /screen/ws) src/turn_stats.rs per-turn analytics sink (one sqlite row per turn at /state/hyperhive-turn-stats.sqlite); schema + best-effort writer @@ -121,14 +120,15 @@ hive-ag3nt/ in-container harness crate; produces TWO binaries src/events.rs LiveEvent + broadcast Bus + sqlite-backed history (/state/hyperhive-events.sqlite) + TurnState + model selection (persisted at /state/hyperhive-model) - src/turn.rs claude --print + stream-json pump; --compact retry + src/turn.rs claude --print + stream-json pump; --compact retry; + proactive compaction + auto session-reset src/mcp.rs embedded MCP server (rmcp): AgentServer + ManagerServer src/login.rs probe /root/.claude/ for a valid session src/login_session.rs drives `claude auth login` over stdio pipes src/bin/hive-ag3nt.rs sub-agent main (Serve + Mcp subcommands) src/bin/hive-m1nd.rs manager main (Serve + Mcp subcommands) assets/ index.html, agent.css, app.js, stats.html, - stats.js (include_str!) + stats.js, screen.html (include_str!) prompts/ static role/tools/settings for claude (include_str!): agent.md — sub-agent system prompt manager.md — manager system prompt @@ -141,22 +141,27 @@ nix/ modules/hive-c0re.nix systemd service + firewall + git wiring; imports hive-forge.nix modules/hive-forge.nix optional in-container Forgejo - (`hyperhive.forge.enable`, default on) + (`hyperhive.forge.enable`, default on); + Catppuccin Mocha theme via tmpfiles C+ copy templates/harness-base.nix shared scaffolding for sub-agents + manager templates/agent-base.nix sub-agent nixosConfiguration templates/manager.nix manager nixosConfiguration templates/weston-vnc.nix optional `hyperhive.gui.enable` — weston + VNC backend systemd unit; writes /etc/hyperhive/gui.json (vnc_port + auth) for - the harness WebSocket relay (issue #51) + the harness WebSocket relay (/screen/ws) + forge-theme/theme-catppuccin-vibec0re.css Catppuccin Mocha forge theme docs/ conventions.md naming, identity=socket, async forms, commit style - gotchas.md NixOS / nspawn lessons learned the hard way + gotchas.md NixOS / nspawn quirks and lessons learned web-ui.md dashboard + per-agent page layouts and endpoints turn-loop.md claude invocation, wake prompt, MCP tool surface approvals.md approval flow, manager policy, helper events persistence.md sqlite dbs, retention, state dir layout + terminal-rendering.md per-agent terminal row taxonomy (as built) + boundary.md operator/agent trust model rationale + damocles-migration.md future migration plan for damocles → hyperhive ``` ## Reading paths @@ -167,9 +172,7 @@ read them à la carte. - **"What does the dashboard look like?"** → [`docs/web-ui.md`](docs/web-ui.md). - **"How does the per-agent terminal classify + colour - events?"** → [`docs/terminal-rendering.md`](docs/terminal-rendering.md) - (as-built row taxonomy + layout contract + markdown + - extra-MCP fallback). + events?"** → [`docs/terminal-rendering.md`](docs/terminal-rendering.md). - **"How does claude get its prompt and what tools does it have?"** → [`docs/turn-loop.md`](docs/turn-loop.md). - **"How do config changes flow from manager to operator to @@ -196,439 +199,9 @@ read them à la carte. - **Actions are factored** between admin socket and dashboard via `actions.rs` and `dashboard.rs::lifecycle_action`, so the two surfaces never drift. - -## Scratchpad - -In-flight or recent context that hasn't earned a section yet. -Prune freely. - -- **Just landed:** proactive context-size compaction. `turn:: - drive_turn` now has two compaction paths. The old *reactive* - one (claude prints `Prompt is too long` → `/compact` + retry) - is unchanged — by then the session is already past the window - and no turn can run on it. The new *proactive* one fires after - a clean turn whose last-inference context - (`Bus::last_ctx_usage().context_tokens()`) crossed a watermark: - `drive_turn` injects one synthetic notes-checkpoint turn - (`CHECKPOINT_PROMPT` — "flush durable state into /state now") - then runs `/compact`, so the agent can persist in-flight state - before the detail collapses into a summary. Watermark is - `HIVE_COMPACT_WATERMARK_TOKENS` (default 150k, ~75% of a 200k - window; `0` disables). New `maybe_checkpoint_and_compact` - helper, all best-effort (a failed checkpoint/compact is a Note, - never fails the turn). Both ag3nt + m1nd loops get it for free - via the shared `drive_turn`. `docs/turn-loop.md` gained a - Compaction section. -- **Just landed:** dashboard side panel + forge-linked - config/approvals + per-bucket model stats. (a) Long content - (file previews, approval diffs, journald logs) opens in a - right-side slide-in panel instead of expanding inline; - markdown `.md` previews render through the vendored `marked` - bundle (`/static/marked.js` route added to the dashboard). - (b) the container-row `agent.nix` viewer + `/api/agent-config` - are gone — each row links to the agent's `agent-configs/` - repo on the forge instead. (c) pending approvals render as a - card (identity / what-changed / decision actions) with a - `commit on forge` deep-link and a 3-way diff base toggle — vs - applied / vs last-approved / vs previous proposal — served by - the new `/api/approval-diff/{id}?base=` endpoint. (d) the - `/stats` page gained a per-bucket turns-by-model stacked bar - (`Snapshot.models` + `Bucket.model_counts`) since model - choice drives token cost. `/api/state` carries `forge_present` - so the links only show when the forge is up. Docs refreshed: - web-ui.md, approvals.md, turn-loop.md, agent.md, manager.md. -- **Just landed:** `hyperhive.forge.enable` (default **true**) — - the forge option moved out of `services.hive-forge.*` into the - `hyperhive.*` namespace, and `hive-c0re.nix` imports - `hive-forge.nix` so the forge ships with the standard core - install. Forgejo settings gained - `migrations.ALLOW_LOCALNETWORKS = true` so an in-hive mirror - of the hyperhive repo can import from a `localhost` source - (Forgejo blocks loopback/RFC-1918 migration sources by - default). -- **Just landed:** failed `nixos-container update` self-documents. - `lifecycle::run` now appends the tail (40 lines) of the target - container's own journal to the bail message when an `update` - fails. `nixos-container`'s own stderr on a reload-phase failure - is terse ("failed to reload container"); the real cause — - which unit failed `switch-to-configuration` — lives in the - *container* journal. Scoped to `update` (container's still up - on the old generation, so `journalctl -M` works); best-effort, - appends nothing if the journal can't be read. The manager's - `update` tool / rebuild errors now carry the failing-unit - detail without a second `get_logs` call. -- **Just landed:** `hyperhive.gui.enable` option (replaces - `hyperhive.westonRdp.enable`). New - `nix/templates/weston-vnc.nix` declares a per-agent bool; - enabling it runs weston with the VNC backend as a systemd - service (software/pixman render). Port is deterministic: - FNV-1a hash of the agent name (from hostname) in - [15900, 16799], mirroring `lifecycle::agent_web_port`. - The ExecStart wrapper script computes the port, writes - `/etc/hyperhive/gui.json` (`{ "vnc_port": N, "auth": "none" }`) - for the harness WebSocket relay (issue #51), then execs - weston. Imported by `harness-base.nix`; an agent opts in - from its own `agent.nix`. `Type = "simple"` so it can never - abort `nixos-container update`. A misconfigured weston - degrades to a restart loop in `journalctl`, not a blocked - rebuild. Old `weston-rdp.nix` / `westonRdp.enable` removed. -- **Just landed:** `get_logs` now resolves the machine name. - `journalctl -M` wants the *machine* name (`h-gui`), not the - logical agent name (`gui`) — `get_logs` was the one manager - verb that passed the name straight through instead of mapping - it via `lifecycle::container_name()` like Kill/Start/Restart/ - Update do. Now consistent: pass the plain agent name, hive-c0re - resolves `h-` (manager stays `hm1nd`). Tool description + - `GetLogs` wire doc updated. -- **Just landed:** applied config repos mirrored to the - forge. New private `agent-configs` Forgejo org (renamed - from the unused `agents` org in `SEEDED_ORGS`); core is the - only principal with access (site admin + private repos + - agents not members). `forge::push_config(name)` mirrors an - agent's hive-c0re-owned applied repo — `main` + every tag - (proposal/approved/building/deployed/failed/denied) — to - `agent-configs/.git` via `git push --force`. The - tokenised URL is passed inline per push, never stored as a - named remote: the applied repo is RO-bind-mounted into the - manager at `/applied`, so a token in `.git/config` would - leak core's admin credential to an agent. Call sites: - `forge::ensure_all` (startup, per agent — catches migrate + - offline-forge drift), the spawn task in `actions::approve` - (+ `ensure_config_repo`), `actions::approve` ApplyCommit - branch, `actions::deny` ApplyCommit branch, and - `manager_server::submit_apply_commit`. All best-effort - (warn + continue). `ensure_repo` refactored to share a - `create_repo` helper with the new `ensure_org_repo`. -- **Just landed:** answer questions inline from the per-agent - web page. Question rows in the loose-ends section grew a - textarea + send button; the operator answers as operator by - POSTing cross-origin to the core dashboard's - `/answer-question/{id}` (CORS shim `with_cors` on that - route), never the per-agent socket — keeps the - operator-authority path off the agent's own socket. See - `docs/boundary.md` for the boundary rationale; the - deployment/gateway/privsep work is tracked as `area:ops` - forge issues. -- **Just landed:** sub-agents get a read-only view of their own - config repo. `set_nspawn_flags` now adds - `--bind-ro={proposed_dir}:/agents//config` for every - sub-agent container (manager unchanged — it already has the whole - `/agents` tree RW). The agent can read `agent.nix` + whatever - extra files the manager split the config into, so it can request - precise changes from the manager instead of guessing. RO is - load-bearing: config edits only ever flow through the manager's - proposed repo + the approval queue. `setup_proposed` seeds the - dir before spawn reaches `set_nspawn_flags`; a defensive - `create_dir_all` keeps a missing repo from becoming a - won't-boot container. Takes effect on next rebuild/restart of - each existing sub-agent. `agent.md` system prompt + `docs/ - persistence.md` updated. -- **Just landed:** `request_apply_commit` fetch fix. The old - `git_fetch_to_tag` built a refspec `:refs/tags/proposal/` - and ran `git fetch :...` — but `git fetch` resolves - the left side of a refspec as a remote *ref name*, and a bare - commit sha is not one ("couldn't find remote ref ..."). Fetching - by sha would need a full 40-hex sha plus - `uploadpack.allow*SHA1InWant` on the remote. Surfaced on the first - real `request_apply_commit` (the `gui` agent bootstrap — initial - `deployed/0` seeding uses a different path). Fix: `git_fetch_to_tag` - now resolves the sha LOCALLY against the proposed repo - (`git rev-parse ^{commit}`), fetches all of proposed's heads - into applied's object db (`+refs/heads/*:refs/remotes/proposal-src/*`), - then `git tag`s the resolved sha — all-local, no upload-pack - sha-want negotiation. Plus: `submit_apply_commit` now shape-checks - `commit_ref` is a 7-40 char hex sha (`validate_commit_ref`) and - rejects branch/tag names so the proposal always pins an immutable - commit. Tool description + `RequestApplyCommit` wire doc + - `docs/approvals.md` updated. 3 new tests in `manager_server::tests`. -- **Just landed:** inbox batching unified into `recv(max?)`. - No separate `recv_batch` tool — the existing `recv` tool - grew an optional `max: u32` arg (default 1, server-side - cap 32) so a single round-trip drains up to N popped rows - with the same delivery + ack bookkeeping per row - (`delivered_at = NOW`, `unacked_ids` list, redelivered - tag from `requeue_inflight`). `wait_seconds` still applies - to the FIRST message; once one lands the call drains up - to `max` in total — long-poll + drain compose. Wake - prompt's pending-inbox hint points at `recv(max: N)`. - Wire shape: `AgentRequest::Recv { wait_seconds, max }` - (added `max`), `AgentResponse::Messages { messages: - Vec }` (collapsed the old - `Message` + `Empty` + `Batch` trio into one always-list - variant — empty vec = idle). `DeliveredMessage` is a flat - shared struct in `hive-sh4re`. `format_recv` renders - single = the historical `from: X\n\nbody` block, multi = - `popped N message(s)` header with `---` separators + - per-message redelivery banners; empty = "(empty)". Broker - primitive: dropped the singular `recv`, kept just - `recv_batch(recipient, max)` and `recv_blocking_batch` - (which long-polls then drains via `recv_batch`). 4 new - broker tests on top of the existing 7 (recv_batch_* - family). Closes the "inbox batching hint" item from the - ergonomics wishlist with one tool instead of two; lower - context bloat in claude's prompt. -- **Just landed:** lease-style message delivery / no-drop - on turn fail. The `messages` table gained an `acked_at` - column (idempotent ALTER + backfill = `delivered_at` so - pre-migration delivered rows count as already-acked). - `Broker::recv` now returns `Delivery { id, redelivered, - message }` — the harness gets the row id back so - `AckTurn` can sweep every popped id at turn-end-OK. Two - new wire arms on both agent + manager surfaces: - `AckTurn` (drains the broker's per-recipient in-memory - `unacked_ids` list and stamps the rows `acked_at = NOW`) - and `RequeueInflight` (one-shot at harness boot: resets - `delivered_at = NULL` on every still-inflight row + - remembers each id so the next `Recv` carries - `redelivered: true`). Both bin loops call - `requeue_inflight` once before entering serve, and - `ack_turn` after every `TurnOutcome::Ok` (Failed + - PromptTooLong intentionally skip the ack so the popped - rows stay in-flight for the next boot's requeue). - `format_recv` + `format_wake_prompt` on both bins - surface a `[redelivered after harness restart — may - already be handled]` banner so claude knows the - side-effects of any previous handling may already have - happened. Lock order: `inflight` mutex first then - `conn` mutex in all three methods (`recv` / `ack_turn` - / `requeue_inflight`) so a concurrent pop can't race - the requeue's DB update vs in-memory populate and - miss the redelivered tag. `vacuum_delivered` filter - flipped from `delivered_at < cutoff` to `acked_at IS - NOT NULL AND acked_at < cutoff` so unacked-but- - delivered rows survive vacuum (they're recoverable via - `requeue_inflight`). 7 new tests in `broker::tests` - cover happy path, crash recovery, idempotency, per- - recipient isolation, batch ack, vacuum preservation, - and FIFO ordering on requeue. Closes the "post-rebuild - system-message missed wake" bug class entirely (any - turn that wakes from a `delivered_at NOT NULL, - acked_at NULL` row resurfaces on next boot). -- **Just landed:** ctx + cost badges split. The per-agent - page now shows TWO chips — `ctx · N` (last inference's - prompt size = actual context window utilisation, parsed - from each `assistant` event's `.message.usage`; the - number to watch for compaction) and `cost · M` (sum - across every inference in the turn, the previous - behaviour now correctly labelled — tool-heavy turns - rebill the cached prefix per call and blow past the - model's window). Both fed by a single - `TokenUsageChanged { ctx, cost }` SSE event at turn-end - via `Bus::record_turn_usage`. `turn_stats` grew four - `last_*_tokens` columns (idempotent ALTER migration) so - cold-load seeds both badges from the most recent row. - Pre-migration rows yield no `ctx` seed (empty badge - until next turn) rather than a misleading zero. -- **Just landed:** per-agent terminal coherence pass. - Unified prefix column (padding-left + negative - text-indent so every row kind aligns); `
` - summaries drop the directional glyph and let CSS - `▸/▾` sit in the prefix column; turn boundaries - de-weighted (border-left rule only, no bold/margin/ - tint); stderr lines render orange `!`, - operator-initiated notes mauve italic, catch-all `.sys` - escalated to orange so unrecognised stream-json - surfaces. Message-bearing tool calls - (`send`/`ask`/`answer`/`recv`) render default-open - with markdown bodies via vendored `marked` v4.0.2 - (`hive-fr0nt::MARKED_JS`); assistant `text` rows also - markdown-rendered. Extra-MCP tools get a generic - args pretty-printer (`fmtArgsGeneric`) instead of - raw JSON. tool_use_id → name map carries through the - stream so `renderToolResult` knows when a result came - from `recv` and should default-open with markdown. - See `docs/terminal-rendering.md` for the as-built - taxonomy. Bonus: ctx badge seeded from `turn_stats` - on cold load via `Bus::seed_usage` so the chip paints - real numbers before the next turn finishes. -- **Just landed:** `open_threads` → `loose_ends` rename - (more honest about what the list is) + new - `cancel_loose_end(kind, id)` MCP tool on both - surfaces. `kind = "question"` posts a `[cancelled by - ]` answer to unblock the asker; `kind = - "reminder"` hard-deletes before fire. Auth: sub-agent - must own the row (`asker == self` / `owner == self`); - manager bypasses for hive-wide cleanup. `LooseEnd` - enum gained a `Reminder { id, owner, message, due_at, - age_seconds }` variant; sub-agent flavour filters by - owner, manager unfiltered. Shared dispatch in - `hive-c0re/src/questions.rs::handle_cancel_loose_end`. - Per-agent web UI's `/api/open-threads` → - `/api/loose-ends`. Closes the agent-side "I have no - way to cancel what I queued" friction from the - ergonomics wishlist. -- **Just landed:** `whoami` MCP tool on both surfaces — - returns `{ name, role, pronouns, hyperhive_rev }`. - Lets an agent self-identify without scraping its own - system prompt; pronouns are pulled from the - `HIVE_OPERATOR_PRONOUNS` env that the system prompt - also substitutes, so the two stay in sync. -- **Just landed:** reminder delivery failures persist + - surface. The 5s scheduler now records the failure - reason on the `reminders` row instead of silently - dropping; the web UI surfaces them in the loose-ends - section so the owner agent can see "this reminder - never landed because the target was destroyed" without - reading journald. -- **Just landed:** path linkify. The broker tags every - message body server-side with `file_refs` at ingest - via `/api/state-file/check` (validates each path, - attaches container→host resolution). Last segment - must look like `name.ext` so directory mentions don't - fire spurious links. Dashboard renders refs as - collapsible path-preview blocks below the message - body; per-agent terminal picks them up from the same - field. Server-side validation means the JS doesn't - re-walk allow-lists on every render. -- **Just landed:** tombstones + meta_inputs as - `DashboardEvent`s. Closes the last two refetch loops - on the dashboard side — `purge-tombstone` and - `meta-update` POSTs now flip to 200 with - `data-no-refresh`. The 5s `/api/state` poll is gone - entirely; everything event-driven. -- **Just landed:** per-agent UI gained an open-threads - section (questions + approvals + reminders pending - against this agent) + the container row on the - dashboard gained a `⏰ N` pending-reminder count - chip. task_started / task_notification stream-json - events now pretty-render in the terminal with the - `⌁` glyph so subagent (Task tool) activity is - visually distinct from main-session tool calls. -- **Just landed:** per-agent extra MCP servers via the - `hyperhive.extraMcpServers.` NixOS option in - `agent.nix`. Declares `{ command, args, env, - allowedTools }`; the module writes the whole map to - `/etc/hyperhive/extra-mcp.json`; the harness reads that - file and merges each entry into both `--mcp-config` - and `--allowedTools` (mapped to `mcp____`). - Unblocks matrix / bitburner / any agent with rich - domain tooling — the agent flake's `inputs` block pulls - the external flake, `agent.nix` references it via - `flakeInputs..packages.${pkgs.system}.default`. -- **Just landed:** per-turn analytics sink. New - `hive-ag3nt::turn_stats` writes one row per claude turn to - `/state/hyperhive-turn-stats.sqlite`: identity (model, - wake_from, result_kind), timing (started/ended_at, - duration_ms), cost (full token-usage breakdown), behaviour - (tool_call_count + per-tool JSON map), and post-turn snapshot - metrics (open_threads_count, open_reminders_count fetched via - the existing GetOpenThreads + new CountPendingReminders RPC). - Both ag3nt + m1nd bin loops capture, both Bus accumulates - tool_use blocks via observe_stream during the stdout pump. - Writes are best-effort. No host-side vacuum yet — TODO under - Telemetry; same shape as events_vacuum, target 90d retention. -- **Just landed:** agent web UI event-driven badges. New - `LiveEvent::StatusChanged / ModelChanged / TokenUsageChanged - / TurnStateChanged` variants replace the per-agent page's - /api/state polling for the state row. Status/model/token/state - badges all update from SSE; /api/state only fetched on cold - load + during the login flow (session output isn't event- - shaped). Per-agent endpoints (`/api/cancel|compact|model| - new-session`, `/login/*`) all flip 303→200. New `alive-badge` - chip carries the harness reachability signal (replaces the - "● harness alive" paragraph); new `ctx-badge` mirrors Claude - Code's bottom-right "N tokens" indicator. Every chip carries - a `title=...` tooltip for hover detail. -- **Just landed:** events_vacuum simplified to age-only — - `KEEP_SECS = 7d`, no row cap. Chatty turn no longer evicts - a quiet day's history sooner than expected. Hourly sweep - unchanged. -- **Just landed:** Phase 6 container events. New - `DashboardEvent::ContainerStateChanged { container }` + - `ContainerRemoved { name }` close the last refetch loop on the - dashboard side. `Coordinator::rescan_containers_and_emit` builds a - fresh `container_view::build_all` snapshot, diffs it against a - cached `last_containers` map, and fires per-row events for the - delta. Called from every mutation site: `actions::approve` - (post-spawn), `actions::destroy`, the `lifecycle_action` wrapper - in `dashboard.rs` (start/stop/restart/rebuild), `auto_update:: - rebuild_agent`, and the existing 10s `crash_watch` poll loop. - `ContainerView` extracted to its own module so coordinator + - dashboard can both build it. Dashboard endpoints (`/restart`, - `/destroy`, `/kill`, `/rebuild`, `/start`, `/update-all`, - `/meta-update`, `/purge-tombstone`) now return 200; matching - forms carry `data-no-refresh` where the event coverage is - complete (purge + meta-update keep the refetch since tombstones - + meta_inputs aren't event-derived yet). Client drops the 5s - periodic `/api/state` poll entirely — initial cold load + SSE - for everything afterwards; pending overlay reads from - `transientsState` since the new event payload doesn't carry it. -- **Just landed:** dashboard event refactor. New `hive-fr0nt` - workspace crate hosts shared frontend assets (palette + terminal - CSS + `window.HiveTerminal.create` JS) so both the dashboard and - the per-agent web UI render their live panes through the same - code; the dashboard's `#msgflow` now feels like the agent's - terminal (sticky-bottom + pill + lit chrome). New unified - `DashboardEvent` channel on `Coordinator` (replaces the - broker-only `/messages/stream`); a background forwarder mirrors - broker traffic onto it as `Sent` / `Delivered` variants, and - the mutation-event variants - (`ApprovalAdded` / `ApprovalResolved`, `QuestionAdded` / - `QuestionResolved`, `TransientSet` / `TransientCleared`) cover - every in-process state change the dashboard cares about. Each - frame carries a monotonic per-process `seq`; snapshot endpoints - return their seq alongside the state, and the terminal's - open-buffer-then-fetch-history dance drops any buffered frame - with `seq <= history_seq` so an event landing between subscribe - and history-fetch is neither shown twice nor lost. Operator - inbox + approvals + questions + transients are now derived - client-side from the event stream (cold-loaded from - `/api/state` for first paint, mutated live from SSE - thereafter); `/op-send` + per-agent `/send` return 200 instead - of 303-and-refetch. Container-list events still pending — - `ContainerView` is sourced from external `nixos-container list`, - so the 5s `/api/state` poll continues to drive the containers - section. Approval diffs are now raw unified-diff text on the - wire (per-line classification happens in JS) so they fit in - SSE payloads without HTML escaping. Bug fix: `LiveEvent::Note` - was a newtype variant that serde silently failed to serialize - — converted to `Note { text: String }` (wire shape matches what - the JS already read). -- **Just landed:** `ask_operator` → `ask` rename + optional - `to: ` param for agent-to-agent structured Q&A. - Recipient defaults to the operator (dashboard); peer - questions land in the target's inbox as `QuestionAsked` - events and the recipient replies via new `answer(id, - answer)` tool. Answer always flows back as - `QuestionAnswered { id, question, answer, answerer }` - (renamed from `OperatorAnswered`; `answerer` distinguishes - operator vs peer vs `ttl-watchdog`). Authorisation: - operator-targeted questions can only be answered by the - operator; agent-targeted by the named target (or the - operator as override). Self-ask rejected. Shared dispatch - lives in `hive-c0re/src/questions.rs`. Dashboard's - `pending()` filters on `target IS NULL` so peer questions - never leak into the operator's queue. -- **Just landed:** dashboard now has a terminal-style - compose textbox under the message-flow stream — `@name` - picks the recipient (sticky in localStorage, auto- - completed from `containers[]`), POSTs `/op-send`. New - per-agent `↻ new session` button drops `--continue` for - one turn. Claude spawns with `cwd = /state` so relative - paths in tool calls land in the durable dir. -- **Just landed (prior overhaul still underneath):** tag- - driven config-apply. Two-repo split (proposed = manager - RW, applied = core-only); `request_apply_commit` fetches - the manager's commit into applied and pins it as - `proposal/`; approve / deny / build walk through - tags on the same commit; `applied/main` only fast- - forwards on `deployed/`. `failed/` + `denied/` are - annotated. See `docs/approvals.md`. -- **Recent (since last compaction):** inline +/- diffs on - Write/Edit, send full body via collapsed details, operator - cancel + ttl on questions, deny-with-reason, dashboard - back-link + last-turn timing + model chip, per-agent inbox - view, bind-retry + SO_REUSEADDR, journald viewer, - agent.nix viewer, server-side TurnState, recv(wait_seconds) - max 180s, runtime /model switch + persistence to /state, - crash watcher + ContainerCrash / NeedsLogin / LoggedIn / - NeedsUpdate events, manager `update` tool, pure-hash - agent_web_port + collision banner + spawn/rebuild preflight, - browser notifications, focus-preserving refresh, generalised -
survival, prompt-on-submit pattern. -- **Open threads:** two-step spawn, unprivileged - containers, Bash allow-list, xterm.js. The - deployment / gateway / privsep cluster is tracked as - `area:ops` forge issues. (Landed since this note was first written: - extra per-agent MCP servers, per-agent send allow-list, - telemetry + the `/stats` page.) +- **Two-step spawn:** `request_init_config` → edit `agent.nix` → + `request_spawn`. Never submit a Spawn approval without first + reviewing the config template. +- **Rate-limit sentinel:** `{state_dir}/hyperhive-rate-limited` + is written by the harness on 429 and cleared on retry. + `ContainerView.rate_limited` reads it for the dashboard badge. diff --git a/hive-ag3nt/prompts/agent.md b/hive-ag3nt/prompts/agent.md index 43222a6..aeefad6 100644 --- a/hive-ag3nt/prompts/agent.md +++ b/hive-ag3nt/prompts/agent.md @@ -3,13 +3,13 @@ You are hyperhive agent `{label}` in a multi-agent system. The operator (recipie Tools (hyperhive surface): - `mcp__hyperhive__recv(wait_seconds?, max?)` — drain inbox messages (returns `(empty)` if nothing pending). Without `wait_seconds` (or with `0`) it returns immediately — a cheap "anything pending?" peek you can sprinkle between tool calls. To **wait** for work when you have nothing else useful to do this turn, call with a long wait (e.g. `wait_seconds: 180`, the max) — incoming messages wake you instantly, otherwise the call returns empty at the timeout. That's strictly better than a fixed `sleep` shell command: lower latency on new work, no busy-loop. `max` (default 1, cap 32) drains several queued messages in one call — the wake prompt tells you the pending count. -- `mcp__hyperhive__send(to, body)` — message a peer (by their name) or the operator (recipient `operator`, surfaces in the dashboard). Use `to: "*"` to broadcast to all agents (they receive a hint that it's a broadcast and may not need action). Some agents have a per-agent allow-list (`hyperhive.allowedRecipients` in their `agent.nix`) — if so the tool refuses recipients outside the list with a clear error; route through the manager (`send(to: "manager", …)`) which is always reachable. +- `mcp__hyperhive__send(to, body, in_reply_to?)` — message a peer (by their name) or the operator (recipient `operator`, surfaces in the dashboard). Use `to: "*"` to broadcast to all agents (they receive a hint that it's a broadcast and may not need action). Optional `in_reply_to: ` threads this message under a prior one — the dashboard and per-agent inbox render it with a `↳ reply` link. Some agents have a per-agent allow-list (`hyperhive.allowedRecipients` in their `agent.nix`) — if so the tool refuses recipients outside the list with a clear error; route through the manager (`send(to: "manager", …)`) which is always reachable. - (some agents only) **extra MCP tools** surfaced as `mcp____` — these are agent-specific (matrix client, scraper, db connector, etc.) declared in your `agent.nix` under `hyperhive.extraMcpServers`. Treat them as first-class tools alongside the hyperhive surface; the operator already auto-approved them at deploy time. - `mcp__hyperhive__ask(question, options?, multi?, ttl_seconds?, to?)` — surface a structured question to the human operator (default, or `to: "operator"`) OR a peer agent (`to: ""`). Returns immediately with a question id — do NOT wait inline. When the recipient answers, a system message with event `question_answered { id, question, answer, answerer }` lands in your inbox; handle it on a future turn. Use this for clarifications, permission for risky actions, choice between options, or peer Q&A without burning regular inbox slots. `options` is advisory: a short fixed-choice list when applicable, otherwise leave empty for free text. `multi: true` lets the answerer pick multiple (checkboxes), answer comes back comma-joined. `ttl_seconds` auto-cancels with answer `[expired]` (and `answerer: "ttl-watchdog"`) when the decision becomes moot. - `mcp__hyperhive__answer(id, answer)` — answer a question that was routed to YOU. You'll see one in your inbox as a `question_asked { id, asker, question, options, multi }` system event when a peer or the manager calls `ask(to: "", ...)`. The answer surfaces in the asker's inbox as a `question_answered` event. Strict authorisation: you can only answer questions where you are the declared target. - `mcp__hyperhive__get_loose_ends()` — list your loose ends: unanswered questions where you're asker (waiting on someone) or target (owing a reply), plus reminders you've scheduled that haven't fired. No args, cheap server-side sweep. Useful at turn start to remember what's outstanding without scanning inbox archaeology. - `mcp__hyperhive__cancel_loose_end(kind, id)` — cancel one of your own open threads. `kind` is `"question"` (the asker — you, in this case — gets a `[cancelled by ]` answer so the waiter unblocks) or `"reminder"` (hard-deleted before it fires). `id` from the matching `get_loose_ends` row or the original submission reply. -- `mcp__hyperhive__remind(message, delay_seconds? | at_unix_timestamp?, file_path?)` — schedule a message to land in your *own* inbox at a future time (sender shows as `reminder`). Set exactly one of `delay_seconds` (relative) or `at_unix_timestamp` (absolute). Use for self-paced follow-ups instead of blocking a whole turn on a long `recv` wait. A large `message` auto-spills to a file under `/agents/{label}/state/reminders/`; pass `file_path` to point at one yourself. +- `mcp__hyperhive__remind(message, delay_seconds? | at_unix_timestamp?, file_path?)` — schedule a message to land in your *own* inbox at a future time (sender shows as `reminder`). Set exactly one of `delay_seconds` (relative) or `at_unix_timestamp` (absolute). Use for self-paced follow-ups instead of blocking a whole turn on a long `recv` wait. A large `message` auto-spills to a file under `/agents/{label}/state/reminders/`; pass `file_path` to point at one yourself. Each agent's pending-reminder count is capped (default 50) — the tool will error if the cap is already reached. - `mcp__hyperhive__whoami()` — self-introspection: returns your canonical agent name (from socket identity, not the prompt-substituted label), role, and current hyperhive rev. No args. Use it when you want a trustworthy identity stamp for state files, commit messages, or cross-agent attribution that won't drift across renames. Need new packages, env vars, or other NixOS config for yourself? You can't edit your own config directly — message the manager (recipient `manager`) describing what you need + why. The manager evaluates the request (it doesn't rubber-stamp), edits `/agents/{label}/config/agent.nix` on your behalf, commits, and submits an approval that the operator can accept on the dashboard; on approve hive-c0re rebuilds your container with the new config. diff --git a/hive-ag3nt/prompts/manager.md b/hive-ag3nt/prompts/manager.md index 15f85a5..3ac7c90 100644 --- a/hive-ag3nt/prompts/manager.md +++ b/hive-ag3nt/prompts/manager.md @@ -4,7 +4,8 @@ Tools (hyperhive surface): - `mcp__hyperhive__recv(wait_seconds?, max?)` — drain inbox messages. Without `wait_seconds` (or with `0`) it returns immediately — a cheap inbox peek you can drop between actions. To **wait** when you have nothing else to do, call with a long wait (e.g. `wait_seconds: 180`, the max) — you'll wake instantly on new work, otherwise return after the timeout. Use that instead of ending the turn or sleeping in a Bash command. `max` (default 1, cap 32) drains several queued messages in one call. - `mcp__hyperhive__send(to, body)` — message an agent (by name), another peer, or the operator (`operator` surfaces in the dashboard). Use `to: "*"` to broadcast to all agents (they receive a hint that it's a broadcast and may not need action). -- `mcp__hyperhive__request_spawn(name, description?)` — queue a brand-new sub-agent for operator approval (≤9 char name). Pass an optional `description` and it appears on the dashboard approval card — no need to send a separate message explaining the request. +- `mcp__hyperhive__request_init_config(name, description?)` — **step 1 of a two-step spawn.** Queues an `InitConfig` approval (≤9 char name). On operator approve, hive-c0re seeds the proposed config repo at `/agents//config/` with a default `agent.nix` template and delivers a `config_ready` system event to your inbox. You then review, edit, and commit `agent.nix` before calling `request_spawn`. +- `mcp__hyperhive__request_spawn(name, description?)` — **step 2 of a two-step spawn.** Queues a Spawn approval; requires the proposed config repo to already exist (from a prior approved `request_init_config`). On operator approve, hive-c0re creates the container. Pass an optional `description` for the dashboard card. - `mcp__hyperhive__kill(name)` — graceful stop on a sub-agent. No approval required. - `mcp__hyperhive__start(name)` — start a stopped sub-agent. No approval required. - `mcp__hyperhive__restart(name)` — stop + start a sub-agent. No approval required. @@ -70,8 +71,9 @@ You're the policy gate between sub-agents and the operator's approval queue — Two ways to talk to the operator: `send(to: "operator", ...)` for fire-and-forget status / pointers (surfaces in the operator inbox), or `ask(question, options?)` when you need a decision (omit `to`, or pass `to: "operator"`). `ask` is non-blocking — it queues the question and returns an id immediately; the answer arrives on a future turn as a `question_answered` system event. Prefer `ask` over an open-ended `send` for anything you actually need to wait on. Same primitive can target a sub-agent (`to: ""`) when you need a structured answer from a peer rather than free-form chat. -Messages from sender `system` are hyperhive helper events (JSON body, `event` field discriminates): `approval_resolved`, `spawned`, `rebuilt`, `killed`, `destroyed`, `container_crash`, `needs_login`, `logged_in`, `needs_update`, `question_asked`, `question_answered`. Use these to react to lifecycle changes: +Messages from sender `system` are hyperhive helper events (JSON body, `event` field discriminates): `approval_resolved`, `config_ready`, `spawned`, `rebuilt`, `killed`, `destroyed`, `container_crash`, `needs_login`, `logged_in`, `needs_update`, `question_asked`, `question_answered`. Use these to react to lifecycle changes: +- `config_ready` — the proposed config repo for a new agent was just seeded (post-`InitConfig` approval). Review and edit `/agents//config/agent.nix`, commit your changes, then call `request_spawn` to proceed to the container-creation approval. - `needs_login` — agent has no claude session yet. You can't help directly (login is interactive OAuth on the operator side); flag the operator if it's been long. - `logged_in` — agent just completed login; first useful turn is imminent. Good time to brief them on what to do. - `needs_update` — agent's flake rev is stale. Call `update(name)` to rebuild — it's idempotent and doesn't need approval. From 939df10a615102c3fb030f0da2d381064dd46e26 Mon Sep 17 00:00:00 2001 From: iris Date: Wed, 20 May 2026 16:55:13 +0200 Subject: [PATCH 3/3] docs: document model/context-window config, dynamic watermarks, rate-limit scoping --- CLAUDE.md | 9 ++++++- docs/turn-loop.md | 61 ++++++++++++++++++++++++++++++++++------------- docs/web-ui.md | 9 +++++-- 3 files changed, 60 insertions(+), 19 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index b88c08b..68296b2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -139,11 +139,14 @@ hive-sh4re/ wire types (HostRequest/Response, AgentRequest/Response, nix/ modules/hive-c0re.nix systemd service + firewall + git wiring; + `contextWindowTokens` attrset (per-model, + injected as env vars into all containers); imports hive-forge.nix modules/hive-forge.nix optional in-container Forgejo (`hyperhive.forge.enable`, default on); Catppuccin Mocha theme via tmpfiles C+ copy - templates/harness-base.nix shared scaffolding for sub-agents + manager + templates/harness-base.nix shared scaffolding for sub-agents + manager; + `hyperhive.model` option (HIVE_DEFAULT_MODEL) templates/agent-base.nix sub-agent nixosConfiguration templates/manager.nix manager nixosConfiguration templates/weston-vnc.nix optional `hyperhive.gui.enable` @@ -205,3 +208,7 @@ read them à la carte. - **Rate-limit sentinel:** `{state_dir}/hyperhive-rate-limited` is written by the harness on 429 and cleared on retry. `ContainerView.rate_limited` reads it for the dashboard badge. +- **Context window:** defaults are in `services.hive-c0re.contextWindowTokens` + (host nix, affects all agents). Per-agent default model via + `hyperhive.model` in `agent.nix`. Watermarks are 75%/50% of the + effective window. diff --git a/docs/turn-loop.md b/docs/turn-loop.md index 8ad00e8..37659d6 100644 --- a/docs/turn-loop.md +++ b/docs/turn-loop.md @@ -21,9 +21,13 @@ Each agent harness (`hive-ag3nt serve` or `hive-m1nd serve`) runs: 6. Wait for claude to exit. Compaction is two-pronged — *reactive* on `Prompt is too long` and *proactive* on a context watermark (see [Compaction](#compaction) below). **Rate-limit detection**: - if stdout contains `429` or `rate_limit` markers, the harness - sets the `rate_limited` sentinel (`Bus::emit_status("rate_limited")`), - sleeps `HIVE_RATE_LIMIT_SLEEP_SECS` (default 300), then retries. + on stderr the harness does a raw-line match for `429` / + `rate_limit` markers; on stdout it only fires on parsed + `{"type":"error"}` JSON events (avoiding false positives when + agents discuss `rate_limit_error` in conversation text). On + detection the harness sets the `rate_limited` sentinel + (`Bus::emit_status("rate_limited")`), sleeps + `HIVE_RATE_LIMIT_SLEEP_SECS` (default 300), then retries. The dashboard and per-agent page show a `⊘ rate limited` badge while the harness is parked. 7. Emit `LiveEvent::TurnEnd { ok, note }`. Sleep `poll_ms` to avoid @@ -40,11 +44,33 @@ claude --print --verbose --output-format stream-json --model \ # wake prompt piped over stdin ``` -`` is read from `Bus::model()` on each turn, default -`haiku`. Operator can flip it at runtime with `/model ` in -the web terminal — the next turn picks it up. The choice is -persisted to `/state/hyperhive-model` so it survives restart; -override path: `HYPERHIVE_MODEL_FILE` env var for tests. +`` is read from `Bus::model()` on each turn. The initial +default is set by `hyperhive.model` in the agent's `agent.nix` +(NixOS option; propagates via `HIVE_DEFAULT_MODEL` env var; falls +back to `"haiku"` if unset). The operator can flip it at runtime +with `/model ` in the web terminal — the next turn picks it +up. The choice is persisted to `/state/hyperhive-model` so it +survives restart; override path: `HYPERHIVE_MODEL_FILE` env var +for tests. + +Context-window size is looked up per-model via +`events::context_window_tokens(model)`. Resolution order (first +match wins): + +1. `HIVE_CONTEXT_WINDOW_TOKENS_` env var, where `KEY` + (lowercased) is a substring of the active model name. Injected + by the meta flake from `services.hive-c0re.contextWindowTokens` + (host-level NixOS option, defaults: haiku=200k, sonnet=1M, + opus=1M). Override these for all agents at once without a + per-agent config change. +2. `HIVE_CONTEXT_WINDOW_TOKENS` — single global override for any + model (useful in dev / test). +3. Hard fallback: `200_000` (conservative; only reached outside + NixOS where the env vars aren't set). + +The effective window drives watermarks and is exposed at runtime +via `/api/state.context_window_tokens` so the UI can show a +percentage-of-window ctx badge. `--continue` keeps a persistent session per agent (claude stores sessions in `~/.claude/projects/`, which is bind-mounted @@ -76,17 +102,20 @@ owns it explicitly in `turn::drive_turn`. There are two triggers: persist in-flight task state, decisions, and file paths before the conversation detail collapses into a summary. -The watermark is `HIVE_COMPACT_WATERMARK_TOKENS` (default `150_000`, -~75% of a 200k window); set it to `0` to disable proactive compaction -entirely (the reactive path always applies). The proactive path is -best-effort — a failed checkpoint turn or `/compact` is surfaced as a -`Note` but never fails the turn that already succeeded. The operator -can also force a compaction any time via `/api/compact`. +The compact watermark defaults to **75% of `context_window_tokens(model)`** +(dynamically derived — 150k for haiku, 750k for sonnet/opus). Override +with `HIVE_COMPACT_WATERMARK_TOKENS` (absolute token count); set to `0` +to disable proactive compaction entirely (the reactive path always +applies). The proactive path is best-effort — a failed checkpoint turn +or `/compact` is surfaced as a `Note` but never fails the turn that +already succeeded. The operator can also force a compaction any time +via `/api/compact`. - **Auto session-reset** — a third path that fires when both conditions hold: context is ≥ a watermark (`HIVE_AUTO_RESET_WATERMARK_TOKENS`, - default `100_000`) AND the time since the last turn exceeds the - assumed prompt-cache TTL (`HIVE_CACHE_TTL_SECS`, default `300`). + default **50% of `context_window_tokens(model)`**) AND the time since + the last turn exceeds the assumed prompt-cache TTL + (`HIVE_CACHE_TTL_SECS`, default `3600`). Claude's prompt cache lives ~5 minutes; if the cache is already cold, resuming with `--continue` pays the full re-upload cost of the current context with no benefit over starting fresh. So: diff --git a/docs/web-ui.md b/docs/web-ui.md index 7e6b815..31b6526 100644 --- a/docs/web-ui.md +++ b/docs/web-ui.md @@ -382,7 +382,9 @@ Layout, top to bottom: (input + cache_read + cache_write of the most recent model call in the just-ended turn). This is the **actual context window utilisation** — the number to watch when - deciding whether to compact. + deciding whether to compact. When `context_window_tokens` + is available from `/api/state`, the badge tooltip shows the + percentage of window used. - Cost badge: `cost · 1.3M` — cumulative tokens billed across **every inference** in the last turn (sum of all per-call prompts). Tool-heavy turns rebill the cached @@ -407,7 +409,10 @@ Layout, top to bottom: Polling: `/api/state` is fetched **once** on cold load, and again while `status === 'needs_login_in_progress'` (login session output isn't event-shaped yet). Every other badge - updates from SSE; no periodic refresh timer runs. + updates from SSE; no periodic refresh timer runs. Snapshot + includes `context_window_tokens` (effective window size for + the agent's current model, from `events::context_window_tokens`) + used to compute percentage-of-window in the ctx badge tooltip. - Inbox `
` block (collapsed): `inbox · N` — last 30 messages addressed to this agent, fetched via `AgentRequest::Recent { limit: 30 }`. Reply messages (those