Compare commits

...
Author SHA1 Message Date
müde
39d8359c10 agent ui: event-driven status / model / token_usage / turn_state
new LiveEvent variants on the per-agent bus —
status_changed / model_changed / token_usage_changed /
turn_state_changed — replace the per-agent web UI's
/api/state polling for the badge row.

emit sites:
- Bus::set_model → model_changed
- Bus::record_usage → token_usage_changed
- Bus::set_state → turn_state_changed
- turn::wait_for_login → status_changed("online") on creds detect
- post_login_start / post_login_cancel → status_changed("needs_login_*")

per-agent endpoints (post_set_model / post_compact / post_new_session
/ post_cancel_turn / post_login_*) now all return 200; client
drops the post-submit refetch except on login transitions, which
still need /api/state to render the OAuth form + session stream.

client adds dispatch on the four new event kinds, threads
`currentLabel` through so the composer re-enables on a live
status flip, and no longer fires refreshState() from turn_end or
postModel — the events carry the same signal faster.

closes the per-agent half of the dashboard event-channel
refactor; TODO entry dropped.
2026-05-17 22:49:55 +02:00
müde
b444dac6e8 agent ui: consolidate status into state-row badges
drop the "● harness alive — turn loop running" paragraph; the
new #alive-badge chip in the state row carries the same signal
across all statuses (loading / online / needs-login / offline)
with colour coding. token-usage chip renamed + restyled as
#ctx-badge — primary number is total context-window tokens
used, mirroring claude code's "N tokens" indicator.

every state-row badge now has hover detail: state-badge gets
per-state tooltips + age suffix, model-chip explains the
/model command, last-turn shows the raw ms duration, ctx-badge
breaks out input / cache_read / cache_write / output.

new todo entry for the per-turn stats sink (start/end/model/
tokens/tool-call-count) the harness should be writing.
2026-05-17 22:36:02 +02:00
9 changed files with 199 additions and 34 deletions

View file

@ -23,7 +23,6 @@
- **Reminder delivery-error surface**: `reminder_scheduler::tick` logs failed deliveries but doesn't persist. Add `last_error TEXT, attempt_count INTEGER` columns + a banner on the dashboard row + a "retry" affordance. Needs a sqlite migration (idempotent ALTER TABLE).
- **Per-agent reminder status / query interface**: surface pending vs. delivered counts per agent (manager + each sub-agent) as a small chip on the container row.
- **Phase 6 follow-ups** — dashboard side is fully event-driven (Phase 6 leftovers landed); the per-agent web UI's lifecycle endpoints (`/api/{cancel,compact,model,new-session}`, `/login/*`) still 303-redirect-and-poll. Convert them to 200 + `data-no-refresh` so the per-agent page stops refetching `/api/state` on every operator click — `LiveEvent::Note` already covers cancel/compact/model/new-session, login state needs its own `NeedsLogin` / `LoggedIn` events on the per-agent bus.
- **Tombstones + meta_inputs events**: not yet event-derived. PURG3 + meta-update still trigger a post-submit `/api/state` refetch on the dashboard. Add `TombstoneAdded`/`TombstoneRemoved` + `MetaInputsChanged` so those forms can drop their refetch too and the cold-load is the only `/api/state` fetch in normal operation.
## Security
@ -76,6 +75,10 @@ how often the friction bites in normal use.
Field is optional, ignored if the referenced id is unknown / cross-
agent / out of retention.
## Telemetry
- **Per-turn stats log**: persist one row per claude turn in a new sqlite table on the per-agent state dir (or the host broker DB, indexed by agent). Columns: `started_at`, `ended_at`, `duration_ms`, `model`, `input_tokens`, `output_tokens`, `cache_read_input_tokens`, `cache_creation_input_tokens`, `tool_call_count`, `tool_call_breakdown` (JSON: `{Read: 12, Bash: 3, ...}`), `bytes_streamed`, `wake_reason` (recv'd message / reminder / operator-kick / manual), `result_kind` (ok / cancelled / failed-mid-turn / compacted), `note` (e.g. failure reason). Powers: per-agent dashboards (avg turn time over time, tool-usage histogram, cost projections from token counts × model rate), debugging stuck loops (look for repeated identical wake_reason + zero tool calls), and operator-visible "this is what your spend looked like this week" rollups. Source data is already mostly in the harness's `TurnState` + the per-event bus; just needs a sink. Keep a retention sweep (host-side) so the table doesn't grow forever.
## Bugs
- **Post-rebuild system-message missed wake**: at 09:13:14 the dashboard showed `system → damocles container rebuilt` as ✓ delivered, but the agent harness never ran a turn for it (no claude invocation, no operator-visible activity). A subsequent `recv()` from inside the agent returned `(empty)`, confirming the message was popped + marked delivered server-side — yet drove no turn. Most likely cause: the agent_server `serve_agent_stdio` task is up and answering MCP/socket calls, but the `hive-ag3nt::serve` long-poll loop that drives `drive_turn` either died silently during rebuild or never restarted. Investigate: (a) does hive-ag3nt's serve loop survive `nixos-container update` cleanly, or does its tokio runtime get torn down mid-loop? (b) is there an early-exit path on a transient socket error during rebuild that drops the serve task without notifying the manager? (c) compare timeline with manager's own post-rebuild wake to see if this is rebuilt-agents-only or universal. Could be related to the `recv_blocking` fix in `e423d57` if the rebuild restarts the broker mid-subscribe.

View file

@ -165,12 +165,38 @@ pre.diff {
font-size: 0.78em;
letter-spacing: 0.04em;
}
.token-usage {
color: var(--muted);
font-size: 0.8em;
/* Context-window badge. Mirrors Claude Code's bottom-right "N tokens"
chip single primary number (total prompt tokens in use), full
breakdown on hover. Sized/coloured like a peer of model-chip so
the state row reads as one row of chrome. */
.ctx-badge {
display: inline-block;
padding: 0.1em 0.6em;
border: 1px solid var(--purple-dim);
border-radius: 999px;
color: var(--green);
font-size: 0.78em;
letter-spacing: 0.04em;
cursor: default;
white-space: pre-line;
}
/* Harness reachability badge. Same chip shape + sizing as
`.state-badge` / `.model-chip` so the state row stays visually
uniform; colour communicates the actual reachability state. */
.status-badge {
display: inline-block;
padding: 0.25em 0.8em;
border: 1px solid;
border-radius: 999px;
font-size: 0.85em;
letter-spacing: 0.05em;
}
.status-badge.status-loading { color: var(--muted); border-color: var(--purple-dim); }
.status-badge.status-online { color: var(--green); border-color: var(--green);
text-shadow: 0 0 6px rgba(166, 227, 161, 0.55); }
.status-badge.status-needs-login { color: var(--amber); border-color: var(--amber); }
.status-badge.status-offline { color: var(--muted); border-color: var(--muted); }
.btn-dashlink {
color: var(--cyan);
border: 1px solid var(--cyan);

View file

@ -91,10 +91,10 @@
document.title = `${label} // hyperhive`;
}
function renderOnline(_label, root) {
root.append(
el('p', { class: 'status-online' }, '● harness alive — turn loop running'),
);
function renderOnline(_label, _root) {
// Online state is conveyed by the `#alive-badge` chip in the
// state row — no longer a separate paragraph in the status
// block (keeps the terminal the star, status row stays compact).
}
function renderNeedsLoginIdle(root) {
@ -168,6 +168,10 @@
// dispatcher to print local-only rows ('help', errors) and to clear
// the terminal on `/clear`.
let termAPI = null;
// Label captured from the first /api/state cold load — used by the
// bus-driven `status_changed` handler so it can re-enable the
// composer without waiting for the next snapshot fetch.
let currentLabel = '';
const SLASH_COMMANDS = [
{ name: '/help', desc: 'list slash commands' },
@ -192,9 +196,9 @@
const text = await resp.text().catch(() => '');
termAPI.row('turn-end-fail', '✗ /model failed: ' + resp.status
+ (text ? ' — ' + text : ''));
} else {
refreshState();
}
// No refreshState — the harness emits `model_changed` on the
// SSE bus and the chip handler picks it up live.
} catch (err) {
if (termAPI) termAPI.row('turn-end-fail', '✗ /model failed: ' + err);
}
@ -348,6 +352,13 @@
const h = Math.floor(m / 60);
return h + 'h ' + (m % 60) + 'm';
}
const STATE_TOOLTIPS = {
loading: 'harness not yet contacted',
offline: 'harness unreachable or claude not logged in',
idle: 'turn loop running, no claude invocation in flight',
thinking: 'claude is executing the current turn',
compacting: 'operator-triggered /compact running on the persistent session',
};
function renderStateBadge() {
const badge = $('state-badge');
if (!badge) return;
@ -355,6 +366,7 @@
const age = fmtAge(Date.now() - stateSince);
badge.textContent = def.glyph + ' ' + def.text + ' · ' + age;
badge.className = 'state-badge state-' + stateName;
badge.title = (STATE_TOOLTIPS[stateName] || '') + '\nin this state for ' + age;
const cancelBtn = $('cancel-btn');
if (cancelBtn) cancelBtn.hidden = stateName !== 'thinking';
}
@ -405,27 +417,53 @@
list.append(li);
}
}
// Harness reachability badge: derived from the same `s.status` the
// status block reads. Each status maps to a glyph + label + colour
// class. Lives in the state row so the operator sees boot/login/
// online without losing terminal real-estate to a paragraph.
const ALIVE_LABELS = {
loading: { glyph: '…', text: 'connecting', cls: 'status-loading' },
online: { glyph: '●', text: 'alive', cls: 'status-online' },
needs_login_idle: { glyph: '◌', text: 'needs login', cls: 'status-needs-login' },
needs_login_in_progress: { glyph: '◌', text: 'logging in', cls: 'status-needs-login' },
offline: { glyph: '○', text: 'offline', cls: 'status-offline' },
};
function renderAliveBadge(status) {
const el_ = $('alive-badge');
if (!el_) return;
const def = ALIVE_LABELS[status] || ALIVE_LABELS.loading;
el_.textContent = def.glyph + ' ' + def.text;
el_.className = 'status-badge ' + def.cls;
}
function renderModelChip(model) {
const el_ = $('model-chip');
if (!el_) return;
if (!model) { el_.hidden = true; return; }
el_.hidden = false;
el_.textContent = 'model · ' + model;
el_.title = `claude --model ${model}\nset via the operator's /model command; persists across turns until changed`;
}
// Context badge — mirrors Claude Code's bottom-right "N tokens"
// indicator. Primary number is total prompt tokens used in the
// current context window (input + both cache axes); hover for the
// breakdown including output. Kept as chrome on the state row so
// the terminal stays the star.
function renderTokenUsage(u) {
const el_ = $('token-usage');
const el_ = $('ctx-badge');
if (!el_) return;
if (!u) { el_.hidden = true; return; }
const ctx = u.input_tokens + u.cache_read_input_tokens + u.cache_creation_input_tokens;
const fmt = (n) => n >= 1000 ? (n / 1000).toFixed(1) + 'k' : String(n);
el_.hidden = false;
el_.title = [
'context window in use',
'input: ' + u.input_tokens,
'output: ' + u.output_tokens,
'cache_read: ' + u.cache_read_input_tokens,
'cache_write: ' + u.cache_creation_input_tokens,
].join(' · ');
el_.textContent = '· ctx ' + fmt(ctx) + ' in · ' + fmt(u.output_tokens) + ' out';
'output (last turn): ' + u.output_tokens,
].join('\n');
el_.textContent = 'ctx · ' + fmt(ctx);
}
function renderLastTurn(ms) {
const el_ = $('last-turn');
@ -435,6 +473,7 @@
else if (ms < 60_000) s = (ms / 1000).toFixed(1) + 's';
else s = Math.floor(ms / 60_000) + 'm ' + Math.floor((ms / 1000) % 60) + 's';
el_.textContent = '· last turn ' + s;
el_.title = `wall-clock duration of the last completed claude turn (${ms} ms)`;
el_.hidden = false;
}
function startStateTicker() {
@ -488,6 +527,7 @@
if (!resp.ok) throw new Error('http ' + resp.status);
const s = await resp.json();
if (!headerSet) { setHeader(s.label, s.dashboard_port); headerSet = true; }
currentLabel = s.label;
renderTermInput(s.label, s.status === 'online');
renderInbox(s.inbox || []);
// Authoritative state comes from the harness via /api/state.
@ -499,6 +539,7 @@
} else if (s.turn_state) {
setStateAbs(s.turn_state, s.turn_state_since);
}
renderAliveBadge(s.status);
renderModelChip(s.model);
renderTokenUsage(s.token_usage);
// Skip the re-render if nothing structurally changed. The most
@ -689,8 +730,6 @@
openTurnsFromHistory = Math.max(0, openTurnsFromHistory - 1);
} else {
setBannerActive(false); setState('idle');
// Login may have just landed (or session re-enters Online).
refreshState();
}
const cls = ev.ok ? 'turn-end-ok' : 'turn-end-fail';
api.row(cls,
@ -702,6 +741,38 @@
const v = Object.assign({}, ev); delete v.kind;
renderStream(v, api);
},
// Bus-driven state/badges. `status_changed` may also need a
// /api/state refresh to render the login `#status` block
// (which carries the OAuth URL + form), so we kick the
// existing refresh path on that transition. Online → only
// the badge updates; no /api/state fetch needed.
status_changed(ev, api) {
if (api.fromHistory) return;
renderAliveBadge(ev.status);
renderTermInput(currentLabel, ev.status === 'online');
// Login-flow transitions need the #status block rebuilt
// (it carries the OAuth URL + form). The existing
// refreshState path also re-arms the in-progress poll for
// session output streaming. Online → only the badge moves;
// no /api/state fetch is necessary.
if (ev.status !== 'online' && ev.status !== lastStatus) {
refreshState();
} else if (ev.status === 'online' && lastStatus !== 'online') {
// Status block stays as-is or shows the previous
// login UI; clear it so the operator sees a clean
// online state without a separate refetch.
const root = $('status');
if (root) root.innerHTML = '';
lastStatus = 'online';
}
},
model_changed(ev, api) { if (!api.fromHistory) renderModelChip(ev.model); },
token_usage_changed(ev, api) {
if (!api.fromHistory) renderTokenUsage(ev.usage);
},
turn_state_changed(ev, api) {
if (!api.fromHistory) setStateAbs(ev.state, ev.since_unix);
},
},
onBackfillDone() {
// If the last replayed turn never closed, the banner shimmer +

View file

@ -14,10 +14,11 @@
</div>
<div id="state-row">
<span id="alive-badge" class="status-badge status-loading" title="harness reachability"></span>
<span id="state-badge" class="state-badge state-loading">… booting</span>
<span id="model-chip" class="model-chip" hidden></span>
<span id="ctx-badge" class="ctx-badge" hidden title="tokens used in the current context window"></span>
<span id="last-turn" class="last-turn" hidden></span>
<span id="token-usage" class="token-usage" hidden></span>
<button type="button" id="cancel-btn" class="btn-cancel-turn" hidden>■ cancel turn</button>
<button type="button" id="new-session-btn" class="btn-new-session"
title="next turn runs without --continue, starting a fresh claude session">↻ new session</button>

View file

@ -101,7 +101,7 @@ async fn main() -> Result<()> {
// Partial-run mode: keep the harness alive (so the web UI
// stays bound) but don't drive the turn loop. Poll the
// claude dir; once a session lands we enter `serve`.
turn::wait_for_login(&claude_dir, login_state.clone(), poll_ms).await;
turn::wait_for_login(&claude_dir, login_state.clone(), &bus, poll_ms).await;
serve(
&cli.socket,
Duration::from_millis(poll_ms),

View file

@ -86,7 +86,7 @@ async fn main() -> Result<()> {
.await
}
LoginState::NeedsLogin => {
turn::wait_for_login(&claude_dir, login_state, poll_ms).await;
turn::wait_for_login(&claude_dir, login_state, &bus, poll_ms).await;
serve(
&cli.socket,
Duration::from_millis(poll_ms),

View file

@ -118,6 +118,30 @@ pub enum LiveEvent {
/// Turn finished. `ok=false` means claude exited non-zero or the
/// harness hit a transport error.
TurnEnd { ok: bool, note: Option<String> },
/// Harness reachability flipped: `"online"` /
/// `"needs_login_idle"` / `"needs_login_in_progress"`. The web UI
/// drives the alive badge from this so the operator sees a login
/// land (or get revoked) without polling. Session detail
/// (`url`/`output`/`finished`) is still served by `/api/state`
/// during the short-lived in-progress window — the client
/// re-fetches only while that flow is active.
StatusChanged { status: String },
/// `/api/model` switched the active claude model. The web UI
/// updates the chip + the per-turn stats sink will key off this
/// to mark the boundary in its log.
ModelChanged { model: String },
/// Final-turn `usage` block landed (input + output + cache
/// counters). Powers the context-window badge + accumulates into
/// the per-turn stats sink.
TokenUsageChanged { usage: TokenUsage },
/// Harness's `TurnState` transitioned (idle / thinking /
/// compacting). `since_unix` matches `Bus::state_snapshot().1`
/// so the client's elapsed-time ticker keeps progressing across
/// SSE reconnects without drift.
TurnStateChanged {
state: TurnState,
since_unix: i64,
},
}
/// sqlite-backed event log. Wraps a `Connection` behind a `Mutex` so the
@ -149,6 +173,10 @@ impl EventStore {
LiveEvent::Stream(_) => "stream",
LiveEvent::Note { .. } => "note",
LiveEvent::TurnEnd { .. } => "turn_end",
LiveEvent::StatusChanged { .. } => "status_changed",
LiveEvent::ModelChanged { .. } => "model_changed",
LiveEvent::TokenUsageChanged { .. } => "token_usage_changed",
LiveEvent::TurnStateChanged { .. } => "turn_state_changed",
};
let payload = serde_json::to_string(event).unwrap_or_else(|_| "null".into());
let conn = self.conn.lock().unwrap();
@ -216,7 +244,7 @@ impl TokenUsage {
/// reads via `/api/state` and renders. Lives alongside the bus
/// because everyone who has a `Bus` already has the right handle to
/// poke the state on transitions.
#[derive(Debug, Clone, Copy, Serialize, PartialEq, Eq)]
#[derive(Debug, Clone, Copy, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum TurnState {
/// Inbox is empty / waiting on `Recv`.
@ -340,11 +368,13 @@ impl Bus {
if let Err(e) = persist_model(&value) {
tracing::warn!(error = ?e, "model: persist failed");
}
self.emit(LiveEvent::ModelChanged { model: value });
}
/// Record the latest token usage from a completed turn.
pub fn record_usage(&self, usage: TokenUsage) {
*self.last_usage.lock().unwrap() = Some(usage);
self.emit(LiveEvent::TokenUsageChanged { usage });
}
/// Last known token usage, or `None` if no turn has completed yet.
@ -356,11 +386,31 @@ impl Bus {
/// Update the harness's authoritative turn-loop state. Records
/// the transition time so `state_snapshot` can return a since-age.
pub fn set_state(&self, next: TurnState) {
let mut guard = self.state.lock().unwrap();
if guard.0 == next {
return;
let since;
{
let mut guard = self.state.lock().unwrap();
if guard.0 == next {
return;
}
*guard = (next, now_unix());
since = guard.1;
}
*guard = (next, now_unix());
self.emit(LiveEvent::TurnStateChanged {
state: next,
since_unix: since,
});
}
/// Broadcast a status flip (online / needs_login_*). Called by
/// the bin entry points + `turn::wait_for_login` + the
/// `post_login_*` handlers — every site that mutates the
/// `Arc<Mutex<LoginState>>` should also call this so the web UI
/// drops its periodic /api/state poll while a turn loop is
/// running.
pub fn emit_status(&self, status: impl Into<String>) {
self.emit(LiveEvent::StatusChanged {
status: status.into(),
});
}
/// Current state + since-when (unix seconds). Snapshot copy, no lock held.

View file

@ -169,7 +169,12 @@ pub fn emit_turn_end(bus: &Bus, outcome: &TurnOutcome) {
/// Block until the bound `~/.claude/` dir contains a session, polling
/// `claude_dir` on a `poll_ms` interval (min 2s). Flips `state` to
/// `Online` when login lands; caller resumes its serve loop.
pub async fn wait_for_login(claude_dir: &Path, state: Arc<Mutex<LoginState>>, poll_ms: u64) {
pub async fn wait_for_login(
claude_dir: &Path,
state: Arc<Mutex<LoginState>>,
bus: &Bus,
poll_ms: u64,
) {
tracing::warn!(
claude_dir = %claude_dir.display(),
"no claude session — staying in partial-run mode (web UI only)"
@ -180,6 +185,7 @@ pub async fn wait_for_login(claude_dir: &Path, state: Arc<Mutex<LoginState>>, po
if login::has_session(claude_dir) {
tracing::info!("claude session detected — entering turn loop");
*state.lock().unwrap() = LoginState::Online;
bus.emit_status("online");
return;
}
}

View file

@ -16,7 +16,7 @@ use axum::{
extract::State,
http::StatusCode,
response::{
IntoResponse, Redirect, Response,
IntoResponse, Response,
sse::{Event, KeepAlive, Sse},
},
routing::{get, post},
@ -388,13 +388,19 @@ async fn post_login_start(State(state): State<AppState>) -> Response {
{
let guard = state.session.lock().unwrap();
if guard.is_some() {
return Redirect::to("/").into_response();
return (axum::http::StatusCode::OK, "ok").into_response();
}
}
match LoginSession::start() {
Ok(session) => {
*state.session.lock().unwrap() = Some(Arc::new(session));
Redirect::to("/").into_response()
// Flip status from needs_login_idle → needs_login_in_progress
// so the web UI's badge + polling kick in (polling is still
// the right tool for the streaming session output during
// the login flow itself; events drop the poll for
// *everything else*).
state.bus.emit_status("needs_login_in_progress");
(axum::http::StatusCode::OK, "ok").into_response()
}
Err(e) => error_response(&format!("login start failed: {e:#}")),
}
@ -413,7 +419,7 @@ async fn post_login_code(State(state): State<AppState>, Form(form): Form<CodeFor
if let Err(e) = session.submit_code(&form.code).await {
return error_response(&format!("submit code failed: {e:#}"));
}
Redirect::to("/").into_response()
(axum::http::StatusCode::OK, "ok").into_response()
}
async fn post_login_cancel(State(state): State<AppState>) -> Response {
@ -422,7 +428,9 @@ async fn post_login_cancel(State(state): State<AppState>) -> Response {
session.close_stdin().await;
session.kill();
}
Redirect::to("/").into_response()
// Back to needs_login_idle (LoginState unchanged, session gone).
state.bus.emit_status("needs_login_idle");
(axum::http::StatusCode::OK, "ok").into_response()
}
/// Operator-initiated session compaction. Spawns `turn::compact_session`
@ -452,7 +460,7 @@ async fn post_set_model(State(state): State<AppState>, Form(form): Form<ModelFor
text: format!("operator: /model — claude model set to '{name}' for future turns"),
});
tracing::info!(%name, "operator set model");
Redirect::to("/").into_response()
(axum::http::StatusCode::OK, "ok").into_response()
}
async fn post_compact(State(state): State<AppState>) -> Response {
@ -483,7 +491,7 @@ async fn post_compact(State(state): State<AppState>) -> Response {
});
}
});
Redirect::to("/").into_response()
(axum::http::StatusCode::OK, "ok").into_response()
}
/// Cancel the in-flight claude turn. Coarse-grained: shells out
@ -504,7 +512,7 @@ async fn post_new_session(State(state): State<AppState>) -> Response {
state.bus.emit(crate::events::LiveEvent::Note {
text: "operator: new session armed — next turn runs without --continue".into(),
});
Redirect::to("/").into_response()
(axum::http::StatusCode::OK, "ok").into_response()
}
async fn post_cancel_turn(State(state): State<AppState>) -> Response {
@ -525,7 +533,7 @@ async fn post_cancel_turn(State(state): State<AppState>) -> Response {
Err(e) => format!("operator: /cancel — pkill failed: {e}"),
};
state.bus.emit(crate::events::LiveEvent::Note { text: note });
Redirect::to("/").into_response()
(axum::http::StatusCode::OK, "ok").into_response()
}
fn error_response(message: &str) -> Response {