diff --git a/TODO.md b/TODO.md index 53016100..fc1d8c45 100644 --- a/TODO.md +++ b/TODO.md @@ -1,23 +1,141 @@ -# Hyperhive TODOs +# TODO -## Architecture / Features +Pick anything from here when relevant. Cross-cutting design notes live in +[CLAUDE.md](CLAUDE.md); high-level project intro in [README.md](README.md). -- Shared space for all agents to access documents/files without manager routing -- Private git forge agents can push to and create new repos in -- Move bind mounts in agents to `/agents//state` so path for agent = path for manager +## Security -## Reminder Tool +- **Unprivileged containers (userns mapping).** Today the nspawn container + runs as a fully privileged root. Goal: `PrivateUsersChown=yes` (or the + nixos-container equivalent) so uid 0 inside maps to an unprivileged uid + on the host, and a container-root compromise lands the attacker on an + ordinary user account, not the host's root. Requires per-agent state + dirs to be chown'd to that uid on the host side. The per-agent git + identity (currently injected via `programs.git.config.user` against + the root user in `setup_applied`'s generated flake) also needs to be + provisioned for whatever non-root user claude runs as, or commits + the manager makes against `/agents//config` will fall back to a + generic `nixos@…` identity. +- **Bash command allow-list.** Replace the blanket `Bash` allow with a + pattern allow-list (`Bash(git *)`, `Bash(nix build .*)`, etc.) per + claude-code's `--allowedTools` extended grammar. Likely lives in + `agent.nix` so each agent can scope its own shell surface. -- Handle text overflow → suggest file_path option for long messages -- Per-agent reminder limits (burst capacity, rate limiting) -- **File path delivery**: currently unused in scheduler delivery loop — implement file write/delivery to /state//reminders/ or similar -- **Orphan reminders**: handle partial failures (e.g. delivery succeeds but mark_reminder_sent fails) to avoid resending -- **Unbounded batches**: implement per-cycle delivery limit so burst of 10k reminders doesn't flood the broker in one cycle -- **Scheduler shutdown**: add graceful shutdown signal when coordinator is destroyed (currently runs forever) -- **DB lock contention**: under high reminder volume, many concurrent mark_reminder_sent calls may serialize behind the Mutex lock — consider batch updates +## Operational hygiene (post-meta-flake) -## Dashboard +- **Tag retention.** Every approval mints up to 5 tags in + `applied//.git` (`proposal/`, `approved/`, `building/`, + `deployed/`, plus `failed/` or `denied/`). Every successful + deploy adds one commit to `/var/lib/hyperhive/meta/.git`. + Both grow unbounded. A retention policy — keep all + `deployed/*` indefinitely, age-out `failed/` + `denied/` + after N days, drop `proposal/` + `approved/` + `building/` + once a terminal sibling lands — would keep the audit + trails browsable without forever-growth. -- Per-agent reminder status (pending, delivered) -- Reminder query interface for debugging -- Display reminder delivery errors (failed sends, mark failures) +- **Inert `nix flake lock` no-args call in `meta::sync_agents`.** + Still valid in current nix (resolves missing inputs without + bumping existing ones) but parallel to the deprecated + `--update-input` we just had to migrate. Worth keeping an + eye on; if it gets renamed too, sync_agents stops being + able to seed a fresh meta repo. + +## Bugs + +- **Pending question doesn't always appear on the dashboard.** + Repro: manager calls `ask_operator`, tool result is + `question queued (id=N)` (so the row is in sqlite), but the + M1ND H4S QU3STI0NS section keeps showing "no pending + questions". Last seen with id=5. Diagnostic step landed: + `api_state` now warn-logs (target=`api_state`) when any of + its source queries fail instead of silently + `unwrap_or_default`-ing — next repro should print the + underlying error in journald and tell us whether this is + sqlite (likely `OperatorQuestions::pending()` row-decode + panic on a migrated column) or dashboard-JS-side + (`renderQuestions` exception). Re-investigate with the new + log once the bug fires. + +## UI / UX + +- **Dashboard layout overhaul.** A 3-column attempt (swarm + / 0per4t0r 1n / m3ss4g3s) landed + was reverted in 74ba8a6 + — looked worse in practice (sticky col-heads fighting the + banner, sub-heads too small, columns too narrow for the + container rows). Sections are now ordered semantically in + a single column (swarm bits first, then decisions, then + messages) which is a no-cost improvement. The bigger + restructure is still worth doing; next attempt should: + - keep current widths usable (don't crunch container + rows < ~36em — they have a lot inline) + - default the heavy-but-rare sections (kept-state, meta- + inputs, msg-flow history) into a collapsed `
` + so they don't dominate when empty + - drop the per-section banner divider lines in favour of + something quieter (a single border-top on the h2?) + - try a *masonry-ish* layout (CSS `grid-template-rows: + masonry` once browsers support it; or just two columns + where messages floats on the right at wide viewports + while the rest stacks left). avoid sticky headers — they + fought the page banner last time. + + + +- **Web UI for config repos + meta deploy log.** Browse + per-agent proposed / applied tags + (`proposal/* / approved/* / building/* / deployed/* / + failed/* / denied/*`) plus the swarm-wide meta repo's git + log on the dashboard. Read-only log + diff + raw-file view + is enough — something lighter than a full forge. The meta + log already answers "what's deployed where + when"; this + surfaces it without an ssh-to-host detour. + +- **xterm.js terminal** embedded per-agent, attached to a PTY exposed by + the harness. Pairs well with the unprivileged-container work — would let + the operator drop into the container without `nixos-container root-login`. + +## Telemetry + +- **Harness stats per agent in sqlite, charted on the agent page.** + bitburner-agent samples 18 series; for hyperhive the generally-applicable + ones are: + - turns/min, tool calls/turn, turn duration p50/p95 + - claude exit code distribution (ok vs `--compact`-retry vs failure) + - inbox depth (current + max-over-window) + - messages sent/received per turn (split by recipient: peer / operator / + manager / system) + - approval queue length (across all agents — dashboard-level) + - per-tool usage counts (Read/Edit/Bash/send/recv/…) + - time-since-last-turn (helps spot stuck agents) + - notes file size growth (cues compaction) + Backend: a `stats` table with `(agent, ts, key, value)` written from + the harness on `TurnEnd`; `GET /api/stats?since=…` returns the + series; agent page renders with a small chart lib (uPlot is light). + +## Spawn flow + +- **Two-step spawn.** Today `request_spawn(name)` is one shot: manager + asks → operator approves → container is created with a default + `agent.nix` and empty `/state/`. Manager has no way to pre-stage + per-agent prompt material, package additions, or initial notes before + the agent first wakes. Split into: + 1. `request_spawn_draft(name)` — host creates the per-agent + `proposed/` repo (initial commit) and `state/` dir with no + container; manager now has `/agents//{config,state}/` to + edit + commit just like an existing agent. + 2. `request_spawn_commit(name, commit_ref)` — submits the queued + approval; operator sees the diff in the dashboard like a normal + `apply_commit`; on approve the container is created from that + commit. + Backwards-compat: keep the existing one-shot `request_spawn` for + trivial agents (operator can still type a name in the dashboard). + Surface "drafts" as a new section between K3PT ST4T3 and approvals. + +## Loop substance + +- **Notes compaction.** `/state/` is bind-mounted persistently and agents + are told (in the system prompt) to keep `/state/notes.md` for durable + knowledge — but we don't currently nudge them to compact when notes + grow. Bitburner-agent's pattern: a short-lived secondary claude session + that takes the existing notes + a "compact this" prompt and rewrites + them in place. Add when the notes start bloating. diff --git a/hive-c0re/src/agent_server.rs b/hive-c0re/src/agent_server.rs index 961de512..a4cad214 100644 --- a/hive-c0re/src/agent_server.rs +++ b/hive-c0re/src/agent_server.rs @@ -194,48 +194,26 @@ async fn dispatch(req: &AgentRequest, agent: &str, coord: &Arc) -> file_path, } => { use hive_sh4re::ReminderTiming; - - // Calculate the due_at timestamp, propagating errors instead of silently - // defaulting to epoch 1970 on overflow/conversion failure. - let due_at_result: Result = match timing { + let due_at = match timing { ReminderTiming::InSeconds { seconds } => { - let now = std::time::SystemTime::now(); - let future = match now.checked_add(std::time::Duration::from_secs(*seconds)) { - Some(t) => t, - None => return AgentResponse::Err { - message: format!("InSeconds overflow: {seconds}s exceeds system time range"), - }, - }; - let duration = match future.duration_since(std::time::UNIX_EPOCH) { - Ok(d) => d, - Err(e) => return AgentResponse::Err { - message: format!("system time before UNIX_EPOCH: {e}"), - }, - }; - match i64::try_from(duration.as_secs()) { - Ok(ts) => Ok(ts), - Err(e) => return AgentResponse::Err { - message: format!("unix timestamp exceeds i64 range: {e}"), - }, - } + std::time::SystemTime::now() + .checked_add(std::time::Duration::from_secs(*seconds)) + .and_then(|t| { + t.duration_since(std::time::UNIX_EPOCH) + .ok() + .and_then(|d| i64::try_from(d.as_secs()).ok()) + }) + .unwrap_or(0) } - ReminderTiming::At { unix_timestamp } => Ok(*unix_timestamp), + ReminderTiming::At { unix_timestamp } => *unix_timestamp, }; - - match due_at_result { - Ok(due_at) => { - match broker.store_reminder(agent, message, file_path.as_deref(), due_at) { - Ok(id) => { - tracing::info!(%id, %agent, %due_at, "reminder scheduled"); - AgentResponse::Ok - } - Err(e) => AgentResponse::Err { - message: format!("failed to store reminder: {e:#}"), - }, - } + match broker.store_reminder(agent, message, file_path.as_deref(), due_at) { + Ok(id) => { + tracing::info!(%id, %agent, %due_at, "reminder scheduled"); + AgentResponse::Ok } Err(e) => AgentResponse::Err { - message: format!("invalid reminder timing: {e:#}"), + message: format!("failed to store reminder: {e:#}"), }, } } diff --git a/hive-c0re/src/broker.rs b/hive-c0re/src/broker.rs index 7935cdeb..20773263 100644 --- a/hive-c0re/src/broker.rs +++ b/hive-c0re/src/broker.rs @@ -253,25 +253,6 @@ impl Broker { .context("query reminders") } - /// Get all due reminders across all agents in a single query. - /// Returns a vec of (agent, id, message, file_path) tuples. - pub fn get_all_due_reminders(&self) -> Result)>> { - let conn = self.conn.lock().unwrap(); - let mut stmt = conn.prepare( - "SELECT agent, id, message, file_path FROM reminders WHERE due_at <= ?1 AND sent_at IS NULL ORDER BY agent, due_at ASC" - )?; - let rows = stmt.query_map(params![now_unix()], |row| { - Ok(( - row.get::<_, String>(0)?, - row.get::<_, i64>(1)?, - row.get::<_, String>(2)?, - row.get::<_, Option>(3)?, - )) - })?; - rows.collect::>>() - .context("query all due reminders") - } - /// Mark a reminder as sent (delivered). pub fn mark_reminder_sent(&self, id: i64) -> Result<()> { let conn = self.conn.lock().unwrap(); diff --git a/hive-c0re/src/main.rs b/hive-c0re/src/main.rs index 8d036822..5908b7b6 100644 --- a/hive-c0re/src/main.rs +++ b/hive-c0re/src/main.rs @@ -163,36 +163,41 @@ async fn main() -> Result<()> { tokio::spawn(async move { use hive_sh4re::Message; loop { - // Query all due reminders in a single DB call - match reminder_coord.broker.get_all_due_reminders() { - Ok(reminders) => { - for (agent, id, message, _file_path) in reminders { - // Deliver as inbox message from "reminder" - if let Err(e) = reminder_coord.broker.send(&Message { - from: "reminder".to_owned(), - to: agent.clone(), - body: message.clone(), - }) { - tracing::warn!( - reminder_id = id, - %agent, - error = ?e, - "failed to deliver reminder" - ); - continue; - } - // Mark as sent - if let Err(e) = reminder_coord.broker.mark_reminder_sent(id) { - tracing::warn!( - reminder_id = id, - error = ?e, - "failed to mark reminder sent" - ); + // Get all agents currently registered + let agents = reminder_coord.list_agents(); + for agent in agents { + match reminder_coord.broker.get_due_reminders(&agent) { + Ok(reminders) => { + for (id, message, _file_path) in reminders { + // Deliver as inbox message from "reminder" + if let Err(e) = reminder_coord.broker.send(&Message { + from: "reminder".to_owned(), + to: agent.clone(), + body: message.clone(), + }) { + tracing::warn!( + reminder_id = id, + %agent, + error = ?e, + "failed to deliver reminder" + ); + continue; + } + // Mark as sent + if let Err(e) = reminder_coord.broker.mark_reminder_sent(id) { + tracing::warn!( + reminder_id = id, + error = ?e, + "failed to mark reminder sent" + ); + } } } - } - Err(e) => { - tracing::warn!(error = ?e, "failed to query due reminders"); + Err(e) => tracing::warn!( + %agent, + error = ?e, + "failed to query due reminders" + ), } } tokio::time::sleep(std::time::Duration::from_secs(5)).await;