TODO.md

@@ -1,23 +1,141 @@
-# Hyperhive TODOs
+# TODO
 
-## Architecture / Features
+Pick anything from here when relevant. Cross-cutting design notes live in
+[CLAUDE.md](CLAUDE.md); high-level project intro in [README.md](README.md).
 
-- Shared space for all agents to access documents/files without manager routing
-- Private git forge agents can push to and create new repos in
-- Move bind mounts in agents to `/agents/<name>/state` so path for agent = path for manager
+## Security
 
-## Reminder Tool
+- **Unprivileged containers (userns mapping).** Today the nspawn container
+  runs as a fully privileged root. Goal: `PrivateUsersChown=yes` (or the
+  nixos-container equivalent) so uid 0 inside maps to an unprivileged uid
+  on the host, and a container-root compromise lands the attacker on an
+  ordinary user account, not the host's root. Requires per-agent state
+  dirs to be chown'd to that uid on the host side. The per-agent git
+  identity (currently injected via `programs.git.config.user` against
+  the root user in `setup_applied`'s generated flake) also needs to be
+  provisioned for whatever non-root user claude runs as, or commits
+  the manager makes against `/agents/<n>/config` will fall back to a
+  generic `nixos@…` identity.
+- **Bash command allow-list.** Replace the blanket `Bash` allow with a
+  pattern allow-list (`Bash(git *)`, `Bash(nix build .*)`, etc.) per
+  claude-code's `--allowedTools` extended grammar. Likely lives in
+  `agent.nix` so each agent can scope its own shell surface.
 
-- Handle text overflow → suggest file_path option for long messages
-- Per-agent reminder limits (burst capacity, rate limiting)
-- **File path delivery**: currently unused in scheduler delivery loop — implement file write/delivery to /state/<agent>/reminders/ or similar
-- **Orphan reminders**: handle partial failures (e.g. delivery succeeds but mark_reminder_sent fails) to avoid resending
-- **Unbounded batches**: implement per-cycle delivery limit so burst of 10k reminders doesn't flood the broker in one cycle
-- **Scheduler shutdown**: add graceful shutdown signal when coordinator is destroyed (currently runs forever)
-- **DB lock contention**: under high reminder volume, many concurrent mark_reminder_sent calls may serialize behind the Mutex lock — consider batch updates
+## Operational hygiene (post-meta-flake)
 
-## Dashboard
+- **Tag retention.** Every approval mints up to 5 tags in
+  `applied/<n>/.git` (`proposal/`, `approved/`, `building/`,
+  `deployed/`, plus `failed/` or `denied/`). Every successful
+  deploy adds one commit to `/var/lib/hyperhive/meta/.git`.
+  Both grow unbounded. A retention policy — keep all
+  `deployed/*` indefinitely, age-out `failed/` + `denied/`
+  after N days, drop `proposal/` + `approved/` + `building/`
+  once a terminal sibling lands — would keep the audit
+  trails browsable without forever-growth.
 
-- Per-agent reminder status (pending, delivered)
-- Reminder query interface for debugging
-- Display reminder delivery errors (failed sends, mark failures)
+- **Inert `nix flake lock` no-args call in `meta::sync_agents`.**
+  Still valid in current nix (resolves missing inputs without
+  bumping existing ones) but parallel to the deprecated
+  `--update-input` we just had to migrate. Worth keeping an
+  eye on; if it gets renamed too, sync_agents stops being
+  able to seed a fresh meta repo.
+
+## Bugs
+
+- **Pending question doesn't always appear on the dashboard.**
+  Repro: manager calls `ask_operator`, tool result is
+  `question queued (id=N)` (so the row is in sqlite), but the
+  M1ND H4S QU3STI0NS section keeps showing "no pending
+  questions". Last seen with id=5. Diagnostic step landed:
+  `api_state` now warn-logs (target=`api_state`) when any of
+  its source queries fail instead of silently
+  `unwrap_or_default`-ing — next repro should print the
+  underlying error in journald and tell us whether this is
+  sqlite (likely `OperatorQuestions::pending()` row-decode
+  panic on a migrated column) or dashboard-JS-side
+  (`renderQuestions` exception). Re-investigate with the new
+  log once the bug fires.
+
+## UI / UX
+
+- **Dashboard layout overhaul.** A 3-column attempt (swarm
+  / 0per4t0r 1n / m3ss4g3s) landed + was reverted in 74ba8a6
+  — looked worse in practice (sticky col-heads fighting the
+  banner, sub-heads too small, columns too narrow for the
+  container rows). Sections are now ordered semantically in
+  a single column (swarm bits first, then decisions, then
+  messages) which is a no-cost improvement. The bigger
+  restructure is still worth doing; next attempt should:
+  - keep current widths usable (don't crunch container
+    rows < ~36em — they have a lot inline)
+  - default the heavy-but-rare sections (kept-state, meta-
+    inputs, msg-flow history) into a collapsed `<details>`
+    so they don't dominate when empty
+  - drop the per-section banner divider lines in favour of
+    something quieter (a single border-top on the h2?)
+  - try a *masonry-ish* layout (CSS `grid-template-rows:
+    masonry` once browsers support it; or just two columns
+    where messages floats on the right at wide viewports
+    while the rest stacks left). avoid sticky headers — they
+    fought the page banner last time.
+
+
+
+- **Web UI for config repos + meta deploy log.** Browse
+  per-agent proposed / applied tags
+  (`proposal/* / approved/* / building/* / deployed/* /
+  failed/* / denied/*`) plus the swarm-wide meta repo's git
+  log on the dashboard. Read-only log + diff + raw-file view
+  is enough — something lighter than a full forge. The meta
+  log already answers "what's deployed where + when"; this
+  surfaces it without an ssh-to-host detour.
+
+- **xterm.js terminal** embedded per-agent, attached to a PTY exposed by
+  the harness. Pairs well with the unprivileged-container work — would let
+  the operator drop into the container without `nixos-container root-login`.
+
+## Telemetry
+
+- **Harness stats per agent in sqlite, charted on the agent page.**
+  bitburner-agent samples 18 series; for hyperhive the generally-applicable
+  ones are:
+  - turns/min, tool calls/turn, turn duration p50/p95
+  - claude exit code distribution (ok vs `--compact`-retry vs failure)
+  - inbox depth (current + max-over-window)
+  - messages sent/received per turn (split by recipient: peer / operator /
+    manager / system)
+  - approval queue length (across all agents — dashboard-level)
+  - per-tool usage counts (Read/Edit/Bash/send/recv/…)
+  - time-since-last-turn (helps spot stuck agents)
+  - notes file size growth (cues compaction)
+  Backend: a `stats` table with `(agent, ts, key, value)` written from
+  the harness on `TurnEnd`; `GET /api/stats?since=…` returns the
+  series; agent page renders with a small chart lib (uPlot is light).
+
+## Spawn flow
+
+- **Two-step spawn.** Today `request_spawn(name)` is one shot: manager
+  asks → operator approves → container is created with a default
+  `agent.nix` and empty `/state/`. Manager has no way to pre-stage
+  per-agent prompt material, package additions, or initial notes before
+  the agent first wakes. Split into:
+  1. `request_spawn_draft(name)` — host creates the per-agent
+     `proposed/` repo (initial commit) and `state/` dir with no
+     container; manager now has `/agents/<name>/{config,state}/` to
+     edit + commit just like an existing agent.
+  2. `request_spawn_commit(name, commit_ref)` — submits the queued
+     approval; operator sees the diff in the dashboard like a normal
+     `apply_commit`; on approve the container is created from that
+     commit.
+  Backwards-compat: keep the existing one-shot `request_spawn` for
+  trivial agents (operator can still type a name in the dashboard).
+  Surface "drafts" as a new section between K3PT ST4T3 and approvals.
+
+## Loop substance
+
+- **Notes compaction.** `/state/` is bind-mounted persistently and agents
+  are told (in the system prompt) to keep `/state/notes.md` for durable
+  knowledge — but we don't currently nudge them to compact when notes
+  grow. Bitburner-agent's pattern: a short-lived secondary claude session
+  that takes the existing notes + a "compact this" prompt and rewrites
+  them in place. Add when the notes start bloating.

hive-c0re/src/agent_server.rs

@@ -194,48 +194,26 @@ async fn dispatch(req: &AgentRequest, agent: &str, coord: &Arc<Coordinator>) ->
             file_path,
         } => {
             use hive_sh4re::ReminderTiming;
-
-            // Calculate the due_at timestamp, propagating errors instead of silently
-            // defaulting to epoch 1970 on overflow/conversion failure.
-            let due_at_result: Result<i64> = match timing {
+            let due_at = match timing {
                 ReminderTiming::InSeconds { seconds } => {
-                    let now = std::time::SystemTime::now();
-                    let future = match now.checked_add(std::time::Duration::from_secs(*seconds)) {
-                        Some(t) => t,
-                        None => return AgentResponse::Err {
-                            message: format!("InSeconds overflow: {seconds}s exceeds system time range"),
-                        },
-                    };
-                    let duration = match future.duration_since(std::time::UNIX_EPOCH) {
-                        Ok(d) => d,
-                        Err(e) => return AgentResponse::Err {
-                            message: format!("system time before UNIX_EPOCH: {e}"),
-                        },
-                    };
-                    match i64::try_from(duration.as_secs()) {
-                        Ok(ts) => Ok(ts),
-                        Err(e) => return AgentResponse::Err {
-                            message: format!("unix timestamp exceeds i64 range: {e}"),
-                        },
-                    }
+                    std::time::SystemTime::now()
+                        .checked_add(std::time::Duration::from_secs(*seconds))
+                        .and_then(|t| {
+                            t.duration_since(std::time::UNIX_EPOCH)
+                                .ok()
+                                .and_then(|d| i64::try_from(d.as_secs()).ok())
+                        })
+                        .unwrap_or(0)
                 }
-                ReminderTiming::At { unix_timestamp } => Ok(*unix_timestamp),
+                ReminderTiming::At { unix_timestamp } => *unix_timestamp,
             };
-
-            match due_at_result {
-                Ok(due_at) => {
-                    match broker.store_reminder(agent, message, file_path.as_deref(), due_at) {
-                        Ok(id) => {
-                            tracing::info!(%id, %agent, %due_at, "reminder scheduled");
-                            AgentResponse::Ok
-                        }
-                        Err(e) => AgentResponse::Err {
-                            message: format!("failed to store reminder: {e:#}"),
-                        },
-                    }
+            match broker.store_reminder(agent, message, file_path.as_deref(), due_at) {
+                Ok(id) => {
+                    tracing::info!(%id, %agent, %due_at, "reminder scheduled");
+                    AgentResponse::Ok
                 }
                 Err(e) => AgentResponse::Err {
-                    message: format!("invalid reminder timing: {e:#}"),
+                    message: format!("failed to store reminder: {e:#}"),
                 },
             }
         }

hive-c0re/src/broker.rs

@@ -253,25 +253,6 @@ impl Broker {
             .context("query reminders")
     }
 
-    /// Get all due reminders across all agents in a single query.
-    /// Returns a vec of (agent, id, message, file_path) tuples.
-    pub fn get_all_due_reminders(&self) -> Result<Vec<(String, i64, String, Option<String>)>> {
-        let conn = self.conn.lock().unwrap();
-        let mut stmt = conn.prepare(
-            "SELECT agent, id, message, file_path FROM reminders WHERE due_at <= ?1 AND sent_at IS NULL ORDER BY agent, due_at ASC"
-        )?;
-        let rows = stmt.query_map(params![now_unix()], |row| {
-            Ok((
-                row.get::<_, String>(0)?,
-                row.get::<_, i64>(1)?,
-                row.get::<_, String>(2)?,
-                row.get::<_, Option<String>>(3)?,
-            ))
-        })?;
-        rows.collect::<rusqlite::Result<Vec<_>>>()
-            .context("query all due reminders")
-    }
-
     /// Mark a reminder as sent (delivered).
     pub fn mark_reminder_sent(&self, id: i64) -> Result<()> {
         let conn = self.conn.lock().unwrap();

hive-c0re/src/main.rs

@@ -163,36 +163,41 @@ async fn main() -> Result<()> {
             tokio::spawn(async move {
                 use hive_sh4re::Message;
                 loop {
-                    // Query all due reminders in a single DB call
-                    match reminder_coord.broker.get_all_due_reminders() {
-                        Ok(reminders) => {
-                            for (agent, id, message, _file_path) in reminders {
-                                // Deliver as inbox message from "reminder"
-                                if let Err(e) = reminder_coord.broker.send(&Message {
-                                    from: "reminder".to_owned(),
-                                    to: agent.clone(),
-                                    body: message.clone(),
-                                }) {
-                                    tracing::warn!(
-                                        reminder_id = id,
-                                        %agent,
-                                        error = ?e,
-                                        "failed to deliver reminder"
-                                    );
-                                    continue;
-                                }
-                                // Mark as sent
-                                if let Err(e) = reminder_coord.broker.mark_reminder_sent(id) {
-                                    tracing::warn!(
-                                        reminder_id = id,
-                                        error = ?e,
-                                        "failed to mark reminder sent"
-                                    );
+                    // Get all agents currently registered
+                    let agents = reminder_coord.list_agents();
+                    for agent in agents {
+                        match reminder_coord.broker.get_due_reminders(&agent) {
+                            Ok(reminders) => {
+                                for (id, message, _file_path) in reminders {
+                                    // Deliver as inbox message from "reminder"
+                                    if let Err(e) = reminder_coord.broker.send(&Message {
+                                        from: "reminder".to_owned(),
+                                        to: agent.clone(),
+                                        body: message.clone(),
+                                    }) {
+                                        tracing::warn!(
+                                            reminder_id = id,
+                                            %agent,
+                                            error = ?e,
+                                            "failed to deliver reminder"
+                                        );
+                                        continue;
+                                    }
+                                    // Mark as sent
+                                    if let Err(e) = reminder_coord.broker.mark_reminder_sent(id) {
+                                        tracing::warn!(
+                                            reminder_id = id,
+                                            error = ?e,
+                                            "failed to mark reminder sent"
+                                        );
+                                    }
                                 }
                             }
-                        }
-                        Err(e) => {
-                            tracing::warn!(error = ?e, "failed to query due reminders");
+                            Err(e) => tracing::warn!(
+                                %agent,
+                                error = ?e,
+                                "failed to query due reminders"
+                            ),
                         }
                     }
                     tokio::time::sleep(std::time::Duration::from_secs(5)).await;