docs(persistence): fix stale facts — reminders/todos moved in-container, vacuum ownership, topology writer, matrix glob, qgroup quotas shipped

docs/persistence.md

@@ -40,20 +40,23 @@ schemas, file layouts, and internal migration mechanics.
 
 ### `/var/lib/hyperhive/db/broker.sqlite` (host)
 
-Seven tables, all in one file — four queues, the schedule
+Seven tables, all in one file — three queues, a small key/value
-header/targets split, and the per-agent power-intent registry:
+table, the schedule header/targets split, and the per-agent
+power-intent registry:
 
 - `messages` — every inter-agent / operator-bound message.
   `sender / recipient / body / sent_at / delivered_at / acked_at /
-  in_reply_to`. `in_reply_to` links a reply to its parent row id;
+  in_reply_to / priority`. `in_reply_to` links a reply to its parent
-  the dashboard and per-agent inbox render these as threaded rows.
+  row id; the dashboard and per-agent inbox render these as threaded
-- `reminders` — `mcp__hyperhive__remind` queue.
+  rows.
-  `agent / message / file_path / due_at / created_at / sent_at /
+- `kv` — small persistent key/value store (`key PK / value`) for
-  attempt_count / last_error`. `file_path` set when a body
+  host-side bookkeeping that doesn't warrant its own table.
-  exceeded the inline soft-cap and got auto-spilled to a file
+
-  under the agent's state dir; the worker delivers a short
+  ⚠️ The `mcp__hyperhive__remind` queue is **not** here any more: it
-  pointer instead. `attempt_count` / `last_error` accumulate
+  moved to a harness-local, per-agent store as part of the
-  on delivery-failed retries.
+  loose-ends-v2 migration — see [`/harness/` contents
+  below](#state-dirs-per-agent) for where reminders (and todos, and
+  the questions mirror) actually live now.
 - `approvals` — the queue. `agent / kind (merge_config_pr | spawn |
   init_config | update_meta_inputs | schedule_prompt) /
   commit_ref / requested_at / status / resolved_at / note`.
@@ -98,8 +101,6 @@ Retention:
 - Approvals and questions are kept indefinitely — both are
   audit trails. `actions::destroy` and answered questions stay
   visible to anything that queries by id.
-- Reminder rows are kept after `sent_at` is set (audit trail);
-  no automatic vacuum today.
 - Scheduled prompts: one-shot rows are deleted on fire by the
   worker; recurring rows live until the operator cancels them
   (`cancel_schedule` MCP / dashboard ✗) which tombstones via
@@ -117,20 +118,26 @@ One table:
 - `events(id, ts, kind, payload_json)` — every `LiveEvent` the
   harness emits during turn loop execution.
 
-The harness writes; the host vacuums. `hive-c0re::events_vacuum`
+The harness both writes and vacuums it — this used to be a host-side
-runs hourly and sweeps every existing agent harness dir. Retention
+sweep, but hive-c0re runs as the unprivileged `hive-core` user under
-is **type-scoped**: it deletes only the verbose `stream` rows (the
+privsep and can't delete agent-owned files (host-side deletes hit
-raw claude `stream-json` deltas — one per text chunk / tool use, the
+`PermissionDenied` on the bash-task trio and a readonly-database error
-bulk of the file's size) older than 14 days, and keeps every other
+here), so cleanup moved in-container. `hive-agent`'s `vacuum::run`
-kind (`turn_start`, `turn_end`, `note`, `status_changed`,
+(`hive-agent/src/vacuum.rs`) sweeps hourly. Retention is
-`model_changed`, `token_usage_changed`, `turn_state_changed`)
+**type-scoped**: it deletes only the verbose `stream` rows (the raw
-indefinitely — those are small and carry the semantic per-turn
+claude `stream-json` deltas — one per text chunk / tool use, the bulk
-history the operator scrolls back through when debugging a
+of the file's size) older than 14 days, and keeps every other kind
-regression. Age-only within the `stream` kind — no row cap — so a
+(`turn_start`, `turn_end`, `note`, `status_changed`, `model_changed`,
-chatty turn doesn't lose its stream history sooner than a quiet one.
+`token_usage_changed`, `turn_state_changed`) indefinitely — those are
-Centralising retention on the host means a misbehaving harness can't
+small and carry the semantic per-turn history the operator scrolls
-disable its own vacuum and agents don't need any cleanup wiring of
+back through when debugging a regression. Age-only within the
-their own.
+`stream` kind — no row cap — so a chatty turn doesn't lose its stream
+history sooner than a quiet one. The trade-off (accepted): a
+misbehaving harness could now skip its own cleanup, which the old
+host-side sweep was meant to prevent — but a compromised harness is
+already inside the container trust boundary
+([`docs/security.md`](security.md)), and these are ephemeral local
+artifacts, so cleaning them up where they live is the honest fix.
 
 Path overridable via `HYPERHIVE_EVENTS_DB` (for dev / no-`/harness`
 setups). On open failure the `Bus` falls back to no-store mode
@@ -211,11 +218,15 @@ the actual failure and leave only the host journal holding the
 complete output. With this table the dashboard can surface the entire
 log.
 
-Two indices:
+Three indices:
 - `(agent, started_at)` — backs the per-agent latest-N lookup used
   by the agent card chip.
 - `(status, finished_at)` — backs the retention sweep that runs
   as part of the existing hourly vacuum.
+- `(node_id)` — added by a later migration so a build log row can be
+  looked up by the job-queue node it belongs to (a `hive_jobq` node is
+  immutable after insert, so the link is recorded on the log row
+  instead); legacy rows predating the column keep `node_id IS NULL`.
 
 Writes are best-effort: `append_stdout` / `append_stderr` / `finish`
 log a warning on sqlite error and let the build continue. A failed
@@ -239,7 +250,7 @@ and inbox messages queue unacked until it's removed (see
 
 Unusually, it's read and written from **both** sides of the harness
 bind-mount, and that's the whole design: the harness stats it
-in-container via `hive_agent::paths::paused_marker`, while hive-c0re
+in-container via `hive-agent`'s `paths::paused_marker`, while hive-c0re
 stats it on the host (`Coordinator::is_paused`) to populate the
 `paused` field on the agent card, and creates/removes it
 (`Coordinator::set_paused`) for `hivectl agent <name> pause|resume` and the
@@ -284,15 +295,26 @@ Under `/var/lib/hyperhive/agents/<name>/`:
   - `bash-tasks/` — task JSON + stdout/stderr files for
     background `mcp__bash__run` jobs. JSON files are
     `<id>.json` (status + tails), `<id>.out` / `<id>.err`
-    (full captured output). `hive-c0re::bash_tasks_vacuum` runs
+    (full captured output). The harness's own hourly sweep
-    hourly and deletes terminal task trios older than 48 hours;
+    (`hive-agent`'s `vacuum::run`, same one that ages out `stream`
-    non-terminal (still-running) tasks are never deleted by vacuum.
+    event rows above) deletes terminal task trios older than 48
-  - `hyperhive-todos.sqlite` — loose-ends-v2 todo store. In-container
+    hours; non-terminal (still-running) tasks are never deleted. This
-    daemons (`hive-bash-daemon`, `hive-matrix-daemon`, `hive-forge-notify`)
+    used to be a host-side `hive-c0re` vacuum, moved in-container for
-    upsert keyed todos here over the harness's in-agent socket
+    the same privsep-ownership reason as the events vacuum above.
-    (`HIVE_AGENT_SOCKET`); the harness merges them into `get_loose_ends`
+  - `hyperhive-state.sqlite` — consolidated loose-ends-v2 store: todos,
-    output and clears a row on `mark_todo_done`. Replaced the old
+    reminders, and a questions mirror, one small table each in a single
-    file-based `mcp-loose-ends/` scanner.
+    file (in-container daemons — `hive-bash-daemon`, `hive-matrix-daemon`,
+    `hive-forge-notify` — upsert keyed todos here over the harness's
+    in-agent socket, `HIVE_AGENT_SOCKET`; the harness merges them into
+    `get_loose_ends` output and clears a row on `mark_todo_done`).
+    Replaces three formerly-separate files
+    (`hyperhive-todos.sqlite`, `hyperhive-reminders.sqlite`, and the
+    old file-based `mcp-loose-ends/` scanner before that) — a one-time
+    boot migration (`db_migrate::run`) folds the legacy files into this
+    path the first time a harness boots after the upgrade. Also backs
+    the `mcp__hyperhive__remind` queue, which moved from a host-side
+    `broker.sqlite` table to this per-agent store as part of the same
+    migration.
 
     The harness itself is also a producer, not just the socket server:
     boot wiring's `spawn_todo_socket` starts `todo_server::run` (the
@@ -358,8 +380,11 @@ Contents:
   audit trail (one commit per successful deploy or hyperhive bump).
 - `topology.json` — parent/child agent graph
   (`{ "alice": "root", "bob": "alice", "root": null }`).
-  Written by `topology::set_parent`; read by the dashboard, the
+  Written by `topology::apply_set_parent` (the pure move-validating
-  renderer, and `<parent>` / `<children>` recipient resolution.
+  transform) via `meta::bulk_commit_topology` (the committer — see the
+  `Reparent` node in [`docs/coordinator.md`](coordinator.md)); read by
+  the dashboard, the renderer, and `<parent>` / `<children>` recipient
+  resolution.
 - `tool-groups.json` — per-agent MCP tool group grants
   (`{ "alice": ["messaging", "inbox", "execution"] }`). Written by
   `tool_groups::set_groups`; injected as `HIVE_TOOL_GROUPS` env
@@ -431,8 +456,12 @@ until an explicit opt-in upgrade.
   actually a subvolume, then the normal `remove_dir_all` sweep covers
   plain-dir agents + the applied dir.
 
-Per-subvolume disk-usage accounting and optional quotas are a
+Per-subvolume disk-usage accounting and optional quotas have since
-follow-up (the qgroup work), not part of the base migration.
+landed as the qgroup work: `hivectl quota-enable` turns on btrfs
+qgroup accounting hive-wide (opt-in, no-op on non-btrfs hosts), and
+`hivectl agent <name> quota show|set` reads/limits one agent's
+subvolume usage through the same hive-priv-mediated path as
+subvolume creation/deletion above.
 
 This is the same subvolume `hivectl agent <name> subvol snapshot push`
 sends to the swarm's snapshot store — see
@@ -506,10 +535,12 @@ auto-injected `extraMcpServers.matrix` entry read).
 **First-boot ordering**: hive-c0re provisions the matrix token AFTER
 agent containers come up. Without the path-trigger sibling
 (`systemd.paths.hive-matrix-daemon`, `PathExistsGlob =
-/agents/*/state/matrix-token`), the daemon would exit 0 quietly the
+/agents/*/state/matrix-token*` — the trailing `*` also catches a
-first time it ran and the MCP would have no backend until the next
+secondary multi-account token like `matrix-token-ccc`), the daemon
-restart. The `.path` unit makes the appearance of the token re-fire
+would exit 0 quietly the first time it ran and the MCP would have no
-the service so the daemon comes alive in the same boot cycle as
+backend until the next restart. The `.path` unit makes the appearance
+of the token re-fire the service so the daemon comes alive in the
+same boot cycle as
 provisioning. The same token watcher also drives avatar setting: on a
 restart the daemon re-runs each account's bring-up, which sets the
 avatar (see below).