docs(persistence): fix stale facts — reminders/todos moved in-container, vacuum ownership, topology writer, matrix glob, qgroup quotas shipped
This commit is contained in:
iris
mara
parent
90be37c0c8
commit
a17cc3612b
1 changed files with 77 additions and 46 deletions
|
|
@ -40,20 +40,23 @@ schemas, file layouts, and internal migration mechanics.
|
|||
|
||||
### `/var/lib/hyperhive/db/broker.sqlite` (host)
|
||||
|
||||
Seven tables, all in one file — four queues, the schedule
|
||||
header/targets split, and the per-agent power-intent registry:
|
||||
Seven tables, all in one file — three queues, a small key/value
|
||||
table, the schedule header/targets split, and the per-agent
|
||||
power-intent registry:
|
||||
|
||||
- `messages` — every inter-agent / operator-bound message.
|
||||
`sender / recipient / body / sent_at / delivered_at / acked_at /
|
||||
in_reply_to`. `in_reply_to` links a reply to its parent row id;
|
||||
the dashboard and per-agent inbox render these as threaded rows.
|
||||
- `reminders` — `mcp__hyperhive__remind` queue.
|
||||
`agent / message / file_path / due_at / created_at / sent_at /
|
||||
attempt_count / last_error`. `file_path` set when a body
|
||||
exceeded the inline soft-cap and got auto-spilled to a file
|
||||
under the agent's state dir; the worker delivers a short
|
||||
pointer instead. `attempt_count` / `last_error` accumulate
|
||||
on delivery-failed retries.
|
||||
in_reply_to / priority`. `in_reply_to` links a reply to its parent
|
||||
row id; the dashboard and per-agent inbox render these as threaded
|
||||
rows.
|
||||
- `kv` — small persistent key/value store (`key PK / value`) for
|
||||
host-side bookkeeping that doesn't warrant its own table.
|
||||
|
||||
⚠️ The `mcp__hyperhive__remind` queue is **not** here any more: it
|
||||
moved to a harness-local, per-agent store as part of the
|
||||
loose-ends-v2 migration — see [`/harness/` contents
|
||||
below](#state-dirs-per-agent) for where reminders (and todos, and
|
||||
the questions mirror) actually live now.
|
||||
- `approvals` — the queue. `agent / kind (merge_config_pr | spawn |
|
||||
init_config | update_meta_inputs | schedule_prompt) /
|
||||
commit_ref / requested_at / status / resolved_at / note`.
|
||||
|
|
@ -98,8 +101,6 @@ Retention:
|
|||
- Approvals and questions are kept indefinitely — both are
|
||||
audit trails. `actions::destroy` and answered questions stay
|
||||
visible to anything that queries by id.
|
||||
- Reminder rows are kept after `sent_at` is set (audit trail);
|
||||
no automatic vacuum today.
|
||||
- Scheduled prompts: one-shot rows are deleted on fire by the
|
||||
worker; recurring rows live until the operator cancels them
|
||||
(`cancel_schedule` MCP / dashboard ✗) which tombstones via
|
||||
|
|
@ -117,20 +118,26 @@ One table:
|
|||
- `events(id, ts, kind, payload_json)` — every `LiveEvent` the
|
||||
harness emits during turn loop execution.
|
||||
|
||||
The harness writes; the host vacuums. `hive-c0re::events_vacuum`
|
||||
runs hourly and sweeps every existing agent harness dir. Retention
|
||||
is **type-scoped**: it deletes only the verbose `stream` rows (the
|
||||
raw claude `stream-json` deltas — one per text chunk / tool use, the
|
||||
bulk of the file's size) older than 14 days, and keeps every other
|
||||
kind (`turn_start`, `turn_end`, `note`, `status_changed`,
|
||||
`model_changed`, `token_usage_changed`, `turn_state_changed`)
|
||||
indefinitely — those are small and carry the semantic per-turn
|
||||
history the operator scrolls back through when debugging a
|
||||
regression. Age-only within the `stream` kind — no row cap — so a
|
||||
chatty turn doesn't lose its stream history sooner than a quiet one.
|
||||
Centralising retention on the host means a misbehaving harness can't
|
||||
disable its own vacuum and agents don't need any cleanup wiring of
|
||||
their own.
|
||||
The harness both writes and vacuums it — this used to be a host-side
|
||||
sweep, but hive-c0re runs as the unprivileged `hive-core` user under
|
||||
privsep and can't delete agent-owned files (host-side deletes hit
|
||||
`PermissionDenied` on the bash-task trio and a readonly-database error
|
||||
here), so cleanup moved in-container. `hive-agent`'s `vacuum::run`
|
||||
(`hive-agent/src/vacuum.rs`) sweeps hourly. Retention is
|
||||
**type-scoped**: it deletes only the verbose `stream` rows (the raw
|
||||
claude `stream-json` deltas — one per text chunk / tool use, the bulk
|
||||
of the file's size) older than 14 days, and keeps every other kind
|
||||
(`turn_start`, `turn_end`, `note`, `status_changed`, `model_changed`,
|
||||
`token_usage_changed`, `turn_state_changed`) indefinitely — those are
|
||||
small and carry the semantic per-turn history the operator scrolls
|
||||
back through when debugging a regression. Age-only within the
|
||||
`stream` kind — no row cap — so a chatty turn doesn't lose its stream
|
||||
history sooner than a quiet one. The trade-off (accepted): a
|
||||
misbehaving harness could now skip its own cleanup, which the old
|
||||
host-side sweep was meant to prevent — but a compromised harness is
|
||||
already inside the container trust boundary
|
||||
([`docs/security.md`](security.md)), and these are ephemeral local
|
||||
artifacts, so cleaning them up where they live is the honest fix.
|
||||
|
||||
Path overridable via `HYPERHIVE_EVENTS_DB` (for dev / no-`/harness`
|
||||
setups). On open failure the `Bus` falls back to no-store mode
|
||||
|
|
@ -211,11 +218,15 @@ the actual failure and leave only the host journal holding the
|
|||
complete output. With this table the dashboard can surface the entire
|
||||
log.
|
||||
|
||||
Two indices:
|
||||
Three indices:
|
||||
- `(agent, started_at)` — backs the per-agent latest-N lookup used
|
||||
by the agent card chip.
|
||||
- `(status, finished_at)` — backs the retention sweep that runs
|
||||
as part of the existing hourly vacuum.
|
||||
- `(node_id)` — added by a later migration so a build log row can be
|
||||
looked up by the job-queue node it belongs to (a `hive_jobq` node is
|
||||
immutable after insert, so the link is recorded on the log row
|
||||
instead); legacy rows predating the column keep `node_id IS NULL`.
|
||||
|
||||
Writes are best-effort: `append_stdout` / `append_stderr` / `finish`
|
||||
log a warning on sqlite error and let the build continue. A failed
|
||||
|
|
@ -239,7 +250,7 @@ and inbox messages queue unacked until it's removed (see
|
|||
|
||||
Unusually, it's read and written from **both** sides of the harness
|
||||
bind-mount, and that's the whole design: the harness stats it
|
||||
in-container via `hive_agent::paths::paused_marker`, while hive-c0re
|
||||
in-container via `hive-agent`'s `paths::paused_marker`, while hive-c0re
|
||||
stats it on the host (`Coordinator::is_paused`) to populate the
|
||||
`paused` field on the agent card, and creates/removes it
|
||||
(`Coordinator::set_paused`) for `hivectl agent <name> pause|resume` and the
|
||||
|
|
@ -284,15 +295,26 @@ Under `/var/lib/hyperhive/agents/<name>/`:
|
|||
- `bash-tasks/` — task JSON + stdout/stderr files for
|
||||
background `mcp__bash__run` jobs. JSON files are
|
||||
`<id>.json` (status + tails), `<id>.out` / `<id>.err`
|
||||
(full captured output). `hive-c0re::bash_tasks_vacuum` runs
|
||||
hourly and deletes terminal task trios older than 48 hours;
|
||||
non-terminal (still-running) tasks are never deleted by vacuum.
|
||||
- `hyperhive-todos.sqlite` — loose-ends-v2 todo store. In-container
|
||||
daemons (`hive-bash-daemon`, `hive-matrix-daemon`, `hive-forge-notify`)
|
||||
upsert keyed todos here over the harness's in-agent socket
|
||||
(`HIVE_AGENT_SOCKET`); the harness merges them into `get_loose_ends`
|
||||
output and clears a row on `mark_todo_done`. Replaced the old
|
||||
file-based `mcp-loose-ends/` scanner.
|
||||
(full captured output). The harness's own hourly sweep
|
||||
(`hive-agent`'s `vacuum::run`, same one that ages out `stream`
|
||||
event rows above) deletes terminal task trios older than 48
|
||||
hours; non-terminal (still-running) tasks are never deleted. This
|
||||
used to be a host-side `hive-c0re` vacuum, moved in-container for
|
||||
the same privsep-ownership reason as the events vacuum above.
|
||||
- `hyperhive-state.sqlite` — consolidated loose-ends-v2 store: todos,
|
||||
reminders, and a questions mirror, one small table each in a single
|
||||
file (in-container daemons — `hive-bash-daemon`, `hive-matrix-daemon`,
|
||||
`hive-forge-notify` — upsert keyed todos here over the harness's
|
||||
in-agent socket, `HIVE_AGENT_SOCKET`; the harness merges them into
|
||||
`get_loose_ends` output and clears a row on `mark_todo_done`).
|
||||
Replaces three formerly-separate files
|
||||
(`hyperhive-todos.sqlite`, `hyperhive-reminders.sqlite`, and the
|
||||
old file-based `mcp-loose-ends/` scanner before that) — a one-time
|
||||
boot migration (`db_migrate::run`) folds the legacy files into this
|
||||
path the first time a harness boots after the upgrade. Also backs
|
||||
the `mcp__hyperhive__remind` queue, which moved from a host-side
|
||||
`broker.sqlite` table to this per-agent store as part of the same
|
||||
migration.
|
||||
|
||||
The harness itself is also a producer, not just the socket server:
|
||||
boot wiring's `spawn_todo_socket` starts `todo_server::run` (the
|
||||
|
|
@ -358,8 +380,11 @@ Contents:
|
|||
audit trail (one commit per successful deploy or hyperhive bump).
|
||||
- `topology.json` — parent/child agent graph
|
||||
(`{ "alice": "root", "bob": "alice", "root": null }`).
|
||||
Written by `topology::set_parent`; read by the dashboard, the
|
||||
renderer, and `<parent>` / `<children>` recipient resolution.
|
||||
Written by `topology::apply_set_parent` (the pure move-validating
|
||||
transform) via `meta::bulk_commit_topology` (the committer — see the
|
||||
`Reparent` node in [`docs/coordinator.md`](coordinator.md)); read by
|
||||
the dashboard, the renderer, and `<parent>` / `<children>` recipient
|
||||
resolution.
|
||||
- `tool-groups.json` — per-agent MCP tool group grants
|
||||
(`{ "alice": ["messaging", "inbox", "execution"] }`). Written by
|
||||
`tool_groups::set_groups`; injected as `HIVE_TOOL_GROUPS` env
|
||||
|
|
@ -431,8 +456,12 @@ until an explicit opt-in upgrade.
|
|||
actually a subvolume, then the normal `remove_dir_all` sweep covers
|
||||
plain-dir agents + the applied dir.
|
||||
|
||||
Per-subvolume disk-usage accounting and optional quotas are a
|
||||
follow-up (the qgroup work), not part of the base migration.
|
||||
Per-subvolume disk-usage accounting and optional quotas have since
|
||||
landed as the qgroup work: `hivectl quota-enable` turns on btrfs
|
||||
qgroup accounting hive-wide (opt-in, no-op on non-btrfs hosts), and
|
||||
`hivectl agent <name> quota show|set` reads/limits one agent's
|
||||
subvolume usage through the same hive-priv-mediated path as
|
||||
subvolume creation/deletion above.
|
||||
|
||||
This is the same subvolume `hivectl agent <name> subvol snapshot push`
|
||||
sends to the swarm's snapshot store — see
|
||||
|
|
@ -506,10 +535,12 @@ auto-injected `extraMcpServers.matrix` entry read).
|
|||
**First-boot ordering**: hive-c0re provisions the matrix token AFTER
|
||||
agent containers come up. Without the path-trigger sibling
|
||||
(`systemd.paths.hive-matrix-daemon`, `PathExistsGlob =
|
||||
/agents/*/state/matrix-token`), the daemon would exit 0 quietly the
|
||||
first time it ran and the MCP would have no backend until the next
|
||||
restart. The `.path` unit makes the appearance of the token re-fire
|
||||
the service so the daemon comes alive in the same boot cycle as
|
||||
/agents/*/state/matrix-token*` — the trailing `*` also catches a
|
||||
secondary multi-account token like `matrix-token-ccc`), the daemon
|
||||
would exit 0 quietly the first time it ran and the MCP would have no
|
||||
backend until the next restart. The `.path` unit makes the appearance
|
||||
of the token re-fire the service so the daemon comes alive in the
|
||||
same boot cycle as
|
||||
provisioning. The same token watcher also drives avatar setting: on a
|
||||
restart the daemon re-runs each account's bring-up, which sets the
|
||||
avatar (see below).
|
||||
|
|
|
|||
Loading…
Reference in a new issue