Commit graph hyperhive/docs/persistence.md
Author SHA1 Message Date
damocles
3188e50ab8 docs(#2628): update bash tool descriptions, docs, and system prompt for the todo model (trim impl details for agent) 2026-07-22 17:34:12 +02:00
damocles
cc67a05974 refactor(#2352): extract standalone hivectl crate, hive-c0re daemon-only 2026-07-15 22:36:13 +02:00
damocles
c2bd7db998 refactor(#2416): remove the non-pr config-change flow (request_apply_commit / applycommit) 2026-07-15 21:03:52 +02:00
müde
4a48ce5024 refactor: nix/host-modules + nix/agent-modules layout, update doc paths 2026-07-13 22:05:49 +02:00
müde
0e4b5a1120 refactor(hive-c0re): group src-root files into submodules
stores/ (sqlite-backed host stores + db helper), stats/, agent_config/,
workers/ — pure git-mv moves; crate-root re-exports keep every
crate::<module> path compiling. flake_check stays at root (synchronous
approval-flow validation, not a background worker)
2026-07-06 22:38:47 +02:00
müde
604e1c2557 docs: job-DAG queue model; fold agent_power table into broker.sqlite
coordinator.md rewrites the queue section (node inventory, DAG shapes,
resources, desired-state reconciliation, boot reconcile); approvals.md
+ persistence.md + hivectl --graceful help updated to match. agent_power
lives in broker.sqlite like approvals/questions (own connection + busy
timeout) instead of a separate db file.
2026-07-06 20:36:57 +02:00
damocles
f6d9ca7f99 fold forge_cursor into hyperhive-harness.json + prose comments (mara/argus review) 2026-07-01 18:52:56 +02:00
damocles
c99fa714d9 feat(#2050): set matrix avatar in the daemon over the live client 2026-06-28 01:38:21 +02:00
damocles
c29dfed9f4 docs(#2050): note per-account matrix avatar sync 2026-06-28 01:38:21 +02:00
iris
20e26ad28d docs(persistence): correct the root-container destroy claim (imperative, not declarative)
The 'non-destroyable (declarative; would fight with host NixOS config)'
claim was stale: the root/bootstrap container is imperative
infrastructure, managed end-to-end by hive-c0re (auto_update::
ensure_root_agent recreates it on startup if absent). The actions::
destroy bail is a soft policy guard, and destroying the container is
transient anyway — it's recreated on the next hive-c0re startup.
2026-06-23 14:48:10 +02:00
iris
7e0190e27c docs: address mara review on #1936
- Drop the redundant 'including the root/bootstrap container' /
  'sub-agents + root' enumerations in the port-range + state-dir
  uniformity statements — 'every agent' / 'all agents' is enough (and
  calling out '+ root' reintroduces the special-case framing).
- Stop describing the root agent's privileged (manager-flavour) socket
  as a current thing (it's being removed): drop the conventions.md
  capability clause, the network.md per-agent-privileged-socket list
  entry, and the persistence.md /run/hyperhive/manager/mcp.sock
  runtime-dir bullet.
2026-06-23 14:48:10 +02:00
iris
adb76f3cdf docs(persistence): reframe stale 'the manager' as the root agent
Config-repo editor, meta RO-mount, non-destroyable container, and the
topology.json example now refer to the root agent / bootstrap container
rather than 'the manager'. The /run/hyperhive/manager/mcp.sock path is a
real runtime path, kept (clarified as the root agent's manager-flavour
socket).
2026-06-23 14:48:10 +02:00
atlas
1f602d5fda hive-c0re: back agent state dirs with btrfs subvolumes
Progressive enhancement: a brand-new agent's state root under
/var/lib/hyperhive/agents is created as a btrfs subvolume when the host
filesystem is btrfs, otherwise it falls back to a plain directory. No
existing agent is auto-migrated — the new path only fires when the root
does not yet exist, so plain-dir agents are left untouched until an
explicit opt-in upgrade.

Two new privileged ops (subvolume create/delete are root-only):
EnsureAgentSubvolume statfs-gates on btrfs, creates the subvolume, and
chowns it to the hive-core user so the normal state/claude/harness
mkdirs succeed inside it; DeleteAgentSubvolume btrfs-subvolume-deletes
the root iff it is actually a subvolume. hive-c0re calls Ensure before
the per-agent dirs are created (spawn/rebuild/InitConfig) and Delete on
the purge path only — destroy keeps the subvolume for revival, matching
plain-dir semantics. btrfs-progs added to the hive-priv unit PATH.

Per-subvolume usage accounting + optional quota is a separate
follow-up.
2026-06-19 13:46:39 +02:00
damocles
fb1f7efbe4 docs: move privsep socket-activation + child-state rw rationale out of code comments 2026-06-08 21:58:12 +02:00
damocles
6e39515669 feat: type-scope events vacuum to prune only stream rows (14d) + drop turn-stats vacuum 2026-06-06 07:57:27 +02:00
damocles
1569d55f78 feat: capture normalised bash command heads for the favorite-tools stat 2026-06-06 00:52:56 +02:00
damocles
14c7b0d406 feat: group host-side /var/lib/hyperhive state into db/ forge/ matrix/ run/ subdirs with startup migration 2026-06-05 23:01:47 +02:00
iris
a650187d23 docs: fix bash-tasks retention + matrix provisioning daemon restart
persistence.md: bash-tasks/ says "persist until container purge" but
bash_tasks_vacuum now runs hourly and deletes terminal task trios older
than 48 hours. Update the description to reflect actual retention policy.

matrix.md: add step 6 to the Provisioning flow section documenting that
hive-c0re restarts hive-matrix-daemon immediately after writing the
access_token (fix for issue #1329). Previously the doc stopped at token
write; the restart + .path-trigger fallback interplay was undocumented.
2026-06-05 15:45:14 +02:00
iris
0d84e11dcd docs(persistence): document hyperhive-status in state/ and mcp-loose-ends/ in harness/ 2026-06-05 15:32:22 +02:00
damocles
f5351eb59c fix(#1194): update Execution tool names + frontend + docs 2026-06-03 21:20:49 +02:00
iris
30c8459add docs(persistence): expand meta/ dir section to list config JSON files
The meta/ section only described the flake. Added entries for
topology.json, tool-groups.json, and capabilities.json — the three
system-level config files also committed there — with their writers,
readers, and injected env vars.
2026-06-02 12:43:59 +02:00
damocles
5e13fcbe94 docs: turn-stats vacuum is already implemented — update stale note 2026-06-01 15:39:43 +02:00
damocles
5265af9589 docs: drop false bash-tasks cleanup claim (argus nit) 2026-06-01 14:39:16 +02:00
damocles
494a326029 docs: fix persistence.md paths for harness-state split 2026-06-01 14:39:16 +02:00
damocles
73bf7bef95 docs: document harness/ dir layout including bash-tasks 2026-06-01 14:39:16 +02:00
damocles
1b8a6be8ce docs: add build_logs.sqlite section to persistence.md, trim build_logs.rs module doc 2026-06-01 11:38:42 +02:00
damocles
fce1f49f6a refactor(#838): consolidate harness state files into hyperhive-harness.json 2026-05-31 20:32:13 +02:00
atlas
a043c61828 docs: extract best-effort oneshot service contract + matrix-avatar (#718 batch 6)
Two new sections housing rationale that was inline in
`harness-base.nix`:

- `docs/conventions.md::Best-effort oneshot services` — shape
  contract shared by `tea-login`, `forge-avatar-sync`, and
  `matrix-avatar-sync`: always exit 0, no `set -e`, skip silently
  on missing prerequisites, wired to multi-user.target,
  re-runnable. Plus the artefact-under-agent-home + service-stays-
  root posture and the operator-visible-via-journalctl trade-off.

- `docs/persistence.md::matrix-avatar-sync` — the two-step `media
  upload` → `set avatar_url` matrix-spec dance + why
  `RemainAfterExit = false` (so the `.path` watcher's re-fire on
  token appearance actually re-executes the unit).

In-code comments trim to one-liner purpose + pointers; the script
bodies stay (they're the actual implementation) but their inline
`#` rationale comments collapse where the docs cover them.

`description = ''…''` blocks (operator-facing options docs)
preserved per iris #718.

`nix flake check` clean; `nix fmt` clean.
2026-05-31 16:14:38 +02:00
iris
bd6b48a883 docs/persistence.md: scrub self-ref cookies (#719 batch 4) 2026-05-31 15:49:32 +02:00
atlas
309879dba0 docs: extract 3 substantive harness-base.nix prose blocks (#718, first pass)
iris's #718 scope: move substantive design context from `#` comment
blocks in `nix/` to corresponding `docs/` files, leave short
references in code. iris handed it back to me on #10114 since
nix/ is my lane + #775 established the pattern.

First pass — three highest-density blocks in harness-base.nix:

1. **First-boot agent-user migration** (~70 lines → `~20 lines code +
   short ref` in the activation script). Substantive prose moves to
   new `docs/persistence.md::First-boot agent-user migration (post-#658)`
   section explaining the 4 steps the script performs + the eventual
   removability of the marker-guarded body.

2. **nix-daemon `sandbox-fallback = true`** (10-line block → 5-line
   ref). New `docs/gotchas.md::Containerized nix-daemon needs
   sandbox-fallback = true` section covers the user-namespaces
   rationale + nixpkgs-default override.

3. **Matrix daemon + token-arrival trigger** (~50 lines across two
   systemd units → ~10 lines code + short refs). New
   `docs/persistence.md::Matrix per-agent daemon + token-arrival
   trigger` covers the socket-path rationale, the runtime-dir
   ownership story, and the first-boot ordering pattern.

Net: harness-base.nix -84 lines, docs +74 lines. Substantive design
context moves to durable docs; in-code refs follow iris's pattern
from her #712 batches (`see docs/<file>::<section>`).

Follow-ups: hive-c0re.nix, hive-forge.nix, hive-matrix.nix (already
trimmed via #775 but a couple of remaining blocks could go), and
the smaller files in #718's scope table. Shipping this first to get
the pattern reviewed before larger batches.

Verified: `nix eval` on agent-base toplevel still resolves.
2026-05-31 14:44:12 +02:00
damocles
fcb5fae345 docs: catch up post-#658/#604/#660 + add operator hivectl section (#124) 2026-05-31 10:13:33 +02:00
damocles
c786b9ec37 docs(persistence): broker.sqlite is six tables now (#124 follow-up) 2026-05-27 23:07:15 +02:00
iris
69604407a9 docs: update for recent commits (rate limiting, reply threading, screen, auto-reset, two-step spawn, ctx chip) 2026-05-20 16:52:18 +02:00
damocles
4715e88fff docs: move backlog to forge issue tracker, extract boundary doc 2026-05-20 12:19:16 +02:00
damocles
1529c2d777 lifecycle: bind each sub-agent's config repo read-only at /agents/<name>/config 2026-05-20 10:05:02 +02:00
müde
d890509be3 docs: turn_stats sink + event-driven agent badges + dashboard event vocabulary 2026-05-17 23:28:34 +02:00
damocles
82b0877c47 ask: rename ask_operator → ask + optional 'to' for agent-to-agent Q&A 2026-05-17 13:20:32 +02:00
müde
14aa7c7acc final docs + cleanup sync for meta-flake era
claude.md flips 'in flight' → 'just landed' for the meta
overhaul + extends the file map with meta.rs and migrate.rs.
docs/approvals.md replaces the in-flight callout with a
proper 'Meta flake' section (two-phase deploy walkthrough,
sync_agents semantics, single-phase variants), updates the
two-repo box diagram to include the /var/lib/hyperhive/meta/
tree and tracks flake.nix in applied, rewrites the
container --flake reference to meta#<name>, replaces the
'Manager view of applied' section with a unified
'/agents + /applied + /meta' inventory listing every useful
git incantation, and explains the in-place no-state-loss
migration that now runs on hive-c0re startup.
docs/persistence.md grows entries for the meta repo + the
.meta-migration-done marker. readme box diagram picks up the
/meta RO bind; approval-flow paragraph rewritten end to end
to describe the meta lock dance.

lifecycle::flake_base deleted — the meta render hardcodes
the manager vs agent-base choice as nix expression.
2026-05-16 00:40:06 +02:00
müde
75e7faff0c docs: full sync ahead of compaction + config-management overhaul
readme: manager mcp surface picks up update; operator-surface
recap mentions /model + last-turn + model chip + the three
collapsibles (inbox / journald / agent.nix).

web-ui.md: details-restore-key story under shape; port-conflict
banner mention on containers; agent.nix viewer alongside journald;
notifications use per-event tags + console.debug log on
block/show; deny endpoint takes note=<reason>; data-prompt /
data-prompt-field generalisation noted.

conventions.md: data-prompt and snapshot/restoreOpenDetails added
to the async-forms section.

persistence.md: operator_questions row picks up deadline_at (ttl)
column with a migration note.

todo.md: new 'Bugs' section captures the manager-question
not-rendering issue with three suspect paths to chase.

claude.md scratchpad rewritten as a clean handoff for the
compaction + the upcoming config-git overhaul. flags the
two-repo (proposed/ + applied/) split as the thing to
reconsider.
2026-05-15 22:12:40 +02:00
müde
62d1a74929 docs sync + revert auto-unfree removal
revert the earlier 'operator must set allowUnfree' move:
per-agent containers evaluate their own nixpkgs and the operator's
host-level allowUnfree doesn't propagate in. restoring the scoped
allowUnfreePredicate inside both the claude-unstable overlay and
harness-base.nix; documented in README + gotchas as 'nothing to
set on the operator side'.

docs:
- claude.md file map adds crash_watch.rs, kick_agent on coordinator,
  /api/model + journald viewer + bind-with-retry references.
- scratchpad rewritten to reflect the recent run.
- web-ui.md: notification row + browser notifications section,
  state row (badge + model chip + last-turn chip + cancel button),
  per-agent inbox, /model slash, /cancel-question + journald
  endpoints, focus-preservation on refresh.
- turn-loop.md: --model is read from Bus::model() per turn (runtime
  override via /model); recv(wait_seconds) up to 180s with the
  rationale; ask_operator gains ttl_seconds; new TurnState section;
  kick_agent inbox-on-startup hint.
- approvals.md: ttl/cancel resolution paths for operator questions.
- persistence.md: /state/hyperhive-model file.
- gotchas.md: web UI port collision policy (rename, don't probe);
  bind retry + SO_REUSEADDR shape; auto-unfree restored.
- todo.md: cleaned up empty sections and stale entries; /model
  shipped, dropped from the list.
2026-05-15 21:26:13 +02:00
müde
8b10731aa4 split claude.md into docs/ — per-topic, human-readable
claude.md was eating 400 lines of subsystem detail that's useful
when you're working on that subsystem and noise the rest of the
time. split into:

- docs/conventions.md   naming, identity, async forms, commit style
- docs/gotchas.md       nspawn / nixos-container quirks
- docs/web-ui.md        dashboard + per-agent layouts and endpoints
- docs/turn-loop.md     claude invocation, wake prompt, mcp surface
- docs/approvals.md     approval flow, manager policy, helper events
- docs/persistence.md   sqlite dbs, retention, state dir layout

claude.md is now the entry point — file map, reading paths
("pick the doc that matches your task"), quick reminders that
fit on one screen, and a small scratchpad section for in-flight
context. references the docs; the docs don't reference claude.md.

no content was lost — the docs/ files cover everything the old
claude.md did, plus things i wrote up better while extracting.
2026-05-15 20:17:11 +02:00