From 7529c67f6bb26a4b29e93534bf730831276c886d Mon Sep 17 00:00:00 2001 From: damocles Date: Sat, 6 Jun 2026 08:56:02 +0200 Subject: [PATCH] docs: slim CLAUDE.md to a lean repo index (drop the drifting per-file map) --- CLAUDE.md | 433 ++++++------------------------------------------------ 1 file changed, 43 insertions(+), 390 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 014ed05c..04fe6dc2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -13,399 +13,54 @@ when you need depth on a subsystem. This file is the index. - Credential isolation + sandbox threat model: **[docs/security.md](docs/security.md)**. -## File map +## Repo map -``` -hive-c0re/ host daemon + sibling operator CLI (lib + 2 bins) - src/lib.rs `pub mod` re-exports for every module; shared by - both binaries so they import from the same - tree (no per-bin module duplication). - src/main.rs hive-c0re binary: clap setup; serve / spawn / - kill / rebuild / list / pending / approve / - deny / destroy [--purge] / request-spawn / - set-parent (--parent / --root); periodic - vacuum tasks. - src/bin/hivectl.rs `hivectl` binary: operator-facing host CLI - for ad-hoc administration that doesn't go - through the broker. Verbs: `forge - create-user ` / `matrix create-user - ` — manual entry points to the same - idempotent provisioning flows c0re runs at - boot; `gateway create-user|delete-user| - list-users` — BCrypt htpasswd management for - gateway HTTP Basic auth; `agents list|kill| - restart|restart-all` — container management - without the broker; `choom [--fresh]` - — exec into `machinectl shell h-` - running `claude --continue` (or fresh) so - the operator can drop into the agent's live - Claude session interactively. - src/server.rs host admin socket (HostRequest → dispatch) - src/client.rs admin-socket client - src/manager_server.rs manager-privileged socket (ManagerRequest) - src/agent_ports.rs writes `/var/lib/hyperhive/agent-ports.json` on - meta sync; gateway reads name→port map for - per-agent `/agent//` routing. - src/agent_sockets.rs writes `/var/lib/hyperhive/agent-sockets.json` on - meta sync; gateway reads name→socket-path - map for unix-domain per-agent UI binding. - Entries are filtered by a `hyperhive-socket-bound` marker file - the harness drops next to its socket after a - successful `bind()` — pre-bind agents stay out - of the map so the gateway never races a - missing-socket upstream. A 10s poll loop - (`spawn_poll`) re-fires the writer so the map - reconverges if a marker appears between - meta-sync ticks. - src/gateway_nginx.rs runtime nginx include-file generator. Writes - `/var/lib/hyperhive/gateway/agents.conf` (one - `location /agent//` block per agent, UDS or - TCP upstream) on every topology change; calls - `systemctl -M hive-gateway reload nginx` to reload - nginx. Idempotent — skips write + reload when - the file content would not change. - src/priv_client.rs async client for the `hive-priv` privileged- - helper socket at `/run/hive/priv.sock`. - One-shot connection-per-call: open socket, - send one JSON line, read response, close. - Exposes one async fn per operation; called by - `lifecycle` for bind-mount manipulation during - spawn / rebuild. - src/agent_server.rs per-sub-agent socket listener (long-poll Recv) - src/broker.rs sqlite Message store + intra-process broadcast - channel (`MessageEvent`) for `recv_blocking_batch` + - the dashboard forwarder; hourly vacuum of acked>30d - src/dashboard_events.rs unified wire-facing event channel feeding - `/dashboard/stream`. Carries broker `Sent` / - `Delivered` (mirrored by the forwarder task - in main.rs) + mutation events - (`ApprovalAdded` / `ApprovalResolved`, - `QuestionAdded` / `QuestionResolved`, - `TransientSet` / `TransientCleared`, - `RebuildQueueChanged`, `SchedulesChanged`, - `RemindersChanged`, `CapabilitiesChanged`, - `ToolGroupsChanged`). - Each frame carries a - monotonic per-process `seq` clients use to - dedupe against snapshot reads. - src/approvals.rs sqlite Approval queue + kinds - src/operator_questions.rs sqlite question queue backing `ask` / - `answer` (both operator + agent-to-agent) - src/questions.rs shared dispatch for `Ask` / `Answer` — - used by both agent + manager surfaces - src/reminder_scheduler.rs 5s poll loop: drains due reminders, - resolves file_path container→host, persists - payload + delivers pointer string - src/scheduled_prompts.rs sqlite layer for scheduled prompts — - schema, CRUD helpers (`cancel_all`, - `cancel_targets`, `reap_cancelled`), catch-up - clamp on resume - src/scheduled_prompts_worker.rs 5s poll loop: fires due rows, - fans out one Message per active target, - re-arms recurring rows, deletes fired one-shots - src/events_vacuum.rs host-side hourly sweep of every agent's - /harness/hyperhive-events.sqlite — type-scoped: - prunes only verbose `stream` rows older than - 14d, keeps semantic events (turn-stats has no - vacuum — it's tiny) - src/bash_tasks_vacuum.rs host-side hourly sweep of every agent's - harness/bash-tasks/ — deletes terminal task - trios (.json/.out/.err) older than 48h - src/build_logs.rs sqlite-backed full build-log capture (stdout + - stderr per `nixos-container` / `nix build` - invocation). Process singleton opened by - `Coordinator`; `lifecycle` appends lines live. - Dashboard surfaces the log on expand. Backed by - `/var/lib/hyperhive/build_logs.sqlite`; hourly - retention sweep via the existing vacuum. - src/hive_stats.rs hive-wide turn-stats aggregation: reads every - agent's hyperhive-turn-stats.sqlite read-only, - rolls up swarm totals + per-agent breakdowns + - model mix. Backs GET /api/stats-hive for the - dashboard's ST4TS tab; missing / unreadable dbs - skipped (one bad db never fails the endpoint). - src/container_stats.rs per-container resource metrics (cpu %, current/peak/max - memory in bytes) polled live from cgroup-v2. Backs - GET /api/container-resources for the dashboard's - load chips. Skips unhealthy / stopped containers - gracefully. - src/crash_watch.rs poll every 10s; fire HelperEvent::ContainerCrash - when a previously-running container disappears - without an operator-initiated transient (or a - RECENT_TRANSIENT_GRACE tombstone within the - last 30s) - src/container_view.rs ContainerView struct + build_all helper; - shared between dashboard.rs (cold-load via - /api/state) and coordinator.rs's - rescan_containers_and_emit - src/coordinator.rs shared state (broker/approvals/operator_questions/ - transient/sockets) + tombstone enumeration + - kick_agent + notify_agent (helper-event push) + - last_containers cache + rescan_and_emit diff helper - src/loose_ends.rs loose-ends aggregator (pending approvals + - unanswered questions + pending reminders) — - for_agent (filtered) and hive_wide (manager - surface). Backs AgentRequest::GetLooseEnds + - ManagerRequest::GetLooseEnds (the - get_loose_ends MCP tool). - src/rebuild_queue.rs global serialised queue for long-running ops - (rebuild / meta_update / spawn / destroy / - startup_sweep). - Single background worker drains FIFO; dedup - collapses re-enqueued still-queued entries. - `QueueEntry` carries id, agent, kind, state, - source, parent_id (cascade grouping), timing, - reason, error, step (current phase label while - Running; cleared on finish). Emits - `RebuildQueueChanged` snapshots on every - mutation. - src/actions.rs approve/deny/destroy (transient-aware) - src/auto_update.rs startup rebuild scan + ensure_manager + - meta::lock_update_hyperhive bump - src/lifecycle.rs `nixos-container` shellouts; per-agent applied - + proposed git repo seeding; tag plumbing - src/meta.rs single hive-c0re-owned flake at /var/lib/ - hyperhive/meta/ — sync_agents, two-phase - prepare/finalize/abort, lock_update_* - src/migrate.rs startup auto-migration from pre-meta layout - (idempotent, marker-guarded phase 4) - src/topology.rs agent parent/child storage at - /var/lib/hyperhive/meta/topology.json — sole - source of truth for who's the parent of whom - (single source the dashboard, render_flake, - and the eventual cap-enforcement plumbing all - read). Reconciled by `meta::sync_agents`; - operator/manager edits land via the - eventual write API. - src/tool_groups.rs per-agent tool-group config — reads/writes - `/var/lib/hyperhive/meta/tool-groups.json`; - `set_groups` + `groups_for` + `remove_agent`. - Wired into `render_flake` for `HIVE_TOOL_GROUPS` - injection; operator write via dashboard action. - src/capabilities.rs per-agent capability grants — reads/writes - `/var/lib/hyperhive/meta/capabilities.json`; - `set_caps` + `caps_for` + `has_cap` + - `remove_agent`. Wired into `render_flake` for - `HIVE_CAPABILITIES` injection; operator write - via dashboard P3RM1SS10NS tab. - src/limits.rs wire-protocol size limits (inline body cap, - reminder spill threshold) shared across agent - + manager sockets. Single source of truth. - src/flake_check.rs pre-apply `flake.lock` validation — rejects - proposals missing `inputs..follows` wiring - (duplicate `original` node check). - src/forge.rs optional Forgejo wiring: per-agent users + - tokens, the `agent-configs` org (`push_config`), - and meta read access; mirrors each applied repo - into `agent-configs/` (core-only); agents are - read-only collaborators on `core/meta` - src/matrix.rs optional matrix-tuwunel wiring: generates the - shared registration token at - `/var/lib/hyperhive/matrix-register-token`, - provisions per-agent matrix accounts via the - matrix-spec UIAA two-leg registration flow, - persists each agent's access_token to - `/matrix-token`. Idempotent — skips - registration when token file already exists. - No-op when `hive-matrix` container absent. - src/dashboard.rs axum HTTP: /api/state JSON + actions - + journald viewer + bind-with-retry (SO_REUSEADDR) - + deployed_sha chip per container + - /dashboard/{stream,history} subscribing to the - unified DashboardEvent channel. Static assets - (HTML/CSS/JS/favicon) served by - tower_http::ServeDir from $HIVE_STATIC_DIR - (= `${frontend}/dashboard` per the c0re module). +One line per crate / top-level dir. **Each module's authoritative, +always-current description lives in its own `//!` doc-comment** — +`grep`/read the module when you need detail. This index is kept +deliberately lean: it auto-loads into every turn's context, and a +hand-maintained per-file tree drifts out of sync with the code. -frontend/ npm workspaces (esbuild → static dist). Built - hermetically by `nix/frontend.nix` - (`packages.${system}.frontend`). - packages/shared/ @hive/shared: terminal pane + Catppuccin palette - + base typography (was hive-fr0nt). ES module - exporting { create, linkify }; pure JS, no IIFE - globals; consumed by dashboard + agent. - packages/dashboard/ @hive/dashboard SPA: src/{index.html, tabs.js, - flow.html, flow.js, logs.html, logs.js, - common.js, stream-worker.js, common.css, - dashboard.css, flow.css, logs.css} + - build.mjs → dist/{index.html, flow.html, - logs.html, static/{tabs.js, flow.js, logs.js, - stream-worker.js, common.css, dashboard.css, - flow.css, logs.css}}. - packages/agent/ @hive/agent default per-container UI: src/ - {index, stats, screen}.html + {app, stats}.js - + agent.css → dist/{*.html, static/*}. +### Rust workspace (`Cargo.toml` members) -hive-ag3nt/ in-container harness crate; produces ONE `hive` - binary for all agents - src/lib.rs re-exports + DEFAULT_SOCKET, DEFAULT_WEB_PORT - src/client.rs generic JSON-line request/response over unix socket - src/web_ui.rs per-container axum HTTP page (incl /api/cancel, - /api/compact, /api/model, /events/history, - /screen, /screen/ws) - src/turn_stats.rs per-turn analytics sink (one sqlite row per - turn at /harness/hyperhive-turn-stats.sqlite); - schema + best-effort writer - src/stats.rs read-side aggregations over turn-stats.sqlite - backing the /stats page (bucketed Snapshot: - turns / duration / tokens / model mix) - src/events.rs LiveEvent + broadcast Bus + sqlite-backed history - (/harness/hyperhive-events.sqlite) + TurnState + - model selection (persisted at /harness/hyperhive-model) - src/turn.rs claude --print + stream-json pump; --compact retry; - proactive compaction + auto session-reset - src/mcp.rs embedded MCP server (rmcp): `AgentServer { socket }` — single role, tool groups gate access - src/forge_notify.rs Forgejo webhook subscriber: formats new-issue / - new-PR / comment / review notifications as - broker messages; embeds a body excerpt - (up to BODY_TRUNCATE bytes, ATX headings - escaped) and surfaces any @mention lines that - fell outside the truncation window in a - trailing `mentions (truncated from body):` - block. - src/identity.rs hive-qualified agent label: `label()` / - `qualified_label()` / `qualify(label)`. Reads - `HYPERHIVE_HIVE_DOMAIN`; falls back to short - name when unset. Display-name accessors: - `hive_name()` / `swarm_name()` read - `HYPERHIVE_HIVE_NAME` / `HYPERHIVE_SWARM_NAME`, - both `Option`. - src/login.rs probe $HOME/.claude/ (= `/home//.claude`) - for a valid session - src/login_session.rs drives `claude auth login` over stdio pipes - src/prompt.rs system prompt renderer: filters the unified - template through `` / - `` markers (always - renders the agent section); owns - `write_system_prompt` (re-exported from - turn.rs for callers). - src/bin/hive.rs unified harness main (Serve + Mcp + Wake - subcommands); privilege boundary enforced - server-side at the broker socket. Static UI assets - served by ServeDir from $HIVE_STATIC_DIR - (= hyperhive.frontend.mergedDist — default agent - dist + per-agent extraFiles, set per the - harness-base module). - prompts/ static role/tools/settings for claude (include_str!): - system.md — unified system prompt; shared - sections plus `` / - `` blocks filtered by - `prompt::render` per flavor - claude-settings.json — --settings JSON +- **`hive-c0re/`** — host daemon (runs as the unprivileged `hive-core` + user) plus two operator CLIs. `src/main.rs` is the `hive-c0re` binary + (serve / spawn / kill / rebuild / approve / destroy / periodic vacuum + loops); `src/bin/hivectl.rs` is the ad-hoc operator admin CLI. Owns + the sqlite broker, approval + question + reminder + schedule queues, + the meta flake, lifecycle (`nixos-container` shellouts), gateway / + forge / matrix provisioning, per-container stats, and the axum + operator dashboard (`dashboard.rs`). Largest crate. +- **`hive-ag3nt/`** — in-container harness; one `hive` binary for every + agent. Turn loop (`turn.rs`), embedded MCP server (`mcp.rs`), per-agent + web UI (`web_ui.rs`), event + turn-stats sqlite sinks, login flow, + system-prompt renderer, forge-notify subscriber. +- **`hive-priv/`** — minimal root privileged-helper, socket-activated at + `/run/hive/priv.sock`; performs the few root operations (bind-mount + edits, nsenter) the unprivileged `hive-c0re` delegates to it. See + `docs/boundary.md`. +- **`hive-forge/`** — `hive-forge` Forgejo CLI wrapper; one module per + verb under `src/verbs/`. +- **`hive-matrix-mcp/`** — per-agent matrix-sdk daemon plus the thin + stdio MCP bridge claude spawns per turn. +- **`hive-bash-mcp/`** — per-agent bash-task runner daemon plus its + stdio MCP bridge; writes task files under `/harness/bash-tasks/` and + the favorite-tools `bash_commands` stat into turn-stats.sqlite. +- **`hive-sh4re/`** — shared wire types (Host / Agent / Manager request + + response, `Message`, `Approval`, `HelperEvent`) used across the + unix sockets. -hive-forge/ Forgejo CLI wrapper (`hive-forge` binary) - src/main.rs clap dispatch over the verbs/ - src/client.rs blocking reqwest client (Forgejo REST API) - src/body.rs body input resolution (--body / --body-file / piped stdin) - src/verbs/.rs one module per verb (view, issue, pr, comment, - comments, comment-show, comment-edit, issue-create, - issue-edit, pr-create, pr-reviews, assign, - close, labels, list, milestone, branches, - tree-sha, diff, subscription, attach-issue, - attach-comment, attachment-get, lint, - pr-status, clone). Replaces the - 600-line hive-forge-tools.nix bash script. +### Other top-level dirs -hive-matrix-mcp/ per-agent matrix-sdk integration. - src/main.rs `hive-matrix-daemon` binary entry — long-running - matrix-sdk Client + sync per agent; serves the - MCP bridge over /run/hive-matrix/socket; emits - hyperhive wake on incoming room events. - src/bin/mcp.rs `hive-matrix-mcp` stdio bridge claude spawns per - turn — connects to the daemon socket, forwards - each tool call (send_message, send_dm, - send_reaction, send_reply, mark_read, list_rooms, - list_room_members, read_room). No matrix-sdk dep - at this entrypoint (pure serde + tokio I/O). - src/{client,handlers,socket,timeline,wake,protocol,paths}.rs - shared library modules (matrix-sdk session - restore, request dispatch, unix socket server, - event handler → wake bridge, wire types, env - resolution). - -hive-sh4re/ wire types (HostRequest/Response, AgentRequest/Response, - ManagerRequest/Response, Message, Approval, HelperEvent) - -nix/ - modules/hive-c0re.nix systemd service + firewall + git wiring; - `contextWindowTokens` attrset (per-model, - injected as env vars into all containers); - top-level `services.hyperhive.domain` option - (nullable; required when matrix.enable); - imports hive-forge.nix + hive-gateway.nix + hive-matrix.nix - modules/hive-forge.nix optional in-container Forgejo - (`services.hyperhive.forge.enable`, default on); - `forge.domain` (default `forge.`) drives - ROOT_URL + gateway vhost; Catppuccin Mocha theme - modules/hive-gateway.nix nginx nixos-container in front of all hyperhive - surfaces (`services.hyperhive.gateway.enable`, - default on); proxies `/` → c0re dashboard, - `/agent//` → per-agent UI (driven by - c0re-written JSON at `gateway.agentPortsFile`), - `forge.` sub-domain → forge; `/matrix/` - → fluffychat-web; `.well-known/matrix/` auto-discovery - modules/hive-matrix.nix optional in-container matrix-tuwunel - homeserver (`services.hyperhive.matrix.enable`, - default off); server_name defaults to bare - `services.hyperhive.domain`; gateway serves - `.well-known/matrix/{client,server}` for - auto-discovery; federation on, e2ee deferred - templates/harness-base.nix shared harness for all containers; single - `hive-ag3nt` service unit; `hyperhive.model` - option (HIVE_DEFAULT_MODEL) - templates/agent-base.nix entry-point for sub-agent containers - templates/manager.nix entry-point for ruth; adds forge noise-reduction - defaults (keepSubscriptions=false etc.) - templates/weston-vnc.nix optional `hyperhive.gui.enable` - — weston + VNC backend systemd unit; writes - /etc/hyperhive/gui.json (vnc_port + auth) for - the harness WebSocket relay (/screen/ws) - docs/ markdown options reference (host + agent); - default.nix `nix build .#docs` → `index.md` + `host.md` - + `agent.md` (CommonMark from nixosOptionsDoc). - HTML+CSS rendered downstream by the website repo - forge-theme/theme-catppuccin-vibec0re.css Catppuccin Mocha forge theme - -docs/ - conventions.md naming, identity=socket, tool groups, capabilities, async forms, commit style - gotchas.md NixOS / nspawn quirks and lessons learned - web-ui.md index → web-ui/shape.md (shared skeleton, SSE, - terminal, listener bind, relative paths, atomic - repaint, side panel), web-ui/dashboard.md (tab - contents, container row, topology, selection bar, - approval card, endpoints + event channel), - web-ui/agent.md (header, terminal, composer, - inbox, live view, per-agent endpoints, stats), - web-ui/css-vars.md (Catppuccin theme variables) - turn-loop.md claude invocation, wake prompt, MCP tool surface - tools/ per-group tool docs (bash.md, lifecycle.md, - scheduling.md, matrix.md, forge.md) - approvals.md approval flow, manager policy, helper events - persistence.md sqlite dbs, retention, state dir layout - terminal-rendering.md per-agent terminal row taxonomy (as built) - boundary.md operator/agent trust model rationale - agent-hierarchy.md tree-shape topology design + manager-privilege audit - gateway.md nginx vhost map, matrix discovery flow, firewall posture, - HIVE_FORGE_URL loopback rationale - forge.md per-agent forge accounts + agent-configs mirror; - forge_notify poller (gates, self-filter, body excerpt - pipeline, wrapper formats, reason drop-list, - auto-unsubscribe) - matrix.md matrix container shape, serverName/gatewayHost split, - firewall + federation, provisioning flow, fluffychat-web build - network.md host-side bridge + dnsmasq resolver, v1/v2 roadmap, - foundation for netns isolation - swarm.md multi-hive swarm config: peer declaration, TLS trust - (CA vs certFingerprint), bilateral setup, identity - env vars, matrix federation pointer - ci.md hive-ci Forgejo Actions runner: options, container - shape, auto-registration flow, nix sandbox note - knowledge.md hive-wide knowledge repo: forge clone, /knowledge - bind-mount, webhook + periodic sync, contribution flow - coordinator.md rebuild queue: queue kinds (Rebuild, MetaUpdate, Spawn), - dedup logic, sources, cascade tracking, step labels; - serialisation rationale -``` +- **`frontend/`** — npm workspaces → static dashboard + per-agent UI + dist, built hermetically by `nix/frontend.nix`. Packages: `shared` + (terminal pane + Catppuccin palette), `dashboard` (the operator SPA), + `agent` (the default per-container UI). +- **`nix/`** — NixOS modules (`modules/hive-{c0re,forge,gateway,matrix, + ci,network}.nix`) + per-container templates (`templates/{harness-base, + agent-base,manager,weston-vnc}.nix`) + the options-doc derivation. +- **`docs/`** — subsystem reference docs (see *Reading paths* below). +- **`branding/`**, **`scripts/`** — static assets + helper scripts. ## Reading paths @@ -468,5 +123,3 @@ The docs below own the details — this section just points at them. window:** → [`docs/turn-loop.md`](docs/turn-loop.md). - **Two-step spawn, approval flow, flake.lock validation:** → [`docs/approvals.md`](docs/approvals.md). - -