docs: README + TODO split; trim CLAUDE.md; fix async form 415
This commit is contained in:
müde
parent
392a448656
commit
970f645461
6 changed files with 262 additions and 684 deletions
513
CLAUDE.md
513
CLAUDE.md
|
|
@ -1,86 +1,50 @@
|
|||
# hyperhive
|
||||
# hyperhive — developer reference
|
||||
|
||||
Multi-Claude-Code-agent orchestration on **nixos-containers**. A host-side Rust
|
||||
daemon (`hive-c0re`) spawns nspawn-isolated agent containers and brokers
|
||||
messages between them. A manager agent (`hm1nd`) coordinates the swarm and
|
||||
gates lifecycle changes on user approval via git commits, surfaced through a
|
||||
vibec0re-styled HTTP dashboard with live SSE message-flow.
|
||||
Operator + dev notes: conventions, gotchas, per-subsystem design.
|
||||
|
||||
**PLAN.md** is the living design doc. Read it for the *why* and the phase
|
||||
roadmap; this file is the operator/developer reference for the *how*.
|
||||
- High-level project intro: **[README.md](README.md)**.
|
||||
- Open work + backlog: **[TODO.md](TODO.md)**.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
host (NixOS, hive-c0re.service)
|
||||
│
|
||||
├── hive-c0re (Rust daemon — coordinator + dashboard + CLI)
|
||||
│ ├── lifecycle — nixos-container CRUD (spawn/kill/rebuild/list)
|
||||
│ ├── broker — sqlite message store + broadcast channel
|
||||
│ ├── approvals — sqlite approval queue
|
||||
│ ├── coordinator — shared state (broker/approvals/agent sockets)
|
||||
│ ├── actions — approve/deny (shared between admin socket & dashboard)
|
||||
│ ├── server — host admin socket (JSON line protocol)
|
||||
│ ├── manager_server — manager-only privileged socket
|
||||
│ ├── agent_server — per-sub-agent sockets
|
||||
│ ├── dashboard — axum HTTP UI + SSE message-flow + approve/deny + T4LK
|
||||
│ └── client — admin-socket client (powers `hive-c0re spawn|kill|…`)
|
||||
│
|
||||
├── /run/hyperhive/
|
||||
│ ├── host.sock — admin CLI ↔ daemon
|
||||
│ ├── manager.sock → hm1nd container at /run/hive/mcp.sock
|
||||
│ └── agents/<name>/mcp.sock → h-<name> container at /run/hive/mcp.sock
|
||||
│
|
||||
├── /var/lib/hyperhive/
|
||||
│ ├── broker.sqlite — messages + approvals tables
|
||||
│ ├── agents/<name>/config/ — proposed repo (manager-editable, RO to hive-c0re)
|
||||
│ └── applied/<name>/ — applied repo (hive-c0re-only, container builds here)
|
||||
│
|
||||
└── nixos-containers
|
||||
├── h-<name> (sub-agents, hive-ag3nt binary)
|
||||
└── hm1nd (manager, hive-m1nd binary)
|
||||
```
|
||||
|
||||
## Crates / file map
|
||||
## File map
|
||||
|
||||
```
|
||||
hive-c0re/ host daemon + CLI (one binary, subcommand-dispatched)
|
||||
src/main.rs clap setup; serve / spawn / kill / rebuild / list /
|
||||
pending / approve / deny
|
||||
pending / approve / deny / destroy / request-spawn
|
||||
src/server.rs host admin socket (HostRequest → dispatch)
|
||||
src/client.rs admin-socket client
|
||||
src/manager_server.rs manager-privileged socket (ManagerRequest)
|
||||
src/agent_server.rs per-sub-agent socket listener
|
||||
src/agent_server.rs per-sub-agent socket listener (long-poll Recv)
|
||||
src/broker.rs sqlite Message store + broadcast channel for SSE
|
||||
src/approvals.rs sqlite Approval queue
|
||||
src/coordinator.rs shared state (broker/approvals/agent_flake/sockets)
|
||||
src/actions.rs approve/deny (admin socket + dashboard both call in)
|
||||
src/lifecycle.rs `nixos-container` shellouts, per-agent flake generator,
|
||||
systemd drop-ins, git helpers, agent_web_port hash
|
||||
src/dashboard.rs axum HTTP UI: containers list, T4LK form, approvals
|
||||
(diff + Approve/Deny buttons), SSE message flow
|
||||
src/approvals.rs sqlite Approval queue + kinds
|
||||
src/coordinator.rs shared state (broker/approvals/transient/sockets)
|
||||
src/actions.rs approve/deny/destroy
|
||||
src/auto_update.rs startup rebuild scan + ensure_manager
|
||||
src/lifecycle.rs `nixos-container` shellouts, per-agent flake generator
|
||||
src/dashboard.rs axum HTTP UI: containers, approvals, async-form actions
|
||||
assets/ CSS + JS shipped via include_str!
|
||||
|
||||
hive-ag3nt/ in-container harness crate; produces TWO binaries
|
||||
src/lib.rs DEFAULT_SOCKET, DEFAULT_WEB_PORT, re-exports
|
||||
src/lib.rs re-exports + DEFAULT_SOCKET, DEFAULT_WEB_PORT
|
||||
src/client.rs generic JSON-line request/response over unix socket
|
||||
src/web_ui.rs per-container axum HTTP page (label + placeholder)
|
||||
src/bin/hive-ag3nt.rs sub-agent CLI (serve/send/recv); turn loop + web UI
|
||||
src/bin/hive-m1nd.rs manager CLI (serve/send/recv/spawn/kill/
|
||||
request-apply-commit); recognises HelperEvent
|
||||
src/web_ui.rs per-container axum HTTP page
|
||||
src/events.rs LiveEvent + broadcast Bus for the SSE stream
|
||||
src/turn.rs claude --print + stream-json pump; --compact retry
|
||||
src/mcp.rs embedded MCP server (rmcp): AgentServer + ManagerServer
|
||||
src/login.rs probe /root/.claude/ for a valid session
|
||||
src/login_session.rs drives `claude auth login` over stdio pipes
|
||||
src/bin/hive-ag3nt.rs sub-agent main
|
||||
src/bin/hive-m1nd.rs manager main
|
||||
assets/ CSS + JS for the per-agent UI
|
||||
|
||||
hive-sh4re/ wire types (HostRequest/Response, AgentRequest/Response,
|
||||
ManagerRequest/Response, Message, Approval, HelperEvent)
|
||||
|
||||
nix/
|
||||
modules/hive-c0re.nix systemd service + firewall + git path wiring
|
||||
templates/agent-base.nix sub-agent nixos-container template
|
||||
templates/manager.nix manager nixos-container template
|
||||
|
||||
tests/roundtrip.sh Phase 3 messaging round-trip
|
||||
tests/approval.sh Phase 5 end-to-end approval flow
|
||||
tests/dashboard.sh Phase 6+7 HTTP dashboard + SSE + orphan GC
|
||||
|
||||
docs/damocles-migration.md options for moving damocles onto hyperhive
|
||||
modules/hive-c0re.nix systemd service + firewall + git wiring
|
||||
templates/harness-base.nix shared scaffolding for sub-agents + manager
|
||||
templates/agent-base.nix sub-agent nixosConfiguration
|
||||
templates/manager.nix manager nixosConfiguration
|
||||
```
|
||||
|
||||
## Conventions
|
||||
|
|
@ -104,9 +68,14 @@ docs/damocles-migration.md options for moving damocles onto hyperhive
|
|||
`applied/<name>/flake.nix`, writes the systemd limits drop-in, then
|
||||
`nixos-container update` + stop + start. Anything that changes per-container
|
||||
state on the host should be re-applied here.
|
||||
- **Actions are factored.** `approve` / `deny` live in `actions.rs`; the admin
|
||||
socket and the dashboard POST handlers both call into them, so the two
|
||||
surfaces never drift.
|
||||
- **Actions are factored.** `approve` / `deny` / `destroy` live in
|
||||
`actions.rs`; the admin socket and the dashboard POST handlers both call
|
||||
into them so the two surfaces never drift.
|
||||
- **Async forms.** Dashboard mutating forms carry `data-async`; the
|
||||
`assets/async_forms.js` helper intercepts submit, shows a spinner, and
|
||||
fetches with `application/x-www-form-urlencoded` (axum `Form` extractor
|
||||
rejects multipart). New mutating forms should add `data-async` and
|
||||
optionally `data-confirm`.
|
||||
|
||||
## Gotchas / lessons learned
|
||||
|
||||
|
|
@ -124,33 +93,28 @@ docs/damocles-migration.md options for moving damocles onto hyperhive
|
|||
UIs (the bind is invisible from the host). We force-clear those vars (and
|
||||
`HOST_ADDRESS6` / `LOCAL_ADDRESS6` / `HOST_BRIDGE`) plus set
|
||||
`PRIVATE_NETWORK=0`.
|
||||
- **systemd service PATH ≠ host PATH.** Our service explicitly sets
|
||||
`path = [ pkgs.git "/run/current-system/sw" ]`. Additionally,
|
||||
`environment.HYPERHIVE_GIT = "${pkgs.git}/bin/git"` bakes the absolute path
|
||||
in (read by `lifecycle::git_command()`) so git resolution doesn't depend on
|
||||
PATH plumbing at all.
|
||||
- **systemd service PATH ≠ host PATH.** The hive-c0re service sets
|
||||
`path = [ pkgs.git "/run/current-system/sw" ]`. In-container harness
|
||||
services do the same so anything an agent adds to its own `agent.nix`
|
||||
(`environment.systemPackages`) is visible to claude's Bash tool without
|
||||
editing the service definition. `environment.HYPERHIVE_GIT` bakes git's
|
||||
absolute path in (read by `lifecycle::git_command()`) for the host.
|
||||
- **`RuntimeDirectoryPreserve = "yes"`** keeps `/run/hyperhive/` (and the
|
||||
per-agent sub-dirs) across `hive-c0re` restarts. Without it, every restart
|
||||
wipes bind sources and existing containers can't be started.
|
||||
- **`register_agent` is idempotent** — drops any prior socket task before
|
||||
rebinding. Required so a `hive-c0re` restart followed by `rebuild alice`
|
||||
recreates the agent's socket without needing a clean reinstall.
|
||||
- **`claude-code` is unfree.** `agent-base.nix` allow-list's it specifically.
|
||||
The flake pins it to **nixpkgs-unstable** via `overlays.claude-unstable`
|
||||
(stable lags too far). The overlay imports unstable with its own
|
||||
`allowUnfreePredicate` so the access inside the overlay doesn't itself trip.
|
||||
- **Claude credentials are stateful and per-container.** No `ANTHROPIC_API_KEY`
|
||||
env var path. Today's stopgap: `nixos-container root-login h-<name>` →
|
||||
`claude` (interactive) → log in once. The harness falls back to echo
|
||||
replies when `claude --print` fails. **Phase 8** moves this to a per-agent
|
||||
persistent dir at `/var/lib/hyperhive/agents/<name>/claude/` bind-mounted
|
||||
into the container, with the interactive login driven from the agent's web
|
||||
UI. Sharing one `~/.claude` across agents is NOT viable — OAuth refresh
|
||||
tokens rotate, so any sibling refresh invalidates all the others.
|
||||
- **Echo guard.** `hive-ag3nt serve` skips auto-reply when the incoming body
|
||||
starts with `"echo: "`. Prevents ping-pong loops when both sides fall back
|
||||
to echo. Real conversations between claude-backed agents *will* runaway —
|
||||
bounding them is the manager's job.
|
||||
- **`claude-code` is unfree.** `harness-base.nix` allow-list's it
|
||||
specifically. The flake pins it to **nixpkgs-unstable** via
|
||||
`overlays.claude-unstable` (stable lags too far). The overlay imports
|
||||
unstable with its own `allowUnfreePredicate` so the access inside the
|
||||
overlay doesn't itself trip.
|
||||
- **Claude credentials are per-agent.** `/var/lib/hyperhive/agents/<name>/claude/`
|
||||
bind-mounts to `/root/.claude` (RW). Sharing one dir across agents is NOT
|
||||
viable — OAuth refresh tokens rotate, so any sibling refresh invalidates
|
||||
all the others. Login flow runs from the per-agent web UI; creds persist
|
||||
across `destroy`/recreate.
|
||||
- **Orphan approvals.** If state dirs are wiped out from under a pending
|
||||
approval (test scripts, manual `rm -rf`), the dashboard's next render
|
||||
marks them `failed` with note `"agent state dir missing"` so they fall out
|
||||
|
|
@ -159,339 +123,130 @@ docs/damocles-migration.md options for moving damocles onto hyperhive
|
|||
## Agent MCP surface + turn loop
|
||||
|
||||
The harness ships an embedded MCP server (rmcp 1.7) that claude launches as
|
||||
a stdio child via `--mcp-config`. Subcommand: `hive-ag3nt mcp`. Tools:
|
||||
a stdio child via `--mcp-config`. Subcommand: `hive-ag3nt mcp` (or
|
||||
`hive-m1nd mcp` for the manager surface).
|
||||
|
||||
Sub-agent tools:
|
||||
- `mcp__hyperhive__send(to, body)` — message a peer or the operator.
|
||||
- `mcp__hyperhive__recv()` — drain one inbox message.
|
||||
|
||||
Both translate to `AgentRequest::Send`/`Recv` against the agent's own
|
||||
`/run/hive/mcp.sock` (the existing hyperhive socket). The MCP surface is
|
||||
just claude's view of that socket — same authority, friendlier protocol.
|
||||
|
||||
The turn loop in `hive-ag3nt serve` writes
|
||||
`/run/hive/claude-mcp-config.json` at boot pointing at
|
||||
`/proc/self/exe mcp` (the running hive-ag3nt binary's nix store path).
|
||||
Each turn invokes:
|
||||
|
||||
```
|
||||
claude --print --model haiku --mcp-config <path> --tools <builtins> --allowedTools <builtins+mcp> <prompt>
|
||||
```
|
||||
|
||||
**Loop control.** The harness pops one inbox message (the wake signal) per
|
||||
cycle and hands claude a prompt naming the agent, the sender, the body,
|
||||
and the MCP tools. Claude drives any further `recv`/`send` itself —
|
||||
harness no longer relays claude's stdout as a reply. Stdout is logged for
|
||||
debugging; the side effects (sends via MCP) are what matter.
|
||||
|
||||
**Operator input** moved from the hive-c0re dashboard's T4LK form to each
|
||||
per-agent page. The per-agent `/send` POST hits the new
|
||||
`AgentRequest::OperatorMsg` / `ManagerRequest::OperatorMsg` wire verb,
|
||||
which enqueues `Message { from: "operator", to: <self>, body }` directly
|
||||
into the broker. No more global recipient dropdown — one input per agent
|
||||
page, scoped to that agent.
|
||||
|
||||
**Live view.** Each agent runs a `hive_ag3nt::events::Bus` (a
|
||||
`tokio::sync::broadcast<LiveEvent>` wrapper). The harness emits:
|
||||
- `TurnStart { from, body }` when a wake-up message is popped.
|
||||
- `Stream(value)` for every line claude prints on stdout (parsed
|
||||
stream-json; flattened under `{kind: "stream", type: ...}` via serde
|
||||
internal tagging).
|
||||
- `Note(text)` for stderr lines and non-JSON stdout (so nothing's lost).
|
||||
- `TurnEnd { ok, note }` when claude exits.
|
||||
|
||||
The web UI subscribes via `/events/stream` (SSE) and a small JS panel on
|
||||
`/` appends rows. No full-page reload — the login form (and anything else
|
||||
the operator is typing into) stays put.
|
||||
|
||||
claude is invoked with `--print --verbose --output-format stream-json` so
|
||||
tool calls + assistant text + tool results all land as structured events.
|
||||
The harness no longer reads claude's text stdout into a reply; claude
|
||||
calls `mcp__hyperhive__send` itself.
|
||||
|
||||
**Tool envelope.** Every MCP tool handler in `hive_ag3nt::mcp::AgentServer`
|
||||
wraps its logic in `run_tool(name, args_debug, async { ... })`. The
|
||||
envelope guarantees:
|
||||
1. Pre-log of the request (tool + args).
|
||||
2. The tool's own logic runs.
|
||||
3. A status line is appended to the result body
|
||||
(`[status] N unread message(s) in inbox`) so claude always sees the
|
||||
current inbox depth without an extra tool call.
|
||||
4. Post-log of the full result.
|
||||
|
||||
`AgentRequest::Status` is the non-mutating peek that powers the status
|
||||
line (broker's `count_pending`). When adding new tools (manager surface,
|
||||
notes/state, etc.), use `run_tool` and they pick up the envelope for free.
|
||||
|
||||
**Tool whitelist** (see `ALLOWED_BUILTIN_TOOLS` in `hive-ag3nt::mcp`):
|
||||
- Allowed built-ins: `Bash`, `Edit`, `Glob`, `Grep`, `Read`, `TodoWrite`,
|
||||
`Write`.
|
||||
- Denied by omission: `WebFetch`, `WebSearch`, `Task`, `NotebookEdit` —
|
||||
no external egress, nested-agent spawning, or Jupyter handling until we
|
||||
have a real policy story.
|
||||
- Allowed MCP tools: `mcp__hyperhive__send`, `mcp__hyperhive__recv`.
|
||||
|
||||
`Bash` is on the allow-list "for now" — pending a finer-grained allow-list
|
||||
system for command patterns (`Bash(git *)`-style). When that lands, the
|
||||
`builtin_tools_arg` shape will probably change to a setting / hooks
|
||||
combo per claude-code's permissions plumbing.
|
||||
|
||||
The manager (`hive-m1nd`) runs the same loop with a `ManagerServer` MCP
|
||||
flavor:
|
||||
- `mcp__hyperhive__send`, `recv` — agent surface.
|
||||
Manager additionally:
|
||||
- `mcp__hyperhive__request_spawn(name)` — queue Spawn approval.
|
||||
- `mcp__hyperhive__kill(name)` — graceful stop of a sub-agent.
|
||||
- `mcp__hyperhive__kill(name)` — graceful stop.
|
||||
- `mcp__hyperhive__request_apply_commit(agent, commit_ref)` — submit a
|
||||
config change for any agent (`hm1nd` for self-modification).
|
||||
config change for any agent (including `hm1nd` for self-mods).
|
||||
|
||||
The shared per-turn plumbing lives in `hive_ag3nt::turn::{write_mcp_config,
|
||||
run_turn}` so both binaries can't drift apart.
|
||||
run_turn, drive_turn, emit_turn_end, wait_for_login}` so the two binaries
|
||||
can't drift.
|
||||
|
||||
Each turn:
|
||||
|
||||
```
|
||||
claude --print --verbose --output-format stream-json --model haiku \
|
||||
--continue --settings '{"autoCompactEnabled":false,"autoMemoryEnabled":false}' \
|
||||
--mcp-config <path> --strict-mcp-config \
|
||||
--tools <builtins> --allowedTools <builtins+mcp>
|
||||
# prompt piped over stdin
|
||||
```
|
||||
|
||||
`--continue` keeps a persistent session per agent (claude stores sessions in
|
||||
`~/.claude/projects/`, which is bind-mounted persistently). Auto-compact and
|
||||
auto-memory are disabled because hyperhive owns compaction (`/compact` on
|
||||
overflow, retry once).
|
||||
|
||||
**Loop control.** The harness pops one inbox message per cycle (the wake
|
||||
signal — Recv long-polls server-side for up to 30s waking instantly on a new
|
||||
broker `Sent` event for this agent) and hands claude a prompt naming the
|
||||
agent, the sender, the body, and the MCP tools. Claude drives any further
|
||||
`recv`/`send` itself.
|
||||
|
||||
**Tool envelope** (`mcp::run_tool_envelope`): every MCP tool handler logs
|
||||
the request, runs the body, appends a status line (e.g.
|
||||
`[status] 3 unread message(s) in inbox` from a non-mutating `Status` peek),
|
||||
logs the result. New tools call this helper.
|
||||
|
||||
**Tool whitelist** (`mcp::ALLOWED_BUILTIN_TOOLS`):
|
||||
- Allowed built-ins: `Bash`, `Edit`, `Glob`, `Grep`, `Read`, `TodoWrite`,
|
||||
`Write`.
|
||||
- Denied by omission: `WebFetch`, `WebSearch`, `Task`, `NotebookEdit`.
|
||||
- Allowed MCP tools: as listed above per flavor.
|
||||
|
||||
`Bash` is on the allow-list pending a finer-grained pattern allow-list
|
||||
(`Bash(git *)`-style) — see [TODO.md](TODO.md).
|
||||
|
||||
**Live view.** Each agent runs an `events::Bus` (a
|
||||
`tokio::sync::broadcast<LiveEvent>` wrapper). The harness emits
|
||||
`TurnStart`, `Stream(value)` (one per parsed stream-json line), `Note`,
|
||||
`TurnEnd`. The web UI subscribes via `/events/stream` (SSE) and a JS panel
|
||||
appends rows. No full-page reload — operator input stays put.
|
||||
|
||||
## Manager (hm1nd) is hive-c0re-managed
|
||||
|
||||
The manager container runs through the **same lifecycle as sub-agents** —
|
||||
no separate code path. On `hive-c0re serve` startup, if `nixos-container
|
||||
list` doesn't include `hm1nd`, hive-c0re creates it. The manager's flake
|
||||
lives at `/var/lib/hyperhive/applied/hm1nd/`; its proposed (manager-editable)
|
||||
config at `/var/lib/hyperhive/agents/hm1nd/config/`. Manager can edit its
|
||||
own `agent.nix` (visible inside the container at `/agents/hm1nd/config/`),
|
||||
commit, and submit `request-apply-commit hm1nd <sha>` for operator
|
||||
approval — same flow as for sub-agents.
|
||||
The manager container runs through the **same lifecycle as sub-agents**.
|
||||
On `hive-c0re serve` startup, if `hm1nd` is missing, hive-c0re creates it.
|
||||
The manager's flake lives at `/var/lib/hyperhive/applied/hm1nd/`; its
|
||||
proposed config at `/var/lib/hyperhive/agents/hm1nd/config/`. Manager can
|
||||
edit its own `agent.nix` (visible inside the container at
|
||||
`/agents/hm1nd/config/`) and submit `request-apply-commit hm1nd <sha>` for
|
||||
operator approval.
|
||||
|
||||
Differences from sub-agents:
|
||||
- `flake.nix` extends `hyperhive.nixosConfigurations.manager` (vs
|
||||
`agent-base`).
|
||||
- `flake.nix` extends `hyperhive.nixosConfigurations.manager`
|
||||
(vs `agent-base`).
|
||||
- Container name is `hm1nd` (no `h-` prefix).
|
||||
- Fixed web UI port (`MANAGER_PORT = 8000`).
|
||||
- `set_nspawn_flags` adds an extra bind: `/var/lib/hyperhive/agents` →
|
||||
`/agents` (RW), so the manager can edit per-agent proposed repos.
|
||||
- First-deploy spawn bypasses the approval queue (manager is required
|
||||
infrastructure).
|
||||
- Per-agent socket is the manager socket at `/run/hyperhive/manager/`, owned
|
||||
by `manager_server::start`. `coordinator::ensure_runtime` returns that
|
||||
path for manager and the usual `/run/hyperhive/agents/<name>/` for the
|
||||
rest.
|
||||
- Per-agent socket lives at `/run/hyperhive/manager/`, owned by
|
||||
`manager_server::start`.
|
||||
|
||||
**Migration note:** drop any `containers.hm1nd = { ... }` block from your
|
||||
host NixOS config. hyperhive creates and updates the manager itself now.
|
||||
**Migration note (for older hosts):** drop any `containers.hm1nd = { ... }`
|
||||
block from your host NixOS config. hyperhive creates and updates the
|
||||
manager itself now.
|
||||
|
||||
## Auto-update on startup
|
||||
|
||||
`hive-c0re serve` runs `auto_update::run` in a background task right after
|
||||
opening the coordinator. It enumerates managed containers and rebuilds any
|
||||
whose recorded hyperhive rev differs from the current one:
|
||||
whose recorded hyperhive rev differs from the current one — sub-agents and
|
||||
manager go through the same `lifecycle::rebuild` path. "Rev" = canonical
|
||||
filesystem path of `cfg.hyperhiveFlake`. Marker file:
|
||||
`/var/lib/hyperhive/applied/.<name>.hyperhive-rev`.
|
||||
|
||||
- **Sub-agents** rebuild via `lifecycle::rebuild` (regenerates
|
||||
`applied/<name>/flake.nix`, sets nspawn flags, `nixos-container update --flake`).
|
||||
- **Manager** runs `nixos-container update hm1nd` (no `--flake`). The
|
||||
manager's config lives in the host's NixOS module; this is belt-and-braces
|
||||
on top of NixOS's own container activation. Idempotent when nothing has
|
||||
actually changed.
|
||||
|
||||
"Rev" = canonical filesystem path of `cfg.hyperhiveFlake` (so `/etc/hyperhive`
|
||||
resolving to a new `/nix/store/...-source` triggers a rebuild). Marker file:
|
||||
`/var/lib/hyperhive/applied/.<name>.hyperhive-rev`. If the flake input has
|
||||
no canonical path (e.g. a `github:` URL), auto-update is a no-op — rebuild
|
||||
manually. The task is async and never blocks the admin socket; failures are
|
||||
logged and don't take the daemon down.
|
||||
If the flake input has no canonical path (e.g. a `github:` URL),
|
||||
auto-update is a no-op — rebuild manually.
|
||||
|
||||
The dashboard surfaces pending updates per agent: a clickable "needs update
|
||||
↻" badge appears whenever the marker differs from current rev. The badge
|
||||
POSTs `/rebuild/<name>`, calling the same `auto_update::rebuild_agent` /
|
||||
`rebuild_manager` path so manual triggers and the startup scan can't drift.
|
||||
|
||||
## Build / deploy / test
|
||||
|
||||
```sh
|
||||
# inside the repo (devshell first; no global cargo)
|
||||
nix develop -c cargo check
|
||||
nix develop -c cargo clippy --workspace --all-targets -- -D warnings
|
||||
nix develop -c cargo build
|
||||
|
||||
# evaluate everything (incl. rust+nix+toml fmt + clippy)
|
||||
nix flake check
|
||||
|
||||
# build only the workspace package
|
||||
nix build .#default
|
||||
./result/bin/{hive-c0re,hive-ag3nt,hive-m1nd}
|
||||
|
||||
# deploy to an existing host that imports hyperhive.nixosModules.hive-c0re
|
||||
cd ~/Repos/<nixos-config-repo>
|
||||
nix flake update --update-input hyperhive
|
||||
sudo nixos-rebuild switch --flake .#<host>
|
||||
sudo systemctl restart hive-c0re # if only env/options changed
|
||||
|
||||
# end-to-end tests (each idempotent; runs as root)
|
||||
sudo bash tests/roundtrip.sh # alice ↔ bob echo round-trip
|
||||
sudo bash tests/approval.sh # manager edit → request → user approve → rebuilt
|
||||
sudo bash tests/dashboard.sh # HTTP UI, approve POST, SSE, orphan GC
|
||||
```
|
||||
|
||||
The host config also needs `hyperhive.overlays.default` applied — the module's
|
||||
default `package = pkgs.hyperhive` requires the overlay to bring the package
|
||||
in. The `claude-unstable` overlay is applied internally to per-agent flakes
|
||||
already.
|
||||
|
||||
## Phase status
|
||||
|
||||
- ✅ Phase 0 — repo + Cargo workspace + flake + agent-base template
|
||||
- ✅ Phase 1 — container lifecycle; `nixos-container update` hot-reload works
|
||||
under the patch stack (validated on muede-lpt2)
|
||||
- ✅ Phase 2 — per-agent sockets, in-memory broker, agent harness round-trips
|
||||
- ✅ Phase 3 — sqlite broker (durable) + claude-or-echo turn loop
|
||||
- ✅ Phase 4 — `hm1nd` manager binary + manager socket + declarative
|
||||
`containers.hm1nd`
|
||||
- ✅ Phase 5 — git-commit approval flow
|
||||
- 5a — sqlite approval queue (`request_apply_commit`/`pending`/`approve`/`deny`)
|
||||
- 5b — per-agent config flakes
|
||||
- 5c — manager edits `proposed`, hive-c0re writes-only `applied`; container
|
||||
builds from `applied`. Approve = read `agent.nix` at the approved commit
|
||||
from `proposed`, copy into `applied`, commit + rebuild. Manager cannot
|
||||
move `applied/main` on its own.
|
||||
- ✅ Phase 6 — per-container web UIs (`HIVE_PORT` deterministic-hash) +
|
||||
hive-c0re dashboard (default 7000, vibec0re aesthetic, deep-linked)
|
||||
- ✅ Phase 7 — polish:
|
||||
- 7a — dashboard Approve/Deny buttons + unified diff (`similar` crate)
|
||||
- 7b — broker broadcast + `/messages/stream` SSE + live message-flow panel
|
||||
- 7c — `ApprovalResolved` helper events into manager inbox
|
||||
- 7d — `MemoryMax=2G` + `CPUQuota=50%` systemd drop-in per container
|
||||
- 7e — damocles migration plan (`docs/damocles-migration.md`)
|
||||
- ✅ Phase 7 follow-ups:
|
||||
- Dashboard **T4LK** form — operator can send messages from the browser
|
||||
(`POST /send`, becomes `from: "operator"` broker message)
|
||||
- Orphan-approval GC on dashboard render (stale entries auto-failed)
|
||||
- `PRIVATE_NETWORK=0` + `HOST_ADDRESS=`/`LOCAL_ADDRESS=` cleared in
|
||||
`set_nspawn_flags` so sub-agent web UI ports are reachable on the host
|
||||
- `HYPERHIVE_GIT` env var (absolute path) bypasses PATH ambiguity
|
||||
|
||||
## Phase 8 — real claude in containers + login UX (in progress)
|
||||
|
||||
See PLAN.md → "Phase 8" for the full design. Summary:
|
||||
|
||||
- **Per-agent persistent creds dir.** Bind
|
||||
`/var/lib/hyperhive/agents/<name>/claude/` → `/root/.claude` (RW) in
|
||||
`set_nspawn_flags`. One OAuth lineage per agent; refresh rotations stay
|
||||
contained to that agent.
|
||||
- **State dirs persist by default.** `destroy` keeps
|
||||
`/var/lib/hyperhive/agents/<name>/` unless the operator passes an explicit
|
||||
wipe flag. Recreating an agent of the same name reuses prior creds.
|
||||
- **First spawn is approval-gated.** New agent names go through the same
|
||||
approval queue as config edits. Manager calls `RequestSpawn` (CLI:
|
||||
`hive-m1nd request-spawn <name>`); operator can also queue from the
|
||||
dashboard or `hive-c0re request-spawn <name>`. The host's direct
|
||||
`hive-c0re spawn <name>` still works as a privileged bypass for tests.
|
||||
Approve runs `lifecycle::spawn` in a background task; the dashboard polls
|
||||
via `<meta refresh>` and renders a spinner row while
|
||||
`nixos-container create` + `update` + `start` is in flight.
|
||||
- **"needs login" partial-run state.** No valid session in `~/.claude/` →
|
||||
harness binds the web UI but does NOT start the turn loop. The harness
|
||||
polls the dir; as soon as a login lands it transitions into the turn loop
|
||||
without a restart. Dashboard surfaces the state per-agent via a `needs
|
||||
login` badge in the container list. "Valid session" today is a heuristic
|
||||
(any regular file inside `/root/.claude/`); we may refine once the
|
||||
filename layout claude writes is locked in.
|
||||
- **Login from the per-agent web UI.** Spawn `claude auth login` with plain
|
||||
stdio pipes (no PTY initially), surface the OAuth URL from stdout on the
|
||||
page, accept the resulting code via a paste field, write it to the process
|
||||
stdin. Once `~/.claude/` populates, the existing needs-login polling loop
|
||||
flips state to Online and starts the turn loop — no separate signaling
|
||||
needed. The exact command is overridable via `HYPERHIVE_LOGIN_CMD` so we
|
||||
can adjust without rebuilding. If pipes turn out to be insufficient
|
||||
(claude refuses without a TTY, raw-mode input, ANSI-only output) we redo
|
||||
the backend with a PTY (e.g. `portable-pty`).
|
||||
|
||||
Implementation order: bind-mount/dir creation → approval-gated spawn +
|
||||
spinner → "needs login" partial run → PTY login endpoint. The login UI has
|
||||
nowhere to live until the partial-run mode exists, so don't ship it earlier.
|
||||
POSTs `/rebuild/<name>`, calling the same `auto_update::rebuild_agent`
|
||||
path so manual triggers and the startup scan can't drift.
|
||||
|
||||
## Approval flow
|
||||
|
||||
End-to-end: manager edits per-agent `proposed` repo → commits → submits commit
|
||||
sha → user approves on host CLI **or** dashboard button → `hive-c0re` reads the
|
||||
sha → user approves on host CLI or dashboard button → `hive-c0re` reads the
|
||||
file at that sha from `proposed`, applies into `applied`, commits there, runs
|
||||
`nixos-container update`. Helper-event JSON lands in the manager's inbox.
|
||||
`nixos-container update`. Helper-event JSON (`ApprovalResolved`) lands in the
|
||||
manager's inbox.
|
||||
|
||||
```
|
||||
# Inside the hm1nd container (manager has /agents bind-mounted RW):
|
||||
cd /agents/alice/config
|
||||
$EDITOR agent.nix # e.g. environment.systemPackages = [ pkgs.htop ];
|
||||
git commit -am "add htop"
|
||||
SHA=$(git rev-parse HEAD)
|
||||
hive-m1nd request-apply-commit alice $SHA
|
||||
exit
|
||||
|
||||
# On the host (CLI):
|
||||
sudo hive-c0re pending # shows queued approval with id N
|
||||
sudo hive-c0re approve N # validates, applies, rebuilds
|
||||
sudo nixos-container run h-alice -- which htop
|
||||
|
||||
# Or on the dashboard (browser):
|
||||
http://<host>:7000/ # ◆ APPR0VE button next to the diff
|
||||
```
|
||||
|
||||
Per-agent layout — two separate git repos:
|
||||
Two separate git repos per agent:
|
||||
|
||||
```
|
||||
/var/lib/hyperhive/agents/<name>/config/ # proposed — manager edits, hive-c0re reads only
|
||||
├── .git/
|
||||
└── agent.nix # the only file the manager can change
|
||||
# (initial commit by hive-c0re on first spawn,
|
||||
# never touched by hive-c0re again)
|
||||
|
||||
/var/lib/hyperhive/applied/<name>/ # applied — hive-c0re-only; container builds here
|
||||
├── .git/
|
||||
├── flake.nix # hive-c0re-managed; references hyperhive_flake
|
||||
├── flake.nix # auto-generated; references hyperhive_flake
|
||||
└── agent.nix # overwritten by approve from the proposed commit
|
||||
```
|
||||
|
||||
The container's `--flake` ref is `<applied_dir>#default`. The flake's
|
||||
`nixosConfigurations.default` extends `hyperhive.nixosConfigurations.agent-base`
|
||||
with `./agent.nix` plus an inline module that sets
|
||||
`environment.etc."gitconfig".text` (committer identity = the agent's name) and
|
||||
`systemd.services.hive-ag3nt.environment.HIVE_PORT`/`HIVE_LABEL`.
|
||||
|
||||
## Security backlog
|
||||
|
||||
- **Unprivileged containers (userns mapping).** Today the nspawn container
|
||||
runs as a fully privileged root. Goal: `PrivateUsersChown=yes` (or the
|
||||
nixos-container equivalent) so uid 0 inside maps to an unprivileged uid
|
||||
on the host, and a container-root compromise lands the attacker on an
|
||||
ordinary user account, not the host's root. Requires per-agent state
|
||||
dirs to be chown'd to that uid on the host side.
|
||||
- **Bash command allow-list.** Replace the blanket `Bash` allow with a
|
||||
pattern allow-list (`Bash(git *)`, `Bash(nix build .*)`, etc.) per
|
||||
claude-code's `--allowedTools` extended grammar. Likely lives in
|
||||
`agent.nix` so each agent can scope its own shell surface.
|
||||
|
||||
## Per-agent settings backlog
|
||||
|
||||
- **Model override.** Hard-coded to `claude-haiku-4-5-20251001` in the turn
|
||||
loop right now. Surface it as a per-agent override: operator via
|
||||
dashboard, manager via `request_apply_commit` setting an attr on the
|
||||
agent's flake (most natural place since the flake already carries
|
||||
per-agent env/identity).
|
||||
|
||||
## Polish backlog
|
||||
|
||||
Not phased — pick when relevant:
|
||||
|
||||
- **Operator inbox view** — drain replies addressed to `operator` and show
|
||||
in the dashboard (today they accumulate in sqlite unread).
|
||||
- **Per-agent UI substance** — show last N inbox messages, last turn timing,
|
||||
link back to dashboard.
|
||||
- **xterm.js terminal** — embed in each per-container UI, attach to a PTY
|
||||
exposed by the harness.
|
||||
- **`destroy` verb** — currently `nixos-container destroy` + manual `rm -rf`.
|
||||
Should be one hive-c0re verb that also purges approvals + state dirs.
|
||||
- **Bounded broker** — cap rows per recipient or auto-vacuum delivered
|
||||
messages older than a threshold.
|
||||
- **Container crash events** — watch `container@*.service` via D-Bus,
|
||||
push `HelperEvent::ContainerCrash` to the manager.
|
||||
|
||||
## Inspirations
|
||||
|
||||
- **`~/Repos/bitburner-agent`** — sibling project, drives Claude Code in a
|
||||
turn loop against a Bitburner CDP session. Patterns to steal as we grow:
|
||||
per-cycle prompt diffing (vs full state), notes compaction as a separate
|
||||
short-lived Claude session, MCP server registering tools from a single
|
||||
`TOOLS` array, dashboard with SSE + xterm.js + sqlite stats sampler,
|
||||
opaque "terminal event" stream that unifies tool-call / sleep / op-notice
|
||||
/ etc.
|
||||
The container's `--flake` ref is `<applied_dir>#default`. The flake extends
|
||||
`hyperhive.nixosConfigurations.{agent-base|manager}` with `./agent.nix` plus
|
||||
an inline module setting `programs.git.config.user` (committer identity =
|
||||
the agent's name) and `systemd.services.<harness>.environment` (HIVE_PORT,
|
||||
HIVE_LABEL, HIVE_DASHBOARD_PORT).
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue