docs: CLAUDE.md + PLAN.md updated for current state (all phases shipped)
This commit is contained in:
müde
parent
fcd6563887
commit
c7b50aa5b7
2 changed files with 210 additions and 121 deletions
261
CLAUDE.md
261
CLAUDE.md
|
|
@ -1,10 +1,10 @@
|
|||
# hyperhive
|
||||
|
||||
Multi-Claude-Code-agent orchestration on **nixos-containers**. A host-side Rust
|
||||
daemon spawns nspawn-isolated agent containers and brokers messages between
|
||||
them. Eventually a manager agent (another Claude Code session in its own
|
||||
container) coordinates the swarm and gates lifecycle changes on user approval
|
||||
via git commits.
|
||||
daemon (`hive-c0re`) spawns nspawn-isolated agent containers and brokers
|
||||
messages between them. A manager agent (`hm1nd`) coordinates the swarm and
|
||||
gates lifecycle changes on user approval via git commits, surfaced through a
|
||||
vibec0re-styled HTTP dashboard with live SSE message-flow.
|
||||
|
||||
**PLAN.md** is the living design doc. Read it for the *why* and the phase
|
||||
roadmap; this file is the operator/developer reference for the *how*.
|
||||
|
|
@ -12,66 +12,101 @@ roadmap; this file is the operator/developer reference for the *how*.
|
|||
## Architecture
|
||||
|
||||
```
|
||||
host
|
||||
├── hive-c0re (Rust daemon, NixOS service)
|
||||
│ ├── lifecycle — nixos-container CRUD
|
||||
│ ├── broker — sqlite message store (/var/lib/hyperhive/broker.sqlite)
|
||||
host (NixOS, hive-c0re.service)
|
||||
│
|
||||
├── hive-c0re (Rust daemon — coordinator + dashboard + CLI)
|
||||
│ ├── lifecycle — nixos-container CRUD (spawn/kill/rebuild/list)
|
||||
│ ├── broker — sqlite message store + broadcast channel
|
||||
│ ├── approvals — sqlite approval queue
|
||||
│ ├── coordinator — shared state (broker/approvals/agent sockets)
|
||||
│ ├── actions — approve/deny (shared between admin socket & dashboard)
|
||||
│ ├── server — host admin socket (JSON line protocol)
|
||||
│ └── agent_server — per-agent MCP-ish sockets
|
||||
│ ├── manager_server — manager-only privileged socket
|
||||
│ ├── agent_server — per-sub-agent sockets
|
||||
│ ├── dashboard — axum HTTP UI + SSE message-flow + approve/deny + T4LK
|
||||
│ └── client — admin-socket client (powers `hive-c0re spawn|kill|…`)
|
||||
│
|
||||
├── /run/hyperhive/
|
||||
│ ├── host.sock — admin CLI ↔ daemon
|
||||
│ └── agents/<name>/mcp.sock — bind-mounted into each container at /run/hive
|
||||
│ ├── manager.sock → hm1nd container at /run/hive/mcp.sock
|
||||
│ └── agents/<name>/mcp.sock → h-<name> container at /run/hive/mcp.sock
|
||||
│
|
||||
├── /var/lib/hyperhive/
|
||||
│ ├── broker.sqlite — messages + approvals tables
|
||||
│ ├── agents/<name>/config/ — proposed repo (manager-editable, RO to hive-c0re)
|
||||
│ └── applied/<name>/ — applied repo (hive-c0re-only, container builds here)
|
||||
│
|
||||
└── nixos-containers
|
||||
├── h-<name> (sub-agents, hive-ag3nt binary)
|
||||
└── hm1nd (manager, hive-m1nd binary — Phase 4+)
|
||||
└── hm1nd (manager, hive-m1nd binary)
|
||||
```
|
||||
|
||||
## Crates / file map
|
||||
|
||||
```
|
||||
hive-c0re/ host daemon + CLI (one binary, subcommand-dispatched)
|
||||
main.rs clap setup; serve vs spawn/kill/rebuild/list
|
||||
server.rs host admin socket
|
||||
client.rs host admin socket client (for spawn/kill/rebuild/list)
|
||||
broker.rs sqlite-backed Message store (rusqlite)
|
||||
agent_server.rs per-agent socket listener
|
||||
coordinator.rs shared runtime state (broker + map<name, AgentSocket>)
|
||||
lifecycle.rs `nixos-container` shellouts (spawn/kill/rebuild/list)
|
||||
src/main.rs clap setup; serve / spawn / kill / rebuild / list /
|
||||
pending / approve / deny
|
||||
src/server.rs host admin socket (HostRequest → dispatch)
|
||||
src/client.rs admin-socket client
|
||||
src/manager_server.rs manager-privileged socket (ManagerRequest)
|
||||
src/agent_server.rs per-sub-agent socket listener
|
||||
src/broker.rs sqlite Message store + broadcast channel for SSE
|
||||
src/approvals.rs sqlite Approval queue
|
||||
src/coordinator.rs shared state (broker/approvals/agent_flake/sockets)
|
||||
src/actions.rs approve/deny (admin socket + dashboard both call in)
|
||||
src/lifecycle.rs `nixos-container` shellouts, per-agent flake generator,
|
||||
systemd drop-ins, git helpers, agent_web_port hash
|
||||
src/dashboard.rs axum HTTP UI: containers list, T4LK form, approvals
|
||||
(diff + Approve/Deny buttons), SSE message flow
|
||||
|
||||
hive-ag3nt/ in-container harness; produces TWO binaries from one crate
|
||||
src/lib.rs DEFAULT_SOCKET, re-exports
|
||||
src/client.rs AgentRequest/AgentResponse over /run/hive/mcp.sock
|
||||
src/bin/hive-ag3nt.rs sub-agent CLI (serve/send/recv)
|
||||
src/bin/hive-m1nd.rs manager placeholder (Phase 4)
|
||||
hive-ag3nt/ in-container harness crate; produces TWO binaries
|
||||
src/lib.rs DEFAULT_SOCKET, DEFAULT_WEB_PORT, re-exports
|
||||
src/client.rs generic JSON-line request/response over unix socket
|
||||
src/web_ui.rs per-container axum HTTP page (label + placeholder)
|
||||
src/bin/hive-ag3nt.rs sub-agent CLI (serve/send/recv); turn loop + web UI
|
||||
src/bin/hive-m1nd.rs manager CLI (serve/send/recv/spawn/kill/
|
||||
request-apply-commit); recognises HelperEvent
|
||||
|
||||
hive-sh4re/ wire types (HostRequest/Response, AgentRequest/Response, Message)
|
||||
hive-sh4re/ wire types (HostRequest/Response, AgentRequest/Response,
|
||||
ManagerRequest/Response, Message, Approval, HelperEvent)
|
||||
|
||||
nix/
|
||||
modules/hive-c0re.nix systemd service wiring
|
||||
templates/agent-base.nix nixos-container template (boot.isNspawnContainer = true)
|
||||
modules/hive-c0re.nix systemd service + firewall + git path wiring
|
||||
templates/agent-base.nix sub-agent nixos-container template
|
||||
templates/manager.nix manager nixos-container template
|
||||
|
||||
tests/roundtrip.sh Phase 3 end-to-end smoke test
|
||||
tests/roundtrip.sh Phase 3 messaging round-trip
|
||||
tests/approval.sh Phase 5 end-to-end approval flow
|
||||
tests/dashboard.sh Phase 6+7 HTTP dashboard + SSE + orphan GC
|
||||
|
||||
docs/damocles-migration.md options for moving damocles onto hyperhive
|
||||
```
|
||||
|
||||
## Conventions
|
||||
|
||||
- **Naming.** Containers are length-bounded (`nixos-container` ≤ 11 chars).
|
||||
Sub-agents are `h-<name>` with `<name>` ≤ 9 chars; the manager is `hm1nd`.
|
||||
`MAX_AGENT_NAME` enforces the cap in `lifecycle.rs`.
|
||||
`MAX_AGENT_NAME` enforces the cap in `lifecycle.rs`. Per-agent web UI port =
|
||||
`WEB_PORT_BASE + FNV1a(name) % WEB_PORT_RANGE` (8100..8999); manager fixed
|
||||
at 8000; dashboard `cfg.dashboardPort` (default 7000).
|
||||
- **Identity = socket.** No auth/tokens on the per-agent sockets. The socket
|
||||
*path* identifies the principal; perms come from "who has the bind-mount."
|
||||
- **Wire protocol.** JSON line-delimited over unix sockets in both directions.
|
||||
See `hive-sh4re` for the types. (Phase 6+ may swap to real MCP stdio.)
|
||||
- **Wire protocol.** JSON line-delimited over unix sockets in both directions
|
||||
(host admin / manager / agent). `/messages/stream` is `text/event-stream`.
|
||||
- **Commit messages.** Short, lowercase, no Co-Authored-By trailer.
|
||||
- **Commit before test.** Stage and commit when work *looks* ready, then run
|
||||
validation (`cargo check`, `nix flake check`, real lpt2 deploy). Failures get
|
||||
a follow-up commit rather than an amend.
|
||||
- **`rebuild` is the reconcile verb.** It rewrites `/etc/nixos-containers/<C>.conf`
|
||||
EXTRA_NSPAWN_FLAGS idempotently *and* does `nixos-container update` *and*
|
||||
stop+start so nspawn-level changes (bind mounts) take effect. Anything that
|
||||
changes per-container state on the host should be re-applied here.
|
||||
- **`rebuild` is the reconcile verb.** Idempotently rewrites
|
||||
`/etc/nixos-containers/<C>.conf` (`PRIVATE_NETWORK=0`, clears
|
||||
HOST_ADDRESS/LOCAL_ADDRESS, sets `EXTRA_NSPAWN_FLAGS`), regenerates
|
||||
`applied/<name>/flake.nix`, writes the systemd limits drop-in, then
|
||||
`nixos-container update` + stop + start. Anything that changes per-container
|
||||
state on the host should be re-applied here.
|
||||
- **Actions are factored.** `approve` / `deny` live in `actions.rs`; the admin
|
||||
socket and the dashboard POST handlers both call into them, so the two
|
||||
surfaces never drift.
|
||||
|
||||
## Gotchas / lessons learned
|
||||
|
||||
|
|
@ -79,16 +114,24 @@ tests/roundtrip.sh Phase 3 end-to-end smoke test
|
|||
`EXTRA_NSPAWN_FLAGS` in `/etc/nixos-containers/<NAME>.conf` — the start
|
||||
script (`/nix/store/.../container_-start`) expands it unquoted into the
|
||||
`systemd-nspawn` invocation. We rewrite this line in `set_nspawn_flags()`.
|
||||
- **`/run/systemd/nspawn/*.nspawn` overrides are *ignored*** by `nixos-container`'s
|
||||
start script (it builds the nspawn cmd line directly). Don't bother.
|
||||
- **`boot.isNspawnContainer = true`**, not `boot.isContainer = true`. The
|
||||
latter was renamed in nixos-25.11+.
|
||||
- **`/run/systemd/nspawn/*.nspawn` overrides are *ignored*** by
|
||||
`nixos-container`'s start script (it builds the nspawn cmd line directly).
|
||||
- **`boot.isNspawnContainer = true`**, not `boot.isContainer = true`. Renamed
|
||||
in nixos-25.11+.
|
||||
- **`nixos-container create` auto-assigns `HOST_ADDRESS`/`LOCAL_ADDRESS`** in
|
||||
the `.conf`. The start script's `if HOST_ADDRESS set → --network-veth`
|
||||
branch then forces a private netns — which is silently fatal for our web
|
||||
UIs (the bind is invisible from the host). We force-clear those vars (and
|
||||
`HOST_ADDRESS6` / `LOCAL_ADDRESS6` / `HOST_BRIDGE`) plus set
|
||||
`PRIVATE_NETWORK=0`.
|
||||
- **systemd service PATH ≠ host PATH.** Our service explicitly sets
|
||||
`path = [ "/run/current-system/sw" ]` so `nixos-container` (which lives in
|
||||
the system profile, not nixpkgs) is reachable.
|
||||
`path = [ pkgs.git "/run/current-system/sw" ]`. Additionally,
|
||||
`environment.HYPERHIVE_GIT = "${pkgs.git}/bin/git"` bakes the absolute path
|
||||
in (read by `lifecycle::git_command()`) so git resolution doesn't depend on
|
||||
PATH plumbing at all.
|
||||
- **`RuntimeDirectoryPreserve = "yes"`** keeps `/run/hyperhive/` (and the
|
||||
agent sub-dirs) across `hive-c0re` restarts. Without it, every restart wipes
|
||||
bind sources and existing containers can't be started.
|
||||
per-agent sub-dirs) across `hive-c0re` restarts. Without it, every restart
|
||||
wipes bind sources and existing containers can't be started.
|
||||
- **`register_agent` is idempotent** — drops any prior socket task before
|
||||
rebinding. Required so a `hive-c0re` restart followed by `rebuild alice`
|
||||
recreates the agent's socket without needing a clean reinstall.
|
||||
|
|
@ -102,18 +145,23 @@ tests/roundtrip.sh Phase 3 end-to-end smoke test
|
|||
`claude --print` fails. Future: bind-mount a shared `~/.claude` dir from the
|
||||
host so creds survive container destroy/recreate.
|
||||
- **Echo guard.** `hive-ag3nt serve` skips auto-reply when the incoming body
|
||||
starts with `"echo: "`. Prevents ping-pong loops when both sides fall back to
|
||||
echo. Real conversations between claude-backed agents *will* runaway — that's
|
||||
the manager's job to bound (Phase 4+).
|
||||
starts with `"echo: "`. Prevents ping-pong loops when both sides fall back
|
||||
to echo. Real conversations between claude-backed agents *will* runaway —
|
||||
bounding them is the manager's job.
|
||||
- **Orphan approvals.** If state dirs are wiped out from under a pending
|
||||
approval (test scripts, manual `rm -rf`), the dashboard's next render
|
||||
marks them `failed` with note `"agent state dir missing"` so they fall out
|
||||
of `pending`. They stay in sqlite for audit.
|
||||
|
||||
## Build / deploy / test
|
||||
|
||||
```sh
|
||||
# inside the repo (devshell first; no global cargo)
|
||||
nix develop -c cargo check
|
||||
nix develop -c cargo clippy --workspace --all-targets -- -D warnings
|
||||
nix develop -c cargo build
|
||||
|
||||
# evaluate everything (incl. fmt check)
|
||||
# evaluate everything (incl. rust+nix+toml fmt + clippy)
|
||||
nix flake check
|
||||
|
||||
# build only the workspace package
|
||||
|
|
@ -124,69 +172,74 @@ nix build .#default
|
|||
cd ~/Repos/<nixos-config-repo>
|
||||
nix flake update --update-input hyperhive
|
||||
sudo nixos-rebuild switch --flake .#<host>
|
||||
sudo systemctl restart hive-c0re # if only env/options changed
|
||||
|
||||
# end-to-end test (lpt2 or any host with the module enabled)
|
||||
sudo bash tests/roundtrip.sh
|
||||
# end-to-end tests (each idempotent; runs as root)
|
||||
sudo bash tests/roundtrip.sh # alice ↔ bob echo round-trip
|
||||
sudo bash tests/approval.sh # manager edit → request → user approve → rebuilt
|
||||
sudo bash tests/dashboard.sh # HTTP UI, approve POST, SSE, orphan GC
|
||||
```
|
||||
|
||||
The host config also needs `hyperhive.overlays.default` applied — the module's
|
||||
default `package = pkgs.hyperhive` requires the overlay to bring the package
|
||||
in.
|
||||
in. The `claude-unstable` overlay is applied internally to per-agent flakes
|
||||
already.
|
||||
|
||||
## Phase status
|
||||
|
||||
- ✅ Phase 0 — repo + Cargo workspace + flake + agent-base template
|
||||
- ✅ Phase 1 — container lifecycle (spawn/kill/rebuild/list); nixos-container update
|
||||
hot-reload works under the patch stack (validated empirically on muede-lpt2)
|
||||
- ✅ Phase 2 — per-agent sockets, in-memory broker, agent harness round-trips messages
|
||||
- ✅ Phase 3 — sqlite broker (durable across restart) + claude-or-echo turn loop
|
||||
- ✅ Phase 4 — `hm1nd` manager binary + manager socket + declarative `containers.hm1nd`
|
||||
- ✅ Phase 5 — git-commit approval flow:
|
||||
- ✅ Phase 1 — container lifecycle; `nixos-container update` hot-reload works
|
||||
under the patch stack (validated on muede-lpt2)
|
||||
- ✅ Phase 2 — per-agent sockets, in-memory broker, agent harness round-trips
|
||||
- ✅ Phase 3 — sqlite broker (durable) + claude-or-echo turn loop
|
||||
- ✅ Phase 4 — `hm1nd` manager binary + manager socket + declarative
|
||||
`containers.hm1nd`
|
||||
- ✅ Phase 5 — git-commit approval flow
|
||||
- 5a — sqlite approval queue (`request_apply_commit`/`pending`/`approve`/`deny`)
|
||||
- 5b — per-agent config flakes (proposed + applied repos)
|
||||
- 5c — split: manager edits `proposed`, hive-c0re writes-only `applied`; the
|
||||
container builds from `applied`. Approve = read `agent.nix` at the
|
||||
approved commit from `proposed`, copy into `applied`, commit + rebuild.
|
||||
Manager cannot move `main` on its own.
|
||||
- ✅ Phase 6 — per-container web UIs + hive-c0re dashboard:
|
||||
- Each `hive-ag3nt` / `hive-m1nd` serves an `axum` HTTP page on `HIVE_PORT`
|
||||
(deterministic hash for sub-agents in 8100–8999; fixed 8000 for the manager).
|
||||
Vibec0re-styled placeholder for now (status / inbox / xterm coming later).
|
||||
- `hive-c0re` serves a dashboard on `cfg.dashboardPort` (default 7000)
|
||||
listing containers (deep-linked to their per-container UI) + pending
|
||||
approvals. Same aesthetic.
|
||||
- Firewall opens 7000, 8000, 8100–8999 when the module is enabled.
|
||||
- 5b — per-agent config flakes
|
||||
- 5c — manager edits `proposed`, hive-c0re writes-only `applied`; container
|
||||
builds from `applied`. Approve = read `agent.nix` at the approved commit
|
||||
from `proposed`, copy into `applied`, commit + rebuild. Manager cannot
|
||||
move `applied/main` on its own.
|
||||
- ✅ Phase 6 — per-container web UIs (`HIVE_PORT` deterministic-hash) +
|
||||
hive-c0re dashboard (default 7000, vibec0re aesthetic, deep-linked)
|
||||
- ✅ Phase 7 — polish:
|
||||
- 7a — dashboard `POST /approve/<id>` / `/deny/<id>` buttons + unified
|
||||
diff (via `similar`) of applied vs proposed `agent.nix`.
|
||||
- 7b — broker broadcast channel + `/messages/stream` SSE + live message-flow
|
||||
panel (cyan `→` sent / green `✓` delivered, 200 row cap).
|
||||
- 7c — `ApprovalResolved` helper events into the manager's inbox
|
||||
(`SYSTEM_SENDER` + `HelperEvent` JSON). Manager harness recognises and
|
||||
logs them distinctly.
|
||||
- 7d — default `MemoryMax=2G` + `CPUQuota=50%` applied to every managed
|
||||
container via `/run/systemd/system/container@<NAME>.service.d/hyperhive-limits.conf`
|
||||
drop-in (regenerated on every spawn / rebuild).
|
||||
- 7e — damocles migration plan (`docs/damocles-migration.md`).
|
||||
- 7a — dashboard Approve/Deny buttons + unified diff (`similar` crate)
|
||||
- 7b — broker broadcast + `/messages/stream` SSE + live message-flow panel
|
||||
- 7c — `ApprovalResolved` helper events into manager inbox
|
||||
- 7d — `MemoryMax=2G` + `CPUQuota=50%` systemd drop-in per container
|
||||
- 7e — damocles migration plan (`docs/damocles-migration.md`)
|
||||
- ✅ Phase 7 follow-ups:
|
||||
- Dashboard **T4LK** form — operator can send messages from the browser
|
||||
(`POST /send`, becomes `from: "operator"` broker message)
|
||||
- Orphan-approval GC on dashboard render (stale entries auto-failed)
|
||||
- `PRIVATE_NETWORK=0` + `HOST_ADDRESS=`/`LOCAL_ADDRESS=` cleared in
|
||||
`set_nspawn_flags` so sub-agent web UI ports are reachable on the host
|
||||
- `HYPERHIVE_GIT` env var (absolute path) bypasses PATH ambiguity
|
||||
|
||||
## Approval flow (Phase 5)
|
||||
## Approval flow
|
||||
|
||||
End-to-end: manager edits per-agent config repo → commits → submits commit sha
|
||||
for approval → user approves on host CLI → `hive-c0re` advances `main` + rebuilds.
|
||||
End-to-end: manager edits per-agent `proposed` repo → commits → submits commit
|
||||
sha → user approves on host CLI **or** dashboard button → `hive-c0re` reads the
|
||||
file at that sha from `proposed`, applies into `applied`, commits there, runs
|
||||
`nixos-container update`. Helper-event JSON lands in the manager's inbox.
|
||||
|
||||
```
|
||||
# Inside the hm1nd container (manager has /agents bind-mounted RW):
|
||||
cd /agents/alice/config
|
||||
$EDITOR agent.nix # add `environment.systemPackages = [ pkgs.htop ];`
|
||||
$EDITOR agent.nix # e.g. environment.systemPackages = [ pkgs.htop ];
|
||||
git commit -am "add htop"
|
||||
SHA=$(git rev-parse HEAD)
|
||||
hive-m1nd request-apply-commit alice $SHA
|
||||
exit
|
||||
|
||||
# On the host:
|
||||
sudo hive-c0re pending # shows the queued approval with id N
|
||||
sudo hive-c0re approve N # validates, advances main, rebuilds h-alice
|
||||
sudo nixos-container run h-alice -- which htop # /run/current-system/sw/bin/htop
|
||||
# On the host (CLI):
|
||||
sudo hive-c0re pending # shows queued approval with id N
|
||||
sudo hive-c0re approve N # validates, applies, rebuilds
|
||||
sudo nixos-container run h-alice -- which htop
|
||||
|
||||
# Or on the dashboard (browser):
|
||||
http://<host>:7000/ # ◆ APPR0VE button next to the diff
|
||||
```
|
||||
|
||||
Per-agent layout — two separate git repos:
|
||||
|
|
@ -206,21 +259,33 @@ Per-agent layout — two separate git repos:
|
|||
|
||||
The container's `--flake` ref is `<applied_dir>#default`. The flake's
|
||||
`nixosConfigurations.default` extends `hyperhive.nixosConfigurations.agent-base`
|
||||
with `./agent.nix` plus an inline module setting `environment.etc."gitconfig".text`
|
||||
with the agent's name as the git committer identity.
|
||||
with `./agent.nix` plus an inline module that sets
|
||||
`environment.etc."gitconfig".text` (committer identity = the agent's name) and
|
||||
`systemd.services.hive-ag3nt.environment.HIVE_PORT`/`HIVE_LABEL`.
|
||||
|
||||
On approve: `git show <commit>:agent.nix` from `proposed/<name>`, write the bytes
|
||||
into `applied/<name>/agent.nix`, commit there as `hive-c0re`, then
|
||||
`nixos-container update`. The manager can only propose; only hive-c0re advances
|
||||
`applied`'s `main`.
|
||||
## Polish backlog
|
||||
|
||||
See PLAN.md for the full design and the deferred-out-of-scope list.
|
||||
Not phased — pick when relevant:
|
||||
|
||||
- **Operator inbox view** — drain replies addressed to `operator` and show
|
||||
in the dashboard (today they accumulate in sqlite unread).
|
||||
- **Per-agent UI substance** — show last N inbox messages, last turn timing,
|
||||
link back to dashboard.
|
||||
- **xterm.js terminal** — embed in each per-container UI, attach to a PTY
|
||||
exposed by the harness.
|
||||
- **`destroy` verb** — currently `nixos-container destroy` + manual `rm -rf`.
|
||||
Should be one hive-c0re verb that also purges approvals + state dirs.
|
||||
- **Bounded broker** — cap rows per recipient or auto-vacuum delivered
|
||||
messages older than a threshold.
|
||||
- **Container crash events** — watch `container@*.service` via D-Bus,
|
||||
push `HelperEvent::ContainerCrash` to the manager.
|
||||
|
||||
## Inspirations
|
||||
|
||||
- **`~/Repos/bitburner-agent`** — sibling project, drives Claude Code in a turn
|
||||
loop against a Bitburner CDP session. Patterns to steal as we grow:
|
||||
- **`~/Repos/bitburner-agent`** — sibling project, drives Claude Code in a
|
||||
turn loop against a Bitburner CDP session. Patterns to steal as we grow:
|
||||
per-cycle prompt diffing (vs full state), notes compaction as a separate
|
||||
short-lived Claude session, MCP server registering tools from a single
|
||||
`TOOLS` array, dashboard with SSE + xterm.js + sqlite stats sampler, opaque
|
||||
"terminal event" stream that unifies tool-call / sleep / op-notice / etc.
|
||||
`TOOLS` array, dashboard with SSE + xterm.js + sqlite stats sampler,
|
||||
opaque "terminal event" stream that unifies tool-call / sleep / op-notice
|
||||
/ etc.
|
||||
|
|
|
|||
64
PLAN.md
64
PLAN.md
|
|
@ -1,6 +1,9 @@
|
|||
# hyperhive — Plan
|
||||
|
||||
> **Status.** Planning doc for a new project. Lives at `~/Repos/nixos-configuration-claude/agent-system/PLAN.md` during planning; **moves to `~/Repos/hyperhive/PLAN.md`** once the repo is created.
|
||||
> **Status.** All phases 0–7 have shipped. This file is the original design
|
||||
> doc; **CLAUDE.md** is the current source of truth for what's actually built,
|
||||
> the file map, gotchas, and operator runbook. Keep this file for the *why*
|
||||
> and the original phase rationale; CLAUDE.md for *how things are today*.
|
||||
>
|
||||
> **Names.**
|
||||
> - Repo: `hyperhive`
|
||||
|
|
@ -10,7 +13,7 @@
|
|||
> - `hive-m1nd` — runs in the manager container (same crate, second `main.rs`, wires the manager tool surface).
|
||||
> - Shared crate between `hive-c0re` and `hive-ag3nt` (wire protocol, MCP verb types, message shapes): **`hive-sh4re`**.
|
||||
>
|
||||
> **Relationship to damocles.** Damocles is a separate, currently-running setup. `hyperhive` is a new, independent system in its own repo. Damocles' existing `claude-container.nix` informs the agent-base template but is not a dependency. Eventually damocles migrates onto `hyperhive` — out of v1 scope.
|
||||
> **Relationship to damocles.** Damocles is a separate, currently-running setup. `hyperhive` is a new, independent system in its own repo. Damocles' existing `claude-container.nix` informs the agent-base template but is not a dependency. Migration options laid out in `docs/damocles-migration.md`; recommendation is to keep them separate for now.
|
||||
|
||||
## What we're building
|
||||
|
||||
|
|
@ -109,39 +112,43 @@ A multi-Claude-Code-agent setup on a single host:
|
|||
|
||||
## Phased path
|
||||
|
||||
### Phase 0 — repo bootstrap
|
||||
All phases ✅ shipped. Each section below is the *original* design with notes
|
||||
on what actually landed and what deviated. See CLAUDE.md → "Phase status" for
|
||||
the canonical summary.
|
||||
|
||||
### ✅ Phase 0 — repo bootstrap
|
||||
- Create `~/Repos/hyperhive/`, init flake.
|
||||
- Cargo workspace: `hive-c0re/`, `hive-sh4re/`, `hive-ag3nt/` (the last with two `[[bin]]` targets — `hive-ag3nt` and `hive-m1nd`). All compile, all do nothing useful.
|
||||
- NixOS module skeleton (`nix/modules/hive-c0re.nix`) that runs the daemon as a systemd service on the host.
|
||||
- Agent base template (`nix/templates/agent-base.nix`) that builds a nixos-container including the `hive-ag3nt` binary.
|
||||
- **Exit:** `nixos-container create test-agent --flake .#agent-base && nixos-container start test-agent` brings up a container whose `hive-ag3nt` prints "hello" and exits.
|
||||
|
||||
### Phase 1 — container lifecycle + Risk 1
|
||||
### ✅ Phase 1 — container lifecycle + Risk 1
|
||||
- `hive-c0re`: open host admin socket (`/run/hyperhive/host.sock`); verbs `spawn(name)`, `kill(name)`, `rebuild(name)`, `list()`. Uses `nixos-container` underneath; container name on the host is `h-<name>` (sub-agents) or `hm1nd` (manager).
|
||||
- CLI tool talking to the admin socket (same `hive-c0re` binary, subcommand-driven).
|
||||
- Manually mutate an agent's config flake, call `rebuild`, observe whether `hive-ag3nt` survives.
|
||||
- **Decision:** if hot-reload doesn't preserve the harness, that becomes a hard requirement of `hive-ag3nt`'s design (resume from disk state). Document the outcome.
|
||||
- **Exit:** spawn / rebuild / kill via CLI is reliable; known behaviour for in-flight rebuilds.
|
||||
|
||||
### Phase 2 — sockets + minimal MCP
|
||||
### ✅ Phase 2 — sockets + minimal MCP
|
||||
- `hive-c0re` opens `manager.sock` and `agents/<name>.sock` (one per spawned agent). Per-socket MCP server with the right tool surface baked in. Types from `hive-sh4re`.
|
||||
- `hive-ag3nt`: MCP client (types from `hive-sh4re`), connects to its socket on startup, exchanges hello.
|
||||
- Tools: agent gets `send(to, body)`, `recv()`. No persistence yet (in-memory).
|
||||
- **Exit:** two test agents exchange messages through `hive-c0re` manually-driven.
|
||||
|
||||
### Phase 3 — broker + turn loop
|
||||
### ✅ Phase 3 — broker + turn loop
|
||||
- `hive-c0re`: sqlite-backed message store (`messages` table; `id, sender, recipient, body, sent_at, delivered_at`). Survives `hive-c0re` restart.
|
||||
- `hive-ag3nt` (lib): real turn loop. Reads from `recv`; feeds new messages as user turns to `claude`; captures output; calls `send` for outbound. Long-running.
|
||||
- **Exit:** two `hive-ag3nt`-driven agents have a back-and-forth conversation through `hive-c0re`.
|
||||
|
||||
### Phase 4 — `hive-m1nd` + privileged surface
|
||||
### ✅ Phase 4 — `hive-m1nd` + privileged surface
|
||||
- `hive-m1nd` binary (second `[[bin]]` in `hive-ag3nt`) wires the manager tool surface.
|
||||
- Manager container (`hm1nd`) declared in host NixOS module (auto-restart). Bind-mount `agents/**` RW.
|
||||
- Manager socket gets the privileged tool surface: `request_spawn`/`request_kill`, `request_apply_commit`, `inject_peer_info`, `send(..., wait_for_reply=true)`.
|
||||
- Smoke: attach a terminal to the manager container (`nixos-container root-login`); ask `hive-m1nd` to spawn an agent and route a message to it.
|
||||
- **Exit:** manager spawns, routes, kills a child agent end-to-end; lifecycle still gated by manual CLI approval (no GUI yet).
|
||||
|
||||
### Phase 5 — git-commit approval flow
|
||||
### ✅ Phase 5 — git-commit approval flow
|
||||
- `state-repo` on host tracks world (agents directory listing, allow-lists, etc.).
|
||||
- Per-agent `config/` flake repos created at spawn time.
|
||||
- Manager's container: bind-mounted clones; uses plain `git` CLI to edit/commit.
|
||||
|
|
@ -149,13 +156,13 @@ A multi-Claude-Code-agent setup on a single host:
|
|||
- Per-agent allow-list for `request_install`: in-list installs become auto-applied commits; novel pkgs become pending commits.
|
||||
- **Exit:** manager adds a package to an agent → user approves on CLI → agent picks it up.
|
||||
|
||||
### Phase 6 — per-agent web UI + dashboard MVP
|
||||
### ✅ Phase 6 — per-agent web UI + dashboard MVP
|
||||
- `hive-ag3nt` web UI module (in the crate's lib): HTTP on a per-container host port (host network): status, last messages, embedded terminal (xterm.js over WebSocket). Both `hive-ag3nt` and `hive-m1nd` binaries expose it.
|
||||
- Dashboard served by `hive-c0re`: agent list, per-agent status, links to each agent's UI, link to manager's UI.
|
||||
- No approval UI yet; users still approve via CLI.
|
||||
- **Exit:** browser is a usable navigation layer over the whole system.
|
||||
|
||||
### Phase 7 — dashboard commit view + polish
|
||||
### ✅ Phase 7 — dashboard commit view + polish
|
||||
- Pending-commits view in the dashboard with diff rendering and Approve/Deny buttons (replaces the CLI approve step).
|
||||
- Live message-flow view (`hive-c0re` sees all MCP relay traffic).
|
||||
- `hive-c0re` event push into `hive-m1nd`'s `next_event` (crashes, OOM, approval resolved).
|
||||
|
|
@ -204,19 +211,36 @@ A multi-Claude-Code-agent setup on a single host:
|
|||
└── PLAN.md # this file
|
||||
```
|
||||
|
||||
## Resolved implementation decisions
|
||||
|
||||
The original open-decisions list, with what we picked:
|
||||
|
||||
- **Wire format.** Custom JSON-line over unix sockets (host admin / manager /
|
||||
per-agent), not real MCP stdio. Simpler and good enough for now; can swap
|
||||
to MCP later. SSE for the dashboard message-flow.
|
||||
- **Per-agent web UI.** `axum` HTTP server inside each container at a port
|
||||
hashed from the agent name (8100–8999); manager at fixed 8000; dashboard
|
||||
at 7000. Plain HTML, no HTMX, no xterm.js yet.
|
||||
- **`state-repo` schema.** Per-agent dir with files; not a single TOML.
|
||||
Realised as two parallel git repos per agent: `proposed` (manager-editable)
|
||||
and `applied` (hive-c0re-only). Container builds from `applied`.
|
||||
- **Manager access to applied state.** *Not* RW-mounted. Manager only has
|
||||
`proposed/` bind-mounted; `applied/` is hive-c0re-only.
|
||||
- **One binary or two.** One: `hive-c0re` is daemon + CLI dispatched by
|
||||
subcommand (`serve` / `spawn` / `kill` / `rebuild` / `list` / `pending` /
|
||||
`approve` / `deny`).
|
||||
|
||||
## Polish backlog (not phased)
|
||||
|
||||
See CLAUDE.md → "Polish backlog" for the live list. Highlights: operator
|
||||
inbox drain, per-agent UI substance, xterm.js terminal embed, `destroy` verb,
|
||||
bounded broker, container-crash events via D-Bus.
|
||||
|
||||
## Explicitly deferred / out of v1 scope
|
||||
|
||||
- Per-agent API keys, cost attribution.
|
||||
- Pooled / pre-warmed containers.
|
||||
- Destroy verb on the `hive-c0re` API (use `rm` on host; `state-repo` records intent).
|
||||
- Destroy verb on the `hive-c0re` API (use `rm` on host).
|
||||
- Backup / replication of `agents/` state.
|
||||
- Migration of existing damocles containers (later, separate effort).
|
||||
- Migration of existing damocles containers (`docs/damocles-migration.md`).
|
||||
- Anything about multiple hosts.
|
||||
|
||||
## Open implementation decisions (resolve during build)
|
||||
|
||||
- MCP relay wire format: stdio MCP shuttled through the socket vs custom JSON-RPC matching MCP semantics. Default: stick with MCP stdio, `hive-c0re` multiplexes.
|
||||
- Per-agent web UI tech: own small `axum` server in `hive-ag3nt`'s lib with embedded HTMX, plus xterm.js over WebSocket for the terminal. Reuse same stack as the dashboard.
|
||||
- `state-repo` schema: filesystem-shaped (`agents/<name>/role.txt`, `agents/<name>/allow-list.txt`) vs single declarative file (`world.toml`). Lean filesystem-shaped — git diffs read naturally per agent.
|
||||
- Whether `hive-m1nd` also auto-mounts `state-repo` RW or only via the `request_apply_commit` path. Lean: only via the verb (keeps the audit trail clean).
|
||||
- Whether `hive-c0re` daemon and `hive-c0re` CLI are one binary (subcommands) or two binaries sharing a crate. Default: one binary, `hive-c0re serve` vs `hive-c0re approve <sha>` etc.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue