7fc426b4dd
Adds swarmctl markdown-docs (a hidden Verb, same clap-markdown + hide=true shape as hivectl markdown-docs) and generates docs/tools/swarmctl-cli.md from it. Wires a swarmctl-docs freshness check into nix/checks.nix, same shape as hivectl-docs, diffing against packages.swarmctl. One real gotcha: PathArgs::resolve() reads required SWARMCTL_AUTHELIA_* deployment env vars and errors if unset - swarmctl markdown-docs must not go through that path (it needs none of those vars, and the docs build runs it outside any real deployment). Restructured main() so resolve() only runs for the User arm, not unconditionally before the match. Also links the new doc from docs/tools/README.md (new 'for the swarm operator' section), CLAUDE.md's swarmctl bullet, and docs/conventions.md's flake-check list. Verified: cargo check/clippy -D warnings/test/fmt -p swarmctl all clean; swarmctl markdown-docs diffs clean against the committed doc (checked against both a plain cargo build and the actual nix build. #swarmctl output); scripts/check-issue-refs.sh clean.
276 lines
16 KiB
Markdown
276 lines
16 KiB
Markdown
# hyperhive — claude entry point
|
||
|
||
Hey claude. This is your starting page. The detailed docs live in
|
||
[`docs/`](docs/) and are written for humans + you both — read them
|
||
when you need depth on a subsystem. This file is the index.
|
||
|
||
- High-level project intro: **[README.md](README.md)**.
|
||
- Open work + backlog: the forge issue tracker at
|
||
**`$HIVE_FORGE_URL/hyperhive/hyperhive/issues`**. Read the host out of
|
||
the env var — inside an agent container `localhost` is the *agent*, not
|
||
the forge, so a hardcoded loopback address fails to connect.
|
||
- Operator/agent trust-boundary design:
|
||
**[docs/boundary.md](docs/boundary.md)** (`area/ops` issues
|
||
for the deployment/gateway/privsep work — the separator is a **slash**;
|
||
`--label area:ops` matches nothing and silently returns *unfiltered*
|
||
results, which reads like a backlog rather than like an error).
|
||
- Agent trust model (trust boundary, prompt-injection threat model,
|
||
capability = accepted risk), credential isolation + sandbox threat model:
|
||
**[docs/security.md](docs/security.md)**.
|
||
|
||
## Repo map
|
||
|
||
One line per crate / top-level dir. **Each module's authoritative,
|
||
always-current description lives in its own `//!` doc-comment** —
|
||
`grep`/read the module when you need detail. This index is kept
|
||
deliberately lean: it auto-loads into every turn's context, and a
|
||
hand-maintained per-file tree drifts out of sync with the code.
|
||
|
||
### Rust workspace (`Cargo.toml` members)
|
||
|
||
- **`hive-c0re/`** — host daemon (runs as the unprivileged `hive-core`
|
||
user). `src/main.rs` is the `hive-c0re` binary — **daemon-only**
|
||
(`serve` + the periodic vacuum/sweep loops); the operator CLI lives in
|
||
the separate `hivectl` crate, which talks to the daemon over the host
|
||
admin socket. Owns the sqlite broker, approval + question + reminder +
|
||
schedule queues, the meta flake, lifecycle (`nixos-container`
|
||
shellouts), gateway / forge / matrix provisioning, per-container stats,
|
||
and the axum operator dashboard (`dashboard.rs`). Largest crate.
|
||
- **`hivectl/`** — standalone operator CLI (`hivectl` binary). Talks to
|
||
the `hive-c0re` daemon over the host admin socket (`hive-host-sock`
|
||
wire types) — does NOT link `hive-c0re`. Full, always-current verb
|
||
reference (CI-enforced against the clap tree, see `docs/conventions.md`):
|
||
[`docs/tools/hivectl-cli.md`](docs/tools/hivectl-cli.md).
|
||
- **`hive-agent/`**, **`hive-agent-mcp/`** —
|
||
in-container harness, two sibling crates for every agent (not a
|
||
single `hive-ag3nt/` dir — that's the runtime/binary-family nickname,
|
||
not a directory).
|
||
- **`hive-agent/`** — the serve-loop binary: turn-loop *policy* layer
|
||
(`turn.rs`) over the `hive-claude` driver, per-agent web UI (`web_ui/`
|
||
module dir), event + turn-stats sqlite sinks, login flow, system-prompt
|
||
renderer.
|
||
- **`hive-agent-mcp/`** — the embedded MCP server (long-lived
|
||
streamable-http listener, `hive-mcp-http` systemd unit) + its claude
|
||
launch-config layer (tool-group/capability → `--allowedTools`,
|
||
`--mcp-config` render).
|
||
- **`hive-jobq/`** — job-DAG scheduler, extracted from hive-c0re's
|
||
in-tree `job_queue` as a domain-agnostic library. **Runtime-only —
|
||
nothing writes the graph to disk**; hive-c0re starts empty each boot
|
||
and re-derives desired state via the reconcile sweep, so node ids and
|
||
timestamps are stable within a run, not across restarts. One
|
||
shared graph for the whole system (not a DAG per job); enqueuing
|
||
inserts a self-contained sub-DAG and returns the ids of the nodes the job
|
||
asked for, in the order it named them. Generic over
|
||
the node payload `N` and the resource name `R`; resource deps are named
|
||
counting semaphores acquired all-or-nothing at node start. `hive-c0re`'s
|
||
remaining `job_queue/` module is the c0re-specific layer *over* this
|
||
crate, and is being removed in favour of it — new scheduler-shaped code
|
||
belongs here, not there.
|
||
- **`hive-jobq-wire/`** — wire types for serving a `hive-jobq` graph to a
|
||
viewer, plus the `WireNode` / `WireResource` traits a host implements to
|
||
say how its `N` and `R` render. Deliberately *not* part of `hive-jobq`:
|
||
that crate is logic, this is presentation, and folded together they
|
||
remix. `GraphWire::wire_snapshot` is blanket-implemented for any
|
||
`Graph<N, R>` whose parameters implement both — so a payload that has
|
||
never said how it displays cannot reach a viewer at all.
|
||
- **`hive-screen-mcp/`** — stdio MCP bridge for GUI agents
|
||
(`hyperhive.gui.enable`): `screenshot` via `grim`, `type_text` /
|
||
`key_press` via `wtype` (Wayland virtual-keyboard protocol), and
|
||
`mouse_move` / `mouse_click` as RFB pointer events to the local neatvnc
|
||
server. All userspace, no daemon of its own.
|
||
- **`hive-priv/`** — minimal root privileged-helper, socket-activated at
|
||
`/run/hive/priv.sock`; performs the few root operations (bind-mount
|
||
edits, nsenter) the unprivileged `hive-c0re` delegates to it. See
|
||
`docs/boundary.md`.
|
||
- **`hive-forge/`** — `hive-forge` Forgejo CLI wrapper; one module per
|
||
verb under `src/verbs/`.
|
||
- **`hive-forge-notify/`** — per-agent notification poller daemons; turns
|
||
unread notification threads into todos on the harness's in-agent
|
||
socket. Two binaries from one crate: `hive-forge-notify` (the hive's
|
||
Forgejo) and `hive-github-notify` (github.com, installed by
|
||
`nix/agent-modules/github.nix`). Was a task inside the `hive-agent`
|
||
serve loop; own process since it needs nothing else from the harness.
|
||
- **`hive-matrix-mcp/`** — per-agent matrix-sdk daemon
|
||
(`hive-matrix-daemon`); serves its MCP tools (`send_message`,
|
||
`read_room`, …) directly over streamable-http (no stdio bridge),
|
||
same shape as `hive-bash-mcp`.
|
||
- **`hive-bash-mcp/`** — per-agent bash-task runner daemon
|
||
(`hive-bash-daemon`); serves its MCP tools (`run`/`status`/`kill`)
|
||
directly over streamable-http (no stdio bridge), writes task files
|
||
under `/harness/bash-tasks/`, and records the favorite-tools
|
||
`bash_commands` stat into turn-stats.sqlite.
|
||
- **`hive-sh4re/`** — shared wire types (Agent / Manager request +
|
||
response, `Message`, `Approval`, `HelperEvent`) used across the unix
|
||
sockets. Host-admin-socket and hive-priv-socket wire types have been
|
||
split out into their own crates (below) so `hivectl` and `hive-priv`
|
||
don't need to pull in the rest of `hive-sh4re`.
|
||
- **`hive-host-sock/`** — wire types for the host admin socket
|
||
(`/run/hyperhive/host.sock`), the protocol `hivectl` speaks to
|
||
`hive-c0re`. Split out of `hive-sh4re` so a standalone `hivectl` only
|
||
depends on this protocol crate, not the whole daemon crate.
|
||
- **`hive-priv-sock/`** — wire types for the `hive-priv` privileged-helper
|
||
socket (`/run/hive/priv.sock`), shared by `hive-priv` (server) and
|
||
`hive-c0re` (client). Also split out of `hive-sh4re`.
|
||
- **`hive-core-agent-sock/`** — wire types for the *host*-served per-agent
|
||
+ manager socket (`/run/hive/mcp.sock`), the protocol an agent's harness
|
||
speaks to `hive-c0re`. Same split rationale as the two above; the shared
|
||
payload types it references stay in `hive-sh4re`.
|
||
- **`hive-agent-sock/`** — wire types for the *in-agent* socket, served by
|
||
the harness to in-container producers — matrix / bash MCP daemons and
|
||
`forge_notify` are the built-in ones, but any user-configured MCP server
|
||
can push todos here too, nothing restricts the `subsystem` set. Carries
|
||
the loose-ends-v2 **todo** ops plus
|
||
harness-local reminders + questions. ⚠️ Distinct from
|
||
`hive-core-agent-sock` above: **this socket never leaves the container**
|
||
and `hive-c0re` is not in the path at all — no broker round-trip, no
|
||
long-poll, no marker files.
|
||
- **`hive-types/`** — zero-dependency (bar serde) leaf crate holding the
|
||
foundational newtypes, chiefly `Ident` (1–63 chars of `[a-z0-9-]`,
|
||
constructed only via the validating parser). Lets every wire-type crate
|
||
and both binaries type their agent-name fields as a validated ident and
|
||
get serde-checked parsing at the socket boundary, without coupling to
|
||
`hive-sh4re`.
|
||
- **`hive-sock-client/`** — the shared JSON-line-over-unix-socket client
|
||
every daemon uses to talk to a hyperhive socket. Generic over the
|
||
request/response types, so the host-served control socket and the
|
||
harness's in-agent socket both use it with their own wire-type crates.
|
||
Retry is a policy value (`Retry::None` for callers already inside a poll
|
||
loop, `Retry::RideOutRestart` for callers with no natural retry), and
|
||
the response is either decoded (`request`) or drained (`notify`).
|
||
Deliberately separate from the `*-sock` crates — those stay
|
||
dependency-free wire types.
|
||
- **`hive-metric/`** — small CLI to push a single labeled metric to the
|
||
OTEL collector via the OpenTelemetry Rust SDK / OTLP HTTP exporter.
|
||
- **`swarm-controller/`** — swarm-level daemon, opt-in per host
|
||
(`services.hyperhive.swarm.controller.enable`). Where `hive-c0re` owns
|
||
the agents on **one** host, this owns what is true **across** hives; a
|
||
swarm runs one of them, so most hives leave it off. Serves HTTP over a
|
||
unix socket the gateway's nginx proxies to — ⚠️ **the socket's
|
||
directory is its access control**; the constraint that governs it is in
|
||
the crate's README, and a unit test pins the path.
|
||
- **`swarmctl/`** — swarm-level operator CLI, installed by the
|
||
swarm-controller module on the host that runs the daemon. Runs as
|
||
**root and acts directly** — no socket, no HTTP route, no priv helper;
|
||
the crate's README records why the rootless shape was examined and
|
||
rejected. Does not link `swarm-controller`, mirroring `hivectl` ÷
|
||
`hive-c0re`. ⚠️ Its user store is **two files, one authoritative**:
|
||
`users.json` is canonical, authelia's `users.yml` is a *rendered
|
||
artifact* that is written and never read back. Full, always-current
|
||
verb reference (CI-enforced against the clap tree, same pattern as
|
||
`hivectl`'s — see `docs/conventions.md`):
|
||
[`docs/tools/swarmctl-cli.md`](docs/tools/swarmctl-cli.md).
|
||
|
||
### External dependencies with no directory here
|
||
|
||
- **`hive-claude`** — reusable, app-agnostic driver for headless
|
||
`claude --print`: spawns the CLI, streams + classifies stream-json,
|
||
parses per-turn `Telemetry`, and drives a durable self-compacting
|
||
`InfiniteSession` (name + `SessionStore` + `CompactionPolicy`). Uses
|
||
`thiserror` (it's a library); the `hive-*` binaries consume it with
|
||
`anyhow`. ⚠️ **Its own repo, not a workspace member** — consumed as a
|
||
dependency in the root `Cargo.toml`, so there is no `hive-claude/`
|
||
directory to read and `grep` here will not find it.
|
||
|
||
### Other top-level dirs
|
||
|
||
- **`frontend/`** — npm workspaces → static dashboard + per-agent UI
|
||
dist, built hermetically by `nix/packages/frontend.nix`. Packages:
|
||
`shared` (terminal pane + Catppuccin palette), `dashboard` (the
|
||
operator SPA), `agent` (the default per-container UI), `swarm-ui`
|
||
(swarm-level UI shell — Preact + wouter + TypeScript + JSX, project-
|
||
bootstrap scope; packaged separately by `nix/packages/swarm-ui.nix`,
|
||
not bundled into `packages.default`'s closure — see that file).
|
||
- **`nix/`** — `host-modules/` (the host stack: hyperhive core options,
|
||
`hive-{c0re,priv,forge,gateway,matrix,network,tls,ci}`, otel, swarm),
|
||
`agent-modules/` (the per-agent harness feature modules),
|
||
`templates/{agent,ruth}.nix` (container entry points), `packages/`
|
||
(flake package outputs), `docs/` (the options-doc derivation), plus
|
||
`sources.nix` / `rust.nix` / `checks.nix` / `devshell.nix` /
|
||
`treefmt.nix` behind the thin `flake.nix`.
|
||
- **`docs/`** — subsystem reference docs (see *Reading paths* below).
|
||
- **`branding/`**, **`scripts/`** — static assets + helper scripts.
|
||
|
||
## Reading paths
|
||
|
||
Pick the doc that matches your task. None depend on the others —
|
||
read them à la carte.
|
||
|
||
- **"How do I bring a fresh hive online (first-run hivectl
|
||
bootstrap)?"** → [`docs/setup.md`](docs/setup.md).
|
||
- **"What does the dashboard look like?"** →
|
||
[`docs/web-ui.md`](docs/web-ui.md) (index; sub-pages:
|
||
[`shape`](docs/web-ui/shape.md),
|
||
[`dashboard`](docs/web-ui/dashboard.md),
|
||
[`agent`](docs/web-ui/agent.md)).
|
||
- **"How does the per-agent terminal classify + colour
|
||
events?"** → [`docs/terminal-rendering.md`](docs/terminal-rendering.md).
|
||
- **"How does claude get its prompt and what tools does it have?"** →
|
||
[`docs/turn-loop/`](docs/turn-loop/README.md) (index: the loop, binary shape,
|
||
turn outcomes; sub-pages:
|
||
[`claude-invocation`](docs/turn-loop/claude-invocation.md),
|
||
[`config`](docs/turn-loop/config.md), [`mcp`](docs/turn-loop/mcp.md)).
|
||
- **"How do config changes flow from manager to operator to
|
||
container?"** → [`docs/approvals.md`](docs/approvals.md).
|
||
- **"What state survives destroy / purge / restart?"** →
|
||
[`docs/persistence.md`](docs/persistence.md).
|
||
- **"Naming, commit style, wire protocol, the `data-async`
|
||
pattern."** → [`docs/conventions.md`](docs/conventions.md).
|
||
- **"Why does the nspawn flag look like that?"** →
|
||
[`docs/gotchas.md`](docs/gotchas.md).
|
||
- **"What nginx vhosts does the gateway serve? How does matrix
|
||
discovery work?"** → [`docs/gateway.md`](docs/gateway.md).
|
||
- **"How do per-agent forge accounts work? What does forge_notify
|
||
poll + how does it format wake messages?"** →
|
||
[`docs/forge.md`](docs/forge.md).
|
||
- **"What verbs does `hive-forge` support? How do I post a comment,
|
||
upload an attachment, manage subscriptions?"** →
|
||
[`docs/tools/forge.md`](docs/tools/forge.md).
|
||
- **"What does `hivectl` do? How do I provision a forge/matrix account,
|
||
manage gateway users, restart containers, or drop into an agent shell?"** →
|
||
[`docs/tools/hivectl.md`](docs/tools/hivectl.md).
|
||
- **"How does the matrix-tuwunel container work? What about
|
||
fluffychat-web and per-agent matrix accounts?"** →
|
||
[`docs/matrix.md`](docs/matrix.md).
|
||
- **"How do I give an agent a GitHub account (`gh` + `git push`)?
|
||
How is the PAT injected?"** → [`docs/github.md`](docs/github.md).
|
||
- **"How does DNS resolution work in agent containers? What's the
|
||
bridge network for?"** → [`docs/network.md`](docs/network.md).
|
||
- **"How do I connect two hives into a swarm? How do I declare peer
|
||
hives and configure TLS trust?"** →
|
||
[`docs/swarm/`](docs/swarm/README.md).
|
||
- **"Where do agent snapshots go? How does the swarm's `btrfs receive`
|
||
endpoint authenticate a pushing hive, and what does a snapshot
|
||
actually contain?"** →
|
||
[`docs/snapshot-store.md`](docs/snapshot-store.md).
|
||
- **"How does the rebuild queue work? What are queue kinds and sources?"** →
|
||
[`docs/coordinator.md`](docs/coordinator.md).
|
||
- **"How does the CI runner work? What's the auto-registration flow?"** →
|
||
[`docs/ci.md`](docs/ci.md).
|
||
- **"What is `/knowledge`? How does the hive-wide knowledge repo sync,
|
||
and how do I contribute a document?"** →
|
||
[`docs/knowledge.md`](docs/knowledge.md).
|
||
- **"How do I export Claude Code metrics (tokens, cost, tool calls) to
|
||
a Prometheus/Grafana collector? What OTEL options are available?"** →
|
||
[`docs/observability.md`](docs/observability.md).
|
||
- **"What does a PR review verdict actually gate? Should a reviewer
|
||
wait for CI before approving? What does arming auto-merge commit an
|
||
operator to?"** → [`docs/pr-review-gate.md`](docs/pr-review-gate.md).
|
||
|
||
## Conventions & process
|
||
|
||
The docs below own the details — this section just points at them.
|
||
|
||
- **Commit style, naming, identity, reconcile verb:** →
|
||
[`docs/conventions.md`](docs/conventions.md).
|
||
- **Never add `#[allow(clippy::…)]`** — fix the lint instead (extract a
|
||
helper, add backticks, etc.). Details + worked examples: →
|
||
[`docs/conventions.md`](docs/conventions.md#building--local-checks).
|
||
- **NixOS / nspawn quirks** (bind mounts, conf flags, etc.): →
|
||
[`docs/gotchas.md`](docs/gotchas.md).
|
||
- **Turn loop, sentinels (rate-limit, auth-failed), context
|
||
window:** → [`docs/turn-loop/`](docs/turn-loop/README.md).
|
||
- **Two-step spawn, approval flow, flake.lock validation:** →
|
||
[`docs/approvals.md`](docs/approvals.md).
|
||
- **Pre-push lint hook** (catches tracker-tag and comment-block failures
|
||
before CI does — install once per clone):
|
||
`ln -sf ../../scripts/pre-push .git/hooks/pre-push`
|