hyperhive/CLAUDE.md
atlas 30fa54cbc6 feat(swarmctl): add agent create, queueing the swarm-controller creation DAG
`swarmctl agent create <name> --hive <hive>` POSTs `/api/agents` to
swarm-controller over the daemon's unix socket and prints the queued
job's node id.

It deliberately does not wait. The endpoint queues a DAG whose last node
*publishes* a deploy message; the hive's `hive-c0re` then converges on
its own clock, out of the controller's sight. So even a fully settled
graph would not mean the agent is up, and there is nothing this CLI
could wait for that would let it claim otherwise. Printing the id is
exactly what the response says and all of what it says.

Transport is a bare hyper HTTP/1.1 client handshaked onto a tokio
`UnixStream` via `hyper_util::rt::TokioIo` — the same crate family
`hivectl/src/watch.rs` and `hive-agent/src/web_ui/proxy.rs` already use,
all of it already workspace-pinned. The request/response shapes are a
local mirror rather than a shared crate: the controller's own types are
private to its binary and this crate does not link it, the same
separation `hivectl` keeps from `hive-c0re`.

Errors are reduced to one actionable line — the controller answers
RFC 9457 problem+json, so an unknown `--hive` reaches the operator as
the roster of hives that would have worked rather than a body dump.
Response `warnings` are printed when non-empty.

The nix module wraps the binary with `SWARM_CONTROLLER_SOCKET`, read
from the same `socketPath` the daemon binds.

Refs #4399
2026-09-14 19:40:23 +02:00

15 KiB
Raw Blame History

hyperhive — claude entry point

Hey claude. This is your starting page. The detailed docs live in docs/ and are written for humans + you both — read them when you need depth on a subsystem. This file is the index.

  • High-level project intro: README.md.
  • Open work + backlog: the forge issue tracker at $HIVE_FORGE_URL/hyperhive/hyperhive/issues. Read the host out of the env var — inside an agent container localhost is the agent, not the forge, so a hardcoded loopback address fails to connect.
  • Operator/agent trust-boundary design: docs/trust-boundary/boundary.md (area/ops issues for the deployment/gateway/privsep work — the separator is a slash, and --label area:ops now fails with did you mean "area/ops"?).
  • Agent trust model (trust boundary, prompt-injection threat model, capability = accepted risk), credential isolation + sandbox threat model: docs/trust-boundary/security.md.

Repo map

One line per crate / top-level dir. Each module's authoritative, always-current description lives in its own //! doc-commentgrep/read the module when you need detail. This index is kept deliberately lean: it auto-loads into every turn's context, and a hand-maintained per-file tree drifts out of sync with the code.

Rust workspace (Cargo.toml members)

  • hive-c0re/ — host daemon (runs as the unprivileged hive-core user). src/main.rs is the hive-c0re binary — daemon-only (serve + the periodic vacuum/sweep loops); the operator CLI lives in the separate hivectl crate, which talks to the daemon over the host admin socket. Owns the sqlite broker, approval + reminder + schedule queues, the meta flake, lifecycle (nixos-container shellouts), gateway / forge / matrix provisioning, per-container stats, and the axum operator dashboard (dashboard.rs). Largest crate.
  • hivectl/ — standalone operator CLI (hivectl binary). Talks to the hive-c0re daemon over the host admin socket (hive-host-sock wire types) — does NOT link hive-c0re. Full, always-current verb reference (CI-enforced against the clap tree, see docs/process/conventions.md): docs/tools/hivectl-cli.md.
  • hive-agent/, hive-agent-mcp/ — in-container harness, two sibling crates for every agent (not a single hive-ag3nt/ dir — that's the runtime/binary-family nickname, not a directory).
  • hive-agent/ — the serve-loop binary: turn-loop policy layer (turn.rs) over the hive-claude driver, per-agent web UI (web_ui/ module dir), event + turn-stats sqlite sinks, login flow, system-prompt renderer.
  • hive-agent-mcp/ — the embedded MCP server (long-lived streamable-http listener, hive-mcp-http systemd unit) + its claude launch-config layer (tool-group/capability → --allowedTools, --mcp-config render).
  • hive-jobq/ — job-DAG scheduler, extracted from hive-c0re's in-tree job_queue as a domain-agnostic library. Runtime-only — nothing writes the graph to disk; hive-c0re starts empty each boot and re-derives desired state via the reconcile sweep, so node ids and timestamps are stable within a run, not across restarts. One shared graph for the whole system (not a DAG per job); enqueuing inserts a self-contained sub-DAG and returns the ids of the nodes the job asked for, in the order it named them. Generic over the node payload N and the resource name R; resource deps are named counting semaphores acquired all-or-nothing at node start. hive-c0re's remaining job_queue/ module is the c0re-specific layer over this crate, and is being removed in favour of it — new scheduler-shaped code belongs here, not there.
  • hive-jobq-wire/ — wire types for serving a hive-jobq graph to a viewer, plus the WireNode / WireResource traits a host implements to say how its N and R render. Deliberately not part of hive-jobq: that crate is logic, this is presentation, and folded together they remix. GraphWire::wire_snapshot is blanket-implemented for any Graph<N, R> whose parameters implement both — so a payload that has never said how it displays cannot reach a viewer at all.
  • hive-screen-mcp/ — stdio MCP bridge for GUI agents (hyperhive.gui.enable): screenshot via grim, type_text / key_press via wtype (Wayland virtual-keyboard protocol), and mouse_move / mouse_click as RFB pointer events to the local neatvnc server. All userspace, no daemon of its own.
  • hive-priv/ — minimal root privileged-helper, socket-activated at /run/hive/priv.sock; performs the few root operations (bind-mount edits, nsenter) the unprivileged hive-c0re delegates to it. See docs/trust-boundary/boundary.md.
  • hive-forge/hive-forge Forgejo CLI wrapper; one module per verb under src/verbs/.
  • hive-forge-notify/ — per-agent notification poller daemons; turns unread notification threads into todos on the harness's in-agent socket. Two binaries from one crate: hive-forge-notify (the hive's Forgejo) and hive-github-notify (github.com, installed by nix/agent-modules/github.nix). Was a task inside the hive-agent serve loop; own process since it needs nothing else from the harness.
  • hive-matrix-mcp/ — per-agent matrix-sdk daemon (hive-matrix-daemon); serves its MCP tools (send_message, read_room, …) directly over streamable-http (no stdio bridge), same shape as hive-bash-mcp.
  • hive-bash-mcp/ — per-agent bash-task runner daemon (hive-bash-daemon); serves its MCP tools (run/status/kill) directly over streamable-http (no stdio bridge), writes task files under /harness/bash-tasks/, and records the favorite-tools bash_commands stat into turn-stats.sqlite.
  • hive-subagent-mcp/ — per-agent claude-subagent runner daemon (hive-subagent-daemon); spawns nested claude sessions on request and serves start/continue/status/interrupt directly over streamable-http (no stdio bridge). Independent of hive-bash-mcp (a subagent is a much heavier capability than a bash command). No task files — the daemon's only state is an in-memory map of currently-running processes, live only as long as the process is; the actual claude session survives a daemon restart independently (see session.rs's module doc).
  • hive-sh4re/ — shared wire types (Agent / Manager request + response, Message, Approval, HelperEvent) used across the unix sockets. Host-admin-socket and hive-priv-socket wire types have been split out into their own crates (below) so hivectl and hive-priv don't need to pull in the rest of hive-sh4re.
  • hive-host-sock/ — wire types for the host admin socket (/run/hyperhive/host.sock), the protocol hivectl speaks to hive-c0re. Split out of hive-sh4re so a standalone hivectl only depends on this protocol crate, not the whole daemon crate.
  • hive-priv-sock/ — wire types for the hive-priv privileged-helper socket (/run/hive/priv.sock), shared by hive-priv (server) and hive-c0re (client). Also split out of hive-sh4re.
  • hive-core-agent-sock/ — wire types for the host-served per-agent
    • manager socket (/run/hive/mcp.sock), the protocol an agent's harness speaks to hive-c0re. Same split rationale as the two above; the shared payload types it references stay in hive-sh4re.
  • hive-agent-sock/ — what's common across the in-container "agent plugins" (matrix/bash/subagent MCP daemons, forge_notify, any user-configured MCP server). Wire types for the in-agent socket, served by the harness to those producers — nothing restricts the subsystem set on the todo ops, any producer can push its own. Carries the loose-ends-v2 todo ops plus harness-local reminders. ⚠️ Distinct from hive-core-agent-sock above: this socket never leaves the container and hive-c0re is not in the path at all — no broker round-trip, no long-poll, no marker files. Also carries extra_mcp: the hyperhive.extraMcpServers spec + parsing, shared by hive-agent and hive-subagent-mcp — not a socket protocol, just the natural home since both already depend on this crate.
  • hive-types/ — zero-dependency (bar serde) leaf crate holding the foundational newtypes, chiefly Ident (163 chars of [a-z0-9-], constructed only via the validating parser). Lets every wire-type crate and both binaries type their agent-name fields as a validated ident and get serde-checked parsing at the socket boundary, without coupling to hive-sh4re.
  • hive-sock-client/ — the shared JSON-line-over-unix-socket client every daemon uses to talk to a hyperhive socket. Generic over the request/response types, so the host-served control socket and the harness's in-agent socket both use it with their own wire-type crates. Retry is a policy value (Retry::None for callers already inside a poll loop, Retry::RideOutRestart for callers with no natural retry), and the response is either decoded (request) or drained (notify). Deliberately separate from the *-sock crates — those stay dependency-free wire types.
  • hive-metric/ — small CLI to push a single labeled metric to the OTEL collector via the OpenTelemetry Rust SDK / OTLP HTTP exporter.
  • swarm-controller/ — swarm-level daemon, opt-in per host (services.hyperhive.deploy.swarm-controller.enable). Where hive-c0re owns the agents on one host, this owns what is true across hives; a swarm runs one of them, so most hives leave it off. Serves HTTP over a unix socket the gateway's nginx proxies to — ⚠️ the socket's directory is its access control; the constraint that governs it is in the crate's README, and a unit test pins the path.
  • swarm-secret-client/ — client for the swarm's secret store, over vaultrs. Owns the agreements both ends of the store must share rather than the HTTP: the path a credential lives at, the field its bytes are in, and the translation from this deployment's environment into a logged-in client. ⚠️ Reads the BAO_ env spellings explicitly — vaultrs's own defaults look for VAULT_ADDR/VAULT_CLIENT_CERT/VAULT_CLIENT_KEY, which no unit in this tree sets, so falling through to them yields a client with no identity and a TLS handshake failure that names no cause.
  • swarmctl/ — swarm-level operator CLI, installed by the swarm-controller module on the host that runs the daemon. Runs as root and acts directly — it serves no socket, publishes no HTTP route and has no priv helper; the crate's README records why the rootless shape was examined and rejected. agent create is the one verb that talks to a service rather than acting locally: it POSTs /api/agents to swarm-controller over the daemon's unix socket and prints the queued job id without waiting (the DAG settling would not mean the agent is up). Does not link swarm-controller — its wire shapes are a local mirror kept in this crate, not a shared crate, so the two binaries can be deployed independently. ⚠️ Its user store is authelia's own users.yml, read and written in place — one file, shared with swarm-authelia-bridge; see that crate's README for what both writers must uphold. Full, always-current verb reference (CI-enforced against the clap tree, same pattern as hivectl's — see docs/process/conventions.md): docs/tools/swarmctl-cli.md.

External dependencies with no directory here

  • hive-claude — reusable, app-agnostic driver for headless claude --print: spawns the CLI, streams + classifies stream-json, parses per-turn Telemetry, and drives a durable self-compacting InfiniteSession (name + SessionStore + CompactionPolicy). Uses thiserror (it's a library); the hive-* binaries consume it with anyhow. ⚠️ Its own repo, not a workspace member — consumed as a dependency in the root Cargo.toml, so there is no hive-claude/ directory to read and grep here will not find it.

Other top-level dirs

  • frontend/ — npm workspaces → static dashboard + per-agent UI dist, built hermetically by nix/packages/frontend.nix. Packages: shared (terminal pane + Catppuccin palette), dashboard (the operator SPA), agent (the default per-container UI), swarm-ui (swarm-level UI shell — Preact + wouter + TypeScript + JSX, project- bootstrap scope; packaged separately by nix/packages/swarm-ui.nix, not bundled into packages.default's closure — see that file).
  • nix/host-modules/ (the host stack: hyperhive core options, hive-{c0re,priv,forge,gateway,matrix,network,tls,ci}, otel, swarm), agent-modules/ (the per-agent harness feature modules), templates/{agent,ruth}.nix (container entry points), packages/ (flake package outputs), docs/ (the options-doc derivation), plus sources.nix / rust.nix / checks.nix / devshell.nix / treefmt.nix behind the thin flake.nix.
  • docs/ — subsystem reference docs (see Reading paths below).
  • branding/, scripts/ — static assets + helper scripts.

Reading paths

docs/README.md is the single index — grouped by topic (getting started, agent lifecycle, trust boundary, integrations, networking, scheduler, process). Read it instead of a second copy here; this file used to carry its own parallel "Reading paths" list, and the two drifted out of sync with each other.

Conventions & process

  • Never add #[allow(clippy::…)] — fix the lint instead (extract a helper, add backticks, etc.). Details + worked examples: → docs/process/conventions.md.
  • Pre-push lint hook (catches tracker-tag and comment-block failures before CI does — install once per clone): ln -sf ../../scripts/pre-push .git/hooks/pre-push
  • Comment/PR-description length: size it to how non-obvious the thing is, not to how much investigation it took to find it — a mechanical fix gets a one-line comment, a genuinely non-obvious invariant or decision earns real prose.
  • Operator-facing doc with real implementation detail mixed in: wrap the implementation part in a <details><summary>…</summary> block (blank line after <summary> so the markdown inside still renders, not a second file) rather than splitting into a sibling page — see docs/integrations/github.md's "Implementation" section. Zero nav churn, works uniformly regardless of whether the doc already had a clean boundary. Caveat: it only collapses in a rendered browser — reading the file as raw text (cat, the Read tool) shows everything, same as today. Also: the <summary> line itself isn't markdown- processed — backticks render as literal characters, not styled code — so keep summary text plain prose, no inline-code formatting.