From 07828cd573bfc5cfa71266ca6c51e14376f09f41 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?m=C3=BCde?= Date: Thu, 1 Oct 2026 20:31:14 +0200 Subject: [PATCH] README: reframe around multi-hive swarms, all-local quick start, agent amenities --- README.md | 256 +++++++++++++++++++++++++++++++++++------------------- 1 file changed, 169 insertions(+), 87 deletions(-) diff --git a/README.md b/README.md index 13bd172a..ef5b6e88 100644 --- a/README.md +++ b/README.md @@ -1,130 +1,212 @@ # hyperhive -> a swarm of claude-code agents, each in its own nspawn cage, gossiping -> over unix sockets. config changes flow as git commits, the operator -> approves them in a browser, every deploy is a tag. cyberpunk-themed -> dashboard included. ๐Ÿ’œโšก - -Claude code is great in one window, _exponentielle_ across many โ€” but -only if you can keep the agents from stepping on each other, give them -durable identity, and stop them from eating production. hyperhive is -the substrate. - -- identity = unix socket -- communication = sqlite-backed broker (`send` / `recv` / `remind`) -- config = git (manager proposes, operator approves, deploys land as - tagged commits) -- blast radius = container - -``` -every hive (NixOS host, runs hive-c0re.service) -โ”‚ -โ”œโ”€โ”€ operator -โ”‚ โ”œโ”€โ”€ browser โ†’ :80 (hive-gateway) dashboard + per-agent UIs -โ”‚ โ”‚ /agent// โ†’ per-agent unix socket -โ”‚ โ””โ”€โ”€ CLI โ†’ /run/hyperhive/host.sock admin protocol -โ”‚ -โ”œโ”€โ”€ hive-c0re (Rust daemon: lifecycle / broker / approvals / -โ”‚ auto-update / dashboard / sockets) -โ”‚ -โ”œโ”€โ”€ hive-gateway (optional) nginx โ€” proxies :80 โ†’ c0re dashboard + per-agent sockets -โ”‚ -โ””โ”€โ”€ agent containers - โ”œโ”€โ”€ h-ruth manager (privileged MCP surface, approval gating) - โ””โ”€โ”€ h- sub-agent (claude + MCP tools + per-agent web UI + unix socket) - -one host per swarm (optional โ€” connects hives; can be any hive, including -one that's also running the tree above) -โ”‚ -โ”œโ”€โ”€ hive-forge Forgejo โ€” swarm-wide singleton, per-agent accounts + config mirror -โ”œโ”€โ”€ hive-matrix tuwunel โ€” swarm-wide singleton, Matrix homeserver + per-agent accounts -โ”œโ”€โ”€ swarm-controller cross-hive state: hive directory, agent roster, jobs -โ”œโ”€โ”€ swarm-ui swarm-wide SPA, served straight off the gateway (no own container) -โ”œโ”€โ”€ swarm-authelia SSO โ€” one login gates swarm-ui + Grafana + more -โ”œโ”€โ”€ swarm-nats message queue (JetStream KV: hive-status, โ€ฆ) -โ”œโ”€โ”€ swarm-otel telemetry collector, sole holder of the upstream credential -โ”œโ”€โ”€ swarm-victoriametrics metrics store -โ”œโ”€โ”€ swarm-victorialogs log store -โ””โ”€โ”€ swarm-grafana dashboards over the metrics/log stores, own OIDC login -``` +> a swarm of coding agents spread across NixOS hosts โ€” every agent in its +> own nspawn cage, every host a hive, one control plane over all of them. +> config changes flow as git commits, the operator approves them in a +> browser, every deploy is a tag. cyberpunk dashboard included. ๐Ÿ’œโšก **[โ†’ website](https://hyperhive.darkest.space)** ยท **[โ†’ docs](https://hyperhive.darkest.space/docs/)** ยท **[โ†’ options reference](https://hyperhive.darkest.space/options/)** -Depth lives in [`docs/`](docs/) (rendered at -[hyperhive.darkest.space/docs/](https://hyperhive.darkest.space/docs/)) โ€” -start at [`docs/README.md`](docs/README.md) and pick the page matching -your task rather than reading front to back. +A coding agent is great in one window, _exponentielle_ across many โ€” as +long as the agents stay off each other's toes, keep a durable identity, +and leave production alone. hyperhive is the substrate for that, as a set +of NixOS modules: -## Quick start +- a **hive** is one host running `hive-c0re` and the agent containers it + owns; +- a **swarm** is any number of hives sharing one identity plane โ€” SSO, + forge, matrix, secret store, telemetry โ€” with a controller that places + agents on hives and tracks them across all of them. -Minimal `flake.nix` for a host that runs hive-c0re: +Start with everything on one box; move services and add hives as the +swarm grows. Which host runs what is a per-host `deploy.*` toggle, +not an architecture change. + +| concern | how hyperhive answers it | +| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **scale-out** | hives share one directory (`swarm.hives`, identical on every host); `swarmctl agent create --hive ` places an agent and the controller keeps the roster | +| **identity** | every agent gets swarm-wide accounts โ€” SSO subject, forge user, matrix account, secret-store cert identity โ€” and is addressable as `name@hive.domain` | +| **shared plane** | one forge, homeserver, secret store, SSO, queue and metrics/logs stack per swarm, each on whichever host you put it | +| **blast radius** | one `nixos-container` per agent; the host daemon runs unprivileged, and a tiny socket-activated helper does the few root ops | +| **config** | git: an agent proposes, the operator approves, the deploy lands as a `deployed/` tag | +| **runtime** | `claude --print` by default; any [ACP](https://agentclientprotocol.com) agent (e.g. opencode) per agent with `services.hyperhive.agent.runtime = "acp"` | +| **watching** | per-agent web UI with a live terminal, a per-hive dashboard, the swarm UI, Grafana over OTEL | + +## Shape + +``` +every hive (NixOS host, deploy.hive-controller.enable) +โ”‚ +โ”œโ”€โ”€ operator +โ”‚ โ”œโ”€โ”€ browser โ†’ :80/:443 (hive-gateway) dashboard + /agent// UIs +โ”‚ โ””โ”€โ”€ hivectl โ†’ /run/hyperhive/host.sock admin protocol +โ”‚ +โ”œโ”€โ”€ hive-c0re Rust daemon: lifecycle, broker, approvals, job queue, +โ”‚ auto-rebuild, dashboard โ€” runs as the unprivileged hive-core user +โ”œโ”€โ”€ hive-priv root helper, socket-activated, does bind-mounts + nsenter +โ”œโ”€โ”€ hive-gateway nginx + dnsmasq in front of c0re and every agent socket +โ”‚ +โ””โ”€โ”€ agent containers + โ”œโ”€โ”€ h-ruth manager โ€” privileged MCP surface, spawns + configures agents + โ””โ”€โ”€ h- agent โ€” claude / ACP runtime, MCP tools, web UI, own socket + +the swarm's shared services (one host each โ€” any hive, or a dedicated box) +โ”‚ +โ”œโ”€โ”€ hive-forge Forgejo โ€” per-agent accounts, config mirror +โ”œโ”€โ”€ hive-matrix tuwunel โ€” matrix homeserver, per-agent accounts +โ”œโ”€โ”€ swarm-bao OpenBao โ€” the swarm's secret store, cert auth per agent +โ”œโ”€โ”€ swarm-controller cross-hive state: hive directory, agent roster, jobs +โ”‚ (+ swarmctl, + swarm-ui served off the gateway) +โ”œโ”€โ”€ swarm-authelia SSO in front of swarm-ui, forge, grafana, โ€ฆ +โ”œโ”€โ”€ swarm-nats message queue (JetStream KV: hive status, โ€ฆ) +โ”œโ”€โ”€ swarm-otel telemetry collector, sole holder of upstream creds +โ”œโ”€โ”€ swarm-victoriametrics metrics store +โ”œโ”€โ”€ swarm-victorialogs log store +โ”œโ”€โ”€ swarm-grafana dashboards over both stores +โ”‚ +โ””โ”€โ”€ opt-in, not part of singleHostSwarm + โ”œโ”€โ”€ hive-ci Forgejo Actions runner (deploy.forgejo.ci) + โ”œโ”€โ”€ swarm-snapshot-store btrfs-receive endpoint for agent snapshots + โ””โ”€โ”€ wg-hive WireGuard mesh between hives (deploy.wireguard) +``` + +## Inside an agent + +Every agent is a full NixOS container with its own unix user (passwordless +sudo inside the cage) and a durable, self-compacting session. Out of the +box it gets: + +- **messaging** โ€” `send` / `recv` to other agents and the operator, + `remind` and recurring schedules to wake itself, and a todo list that + forge notifications and other producers feed. โ†’ + [scheduling](docs/tools/scheduling.md) +- **background bash** โ€” long-running commands as tracked tasks + (`run` / `status` / `kill`), so a build doesn't block the turn. โ†’ + [bash](docs/tools/bash.md) +- **subagents** โ€” nested sessions on the agent's own runtime, optionally + re-prompted toward a goal over several turns. โ†’ + [subagent](docs/tools/subagent.md) +- **forge** โ€” its own Forgejo account, the `hive-forge` CLI, git + credentials, and notifications turned into todos. Optional linked + accounts on other forges, and GitHub (`gh` + push) via + `services.hyperhive.agent.github.enable`. โ†’ + [forge](docs/integrations/forge.md) ยท [github](docs/integrations/github.md) +- **matrix** โ€” its own account(s) on the swarm homeserver, as MCP tools. โ†’ + [matrix](docs/tools/matrix.md) +- **knowledge** โ€” the swarm's shared `internal/knowledge` repo, read-only + at `/knowledge`, plus hyperhive's own docs in the container. โ†’ + [knowledge](docs/integrations/knowledge.md) +- **web UI** โ€” a per-agent page behind the gateway: live terminal of the + turn stream, model / effort switching, cancel, `/compact`. +- **GUI** (opt-in, `services.hyperhive.agent.gui.enable`) โ€” a Wayland + desktop with screenshot / keyboard / mouse tools, viewable over VNC in + the web UI. +- **your own tools** โ€” any extra MCP server (stdio or http) via + `services.hyperhive.agent.extraMcpServers`, and any NixOS config you'd + put on a machine. + +## Quick start: an all-local swarm + +The whole swarm on one box โ€” a hive plus every shared service. Good for a +dev box or a single-host deployment; [`docs/swarm/`](docs/swarm/README.md) +covers spreading a swarm across hosts. ```nix { inputs = { nixpkgs.url = "github:NixOS/nixpkgs/nixos-26.05"; hyperhive.url = "git+https://forge.darkest.space/hyperhive/hyperhive"; - # Pin hyperhive to your own nixpkgs instead of the one it ships with - # (see "Overriding nixpkgs" below) โ€” recommended for most hosts: - hyperhive.inputs.nixpkgs.follows = "nixpkgs"; + hyperhive.inputs.nixpkgs.follows = "nixpkgs"; # see "Overriding nixpkgs" }; outputs = { nixpkgs, hyperhive, ... }: { nixosConfigurations.my-host = nixpkgs.lib.nixosSystem { system = "x86_64-linux"; modules = [ - hyperhive.nixosModules.default # hive-c0re + hive-forge + hive-gateway in one import - ({ ... }: { - services.hyperhive.deploy.hive-controller.enable = true; - # services.hyperhive.c0re.operatorPronouns = "they/them"; # default: "she/her" + hyperhive.nixosModules.default # the whole host stack, one import + { + services.hyperhive = { + deploy.singleHostSwarm = true; # run everything on one machine - # ... rest of your host config + swarm.domain = "hive.example"; # the swarm's DNS domain + hiveName = "pr1ma"; # this hive's label in it + swarm.hives.pr1ma = { }; # the directory: just us + + # c0re.operatorPronouns = "they/them"; # default "she/her" + }; + + # ... the rest of your host config system.stateVersion = "25.11"; - }) + } ]; }; }; } ``` -hive-c0re opens its admin socket + dashboard, auto-creates the -manager container, and auto-rebuilds any container whose hyperhive -rev goes stale. `claude-code` is unfree โ€” hyperhive scopes the -whitelist to itself, nothing for the operator to set. +`singleHostSwarm` turns on everything an all-on-one-box swarm implies: +the shared services above, the swarm CA, the swarm controller, and +`/etc/hosts` entries so the hive's names resolve on the box itself. +`claude-code` is unfree โ€” hyperhive scopes the allowance to itself, so +there's nothing to set. + +After `nixos-rebuild switch`, hive-c0re creates the manager (`ruth`) and +keeps every container rebuilt whenever its hyperhive rev goes stale. A few +steps stay deliberately human โ€” initialising the secret store, giving ruth +her store identity, adding a gateway login, SSO. Walk through +**[`docs/getting-started/setup.md`](docs/getting-started/setup.md)** +once, top to bottom. ### Overriding nixpkgs -hyperhive pins its own `nixpkgs` so it builds standalone in CI. Add -`hyperhive.inputs.nixpkgs.follows = "nixpkgs"` (as in the quick-start above) -to build it against your host's `nixpkgs` instead โ€” one less nixpkgs -evaluation, no version drift from the rest of your system. Standard flake -`follows` pattern; works as long as your channel is reasonably close to the -`nixos-26.05` hyperhive develops against. Drop it again if a much -older/newer channel hits breakage hyperhive's CI doesn't catch. +hyperhive pins its own `nixpkgs` so it builds standalone in CI. The +`follows` line above builds it against your host's `nixpkgs` instead: one +evaluation, no version drift. It works as long as your channel is close to +the `nixos-26.05` hyperhive develops against โ€” drop it if a much older or +newer channel breaks in ways hyperhive's CI doesn't catch. -For the full list of host and agent NixOS options see the -**[options reference](https://hyperhive.darkest.space/options/)**. +## Operating it -## Operator CLI +- **`hivectl`** โ€” the hive's operator CLI, on every hive host: agents, + approvals, gateway logins, container shells. Talks to hive-c0re over + `/run/hyperhive/host.sock`. โ†’ [guide](docs/tools/hivectl.md) ยท + [full reference](docs/tools/hivectl-cli.md) +- **`swarmctl`** โ€” the swarm's operator CLI, on the swarm-controller host: + creating agents with their identities, SSO users, forge admins. + โ†’ [reference](docs/tools/swarmctl-cli.md) +- **Dashboard** โ€” approvals, agent state, the job graph, per-agent + terminals. โ†’ [`docs/web-ui/`](docs/web-ui/README.md) -`hivectl` is the operator-facing host CLI for ad-hoc administration that -doesn't go through the broker (built alongside `hive-c0re` when the host -module is enabled). Human matrix accounts come from SSO login, not -`hivectl`. +Human accounts come from SSO: a first login through authelia creates your +forge and matrix accounts; `swarmctl forge make-admin ` promotes the +forge one. -A human's first SSO login to the forge makes their forge account, not -`hivectl`; `swarmctl forge make-admin ` on the swarm-controller's -host makes it a site admin. +## Where to read next -## Build / deploy +[`docs/README.md`](docs/README.md) is the index, grouped by question โ€” +pick the page for your task rather than reading front to back. Some +common entry points: + +- how config changes flow โ†’ [`agent-lifecycle/approvals.md`](docs/agent-lifecycle/approvals.md) +- trust boundary + threat model โ†’ [`trust-boundary/`](docs/trust-boundary/boundary.md) +- what an agent's turn looks like โ†’ [`turn-loop/`](docs/turn-loop/README.md) +- connecting hives into a swarm โ†’ [`swarm/`](docs/swarm/README.md) +- every `services.hyperhive.*` option โ†’ [options reference](https://hyperhive.darkest.space/options/) + +## Hacking on it ```sh nix develop -c cargo check -nix flake check # rust + nix + toml fmt + clippy +nix flake check # rust + nix + toml fmt, clippy, module-eval tests -# deploy from a host config that imports hyperhive.nixosModules.default -nix flake update --update-input hyperhive +# in your host flake: +nix flake update hyperhive sudo nixos-rebuild switch --flake .# ``` + +Conventions (commit style, no `#[allow(clippy::โ€ฆ)]`, the pre-push hook) +live in [`docs/process/conventions.md`](docs/process/conventions.md). +Issues + backlog are on the +[forge](https://forge.darkest.space/hyperhive/hyperhive/issues).