diff --git a/README.md b/README.md index ef5b6e88..1bab64e0 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,10 @@ # hyperhive -> a swarm of coding agents spread across NixOS hosts β€” every agent in its -> own nspawn cage, every host a hive, one control plane over all of them. -> config changes flow as git commits, the operator approves them in a -> browser, every deploy is a tag. cyberpunk dashboard included. πŸ’œβš‘ +> a swarm of coding agents with one control plane: one identity, one +> forge, one homeserver, one secret store, one UI. agents run caged on +> whichever host has room. config changes flow as git commits, the +> operator approves them in a browser, every deploy is a tag. cyberpunk +> dashboards included. πŸ’œβš‘ **[β†’ website](https://hyperhive.darkest.space)** Β· **[β†’ docs](https://hyperhive.darkest.space/docs/)** Β· @@ -11,55 +12,42 @@ A coding agent is great in one window, _exponentielle_ across many β€” as long as the agents stay off each other's toes, keep a durable identity, -and leave production alone. hyperhive is the substrate for that, as a set -of NixOS modules: +and leave production alone. hyperhive is a set of NixOS modules that runs +that many-agent setup as a **swarm**: -- a **hive** is one host running `hive-c0re` and the agent containers it - owns; -- a **swarm** is any number of hives sharing one identity plane β€” SSO, - forge, matrix, secret store, telemetry β€” with a controller that places - agents on hives and tracks them across all of them. +- **the swarm is where things live** β€” agent identities and accounts, + secrets, the job graph that creates and places agents, telemetry, and + the UI you drive it all from. +- **hives are the substrate** β€” NixOS hosts that run agent containers on + the swarm's behalf. Add a hive to add capacity. Start with everything on one box; move services and add hives as the -swarm grows. Which host runs what is a per-host `deploy.*` toggle, -not an architecture change. +swarm grows. Which host runs what is a per-host `deploy.*` toggle, not an +architecture change. -| concern | how hyperhive answers it | -| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | -| **scale-out** | hives share one directory (`swarm.hives`, identical on every host); `swarmctl agent create --hive ` places an agent and the controller keeps the roster | -| **identity** | every agent gets swarm-wide accounts β€” SSO subject, forge user, matrix account, secret-store cert identity β€” and is addressable as `name@hive.domain` | -| **shared plane** | one forge, homeserver, secret store, SSO, queue and metrics/logs stack per swarm, each on whichever host you put it | -| **blast radius** | one `nixos-container` per agent; the host daemon runs unprivileged, and a tiny socket-activated helper does the few root ops | -| **config** | git: an agent proposes, the operator approves, the deploy lands as a `deployed/` tag | -| **runtime** | `claude --print` by default; any [ACP](https://agentclientprotocol.com) agent (e.g. opencode) per agent with `services.hyperhive.agent.runtime = "acp"` | -| **watching** | per-agent web UI with a live terminal, a per-hive dashboard, the swarm UI, Grafana over OTEL | +| concern | how hyperhive answers it | +| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **control plane** | `swarm-controller` holds the hive directory, the agent roster and the job graph. Create an agent from the swarm UI or `swarmctl agent create --hive `: it provisions the SSO identity, forge user and config repo, then deploys the agent onto that hive | +| **identity** | every agent is a swarm-wide principal β€” SSO subject, forge user, matrix account, secret-store cert identity β€” addressable as `name@hive.domain` | +| **secrets** | one OpenBao store; each agent fetches its own credentials under its own identity, nothing copied between hosts by hand | +| **shared services** | one forge, homeserver, SSO, message queue and metrics/logs stack per swarm, each on whichever host you put it | +| **config** | git: an agent proposes, the operator approves, the deploy lands as a `deployed/` tag | +| **runtime** | `claude --print` by default; any [ACP](https://agentclientprotocol.com) agent (e.g. opencode) per agent with `services.hyperhive.agent.runtime = "acp"` | +| **substrate** | each hive runs one `nixos-container` per agent, under an unprivileged daemon with a tiny socket-activated helper for the few root ops | +| **watching** | the swarm UI (hives, agents with live terminals, jobs, a cross-repo issue report), Grafana over OTEL, and a per-hive dashboard for host-level detail | ## Shape ``` -every hive (NixOS host, deploy.hive-controller.enable) +the swarm β€” one of each, each on whichever host you choose β”‚ -β”œβ”€β”€ operator -β”‚ β”œβ”€β”€ browser β†’ :80/:443 (hive-gateway) dashboard + /agent// UIs -β”‚ └── hivectl β†’ /run/hyperhive/host.sock admin protocol -β”‚ -β”œβ”€β”€ hive-c0re Rust daemon: lifecycle, broker, approvals, job queue, -β”‚ auto-rebuild, dashboard β€” runs as the unprivileged hive-core user -β”œβ”€β”€ hive-priv root helper, socket-activated, does bind-mounts + nsenter -β”œβ”€β”€ hive-gateway nginx + dnsmasq in front of c0re and every agent socket -β”‚ -└── agent containers - β”œβ”€β”€ h-ruth manager β€” privileged MCP surface, spawns + configures agents - └── h- agent β€” claude / ACP runtime, MCP tools, web UI, own socket - -the swarm's shared services (one host each β€” any hive, or a dedicated box) -β”‚ -β”œβ”€β”€ hive-forge Forgejo β€” per-agent accounts, config mirror +β”œβ”€β”€ swarm-controller hive directory, agent roster, job graph +β”‚ β”œβ”€β”€ swarm-ui the operator's UI, served off the gateway on the swarm apex +β”‚ └── swarmctl its CLI, on the controller's host +β”œβ”€β”€ swarm-authelia SSO in front of the swarm UI, forge, grafana, … +β”œβ”€β”€ swarm-bao OpenBao β€” secrets, a cert identity per agent +β”œβ”€β”€ hive-forge Forgejo β€” per-agent accounts, config repos β”œβ”€β”€ hive-matrix tuwunel β€” matrix homeserver, per-agent accounts -β”œβ”€β”€ swarm-bao OpenBao β€” the swarm's secret store, cert auth per agent -β”œβ”€β”€ swarm-controller cross-hive state: hive directory, agent roster, jobs -β”‚ (+ swarmctl, + swarm-ui served off the gateway) -β”œβ”€β”€ swarm-authelia SSO in front of swarm-ui, forge, grafana, … β”œβ”€β”€ swarm-nats message queue (JetStream KV: hive status, …) β”œβ”€β”€ swarm-otel telemetry collector, sole holder of upstream creds β”œβ”€β”€ swarm-victoriametrics metrics store @@ -70,6 +58,18 @@ the swarm's shared services (one host each β€” any hive, or a dedicated box) β”œβ”€β”€ hive-ci Forgejo Actions runner (deploy.forgejo.ci) β”œβ”€β”€ swarm-snapshot-store btrfs-receive endpoint for agent snapshots └── wg-hive WireGuard mesh between hives (deploy.wireguard) + +the hives β€” the substrate, as many as you like (deploy.hive-controller.enable) +β”‚ +β”œβ”€β”€ hive-c0re runs the agent containers: lifecycle, approvals, broker, +β”‚ auto-rebuild, the per-hive dashboard; unprivileged +β”œβ”€β”€ hive-priv root helper, socket-activated, does bind-mounts + nsenter +β”œβ”€β”€ hive-gateway nginx + dnsmasq on :80/:443, fronts every agent's web UI +β”œβ”€β”€ hivectl host-level CLI +β”‚ +└── agent containers + β”œβ”€β”€ h-ruth manager β€” privileged MCP surface, configures agents + └── h- agent β€” claude / ACP runtime, MCP tools, web UI ``` ## Inside an agent @@ -117,8 +117,10 @@ covers spreading a swarm across hosts. { inputs = { nixpkgs.url = "github:NixOS/nixpkgs/nixos-26.05"; - hyperhive.url = "git+https://forge.darkest.space/hyperhive/hyperhive"; - hyperhive.inputs.nixpkgs.follows = "nixpkgs"; # see "Overriding nixpkgs" + hyperhive = { + url = "git+https://forge.darkest.space/hyperhive/hyperhive"; + inputs.nixpkgs.follows = "nixpkgs"; # see "Overriding nixpkgs" + }; }; outputs = { nixpkgs, hyperhive, ... }: { @@ -128,7 +130,8 @@ covers spreading a swarm across hosts. hyperhive.nixosModules.default # the whole host stack, one import { services.hyperhive = { - deploy.singleHostSwarm = true; # run everything on one machine + deploy.singleHostSwarm = true; # every swarm service on this machine + deploy.hive-controller.enable = true; # …and a hive to run agents on swarm.domain = "hive.example"; # the swarm's DNS domain hiveName = "pr1ma"; # this hive's label in it @@ -152,10 +155,10 @@ the shared services above, the swarm CA, the swarm controller, and `claude-code` is unfree β€” hyperhive scopes the allowance to itself, so there's nothing to set. -After `nixos-rebuild switch`, hive-c0re creates the manager (`ruth`) and -keeps every container rebuilt whenever its hyperhive rev goes stale. A few -steps stay deliberately human β€” initialising the secret store, giving ruth -her store identity, adding a gateway login, SSO. Walk through +After `nixos-rebuild switch` the swarm UI answers on the swarm domain, and +the hive brings up the manager agent (`ruth`). A few steps stay +deliberately human β€” initialising the secret store, giving ruth her store +identity, creating your SSO account. Walk through **[`docs/getting-started/setup.md`](docs/getting-started/setup.md)** once, top to bottom. @@ -169,19 +172,22 @@ newer channel breaks in ways hyperhive's CI doesn't catch. ## Operating it -- **`hivectl`** β€” the hive's operator CLI, on every hive host: agents, - approvals, gateway logins, container shells. Talks to hive-c0re over - `/run/hyperhive/host.sock`. β†’ [guide](docs/tools/hivectl.md) Β· - [full reference](docs/tools/hivectl-cli.md) -- **`swarmctl`** β€” the swarm's operator CLI, on the swarm-controller host: - creating agents with their identities, SSO users, forge admins. +- **swarm UI** β€” the day-to-day surface, on the swarm apex behind SSO + (`admins` group): hive roster and status, every agent with its live + terminal and wanted state, creating agents, linking external forge and + matrix accounts, the job graph, a cross-repo issue report. β†’ + [`docs/swarm/ui.md`](docs/swarm/ui.md) +- **`swarmctl`** β€” the swarm's CLI, on the swarm-controller host: create + agents and mint their identities, manage SSO users, make forge admins. β†’ [reference](docs/tools/swarmctl-cli.md) -- **Dashboard** β€” approvals, agent state, the job graph, per-agent - terminals. β†’ [`docs/web-ui/`](docs/web-ui/README.md) +- **per-hive** β€” each hive's dashboard (approvals, container state, + rebuild queue) and `hivectl` for host-level administration and + container shells. β†’ [`docs/web-ui/`](docs/web-ui/README.md) Β· + [hivectl](docs/tools/hivectl.md) -Human accounts come from SSO: a first login through authelia creates your -forge and matrix accounts; `swarmctl forge make-admin ` promotes the -forge one. +You are an SSO user too: `swarmctl user add --group admins`, and a +first login through authelia creates your forge and matrix accounts; +`swarmctl forge make-admin ` promotes the forge one. ## Where to read next @@ -189,10 +195,10 @@ forge one. pick the page for your task rather than reading front to back. Some common entry points: +- connecting hives into a swarm β†’ [`swarm/`](docs/swarm/README.md) - how config changes flow β†’ [`agent-lifecycle/approvals.md`](docs/agent-lifecycle/approvals.md) - trust boundary + threat model β†’ [`trust-boundary/`](docs/trust-boundary/boundary.md) - what an agent's turn looks like β†’ [`turn-loop/`](docs/turn-loop/README.md) -- connecting hives into a swarm β†’ [`swarm/`](docs/swarm/README.md) - every `services.hyperhive.*` option β†’ [options reference](https://hyperhive.darkest.space/options/) ## Hacking on it