Watch
0
0
Fork
You've already forked hyperhive
0
hyperhive/swarm-controller
Repository files (latest commit first)
Filename Latest commit message Latest commit date
atlas 29aa242201 controller stores: recover from a record they can't decode
After a rollback, a record a newer build wrote (a new enum variant, a
changed required field) stopped decoding, and the strict
`schedules::read` turned that into an error on every read and write of
the key until someone deleted it by hand.

- agent_limits / inference (one document per agent): an undecodable
  document reads as unset, with a WARN naming the bucket, key and error.
  The PUT's "equal to stored, skip the write" shortcut no longer applies
  to an undecodable document, so a PUT of the defaults replaces it too.
- schedule definitions (one document, every schedule): salvaged per
  definition, as the review-classifier run list is. An undecodable
  definition is kept as its raw JSON, hidden from the list, edits and
  projections, written back by every commit after the readable ones, and
  its id is never minted again. A document with no list of schedules
  stays an error: reading it as empty would empty every agent's key.
- per-agent agent-schedules: an undecodable key reads as no schedules
  and is rewritten on the next projection, even when the agent has none.

`schedules::read` is replaced by `read_or_default`; `wanted` keeps its
own strict read.

Closes #5193
2026-10-13 08:12:50 +02:00
..
src controller stores: recover from a record they can't decode 2026-10-13 08:12:50 +02:00
Cargo.toml swarm-controller: record the review classifier's runs and list them 2026-10-12 21:48:51 +02:00
README.md review classifier: model, effort and review concurrency as swarm-wide settings 2026-10-13 01:45:19 +02:00

swarm-controller

The swarm-level daemon. Where hive-c0re owns the agents on one host, this owns what is true across hives — so a swarm runs one of them and most hives leave it off.

Opt-in per host via services.hyperhive.deploy.swarm-controller.enable, which is deliberately not derived from services.hyperhive.deploy.hive-controller.enable: turning it on is a statement about swarm topology, not about whether this host runs a hive.

What it does

The swarm's control plane. The swarm UI and swarmctl are its clients.

  • Hive directory — serves swarm.hives (GET /api/hives) and what each hive last published about itself (GET /api/hives/status).
  • Agent roster and wanted state — every agent the swarm knows (GET /api/agents, /api/agents/status), and the state it declares for each one on its hive (up/offline/paused/destroyed, PUT /api/hives/{hive}/agents/{agent}/state).
  • Job graph — a hive-jobq scheduler, served at GET /api/jobq/graph. Every provisioning step below runs as a node in it.
  • Agent creation — POST /api/agents queues the SSO identity (through swarm-authelia-bridge), forge user, config repo, store identity, forge token and matrix account, declares the agent paused, then sends its hive a deploy message.
  • Agent credentials — at start and every five minutes it re-checks every agent's forge token and matrix account, and renews store certificates and queue secrets as they age.
  • Linked external accounts — an operator-supplied matrix, forge or GitHub account for one agent, stored in the swarm secret store (PUT /api/hives/{hive}/agents/{agent}/matrix-accounts/{account}, .../forge-accounts/{label}, .../github-account); distinct from the agent's own swarm-minted accounts above. GET .../linked-accounts lists them, plus the agent's own main matrix account, by kind, name and host, never with a credential. Each of the three also takes a DELETE; matrix's also takes ?revoke=true, which logs the stored token out at its homeserver first and leaves the account in place if that fails.
  • Config PR status — each agent's open config-repo PR, cached from forge webhooks (GET /api/config-prs, /api/agents/{name}/config-pr).
  • Swarm-wide forge objects and webhooks → docs/swarm/README.md.
  • Scheduled prompts — named prompts for one or more agents, kept in the schedule-definitions KV bucket and written into each agent's key of agent-schedules, both of which this daemon alone writes (GET/POST /api/schedules, PATCH/DELETE /api/schedules/{id}); each agent fires them itself → docs/swarm/schedules.md.
  • Inference settings — each agent's model, effort and subagent models, one key per agent in the agent-inference KV bucket, which this daemon alone writes (GET/PUT/DELETE /api/agents/{name}/inference); each agent applies them itself. GET also answers the model and effort the agent last reported in its turn-state header, kept in memory → docs/swarm/inference.md.
  • Resource caps — each agent's CPU and memory cap, one key per agent in the agent-limits KV bucket, which this daemon alone writes (GET/PUT/DELETE /api/agents/{name}/limits); the hive running the agent reads it, and each cap replaces the hive's own answer for that field → docs/scheduler/coordinator.md.
  • Review classifier — try-outs from the swarm UI and reviews on review requests, and the newest 100 finished runs, one document in the review-classifier-runs KV bucket, which this daemon alone writes (POST /api/review-classifier/try-out, GET /api/review-classifier/try-out/{id}, GET /api/review-classifier/rules, GET /api/review-classifier/runs) → docs/swarm/review-classifier.md.
  • Review classifier settings — the classifier's model, effort and how many requested reviews run at once, one document in the review-classifier-settings KV bucket, which this daemon alone reads and writes (GET/PUT /api/review-classifier/settings) → docs/swarm/review-classifier.md.
  • Relays — each agent's terminal and turn-state header as SSE, agent icons, the cross-repo issue report, and the UI's quick links.

It reads its configuration once at startup, from the environment the nix module sets. It persists webhook-secret and operator-matrix-admins.json in its state directory. The job graph lives in memory; hive status and wanted state live in the swarm queue, so both survive a restart.

Why a unix socket, not a port

The hive-gateway's nginx is the only intended client and reaches the socket through a bind-mount. A listener that is never bound to an address cannot be reached from off-host by mistake.

The socket path is services.hyperhive.deploy.swarm-controller.socketPath, default /run/swarm-controller/controller.sock, exported to the process as SWARM_CONTROLLER_SOCKET.

⚠️ The socket's directory is its access control

The socket is 0666. It has to be: nginx runs as a different user and connect(2) needs write. This matches how hive-c0re publishes the per-agent sockets, and rests on the same argument — "the bind source dir is per-agent on host so blast radius is unchanged."

What keeps that safe is that the directory holds one socket. So:

Never point socketPath at a directory that carries anything else. /run/hyperhive above all — it holds host.sock, the host admin socket. Pointing nginx at that directory to reach this socket would put the admin socket within its reach too.

nginx is a host service, so nothing narrows what it can reach except the directory itself — that is the whole of the access control. A unit test pins the default path so a tidying edit fails instead of reviewing cleanly.

RuntimeDirectoryPreserve=yes and the daemon's stale-socket unlink on start are a pair: preserving the directory without the unlink means bind fails with EADDRINUSE after a restart.

Packaging

Built by the workspace derivation and extracted as its own package (nix build .#swarm-controller). Deliberately not in nix/packages' daemonBins — that list is the core stack and drives the bundle services.hyperhive.c0re.package points at, so folding this in would put a swarm-scoped service into every hive's closure.