hyperhive/swarm-controller

swarm-controller

The swarm-level daemon. Where hive-c0re owns the agents on one host, this owns what is true across hives — so a swarm runs one of them and most hives leave it off.

Opt-in per host via services.hyperhive.deploy.swarm-controller.enable, which is deliberately not derived from services.hyperhive.deploy.hive-controller.enable: turning it on is a statement about swarm topology, not about whether this host runs a hive.

What it does

The swarm's control plane. The swarm UI and swarmctl are its clients.

• Hive directory — serves swarm.hives (GET /api/hives) and what each hive last published about itself (GET /api/hives/status).
• Agent roster and wanted state — every agent the swarm knows (GET /api/agents, /api/agents/status), and the state it declares for each one on its hive (up/offline/paused/destroyed, PUT /api/hives/{hive}/agents/{agent}/state). An agent that has never reported takes its hive and declared state in /api/agents/status from the one hive whose declaration places it.
• Job graph — a hive-jobq scheduler, served at GET /api/jobq/graph. Every provisioning step below runs as a node in it. A node that fails logs one warn line, swarm jobq: node failed, with its node id, kind and error.
• Agent creation — POST /api/agents queues the SSO identity (through swarm-authelia-bridge), forge user, config repo, store identity, forge token and matrix account, declares the agent paused, then sends its hive a deploy message.
• Agent credentials — at start and every five minutes it re-checks every agent's forge token and matrix account, and renews store certificates and queue secrets as they age.
• Linked external accounts — an operator-supplied matrix, forge or GitHub account, or backend environment, for one agent, stored in the swarm secret store (PUT /api/hives/{hive}/agents/{agent}/matrix-accounts/{account}, .../forge-accounts/{label}, .../github-account, .../backend-env, whose save replaces what is stored); distinct from the agent's own swarm-minted accounts above. GET .../linked-accounts lists them, plus the agent's own main matrix account, by kind, name and host, never with a credential. Each of the four also takes a DELETE; matrix's also takes ?revoke=true, which logs the stored token out at its homeserver first and leaves the account in place if that fails.
• Config PR status — each agent's open config-repo PR, cached from forge webhooks (GET /api/config-prs, /api/agents/{name}/config-pr).
• Swarm-wide forge objects and webhooks → docs/swarm/README.md.
• Scheduled prompts — named prompts for one or more agents, kept in the schedule-definitions KV bucket and written into each agent's key of agent-schedules, both of which this daemon alone writes (GET/POST /api/schedules, PATCH/DELETE /api/schedules/{id}); each agent fires them itself → docs/swarm/schedules.md.
• Inference settings — each agent's model, effort, subagent models and cap on concurrently running subagents, one key per agent in the agent-inference KV bucket, which this daemon alone writes (GET/PUT/DELETE /api/agents/{name}/inference); each agent applies them itself. GET also answers the model and effort the agent last reported in its turn-state header, kept in memory → docs/swarm/inference.md.
• Resource caps — each agent's CPU and memory cap, one key per agent in the agent-limits KV bucket, which this daemon alone writes (GET/PUT/DELETE /api/agents/{name}/limits); the hive running the agent reads it, and each cap replaces the hive's own answer for that field → docs/scheduler/coordinator.md.
• Egress allow-list — the destinations agents may reach through their hive's egress proxy, each for every agent or for one, as one value in the egress-allow KV bucket, which this daemon alone writes (GET/POST/DELETE /api/egress/allowed-domains); every hive reads it and renders the proxy's allow-list files, which add to the names the hive's own configuration lists. It also keeps a derived entry per agent for the host of its stored backend environment's ANTHROPIC_BASE_URL → docs/networking/network.md.
• Agent instructions — each agent's AGENTS.md as its harness last published it into the agents-md KV bucket (GET /api/agents/{name}/agents-md), and edits to it as git diffs, which this daemon hands to the agent's harness to apply (PUT /api/agents/{name}/agents-md). POST /api/agents can set the text the file starts with (agents_md), stored in the agents-md-initial bucket for the harness to write when no file exists → docs/swarm/README.md.
• Review classifier — try-outs from the swarm UI and reviews on review requests, the requested reviews not yet finished, in the review-requests work-queue stream, and the newest 100 finished runs, one document in the review-classifier-runs KV bucket, both of which this daemon alone writes (POST /api/review-classifier/try-out, GET /api/review-classifier/try-out/{id}, GET /api/review-classifier/rules, GET /api/review-classifier/runs), and the try-out page's branch for a PR, pushed as the classifier's forge account (POST /api/review-classifier/open-pr) → docs/swarm/review-classifier.md.
• Review classifier settings — the classifier's runtime, model, effort and how many requested reviews run at once, one document in the review-classifier-settings KV bucket, which this daemon alone reads and writes (GET/PUT /api/review-classifier/settings) → docs/swarm/review-classifier.md.
• Notification flows — which of the notification router's built-in flows are switched off, and the custom flows, as one document in the notify-flows KV bucket, which this daemon alone writes (GET/PUT /api/notify/flows); the router reads it. GET /api/notify/deliveries and GET /api/notify/dry-run ask the router for its deliveries log and a dry run of the saved flows → docs/swarm/notify-router.md.
• Operator profiles — each operator's name and the pronouns on their forge profile, for every member of authelia's operator group, as one list under the one key of the operator-profiles KV bucket. This daemon alone writes it, at start and every five minutes, and only when the list changed. A forge read that fails keeps that operator's stored entry, and a name that leaves the group drops out of the list. Every agent reads the key into its system prompt → docs/turn-loop/claude-invocation.md.
• Builder runs — the swarm builder's newest 100 finished builds of agent config PRs, with nix's tail for a failed one, one document in the builder-runs KV bucket, which this daemon alone writes (GET /api/builder/runs, GET /api/builder/runs/{id}, the link on each swarm-builder commit status) → docs/swarm/builder.md.
• Meta fan-out — the hyperhive pin of internal/meta that this daemon last sent the hives a deploy for, one key in the meta-fanout KV bucket, which this daemon alone reads and writes → docs/swarm/builder.md.
• Relays — each agent's terminal and turn-state header as SSE, agent icons, the cross-repo issue report, and the UI's quick links.
• Alerts — the swarm Grafana's current alerts (GET /api/alerts), read at request time with a Viewer service-account token from the secret store → docs/swarm/credentials-reference.md.

It reads its configuration once at startup, from the environment the nix module sets. It keeps the webhook HMAC secret in the secret store and operator-matrix-admins.json in its state directory. The job graph lives in memory; hive status and wanted state live in the swarm queue, so both survive a restart.

Why a unix socket, not a port

The hive-gateway's nginx is the only intended client (swarmctl runs as root). A listener that is never bound to an address cannot be reached from off-host by mistake.

The socket path is services.hyperhive.deploy.swarm-controller.socketPath, default /run/swarm-controller/controller.sock, exported to the process as SWARM_CONTROLLER_SOCKET.

⚠️ The socket's directory is its access control

The socket is 0666. It has to be: nginx runs as a different user and connect(2) needs write. This matches how hive-c0re publishes the per-agent sockets, and rests on the same argument — "the bind source dir is per-agent on host so blast radius is unchanged."

What keeps that safe is that the directory is 0750, owned by the swarm-controller-clients group, and holds one socket. nginx is in that group through its unit's SupplementaryGroups; root bypasses mode bits, so swarmctl needs no membership. Every other local user gets EACCES at the directory. The group is not swarm-controller's own, which also owns the state directory. So:

Never point socketPath at a directory that carries anything else. /run/hyperhive above all — it holds host.sock, the host admin socket. Pointing nginx at that directory to reach this socket would put the admin socket within its reach too.

nginx is a host service, so nothing narrows what it can reach except the directory itself — that is the whole of the access control. The unit sets the group with a root ExecStartPre (chgrp), because systemd chowns the runtime directory to the unit's own group on every start. A unit test pins the default path so a tidying edit fails instead of reviewing cleanly.

RuntimeDirectoryPreserve=yes and the daemon's stale-socket unlink on start are a pair: preserving the directory without the unlink means bind fails with EADDRINUSE after a restart.

Packaging

Built by the workspace derivation and extracted as its own package (nix build .#swarm-controller). Deliberately not in nix/packages' daemonBins — that list is the core stack and drives the bundle services.hyperhive.deploy.hive-controller.package points at, so folding this in would put a swarm-scoped service into every hive's closure.