hyperhive/docs/2-handbook/swarm/services.md

Swarm-wide services

Some things exist once per swarm rather than once per hive: the forge, the matrix homeserver, SSO, the secret store, the queue, and the metrics and log stack. This page says which host runs them, what a hive that runs none of them configures instead, and what each service is for. The all-local quick start sets everything with one line → README.

Two options say where they live, and everything else derives:

services.hyperhive.deploy.singleHostSwarm = true;   # everything on this box
# or, for a dedicated services host with hives elsewhere:
services.hyperhive.deploy.allSwarmServices = true;

deploy.allSwarmServices is what "the swarm's shared services run here" means: every once-per-swarm service takes its enable from it. The service sections below don't repeat that. A service that needs the secret store comes on only where the host has the service's store identity, so a services host with the store elsewhere gets it once it holds a copy of that identity; set the service's own enable to override either way.

singleHostSwarm is the all-on-one-box mode above it. It defaults deploy.allSwarmServices, the swarm controller (deploy.swarm-controller.enable), the host's /etc/hosts entries for the names it serves (gateway.localHostsEntry) and where the secret store's bootstrap token goes (deploy.bao.bootstrapTokenFile). You can still set each derived toggle on its own, and that setting wins.

Both default to off, because a host can't tell whether it's meant to be the swarm's service host. With them off, a hive is a client of those services: it configures how to reach them and runs none of them.

That includes the forge (deploy.forgejo.enable). Every swarm needs one, being the canonical store for the meta flake and every agent's config repo, so exactly one host must turn it on: singleHostSwarm, allSwarmServices, or deploy.forgejo.enable = true set by hand.

A hive can't start or stop any of these. hivectl stop, start and restart act on the hive's agents and its gateway only, even on the host that runs the swarm's services. The forge and matrix containers are systemd units on that host (container@hive-forge.service, container@hive-matrix.service), and you manage them there with systemctl.

A hive that runs none of these reaches each one by name, the forge at forge.<swarm-domain> for example. Only the host running a service answers its name from its own resolver, so on a swarm spread over more than one host, your DNS has to resolve those names to that host.

Moving an existing hive's forge to the swarm's

A hive that stops running the forge keeps the old container's state at /var/lib/nixos-containers/hive-forge/. Nothing moves it to the swarm's forge: push anything worth keeping there by hand.

Deployment shapes

• All-local. Everything on one machine: singleHostSwarm = true, plus deploy.hive-controller.enable = true for a hive to run agents on. After the first switch, the steps in setup.md remain.
• Services on the swarm controller host. Set deploy.allSwarmServices and deploy.swarm-controller.enable there, with hives elsewhere. The controller doesn't derive from allSwarmServices.
• Fully spread out. One container, VM or machine per service.
• Semi-spread. The services split over a few machines by weight, hives on their own, a VPN between them, and most vhosts answering on the VPN only. A worked example with its NixOS files: semi-spread-deploy.md.

These are a set, not a ladder with a correct top, and in-between shapes work: you can set each derived toggle on its own, which makes "all local except X" a configuration rather than an unsupported edge case. Co-location is normal, and where one service has something to say about sharing a host, that belongs in the service's own doc.

Services

An operator who isn't spreading the swarm over more than one host can skip most of this page: singleHostSwarm turns on the shared services below, with two exceptions. The CI runner stays off until you set deploy.ci-runner.enable, and the builder and binary cache come on only where the host has their store identities. The builder and the CI runner are the two that cost real resources (they build and run pull-request code), so they're the two worth a look before you switch them on.

Service	What it does	Option	Details
Forge	Holds the meta flake and every agent's config repo.	deploy.forgejo.enable	forge.md
Matrix	The chat homeserver agents and the operator talk over.	deploy.matrix.enable	matrix.md
Secret store	Keeps the swarm's credentials.	deploy.bao.enable	credentials.md
Queue	Carries hive status and swarm events.	deploy.nats.enable	Deployment shapes
SSO	One login for the operator and the agents.	deploy.authelia.enable	SSO
Metrics	Stores telemetry and shows it in Grafana.	deploy.victoriametrics.enable, deploy.grafana.enable	Metrics
Logs	Stores every journal for search in Grafana.	deploy.victorialogs.enable	Logs
Telemetry collector	Routes metrics and logs to the stores and an optional upstream.	deploy.swarm-otel.enable	Telemetry collector
CI runner	Runs Forgejo Actions for pull requests. Heavy.	deploy.ci-runner.enable	CI runner
Builder	Evaluates and builds agent config PRs. Heavy.	deploy.builder.enable	Builder
Binary cache	Serves built closures to the hives.	deploy.attic.enable	Binary cache
Backup archive	Keeps every agent's state backups, one repository each.	deploy.archive.enable	Backup archive
CI metrics	Records CI run times in the metrics store.	deploy.ci-metrics.enable	CI metrics
Review classifier	Checks PRs against a repo's review rules.	deploy.review-classifier.enable	Review classifier
Claude login minter	Runs the log in button on an agent's card.	deploy.claude-login.enable	Claude login minter
Notification router	Turns forge events into notifications for agents.	deploy.notify-router.enable	Notification router
Speech to text	Turns voice memos into text for agents.	deploy.stt.enable	Speech to text

SSO (authelia)

One authelia per swarm, in a swarm-authelia container at auth.<swarm-domain>. The operator and the agents are both subjects of the same provider, told apart by roles and claims. Set deploy.authelia.enable to run it here. Every hive sets swarm.authelia.url, the same value on all of them: https://<swarm.authelia.domain> by default, or the other name when you join a swarm whose IdP lives elsewhere. Its keys live in the secret store, and nothing rotates them automatically: replacing one invalidates the sessions and the encrypted store, so that's an action you take on purpose.

Bootstrapping the first user and the login flow: sso.md. The container, its sockets, secrets and metrics: Host setup.

Metrics (VictoriaMetrics + Grafana)

The swarm's telemetry lands in one VictoriaMetrics, and one Grafana reads it, at metrics.<swarm-domain> and grafana.<swarm-domain>. They sit in two containers so you can restart or break Grafana without losing the time-series database. Enabling either enables both; to run one alone, set it directly with deploy.victoriametrics.enable or deploy.grafana.enable.

Log in to Grafana with your swarm SSO account. Grafana provisions its dashboards from the repo, so edits made in the UI aren't kept. Each service has its own board; open the folder to see them.

⚠️ The metrics store grows for as long as the swarm runs. Options to touch:

Option	When you'd touch it
deploy.victoriametrics.retentionPeriod	Default 5y. Lower it once you know how fast the swarm fills a disk. A too-short value discards history you can't get back.
swarm.grafana.oidc.role	Default Admin for everyone who logs in. Lower it to Viewer or Editor if some operators shouldn't reconfigure Grafana.

The dashboard list, the "No data" cases, and how the gateway guards the store: telemetry.md.

Logs (VictoriaLogs)

Each hive ships its journals to one VictoriaLogs at logs.<swarm-domain>, behind the same SSO as everything else: the swarm's service containers, the hive's daemons and the harness units in every agent container.

To read them, open Grafana, pick Explore and choose the VictoriaLogs datasource. Agents query it with swarm-logs, and need the logs permission.

Option	When you'd touch it
deploy.victorialogs.retentionPeriod	Default 30d, far shorter than the metrics store's. Raise it to answer questions about last quarter.

The other options, who may query, and the gateway rules: telemetry.md.

Telemetry collector (OTEL)

The swarm's collector receives from every hive's own collector and decides where telemetry goes. It always writes the metrics and log stores, and exports to deploy.swarm-otel.upstream.endpoint as well when you set one. It holds the upstream credential, so no hive and no agent needs one. Every hive reaches it by its gateway name, otel.<swarm-domain>, so a hive that doesn't run the swarm's services needs nothing set.

How the collector connects, its per-service routes and the homeserver's stand-in metrics: telemetry.md.

CI runner

One Forgejo Actions runner, in a swarm-ci-runner container on the host that sets deploy.ci-runner.enable. It doesn't derive from allSwarmServices: a runner executes pull-request code, so running one is a decision of its own. It needs neither the forge nor a hive on its host, and registers itself with the forge. Its ci runner board in Grafana shows its jobs and capacity, and the forge's CI runs tab shows the run times of every run. Registration, the metrics it exports and the threat model: Host setup and the rest of ci.md.

Builder

The swarm builder, in a swarm-builder container on the host that sets deploy.builder.enable; set it to false to opt out. It evaluates and builds unreviewed agent config PRs, and the result shows as a swarm-builder commit status on the PR. A red status blocks nothing: the merge still deploys. It also builds each agent before the swarm sends its deploy, and a failed build holds that deploy back. The jobs show on the swarm UI's Jobs page. It builds through its host's nix daemon, so it costs real CPU and disk. Host setup, and what it builds and why: builder.md.

Binary cache

The swarm's attic server, in a swarm-attic container on the host that sets deploy.attic.enable. The builder pushes green builds to it, and the hives pull closures from it at swarm.attic.domain (default attic.<swarm-domain>). For an external cache, override swarm.attic.{domain,cache} and don't set deploy.attic.enable anywhere, or set it to lib.mkForce false under allSwarmServices. Its keys and tokens create themselves in the secret store. How it works: Binary cache host setup and builder.md.

CI metrics

The CI run poller, a swarm-ci-metrics timer unit on the host that sets deploy.ci-metrics.enable; set it to false to opt out. Every five minutes it pushes the run time and queue wait of each newly finished forge Actions run to the metrics store. What it reads and pushes: ci-metrics.md.

Review classifier

The review classifier's try-out service, in a swarm-review-classifier container on the host that sets deploy.review-classifier.enable. You set the key for the runtime the swarm settings choose in the swarm UI: the classifier settings card on /jobs takes a key for that runtime and shows whether one exists and when it changed, never the key. "Set" means a key exists, not that it works, and every run spends money on that key; until the chosen one exists, every run fails with no API key… or no ACP provider key…. The service reads the key on every run, so writing or removing it takes effect on the next run. The store paths for the two keys, and what a run is and costs: Host setup and review-classifier.md.

Claude login minter

The claude login minter, in a swarm-claude-login container on the host that sets deploy.claude-login.enable. Run one: a login in progress lives in the service's memory between its start and its code. An operator logs an agent in from the log in button on its card in the swarm UI, and the agent's harness installs the new login. The flow and the version rule: claude-login.md. The service's host setup: Host setup.

Notification router

The notification router, in a swarm-notify-router container on the host that sets deploy.notify-router.enable. Run one: it handles each issue's events in order. It turns forge events (assigning an issue or pull request, closing it, a failed CI run on a pull request) into todos for the agents they concern, and an agent that's down gets its todos when it's back. The flows and the delivery path: notify-router.md, and Host setup.

Speech to text

The speech-to-text service, in a swarm-stt container on the host that sets deploy.stt.enable. Run it where the inference hardware is: it transcribes with whisper.cpp. An agent that receives a voice memo sends the audio to it over the queue and gets the text back, and a memo sent to several agents costs one run. deploy.stt.settings.modelSize picks the model (small by default, or base). The audio comes from rooms with outside people, so the service runs under a tight sandbox, but its network isn't restricted, which the design page spells out. The settings, limits and reasons: stt.md and Host setup.