140 lines
7.6 KiB
Markdown
140 lines
7.6 KiB
Markdown
# The swarm
|
|
|
|
The **swarm** is where things live: agent identities and accounts, secrets,
|
|
the job graph that creates and places agents, telemetry, and the UI you
|
|
drive it all from. **Hives are the substrate** — NixOS hosts that run agent
|
|
containers on the swarm's behalf. Every hive belongs to a swarm; a single
|
|
host is a swarm of one.
|
|
|
|
This page is for the operator. It covers the control plane, how a hive
|
|
joins the directory, and how hives and agents report upward. The steps for a
|
|
fresh swarm are in [`setup.md`](../../1-getting-started/setup.md); the all-local
|
|
config is the [README quick start](../../../README.md#quick-start-an-all-local-swarm).
|
|
|
|
Option reference: `services.hyperhive.swarm.*` (swarm-wide facts, identical
|
|
on every host) and `services.hyperhive.deploy.*` (whether _this_ host runs
|
|
grafana, the controller, authelia, …) →
|
|
[options reference](https://hyperhive.darkest.space/options/), or
|
|
`nix build .#docs-swarm` / `.#docs-deploy`.
|
|
|
|
## Where each piece lives
|
|
|
|
- **control plane** — `swarm-controller`: hive directory, agent list, job
|
|
graph, agent creation. → [below](#swarm-controller)
|
|
- **scheduled prompts** — named prompts for one or more agents at set times,
|
|
kept by the swarm and fired in each agent's container. → [`schedules.md`](schedules.md)
|
|
- **inference settings** — the model and effort the swarm sets for an agent,
|
|
over its own. → [`inference.md`](../../4-implementation/swarm/inference.md)
|
|
- **resource caps** — the CPU and memory cap the swarm sets for an agent,
|
|
over its hive's. →
|
|
[`coordinator.md`](../../4-implementation/scheduler/coordinator.md#container-resource-limits)
|
|
- **review classifier** — checks a PR against its repo's own yes/no rules
|
|
and posts the result as a review. →
|
|
[`review-classifier.md`](../../4-implementation/swarm/review-classifier.md)
|
|
- **notification router** — forge events turned into todos on the agents
|
|
they concern: assigned, unassigned, cleared on close; and the classifier's
|
|
review requested on a push. →
|
|
[`notify-router.md`](../../4-implementation/swarm/notify-router.md)
|
|
- **speech-to-text** — voice memos agents receive, turned into text once per
|
|
memo for the whole swarm. →
|
|
[`stt.md`](../../4-implementation/swarm/stt.md)
|
|
- **swarm UI** — the operator's day-to-day surface, on the swarm apex,
|
|
`admins` only. → [`ui.md`](ui.md)
|
|
- **shared services** — one forge, homeserver, SSO, queue and metrics/logs
|
|
stack, each on whichever host you put it. → [`services.md`](services.md)
|
|
- **forge** — the swarm's Forgejo: per-agent accounts, notifications and the
|
|
CLI agents use. → [`forge.md`](forge.md)
|
|
- **matrix** — the swarm's homeserver, several accounts per agent, and
|
|
federation. → [`matrix.md`](matrix.md)
|
|
- **backup archive** — where the hives back up each agent's state, how long it
|
|
stays and how to check it. → [`archive.md`](archive.md)
|
|
- **knowledge repo** — the reference docs every agent reads, and how to add
|
|
one. → [`knowledge.md`](knowledge.md)
|
|
- **`swarmctl`** — the swarm operator's CLI. → [`swarmctl-cli.md`](swarmctl-cli.md)
|
|
- **SSO** — the secrets authelia generates and how each reaches its reader.
|
|
→ [`sso.md`](sso.md)
|
|
- **secrets** — the rules a credential follows → [`credentials.md`](credentials.md);
|
|
every credential the swarm holds, who mints it, who reads it, how it renews
|
|
and where it lives → [`credentials-reference.md`](../../4-implementation/swarm/credentials-reference.md) ·
|
|
how the store comes up, who writes its grants and how it unseals →
|
|
[`bao.md`](../../4-implementation/swarm/bao.md)
|
|
- **swarm CA** — the store root every hive's gateway leaf chains to, and what to
|
|
hand a peer. → [`ca.md`](../../4-implementation/swarm/ca.md)
|
|
|
|
## Swarm controller
|
|
|
|
`swarm-controller` is the swarm's control plane: it holds the hive directory,
|
|
the agent list and their wanted state, and the job graph. Creating an agent —
|
|
from the swarm UI or `swarmctl agent create --hive <h>` — queues its SSO
|
|
identity, forge user, config repo, store identity and matrix account, then
|
|
sends the hive a deploy message. A new agent starts `paused`. A store identity
|
|
that fails to mint doesn't stop the creation; the controller mints it later
|
|
([how](../../4-implementation/swarm/controller.md#agent-identity-minting)).
|
|
|
|
`services.hyperhive.deploy.swarm-controller.enable` runs it on this host;
|
|
`singleHostSwarm` turns it on. **Otherwise off: set it on the one host that
|
|
runs the controller.** A swarm has one controller, so enabling it states a
|
|
fact about swarm topology, not about whether this host runs a hive.
|
|
|
|
What it serves, why it's a unix socket rather than a port, and the
|
|
socket-directory constraint that governs where `socketPath` may point:
|
|
[`swarm-controller/README.md`](../../../swarm-controller/README.md).
|
|
|
|
### What the controller keeps
|
|
|
|
- **Hive status.** Each hive publishes its state upward through the swarm queue.
|
|
The swarm UI's hive list shows when each last reported: `fresh`, `stale`,
|
|
`never_reported` or `unknown`.
|
|
`services.hyperhive.swarm.controller.staleAfterSeconds` (default `120`) sets
|
|
how long a hive may stay silent before it reads `stale`.
|
|
- **Agent queue access.** Each agent connects to the queue as itself, with a
|
|
credential the controller mints when it creates the agent.
|
|
- **An agent's `AGENTS.md`.** The swarm holds the revisions and the harness
|
|
applies them.
|
|
- **Forge objects and webhooks.** Swarm-wide labels, repos and webhooks the
|
|
controller registers on the forge.
|
|
|
|
The options, wire formats and failure modes of each:
|
|
[`controller.md`](../../4-implementation/swarm/controller.md).
|
|
|
|
### Which agents exist
|
|
|
|
The swarm decides. An agent exists in the swarm if and only if it has an
|
|
identity in the swarm's authelia identity store, which `swarmctl agent create`
|
|
or the swarm UI writes once. `GET /api/agents` on the swarm controller lists
|
|
them. A hive creates nothing at boot, so every agent, the manager `ruth`
|
|
included, comes from the swarm UI or `swarmctl agent create`, and destroying
|
|
one is as final for her as for any other.
|
|
|
|
The swarm UI's setup assistant creates `ruth` from the
|
|
[`ruth` archetype](archetypes.md#the-built-in-archetypes), which grants her
|
|
`manage_root_agent` and sets her model to `sonnet`. A ruth created before the
|
|
archetype keeps her model; set `sonnet` in her inference settings in the swarm UI
|
|
if you want it. What that grant lets an agent reach on a hive is on the
|
|
[agent permissions page](../agents/agent-permissions.md).
|
|
|
|
## Hives: the substrate
|
|
|
|
- **hive** — one host running `hive-c0re` and its agent containers
|
|
(`deploy.hive-controller.enable`). Addressed as `<hiveName>.<swarm.domain>`.
|
|
- **swarm** — every hive in `services.hyperhive.swarm.hives`, plus the
|
|
shared services and controller. You can qualify an agent as
|
|
`agent@hive-domain`.
|
|
- **peer hive** — any hive in the directory other than this one. Peers are
|
|
_derived_, not declared: the directory lists every hive including
|
|
yourself, and `hiveName` says which one you are.
|
|
|
|
Each hive carries an identity (`hiveName`, the `swarm.hives` directory) that
|
|
says which hive it's and how to reach the others; the directory feeds the
|
|
swarm UI's hive list and matrix federation. →
|
|
[the hive directory](hive-directory.md).
|
|
|
|
## Cross-references
|
|
|
|
- [`ui.md`](ui.md) — the swarm UI, the operator surface for "what hives exist"
|
|
- [`5-project/process/conventions.md`](../../5-project/process/conventions.md) § Hive identity —
|
|
env vars, qualified labels
|
|
- [`2-handbook/swarm/matrix.md`](matrix.md) — matrix
|
|
federation, TLS cert autogeneration, firewall posture
|
|
- [`4-implementation/networking/gateway.md`](../../4-implementation/networking/gateway.md) — nginx vhosts and
|
|
the `.well-known/matrix/` autodiscovery scheme
|