A replay consumer now has a 5s idle heartbeat, and the row stream ends at the first consumer error instead of skipping it. When the server drops the consumer, as on a restart, two missed heartbeats end the SSE response and the browser reconnects with Last-Event-ID. Before, the response stayed open and delivered nothing. A Last-Event-ID above the stream's newest sequence now replays as a fresh reader. The server accepts a start past the end and delivers nothing until the stream reaches it, so a stale id stalled the replay. swarm-controller's stream tests run against a real nats-server on loopback (`test_nats`, binary from HIVE_NATS_SERVER, exported by checks.nix and devshell.nix). They cover: ensuring the stream twice, an existing stream with other limits being opened as it is, the replay arriving before live rows, a reconnect getting every later row exactly once, and a dropped consumer ending the row stream. The limits test now compares against the subagent stream's own config. Two comments that described the agent's terminal route as live-only are corrected.
18 KiB
Swarm UI
The swarm's own web surface and the operator's day-to-day view: served by
the gateway on the swarm apex (services.hyperhive.swarm.domain),
readable only by operators. The per-hive dashboard, on each hive's own
domain, covers host-level detail for one hive.
What it shows
| route | what |
|---|---|
/ |
the hive directory, each hive with its last reported status |
/agents |
every agent: status, config PR, wanted state, inference settings; create agents, link forge, matrix and GitHub accounts |
/agents/<name>/terminal |
one agent's live terminal |
/agents/<name>/subagents/<subagent>/terminal |
one of that agent's subagents' live terminal |
/jobs |
the controller's job graph — where agent creation and credential mints show progress |
/schedules |
the scheduled prompts: add, edit, pause, resume and remove |
/issues |
a cross-repo issue report |
Everything it shows comes from swarm-controller.
An agent created here or with swarmctl agent create starts paused; set it
up from its card.
Agent and subagent terminals
An agent's detail panel on /agents shows a small live preview of its
terminal, and below it the agent's subagents: every subagent the swarm
queue holds terminal rows for from the last 24 hours. Picking one shows its
terminal in the same preview, and expand opens it in its own tab, as it
does for the agent. The list refreshes every 10 seconds, so a subagent the
agent spawns shows up once it writes output.
An agent's terminal opens with its last 100 rows from the past 24 hours, then follows new ones; a reconnect picks up after the last row it showed. A subagent terminal shows rows published while it's open, and has no turn-state badges.
Scheduled prompts
/schedules has one card per scheduled prompt. The name is the card's
title, with the agents it fires for, the message, how often it repeats and
when it fires next. edit turns the card into its own form; pause and
resume stop and restart it; remove deletes it after a confirmation.
+ schedule opens an empty form above the list, and the filters narrow the
list by name, agent or state (active, paused, or past for a one-shot
whose time has passed). past is the controller's clock against the start time,
not a record that an agent fired, so you can still pause a past schedule.
A schedule repeats every some number of seconds, minutes, hours or days, or fires once. Its start time is the moment a one-shot fires, and the origin of the fire times of a repeating one. The form previews the next three fire times. Changing the start or the repeat moves them; editing only the name, message or agents doesn't, and the form warns when an edit would.
A start in the past fires the latest missed time once, right away, in every agent chosen, and the form says so.
Only an operator can change a schedule here or with swarmctl schedule; an
agent has no way to add one. The page needs the controller's /api/schedules
routes.
Sending a message to an agent
Under an agent's terminal, in the preview and in its own tab, is a message box. send hands the text to the agent's harness, and the agent takes it as an inbox message from your authelia username. sent means the harness was listening and holds the message. If the harness isn't running, or isn't connected to the swarm queue, the box shows an error and the agent gets nothing. Subagents have no message box.
The message doesn't go through the agent's hive, so the hive dashboard's
message history doesn't list it. The agent's answer shows in its terminal; an
agent that calls send to your username gets an unknown-recipient error. A
harness that restarts before taking its turn loses the message.
How a message reaches the agent
The gateway sets Remote-User from authelia's answer on every /api/
request, replacing any the browser sent. POST /api/agents/<name>/input
takes that header as the sender and sends {from, body} as a NATS request on
$SWARM.input.<name>. swarm-controller is the only queue client allowed to
publish there. The agent's harness subscribes to its own subject and queues
each message for its turn loop with its reminders. Only the agent's own queue
credential may subscribe there. The harness doesn't reply, so swarm-controller
answers 202 when the request times out after two seconds, and 503 when the
queue reports no subscriber.
Where the subagent list and rows come from
The agent's subagent daemon publishes each subagent's terminal rows on
$SWARM.term.<agent>.sub.<subagent> with the agent's own queue credential,
into a stream named term-sub-<agent> that swarm-controller creates for every
declared agent. The stream keeps rows for 24 hours. swarm-controller serves the list as
GET /api/agents/<name>/subagents, the subagents named by the subjects in that
stream, each with last_row_unix, the stream's publish time of its newest row,
or null when that row isn't read within two seconds. The lookups run
concurrently, sixteen at a time, so the list waits at most two seconds for
each sixteen subagents.
swarm-controller also relays one subagent's rows as SSE on
GET /api/agents/<name>/subagents/<subagent>/term/stream. An agent with no
store identity, or no queue address, publishes no subagent terminals.
Where an agent's terminal history comes from
The harness publishes the agent's terminal rows on $SWARM.term.<agent>, into
a stream named term-main-<agent> that swarm-controller creates for every
declared agent, with the same 24 hours or 64 MiB as the subagent stream. That
stream holds this one subject, and the subagent stream holds only subjects
below $SWARM.term.<agent>.sub, so they never overlap.
GET /api/agents/<name>/term/stream creates a short-lived consumer on it
that starts 100 rows back, so the SSE stream sends those rows and then each
new one. Each event's id is the row's stream sequence, and a request with
Last-Event-ID resumes after it. An id past the newest row of the stream, as
after swarm-controller recreates the stream, starts over as a fresh reader. When the
consumer stops sending its heartbeat, as after a queue restart, the route ends
the response and the browser reconnects. Until the stream exists, the route sends
live rows only.
Inference settings
The agent's detail panel shows its model and effort under inference, each
with where it comes from: the swarm, its agent.nix, its own pick, or the
default. Below them is the subagent model list the swarm sets, or "its own"
when it sets none. edit opens a dialog with the three fields, filled with
what the swarm stores; saving does a PUT /api/agents/{agent}/inference of
the whole value. A field you leave empty stays out of it, so the agent keeps
its own value for that field. clear the swarm's setting sends the DELETE.
The panel reads GET /api/agents/{agent}/inference; before the agent's first
report reaches the controller, it shows the swarm's stored values only.
inference describes the settings themselves.
Linking external accounts
Each agent on /agents opens three dialogs that write a credential for it
into the swarm secret store through swarm-controller. All three are blind
set actions: no route hands a token back. swarm-controller refuses to link a
name that already holds an account; delete that account first to replace it.
The agent's detail panel lists its linked accounts under accounts, one row
per account: kind, name and host. Opening an agent makes one request,
GET /api/hives/{hive}/agents/{agent}/linked-accounts, which returns every
account of that agent as names and hosts and never a credential.
| kind | one row per | name | host |
|---|---|---|---|
| matrix | swarm/agents/<agent>/matrix/<account> |
the account | its homeserver, if set |
| forgejo | swarm/agents/<agent>/forge/<label> |
the label | its url |
| github | swarm/agents/<agent>/github-token, if any |
github |
github.com |
The matrix main row is the agent's own account, which the swarm mints;
it carries an own account badge. The panel requests the list again when
a link dialog closes.
- link a matrix account —
PUT /api/hives/{hive}/agents/{agent}/matrix-accounts/{account}, either a pasted bearer token or a user id + password that swarm-controller logs in with, storing the token it gets back. - link a forge account —
PUT /api/hives/{hive}/agents/{agent}/forge-accounts/{label}with a forge URL and an access token, stored atswarm/agents/<agent>/forge/<label>. The agent'shive-agent-forge-accountsunit fetches it into<state>/forge-<label>-tokenand<state>/forge-<label>.json, the fileshive-forge -f <label>reads. The unit rewrites a pair whenever the store's account for its label differs, and never deletes one: a pair whose label the store doesn't list stays untouched. swarm-controller answers 409 to a link for a label that already holds an account, so replacing one takes a delete, then a new link. - link a github account —
PUT /api/hives/{hive}/agents/{agent}/github-accountwith a personal access token, stored atswarm/agents/<agent>/github-token. One token per agent. The agent'shive-agent-github-tokenunit fetches it into<state>/github-token, the file itsghwrapper, git credential helper and GitHub notification poller read. Agithub-tokenalready in place stays when the store holds none. What the token needs and how the agent uses it: GitHub accounts.
Deleting a linked account
Every row except the matrix main row has a delete action. Its
confirmation names the account and its host, and confirming sends one
request:
| kind | route |
|---|---|
| matrix | DELETE /api/hives/{hive}/agents/{agent}/matrix-accounts/{account} |
| forgejo | DELETE /api/hives/{hive}/agents/{agent}/forge-accounts/{label} |
| github | DELETE /api/hives/{hive}/agents/{agent}/github-account |
swarm-controller removes every version of the entry from the swarm secret
store, and answers 404 when the store holds nothing there. It refuses main: the
swarm mints that account and would re-mint it. The panel requests the list
again after a delete, including a 404 one: the account being already gone is
the end state the delete wanted, so the dialog closes without an error.
The matrix confirmation has a checkbox, off by default, that logs the token
out at its homeserver first (?revoke=true). A 401 M_UNKNOWN_TOKEN from
that logout means the homeserver already doesn't recognise the token, so the
delete proceeds as if it had succeeded. Any other failed logout, or an account
with no homeserver stored, leaves the account in the store and the dialog
shows why. Deleting a forge account or GitHub token leaves the token valid at
its provider; revoke it there.
The agent isn't told about a delete. Its matrix daemon drops the account when
it next lists the store, within two minutes. Its forge and GitHub units never
delete a file, so a token already fetched into <state> stays there.
Where each credential lives and who reads it:
credentials.md.
Enabling
services.hyperhive.deploy.swarm-ui.enable = true; # defaults to deploy.swarm-controller.enable
Derived from the controller rather than from allSwarmServices:
the UI is a view onto the controller's state and reaches it over that
daemon's socket, so the host that runs the controller is the host that
can serve the UI. A hive that merely uses a swarm has nothing to serve.
The UI answers on services.hyperhive.swarm.domain and nothing else. It
shares that name with the swarm-controller it fronts — one service to a
reader and to a certificate — so there is no separate option to pin.
The apex must differ from
services.hyperhive.domain. The gateway's default server already answers for the hive domain, and two vhosts claiming oneserver_namedon't error — nginx picks one — so this is an assertion rather than a runtime surprise.
🔑 You must be in the admins group
This is the step that separates "protected" from "locked out." The
vhost's auth_request asks authelia "is there a session"; the rule
that makes it mean "is this an operator" is an access_control entry
requiring group:admins. An account without that group authenticates
fine and still gets bounced.
swarmctl user add <you> --email <you>@example.com --group admins
swarmctl user update <you> --add-group admins # an account that already exists
--email isn't needed for the UI, but the forge won't create your account
without one → setup.md § 2.
Why a group and not "any session": agents are authelia subjects too, so authenticated includes every agent in the swarm. The group is the only thing standing between "an operator's page" and "anyone with a session."
What it costs to be reachable
hyperhive publishes the apex to the hive's resolver like every other swarm service, so agent containers can resolve it. That's deliberate and it's not a hole: reachability isn't the access control here. An agent that resolves the name and connects still has no operator session, and the subrequest denies it.
Quick links
The swarm UI's header carries a single 🔗 button, visible on every route,
opening a popover of links to other swarm-wide services. Backed by
GET /api/links (swarm-controller), which serves
services.hyperhive.swarm.controller.links (a listOf { label, icon, url },
same shape as the per-agent services.hyperhive.agent.dashboardLinks).
Links to swarm services come from swarm-level options, so the list is the same whichever host runs each service:
| link | URL built from |
|---|---|
| Forge | services.hyperhive.swarm.forge.domain |
| Authelia | services.hyperhive.swarm.authelia.domain |
| Grafana | services.hyperhive.swarm.grafana.domain |
| Metrics | services.hyperhive.swarm.victoriametrics.domain |
| Logs | services.hyperhive.swarm.victorialogs.domain |
| Matrix | services.hyperhive.swarm.matrix.gatewayHost |
| Bao | services.hyperhive.swarm.bao.ui.domain |
Each entry is present whenever its option has a value — every option in the table defaults to one, so the link appears even on a swarm that runs no instance of that service.
nix/host-modules/swarm-controller.nix builds these entries. swarm-ui.nix
adds one more, for this UI's own API docs. An operator can add entries
directly. An empty list hides the button.
The Matrix entry opens the swarm's matrix web client (fluffychat,
services.hyperhive.deploy.matrix.gui.package) at the homeserver's
gateway host, chat.<swarm domain> by default. The homeserver's host
always serves the client at / on that vhost, so the entry is present
whenever the swarm names a gateway host.
The Bao entry opens the secret store's web UI, which admits members of
authelia's admins group only; see secrets.md.
Adding a swarm service name: the two wiring sites
Adding a swarm service name means touching two things. Missing the second ships as a different flavour of "works from the host, broken from a container":
| site | file |
|---|---|
vhost + gateway.localNames |
the service's own module (for example nix/host-modules/swarm-ui.nix) |
| certificate name | nix/host-modules/swarm.nix (serviceDomains) |
The DNS record and the local-dev /etc/hosts entry need no separate
edit: both derive from services.hyperhive.gateway.localNames,
which a service's own module already has to push its domain into to be
resolvable — see nix/host-modules/hive-gateway/dnsmasq.nix and
.../default.nix's networking.hosts. vhosts.nix itself is scoped to
the surface the hive's own domain serves (dashboard, per-agent routing,
matrix discovery); a swarm service declares its own vhost next to its
own options, the way swarm-ui.nix and swarm-authelia.nix do.
⚠️ The certificate one is the hardest to predict and the most visible when
missed. serviceDomains is both the allowed_domains the secret
store's pki/roles/swarm-services narrows to and the list each gateway's
leaf draws its SANs from (it carries the ones that host fronts),
and the apex is a sibling of forge.<swarm> / chat.<swarm> /
auth.<swarm>, not a parent — no CA in the hierarchy issues for it
implicitly. Left out, the vhost falls back to the hive leaf and the
swarm's front page opens with a name mismatch.
Cross-references
sso.md— the authelia instance itself, and the user store.../networking/gateway.md— the full vhost map and TLS modes.