d81fb2236f
GET /api/agents/{name}/subagents reads each subagent's newest row with
get_last_raw_message_by_subject, which publishes
$JS.API.STREAM.MSG.GET.term-sub-<agent>. The controller's NATS grant had
no STREAM.MSG.GET, so every lookup was refused and waited the 10s request
timeout, one subagent after another; six subagents passed nginx's 60s.
Grant `$JS.API.STREAM.MSG.GET.*`. A wildcard matches a whole subject
token only, so a `term-sub-*` spelling would name one stream literally;
`*` is the narrowest grant that covers the per-agent streams, the same
width as the controller's STREAM.INFO.* grant.
The lookups now run concurrently (16 at a time) with a 2s timeout each.
A lookup that fails or times out lists its subagent with a null
last_row_unix, and one warn per request names how many failed and the
first error.
307 lines
16 KiB
Markdown
307 lines
16 KiB
Markdown
# Swarm UI
|
|
|
|
The swarm's own web surface and the operator's day-to-day view: served by
|
|
the gateway on the **swarm apex** (`services.hyperhive.swarm.domain`),
|
|
readable only by operators. The per-hive dashboard, on each hive's own
|
|
domain, covers host-level detail for one hive.
|
|
|
|
## What it shows
|
|
|
|
| route | what |
|
|
| ---------------------------------------------- | --------------------------------------------------------------------------------------------------- |
|
|
| `/` | the hive directory, each hive with its last reported status |
|
|
| `/agents` | every agent: status, config PR, wanted state; create agents, link forge, matrix and GitHub accounts |
|
|
| `/agents/<name>/terminal` | one agent's live terminal |
|
|
| `/agents/<name>/subagents/<subagent>/terminal` | one of that agent's subagents' live terminal |
|
|
| `/jobs` | the controller's job graph — where agent creation and credential mints show progress |
|
|
| `/schedules` | the scheduled prompts: add, edit, pause, resume and remove |
|
|
| `/issues` | a cross-repo issue report |
|
|
|
|
Everything it shows comes from [`swarm-controller`](../../swarm-controller/README.md).
|
|
An agent created here or with `swarmctl agent create` starts `paused`; set it
|
|
`up` from its card.
|
|
|
|
### Agent and subagent terminals
|
|
|
|
An agent's detail panel on `/agents` shows a small live preview of its
|
|
terminal, and below it the agent's **subagents**: every subagent the swarm
|
|
queue holds terminal rows for from the last 24 hours. Picking one shows its
|
|
terminal in the same preview, and **expand** opens it in its own tab, as it
|
|
does for the agent. The list refreshes every 10 seconds, so a subagent the
|
|
agent spawns shows up once it writes output.
|
|
|
|
A preview shows rows published while it's open. A subagent terminal has no
|
|
turn-state badges.
|
|
|
|
### Scheduled prompts
|
|
|
|
`/schedules` has one card per scheduled prompt. The **name** is the card's
|
|
title, with the **agents** it fires for, the message, how often it repeats and
|
|
when it fires next. **edit** turns the card into its own form; **pause** and
|
|
**resume** stop and restart it; **remove** deletes it after a confirmation.
|
|
**+ schedule** opens an empty form above the list, and the filters narrow the
|
|
list by name, agent or state (`active`, `paused`, or `past` for a one-shot
|
|
whose time has passed). `past` is the controller's clock against the start time,
|
|
not a record that an agent fired, so you can still pause a `past` schedule.
|
|
|
|
A schedule repeats **every** some number of seconds, minutes, hours or days, or
|
|
fires **once**. Its start time is the moment a one-shot fires, and the origin of
|
|
the fire times of a repeating one. The form previews the next three fire times.
|
|
Changing the start or the repeat moves them; editing only the name, message or
|
|
agents doesn't, and the form warns when an edit would.
|
|
|
|
A start in the past fires the latest missed time once, right away, in every
|
|
agent chosen, and the form says so.
|
|
|
|
Only an operator can change a schedule here or with `swarmctl schedule`; an
|
|
agent has no way to add one. The page needs the controller's `/api/schedules`
|
|
routes.
|
|
|
|
### Sending a message to an agent
|
|
|
|
Under an agent's terminal, in the preview and in its own tab, is a **message**
|
|
box. **send** hands the text to the agent's harness, and the agent takes it as
|
|
an inbox message from your authelia username. **sent** means the harness was
|
|
listening and holds the message. If the harness isn't running, or isn't
|
|
connected to the swarm queue, the box shows an error and the agent gets nothing.
|
|
Subagents have no message box.
|
|
|
|
The message doesn't go through the agent's hive, so the hive dashboard's
|
|
message history doesn't list it. The agent's answer shows in its terminal; an
|
|
agent that calls `send` to your username gets an unknown-recipient error. A
|
|
harness that restarts before taking its turn loses the message.
|
|
|
|
<details><summary>How a message reaches the agent</summary>
|
|
|
|
The gateway sets `Remote-User` from authelia's answer on every `/api/`
|
|
request, replacing any the browser sent. `POST /api/agents/<name>/input`
|
|
takes that header as the sender and sends `{from, body}` as a NATS request on
|
|
`$SWARM.input.<name>`. swarm-controller is the only queue client allowed to
|
|
publish there. The agent's harness subscribes to its own subject and queues
|
|
each message for its turn loop with its reminders. Only an agent connected with
|
|
its own queue credential may subscribe there; an agent on its hive's shared
|
|
client takes no input this way, and the request gets the 503 below. The harness doesn't reply,
|
|
so swarm-controller answers 202 when the request times out after two seconds,
|
|
and 503 when the queue reports no subscriber.
|
|
|
|
</details>
|
|
|
|
<details><summary>Where the subagent list and rows come from</summary>
|
|
|
|
The agent's subagent daemon publishes each subagent's terminal rows on
|
|
`$SWARM.term.<agent>.sub.<subagent>` with the agent's own queue credential,
|
|
into a stream named `term-sub-<agent>` that swarm-controller creates for every
|
|
declared agent. The stream keeps rows for 24 hours. swarm-controller serves the list as
|
|
`GET /api/agents/<name>/subagents`, the subagents named by the subjects in that
|
|
stream, each with `last_row_unix`, the stream's publish time of its newest row,
|
|
or `null` when that row isn't read within two seconds. The lookups run
|
|
concurrently, sixteen at a time, so the list waits at most two seconds for
|
|
each sixteen subagents.
|
|
swarm-controller also relays one subagent's rows as SSE on
|
|
`GET /api/agents/<name>/subagents/<subagent>/term/stream`. An agent with no
|
|
store identity, or no queue address, publishes no subagent terminals.
|
|
|
|
</details>
|
|
|
|
### Linking external accounts
|
|
|
|
Each agent on `/agents` opens three dialogs that write a credential for it
|
|
into the swarm secret store through swarm-controller. All three are blind
|
|
set actions: no route hands a token back. swarm-controller refuses to link a
|
|
name that already holds an account; delete that account first to replace it.
|
|
|
|
The agent's detail panel lists its linked accounts under **accounts**, one row
|
|
per account: kind, name and host. Opening an agent makes one request,
|
|
`GET /api/hives/{hive}/agents/{agent}/linked-accounts`, which returns every
|
|
account of that agent as names and hosts and never a credential.
|
|
|
|
| kind | one row per | name | host |
|
|
| ------- | ------------------------------------------- | ----------- | ------------------------ |
|
|
| matrix | `swarm/agents/<agent>/matrix/<account>` | the account | its `homeserver`, if set |
|
|
| forgejo | `swarm/agents/<agent>/forge/<label>` | the label | its `url` |
|
|
| github | `swarm/agents/<agent>/github-token`, if any | `github` | `github.com` |
|
|
|
|
The matrix `main` row is the agent's own account, which the swarm mints;
|
|
it carries an **own account** badge. The panel requests the list again when
|
|
a link dialog closes.
|
|
|
|
- **link a matrix account** — `PUT /api/hives/{hive}/agents/{agent}/matrix-accounts/{account}`,
|
|
either a pasted bearer token or a user id + password that
|
|
swarm-controller logs in with, storing the token it gets back.
|
|
- **link a forge account** — `PUT /api/hives/{hive}/agents/{agent}/forge-accounts/{label}`
|
|
with a forge URL and an access token, stored at
|
|
`swarm/agents/<agent>/forge/<label>`. The agent's
|
|
`hive-agent-forge-accounts` unit fetches it into
|
|
`<state>/forge-<label>-token` and `<state>/forge-<label>.json`, the files
|
|
`hive-forge -f <label>` reads. The unit rewrites a pair whenever the
|
|
store's account for its label differs, and never deletes one: a pair whose
|
|
label the store doesn't list stays untouched. swarm-controller answers 409
|
|
to a link for a label that already holds an account, so replacing one
|
|
takes a delete, then a new link.
|
|
- **link a github account** — `PUT /api/hives/{hive}/agents/{agent}/github-account`
|
|
with a personal access token, stored at `swarm/agents/<agent>/github-token`.
|
|
One token per agent. The agent's
|
|
`hive-agent-github-token` unit fetches it into `<state>/github-token`,
|
|
the file its `gh` wrapper, git credential helper and GitHub notification
|
|
poller read. A `github-token` already in place stays when the store holds
|
|
none. What the token needs and how the agent uses it:
|
|
[GitHub accounts](../integrations/github.md).
|
|
|
|
#### Deleting a linked account
|
|
|
|
Every row except the matrix `main` row has a **delete** action. Its
|
|
confirmation names the account and its host, and confirming sends one
|
|
request:
|
|
|
|
| kind | route |
|
|
| ------- | ------------------------------------------------------------------- |
|
|
| matrix | `DELETE /api/hives/{hive}/agents/{agent}/matrix-accounts/{account}` |
|
|
| forgejo | `DELETE /api/hives/{hive}/agents/{agent}/forge-accounts/{label}` |
|
|
| github | `DELETE /api/hives/{hive}/agents/{agent}/github-account` |
|
|
|
|
swarm-controller removes every version of the entry from the swarm secret
|
|
store, and answers 404 when the store holds nothing there. It refuses `main`: the
|
|
swarm mints that account and would re-mint it. The panel requests the list
|
|
again after a delete, including a 404 one: the account being already gone is
|
|
the end state the delete wanted, so the dialog closes without an error.
|
|
|
|
The matrix confirmation has a checkbox, off by default, that logs the token
|
|
out at its homeserver first (`?revoke=true`). A 401 `M_UNKNOWN_TOKEN` from
|
|
that logout means the homeserver already doesn't recognise the token, so the
|
|
delete proceeds as if it had succeeded. Any other failed logout, or an account
|
|
with no homeserver stored, leaves the account in the store and the dialog
|
|
shows why. Deleting a forge account or GitHub token leaves the token valid at
|
|
its provider; revoke it there.
|
|
|
|
The agent isn't told about a delete. Its matrix daemon drops the account when
|
|
it next lists the store, within two minutes. Its forge and GitHub units never
|
|
delete a file, so a token already fetched into `<state>` stays there.
|
|
|
|
Where each credential lives and who reads it:
|
|
[`credentials.md`](credentials.md).
|
|
|
|
## Enabling
|
|
|
|
```nix
|
|
services.hyperhive.deploy.swarm-ui.enable = true; # defaults to deploy.swarm-controller.enable
|
|
```
|
|
|
|
Derived from the controller rather than from `allSwarmServices`:
|
|
the UI is a view onto the controller's state and reaches it over that
|
|
daemon's socket, so the host that runs the controller is the host that
|
|
can serve the UI. A hive that merely _uses_ a swarm has nothing to serve.
|
|
|
|
The UI answers on `services.hyperhive.swarm.domain` and nothing else. It
|
|
shares that name with the swarm-controller it fronts — one service to a
|
|
reader and to a certificate — so there is no separate option to pin.
|
|
|
|
> The apex must differ from `services.hyperhive.domain`. The gateway's
|
|
> default server already answers for the hive domain, and two vhosts
|
|
> claiming one `server_name` don't error — nginx picks one — so this is
|
|
> an assertion rather than a runtime surprise.
|
|
|
|
## 🔑 You must be in the `admins` group
|
|
|
|
**This is the step that separates "protected" from "locked out."** The
|
|
vhost's `auth_request` asks authelia _"is there a session"_; the rule
|
|
that makes it mean _"is this an operator"_ is an `access_control` entry
|
|
requiring `group:admins`. An account without that group authenticates
|
|
fine and still gets bounced.
|
|
|
|
```sh
|
|
swarmctl user add <you> --email <you>@example.com --group admins
|
|
swarmctl user update <you> --add-group admins # an account that already exists
|
|
```
|
|
|
|
`--email` isn't needed for the UI, but the forge won't create your account
|
|
without one → [setup.md § 2](../getting-started/setup.md#2--your-sso-account).
|
|
|
|
Why a group and not "any session": agents are authelia subjects too, so
|
|
_authenticated_ includes every agent in the swarm. The group is the only
|
|
thing standing between "an operator's page" and "anyone with a session."
|
|
|
|
## What it costs to be reachable
|
|
|
|
hyperhive publishes the apex to the hive's resolver like every other
|
|
swarm service, so agent containers can resolve it. That's deliberate and it's
|
|
not a hole: **reachability isn't the access control here.** An agent
|
|
that resolves the name and connects still has no operator session, and
|
|
the subrequest denies it.
|
|
|
|
## Quick links
|
|
|
|
The swarm UI's header carries a single 🔗 button, visible on every route,
|
|
opening a popover of links to other swarm-wide services. Backed by
|
|
`GET /api/links` (swarm-controller), which serves
|
|
`services.hyperhive.swarm.controller.links` (a `listOf { label, icon, url }`,
|
|
same shape as the per-agent `services.hyperhive.agent.dashboardLinks`).
|
|
|
|
Links to swarm services come from swarm-level options, so the list is the
|
|
same whichever host runs each service:
|
|
|
|
| link | URL built from |
|
|
| -------- | ------------------------------------------------- |
|
|
| Forge | `services.hyperhive.swarm.forge.domain` |
|
|
| Authelia | `services.hyperhive.swarm.authelia.domain` |
|
|
| Grafana | `services.hyperhive.swarm.grafana.domain` |
|
|
| Metrics | `services.hyperhive.swarm.victoriametrics.domain` |
|
|
| Logs | `services.hyperhive.swarm.victorialogs.domain` |
|
|
| Matrix | `services.hyperhive.swarm.matrix.gatewayHost` |
|
|
| Bao | `services.hyperhive.swarm.bao.ui.domain` |
|
|
|
|
Each entry is present whenever its option has a value — every option in
|
|
the table defaults to one, so the link appears even on a swarm that runs
|
|
no instance of that service.
|
|
|
|
`nix/host-modules/swarm-controller.nix` builds these entries. `swarm-ui.nix`
|
|
adds one more, for this UI's own API docs. An operator can add entries
|
|
directly. An empty list hides the button.
|
|
|
|
The **Matrix** entry opens the swarm's matrix web client (fluffychat,
|
|
`services.hyperhive.deploy.matrix.gui.package`) at the homeserver's
|
|
gateway host, `chat.<swarm domain>` by default. The homeserver's host
|
|
always serves the client at `/` on that vhost, so the entry is present
|
|
whenever the swarm names a gateway host.
|
|
|
|
The **Bao** entry opens the secret store's web UI, which admits members of
|
|
authelia's `admins` group only; see [`secrets.md`](secrets.md#the-browser-ui).
|
|
|
|
<details><summary>Adding a swarm service name: the two wiring sites</summary>
|
|
|
|
Adding a swarm service name means touching two things. Missing the
|
|
second ships as a different flavour of "works from the host, broken from
|
|
a container":
|
|
|
|
| site | file |
|
|
| ---------------------------- | ---------------------------------------------------------------------- |
|
|
| vhost + `gateway.localNames` | the service's own module (for example `nix/host-modules/swarm-ui.nix`) |
|
|
| **certificate name** | `nix/host-modules/swarm.nix` (`serviceDomains`) |
|
|
|
|
<!-- vale write-good.Passive = NO -->
|
|
|
|
The DNS record and the local-dev `/etc/hosts` entry need no separate
|
|
edit: both derive from `services.hyperhive.gateway.localNames`,
|
|
which a service's own module already has to push its domain into to be
|
|
resolvable — see `nix/host-modules/hive-gateway/dnsmasq.nix` and
|
|
`.../default.nix`'s `networking.hosts`. `vhosts.nix` itself is scoped to
|
|
the surface the hive's own domain serves (dashboard, per-agent routing,
|
|
matrix discovery); a swarm service declares its own vhost next to its
|
|
own options, the way `swarm-ui.nix` and `swarm-authelia.nix` do.
|
|
|
|
<!-- vale write-good.Passive = YES -->
|
|
|
|
⚠️ The certificate one is the hardest to predict and the most visible when
|
|
missed. `serviceDomains` is _both_ the `allowed_domains` the secret
|
|
store's `pki/roles/swarm-services` narrows to and the list each gateway's
|
|
leaf draws its SANs from (it carries the ones that host fronts),
|
|
and the apex is a **sibling** of `forge.<swarm>` / `chat.<swarm>` /
|
|
`auth.<swarm>`, not a parent — no CA in the hierarchy issues for it
|
|
implicitly. Left out, the vhost falls back to the hive leaf and the
|
|
swarm's front page opens with a name mismatch.
|
|
|
|
</details>
|
|
|
|
## Cross-references
|
|
|
|
- [`sso.md`](sso.md) — the authelia instance itself, and the user store.
|
|
- [`../networking/gateway.md`](../networking/gateway.md) — the full vhost map and TLS modes.
|