Watch
0
0
Fork
You've already forked hyperhive
0

swarm: per-agent inference settings (model, effort, subagent_models) in KV

swarm-controller keeps each agent's inference settings in a new
`agent-inference` KV bucket, one key per agent, and is its only writer
(`GET`/`PUT`/`DELETE /api/agents/{name}/inference`; `swarmctl agent
inference get|set|clear`). Every field is optional. Each agent reads its
own key every 10 s through a direct-get grant on that key alone, and a
set `model` or `effort` replaces the Bus value, so the next turn runs on
it and the badge follows.

The swarm value is applied without persisting it: `Bus::set_model` and
`set_effort` are split into a persisting setter and a non-persisting
`apply_*`, and the poller uses the latter. Otherwise the swarm value
would overwrite `hyperhive-model`/`hyperhive-effort`, and clearing it, or
a restart, would no longer find the agent's own value. An unset field
changes nothing; a field cleared after being applied goes back to the
local value (`harness_state::local_model`/`local_effort`, the same
resolution `Bus::new` uses), read again at the clear.

`subagent_models` is validated, stored and shown by `get`; no agent
applies it yet.

Refs #5034
This commit is contained in:
atlas 2026-10-09 23:11:58 +02:00
commit 916fdb35b8
24 changed files with 1432 additions and 34 deletions

View file

@ -84,6 +84,8 @@ declarations.
sealed?** → [`swarm/bao.md`](swarm/bao.md).
- **How does an agent get a prompt at set times, and why doesn't it fire
twice after downtime?** → [`swarm/schedules.md`](swarm/schedules.md).
- **How do I set an agent's model or effort from the swarm, and what happens
to its own pick?** → [`swarm/inference.md`](swarm/inference.md).
## Scheduler, CI, observability

View file

@ -23,6 +23,8 @@ grafana, the controller, authelia, …) →
graph, agent creation. → [below](#swarm-controller)
- **scheduled prompts** — named prompts for one or more agents at set times,
kept by the swarm and fired in each agent's container. → [`schedules.md`](schedules.md)
- **inference settings** — the model and effort the swarm sets for an agent,
over its own. → [`inference.md`](inference.md)
- **swarm UI** — the operator's day-to-day surface, on the swarm apex,
`admins` only. → [`ui.md`](ui.md)
- **shared services** — one forge, homeserver, SSO, queue and metrics/logs

68
docs/swarm/inference.md Normal file
View file

@ -0,0 +1,68 @@
# Per-agent inference settings
The swarm can set the model and reasoning effort each agent's turns run on,
plus the models it may start subagents on. A field you set wins over the
agent's own value; a field you leave unset keeps the agent on its own value,
so an agent you set nothing for runs as it does today.
## Setting them
```sh
swarmctl agent inference set iris --model opus --effort high
swarmctl agent inference get iris
swarmctl agent inference clear iris
```
- `set` changes only the flags you pass; the other fields keep what's
stored. A hive must declare the agent.
- `--effort` is one of `low`, `medium`, `high`, `xhigh`, `max`.
- `--subagent-models` takes a comma-separated list. It's stored and shown,
but agents don't apply it yet.
- `get` shows each field, naming the unset ones.
- `clear` removes every field; the agent goes back to its own values.
The full flag reference is [`swarmctl-cli.md`](../tools/swarmctl-cli.md).
## When an agent picks a change up
Each agent reads its settings every 10 seconds and switches on the next
read: its next turn runs on the new model or effort, and its web UI badge
shows it. A turn already running keeps what it started with.
- **Its own value is the one it runs without the swarm**, from a `/model`
or `/effort` pick in its web UI or from its agent.nix. Setting a swarm
value doesn't overwrite it.
- **Clearing a field** (or the whole key) puts the agent back on its own
value on the next read.
- **A `/model` or `/effort` pick while the swarm sets that field** changes
the agent's own value, but the swarm value comes back on the next read.
- **After a restart**, the agent runs its own values until its first read
of its settings succeeds.
<details><summary>Implementation</summary>
The settings live in the `agent-inference` KV bucket, one key per agent,
which swarm-controller alone writes:
```json
{ "model": "opus", "effort": "high", "subagent_models": ["haiku", "sonnet"] }
```
Every field is optional, and `{}` reads the same as no key.
| method | path | body |
| -------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| `GET` | `/api/agents/<agent>/inference` | —; answers the stored value, `{}` when none |
| `PUT` | `/api/agents/<agent>/inference` | the whole value, replacing the stored one; the controller refuses unknown fields, an unknown effort, or a model name with whitespace |
| `DELETE` | `/api/agents/<agent>/inference` | —; answers 204 |
An agent's own queue credential grants it
`$JS.API.DIRECT.GET.KV_agent-inference.$KV.agent-inference.<agent>`, the
direct read of its own key, and nothing else in this bucket — the same shape
as its schedule key ([`schedules.md`](schedules.md)).
The harness (`hive-agent`'s `swarm_inference`) applies a set field to the
in-memory model or effort without writing `hyperhive-model` or
`hyperhive-effort`, which keep the agent's own value.
</details>

View file

@ -132,8 +132,8 @@ Each prompt they answer with has `id`, `name`, `agents` (sorted), `body`,
`next_fire_unix`: the next time on the prompt's grid, as of the moment the
controller answers, `null` when paused or a one-shot whose time has passed.
An agent's own queue credential grants exactly one subject in the swarm's
JetStream: `$JS.API.DIRECT.GET.KV_agent-schedules.$KV.agent-schedules.<agent>`,
An agent's own queue credential grants exactly one subject in this bucket:
`$JS.API.DIRECT.GET.KV_agent-schedules.$KV.agent-schedules.<agent>`,
the direct read of its own key. It can't read another agent's key, open or
watch the bucket, or write to it. A KV watch would need a consumer, and a
consumer create subject doesn't name a key, so granting one would let the

View file

@ -9,6 +9,10 @@ This document contains the help content for the `swarmctl` command-line program.
* [`swarmctl agent create`↴](#swarmctl-agent-create)
* [`swarmctl agent mint-identity`↴](#swarmctl-agent-mint-identity)
* [`swarmctl agent mint-forge-token`↴](#swarmctl-agent-mint-forge-token)
* [`swarmctl agent inference`↴](#swarmctl-agent-inference)
* [`swarmctl agent inference get`↴](#swarmctl-agent-inference-get)
* [`swarmctl agent inference set`↴](#swarmctl-agent-inference-set)
* [`swarmctl agent inference clear`↴](#swarmctl-agent-inference-clear)
* [`swarmctl schedule`↴](#swarmctl-schedule)
* [`swarmctl schedule add`↴](#swarmctl-schedule-add)
* [`swarmctl schedule list`↴](#swarmctl-schedule-list)
@ -59,6 +63,7 @@ Manage agents across the swarm
* `create` — Queue creation of a new agent on a hive in this swarm
* `mint-identity` — Queue a re-mint of an existing agent's identity at the swarm's secret store
* `mint-forge-token` — Check one agent's forge token, and mint it if it's missing or stale
* `inference` — Show or change the model, effort and subagent models the swarm sets for an agent
@ -137,6 +142,82 @@ Queues and returns, the same way `agent create` does — watch the swarm UI's jo
## `swarmctl agent inference`
Show or change the model, effort and subagent models the swarm sets for an agent.
A field the swarm sets wins over the agent's own value (from its `/model` or `/effort` pick or its agent.nix) from its next turn on; the agent reads its settings every ten seconds. A field the swarm doesn't set, or clears, leaves the agent on its own value.
**Usage:** `swarmctl agent inference <COMMAND>`
###### **Subcommands:**
* `get` — Show what the swarm sets for an agent, naming the fields it doesn't set
* `set` — Set fields for an agent. Only the flags given change; the others keep what's stored
* `clear` — Clear every field for an agent, so it goes back to its own values
## `swarmctl agent inference get`
Show what the swarm sets for an agent, naming the fields it doesn't set
**Usage:** `swarmctl agent inference get [OPTIONS] <AGENT>`
###### **Arguments:**
* `<AGENT>` — Agent name
###### **Options:**
* `--controller-socket <PATH>` — swarm-controller's unix socket.
Supplied by the nix module that installs this binary, from the same `socketPath` option the daemon binds; falls back to `SWARM_CONTROLLER_SOCKET`.
## `swarmctl agent inference set`
Set fields for an agent. Only the flags given change; the others keep what's stored
**Usage:** `swarmctl agent inference set [OPTIONS] <AGENT>`
###### **Arguments:**
* `<AGENT>` — Agent name. A hive must declare it
###### **Options:**
* `--model <MODEL>` — The model its turns run on, as claude's `--model` takes it
* `--effort <LEVEL>` — The reasoning effort its turns run at
Possible values: `low`, `medium`, `high`, `xhigh`, `max`
* `--subagent-models <MODELS>` — The models it may start subagents on, comma-separated. Stored and shown; the agent doesn't apply it yet
* `--controller-socket <PATH>` — swarm-controller's unix socket.
Supplied by the nix module that installs this binary, from the same `socketPath` option the daemon binds; falls back to `SWARM_CONTROLLER_SOCKET`.
## `swarmctl agent inference clear`
Clear every field for an agent, so it goes back to its own values
**Usage:** `swarmctl agent inference clear [OPTIONS] <AGENT>`
###### **Arguments:**
* `<AGENT>` — Agent name
###### **Options:**
* `--controller-socket <PATH>` — swarm-controller's unix socket.
Supplied by the nix module that installs this binary, from the same `socketPath` option the daemon binds; falls back to `SWARM_CONTROLLER_SOCKET`.
## `swarmctl schedule`
Manage scheduled prompts.

View file

@ -54,7 +54,9 @@ back to `"haiku"` if unset). The operator can flip it at runtime
with `/model <name>` in the web terminal — the next turn picks it
up. The harness persists the choice to `/harness/hyperhive-model` so it
survives restart; override path: `HYPERHIVE_MODEL_FILE` env var
for tests.
for tests. A model or effort the swarm sets for the agent replaces the
`Bus` value, and the harness doesn't persist it; clearing it goes back to
the persisted or nix value → [`swarm/inference.md`](../swarm/inference.md).
`harness_state::context_window_tokens(model)` looks up
context-window size per-model. Resolution order (first