461795b2cf
docs/swarm/inference.md had a table-row suffix pasted before the H1 by a bad substitution, and the PUT row had lost it; the text is back in the row. A failed read of the agent-inference key already applied no cap (an unreachable swarm must not block subagent work) and warned, but the warning did not say which agent. It now names the agent and the error, a test pins that, and subagent.md says a failed read starts anyway. The max-subagents field stays a text field with a digit pattern: the review-settings page's numeric field (concurrent reviews) is a TextField with pattern="[0-9]+" and validation, and no field in swarm-ui uses type="number". The new field gets the same pattern.
133 lines
6.6 KiB
Markdown
133 lines
6.6 KiB
Markdown
# Per-agent inference settings
|
|
|
|
The swarm can set the model and reasoning effort each agent's turns run on,
|
|
plus the models it may start subagents on and how many subagents it may run
|
|
at once. A field you set wins over the
|
|
agent's own value; a field you leave unset keeps the agent on its own value,
|
|
so an agent you set nothing for runs as it does today.
|
|
|
|
## Setting them
|
|
|
|
```sh
|
|
swarmctl agent inference set iris --model opus --effort high
|
|
swarmctl agent inference get iris
|
|
swarmctl agent inference clear iris
|
|
```
|
|
|
|
- `set` changes only the flags you pass; the other fields keep what's
|
|
stored. A hive must declare the agent.
|
|
- `--effort` is one of `low`, `medium`, `high`, `xhigh`, `max`.
|
|
- `--subagent-models` takes a comma-separated list. It replaces the
|
|
agent's `availableModels` as the models its subagent `start` and
|
|
`continue` calls may name, read on each call
|
|
([`subagent.md`](../tools/subagent.md#which-models-a-subagent-may-run-on)).
|
|
- You set the subagent cap, `max_subagents`, in the swarm UI's inference
|
|
dialog; `set` has no flag for it and keeps the stored one. It's a whole
|
|
number from 1 to 32; unset means no cap
|
|
([`subagent.md`](../tools/subagent.md#how-many-subagents-may-run-at-once)).
|
|
- `get` shows each field, naming the unset ones (or says the swarm sets
|
|
nothing for the agent), and then the model and effort the agent reports
|
|
running on, each with where it comes from.
|
|
- `clear` removes every field; the agent goes back to its own values.
|
|
|
|
The full flag reference is [`swarmctl-cli.md`](../tools/swarmctl-cli.md). The
|
|
swarm UI's agents page edits the same settings; see [UI](ui.md). The
|
|
swarm is the only place to change an agent's model or effort at runtime:
|
|
its web UI shows both but has no picker and no `/model` or `/effort`
|
|
command.
|
|
|
|
## What an agent runs on
|
|
|
|
`get` shows what the agent runs on next to what the swarm sets:
|
|
|
|
```text
|
|
model: opus
|
|
effort: (not set; the agent's own)
|
|
subagent_models: (not set; the agent's own)
|
|
max_subagents: (not set; no cap)
|
|
running model: opus (swarm)
|
|
running effort: medium (local_file)
|
|
```
|
|
|
|
Each running value names where it comes from:
|
|
|
|
| source | meaning |
|
|
| ------------ | -------------------------------------------------------------------------- |
|
|
| `swarm` | the value set here |
|
|
| `agent_nix` | `services.hyperhive.agent.model` or `effortLevel` in the agent's agent.nix |
|
|
| `local_file` | a `hyperhive-model` or `hyperhive-effort` file in the agent's harness dir |
|
|
| `default` | the harness's built-in default (`haiku`, `medium`) |
|
|
|
|
When the agent starts without a swarm value, its agent.nix model wins over
|
|
a `hyperhive-model` file, and a `hyperhive-effort` file wins over its
|
|
agent.nix effort. These files come from runtime picks the agent's web UI no
|
|
longer offers; the agent still reads them but never writes them, so a
|
|
`local_file` value stays until the swarm sets that field or someone deletes
|
|
the file.
|
|
|
|
The agent reports these whenever they change, and again once a minute.
|
|
swarm-controller keeps only what it has received since it started, so for up
|
|
to a minute after a controller restart `get` shows `running: not reported…`.
|
|
|
|
## When an agent picks a change up
|
|
|
|
Each agent reads its settings every 10 seconds and switches on the next
|
|
read: its next turn runs on the new model or effort, and its web UI badge
|
|
shows it. A turn already running keeps what it started with.
|
|
|
|
- **Its own value is the one it runs without the swarm**: its agent.nix or a
|
|
`hyperhive-model` / `hyperhive-effort` file. Setting a swarm value doesn't
|
|
overwrite it.
|
|
- **Clearing a field** (or the whole key) puts the agent back on its own
|
|
value on the next read.
|
|
- **After a restart**, the agent runs its own values until its first read
|
|
of its settings succeeds.
|
|
|
|
<details><summary>Implementation</summary>
|
|
|
|
The settings live in the `agent-inference` KV bucket, one key per agent,
|
|
which swarm-controller alone writes:
|
|
|
|
```json
|
|
{
|
|
"model": "opus",
|
|
"effort": "high",
|
|
"subagent_models": ["haiku", "sonnet"],
|
|
"max_subagents": 3
|
|
}
|
|
```
|
|
|
|
Every field is optional. The agent runs the same with `{}` as with no key;
|
|
`GET` answers `"configured": null` for no key (or a cleared one) and `{}` for
|
|
a stored value with no fields. A stored value the controller can't decode,
|
|
such as one a newer build wrote, also answers `"configured": null`, with a
|
|
warning in its log, until the next `PUT` replaces it.
|
|
|
|
| method | path | body |
|
|
| -------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `GET` | `/api/agents/<agent>/inference` | —; answers `{"configured": <the stored value, or null>, "effective": <what the agent reported, or null>}` |
|
|
| `PUT` | `/api/agents/<agent>/inference` | the whole value, replacing the stored one; the controller refuses unknown fields, an unknown effort, or a model name with whitespace or a leading `-`, or a `max_subagents` outside 1 to 32 |
|
|
| `DELETE` | `/api/agents/<agent>/inference` | —; answers 204 |
|
|
|
|
An agent's own queue credential grants it
|
|
`$JS.API.DIRECT.GET.KV_agent-inference.$KV.agent-inference.<agent>`, the
|
|
direct read of its own key, and nothing else in this bucket — the same shape
|
|
as its schedule key ([`schedules.md`](schedules.md)).
|
|
|
|
The harness (`hive-agent`'s `swarm_inference`) applies a set field to the
|
|
in-memory model or effort without writing `hyperhive-model` or
|
|
`hyperhive-effort`, which keep the agent's own value.
|
|
|
|
The subagent daemon (`hive-subagent-mcp`'s `swarm_models`) reads
|
|
`subagent_models` and `max_subagents` with the same direct get, under the
|
|
same credential, on each `start` or `continue`.
|
|
|
|
`effective` comes from the agent's turn-state header on
|
|
`$SWARM.agent-state.<agent>` ([`README.md`](README.md)), which carries
|
|
`model`, `effort`, `model_source` and `effort_source`. swarm-controller
|
|
subscribes to every agent's header with the reader grant it already holds for
|
|
the state relay and keeps the latest per agent in memory. A header without
|
|
those fields, from an agent that doesn't report them, leaves `effective`
|
|
`null`.
|
|
|
|
</details>
|