133 lines
6.6 KiB
Markdown
133 lines
6.6 KiB
Markdown
# Per-agent inference settings
|
|
|
|
The swarm can set the model and reasoning effort each agent's turns run on,
|
|
plus the models it may start subagents on and how many subagents it may run
|
|
at once. A field you set wins over the
|
|
agent's own value; a field you leave unset keeps the agent on its own value,
|
|
so an agent you set nothing for runs as it does today.
|
|
|
|
## Setting them
|
|
|
|
```sh
|
|
swarmctl agent inference set iris --model opus --effort high
|
|
swarmctl agent inference get iris
|
|
swarmctl agent inference clear iris
|
|
```
|
|
|
|
- `set` changes only the flags you pass; the other fields keep what's
|
|
stored. A hive must declare the agent.
|
|
- `--effort` is one of `low`, `medium`, `high`, `xhigh`, `max`.
|
|
- `--subagent-models` takes a comma-separated list. It replaces the
|
|
agent's `availableModels` as the models its subagent `start` and
|
|
`continue` calls may name, read on each call
|
|
([`subagent.md`](../tools/subagent.md#which-models-a-subagent-may-run-on)).
|
|
- You set the subagent cap, `max_subagents`, in the swarm UI's inference
|
|
dialog; `set` has no flag for it and keeps the stored one. It's a whole
|
|
number from 0 to 32, where 0 allows no subagents; unset means no cap
|
|
([`subagent.md`](../tools/subagent.md#how-many-subagents-may-run-at-once)).
|
|
- `get` shows each field, naming the unset ones (or says the swarm sets
|
|
nothing for the agent), and then the model and effort the agent reports
|
|
running on, each with where it comes from.
|
|
- `clear` removes every field; the agent goes back to its own values.
|
|
|
|
The full flag reference is [`swarmctl-cli.md`](../tools/swarmctl-cli.md). The
|
|
swarm UI's agents page edits the same settings; see [UI](ui.md). The
|
|
swarm is the only place to change an agent's model or effort at runtime:
|
|
its web UI shows both but has no picker and no `/model` or `/effort`
|
|
command.
|
|
|
|
## What an agent runs on
|
|
|
|
`get` shows what the agent runs on next to what the swarm sets:
|
|
|
|
```text
|
|
model: opus
|
|
effort: (not set; the agent's own)
|
|
subagent_models: (not set; the agent's own)
|
|
max_subagents: (not set; no cap)
|
|
running model: opus (swarm)
|
|
running effort: medium (local_file)
|
|
```
|
|
|
|
Each running value names where it comes from:
|
|
|
|
| source | meaning |
|
|
| ------------ | -------------------------------------------------------------------------- |
|
|
| `swarm` | the value set here |
|
|
| `agent_nix` | `services.hyperhive.agent.model` or `effortLevel` in the agent's agent.nix |
|
|
| `local_file` | a `hyperhive-model` or `hyperhive-effort` file in the agent's harness dir |
|
|
| `default` | the harness's built-in default (`haiku`, `medium`) |
|
|
|
|
When the agent starts without a swarm value, its agent.nix model wins over
|
|
a `hyperhive-model` file, and a `hyperhive-effort` file wins over its
|
|
agent.nix effort. These files come from runtime picks the agent's web UI no
|
|
longer offers; the agent still reads them but never writes them, so a
|
|
`local_file` value stays until the swarm sets that field or someone deletes
|
|
the file.
|
|
|
|
The agent reports these whenever they change, and again once a minute.
|
|
swarm-controller keeps only what it has received since it started, so for up
|
|
to a minute after a controller restart `get` shows `running: not reported…`.
|
|
|
|
## When an agent picks a change up
|
|
|
|
Each agent reads its settings every 10 seconds and switches on the next
|
|
read: its next turn runs on the new model or effort, and its web UI badge
|
|
shows it. A turn already running keeps what it started with.
|
|
|
|
- **Its own value is the one it runs without the swarm**: its agent.nix or a
|
|
`hyperhive-model` / `hyperhive-effort` file. Setting a swarm value doesn't
|
|
overwrite it.
|
|
- **Clearing a field** (or the whole key) puts the agent back on its own
|
|
value on the next read.
|
|
- **After a restart**, the agent runs its own values until its first read
|
|
of its settings succeeds.
|
|
|
|
<details><summary>Implementation</summary>
|
|
|
|
The settings live in the `agent-inference` KV bucket, one key per agent,
|
|
which swarm-controller alone writes:
|
|
|
|
```json
|
|
{
|
|
"model": "opus",
|
|
"effort": "high",
|
|
"subagent_models": ["haiku", "sonnet"],
|
|
"max_subagents": 3
|
|
}
|
|
```
|
|
|
|
Every field is optional. The agent runs the same with `{}` as with no key;
|
|
`GET` answers `"configured": null` for no key (or a cleared one) and `{}` for
|
|
a stored value with no fields. A stored value the controller can't decode,
|
|
such as one a newer build wrote, also answers `"configured": null`, with a
|
|
warning in its log, until the next `PUT` replaces it.
|
|
|
|
| method | path | body |
|
|
| -------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
| `GET` | `/api/agents/<agent>/inference` | —; answers `{"configured": <the stored value, or null>, "effective": <what the agent reported, or null>}` |
|
|
| `PUT` | `/api/agents/<agent>/inference` | the whole value, replacing the stored one; the controller refuses unknown fields, an unknown effort, or a model name with whitespace or a leading `-`, or a `max_subagents` above 32 |
|
|
| `DELETE` | `/api/agents/<agent>/inference` | —; answers 204 |
|
|
|
|
An agent's own queue credential grants it
|
|
`$JS.API.DIRECT.GET.KV_agent-inference.$KV.agent-inference.<agent>`, the
|
|
direct read of its own key, and nothing else in this bucket — the same shape
|
|
as its schedule key ([`schedules.md`](schedules.md)).
|
|
|
|
The harness (`hive-agent`'s `swarm_inference`) applies a set field to the
|
|
in-memory model or effort without writing `hyperhive-model` or
|
|
`hyperhive-effort`, which keep the agent's own value.
|
|
|
|
The subagent daemon (`hive-subagent-mcp`'s `swarm_models`) reads
|
|
`subagent_models` and `max_subagents` with the same direct get, under the
|
|
same credential, on each `start` or `continue`.
|
|
|
|
`effective` comes from the agent's turn-state header on
|
|
`$SWARM.agent-state.<agent>` ([`README.md`](README.md)), which carries
|
|
`model`, `effort`, `model_source` and `effort_source`. swarm-controller
|
|
subscribes to every agent's header with the reader grant it already holds for
|
|
the state relay and keeps the latest per agent in memory. A header without
|
|
those fields, from an agent that doesn't report them, leaves `effective`
|
|
`null`.
|
|
|
|
</details>
|