594693f4c7
The per-agent web UI no longer sets the model: the `/model` slash command, the model picker, `POST /api/model`, `Bus::set_model` and the `hyperhive-model` writer are gone. The model chip is display-only and shows where the model comes from (`model_source` in `/api/state`). The model is set on the swarm (swarm UI agents page, `swarmctl agent inference set`). The read side is unchanged: with no swarm value an agent runs its agent.nix model, else an existing `hyperhive-model` file, else haiku. Existing files are not touched. With the picker gone, its model list goes too: `available_models` in `/api/state`, the ACP model-list filter, and the harness's global `HIVE_AVAILABLE_MODELS`. The subagent daemon keeps its own `HIVE_AVAILABLE_MODELS` from mcp.nix, and the `availableModels` assertion stays. `/effort` (command, picker, route, file) is unchanged. Refs #5034
120 lines
6 KiB
Markdown
120 lines
6 KiB
Markdown
# Per-agent inference settings
|
|
|
|
The swarm can set the model and reasoning effort each agent's turns run on,
|
|
plus the models it may start subagents on. A field you set wins over the
|
|
agent's own value; a field you leave unset keeps the agent on its own value,
|
|
so an agent you set nothing for runs as it does today.
|
|
|
|
## Setting them
|
|
|
|
```sh
|
|
swarmctl agent inference set iris --model opus --effort high
|
|
swarmctl agent inference get iris
|
|
swarmctl agent inference clear iris
|
|
```
|
|
|
|
- `set` changes only the flags you pass; the other fields keep what's
|
|
stored. A hive must declare the agent.
|
|
- `--effort` is one of `low`, `medium`, `high`, `xhigh`, `max`.
|
|
- `--subagent-models` takes a comma-separated list. It replaces the
|
|
agent's `availableModels` as the models its subagent `start` and
|
|
`continue` calls may name, read on each call
|
|
([`subagent.md`](../tools/subagent.md#which-models-a-subagent-may-run-on)).
|
|
- `get` shows each field, naming the unset ones (or says the swarm sets
|
|
nothing for the agent), and then the model and effort the agent reports
|
|
running on, each with where it comes from.
|
|
- `clear` removes every field; the agent goes back to its own values.
|
|
|
|
The full flag reference is [`swarmctl-cli.md`](../tools/swarmctl-cli.md). The
|
|
swarm UI's agents page edits the same settings; see [UI](ui.md). The
|
|
swarm is the only place to change an agent's model at runtime: its web UI
|
|
shows the model but has no picker and no `/model` command.
|
|
|
|
## What an agent runs on
|
|
|
|
`get` shows what the agent runs on next to what the swarm sets:
|
|
|
|
```text
|
|
model: opus
|
|
effort: (not set; the agent's own)
|
|
subagent_models: (not set; the agent's own)
|
|
running model: opus (swarm)
|
|
running effort: medium (local_file)
|
|
```
|
|
|
|
Each running value names where it comes from:
|
|
|
|
| source | meaning |
|
|
| ------------ | -------------------------------------------------------------------------- |
|
|
| `swarm` | the value set here |
|
|
| `agent_nix` | `services.hyperhive.agent.model` or `effortLevel` in the agent's agent.nix |
|
|
| `local_file` | an `/effort` pick in the agent's web UI, or a `hyperhive-model` file |
|
|
| `default` | the harness's built-in default (`haiku`, `medium`) |
|
|
|
|
When the agent starts without a swarm value, its agent.nix model wins over
|
|
a `hyperhive-model` file, and its `/effort` pick wins over its agent.nix
|
|
effort. A `hyperhive-model` file comes from a runtime model pick the
|
|
agent's web UI no longer offers; the agent still reads it but never writes
|
|
it.
|
|
|
|
The agent reports these whenever they change, and again once a minute.
|
|
swarm-controller keeps only what it has received since it started, so for up
|
|
to a minute after a controller restart `get` shows `running: not reported…`.
|
|
|
|
## When an agent picks a change up
|
|
|
|
Each agent reads its settings every 10 seconds and switches on the next
|
|
read: its next turn runs on the new model or effort, and its web UI badge
|
|
shows it. A turn already running keeps what it started with.
|
|
|
|
- **Its own value is the one it runs without the swarm**: its agent.nix, an
|
|
`/effort` pick in its web UI, or a `hyperhive-model` file. Setting a swarm
|
|
value doesn't overwrite it.
|
|
- **Clearing a field** (or the whole key) puts the agent back on its own
|
|
value on the next read.
|
|
- **An `/effort` pick while the swarm sets effort** changes the agent's own
|
|
value, but the swarm value comes back on the next read.
|
|
- **After a restart**, the agent runs its own values until its first read
|
|
of its settings succeeds.
|
|
|
|
<details><summary>Implementation</summary>
|
|
|
|
The settings live in the `agent-inference` KV bucket, one key per agent,
|
|
which swarm-controller alone writes:
|
|
|
|
```json
|
|
{ "model": "opus", "effort": "high", "subagent_models": ["haiku", "sonnet"] }
|
|
```
|
|
|
|
Every field is optional. The agent runs the same with `{}` as with no key;
|
|
`GET` answers `"configured": null` for no key (or a cleared one) and `{}` for
|
|
a stored value with no fields.
|
|
|
|
| method | path | body |
|
|
| -------- | ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `GET` | `/api/agents/<agent>/inference` | —; answers `{"configured": <the stored value, or null>, "effective": <what the agent reported, or null>}` |
|
|
| `PUT` | `/api/agents/<agent>/inference` | the whole value, replacing the stored one; the controller refuses unknown fields, an unknown effort, or a model name with whitespace or a leading `-` |
|
|
| `DELETE` | `/api/agents/<agent>/inference` | —; answers 204 |
|
|
|
|
An agent's own queue credential grants it
|
|
`$JS.API.DIRECT.GET.KV_agent-inference.$KV.agent-inference.<agent>`, the
|
|
direct read of its own key, and nothing else in this bucket — the same shape
|
|
as its schedule key ([`schedules.md`](schedules.md)).
|
|
|
|
The harness (`hive-agent`'s `swarm_inference`) applies a set field to the
|
|
in-memory model or effort without writing `hyperhive-model` or
|
|
`hyperhive-effort`, which keep the agent's own value.
|
|
|
|
The subagent daemon (`hive-subagent-mcp`'s `swarm_models`) reads
|
|
`subagent_models` with the same direct get, under the same credential, on
|
|
each `start` or `continue` that names a model.
|
|
|
|
`effective` comes from the agent's turn-state header on
|
|
`$SWARM.agent-state.<agent>` ([`README.md`](README.md)), which carries
|
|
`model`, `effort`, `model_source` and `effort_source`. swarm-controller
|
|
subscribes to every agent's header with the reader grant it already holds for
|
|
the state relay and keeps the latest per agent in memory. A header without
|
|
those fields, from an agent that doesn't report them, leaves `effective`
|
|
`null`.
|
|
|
|
</details>
|