90e6b984eb
A `start` or `continue` naming a `model` now reads this agent's key in the `agent-inference` bucket and, when `subagent_models` is set, checks the model against that list instead of HIVE_AVAILABLE_MODELS (agent.nix `availableModels`). An unset field, an absent key or an absent bucket keeps today's check. A read that fails or takes over 2 s falls back to HIVE_AVAILABLE_MODELS with a warning, so a subagent start never waits on the queue. A call without a `model` is not checked and does not read. The read uses the agent's own queue credential through swarm_term's connect, and the one direct-get subject the agent is granted in that bucket. The refusal names the list and that it came from the swarm. Refs #5034 (S5)
75 lines
3.7 KiB
Markdown
75 lines
3.7 KiB
Markdown
# Per-agent inference settings
|
|
|
|
The swarm can set the model and reasoning effort each agent's turns run on,
|
|
plus the models it may start subagents on. A field you set wins over the
|
|
agent's own value; a field you leave unset keeps the agent on its own value,
|
|
so an agent you set nothing for runs as it does today.
|
|
|
|
## Setting them
|
|
|
|
```sh
|
|
swarmctl agent inference set iris --model opus --effort high
|
|
swarmctl agent inference get iris
|
|
swarmctl agent inference clear iris
|
|
```
|
|
|
|
- `set` changes only the flags you pass; the other fields keep what's
|
|
stored. A hive must declare the agent.
|
|
- `--effort` is one of `low`, `medium`, `high`, `xhigh`, `max`.
|
|
- `--subagent-models` takes a comma-separated list. It replaces the
|
|
agent's `availableModels` as the models its subagent `start` and
|
|
`continue` calls may name, read on each call
|
|
([`subagent.md`](../tools/subagent.md#which-models-a-subagent-may-run-on)).
|
|
The web UI's model picker doesn't use it.
|
|
- `get` shows each field, naming the unset ones.
|
|
- `clear` removes every field; the agent goes back to its own values.
|
|
|
|
The full flag reference is [`swarmctl-cli.md`](../tools/swarmctl-cli.md).
|
|
|
|
## When an agent picks a change up
|
|
|
|
Each agent reads its settings every 10 seconds and switches on the next
|
|
read: its next turn runs on the new model or effort, and its web UI badge
|
|
shows it. A turn already running keeps what it started with.
|
|
|
|
- **Its own value is the one it runs without the swarm**, from a `/model`
|
|
or `/effort` pick in its web UI or from its agent.nix. Setting a swarm
|
|
value doesn't overwrite it.
|
|
- **Clearing a field** (or the whole key) puts the agent back on its own
|
|
value on the next read.
|
|
- **A `/model` or `/effort` pick while the swarm sets that field** changes
|
|
the agent's own value, but the swarm value comes back on the next read.
|
|
- **After a restart**, the agent runs its own values until its first read
|
|
of its settings succeeds.
|
|
|
|
<details><summary>Implementation</summary>
|
|
|
|
The settings live in the `agent-inference` KV bucket, one key per agent,
|
|
which swarm-controller alone writes:
|
|
|
|
```json
|
|
{ "model": "opus", "effort": "high", "subagent_models": ["haiku", "sonnet"] }
|
|
```
|
|
|
|
Every field is optional, and `{}` reads the same as no key.
|
|
|
|
| method | path | body |
|
|
| -------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
|
|
| `GET` | `/api/agents/<agent>/inference` | —; answers the stored value, `{}` when none |
|
|
| `PUT` | `/api/agents/<agent>/inference` | the whole value, replacing the stored one; the controller refuses unknown fields, an unknown effort, or a model name with whitespace |
|
|
| `DELETE` | `/api/agents/<agent>/inference` | —; answers 204 |
|
|
|
|
An agent's own queue credential grants it
|
|
`$JS.API.DIRECT.GET.KV_agent-inference.$KV.agent-inference.<agent>`, the
|
|
direct read of its own key, and nothing else in this bucket — the same shape
|
|
as its schedule key ([`schedules.md`](schedules.md)).
|
|
|
|
The harness (`hive-agent`'s `swarm_inference`) applies a set field to the
|
|
in-memory model or effort without writing `hyperhive-model` or
|
|
`hyperhive-effort`, which keep the agent's own value.
|
|
|
|
The subagent daemon (`hive-subagent-mcp`'s `swarm_models`) reads
|
|
`subagent_models` with the same direct get, under the same credential, on
|
|
each `start` or `continue` that names a model.
|
|
|
|
</details>
|