7364ccdc98
`GET /api/agents/{name}/inference` answered `"configured": {}` both for
an agent with no key in `agent-inference` and for a stored value with no
fields, so a reader could not tell "the swarm sets nothing" from "a
setting with every field unset". It now reads the key as
`Option<AgentInference>`: an absent or deleted key is `null`, a stored
`{}` stays `{}`. The shared `schedules::read` helper is unchanged.
`swarmctl agent inference get` prints "no swarm setting; the agent runs
on its own model and effort" for `null`; `set` merges into an empty
value in that case, as before.
The agent itself still treats no key and `{}` the same.
Refs #5034
115 lines
5.7 KiB
Markdown
115 lines
5.7 KiB
Markdown
# Per-agent inference settings
|
|
|
|
The swarm can set the model and reasoning effort each agent's turns run on,
|
|
plus the models it may start subagents on. A field you set wins over the
|
|
agent's own value; a field you leave unset keeps the agent on its own value,
|
|
so an agent you set nothing for runs as it does today.
|
|
|
|
## Setting them
|
|
|
|
```sh
|
|
swarmctl agent inference set iris --model opus --effort high
|
|
swarmctl agent inference get iris
|
|
swarmctl agent inference clear iris
|
|
```
|
|
|
|
- `set` changes only the flags you pass; the other fields keep what's
|
|
stored. A hive must declare the agent.
|
|
- `--effort` is one of `low`, `medium`, `high`, `xhigh`, `max`.
|
|
- `--subagent-models` takes a comma-separated list. It replaces the
|
|
agent's `availableModels` as the models its subagent `start` and
|
|
`continue` calls may name, read on each call
|
|
([`subagent.md`](../tools/subagent.md#which-models-a-subagent-may-run-on)).
|
|
The web UI's model picker doesn't use it.
|
|
- `get` shows each field, naming the unset ones (or says the swarm sets
|
|
nothing for the agent), and then the model and effort the agent reports
|
|
running on, each with where it comes from.
|
|
- `clear` removes every field; the agent goes back to its own values.
|
|
|
|
The full flag reference is [`swarmctl-cli.md`](../tools/swarmctl-cli.md).
|
|
|
|
## What an agent runs on
|
|
|
|
`get` shows what the agent runs on next to what the swarm sets:
|
|
|
|
```text
|
|
model: opus
|
|
effort: (not set; the agent's own)
|
|
subagent_models: (not set; the agent's own)
|
|
running model: opus (swarm)
|
|
running effort: medium (local_file)
|
|
```
|
|
|
|
Each running value names where it comes from:
|
|
|
|
| source | meaning |
|
|
| ------------ | -------------------------------------------------------------------------- |
|
|
| `swarm` | the value set here |
|
|
| `agent_nix` | `services.hyperhive.agent.model` or `effortLevel` in the agent's agent.nix |
|
|
| `local_file` | a `/model` or `/effort` pick in the agent's web UI |
|
|
| `default` | the harness's built-in default (`haiku`, `medium`) |
|
|
|
|
When the agent starts without a swarm value, its agent.nix model wins over
|
|
its `/model` pick, and its `/effort` pick wins over its agent.nix effort.
|
|
|
|
The agent reports these whenever they change, and again once a minute.
|
|
swarm-controller keeps only what it has received since it started, so for up
|
|
to a minute after a controller restart `get` shows `running: not reported…`.
|
|
|
|
## When an agent picks a change up
|
|
|
|
Each agent reads its settings every 10 seconds and switches on the next
|
|
read: its next turn runs on the new model or effort, and its web UI badge
|
|
shows it. A turn already running keeps what it started with.
|
|
|
|
- **Its own value is the one it runs without the swarm**, from a `/model`
|
|
or `/effort` pick in its web UI or from its agent.nix. Setting a swarm
|
|
value doesn't overwrite it.
|
|
- **Clearing a field** (or the whole key) puts the agent back on its own
|
|
value on the next read.
|
|
- **A `/model` or `/effort` pick while the swarm sets that field** changes
|
|
the agent's own value, but the swarm value comes back on the next read.
|
|
- **After a restart**, the agent runs its own values until its first read
|
|
of its settings succeeds.
|
|
|
|
<details><summary>Implementation</summary>
|
|
|
|
The settings live in the `agent-inference` KV bucket, one key per agent,
|
|
which swarm-controller alone writes:
|
|
|
|
```json
|
|
{ "model": "opus", "effort": "high", "subagent_models": ["haiku", "sonnet"] }
|
|
```
|
|
|
|
Every field is optional. The agent runs the same with `{}` as with no key;
|
|
`GET` answers `"configured": null` for no key (or a cleared one) and `{}` for
|
|
a stored value with no fields.
|
|
|
|
| method | path | body |
|
|
| -------- | ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `GET` | `/api/agents/<agent>/inference` | —; answers `{"configured": <the stored value, or null>, "effective": <what the agent reported, or null>}` |
|
|
| `PUT` | `/api/agents/<agent>/inference` | the whole value, replacing the stored one; the controller refuses unknown fields, an unknown effort, or a model name with whitespace or a leading `-` |
|
|
| `DELETE` | `/api/agents/<agent>/inference` | —; answers 204 |
|
|
|
|
An agent's own queue credential grants it
|
|
`$JS.API.DIRECT.GET.KV_agent-inference.$KV.agent-inference.<agent>`, the
|
|
direct read of its own key, and nothing else in this bucket — the same shape
|
|
as its schedule key ([`schedules.md`](schedules.md)).
|
|
|
|
The harness (`hive-agent`'s `swarm_inference`) applies a set field to the
|
|
in-memory model or effort without writing `hyperhive-model` or
|
|
`hyperhive-effort`, which keep the agent's own value.
|
|
|
|
The subagent daemon (`hive-subagent-mcp`'s `swarm_models`) reads
|
|
`subagent_models` with the same direct get, under the same credential, on
|
|
each `start` or `continue` that names a model.
|
|
|
|
`effective` comes from the agent's turn-state header on
|
|
`$SWARM.agent-state.<agent>` ([`README.md`](README.md)), which carries
|
|
`model`, `effort`, `model_source` and `effort_source`. swarm-controller
|
|
subscribes to every agent's header with the reader grant it already holds for
|
|
the state relay and keeps the latest per agent in memory. A header without
|
|
those fields, from an agent that doesn't report them, leaves `effective`
|
|
`null`.
|
|
|
|
</details>
|