swarm: per-agent inference settings (model, effort, subagent_models) in KV
swarm-controller keeps each agent's inference settings in a new
`agent-inference` KV bucket, one key per agent, and is its only writer
(`GET`/`PUT`/`DELETE /api/agents/{name}/inference`; `swarmctl agent
inference get|set|clear`). Every field is optional. Each agent reads its
own key every 10 s through a direct-get grant on that key alone, and a
set `model` or `effort` replaces the Bus value, so the next turn runs on
it and the badge follows.
The swarm value is applied without persisting it: `Bus::set_model` and
`set_effort` are split into a persisting setter and a non-persisting
`apply_*`, and the poller uses the latter. Otherwise the swarm value
would overwrite `hyperhive-model`/`hyperhive-effort`, and clearing it, or
a restart, would no longer find the agent's own value. An unset field
changes nothing; a field cleared after being applied goes back to the
local value (`harness_state::local_model`/`local_effort`, the same
resolution `Bus::new` uses), read again at the clear.
`subagent_models` is validated, stored and shown by `get`; no agent
applies it yet.
Refs #5034
This commit is contained in:
parent
29d956140e
commit
916fdb35b8
24 changed files with 1432 additions and 34 deletions
68
docs/swarm/inference.md
Normal file
68
docs/swarm/inference.md
Normal file
|
|
@ -0,0 +1,68 @@
|
|||
# Per-agent inference settings
|
||||
|
||||
The swarm can set the model and reasoning effort each agent's turns run on,
|
||||
plus the models it may start subagents on. A field you set wins over the
|
||||
agent's own value; a field you leave unset keeps the agent on its own value,
|
||||
so an agent you set nothing for runs as it does today.
|
||||
|
||||
## Setting them
|
||||
|
||||
```sh
|
||||
swarmctl agent inference set iris --model opus --effort high
|
||||
swarmctl agent inference get iris
|
||||
swarmctl agent inference clear iris
|
||||
```
|
||||
|
||||
- `set` changes only the flags you pass; the other fields keep what's
|
||||
stored. A hive must declare the agent.
|
||||
- `--effort` is one of `low`, `medium`, `high`, `xhigh`, `max`.
|
||||
- `--subagent-models` takes a comma-separated list. It's stored and shown,
|
||||
but agents don't apply it yet.
|
||||
- `get` shows each field, naming the unset ones.
|
||||
- `clear` removes every field; the agent goes back to its own values.
|
||||
|
||||
The full flag reference is [`swarmctl-cli.md`](../tools/swarmctl-cli.md).
|
||||
|
||||
## When an agent picks a change up
|
||||
|
||||
Each agent reads its settings every 10 seconds and switches on the next
|
||||
read: its next turn runs on the new model or effort, and its web UI badge
|
||||
shows it. A turn already running keeps what it started with.
|
||||
|
||||
- **Its own value is the one it runs without the swarm**, from a `/model`
|
||||
or `/effort` pick in its web UI or from its agent.nix. Setting a swarm
|
||||
value doesn't overwrite it.
|
||||
- **Clearing a field** (or the whole key) puts the agent back on its own
|
||||
value on the next read.
|
||||
- **A `/model` or `/effort` pick while the swarm sets that field** changes
|
||||
the agent's own value, but the swarm value comes back on the next read.
|
||||
- **After a restart**, the agent runs its own values until its first read
|
||||
of its settings succeeds.
|
||||
|
||||
<details><summary>Implementation</summary>
|
||||
|
||||
The settings live in the `agent-inference` KV bucket, one key per agent,
|
||||
which swarm-controller alone writes:
|
||||
|
||||
```json
|
||||
{ "model": "opus", "effort": "high", "subagent_models": ["haiku", "sonnet"] }
|
||||
```
|
||||
|
||||
Every field is optional, and `{}` reads the same as no key.
|
||||
|
||||
| method | path | body |
|
||||
| -------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `GET` | `/api/agents/<agent>/inference` | —; answers the stored value, `{}` when none |
|
||||
| `PUT` | `/api/agents/<agent>/inference` | the whole value, replacing the stored one; the controller refuses unknown fields, an unknown effort, or a model name with whitespace |
|
||||
| `DELETE` | `/api/agents/<agent>/inference` | —; answers 204 |
|
||||
|
||||
An agent's own queue credential grants it
|
||||
`$JS.API.DIRECT.GET.KV_agent-inference.$KV.agent-inference.<agent>`, the
|
||||
direct read of its own key, and nothing else in this bucket — the same shape
|
||||
as its schedule key ([`schedules.md`](schedules.md)).
|
||||
|
||||
The harness (`hive-agent`'s `swarm_inference`) applies a set field to the
|
||||
in-memory model or effort without writing `hyperhive-model` or
|
||||
`hyperhive-effort`, which keep the agent's own value.
|
||||
|
||||
</details>
|
||||
Loading…
Reference in a new issue