docs: sort the top-level directories into five sections by what the reader wants
Every docs directory moves under 1-getting-started, 2-handbook, 4-implementation or 5-project; the options reference (section 3) comes from the website and has no directory here. trust-boundary stays whole under the handbook for now. web-ui, swarm and tools split between handbook and implementation, and the command-line references move to 2-handbook/reference. Links, path references in code and nix, and the docs index follow the moves; no page content changes.
This commit is contained in:
iris
mara
parent
53ccbcb4b3
commit
6739bb903d
1 changed files with 736 additions and 701 deletions
133
docs/4-implementation/swarm/inference.md
Normal file
133
docs/4-implementation/swarm/inference.md
Normal file
|
|
@ -0,0 +1,133 @@
|
|||
# Per-agent inference settings
|
||||
|
||||
The swarm can set the model and reasoning effort each agent's turns run on,
|
||||
plus the models it may start subagents on and how many subagents it may run
|
||||
at once. A field you set wins over the
|
||||
agent's own value; a field you leave unset keeps the agent on its own value,
|
||||
so an agent you set nothing for runs as it does today.
|
||||
|
||||
## Setting them
|
||||
|
||||
```sh
|
||||
swarmctl agent inference set iris --model opus --effort high
|
||||
swarmctl agent inference get iris
|
||||
swarmctl agent inference clear iris
|
||||
```
|
||||
|
||||
- `set` changes only the flags you pass; the other fields keep what's
|
||||
stored. A hive must declare the agent.
|
||||
- `--effort` is one of `low`, `medium`, `high`, `xhigh`, `max`.
|
||||
- `--subagent-models` takes a comma-separated list. It replaces the
|
||||
agent's `availableModels` as the models its subagent `start` and
|
||||
`continue` calls may name, read on each call
|
||||
([`subagent.md`](../../2-handbook/tools/subagent.md#which-models-a-subagent-may-run-on)).
|
||||
- You set the subagent cap, `max_subagents`, in the swarm UI's inference
|
||||
dialog; `set` has no flag for it and keeps the stored one. It's a whole
|
||||
number from 0 to 32, where 0 allows no subagents; unset means no cap
|
||||
([`subagent.md`](../../2-handbook/tools/subagent.md#how-many-subagents-may-run-at-once)).
|
||||
- `get` shows each field, naming the unset ones (or says the swarm sets
|
||||
nothing for the agent), and then the model and effort the agent reports
|
||||
running on, each with where it comes from.
|
||||
- `clear` removes every field; the agent goes back to its own values.
|
||||
|
||||
The full flag reference is [`swarmctl-cli.md`](../../2-handbook/reference/swarmctl-cli.md). The
|
||||
swarm UI's agents page edits the same settings; see [UI](../../2-handbook/swarm/ui.md). The
|
||||
swarm is the only place to change an agent's model or effort at runtime:
|
||||
its web UI shows both but has no picker and no `/model` or `/effort`
|
||||
command.
|
||||
|
||||
## What an agent runs on
|
||||
|
||||
`get` shows what the agent runs on next to what the swarm sets:
|
||||
|
||||
```text
|
||||
model: opus
|
||||
effort: (not set; the agent's own)
|
||||
subagent_models: (not set; the agent's own)
|
||||
max_subagents: (not set; no cap)
|
||||
running model: opus (swarm)
|
||||
running effort: medium (local_file)
|
||||
```
|
||||
|
||||
Each running value names where it comes from:
|
||||
|
||||
| source | meaning |
|
||||
| ------------ | -------------------------------------------------------------------------- |
|
||||
| `swarm` | the value set here |
|
||||
| `agent_nix` | `services.hyperhive.agent.model` or `effortLevel` in the agent's agent.nix |
|
||||
| `local_file` | a `hyperhive-model` or `hyperhive-effort` file in the agent's harness dir |
|
||||
| `default` | the harness's built-in default (`haiku`, `medium`) |
|
||||
|
||||
When the agent starts without a swarm value, its agent.nix model wins over
|
||||
a `hyperhive-model` file, and a `hyperhive-effort` file wins over its
|
||||
agent.nix effort. These files come from runtime picks the agent's web UI no
|
||||
longer offers; the agent still reads them but never writes them, so a
|
||||
`local_file` value stays until the swarm sets that field or someone deletes
|
||||
the file.
|
||||
|
||||
The agent reports these whenever they change, and again once a minute.
|
||||
swarm-controller keeps only what it has received since it started, so for up
|
||||
to a minute after a controller restart `get` shows `running: not reported…`.
|
||||
|
||||
## When an agent picks a change up
|
||||
|
||||
Each agent reads its settings every 10 seconds and switches on the next
|
||||
read: its next turn runs on the new model or effort, and its web UI badge
|
||||
shows it. A turn already running keeps what it started with.
|
||||
|
||||
- **Its own value is the one it runs without the swarm**: its agent.nix or a
|
||||
`hyperhive-model` / `hyperhive-effort` file. Setting a swarm value doesn't
|
||||
overwrite it.
|
||||
- **Clearing a field** (or the whole key) puts the agent back on its own
|
||||
value on the next read.
|
||||
- **After a restart**, the agent runs its own values until its first read
|
||||
of its settings succeeds.
|
||||
|
||||
<details><summary>Implementation</summary>
|
||||
|
||||
The settings live in the `agent-inference` KV bucket, one key per agent,
|
||||
which swarm-controller alone writes:
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "opus",
|
||||
"effort": "high",
|
||||
"subagent_models": ["haiku", "sonnet"],
|
||||
"max_subagents": 3
|
||||
}
|
||||
```
|
||||
|
||||
Every field is optional. The agent runs the same with `{}` as with no key;
|
||||
`GET` answers `"configured": null` for no key (or a cleared one) and `{}` for
|
||||
a stored value with no fields. A stored value the controller can't decode,
|
||||
such as one a newer build wrote, also answers `"configured": null`, with a
|
||||
warning in its log, until the next `PUT` replaces it.
|
||||
|
||||
| method | path | body |
|
||||
| -------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `GET` | `/api/agents/<agent>/inference` | —; answers `{"configured": <the stored value, or null>, "effective": <what the agent reported, or null>}` |
|
||||
| `PUT` | `/api/agents/<agent>/inference` | the whole value, replacing the stored one; the controller refuses unknown fields, an unknown effort, or a model name with whitespace or a leading `-`, or a `max_subagents` above 32 |
|
||||
| `DELETE` | `/api/agents/<agent>/inference` | —; answers 204 |
|
||||
|
||||
An agent's own queue credential grants it
|
||||
`$JS.API.DIRECT.GET.KV_agent-inference.$KV.agent-inference.<agent>`, the
|
||||
direct read of its own key, and nothing else in this bucket — the same shape
|
||||
as its schedule key ([`schedules.md`](../../2-handbook/swarm/schedules.md)).
|
||||
|
||||
The harness (`hive-agent`'s `swarm_inference`) applies a set field to the
|
||||
in-memory model or effort without writing `hyperhive-model` or
|
||||
`hyperhive-effort`, which keep the agent's own value.
|
||||
|
||||
The subagent daemon (`hive-subagent-mcp`'s `swarm_models`) reads
|
||||
`subagent_models` and `max_subagents` with the same direct get, under the
|
||||
same credential, on each `start` or `continue`.
|
||||
|
||||
`effective` comes from the agent's turn-state header on
|
||||
`$SWARM.agent-state.<agent>` ([`README.md`](../../2-handbook/swarm/README.md)), which carries
|
||||
`model`, `effort`, `model_source` and `effort_source`. swarm-controller
|
||||
subscribes to every agent's header with the reader grant it already holds for
|
||||
the state relay and keeps the latest per agent in memory. A header without
|
||||
those fields, from an agent that doesn't report them, leaves `effective`
|
||||
`null`.
|
||||
|
||||
</details>
|
||||
Loading…
Reference in a new issue