vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity
hyperhive/docs/swarm/inference.md
atlas 916fdb35b8 swarm: per-agent inference settings (model, effort, subagent_models) in KV 
swarm-controller keeps each agent's inference settings in a new
`agent-inference` KV bucket, one key per agent, and is its only writer
(`GET`/`PUT`/`DELETE /api/agents/{name}/inference`; `swarmctl agent
inference get|set|clear`). Every field is optional. Each agent reads its
own key every 10 s through a direct-get grant on that key alone, and a
set `model` or `effort` replaces the Bus value, so the next turn runs on
it and the badge follows.

The swarm value is applied without persisting it: `Bus::set_model` and
`set_effort` are split into a persisting setter and a non-persisting
`apply_*`, and the poller uses the latter. Otherwise the swarm value
would overwrite `hyperhive-model`/`hyperhive-effort`, and clearing it, or
a restart, would no longer find the agent's own value. An unset field
changes nothing; a field cleared after being applied goes back to the
local value (`harness_state::local_model`/`local_effort`, the same
resolution `Bus::new` uses), read again at the clear.

`subagent_models` is validated, stored and shown by `get`; no agent
applies it yet.

Refs #5034
2026-10-09 23:13:13 +02:00

3.3 KiB
Raw Blame History

Per-agent inference settings

The swarm can set the model and reasoning effort each agent's turns run on, plus the models it may start subagents on. A field you set wins over the agent's own value; a field you leave unset keeps the agent on its own value, so an agent you set nothing for runs as it does today.

Setting them

swarmctl agent inference set iris --model opus --effort high
swarmctl agent inference get iris
swarmctl agent inference clear iris
  • set changes only the flags you pass; the other fields keep what's stored. A hive must declare the agent.
  • --effort is one of low, medium, high, xhigh, max.
  • --subagent-models takes a comma-separated list. It's stored and shown, but agents don't apply it yet.
  • get shows each field, naming the unset ones.
  • clear removes every field; the agent goes back to its own values.

The full flag reference is swarmctl-cli.md.

When an agent picks a change up

Each agent reads its settings every 10 seconds and switches on the next read: its next turn runs on the new model or effort, and its web UI badge shows it. A turn already running keeps what it started with.

  • Its own value is the one it runs without the swarm, from a /model or /effort pick in its web UI or from its agent.nix. Setting a swarm value doesn't overwrite it.
  • Clearing a field (or the whole key) puts the agent back on its own value on the next read.
  • A /model or /effort pick while the swarm sets that field changes the agent's own value, but the swarm value comes back on the next read.
  • After a restart, the agent runs its own values until its first read of its settings succeeds.
Implementation

The settings live in the agent-inference KV bucket, one key per agent, which swarm-controller alone writes:

{ "model": "opus", "effort": "high", "subagent_models": ["haiku", "sonnet"] }

Every field is optional, and {} reads the same as no key.

method path body
GET /api/agents/<agent>/inference —; answers the stored value, {} when none
PUT /api/agents/<agent>/inference the whole value, replacing the stored one; the controller refuses unknown fields, an unknown effort, or a model name with whitespace
DELETE /api/agents/<agent>/inference —; answers 204

An agent's own queue credential grants it $JS.API.DIRECT.GET.KV_agent-inference.$KV.agent-inference.<agent>, the direct read of its own key, and nothing else in this bucket — the same shape as its schedule key (schedules.md).

The harness (hive-agent's swarm_inference) applies a set field to the in-memory model or effort without writing hyperhive-model or hyperhive-effort, which keep the agent's own value.