vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity
hyperhive/docs/swarm/inference.md
atlas 29aa242201 controller stores: recover from a record they can't decode 
After a rollback, a record a newer build wrote (a new enum variant, a
changed required field) stopped decoding, and the strict
`schedules::read` turned that into an error on every read and write of
the key until someone deleted it by hand.

- agent_limits / inference (one document per agent): an undecodable
  document reads as unset, with a WARN naming the bucket, key and error.
  The PUT's "equal to stored, skip the write" shortcut no longer applies
  to an undecodable document, so a PUT of the defaults replaces it too.
- schedule definitions (one document, every schedule): salvaged per
  definition, as the review-classifier run list is. An undecodable
  definition is kept as its raw JSON, hidden from the list, edits and
  projections, written back by every commit after the readable ones, and
  its id is never minted again. A document with no list of schedules
  stays an error: reading it as empty would empty every agent's key.
- per-agent agent-schedules: an undecodable key reads as no schedules
  and is rewritten on the next projection, even when the agent has none.

`schedules::read` is replaced by `read_or_default`; `wanted` keeps its
own strict read.

Closes #5193
2026-10-13 08:12:50 +02:00

6.1 KiB
Raw Blame History

Per-agent inference settings

The swarm can set the model and reasoning effort each agent's turns run on, plus the models it may start subagents on. A field you set wins over the agent's own value; a field you leave unset keeps the agent on its own value, so an agent you set nothing for runs as it does today.

Setting them

swarmctl agent inference set iris --model opus --effort high
swarmctl agent inference get iris
swarmctl agent inference clear iris
  • set changes only the flags you pass; the other fields keep what's stored. A hive must declare the agent.
  • --effort is one of low, medium, high, xhigh, max.
  • --subagent-models takes a comma-separated list. It replaces the agent's availableModels as the models its subagent start and continue calls may name, read on each call (subagent.md).
  • get shows each field, naming the unset ones (or says the swarm sets nothing for the agent), and then the model and effort the agent reports running on, each with where it comes from.
  • clear removes every field; the agent goes back to its own values.

The full flag reference is swarmctl-cli.md. The swarm UI's agents page edits the same settings; see UI. The swarm is the only place to change an agent's model or effort at runtime: its web UI shows both but has no picker and no /model or /effort command.

What an agent runs on

get shows what the agent runs on next to what the swarm sets:

model: opus
effort: (not set; the agent's own)
subagent_models: (not set; the agent's own)
running model: opus (swarm)
running effort: medium (local_file)

Each running value names where it comes from:

source meaning
swarm the value set here
agent_nix services.hyperhive.agent.model or effortLevel in the agent's agent.nix
local_file a hyperhive-model or hyperhive-effort file in the agent's harness dir
default the harness's built-in default (haiku, medium)

When the agent starts without a swarm value, its agent.nix model wins over a hyperhive-model file, and a hyperhive-effort file wins over its agent.nix effort. These files come from runtime picks the agent's web UI no longer offers; the agent still reads them but never writes them, so a local_file value stays until the swarm sets that field or someone deletes the file.

The agent reports these whenever they change, and again once a minute. swarm-controller keeps only what it has received since it started, so for up to a minute after a controller restart get shows running: not reported….

When an agent picks a change up

Each agent reads its settings every 10 seconds and switches on the next read: its next turn runs on the new model or effort, and its web UI badge shows it. A turn already running keeps what it started with.

  • Its own value is the one it runs without the swarm: its agent.nix or a hyperhive-model / hyperhive-effort file. Setting a swarm value doesn't overwrite it.
  • Clearing a field (or the whole key) puts the agent back on its own value on the next read.
  • After a restart, the agent runs its own values until its first read of its settings succeeds.
Implementation

The settings live in the agent-inference KV bucket, one key per agent, which swarm-controller alone writes:

{ "model": "opus", "effort": "high", "subagent_models": ["haiku", "sonnet"] }

Every field is optional. The agent runs the same with {} as with no key; GET answers "configured": null for no key (or a cleared one) and {} for a stored value with no fields. A stored value the controller can't decode, such as one a newer build wrote, also answers "configured": null, with a warning in its log, until the next PUT replaces it.

method path body
GET /api/agents/<agent>/inference —; answers {"configured": <the stored value, or null>, "effective": <what the agent reported, or null>}
PUT /api/agents/<agent>/inference the whole value, replacing the stored one; the controller refuses unknown fields, an unknown effort, or a model name with whitespace or a leading -
DELETE /api/agents/<agent>/inference —; answers 204

An agent's own queue credential grants it $JS.API.DIRECT.GET.KV_agent-inference.$KV.agent-inference.<agent>, the direct read of its own key, and nothing else in this bucket — the same shape as its schedule key (schedules.md).

The harness (hive-agent's swarm_inference) applies a set field to the in-memory model or effort without writing hyperhive-model or hyperhive-effort, which keep the agent's own value.

The subagent daemon (hive-subagent-mcp's swarm_models) reads subagent_models with the same direct get, under the same credential, on each start or continue that names a model.

effective comes from the agent's turn-state header on $SWARM.agent-state.<agent> (README.md), which carries model, effort, model_source and effort_source. swarm-controller subscribes to every agent's header with the reader grant it already holds for the state relay and keeps the latest per agent in memory. A header without those fields, from an agent that doesn't report them, leaves effective null.