8.7 KiB
Per-agent inference settings
The swarm can set the model and reasoning effort each agent's turns run on, plus the models it may start subagents on, how many subagents it may run at once, and a soft limit on how full its context may get before it compacts. A field you set wins over the agent's own value; a field you leave unset keeps the agent on its own value, so an agent you set nothing for runs as it does today.
Setting them
swarmctl agent inference set iris --model opus --effort high
swarmctl agent inference get iris
swarmctl agent inference clear iris
setchanges only the flags you pass; the other fields keep what's stored. A hive must declare the agent.--effortis one oflow,medium,high,xhigh,max.--subagent-modelstakes a comma-separated list. It replaces the agent'savailableModelsas the models its subagentstartandcontinuecalls may name, read on each call (subagent.md).- You set the subagent cap,
max_subagents, in the swarm UI's inference dialog;sethas no flag for it and keeps the stored one. It's a whole number from 0 to 32, where 0 allows no subagents; unset means no cap (subagent.md). - You set the soft context limit,
soft_context_percent, in the same dialog;sethas no flag for it either. It's the percent of the model's context window at which the agent compacts at the end of a turn, in place of the harness's own 75%: 20 on a 1M window compacts once the context passes 200k. A turn can still run past it, and it changes nothing else (the context shown in the UI). It's a whole number from 5 to 100; unset keeps the harness's own watermark (claude-invocation.md). - You set the fresh-session threshold,
fresh_session_percent, with it: the percent of the context window above which a turn that finds the prompt cache cold starts a fresh session instead of resuming, in place of the harness's own 50%. It's a whole number from 5 to 100, and not above the soft limit when you set both, since the agent compacts at the soft limit and the context would never reach a higher threshold; the controller refuses a pair that crosses. With no soft limit set, the harness's own 75% applies, and the dialog keeps the threshold at or below it.HIVE_AUTO_RESET_WATERMARK_TOKENSin the agent's environment outranks the threshold.sethas no flag for it either, and the dialog edits both on one slider. getshows each field, naming the unset ones (or says the swarm sets nothing for the agent), and then the model and effort the agent reports running on, each with where it comes from.clearremoves every field; the agent goes back to its own values.
The full flag reference is swarmctl-cli.md. The
swarm UI's agents page edits the same settings; see UI. The
swarm is the only place to change an agent's model or effort at runtime:
its web UI shows both but has no picker and no /model or /effort
command.
What an agent runs on
get shows what the agent runs on next to what the swarm sets:
model: opus
effort: (not set; the agent's own)
subagent_models: (not set; the agent's own)
max_subagents: (not set; no cap)
soft_context_percent: (not set; the harness's own watermark)
fresh_session_percent: (not set; the harness's own half)
running model: opus (swarm)
running effort: medium (local_file)
Each running value names where it comes from:
| source | meaning |
|---|---|
swarm |
the value set here |
agent_nix |
services.hyperhive.agent.model or effortLevel in the agent's agent.nix |
local_file |
a hyperhive-model or hyperhive-effort file in the agent's harness dir |
default |
the harness's built-in default (haiku, medium) |
When the agent starts without a swarm value, its agent.nix model wins over
a hyperhive-model file, and a hyperhive-effort file wins over its
agent.nix effort. These files come from runtime picks the agent's web UI no
longer offers; the agent still reads them but never writes them, so a
local_file value stays until the swarm sets that field or someone deletes
the file.
The agent reports these whenever they change, and again once a minute.
swarm-controller keeps only what it has received since it started, so for up
to a minute after a controller restart get shows running: not reported….
When an agent picks a change up
Each agent reads its settings every 10 seconds and switches on the next read: its next turn runs on the new model or effort, and its web UI badge shows it. A turn already running keeps what it started with.
- Its own value is the one it runs without the swarm: its agent.nix or a
hyperhive-model/hyperhive-effortfile. Setting a swarm value doesn't overwrite it. - Clearing a field (or the whole key) puts the agent back on its own value on the next read.
- After a restart, the agent runs its own values until its first read of its settings succeeds.
Implementation
The settings live in the agent-inference KV bucket, one key per agent,
which swarm-controller alone writes:
{
"model": "opus",
"effort": "high",
"subagent_models": ["haiku", "sonnet"],
"max_subagents": 3,
"soft_context_percent": 20,
"fresh_session_percent": 10
}
Every field is optional. The agent runs the same with {} as with no key;
GET answers "configured": null for no key (or a cleared one) and {} for
a stored value with no fields. A stored value the controller can't decode,
such as one a newer build wrote, also answers "configured": null, with a
warning in its log, until the next PUT replaces it.
| method | path | body |
|---|---|---|
GET |
/api/agents/<agent>/inference |
—; answers {"configured": <the stored value, or null>, "effective": <what the agent reported, or null>} |
PUT |
/api/agents/<agent>/inference |
the whole value, replacing the stored one; the controller refuses unknown fields, an unknown effort, or a model name with whitespace or a leading -, a max_subagents above 32, or a soft_context_percent outside 5 to 100 |
DELETE |
/api/agents/<agent>/inference |
—; answers 204 |
An agent's own queue credential grants it
$JS.API.DIRECT.GET.KV_agent-inference.$KV.agent-inference.<agent>, the
direct read of its own key, and nothing else in this bucket — the same shape
as its schedule key (schedules.md).
The harness (hive-agent's swarm_inference) applies a set field to the
in-memory model or effort without writing hyperhive-model or
hyperhive-effort, which keep the agent's own value.
The same loop hands soft_context_percent to the Bus
(set_soft_context_percent), which the compaction policy reads when it decides
at turn end, and fresh_session_percent (set_fresh_session_percent), which
the pre-turn autoreset reads; a key with no limit, or no key, clears it on the
next read.
The subagent daemon (hive-subagent-mcp's swarm_models) reads
subagent_models and max_subagents with the same direct get, under the
same credential, on each start or continue.
effective comes from the agent's turn-state header on
$SWARM.agent-state.<agent> (README.md), which carries
model, effort, model_source and effort_source. swarm-controller
subscribes to every agent's header with the reader grant it already holds for
the state relay and keeps the latest per agent in memory. A header without
those fields, from an agent that doesn't report them, leaves effective
null.