Watch
0
0
Fork
You've already forked hyperhive
0

swarm: a soft context limit per agent, as a percent of the window, applied at turn end

This commit is contained in:
damocles 2026-11-15 15:13:50 +01:00 • committed by mara
commit 1e03979836
11 changed files with 214 additions and 35 deletions

View file

@ -83,7 +83,7 @@ Manage agents across the swarm
* `mint-forge-token` — Check one agent's forge token, and mint it if it's missing or stale
* `set-state` — Declare the state a hive converges an agent to: `up`, `offline`, `paused` or `destroyed`
* `watch` — Follow an agent's terminal rows live: the newest retained rows, then each new one, one line per row, until Ctrl-C or the stream ends
* `inference` — Show the model, effort, subagent models and subagent cap the swarm sets for an agent, or change the model, effort and subagent models
* `inference` — Show the model, effort, subagent models, subagent cap and soft context limit the swarm sets for an agent, or change the model, effort and subagent models
* `permissions` — Show or change the permissions the swarm grants an agent
* `agents-md` — Show an agent's AGENTS.md, or edit it with a diff
@ -192,9 +192,9 @@ The swarm-level counterpart of `hivectl agent watch`, read from the swarm-contro
## `swarmctl agent inference`
Show the model, effort, subagent models and subagent cap the swarm sets for an agent, or change the model, effort and subagent models.
Show the model, effort, subagent models, subagent cap and soft context limit the swarm sets for an agent, or change the model, effort and subagent models.
`get` prints the subagent cap (`max_subagents`); `set` has no flag for it and keeps a stored one. Only swarm-ui sets it.
`get` prints the subagent cap (`max_subagents`) and the soft context limit (`soft_context_percent`); `set` has no flag for them and keeps stored ones. Only swarm-ui sets them.
A field the swarm sets wins over the agent's own value (its agent.nix or a `hyperhive-model`/`hyperhive-effort` file) from its next turn on; the agent reads its settings every ten seconds. A field the swarm doesn't set, or clears, leaves the agent on its own value.

View file

@ -1,8 +1,8 @@
# Per-agent inference settings
The swarm can set the model and reasoning effort each agent's turns run on,
plus the models it may start subagents on and how many subagents it may run
at once. A field you set wins over the
plus the models it may start subagents on, how many subagents it may run
at once, and a soft limit on how full its context may get before it compacts. A field you set wins over the
agent's own value; a field you leave unset keeps the agent on its own value,
so an agent you set nothing for runs as it does today.
@ -25,6 +25,14 @@ swarmctl agent inference clear iris
dialog; `set` has no flag for it and keeps the stored one. It's a whole
number from 0 to 32, where 0 allows no subagents; unset means no cap
([`subagent.md`](../agent-tools/subagent.md#how-many-subagents-may-run-at-once)).
- You set the soft context limit, `soft_context_percent`, in the same
dialog; `set` has no flag for it either. It is the percent of the model's
context window at which the agent compacts at the end of a turn, in place of
the harness's own 75%: 20 on a 1M window compacts once the context passes
200k. A turn can still run past it, and it changes nothing else (the idle
auto-reset, the context shown in the UI). It's a whole number from 5 to 100;
unset keeps the harness's own watermark
([`claude-invocation.md`](../turn-loop/claude-invocation.md)).
- `get` shows each field, naming the unset ones (or says the swarm sets
nothing for the agent), and then the model and effort the agent reports
running on, each with where it comes from.
@ -45,6 +53,7 @@ model: opus
effort: (not set; the agent's own)
subagent_models: (not set; the agent's own)
max_subagents: (not set; no cap)
soft_context_percent: (not set; the harness's own watermark)
running model: opus (swarm)
running effort: medium (local_file)
```
@ -93,7 +102,8 @@ which swarm-controller alone writes:
"model": "opus",
"effort": "high",
"subagent_models": ["haiku", "sonnet"],
"max_subagents": 3
"max_subagents": 3,
"soft_context_percent": 20
}
```
@ -106,7 +116,7 @@ warning in its log, until the next `PUT` replaces it.
| method | path | body |
| -------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `GET` | `/api/agents/<agent>/inference` | —; answers `{"configured": <the stored value, or null>, "effective": <what the agent reported, or null>}` |
| `PUT` | `/api/agents/<agent>/inference` | the whole value, replacing the stored one; the controller refuses unknown fields, an unknown effort, or a model name with whitespace or a leading `-`, or a `max_subagents` above 32 |
| `PUT` | `/api/agents/<agent>/inference` | the whole value, replacing the stored one; the controller refuses unknown fields, an unknown effort, or a model name with whitespace or a leading `-`, a `max_subagents` above 32, or a `soft_context_percent` outside 5 to 100 |
| `DELETE` | `/api/agents/<agent>/inference` | —; answers 204 |
An agent's own queue credential grants it
@ -118,6 +128,10 @@ The harness (`hive-agent`'s `swarm_inference`) applies a set field to the
in-memory model or effort without writing `hyperhive-model` or
`hyperhive-effort`, which keep the agent's own value.
The same loop hands `soft_context_percent` to the Bus
(`set_soft_context_percent`), which the compaction policy reads when it decides
at turn end; a key with no limit, or no key, clears it on the next read.
The subagent daemon (`hive-subagent-mcp`'s `swarm_models`) reads
`subagent_models` and `max_subagents` with the same direct get, under the
same credential, on each `start` or `continue`.

View file

@ -164,7 +164,10 @@ The **when** for hyperhive's own policy is a `hive_claude::CompactionPolicy` inj
`turn::make_session` builds a `PercentPolicy` that compacts once the
model-reported context fill reaches `HIVE_COMPACT_WATERMARK_PERCENT`
(default **75%**), falling back to `harness_state::context_window_tokens(model)`
for the window on turns the model didn't report one. `0` disables proactive
for the window on turns the model didn't report one. The swarm can replace
that percent per agent with its `soft_context_percent`
([`inference.md`](../swarm/inference.md)); `turn::SoftLimitPolicy` reads it
for each decision. `0` disables proactive
compaction (the reactive path always applies). The proactive path is
best-effort — a failed checkpoint or `/compact` never fails the turn that
already succeeded. An ACP agent gets the same policy; how it compacts is in

View file

@ -367,6 +367,11 @@ pub struct Bus {
/// limit the model enforces, which may differ from the default
/// (e.g. 200 k active window on a 1 M cache-enabled model).
api_context_window: Arc<Mutex<Option<u64>>>,
/// The swarm's soft context limit, as a percent of the context window
/// (`soft_context_percent` of the agent's `agent-inference` key): the
/// compaction policy compacts at it instead of the harness's own
/// watermark. `None` until the swarm sets one.
soft_context_percent: Arc<Mutex<Option<u8>>>,
/// Resolved model id from the most recent turn's `assistant` events
/// (the API-echoed `message.model`, e.g. `claude-opus-4-8`), as
/// opposed to the requested `--model` name which may be a short
@ -419,6 +424,7 @@ impl Bus {
tool_calls: Arc::new(Mutex::new(std::collections::HashMap::new())),
last_turn_ended_unix: Arc::new(AtomicI64::new(0)),
api_context_window: Arc::new(Mutex::new(None)),
soft_context_percent: Arc::new(Mutex::new(None)),
last_resolved_model: Arc::new(Mutex::new(None)),
}
}
@ -700,6 +706,17 @@ impl Bus {
*self.api_context_window.lock().unwrap()
}
/// Set (or clear, with `None`) the swarm's soft context limit.
pub fn set_soft_context_percent(&self, percent: Option<u8>) {
*self.soft_context_percent.lock().unwrap() = percent;
}
/// The swarm's soft context limit, as a percent of the window.
#[must_use]
pub fn soft_context_percent(&self) -> Option<u8> {
*self.soft_context_percent.lock().unwrap()
}
/// The effective context window for `model`: the API-reported window if a
/// turn has completed (`api_context_window`), else the per-model default
/// ([`context_window_tokens`]). Single accessor so the state + dashboard

View file

@ -20,7 +20,8 @@
//! A failed read changes nothing. Until the first successful read after a
//! start, the agent runs on its local values.
//!
//! `subagent_models` and `max_subagents` are not read here.
//! `subagent_models` and `max_subagents` are not read here. `soft_context_percent`
//! is handed to the Bus, which the compaction policy reads at turn end.
use std::time::Duration;
@ -143,6 +144,7 @@ pub async fn run(bus: Bus) {
loop {
if let Some(read) = read(&client, &agent).await {
overridden.apply(&read, &target);
target.bus.set_soft_context_percent(read.soft_context_percent);
}
tokio::time::sleep(POLL_INTERVAL).await;
}
@ -229,6 +231,7 @@ mod tests {
effort,
subagent_models: None,
max_subagents: None,
soft_context_percent: None,
}
}

View file

@ -12,6 +12,7 @@ use hive_runtime::{
AcpRuntime, AgentRuntime, ClaudeRuntime, Config, PercentPolicy, PermissionAsk,
PermissionPolicy, Runtime, RuntimeSpec, Sink,
};
use hive_claude::{CompactionPolicy, Usage};
use serde_json::Value;
use crate::events::{Bus, LiveEvent};
@ -316,23 +317,58 @@ const ACP_SESSION_FILE: &str = "hyperhive-acp-session";
/// [`hive_claude::InfiniteSession`]. Built once by the serve loop (see
/// [`make_session`]) and threaded through the turns, rather than rebuilt each
/// time.
pub type AgentSession = AgentRuntime<PercentPolicy>;
pub type AgentSession = AgentRuntime<SoftLimitPolicy>;
/// [`PercentPolicy`] with the watermark the swarm sets, when it sets one.
///
/// The swarm's `soft_context_percent` replaces the harness's own percent
/// (`compact_percent`) for the compaction decision at turn end, as a share of
/// the window the model reports; a turn can still run past it. Read per
/// decision, because the swarm changes it while the session lives. A harness
/// percent of `0` (proactive compaction off) stays off.
#[derive(Clone)]
pub struct SoftLimitPolicy {
percent: PercentPolicy,
bus: Bus,
}
impl CompactionPolicy for SoftLimitPolicy {
fn should_compact(&self, usage: Usage) -> bool {
let percent = match self.bus.soft_context_percent() {
Some(soft) if self.percent.percent != 0 => soft,
_ => self.percent.percent,
};
PercentPolicy {
percent,
..self.percent.clone()
}
.should_compact(usage)
}
fn checkpoint_prompt(&self) -> Option<&str> {
self.percent.checkpoint_prompt()
}
}
/// Construct the agent's durable session, with a percent-of-window compaction
/// policy that checkpoints (`CHECKPOINT_PROMPT`) before compacting; on claude
/// also a constant title and the on-disk store. Called once at serve-loop start.
/// `percent` comes from a boot-time env var and `default_window` is only a
/// fallback for turns where the model didn't report a window, so a single
/// build at startup is fine.
/// build at startup is fine; the swarm's soft limit is read from the bus per
/// decision.
///
/// # Errors
///
/// Returns an error if the runtime selection in the environment is invalid.
pub fn make_session(bus: &Bus) -> Result<AgentSession> {
let policy = PercentPolicy {
percent: compact_percent(),
default_window: Some(effective_context_window(bus)),
checkpoint_prompt: Some(CHECKPOINT_PROMPT.to_string()),
let policy = SoftLimitPolicy {
percent: PercentPolicy {
percent: compact_percent(),
default_window: Some(effective_context_window(bus)),
checkpoint_prompt: Some(CHECKPOINT_PROMPT.to_string()),
},
bus: bus.clone(),
};
Ok(match RuntimeSpec::from_env()? {
RuntimeSpec::Claude => {
@ -860,9 +896,48 @@ mod tests {
error_to_turn, nonzero_or, parse_u64, report_compact, runtime_error_to_turn,
with_turn_end_compact,
};
use super::{PercentPolicy, SoftLimitPolicy};
use crate::events::{Bus, LiveEvent};
use hive_claude::{CompactionPolicy, Usage};
use hive_runtime::{AcpError, Error};
/// The swarm's soft limit replaces the harness's percent: 20% of a
/// reported 1M window is 200k, below the 75% default, and clearing it
/// restores the default. A harness percent of 0 stays off whatever the
/// swarm sets.
#[test]
fn the_swarm_soft_limit_replaces_the_compaction_percent() {
let bus = Bus::new();
let policy = |percent| SoftLimitPolicy {
percent: PercentPolicy {
percent,
default_window: None,
checkpoint_prompt: None,
},
bus: bus.clone(),
};
let usage = Usage {
context_tokens: 250_000,
context_window: Some(1_000_000),
};
assert!(!policy(75).should_compact(usage));
bus.set_soft_context_percent(Some(20));
assert!(policy(75).should_compact(usage));
assert!(!policy(0).should_compact(usage));
// A soft limit above the default is honoured too.
bus.set_soft_context_percent(Some(90));
let high = Usage {
context_tokens: 800_000,
..usage
};
assert!(!policy(75).should_compact(high));
bus.set_soft_context_percent(None);
assert!(policy(75).should_compact(high));
}
#[test]
fn a_turn_that_parsed_nothing_does_not_keep_the_previous_resolved_model() {
let resolved = crate::paths::with_harness_dir(|| {

View file

@ -8,6 +8,14 @@ pub use swarm_wire::agent_inference::{Effort, Source};
/// typo rather than a cap.
pub const MAX_SUBAGENTS: u32 = 32;
/// The lowest `soft_context_percent` a `PUT` may set. Below it the system
/// prompt and tools alone approach the mark on a large window, so every turn
/// would compact.
pub const MIN_SOFT_CONTEXT_PERCENT: u8 = 5;
/// The highest `soft_context_percent` a `PUT` may set: the whole window.
pub const MAX_SOFT_CONTEXT_PERCENT: u8 = 100;
/// One agent's inference settings. An absent field is not set at swarm level,
/// and the agent keeps its own value for it.
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
@ -28,6 +36,12 @@ pub struct InferenceSettings {
/// subagents) to [`MAX_SUBAGENTS`]. Absent is no cap.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub max_subagents: Option<u32>,
/// The percent of the model's context window at which the agent compacts
/// at the end of a turn, from [`MIN_SOFT_CONTEXT_PERCENT`] to
/// [`MAX_SOFT_CONTEXT_PERCENT`]. A soft limit: a turn can run past it.
/// Absent keeps the harness's own watermark.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub soft_context_percent: Option<u8>,
}
/// What `GET` answers for one agent.
@ -72,8 +86,9 @@ mod tests {
effort: Some(Effort::Xhigh),
subagent_models: Some(vec!["haiku".to_owned()]),
max_subagents: Some(3),
soft_context_percent: Some(20),
},
r#"{"model":"opus","effort":"xhigh","subagent_models":["haiku"],"max_subagents":3}"#,
r#"{"model":"opus","effort":"xhigh","subagent_models":["haiku"],"max_subagents":3,"soft_context_percent":20}"#,
),
] {
assert_eq!(serde_json::to_string(&value).unwrap(), wire);

View file

@ -1,5 +1,5 @@
//! `/api/agents/{name}/inference` — the model, effort, subagent models and
//! subagent cap the swarm sets for one agent, and what the agent reports
//! `/api/agents/{name}/inference` — the model, effort, subagent models,
//! subagent cap and soft context limit the swarm sets for one agent, and what the agent reports
//! running on.
//!
//! Each agent's settings are one value under its own key of `agent-inference`
@ -28,7 +28,8 @@ use axum::http::StatusCode;
use futures_util::StreamExt as _;
use problem_details::ProblemDetails;
use swarm_controller_api::inference::{
AgentInferenceView, EffectiveInference, InferenceSettings, MAX_SUBAGENTS,
AgentInferenceView, EffectiveInference, InferenceSettings, MAX_SOFT_CONTEXT_PERCENT,
MAX_SUBAGENTS, MIN_SOFT_CONTEXT_PERCENT,
};
use swarm_queue_client::agent_inference;
use swarm_wire::agent_inference::{AgentInference, Effective};
@ -47,6 +48,7 @@ fn settings_view(v: AgentInference) -> InferenceSettings {
effort: v.effort,
subagent_models: v.subagent_models,
max_subagents: v.max_subagents,
soft_context_percent: v.soft_context_percent,
}
}
@ -91,11 +93,19 @@ pub(crate) fn into_stored(settings: InferenceSettings) -> Result<AgentInference,
"max_subagents is {max}; it is from 0 (no subagents) to {MAX_SUBAGENTS}, or left out for no cap"
));
}
if let Some(percent) = settings.soft_context_percent
&& !(MIN_SOFT_CONTEXT_PERCENT..=MAX_SOFT_CONTEXT_PERCENT).contains(&percent)
{
return Err(format!(
"soft_context_percent is {percent}; it is from {MIN_SOFT_CONTEXT_PERCENT} to {MAX_SOFT_CONTEXT_PERCENT}, or left out for the harness's own watermark"
));
}
Ok(AgentInference {
model: settings.model,
effort: settings.effort,
subagent_models: settings.subagent_models,
max_subagents: settings.max_subagents,
soft_context_percent: settings.soft_context_percent,
})
}
@ -251,7 +261,8 @@ pub(crate) async fn get_agent_inference(
ones whole", body = InferenceSettings),
(status = 400, description = "not an identifier, not declared on any hive, or a \
model name is empty, has whitespace, starts with `-`, or is listed \
twice, or `max_subagents` is above 32 (problem+json)",
twice, or `max_subagents` is above 32, or `soft_context_percent` is outside \
its range (problem+json)",
body = String),
(status = 422, description = "an unknown field, or an effort that is not one of \
the five levels"),
@ -394,6 +405,7 @@ mod tests {
effort: Some(Effort::High),
subagent_models: Some(vec!["haiku".to_owned()]),
max_subagents: None,
soft_context_percent: None,
}
);
assert_eq!(
@ -555,6 +567,25 @@ mod tests {
assert!(settings(r#"{"max_subagents":-1}"#).is_err());
}
#[test]
fn soft_context_percent_outside_its_range_is_refused() {
for percent in [0, MIN_SOFT_CONTEXT_PERCENT - 1, MAX_SOFT_CONTEXT_PERCENT + 1] {
let json = format!(r#"{{"soft_context_percent":{percent}}}"#);
assert!(into_stored(settings(&json).unwrap()).is_err(), "{json}");
}
assert!(settings(r#"{"soft_context_percent":-1}"#).is_err());
assert!(settings(r#"{"soft_context_percent":256}"#).is_err());
}
#[test]
fn soft_context_percent_in_range_is_stored() {
for percent in [MIN_SOFT_CONTEXT_PERCENT, 20, MAX_SOFT_CONTEXT_PERCENT] {
let json = format!(r#"{{"soft_context_percent":{percent}}}"#);
let stored = into_stored(settings(&json).unwrap()).unwrap();
assert_eq!(stored.soft_context_percent, Some(percent));
}
}
#[test]
fn max_subagents_in_range_is_stored() {
for max in [0, 1, 3, MAX_SUBAGENTS] {

View file

@ -50,6 +50,10 @@ pub struct AgentInference {
/// cap.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub max_subagents: Option<u32>,
/// The percent of the model's context window at which the agent compacts,
/// in place of the harness's own watermark. `None` keeps the harness's.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub soft_context_percent: Option<u8>,
}
/// A claude `--effort` level, ascending.
@ -142,11 +146,12 @@ mod tests {
effort: Some(Effort::Xhigh),
subagent_models: Some(vec!["haiku".to_owned(), "sonnet".to_owned()]),
max_subagents: Some(3),
soft_context_percent: Some(20),
};
let encoded = serde_json::to_string(&value).unwrap();
assert_eq!(
encoded,
r#"{"model":"opus","effort":"xhigh","subagent_models":["haiku","sonnet"],"max_subagents":3}"#
r#"{"model":"opus","effort":"xhigh","subagent_models":["haiku","sonnet"],"max_subagents":3,"soft_context_percent":20}"#
);
assert_eq!(
serde_json::from_str::<AgentInference>(&encoded).unwrap(),
@ -171,13 +176,15 @@ mod tests {
);
}
/// A value written before `max_subagents` existed decodes with no cap.
/// A value written before `max_subagents` or `soft_context_percent`
/// existed decodes with neither set.
#[test]
fn a_value_without_max_subagents_has_no_cap() {
fn a_value_without_the_caps_has_neither_set() {
let decoded: AgentInference =
serde_json::from_str(r#"{"model":"opus","effort":"high","subagent_models":["haiku"]}"#)
.unwrap();
assert_eq!(decoded.max_subagents, None);
assert_eq!(decoded.soft_context_percent, None);
assert_eq!(decoded.model.as_deref(), Some("opus"));
}

View file

@ -36,6 +36,7 @@ impl SetFlags {
effort: effort.or(current.effort),
subagent_models: self.subagent_models.or(current.subagent_models),
max_subagents: current.max_subagents,
soft_context_percent: current.soft_context_percent,
})
}
}
@ -130,14 +131,18 @@ fn render_configured(s: Option<&InferenceSettings>) -> String {
fn render(s: &InferenceSettings) -> String {
const UNSET: &str = "(not set; the agent's own)";
format!(
"model: {}\neffort: {}\nsubagent_models: {}\nmax_subagents: {}\n",
"model: {}\neffort: {}\nsubagent_models: {}\nmax_subagents: {}\nsoft_context_percent: {}\n",
s.model.as_deref().unwrap_or(UNSET),
s.effort.map_or(UNSET, Effort::as_str),
s.subagent_models
.as_ref()
.map_or_else(|| UNSET.to_owned(), |m| m.join(", ")),
s.max_subagents
.map_or_else(|| "(not set; no cap)".to_owned(), |n| n.to_string())
.map_or_else(|| "(not set; no cap)".to_owned(), |n| n.to_string()),
s.soft_context_percent.map_or_else(
|| "(not set; the harness's own watermark)".to_owned(),
|n| format!("{n}%")
)
)
}
@ -198,10 +203,12 @@ mod tests {
assert_eq!(merged.effort, Some(Effort::High));
}
/// `set` has no flag for the cap, and a stored one survives it.
/// `set` has no flag for the cap or the soft limit, and stored ones
/// survive it.
#[test]
fn set_keeps_the_stored_max_subagents() {
let current: InferenceSettings = serde_json::from_str(r#"{"max_subagents":3}"#).unwrap();
fn set_keeps_the_stored_cap_and_soft_limit() {
let current: InferenceSettings =
serde_json::from_str(r#"{"max_subagents":3,"soft_context_percent":20}"#).unwrap();
let merged = SetFlags {
model: Some("opus".to_owned()),
..flags()
@ -209,6 +216,7 @@ mod tests {
.merge(current)
.unwrap();
assert_eq!(merged.max_subagents, Some(3));
assert_eq!(merged.soft_context_percent, Some(20));
}
/// An effort the controller would answer 422 for fails before the request.
@ -259,15 +267,19 @@ mod tests {
serde_json::from_str(r#"{"configured":{},"effective":null}"#).unwrap();
assert_eq!(
render_configured(empty.configured.as_ref()).lines().count(),
4
5
);
}
#[test]
fn an_empty_value_renders_every_field_as_unset() {
let row = render(&serde_json::from_str("{}").unwrap());
assert_eq!(row.lines().count(), 4);
assert!(row.ends_with("max_subagents: (not set; no cap)\n"), "{row}");
assert_eq!(row.lines().count(), 5);
assert!(
row.ends_with("soft_context_percent: (not set; the harness's own watermark)\n"),
"{row}"
);
assert!(row.contains("max_subagents: (not set; no cap)\n"), "{row}");
assert!(
row.lines()
.take(3)

View file

@ -386,11 +386,13 @@ enum AgentVerb {
/// show what it last did. Fails when the swarm has never heard from the
/// agent.
Watch(AgentWatchArgs),
/// Show the model, effort, subagent models and subagent cap the swarm sets
/// for an agent, or change the model, effort and subagent models.
/// Show the model, effort, subagent models, subagent cap and soft context
/// limit the swarm sets for an agent, or change the model, effort and
/// subagent models.
///
/// `get` prints the subagent cap (`max_subagents`); `set` has no flag for
/// it and keeps a stored one. Only swarm-ui sets it.
/// `get` prints the subagent cap (`max_subagents`) and the soft context
/// limit (`soft_context_percent`); `set` has no flag for them and keeps
/// stored ones. Only swarm-ui sets them.
///
/// A field the swarm sets wins over the agent's own value (its agent.nix
/// or a `hyperhive-model`/`hyperhive-effort` file) from its next turn on;