Watch
0
0
Fork
You've already forked hyperhive
0

swarm: per-agent inference settings (model, effort, subagent_models) in KV

swarm-controller keeps each agent's inference settings in a new
`agent-inference` KV bucket, one key per agent, and is its only writer
(`GET`/`PUT`/`DELETE /api/agents/{name}/inference`; `swarmctl agent
inference get|set|clear`). Every field is optional. Each agent reads its
own key every 10 s through a direct-get grant on that key alone, and a
set `model` or `effort` replaces the Bus value, so the next turn runs on
it and the badge follows.

The swarm value is applied without persisting it: `Bus::set_model` and
`set_effort` are split into a persisting setter and a non-persisting
`apply_*`, and the poller uses the latter. Otherwise the swarm value
would overwrite `hyperhive-model`/`hyperhive-effort`, and clearing it, or
a restart, would no longer find the agent's own value. An unset field
changes nothing; a field cleared after being applied goes back to the
local value (`harness_state::local_model`/`local_effort`, the same
resolution `Bus::new` uses), read again at the clear.

`subagent_models` is validated, stored and shown by `get`; no agent
applies it yet.

Refs #5034
This commit is contained in:
atlas 2026-10-09 23:11:58 +02:00
commit 916fdb35b8
24 changed files with 1432 additions and 34 deletions

View file

@ -84,6 +84,8 @@ declarations.
sealed?** → [`swarm/bao.md`](swarm/bao.md). sealed?** → [`swarm/bao.md`](swarm/bao.md).
- **How does an agent get a prompt at set times, and why doesn't it fire - **How does an agent get a prompt at set times, and why doesn't it fire
twice after downtime?** → [`swarm/schedules.md`](swarm/schedules.md). twice after downtime?** → [`swarm/schedules.md`](swarm/schedules.md).
- **How do I set an agent's model or effort from the swarm, and what happens
to its own pick?** → [`swarm/inference.md`](swarm/inference.md).
## Scheduler, CI, observability ## Scheduler, CI, observability

View file

@ -23,6 +23,8 @@ grafana, the controller, authelia, …) →
graph, agent creation. → [below](#swarm-controller) graph, agent creation. → [below](#swarm-controller)
- **scheduled prompts** — named prompts for one or more agents at set times, - **scheduled prompts** — named prompts for one or more agents at set times,
kept by the swarm and fired in each agent's container. → [`schedules.md`](schedules.md) kept by the swarm and fired in each agent's container. → [`schedules.md`](schedules.md)
- **inference settings** — the model and effort the swarm sets for an agent,
over its own. → [`inference.md`](inference.md)
- **swarm UI** — the operator's day-to-day surface, on the swarm apex, - **swarm UI** — the operator's day-to-day surface, on the swarm apex,
`admins` only. → [`ui.md`](ui.md) `admins` only. → [`ui.md`](ui.md)
- **shared services** — one forge, homeserver, SSO, queue and metrics/logs - **shared services** — one forge, homeserver, SSO, queue and metrics/logs

68
docs/swarm/inference.md Normal file
View file

@ -0,0 +1,68 @@
# Per-agent inference settings
The swarm can set the model and reasoning effort each agent's turns run on,
plus the models it may start subagents on. A field you set wins over the
agent's own value; a field you leave unset keeps the agent on its own value,
so an agent you set nothing for runs as it does today.
## Setting them
```sh
swarmctl agent inference set iris --model opus --effort high
swarmctl agent inference get iris
swarmctl agent inference clear iris
```
- `set` changes only the flags you pass; the other fields keep what's
stored. A hive must declare the agent.
- `--effort` is one of `low`, `medium`, `high`, `xhigh`, `max`.
- `--subagent-models` takes a comma-separated list. It's stored and shown,
but agents don't apply it yet.
- `get` shows each field, naming the unset ones.
- `clear` removes every field; the agent goes back to its own values.
The full flag reference is [`swarmctl-cli.md`](../tools/swarmctl-cli.md).
## When an agent picks a change up
Each agent reads its settings every 10 seconds and switches on the next
read: its next turn runs on the new model or effort, and its web UI badge
shows it. A turn already running keeps what it started with.
- **Its own value is the one it runs without the swarm**, from a `/model`
or `/effort` pick in its web UI or from its agent.nix. Setting a swarm
value doesn't overwrite it.
- **Clearing a field** (or the whole key) puts the agent back on its own
value on the next read.
- **A `/model` or `/effort` pick while the swarm sets that field** changes
the agent's own value, but the swarm value comes back on the next read.
- **After a restart**, the agent runs its own values until its first read
of its settings succeeds.
<details><summary>Implementation</summary>
The settings live in the `agent-inference` KV bucket, one key per agent,
which swarm-controller alone writes:
```json
{ "model": "opus", "effort": "high", "subagent_models": ["haiku", "sonnet"] }
```
Every field is optional, and `{}` reads the same as no key.
| method | path | body |
| -------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| `GET` | `/api/agents/<agent>/inference` | —; answers the stored value, `{}` when none |
| `PUT` | `/api/agents/<agent>/inference` | the whole value, replacing the stored one; the controller refuses unknown fields, an unknown effort, or a model name with whitespace |
| `DELETE` | `/api/agents/<agent>/inference` | —; answers 204 |
An agent's own queue credential grants it
`$JS.API.DIRECT.GET.KV_agent-inference.$KV.agent-inference.<agent>`, the
direct read of its own key, and nothing else in this bucket — the same shape
as its schedule key ([`schedules.md`](schedules.md)).
The harness (`hive-agent`'s `swarm_inference`) applies a set field to the
in-memory model or effort without writing `hyperhive-model` or
`hyperhive-effort`, which keep the agent's own value.
</details>

View file

@ -132,8 +132,8 @@ Each prompt they answer with has `id`, `name`, `agents` (sorted), `body`,
`next_fire_unix`: the next time on the prompt's grid, as of the moment the `next_fire_unix`: the next time on the prompt's grid, as of the moment the
controller answers, `null` when paused or a one-shot whose time has passed. controller answers, `null` when paused or a one-shot whose time has passed.
An agent's own queue credential grants exactly one subject in the swarm's An agent's own queue credential grants exactly one subject in this bucket:
JetStream: `$JS.API.DIRECT.GET.KV_agent-schedules.$KV.agent-schedules.<agent>`, `$JS.API.DIRECT.GET.KV_agent-schedules.$KV.agent-schedules.<agent>`,
the direct read of its own key. It can't read another agent's key, open or the direct read of its own key. It can't read another agent's key, open or
watch the bucket, or write to it. A KV watch would need a consumer, and a watch the bucket, or write to it. A KV watch would need a consumer, and a
consumer create subject doesn't name a key, so granting one would let the consumer create subject doesn't name a key, so granting one would let the

View file

@ -9,6 +9,10 @@ This document contains the help content for the `swarmctl` command-line program.
* [`swarmctl agent create`↴](#swarmctl-agent-create) * [`swarmctl agent create`↴](#swarmctl-agent-create)
* [`swarmctl agent mint-identity`↴](#swarmctl-agent-mint-identity) * [`swarmctl agent mint-identity`↴](#swarmctl-agent-mint-identity)
* [`swarmctl agent mint-forge-token`↴](#swarmctl-agent-mint-forge-token) * [`swarmctl agent mint-forge-token`↴](#swarmctl-agent-mint-forge-token)
* [`swarmctl agent inference`↴](#swarmctl-agent-inference)
* [`swarmctl agent inference get`↴](#swarmctl-agent-inference-get)
* [`swarmctl agent inference set`↴](#swarmctl-agent-inference-set)
* [`swarmctl agent inference clear`↴](#swarmctl-agent-inference-clear)
* [`swarmctl schedule`↴](#swarmctl-schedule) * [`swarmctl schedule`↴](#swarmctl-schedule)
* [`swarmctl schedule add`↴](#swarmctl-schedule-add) * [`swarmctl schedule add`↴](#swarmctl-schedule-add)
* [`swarmctl schedule list`↴](#swarmctl-schedule-list) * [`swarmctl schedule list`↴](#swarmctl-schedule-list)
@ -59,6 +63,7 @@ Manage agents across the swarm
* `create` — Queue creation of a new agent on a hive in this swarm * `create` — Queue creation of a new agent on a hive in this swarm
* `mint-identity` — Queue a re-mint of an existing agent's identity at the swarm's secret store * `mint-identity` — Queue a re-mint of an existing agent's identity at the swarm's secret store
* `mint-forge-token` — Check one agent's forge token, and mint it if it's missing or stale * `mint-forge-token` — Check one agent's forge token, and mint it if it's missing or stale
* `inference` — Show or change the model, effort and subagent models the swarm sets for an agent
@ -137,6 +142,82 @@ Queues and returns, the same way `agent create` does — watch the swarm UI's jo
## `swarmctl agent inference`
Show or change the model, effort and subagent models the swarm sets for an agent.
A field the swarm sets wins over the agent's own value (from its `/model` or `/effort` pick or its agent.nix) from its next turn on; the agent reads its settings every ten seconds. A field the swarm doesn't set, or clears, leaves the agent on its own value.
**Usage:** `swarmctl agent inference <COMMAND>`
###### **Subcommands:**
* `get` — Show what the swarm sets for an agent, naming the fields it doesn't set
* `set` — Set fields for an agent. Only the flags given change; the others keep what's stored
* `clear` — Clear every field for an agent, so it goes back to its own values
## `swarmctl agent inference get`
Show what the swarm sets for an agent, naming the fields it doesn't set
**Usage:** `swarmctl agent inference get [OPTIONS] <AGENT>`
###### **Arguments:**
* `<AGENT>` — Agent name
###### **Options:**
* `--controller-socket <PATH>` — swarm-controller's unix socket.
Supplied by the nix module that installs this binary, from the same `socketPath` option the daemon binds; falls back to `SWARM_CONTROLLER_SOCKET`.
## `swarmctl agent inference set`
Set fields for an agent. Only the flags given change; the others keep what's stored
**Usage:** `swarmctl agent inference set [OPTIONS] <AGENT>`
###### **Arguments:**
* `<AGENT>` — Agent name. A hive must declare it
###### **Options:**
* `--model <MODEL>` — The model its turns run on, as claude's `--model` takes it
* `--effort <LEVEL>` — The reasoning effort its turns run at
Possible values: `low`, `medium`, `high`, `xhigh`, `max`
* `--subagent-models <MODELS>` — The models it may start subagents on, comma-separated. Stored and shown; the agent doesn't apply it yet
* `--controller-socket <PATH>` — swarm-controller's unix socket.
Supplied by the nix module that installs this binary, from the same `socketPath` option the daemon binds; falls back to `SWARM_CONTROLLER_SOCKET`.
## `swarmctl agent inference clear`
Clear every field for an agent, so it goes back to its own values
**Usage:** `swarmctl agent inference clear [OPTIONS] <AGENT>`
###### **Arguments:**
* `<AGENT>` — Agent name
###### **Options:**
* `--controller-socket <PATH>` — swarm-controller's unix socket.
Supplied by the nix module that installs this binary, from the same `socketPath` option the daemon binds; falls back to `SWARM_CONTROLLER_SOCKET`.
## `swarmctl schedule` ## `swarmctl schedule`
Manage scheduled prompts. Manage scheduled prompts.

View file

@ -54,7 +54,9 @@ back to `"haiku"` if unset). The operator can flip it at runtime
with `/model <name>` in the web terminal — the next turn picks it with `/model <name>` in the web terminal — the next turn picks it
up. The harness persists the choice to `/harness/hyperhive-model` so it up. The harness persists the choice to `/harness/hyperhive-model` so it
survives restart; override path: `HYPERHIVE_MODEL_FILE` env var survives restart; override path: `HYPERHIVE_MODEL_FILE` env var
for tests. for tests. A model or effort the swarm sets for the agent replaces the
`Bus` value, and the harness doesn't persist it; clearing it goes back to
the persisted or nix value → [`swarm/inference.md`](../swarm/inference.md).
`harness_state::context_window_tokens(model)` looks up `harness_state::context_window_tokens(model)` looks up
context-window size per-model. Resolution order (first context-window size per-model. Resolution order (first

View file

@ -37,10 +37,14 @@ rusqlite.workspace = true
schemars.workspace = true schemars.workspace = true
serde.workspace = true serde.workspace = true
serde_json.workspace = true serde_json.workspace = true
# `agent-schedules` alone: the schedule ticker reads its own key with one direct # `agent-schedules` and `agent-inference`: the schedule ticker and the
# get. `kv`/`notices` name buckets and streams this harness opens neither end # inference poller each read their own key with one direct get. `kv`/`notices`
# of, and the terminal publisher is a plain core-subject publish. # name buckets and streams this harness opens neither end of, and the terminal
swarm-queue-client = { workspace = true, features = ["agent-schedules"] } # publisher is a plain core-subject publish.
swarm-queue-client = { workspace = true, features = [
"agent-schedules",
"agent-inference",
] }
# This agent's own queue secret, read from the store under its own certificate. # This agent's own queue secret, read from the store under its own certificate.
swarm-secret-client.workspace = true swarm-secret-client.workspace = true
tokio.workspace = true tokio.workspace = true

View file

@ -19,9 +19,8 @@ use serde::{Deserialize, Serialize};
use tokio::sync::broadcast; use tokio::sync::broadcast;
use crate::harness_state::{ use crate::harness_state::{
DEFAULT_EFFORT, DEFAULT_MODEL, configured_effort, configured_model, context_window_tokens, configured_effort, configured_model, context_window_tokens, local_effort, local_model,
load_effort, load_model, persist_effort, persist_model, read_harness_state, persist_effort, persist_model, read_harness_state, write_harness_state,
write_harness_state,
}; };
const CHANNEL_CAPACITY: usize = 256; const CHANNEL_CAPACITY: usize = 256;
@ -394,21 +393,8 @@ impl Bus {
} }
}; };
let (tx, _) = broadcast::channel(CHANNEL_CAPACITY); let (tx, _) = broadcast::channel(CHANNEL_CAPACITY);
// Priority: HIVE_DEFAULT_MODEL (from services.hyperhive.agent.model in agent.nix) > let initial_model = local_model(configured_model());
// persisted runtime override > compiled-in DEFAULT_MODEL. let initial_effort = local_effort(configured_effort());
// The nix config always wins on rebuild; the persisted file is kept
// for within-session tracking only (see persist_model / set_model).
let initial_model = configured_model().map_or_else(
|| load_model().unwrap_or_else(|| DEFAULT_MODEL.to_owned()),
str::to_owned,
);
// Effort precedence is the inverse of model: a persisted operator
// pick wins over the nix `HIVE_DEFAULT_EFFORT` baseline so the
// runtime choice survives harness restart (it only resets on
// `--purge`). nix is the default for a never-touched agent.
let initial_effort = load_effort()
.or_else(|| configured_effort().map(str::to_owned))
.unwrap_or_else(|| DEFAULT_EFFORT.to_owned());
// Restore rate_limited (and needs_login) from the consolidated // Restore rate_limited (and needs_login) from the consolidated
// harness state file so the dashboard shows the correct status // harness state file so the dashboard shows the correct status
// on cold load if the harness crashed while parked. // on cold load if the harness crashed while parked.
@ -590,10 +576,21 @@ impl Bus {
/// Panics if the internal lock is poisoned. /// Panics if the internal lock is poisoned.
pub fn set_model(&self, name: impl Into<String>) { pub fn set_model(&self, name: impl Into<String>) {
let value: String = name.into(); let value: String = name.into();
self.model.lock().unwrap().clone_from(&value);
if let Err(e) = persist_model(&value) { if let Err(e) = persist_model(&value) {
tracing::warn!(error = ?e, "model: persist failed"); tracing::warn!(error = ?e, "model: persist failed");
} }
self.apply_model(value);
}
/// Switch the model for future turns without persisting it, so
/// `hyperhive-model` keeps the agent's local value. For a value the
/// swarm sets (`crate::swarm_inference`).
///
/// # Panics
///
/// Panics if the internal lock is poisoned.
pub fn apply_model(&self, value: String) {
self.model.lock().unwrap().clone_from(&value);
// Mirror the resolved model into hyperhive-harness.json so // Mirror the resolved model into hyperhive-harness.json so
// hive-c0re can surface it on the dashboard badge without reading // hive-c0re can surface it on the dashboard badge without reading
// the hyperhive-model override file directly. // the hyperhive-model override file directly.
@ -624,10 +621,21 @@ impl Bus {
/// Panics if the internal lock is poisoned. /// Panics if the internal lock is poisoned.
pub fn set_effort(&self, level: impl Into<String>) { pub fn set_effort(&self, level: impl Into<String>) {
let value: String = level.into(); let value: String = level.into();
self.effort.lock().unwrap().clone_from(&value);
if let Err(e) = persist_effort(&value) { if let Err(e) = persist_effort(&value) {
tracing::warn!(error = ?e, "effort: persist failed"); tracing::warn!(error = ?e, "effort: persist failed");
} }
self.apply_effort(value);
}
/// Switch the effort level without persisting it, so `hyperhive-effort`
/// keeps the agent's local value. For a value the swarm sets
/// (`crate::swarm_inference`).
///
/// # Panics
///
/// Panics if the internal lock is poisoned.
pub fn apply_effort(&self, value: String) {
self.effort.lock().unwrap().clone_from(&value);
self.emit(LiveEvent::EffortChanged { effort: value }); self.emit(LiveEvent::EffortChanged { effort: value });
} }

View file

@ -53,6 +53,27 @@ pub(crate) fn load_effort() -> Option<String> {
} }
} }
/// The model this agent runs when the swarm sets none. Priority:
/// `configured` (`HIVE_DEFAULT_MODEL`, from `services.hyperhive.agent.model`
/// in agent.nix) > the persisted runtime override > [`DEFAULT_MODEL`].
pub(crate) fn local_model(configured: Option<&str>) -> String {
configured.map_or_else(
|| load_model().unwrap_or_else(|| DEFAULT_MODEL.to_owned()),
str::to_owned,
)
}
/// The effort this agent runs at when the swarm sets none. The inverse of
/// [`local_model`]: a persisted operator pick wins over `configured`
/// (`HIVE_DEFAULT_EFFORT`), so the runtime choice survives a harness restart
/// (it only resets on `--purge`); nix is the default for a never-touched
/// agent.
pub(crate) fn local_effort(configured: Option<&str>) -> String {
load_effort()
.or_else(|| configured.map(str::to_owned))
.unwrap_or_else(|| DEFAULT_EFFORT.to_owned())
}
pub(crate) fn persist_effort(level: &str) -> std::io::Result<()> { pub(crate) fn persist_effort(level: &str) -> std::io::Result<()> {
let path = effort_file_path(); let path = effort_file_path();
if let Some(parent) = path.parent() { if let Some(parent) = path.parent() {

View file

@ -32,6 +32,7 @@ mod state_entry_watch;
mod stats; mod stats;
mod swarm_agent_icon; mod swarm_agent_icon;
mod swarm_agent_state; mod swarm_agent_state;
mod swarm_inference;
mod swarm_input; mod swarm_input;
mod swarm_queue; mod swarm_queue;
mod swarm_term; mod swarm_term;
@ -532,6 +533,8 @@ async fn serve_main<S: Surface>(socket: &Path, poll_ms: u64) -> Result<()> {
swarm_agent_state::spawn(&bus); swarm_agent_state::spawn(&bus);
// Once per start: a new icon only arrives with a new harness process. // Once per start: a new icon only arrives with a new harness process.
swarm_agent_icon::spawn(); swarm_agent_icon::spawn();
// The model and effort the swarm sets for this agent, over its local ones.
tokio::spawn(swarm_inference::run(bus.clone()));
// Set by the web UI's `/api/cancel` on a successful SIGINT, read-and- // Set by the web UI's `/api/cancel` on a successful SIGINT, read-and-
// cleared by `handle_turn` before building the next wake prompt — see // cleared by `handle_turn` before building the next wake prompt — see
// `hive_sh4re::inbox::INTERRUPTED_HINT`. Shared between the web server task and // `hive_sh4re::inbox::INTERRUPTED_HINT`. Shared between the web server task and

View file

@ -0,0 +1,316 @@
//! Applies the model and effort the swarm sets for this agent.
//!
//! swarm-controller keeps the agent's settings under its key in the
//! `agent-inference` KV bucket. This task reads that key every
//! [`POLL_INTERVAL`] with the one subject the agent's own queue credential is
//! granted there (`swarm_queue_client::agent_inference::read_own`).
//!
//! **A set field wins**: it replaces the Bus value, so the next turn runs on
//! it and the web UI's badge follows. It is not persisted: `hyperhive-model`
//! and `hyperhive-effort` keep the agent's local value.
//!
//! **An unset field changes nothing**, so an agent the swarm sets nothing for
//! runs on its local value ([`crate::harness_state::local_model`] and
//! [`crate::harness_state::local_effort`]). A field that was set and then
//! cleared goes back to that local value, read again at that moment.
//!
//! A failed read changes nothing. Until the first successful read after a
//! start, the agent runs on its local values.
//!
//! `subagent_models` is not read here.
use std::time::Duration;
use swarm_queue_client::agent_inference::{AgentInference, Effort, Read, read_own};
use crate::events::Bus;
use crate::harness_state::{configured_effort, configured_model, local_effort, local_model};
/// How often the key is read. Also the longest a change can take to reach
/// the agent's next turn.
const POLL_INTERVAL: Duration = Duration::from_secs(10);
/// Where the settings are applied, and where the local values come from.
trait Target {
fn model(&self) -> String;
fn effort(&self) -> String;
fn apply_model(&self, value: String);
fn apply_effort(&self, value: String);
fn local_model(&self) -> String;
fn local_effort(&self) -> String;
}
/// The harness's Bus, with the nix values read once at start.
struct Harness {
bus: Bus,
nix_model: Option<&'static str>,
nix_effort: Option<&'static str>,
}
impl Target for Harness {
fn model(&self) -> String {
self.bus.model()
}
fn effort(&self) -> String {
self.bus.effort()
}
fn apply_model(&self, value: String) {
self.bus.apply_model(value);
}
fn apply_effort(&self, value: String) {
self.bus.apply_effort(value);
}
fn local_model(&self) -> String {
local_model(self.nix_model)
}
fn local_effort(&self) -> String {
local_effort(self.nix_effort)
}
}
/// Which fields currently carry the swarm's value.
#[derive(Debug, Default)]
struct Overridden {
model: bool,
effort: bool,
}
/// What one field should become: the swarm's value while it sets one, the
/// local value on the read after it stops, and otherwise `None` (left alone).
/// Also `None` when the value is already what it should be.
fn next(
swarm: Option<&str>,
overridden: &mut bool,
current: &str,
local: impl FnOnce() -> String,
) -> Option<String> {
let wanted = match swarm {
Some(value) => {
*overridden = true;
value.to_owned()
}
None if std::mem::take(overridden) => local(),
None => return None,
};
(wanted != current).then_some(wanted)
}
impl Overridden {
/// Apply one read of the key to `target`.
fn apply(&mut self, read: &AgentInference, target: &impl Target) {
if let Some(model) = next(
read.model.as_deref(),
&mut self.model,
&target.model(),
|| target.local_model(),
) {
tracing::info!(%model, swarm = self.model, "inference: switching model");
target.apply_model(model);
}
if let Some(effort) = next(
read.effort.map(Effort::as_str),
&mut self.effort,
&target.effort(),
|| target.local_effort(),
) {
tracing::info!(%effort, swarm = self.effort, "inference: switching effort");
target.apply_effort(effort);
}
}
}
/// Run the poller for the harness's lifetime, if this agent has a label to
/// name its key with.
pub async fn run(bus: Bus) {
let agent = crate::identity::label();
if agent.is_empty() {
tracing::warn!("this agent has no label; not reading its swarm inference settings");
return;
}
let target = Harness {
bus,
nix_model: configured_model(),
nix_effort: configured_effort(),
};
let client = crate::swarm_queue::client().await;
tracing::info!(%agent, "reading swarm inference settings from the swarm queue");
let mut overridden = Overridden::default();
loop {
if let Some(read) = read(&client, &agent).await {
overridden.apply(&read, &target);
}
tokio::time::sleep(POLL_INTERVAL).await;
}
}
/// The key's current value, or `None` when it could not be read.
async fn read(client: &async_nats::Client, agent: &str) -> Option<AgentInference> {
// An unconnected client does not fail a JetStream request, it hangs on it.
swarm_queue_client::ensure_connected(client).ok()?;
match read_own(client, agent).await {
Ok(Read::Current(current)) => Some(current),
Ok(Read::NoBucket) => {
tracing::debug!("inference: the agent-inference bucket does not answer yet");
None
}
Err(e) => {
tracing::warn!(error = %e, "inference: reading this agent's settings failed");
None
}
}
}
#[cfg(test)]
mod tests {
use std::cell::RefCell;
use super::*;
/// A Bus stand-in recording every apply.
struct Fake {
model: RefCell<String>,
effort: RefCell<String>,
local_model: RefCell<String>,
local_effort: String,
applies: RefCell<usize>,
}
impl Fake {
fn new() -> Self {
Self {
model: RefCell::new("sonnet".to_owned()),
effort: RefCell::new("medium".to_owned()),
local_model: RefCell::new("sonnet".to_owned()),
local_effort: "medium".to_owned(),
applies: RefCell::new(0),
}
}
fn now(&self) -> (String, String) {
(self.model.borrow().clone(), self.effort.borrow().clone())
}
}
impl Target for Fake {
fn model(&self) -> String {
self.model.borrow().clone()
}
fn effort(&self) -> String {
self.effort.borrow().clone()
}
fn apply_model(&self, value: String) {
*self.applies.borrow_mut() += 1;
*self.model.borrow_mut() = value;
}
fn apply_effort(&self, value: String) {
*self.applies.borrow_mut() += 1;
*self.effort.borrow_mut() = value;
}
fn local_model(&self) -> String {
self.local_model.borrow().clone()
}
fn local_effort(&self) -> String {
self.local_effort.clone()
}
}
fn set(model: Option<&str>, effort: Option<Effort>) -> AgentInference {
AgentInference {
model: model.map(str::to_owned),
effort,
subagent_models: None,
}
}
fn pair(model: &str, effort: &str) -> (String, String) {
(model.to_owned(), effort.to_owned())
}
#[test]
fn a_set_field_replaces_the_bus_value() {
let fake = Fake::new();
let mut o = Overridden::default();
o.apply(&set(Some("opus"), Some(Effort::High)), &fake);
assert_eq!(fake.now(), pair("opus", "high"));
// Re-reading the same value applies nothing again.
o.apply(&set(Some("opus"), Some(Effort::High)), &fake);
assert_eq!(*fake.applies.borrow(), 2);
}
/// An absent key, and a key with every field unset, leave the agent on
/// whatever it runs now — even when that differs from its local value.
#[test]
fn an_absent_key_leaves_the_bus_alone() {
let fake = Fake::new();
*fake.model.borrow_mut() = "haiku".to_owned();
let mut o = Overridden::default();
for _ in 0..3 {
o.apply(&AgentInference::default(), &fake);
}
assert_eq!(fake.now(), pair("haiku", "medium"));
assert_eq!(*fake.applies.borrow(), 0);
}
/// Only the field the swarm sets moves; the other keeps its value.
#[test]
fn an_unset_field_beside_a_set_one_is_left_alone() {
let fake = Fake::new();
*fake.effort.borrow_mut() = "max".to_owned();
let mut o = Overridden::default();
o.apply(&set(Some("opus"), None), &fake);
assert_eq!(fake.now(), pair("opus", "max"));
}
#[test]
fn a_cleared_field_goes_back_to_the_local_value() {
let fake = Fake::new();
let mut o = Overridden::default();
o.apply(&set(Some("opus"), Some(Effort::Xhigh)), &fake);
assert_eq!(fake.now(), pair("opus", "xhigh"));
// The effort is cleared, the model still set.
o.apply(&set(Some("opus"), None), &fake);
assert_eq!(fake.now(), pair("opus", "medium"));
// The local value is read at the clear, not at start.
"haiku".clone_into(&mut fake.local_model.borrow_mut());
o.apply(&AgentInference::default(), &fake);
assert_eq!(fake.now(), pair("haiku", "medium"));
// Once reverted, a later unset read changes nothing.
*fake.model.borrow_mut() = "sonnet".to_owned();
o.apply(&AgentInference::default(), &fake);
assert_eq!(fake.now(), pair("sonnet", "medium"));
}
/// On the real Bus: the swarm's value reaches the Bus but not the
/// persisted files, so clearing it, or a restart, finds the local value.
#[test]
fn the_swarm_value_is_not_persisted_over_the_local_one() {
let dir = tempfile::tempdir().unwrap();
let prev = std::env::var_os("HYPERHIVE_HARNESS_DIR");
// SAFETY: no other test in this crate reads or sets this variable.
unsafe { std::env::set_var("HYPERHIVE_HARNESS_DIR", dir.path()) };
let bus = Bus::new();
bus.set_model("sonnet");
bus.set_effort("low");
let target = Harness {
bus: bus.clone(),
nix_model: None,
nix_effort: None,
};
let mut o = Overridden::default();
o.apply(&set(Some("opus"), Some(Effort::Max)), &target);
let swarm = (bus.model(), bus.effort());
let files = (
crate::harness_state::load_model(),
crate::harness_state::load_effort(),
);
o.apply(&AgentInference::default(), &target);
let cleared = (bus.model(), bus.effort());
match prev {
// SAFETY: as above.
Some(v) => unsafe { std::env::set_var("HYPERHIVE_HARNESS_DIR", v) },
None => unsafe { std::env::remove_var("HYPERHIVE_HARNESS_DIR") },
}
assert_eq!(swarm, pair("opus", "max"));
assert_eq!(files, (Some("sonnet".to_owned()), Some("low".to_owned())));
assert_eq!(cleared, pair("sonnet", "low"));
}
}

View file

@ -739,6 +739,9 @@ in
# `CONSUMER.CREATE` (a watch's create subject carries no key, # `CONSUMER.CREATE` (a watch's create subject carries no key,
# so it would cover every agent's) and no `$KV` write. # so it would cover every agent's) and no `$KV` write.
"--agent-token-publish-subject ${lib.escapeShellArg "\$\$JS.API.DIRECT.GET.KV_agent-schedules.\$\$KV.agent-schedules.{agent}"}" "--agent-token-publish-subject ${lib.escapeShellArg "\$\$JS.API.DIRECT.GET.KV_agent-schedules.\$\$KV.agent-schedules.{agent}"}"
# Its own key of the `agent-inference` KV bucket
# (`swarm_queue_client::agent_inference`), on the same terms.
"--agent-token-publish-subject ${lib.escapeShellArg "\$\$JS.API.DIRECT.GET.KV_agent-inference.\$\$KV.agent-inference.{agent}"}"
"--store-cert-role ${lib.escapeShellArg authCertRole}" "--store-cert-role ${lib.escapeShellArg authCertRole}"
]; ];
# The store identity arrives by `LoadCredential` and is named # The store identity arrives by `LoadCredential` and is named

View file

@ -295,7 +295,7 @@ let
# The whole per-agent grant, compared as a list rather than by infix: a # The whole per-agent grant, compared as a list rather than by infix: a
# wider subject added beside these (`$$JS.API.>`, another stream's name) # wider subject added beside these (`$$JS.API.>`, another stream's name)
# would pass every presence check above. # would pass every presence check above.
name = "a verified agent's grant is exactly its own subjects, and its one JetStream API subject reads its own schedule key"; name = "a verified agent's grant is exactly its own subjects, and its JetStream API subjects read its own schedule and inference keys";
ok = ok =
let let
args = lib.splitString " " (responderOf natsOldPath).serviceConfig.ExecStart; args = lib.splitString " " (responderOf natsOldPath).serviceConfig.ExecStart;
@ -314,6 +314,7 @@ let
"'$$KV.agent-icons.{agent}'" "'$$KV.agent-icons.{agent}'"
"'$$SWARM.term.{agent}.sub.>'" "'$$SWARM.term.{agent}.sub.>'"
"'$$JS.API.DIRECT.GET.KV_agent-schedules.$$KV.agent-schedules.{agent}'" "'$$JS.API.DIRECT.GET.KV_agent-schedules.$$KV.agent-schedules.{agent}'"
"'$$JS.API.DIRECT.GET.KV_agent-inference.$$KV.agent-inference.{agent}'"
]; ];
} }
{ {

View file

@ -46,6 +46,10 @@ The swarm's control plane. The swarm UI and `swarmctl` are its clients.
`agent-schedules`, both of which this daemon alone writes `agent-schedules`, both of which this daemon alone writes
(`GET`/`POST /api/schedules`, `PATCH`/`DELETE /api/schedules/{id}`); each (`GET`/`POST /api/schedules`, `PATCH`/`DELETE /api/schedules/{id}`); each
agent fires them itself → [`docs/swarm/schedules.md`](../docs/swarm/schedules.md). agent fires them itself → [`docs/swarm/schedules.md`](../docs/swarm/schedules.md).
- **Inference settings** — each agent's model, effort and subagent models,
one key per agent in the `agent-inference` KV bucket, which this daemon
alone writes (`GET`/`PUT`/`DELETE /api/agents/{name}/inference`); each
agent applies them itself → [`docs/swarm/inference.md`](../docs/swarm/inference.md).
- **Relays** — each agent's terminal and turn-state header as SSE, agent - **Relays** — each agent's terminal and turn-state header as SSE, agent
icons, the cross-repo issue report, and the UI's quick links. icons, the cross-repo issue report, and the UI's quick links.

View file

@ -0,0 +1,295 @@
//! `/api/agents/{name}/inference` — the model, effort and subagent models the
//! swarm sets for one agent.
//!
//! Each agent's settings are one value under its own key of `agent-inference`
//! (`swarm_queue_client::agent_inference`), which this daemon alone writes.
//! The agent reads its key and applies each set field over its local value;
//! an unset field leaves the local value in force.
use std::collections::BTreeSet;
use axum::Json;
use axum::extract::{Path, State};
use axum::http::StatusCode;
use problem_details::ProblemDetails;
use serde::{Deserialize, Serialize};
use swarm_queue_client::agent_inference::{self, AgentInference, Effort};
use utoipa::ToSchema;
use crate::schedules::{live, queue, read, refused, write};
use crate::{AppState, error_problem, wanted_error_status};
/// How many times a losing writer re-reads and re-applies, as in
/// `wanted::MAX_ATTEMPTS`.
const MAX_ATTEMPTS: usize = 5;
/// One agent's inference settings. An absent field is not set at swarm level,
/// and the agent keeps its own value for it.
#[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize, ToSchema)]
#[serde(deny_unknown_fields)]
pub struct InferenceSettings {
/// The model the agent's turns run on, as claude's `--model` takes it.
#[serde(default, skip_serializing_if = "Option::is_none")]
model: Option<String>,
/// One of `low`, `medium`, `high`, `xhigh`, `max`.
#[serde(default, skip_serializing_if = "Option::is_none")]
#[schema(value_type = Option<String>)]
effort: Option<Effort>,
/// The models the agent may start its subagents on. At least one.
#[serde(default, skip_serializing_if = "Option::is_none")]
subagent_models: Option<Vec<String>>,
}
impl From<AgentInference> for InferenceSettings {
fn from(v: AgentInference) -> Self {
Self {
model: v.model,
effort: v.effort,
subagent_models: v.subagent_models,
}
}
}
/// A model name as the agent passes it to claude: one non-empty word.
fn check_model(name: &str, what: &str) -> Result<(), String> {
if name.is_empty() {
return Err(format!("the {what} is empty"));
}
if name.chars().any(|c| c.is_whitespace() || c.is_control()) {
return Err(format!("the {what} {name:?} contains whitespace"));
}
Ok(())
}
impl InferenceSettings {
/// The value to store, or why it is refused.
fn into_stored(self) -> Result<AgentInference, String> {
if let Some(model) = &self.model {
check_model(model, "model")?;
}
if let Some(models) = &self.subagent_models {
if models.is_empty() {
return Err(
"subagent_models is empty; leave it out to keep the agent's own list"
.to_owned(),
);
}
let mut seen = BTreeSet::new();
for model in models {
check_model(model, "subagent model")?;
if !seen.insert(model) {
return Err(format!("subagent model {model:?} is listed twice"));
}
}
}
Ok(AgentInference {
model: self.model,
effort: self.effort,
subagent_models: self.subagent_models,
})
}
}
fn agent_name(name: &str) -> Result<String, (StatusCode, String)> {
hive_types::Ident::parse(name)
.map(hive_types::Ident::into_string)
.map_err(|reason| (StatusCode::BAD_REQUEST, reason.to_owned()))
}
fn queue_problem(e: &anyhow::Error) -> ProblemDetails {
tracing::warn!(error = %format!("{e:#}"), "inference: queue failure");
error_problem(wanted_error_status(e), &format!("{e:#}"))
}
async fn open(
client: &async_nats::Client,
) -> Result<async_nats::jetstream::kv::Store, ProblemDetails> {
agent_inference::open_or_create(client)
.await
.map_err(|e| queue_problem(&anyhow::Error::new(e)))
}
#[utoipa::path(
get,
path = "/api/agents/{name}/inference",
params(("name" = String, Path, description = "agent name")),
responses(
(status = 200, description = "the agent's settings; `{}` when none is set",
body = InferenceSettings),
(status = 400, description = "not an identifier (problem+json)", body = String),
(status = 503, description = "no swarm queue is configured on this host, or it is \
not connected (problem+json)", body = String),
(status = 500, description = "the settings could not be read (problem+json)",
body = String),
),
tag = "agents"
)]
pub(crate) async fn get_agent_inference(
State(state): State<AppState>,
Path(name): Path<String>,
) -> Result<Json<InferenceSettings>, ProblemDetails> {
let agent = agent_name(&name).map_err(refused)?;
let client = queue(&state).map_err(refused)?;
let store = open(&client).await?;
let (current, _) = read::<AgentInference>(&store, &agent)
.await
.map_err(|e| queue_problem(&e))?;
Ok(Json(current.into()))
}
#[utoipa::path(
put,
path = "/api/agents/{name}/inference",
params(("name" = String, Path, description = "agent name")),
request_body = InferenceSettings,
responses(
(status = 200, description = "the settings as stored; they replace the previous \
ones whole", body = InferenceSettings),
(status = 400, description = "not an identifier, not declared on any hive, or a \
model name is empty, has whitespace, or is listed twice (problem+json)",
body = String),
(status = 422, description = "an unknown field, or an effort that is not one of \
the five levels"),
(status = 503, description = "no swarm queue is configured on this host, or it is \
not connected (problem+json)", body = String),
(status = 500, description = "the settings could not be written (problem+json)",
body = String),
),
tag = "agents"
)]
pub(crate) async fn put_agent_inference(
State(state): State<AppState>,
Path(name): Path<String>,
Json(req): Json<InferenceSettings>,
) -> Result<Json<InferenceSettings>, ProblemDetails> {
let agent = agent_name(&name).map_err(refused)?;
let value = req
.into_stored()
.map_err(|why| error_problem(StatusCode::BAD_REQUEST, &why))?;
let client = queue(&state).map_err(refused)?;
if !live(&state).await?.contains(&agent) {
return Err(error_problem(
StatusCode::BAD_REQUEST,
&format!("no hive's wanted state declares an agent {agent:?} that is not destroyed"),
));
}
let store = open(&client).await?;
for _ in 0..MAX_ATTEMPTS {
let (current, revision) = read::<AgentInference>(&store, &agent)
.await
.map_err(|e| queue_problem(&e))?;
if current == value
|| write(&store, &agent, &value, revision)
.await
.map_err(|e| queue_problem(&e))?
{
tracing::info!(%agent, ?value, "set inference settings");
return Ok(Json(value.into()));
}
}
Err(queue_problem(&anyhow::anyhow!(
"gave up writing the inference settings of {agent} after {MAX_ATTEMPTS} conflicting \
writes"
)))
}
#[utoipa::path(
delete,
path = "/api/agents/{name}/inference",
params(("name" = String, Path, description = "agent name")),
responses(
(status = 204, description = "cleared; the agent goes back to its own values"),
(status = 400, description = "not an identifier (problem+json)", body = String),
(status = 503, description = "no swarm queue is configured on this host, or it is \
not connected (problem+json)", body = String),
(status = 500, description = "the settings could not be cleared (problem+json)",
body = String),
),
tag = "agents"
)]
pub(crate) async fn delete_agent_inference(
State(state): State<AppState>,
Path(name): Path<String>,
) -> Result<StatusCode, ProblemDetails> {
let agent = agent_name(&name).map_err(refused)?;
let client = queue(&state).map_err(refused)?;
let store = open(&client).await?;
store
.delete(&agent)
.await
.map_err(|e| queue_problem(&anyhow::Error::new(e)))?;
tracing::info!(%agent, "cleared inference settings");
Ok(StatusCode::NO_CONTENT)
}
#[cfg(test)]
mod tests {
use super::*;
fn settings(json: &str) -> serde_json::Result<InferenceSettings> {
serde_json::from_str(json)
}
#[test]
fn a_full_put_is_stored_as_sent() {
let stored = settings(r#"{"model":"opus","effort":"high","subagent_models":["haiku"]}"#)
.unwrap()
.into_stored()
.unwrap();
assert_eq!(
stored,
AgentInference {
model: Some("opus".to_owned()),
effort: Some(Effort::High),
subagent_models: Some(vec!["haiku".to_owned()]),
}
);
assert_eq!(
serde_json::to_value(InferenceSettings::from(stored)).unwrap(),
serde_json::json!({"model": "opus", "effort": "high", "subagent_models": ["haiku"]})
);
}
/// Each field is optional, and one left out stays out of the stored value.
#[test]
fn a_partial_put_leaves_the_other_fields_unset() {
let stored = settings(r#"{"effort":"max"}"#)
.unwrap()
.into_stored()
.unwrap();
assert_eq!(
stored,
AgentInference {
effort: Some(Effort::Max),
..AgentInference::default()
}
);
}
#[test]
fn an_unknown_field_or_effort_does_not_decode() {
for json in [
r#"{"modle":"opus"}"#,
r#"{"available_models":["opus"]}"#,
r#"{"effort":"extreme"}"#,
r#"{"effort":"HIGH"}"#,
] {
assert!(settings(json).is_err(), "{json}");
}
// Control: the same shape with a known field and level decodes.
assert!(settings(r#"{"model":"opus","effort":"xhigh"}"#).is_ok());
}
#[test]
fn bad_model_names_are_refused() {
for json in [
r#"{"model":""}"#,
r#"{"model":"opus 4"}"#,
r#"{"subagent_models":[]}"#,
r#"{"subagent_models":["haiku",""]}"#,
r#"{"subagent_models":["haiku","haiku"]}"#,
] {
assert!(settings(json).unwrap().into_stored().is_err(), "{json}");
}
}
}

View file

@ -52,6 +52,7 @@ mod config_pr;
mod forge; mod forge;
mod forge_account; mod forge_account;
mod github_account; mod github_account;
mod inference;
mod issue_report; mod issue_report;
mod linked_accounts; mod linked_accounts;
mod matrix_account; mod matrix_account;
@ -3211,6 +3212,11 @@ fn build_app(state: AppState) -> axum::Router {
.routes(routes!(get_jobq_rollup)) .routes(routes!(get_jobq_rollup))
.routes(routes!(get_agent_config_pr)) .routes(routes!(get_agent_config_pr))
.routes(routes!(get_agent_icon)) .routes(routes!(get_agent_icon))
.routes(routes!(
inference::get_agent_inference,
inference::put_agent_inference,
inference::delete_agent_inference
))
.routes(routes!(get_config_prs)) .routes(routes!(get_config_prs))
.routes(routes!(create_agent)) .routes(routes!(create_agent))
.routes(routes!(mint_agent_identity)) .routes(routes!(mint_agent_identity))
@ -5479,6 +5485,59 @@ mod tests {
); );
} }
/// Every inference verb refuses a name that is not an identifier with a
/// 400 and answers 503 without a queue; a PUT whose body is refused is a
/// 400 before the queue is asked.
#[tokio::test]
async fn the_inference_routes_check_the_name_and_body_before_the_queue() {
use super::inference::{delete_agent_inference, get_agent_inference, put_agent_inference};
use axum::extract::{Path, State};
use axum::http::StatusCode;
fn status<T>(r: Result<T, problem_details::ProblemDetails>) -> Option<StatusCode> {
r.map(|_| ()).expect_err("must not succeed").status
}
let (state, _sched) = state_with_roster();
let body = |json: &str| axum::Json(serde_json::from_str(json).expect("decodes"));
for name in ["iris", "Not An Ident"] {
let want = Some(if name == "iris" {
StatusCode::SERVICE_UNAVAILABLE
} else {
StatusCode::BAD_REQUEST
});
let path = || Path(name.to_owned());
assert_eq!(
status(get_agent_inference(State(state.clone()), path()).await),
want,
"GET {name}"
);
assert_eq!(
status(
put_agent_inference(State(state.clone()), path(), body(r#"{"model":"opus"}"#))
.await
),
want,
"PUT {name}"
);
assert_eq!(
status(delete_agent_inference(State(state.clone()), path()).await),
want,
"DELETE {name}"
);
}
assert_eq!(
status(
put_agent_inference(
State(state),
Path("iris".to_owned()),
body(r#"{"model":"opus 4"}"#)
)
.await
),
Some(StatusCode::BAD_REQUEST)
);
}
#[tokio::test] #[tokio::test]
async fn listing_subagents_needs_a_queue_and_an_identifier() { async fn listing_subagents_needs_a_queue_and_an_identifier() {
let (state, _sched) = state_with_roster(); let (state, _sched) = state_with_roster();

View file

@ -404,7 +404,7 @@ async fn open_or_create_definitions(client: &async_nats::Client) -> anyhow::Resu
/// The value of `key` decoded, with the revision to write it back against. /// The value of `key` decoded, with the revision to write it back against.
/// An absent or deleted key is the default value. /// An absent or deleted key is the default value.
async fn read<T: Default + serde::de::DeserializeOwned>( pub(crate) async fn read<T: Default + serde::de::DeserializeOwned>(
store: &Store, store: &Store,
key: &str, key: &str,
) -> anyhow::Result<(T, Option<u64>)> { ) -> anyhow::Result<(T, Option<u64>)> {
@ -425,7 +425,7 @@ async fn read<T: Default + serde::de::DeserializeOwned>(
/// Write `value` to `key` against `revision`. `Ok(false)` when another write /// Write `value` to `key` against `revision`. `Ok(false)` when another write
/// landed first. /// landed first.
async fn write<T: Serialize>( pub(crate) async fn write<T: Serialize>(
store: &Store, store: &Store,
key: &str, key: &str,
value: &T, value: &T,
@ -549,7 +549,7 @@ pub(crate) fn spawn_reconcile(
} }
/// The connected queue client, or the problem. /// The connected queue client, or the problem.
fn queue(state: &AppState) -> Result<async_nats::Client, (StatusCode, String)> { pub(crate) fn queue(state: &AppState) -> Result<async_nats::Client, (StatusCode, String)> {
let status = state.status.as_ref().ok_or_else(|| { let status = state.status.as_ref().ok_or_else(|| {
( (
StatusCode::SERVICE_UNAVAILABLE, StatusCode::SERVICE_UNAVAILABLE,
@ -566,12 +566,12 @@ fn queue(state: &AppState) -> Result<async_nats::Client, (StatusCode, String)> {
Ok(client) Ok(client)
} }
fn refused((status, detail): (StatusCode, String)) -> ProblemDetails { pub(crate) fn refused((status, detail): (StatusCode, String)) -> ProblemDetails {
error_problem(status, &detail) error_problem(status, &detail)
} }
/// Every agent some hive declares and has not destroyed. /// Every agent some hive declares and has not destroyed.
async fn live(state: &AppState) -> Result<BTreeSet<String>, ProblemDetails> { pub(crate) async fn live(state: &AppState) -> Result<BTreeSet<String>, ProblemDetails> {
let Some(wanted) = state.wanted.as_ref() else { let Some(wanted) = state.wanted.as_ref() else {
return Err(error_problem( return Err(error_problem(
StatusCode::SERVICE_UNAVAILABLE, StatusCode::SERVICE_UNAVAILABLE,

View file

@ -507,12 +507,14 @@ mod tests {
"$SWARM.term.{agent}.sub.>", "$SWARM.term.{agent}.sub.>",
"--agent-token-publish-subject", "--agent-token-publish-subject",
"$JS.API.DIRECT.GET.KV_agent-schedules.$KV.agent-schedules.{agent}", "$JS.API.DIRECT.GET.KV_agent-schedules.$KV.agent-schedules.{agent}",
"--agent-token-publish-subject",
"$JS.API.DIRECT.GET.KV_agent-inference.$KV.agent-inference.{agent}",
"--store-cert-role", "--store-cert-role",
"swarm-nats-auth", "swarm-nats-auth",
]) ])
.expect("the unit's own argument vector must parse"); .expect("the unit's own argument vector must parse");
assert_eq!(args.agent_client_suffix, "-agent"); assert_eq!(args.agent_client_suffix, "-agent");
assert_eq!(args.agent_token_publish_subjects.len(), 5); assert_eq!(args.agent_token_publish_subjects.len(), 6);
} }
/// A request with no `Nats-Server-Xkey` is dropped before anything reads /// A request with no `Nats-Server-Xkey` is dropped before anything reads

View file

@ -1440,6 +1440,39 @@ mod tests {
assert!(covers("$KV.*.>", &key_subject("argus"))); assert!(covers("$KV.*.>", &key_subject("argus")));
} }
/// The inference template as `swarm-nats.nix` spells it reads one key, the
/// agent's own, and nothing that writes, watches or opens the bucket.
#[test]
fn an_agent_may_read_only_its_own_inference_key() {
use swarm_queue_client::agent_inference::{STREAM, direct_get_subject, key_subject};
let p = policy()
.with_agent_token_subjects(vec![
"$JS.API.DIRECT.GET.KV_agent-inference.$KV.agent-inference.{agent}".to_owned(),
])
.expect("a per-agent template is valid");
let atlas = p.agent_token_permissions("atlas").expect("configured");
assert_eq!(atlas.publish, vec![direct_get_subject("atlas")]);
let argus = p.agent_token_permissions("argus").expect("configured");
assert_eq!(argus.publish, vec![direct_get_subject("argus")]);
for refused in [
direct_get_subject("argus"),
format!("$JS.API.DIRECT.GET.{STREAM}"),
format!("$JS.API.STREAM.INFO.{STREAM}"),
format!("$JS.API.CONSUMER.CREATE.{STREAM}"),
key_subject("atlas"),
swarm_queue_client::agent_schedules::direct_get_subject("atlas"),
] {
assert!(
!atlas.publish.iter().any(|g| covers(g, &refused)),
"atlas's grant covers {refused}"
);
}
// Control: the matcher matches the one subject granted.
assert!(covers(&atlas.publish[0], &direct_get_subject("atlas")));
}
#[test] #[test]
fn an_agent_token_subject_without_the_placeholder_is_refused() { fn an_agent_token_subject_without_the_placeholder_is_refused() {
let err = policy() let err = policy()

View file

@ -37,6 +37,9 @@ subagent-term = ["async-nats/jetstream"]
# schedule key without opening the bucket. Explicit for the same reason as # schedule key without opening the bucket. Explicit for the same reason as
# `notices`. # `notices`.
agent-schedules = ["async-nats/jetstream"] agent-schedules = ["async-nats/jetstream"]
# `agent_inference::read_own`, for the agent harness, which reads its own
# inference key the same way. Explicit for the same reason as `notices`.
agent-inference = ["async-nats/jetstream"]
[dependencies] [dependencies]
# Bare (no `kv`/`jetstream`) unless a consumer opts into the `kv` feature # Bare (no `kv`/`jetstream`) unless a consumer opts into the `kv` feature

View file

@ -0,0 +1,235 @@
//! The per-agent inference-settings KV bucket: one key per agent, holding the
//! model, effort and subagent model list the swarm sets for it.
//!
//! **swarm-controller is the only writer**, and creates the bucket on its
//! first write. Every field is optional: an absent field leaves the agent on
//! its own local value, so an agent with no key runs exactly as it would with
//! no swarm.
//!
//! **An agent reads with one `DIRECT.GET` on its own key, not a watch**, for
//! the reason [`crate::agent_schedules`] gives: the direct get names the key
//! in its subject ([`crate::agent_inference::direct_get_subject`]), which is
//! what lets the queue grant scope the read to one agent.
//!
//! Keyed by the agent alone, like [`crate::agent_schedules`]: an agent can
//! move between hives.
/// The KV bucket inference settings are published into, keyed by agent name.
pub const BUCKET: &str = "agent-inference";
/// The stream backing [`BUCKET`]. `create_key_value` prefixes `KV_`.
pub const STREAM: &str = "KV_agent-inference";
/// The subject `agent`'s entry is stored under.
#[must_use]
pub fn key_subject(agent: &str) -> String {
format!("$KV.{BUCKET}.{agent}")
}
/// The request subject that reads `agent`'s current entry, and so the one
/// subject the queue grants that agent in this bucket.
#[must_use]
pub fn direct_get_subject(agent: &str) -> String {
format!("$JS.API.DIRECT.GET.{STREAM}.{}", key_subject(agent))
}
/// One agent's inference settings: the value under `BUCKET`/`<agent>`.
/// `None` is "not set at swarm level", never "use a default".
#[derive(Debug, Clone, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
pub struct AgentInference {
/// The model the agent's turns run on.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub model: Option<String>,
/// The reasoning effort the agent's turns run at.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub effort: Option<Effort>,
/// The models the agent may start its subagents on.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub subagent_models: Option<Vec<String>>,
}
/// A claude `--effort` level, ascending.
#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum Effort {
Low,
Medium,
High,
Xhigh,
Max,
}
impl Effort {
/// The level as claude's `--effort` takes it, and as it is stored.
#[must_use]
pub const fn as_str(self) -> &'static str {
match self {
Self::Low => "low",
Self::Medium => "medium",
Self::High => "high",
Self::Xhigh => "xhigh",
Self::Max => "max",
}
}
}
/// Open the inference bucket for writing, creating it if nothing has yet.
///
/// **Controller-side only.** `history: 1`: an agent wants its current value.
#[cfg(feature = "kv")]
pub async fn open_or_create(
client: &async_nats::Client,
) -> Result<async_nats::jetstream::kv::Store, crate::Error> {
let js = async_nats::jetstream::new(client.clone());
crate::kv::open_or_create(
&js,
BUCKET,
"agent-inference",
"Inference settings the swarm controller sets for each agent".to_owned(),
)
.await
}
/// What reading one agent's key found.
#[cfg(feature = "agent-inference")]
#[derive(Debug)]
pub enum Read {
/// The bucket answered; an absent or deleted key has every field unset.
Current(AgentInference),
/// Nothing answers for the bucket's stream: no setting has been written
/// yet, or `JetStream` is not serving it right now. Not an unset value,
/// since the second case says nothing about the agent's settings.
NoBucket,
}
/// Read `agent`'s current settings with the one subject its grant covers.
///
/// # Errors
///
/// A failed request or an undecodable value. A request the queue refused for
/// permissions reaches the client as a timeout, not as a refusal.
#[cfg(feature = "agent-inference")]
pub async fn read_own(
client: &async_nats::Client,
agent: &str,
) -> Result<Read, Box<dyn std::error::Error + Send + Sync>> {
use async_nats::jetstream::stream::DirectGetErrorKind;
let js = async_nats::jetstream::new(client.clone());
let stream = js.get_stream_no_info(STREAM).await?;
let message = match stream.direct_get_last_for_subject(key_subject(agent)).await {
Ok(message) => message,
Err(e) if matches!(e.kind(), DirectGetErrorKind::NotFound) => {
return Ok(Read::Current(AgentInference::default()));
}
Err(e) if is_no_responders(&e) => return Ok(Read::NoBucket),
Err(e) => return Err(e.into()),
};
// A KV delete or purge is stored as an empty marker carrying this header.
let removed = message
.headers
.get("KV-Operation")
.is_some_and(|op| matches!(op.as_str(), "DEL" | "PURGE"));
if removed {
return Ok(Read::Current(AgentInference::default()));
}
Ok(Read::Current(serde_json::from_slice(&message.payload)?))
}
#[cfg(feature = "agent-inference")]
fn is_no_responders(e: &async_nats::jetstream::stream::DirectGetError) -> bool {
matches!(
e.kind(),
async_nats::jetstream::stream::DirectGetErrorKind::ErrorResponse(status, _)
if status == async_nats::StatusCode::NO_RESPONDERS
)
}
#[cfg(test)]
mod tests {
use super::*;
/// The names the grant in `swarm-nats.nix` spells with `{agent}`.
#[test]
fn names_match_the_queue_grant() {
assert_eq!(STREAM, format!("KV_{BUCKET}"));
assert_eq!(key_subject("iris"), "$KV.agent-inference.iris");
assert_eq!(
direct_get_subject("iris"),
"$JS.API.DIRECT.GET.KV_agent-inference.$KV.agent-inference.iris"
);
}
#[test]
fn a_full_value_round_trips() {
let value = AgentInference {
model: Some("opus".to_owned()),
effort: Some(Effort::Xhigh),
subagent_models: Some(vec!["haiku".to_owned(), "sonnet".to_owned()]),
};
let encoded = serde_json::to_string(&value).unwrap();
assert_eq!(
encoded,
r#"{"model":"opus","effort":"xhigh","subagent_models":["haiku","sonnet"]}"#
);
assert_eq!(
serde_json::from_str::<AgentInference>(&encoded).unwrap(),
value
);
}
/// An unset field is left out on the wire, and `{}` is every field unset.
#[test]
fn unset_fields_are_absent() {
let value = AgentInference {
effort: Some(Effort::Low),
..AgentInference::default()
};
assert_eq!(
serde_json::to_string(&value).unwrap(),
r#"{"effort":"low"}"#
);
assert_eq!(
serde_json::from_str::<AgentInference>("{}").unwrap(),
AgentInference::default()
);
}
#[test]
fn every_effort_is_stored_as_its_cli_spelling() {
for effort in [
Effort::Low,
Effort::Medium,
Effort::High,
Effort::Xhigh,
Effort::Max,
] {
assert_eq!(
serde_json::to_string(&effort).unwrap(),
format!("\"{}\"", effort.as_str())
);
}
}
#[test]
fn an_unknown_effort_is_refused() {
for doc in [
r#"{"effort":"extreme"}"#,
r#"{"effort":"High"}"#,
r#"{"effort":""}"#,
] {
assert!(
serde_json::from_str::<AgentInference>(doc).is_err(),
"{doc}"
);
}
}
/// Fields a newer controller adds must not cost an older agent its value.
#[test]
fn decoding_tolerates_unknown_fields() {
let decoded: AgentInference =
serde_json::from_str(r#"{"model":"opus","temperature":1}"#).unwrap();
assert_eq!(decoded.model.as_deref(), Some("opus"));
}
}

View file

@ -217,6 +217,11 @@ pub mod agent_icon;
/// writing needs `kv` and reading needs `agent-schedules`. /// writing needs `kv` and reading needs `agent-schedules`.
pub mod agent_schedules; pub mod agent_schedules;
/// The bucket swarm-controller sets each agent's model, effort and subagent
/// models in — one key per agent, read by that agent alone. The names are
/// unconditional; writing needs `kv` and reading needs `agent-inference`.
pub mod agent_inference;
#[cfg(feature = "kv")] #[cfg(feature = "kv")]
mod kv; mod kv;

182
swarmctl/src/inference.rs Normal file
View file

@ -0,0 +1,182 @@
//! `swarmctl agent inference` — the model, effort and subagent models the
//! swarm sets for one agent, through swarm-controller's
//! `/api/agents/{name}/inference` routes.
//!
//! The agent picks a change up on its next read of its key, within ten
//! seconds, and its next turn runs on it. A field the swarm does not set
//! leaves the agent on its own value.
//!
//! The shape below mirrors the controller's `InferenceSettings`, for the
//! reason [`crate::agent`]'s module doc gives.
use std::path::Path;
use anyhow::{Context as _, Result, bail};
use serde::{Deserialize, Serialize};
use crate::agent::{parse_ident, send};
/// One agent's settings; an absent field is not set at swarm level.
#[derive(Debug, Default, PartialEq, Eq, Serialize, Deserialize)]
struct Settings {
#[serde(default, skip_serializing_if = "Option::is_none")]
model: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
effort: Option<String>,
#[serde(default, skip_serializing_if = "Option::is_none")]
subagent_models: Option<Vec<String>>,
}
/// What `agent inference set` was asked to change.
pub(crate) struct SetFlags {
pub model: Option<String>,
pub effort: Option<String>,
pub subagent_models: Option<Vec<String>>,
}
impl SetFlags {
/// `current` with each given flag written over it.
fn merge(self, current: Settings) -> Result<Settings> {
if self.model.is_none() && self.effort.is_none() && self.subagent_models.is_none() {
bail!("nothing to change: pass --model, --effort or --subagent-models");
}
Ok(Settings {
model: self.model.or(current.model),
effort: self.effort.or(current.effort),
subagent_models: self.subagent_models.or(current.subagent_models),
})
}
}
fn runtime() -> Result<tokio::runtime::Runtime> {
tokio::runtime::Builder::new_current_thread()
.enable_io()
.build()
.context("starting a tokio runtime for the controller request")
}
/// The agent's settings path. The name is checked first, so it cannot
/// reshape the URI.
fn member(agent: &str) -> Result<String> {
let agent = parse_ident(agent, "agent name")?;
Ok(format!("/api/agents/{agent}/inference"))
}
/// Run `swarmctl agent inference get`.
pub(crate) fn get(socket: &Path, agent: &str) -> Result<()> {
let uri = member(agent)?;
let current: Settings = runtime()?.block_on(send(
socket,
hyper::Method::GET,
&uri,
None::<&()>,
"inference-get",
))?;
print!("{}", render(&current));
Ok(())
}
/// Run `swarmctl agent inference set`: read the agent's settings, write the
/// flags over them, and store the result.
pub(crate) fn set(socket: &Path, agent: &str, flags: SetFlags) -> Result<()> {
let uri = member(agent)?;
let rt = runtime()?;
let current: Settings = rt.block_on(send(
socket,
hyper::Method::GET,
&uri,
None::<&()>,
"inference-get",
))?;
let wanted = flags.merge(current)?;
let stored: Settings = rt.block_on(send(
socket,
hyper::Method::PUT,
&uri,
Some(&wanted),
"inference-set",
))?;
print!("{}", render(&stored));
Ok(())
}
/// Run `swarmctl agent inference clear`.
pub(crate) fn clear(socket: &Path, agent: &str) -> Result<()> {
let uri = member(agent)?;
runtime()?.block_on(send::<(), ()>(
socket,
hyper::Method::DELETE,
&uri,
None,
"inference-clear",
))?;
println!("cleared; {agent} goes back to its own model and effort");
Ok(())
}
/// One line per field, naming the unset ones.
fn render(s: &Settings) -> String {
const UNSET: &str = "(not set; the agent's own)";
format!(
"model: {}\neffort: {}\nsubagent_models: {}\n",
s.model.as_deref().unwrap_or(UNSET),
s.effort.as_deref().unwrap_or(UNSET),
s.subagent_models
.as_ref()
.map_or_else(|| UNSET.to_owned(), |m| m.join(", "))
)
}
#[cfg(test)]
mod tests {
use super::*;
fn flags() -> SetFlags {
SetFlags {
model: None,
effort: None,
subagent_models: None,
}
}
/// A flag replaces its field and every other field keeps what is stored.
#[test]
fn set_merges_into_the_stored_value() {
let current: Settings =
serde_json::from_str(r#"{"model":"sonnet","subagent_models":["haiku"]}"#).unwrap();
let merged = SetFlags {
effort: Some("high".to_owned()),
..flags()
}
.merge(current)
.unwrap();
assert_eq!(
serde_json::to_string(&merged).unwrap(),
r#"{"model":"sonnet","effort":"high","subagent_models":["haiku"]}"#
);
let merged = SetFlags {
model: Some("opus".to_owned()),
..flags()
}
.merge(merged)
.unwrap();
assert_eq!(merged.model.as_deref(), Some("opus"));
assert_eq!(merged.effort.as_deref(), Some("high"));
}
#[test]
fn set_with_no_flag_is_refused() {
assert!(flags().merge(Settings::default()).is_err());
}
#[test]
fn an_empty_value_renders_every_field_as_unset() {
let row = render(&serde_json::from_str("{}").unwrap());
assert_eq!(row.lines().count(), 3);
assert!(
row.lines()
.all(|l| l.ends_with("(not set; the agent's own)")),
"{row}"
);
}
}

View file

@ -29,6 +29,7 @@
mod agent; mod agent;
mod forge; mod forge;
mod inference;
mod schedule; mod schedule;
mod users; mod users;
@ -210,6 +211,55 @@ enum AgentVerb {
/// Queues and returns, the same way `agent create` does — watch the /// Queues and returns, the same way `agent create` does — watch the
/// swarm UI's job view for the outcome. /// swarm UI's job view for the outcome.
MintForgeToken(AgentMintForgeTokenArgs), MintForgeToken(AgentMintForgeTokenArgs),
/// Show or change the model, effort and subagent models the swarm sets
/// for an agent.
///
/// A field the swarm sets wins over the agent's own value (from its
/// `/model` or `/effort` pick or its agent.nix) from its next turn on;
/// the agent reads its settings every ten seconds. A field the swarm
/// doesn't set, or clears, leaves the agent on its own value.
Inference {
#[command(subcommand)]
command: InferenceVerb,
},
}
#[derive(Subcommand)]
enum InferenceVerb {
/// Show what the swarm sets for an agent, naming the fields it doesn't
/// set.
Get(InferenceAgentArgs),
/// Set fields for an agent. Only the flags given change; the others keep
/// what's stored.
Set(InferenceSetArgs),
/// Clear every field for an agent, so it goes back to its own values.
Clear(InferenceAgentArgs),
}
#[derive(Args)]
struct InferenceAgentArgs {
/// Agent name.
agent: String,
#[command(flatten)]
socket: ControllerSocketArg,
}
#[derive(Args)]
struct InferenceSetArgs {
/// Agent name. A hive must declare it.
agent: String,
/// The model its turns run on, as claude's `--model` takes it.
#[arg(long, value_name = "MODEL")]
model: Option<String>,
/// The reasoning effort its turns run at.
#[arg(long, value_name = "LEVEL", value_parser = ["low", "medium", "high", "xhigh", "max"])]
effort: Option<String>,
/// The models it may start subagents on, comma-separated. Stored and
/// shown; the agent doesn't apply it yet.
#[arg(long, value_name = "MODELS", value_delimiter = ',')]
subagent_models: Option<Vec<String>>,
#[command(flatten)]
socket: ControllerSocketArg,
} }
#[derive(Args)] #[derive(Args)]
@ -496,6 +546,9 @@ fn main() -> Result<()> {
let socket = path_from(args.controller_socket, "SWARM_CONTROLLER_SOCKET")?; let socket = path_from(args.controller_socket, "SWARM_CONTROLLER_SOCKET")?;
agent::mint_forge_token(&socket, &args.name) agent::mint_forge_token(&socket, &args.name)
} }
Verb::Agent {
command: AgentVerb::Inference { command },
} => inference(command),
// Same socket-resolution reasoning as `Create` above. // Same socket-resolution reasoning as `Create` above.
Verb::Forge { Verb::Forge {
command: ForgeVerb::MakeAdmin(args), command: ForgeVerb::MakeAdmin(args),
@ -544,6 +597,22 @@ fn main() -> Result<()> {
} }
} }
fn inference(command: InferenceVerb) -> Result<()> {
match command {
InferenceVerb::Get(a) => inference::get(&a.socket.resolve()?, &a.agent),
InferenceVerb::Set(a) => inference::set(
&a.socket.resolve()?,
&a.agent,
inference::SetFlags {
model: a.model,
effort: a.effort,
subagent_models: a.subagent_models,
},
),
InferenceVerb::Clear(a) => inference::clear(&a.socket.resolve()?, &a.agent),
}
}
fn schedule(command: ScheduleVerb) -> Result<()> { fn schedule(command: ScheduleVerb) -> Result<()> {
match command { match command {
ScheduleVerb::Add(a) => schedule::add( ScheduleVerb::Add(a) => schedule::add(