Watch
0
0
Fork
You've already forked hyperhive
0

hive-agent: export ACP-reported cost and context fill over OTLP

An ACP agent's `usage_update` carries `cost.{amount,currency}`, the
session's running total (opencode sums every assistant message in the
session). hive-runtime now reads it and turns the running total into
what each report added: a new session counts from zero, a session loaded
into a freshly started agent only baselines on its first report, and a
falling total adds nothing. The spend is held on the runtime until
`Runtime::take_reported_cost` drains it; claude's runtime reports none,
since the claude binary already exports `claude_code.cost.usage`.

hive-agent's existing turn-metrics meter records three new instruments:

- `hyperhive.agent.cost.usage` (counter, `model` + `currency`), ACP only;
- `hyperhive.agent.context.used` / `.size` (gauges, no attributes), for
  every backend: the two numbers the web UI's ctx% divides.

The `hyperhive · agents` dashboard gets ACP cost panels on its cost tab
and a context-fill panel on its health tab.

Refs #4845
This commit is contained in:
atlas 2026-09-30 22:23:01 +02:00
commit c2bdf30e05
8 changed files with 605 additions and 26 deletions

View file

@ -355,14 +355,16 @@ When OTEL is enabled, the harness itself (`hive-agent`) exports one small set
of metrics per claude turn, recorded the moment the turn ends (not polled). of metrics per claude turn, recorded the moment the turn ends (not polled).
These are deliberately the fields Claude Code's own built-in export (see These are deliberately the fields Claude Code's own built-in export (see
above) can't know about — the harness's own wall-clock timing, what woke the above) can't know about — the harness's own wall-clock timing, what woke the
turn, its own outcome classification, the loose-ends backlog, and session turn, its own outcome classification, the loose-ends backlog, session
boundaries. Token usage, cost, and tool-call counts are **not** duplicated boundaries, and how full the context window is. Token usage and tool-call
here; that's already covered by Claude's own export. counts are **not** duplicated here; that's already covered by Claude's own
export. Cost is here only for agents on an ACP runtime, which have no export
of their own — see [ACP-reported cost](#acp-reported-cost).
<!-- vale write-good.Passive = YES --> <!-- vale write-good.Passive = YES -->
| Metric | Unit | Kind | Attributes | | Metric | Unit | Kind | Attributes |
| ---------------------------------------- | ---- | --------- | ------------------------------------------------------------------------------------------------------------------ | | ---------------------------------------- | --------- | --------- | ------------------------------------------------------------------------------------------------------------------ |
| `hyperhive.agent.turn.duration` | `ms` | histogram | `wake_from`, `result_kind`, `model` | | `hyperhive.agent.turn.duration` | `ms` | histogram | `wake_from`, `result_kind`, `model` |
| `hyperhive.agent.turn.count` | — | counter | `wake_from`, `result_kind`, `model` | | `hyperhive.agent.turn.count` | — | counter | `wake_from`, `result_kind`, `model` |
| `hyperhive.agent.session.count` | — | counter | `model` (incremented once per fresh, non-`--continue`'d session) | | `hyperhive.agent.session.count` | — | counter | `model` (incremented once per fresh, non-`--continue`'d session) |
@ -371,6 +373,9 @@ here; that's already covered by Claude's own export.
| `hyperhive.agent.claude_md.lines` | — | gauge | none — recorded from the `CLAUDE.md`-size watch's own ~15-minute tick, **not** per turn like the rows above | | `hyperhive.agent.claude_md.lines` | — | gauge | none — recorded from the `CLAUDE.md`-size watch's own ~15-minute tick, **not** per turn like the rows above |
| `hyperhive.agent.claude_usage.percent` | `%` | gauge | `window` (`five_hour`, `seven_day`, … as the usage endpoint names them) — polled every 5 minutes, **not** per turn | | `hyperhive.agent.claude_usage.percent` | `%` | gauge | `window` (`five_hour`, `seven_day`, … as the usage endpoint names them) — polled every 5 minutes, **not** per turn |
| `hyperhive.agent.claude_usage.resets_at` | `s` | gauge | `window` — unix seconds at which that window resets; same 5-minute poll | | `hyperhive.agent.claude_usage.resets_at` | `s` | gauge | `window` — unix seconds at which that window resets; same 5-minute poll |
| `hyperhive.agent.context.used` | `{token}` | gauge | none — tokens in the context window at turn end, the numerator of the web UI's ctx% |
| `hyperhive.agent.context.size` | `{token}` | gauge | none — the context window those tokens fill, ctx%'s denominator |
| `hyperhive.agent.cost.usage` | — | counter | `model`, `currency` — ACP agents only, see below |
For the two `claude_usage` gauges the harness polls the Claude subscription For the two `claude_usage` gauges the harness polls the Claude subscription
usage endpoint (`GET /api/oauth/usage`, the one behind claude's own `/usage`) usage endpoint (`GET /api/oauth/usage`, the one behind claude's own `/usage`)
@ -381,6 +386,12 @@ session (API-key backends, not yet logged in) or an expired token skips the
poll, so a gauge keeps its last value until the next successful poll — a poll, so a gauge keeps its last value until the next successful poll — a
`resets_at` in the past means the paired `percent` is stale. `resets_at` in the past means the paired `percent` is stale.
The two `context` gauges hold the two numbers the web UI divides for its ctx%:
the last turn's context tokens (input, cache read and cache creation), and the
API-reported window, else the model's default. A turn that parsed no usage
leaves both where the previous turn put them. The dashboard's **Context
window used by agent** panel (health tab) divides one by the other.
Resource attributes (`service.name`, `agent`, `hive`, `swarm`) come from the Resource attributes (`service.name`, `agent`, `hive`, `swarm`) come from the
same container-wide `OTEL_RESOURCE_ATTRIBUTES` as everything else in this same container-wide `OTEL_RESOURCE_ATTRIBUTES` as everything else in this
section — nothing extra to configure. Cadence follows section — nothing extra to configure. Cadence follows
@ -389,6 +400,32 @@ section — nothing extra to configure. Cadence follows
the harness flushes the batched points to the collector, not how often it the harness flushes the batched points to the collector, not how often it
records them (every turn, always). records them (every turn, always).
### ACP-reported cost
An ACP agent reports what its session has cost so far in the `cost` field of
its `usage_update` notifications — a running total, not a per-turn figure
(opencode sums every assistant message in the session). The harness counts
what each report adds to the last one and exports that as
`hyperhive.agent.cost.usage`, so `increase()` over it gives the money spent
in a range. It follows a few rules:
- A new session's first report counts in full, since the session started at
zero.
- A session loaded into a freshly started agent (after a harness restart or
an agent crash) has an unknown total until it reports one, so that first
report only sets the baseline — the turn it covers goes uncounted.
- A total lower than the previous one means the agent dropped part of the
session's history; it adds nothing and becomes the new baseline.
- Cost from a compaction the harness runs between turns counts toward the
next turn.
The value is in whatever `currency` the agent names; opencode always sends
`USD`, and the `hyperhive · agents` dashboard's ACP cost panels (cost tab)
filter on it. A claude agent never records this metric: claude's own export
already carries its cost as `claude_code.cost.usage`, and keeping the two
apart means no turn is ever counted twice. Add both names together for a
whole-swarm spend figure.
## Hive-scoped metrics (hive-c0re) ## Hive-scoped metrics (hive-c0re)
<!-- vale write-good.Passive = NO --> <!-- vale write-good.Passive = NO -->

View file

@ -1001,6 +1001,7 @@ async fn handle_turn<S: Surface>(
let ended_at = chrono::Utc::now().timestamp(); let ended_at = chrono::Utc::now().timestamp();
let duration_ms = i64::try_from(started_instant.elapsed().as_millis()).unwrap_or(i64::MAX); let duration_ms = i64::try_from(started_instant.elapsed().as_millis()).unwrap_or(i64::MAX);
let (open_threads, open_reminders) = S::post_turn_counts(socket).await; let (open_threads, open_reminders) = S::post_turn_counts(socket).await;
let context_window = bus.effective_context_window(&model_at_start);
let row = serve_common::build_row(serve_common::TurnRowArgs { let row = serve_common::build_row(serve_common::TurnRowArgs {
started_at, started_at,
ended_at, ended_at,
@ -1014,10 +1015,17 @@ async fn handle_turn<S: Surface>(
open_reminders_count: open_reminders, open_reminders_count: open_reminders,
}); });
// Harness-only OTEL metrics (duration/wake_from/result_kind/loose-ends/ // Harness-only OTEL metrics (duration/wake_from/result_kind/loose-ends/
// session boundaries) — independent of the sqlite sink below, and a // session boundaries/context fill) — independent of the sqlite sink
// cheap no-op when OTEL isn't configured. See `otel_turn_metrics`'s // below, and a cheap no-op when OTEL isn't configured. See
// module doc for why token/cost/tool-count are deliberately not here. // `otel_turn_metrics`'s module doc for why token/tool-count are
// deliberately not here, and why cost is only ever an ACP agent's.
otel_turn_metrics::record(&row, fresh_session); otel_turn_metrics::record(&row, fresh_session);
if let Some(ctx) = bus.last_ctx_usage() {
otel_turn_metrics::record_context(ctx.context_tokens(), context_window);
}
for cost in hive_runtime::Runtime::take_reported_cost(session) {
otel_turn_metrics::record_reported_cost(&row.model, &cost);
}
if let Some(stats) = stats { if let Some(stats) = stats {
stats.record(&row); stats.record(&row);
} }

View file

@ -6,8 +6,8 @@
//! only the harness-only concepts: wall-clock turn duration as *this harness* //! only the harness-only concepts: wall-clock turn duration as *this harness*
//! measures it (not Claude's own per-request latency), what woke the turn, //! measures it (not Claude's own per-request latency), what woke the turn,
//! the harness's own outcome classification, the loose-ends backlog at turn //! the harness's own outcome classification, the loose-ends backlog at turn
//! end, and session boundaries (fresh vs. `--continue`'d). Scoped this way //! end, session boundaries (fresh vs. `--continue`'d), context-window fill,
//! per the tracker discussion on the "emit agent stats as OTEL metrics" issue. //! and the cost an ACP agent reports — claude's runtime reports none.
//! //!
//! Synchronous instruments, not observable ones: unlike hive-c0re's //! Synchronous instruments, not observable ones: unlike hive-c0re's
//! container-resource gauges (`hive-c0re/src/stats/otel_metrics.rs`), which //! container-resource gauges (`hive-c0re/src/stats/otel_metrics.rs`), which
@ -62,6 +62,11 @@ struct Instruments {
/// `window`. /// `window`.
claude_usage_percent: Gauge<f64>, claude_usage_percent: Gauge<f64>,
claude_usage_resets_at: Gauge<u64>, claude_usage_resets_at: Gauge<u64>,
context_used: Gauge<u64>,
context_size: Gauge<u64>,
/// Keyed by `model` and `currency`: an ACP agent reports cost in a
/// currency of its choosing.
reported_cost: Counter<f64>,
} }
/// Lazily built on the first call to [`record`]. `None` when OTEL isn't /// Lazily built on the first call to [`record`]. `None` when OTEL isn't
@ -129,6 +134,30 @@ pub fn record_claude_usage(window: &str, percent: f64, resets_at: Option<u64>) {
} }
} }
/// Record the context window's fill at turn end: `used` tokens of `size`,
/// the two numbers the web UI's ctx% divides. No-op when OTEL isn't
/// configured, same as [`record`].
pub fn record_context(used: u64, size: u64) {
let Some(inst) = INSTRUMENTS.get_or_init(build).as_ref() else {
return;
};
inst.context_used.record(used, &[]);
inst.context_size.record(size, &[]);
}
/// Add what an ACP agent reported spending since the last call. No-op when
/// OTEL isn't configured, same as [`record`].
pub fn record_reported_cost(model: &str, cost: &hive_runtime::ReportedCost) {
let Some(inst) = INSTRUMENTS.get_or_init(build).as_ref() else {
return;
};
let attrs = [
KeyValue::new("model", model.to_owned()),
KeyValue::new("currency", cost.currency.clone()),
];
inst.reported_cost.add(cost.amount, &attrs);
}
fn build() -> Option<Instruments> { fn build() -> Option<Instruments> {
if !enabled() { if !enabled() {
tracing::debug!("otel turn-metrics: no endpoint configured, exporter disabled"); tracing::debug!("otel turn-metrics: no endpoint configured, exporter disabled");
@ -161,6 +190,15 @@ fn build() -> Option<Instruments> {
.u64_gauge("hyperhive.agent.claude_usage.resets_at") .u64_gauge("hyperhive.agent.claude_usage.resets_at")
.with_unit("s") .with_unit("s")
.build(), .build(),
context_used: meter
.u64_gauge("hyperhive.agent.context.used")
.with_unit("{token}")
.build(),
context_size: meter
.u64_gauge("hyperhive.agent.context.size")
.with_unit("{token}")
.build(),
reported_cost: meter.f64_counter("hyperhive.agent.cost.usage").build(),
_provider: provider, _provider: provider,
}) })
} }

View file

@ -8,7 +8,7 @@
mod rpc; mod rpc;
mod stream; mod stream;
use std::collections::{HashMap, VecDeque}; use std::collections::{BTreeMap, HashMap, VecDeque};
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::pin::Pin; use std::pin::Pin;
use std::sync::atomic::{AtomicBool, Ordering}; use std::sync::atomic::{AtomicBool, Ordering};
@ -229,6 +229,67 @@ impl Choices {
} }
} }
/// An amount of money an agent reported, in an ISO 4217 `currency`.
#[derive(Debug, Clone, PartialEq)]
pub struct ReportedCost {
pub amount: f64,
pub currency: String,
}
/// What the agent has reported spending, shared between the runtime and its
/// [`Live`] agent so an amount outlives a respawn until it is taken.
#[derive(Clone, Default)]
struct Spend(Arc<std::sync::Mutex<Spent>>);
#[derive(Default)]
struct Spent {
/// The attached session's running total at its last report; `None` until
/// it reports one.
total: Option<ReportedCost>,
/// The attached session is new, so its total started at zero.
fresh: bool,
/// Spent since the last [`Spend::take`], per currency.
untaken: BTreeMap<String, f64>,
}
impl Spend {
/// Start following a newly attached session. A loaded session's total
/// before its first report is unknown, so that report only sets the
/// baseline.
fn attach(&self, fresh: bool) {
let mut spent = self.lock();
spent.total = None;
spent.fresh = fresh;
}
/// Fold in a running total the attached session reported. A total lower
/// than the last one means the agent dropped part of the session's
/// history, so it adds nothing.
fn observe(&self, total: ReportedCost) {
let mut spent = self.lock();
let added = match &spent.total {
Some(last) if last.currency == total.currency => total.amount - last.amount,
None if spent.fresh => total.amount,
_ => 0.0,
};
if added > 0.0 {
*spent.untaken.entry(total.currency.clone()).or_default() += added;
}
spent.total = Some(total);
}
fn take(&self) -> Vec<ReportedCost> {
std::mem::take(&mut self.lock().untaken)
.into_iter()
.map(|(currency, amount)| ReportedCost { amount, currency })
.collect()
}
fn lock(&self) -> std::sync::MutexGuard<'_, Spent> {
self.0.lock().unwrap_or_else(PoisonError::into_inner)
}
}
fn find<'a>( fn find<'a>(
options: &'a [stream::ConfigOption], options: &'a [stream::ConfigOption],
category: &str, category: &str,
@ -291,6 +352,7 @@ pub struct AcpRuntime<P: CompactionPolicy> {
cancel_grace: Duration, cancel_grace: Duration,
compact_idle: Duration, compact_idle: Duration,
choices: Choices, choices: Choices,
spend: Spend,
} }
/// The running agent process and the session loaded into it. /// The running agent process and the session loaded into it.
@ -305,6 +367,7 @@ struct Live {
commands: Option<Vec<String>>, commands: Option<Vec<String>>,
/// The config options `loaded` last reported. /// The config options `loaded` last reported.
choices: Choices, choices: Choices,
spend: Spend,
} }
impl Live { impl Live {
@ -340,6 +403,7 @@ impl<P: CompactionPolicy> AcpRuntime<P> {
cancel_grace: CANCEL_GRACE, cancel_grace: CANCEL_GRACE,
compact_idle: COMPACT_IDLE, compact_idle: COMPACT_IDLE,
choices: Choices::default(), choices: Choices::default(),
spend: Spend::default(),
} }
} }
@ -381,6 +445,7 @@ impl<P: CompactionPolicy> AcpRuntime<P> {
model: None, model: None,
commands: None, commands: None,
choices: self.choices.clone(), choices: self.choices.clone(),
spend: self.spend.clone(),
}) })
} }
@ -414,6 +479,7 @@ impl<P: CompactionPolicy> AcpRuntime<P> {
live.model = stream::session_model(&response); live.model = stream::session_model(&response);
live.offer(stream::config_options(&response).unwrap_or_default()); live.offer(stream::config_options(&response).unwrap_or_default());
live.loaded = Some(id.clone()); live.loaded = Some(id.clone());
live.spend.attach(false);
live.commands = None; live.commands = None;
discard_stale(live)?; discard_stale(live)?;
return Ok((id, new)); return Ok((id, new));
@ -436,6 +502,7 @@ impl<P: CompactionPolicy> AcpRuntime<P> {
live.model = stream::session_model(&response); live.model = stream::session_model(&response);
live.offer(stream::config_options(&response).unwrap_or_default()); live.offer(stream::config_options(&response).unwrap_or_default());
live.loaded = Some(id.clone()); live.loaded = Some(id.clone());
live.spend.attach(true);
live.commands = None; live.commands = None;
write_id(&self.pending_file(), &id)?; write_id(&self.pending_file(), &id)?;
Ok((id, true)) Ok((id, true))
@ -709,6 +776,10 @@ impl<P: CompactionPolicy> Runtime for AcpRuntime<P> {
Some(self.choices.clone()) Some(self.choices.clone())
} }
fn take_reported_cost(&self) -> Vec<ReportedCost> {
self.spend.take()
}
/// Moves the session file aside; the agent keeps its own copy of the /// Moves the session file aside; the agent keeps its own copy of the
/// session, so nothing is lost. A new session whose first prompt was never /// session, so nothing is lost. A new session whose first prompt was never
/// answered is dropped too. /// answered is dropped too.
@ -761,8 +832,11 @@ fn deliver(
} }
/// Keep the commands and config options an `update` for the loaded session /// Keep the commands and config options an `update` for the loaded session
/// advertises. /// advertises, and the cost it reports.
fn keep_advertised(agent: &mut Live, update: &Value) { fn keep_advertised(agent: &mut Live, update: &Value) {
if let Some(cost) = stream::reported_cost(update) {
agent.spend.observe(cost);
}
if let Some(advertised) = stream::advertised_commands(update) { if let Some(advertised) = stream::advertised_commands(update) {
agent.commands = Some(advertised); agent.commands = Some(advertised);
} }
@ -775,7 +849,7 @@ fn keep_advertised(agent: &mut Live, update: &Value) {
/// replays (the caller already has it), and anything that arrived after the /// replays (the caller already has it), and anything that arrived after the
/// previous turn settled, which must not be shown as part of the next one. /// previous turn settled, which must not be shown as part of the next one.
/// The commands and config options the agent advertises for the loaded /// The commands and config options the agent advertises for the loaded
/// session are kept. /// session, and the cost it reports, are kept.
fn discard_stale(live: &mut Live) -> std::result::Result<(), AcpError> { fn discard_stale(live: &mut Live) -> std::result::Result<(), AcpError> {
while let Ok(incoming) = live.conn.incoming.try_recv() { while let Ok(incoming) = live.conn.incoming.try_recv() {
between_turns(live, incoming)?; between_turns(live, incoming)?;
@ -941,7 +1015,7 @@ mod tests {
use hive_claude::{Config, NoopSink, PercentPolicy, Sink}; use hive_claude::{Config, NoopSink, PercentPolicy, Sink};
use super::{AcpError, AcpRuntime, Choice, SessionChoices, TurnKind}; use super::{AcpError, AcpRuntime, Choice, ReportedCost, SessionChoices, Spend, TurnKind};
use crate::{AcpCommand, Error, Runtime}; use crate::{AcpCommand, Error, Runtime};
/// An ACP agent in plain `sh`. It answers `initialize`, numbers its /// An ACP agent in plain `sh`. It answers `initialize`, numbers its
@ -971,7 +1045,9 @@ mod tests {
/// and `m/plain`, and effort levels `low` (the default) and `high` on /// and `m/plain`, and effort levels `low` (the default) and `high` on
/// `m/think` only; it appends each `session/set_config_option` to /// `m/think` only; it appends each `session/set_config_option` to
/// `<log>.sets` as `id=value`. With `REFUSE` set, it answers the first /// `<log>.sets` as `id=value`. With `REFUSE` set, it answers the first
/// `session/set_config_option` to that value with an error. /// `session/set_config_option` to that value with an error. With `COST`
/// set, each prompt adds that many dollars to its session's total (from zero,
/// kept in `<log>.cost.<session>` across restarts) and reports it.
const AGENT: &str = r#" const AGENT: &str = r#"
n=0 prompt= model=m/think effort=low n=0 prompt= model=m/think effort=low
printf 'start\n' >> "$1.methods" printf 'start\n' >> "$1.methods"
@ -1007,6 +1083,7 @@ while IFS= read -r line; do
printf '{"jsonrpc":"2.0","id":%s,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":true,"mcpCapabilities":{"http":true}}}}\n' "$id" ;; printf '{"jsonrpc":"2.0","id":%s,"result":{"protocolVersion":1,"agentCapabilities":{"loadSession":true,"mcpCapabilities":{"http":true}}}}\n' "$id" ;;
session/new) session/new)
n=$((n+1)) n=$((n+1))
rm -f "$1.cost.s$n"
printf '{"jsonrpc":"2.0","id":%s,"result":{"sessionId":"s%s"%s}}\n' "$id" "$n" "$(options)" printf '{"jsonrpc":"2.0","id":%s,"result":{"sessionId":"s%s"%s}}\n' "$id" "$n" "$(options)"
advertise "s$n" ;; advertise "s$n" ;;
session/load) session/load)
@ -1026,6 +1103,11 @@ while IFS= read -r line; do
session/prompt) session/prompt)
printf '%s\n' "$line" >> "$1" printf '%s\n' "$line" >> "$1"
[ -n "$USED" ] && [ "$sid" = s1 ] && update "$sid" "{\"sessionUpdate\":\"usage_update\",\"used\":$USED,\"size\":1000}" [ -n "$USED" ] && [ "$sid" = s1 ] && update "$sid" "{\"sessionUpdate\":\"usage_update\",\"used\":$USED,\"size\":1000}"
if [ -n "$COST" ]; then
total=$(( $(cat "$1.cost.$sid" 2>/dev/null || echo 0) + COST ))
printf '%s' "$total" > "$1.cost.$sid"
update "$sid" "{\"sessionUpdate\":\"usage_update\",\"cost\":{\"amount\":$total,\"currency\":\"USD\"}}"
fi
case $line in case $line in
*'"text":"/compact"'*) [ -n "$STALL_COMPACT" ] && prompt=$id && continue ;; *'"text":"/compact"'*) [ -n "$STALL_COMPACT" ] && prompt=$id && continue ;;
esac esac
@ -1639,4 +1721,93 @@ done
Some(choice("m/think", &["m/think", "m/plain"])) Some(choice("m/think", &["m/think", "m/plain"]))
); );
} }
fn usd(amount: f64) -> ReportedCost {
ReportedCost {
amount,
currency: "USD".into(),
}
}
#[test]
fn a_new_session_s_first_report_is_all_spent() {
let spend = Spend::default();
spend.attach(true);
spend.observe(usd(0.5));
spend.observe(usd(1.25));
assert_eq!(spend.take(), [usd(1.25)]);
assert_eq!(spend.take(), []);
spend.observe(usd(2.0));
assert_eq!(spend.take(), [usd(0.75)]);
}
#[test]
fn a_loaded_session_s_first_report_only_sets_the_baseline() {
let spend = Spend::default();
spend.attach(false);
spend.observe(usd(10.0));
assert_eq!(spend.take(), []);
spend.observe(usd(10.5));
assert_eq!(spend.take(), [usd(0.5)]);
}
#[test]
fn a_falling_total_adds_nothing_and_becomes_the_baseline() {
let spend = Spend::default();
spend.attach(true);
spend.observe(usd(3.0));
spend.observe(usd(1.0));
spend.observe(usd(1.5));
assert_eq!(spend.take(), [usd(3.5)]);
}
#[test]
fn untaken_spend_survives_attaching_another_session() {
let spend = Spend::default();
spend.attach(true);
spend.observe(usd(1.0));
spend.attach(true);
spend.observe(usd(0.25));
assert_eq!(spend.take(), [usd(1.25)]);
}
#[test]
fn a_changed_currency_only_sets_a_new_baseline() {
let spend = Spend::default();
spend.attach(true);
spend.observe(usd(1.0));
let eur = |amount| ReportedCost {
amount,
currency: "EUR".into(),
};
spend.observe(eur(0.75));
spend.observe(eur(1.25));
assert_eq!(spend.take(), [eur(0.5), usd(1.0)]);
}
#[tokio::test]
async fn reported_cost_is_what_the_session_added_since_last_taken() {
let dir = tempfile::tempdir().unwrap();
let config = config(dir.path());
let env = [("COST", "2")];
let before = agent(dir.path(), "ok", &env, PercentPolicy::default());
before.run(&config, "one", &NoopSink).await.unwrap();
before.run(&config, "two", &NoopSink).await.unwrap();
assert_eq!(before.take_reported_cost(), [usd(4.0)]);
assert_eq!(before.take_reported_cost(), []);
drop(before);
let after = agent(dir.path(), "ok", &env, PercentPolicy::default());
after.run(&config, "three", &NoopSink).await.unwrap();
assert_eq!(after.take_reported_cost(), []);
after.run(&config, "four", &NoopSink).await.unwrap();
assert_eq!(after.take_reported_cost(), [usd(2.0)]);
after.archive().unwrap();
after.run(&config, "five", &NoopSink).await.unwrap();
assert_eq!(after.take_reported_cost(), [usd(2.0)]);
assert_eq!(count(dir.path(), "session/load"), 1);
assert_eq!(count(dir.path(), "session/new"), 2);
}
} }

View file

@ -7,6 +7,8 @@ use std::collections::HashMap;
use hive_claude::{Telemetry, TokenUsage}; use hive_claude::{Telemetry, TokenUsage};
use serde_json::{Map, Value, json}; use serde_json::{Map, Value, json};
use super::ReportedCost;
/// Turns one prompt's `session/update` notifications into claude /// Turns one prompt's `session/update` notifications into claude
/// `stream-json` events. /// `stream-json` events.
/// ///
@ -258,6 +260,19 @@ pub(super) fn config_options(value: &Value) -> Option<Vec<ConfigOption>> {
) )
} }
/// The running total cost a `usage_update` reports for its session, or
/// `None` for any other `update` and for one that reports no cost.
pub(super) fn reported_cost(update: &Value) -> Option<ReportedCost> {
if update.get("sessionUpdate").and_then(Value::as_str) != Some("usage_update") {
return None;
}
let cost = update.get("cost")?;
Some(ReportedCost {
amount: cost.get("amount")?.as_f64()?,
currency: cost.get("currency")?.as_str()?.to_owned(),
})
}
/// The options a `config_option_update` sets, or `None` for any other /// The options a `config_option_update` sets, or `None` for any other
/// `update`. Each replaces the session's previous options. /// `update`. Each replaces the session's previous options.
pub(super) fn updated_config_options(update: &Value) -> Option<Vec<ConfigOption>> { pub(super) fn updated_config_options(update: &Value) -> Option<Vec<ConfigOption>> {
@ -402,7 +417,8 @@ fn tool_result(id: &str, output: &str, is_error: bool) -> Value {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::{ use super::{
ConfigOption, StreamMapper, canonical_tool_name, config_options, mcp_servers, session_model, ConfigOption, ReportedCost, StreamMapper, canonical_tool_name, config_options, mcp_servers,
reported_cost, session_model,
}; };
use serde_json::{Value, json}; use serde_json::{Value, json};
@ -537,6 +553,33 @@ mod tests {
assert_eq!(t.model.as_deref(), Some("provider/model")); assert_eq!(t.model.as_deref(), Some("provider/model"));
} }
#[test]
fn a_usage_update_reports_its_cost_when_it_carries_one() {
assert_eq!(
reported_cost(&json!({ "sessionUpdate": "usage_update", "used": 5000,
"size": 262_144, "cost": { "amount": 0.045, "currency": "USD" } })),
Some(ReportedCost {
amount: 0.045,
currency: "USD".into()
})
);
assert_eq!(
reported_cost(&json!({ "sessionUpdate": "usage_update", "used": 5000,
"size": 262_144 })),
None
);
assert_eq!(
reported_cost(&json!({ "sessionUpdate": "usage_update",
"cost": { "amount": 0.045 } })),
None
);
assert_eq!(
reported_cost(&json!({ "sessionUpdate": "plan",
"cost": { "amount": 0.045, "currency": "USD" } })),
None
);
}
#[test] #[test]
fn updates_with_nothing_to_show_emit_nothing() { fn updates_with_nothing_to_show_emit_nothing() {
let out = feed( let out = feed(

View file

@ -4,7 +4,7 @@ use std::path::PathBuf;
use hive_claude::{CompactionPolicy, Config, InfiniteSession, Progress, SessionStore, Sink}; use hive_claude::{CompactionPolicy, Config, InfiniteSession, Progress, SessionStore, Sink};
use crate::{Canceller, Choices, Result, Runtime}; use crate::{Canceller, Choices, ReportedCost, Result, Runtime};
/// `claude --print` turns on a titled [`InfiniteSession`], which owns /// `claude --print` turns on a titled [`InfiniteSession`], which owns
/// resume-or-create and compaction. /// resume-or-create and compaction.
@ -47,4 +47,8 @@ impl<P: CompactionPolicy> Runtime for ClaudeRuntime<P> {
fn choices(&self) -> Option<Choices> { fn choices(&self) -> Option<Choices> {
None None
} }
fn take_reported_cost(&self) -> Vec<ReportedCost> {
Vec::new()
}
} }

View file

@ -23,7 +23,7 @@ mod spec;
pub use acp::{ pub use acp::{
AcpError, AcpRuntime, Canceller, Choice, Choices, PermissionAsk, PermissionPolicy, AcpError, AcpRuntime, Canceller, Choice, Choices, PermissionAsk, PermissionPolicy,
SessionChoices, ReportedCost, SessionChoices,
}; };
pub use claude::ClaudeRuntime; pub use claude::ClaudeRuntime;
pub use hive_claude::{ pub use hive_claude::{
@ -65,6 +65,11 @@ pub trait Runtime {
/// pick. `None` for a runtime whose session offers none to read: claude's /// pick. `None` for a runtime whose session offers none to read: claude's
/// `--model` and `--effort` are passed through unchecked. /// `--model` and `--effort` are passed through unchecked.
fn choices(&self) -> Option<Choices>; fn choices(&self) -> Option<Choices>;
/// What the agent reported spending since the last call, one entry per
/// currency. Empty for a runtime whose agent reports no cost: claude
/// exports its own.
fn take_reported_cost(&self) -> Vec<ReportedCost>;
} }
/// The runtime an agent was configured with, chosen at startup from a /// The runtime an agent was configured with, chosen at startup from a
@ -109,6 +114,13 @@ impl<P: CompactionPolicy> Runtime for AgentRuntime<P> {
Self::Acp(r) => r.choices(), Self::Acp(r) => r.choices(),
} }
} }
fn take_reported_cost(&self) -> Vec<ReportedCost> {
match self {
Self::Claude(r) => r.take_reported_cost(),
Self::Acp(r) => r.take_reported_cost(),
}
}
} }
/// Why a runtime operation did not complete. /// Why a runtime operation did not complete.

View file

@ -2126,6 +2126,233 @@
} }
} }
} }
},
"panel-65": {
"kind": "Panel",
"spec": {
"id": 65,
"title": "ACP agent cost by agent",
"description": "USD an ACP agent reported spending in the selected range, busiest first.",
"links": [],
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "DataQuery",
"group": "prometheus",
"version": "v0",
"datasource": {
"name": "@datasourceUid@"
},
"spec": {
"expr": "sort_desc(sum by (agent) (increase({__name__=\"hyperhive.agent.cost.usage\", hive=~\"$hive\", agent=~\"$agent\", currency=\"USD\"}[$__range])))",
"instant": true,
"legendFormat": "{{agent}}"
}
},
"refId": "A",
"hidden": false
}
}
],
"transformations": [],
"queryOptions": {}
}
},
"vizConfig": {
"kind": "VizConfig",
"group": "bargauge",
"version": "",
"spec": {
"options": {
"displayMode": "basic",
"orientation": "horizontal",
"reduceOptions": {
"calcs": ["lastNotNull"],
"fields": "",
"values": false
},
"showUnfilled": true,
"valueMode": "text"
},
"fieldConfig": {
"defaults": {
"unit": "currencyUSD",
"decimals": 2,
"thresholds": {
"mode": "absolute",
"steps": [
{
"value": null,
"color": "purple"
}
]
},
"color": {
"mode": "fixed",
"fixedColor": "purple"
}
},
"overrides": []
}
}
}
}
},
"panel-66": {
"kind": "Panel",
"spec": {
"id": 66,
"title": "ACP agent cost rate by agent",
"description": "USD per hour an ACP agent reported spending.",
"links": [],
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "DataQuery",
"group": "prometheus",
"version": "v0",
"datasource": {
"name": "@datasourceUid@"
},
"spec": {
"expr": "sum by (agent) (rate({__name__=\"hyperhive.agent.cost.usage\", hive=~\"$hive\", agent=~\"$agent\", currency=\"USD\"}[$__rate_interval])) * 3600",
"legendFormat": "{{agent}}"
}
},
"refId": "A",
"hidden": false
}
}
],
"transformations": [],
"queryOptions": {
"interval": "10m"
}
}
},
"vizConfig": {
"kind": "VizConfig",
"group": "timeseries",
"version": "",
"spec": {
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"fieldConfig": {
"defaults": {
"unit": "currencyUSD",
"decimals": 2,
"color": {
"mode": "palette-classic"
},
"custom": {
"axisSoftMin": 0,
"drawStyle": "line",
"fillOpacity": 0,
"lineWidth": 1,
"showPoints": "never",
"spanNulls": false
}
},
"overrides": []
}
}
}
}
},
"panel-34": {
"kind": "Panel",
"spec": {
"id": 34,
"title": "Context window used by agent",
"description": "Share of each agent's context window filled at its last turn end.",
"links": [],
"data": {
"kind": "QueryGroup",
"spec": {
"queries": [
{
"kind": "PanelQuery",
"spec": {
"query": {
"kind": "DataQuery",
"group": "prometheus",
"version": "v0",
"datasource": {
"name": "@datasourceUid@"
},
"spec": {
"expr": "max by (agent) ({__name__=\"hyperhive.agent.context.used\", hive=~\"$hive\", agent=~\"$agent\"}) / max by (agent) ({__name__=\"hyperhive.agent.context.size\", hive=~\"$hive\", agent=~\"$agent\"}) * 100",
"legendFormat": "{{agent}}"
}
},
"refId": "A",
"hidden": false
}
}
],
"transformations": [],
"queryOptions": {}
}
},
"vizConfig": {
"kind": "VizConfig",
"group": "timeseries",
"version": "",
"spec": {
"options": {
"legend": {
"calcs": [],
"displayMode": "list",
"placement": "bottom",
"showLegend": true
},
"tooltip": {
"mode": "multi",
"sort": "desc"
}
},
"fieldConfig": {
"defaults": {
"unit": "percent",
"decimals": 0,
"color": {
"mode": "palette-classic"
},
"custom": {
"axisSoftMin": 0,
"drawStyle": "line",
"fillOpacity": 0,
"lineWidth": 1,
"showPoints": "never",
"spanNulls": false,
"axisSoftMax": 100
}
},
"overrides": []
}
}
}
}
} }
}, },
"layout": { "layout": {
@ -2373,6 +2600,32 @@
"name": "panel-64" "name": "panel-64"
} }
} }
},
{
"kind": "GridLayoutItem",
"spec": {
"x": 0,
"y": 30,
"width": 12,
"height": 8,
"element": {
"kind": "ElementReference",
"name": "panel-65"
}
}
},
{
"kind": "GridLayoutItem",
"spec": {
"x": 12,
"y": 30,
"width": 12,
"height": 8,
"element": {
"kind": "ElementReference",
"name": "panel-66"
}
}
} }
] ]
} }
@ -2438,6 +2691,19 @@
"name": "panel-33" "name": "panel-33"
} }
} }
},
{
"kind": "GridLayoutItem",
"spec": {
"x": 0,
"y": 23,
"width": 24,
"height": 8,
"element": {
"kind": "ElementReference",
"name": "panel-34"
}
}
} }
] ]
} }