Watch
0
0
Fork
You've already forked hyperhive
0

hive-c0re: hive warnings reach the operator as metrics and alerts, not a banner

The dashboard's top-of-page warning banner is gone. The warning registry
stays, because /health/ready, the swarm status view and the OTEL gauges
read it. What it feeds is now alerted on in Grafana:

- hyperhive.hive.warning{kind,level}: one series per raised warning, so an
  alert names the cause. Raising a warning also logs one `hive warning
  raised` line with the message.
- hyperhive.hive.nix_store.utilization: the used fraction of the nix
  store's filesystem, probed in the gauge callback.
- Alerts: hh-hive-degraded on the existing degraded gauge, hh-hive-warning
  per kind (disk excluded), hh-host-disk-high at 85% and
  hh-host-disk-critical at 95%.
- pending_logins is now a registry warning, refreshed from the container
  rescan, so it reaches the per-kind gauge.

Frontend: server_warnings leaves the state snapshot, and the dashboard,
flow and stats pages keep only the sticky chrome wrapper. The shared
hive-warn and WarnBanner components stay, since other views use them.

A module-eval case checks each alert's metric name against the exporter's
source, so a rename on either side fails the build.
This commit is contained in:
iris 2026-10-21 17:52:53 +02:00
commit 4dbcf6fa80
35 changed files with 468 additions and 322 deletions

View file

@ -102,12 +102,6 @@ pub(super) struct StateSnapshot {
/// var, set from `services.hyperhive.swarm.name`. `None` when
/// unset — chrome omits the swarm segment of the breadcrumb.
swarm_name: Option<String>,
/// Server-level warnings for the dashboard's top-of-page banner
/// (currently host disk-pressure; more producers can be added
/// backend-side). Empty when all clear. Built by
/// `host_stats::server_warnings`; the frontend renders this list
/// generically, so new warning kinds need no frontend change.
server_warnings: Vec<crate::host_stats::ServerWarning>,
}
#[derive(Serialize)]
@ -193,15 +187,6 @@ pub(super) async fn api_state(
let tombstones = build_tombstone_views(&containers);
let port_conflicts = build_port_conflicts(&containers);
// Banner warnings: host probes (disk) + agent-state (pending logins).
// Built before the response struct because the agent-state producer
// borrows `containers`, which moves in below.
let server_warnings = {
let mut w = crate::host_stats::server_warnings();
w.extend(crate::host_stats::agent_state_warnings(&containers));
w
};
axum::Json(StateSnapshot {
seq,
hostname,
@ -233,7 +218,6 @@ pub(super) async fn api_state(
swarm_name: std::env::var("HYPERHIVE_SWARM_NAME")
.ok()
.filter(|s| !s.is_empty()),
server_warnings,
})
}