hive-c0re: hive warnings reach the operator as metrics and alerts, not a banner
The dashboard's top-of-page warning banner is gone. The warning registry
stays, because /health/ready, the swarm status view and the OTEL gauges
read it. What it feeds is now alerted on in Grafana:
- hyperhive.hive.warning{kind,level}: one series per raised warning, so an
alert names the cause. Raising a warning also logs one `hive warning
raised` line with the message.
- hyperhive.hive.nix_store.utilization: the used fraction of the nix
store's filesystem, probed in the gauge callback.
- Alerts: hh-hive-degraded on the existing degraded gauge, hh-hive-warning
per kind (disk excluded), hh-host-disk-high at 85% and
hh-host-disk-critical at 95%.
- pending_logins is now a registry warning, refreshed from the container
rescan, so it reaches the per-kind gauge.
Frontend: server_warnings leaves the state snapshot, and the dashboard,
flow and stats pages keep only the sticky chrome wrapper. The shared
hive-warn and WarnBanner components stay, since other views use them.
A module-eval case checks each alert's metric name against the exporter's
source, so a rename on either side fails the build.
This commit is contained in:
parent
013bbe5adc
commit
4dbcf6fa80
35 changed files with 468 additions and 322 deletions
|
|
@ -102,12 +102,6 @@ pub(super) struct StateSnapshot {
|
|||
/// var, set from `services.hyperhive.swarm.name`. `None` when
|
||||
/// unset — chrome omits the swarm segment of the breadcrumb.
|
||||
swarm_name: Option<String>,
|
||||
/// Server-level warnings for the dashboard's top-of-page banner
|
||||
/// (currently host disk-pressure; more producers can be added
|
||||
/// backend-side). Empty when all clear. Built by
|
||||
/// `host_stats::server_warnings`; the frontend renders this list
|
||||
/// generically, so new warning kinds need no frontend change.
|
||||
server_warnings: Vec<crate::host_stats::ServerWarning>,
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
|
|
@ -193,15 +187,6 @@ pub(super) async fn api_state(
|
|||
let tombstones = build_tombstone_views(&containers);
|
||||
let port_conflicts = build_port_conflicts(&containers);
|
||||
|
||||
// Banner warnings: host probes (disk) + agent-state (pending logins).
|
||||
// Built before the response struct because the agent-state producer
|
||||
// borrows `containers`, which moves in below.
|
||||
let server_warnings = {
|
||||
let mut w = crate::host_stats::server_warnings();
|
||||
w.extend(crate::host_stats::agent_state_warnings(&containers));
|
||||
w
|
||||
};
|
||||
|
||||
axum::Json(StateSnapshot {
|
||||
seq,
|
||||
hostname,
|
||||
|
|
@ -233,7 +218,6 @@ pub(super) async fn api_state(
|
|||
swarm_name: std::env::var("HYPERHIVE_SWARM_NAME")
|
||||
.ok()
|
||||
.filter(|s| !s.is_empty()),
|
||||
server_warnings,
|
||||
})
|
||||
}
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue