Watch
0
0
Fork
You've already forked hyperhive
0

hive-c0re: hive warnings reach the operator as metrics and alerts, not a banner

The dashboard's top-of-page warning banner is gone. The warning registry
stays, because /health/ready, the swarm status view and the OTEL gauges
read it. What it feeds is now alerted on in Grafana:

- hyperhive.hive.warning{kind,level}: one series per raised warning, so an
  alert names the cause. Raising a warning also logs one `hive warning
  raised` line with the message.
- hyperhive.hive.nix_store.utilization: the used fraction of the nix
  store's filesystem, probed in the gauge callback.
- Alerts: hh-hive-degraded on the existing degraded gauge, hh-hive-warning
  per kind (disk excluded), hh-host-disk-high at 85% and
  hh-host-disk-critical at 95%.
- pending_logins is now a registry warning, refreshed from the container
  rescan, so it reaches the per-kind gauge.

Frontend: server_warnings leaves the state snapshot, and the dashboard,
flow and stats pages keep only the sticky chrome wrapper. The shared
hive-warn and WarnBanner components stay, since other views use them.

A module-eval case checks each alert's metric name against the exporter's
source, so a rename on either side fails the build.
This commit is contained in:
iris 2026-10-21 17:52:53 +02:00
commit 4dbcf6fa80
35 changed files with 468 additions and 322 deletions

View file

@ -367,11 +367,11 @@ pub async fn agents_for_meta_listing() -> Result<Vec<crate::meta::AgentSpec>> {
agents_for_meta(None).await
}
/// Per-agent "stop failed" banner guards, keyed by agent name. Cleared the
/// Per-agent "stop failed" warning guards, keyed by agent name. Cleared the
/// next time that agent's container stops cleanly — a `Mutex<HashMap<...>>`
/// rather than a bare `set_boot_warning` because this condition (unlike a
/// one-shot boot step) genuinely resolves later, once an operator has
/// intervened, and the banner should clear with it instead of surviving
/// intervened, and the warning should clear with it instead of surviving
/// until the next hive-c0re restart.
fn stop_failed_guards()
-> &'static std::sync::Mutex<std::collections::HashMap<String, crate::warnings::WarningGuard>> {
@ -402,7 +402,7 @@ pub async fn kill(name: &str) -> Result<()> {
.unwrap_or_else(std::sync::PoisonError::into_inner);
match result {
Ok(()) => {
// A clean stop clears any earlier "this looked wedged" banner
// A clean stop clears any earlier "this looked wedged" warning
// for the same agent.
guards.remove(name);
Ok(())