Watch
0
0
Fork
You've already forked hyperhive
0
hyperhive/hive-c0re/src/host_stats.rs
iris 5d0f3d060b feat(dashboard): generic server-warnings banner on every page (#1518)
Per operator request: instead of a disk-specific alert, surface a generic
server-warnings banner at the very top of every page, so new system
warnings can be added backend-side with no frontend change.

- hive-c0re `host_stats`: `server_warnings() -> Vec<ServerWarning>`
  (`{ kind, level, message }`). The threshold logic lives server-side; the
  host disk-pressure check (a `statvfs` probe of `/nix`: ≥85% used → warn,
  ≥95% → crit) is the first and only producer today. No new deps (libc).
- `/api/state` carries `server_warnings` (replaces the disk-specific
  field). Empty when all clear.
- frontend: `renderServerWarnings` / `initServerWarnings` in `common.js`
  inject a sticky top-of-<body> banner and render the list, coloured by
  `level`. Wired on every page — dashboard (live, via refreshState),
  FL0W, L0GS, H0M3. No per-warning frontend code; adding a warning kind
  is a pure backend change.

cargo check/clippy/fmt + npm run build green. Closes #1518.
2026-06-08 20:55:15 +02:00

124 lines
5 KiB
Rust
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

//! Host-system probes and the derived **server-warning** list that the
//! dashboard renders as a top-of-page banner.
//!
//! The first (and currently only) producer is a host disk-usage check:
//! the nix store filling up is what ENOSPC-failed CI before the GC
//! guardrails were documented. hive-c0re runs on the host (not inside a
//! container), so it can `statvfs` the store path directly and warn the
//! operator *before* an ENOSPC, not after.
//!
//! [`server_warnings`] is the public surface: it returns a flat list of
//! [`ServerWarning`]s for `/api/state`. The dashboard renders whatever it
//! returns, coloured by `level`, so adding a new system warning (memory
//! pressure, a failed unit, …) is a backend-only change — no frontend
//! edit. Keep producers cheap; this runs on every `/api/state` assembly.
use serde::Serialize;
/// One server-level warning for the dashboard's top-of-page banner.
#[derive(Debug, Clone, Serialize)]
pub struct ServerWarning {
/// Stable kind id (e.g. `"disk_pressure"`) — lets the frontend dedupe
/// or special-case without parsing the message.
pub kind: &'static str,
/// Severity: `"warn"` (amber) or `"crit"` (red). The banner picks its
/// colour from this; everything else is just the message text.
pub level: &'static str,
/// Human-readable, already-formatted message shown in the banner.
pub message: String,
}
/// Percent-used past which the host nix store earns a disk-pressure
/// warning; above [`DISK_CRIT_PCT`] it escalates to `crit`.
const DISK_WARN_PCT: f64 = 85.0;
const DISK_CRIT_PCT: f64 = 95.0;
/// Collect the current server-level warnings for the dashboard banner.
/// Each producer pushes zero or more [`ServerWarning`]s; the frontend
/// renders whatever this returns. Cheap to call on every `/api/state`
/// assembly (currently a single `statvfs`).
#[must_use]
pub fn server_warnings() -> Vec<ServerWarning> {
let mut out = Vec::new();
if let Some(d) = nix_disk_usage()
&& d.used_pct >= DISK_WARN_PCT
{
#[allow(
clippy::cast_precision_loss,
reason = "byte counts stay well under f64's 2^53 exact-integer range, so this GiB conversion loses no precision"
)]
let free_gib = d.free_bytes as f64 / (1024.0 * 1024.0 * 1024.0);
out.push(ServerWarning {
kind: "disk_pressure",
level: if d.used_pct >= DISK_CRIT_PCT {
"crit"
} else {
"warn"
},
message: format!(
"host nix store {:.0}% full ({free_gib:.1} GiB free) \
— garbage-collect the store before it runs out of space",
d.used_pct
),
});
}
out
}
/// Disk usage for the filesystem backing the host nix store — internal to
/// the disk-pressure producer above.
struct DiskUsage {
/// Space available to unprivileged writers, in bytes.
free_bytes: u64,
/// Percentage used, 0–100. Mirrors `df`'s use% — `used / (used +
/// available)` — so the threshold means what the operator sees in `df`.
used_pct: f64,
}
/// Probe disk usage for the filesystem containing the nix store (`/nix`),
/// falling back to `/` when `/nix` isn't its own mount. Returns `None` if
/// the `statvfs` syscall fails (path missing, permission, etc.).
fn nix_disk_usage() -> Option<DiskUsage> {
disk_usage("/nix").or_else(|| disk_usage("/"))
}
fn disk_usage(path: &str) -> Option<DiskUsage> {
let c_path = std::ffi::CString::new(path).ok()?;
// SAFETY: `statvfs` reads only through the valid NUL-terminated
// `c_path` pointer and writes into the zeroed `stat` we own. We check
// the return code before reading any field.
let mut stat: libc::statvfs = unsafe { std::mem::zeroed() };
let rc = unsafe { libc::statvfs(c_path.as_ptr(), &raw mut stat) };
if rc != 0 {
return None;
}
// statvfs fields are `c_ulong` (== u64 on the x86_64-linux host this
// runs on); the arithmetic below stays in that native width.
let frsize = stat.f_frsize;
let total_blocks = stat.f_blocks;
let free_blocks = stat.f_bfree;
let avail_blocks = stat.f_bavail;
if total_blocks == 0 || frsize == 0 {
return None;
}
let free_bytes = avail_blocks.saturating_mul(frsize);
// df's use%: used / (used + available). `used` counts root-reserved
// blocks (total - bfree); `available` is the unprivileged free
// (bavail), so the percentage matches what `df` reports.
let used_blocks = total_blocks.saturating_sub(free_blocks);
let capacity = used_blocks.saturating_add(avail_blocks);
let used_pct = if capacity == 0 {
0.0
} else {
#[allow(
clippy::cast_precision_loss,
reason = "block counts stay well under f64's 2^53 exact-integer range, so this percentage computation loses no precision"
)]
let raw = used_blocks as f64 / capacity as f64 * 100.0;
raw
};
Some(DiskUsage {
free_bytes,
used_pct,
})
}