feat(hive-c0re): export metrics whose subject is the hive, not an agent
Every hive-labelled series in the store also carries an agent label, so a
hive is only ever visible as the sum of its agents — and a hive whose c0re
has stopped is indistinguishable from one that simply hosts none.
Adds three instruments to the exporter hive-c0re already runs, each a
projection of a value the process computes anyway: process.uptime (the
semconv name — the spec defines it as a double gauge in seconds, which is
exactly this instrument), hyperhive.hive.degraded, and
hyperhive.hive.warnings split by level. None carries an agent attribute;
that absence is what makes them selectable as hive-scoped.
The health pair reads warnings::readiness() rather than deriving its own
verdict, and degraded ships as a series instead of being left for a
dashboard query to compute from warnings{level="crit"} — either would put
the "what counts as unhealthy" rule in a second place that disagrees
silently the first time a degrading condition is added.
This commit is contained in:
parent
f51fa921b4
commit
5eca0cc516
3 changed files with 222 additions and 2 deletions
|
|
@ -97,6 +97,20 @@ pub const STATUS_OK: &str = "ok";
|
|||
/// `status` value for a hive with at least one `crit`-level warning set.
|
||||
pub const STATUS_DEGRADED: &str = "degraded";
|
||||
|
||||
/// Warning level meaning "an operator should look" — does not degrade the
|
||||
/// hive's readiness.
|
||||
pub const LEVEL_WARN: &str = "warn";
|
||||
/// Warning level meaning "this hive is not healthy" — degrades readiness.
|
||||
pub const LEVEL_CRIT: &str = "crit";
|
||||
|
||||
/// Every level a warning can carry, in increasing severity.
|
||||
///
|
||||
/// Named because a consumer that *reports on* levels has to enumerate them,
|
||||
/// and enumerating them as literals in another module is the agreement
|
||||
/// nothing checks: the day a third level is added, that consumer keeps
|
||||
/// compiling and silently stops covering it.
|
||||
pub const LEVELS: [&str; 2] = [LEVEL_WARN, LEVEL_CRIT];
|
||||
|
||||
/// What this hive currently says about its own health.
|
||||
///
|
||||
/// One type with one producer ([`readiness`]) because there is more than
|
||||
|
|
@ -133,7 +147,7 @@ impl Readiness {
|
|||
#[must_use]
|
||||
pub fn readiness() -> Readiness {
|
||||
let warnings = snapshot();
|
||||
let status = if warnings.iter().any(|w| w.level == "crit") {
|
||||
let status = if warnings.iter().any(|w| w.level == LEVEL_CRIT) {
|
||||
STATUS_DEGRADED
|
||||
} else {
|
||||
STATUS_OK
|
||||
|
|
|
|||
Loading…
Reference in a new issue