vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(hive-c0re): export metrics whose subject is the hive, not an agent

Browse source 
Every hive-labelled series in the store also carries an agent label, so a
hive is only ever visible as the sum of its agents — and a hive whose c0re
has stopped is indistinguishable from one that simply hosts none.

Adds three instruments to the exporter hive-c0re already runs, each a
projection of a value the process computes anyway: process.uptime (the
semconv name — the spec defines it as a double gauge in seconds, which is
exactly this instrument), hyperhive.hive.degraded, and
hyperhive.hive.warnings split by level. None carries an agent attribute;
that absence is what makes them selectable as hive-scoped.

The health pair reads warnings::readiness() rather than deriving its own
verdict, and degraded ships as a series instead of being left for a
dashboard query to compute from warnings{level="crit"} — either would put
the "what counts as unhealthy" rule in a second place that disagrees
silently the first time a degrading condition is added.
This commit is contained in:
atlas 2026-08-24 19:13:09 +02:00 • committed by mara
commit 5eca0cc516
3 changed files with 222 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

25
docs/observability.md
View file

@ -245,6 +245,31 @@ section — nothing extra to configure. Cadence follows
the batched points are flushed to the collector, not how often they're
recorded (every turn, always).
## Hive-scoped metrics (hive-c0re)
Everything above is measured **per agent**, tagged with the hive it runs in.
These three are measured per **hive**, and carry no `agent` label — so a hive
that hosts no agents still reports, and "this hive is quiet" is
distinguishable from "this hive is gone". Select them with
`{hive!="",agent=""}`.
| Metric | Unit | Kind | Meaning |
|--------|------|------|---------|
| `process.uptime` | `s` | gauge | seconds since this hive's `hive-c0re` started exporting; a restart reads as a drop to ~0 |
| `hyperhive.hive.degraded` | `1` | gauge | `1` while the hive reports itself unhealthy — the same verdict `/health/ready` gives and the swarm status view shows |
| `hyperhive.hive.warnings` | `1` | gauge | how many warnings are currently raised, split by a `level` attribute (`warn`, `crit`) |
Both levels are reported every cycle, `0` included, so a healthy hive is
visible as zeros rather than as missing series.
`hyperhive.hive.degraded` is what a dashboard should alert on: it is
`hive-c0re`'s own readiness verdict, so it stays in step with `/health/ready`
and with what the swarm controller sees. `hyperhive.hive.warnings` is the
detail behind it — `warn`-level entries mean "an operator should look" and do
**not** set `degraded`.
Same cadence, transport and resource labels as the container metrics above.
## Agent-emitted custom metrics (`hive-metric`)
Agents can push arbitrary labeled metrics to the same OTEL collector via the