Watch
0
0
Fork
You've already forked hyperhive
0
Commit graph hyperhive/hive-c0re/src/stats/otel_metrics.rs
Author SHA1 Message Date
atlas
e15c499a31 fix(#3245): resolve the remaining intra-doc links in hive-c0re
Takes the crate from 26 rustdoc warnings to 1, on top of the ten in the
previous commit.

argus's review findings:
- agent_sockets.rs: [`write`] was still ambiguous (function vs macro).
  The previous change narrowed the qualifier and left the ambiguity;
  [`write()`] is what resolves it.
- forge/users.rs <hex> and stats/container_stats.rs <name>: unclosed
  HTML tags in prose, now backticked.

The rest of the crate, so the count actually reaches zero:
- job_queue/mod.rs: Queue::graph_snapshot -> JobQueue::graph_snapshot
  (there is no Queue type), and super::scheduler -> scheduler (mod.rs
  *is* job_queue, so super:: pointed outside it)
- job_queue/resource.rs: NodeKind -> super::model::NodeKind
- matrix.rs: password_path(name) -> password_path; and
  forge::provision_user_token -> crate::forge::provision_user_token.
  Note the path has no `users` segment: forge/mod.rs declares `mod
  users` private and re-exports it, so the canonical path comes from the
  re-export rather than the directory tree.
- socket_server/lifecycle_handlers.rs: InfraContainer ->
  hive_priv_sock::InfraContainer
- stats/otel_metrics.rs: crate::meta::otel_config is a private fn no
  path can name from another module, so it becomes prose
- main.rs: redundant explicit link target dropped

coordinator.rs:405 (CrashWatchGuard) is deliberately untouched: #3244
deletes that doc block, so fixing it here would conflict with an open PR
and repair a symbol that is about to stop existing.
2026-08-14 00:25:35 +02:00
atlas
d09ec31431 feat(#2899): add host.arch + service.version to container metrics
The container-resource exporter identified the samples it sent by
container and by hive, but not by machine or by build — so a sample
could not be attributed to the host it came from or the deploy that
produced it.

Both go on the OTEL resource rather than on each data point, for the
same reason `hive` / `swarm` already do: they are constant across one
hive-c0re. Only per-container facts stay per-data-point.

- `host.arch` — mapped to the semconv spelling, not forwarded from
  rust's. The two disagree on exactly the architectures this runs on
  (`x86_64` / `aarch64` vs `amd64` / `arm64`), and the failure mode is
  silent: a dashboard filtering the standard value matches nothing.
  A test pins this, since nothing else would catch it.
- `service.version` — the running flake rev, via
  `auto_update::current_flake_rev`, the same source the dashboard
  snapshot and `get_agent_meta` already use. NOT the crate version:
  that's a workspace constant that never moves between deploys, so it
  could not answer "which build produced this sample?". Omitted rather
  than guessed when the flake ref carries no rev.

`spawn_exporter` takes the flake ref to reach the rev — the string it
needs, not the whole `Coordinator`, so the module's coupling doesn't
widen for one attribute.

The issue's third item, `container`, needs no change: `attrs()` has
emitted `container.name` per data point since this exporter landed.

Checked with clippy (`--all-targets -D warnings`), `cargo test -p
hive-c0re` (321 passed) and `nix fmt`. No option surface is touched, so
no nix-eval gate.
2026-08-01 14:21:10 +02:00
atlas
419c9659a3 feat(#2007): export per-agent container cpu/mem/disk via otel
hive-c0re already samples each agent container's cgroup load for the
dashboard (stats/container_stats.rs); this rides those gauges out to the
configured OTLP endpoint, reusing the existing services.hyperhive.otel
config (endpoint + auth header) — no new toggle.

- New stats/otel_metrics.rs: exports via the OpenTelemetry Rust SDK (same
  crates as hive-metric) with the semconv container.* metric names +
  container.name attribute so off-the-shelf OTel/Grafana dashboards work,
  plus the hive agent label. container.cpu.time (counter, s, from cumulative
  cpu.stat usage_usec), container.memory.usage, container.memory.usage.limit;
  memory peak / on-disk storage / instantaneous cpu percent stay hyperhive.*
  custom (no semconv equivalent). Observable instruments read a shared
  snapshot an async task refreshes (gather() is async; SDK callbacks sync).
- container_stats: expose cpu_time_usec (cumulative) on ContainerResource.
- The OTLP auth header is loaded onto hive-c0re's own unit via systemd
  LoadCredential and read from $CREDENTIALS_DIRECTORY/otel-headers.
- docs/observability.md documents the host-emitted semconv metrics.

Host-side export, so it covers containers even when their agent is idle.
2026-07-15 22:51:31 +02:00