Review catch: this PR relaxed the `otel.endpoint` assertion and staled the
canonical OTEL reference in the same stroke — `docs/observability.md` is
what CLAUDE.md points readers at for "what OTEL options are available",
and it still said required-full-stop while the new swarm/services.md
section said a local store satisfies it.
Also corrects the option's own description in otel.nix, which said the
same thing and renders into the generated options doc. Grepping the
reviewer's phrasing did not find that one; grepping the claim did.
Records the second destination where the "endpoint is where telemetry
ultimately goes" paragraph makes its claim, rather than only in the new
section a reader may not reach.
argus on #3280: the receiver has no auth extension - the nixpkgs module
passes settings straight through and nothing wires one on - so
'unauthenticated to a bridge address' means reachable from an agent
container, not presents a credential. The firewall is the whole access
control.
Consequence, stated because it is a choice rather than an oversight:
any agent can push arbitrary OTLP and the collector forwards it under
the operator's credential, including data smuggled out in resource
attributes. That is a different risk from the one the collector fixes,
and strictly smaller than what preceded it - before, every agent held
the credential itself and could do all of that plus use the token
anywhere else. The collector removes the token and keeps the pipe.
Same principle this PR already applies to the availability trade: state
it where the reader is, rather than let it be discovered.
The headersCredential section still said hive-c0re forwards the file
into each agent container - that is exactly the behaviour this change
removes, so the doc contradicted the code rather than merely lagging
it. Same for the collector section, which presented the two paths the
change collapses.
Also drops the dangling collector.enable reference in the network
section.
argus's non-blocking note on #3278: docs/observability.md is what this
repo's reading-paths index names as 'what OTEL options are available',
and it did not mention collector.enable/port/upstreamHeaderName at all.
The nix docstrings covered it, but not where a reader following the
established path would look.
Carries the two things a docstring is a poor home for: that endpoint
keeps meaning 'where telemetry ultimately goes' (the agent-facing value
is derived, so an existing deployment is unaffected), and the
availability trade the collector makes against the direct-export
property this page already promises.
Two sites spelled it as a brace group, services.hyperhive.{hiveName,
swarmName}, which no anchored rewrite can handle correctly now that only
one of the two moves; both are written out separately. One of them is an
MCP tool description, so it is rendered into every agent's system prompt.
hive-c0re already samples each agent container's cgroup load for the
dashboard (stats/container_stats.rs); this rides those gauges out to the
configured OTLP endpoint, reusing the existing services.hyperhive.otel
config (endpoint + auth header) — no new toggle.
- New stats/otel_metrics.rs: exports via the OpenTelemetry Rust SDK (same
crates as hive-metric) with the semconv container.* metric names +
container.name attribute so off-the-shelf OTel/Grafana dashboards work,
plus the hive agent label. container.cpu.time (counter, s, from cumulative
cpu.stat usage_usec), container.memory.usage, container.memory.usage.limit;
memory peak / on-disk storage / instantaneous cpu percent stay hyperhive.*
custom (no semconv equivalent). Observable instruments read a shared
snapshot an async task refreshes (gather() is async; SDK callbacks sync).
- container_stats: expose cpu_time_usec (cumulative) on ContainerResource.
- The OTLP auth header is loaded onto hive-c0re's own unit via systemd
LoadCredential and read from $CREDENTIALS_DIRECTORY/otel-headers.
- docs/observability.md documents the host-emitted semconv metrics.
Host-side export, so it covers containers even when their agent is idle.
Add docs/observability.md covering all services.hyperhive.otel.*
options: enable, endpoint, protocol, headersCredential,
extraResourceAttributes, debug (new in cb0a66147a), and
metricIntervalMs.
Includes:
- Built-in OTEL_RESOURCE_ATTRIBUTES labels (service.name, agent, hive, swarm)
- Cumulative temporality note (avoids Prometheus DELTA drop)
- Network note for host-side collectors on non-standard ports,
cross-referencing docs/network.md exposeHostPorts
Also:
- CLAUDE.md: add reading-path entry for the new doc
- docs/network.md: link the OTEL mention to observability.md
Closes no issue — gap found during doc sweep.
