diff --git a/docs/observability.md b/docs/observability.md index 5a3fbea0..5e4f5b01 100644 --- a/docs/observability.md +++ b/docs/observability.md @@ -16,8 +16,11 @@ services.hyperhive.otel = { }; ``` -`enable` is the single gate. `endpoint` (required when enabled) is where -telemetry ultimately goes. +`enable` is the single gate. `endpoint` is where telemetry goes upstream — +required when enabled *unless* this host runs the swarm's own metrics store +(`swarm.victoriametrics.enable`), which is a destination in its own right. With +both, telemetry goes to both. See +[`swarm/services.md`](swarm/services.md#metrics-victoriametrics--grafana). **There is exactly one way telemetry leaves a hive: through the collector that `enable` starts on the host.** Agents never talk to `endpoint` themselves — @@ -58,11 +61,16 @@ closed by this design, and nothing here should be read as closing it. Master switch. When true, all other options below take effect. -### `services.hyperhive.otel.endpoint` — string, required when enabled +### `services.hyperhive.otel.endpoint` — string, required when enabled unless the swarm store runs here -OTLP collector endpoint URL. Set as `OTEL_EXPORTER_OTLP_ENDPOINT` for every +Upstream OTLP endpoint URL. Set as `OTEL_EXPORTER_OTLP_ENDPOINT` for every agent. Example: `"https://collector.example.com/otel"`. +Leave it empty **only** on a host running `swarm.victoriametrics.enable` — the +local store is then the destination and the collector writes there instead. +With neither, `enable` is refused at eval: telemetry with nowhere to go is a +misconfiguration, not a quiet no-op. + ### `services.hyperhive.otel.protocol` — enum, default `"http/protobuf"` OTLP wire protocol, passed as `OTEL_EXPORTER_OTLP_PROTOCOL`. Accepted values: @@ -142,12 +150,16 @@ protects it from other containers, not from the agent itself. As long as the direct path stays *selectable*, that hole stays selectable; an option that can reintroduce it is a hole with extra steps. -**`endpoint` keeps meaning "where telemetry ultimately goes."** The collector +**`endpoint` keeps meaning "where telemetry goes upstream."** The collector does not redefine it — the agent-facing value is *derived* (`http://:`), so an existing deployment's `endpoint` keeps working unchanged. The bridge port is contributed to `exposeHostPorts` automatically; there is nothing to open by hand. +What the collector *added* is a second destination: on a host running the +swarm's metrics store it writes there too, so `endpoint` is no longer the only +place telemetry can land — and no longer the only way to have one. + ### `services.hyperhive.otel.collector.port` — port, default `4318` The OTLP/HTTP port the collector listens on, bound to the bridge IP only. diff --git a/nix/host-modules/otel.nix b/nix/host-modules/otel.nix index ca159c84..7d67d5aa 100644 --- a/nix/host-modules/otel.nix +++ b/nix/host-modules/otel.nix @@ -36,8 +36,15 @@ default = ""; example = "https://collector.example.com/otel"; description = '' - OTLP collector endpoint, set as `OTEL_EXPORTER_OTLP_ENDPOINT` - for every agent. Required when `enable` is true. + Upstream OTLP endpoint, set as `OTEL_EXPORTER_OTLP_ENDPOINT` for + every agent. + + Required when `enable` is true, **unless** this host runs the + swarm's metrics store + ({option}`services.hyperhive.swarm.victoriametrics.enable`) — that + store is a destination in its own right, and with both configured + telemetry goes to both. With neither, `enable` is refused rather + than silently exporting nowhere. ''; };