From 7da791515038cedbb9b6305f1963a241417072f9 Mon Sep 17 00:00:00 2001 From: atlas Date: Wed, 19 Aug 2026 15:11:37 +0200 Subject: [PATCH] otel: refuse a half-configured escape hatch instead of 404ing silently MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Turning ingest auth off without clearing a hive's credential leaves that hive's collector authenticating and addressing its own path, while an unauthenticated swarm tier serves one catch-all and forwards the URI unchanged. The receiver is asked for a path it does not serve, so telemetry stops with 404s and retries — no 401, no assertion, nothing in any log naming auth. Only reachable by overriding one side without the other, since both defaults derive from the same flag. That is what makes it worth a build error rather than a caveat: an operator who flips the documented escape hatch has no reason to suspect the sending half. Found in review by argus. --- docs/observability.md | 6 ++++++ nix/host-modules/otel.nix | 35 +++++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+) diff --git a/docs/observability.md b/docs/observability.md index 20a51a47..7384ba0c 100644 --- a/docs/observability.md +++ b/docs/observability.md @@ -212,8 +212,14 @@ trusted, or a swarm with no authelia: ```nix services.hyperhive.swarm.otel.requireHiveIdentity = false; +services.hyperhive.otel.clientSecretFile = null; # on each hive that had one ``` +Both halves, because a collector that authenticates also addresses its hive's +own path, and an unauthenticated swarm tier serves no per-hive paths. Set only +the first and that hive's samples 404 instead of arriving. On a host running +both tiers the build says so; on a split host it is yours to keep in step. + ⚠️ That reopens the original hole rather than merely skipping a check: while an unauthenticated port is listening, anything that can reach the collector can attribute metrics to any hive. diff --git a/nix/host-modules/otel.nix b/nix/host-modules/otel.nix index ebeefc9b..0c1ef648 100644 --- a/nix/host-modules/otel.nix +++ b/nix/host-modules/otel.nix @@ -403,6 +403,41 @@ in false to accept unauthenticated ingest. ''; } + { + # The mirror of the assertion above, and the reason it exists is + # that the failure is SILENT rather than loud. With ingest + # unauthenticated the swarm tier serves one catch-all location + # and passes the URI through unchanged — but this tier still + # appends `/` whenever it holds a credential, so the + # receiver is asked for `//v1/metrics`, a path it does not + # serve. The result is 404s and retries: no 401, no assertion, no + # log anywhere saying telemetry stopped. + # + # Only reachable by overriding one side without the other, since + # both defaults derive from `swarm.authelia.enable` and move + # together. That is exactly why it is worth a build error — an + # operator who flips the escape hatch has no reason to suspect + # the sending half. + assertion = !(swarmOtelCfg.enable && !swarmOtelCfg.requireHiveIdentity && senderAuth); + message = '' + This host accepts unauthenticated telemetry ingest + (services.hyperhive.swarm.otel.requireHiveIdentity = false), + but its own collector still holds a credential: + + services.hyperhive.otel.clientSecretFile = ${ + if otel.clientSecretFile == null then "null" else otel.clientSecretFile + } + + A collector that authenticates also addresses its hive's own + path, and an unauthenticated swarm tier serves no per-hive + paths — so this hive's samples would 404 rather than be + refused, which no log names as an auth problem. + + Set services.hyperhive.otel.clientSecretFile = null to send + unauthenticated too, or drop the requireHiveIdentity + override. + ''; + } ]; } );