From 94ef8428e03d2aaf57ea3bb622cd62e528819715 Mon Sep 17 00:00:00 2001 From: atlas Date: Mon, 24 Aug 2026 13:36:32 +0200 Subject: [PATCH] refactor(swarm-otel): one declaration for a published scrape target MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `collectorAudiences` was a list of URLs services contributed so the collector's client could be registered for them. Slice C needs the same URLs as scrape jobs, and a job needs a name the audience list has no room for — so services would have contributed to two options that must agree. They now contribute `publishedScrapeTargets` once, as ` = ""`, and the client's audiences derive from it. The drift that would have needed maintaining is gone, and its failure mode was the quiet one: a target whose audience was forgotten authenticates against nothing and reads as a broken scrape rather than a missing registration. Adds an assertion for the one collision the module system cannot catch. Two definitions of the same key within one option are already refused (measured); across the two scrape options nothing arbitrates, and both entries would render into a single scrape_configs list under one job_name. --- nix/host-modules/hive-forge/default.nix | 15 ++-- nix/host-modules/swarm-otel.nix | 100 ++++++++++++++++-------- 2 files changed, 77 insertions(+), 38 deletions(-) diff --git a/nix/host-modules/hive-forge/default.nix b/nix/host-modules/hive-forge/default.nix index e319da0c..4367fd28 100644 --- a/nix/host-modules/hive-forge/default.nix +++ b/nix/host-modules/hive-forge/default.nix @@ -442,17 +442,20 @@ in url = "https://${cfg.domain}/"; }; - # The address a scraper has to hold a token for. Same `behindGateway` - # guard, and for a stronger reason than the two above: with it off - # there is no `= /metrics` location and no `auth_request` in front of - # it, so the URL this names does not exist to be authorised. + # The metrics endpoint, declared once: the collector both scrapes this + # URL and derives from it the audience its token is minted for. Same + # `behindGateway` guard, and for a stronger reason than the two above: + # with it off there is no `= /metrics` location and no `auth_request` + # in front of it, so the URL this names does not exist to be scraped + # or authorised. # # ⚠️ Written as the exact URL a collector requests, because that is # what authelia compares against — this string agreeing with the # `location` block above it is the whole mechanism. A near miss is a # correctly minted token refused at the target. - services.hyperhive.swarm.otel.collectorAudiences = - lib.optional cfg.behindGateway "https://${cfg.domain}/metrics"; + services.hyperhive.swarm.otel.publishedScrapeTargets = lib.optionalAttrs cfg.behindGateway { + forgejo = "https://${cfg.domain}/metrics"; + }; # `server_name = forge.domain`, proxies all `/` → forgejo. Tuned for # git: `client_max_body_size 1G`, `proxy_read_timeout 1h` (multi-GB diff --git a/nix/host-modules/swarm-otel.nix b/nix/host-modules/swarm-otel.nix index c02563c4..c1b7fea2 100644 --- a/nix/host-modules/swarm-otel.nix +++ b/nix/host-modules/swarm-otel.nix @@ -224,36 +224,35 @@ in ''; }; - collectorAudiences = lib.mkOption { - type = lib.types.listOf lib.types.str; - default = [ ]; - example = lib.literalExpression ''[ "https://forge.example.com/metrics" ]''; + publishedScrapeTargets = lib.mkOption { + type = lib.types.attrsOf lib.types.str; + default = { }; + example = lib.literalExpression ''{ forge = "https://forge.example.com/metrics"; }''; description = '' - URLs this collector may mint an access token for, contributed by - the modules that publish them. + Prometheus endpoints this collector scrapes **by name, with a + credential**, as ` = ""`. - A service module says **which URL it exposes**; this module turns - the set into the collector's single OAuth2 client — the same split - as `services.hyperhive.gateway.localNames`, and for the same - reason. The collector has no business knowing another service's - routes, and a service cannot reach into the client list to add - itself: client definitions concatenate rather than merging into a - shared entry, so the entry has to be built in one place from a - list many modules can write. + A service module declares its own entry, from its own module, the + same way it does for `scrapeTargets` — and the collector's OAuth2 + client derives its permitted audiences from these URLs, so a + target and the authorisation to reach it are **one declaration**. + Two lists that must agree would be a drift to maintain, and its + failure mode is the bad one: a target whose audience was forgotten + authenticates against nothing and reads as a scrape failure rather + than a config mistake. - ⚠️ An audience is a **URL**, not a label. Authelia validates a - bearer token against the address being requested, so a token - minted for one target is refused at another and a client may only - request an audience it is registered for — registration is the - authorisation. A near miss (a trailing slash, `http` for `https`) - presents as a valid token rejected at the target, several layers - from its cause. + ⚠️ A **full URL**, not `host:port`. Authelia validates a bearer + token against the address being requested, so the string here is + also the audience the token is minted for; a near miss (a trailing + slash, `http` for `https`) presents as a valid token rejected at + the target, several layers from its cause. - ⚠️ Deliberately not part of `scrapeTargets`. An entry there is - trusted because the scraper and the target share a host; an entry - here is trusted because it presents a credential. Folding the two - together would leave a reader unable to tell which of those a - given target relies on. + ⚠️ Deliberately separate from `scrapeTargets`. An entry there is + trusted because the scraper and the target share a host — that + option is loopback-and-unauthenticated by contract. An entry here + is trusted because it presents a credential. One shape for both + would leave a reader unable to tell which of those a given target + relies on. ''; }; @@ -328,14 +327,14 @@ in # render nothing while reading as done; a swarm whose authelia lives # elsewhere registers it there. # - # ⚠️ Also conditional on the audience set being non-empty, and that is - # the shipped case rather than an edge — nothing contributes a URL - # until some service publishes one behind the gateway. Authelia - # refuses a bearer-authz client with no audience, so an unconditional + # ⚠️ Also conditional on there being a published target, and that is + # the shipped case rather than an edge — nothing declares one until + # some service publishes behind the gateway. Authelia refuses a + # bearer-authz client with no audience, so an unconditional # declaration would break every hive that runs a collector and # publishes nothing. services.hyperhive.swarm.authelia.oidc.clients = - lib.mkIf (autheliaCfg.enable && cfg.collectorAudiences != [ ]) + lib.mkIf (autheliaCfg.enable && cfg.publishedScrapeTargets != { }) [ { id = cfg.clientId; @@ -345,7 +344,13 @@ in # endpoint refuses an otherwise valid token and blames the # token rather than the missing grant. bearerAuthz = true; - audience = cfg.collectorAudiences; + # DERIVED from the targets rather than contributed alongside + # them. A service declares a URL once and this is the + # permission to reach it; two lists that had to agree would be + # a drift to maintain, and the failure mode is the quiet one — + # a target whose audience was forgotten authenticates against + # nothing and looks like a broken scrape. + audience = lib.attrValues cfg.publishedScrapeTargets; # Stated rather than left on authelia's default, because the # two agreeing today is not the same as this being the # required value: authelia permits only basic / JWT methods @@ -416,6 +421,37 @@ in Rename the hive. Reserved: ${lib.concatMapStringsSep ", " (n: "'${n}'") reservedOwners}. ''; } + { + # A job name used by BOTH scrape options. Within one option this + # cannot happen — the module system refuses two definitions of the + # same key with different values, measured — but the two options + # are separate, so nothing arbitrates between them, and they + # render into a single `scrape_configs` LIST where nothing + # overwrites anything: both entries ship under one `job_name`. + # + # The message names which side each collision came from, which is + # the part a rendered-config error could not tell an operator. + assertion = + lib.intersectLists (lib.attrNames cfg.scrapeTargets) (lib.attrNames cfg.publishedScrapeTargets) + == [ ]; + message = '' + services.hyperhive.swarm.otel: ${ + lib.concatMapStringsSep ", " (j: "'${j}'") ( + lib.intersectLists (lib.attrNames cfg.scrapeTargets) (lib.attrNames cfg.publishedScrapeTargets) + ) + } is declared as both a loopback + scrapeTargets job and a publishedScrapeTargets job. + + They render into one prometheus scrape_configs list, so both + entries would ship under the same job_name — nothing overwrites + anything, and the two are scraped by different rules with + different trust. + + Rename one. A target is either reachable on loopback because it + shares this host, or published and reached with a credential; it + should not be described as both. + ''; + } { # A hive proves who it is with a token this provider mints, so # there is no version of this collector that runs without one.