vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

swarm-otel: scrape grafana through its published name

Browse source 
Grafana's own metrics were scraped over loopback through the
`grafana-metrics` nginx vhost, declared only where grafana and the
collector share a host, so a collector on any other host had no job for
it. Its target is now declared on every host as
`https://<swarm.grafana.domain>/metrics`, next to the other published
targets, and grafana's vhost serves `= /metrics` behind the collector's
auth subrequest (`/__grafana_metrics_authz`, the forge's shape). The
rules derived from `publishedScrapeTargets` admit the collector's client
there and deny every other subject.

The loopback scrape is gone with the vhost that served it and its
`swarm.grafana.metricsPort` option, so a single host still scrapes
grafana once, now by name. A config that sets that option now fails
evaluation with "option does not exist".

On the published vhost `/metrics` was answered with 403 for everyone.
It now answers the collector's token, and a plain session gets authelia's
`deny`. The design note that ruled the published vhost out for scraping
predates the collector's credential; it is replaced by the location's
own comment and a line on `swarm.grafana.domain`.

Part of #5205
This commit is contained in:
atlas 2026-10-14 19:13:53 +02:00 • committed by mara
commit 54f6061d41
1 changed files with 145 additions and 99 deletions
Download patch file Download diff file Expand all files Collapse all files

24
nix/host-modules/glue-otel-scrape-targets.nix
View file

@ -1,21 +1,20 @@
# Glue: declare the loopback scrape targets of grafana and nats on the swarm's
# OTEL collector, wherever one of them shares a host with it. Authelia is a
# `publishedScrapeTargets` job (./swarm-otel-service.nix); a loopback job of
# Glue: declare the loopback scrape target of nats on the swarm's OTEL
# collector, wherever it shares a host with it. Authelia and grafana are
# `publishedScrapeTargets` jobs (./swarm-otel-service.nix); a loopback job of
# the same name fails ./swarm-otel.nix's job-name assertion.
#
# ONE FILE, TWO PAIRINGS — each service ← the collector, with the same shape.
# Deleting this leaves the collector scraping none of them: no error, no
# metrics.
# ONE PAIRING — nats ← the collector. Deleting this leaves the collector not
# scraping nats: no error, no metrics.
#
# Each target is gated on the service AND the collector being enabled here. The
# The target is gated on the service AND the collector being enabled here. The
# loopback address is only honest when both run on this host; a target nobody
# can read asserts a collection that is not happening. Both services default
# from `deploy.allSwarmServices`, but `mkDefault` is an invitation to override,
# so co-location is a property of the auto-deployed topology, not the modules.
# can read asserts a collection that is not happening. Both default from
# `deploy.allSwarmServices`, but `mkDefault` is an invitation to override, so
# co-location is a property of the auto-deployed topology, not the modules.
#
# ⚠️ Silent when split: separate hosts are separate evaluations with no shared
# context, so a host running a service and no collector declares nothing and
# the collector's host never learns of the service. No error, no warning — a
# context, so a host running nats and no collector declares nothing and
# the collector's host never learns of it. No error, no warning — a
# healthy exporter and an empty dashboard. Not guardable from here.
#
# ⚠️ `swarm.otel`, not `hyperhive.otel` — two collectors one word apart, and only
@ -33,7 +32,6 @@ let
in
{
config.services.hyperhive.swarm.otel.scrapeTargets = {
grafana = withCollector "grafana" "127.0.0.1:${toString swarmCfg.grafana.metricsPort}";
nats = withCollector "nats" "127.0.0.1:${toString swarmCfg.nats.metricsPort}";
};
}