grafana: the published vhost refuses /metrics
Grafana runs with metrics.enabled and no basic_auth_*, so its Prometheus exposition was served to any anonymous client of grafana.<swarm> through the published vhost's `location "/"`. Add an exact-match `= /metrics` location returning 403 there. The loopback `grafana-metrics` vhost and the collector's scrape of it are unchanged. Comments on the metricsPort option and the loopback vhost now state that the published vhost refuses /metrics; the gateway doc says the same. A module-eval case pins the refusing location and that `/` still proxies.
This commit is contained in:
parent
eba6428b6b
commit
6c9c2cef45
4 changed files with 28 additions and 3 deletions
|
|
@ -52,7 +52,7 @@ The gateway always terminates TLS. Which certificate it serves depends on what y
|
||||||
|
|
||||||
`tls.certDir` together with `tls.acme.enable` fails an assertion — pick one.
|
`tls.certDir` together with `tls.acme.enable` fails an assertion — pick one.
|
||||||
|
|
||||||
⚠️ One vhost class has no TLS: a vhost bound to loopback for a local consumer. `grafana-metrics` (`nix/host-modules/swarm-grafana.nix`) is the one in the tree: it listens on `127.0.0.1` only and serves `= /metrics` from grafana's unix socket for this host's collector. Everything with a routable name follows the table above.
|
⚠️ One vhost class has no TLS: a vhost bound to loopback for a local consumer. `grafana-metrics` (`nix/host-modules/swarm-grafana.nix`) is the one in the tree: it listens on `127.0.0.1` only and serves `= /metrics` from grafana's unix socket for this host's collector. The published `grafana.<swarm>` vhost answers `/metrics` with 403. Everything with a routable name follows the table above.
|
||||||
|
|
||||||
### ACME / Let's Encrypt (`tls.acme`)
|
### ACME / Let's Encrypt (`tls.acme`)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -72,7 +72,9 @@ in
|
||||||
{option}`services.hyperhive.swarm.otel.scrapeTargets` has: those
|
{option}`services.hyperhive.swarm.otel.scrapeTargets` has: those
|
||||||
targets are trusted by *proximity* rather than by credential.
|
targets are trusted by *proximity* rather than by credential.
|
||||||
Deliberately **not** the published `grafana.<domain>` vhost, which
|
Deliberately **not** the published `grafana.<domain>` vhost, which
|
||||||
would put an authorization decision in front of a scrape.
|
would put an authorization decision in front of a scrape: that vhost
|
||||||
|
answers `/metrics` with 403, because Grafana serves it without
|
||||||
|
authentication.
|
||||||
|
|
||||||
The number itself is arbitrary and free today;
|
The number itself is arbitrary and free today;
|
||||||
`state/eval-port-collisions.sh` is what keeps it that way, since a
|
`state/eval-port-collisions.sh` is what keeps it that way, since a
|
||||||
|
|
|
||||||
|
|
@ -458,6 +458,13 @@ in
|
||||||
services.nginx.virtualHosts."${cfg.domain}" = (gatewayCfg.lib.tlsFor cfg.domain) // {
|
services.nginx.virtualHosts."${cfg.domain}" = (gatewayCfg.lib.tlsFor cfg.domain) // {
|
||||||
listen = gatewayCfg.lib.listen;
|
listen = gatewayCfg.lib.listen;
|
||||||
extraConfig = gatewayCfg.lib.securityHeaders;
|
extraConfig = gatewayCfg.lib.securityHeaders;
|
||||||
|
# Grafana serves its Prometheus exposition to anyone, signed in or not
|
||||||
|
# (`metrics.enabled` below, no `basic_auth_*`), so the published name
|
||||||
|
# refuses it; the scrape goes through the loopback `grafana-metrics`
|
||||||
|
# vhost instead. `=` is enough: nginx matches the decoded, normalised
|
||||||
|
# path, so `//metrics`, `/./metrics`, `/%6detrics` and `/metrics?x` all
|
||||||
|
# land here, and Grafana itself answers `/metrics/` with a login redirect.
|
||||||
|
locations."= /metrics".return = "403";
|
||||||
locations."/" = {
|
locations."/" = {
|
||||||
proxyPass = "http://unix:${socketPath}:/";
|
proxyPass = "http://unix:${socketPath}:/";
|
||||||
proxyWebsockets = true;
|
proxyWebsockets = true;
|
||||||
|
|
@ -484,7 +491,8 @@ in
|
||||||
# ⚠️ `= /metrics` is an EXACT match, not a prefix: it is the difference
|
# ⚠️ `= /metrics` is an EXACT match, not a prefix: it is the difference
|
||||||
# between exposing one endpoint and re-serving the whole of Grafana on an
|
# between exposing one endpoint and re-serving the whole of Grafana on an
|
||||||
# unauthenticated port. Widening it would be a silent authorization bypass,
|
# unauthenticated port. Widening it would be a silent authorization bypass,
|
||||||
# since nothing else about this block would have to change.
|
# since nothing else about this block would have to change. The published
|
||||||
|
# vhost above refuses `/metrics`, so this listener is the only way to it.
|
||||||
services.nginx.virtualHosts."grafana-metrics" = {
|
services.nginx.virtualHosts."grafana-metrics" = {
|
||||||
listen = [
|
listen = [
|
||||||
{
|
{
|
||||||
|
|
|
||||||
|
|
@ -121,6 +121,21 @@ let
|
||||||
grafanaOldPath.services.nginx.virtualHosts."grafana-metrics".locations."= /metrics".proxyPass
|
grafanaOldPath.services.nginx.virtualHosts."grafana-metrics".locations."= /metrics".proxyPass
|
||||||
== "http://unix:/run/swarm-grafana/grafana.sock:/metrics";
|
== "http://unix:/run/swarm-grafana/grafana.sock:/metrics";
|
||||||
}
|
}
|
||||||
|
{
|
||||||
|
# Grafana serves `/metrics` unauthenticated, so the published name must
|
||||||
|
# answer it itself; the scrape reads the loopback vhost above instead.
|
||||||
|
# `=` is what catches the normalised variants and still leaves the rest
|
||||||
|
# of Grafana proxied.
|
||||||
|
name = "grafana's published vhost refuses /metrics and still proxies everything else";
|
||||||
|
ok =
|
||||||
|
let
|
||||||
|
g = grafanaOldPath.services.hyperhive.swarm.grafana;
|
||||||
|
locs = grafanaOldPath.services.nginx.virtualHosts.${g.domain}.locations;
|
||||||
|
in
|
||||||
|
locs."= /metrics".return == "403"
|
||||||
|
&& locs."= /metrics".proxyPass == null
|
||||||
|
&& lib.hasPrefix "http://unix:" locs."/".proxyPass;
|
||||||
|
}
|
||||||
{
|
{
|
||||||
# This fixture enables grafana and NOT authelia, so the login-form toggle
|
# This fixture enables grafana and NOT authelia, so the login-form toggle
|
||||||
# has to follow "grafana requires SSO" rather than "both services are on
|
# has to follow "grafana requires SSO" rather than "both services are on
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue