Compare commits

...
Author SHA1 Message Date
atlas
b2ee674415 fix(#3517): metrics are not optional, and the endpoint is denied by default
Two review findings, folded together.

mara: a swarm integrated auto deployed forge always has metrics, so the
toggle is gone. The endpoint follows behindGateway instead, which is the
swarm-integrated shape and the condition the protected location lives
under. Serving it without that location would put it on a listener
openFirewall can expose with nothing in front.

argus: /metrics matched no access_control rule, so default_policy
one_factor governed it. That is any authenticated subject, which today
means any operator and tomorrow any agent. My audience argument covered
the Bearer path only; the same endpoint also accepts CookieSession, and a
cookie carries no audience at all, so the audience was never what stood
between a browser session and this data.

The rule is deny rather than a client-scoped allow because the collector's
client does not exist yet. Authelia refuses a subject naming an
unregistered client, and does so in a preStart validator rather than at
build time, so naming one early yields a green nixos-rebuild and dead
swarm SSO on the next restart. Denying until the client is registered
makes publishing the endpoint safe on its own; registering it is a
one-line change from deny to that allow.

Rule order is load-bearing: authelia takes the first match.
2026-08-24 12:06:51 +02:00
atlas
4e17deada5 feat(#3517): let a machine authenticate to the authz endpoint with a bearer token
The gateway authenticates scrapers so services do not each grow a static
bearer of their own, but authelia's auth_request endpoint ran its default
strategies, which are cookie-only. A scraper's OAuth2 access token was
refused no matter how it was minted.

CookieSession is listed explicitly because authn_strategies replaces the
defaults rather than extending them. Omitting it evaluates, renders and
starts, and silently ends every operator session on the swarm UI, which
uses this same endpoint.

Unconditional rather than keyed to whichever service is scraped today:
this makes a scheme available, not an authorisation. Authelia refuses a
token carrying no audience for the requested URL and only issues a client
audiences it is registered for, so nothing passes until a client is
registered against a specific URL.
2026-08-24 12:06:51 +02:00
atlas
ae129835ae feat(#3517): publish forgejo's metrics behind the gateway
Forgejo can serve prometheus metrics but nothing turned them on, and
turning them on alone would have published them: forgejo serves
`/metrics` on its normal listener and the gateway vhost proxies `/` to
that listener, so the existing catch-all would have carried the endpoint
to anyone. The option is therefore one switch for both halves, and the
`= /metrics` location is an exact match so it outranks that prefix.

Authentication is the gateway's rather than forgejo's own `[metrics]
TOKEN`: a scraper presents an audience-scoped authelia token which nginx
checks via auth_request, so the swarm keeps one identity system instead
of gaining a static bearer per service.

The subrequest deliberately omits the `error_page 401 =302` that
swarm-ui uses. That redirect sends a browser to a login page; a scraper
would follow it and parse HTML as metrics.
2026-08-24 12:06:51 +02:00
2 changed files with 127 additions and 14 deletions

View file

@ -450,15 +450,54 @@ in
"${cfg.domain}" = (gatewayCfg.lib.tlsFor cfg.domain) // {
listen = gatewayCfg.lib.listen;
extraConfig = gatewayCfg.lib.securityHeaders;
locations."/" = {
proxyPass = "http://127.0.0.1:${toString cfg.httpPort}/";
proxyWebsockets = true;
extraConfig = ''
proxy_buffering off;
client_max_body_size 1G;
proxy_read_timeout 1h;
proxy_send_timeout 1h;
'';
locations = {
"/" = {
proxyPass = "http://127.0.0.1:${toString cfg.httpPort}/";
proxyWebsockets = true;
extraConfig = ''
proxy_buffering off;
client_max_body_size 1G;
proxy_read_timeout 1h;
proxy_send_timeout 1h;
'';
};
# ⚠️ EXACT match, and that is what makes this safe. Forgejo
# serves `/metrics` on the same listener the `/` prefix above
# already proxies, so without a more specific location the
# endpoint would ride that catch-all to anyone. `= /metrics`
# outranks the `/` prefix in nginx, so this location — and its
# auth — is the one that runs.
"= /metrics" = {
proxyPass = "http://127.0.0.1:${toString cfg.httpPort}/metrics";
extraConfig = ''
auth_request /__forge_metrics_authz;
'';
};
# The subrequest. Same implementation and header set as
# `swarm-ui.nix` uses, for the same reason: `X-Original-URL`
# and `X-Original-Method` are what authelia's `auth-request`
# implementation reads.
#
# ⚠️ NO `error_page 401 =302` here, and its absence is the
# whole point. The swarm UI redirects an unauthenticated
# browser to a login page; a scraper handed that 302 would
# follow it and parse an HTML page as metrics. A machine-facing
# location lets the 401 reach the client unchanged.
"= /__forge_metrics_authz" = {
proxyPass = "http://127.0.0.1:${toString autheliaCfg.port}/api/authz/auth-request";
extraConfig = ''
internal;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header X-Original-Method $request_method;
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
'';
};
};
};
};
@ -674,6 +713,20 @@ in
DEFAULT_BRANCH = "main";
DEFAULT_PRIVATE = "private";
};
# Not an option: a swarm-integrated, auto-deployed forge
# always has metrics. Tied to `behindGateway` because that
# IS the swarm-integrated shape — it is the condition under
# which the protected `= /metrics` location below exists.
# Serving the endpoint without that location would put it on
# a listener `openFirewall` can expose, with nothing in
# front of it.
#
# No `TOKEN` here on purpose. Forgejo can guard this itself
# with a static bearer, but the swarm authenticates the
# scraper at the gateway, so a second credential system per
# service would buy nothing and would be the one that stops
# getting rotated.
metrics.ENABLED = cfg.behindGateway;
# Repo migrations / pull-mirrors fetch from the source
# URL *inside* Forgejo. hyperhive code is synced from
# `localhost` (and the host LAN), which Forgejo's

View file

@ -42,6 +42,7 @@ let
hyperhiveDomain = hyperhiveCfg.domain;
swarmDomain = hyperhiveCfg.swarm.domain;
uiCfg = hyperhiveCfg.swarm.ui;
forgeCfg = hyperhiveCfg.swarm.forge;
# Group an account must hold to reach operator-only surfaces. Named
# here because this module writes the rule that enforces it and
@ -1078,6 +1079,35 @@ in
settings = {
theme = "dark";
server.address = "tcp://127.0.0.1:${toString cfg.port}";
# Let a machine present an OAuth2 access token to the same
# `auth_request` endpoint browsers use, so a scraper can be
# authenticated by the gateway instead of every service
# growing its own static bearer.
#
# ⚠️ `authn_strategies` REPLACES the defaults rather than
# adding to them, so `CookieSession` is listed explicitly.
# Dropping it does not fail to evaluate and does not fail to
# start — it silently ends every operator session on the
# swarm UI, which rides this same endpoint.
#
# Unconditional, and not keyed to whichever service is
# currently scraped: this only makes a *scheme* available.
# Authorisation is the audience — authelia refuses a token
# that carries no audience for the requested URL, and a
# client may only be issued audiences it is registered for.
# So enabling the scheme grants nobody anything until a
# client is registered for a specific URL.
server.endpoints.authz.auth-request = {
implementation = "AuthRequest";
authn_strategies = [
{
name = "HeaderAuthorization";
schemes = [ "Bearer" ];
}
{ name = "CookieSession"; }
];
};
log.level = "info";
# `watch` is load-bearing, not a convenience: authelia reads
@ -1105,11 +1135,41 @@ in
# the account to disagree silently.
access_control = {
default_policy = "one_factor";
rules = lib.optional uiCfg.enable {
domain = uiCfg.domain;
subject = [ "group:${operatorGroup}" ];
policy = "one_factor";
};
# ⚠️ ORDER MATTERS — authelia takes the FIRST matching rule.
# The metrics rule is listed first so it cannot be shadowed
# by a broader domain rule added later.
rules =
# The forge's metrics endpoint. `deny` is deliberate and
# is the whole protection right now: the endpoint is
# always served (a swarm-integrated forge always has
# metrics), and `default_policy` is `one_factor`, which
# means *any* authenticated subject — every operator
# today, every agent once they hold authelia accounts.
#
# Being reachable by a Bearer token is not sufficient on
# its own: `authn_strategies` on this endpoint also
# accepts `CookieSession`, and a cookie carries no
# audience, so the audience is not what stands between a
# browser session and this data.
#
# The collector gets in by REPLACING this with a
# client-scoped allow (`subject = ["oauth2:client:<id>"]`)
# once such a client is registered. Denying until then is
# what makes publishing the endpoint safe on its own —
# authelia refuses a subject naming a client that is not
# registered, and it does so in a `preStart` validator,
# so naming one early takes the whole SSO service down on
# the next restart rather than failing the build.
lib.optional forgeCfg.behindGateway {
domain = forgeCfg.domain;
resources = [ "^/metrics$" ];
policy = "deny";
}
++ lib.optional uiCfg.enable {
domain = uiCfg.domain;
subject = [ "group:${operatorGroup}" ];
policy = "one_factor";
};
};
# The cookie domain is the SWARM's domain, NOT authelia's