fix(#3554): push to the swarm's stores by domain, authenticated

The collector's store exporters were gated on deploy.victoriametrics.enable /
deploy.victorialogs.enable — options that say "this host RUNS the store", not
"the swarm has one". A collector that did not share a host with the stores
rendered no exporter at all and dropped everything it received, from every
hive, silently: an absent exporter is not an error.

mara ruled the premise rather than the mechanism ("the swarm always has a
store"), so there is no gate and no new option for that. Both exporters are
unconditional and address the stores by domain, per the cross-host rule from
the OIDC client and secret-delivery unit #3517 already built. The logs exporter
had the identical bug and is fixed here too.

Both stores gained a machine ingest route, required in the same change: the
exporter now targets https://logs.<domain>/insert/..., and that vhost is
browser-shaped, so shipping the collector half alone would have regressed logs
ingestion that works today. Neither ingest location carries
`error_page 401 =302` — a pusher handed a redirect follows it and POSTs at a
login page, which answers 200.

Whether the collector authenticates follows the CREDENTIAL, never another
service's placement: `clientSecretFile` is a nullable option, and the delivery
unit — the one thing here that may know where authelia runs, since it copies
out of its container — sets it by mkDefault. An earlier revision gated this on
deploy.authelia.enable directly, which put a different service's co-location in
the collector's own config.

Also removed rather than relaxed: the assertion that this collector has
"somewhere to send". It read the store's per-host enable, so it rejected at
eval exactly the deployment reaching the stores by domain exists for.
Deliberately not replaced with an authentication assertion — a collector on a
host of its own is a supported shape, and refusing to build it would make this
fix illegal where the bug bites hardest.

Knock-on worth review: collectLogs is now always satisfied, so journald
collection is unconditional.

Config shape validated against otelcol-contrib 0.151.0 `validate`, with a
bogus-key control confirming the validator checks the extension schema.
module-eval: 31 properties.
This commit is contained in:
atlas 2026-08-31 23:14:41 +02:00
commit 5478e0bf67
4 changed files with 446 additions and 181 deletions

View file

@ -21,6 +21,7 @@ let
networkCfg = config.services.hyperhive.network;
hyperhiveCfg = config.services.hyperhive;
gatewayCfg = hyperhiveCfg.gateway;
autheliaCfg = hyperhiveCfg.swarm.authelia;
swarmDomain = hyperhiveCfg.swarm.domain;
# Total on a null swarm domain for the same reason every sibling module is:
@ -123,17 +124,58 @@ in
# named it runs, which is what keeps scraper and target on one host by
# construction instead of by luck.
#
# The loopback literal introduces no new assumption — it is the address
# this module already pins the listener to, and the same one the
# collector's `otlphttp/victoriametrics` exporter already writes to. If
# that reach is ever wrong, it is wrong for the write path first.
# The loopback literal is safe for this option and NOT for the write path,
# which is the distinction that matters now that the two differ. A scrape
# target is only ever read by a collector on this host, so loopback states
# a fact. The collector's push goes to the swarm name through the gateway,
# because the collector need not be here at all.
services.hyperhive.swarm.otel.scrapeTargets.victoriametrics = "127.0.0.1:${toString cfg.port}";
services.nginx.virtualHosts."${cfg.domain}" = (gatewayCfg.lib.tlsFor cfg.domain) // {
listen = gatewayCfg.lib.listen;
extraConfig = gatewayCfg.lib.securityHeaders;
locations."/" = {
proxyPass = "http://127.0.0.1:${toString cfg.port}/";
locations = {
"/" = {
proxyPass = "http://127.0.0.1:${toString cfg.port}/";
};
# The swarm collector's ingest route, and the only authenticated thing
# on this vhost. `=` so it outranks the `/` prefix above, which would
# otherwise carry these writes with no check at all.
#
# ⚠️ No `error_page 401 =302` here, and its absence is the point: a
# redirect is right for a browser and wrong for a pusher, which would
# follow it and POST its batch at a login page that answers 200 —
# ingest reporting healthy while storing nothing.
"= /opentelemetry/api/v1/push" = {
proxyPass = "http://127.0.0.1:${toString cfg.port}/opentelemetry/api/v1/push";
extraConfig = ''
auth_request /__metrics_push_authz;
'';
};
# The subrequest. Same target and header set as the sibling log
# store's, which took them from `swarm-ui.nix` — `X-Original-URL` and
# `X-Original-Method` are what authelia's auth-request implementation
# reads, and the address it compares the token's audience against.
"= /__metrics_push_authz" = {
proxyPass = "https://${autheliaCfg.domain}/api/authz/auth-request";
# nixpkgs appends its OWN `Host $host` after extraConfig, which
# would override verifiedProxyTo's — see the comment on
# verifiedProxyTo in hive-gateway/vhost-lib.nix.
recommendedProxySettings = false;
extraConfig = ''
internal;
${gatewayCfg.lib.verifiedProxyTo autheliaCfg.domain}
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header X-Original-Method $request_method;
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
'';
};
};
};
@ -192,5 +234,8 @@ in
# deployment ever needs them.
#
# ⚠️ That endpoint is unauthenticated, which is why `listenAddress` above is
# loopback and why the collector — not agents — is the writer.
# loopback: the only route to it from off-host is the vhost's ingest
# location, which authenticates. The collector is still the only writer, but
# it now arrives by the swarm name rather than over loopback, because it need
# not share a host with this store.
}