fix(#3554): push to the swarm's stores by domain, authenticated
The collector's store exporters were gated on deploy.victoriametrics.enable /
deploy.victorialogs.enable — options that say "this host RUNS the store", not
"the swarm has one". A collector that did not share a host with the stores
rendered no exporter at all and dropped everything it received, from every
hive, silently: an absent exporter is not an error.
mara ruled the premise rather than the mechanism ("the swarm always has a
store"), so there is no gate and no new option for that. Both exporters are
unconditional and address the stores by domain, per the cross-host rule from
the OIDC client and secret-delivery unit #3517 already built. The logs exporter
had the identical bug and is fixed here too.
Both stores gained a machine ingest route, required in the same change: the
exporter now targets https://logs.<domain>/insert/..., and that vhost is
browser-shaped, so shipping the collector half alone would have regressed logs
ingestion that works today. Neither ingest location carries
`error_page 401 =302` — a pusher handed a redirect follows it and POSTs at a
login page, which answers 200.
Whether the collector authenticates follows the CREDENTIAL, never another
service's placement: `clientSecretFile` is a nullable option, and the delivery
unit — the one thing here that may know where authelia runs, since it copies
out of its container — sets it by mkDefault. An earlier revision gated this on
deploy.authelia.enable directly, which put a different service's co-location in
the collector's own config.
Also removed rather than relaxed: the assertion that this collector has
"somewhere to send". It read the store's per-host enable, so it rejected at
eval exactly the deployment reaching the stores by domain exists for.
Deliberately not replaced with an authentication assertion — a collector on a
host of its own is a supported shape, and refusing to build it would make this
fix illegal where the bug bites hardest.
Knock-on worth review: collectLogs is now always satisfied, so journald
collection is unconditional.
Config shape validated against otelcol-contrib 0.151.0 `validate`, with a
bogus-key control confirming the validator checks the extension schema.
module-eval: 31 properties.
This commit is contained in:
parent
a3612f5168
commit
5478e0bf67
4 changed files with 446 additions and 181 deletions
|
|
@ -21,6 +21,7 @@ let
|
|||
networkCfg = config.services.hyperhive.network;
|
||||
hyperhiveCfg = config.services.hyperhive;
|
||||
gatewayCfg = hyperhiveCfg.gateway;
|
||||
autheliaCfg = hyperhiveCfg.swarm.authelia;
|
||||
swarmDomain = hyperhiveCfg.swarm.domain;
|
||||
|
||||
# Total on a null swarm domain for the same reason every sibling module is:
|
||||
|
|
@ -123,17 +124,58 @@ in
|
|||
# named it runs, which is what keeps scraper and target on one host by
|
||||
# construction instead of by luck.
|
||||
#
|
||||
# The loopback literal introduces no new assumption — it is the address
|
||||
# this module already pins the listener to, and the same one the
|
||||
# collector's `otlphttp/victoriametrics` exporter already writes to. If
|
||||
# that reach is ever wrong, it is wrong for the write path first.
|
||||
# The loopback literal is safe for this option and NOT for the write path,
|
||||
# which is the distinction that matters now that the two differ. A scrape
|
||||
# target is only ever read by a collector on this host, so loopback states
|
||||
# a fact. The collector's push goes to the swarm name through the gateway,
|
||||
# because the collector need not be here at all.
|
||||
services.hyperhive.swarm.otel.scrapeTargets.victoriametrics = "127.0.0.1:${toString cfg.port}";
|
||||
|
||||
services.nginx.virtualHosts."${cfg.domain}" = (gatewayCfg.lib.tlsFor cfg.domain) // {
|
||||
listen = gatewayCfg.lib.listen;
|
||||
extraConfig = gatewayCfg.lib.securityHeaders;
|
||||
locations."/" = {
|
||||
proxyPass = "http://127.0.0.1:${toString cfg.port}/";
|
||||
locations = {
|
||||
"/" = {
|
||||
proxyPass = "http://127.0.0.1:${toString cfg.port}/";
|
||||
};
|
||||
|
||||
# The swarm collector's ingest route, and the only authenticated thing
|
||||
# on this vhost. `=` so it outranks the `/` prefix above, which would
|
||||
# otherwise carry these writes with no check at all.
|
||||
#
|
||||
# ⚠️ No `error_page 401 =302` here, and its absence is the point: a
|
||||
# redirect is right for a browser and wrong for a pusher, which would
|
||||
# follow it and POST its batch at a login page that answers 200 —
|
||||
# ingest reporting healthy while storing nothing.
|
||||
"= /opentelemetry/api/v1/push" = {
|
||||
proxyPass = "http://127.0.0.1:${toString cfg.port}/opentelemetry/api/v1/push";
|
||||
extraConfig = ''
|
||||
auth_request /__metrics_push_authz;
|
||||
'';
|
||||
};
|
||||
|
||||
# The subrequest. Same target and header set as the sibling log
|
||||
# store's, which took them from `swarm-ui.nix` — `X-Original-URL` and
|
||||
# `X-Original-Method` are what authelia's auth-request implementation
|
||||
# reads, and the address it compares the token's audience against.
|
||||
"= /__metrics_push_authz" = {
|
||||
proxyPass = "https://${autheliaCfg.domain}/api/authz/auth-request";
|
||||
# nixpkgs appends its OWN `Host $host` after extraConfig, which
|
||||
# would override verifiedProxyTo's — see the comment on
|
||||
# verifiedProxyTo in hive-gateway/vhost-lib.nix.
|
||||
recommendedProxySettings = false;
|
||||
extraConfig = ''
|
||||
internal;
|
||||
${gatewayCfg.lib.verifiedProxyTo autheliaCfg.domain}
|
||||
proxy_pass_request_body off;
|
||||
proxy_set_header Content-Length "";
|
||||
proxy_set_header X-Original-Method $request_method;
|
||||
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_set_header X-Forwarded-Host $http_host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
'';
|
||||
};
|
||||
};
|
||||
};
|
||||
|
||||
|
|
@ -192,5 +234,8 @@ in
|
|||
# deployment ever needs them.
|
||||
#
|
||||
# ⚠️ That endpoint is unauthenticated, which is why `listenAddress` above is
|
||||
# loopback and why the collector — not agents — is the writer.
|
||||
# loopback: the only route to it from off-host is the vhost's ingest
|
||||
# location, which authenticates. The collector is still the only writer, but
|
||||
# it now arrives by the swarm name rather than over loopback, because it need
|
||||
# not share a host with this store.
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in a new issue