vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

otel: the hive tier presents its own identity to the swarm collector

Browse source 
The receiving half authenticates per hive, so this half has to prove which
hive it is. It mints a token against the swarm's authelia with this hive's
client and posts to that hive's path on the collector's gateway name.

Holding a credential is what decides whether this tier authenticates —
`clientSecretFile` non-null — rather than a second switch that could
disagree with it. The default is the secret this host's own authelia
minted, which is right exactly when the IdP runs here; a hive that is not
that host names wherever the file landed, the same manual-copy shape the
identities option already documents as unsolved.

Two things that a diff will not explain:

`endpoint_params.audience` is not redundant with the client's registered
audience. Registering only makes an audience permissible; a token minted
without asking for one carries `aud: []` and every receiver refuses it,
with a config that reads correctly at both ends.

`client_secret_file` keeps the secret out of nix altogether — the
collector opens the file itself. It is a real key of this extension,
checked against the shipped binary with a deliberate typo rejected in the
same run, so "accepted" is distinguishable from "ignores everything". The
path comes from systemd's `CREDENTIALS_DIRECTORY`, so nothing hardcodes a
`/run/credentials` layout.

An assertion covers the one deployment where this can go wrong silently:
a host running both tiers with ingest authenticated and no credential to
present would 401 against a collector on the same machine.
This commit is contained in:
atlas 2026-08-19 14:53:10 +02:00 • committed by mara
commit cb787997bd
1 changed files with 118 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

120
nix/host-modules/otel.nix
View file

@ -193,6 +193,40 @@ in
'';
};
clientSecretFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default =
if
config.services.hyperhive.swarm.authelia.enable && config.services.hyperhive.hiveName != null
then
"${config.services.hyperhive.swarm.authelia.hostClientSecretDir}/"
+ "${config.services.hyperhive.swarm.authelia.hiveClientPrefix}${config.services.hyperhive.hiveName}.secret"
else
null;
defaultText = lib.literalExpression ''"''${swarm.authelia.hostClientSecretDir}/''${swarm.authelia.hiveClientPrefix}''${hiveName}.secret" when this host runs the swarm's IdP, else null'';
example = "/var/lib/secrets/hive-telemetry.secret";
description = ''
Absolute path to this hive's OAuth2 client secret, used to
authenticate to the swarm's collector as this hive.
**Whether this is set is what decides whether this tier
authenticates at all.** A credential is the only thing that makes
authenticated egress possible, so its presence is the condition
rather than a second switch that could disagree with it.
Defaults to the secret this host's own authelia minted, which is
correct exactly when the swarm's IdP runs here. On a hive that is
not that host, the file has to arrive some other way and this
option names wherever it landed — the same manual-copy shape
`services.hyperhive.swarm.authelia.oidc.hiveIdentities` documents,
where delivering a secret to a hive that is not this host is
deliberately not solved.
Read by `LoadCredential`, so it is never evaluated by nix, never
copied into the store and never passed in argv.
'';
};
metricIntervalMs = lib.mkOption {
type = lib.types.nullOr lib.types.ints.positive;
default = null;
@ -214,8 +248,28 @@ in
config = lib.mkIf config.services.hyperhive.otel.enable (
let
otel = config.services.hyperhive.otel;
autheliaCfg = config.services.hyperhive.swarm.authelia;
swarmOtelCfg = config.services.hyperhive.swarm.otel;
hiveName = config.services.hyperhive.hiveName;
listen = "${config.services.hyperhive.network.bridgeIp}:${toString otel.collector.port}";
swarmName = "otlphttp/swarm";
authName = "oauth2client/swarm";
# Holding a credential IS the condition — see `clientSecretFile`.
senderAuth = otel.clientSecretFile != null && hiveName != null;
# This hive's client id, and also the audience it must ASK for. Both
# are `hiveClientPrefix` + the hive's name because that is the one
# name the swarm already agrees on; the receiver one tier up derives
# the same string.
hiveClient = "${autheliaCfg.hiveClientPrefix}${toString hiveName}";
# systemd exports `CREDENTIALS_DIRECTORY` to any unit with
# `LoadCredential`, and the collector expands `${env:…}` at load. So
# the secret reaches the process as a PATH resolved at runtime — nix
# renders neither the value nor the directory, and nothing has to
# hardcode `/run/credentials/<unit>`.
credName = "swarm-client.secret";
in
{
# Reachable from agent containers and nowhere else: this opens
@ -279,15 +333,77 @@ in
# change. `https://` because that name resolves through the
# gateway even on a co-located host — see `caTrust` above for
# the trust half that makes this verify.
endpoint = "https://${config.services.hyperhive.swarm.otel.domain}";
};
# The hive's own path under the collector's single name. The
# swarm tier gives each hive its own authenticated receiver and
# routes to it by this prefix, so the path is not decoration —
# it selects WHICH receiver, and therefore which hive the
# samples get labelled as.
endpoint = "https://${swarmOtelCfg.domain}" + lib.optionalString senderAuth "/${toString hiveName}";
}
// lib.optionalAttrs senderAuth { auth.authenticator = authName; };
# ⚠️ An extension not listed here is INERT: the collector starts
# clean and the exporter naming it sends nothing authenticated.
service.extensions = lib.optional senderAuth authName;
service.pipelines.metrics = {
receivers = [ "otlp" ];
exporters = [ swarmName ];
};
}
// lib.optionalAttrs senderAuth {
extensions.${authName} = {
client_id = hiveClient;
# A real key, measured against this collector version rather
# than assumed — with a deliberate typo rejected in the same
# run, so "accepted" is distinguishable from "ignores
# everything". Keeps the secret out of nix entirely: the
# collector opens the file itself.
client_secret_file = "\${env:CREDENTIALS_DIRECTORY}/${credName}";
token_url = "${toString autheliaCfg.url}/api/oidc/token";
# ⚠️ THE AUDIENCE HAS TO BE REQUESTED, not merely granted.
# Registering it on the client only makes it permissible; a
# token minted without asking carries `aud: []` and every
# receiver refuses it — with a config that reads perfectly at
# both ends. Measured against authelia 4.39.20.
endpoint_params.audience = hiveClient;
};
};
};
# `LoadCredential` and not a copy-oneshot: this collector is a HOST
# unit, so there is no container boundary to cross and therefore no
# reason for a second on-disk copy of the secret. systemd hands it to
# the process in a private tmpfs and exports the directory, which is
# what the config above names.
systemd.services.opentelemetry-collector.serviceConfig = lib.optionalAttrs senderAuth {
LoadCredential = [ "${credName}:${otel.clientSecretFile}" ];
};
assertions = [
{
# Only checkable on a host that runs BOTH tiers — which is the
# deployment where it can actually go wrong silently. A remote
# hive cannot see the swarm tier's config at all, so its operator
# sets the path explicitly and this says nothing.
assertion = !(swarmOtelCfg.enable && swarmOtelCfg.requireHiveIdentity) || senderAuth;
message = ''
This host runs the swarm's telemetry collector with
services.hyperhive.swarm.otel.requireHiveIdentity = true, so
ingest is authenticated per hive — but this hive's own
collector has no credential to present:
services.hyperhive.otel.clientSecretFile = ${
if otel.clientSecretFile == null then "null" else otel.clientSecretFile
}
services.hyperhive.hiveName = ${if hiveName == null then "null" else hiveName}
Its samples would be refused with a 401 by the collector
running beside it. Set both, or set requireHiveIdentity =
false to accept unauthenticated ingest.
'';
}
];
}
);
}