swarm-otel: deliver the OIDC client secret through the secret store

The swarm collector's OIDC client secret only existed where authelia
did: `swarm-otel-oidc-secret.service` copied the minted plaintext out
of authelia's container tree, reachable only because the two share a
host's network namespace. A swarm that placed authelia elsewhere
delivered nothing, and the option's own description said so —
"a deployment that places authelia elsewhere points this at a file it
delivers itself." Same gap as #3853 and #4234, and this is the
swarm-otel twin of #4234's fix for Grafana.

Mirrors PR #4361 (Grafana) almost exactly:

- `swarm-bao-otel-oidc.service` reads
  `swarm/services/<client-id>/oidc/client` out of the store, in every
  deployment, replacing the co-located copy unit outright — one
  delivery route, not two, per the ruling that landed under #4234.
- Client registration moved out of `swarm-otel.nix`'s own `config`
  block (gated on this host running the collector) into
  `glue-swarm-otel-oidc-client.nix` (gated on this host running
  authelia), the same split `glue-grafana-oidc-client.nix` made. It
  was broken the same way: a split deployment registered the client
  nowhere at all, so authelia never minted a secret for the publisher
  to send on.
- The publisher's `services` prefix (write grant in `swarm-bao.nix`,
  hive read grant in `policy::render`) already covers any service's
  path — nothing to add there. `swarm-secret-publisher.nix` only grew
  `serviceClientIds` by one entry.

One judgement call, stated rather than buried: the store-reading unit
renders only where this host holds a client identity
(`deploy.bao.clientCertFile`/`clientKeyFile`), rather than asserting
it the way `swarm-grafana.nix` does. Grafana's local login form is
disabled unconditionally, so a Grafana with no OIDC secret has no way
in at all — that earns a hard refusal. This collector without a
credential still receives every hive's telemetry; only its own pushes
to the stores go out unauthenticated and get refused there, an
already-supported degrade the module's own `haveCollectorSecret` flag
named before this change. So the reading unit follows the shape
`glue-matrix-bao-token.nix` and `glue-queue-agent-credential.nix` use
for their own optional readers: no unit when the identity is absent,
not a build refusal.

Fixtures mirror #4361's: `otelBaoWithAuthelia`/`otelBaoRemoteAuthelia`
are the positive pair (co-located and split, both reading through the
store), `otelNoIdentity` is the negative — no reading unit, no
assertion firing, `clientSecretFile` left null.

Refs #4258
This commit is contained in:
atlas 2026-09-13 20:23:39 +02:00 committed by mara
commit 0ff5c8110b
7 changed files with 409 additions and 148 deletions

View file

@ -504,6 +504,35 @@ let
otelSettings =
machine: machine.containers.swarm-otel.config.services.opentelemetry-collector.settings;
# The collector beside authelia, reading its own OIDC secret out of the
# store like every other collector — the cert pair here is not scenery, it
# is the arm that would catch the deleted co-located copy unit coming back.
otelBaoWithAuthelia = hive {
deploy.swarm-otel.enable = true;
deploy.authelia.enable = true;
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
};
# The same collector with the IdP on ANOTHER host and a store leaf placed by
# hand. Identical to the fixture above in everything the delivery path
# reads, which is the point.
otelBaoRemoteAuthelia = hive {
deploy.swarm-otel.enable = true;
swarm.authelia.url = "https://auth.example.invalid";
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
};
# A collector holding no store identity at all. Unlike Grafana's mirror
# image, this is not a refused shape: the collector still receives every
# hive's telemetry with nothing to push authenticated with, which is the
# already-supported degrade `haveCollectorSecret` names above the module's
# `let`. What this fixture is for is checking the reading unit itself does
# not render, rather than rendering with an env var nothing filled in.
otelNoIdentity = hive {
deploy.swarm-otel.enable = true;
swarm.authelia.url = "https://auth.example.invalid";
};
# authelia somewhere else, the credential delivered by hand. Whether this
# collector authenticates must follow the credential, never another
# service's placement.
@ -1031,6 +1060,71 @@ let
lib.elem "swarm-grafana" (clients secretPublisherHere)
&& !(lib.elem "swarm-grafana" (clients grafanaRemoteAuthelia));
}
{
# The collector's half of the same defect and the same fix: this used to
# be gated on `deploy.swarm-otel.enable`, so a split deployment
# registered the client nowhere and authelia minted nothing to publish.
name = "the swarm's collector client is registered wherever authelia runs";
ok =
let
clients = m: map (c: c.id) m.services.hyperhive.swarm.authelia.oidc.clients;
in
lib.elem "swarm-collector" (clients secretPublisherHere)
&& !(lib.elem "swarm-collector" (clients otelBaoRemoteAuthelia));
}
{
# 🩸 The arm that guards the ruling this slice landed under, the
# collector's half of ./swarm-grafana.nix's own. There is ONE delivery
# route: the store reader, on every host that runs the collector and
# holds a store identity. The negative names the deleted unit rather
# than a generic absence, because the way this regresses is someone
# re-adding the co-located copy as an optimisation.
name = "the collector's OIDC secret has exactly one delivery unit, the store reader, in both topologies";
ok =
let
local = otelBaoWithAuthelia.systemd.services;
remote = otelBaoRemoteAuthelia.systemd.services;
in
local ? swarm-bao-otel-oidc
&& remote ? swarm-bao-otel-oidc
&& !(local ? swarm-otel-oidc-secret)
&& !(remote ? swarm-otel-oidc-secret);
}
{
# Same 403-not-a-miss reason as grafana's arm above: the reader's grant
# covers the `services` prefix, so a path outside it is refused rather
# than empty, however correct it reads.
name = "the collector's OIDC secret is read from the prefix the publisher writes";
ok =
let
s = otelBaoRemoteAuthelia.systemd.services.swarm-bao-otel-oidc.script;
in
lib.hasInfix "secret/swarm/services/swarm-collector/oidc/client" s
&& !(lib.hasInfix "secret/swarm/hives/" s);
}
{
# Both ends of a wire nothing at eval time carries end to end: the
# publisher on authelia's host writes the path the reader on the
# collector's host reads, and the two files agree only because both
# compose it from the same swarm-wide client id. `secretPublisherHere`
# already grew this client when `serviceClientIds` did.
name = "the publisher writes the swarm service path the collector reads";
ok = lib.hasInfix "secret/swarm/services/swarm-collector/oidc/client" (
secretPublisherHere.systemd.services.swarm-secret-publish.script
);
}
{
# The collector's non-assertion, the deliberate mirror of Grafana's
# assertion two cases up: a host with no store identity is a supported,
# merely degraded shape here, so the reading unit simply does not exist
# rather than refusing the build. `haveCollectorSecret` is what the
# degrade already reads, unchanged by this slice.
name = "a collector with no store identity renders no reading unit, and is not refused";
ok =
!(otelNoIdentity.systemd.services ? swarm-bao-otel-oidc)
&& otelNoIdentity.services.hyperhive.deploy.swarm-otel.clientSecretFile == null
&& !(lib.any (a: !a.assertion) otelNoIdentity.assertions);
}
{
# Reads the daemon's rendered unit, not the options: the queue address
# arrives as an env var whose whole attrset is guarded on `natsUrl`, and