The swarm-nats container now runs its own otel collector, which reads the
container's journal and pushes it to the swarm collector at
https://<swarm.otel.domain>/swarm-nats, authenticated as the queue's
existing OIDC client (swarm.nats.clientId). Records carry
service.name=nats and no hive label.
The collector lives in a reusable container module,
nix/container-modules/swarm-journal-forwarder.nix, which follows
swarm-bao's forwarder: a journald receiver over /var/log/journal with
merge and a file_storage cursor, an otlphttp exporter with an
oauth2client authenticator that requests its own client id as audience,
self-metrics on a loopback telemetry port, and the client secret handed
in through LoadCredential. Unlike swarm-bao, the secret is fetched inside
the container by swarm-journal-forwarder-oidc.service, under the
container's own store identity (the swarm-nats-auth leaf, which already
holds a read grant on secret/swarm/services/swarm-nats/oidc/client).
The fetch retries until the store answers, and the collector restarts
until the file exists. A 10-minute timer re-reads the secret and restarts
the collector when it changed.
The swarm collector's authenticated producer route was hard-wired to
swarm-bao. It is now rendered from swarm.otel.serviceProducers
(internal), one receiver, oidc authenticator, gateway location and
pipeline entry per service: swarm-bao (port 4391, unchanged names) and
swarm-nats (port 4392). The reserved-fragment and port-collision
assertions cover every entry.
The queue's OIDC client registration gains its own id as audience and
RS256-signed access tokens, which the receiver verifies offline.
swarm-nats drops --link-journal=host, so its journal stays in the guest
where the forwarder reads it, as swarm-bao did in 44009dc51. The new
swarm.nats.otel.telemetryPort (8891) keeps the forwarder off the
collectors' 8888-8890 in the shared host netns, with an assertion.
module-eval-nats-authelia gains four cases: the container's forwarder
and fetch unit on the store host and on a queue-only host, the
responder role's read grant, the swarm collector's swarm-nats receiver
on a host without the queue, and the client registration.
Part of #4499
Part of #5342
The three client registrations move into glue-{controller,nats,matrix}-oidc-client.nix,
gated only on deploy.authelia.enable and reading their ids from swarm.* options,
so the host running authelia registers every one of them whichever host runs
the service. The matrix callback is a new read-only swarm.matrix.sso.redirectUri,
read by both the homeserver's config and its client entry.
swarm-secret-publish now also publishes the queue's and the homeserver's minted
secrets to swarm/services/<id>/oidc/client. The queue host reads its responder's
secret there with swarm-bao-nats-auth-oidc under the responder's own leaf
(swarm-nats-auth role gains that one read path), and the homeserver's host with
swarm-bao-matrix-oidc under matrix-ctl's leaf (swarm-matrix-ctl role gains its
one read path). The host-file copies are gone: swarm-nats-auth-secrets now
delivers only the callout seeds and the store identity, and
hive-matrix-oidc-secret is removed. A fetch unit whose leaf is unset still
renders and fails naming the option pair.
Closes#4943
