vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

swarm-nats: forward the queue container's journal to the swarm collector

Browse source 
The swarm-nats container now runs its own otel collector, which reads the
container's journal and pushes it to the swarm collector at
https://<swarm.otel.domain>/swarm-nats, authenticated as the queue's
existing OIDC client (swarm.nats.clientId). Records carry
service.name=nats and no hive label.

The collector lives in a reusable container module,
nix/container-modules/swarm-journal-forwarder.nix, which follows
swarm-bao's forwarder: a journald receiver over /var/log/journal with
merge and a file_storage cursor, an otlphttp exporter with an
oauth2client authenticator that requests its own client id as audience,
self-metrics on a loopback telemetry port, and the client secret handed
in through LoadCredential. Unlike swarm-bao, the secret is fetched inside
the container by swarm-journal-forwarder-oidc.service, under the
container's own store identity (the swarm-nats-auth leaf, which already
holds a read grant on secret/swarm/services/swarm-nats/oidc/client).
The fetch retries until the store answers, and the collector restarts
until the file exists. A 10-minute timer re-reads the secret and restarts
the collector when it changed.

The swarm collector's authenticated producer route was hard-wired to
swarm-bao. It is now rendered from swarm.otel.serviceProducers
(internal), one receiver, oidc authenticator, gateway location and
pipeline entry per service: swarm-bao (port 4391, unchanged names) and
swarm-nats (port 4392). The reserved-fragment and port-collision
assertions cover every entry.

The queue's OIDC client registration gains its own id as audience and
RS256-signed access tokens, which the receiver verifies offline.

swarm-nats drops --link-journal=host, so its journal stays in the guest
where the forwarder reads it, as swarm-bao did in 44009dc51. The new
swarm.nats.otel.telemetryPort (8891) keeps the forwarder off the
collectors' 8888-8890 in the shared host netns, with an assertion.

module-eval-nats-authelia gains four cases: the container's forwarder
and fetch unit on the store host and on a queue-only host, the
responder role's read grant, the swarm collector's swarm-nats receiver
on a host without the queue, and the client registration.

Part of #4499
Part of #5342
This commit is contained in:
atlas 2026-10-16 02:04:02 +02:00 • committed by mara
commit 40a2bd047b
9 changed files with 660 additions and 105 deletions
Download patch file Download diff file Expand all files Collapse all files

18
nix/host-modules/swarm-nats-service.nix
View file

@ -107,6 +107,24 @@ in
'';
};
otel.telemetryPort = lib.mkOption {
type = lib.types.port;
default = 8891;
description = ''
Loopback port the journal forwarder inside the queue's container
serves its own metrics on (refused and dropped records, exporter
failures).
⚠️ Not 8888, 8889 or 8890: the hive collector, the swarm collector
and the store's forwarder bind those
({option}`services.hyperhive.otel.telemetryPort`,
{option}`services.hyperhive.swarm.otel.telemetryPort`,
{option}`services.hyperhive.swarm.bao.otel.telemetryPort`), and this
container shares the host netns with them when co-located. The second
collector to bind a port dies with `bind: address already in use`.
'';
};
clientId = lib.mkOption {
type = lib.types.str;
default = "swarm-nats";