vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

swarm-nats: forward the queue container's journal to the swarm collector

Browse source 
The swarm-nats container now runs its own otel collector, which reads the
container's journal and pushes it to the swarm collector at
https://<swarm.otel.domain>/swarm-nats, authenticated as the queue's
existing OIDC client (swarm.nats.clientId). Records carry
service.name=nats and no hive label.

The collector lives in a reusable container module,
nix/container-modules/swarm-journal-forwarder.nix, which follows
swarm-bao's forwarder: a journald receiver over /var/log/journal with
merge and a file_storage cursor, an otlphttp exporter with an
oauth2client authenticator that requests its own client id as audience,
self-metrics on a loopback telemetry port, and the client secret handed
in through LoadCredential. Unlike swarm-bao, the secret is fetched inside
the container by swarm-journal-forwarder-oidc.service, under the
container's own store identity (the swarm-nats-auth leaf, which already
holds a read grant on secret/swarm/services/swarm-nats/oidc/client).
The fetch retries until the store answers, and the collector restarts
until the file exists. A 10-minute timer re-reads the secret and restarts
the collector when it changed.

The swarm collector's authenticated producer route was hard-wired to
swarm-bao. It is now rendered from swarm.otel.serviceProducers
(internal), one receiver, oidc authenticator, gateway location and
pipeline entry per service: swarm-bao (port 4391, unchanged names) and
swarm-nats (port 4392). The reserved-fragment and port-collision
assertions cover every entry.

The queue's OIDC client registration gains its own id as audience and
RS256-signed access tokens, which the receiver verifies offline.

swarm-nats drops --link-journal=host, so its journal stays in the guest
where the forwarder reads it, as swarm-bao did in 44009dc51. The new
swarm.nats.otel.telemetryPort (8891) keeps the forwarder off the
collectors' 8888-8890 in the shared host netns, with an assertion.

module-eval-nats-authelia gains four cases: the container's forwarder
and fetch unit on the store host and on a queue-only host, the
responder role's read grant, the swarm collector's swarm-nats receiver
on a host without the queue, and the client registration.

Part of #4499
Part of #5342
This commit is contained in:
atlas 2026-10-16 02:04:02 +02:00 • committed by mara
commit 40a2bd047b
9 changed files with 660 additions and 105 deletions
Download patch file Download diff file Expand all files Collapse all files

56
nix/host-modules/swarm-otel-service.nix
View file

@ -166,6 +166,48 @@ in
'';
};
serviceProducers = lib.mkOption {
internal = true;
type = lib.types.attrsOf (
lib.types.submodule {
options = {
clientId = lib.mkOption {
type = lib.types.str;
description = ''
The producer's OAuth2 client id: the one audience its
receiver's `oidc/<name>` authenticator accepts, and the one
the forwarder requests.
'';
};
port = lib.mkOption {
type = lib.types.port;
description = ''
Loopback port of the producer's receiver on the collector's
host. In the port-collision assertion with every other
receiver there.
'';
};
};
}
);
default = { };
description = ''
Swarm services whose own container forwards to this collector, keyed
by the route name: each gets a receiver `otlp/<name>`, an
authenticator `oidc/<name>` and a gateway location
`https://''${domain}/<name>/`, and its records land in the swarm
tier's pipelines with no `hive` label.
A receiver per producer because an `oidc` authenticator checks one
audience, and each service mints its token as its own client.
Declared below for every host, under no `enable`: the collector's
host renders the receiver whether or not the service runs there.
Every name must contain a fragment of
`nix/reserved-hive-fragments.nix`, so no hive can own its components.
'';
};
storeProducerName = lib.mkOption {
type = lib.types.str;
readOnly = true;
@ -346,6 +388,20 @@ in
};
};
config.services.hyperhive.swarm.otel.serviceProducers = {
# The forwarder in ./swarm-bao.nix.
${cfg.storeProducerName} = {
clientId = baoCfg.otel.clientId;
port = cfg.storeProducerPort;
};
# The forwarder in ./swarm-nats.nix, which addresses this route by the
# queue container's name.
swarm-nats = {
clientId = hyperhiveCfg.swarm.nats.clientId;
port = lib.mkDefault 4392;
};
};
# Not under any `enable`: see `publishedScrapeTargets` for why every host
# declares every entry.
#