fix(#3125): the swarm tier's self-metrics must not fight the hive tier's

A collector serves its own metrics on localhost:8888 unless told
otherwise, and co-located tiers share a network namespace, so the second
one to start dies with 'bind: address already in use'.

The port appears in neither config - it is a default inside the binary -
so comparing the ports the configs name reports them distinct. A
behavioural probe found it by being unable to start the chain.

metrics.address is the spelling that looks right and is rejected by this
version ('migration.MetricsConfigV030' has invalid keys: address);
readers is the schema it accepts.
This commit is contained in:
atlas 2026-08-18 13:24:59 +02:00 committed by mara
commit 636dba8d61

View file

@ -77,6 +77,26 @@ in
'';
};
telemetryPort = lib.mkOption {
type = lib.types.port;
default = 8889;
description = ''
Port this collector serves its **own** metrics on queue depth,
refused and dropped samples, exporter failures. How you find out
that telemetry is being lost, so it is worth keeping rather than
switching off.
**Deliberately not 8889's neighbour 8888**, which is the
collector's built-in default and therefore what the hive tier
already binds. Two collectors share a network namespace whenever
they are co-located, and unlike the OTLP port this one appears
nowhere in either config it is a default inside the binary, so
nothing that compares configured ports can see the clash. The
second collector to start simply dies with
`bind: address already in use`.
'';
};
url = lib.mkOption {
type = lib.types.str;
default = "http://127.0.0.1:${toString cfg.port}";
@ -185,6 +205,23 @@ in
// lib.optionalAttrs (otelCfg.protocol == "http/json") { encoding = "json"; };
};
# Moves this collector's self-metrics off the built-in
# default of `localhost:8888`, which the hive tier holds.
#
# ⚠️ `metrics.address` is the spelling that looks right and
# is REJECTED by this collector version — measured, not
# read: `'migration.MetricsConfigV030' has invalid keys:
# address`. `readers` is the schema it accepts, and the
# difference is a startup failure rather than a warning.
service.telemetry.metrics.readers = [
{
pull.exporter.prometheus = {
host = "127.0.0.1";
port = cfg.telemetryPort;
};
}
];
service.pipelines.metrics = {
receivers = [ "otlp" ];
# Fan-out, not a choice: with both configured the same