swarm-builder: export build counts, durations, queue wait and publish failures through the host's collector
The builder pushes its build metrics (builds by kind and outcome, build duration, queue depth and wait, reply publish failures) to its forwarder's OTLP receiver, which relays them to the swarm collector. It also registers the standard store client-certificate gauges (bao.client_cert.*, cert=builder) on the same exporter. They are the builder's liveness signal: exported every minute whether or not a build runs. The cert-metrics table in docs/scheduler/observability.md gains the builder row.
This commit is contained in:
parent
3b8f7ff630
commit
e30f5849d5
1 changed files with 740 additions and 32 deletions
|
|
@ -188,6 +188,19 @@ let
|
|||
&& svc.serviceConfig.DynamicUser
|
||||
&& !(lib.any (lib.hasSuffix "TOKEN") (lib.attrNames env));
|
||||
}
|
||||
{
|
||||
# The daemon pushes its build metrics to the forwarder's OTLP receiver on
|
||||
# the container's loopback, which relays them to the swarm collector under
|
||||
# the builder's own client.
|
||||
name = "the builder exports its metrics through the forwarder's OTLP receiver on the container's loopback";
|
||||
ok =
|
||||
let
|
||||
f = inner.services.hyperhive.swarmJournalForwarder;
|
||||
in
|
||||
f.otlpMetricsPort != null
|
||||
&& env.OTEL_EXPORTER_OTLP_ENDPOINT == "http://127.0.0.1:${toString f.otlpMetricsPort}"
|
||||
&& lib.hasInfix "service.name=hyperhive-swarm-builder" env.OTEL_RESOURCE_ATTRIBUTES;
|
||||
}
|
||||
{
|
||||
# The closure is pushed with `attic` from the unit, whose config file holds
|
||||
# the push token: it goes in a runtime directory only the dynamic user can
|
||||
|
|
|
|||
Loading…
Reference in a new issue