Watch
0
0
Fork
You've already forked hyperhive
0

nix: ship the journals of the units an apply can leave failed

The units on the path a deploy takes to TLS, the store's grants and the
swarm collector itself were not on the host collector's journald
allowlist, so an ingest outage one of them caused showed in the store
only as every source going quiet at once.

Each module names its own units, per the option's rule:
- hive-tls.nix: hive-tls-ca, swarm-services-cert
- hive-gateway: hive-gateway-self-signed-cert (self-signed mode only)
- swarm-bao.nix: the seven grant units beside
  swarm-bao-services-issuer-policy
- swarm-otel.nix: container@<machine>, and
  nixos-rebuild-switch-to-configuration, the transient unit nixos-rebuild
  runs the activation in and whose syslog lines carry its status

The module-eval arm pins each unit as both listed and defined, since a
listed name that matches nothing is silent.
This commit is contained in:
atlas 2026-09-24 13:07:44 +02:00 • committed by mara
commit aa719da571
5 changed files with 77 additions and 3 deletions

View file

@ -61,7 +61,54 @@ let
deploy.bao.otelOidcClientCertFile = "/etc/pki/bao-otel-oidc.pem";
deploy.bao.otelOidcClientKeyFile = "/etc/pki/bao-otel-oidc-key.pem";
};
# The collector beside the store holding a bootstrap token: the one shape in
# which every unit an apply can leave failed renders on the same host.
otelApplyPath = hive {
deploy.swarm-otel.enable = true;
deploy.authelia.enable = true;
deploy.bao.enable = true;
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
};
# Units on the path a deploy takes to TLS, the store's grants and the
# collector itself. A failure among them silences ingest, so without their
# journals the store can show that ingest stopped but not which unit
# stopped it.
applyPathUnits = [
"container@swarm-otel"
"hive-tls-ca"
"swarm-services-cert"
"hive-gateway-self-signed-cert"
"swarm-bao-controller-policy"
"swarm-bao-secret-publisher-policy"
"swarm-bao-matrix-ctl-policy"
"swarm-bao-matrix-token-policy"
"swarm-bao-queue-agent-policy"
"swarm-bao-grafana-oidc-policy"
"swarm-bao-otel-oidc-policy"
"swarm-bao-services-issuer-policy"
];
cases = [
{
# Listed AND defined, because a name that matches nothing is not an
# error anywhere: a unit renamed out from under its entry would pass a
# membership check and still never reach the store.
name = "every unit on the apply path is defined and ships its journal";
ok =
let
m = otelApplyPath;
in
lib.all (
u: builtins.elem u m.services.hyperhive.swarm.otel.journaldUnits && m.systemd.services ? ${u}
) applyPathUnits;
}
{
# Transient, so nothing here defines it and only membership can be
# pinned: nixos-rebuild names the unit it runs the activation in.
name = "the activation's journal ships beside the units it starts";
ok = builtins.elem "nixos-rebuild-switch-to-configuration" otelApplyPath.services.hyperhive.swarm.otel.journaldUnits;
}
{
# 🩸 The arm that guards the ruling this slice landed under, the
# collector's half of ./swarm-grafana.nix's own. There is ONE delivery