From 5cec69bd2193b8c2ea508695f1430bef03ba60eb Mon Sep 17 00:00:00 2001 From: atlas Date: Wed, 23 Sep 2026 17:08:20 +0200 Subject: [PATCH] otel.nix: trim the StartLimit comment block to the load-bearing points Cut ~27 lines of blackout-measurement and cross-reference narrative (already in the PR body / issue) down to the three things a reader actually needs at this call site: the [Unit]-vs-[Service] trap, why Restart is absent, and the window-vs-burst constraint. --- nix/host-modules/otel.nix | 35 ++++++----------------------------- 1 file changed, 6 insertions(+), 29 deletions(-) diff --git a/nix/host-modules/otel.nix b/nix/host-modules/otel.nix index ca9ff7a5..84e6a81a 100644 --- a/nix/host-modules/otel.nix +++ b/nix/host-modules/otel.nix @@ -627,35 +627,12 @@ in }; systemd.services.opentelemetry-collector = lib.mkMerge [ - # Survive a restart that races the outgoing process off - # `telemetryPort`. Every deploy, the replacement collector has been - # reaching `bind()` while the old one still holds `127.0.0.1:8888`, - # and nixpkgs' `Restart = "always"` with no `RestartSec` turns that - # into the worst possible response: five retries inside two seconds, - # `start-limit-hit`, and no telemetry at all until something else - # pokes the unit. Measured at ~27s of blackout per deploy. - # - # 🔑 Same inversion the credential wait below records — `always` - # reads like it makes this self-healing and does the opposite. A - # FAST-failing unit exhausts its allowance before the thing it is - # waiting for (here, an exiting process letting go of a socket) can - # possibly have happened. Nothing here removes that race; it makes - # the unit outlast it instead. - # - # Sized for a socket held by a process already on its way out — - # seconds, not an unseal's hours. 5s ends the blackout within one - # interval of the port coming free; 12 attempts ride out 55s of it - # being held, twice the worst gap observed. `Restart` itself is - # deliberately NOT set, same as both container collectors: nixpkgs' - # own module defines it at the normal priority, so a second - # definition is a conflict the module system refuses. - # - # ⚠️ `StartLimit*` are `[Unit]` settings and systemd IGNORES them - # under `[Service]` — written into `serviceConfig` they render, - # deploy and do nothing. These two options are the `[Unit]` - # spelling; `checks.module-eval-hive-otel` asserts they land in - # `unitConfig` and not beside `RestartSec`. The window has to - # exceed `RestartSec × burst`. + # ⚠️ `StartLimit*` are `[Unit]` settings; systemd IGNORES them under + # `[Service]` — in `serviceConfig` they render, deploy and do + # nothing. `checks.module-eval-hive-otel` pins them in `unitConfig`. + # `Restart` is deliberately NOT set: nixpkgs' own module defines it + # at normal priority, so a second definition is a module-system + # conflict. The window must exceed `RestartSec × burst`. { startLimitBurst = 12; startLimitIntervalSec = 120;