credential units: restart consumers on a changed credential; fix the ordering claim
The previous commit's comments said a unit in auto-restart keeps its start job, so anything ordered after it waits for the whole 24h retry window. That is wrong under the default RestartMode=normal: each failed attempt passes through `failed`, which ends that start job. `After=` dependents proceed after one attempt, `Requires=` dependents fail with `dependency`, and the retries continue as fresh start jobs. The 2026-09-24 journal shows it with the already-2880 swarm-services-cert: nginx got "Dependency failed" 1ms after the first failure, and switch-to-configuration exited before the first restart was scheduled. The comments in lib/store-retry.nix, glue-matrix-bao-token.nix, glue-queue-agent-credential.nix, swarm-otel.nix and swarm-grafana.nix now say that, and so does docs/swarm/credentials.md. Because dependents start after one attempt, a consumer that loads its credential at start never sees a value a later attempt lands, or a rotated one. nix/host-modules/lib/refresh-consumer.nix adds `secret_differs` and `refresh_consumer`, and the four fetch units whose consumers take a start-time copy call them after the write, only when the value changed: - swarm-bao-matrix-token -> tuwunel.service in hive-matrix - swarm-bao-otel-oidc -> opentelemetry-collector.service in swarm-otel - swarm-bao-grafana-oidc -> grafana.service in the grafana container - swarm-bao-forwarder-oidc -> opentelemetry-collector.service in swarm-bao A running consumer is try-restarted, a failed one is reset and started, all with --no-block. Inline in the fetch script rather than a PathChanged path unit because the fetch script is the only writer and already knows whether the value changed, and it is the same shape as this PR's nginx hook and swarm-bao-nats-tls's restart of nats. module-eval-bao-grants gains one case per consumer. Refs #4662
This commit is contained in:
parent
b3b42d3279
commit
7eb966fe2b
9 changed files with 178 additions and 22 deletions
|
|
@ -82,6 +82,7 @@ let
|
|||
matrixMachine = "hive-matrix";
|
||||
|
||||
atomicWriteSecret = import ./lib/atomic-write-secret.nix { };
|
||||
refreshConsumer = import ./lib/refresh-consumer.nix { };
|
||||
storeRetry = import ./lib/store-retry.nix { };
|
||||
in
|
||||
{
|
||||
|
|
@ -151,11 +152,12 @@ in
|
|||
path = [
|
||||
deployCfg.bao.package
|
||||
pkgs.coreutils
|
||||
pkgs.systemd
|
||||
];
|
||||
# ./lib/store-retry.nix. ⚠️ This unit is `Before=` the homeserver's
|
||||
# container, and an auto-restarting unit is still `activating`, so the
|
||||
# homeserver waits for as long as the login below keeps failing — up to
|
||||
# the full 24h on a store that stays sealed.
|
||||
# ./lib/store-retry.nix. This unit is `Before=` the homeserver's
|
||||
# container, which waits for one attempt and then starts on the token it
|
||||
# already has; a token a later attempt lands restarts the homeserver
|
||||
# (below).
|
||||
inherit (storeRetry) startLimitBurst startLimitIntervalSec;
|
||||
serviceConfig = storeRetry.serviceConfig // {
|
||||
Type = "oneshot";
|
||||
|
|
@ -179,14 +181,14 @@ in
|
|||
set -euo pipefail
|
||||
|
||||
${atomicWriteSecret}
|
||||
${refreshConsumer}
|
||||
|
||||
# A sealed or uninitialised store answers on the port and never
|
||||
# answers the read, so "the store is up" is not the same as "the
|
||||
# store can answer". `TimeoutStartSec` above bounds each attempt and
|
||||
# a timed-out attempt is retried like any other failure; the
|
||||
# homeserver only `Wants=` this unit, so a retry budget spent
|
||||
# degrades to keeping the local token rather than failing the
|
||||
# container.
|
||||
# homeserver only `Wants=` this unit, so a failed attempt lets it
|
||||
# start on the local token rather than failing it.
|
||||
# `bao`'s own message is the only thing separating a missing value
|
||||
# from a refused identity from an unreachable host. This unit's
|
||||
# degraded mode is correct for all three, so it reports which one
|
||||
|
|
@ -232,6 +234,8 @@ in
|
|||
exit 0
|
||||
fi
|
||||
|
||||
changed=0
|
||||
if secret_differs ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)} "$token"; then changed=1; fi
|
||||
atomic_write_secret 0600 "" ${lib.escapeShellArg (toString deployCfg.matrix.appserviceTokenFile)} "$token"
|
||||
|
||||
# Re-stamp the registration file from the token just written. The
|
||||
|
|
@ -245,6 +249,13 @@ in
|
|||
# hive-matrix's own renderer rather than a `printf` here, so the
|
||||
# registration's shape has one home.
|
||||
${deployCfg.matrix.appserviceRegistrationScript}
|
||||
|
||||
# tuwunel loads the registration through `LoadCredential`, a copy
|
||||
# taken at start, while hive-c0re reads the token file on every call:
|
||||
# a changed token splits the two until the homeserver restarts.
|
||||
if [ "$changed" = 1 ]; then
|
||||
refresh_consumer ${lib.escapeShellArg matrixMachine} tuwunel.service
|
||||
fi
|
||||
'';
|
||||
};
|
||||
})
|
||||
|
|
|
|||
Loading…
Reference in a new issue