nix: unbreak the swarm-services leaf the gateway waits on
Three defects in the store-issued path, each of which alone kept nginx from starting at all. The gateway's cert import `Requires=` this leaf, so a leaf that is never issued is not a name mismatch — it is an empty listener, and the swarm's own forge stopped answering on :443. `swarm-services-cert` declared `Before=hive-tls-ca` for the trust bundle's sake while also being `After=` the store's container, which is itself `After=hive-tls-ca`. systemd resolved the cycle the only way it can, by deleting the job, so the leaf went unissued on every activation. The edge is gone; the bundle converges the other way round, through the restart this unit already performed when the root it wrote was new. The `pki` mount was enabled without `-max-lease-ttl`, so bao clamped the 30-year root to the 768h default and then refused every issue call, because a leaf of the mount's own default length would outlive the CA signing it. The mount is tuned on every run, the role pins a 720h leaf, and a root that can no longer cover one is replaced rather than left to refuse forever. A hive tests its own copy of that certificate against the same threshold, so both ends reach a fresh leaf without signalling. `swarm-bao-pki` mints the services-issuer leaf that opens the mount, but only `swarm-bao-certs` required it. `RemainAfterExit` plus an already-active unit means an activation that ADDS a leaf mints nothing — which is how a host whose config named `services-issuer.pem` came to have no such file. A target wants it now, like every sibling granting unit.
This commit is contained in:
müde
parent
22a87f7268
commit
5704c0c583
3 changed files with 118 additions and 18 deletions
|
|
@ -386,14 +386,19 @@ in
|
|||
# to wait for, so the dependency is conditional rather than a unit
|
||||
# that exists and does nothing.
|
||||
#
|
||||
# `swarm-services-cert` is ordered before this one rather than the
|
||||
# other way round, and not because this unit needs the leaf — it
|
||||
# never touches it. The trust bundle written at the end of this
|
||||
# script anchors on every root a consumer must hold, and the
|
||||
# services root is one of them, so the file has to be on disk
|
||||
# before the `cat`. Get it wrong and the bundle is a boot behind:
|
||||
# correct-looking, and missing the anchor for exactly the names the
|
||||
# gateway serves with it.
|
||||
# `swarm-services-cert` is NOT ordered against this unit in either
|
||||
# direction, and the absence is load-bearing — see the cycle spelled
|
||||
# out at that unit. It runs after the store's container, the container
|
||||
# runs after this unit, so an edge back to here is a loop and systemd
|
||||
# resolves a loop by dropping a job.
|
||||
#
|
||||
# What that costs: the trust bundle written at the end of this script
|
||||
# anchors on every root a consumer must hold, and the services root is
|
||||
# one of them, so on a first boot the `cat` below runs before that file
|
||||
# exists and the bundle is one restart short of complete. The `if [ -s
|
||||
# "$svcroot" ]` guard is what makes that state legal rather than fatal,
|
||||
# and `swarm-services-cert` restarts this unit once the root it wrote
|
||||
# is new — so the bundle converges instead of being a boot behind.
|
||||
after = lib.optionals swarmCaCfg.autoConfigure [ "swarm-ca.service" ];
|
||||
requires = lib.optionals swarmCaCfg.autoConfigure [ "swarm-ca.service" ];
|
||||
path = [ pkgs.openssl ];
|
||||
|
|
@ -647,18 +652,38 @@ in
|
|||
# coincidence, and `requiredBy` is what makes a store that never
|
||||
# comes up an outage that says so instead of a gateway serving the
|
||||
# wrong name.
|
||||
before = [
|
||||
"hive-tls-ca.service"
|
||||
"hive-gateway-self-signed-cert.service"
|
||||
];
|
||||
#
|
||||
# 🩸 NOT `Before=hive-tls-ca.service`, however much the trust bundle
|
||||
# wants it. That edge closed a cycle — the store's container is
|
||||
# `After=hive-tls-ca` (it serves the material that unit writes), and
|
||||
# this unit is `After=` the container — so systemd broke the loop the
|
||||
# only way it can, by deleting a job:
|
||||
#
|
||||
# Found ordering cycle: container@swarm-bao.service/start after
|
||||
# hive-tls-ca.service/start after swarm-services-cert.service/start
|
||||
# Job swarm-services-cert.service/start deleted to break ordering cycle
|
||||
#
|
||||
# With the job gone the leaf was never issued, `Requires=` took the
|
||||
# gateway's cert import down with it, and nginx never started at all —
|
||||
# so the swarm served nothing rather than serving the wrong name. The
|
||||
# bundle reaches the root the other way round, at the end of the script
|
||||
# below: it restarts `hive-tls-ca` when the root CHANGED, which is the
|
||||
# same path that already covered a store coming up hours late.
|
||||
before = [ "hive-gateway-self-signed-cert.service" ];
|
||||
requiredBy = [ "hive-gateway-self-signed-cert.service" ];
|
||||
# The store's container, where it runs here. On a hive that reads a
|
||||
# store hosted elsewhere no such unit exists and systemd ignores
|
||||
# the name, which is the correct behaviour rather than a gap: what
|
||||
# this unit actually needs is the store reachable, and on a remote
|
||||
# store nothing local can order against that.
|
||||
#
|
||||
# `swarm-bao-pki` mints the services-issuer leaf the login below
|
||||
# presents, and is absent on a hive holding an operator-placed one —
|
||||
# ignored there for the same reason, rather than being a dependency
|
||||
# only some hives may declare.
|
||||
after = [
|
||||
"container@${baoCfg.machine}.service"
|
||||
"swarm-bao-pki.service"
|
||||
"swarm-bao-services-issuer-policy.service"
|
||||
];
|
||||
wants = [ "container@${baoCfg.machine}.service" ];
|
||||
|
|
@ -723,6 +748,16 @@ in
|
|||
covers "$svcleaf" "$svcNames" || reissue=1
|
||||
[ -s "$svcroot" ] || reissue=1
|
||||
|
||||
# The ROOT's window, not the leaf's, and it is what keeps the two
|
||||
# sides of this from needing to talk. The store replaces an issuer
|
||||
# that has less than a leaf's window left (./swarm-bao.nix's
|
||||
# regeneration guard) — so a hive testing its own copy of that same
|
||||
# certificate against the same threshold asks for a new leaf on the
|
||||
# same activation, and gets the replacement root back with it.
|
||||
# Without this a leaf stays "valid" while the anchor it chains to no
|
||||
# longer exists in the mount, which no expiry check would ever catch.
|
||||
openssl x509 -in "$svcroot" -noout -checkend 2592000 >/dev/null 2>&1 || reissue=1
|
||||
|
||||
if [ "$reissue" = 0 ]; then
|
||||
echo "swarm-services leaf valid and covering the configured names — leaving it alone"
|
||||
exit 0
|
||||
|
|
@ -857,11 +892,12 @@ in
|
|||
systemctl reload --no-block nginx.service
|
||||
fi
|
||||
|
||||
# The bundle is assembled by `hive-tls-ca`, which is ordered
|
||||
# after this unit and therefore already holds the current root on
|
||||
# any normal boot. On the retry path it ran a long time ago, so a
|
||||
# CHANGED root has to reach it — and only a changed one, or every
|
||||
# boot would bounce a unit with nothing to do.
|
||||
# The bundle is assembled by `hive-tls-ca`, which runs BEFORE this
|
||||
# unit — it has to, or the store's container cannot start and there
|
||||
# is nothing to ask for a leaf. So on a first boot it wrote a bundle
|
||||
# without the root below, and this restart is what completes it
|
||||
# rather than a retry-path fallback. Only on a CHANGED root: every
|
||||
# boot otherwise bounces a unit with nothing to do.
|
||||
if [ "$rootchanged" = 1 ] && systemctl is-active --quiet hive-tls-ca.service; then
|
||||
echo "the services root changed — rebuilding the trust bundle"
|
||||
systemctl restart --no-block hive-tls-ca.service
|
||||
|
|
|
|||
Loading…
Reference in a new issue