From 0befa4f36dcbea2b19c331bbb03872eda13561e4 Mon Sep 17 00:00:00 2001 From: atlas Date: Wed, 5 Aug 2026 18:44:03 +0200 Subject: [PATCH 1/3] feat(tls): adopt the swarm CA automatically, but only where it is safe MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A hive whose CA predates the swarm root keeps a self-signed CA, and nothing re-roots it. That is what makes the hierarchy non-disruptive, and it also means the O(1)-trust payoff never arrives for that hive. Adoption now happens by itself where this host owns the root, and nowhere else. The split is the whole design: adoption invalidates an anchor consumers already trust, and they refresh on their own schedule — on one box that schedule is knowable, across hosts it is not. So the all-local case migrates itself and every other case gets a failure that names both files, gives the two-command recipe, says why it is not automatic, and offers the marker as a deliberate opt-out. The previous CA rides in the trust bundle afterwards. Consumers read the bundle rather than ca.pem, so adoption is additive before it is subtractive; agents pick up new trust only when their container restarts, which is a window even on a single host. Dropping the old anchor stays a separate, deliberate step. --- nix/host-modules/hive-tls.nix | 65 ++++++++++++++++++++++++++++++++--- 1 file changed, 60 insertions(+), 5 deletions(-) diff --git a/nix/host-modules/hive-tls.nix b/nix/host-modules/hive-tls.nix index 3de3169b..8f785282 100644 --- a/nix/host-modules/hive-tls.nix +++ b/nix/host-modules/hive-tls.nix @@ -248,6 +248,49 @@ in root=${lib.escapeShellArg "${swarmCaCfg.stateDir}/root.pem"} rootk=${lib.escapeShellArg "${swarmCaCfg.stateDir}/root-key.pem"} + prev="$d/ca-previous.pem" + marker="$d/.swarm-ca-adopted" + + # --- Adoption: a hive whose CA predates the swarm root. + # + # Runs ONLY where this host also owns the root (`autoConfigure`), + # because adoption invalidates an anchor that consumers already + # trust and they refresh on their own schedule. On one box that + # schedule is knowable; across hosts it is not, so there the + # operator does it and this unit only says so, loudly. + # + # Guarded by a marker rather than by the state of the world: the + # marker's ABSENCE is the trigger, so this fires once per hive + # instead of re-deciding every activation. That is the difference + # between a migration and a kicking machine. + if [ -s "$root" ] && [ -s "$ca" ] && [ ! -e "$marker" ] \ + && ! openssl verify -CAfile "$root" "$ca" >/dev/null 2>&1; then + ${ + if swarmCaCfg.autoConfigure then + '' + echo "adopting the swarm CA: $ca does not chain to $root" >&2 + # Keep the old CA as an anchor across the overlap. Consumers + # read the bundle, so adoption is additive before it is + # subtractive — agents pick up new trust only when their + # container restarts, which is a window even on one host. + cp "$ca" "$prev" + chmod 0644 "$prev" + rm -f "$ca" "$cak" + touch "$marker"'' + else + '' + echo "hive-tls: this hive's CA does not chain to the swarm root." >&2 + echo " hive CA: $ca" >&2 + echo " swarm root: $root" >&2 + echo "Adopting the hierarchy is not automatic here: it invalidates an" >&2 + echo "anchor that peers and agents on OTHER hosts still trust, and they" >&2 + echo "refresh on their own schedule — only you know when that is safe." >&2 + echo "To adopt: rm $ca $cak && systemctl restart hive-tls-ca.service" >&2 + echo "To keep the current CA deliberately: touch $marker" >&2 + exit 1'' + } + fi + # --- CA: generated once and reused across leaf rotations, in # whichever of the two shapes `caGenScript` selected. # Regenerated only if missing or already expired (checkend 0). @@ -298,12 +341,24 @@ in # Written IN PLACE, never renamed into position: containers bind # -mount this file, and a bind mount follows the inode. A # rename would leave every consumer holding the old one. + # `ca-previous.pem` rides along after an adoption for the same + # reason: a consumer that has not restarted yet is still pinned to + # the old anchor, and dropping it the instant the new one exists + # would break exactly the hosts the migration was supposed to + # spare. Removing it is a deliberate later step, not this unit's + # call — "long enough" is a deployment fact. + # + # Built as an explicit anchor ARRAY with `if` guards, not + # `[ -s f ] && anchors+=(f)`: under `set -e` an && list whose test + # fails IS a failing command and kills the unit, so the tidier + # form would break precisely on the hive where the optional file + # is legitimately absent. The `|| true` patch for that is worse — + # it would also swallow a real failure to read the hive CA. bundle="$d/trust-bundle.pem" - if [ -s "$root" ]; then - cat "$ca" "$root" > "$bundle" - else - cat "$ca" > "$bundle" - fi + anchors=("$ca") + if [ -s "$prev" ]; then anchors+=("$prev"); fi + if [ -s "$root" ]; then anchors+=("$root"); fi + cat "''${anchors[@]}" > "$bundle" chmod 0644 "$bundle" ''; }; From 591d0e789f2bbc5320346de55ea6ee795df33651 Mon Sep 17 00:00:00 2001 From: atlas Date: Wed, 5 Aug 2026 18:48:21 +0200 Subject: [PATCH 2/3] docs(swarm): document the two adoption paths The CA page described adoption as a documented `rm`, which is now only half true: where the host owns the root it happens by itself, and the split between the two cases is the part worth explaining rather than the commands. --- docs/swarm/ca.md | 58 +++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 47 insertions(+), 11 deletions(-) diff --git a/docs/swarm/ca.md b/docs/swarm/ca.md index 23182dc4..9d176946 100644 --- a/docs/swarm/ca.md +++ b/docs/swarm/ca.md @@ -78,15 +78,51 @@ on. ## Adopting the hierarchy on an existing hive -An existing `ca.pem` is never re-rooted automatically — swapping it -would break every consumer that already trusts it, and agents only pick -up new trust when their container restarts. To adopt, delete `ca.pem` + -`ca-key.pem` under `tls.stateDir` and restart `hive-tls-ca.service`; -the CA is re-issued under the root and the leaf re-signed. Until then -the hive serves TLS exactly as before and is simply not part of the -swarm's trust hierarchy. +A hive that predates the swarm root carries a self-signed `ca.pem`, and +adopting the hierarchy means replacing it. That invalidates an anchor +consumers already trust, and they refresh on their own schedule — agents +only pick up new trust when their container restarts, peers only on +their own rebuild. **Who is allowed to decide that is what splits the +two cases.** -Making that adoption a first-class, non-disruptive operation — rather -than a documented `rm` — is tracked separately; the mechanism it needs -(carrying the previous CA in the trust bundle across the overlap) -already exists. +### Where this host owns the root (`autoConfigure`) + +Adoption happens by itself, once. `hive-tls-ca.service` notices that +`ca.pem` does not chain to the root, keeps the old certificate as +`ca-previous.pem`, and re-issues under the root; the next leaf is signed +by the new CA. + +It is safe to automate here precisely because this is the +all-on-one-host shape: every consumer is on this box, so "when will they +have refreshed" is knowable rather than guessed. + +The old CA stays in `trust-bundle.pem` afterwards, so adoption is +**additive to the anchor set before it is subtractive** — a container +that has not restarted yet still validates. Removing `ca-previous.pem` +is a deliberate later step: how long is long enough is a property of the +deployment, not something the unit can know. + +A marker file (`.swarm-ca-adopted`) records that this ran. Its *absence* +is the trigger, so adoption fires once per hive rather than being +re-decided on every activation. + +### Everywhere else + +No automatic adoption. `hive-tls-ca.service` fails, loudly, naming both +certificates and giving the two-command recipe: + +``` +rm /ca.pem /ca-key.pem +systemctl restart hive-tls-ca.service +``` + +Failing rather than warning is deliberate: a hive whose CA does not +chain to the root it has been given is misconfigured, and a warning in a +build log is not something anyone reads twice. + +To keep the current CA on purpose — a hive that deliberately stays +outside the hierarchy, or one mid-migration — `touch` the marker file +named in the message. That is a decision, and it is recorded as one. + +A hive with **no** root configured at all is not affected by any of +this: it self-signs exactly as it always has. From 01680ee96285c8742230fe8451adac5957f38039 Mon Sep 17 00:00:00 2001 From: atlas Date: Wed, 5 Aug 2026 18:49:35 +0200 Subject: [PATCH 3/3] docs(tls): move the bundle rationale to the CA page MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The comment block tripped the 30-line lint, and the lint was right: the openssl chain-termination explanation had just been written into docs/swarm/ca.md, so the comment was a second copy of it. What stays is the part the code cannot say — the inode constraint, why the previous anchor is not dropped here, and the set -e alternative that was rejected. --- nix/host-modules/hive-tls.nix | 42 ++++++++++++----------------------- 1 file changed, 14 insertions(+), 28 deletions(-) diff --git a/nix/host-modules/hive-tls.nix b/nix/host-modules/hive-tls.nix index 8f785282..553e5251 100644 --- a/nix/host-modules/hive-tls.nix +++ b/nix/host-modules/hive-tls.nix @@ -322,38 +322,24 @@ in fi # --- Trust bundle: what a consumer must TRUST, as opposed to - # `ca.pem`, which is what this host SIGNS with. The two were the - # same file while the hive CA was self-signed, and stopped being - # the same file the moment it became an intermediate: openssl - # refuses to end a chain at a trusted cert that isn't - # self-signed (that's what `-partial_chain` is for), so an - # agent's curl handed only `ca.pem` fails with "unable to get - # issuer certificate". Verified in both directions before this - # was written — rustls and Go accept a trusted intermediate, - # which is what makes the breakage partial and easy to miss. + # `ca.pem`, which is what this host SIGNS with. Why they stopped + # being the same file, and why `ca-previous.pem` stays in the set + # after an adoption, are in docs/swarm/ca.md. Three constraints + # the code cannot state: # - # Consumers therefore trust hive CA + swarm root; on a hive that - # still has a self-signed CA the bundle is just that CA, so the - # consumer side needs no condition at all. Trusting the root is - # also the point of the hierarchy — it is what lets a peer hive - # validate without being hand-pinned. + # Written IN PLACE, never renamed into position — containers + # bind-mount this file and a bind mount follows the inode, so a + # rename leaves every consumer holding the old one. # - # Written IN PLACE, never renamed into position: containers bind - # -mount this file, and a bind mount follows the inode. A - # rename would leave every consumer holding the old one. - # `ca-previous.pem` rides along after an adoption for the same - # reason: a consumer that has not restarted yet is still pinned to - # the old anchor, and dropping it the instant the new one exists - # would break exactly the hosts the migration was supposed to - # spare. Removing it is a deliberate later step, not this unit's - # call — "long enough" is a deployment fact. + # Dropping `ca-previous.pem` is deliberately NOT done here: + # "long enough" is a deployment fact, not a unit's call. # - # Built as an explicit anchor ARRAY with `if` guards, not + # Built as an array with `if` guards, not # `[ -s f ] && anchors+=(f)`: under `set -e` an && list whose test - # fails IS a failing command and kills the unit, so the tidier - # form would break precisely on the hive where the optional file - # is legitimately absent. The `|| true` patch for that is worse — - # it would also swallow a real failure to read the hive CA. + # fails IS a failing command and kills the unit — on exactly the + # hive where the optional file is legitimately absent. The + # reflexive `|| true` is worse; it also swallows a real failure to + # read the hive CA. bundle="$d/trust-bundle.pem" anchors=("$ca") if [ -s "$prev" ]; then anchors+=("$prev"); fi