Watch
0
0
Fork
You've already forked hyperhive
0

hive TLS: reload the trust bundle's unit on a switch, name the missing root when it's empty

A switch that changed `hive-tls-ca` stopped and started it, and the stop
propagated through `Requires=` to the store's container (sealed again under
the shamir seal), every container binding the bundle, and every
`<name>-ca-bundle` unit with its consumers. `reloadIfChanged` with the same
script as `ExecReload` rewrites the bundle in place and restarts none of them.
`Requires=` stays, so a failed assembly still keeps them from starting.

With `swarm-services-root.pem` missing, `hive-tls-ca` writes an empty bundle
and every `<name>-ca-bundle` unit fails. Its log line now names the root's
path. docs/swarm/ca.md describes both.
This commit is contained in:
atlas 2026-10-05 20:47:20 +02:00 • committed by mara
commit 1a4dc16053
3 changed files with 53 additions and 3 deletions

View file

@ -77,8 +77,21 @@ issues it.
`hive-tls-ca.service` writes `<deploy.hive-controller.tls.stateDir>/trust-bundle.pem`, `hive-tls-ca.service` writes `<deploy.hive-controller.tls.stateDir>/trust-bundle.pem`,
the file every consumer anchors on. It holds one certificate, a copy of the file every consumer anchors on. It holds one certificate, a copy of
`swarm-services-root.pem`: the store's `pki` root, which the gateway leaf and `swarm-services-root.pem`: the store's `pki` root, which the gateway leaf and
the swarm-services leaf both chain to. It's the only root the swarm has. Until the swarm-services leaf both chain to. It's the only root the swarm has.
the first fetch of that root lands, the bundle is empty.
**Without the store's root, the bundle is empty and its readers go red.**
While `<deploy.hive-controller.tls.stateDir>/swarm-services-root.pem` is
missing or empty, as on a first boot before the store answers,
`hive-tls-ca.service` writes an empty bundle and logs that file's path. Every
`<name>-ca-bundle` unit then fails with `<bundle> contributed no certificate
to the bundle`, and the consumers that `Requires=` it don't start.
`swarm-services-cert.service` restarts `hive-tls-ca.service` when it writes a
new root.
A deploy that changes `hive-tls-ca.service` reloads it: the same script
rewrites the bundle in place, and nothing that `Requires=` the unit restarts,
including the store's container. A `<name>-ca-bundle` unit keeps the copy it
made when it started.
⚠️ **A consumer reads its trust when it starts.** An agent or service ⚠️ **A consumer reads its trust when it starts.** An agent or service
container that last started before the store's root reached the bundle container that last started before the store's root reached the bundle

View file

@ -362,6 +362,17 @@ in
# "$svcroot" ]` guard is what makes that state legal rather than fatal, # "$svcroot" ]` guard is what makes that state legal rather than fatal,
# and `swarm-services-cert` restarts this unit once the root it wrote # and `swarm-services-cert` restarts this unit once the root it wrote
# is new — so the bundle converges instead of being a boot behind. # is new — so the bundle converges instead of being a boot behind.
#
# ⚠️ A switch that changes this unit reloads it, running the same
# script, rather than stopping and starting it. A stop propagates to
# every unit that `Requires=` this one: the store's container, which
# comes back sealed under the `shamir` seal, every container that
# binds the bundle, and every `<name>-ca-bundle` unit with its
# consumers. A reload rewrites the bundle in place and touches none of
# them, so a `<name>-ca-bundle` unit keeps the copy it made at start
# until it restarts.
reloadIfChanged = true;
reload = config.systemd.services.hive-tls-ca.script;
serviceConfig = { serviceConfig = {
Type = "oneshot"; Type = "oneshot";
RemainAfterExit = true; RemainAfterExit = true;
@ -387,7 +398,7 @@ in
if [ -s "$svcroot" ]; then if [ -s "$svcroot" ]; then
cat "$svcroot" > "$bundle" cat "$svcroot" > "$bundle"
else else
echo "no trust anchor on disk yet — $bundle is empty until the store's root is fetched" >&2 echo "the store's root $svcroot is missing or empty, so $bundle is empty and every *-ca-bundle unit fails until swarm-services-cert fetches it" >&2
: > "$bundle" : > "$bundle"
fi fi
chmod 0644 "$bundle" chmod 0644 "$bundle"

View file

@ -281,6 +281,32 @@ let
&& !(lib.hasInfix "anchors+=(" caUnit.script) && !(lib.hasInfix "anchors+=(" caUnit.script)
&& !(lib.hasInfix "ca.pem" caUnit.script); && !(lib.hasInfix "ca.pem" caUnit.script);
} }
{
# A reload does not propagate through `Requires=`; a stop and start
# restarts the store's container, which comes back sealed. The control
# is that container still requiring the unit, so a failed assembly
# still keeps it from starting.
name = "a switch that changes the trust bundle's unit reloads it with the same script, so the store's container is not restarted";
ok =
let
text = allLocal.systemd.units."hive-tls-ca.service".text;
store = units."container@swarm-bao";
in
caUnit.reloadIfChanged
&& caUnit.reload == caUnit.script
&& lib.hasInfix "X-ReloadIfChanged=true" text
&& lib.hasInfix "ExecReload=" text
&& lib.elem "hive-tls-ca.service" store.requires
&& lib.elem "hive-tls-ca.service" store.after;
}
{
# A missing root is the one case where the bundle is empty, and every
# `<name>-ca-bundle` unit fails on it; the log names the file.
name = "an empty trust bundle is logged naming the store's root it is copied from";
ok = lib.hasInfix ''echo "the store's root $svcroot is missing or empty'' (
lib.last (lib.splitString ''if [ -s "$svcroot" ]; then'' caUnit.script)
);
}
{ {
# The control is the all-local host, which defines none of them. # The control is the all-local host, which defines none of them.
name = "a definition of a removed swarm.ca option fails evaluation, naming the option and the store's root"; name = "a definition of a removed swarm.ca option fails evaluation, naming the option and the store's root";