5a83c40dca
The swarm's service names cannot go on the hive leaf: the hive CA is name-constrained to the hive domain and those names are siblings of it. So there is a second leaf, signed by the services sub-CA. signLeafScript is parameterised rather than duplicated -- same ceremony, different issuer and names -- so the two cannot drift in how they are built. The name list itself is derived once, as a read-only swarm.serviceDomains, and read by both the sub-CA that name-constrains those names and the leaf that carries them as SANs: two modules each assembling the list is how they stop agreeing. The renewal unit is the point of this commit as much as the leaf. hive-tls-resign now knows about both, because a leaf that first-boot issuance creates and weekly renewal ignores looks perfect for its entire validity and then expires with no warning -- the failure is invisible until it is total. The freshness test became a function over a leaf rather than a check of one, so adding a third leaf is a line rather than a rewrite. The services leaf is skipped where the sub-CA is absent: it exists only where the swarm CA is autoconfigured, and on a hive whose certs come from its operator the correct state is no leaf, not a stale one. Also drops a comment that documented signLeafScript's old signature from above an unrelated binding.
287 lines
12 KiB
Nix
287 lines
12 KiB
Nix
# The swarm root CA: the anchor a whole swarm shares, and the issuer of
|
|
# each hive's own CA (which is where it gets used — see ./hive-tls.nix).
|
|
#
|
|
# Why a hierarchy at all: cross-hive trust used to be hand-pinned per
|
|
# peer, so every hive had to name every other one — O(n²) configuration
|
|
# that a new hive could only join by editing all the others. One root
|
|
# makes it O(1): trust the root once and every present *and future* peer
|
|
# validates. The per-peer CA field is gone precisely because this
|
|
# replaced it.
|
|
#
|
|
# Two provisioning modes, ONE structure — what differs is who puts the
|
|
# artifacts on disk, never what the artifacts are:
|
|
#
|
|
# - autoconfigured (`services.hyperhive.enableAllLocalDefaults`, or
|
|
# `autoConfigure` set directly): the unit below generates the root
|
|
# here on first boot.
|
|
# - operator-provided (multi-host): the operator installs the root cert
|
|
# — and, on a host that does not hold the root key, the hive CA too —
|
|
# into the state dirs, and this unit does nothing. Splitting a
|
|
# single-host swarm across hosts is then a matter of moving the
|
|
# service dirs and turning `autoConfigure` off, not of a second code
|
|
# path that has to be kept agreeing with the first.
|
|
#
|
|
# The root KEY is the reason this is a runtime file and not a nix option:
|
|
# the store is world-readable and content-addressed, so a key committed
|
|
# to a flake is a key *published* to every consumer of that flake.
|
|
#
|
|
# ⚠️ The root CERT is therefore a runtime file too, which costs something:
|
|
# nothing whose trust store is built at build time can name it. See
|
|
# docs/swarm/ca.md § "Distributing the root".
|
|
{
|
|
lib,
|
|
config,
|
|
pkgs,
|
|
...
|
|
}:
|
|
let
|
|
cfg = config.services.hyperhive.swarm.ca;
|
|
hyperhiveCfg = config.services.hyperhive;
|
|
|
|
# The subject CN is a label for a human reading a chain, not an
|
|
# identity anything authenticates against. Fall through swarm name →
|
|
# hive domain → a constant so a hive that has set neither still
|
|
# evaluates; a missing `domain` is reported by its own assertion in
|
|
# hive-network.nix, and shouldn't also surface here as a null.
|
|
swarmLabel =
|
|
if hyperhiveCfg.swarm.name != null then
|
|
hyperhiveCfg.swarm.name
|
|
else if hyperhiveCfg.domain != null then
|
|
hyperhiveCfg.domain
|
|
else
|
|
"hyperhive";
|
|
|
|
# Derived once in ./swarm.nix, read here and by ./hive-tls.nix: the
|
|
# CA that name-constrains these and the leaf that carries them as SANs
|
|
# must agree exactly, and two modules each assembling the list is how
|
|
# they stop agreeing. It is also this unit's *rotation trigger* below,
|
|
# which is why the ordering is stable there rather than here.
|
|
serviceDomains = hyperhiveCfg.swarm.serviceDomains;
|
|
in
|
|
{
|
|
options.services.hyperhive.swarm.ca = {
|
|
autoConfigure = lib.mkOption {
|
|
type = lib.types.bool;
|
|
default = false;
|
|
example = true;
|
|
description = ''
|
|
Run the whole swarm CA on this one host: generate the swarm
|
|
root when it is missing, and issue this hive's CA under it.
|
|
|
|
`services.hyperhive.enableAllLocalDefaults` turns this on as
|
|
part of the all-on-one-box mode. Set it here directly to run the
|
|
CA on a host that is not otherwise all-local.
|
|
|
|
**Off by default, deliberately.** A swarm's services and its
|
|
hives can live on different hosts, and this host has no way to
|
|
tell whether it is the one holding the root — so the swarm CA
|
|
is something an operator sets up, not something a host decides
|
|
it is. Turn this on for an all-on-one-host deployment (dev
|
|
boxes, single-hive swarms) and get the hierarchy for free.
|
|
|
|
With it off, both artifacts are operator-provided: the root
|
|
under `stateDir`, and this hive's CA under
|
|
`services.hyperhive.tls.stateDir`. A hive given neither keeps
|
|
the self-signed CA it has always had — it simply isn't part of
|
|
a swarm's trust hierarchy, which is the correct outcome for a
|
|
hive nobody has federated yet.
|
|
'';
|
|
};
|
|
|
|
stateDir = lib.mkOption {
|
|
type = lib.types.str;
|
|
default = "/var/lib/swarm-ca";
|
|
description = ''
|
|
Host directory holding the swarm root CA: `root.pem` (the
|
|
anchor, safe to distribute — copy it to this same path on every
|
|
other host in the swarm) and `root-key.pem`
|
|
(0600, the one file that must never reach the nix store or
|
|
another host). The directory itself is 0700: nothing reads
|
|
out of it but the hive CA issuance in `hive-tls.nix`.
|
|
|
|
Moving the swarm CA to its own host is a matter of moving this
|
|
directory and setting `autoConfigure = false` here.
|
|
'';
|
|
};
|
|
|
|
servicesValidityDays = lib.mkOption {
|
|
type = lib.types.int;
|
|
default = 1825;
|
|
description = ''
|
|
Validity window of the swarm-services sub-CA in days (~5y).
|
|
Deliberately far shorter than the root's: this CA is *meant* to
|
|
be re-issued — adding a swarm service changes its name
|
|
constraints and rotates it — so a long window buys nothing, and
|
|
a short one keeps the rotation path exercised rather than
|
|
theoretical.
|
|
|
|
Rotating it is cheap in the way rotating the root is not: it
|
|
touches only the swarm-service vhosts, and no peer hive holds it
|
|
as an anchor.
|
|
'';
|
|
};
|
|
|
|
validityDays = lib.mkOption {
|
|
type = lib.types.int;
|
|
default = 10950;
|
|
description = ''
|
|
Validity window of the swarm root CA in days (default ~30y).
|
|
Deliberately longer than `services.hyperhive.tls.caValidityDays`:
|
|
the root must outlive the hive CAs it issues, or those chains
|
|
expire out from under hives that are still perfectly happy with
|
|
their own intermediate. Rotating a root is the one operation in
|
|
this system with no partial-failure mode — it invalidates every
|
|
peer at once, paced by the slowest peer's rebuild — so it is
|
|
never automatic and this window is meant to be uneventful.
|
|
'';
|
|
};
|
|
};
|
|
|
|
config = lib.mkIf (hyperhiveCfg.enable && cfg.autoConfigure) {
|
|
systemd.services.swarm-ca = {
|
|
description = "Generate the swarm root CA when absent";
|
|
wantedBy = [ "multi-user.target" ];
|
|
path = [ pkgs.openssl ];
|
|
serviceConfig = {
|
|
Type = "oneshot";
|
|
RemainAfterExit = true;
|
|
UMask = "0077";
|
|
# Pin the journal identity (else it's the `script` store-path wrapper).
|
|
SyslogIdentifier = "swarm-ca";
|
|
};
|
|
script = ''
|
|
set -euo pipefail
|
|
d=${lib.escapeShellArg cfg.stateDir}
|
|
install -d -m 0700 "$d"
|
|
|
|
root="$d/root.pem"
|
|
rootk="$d/root-key.pem"
|
|
|
|
# Note the asymmetry with the hive CA in hive-tls.nix, which
|
|
# regenerates itself once expired: a root is never replaced
|
|
# automatically, not even an expired one. Consumers hold this
|
|
# cert, so replacing it is a swarm-wide flag day that wants an
|
|
# operator running it deliberately, with both roots trusted
|
|
# across the overlap.
|
|
if [ -s "$root" ] && [ -s "$rootk" ]; then
|
|
echo "swarm root CA already present at $root — leaving it alone"
|
|
exit 0
|
|
fi
|
|
|
|
# Half a root is not a root. Generating a fresh key beside an
|
|
# already-distributed cert (or the reverse) leaves every
|
|
# consumer trusting an anchor that no longer signs anything —
|
|
# and it would look like it worked.
|
|
if [ -e "$root" ] || [ -e "$rootk" ]; then
|
|
echo "swarm root CA half-provisioned ($root / $rootk) — refusing to generate over it" >&2
|
|
exit 1
|
|
fi
|
|
|
|
echo "generating swarm root CA at $root"
|
|
# pathlen:1 — the root signs hive CAs, which sign leaves. One
|
|
# intermediate below the root and no deeper.
|
|
openssl req -x509 -newkey rsa:4096 -nodes -sha256 \
|
|
-days ${toString cfg.validityDays} \
|
|
-keyout "$rootk" -out "$root" \
|
|
-subj "/CN=swarm-ca ${swarmLabel}" \
|
|
-addext "basicConstraints=critical,CA:TRUE,pathlen:1" \
|
|
-addext "keyUsage=critical,keyCertSign,cRLSign"
|
|
chmod 0600 "$rootk"
|
|
chmod 0644 "$root"
|
|
'';
|
|
};
|
|
|
|
# The swarm-services sub-CA: issues leaves for the swarm's own
|
|
# service names, which no hive CA can sign — each of those is
|
|
# name-constrained to its own hive's domain, and the service names
|
|
# are siblings of it, not children.
|
|
#
|
|
# Rotation is the point of it being separate (mara: "swarm services
|
|
# sub ca that can rotate independently of swarm root ca"): the
|
|
# constraint enumerates the exact service names, so adding a service
|
|
# re-issues *this* and never touches the root or any hive CA.
|
|
systemd.services.swarm-services-ca = {
|
|
description = "Issue the swarm-services sub-CA under the swarm root";
|
|
wantedBy = [ "multi-user.target" ];
|
|
after = [ "swarm-ca.service" ];
|
|
requires = [ "swarm-ca.service" ];
|
|
path = [ pkgs.openssl ];
|
|
serviceConfig = {
|
|
Type = "oneshot";
|
|
RemainAfterExit = true;
|
|
UMask = "0077";
|
|
SyslogIdentifier = "swarm-services-ca";
|
|
};
|
|
script = ''
|
|
set -euo pipefail
|
|
d=${lib.escapeShellArg cfg.stateDir}
|
|
root="$d/root.pem"
|
|
rootk="$d/root-key.pem"
|
|
ca="$d/services-ca.pem"
|
|
cak="$d/services-ca-key.pem"
|
|
# The name set this CA was last issued for. Comparing against it
|
|
# is what makes re-issuance happen exactly when the service
|
|
# names change — not every boot, and not never.
|
|
names="$d/services-ca.names"
|
|
want=${lib.escapeShellArg (lib.concatStringsSep "\n" serviceDomains)}
|
|
|
|
if [ -z "$want" ]; then
|
|
echo "no swarm service domains configured — nothing to issue for"
|
|
exit 0
|
|
fi
|
|
|
|
# Same half-provisioned guard as the root: a key beside a cert
|
|
# that did not sign it looks like it works and issues nothing
|
|
# anyone will trust.
|
|
if { [ -e "$ca" ] && [ ! -e "$cak" ]; } || { [ -e "$cak" ] && [ ! -e "$ca" ]; }; then
|
|
echo "services sub-CA half-provisioned ($ca / $cak) — refusing to generate over it" >&2
|
|
exit 1
|
|
fi
|
|
|
|
if [ -s "$ca" ] && [ -s "$cak" ] && [ -f "$names" ] \
|
|
&& [ "$(cat "$names")" = "$want" ]; then
|
|
echo "services sub-CA present and covers the configured names — leaving it alone"
|
|
exit 0
|
|
fi
|
|
|
|
if [ ! -s "$root" ] || [ ! -s "$rootk" ]; then
|
|
echo "no swarm root CA at $root — cannot issue the services sub-CA under it" >&2
|
|
exit 1
|
|
fi
|
|
|
|
echo "issuing services sub-CA at $ca for: $(echo "$want" | tr '\n' ' ')"
|
|
csr="$(mktemp "$d/services-ca.csr.XXXXXX")"
|
|
ext="$(mktemp "$d/services-ca.ext.XXXXXX")"
|
|
trap 'rm -f "$csr" "$ext"' EXIT
|
|
|
|
openssl req -newkey rsa:4096 -nodes -sha256 \
|
|
-keyout "$cak" -out "$csr" \
|
|
-subj "/CN=swarm-services-ca ${swarmLabel}"
|
|
|
|
{
|
|
# pathlen:0 — this signs leaves and delegates no further.
|
|
printf 'basicConstraints=critical,CA:TRUE,pathlen:0\n'
|
|
printf 'keyUsage=critical,keyCertSign,cRLSign\n'
|
|
printf 'subjectKeyIdentifier=hash\n'
|
|
printf 'authorityKeyIdentifier=keyid:always\n'
|
|
# Constrained to the exact service names, not to the whole
|
|
# swarm domain: a leaked services CA should mint `forge.`,
|
|
# `chat.`, `auth.` and nothing else. The IP exclusions are not
|
|
# redundant — a DNS constraint says nothing about an
|
|
# iPAddress SAN, and an unconstrained name type is a name
|
|
# type this CA is unconstrained for.
|
|
printf 'nameConstraints=critical,permitted;%s,excluded;IP:0.0.0.0/0.0.0.0,excluded;IP:0:0:0:0:0:0:0:0/0:0:0:0:0:0:0:0\n' \
|
|
"$(echo "$want" | sed 's/^/DNS:/' | paste -sd, -)"
|
|
} > "$ext"
|
|
|
|
openssl x509 -req -in "$csr" -CA "$root" -CAkey "$rootk" \
|
|
-CAcreateserial -days ${toString cfg.servicesValidityDays} -sha256 \
|
|
-extfile "$ext" -out "$ca"
|
|
|
|
printf '%s' "$want" > "$names"
|
|
chmod 0600 "$cak"
|
|
chmod 0644 "$ca" "$names"
|
|
'';
|
|
};
|
|
};
|
|
}
|