97a7b518ea
Same defect as the switch below it, one tier up: it sat at the TOP of `services.hyperhive`, a namespace that is meant to be everything about hyperhive rather than the settings of a single hive. Whether this box is the whole deployment is as per-host as a decision gets. The name follows mara's sentence for what it means — "everything in the swarm is running on this host" — rather than naming its mechanism. "Defaults" was doing no work: it is not a defaults toggle, it is a claim about where the swarm lives, and the pair now reads as the containment it already was, singleHostSwarm implying allSwarmServices plus this hive. One site was a setter rather than a reference: module-eval's `allLocal` fixture passes an attrset merged into `services.hyperhive`, so its key carries the path and had to become `deploy.singleHostSwarm`. A rename by bare identifier is right for the twelve prose mentions and wrong for exactly this one, which is worth knowing before the next rename.
303 lines
13 KiB
Nix
303 lines
13 KiB
Nix
# The swarm root CA: the anchor a whole swarm shares, and the issuer of
|
||
# each hive's own CA (which is where it gets used — see ./hive-tls.nix).
|
||
#
|
||
# Why a hierarchy at all: cross-hive trust used to be hand-pinned per
|
||
# peer, so every hive had to name every other one — O(n²) configuration
|
||
# that a new hive could only join by editing all the others. One root
|
||
# makes it O(1): trust the root once and every present *and future* peer
|
||
# validates. The per-peer CA field is gone precisely because this
|
||
# replaced it.
|
||
#
|
||
# Two provisioning modes, ONE structure — what differs is who puts the
|
||
# artifacts on disk, never what the artifacts are:
|
||
#
|
||
# - autoconfigured (`services.hyperhive.deploy.singleHostSwarm`, or
|
||
# `autoConfigure` set directly): the unit below generates the root
|
||
# here on first boot.
|
||
# - operator-provided (multi-host): the operator installs the root cert
|
||
# — and, on a host that does not hold the root key, the hive CA too —
|
||
# into the state dirs, and this unit does nothing. Splitting a
|
||
# single-host swarm across hosts is then a matter of moving the
|
||
# service dirs and turning `autoConfigure` off, not of a second code
|
||
# path that has to be kept agreeing with the first.
|
||
#
|
||
# The root KEY is the reason this is a runtime file and not a nix option:
|
||
# the store is world-readable and content-addressed, so a key committed
|
||
# to a flake is a key *published* to every consumer of that flake.
|
||
#
|
||
# ⚠️ The root CERT is therefore a runtime file too, which costs something:
|
||
# nothing whose trust store is built at build time can name it. See
|
||
# docs/swarm/ca.md § "Distributing the root".
|
||
{
|
||
lib,
|
||
config,
|
||
pkgs,
|
||
...
|
||
}:
|
||
let
|
||
cfg = config.services.hyperhive.swarm.ca;
|
||
hyperhiveCfg = config.services.hyperhive;
|
||
|
||
# The subject CN is a label for a human reading a chain, not an
|
||
# identity anything authenticates against. Fall through swarm name →
|
||
# hive domain → a constant so a hive that has set neither still
|
||
# evaluates; a missing `domain` is reported by its own assertion in
|
||
# hive-network.nix, and shouldn't also surface here as a null.
|
||
swarmLabel =
|
||
if hyperhiveCfg.swarm.name != null then
|
||
hyperhiveCfg.swarm.name
|
||
else if hyperhiveCfg.domain != null then
|
||
hyperhiveCfg.domain
|
||
else
|
||
"hyperhive";
|
||
|
||
# Derived once in ./swarm.nix, read here and by ./hive-tls.nix: the
|
||
# CA that name-constrains these and the leaf that carries them as SANs
|
||
# must agree exactly, and two modules each assembling the list is how
|
||
# they stop agreeing. It is also this unit's *rotation trigger* below,
|
||
# which is why the ordering is stable there rather than here.
|
||
serviceDomains = hyperhiveCfg.swarm.serviceDomains;
|
||
in
|
||
{
|
||
options.services.hyperhive.swarm.ca = {
|
||
autoConfigure = lib.mkOption {
|
||
type = lib.types.bool;
|
||
default = false;
|
||
example = true;
|
||
description = ''
|
||
Run the whole swarm CA on this one host: generate the swarm
|
||
root when it is missing, and issue this hive's CA under it.
|
||
|
||
`services.hyperhive.deploy.singleHostSwarm` turns this on as
|
||
part of the all-on-one-box mode. Set it here directly to run the
|
||
CA on a host that is not otherwise all-local.
|
||
|
||
**Off by default, deliberately.** A swarm's services and its
|
||
hives can live on different hosts, and this host has no way to
|
||
tell whether it is the one holding the root — so the swarm CA
|
||
is something an operator sets up, not something a host decides
|
||
it is. Turn this on for an all-on-one-host deployment (dev
|
||
boxes, single-hive swarms) and get the hierarchy for free.
|
||
|
||
With it off, both artifacts are operator-provided: the root
|
||
under `stateDir`, and this hive's CA under
|
||
`services.hyperhive.tls.stateDir`. A hive given neither keeps
|
||
the self-signed CA it has always had — it simply isn't part of
|
||
a swarm's trust hierarchy, which is the correct outcome for a
|
||
hive nobody has federated yet.
|
||
'';
|
||
};
|
||
|
||
stateDir = lib.mkOption {
|
||
type = lib.types.str;
|
||
default = "/var/lib/swarm-ca";
|
||
description = ''
|
||
Host directory holding the swarm root CA: `root.pem` (the
|
||
anchor, safe to distribute — copy it to this same path on every
|
||
other host in the swarm) and `root-key.pem`
|
||
(0600, the one file that must never reach the nix store or
|
||
another host). The directory itself is 0700: nothing reads
|
||
out of it but the hive CA issuance in `hive-tls.nix`.
|
||
|
||
Moving the swarm CA to its own host is a matter of moving this
|
||
directory and setting `autoConfigure = false` here.
|
||
'';
|
||
};
|
||
|
||
servicesValidityDays = lib.mkOption {
|
||
type = lib.types.int;
|
||
default = 1825;
|
||
description = ''
|
||
Validity window of the swarm-services sub-CA in days (~5y).
|
||
Deliberately far shorter than the root's: this CA is *meant* to
|
||
be re-issued — adding a swarm service changes its name
|
||
constraints and rotates it — so a long window buys nothing, and
|
||
a short one keeps the rotation path exercised rather than
|
||
theoretical.
|
||
|
||
Rotating it is cheap in the way rotating the root is not: it
|
||
touches only the swarm-service vhosts, and no peer hive holds it
|
||
as an anchor.
|
||
'';
|
||
};
|
||
|
||
validityDays = lib.mkOption {
|
||
type = lib.types.int;
|
||
default = 10950;
|
||
description = ''
|
||
Validity window of the swarm root CA in days (default ~30y).
|
||
Deliberately longer than `services.hyperhive.tls.caValidityDays`:
|
||
the root must outlive the hive CAs it issues, or those chains
|
||
expire out from under hives that are still perfectly happy with
|
||
their own intermediate. Rotating a root is the one operation in
|
||
this system with no partial-failure mode — it invalidates every
|
||
peer at once, paced by the slowest peer's rebuild — so it is
|
||
never automatic and this window is meant to be uneventful.
|
||
'';
|
||
};
|
||
};
|
||
|
||
config = lib.mkIf (hyperhiveCfg.enable && cfg.autoConfigure) {
|
||
# A CA that fails to issue is invisible until something makes a TLS call
|
||
# hours later, so these two oneshots are worth more than most services.
|
||
services.hyperhive.swarm.otel.journaldUnits = [
|
||
"swarm-ca"
|
||
"swarm-services-ca"
|
||
];
|
||
|
||
systemd.services.swarm-ca = {
|
||
description = "Generate the swarm root CA when absent";
|
||
wantedBy = [ "multi-user.target" ];
|
||
path = [ pkgs.openssl ];
|
||
serviceConfig = {
|
||
Type = "oneshot";
|
||
RemainAfterExit = true;
|
||
UMask = "0077";
|
||
# Pin the journal identity (else it's the `script` store-path wrapper).
|
||
SyslogIdentifier = "swarm-ca";
|
||
};
|
||
script = ''
|
||
set -euo pipefail
|
||
d=${lib.escapeShellArg cfg.stateDir}
|
||
install -d -m 0700 "$d"
|
||
|
||
root="$d/root.pem"
|
||
rootk="$d/root-key.pem"
|
||
|
||
# Note the asymmetry with the hive CA in hive-tls.nix, which
|
||
# regenerates itself once expired: a root is never replaced
|
||
# automatically, not even an expired one. Consumers hold this
|
||
# cert, so replacing it is a swarm-wide flag day that wants an
|
||
# operator running it deliberately, with both roots trusted
|
||
# across the overlap.
|
||
if [ -s "$root" ] && [ -s "$rootk" ]; then
|
||
echo "swarm root CA already present at $root — leaving it alone"
|
||
exit 0
|
||
fi
|
||
|
||
# Half a root is not a root. Generating a fresh key beside an
|
||
# already-distributed cert (or the reverse) leaves every
|
||
# consumer trusting an anchor that no longer signs anything —
|
||
# and it would look like it worked.
|
||
if [ -e "$root" ] || [ -e "$rootk" ]; then
|
||
echo "swarm root CA half-provisioned ($root / $rootk) — refusing to generate over it" >&2
|
||
exit 1
|
||
fi
|
||
|
||
echo "generating swarm root CA at $root"
|
||
# pathlen:1 — the root signs hive CAs, which sign leaves. One
|
||
# intermediate below the root and no deeper.
|
||
openssl req -x509 -newkey rsa:4096 -nodes -sha256 \
|
||
-days ${toString cfg.validityDays} \
|
||
-keyout "$rootk" -out "$root" \
|
||
-subj "/CN=swarm-ca ${swarmLabel}" \
|
||
-addext "basicConstraints=critical,CA:TRUE,pathlen:1" \
|
||
-addext "keyUsage=critical,keyCertSign,cRLSign"
|
||
chmod 0600 "$rootk"
|
||
chmod 0644 "$root"
|
||
'';
|
||
};
|
||
|
||
# The swarm-services sub-CA: issues leaves for the swarm's own
|
||
# service names, which no hive CA can sign — each of those is
|
||
# name-constrained to its own hive's domain, and the service names
|
||
# are siblings of it, not children.
|
||
#
|
||
# Rotation is the point of it being separate (mara: "swarm services
|
||
# sub ca that can rotate independently of swarm root ca"): the
|
||
# constraint enumerates the exact service names, so adding a service
|
||
# re-issues *this* and never touches the root or any hive CA.
|
||
systemd.services.swarm-services-ca = {
|
||
description = "Issue the swarm-services sub-CA under the swarm root";
|
||
wantedBy = [ "multi-user.target" ];
|
||
after = [ "swarm-ca.service" ];
|
||
requires = [ "swarm-ca.service" ];
|
||
path = [ pkgs.openssl ];
|
||
serviceConfig = {
|
||
Type = "oneshot";
|
||
RemainAfterExit = true;
|
||
UMask = "0077";
|
||
SyslogIdentifier = "swarm-services-ca";
|
||
};
|
||
script = ''
|
||
set -euo pipefail
|
||
d=${lib.escapeShellArg cfg.stateDir}
|
||
root="$d/root.pem"
|
||
rootk="$d/root-key.pem"
|
||
ca="$d/services-ca.pem"
|
||
cak="$d/services-ca-key.pem"
|
||
# The name set this CA was last issued for. Comparing against it
|
||
# is what makes re-issuance happen exactly when the service
|
||
# names change — not every boot, and not never.
|
||
names="$d/services-ca.names"
|
||
want=${lib.escapeShellArg (lib.concatStringsSep "\n" serviceDomains)}
|
||
|
||
if [ -z "$want" ]; then
|
||
echo "no swarm service domains configured — nothing to issue for"
|
||
exit 0
|
||
fi
|
||
|
||
# Same half-provisioned guard as the root: a key beside a cert
|
||
# that did not sign it looks like it works and issues nothing
|
||
# anyone will trust.
|
||
if { [ -e "$ca" ] && [ ! -e "$cak" ]; } || { [ -e "$cak" ] && [ ! -e "$ca" ]; }; then
|
||
echo "services sub-CA half-provisioned ($ca / $cak) — refusing to generate over it" >&2
|
||
exit 1
|
||
fi
|
||
|
||
if [ -s "$ca" ] && [ -s "$cak" ] && [ -f "$names" ] \
|
||
&& [ "$(cat "$names")" = "$want" ]; then
|
||
echo "services sub-CA present and covers the configured names — leaving it alone"
|
||
exit 0
|
||
fi
|
||
|
||
if [ ! -s "$root" ] || [ ! -s "$rootk" ]; then
|
||
echo "no swarm root CA at $root — cannot issue the services sub-CA under it" >&2
|
||
exit 1
|
||
fi
|
||
|
||
echo "issuing services sub-CA at $ca for: $(echo "$want" | tr '\n' ' ')"
|
||
csr="$(mktemp "$d/services-ca.csr.XXXXXX")"
|
||
ext="$(mktemp "$d/services-ca.ext.XXXXXX")"
|
||
trap 'rm -f "$csr" "$ext"' EXIT
|
||
|
||
openssl req -newkey rsa:4096 -nodes -sha256 \
|
||
-keyout "$cak" -out "$csr" \
|
||
-subj "/CN=swarm-services-ca ${swarmLabel}"
|
||
|
||
{
|
||
# pathlen:0 — this signs leaves and delegates no further.
|
||
printf 'basicConstraints=critical,CA:TRUE,pathlen:0\n'
|
||
printf 'keyUsage=critical,keyCertSign,cRLSign\n'
|
||
printf 'subjectKeyIdentifier=hash\n'
|
||
printf 'authorityKeyIdentifier=keyid:always\n'
|
||
# Constrained to the exact service names, not to the whole
|
||
# swarm domain: a leaked services CA should mint `forge.`,
|
||
# `chat.`, `auth.` and nothing else. The IP exclusions are not
|
||
# redundant — a DNS constraint says nothing about an
|
||
# iPAddress SAN, and an unconstrained name type is a name
|
||
# type this CA is unconstrained for.
|
||
#
|
||
# ⚠️ EVERY entry carries its own `permitted;` / `excluded;`
|
||
# prefix. openssl's parser takes the qualifier per subtree, not
|
||
# once for a run of them: `permitted;DNS:a,DNS:b` is rejected
|
||
# outright with `v2i_NAME_CONSTRAINTS: invalid syntax`, which
|
||
# fails the whole unit. The hive CA next door emits exactly one
|
||
# permitted name, so the missing-prefix form is accidentally
|
||
# valid there and does NOT generalise — this list is always
|
||
# longer than one.
|
||
printf 'nameConstraints=critical,%s,excluded;IP:0.0.0.0/0.0.0.0,excluded;IP:0:0:0:0:0:0:0:0/0:0:0:0:0:0:0:0\n' \
|
||
"$(echo "$want" | sed 's/^/permitted;DNS:/' | paste -sd, -)"
|
||
} > "$ext"
|
||
|
||
openssl x509 -req -in "$csr" -CA "$root" -CAkey "$rootk" \
|
||
-CAcreateserial -days ${toString cfg.servicesValidityDays} -sha256 \
|
||
-extfile "$ext" -out "$ca"
|
||
|
||
printf '%s' "$want" > "$names"
|
||
chmod 0600 "$cak"
|
||
chmod 0644 "$ca" "$names"
|
||
'';
|
||
};
|
||
};
|
||
}
|