vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(nix): make the swarm CA hierarchy opt-in

Browse source 
A swarm's services and its hives can live on different hosts, and a host
has no way to tell whether it is the one holding the root — so setting
the swarm CA up is an operator action, not something a host infers.
`swarm.ca.autoConfigure` is off by default and is the only thing that
issues a hive sub-CA, which is also the only case that *can*: signing
one needs the root's private key, and that key is deliberately not on a
hive host otherwise.

With the flag off, a hive self-signs its CA exactly as it always has.
That keeps a plain hive working out of the box; what it gives up is
membership of a swarm's trust hierarchy, which is the right thing to
give up for a hive nobody has federated.

Replaces a default derived from `swarm.peers == { }`. That read "no
peers declared" as "everything is local", which is not the same claim —
a hive can belong to a swarm it has not declared yet, or to one whose
services run elsewhere entirely.
This commit is contained in:
atlas 2026-08-05 15:11:22 +02:00 • committed by mara
commit cc74a2c1f6
3 changed files with 118 additions and 78 deletions
Download patch file Download diff file Expand all files Collapse all files

27
docs/swarm.md
View file

@ -66,16 +66,25 @@ artifacts on disk:
| | swarm root | this hive's CA |
| --- | --- | --- |
| all on one host (default) | generated by `swarm-ca.service` on first boot | issued by `hive-tls-ca.service` under the root |
| split across hosts | operator-provided | operator-provided |
| default | operator-provided | operator-provided, else self-signed as before |
| `autoConfigure = true` (all on one host) | generated by `swarm-ca.service` on first boot | issued by `hive-tls-ca.service` under the root |
`services.hyperhive.swarm.ca.autoConfigure` selects between them. It
defaults to true exactly while this hive declares no `swarm.peers`, so a
single-host swarm costs no configuration and declaring a peer stops the
host from minting a root that could not be the swarm's. Moving the swarm
CA onto its own host is then a matter of moving
`services.hyperhive.swarm.ca.stateDir` and setting `autoConfigure =
false` — there is no second code path to switch to.
`services.hyperhive.swarm.ca.autoConfigure` selects between them, and is
**off by default**: a swarm's services and its hives can live on
different hosts, and a host cannot tell whether it is the one holding
the root, so setting the swarm CA up is an operator action rather than
something a host assumes. Turn it on for an all-on-one-host deployment
and the hierarchy costs no configuration.
**A hive given neither artifact keeps the self-signed CA it has always
had.** It serves TLS exactly as before and simply isn't part of a
swarm's trust hierarchy — the right outcome for a hive nobody has
federated yet. Only `autoConfigure` issues a hive sub-CA, because only
that case can: signing one needs the root's private key.
Moving the swarm CA onto its own host is then a matter of moving
`services.hyperhive.swarm.ca.stateDir` and leaving `autoConfigure` off —
there is no second code path to switch to.
The root's private key never reaches the nix store: the store is
world-readable, so a key committed to a flake is a key published to

137
nix/host-modules/hive-tls.nix
View file

@ -27,6 +27,76 @@ let
# when-to-sign condition. The leaf covers the bare hive domain plus
# `forge.`, `matrix.` and `*.<domain>` so all sub-domains validate
# under the same cert + the hive CA.
# How this hive's CA comes into existence when it is missing — and it
# is one of exactly two things, chosen by config rather than by what
# happens to be on disk.
#
# Issuing a sub-CA requires the swarm root's PRIVATE key, so it can
# only happen where that key legitimately lives: the single-host
# deployment that `swarm.ca.autoConfigure` describes. A host cannot
# infer that it is that host — a swarm's services and its hives can
# sit anywhere — so with the flag off this hive self-signs exactly as
# it always has, and an operator who wants it in the hierarchy
# installs the CA themselves. Falling back to self-signed rather than
# failing keeps a plain hive working out of the box; what it loses is
# membership of a swarm's trust, which is the correct thing to lose
# for a hive nobody has federated.
caGenScript =
if swarmCaCfg.autoConfigure then
''
# The root should exist — `swarm-ca.service` runs before this and
# is required by it. If it doesn't, something upstream failed and
# signing with a half-provisioned root would be worse than stopping.
if [ ! -s "$root" ] || [ ! -s "$rootk" ]; then
echo "swarm.ca.autoConfigure is set but there is no root CA key at $rootk" >&2
echo "(swarm-ca.service should have generated it) — refusing to issue a hive CA." >&2
exit 1
fi
echo "issuing fresh hive CA at $ca under the swarm root"
cacsr="$(mktemp "$d/ca.csr.XXXXXX")"
caext="$(mktemp "$d/ca.ext.XXXXXX")"
trap 'rm -f "$cacsr" "$caext"' EXIT
openssl req -newkey rsa:4096 -nodes -sha256 \
-keyout "$cak" -out "$cacsr" \
-subj "/CN=hive-ca ${domain}"
# printf (not a heredoc) so the ext-file lines carry no leading
# whitespace once nix has stripped the indented-string indent.
{
printf 'basicConstraints=critical,CA:TRUE,pathlen:0\n'
printf 'keyUsage=critical,keyCertSign,cRLSign\n'
printf 'subjectKeyIdentifier=hash\n'
printf 'authorityKeyIdentifier=keyid:always\n'
# The constraint is the point of the hierarchy, not a
# flourish: without it a leaked hive CA mints any name in
# the swarm, and it is verifiers that enforce this, not our
# good behaviour. The IP exclusions are not redundant — a
# DNS constraint says nothing about an iPAddress SAN, and a
# name type nobody constrained is a name type this CA is
# unconstrained for.
printf 'nameConstraints=critical,permitted;DNS:%s,excluded;IP:0.0.0.0/0.0.0.0,excluded;IP:0:0:0:0:0:0:0:0/0:0:0:0:0:0:0:0\n' \
${lib.escapeShellArg domain}
} > "$caext"
openssl x509 -req -in "$cacsr" -CA "$root" -CAkey "$rootk" \
-CAcreateserial -days ${toString cfg.caValidityDays} -sha256 \
-extfile "$caext" -out "$ca"
rm -f "$cacsr" "$caext"
trap - EXIT
''
else
''
echo "generating fresh self-signed hive CA at $ca"
openssl req -x509 -newkey rsa:4096 -nodes -sha256 \
-days ${toString cfg.caValidityDays} \
-keyout "$cak" -out "$ca" \
-subj "/CN=hive-ca ${domain}" \
-addext "basicConstraints=critical,CA:TRUE,pathlen:0" \
-addext "keyUsage=critical,keyCertSign,cRLSign"
'';
signLeafScript = pkgs.writeShellScript "hive-tls-sign-leaf" ''
set -euo pipefail
d="$1"
@ -86,12 +156,13 @@ in
# and leaf rotation never re-breaks them. See `docs/gateway.md`
# ("Self-signed TLS").
#
# The hive CA is itself an intermediate, issued under the swarm root
# (./swarm-ca.nix, which owns the why) and name-constrained to this
# hive's domain — so it stays the anchor agents pin, while a peer that
# trusts only the root can still validate everything this hive serves.
# Hives that predate the root keep their self-signed CA until an
# operator drops it; see the issuance comment below.
# That CA is self-signed by default. Under
# `swarm.ca.autoConfigure` — the all-on-one-host case — it is instead
# an intermediate issued under the swarm root (./swarm-ca.nix owns the
# why) and name-constrained to this hive's domain, so it stays the
# anchor agents pin while a peer holding only the root can validate
# everything this hive serves. Either way an existing CA is left
# alone; see the issuance comment below.
options.services.hyperhive.tls = {
stateDir = lib.mkOption {
@ -177,11 +248,11 @@ in
root=${lib.escapeShellArg "${swarmCaCfg.stateDir}/root.pem"}
rootk=${lib.escapeShellArg "${swarmCaCfg.stateDir}/root-key.pem"}
# --- CA: an intermediate under the swarm root, generated once
# and reused across leaf rotations. Regenerated only if missing
# or already expired (checkend 0). A new CA means every consumer
# must re-trust, so the leaf is dropped to force a re-sign under
# the fresh CA.
# --- CA: generated once and reused across leaf rotations, in
# whichever of the two shapes `caGenScript` selected.
# Regenerated only if missing or already expired (checkend 0).
# A new CA means every consumer must re-trust, so the leaf is
# dropped to force a re-sign under the fresh CA.
#
# An existing CA is never re-rooted here. A hive predating the
# swarm root carries a self-signed `ca.pem`, and swapping it for
@ -193,49 +264,7 @@ in
# to hives that never adopt it.
if [ ! -s "$ca" ] || [ ! -s "$cak" ] \
|| ! openssl x509 -in "$ca" -noout -checkend 0 >/dev/null 2>&1; then
# Signing needs the root's private key, which on a multi-host
# swarm is deliberately somewhere else. There is nothing to
# fall back to: a self-signed CA here would still serve TLS
# and would silently not be part of the swarm's trust.
if [ ! -s "$root" ] || [ ! -s "$rootk" ]; then
echo "no swarm root CA key at $rootk — cannot issue this hive's CA." >&2
echo "Either set services.hyperhive.swarm.ca.autoConfigure (single-host swarm)," >&2
echo "or install the operator-issued hive CA at $ca + $cak." >&2
exit 1
fi
echo "issuing fresh hive CA at $ca under the swarm root"
cacsr="$(mktemp "$d/ca.csr.XXXXXX")"
caext="$(mktemp "$d/ca.ext.XXXXXX")"
trap 'rm -f "$cacsr" "$caext"' EXIT
openssl req -newkey rsa:4096 -nodes -sha256 \
-keyout "$cak" -out "$cacsr" \
-subj "/CN=hive-ca ${domain}"
# printf (not a heredoc) so the ext-file lines carry no leading
# whitespace once nix has stripped the indented-string indent.
{
printf 'basicConstraints=critical,CA:TRUE,pathlen:0\n'
printf 'keyUsage=critical,keyCertSign,cRLSign\n'
printf 'subjectKeyIdentifier=hash\n'
printf 'authorityKeyIdentifier=keyid:always\n'
# The constraint is the point of the hierarchy, not a
# flourish: without it a leaked hive CA mints any name in
# the swarm, and it is verifiers that enforce this, not our
# good behaviour. The IP exclusions are not redundant — a
# DNS constraint says nothing about an iPAddress SAN, and a
# name type nobody constrained is a name type this CA is
# unconstrained for.
printf 'nameConstraints=critical,permitted;DNS:%s,excluded;IP:0.0.0.0/0.0.0.0,excluded;IP:0:0:0:0:0:0:0:0/0:0:0:0:0:0:0:0\n' \
${lib.escapeShellArg domain}
} > "$caext"
openssl x509 -req -in "$cacsr" -CA "$root" -CAkey "$rootk" \
-CAcreateserial -days ${toString cfg.caValidityDays} -sha256 \
-extfile "$caext" -out "$ca"
rm -f "$cacsr" "$caext"
trap - EXIT
${caGenScript}
chmod 0600 "$cak"
chmod 0644 "$ca"
rm -f "$leaf" "$leafk"

30
nix/host-modules/swarm-ca.nix
View file

@ -52,23 +52,25 @@ in
options.services.hyperhive.swarm.ca = {
autoConfigure = lib.mkOption {
type = lib.types.bool;
default = hyperhiveCfg.swarm.peers == { };
defaultText = lib.literalExpression "services.hyperhive.swarm.peers == { }";
example = false;
default = false;
example = true;
description = ''
Generate the swarm root CA on this host when it is missing.
Run the whole swarm CA on this one host: generate the swarm
root when it is missing, and issue this hive's CA under it.
Defaults to true exactly while this hive declares no peers —
an all-on-one-host swarm has no cross-hive trust to establish,
so it should cost no configuration. Declaring a peer turns it
off, because a hive that has peers is by definition not the
only place a root could come from, and a second independently
generated root is not a swarm root at all. Set it explicitly
to `true` on the host that does hold the root, or to `false`
to require the operator to provide it.
**Off by default, deliberately.** A swarm's services and its
hives can live on different hosts, and this host has no way to
tell whether it is the one holding the root — so the swarm CA
is something an operator sets up, not something a host decides
it is. Turn this on for an all-on-one-host deployment (dev
boxes, single-hive swarms) and get the hierarchy for free.
Turning this off does not disable anything else: the root is
read from the same `stateDir` either way.
With it off, both artifacts are operator-provided: the root
under `stateDir`, and this hive's CA under
`services.hyperhive.tls.stateDir`. A hive given neither keeps
the self-signed CA it has always had — it simply isn't part of
a swarm's trust hierarchy, which is the correct outcome for a
hive nobody has federated yet.
'';
};