vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

swarm CA: retire the host-held root CA, the store holds the only anchor

Browse source 
The secret store's pki root issues every hive's gateway leaf and the
swarm's service names, so the root swarm-ca.service generated on the
host anchors nothing and its key is a secret outside the store.

- swarm-ca.nix is gone: swarm-ca.service, root.pem and root-key.pem,
  and the swarm.ca.{autoConfigure,stateDir,validityDays} options. A
  definition of any of them fails evaluation with a removal message
  (swarm-ca-removed.nix), and a tmpfiles rule deletes /var/lib/swarm-ca
  on every host. singleHostSwarm no longer sets a swarm.* option.
- hive-tls-ca drops the $root slot: trust-bundle.pem holds the store's
  pki root and a hive CA's public ca.pem/ca-previous.pem only.
- hivectl peer-config no longer prints the scp line for the swarm root;
  the store's pki root is the one step.
- The gateway's cert import no longer stands a hive leaf or a throwaway
  in for a missing swarm-services leaf where it Requires= the issuing
  unit: it fails, naming swarm-services-cert.service. The stand-in stays
  where the gateway does not wait (no swarm name, or the store's UI
  only, whose passthrough must start with the store sealed).

Comments and docs that named swarm-ca.nix, swarm.ca.* or the host root
now name the store's pki root.

Refs #4588
This commit is contained in:
atlas 2026-10-05 19:37:35 +02:00 • committed by mara
commit 93f1a76ea7
22 changed files with 305 additions and 395 deletions
Download patch file Download diff file Expand all files Collapse all files

2
README.md
View file

@ -150,7 +150,7 @@ covers spreading a swarm across hosts.
```
`singleHostSwarm` turns on everything an all-on-one-box swarm implies:
the shared services above, the swarm CA, the swarm controller, and
the shared services above, the swarm controller, and
`/etc/hosts` entries so the hive's names resolve on the box itself.
`claude-code` is unfree — hyperhive scopes the allowance to itself, so
there's nothing to set.

4
docs/networking/gateway.md
View file

@ -76,11 +76,11 @@ On by default, listening on `httpsPort` (default 443) beside the plain-http `por
The issuer is the **secret store's `pki` mount**, not a bare self-signed leaf. `hive-tls-resign.service` (from `hive-tls.nix`) asks the store for a gateway **leaf** through this hive's own role, presenting the hive's own store identity (`services.hyperhive.deploy.hive-controller.tls.baoHiveIdentityCertFile` / `baoHiveIdentityKeyFile`), and writes it under `…tls.stateDir` (default `/var/lib/hive-tls`). `hive-gateway-self-signed-cert` then imports the leaf into nginx's state dir (`/var/lib/hive-gateway/tls/{cert,key}.pem`). On a host that runs no hive the import unit installs the swarm-services leaf there instead, or a throwaway self-signed pair until that leaf lands. [`swarm/ca.md`](../swarm/ca.md) has the role, the identity and what a hive without one does.
⚠️ **Keep the import unit.** It does two jobs nginx needs: it re-modes the key to `0640 root:nginx` (nginx's pre-start `nginx -t` runs as the nginx user and fails on the source's `0600 root:root` key), and it makes sure **every cert path the config names exists** — when the swarm-services leaf is missing it installs the hive leaf in its place, or on a host that runs no hive a throwaway self-signed pair. nginx refuses a config naming a missing cert file, so without that fallback one missing leaf takes down every vhost, not just one.
⚠️ **Keep the import unit.** It does two jobs nginx needs: it re-modes the key to `0640 root:nginx` (nginx's pre-start `nginx -t` runs as the nginx user and fails on the source's `0600 root:root` key), and it puts a file at **every cert path the config names**, because nginx refuses a config naming a missing cert file and one missing leaf would take down every vhost, not just one. Where the gateway waits for the swarm-services leaf, a missing one fails the import unit, naming `swarm-services-cert.service`, and nginx stays down until that unit issues it. Where it doesn't — a host fronting no swarm name, or only the store's UI, which starts its passthrough with the store sealed — the hive leaf stands in until the services leaf lands, or on a host that runs no hive a throwaway self-signed pair.
**Why an issuer, not a bare leaf**: a bare self-signed leaf is its own trust anchor, so every regeneration is a new anchor every consumer must re-trust, and an agent's build sets its trust store once, before any leaf exists. Agents and federation peers trust the store's root once; leaf rotation never breaks them.
**What consumers trust**: `trust-bundle.pem` in the same state dir. It carries the store's `pki` root, plus a hive CA's public `ca.pem` and the swarm root where they exist ([`swarm/ca.md`](../swarm/ca.md#the-trust-bundle)). nginx serves the leaf alone, since its issuer is a self-signed root. Agents (via `security.pki.certificateFiles`), the CI and forge containers and federating peers all read the bundle.
**What consumers trust**: `trust-bundle.pem` in the same state dir. It carries the store's `pki` root, plus a hive CA's public `ca.pem` where one exists ([`swarm/ca.md`](../swarm/ca.md#the-trust-bundle)). nginx serves the leaf alone, since its issuer is a self-signed root. Agents (via `security.pki.certificateFiles`), the CI and forge containers and federating peers all read the bundle.
**Why on by default**: matrix-dart-sdk (FluffyChat's SDK) fetches `https://<host>/.well-known/matrix/client` and never falls back to plain http, so without TLS the browser client can't bootstrap.

47
docs/swarm/ca.md
View file

@ -83,8 +83,9 @@ the file every consumer anchors on. It holds whichever of these exist:
issues under that CA; they stay in the bundle so a leaf it signed verifies
until `hive-tls-resign` replaces it. Removing them is a deliberate later
step.
- the swarm root, `<swarm.ca.stateDir>/root.pem`, on the host that generates
it.
No other anchor goes in: the store's `pki` root is the only root the swarm
has.
No hive generates a CA or signs with a CA key. A
`<deploy.hive-controller.tls.stateDir>/ca-key.pem` stays on disk, unread:
@ -163,46 +164,34 @@ role's `allowed_domains`, enforced by the store at issue time: a hive's
identity that leaks can get certificates for that hive's names and no
others.
The swarm root's private key never reaches the nix store: the store is
world-readable and content-addressed, so a key committed to a flake is a
key published to everyone who builds it. The swarm distributes only
certificates.
No hive holds a root CA or its key, and nothing under `/var/lib/swarm-ca`
exists: every host deletes that directory at boot and on every deploy. The
swarm distributes only certificates.
## What to hand a peer
`hivectl peer-config` prints the `services.hyperhive.swarm.hives."<name>"`
block a peer operator pastes into their own config. When this hive's
gateway serves a self-signed leaf (detected by the presence of
`<deploy.hive-controller.tls.stateDir>/trust-bundle.pem`), it also prints a
one-time `scp` line installing the **swarm root** —
`<swarm.ca.stateDir>/root.pem`, not this hive's own CA — on the peer's
host:
```
scp /var/lib/swarm-ca/root.pem <peer-host>:/var/lib/swarm-ca/root.pem
```
It then prints the secret store's `pki` root itself, read from
`<deploy.hive-controller.tls.stateDir>/trust-bundle.pem`), it also prints the
secret store's `pki` root itself, read from
`/var/lib/hive-tls/swarm-services-root.pem`. The store issues every hive's
gateway leaf and the swarm's service names under that root, so a peer host
trusting only the swarm root can't verify them.
gateway leaf and the swarm's service names under that root, so it's the one
anchor a peer host needs.
The operator installs the roots **once per swarm host**, not once per peer,
The operator installs the root **once per swarm host**, not once per peer,
so a hive joining later needs no edit on the hives already running. A hive
whose cert already chains to a public CA has nothing to install —
`peer-config` says so instead.
## Distributing the root
The root **key** is a runtime file for the reason in
[Constraints on the material](#constraints-on-the-material).
The root **certificate** is a runtime file as a _consequence_ — it lives
beside the key under `swarm.ca.stateDir` — and that has a cost worth
The root **certificate** is a runtime file: each hive fetches it from the
store at boot, so nothing can name it at build time. That has a cost worth
naming, because it's subtle and it bites at a distance:
> **Nothing whose trust store nix assembles at build time can reference
> the swarm root.** The derivation reads `security.pki.certificateFiles`
> the store's root.** The derivation reads `security.pki.certificateFiles`
> inside itself; the root doesn't exist there.
Two consumers, and only one of them is fine:
@ -231,7 +220,7 @@ Two consumers, and only one of them is fine:
> fixed. The same caveat applies to `hive-forge` (Go) for the same
> reason.
To put the root on another host, copy the certificate to the same path
there (`scp <stateDir>/root.pem <host>:<stateDir>/root.pem`). One anchor
per host, not one per peer: a hive joining later needs no edit on the
hives already running, which is the whole point of the hierarchy.
A hive in the swarm fetches the root itself. A host outside it gets the
certificate from `hivectl peer-config` ([What to hand a peer](#what-to-hand-a-peer)).
One anchor per host, not one per peer: a hive joining later needs no edit
on the hives already running.

19
docs/swarm/secrets.md
View file

@ -43,8 +43,6 @@ neither is a renaming of the other.
| secret | generated by | lives at | hive elsewhere |
| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| swarm root CA cert | `swarm-ca.nix` first-boot unit, when the operator sets `autoConfigure` | `/var/lib/swarm-ca/root.pem` | operator copies the **cert** in; it's public |
| swarm root CA key | same unit | `/var/lib/swarm-ca/root-key.pem`, `0600` | stays on whichever host holds it — see the constraint below |
| swarm-services root CA (cert + key) | `swarm-bao.nix`'s granting unit, inside the store's `pki` mount | the **key** never leaves the mount; the **cert** lands at `/var/lib/swarm-bao-services-pki/services-root.pem` (0644) and at `/var/lib/hive-tls/swarm-services-root.pem` on every hive | nothing to copy — every hive receives the cert with the leaf it asks the store for, or from `pki/ca/pem` where it fronts no swarm name and needs a store identity |
| authelia session, JWT and storage-encryption keys | authelia's first-boot unit, in-container | `/var/lib/authelia-swarm/{session,jwt,storage-encryption}.key` | generated in place; nothing outside that container reads them |
| authelia OIDC HMAC key | same unit | `/var/lib/authelia-swarm/oidc-hmac.key` | same |
@ -445,17 +443,12 @@ the hive domain rather than a child** — `forge.<swarm>` next to `<hive>.<swarm
That role can't issue a certificate for a swarm service; the store refuses it.
The store's `swarm-services` role issues them instead, from the root inside its
`pki` mount. The swarm root (`swarm.ca.*`) is still generated on one host, and
two things follow for it:
- **The root's private key is a runtime file and must never enter the nix
store**, so nothing build-time can name it — the system reads
`security.pki.certificateFiles`' value at build time, which makes it the
wrong tool here. Trust reaches containers through a bind-mounted bundle
assembled at boot instead.
- **On any topology other than all-local, placing that key is an operations
decision**, not something this module tree makes for you. A hive that hosts no
swarm services needs only the root's _cert_, to trust what others issue.
`pki` mount. That root's private key never leaves the mount, and its
certificate is the only anchor the swarm has. Each hive fetches the certificate
from the store at boot, so nothing build-time can name it — the system reads
`security.pki.certificateFiles`' value at build time, which makes it the wrong
tool here. Trust reaches containers through a bind-mounted bundle assembled at
boot instead.
## Adding a secret

3
docs/swarm/services.md
View file

@ -20,8 +20,7 @@ sections below don't repeat it, so a service that stops deriving is a
visible difference rather than one more paragraph saying the same thing.
`singleHostSwarm` is the all-on-one-box mode above it. It defaults
`deploy.allSwarmServices`, the swarm CA (`swarm.ca.autoConfigure`, generated
on this host), the swarm controller (`deploy.swarm-controller.enable`), the
`deploy.allSwarmServices`, the swarm controller (`deploy.swarm-controller.enable`), the
host's `/etc/hosts` entries for the names it serves
(`gateway.localHostsEntry`), the queue's auth-callout keys
(`deploy.nats.autoGenerateCallout`) and where the secret store's bootstrap

47
hivectl/src/wg.rs
View file

@ -16,31 +16,17 @@ const WG_INTERFACE: &str = "wg-hive";
/// Host path of this hive's TLS trust bundle (matches the
/// `services.hyperhive.deploy.hive-controller.tls.stateDir` default in
/// hive-tls.nix). Its
/// existence means the gateway serves a self-signed, hive-CA-signed
/// leaf, so a federating peer needs an anchor for it — which is now the
/// swarm root, not a per-hive CA. Absent = ACME / operator cert, trusted
/// by the default CA bundle with nothing to distribute.
///
/// The bundle rather than `ca.pem`: the hive CA is an intermediate under
/// the swarm root, so `ca.pem` alone is not a chain a peer can validate
/// against (openssl will not stop at a trusted non-self-signed cert).
/// The bundle is whatever this hive is currently rooted at — the CA
/// alone on a hive that predates the swarm root — so this stays a single
/// path with no mode to branch on.
/// hive-tls.nix). Its existence means the gateway is in self-signed mode,
/// serving a leaf the secret store issued, so a federating peer needs an
/// anchor for it. Absent = ACME / operator cert, trusted by the default CA
/// bundle with nothing to distribute.
const HIVE_TLS_TRUST_BUNDLE_PATH: &str = "/var/lib/hive-tls/trust-bundle.pem";
/// Host path of the swarm root CA cert (matches the
/// `services.hyperhive.swarm.ca.stateDir` default in swarm-ca.nix). The
/// anchor a whole swarm shares: install it once per host and every
/// present *and future* hive under it validates, which is what replaced
/// the per-hive CA pinning.
const SWARM_CA_ROOT_PATH: &str = "/var/lib/swarm-ca/root.pem";
/// Host path of the secret store's `pki` root (hive-tls.nix's
/// `$svcroot`, folded into the trust bundle). The swarm's service names
/// are issued under it, so a peer trusting only the swarm root above
/// cannot validate them.
/// `$svcroot`, folded into the trust bundle). The anchor a whole swarm
/// shares: every hive's gateway leaf and the swarm's service names are
/// issued under it, so a peer host installs it once and every present
/// *and future* hive validates.
const STORE_PKI_ROOT_PATH: &str = "/var/lib/hive-tls/swarm-services-root.pem";
/// Attrset key for a hive in `services.hyperhive.swarm.hives`, derived
@ -190,8 +176,8 @@ pub(crate) fn wg_peer(domain: &str, pubkey: &str, address: &str, endpoint: Optio
/// certificate itself, or where it will appear when this hive has not
/// fetched it yet.
fn store_root_step(pem: Option<&str>) -> String {
const HEAD: &str = "# 2. trust the SECRET STORE's pki root on the peer host (once, not per hive);\n\
# the swarm's service names are issued under it:\n";
const HEAD: &str = "# 1. trust the SECRET STORE's pki root on the peer host (once, not per hive);\n\
# every hive's gateway leaf and the swarm's service names are issued under it:\n";
match pem.map(str::trim_end).filter(|p| !p.is_empty()) {
Some(p) => format!(
"{HEAD}{p}\n\
@ -205,8 +191,8 @@ fn store_root_step(pem: Option<&str>) -> String {
}
/// `peer-config` — print the `swarm.hives."<name>"` block a peer
/// operator pastes to federate with THIS hive, plus the swarm-root and
/// store-root steps when this hive serves a self-signed chain. Reads
/// operator pastes to federate with THIS hive, plus the store-root step
/// when this hive serves a self-signed chain. Reads
/// local state only (the TLS trust bundle, the store root + the wg key);
/// prints, never mutates.
pub(crate) fn peer_config(domain: &str, wg_address: Option<&str>, wg_endpoint: Option<&str>) {
@ -219,17 +205,10 @@ pub(crate) fn peer_config(domain: &str, wg_address: Option<&str>, wg_endpoint: O
.and_then(|k| wg_pubkey(&k).ok());
if self_signed {
// One anchor for the whole swarm, installed once per host — not
// a file per peer. That is the point of the hierarchy: a hive
// joining later needs no edit on the hives already running.
println!("# 1. install the SWARM ROOT on the peer host (once, not per hive):");
println!("scp {SWARM_CA_ROOT_PATH} <peer-host>:{SWARM_CA_ROOT_PATH}");
println!("# (skip if that host already has the swarm root)");
println!();
let store_root = std::fs::read_to_string(STORE_PKI_ROOT_PATH).ok();
print!("{}", store_root_step(store_root.as_deref()));
println!();
println!("# 3. paste into the peer hive's NixOS config:");
println!("# 2. paste into the peer hive's NixOS config:");
} else {
println!("# paste into the peer hive's NixOS config:");
}

2
nix/host-modules/default.nix
View file

@ -47,7 +47,6 @@
./swarm-authelia.nix
./swarm-bao-service.nix
./swarm-bao.nix
./swarm-ca.nix
./swarm-ci-runner.nix
./swarm-secret-publisher.nix
./swarm-nats-service.nix
@ -65,6 +64,7 @@
./swarm-victoriametrics.nix
./swarm-wireguard.nix
./swarm.nix
./swarm-ca-removed.nix
./swarm-peers-removed.nix
./swarm-required-services.nix
];

2
nix/host-modules/deploy.nix
View file

@ -4,7 +4,7 @@
# different kinds of fact and only one of them varies per machine:
#
# swarm.* — swarm-wide truth. The swarm's name, domain, hives, peers,
# CA, and where each service lives. **Identical on every
# and where each service lives. **Identical on every
# host**, byte for byte; a hive needs all of it to be a
# *client* of the swarm.
# deploy.* — this machine's deployment decisions. Necessarily different

8
nix/host-modules/hive-c0re/default.nix
View file

@ -128,10 +128,10 @@ in
./theme.nix
# Hive-CA trust for this daemon's outbound TLS. The matrix provisioning
# URL (`swarm.matrix.apiUrl`) defaults to https on the homeserver's
# gateway vhost, which serves a leaf chaining to the store's services
# root or the swarm root — CAs generated at runtime, which no
# build-time trust store can contain. The queue's TLS is verified
# against `HIVE_C0RE_OIDC_CA_FILE` (../hive-tls.nix), not this bundle.
# gateway vhost, which serves a leaf chaining to the store's pki root —
# a CA generated at runtime, which no build-time trust store can
# contain. The queue's TLS is verified against `HIVE_C0RE_OIDC_CA_FILE`
# (../hive-tls.nix), not this bundle.
#
# ⚠️ `SSL_CERT_FILE` REPLACES the trust store, and unlike the other
# consumers of this helper, hive-c0re already makes a *public*-TLS call:

4
nix/host-modules/hive-forge/service.nix
View file

@ -100,8 +100,8 @@ in
⚠️ To use the per-hive name `forge.''${services.hyperhive.domain}`
instead, pin it here. Certificates follow either way: the
swarm-services sub-CA is name-constrained to the configured
names (see `./swarm-ca.nix`), not to a fixed tree.
store's `swarm-services` role issues for the configured names,
not for a fixed tree.
Set to a full hostname (`git.example.com`,
`forge.internal.lan`, etc.) for a bespoke vhost shape — the

81
nix/host-modules/hive-gateway/default.nix
View file

@ -29,6 +29,16 @@ let
# The hive leaf exists only where a hive runs; ../hive-tls.nix issues it
# under that same condition.
runsHive = config.services.hyperhive.deploy.hive-controller.enable;
# Whether the cert import `Requires=` the services leaf's issuing unit
# (../hive-tls.nix decides). Where it does, a missing leaf fails the
# import rather than anything standing in for it.
svcLeafRequired = lib.elem "hive-gateway-self-signed-cert.service" (
config.systemd.services.swarm-services-cert.requiredBy or [ ]
);
svcLeafMissing = ''
echo "no swarm-services leaf at ${tlsStateDir}/swarm-services.pem, and this gateway serves swarm names only with it." >&2
echo "swarm-services-cert.service issues it from the secret store's pki mount; its journal says why it has not." >&2
exit 1'';
# Every vhost claiming `default_server`, ours and the operator's
# alike. Computed once so the assertion below and the message it
@ -252,10 +262,9 @@ in
# 🚨 DO NOT "simplify" this into pointing nginx at the CA dir. It does
# TWO jobs: it re-modes the leaf to the 0640 root:nginx copy below (a
# 0600 key fails nginx's pre-start config test and blocks the unit),
# and it guarantees every cert path the config names exists — which is
# what the swarm-services fallback at the bottom of the script is for.
# A leaf that never issues therefore takes the WHOLE gateway down, not
# one vhost. Both mechanisms, and what it cost the time this was
# and it puts a file at every cert path the config names or fails: one
# nginx cannot find takes the WHOLE gateway down, not one vhost. Both
# mechanisms, and what it cost the time this was
# collapsed: `docs/networking/gateway.md::Self-signed TLS (default)`.
#
# ALWAYS runs (no ConditionPathExists) and is idempotent, because it
@ -307,49 +316,51 @@ in
# because the hive leaf's role allows only its own hive's
# domain, and the service names are siblings of it.
#
# Absent is a normal state, not a failure: a host fronting
# no swarm name has none, and issuance can also fail on a
# host that wants one.
#
# ⚠️ When it is absent the HIVE leaf goes to this path
# anyway, and that fallback is load-bearing rather than
# tidy. nginx refuses to load a config naming a cert file
# that does not exist — `cannot load certificate … no such
# file` fails the pre-start test, so the vhost does not
# degrade, the ENTIRE gateway dies and takes the forge, the
# dashboard and matrix with it. Serving the hive leaf on a
# swarm-service name is a name mismatch: browsers warn,
# strict clients refuse, everything else keeps working, and
# the operator gets a bad cert instead of no hive.
#
# Measured, not theorised: this exact path took pr1ma's
# gateway down when the services sub-CA failed to issue.
# Where the gateway waits for it, a missing leaf fails this
# unit and nginx with it. Elsewhere — no swarm name here, or
# only the store's UI, whose host must start its passthrough
# with the store sealed — the hive leaf stands in, because
# nginx refuses a config naming a missing cert file, until
# `swarm-services-cert` issues one and re-runs this unit.
if [ -s ${tlsStateDir}/swarm-services.pem ]; then
install -m 0644 ${tlsStateDir}/swarm-services.pem ${svcCert}
install -m 0640 -g nginx ${tlsStateDir}/swarm-services-key.pem ${svcKey}
else
echo "no swarm-services leaf — serving the hive leaf on those names (mismatch, not an outage)" >&2
install -m 0644 ${tlsStateDir}/gateway.pem ${svcCert}
install -m 0640 -g nginx ${tlsStateDir}/gateway-key.pem ${svcKey}
${
if svcLeafRequired then
svcLeafMissing
else
''
echo "no swarm-services leaf yet — serving the hive leaf on those names until swarm-services-cert.service issues one" >&2
install -m 0644 ${tlsStateDir}/gateway.pem ${svcCert}
install -m 0640 -g nginx ${tlsStateDir}/gateway-key.pem ${svcKey}''
}
fi''
else
''
# No hive here, so no hive leaf: every vhost, the catch-all
# included, serves the swarm-services leaf. Until it lands a
# throwaway self-signed pair stands in, because nginx refuses
# a config naming a missing cert file. A store-only host does
# not wait for the leaf, so its stream passthrough comes up with
# the store sealed. `swarm-services-cert` re-runs this unit
# once the leaf lands.
# included, serves the swarm-services leaf. Where the gateway
# waits for it, a missing leaf fails this unit. A store-only
# host does not wait, so its stream passthrough comes up with
# the store sealed: a throwaway self-signed pair stands in,
# because nginx refuses a config naming a missing cert file,
# until `swarm-services-cert` issues the leaf and re-runs this
# unit.
if [ -s ${tlsStateDir}/swarm-services.pem ] && [ -s ${tlsStateDir}/swarm-services-key.pem ]; then
cert=${tlsStateDir}/swarm-services.pem
key=${tlsStateDir}/swarm-services-key.pem
else
echo "no swarm-services leaf yet — serving a throwaway self-signed certificate until it lands" >&2
cert=${tlsDir}/fallback.pem
key=${tlsDir}/fallback-key.pem
(umask 077; openssl req -x509 -newkey rsa:2048 -nodes -sha256 -days 30 \
-subj "/CN=hyperhive-gateway-fallback" -keyout "$key" -out "$cert")
${
if svcLeafRequired then
svcLeafMissing
else
''
echo "no swarm-services leaf yet — serving a throwaway self-signed certificate until it lands" >&2
cert=${tlsDir}/fallback.pem
key=${tlsDir}/fallback-key.pem
(umask 077; openssl req -x509 -newkey rsa:2048 -nodes -sha256 -days 30 \
-subj "/CN=hyperhive-gateway-fallback" -keyout "$key" -out "$cert")''
}
fi
install -m 0644 "$cert" ${tlsCert}
install -m 0640 -g nginx "$key" ${tlsKey}

9
nix/host-modules/hive-matrix.nix
View file

@ -12,7 +12,7 @@ let
hyperhiveDomain = config.services.hyperhive.domain;
# Same runtime→build-time bridge hive-forge and swarm-ci-runner cross:
# binds the hive trust bundle (which folds in the swarm root) into the
# binds the hive trust bundle (which folds in the store's root) into the
# container and orders the container after `hive-tls-ca.service`. The
# assembled bundle itself comes from `caTrust.trustBundle`, imported in
# the container config below.
@ -783,10 +783,9 @@ in
# Swarm-internal trust reaches tuwunel at RUNTIME, not via
# `security.pki.certificateFiles`. That option is read when the
# system is BUILT, and the swarm root is deliberately a runtime
# file (`swarm.ca.stateDir`) because its key must never enter the
# store — so there is nothing build-time to name. The bind-mount
# above plus the bundle service below are what replaced it.
# system is BUILT, and the store's root is a runtime file fetched
# from the store at boot — so there is nothing build-time to name.
# The bind-mount above plus the bundle service below carry it.
# tuwunel hard-fails to boot if `/etc/resolv.conf` has no
# `nameserver` line (`Failed to configure DNS resolver ... no

20
nix/host-modules/hive-tls.nix
View file

@ -8,7 +8,6 @@ let
cfg = config.services.hyperhive.deploy.hive-controller.tls;
hyperhiveCfg = config.services.hyperhive;
gatewayCfg = config.services.hyperhive.gateway;
swarmCaCfg = config.services.hyperhive.swarm.ca;
domain = hyperhiveCfg.domain;
baoCfg = hyperhiveCfg.swarm.bao;
baoDeploy = hyperhiveCfg.deploy.bao;
@ -346,15 +345,6 @@ in
};
config = lib.mkIf active {
# 🚫 There is deliberately NO warning that these service names are
# outside this hive's domain and this host cannot sign for them. The
# services leaf comes from the secret store's `pki` mount, which every
# hive reaches over the network with its own identity, so holding the
# swarm root key does not decide whether a hive can serve a
# swarm-service name. Such a warning would fire on every rebuild of
# every hive that is not the CA host, about a fallback that does not
# happen.
# The state dir and the trust bundle. Never waits on the store: the
# store's container is ordered after this unit, which writes the
# bundle it bind-mounts, so the leaf is `hive-tls-resign`'s.
@ -366,8 +356,6 @@ in
# turn `Requires=`. Order against the unit that reads the dir.
before = [ "hive-gateway-self-signed-cert.service" ];
requiredBy = [ "hive-gateway-self-signed-cert.service" ];
# The bundle carries the swarm root where this host generates it.
#
# `swarm-services-cert` is NOT ordered against this unit in either
# direction, and the absence is load-bearing — see the cycle spelled
# out at that unit. It runs after the store's container, the container
@ -381,7 +369,6 @@ in
# "$svcroot" ]` guard is what makes that state legal rather than fatal,
# and `swarm-services-cert` restarts this unit once the root it wrote
# is new — so the bundle converges instead of being a boot behind.
after = lib.optionals swarmCaCfg.autoConfigure [ "swarm-ca.service" ];
serviceConfig = {
Type = "oneshot";
RemainAfterExit = true;
@ -396,7 +383,6 @@ in
ca="$d/ca.pem"
prev="$d/ca-previous.pem"
root=${lib.escapeShellArg "${swarmCaCfg.stateDir}/root.pem"}
svcroot="$d/swarm-services-root.pem"
# --- Trust bundle: what a consumer must TRUST. Why it is a set of
@ -423,7 +409,6 @@ in
anchors=()
if [ -s "$ca" ]; then anchors+=("$ca"); fi
if [ -s "$prev" ]; then anchors+=("$prev"); fi
if [ -s "$root" ]; then anchors+=("$root"); fi
if [ -s "$svcroot" ]; then anchors+=("$svcroot"); fi
# `cat` with no arguments reads stdin, so no anchor at all is an
# empty file until `swarm-services-cert` writes the root.
@ -834,10 +819,7 @@ in
# flattening the chain: the issuer here is a self-signed root, so
# appending it would ship an anchor a client either already holds
# — in which case it is redundant — or does not, in which case
# shipping it proves nothing. The old file appended the services
# sub-CA because that WAS an intermediate, and a client anchored
# on the swarm root could not build the middle of the chain by
# itself.
# shipping it proves nothing.
mv -f "$d/swarm-services-leaf.new" "$svcleaf"
mv -f "$d/swarm-services-key.pem.new" "$d/swarm-services-key.pem"
chmod 0644 "$svcleaf"

27
nix/host-modules/local-defaults.nix
View file

@ -32,7 +32,6 @@ in
toggles that an all-on-one-box deployment implies: the swarm's
shared services
(`services.hyperhive.deploy.allSwarmServices`), the swarm
CA (`services.hyperhive.swarm.ca.autoConfigure`), the swarm
controller (`services.hyperhive.deploy.swarm-controller.enable`), and the
host's `/etc/hosts` entries for the names this hive serves
(`services.hyperhive.gateway.localHostsEntry`) — with no real DNS
@ -71,11 +70,8 @@ in
# break every agent's access to the forge.
config.services.hyperhive.gateway.localHostsEntry = lib.mkDefault cfg.deploy.singleHostSwarm;
# Out of the `swarm` attrset below, because it is a `deploy.*` option now
# (./deploy.nix): "does THIS host run the swarm's services" is a per-host
# decision. Written as a path rather than folded into a second
# `config.services.hyperhive.deploy = { … }` attrset, for the same reason
# the ⚠️ below gives about `swarm`.
# A `deploy.*` option (./deploy.nix): "does THIS host run the swarm's
# services" is a per-host decision.
config.services.hyperhive.deploy.allSwarmServices = lib.mkDefault cfg.deploy.singleHostSwarm;
# The queue's auth-callout nkeys. Generating them is safe exactly
@ -86,15 +82,6 @@ in
config.services.hyperhive.deploy.nats.autoGenerateCallout =
lib.mkDefault cfg.deploy.singleHostSwarm;
# ⚠️ Every `swarm.*` default this mode sets lives INSIDE this attrset, not
# as a second `config.services.hyperhive.swarm.…` path beside it — written
# that way the two definitions of `swarm` collide and the nested one is
# silently lost. The gate caught exactly that on the controller's queue URL,
# when this mode still set it: mode on, `natsUrl` still "".
config.services.hyperhive.swarm = {
ca.autoConfigure = lib.mkDefault cfg.deploy.singleHostSwarm;
};
# The controller is asserted by the MODE and by nothing else. Its own
# option stays `default = false` precisely because running it is a
# statement about swarm topology — but "this box is the whole
@ -103,11 +90,8 @@ in
# instead would be wrong: a hive in a larger swarm can legitimately
# want the shared services without being the host that controls them.
#
# Sits outside the `swarm` attrset above because it is a `deploy.*`
# option (./deploy.nix): "does THIS host run the controller" is exactly
# the per-host fact `swarm.*` may not carry. The ⚠️ collision note above
# does not apply here — that one is about two definitions of `swarm`
# itself, and this is a different top-level path.
# A `deploy.*` option (./deploy.nix): "does THIS host run the controller"
# is exactly the per-host fact `swarm.*` may not carry.
config.services.hyperhive.deploy.swarm-controller.enable = lib.mkDefault cfg.deploy.singleHostSwarm;
# Where the operator drops the token that writes the swarm's first bao
@ -122,8 +106,7 @@ in
# substitute for the other: one unseals the store, this one authorises the
# first write.
#
# Outside the `swarm` attrset above for the same reason the controller's
# options are — a path on this host is a `deploy.*` fact.
# A path on this host is a `deploy.*` fact, like the controller's options.
config.services.hyperhive.deploy.bao.bootstrapTokenFile = lib.mkIf cfg.deploy.singleHostSwarm (
lib.mkDefault "/var/lib/swarm-bao-bootstrap/grant.token"
);

47
nix/host-modules/swarm-bao.nix
View file

@ -597,17 +597,15 @@ let
#
# The mount's own root is generated into it by the bootstrap unit below and
# its private key never leaves — `root/generate/internal` keeps it inside
# the store, which is the whole reason the mount exists rather than another
# key beside ./swarm-ca.nix's.
# the store.
#
# An option rather than a literal because ./hive-tls.nix issues through it
# and has to spell it the same way.
servicesPkiMountPath = baoDeploy.servicesPkiMountPath;
# Subject of the root generated into that mount. A label for a human reading
# a chain, not an identity anything authenticates against — same fall-through
# ./swarm-ca.nix:29-37 uses, and for the same reason: a hive that has set
# neither a swarm name nor a domain must still evaluate.
# a chain, not an identity anything authenticates against. Falls through so
# a hive that has set neither a swarm name nor a domain still evaluates.
servicesPkiRootLabel =
if hyperhiveCfg.swarm.name != null then
hyperhiveCfg.swarm.name
@ -624,11 +622,9 @@ let
servicesPkiRootDir = "/var/lib/swarm-bao-services-pki";
servicesPkiRootFile = "${servicesPkiRootDir}/services-root.pem";
# ~30y, matching ./swarm-ca.nix's root rather than its services sub-CA. This
# is an ANCHOR: an operator adds it to a browser's trust store by hand, and
# every rotation is a manual re-trust on every machine that holds it. The
# sub-CA's short window bought cheap rotation because the root above it
# absorbed the churn; nothing sits above this one.
# ~30y. This is an ANCHOR: an operator adds it to a browser's trust store by
# hand, and every rotation is a manual re-trust on every machine that holds
# it; nothing sits above this one to absorb the churn.
servicesPkiRootTtl = "262800h";
# The leaf's window, pinned on the role rather than left to the mount's
@ -645,16 +641,16 @@ let
# The role every swarm-service certificate will be issued through, and the
# whole narrowing of this mount. A role rather than a name-constrained
# sub-CA: the engine enforces the same "these names and no others" at issue
# time that ./swarm-ca.nix builds into x509 `nameConstraints` today, so a
# second CA in the chain would re-implement a check that already happens.
# sub-CA: the engine enforces "these names and no others" at issue time, so
# x509 `nameConstraints` on a second CA in the chain would re-implement a
# check that already happens.
servicesPkiRoleName = baoDeploy.servicesPkiRoleName;
# The names that role may issue for. Read out of the swarm-tier option
# rather than assembled here, exactly as ./swarm-ca.nix:45 and
# ./hive-tls.nix:16 read it — the thing that narrows what may be issued and
# the leaf that carries those names as SANs must agree, and a second module
# composing its own list is how they stop agreeing.
# rather than assembled here, exactly as ./hive-tls.nix reads it — the thing
# that narrows what may be issued and the leaf that carries those names as
# SANs must agree, and a second module composing its own list is how they
# stop agreeing.
swarmServiceDomains = hyperhiveCfg.swarm.serviceDomains;
# The identity that asks this mount for a certificate. A FOURTH principal
@ -1374,7 +1370,7 @@ in
clientCaFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-ca/root.pem";
example = "/var/lib/swarm-bao-pki/ca.pem";
description = ''
Authority the store validates hive **client** certificates against.
@ -1385,7 +1381,7 @@ in
On a hive that runs the store, a glue module supplies the CA it
minted for exactly this, as a `mkDefault`. Point this at something
else — the swarm root, an operator's own CA — and yours wins.
else — an operator's own CA — and yours wins.
`null` leaves client-certificate verification off, which is only
appropriate where something else authenticates the connection.
@ -2806,21 +2802,18 @@ in
#
# Every narrowing is load-bearing, and three of them turn OFF a
# default that is on:
# * `allow_subdomains` — the x509 constraint ./swarm-ca.nix writes
# permits a whole subtree per name; a swarm service is an exact
# hostname, so the role is the narrower of the two.
# * `allow_subdomains` — a swarm service is an exact hostname, not
# a subtree.
# * `allow_localhost` — bao's own default is `true`, which would
# make this role issue for a name nobody configured.
# * `allow_ip_sans` — the sub-CA excludes both IP subtrees
# explicitly (./swarm-ca.nix:260-276); an unconstrained name type
# is a name type this issuer is unconstrained for.
# * `allow_ip_sans` — an unconstrained name type is a name type
# this issuer is unconstrained for.
# `server_flag` alone because that is what today's leaf carries
# (`extendedKeyUsage=serverAuth`, ./hive-tls.nix:144), and rsa:4096
# because that is the key the current implementation generates.
#
# An empty `swarm.serviceDomains` renders an empty `allowed_domains`,
# which with `allow_any_name=false` is a role that issues nothing —
# the same outcome as ./swarm-ca.nix's "nothing to issue for" exit,
# which with `allow_any_name=false` is a role that issues nothing,
# reached without a branch.
bao write ${lib.escapeShellArg "${servicesPkiMountPath}/roles/${servicesPkiRoleName}"} \
allowed_domains=${lib.escapeShellArg (lib.concatStringsSep "," swarmServiceDomains)} \

33
nix/host-modules/swarm-ca-removed.nix Normal file
View file

@ -0,0 +1,33 @@
# Removal shim for `services.hyperhive.swarm.ca.*` and the host-held swarm
# root it generated.
#
# The secret store's `pki` root is the only trust anchor: it issues every
# hive's gateway leaf and the swarm's service names, and its key never leaves
# the store. Nothing generates, reads or bundles a root under
# `/var/lib/swarm-ca`, so a definition of these options fails evaluation with
# a message, and the directory is deleted on every host.
#
# Self-contained and deletable, like ./swarm-peers-removed.nix: one file to
# remove when the deprecation window closes.
{ lib, ... }:
let
removed =
name:
lib.mkRemovedOptionModule [ "services" "hyperhive" "swarm" "ca" name ] ''
The host-held swarm root CA is gone: the secret store's pki root is the
only trust anchor, and nothing generates or reads a root under
/var/lib/swarm-ca. Remove this definition. See docs/swarm/ca.md.
'';
in
{
imports = map removed [
"autoConfigure"
"stateDir"
"validityDays"
];
# The root's private key is a secret outside the store, and the
# certificate beside it is an anchor nothing issues under. The option's
# default path only: a relocated `stateDir` is the operator's to delete.
systemd.tmpfiles.rules = [ "R /var/lib/swarm-ca" ];
}

161
nix/host-modules/swarm-ca.nix
View file

@ -1,161 +0,0 @@
# The swarm root CA: the anchor a whole swarm shares, and the issuer of
# each hive's own CA (which is where it gets used — see ./hive-tls.nix).
#
# One root rather than per-peer pinning, because cross-hive trust is then
# O(1): trust the root once and every present *and future* peer validates,
# instead of every hive having to name every other one.
#
# Two provisioning modes share ONE structure — what differs is who puts
# the artifacts on disk, never what the artifacts are. Which mode does
# what, and what moving between them costs:
# `docs/swarm/ca.md::Two provisioning modes, one structure`.
#
# ⚠️ Root key AND root cert are runtime files, never nix options. The key
# for the obvious reason; the cert as a consequence, and that one costs
# something real — nothing whose trust store is built at build time can
# name it: `docs/swarm/ca.md::Distributing the root`.
#
# ⚠️ This root does not issue the swarm's SERVICE certificates; there is no
# `swarm-services-ca` sub-CA under it. Those come out of the secret store's
# own `pki` mount, anchored on a root generated inside it (./swarm-bao.nix),
# and ./hive-tls.nix asks for them over the network instead of signing them
# here. This root has one job: the anchor each hive's own CA is issued under.
{
lib,
config,
pkgs,
...
}:
let
cfg = config.services.hyperhive.swarm.ca;
hyperhiveCfg = config.services.hyperhive;
# The subject CN is a label for a human reading a chain, not an
# identity anything authenticates against. Fall through swarm name →
# hive domain → a constant so a hive that has set neither still
# evaluates; a missing `domain` is reported by its own assertion in
# hive-network.nix, and shouldn't also surface here as a null.
swarmLabel =
if hyperhiveCfg.swarm.name != null then
hyperhiveCfg.swarm.name
else if hyperhiveCfg.domain != null then
hyperhiveCfg.domain
else
"hyperhive";
in
{
options.services.hyperhive.swarm.ca = {
autoConfigure = lib.mkOption {
type = lib.types.bool;
default = false;
example = true;
description = ''
Run the whole swarm CA on this one host: generate the swarm
root when it is missing, and issue this hive's CA under it.
`services.hyperhive.deploy.singleHostSwarm` turns this on as
part of the all-on-one-box mode. Set it here directly to run the
CA on a host that is not otherwise all-local.
**Off by default, deliberately.** A swarm's services and its
hives can live on different hosts, and this host has no way to
tell whether it is the one holding the root — so the swarm CA
is something an operator sets up, not something a host decides
it is. Turn this on for an all-on-one-host deployment (dev
boxes, single-hive swarms) and get the hierarchy for free.
With it off, both artifacts are operator-provided: the root
under `stateDir`, and this hive's CA under
`services.hyperhive.deploy.hive-controller.tls.stateDir`. A hive
given neither keeps the self-signed CA it has always had — it
simply isn't part of a swarm's trust hierarchy, which is the
correct outcome for a hive nobody has federated yet.
'';
};
stateDir = lib.mkOption {
type = lib.types.str;
default = "/var/lib/swarm-ca";
description = ''
Host directory holding the swarm root CA: `root.pem` (the
anchor, safe to distribute — copy it to this same path on every
other host in the swarm) and `root-key.pem`
(0600, the one file that must never reach the nix store or
another host). The directory itself is 0700: nothing reads
out of it but the hive CA issuance in `hive-tls.nix`.
Moving the swarm CA to its own host is a matter of moving this
directory and setting `autoConfigure = false` here.
'';
};
validityDays = lib.mkOption {
type = lib.types.int;
default = 10950;
description = ''
Validity window of the swarm root CA in days (default ~30y).
Deliberately long: the root must outlive the hive CAs it issued, or those chains
expire out from under hives that are still perfectly happy with
their own intermediate. Rotating a root is the one operation in
this system with no partial-failure mode — it invalidates every
peer at once, paced by the slowest peer's rebuild — so it is
never automatic and this window is meant to be uneventful.
'';
};
};
config = lib.mkIf cfg.autoConfigure {
systemd.services.swarm-ca = {
description = "Generate the swarm root CA when absent";
wantedBy = [ "multi-user.target" ];
path = [ pkgs.openssl ];
serviceConfig = {
Type = "oneshot";
RemainAfterExit = true;
UMask = "0077";
# Pin the journal identity (else it's the `script` store-path wrapper).
SyslogIdentifier = "swarm-ca";
};
script = ''
set -euo pipefail
d=${lib.escapeShellArg cfg.stateDir}
install -d -m 0700 "$d"
root="$d/root.pem"
rootk="$d/root-key.pem"
# Note the asymmetry with the hive CA in hive-tls.nix, which
# regenerates itself once expired: a root is never replaced
# automatically, not even an expired one. Consumers hold this
# cert, so replacing it is a swarm-wide flag day that wants an
# operator running it deliberately, with both roots trusted
# across the overlap.
if [ -s "$root" ] && [ -s "$rootk" ]; then
echo "swarm root CA already present at $root — leaving it alone"
exit 0
fi
# Half a root is not a root. Generating a fresh key beside an
# already-distributed cert (or the reverse) leaves every
# consumer trusting an anchor that no longer signs anything —
# and it would look like it worked.
if [ -e "$root" ] || [ -e "$rootk" ]; then
echo "swarm root CA half-provisioned ($root / $rootk) — refusing to generate over it" >&2
exit 1
fi
echo "generating swarm root CA at $root"
# pathlen:1 — the root signs hive CAs, which sign leaves. One
# intermediate below the root and no deeper.
openssl req -x509 -newkey rsa:4096 -nodes -sha256 \
-days ${toString cfg.validityDays} \
-keyout "$rootk" -out "$root" \
-subj "/CN=swarm-ca ${swarmLabel}" \
-addext "basicConstraints=critical,CA:TRUE,pathlen:1" \
-addext "keyUsage=critical,keyCertSign,cRLSign"
chmod 0600 "$rootk"
chmod 0644 "$root"
'';
};
};
}

2
nix/host-modules/swarm-controller.nix
View file

@ -669,7 +669,7 @@ in
hiveClientCaFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-ca/root.pem";
example = "/var/lib/swarm-bao-pki/ca.pem";
description = ''
Authority whose leaves are hive client certificates, so that this
daemon can create each hive's cert-auth role. The store matches a

9
nix/host-modules/swarm-peers-removed.nix
View file

@ -57,11 +57,10 @@ in
services.hyperhive.swarm.peers.<hive>.caCert is removed and
ignored, and has no replacement in services.hyperhive.swarm.hives.
Trust inside a swarm now derives from the swarm root CA
(services.hyperhive.swarm.ca — see docs/swarm/ca.md): every hive
under it chains to it, so a per-hive CA is dead weight. What this
genuinely drops is trusting a hive whose root this swarm does NOT
own — another swarm's, or one keeping its own CA.
Trust inside a swarm derives from the secret store's pki root (see
docs/swarm/ca.md): every hive chains to it, so a per-hive CA is dead
weight. What this genuinely drops is trusting a hive whose root this
swarm does NOT own — another swarm's, or one keeping its own CA.
Still set on: ${lib.concatStringsSep ", " withCaCert}
'';

26
nix/host-modules/swarm.nix
View file

@ -72,13 +72,13 @@ let
# `serviceDomains` below is this set sorted + deduplicated.
#
# ⚠️ These are NOT required to be under `swarm.domain`. An earlier
# revision asserted that, reasoning that the services sub-CA is
# constrained to the swarm's tree — but the sub-CA is constrained to
# the **configured names** (./swarm-ca.nix) and the swarm root carries
# no name constraints at all, so any configured name is issuable. The
# assertion encoded an intended shape, not a property of the code, and
# it rejected the supported migration path: a hive pinning its old
# `forge.<hive domain>` while joining a swarm at a different apex.
# revision asserted that, reasoning that the services issuer is
# constrained to the swarm's tree — but the store's `swarm-services`
# role admits exactly the **configured names** (./swarm-bao.nix), so any
# configured name is issuable. The assertion encoded an intended shape,
# not a property of the code, and it rejected the supported migration
# path: a hive pinning its old `forge.<hive domain>` while joining a swarm
# at a different apex.
#
# Every name is listed whether or not this host runs the service: the
# store's issuing role is built from this list on the store's host and
@ -226,9 +226,9 @@ in
visible = false;
internal = true;
description = ''
Removed. Trust inside a swarm comes from the swarm root CA
(`services.hyperhive.swarm.ca`, docs/swarm/ca.md), which
replaces per-hive leaf pinning entirely.
Removed. Trust inside a swarm comes from the secret store's
pki root (docs/swarm/ca.md), which replaces per-hive leaf
pinning entirely.
'';
};
};
@ -369,9 +369,9 @@ in
existed to serve — nothing else ever consumed it.
There is no replacement, and none is needed for a hive inside
this swarm: trust comes from the swarm root CA
(services.hyperhive.swarm.ca — see docs/swarm/ca.md), which
every hive chains to, so one anchor replaces per-hive pinning.
this swarm: trust comes from the secret store's pki root (see
docs/swarm/ca.md), which every hive chains to, so one anchor
replaces per-hive pinning.
What that genuinely drops is trusting a hive whose root this
swarm does NOT own — another swarm's, or one keeping its own
CA. That is a cross-swarm problem and wants a mechanism

109
nix/module-eval/hive-tls.nix
View file

@ -4,7 +4,8 @@
# The swarm-services leaf's renewal: a timer that really re-runs the
# issuance, at a threshold read off the store role's lifetime. And the
# store's root on a hive that asks it for no leaf. And the gateway leaf,
# issued by the store through the hive's own role, with no hive CA.
# issued by the store through the hive's own role, with no hive CA and no
# host-held swarm root.
{
pkgs,
lib,
@ -21,6 +22,7 @@ let
nixosSystem
;
})
hive
hiveWithController
runGroup
;
@ -208,10 +210,9 @@ let
!(holds noService) && holds allLocal;
}
{
# The controls are the store's PKI unit, which still mints its own CA
# with `openssl req`, and the swarm root's unit, which still reads
# `root-key.pem`: absence here is not a matcher that cannot match.
name = "no hive TLS unit generates a CA, signs with a CA key or reads the swarm root key";
# The control is the store's PKI unit, which still mints its own CA
# with `openssl req`: absence here is not a matcher that cannot match.
name = "no hive TLS unit generates a CA or signs with a CA key";
ok =
let
hasCaWork =
@ -219,16 +220,106 @@ let
lib.any (p: lib.hasInfix p s) [
"openssl req"
"-CAkey"
"root-key.pem"
];
pkiScript = units.swarm-bao-pki.script;
rootScript = units.swarm-ca.script;
in
!(hasCaWork caUnit.script)
&& !(hasCaWork leafUnit.script)
&& !(hasCaWork noService.systemd.services.hive-tls-ca.script)
&& lib.hasInfix "openssl req" pkiScript
&& lib.hasInfix "root-key.pem" rootScript;
&& lib.hasInfix "openssl req" pkiScript;
}
{
# Every unit's script, environment and ordering, on the all-local host
# and on a hive that runs no store. The control is the same scan finding
# the store root's file, so an empty result is not a scan that reads
# nothing.
name = "no unit generates, reads or waits on a host-held swarm root, and every host deletes its directory";
ok =
let
texts =
m:
lib.concatMap (u: [
(u.script or "")
(builtins.toJSON (u.environment or { }))
(toString ((u.after or [ ]) ++ (u.requires or [ ]) ++ (u.wants or [ ])))
]) (lib.attrValues m.systemd.services);
mentions = m: needle: lib.any (lib.hasInfix needle) (texts m);
gone =
m:
!(m.systemd.services ? swarm-ca)
&& !(mentions m "swarm-ca.service")
&& !(mentions m "/var/lib/swarm-ca")
&& !(mentions m "root-key.pem")
&& lib.elem "R /var/lib/swarm-ca" m.systemd.tmpfiles.rules;
in
gone allLocal
&& gone noService
&& mentions allLocal "swarm-services-root.pem"
&& mentions noService "swarm-services-root.pem";
}
{
# Three slots and no fourth: a root slot beside them is an anchor
# nothing issues under.
name = "the trust bundle anchors on the store's root and a hive CA's public certificates only";
ok =
let
slots = lib.filter (l: lib.hasPrefix "if [ -s " l && lib.hasInfix "anchors+=(" l) (
lib.splitString "\n" caUnit.script
);
in
lib.length slots == 3
&& lib.hasInfix ''if [ -s "$svcroot" ]; then anchors+=("$svcroot"); fi'' caUnit.script
&& !(lib.hasInfix ''anchors+=("$root")'' caUnit.script);
}
{
# The control is the all-local host, which defines none of them.
name = "a definition of a removed swarm.ca option fails evaluation, naming the option and the store's root";
ok =
let
refused =
m: opt:
lib.any (
a:
!a.assertion
&& lib.hasInfix "services.hyperhive.swarm.ca.${opt}" a.message
&& lib.hasInfix "pki root is the" a.message
) m.assertions;
setting = {
autoConfigure = hive { swarm.ca.autoConfigure = true; };
stateDir = hive { swarm.ca.stateDir = "/var/lib/swarm-ca"; };
validityDays = hive { swarm.ca.validityDays = 10950; };
};
in
lib.all (opt: refused setting.${opt} opt) (lib.attrNames setting)
&& !(lib.any (opt: refused allLocal opt) (lib.attrNames setting));
}
{
# Where the import `Requires=` the services leaf's issuing unit, a
# missing leaf fails it; elsewhere the hive leaf stands in. The control
# is a hive fronting no swarm name, which does not wait.
name = "a hive gateway that waits for the services leaf fails its cert import without it, naming the issuing unit";
ok =
let
imp = m: m.systemd.services.hive-gateway-self-signed-cert.script;
waits =
m:
lib.elem "hive-gateway-self-signed-cert.service" m.systemd.services.swarm-services-cert.requiredBy;
standIn = "install -m 0644 /var/lib/hive-tls/gateway.pem /var/lib/hive-gateway/tls/swarm-services.pem";
refusal = "swarm-services-cert.service issues it";
# The `else` branch alone, up to the `fi` that closes it.
missingBranch =
m:
builtins.head (
lib.splitString "\nfi" (lib.last (lib.splitString "swarm-services.pem ]; then" (imp m)))
);
in
waits allLocal
&& lib.hasInfix refusal (missingBranch allLocal)
&& lib.hasSuffix "exit 1" (lib.removeSuffix "\n" (lib.trim (missingBranch allLocal)))
&& !(lib.hasInfix standIn (imp allLocal))
&& !(waits noService)
&& lib.hasInfix standIn (missingBranch noService)
&& !(lib.hasInfix refusal (imp noService));
}
{
# Nothing reads or removes a `ca-key.pem`, so a rollback finds the

38
nix/module-eval/swarm-services-switch.nix
View file

@ -287,30 +287,50 @@ let
lib.elem "hive-gateway-self-signed-cert.service" s.swarm-services-cert.requiredBy
&& lib.elem "hive-gateway-self-signed-cert.service" s.swarm-services-cert.before;
# The cert it installs on the default and swarm-services paths:
# the services leaf when present, a throwaway pair otherwise.
importsServicesLeafOrFallback =
# the services leaf when present.
importsServicesLeaf =
m:
let
imp = m.systemd.services.hive-gateway-self-signed-cert;
in
lib.hasInfix "cert=/var/lib/hive-tls/swarm-services.pem" imp.script
&& lib.hasInfix "openssl req -x509" imp.script
&& lib.hasInfix "cert=/var/lib/hive-gateway/tls/fallback.pem" imp.script
&& lib.hasInfix ''install -m 0644 "$cert" /var/lib/hive-gateway/tls/cert.pem'' imp.script
&& lib.hasInfix ''install -m 0644 "$cert" /var/lib/hive-gateway/tls/swarm-services.pem'' imp.script
&& lib.hasInfix ''install -m 0644 "$cert" /var/lib/hive-gateway/tls/swarm-services.pem'' imp.script;
# Without it: a throwaway pair where the gateway does not wait for
# the leaf, a failure naming the issuing unit where it does.
standsInThrowaway =
m:
let
imp = m.systemd.services.hive-gateway-self-signed-cert;
in
lib.hasInfix "openssl req -x509" imp.script
&& lib.hasInfix "cert=/var/lib/hive-gateway/tls/fallback.pem" imp.script
&& lib.any (p: lib.hasPrefix "openssl" (p.pname or (p.name or ""))) imp.path;
failsWithoutLeaf =
m:
let
s = m.systemd.services.hive-gateway-self-signed-cert.script;
in
lib.hasInfix "swarm-services-cert.service issues it" s
&& lib.hasInfix "exit 1" s
&& !(lib.hasInfix "fallback.pem" s);
store = storeOnlyNoHive;
hs = swarmServicesHere.systemd.services;
in
noHive swarmServicesOnly
&& noHive swarmServicesOnlyNoStore
&& noHive store
&& importsServicesLeafOrFallback swarmServicesOnly
&& importsServicesLeafOrFallback swarmServicesOnlyNoStore
&& importsServicesLeafOrFallback store
# Fronting many swarm names: the gateway waits for their leaf.
&& importsServicesLeaf swarmServicesOnly
&& importsServicesLeaf swarmServicesOnlyNoStore
&& importsServicesLeaf store
# Fronting many swarm names: the gateway waits for their leaf, and
# its import fails without it.
&& waitsForLeaf swarmServicesOnly
&& waitsForLeaf swarmServicesOnlyNoStore
&& failsWithoutLeaf swarmServicesOnly
&& failsWithoutLeaf swarmServicesOnlyNoStore
&& standsInThrowaway store
&& !(failsWithoutLeaf store)
# The store's UI alone: its nginx carries the passthrough to the
# store, so it starts on the fallback with the store sealed.
&&