vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

swarm-bao: make the reader's identity declarable, not just the store's

Browse source 
`swarm-bao.nix` declared the store's half of the mTLS pair as options —
`serverCertFile`, `serverKeyFile`, `clientCaFile` — and left the reader's
half as a literal inside `glue-bao-tls.nix`, which only runs where
`deploy.bao.enable` is set. A hive that did not host the store therefore
could not read from it and could not be pointed at a certificate even
when one had been placed by hand.

Adds `clientCertFile`, `clientKeyFile` and `serverCaFile` beside their
three server siblings, `mkDefault`ed by the glue to the leaf it already
mints, and moves `glue-matrix-bao-token.nix` onto them. Its gate becomes
"this host holds an identity" rather than "the store is a neighbour",
and the unit ordering that names store-local units is now conditional --
`Requires=` on an absent unit fails the job.

`serverCaFile` is separate from `clientCaFile` on purpose: one is the
store choosing which readers to trust, the other a reader choosing which
store to trust. Self-signing collapses them to one file, which is a
property of that deployment and not of the pairing.

Closes #3855.
This commit is contained in:
atlas 2026-08-31 19:40:08 +02:00 • committed by mara
commit 694abf4439
6 changed files with 126 additions and 19 deletions
Download patch file Download diff file Expand all files Collapse all files

7
docs/setup.md
View file

@ -67,11 +67,14 @@ Whether anything more is needed depends on
The store serves TLS, and on a hive that deploys it you need do nothing: a
first-boot unit mints a CA of the store's own plus the two leaves it signs —
the store's server certificate and this host's client certificate — and points
`deploy.bao.serverCertFile`, `.serverKeyFile` and `.clientCaFile` at them.
`deploy.bao.serverCertFile`, `.serverKeyFile` and `.clientCaFile` at the store's
half, `.clientCertFile`, `.clientKeyFile` and `.serverCaFile` at the reader's.
Those are `mkDefault`s, so naming your own paths wins. Do that when your
certificates come from a real internal CA; the store has no opinion about
which. ⚠️ Not the gateway's HTTPS certificates and not the hive CA — this is
which. A hive that does **not** deploy the store names the reader's three
itself: that leaf is issued out of band, and it is the one credential the store
cannot hand you, being what opens it. ⚠️ Not the gateway's HTTPS certificates and not the hive CA — this is
**mTLS between services and the store**, a separate trust domain, because a
store that took its identity from an authority it will itself distribute could
never come up before that authority.

4
docs/swarm/secrets.md
View file

@ -156,6 +156,10 @@ mints a CA that signs exactly two things, the store's server certificate and a
reader's client certificate, and distributes nothing. A deployment with a real
internal CA deletes that file and names its own paths in
`deploy.bao.serverCertFile` / `clientCaFile`; the store itself has no opinion.
A hive that reads from a store on **another** machine names the reader's half —
`clientCertFile`, `clientKeyFile`, `serverCaFile` — and places that leaf by hand.
It is the one credential that cannot come out of the store, being what opens it;
everything else a hive needs does.
## The constraint that decides where the root lives

8
nix/host-modules/glue-bao-tls.nix
View file

@ -73,6 +73,14 @@ in
serverCertFile = lib.mkDefault "${pkiDir}/server.pem";
serverKeyFile = lib.mkDefault "${pkiDir}/server-key.pem";
clientCaFile = lib.mkDefault "${pkiDir}/ca.pem";
# A reader on this host, which happens to be the host that mints. Only
# these three are what a reader elsewhere needs placed by hand; that they
# collapse to the same CA file here is a property of self-signing, not of
# the pairing.
clientCertFile = lib.mkDefault "${pkiDir}/client.pem";
clientKeyFile = lib.mkDefault "${pkiDir}/client-key.pem";
serverCaFile = lib.mkDefault "${pkiDir}/ca.pem";
};
# Idempotent on ABSENCE, never on content. Re-issuing the CA invalidates

43
nix/host-modules/glue-matrix-bao-token.nix
View file

@ -16,11 +16,11 @@
# overwrites it with the swarm's copy when the store has one. A store that is
# empty or unreachable leaves a working hive with a local token.
#
# 📌 LIMIT, stated rather than hidden: this gates on the store running HERE.
# A hive reading from a store on another machine needs the same unit with that
# machine's address and a client leaf issued out of band — the mechanism is
# identical, only `-address` and the cert's provenance differ. Deferred until
# there is a second hive to test it against, rather than shipped untested.
# 📌 This runs wherever a client identity is configured, NOT only where the
# store is. `deploy.bao.enable` would have been the co-location assumption
# itself; the reader needs a certificate, not a neighbour. On the store's own
# host ./glue-bao-tls.nix supplies one as a `mkDefault` and nothing changes;
# elsewhere an operator places the leaf and names it, and the same unit works.
{
pkgs,
lib,
@ -33,11 +33,12 @@ let
baoCfg = hyperhiveCfg.swarm.bao;
matrixCfg = hyperhiveCfg.swarm.matrix;
# Owned by ./glue-bao-tls.nix, which mints them. Named here rather than
# shared through a `let`: a cross-module binding would make these two files
# one file with a gap in the middle, and the whole point of a glue module is
# that it can be deleted on its own.
pkiDir = "/var/lib/swarm-bao-pki";
baoDeploy = deployCfg.bao;
# What decides whether this unit exists at all. A reader is defined by holding
# a certificate the store accepts, and that is true on the store's own host
# and on a hive three networks away for exactly the same reason.
haveClientIdentity = baoDeploy.clientCertFile != null && baoDeploy.clientKeyFile != null;
# Where the token lives in the store. A path, not a convention to guess at:
# whoever writes it and whoever reads it must agree, and the agreement
@ -52,15 +53,19 @@ let
matrixMachine = "hive-matrix";
in
{
config = lib.mkIf (hyperhiveCfg.enable && deployCfg.bao.enable && deployCfg.matrix.enable) {
config = lib.mkIf (hyperhiveCfg.enable && haveClientIdentity && deployCfg.matrix.enable) {
systemd.services.swarm-bao-matrix-token = {
description = "fetch the matrix registration token from the swarm secret store";
after = [
# Every one of these names a unit that exists only where the store runs.
# `Requires=` on an absent unit fails the job outright, so the ordering is
# conditional even though the read is not: off-host there is nothing local
# to wait for, and the timeout below is what bounds the attempt instead.
after = lib.optionals baoDeploy.enable [
"swarm-bao-pki.service"
"container@${baoCfg.machine}.service"
];
wants = [ "container@${baoCfg.machine}.service" ];
requires = [ "swarm-bao-pki.service" ];
wants = lib.optionals baoDeploy.enable [ "container@${baoCfg.machine}.service" ];
requires = lib.optionals baoDeploy.enable [ "swarm-bao-pki.service" ];
before = [ "container@${matrixMachine}.service" ];
wantedBy = [ "container@${matrixMachine}.service" ];
path = [
@ -77,9 +82,13 @@ in
};
environment = {
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
BAO_CACERT = "${pkiDir}/ca.pem";
BAO_CLIENT_CERT = "${pkiDir}/client.pem";
BAO_CLIENT_KEY = "${pkiDir}/client-key.pem";
BAO_CLIENT_CERT = baoDeploy.clientCertFile;
BAO_CLIENT_KEY = baoDeploy.clientKeyFile;
}
# Absent means the system trust store, which is what a deployment with a
# real CA wants and what a self-signed one must not be left with.
// lib.optionalAttrs (baoDeploy.serverCaFile != null) {
BAO_CACERT = baoDeploy.serverCaFile;
};
script = ''
set -euo pipefail

48
nix/host-modules/swarm-bao.nix
View file

@ -268,6 +268,54 @@ in
'';
};
clientCertFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-bao/client.pem";
description = ''
Certificate a **reader on this machine** presents to the store.
The counterpart to {option}`services.hyperhive.deploy.bao.clientCaFile`,
which is the store's side of the same handshake. Only the store's side
was declarable, so a hive that did not run the store had no way to be
pointed at a certificate even when one was placed for it.
On a hive that runs the store, a glue module supplies the leaf it minted,
as a `mkDefault`. Everywhere else this is the credential an operator
places by hand — the one secret that cannot come out of the store,
because it is what opens it.
A path, never a value.
'';
};
clientKeyFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-bao/client-key.pem";
description = ''
Private key for {option}`services.hyperhive.deploy.bao.clientCertFile`.
Both or neither — a certificate with no key authenticates nothing.
'';
};
serverCaFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-bao/ca.pem";
description = ''
Authority a **reader on this machine** validates the store's certificate
against.
Not {option}`services.hyperhive.deploy.bao.clientCaFile` with the words
rearranged: that one is the store choosing which readers to trust, this
one is a reader choosing which store to trust. A deployment that
self-signs both ends points them at the same file and reads that as
confirmation they are interchangeable — they are not, and they diverge
the moment either end gets a real CA.
'';
};
extraListenAddresses = lib.mkOption {
type = lib.types.listOf lib.types.str;
default = [ ];

35
nix/module-eval.nix
View file

@ -90,6 +90,18 @@ let
deploy.bao.enable = true;
deploy.matrix.enable = true;
};
# A hive that reads from a store it does not run: no `deploy.bao.enable`, so
# nothing here mints a leaf and the operator names one placed by hand. The
# deployment this pairing exists to serve, and the one that was previously
# inexpressible — the gate asked whether the store was a neighbour.
baoRemoteReader = hive {
deploy.matrix.enable = true;
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
};
# The same hive with the identity taken away, which separates "a homeserver
# is deployed" from "this host can authenticate to the store".
matrixNoBaoIdentity = hive { deploy.matrix.enable = true; };
# A priority collision is a property of the *option*, not
# of the merged value's interior — nix throws the moment the value is
@ -205,6 +217,29 @@ let
name = "a store with no homeserver beside it renders no token reader";
ok = !(baoPkcs11.systemd.services ? swarm-bao-matrix-token);
}
{
name = "a hive that names a client identity reads from a store it does not run";
ok = baoRemoteReader.systemd.services ? swarm-bao-matrix-token;
}
{
# Absence arm for the one above, and the reason the gate is the identity
# rather than the homeserver: without it, deploying matrix anywhere would
# render a reader that cannot authenticate.
name = "a homeserver with no way to authenticate to the store renders no token reader";
ok = !(matrixNoBaoIdentity.systemd.services ? swarm-bao-matrix-token);
}
{
# `Requires=` on a unit that does not exist fails the job, and nothing
# local mints certificates off-host — so this orders against nothing.
# Eval cannot see that failure; only the empty list here stands in for it.
name = "an off-host reader requires no unit the store's host would have provided";
ok = baoRemoteReader.systemd.services.swarm-bao-matrix-token.requires == [ ];
}
{
# Presence control for the case above: the list is conditional, not gone.
name = "a co-located reader still orders after the local pki unit";
ok = baoWithMatrix.systemd.services.swarm-bao-matrix-token.requires == [ "swarm-bao-pki.service" ];
}
];
bad = builtins.filter (c: !c.ok) cases;