swarm-tls: narrow each gateway's services leaf to the names it fronts

Every gateway asked the store's `pki/issue/swarm-services` for the whole
swarm's service set, so a private key on any gateway host could serve a
valid certificate for services that host does not front and never has.

`swarm.localServiceDomains` derives the per-host subset by filtering
`swarm.serviceDomains` against the vhosts this host actually renders —
the deploy flags those vhosts are already guarded on, read once rather
than copied into a second filter. The leaf request and the coverage
guard that decides whether to re-issue both read it, so they cannot
disagree about which names the leaf owes.

The sub-CA's name constraint and the role's `allowed_domains` stay the
swarm-wide set: every host's subset is inside it, and narrowing the
constraint per host would turn one signing into N.
This commit is contained in:
atlas 2026-09-23 22:31:57 +02:00 • committed by mara
commit ed2ec52fe5
5 changed files with 89 additions and 20 deletions

View file

@ -54,10 +54,12 @@ copies from. Every hive does this with its own identity, so holding the
swarm root's private key stopped being what decides whether a hive can
serve its swarm's names.
The same `services.hyperhive.swarm.serviceDomains` that builds the SANs
also populates the role's `allowed_domains`, so asking for a name nobody
configured is a refusal from the store naming that name — not a
certificate quietly issued for it.
`services.hyperhive.swarm.serviceDomains` populates the role's
`allowed_domains`, so asking for a name nobody configured is a refusal
from the store naming that name — not a certificate quietly issued for
it. A host's own SANs are narrower still: it asks only for the names it
fronts a vhost for (`swarm.localServiceDomains`), so the key on one
gateway can't serve a swarm service that runs behind another.
**The root's public certificate is a file, on every hive:**
`/var/lib/hive-tls/swarm-services-root.pem` (0644), written beside the

View file

@ -92,7 +92,8 @@ own options, the way `swarm-ui.nix` and `swarm-authelia.nix` do.
⚠️ The certificate one is the hardest to predict and the most visible when
missed. `serviceDomains` is _both_ the `allowed_domains` the secret
store's `pki/roles/swarm-services` narrows to and the leaf's SAN list,
store's `pki/roles/swarm-services` narrows to and the list each gateway's
leaf draws its SANs from (it carries the ones that host fronts),
and the apex is a **sibling** of `forge.<swarm>` / `chat.<swarm>` /
`auth.<swarm>`, not a parent — no CA in the hierarchy issues for it
implicitly. Left out, the vhost falls back to the hive leaf and the

View file

@ -21,10 +21,19 @@ let
baoServicesPkiMount = baoDeploy.servicesPkiMountPath;
baoServicesPkiRole = baoDeploy.servicesPkiRoleName;
# Derived once in ./swarm.nix and read here + in ./swarm-bao.nix, so
# the names this leaf carries as SANs and the names the store's
# `pki/roles/swarm-services` is narrowed to cannot disagree.
swarmServiceDomains = hyperhiveCfg.swarm.serviceDomains;
# Derived once in ./swarm.nix: the swarm service names THIS host fronts
# a vhost for. Every one of them is inside the swarm-wide set that
# ./swarm-bao.nix narrows `pki/roles/swarm-services` to, so the leaf
# stays issuable while carrying no name this gateway does not serve.
localServiceDomains = hyperhiveCfg.swarm.localServiceDomains;
# The one name the request's `common_name` carries; the rest ride as
# SANs. Not `builtins.head`: a host fronting no swarm service renders
# this script too — its unit exits on `want_svc` long before running
# the request, but a bare `head []` would have failed the EVALUATION.
leafCommonName = lib.optionalString (localServiceDomains != [ ]) (
builtins.head localServiceDomains
);
# The host-managed hive CA is the trust anchor for self-signed mode.
# It is only stood up when the gateway actually serves a self-signed
@ -195,15 +204,16 @@ let
svcleaf="$d/swarm-services.pem"
svcroot="$d/swarm-services-root.pem"
hiveNames=${lib.escapeShellArg "${domain} *.${domain}"}
svcNames=${lib.escapeShellArg (lib.concatStringsSep " " swarmServiceDomains)}
svcNames=${lib.escapeShellArg (lib.concatStringsSep " " localServiceDomains)}
# A hive with no configured service names has no services leaf to
# hold, and its absence there is the correct state rather than a
# stale one. Anywhere else it is expected: the leaf comes from the
# A host fronting none of the swarm's service names has no services
# leaf to hold, and its absence there is the correct state rather
# than a stale one — its own vhosts are covered by the hive leaf
# above. Anywhere else it is expected: the leaf comes from the
# secret store's `pki` mount now, not from a sub-CA that exists on
# one host in the swarm, so "this host cannot issue it" is no longer
# one of the answers.
want_svc=${if swarmServiceDomains == [ ] then "0" else "1"}
want_svc=${if localServiceDomains == [ ] then "0" else "1"}
# Expiry is not the only way a leaf goes wrong. One signed when the
# configured name set was smaller stays valid for its whole lifetime
@ -681,10 +691,11 @@ in
# for it to be an intermediate OF.
#
# The names it may carry are not this unit's to assert: the role
# refuses anything outside `allowed_domains`, which is read from the
# same `swarm.serviceDomains` the SANs below are built from. A
# mismatch is a refusal from the store naming the offending name,
# not a certificate quietly issued for something nobody configured.
# refuses anything outside `allowed_domains`, read from the
# swarm-wide `swarm.serviceDomains` that the SANs below are a
# per-host SUBSET of. A mismatch is a refusal from the store naming
# the offending name, not a certificate quietly issued for something
# nobody configured.
systemd.services.swarm-services-cert = {
description = "Issue the swarm-services TLS leaf from the secret store's PKI";
wantedBy = [ "multi-user.target" ];
@ -847,8 +858,8 @@ in
# the store, not this host, decides what the certificate says.
if ! bao write -format=json \
${lib.escapeShellArg "${baoServicesPkiMount}/issue/${baoServicesPkiRole}"} \
common_name=${lib.escapeShellArg (builtins.head swarmServiceDomains)} \
alt_names=${lib.escapeShellArg (lib.concatStringsSep "," swarmServiceDomains)} \
common_name=${lib.escapeShellArg leafCommonName} \
alt_names=${lib.escapeShellArg (lib.concatStringsSep "," localServiceDomains)} \
> "$resp" 2>"$err"; then
echo "the store refused to issue the swarm-services certificate." >&2
cat "$err" >&2

View file

@ -299,6 +299,27 @@ in
'';
};
options.services.hyperhive.swarm.localServiceDomains = lib.mkOption {
type = lib.types.listOf lib.types.str;
readOnly = true;
internal = true;
description = ''
Read-only: the names in `serviceDomains` **this host actually
fronts**, which is what its services leaf may carry. The swarm-wide
set stays what the sub-CA is name-constrained to and what the
store's issuing role permits; narrowing the leaf is what stops a
private key on one gateway presenting a valid certificate for
services that host does not serve and never has.
Read off the rendered `services.nginx.virtualHosts` rather than off
the per-service deploy flags: those flags are what the vhosts are
already guarded on, so a second reading of them is a copy that
drifts from the thing it claims to describe. Filtering
`serviceDomains` rather than assembling a list beside it is that
option's own constraint, and it applies here for the same reason.
'';
};
config = {
services.hyperhive.swarm.peerHives = lib.filterAttrs (name: _: name != cfg.hiveName) swarmCfg.hives;
@ -306,6 +327,10 @@ in
lib.unique (lib.filter (d: d != null && d != "") serviceDomains')
);
services.hyperhive.swarm.localServiceDomains = lib.filter (
d: config.services.nginx.virtualHosts ? ${d}
) swarmCfg.serviceDomains;
assertions = [
{
# An EMPTY `hives` fires this too, deliberately: since

View file

@ -52,6 +52,10 @@ let
deploy.forgejo.ci.enable = true;
};
# A hive whose one gateway-published swarm service sits on another host:
# every swarm name still configured, not one of them served here.
forgeElsewhere = hive { deploy.forgejo.behindGateway = false; };
# A priority collision is a property of the *option*, not
# of the merged value's interior — nix throws the moment the value is
# demanded at all, so `seq`-ing each `serviceConfig` value to WHNF is
@ -199,6 +203,32 @@ let
in
(l.tlsFor "t.local").sslCertificate != (l.tlsFor "_").sslCertificate;
}
{
# A leaf is a key that can SERVE every name in it, so the names it
# asks for are that key's blast radius. `bare` fronts forge and
# nothing else; the last conjunct is the control, since `auth.t.local`
# is in the swarm's set and an unnarrowed request would carry it here
# too.
name = "a gateway's services leaf asks for only the swarm names that host fronts";
ok =
let
s = bare.services.hyperhive.swarm.serviceDomains;
u = bare.systemd.services.swarm-services-cert.script;
in
lib.hasInfix "alt_names=forge.t.local" u
&& !(lib.hasInfix "auth.t.local" u)
&& lib.elem "auth.t.local" s;
}
{
# The narrowing's floor: no names left means no request, rather than a
# request for an empty SAN set that the store would refuse. Its own
# vhosts are the hive leaf's, which this host still signs.
name = "a host fronting none of the swarm's service names requests no services leaf";
ok =
forgeElsewhere.services.hyperhive.swarm.localServiceDomains == [ ]
&& forgeElsewhere.services.hyperhive.swarm.serviceDomains != [ ]
&& lib.hasInfix "want_svc=0" forgeElsewhere.systemd.services.swarm-services-cert.script;
}
{
# nixos asserts when a vhost declares both, so this is also a
# statement that the `removeAttrs` upstream of it still happens.