Watch
0
0
Fork
You've already forked hyperhive
0

swarm-tls: narrow each gateway's services leaf to the names it fronts

Every gateway asked the store's `pki/issue/swarm-services` for the whole
swarm's service set, so a private key on any gateway host could serve a
valid certificate for services that host does not front and never has.

`swarm.localServiceDomains` derives the per-host subset by filtering
`swarm.serviceDomains` against the vhosts this host actually renders —
the deploy flags those vhosts are already guarded on, read once rather
than copied into a second filter. The leaf request and the coverage
guard that decides whether to re-issue both read it, so they cannot
disagree about which names the leaf owes.

The sub-CA's name constraint and the role's `allowed_domains` stay the
swarm-wide set: every host's subset is inside it, and narrowing the
constraint per host would turn one signing into N.
This commit is contained in:
atlas 2026-09-23 22:31:57 +02:00 • committed by mara
commit ed2ec52fe5
5 changed files with 89 additions and 20 deletions

View file

@ -54,10 +54,12 @@ copies from. Every hive does this with its own identity, so holding the
swarm root's private key stopped being what decides whether a hive can swarm root's private key stopped being what decides whether a hive can
serve its swarm's names. serve its swarm's names.
The same `services.hyperhive.swarm.serviceDomains` that builds the SANs `services.hyperhive.swarm.serviceDomains` populates the role's
also populates the role's `allowed_domains`, so asking for a name nobody `allowed_domains`, so asking for a name nobody configured is a refusal
configured is a refusal from the store naming that name — not a from the store naming that name — not a certificate quietly issued for
certificate quietly issued for it. it. A host's own SANs are narrower still: it asks only for the names it
fronts a vhost for (`swarm.localServiceDomains`), so the key on one
gateway can't serve a swarm service that runs behind another.
**The root's public certificate is a file, on every hive:** **The root's public certificate is a file, on every hive:**
`/var/lib/hive-tls/swarm-services-root.pem` (0644), written beside the `/var/lib/hive-tls/swarm-services-root.pem` (0644), written beside the

View file

@ -92,7 +92,8 @@ own options, the way `swarm-ui.nix` and `swarm-authelia.nix` do.
⚠️ The certificate one is the hardest to predict and the most visible when ⚠️ The certificate one is the hardest to predict and the most visible when
missed. `serviceDomains` is _both_ the `allowed_domains` the secret missed. `serviceDomains` is _both_ the `allowed_domains` the secret
store's `pki/roles/swarm-services` narrows to and the leaf's SAN list, store's `pki/roles/swarm-services` narrows to and the list each gateway's
leaf draws its SANs from (it carries the ones that host fronts),
and the apex is a **sibling** of `forge.<swarm>` / `chat.<swarm>` / and the apex is a **sibling** of `forge.<swarm>` / `chat.<swarm>` /
`auth.<swarm>`, not a parent — no CA in the hierarchy issues for it `auth.<swarm>`, not a parent — no CA in the hierarchy issues for it
implicitly. Left out, the vhost falls back to the hive leaf and the implicitly. Left out, the vhost falls back to the hive leaf and the

View file

@ -21,10 +21,19 @@ let
baoServicesPkiMount = baoDeploy.servicesPkiMountPath; baoServicesPkiMount = baoDeploy.servicesPkiMountPath;
baoServicesPkiRole = baoDeploy.servicesPkiRoleName; baoServicesPkiRole = baoDeploy.servicesPkiRoleName;
# Derived once in ./swarm.nix and read here + in ./swarm-bao.nix, so # Derived once in ./swarm.nix: the swarm service names THIS host fronts
# the names this leaf carries as SANs and the names the store's # a vhost for. Every one of them is inside the swarm-wide set that
# `pki/roles/swarm-services` is narrowed to cannot disagree. # ./swarm-bao.nix narrows `pki/roles/swarm-services` to, so the leaf
swarmServiceDomains = hyperhiveCfg.swarm.serviceDomains; # stays issuable while carrying no name this gateway does not serve.
localServiceDomains = hyperhiveCfg.swarm.localServiceDomains;
# The one name the request's `common_name` carries; the rest ride as
# SANs. Not `builtins.head`: a host fronting no swarm service renders
# this script too — its unit exits on `want_svc` long before running
# the request, but a bare `head []` would have failed the EVALUATION.
leafCommonName = lib.optionalString (localServiceDomains != [ ]) (
builtins.head localServiceDomains
);
# The host-managed hive CA is the trust anchor for self-signed mode. # The host-managed hive CA is the trust anchor for self-signed mode.
# It is only stood up when the gateway actually serves a self-signed # It is only stood up when the gateway actually serves a self-signed
@ -195,15 +204,16 @@ let
svcleaf="$d/swarm-services.pem" svcleaf="$d/swarm-services.pem"
svcroot="$d/swarm-services-root.pem" svcroot="$d/swarm-services-root.pem"
hiveNames=${lib.escapeShellArg "${domain} *.${domain}"} hiveNames=${lib.escapeShellArg "${domain} *.${domain}"}
svcNames=${lib.escapeShellArg (lib.concatStringsSep " " swarmServiceDomains)} svcNames=${lib.escapeShellArg (lib.concatStringsSep " " localServiceDomains)}
# A hive with no configured service names has no services leaf to # A host fronting none of the swarm's service names has no services
# hold, and its absence there is the correct state rather than a # leaf to hold, and its absence there is the correct state rather
# stale one. Anywhere else it is expected: the leaf comes from the # than a stale one — its own vhosts are covered by the hive leaf
# above. Anywhere else it is expected: the leaf comes from the
# secret store's `pki` mount now, not from a sub-CA that exists on # secret store's `pki` mount now, not from a sub-CA that exists on
# one host in the swarm, so "this host cannot issue it" is no longer # one host in the swarm, so "this host cannot issue it" is no longer
# one of the answers. # one of the answers.
want_svc=${if swarmServiceDomains == [ ] then "0" else "1"} want_svc=${if localServiceDomains == [ ] then "0" else "1"}
# Expiry is not the only way a leaf goes wrong. One signed when the # Expiry is not the only way a leaf goes wrong. One signed when the
# configured name set was smaller stays valid for its whole lifetime # configured name set was smaller stays valid for its whole lifetime
@ -681,10 +691,11 @@ in
# for it to be an intermediate OF. # for it to be an intermediate OF.
# #
# The names it may carry are not this unit's to assert: the role # The names it may carry are not this unit's to assert: the role
# refuses anything outside `allowed_domains`, which is read from the # refuses anything outside `allowed_domains`, read from the
# same `swarm.serviceDomains` the SANs below are built from. A # swarm-wide `swarm.serviceDomains` that the SANs below are a
# mismatch is a refusal from the store naming the offending name, # per-host SUBSET of. A mismatch is a refusal from the store naming
# not a certificate quietly issued for something nobody configured. # the offending name, not a certificate quietly issued for something
# nobody configured.
systemd.services.swarm-services-cert = { systemd.services.swarm-services-cert = {
description = "Issue the swarm-services TLS leaf from the secret store's PKI"; description = "Issue the swarm-services TLS leaf from the secret store's PKI";
wantedBy = [ "multi-user.target" ]; wantedBy = [ "multi-user.target" ];
@ -847,8 +858,8 @@ in
# the store, not this host, decides what the certificate says. # the store, not this host, decides what the certificate says.
if ! bao write -format=json \ if ! bao write -format=json \
${lib.escapeShellArg "${baoServicesPkiMount}/issue/${baoServicesPkiRole}"} \ ${lib.escapeShellArg "${baoServicesPkiMount}/issue/${baoServicesPkiRole}"} \
common_name=${lib.escapeShellArg (builtins.head swarmServiceDomains)} \ common_name=${lib.escapeShellArg leafCommonName} \
alt_names=${lib.escapeShellArg (lib.concatStringsSep "," swarmServiceDomains)} \ alt_names=${lib.escapeShellArg (lib.concatStringsSep "," localServiceDomains)} \
> "$resp" 2>"$err"; then > "$resp" 2>"$err"; then
echo "the store refused to issue the swarm-services certificate." >&2 echo "the store refused to issue the swarm-services certificate." >&2
cat "$err" >&2 cat "$err" >&2

View file

@ -299,6 +299,27 @@ in
''; '';
}; };
options.services.hyperhive.swarm.localServiceDomains = lib.mkOption {
type = lib.types.listOf lib.types.str;
readOnly = true;
internal = true;
description = ''
Read-only: the names in `serviceDomains` **this host actually
fronts**, which is what its services leaf may carry. The swarm-wide
set stays what the sub-CA is name-constrained to and what the
store's issuing role permits; narrowing the leaf is what stops a
private key on one gateway presenting a valid certificate for
services that host does not serve and never has.
Read off the rendered `services.nginx.virtualHosts` rather than off
the per-service deploy flags: those flags are what the vhosts are
already guarded on, so a second reading of them is a copy that
drifts from the thing it claims to describe. Filtering
`serviceDomains` rather than assembling a list beside it is that
option's own constraint, and it applies here for the same reason.
'';
};
config = { config = {
services.hyperhive.swarm.peerHives = lib.filterAttrs (name: _: name != cfg.hiveName) swarmCfg.hives; services.hyperhive.swarm.peerHives = lib.filterAttrs (name: _: name != cfg.hiveName) swarmCfg.hives;
@ -306,6 +327,10 @@ in
lib.unique (lib.filter (d: d != null && d != "") serviceDomains') lib.unique (lib.filter (d: d != null && d != "") serviceDomains')
); );
services.hyperhive.swarm.localServiceDomains = lib.filter (
d: config.services.nginx.virtualHosts ? ${d}
) swarmCfg.serviceDomains;
assertions = [ assertions = [
{ {
# An EMPTY `hives` fires this too, deliberately: since # An EMPTY `hives` fires this too, deliberately: since

View file

@ -52,6 +52,10 @@ let
deploy.forgejo.ci.enable = true; deploy.forgejo.ci.enable = true;
}; };
# A hive whose one gateway-published swarm service sits on another host:
# every swarm name still configured, not one of them served here.
forgeElsewhere = hive { deploy.forgejo.behindGateway = false; };
# A priority collision is a property of the *option*, not # A priority collision is a property of the *option*, not
# of the merged value's interior — nix throws the moment the value is # of the merged value's interior — nix throws the moment the value is
# demanded at all, so `seq`-ing each `serviceConfig` value to WHNF is # demanded at all, so `seq`-ing each `serviceConfig` value to WHNF is
@ -199,6 +203,32 @@ let
in in
(l.tlsFor "t.local").sslCertificate != (l.tlsFor "_").sslCertificate; (l.tlsFor "t.local").sslCertificate != (l.tlsFor "_").sslCertificate;
} }
{
# A leaf is a key that can SERVE every name in it, so the names it
# asks for are that key's blast radius. `bare` fronts forge and
# nothing else; the last conjunct is the control, since `auth.t.local`
# is in the swarm's set and an unnarrowed request would carry it here
# too.
name = "a gateway's services leaf asks for only the swarm names that host fronts";
ok =
let
s = bare.services.hyperhive.swarm.serviceDomains;
u = bare.systemd.services.swarm-services-cert.script;
in
lib.hasInfix "alt_names=forge.t.local" u
&& !(lib.hasInfix "auth.t.local" u)
&& lib.elem "auth.t.local" s;
}
{
# The narrowing's floor: no names left means no request, rather than a
# request for an empty SAN set that the store would refuse. Its own
# vhosts are the hive leaf's, which this host still signs.
name = "a host fronting none of the swarm's service names requests no services leaf";
ok =
forgeElsewhere.services.hyperhive.swarm.localServiceDomains == [ ]
&& forgeElsewhere.services.hyperhive.swarm.serviceDomains != [ ]
&& lib.hasInfix "want_svc=0" forgeElsewhere.systemd.services.swarm-services-cert.script;
}
{ {
# nixos asserts when a vhost declares both, so this is also a # nixos asserts when a vhost declares both, so this is also a
# statement that the `removeAttrs` upstream of it still happens. # statement that the `removeAttrs` upstream of it still happens.