vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

hive-tls: fail swarm-services-cert when the store identity is absent

Browse source 
A hive fronting no swarm name fetches the store's pki root presenting its
reader certificate. When deploy.bao.clientCertFile or clientKeyFile is unset,
or either named file is not readable, the unit now exits 1 with a message
naming both options and their paths, instead of exiting 0 with the root
missing from the trust bundle.

The module-eval cases that asserted the quiet skip assert the failure
instead, and a new case covers the named-but-absent files.
This commit is contained in:
atlas 2026-10-04 08:35:12 +02:00 • committed by mara
commit 849bf46550
4 changed files with 52 additions and 16 deletions
Download patch file Download diff file Expand all files Collapse all files

5
docs/swarm/ca.md
View file

@ -65,7 +65,10 @@ gateway can't serve a swarm service that runs behind another.
`/var/lib/hive-tls/swarm-services-root.pem` (0644), written beside the `/var/lib/hive-tls/swarm-services-root.pem` (0644), written beside the
leaf it signed and folded into `trust-bundle.pem`. A hive fronting no leaf it signed and folded into `trust-bundle.pem`. A hive fronting no
swarm name asks for no leaf and reads the root alone, from the store's swarm name asks for no leaf and reads the root alone, from the store's
`pki/ca/pem`, when it holds a store identity (`deploy.bao.clientCertFile`). `pki/ca/pem`, presenting its store identity
(`deploy.bao.clientCertFile` / `clientKeyFile`) for the handshake. Every hive
must hold one: without it, or with either file absent,
`swarm-services-cert.service` fails with a message naming both options.
On the host running On the host running
the store it's also at the store it's also at
`/var/lib/swarm-bao-services-pki/services-root.pem`. That's the file to `/var/lib/swarm-bao-services-pki/services-root.pem`. That's the file to

2
docs/swarm/secrets.md
View file

@ -45,7 +45,7 @@ neither is a renaming of the other.
| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| swarm root CA cert | `swarm-ca.nix` first-boot unit, when the operator sets `autoConfigure` | `/var/lib/swarm-ca/root.pem` | operator copies the **cert** in; it's public | | swarm root CA cert | `swarm-ca.nix` first-boot unit, when the operator sets `autoConfigure` | `/var/lib/swarm-ca/root.pem` | operator copies the **cert** in; it's public |
| swarm root CA key | same unit | `/var/lib/swarm-ca/root-key.pem`, `0600` | stays on whichever host holds it — see the constraint below | | swarm root CA key | same unit | `/var/lib/swarm-ca/root-key.pem`, `0600` | stays on whichever host holds it — see the constraint below |
| swarm-services root CA (cert + key) | `swarm-bao.nix`'s granting unit, inside the store's `pki` mount | the **key** never leaves the mount; the **cert** lands at `/var/lib/swarm-bao-services-pki/services-root.pem` (0644) and at `/var/lib/hive-tls/swarm-services-root.pem` on every hive | nothing to copy — every hive receives the cert with the leaf it asks the store for, or from `pki/ca/pem` where it fronts no swarm name and holds a store identity | | swarm-services root CA (cert + key) | `swarm-bao.nix`'s granting unit, inside the store's `pki` mount | the **key** never leaves the mount; the **cert** lands at `/var/lib/swarm-bao-services-pki/services-root.pem` (0644) and at `/var/lib/hive-tls/swarm-services-root.pem` on every hive | nothing to copy — every hive receives the cert with the leaf it asks the store for, or from `pki/ca/pem` where it fronts no swarm name and needs a store identity |
| authelia session, JWT and storage-encryption keys | authelia's first-boot unit, in-container | `/var/lib/authelia-swarm/{session,jwt,storage-encryption}.key` | generated in place; nothing outside that container reads them | | authelia session, JWT and storage-encryption keys | authelia's first-boot unit, in-container | `/var/lib/authelia-swarm/{session,jwt,storage-encryption}.key` | generated in place; nothing outside that container reads them |
| authelia OIDC HMAC key | same unit | `/var/lib/authelia-swarm/oidc-hmac.key` | same | | authelia OIDC HMAC key | same unit | `/var/lib/authelia-swarm/oidc-hmac.key` | same |
| authelia OIDC issuer key (RSA) | same unit | `/var/lib/authelia-swarm/oidc-issuer.key` | same — relying parties verify against the **public** half at `/jwks.json` | | authelia OIDC issuer key (RSA) | same unit | `/var/lib/authelia-swarm/oidc-issuer.key` | same — relying parties verify against the **public** half at `/jwks.json` |

27
nix/host-modules/hive-tls.nix
View file

@ -797,12 +797,26 @@ in
# `ca/pem` takes no token; the TLS client certificate is the # `ca/pem` takes no token; the TLS client certificate is the
# store's listener's demand, not bao's, so this presents the # store's listener's demand, not bao's, so this presents the
# hive's reader identity and never logs in with it. # hive's reader identity and never logs in with it.
# A hive with no reader identity has no certificate to present, so it # A hive without a readable reader identity fails here, naming the
# skips the fetch and the bundle goes without this root. # options: every hive must hold one.
if [ "$want_svc" = 0 ]; then if [ "$want_svc" = 0 ]; then
${ ${
if haveReaderIdentity then if haveReaderIdentity then
'' ''
missing=""
for f in ${
lib.escapeShellArgs [
baoDeploy.clientCertFile
baoDeploy.clientKeyFile
]
}; do
[ -r "$f" ] || missing="$missing $f"
done
if [ -n "$missing" ]; then
echo "this hive has no usable store identity: cannot read$missing." >&2
echo "place the files named by services.hyperhive.deploy.bao.clientCertFile (${baoDeploy.clientCertFile}) and services.hyperhive.deploy.bao.clientKeyFile (${baoDeploy.clientKeyFile})." >&2
exit 1
fi
echo "no swarm service domains configured — fetching only the store's pki root" echo "no swarm service domains configured — fetching only the store's pki root"
err="$(mktemp)" err="$(mktemp)"
raw="$(mktemp "$d/swarm-services-root.XXXXXX")" raw="$(mktemp "$d/swarm-services-root.XXXXXX")"
@ -838,8 +852,13 @@ in
exit 0'' exit 0''
else else
'' ''
echo "store pki root not staged: this hive has no store identity (services.hyperhive.deploy.bao.clientCertFile/clientKeyFile)" echo "this hive has no store identity, so it cannot fetch the store's ${baoServicesPkiMount} root." >&2
exit 0'' echo "set services.hyperhive.deploy.bao.clientCertFile (${
if baoDeploy.clientCertFile == null then "unset" else baoDeploy.clientCertFile
}) and services.hyperhive.deploy.bao.clientKeyFile (${
if baoDeploy.clientKeyFile == null then "unset" else baoDeploy.clientKeyFile
}) and place both files." >&2
exit 1''
} }
fi fi

34
nix/module-eval/hive-tls.nix
View file

@ -50,11 +50,12 @@ let
noIdentity = hive { }; noIdentity = hive { };
noIdFetch = noIdentity.systemd.services.swarm-services-cert.script; noIdFetch = noIdentity.systemd.services.swarm-services-cert.script;
# The `want_svc=0` branch of a rendered script, up to its closing `fi`. # The `want_svc=0` branch of a rendered script, up to the `fi` that closes it (the
# only one followed by a blank line).
noSvcBranch = noSvcBranch =
script: script:
builtins.head ( builtins.head (
lib.splitString "\nfi\n" (lib.last (lib.splitString ''if [ "$want_svc" = 0 ]; then'' script)) lib.splitString "\nfi\n\n" (lib.last (lib.splitString ''if [ "$want_svc" = 0 ]; then'' script))
); );
units = allLocal.systemd.services; units = allLocal.systemd.services;
@ -159,20 +160,33 @@ let
) noSvcFetch; ) noSvcFetch;
} }
{ {
# Nothing to present means no fetch to fail: the unit stays green and # No identity to present: the unit fails, naming both options, rather
# says why the root is missing. The control is the same hive with an # than leaving the bundle without the root behind a green unit. The
# identity, whose branch does fetch and can fail. # control is the same hive with an identity, whose branch fetches.
name = "a hive fronting no swarm name with no store identity skips the fetch and cannot fail on it"; name = "a hive fronting no swarm name with no store identity fails the unit, naming both options";
ok = ok =
noIdentity.services.hyperhive.swarm.localServiceDomains == [ ] noIdentity.services.hyperhive.swarm.localServiceDomains == [ ]
&& noIdentity.services.hyperhive.deploy.bao.clientCertFile == null && noIdentity.services.hyperhive.deploy.bao.clientCertFile == null
&& !(lib.hasInfix "/v1/pki/ca/pem" noIdFetch) && !(lib.hasInfix "/v1/pki/ca/pem" noIdFetch)
&& lib.hasInfix "store pki root not staged" (noSvcBranch noIdFetch) && lib.hasInfix "services.hyperhive.deploy.bao.clientCertFile" (noSvcBranch noIdFetch)
&& lib.hasInfix "deploy.bao.clientCertFile" (noSvcBranch noIdFetch) && lib.hasInfix "services.hyperhive.deploy.bao.clientKeyFile" (noSvcBranch noIdFetch)
&& !(lib.hasInfix "exit 1" (noSvcBranch noIdFetch)) && lib.hasSuffix "exit 1" (noSvcBranch noIdFetch)
&& lib.hasInfix "exit 1" (noSvcBranch noSvcFetch) && !(lib.hasInfix "exit 0" (noSvcBranch noIdFetch))
&& !(lib.hasInfix "not staged" noIdFetch)
&& lib.hasInfix "/v1/pki/ca/pem" (noSvcBranch noSvcFetch); && lib.hasInfix "/v1/pki/ca/pem" (noSvcBranch noSvcFetch);
} }
{
# An identity named but not on disk fails before curl, naming the paths.
name = "a hive fronting no swarm name whose identity files are absent fails the unit, naming the paths";
ok =
lib.hasInfix "for f in /etc/pki/bao-client.pem /etc/pki/bao-client-key.pem; do" (
noSvcBranch noSvcFetch
)
&& lib.hasInfix ''[ -r "$f" ] || missing="$missing $f"'' (noSvcBranch noSvcFetch)
&& lib.hasInfix "cannot read$missing" (noSvcBranch noSvcFetch)
&& lib.hasInfix "clientCertFile (/etc/pki/bao-client.pem)" (noSvcBranch noSvcFetch)
&& lib.hasInfix "clientKeyFile (/etc/pki/bao-client-key.pem)" (noSvcBranch noSvcFetch);
}
{ {
# The fetch retries against a sealed store; a gateway `Requires=` on it # The fetch retries against a sealed store; a gateway `Requires=` on it
# would go down with each attempt. The control is a host fronting # would go down with each attempt. The control is a host fronting