hive-tls: the gateway leaf uses the services-issuer identity, not a hive's

swarm-services-cert issues the swarm-services leaf a self-signed gateway
serves, and on a host fronting no swarm name fetches the store's pki root.
It is gated on the gateway, not on a hive, yet its leaf pair was named
under deploy.hive-controller.tls and defaulted from the hive reader's cert
directory, and the root fetch presented the hive's own identity. A gateway
host with no hive had no such identity, so that branch was red by
construction.

- The pair moves to deploy.bao.servicesIssuerClient{Cert,Key}File, beside
  the store's other host-unit readers, and glue-bao-tls.nix defaults it on
  the store's host from the leaf it already mints. The old
  deploy.hive-controller.tls.baoClient{Cert,Key}File paths are
  mkRenamedOptionModule aliases. glue-services-issuer-bao-identity.nix,
  which derived the default from deploy.bao.clientCertFile, is gone.
- The no-swarm-name root fetch presents the services-issuer pair. The hive
  identity is now only hive-tls-resign's.
- A self-signed gateway with no pair fails evaluation, naming both options.
  A pair named but unreadable still fails the unit, naming the paths.

Refs #4921

docs/swarm/ca.md

@@ -39,11 +39,10 @@ from the store's own client CA, with the subject
   and set both options, before the host's next deploy.
 
 Without it, `hive-tls-resign.service` fails with a message naming both
-options, and so does `swarm-services-cert.service` on a hive fronting no
+options. The unit checks the identity before the leaf, so it fails even while a
-swarm name. The unit checks the identity before the leaf, so it fails even while a
 valid leaf keeps the gateway up. The identity is the hive's alone:
-it issues this hive's leaf and fetches the store's root, and it's never the
+it issues this hive's leaf and nothing else, and it's never the identity a
-identity a swarm service uses.
+swarm service uses.
 
 **When it issues.** The unit runs at boot, on every deploy, and from
 `hive-tls-resign.timer` once a day. It asks the store for a new leaf when
@@ -108,12 +107,33 @@ domain, so it can't cover them.
 
 The secret store's `pki` mount issues them a leaf of their own, through its
 `swarm-services` role. `swarm-services-cert.service` logs in with
-this host's services-issuer certificate, calls
+the services-issuer identity, calls
 `pki/issue/swarm-services`, and writes the result to
 `/var/lib/hive-tls/swarm-services.pem` — the path the gateway already
-copies from. Every hive does this with its own identity, so the store's
+copies from. The store's role, not a key on the host, decides which names a
-role, not a key on the hive, decides whether a hive can serve its swarm's
+gateway can serve.
-names.
+
+**Every gateway that serves the self-signed leaf needs the services-issuer
+identity**, whether or not a hive runs on its host. It's a client certificate
+from the store's own client CA, with the subject
+`deploy.bao.servicesIssuerCommonName` (`swarm-services-issuer` by default),
+and it's named by `services.hyperhive.deploy.bao.servicesIssuerClientCertFile`
+and `servicesIssuerClientKeyFile`. Its policy may issue from
+`pki/issue/swarm-services` and nothing else, and it's the same certificate on
+every gateway host:
+
+- On the host running the store, `swarm-bao-pki.service` mints it at
+  `/var/lib/swarm-bao-pki/services-issuer.pem` and the two options default to
+  it.
+- On every other gateway host, copy `services-issuer.pem` and
+  `services-issuer-key.pem` from the store's host and set both options.
+
+A host whose gateway serves the self-signed leaf without both options set
+fails evaluation, naming them. One that names files it can't read fails
+`swarm-services-cert.service`, naming the paths. The old spellings,
+`deploy.hive-controller.tls.baoClientCertFile` and `baoClientKeyFile`, are
+renames of these two: a config that still sets them evaluates with a warning
+naming both paths.
 
 `services.hyperhive.swarm.serviceDomains`, every swarm service name and the
 same list on every host, populates the role's
@@ -123,14 +143,12 @@ it. A host's own SANs are narrower still: it asks only for the names it
 fronts a vhost for (`swarm.localServiceDomains`), so the key on one
 gateway can't serve a swarm service that runs behind another.
 
-**The root's public certificate is a file, on every hive:**
+**The root's public certificate is a file, on every self-signed gateway host:**
 `/var/lib/hive-tls/swarm-services-root.pem` (0644), written beside the
-leaf it signed and folded into `trust-bundle.pem`. A hive fronting no
+leaf it signed and folded into `trust-bundle.pem`. A host fronting no
 swarm name asks for no services leaf and reads the root alone, from the store's
-`pki/ca/pem`, presenting its own store identity
+`pki/ca/pem`, presenting the services-issuer identity for the handshake
-(`deploy.hive-controller.tls.baoHiveIdentityCertFile` /
+without logging in with it. With either file absent,
-`baoHiveIdentityKeyFile`) for the handshake. Every hive must hold one:
-without it, or with either file absent,
 `swarm-services-cert.service` fails with a message naming both options.
 On the host running
 the store it's also at

nix/host-modules/default.nix

@@ -40,7 +40,6 @@
     ./glue-nats-oidc-client.nix
     ./glue-queue-agent-credential.nix
     ./glue-secret-publisher-bao-identity.nix
-    ./glue-services-issuer-bao-identity.nix
     ./glue-swarm-bao-otel-oidc-client.nix
     ./glue-swarm-otel-oidc-client.nix
     ./swarm-authelia-service.nix

nix/host-modules/deploy.nix

@@ -174,6 +174,17 @@ in
       [ "services" "hyperhive" "deploy" "ci-runner" "jobTimeout" ]
     )
 
+    # The gateway's services-issuer pair is a swarm-service identity, not a
+    # hive's, so it sits with the store's other host-unit readers.
+    (lib.mkRenamedOptionModule
+      [ "services" "hyperhive" "deploy" "hive-controller" "tls" "baoClientCertFile" ]
+      [ "services" "hyperhive" "deploy" "bao" "servicesIssuerClientCertFile" ]
+    )
+    (lib.mkRenamedOptionModule
+      [ "services" "hyperhive" "deploy" "hive-controller" "tls" "baoClientKeyFile" ]
+      [ "services" "hyperhive" "deploy" "bao" "servicesIssuerClientKeyFile" ]
+    )
+
     # The runner toggle sat under `swarm.forge.ci` before `deploy.forgejo.ci`.
     # Both were per-host, so each forwards straight to its final path.
     (lib.mkRenamedOptionModule

nix/host-modules/glue-bao-tls.nix

@@ -107,6 +107,8 @@ in
       forwarderOidcClientKeyFile = lib.mkDefault "${pkiDir}/forwarder-oidc-key.pem";
       ciRunnerClientCertFile = lib.mkDefault "${pkiDir}/ci-runner.pem";
       ciRunnerClientKeyFile = lib.mkDefault "${pkiDir}/ci-runner-key.pem";
+      servicesIssuerClientCertFile = lib.mkDefault "${pkiDir}/services-issuer.pem";
+      servicesIssuerClientKeyFile = lib.mkDefault "${pkiDir}/services-issuer-key.pem";
 
       # The granter: every `swarm-bao-*-policy` unit on this host logs in with
       # it. ⚠️ Unlike every other leaf here, never the file an operator copies:
@@ -224,8 +226,8 @@ in
         [ -s ${pkiDir}/forwarder-oidc.pem ] || ${signLeaf} ${pkiDir} forwarder-oidc \
           ${lib.escapeShellArg deployCfg.bao.forwarderOidcCommonName} "" clientAuth
 
-        # The identity a hive presents to ask the store's `pki` mount for the
+        # The identity a gateway host presents to ask the store's `pki` mount
-        # swarm-services certificate its gateway serves. Minted here like the
+        # for the swarm-services certificate it serves. Minted here like the
         # three above, and the reason is the sharpest of the four: this leaf
         # is what OPENS the mint, so it cannot come out of it. A certificate
         # authority that issues the credential admitting you to it is the

nix/host-modules/glue-hive-tls-bao-identity.nix

@@ -6,7 +6,7 @@
 # that credential, which is what every hive on a host without the store does.
 #
 # ⚠️ The minting is NOT here. ./glue-bao-tls.nix holds the CA and signs the
-# leaf, for the reason ./glue-services-issuer-bao-identity.nix gives.
+# leaf, because the thing that owns a private key owns issuing from it.
 #
 # Gated on the minted PKI existing, as that file is, and on a hive running
 # here: a host without one has no hive leaf to ask for.
@@ -21,7 +21,7 @@ let
   baoDeploy = config.services.hyperhive.deploy.bao;
 
   # Where ./glue-bao-tls.nix puts the leaves, derived from the reader's own
-  # path as ./glue-services-issuer-bao-identity.nix derives it.
+  # path rather than repeating that file's directory literal.
   haveMintedPki = baoDeploy.clientCertFile != null;
   pkiDir = if haveMintedPki then builtins.dirOf baoDeploy.clientCertFile else null;
 in

nix/host-modules/glue-services-issuer-bao-identity.nix

@@ -1,43 +0,0 @@
-# Glue: point the swarm-services certificate unit at the bao leaf minted for
-# it.
-#
-# ONE PAIRING PER FILE — the services-issuer principal ← bao, and nothing
-# else. Deleting this leaves a hive that takes an operator-provided path to
-# that credential, which is what any deployment not minting its own already
-# does.
-#
-# ⚠️ The minting is NOT here. ./glue-bao-tls.nix holds the CA and signs the
-# leaf, because the thing that owns a private key owns issuing from it. What
-# belongs here is the pairing: which paths `swarm-services-cert` reads.
-#
-# ⚠️ Gated on the leaf existing, not on the store running here — the same rule
-# ./glue-controller-bao-identity.nix states, and it matters more for this
-# principal than for any of its siblings: every hive in a swarm serves the
-# swarm's service names and therefore needs this certificate, while exactly
-# one of them hosts the store.
-#
-# Everything is `mkDefault`. An operator naming their own paths wins.
-{
-  lib,
-  config,
-  ...
-}:
-let
-  hyperhiveCfg = config.services.hyperhive;
-  deployCfg = hyperhiveCfg.deploy;
-  baoDeploy = deployCfg.bao;
-
-  # Where ./glue-bao-tls.nix puts the leaves, derived from the reader's own
-  # path rather than repeating that file's directory literal: an operator who
-  # moves the PKI moves both, and the two cannot drift apart.
-  haveMintedPki = baoDeploy.clientCertFile != null;
-  pkiDir = if haveMintedPki then builtins.dirOf baoDeploy.clientCertFile else null;
-in
-{
-  config = lib.mkIf haveMintedPki {
-    services.hyperhive.deploy.hive-controller.tls = {
-      baoClientCertFile = lib.mkDefault "${pkiDir}/services-issuer.pem";
-      baoClientKeyFile = lib.mkDefault "${pkiDir}/services-issuer-key.pem";
-    };
-  };
-}

nix/host-modules/hive-tls.nix

@@ -72,41 +72,77 @@ let
     else
       "${baoDeploy.hiveLeafPkiRolePrefix}-${hyperhiveCfg.hiveName}";
 
-  # The hive's own store identity: its subject is
-  # `<hiveLeafCommonNamePrefix>-<hiveName>`, and its policy may issue from
-  # `hiveLeafRole` alone. It also carries the root fetch's handshake.
-  haveHiveIdentity = cfg.baoHiveIdentityCertFile != null && cfg.baoHiveIdentityKeyFile != null;
   showPath = p: if p == null then "unset" else p;
-  hiveIdentityOptions = "services.hyperhive.deploy.hive-controller.tls.baoHiveIdentityCertFile (${showPath cfg.baoHiveIdentityCertFile}) and services.hyperhive.deploy.hive-controller.tls.baoHiveIdentityKeyFile (${showPath cfg.baoHiveIdentityKeyFile})";
 
-  # Fails the unit, naming both options, when the identity is unset or a file
+  # Fails the unit, naming both options, when an identity is unset or a file
-  # is unreadable. Every hive must hold one.
+  # is unreadable.
-  requireHiveIdentity =
+  requireIdentity =
+    {
+      holder,
+      what,
+      certFile,
+      keyFile,
+      certOption,
+      keyOption,
+      unsetHint ? "",
+    }:
     purpose:
-    if haveHiveIdentity then
+    let
+      options = "${certOption} (${showPath certFile}) and ${keyOption} (${showPath keyFile})";
+    in
+    if certFile != null && keyFile != null then
       ''
         missing=""
         for f in ${
           lib.escapeShellArgs [
-            cfg.baoHiveIdentityCertFile
+            certFile
-            cfg.baoHiveIdentityKeyFile
+            keyFile
           ]
         }; do
           [ -r "$f" ] || missing="$missing $f"
         done
         if [ -n "$missing" ]; then
-          echo "this hive has no usable store identity, so it cannot ${purpose}: cannot read$missing." >&2
+          echo "${holder} has no usable ${what}, so it cannot ${purpose}: cannot read$missing." >&2
-          echo "place the files named by ${hiveIdentityOptions}." >&2
+          echo "place the files named by ${options}." >&2
           exit 1
         fi
       ''
     else
       ''
-        echo "this hive has no store identity, so it cannot ${purpose}." >&2
+        echo "${holder} has no ${what}, so it cannot ${purpose}." >&2
-        echo "set ${hiveIdentityOptions} and place both files." >&2
+        echo "set ${options} and place both files.${unsetHint}" >&2
         exit 1
       '';
 
+  # The hive's own store identity: its subject is
+  # `<hiveLeafCommonNamePrefix>-<hiveName>`, and its policy may issue from
+  # `hiveLeafRole` alone. Every hive must hold one.
+  requireHiveIdentity = requireIdentity {
+    holder = "this hive";
+    what = "store identity";
+    certFile = cfg.baoHiveIdentityCertFile;
+    keyFile = cfg.baoHiveIdentityKeyFile;
+    certOption = "services.hyperhive.deploy.hive-controller.tls.baoHiveIdentityCertFile";
+    keyOption = "services.hyperhive.deploy.hive-controller.tls.baoHiveIdentityKeyFile";
+  };
+
+  # The swarm-wide services-issuer identity `swarm-services-cert` presents,
+  # whether or not a hive runs here. Its subject is
+  # `servicesIssuerCommonName`, and its policy may issue from
+  # `pki/issue/swarm-services` alone.
+  servicesIssuerCert = baoDeploy.servicesIssuerClientCertFile;
+  servicesIssuerKey = baoDeploy.servicesIssuerClientKeyFile;
+  haveServicesIssuer = servicesIssuerCert != null && servicesIssuerKey != null;
+  requireServicesIssuer = requireIdentity {
+    holder = "this host";
+    what = "services-issuer identity";
+    certFile = servicesIssuerCert;
+    keyFile = servicesIssuerKey;
+    certOption = "services.hyperhive.deploy.bao.servicesIssuerClientCertFile";
+    keyOption = "services.hyperhive.deploy.bao.servicesIssuerClientKeyFile";
+    unsetHint = " Its subject must be CN=${baoDeploy.servicesIssuerCommonName}.";
+  };
+
   # Shared by every unit that decides whether to re-issue a leaf. It
   # lives here rather than in one of them because the guards have to
   # agree: they answer the same question for different leaves
@@ -195,11 +231,11 @@ let
   servicesCertEnvironment = {
     BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
   }
-  // lib.optionalAttrs (cfg.baoClientCertFile != null) {
+  // lib.optionalAttrs (servicesIssuerCert != null) {
-    BAO_CLIENT_CERT = cfg.baoClientCertFile;
+    BAO_CLIENT_CERT = servicesIssuerCert;
   }
-  // lib.optionalAttrs (cfg.baoClientKeyFile != null) {
+  // lib.optionalAttrs (servicesIssuerKey != null) {
-    BAO_CLIENT_KEY = cfg.baoClientKeyFile;
+    BAO_CLIENT_KEY = servicesIssuerKey;
   }
   # Absent means the system trust store, which is what a deployment
   # with a real CA wants and what a self-signed one must not be left
@@ -208,9 +244,8 @@ let
     BAO_CACERT = baoDeploy.serverCaFile;
   };
 
-  # The root-only fetch. Same store and server CA as `servicesCertEnvironment`,
+  # The root-only fetch: the same store, server CA and services-issuer pair
-  # but the hive's own identity: a host fronting no swarm name holds no
+  # as `servicesCertEnvironment`, handed to curl.
-  # services-issuer pair.
   storeRootCurlTls = lib.escapeShellArgs (
     lib.optionals (baoDeploy.serverCaFile != null) [
       "--cacert"
@@ -218,9 +253,9 @@ let
     ]
     ++ [
       "--cert"
-      cfg.baoHiveIdentityCertFile
+      servicesIssuerCert
       "--key"
-      cfg.baoHiveIdentityKeyFile
+      servicesIssuerKey
     ]
   );
 
@@ -262,62 +297,21 @@ in
       '';
     };
 
-    baoClientCertFile = lib.mkOption {
-      type = lib.types.nullOr lib.types.str;
-      default = null;
-      example = "/var/lib/swarm-bao-pki/services-issuer.pem";
-      description = ''
-        Certificate `swarm-services-cert` presents to the secret store when it
-        asks for the swarm-services leaf the gateway serves.
-
-        Its subject has to be
-        {option}`services.hyperhive.deploy.bao.servicesIssuerCommonName` — cert
-        auth matches on the common name, and this is the one role whose policy
-        permits `pki/issue/swarm-services`.
-
-        A **separate** credential from
-        {option}`services.hyperhive.deploy.bao.clientCertFile`, which is this
-        hive's own reader identity and whose policy reads every secret the
-        hive is entitled to, and from
-        {option}`services.hyperhive.deploy.hive-controller.tls.baoHiveIdentityCertFile`,
-        which the root fetch on a hive fronting no swarm name presents.
-
-        On a hive that mints its own store PKI a glue module supplies the leaf
-        as a `mkDefault`; everywhere else it is the credential an operator
-        places, for the reason the store's own client certificate is.
-
-        A path, never a value.
-      '';
-    };
-
-    baoClientKeyFile = lib.mkOption {
-      type = lib.types.nullOr lib.types.str;
-      default = null;
-      example = "/var/lib/swarm-bao-pki/services-issuer-key.pem";
-      description = ''
-        Private key for
-        {option}`services.hyperhive.deploy.hive-controller.tls.baoClientCertFile`.
-        Both or neither — a certificate with no key authenticates nothing.
-      '';
-    };
-
     baoHiveIdentityCertFile = lib.mkOption {
       type = lib.types.nullOr lib.types.str;
       default = null;
       example = "/var/lib/swarm-bao-pki/hive-leaf.pem";
       description = ''
         This hive's own store identity. `hive-tls-resign` presents it to ask
-        the secret store for the gateway leaf, and `swarm-services-cert` on a
+        the secret store for the gateway leaf.
-        hive fronting no swarm name presents it for the handshake of the root
-        fetch.
 
         Its subject has to be
         `<services.hyperhive.deploy.bao.hiveLeafCommonNamePrefix>-<hiveName>`;
         that cert-auth role's policy may issue from this hive's gateway-leaf
         role and nothing else.
 
-        Every hive with a self-signed gateway needs it: without it both units
+        Every hive with a self-signed gateway needs it: without it that unit
-        fail, naming this option and
+        fails, naming this option and
         {option}`services.hyperhive.deploy.hive-controller.tls.baoHiveIdentityKeyFile`.
         On the host that runs the store a glue module supplies the leaf as a
         `mkDefault`; on every other host an operator places it.
@@ -339,6 +333,30 @@ in
   };
 
   config = lib.mkIf active {
+    assertions = [
+      {
+        assertion = haveServicesIssuer;
+        message = ''
+          services.hyperhive.gateway serves the self-signed leaf on this host,
+          which needs the swarm-services issuer's store identity: set both
+
+            services.hyperhive.deploy.bao.servicesIssuerClientCertFile
+            services.hyperhive.deploy.bao.servicesIssuerClientKeyFile
+
+          swarm-services-cert.service presents it to ask the swarm secret
+          store for the swarm-services leaf and to fetch the store's pki root.
+
+          Both default to the pair glue-bao-tls.nix mints when
+          services.hyperhive.deploy.bao.enable is true (the host that runs
+          the store). Otherwise copy services-issuer.pem and
+          services-issuer-key.pem from the store host's /var/lib/swarm-bao-pki/
+          and name them in those two options, or give the gateway a
+          certificate of its own (services.hyperhive.gateway.tls.certDir or
+          services.hyperhive.gateway.tls.acme.enable).
+        '';
+      }
+    ];
+
     # The state dir and the trust bundle. Never waits on the store: the
     # store's container is ordered after this unit, which writes the
     # bundle it bind-mounts, so the leaf is `hive-tls-resign`'s.
@@ -603,9 +621,9 @@ in
       # store nothing local can order against that.
       #
       # `swarm-bao-pki` mints the services-issuer leaf the login below
-      # presents, and is absent on a hive holding an operator-placed one —
+      # presents, and is absent on a host holding an operator-placed one —
       # ignored there for the same reason, rather than being a dependency
-      # only some hives may declare.
+      # only some hosts may declare.
       after = [
         "container@${baoCfg.machine}.service"
         "swarm-bao-pki.service"
@@ -643,13 +661,12 @@ in
         # bundle: every other hive's swarm-service names chain to it.
         # `ca/pem` takes no token; the TLS client certificate is the
         # store's listener's demand, not bao's, so this presents the
-        # hive's own identity and never logs in with it.
+        # services-issuer pair and never logs in with it. Without a
-        # A hive without a readable identity fails here, naming the
+        # readable pair the unit fails here, naming the options.
-        # options: every hive must hold one.
         if [ "$want_svc" = 0 ]; then
         ${
-          if haveHiveIdentity then
+          if haveServicesIssuer then
-            requireHiveIdentity "fetch the store's ${baoServicesPkiMount} root"
+            requireServicesIssuer "fetch the store's ${baoServicesPkiMount} root"
             + ''
               echo "no swarm service domains configured — fetching only the store's pki root"
               err="$(mktemp)"
@@ -683,7 +700,7 @@ in
               fi
               exit 0''
           else
-            requireHiveIdentity "fetch the store's ${baoServicesPkiMount} root"
+            requireServicesIssuer "fetch the store's ${baoServicesPkiMount} root"
         }
         fi
 
@@ -725,17 +742,7 @@ in
           exit 0
         fi
 
-        ${
+        ${requireServicesIssuer "ask the store for the swarm-services leaf"}
-          if cfg.baoClientCertFile == null || cfg.baoClientKeyFile == null then
-            ''
-              echo "no bao client certificate configured for this host, so the gateway's" >&2
-              echo "swarm-services certificate cannot be requested from the store." >&2
-              echo "Set services.hyperhive.deploy.hive-controller.tls.baoClient{Cert,Key}File" >&2
-              echo "to a leaf the store's CA signed with CN=${baoDeploy.servicesIssuerCommonName}." >&2
-              exit 1''
-          else
-            ""
-        }
 
         # `bao`'s own message is the only thing separating an unreachable
         # store from a refused identity from a role that would not issue

nix/host-modules/swarm-bao.nix

@@ -1950,6 +1950,43 @@ in
       '';
     };
 
+    servicesIssuerClientCertFile = lib.mkOption {
+      type = lib.types.nullOr lib.types.str;
+      default = null;
+      example = "/var/lib/swarm-bao-pki/services-issuer.pem";
+      description = ''
+        Certificate `swarm-services-cert` presents to the store: to log in and
+        ask for the swarm-services leaf the gateway serves, and for the
+        handshake of the root fetch on a host fronting no swarm name. Its
+        subject must be
+        {option}`services.hyperhive.deploy.bao.servicesIssuerCommonName`.
+
+        ⚠️ Neither the reader's
+        {option}`services.hyperhive.deploy.bao.clientCertFile` nor a hive's own
+        identity: this one's policy permits `update` on
+        `pki/issue/swarm-services` and nothing else, and it is the same on
+        every gateway host, whether or not a hive runs there.
+
+        Required on every host whose gateway serves the self-signed leaf;
+        unset fails evaluation.
+
+        ./glue-bao-tls.nix points it at the leaf it mints on the store's host.
+        On a gateway host without the store, it is a file the operator copies
+        from the store's host. A path, never a value.
+      '';
+    };
+
+    servicesIssuerClientKeyFile = lib.mkOption {
+      type = lib.types.nullOr lib.types.str;
+      default = null;
+      example = "/var/lib/swarm-bao-pki/services-issuer-key.pem";
+      description = ''
+        Private key for
+        {option}`services.hyperhive.deploy.bao.servicesIssuerClientCertFile`.
+        A path, never a value.
+      '';
+    };
+
     granterCommonName = lib.mkOption {
       type = lib.types.str;
       default = "bao-granter";

nix/module-eval/bao-grants.nix

@@ -1219,7 +1219,9 @@ let
         && t.baoHiveIdentityKeyFile == "/var/lib/swarm-bao-pki/hive-leaf-key.pem"
         # Its own leaf, not the reader's or the services issuer's.
         && t.baoHiveIdentityCertFile != baoGrantHere.services.hyperhive.deploy.bao.clientCertFile
-        && t.baoHiveIdentityCertFile != t.baoClientCertFile;
+        &&
+          t.baoHiveIdentityCertFile
+          != baoGrantHere.services.hyperhive.deploy.bao.servicesIssuerClientCertFile;
     }
     {
       name = "a gateway-leaf role prefix outside swarm-* is refused, naming the option";

nix/module-eval/forge-placement.nix

@@ -73,6 +73,8 @@ let
     deploy.forgejo.ci.jobTimeout = "2h";
     deploy.bao.ciRunnerClientCertFile = "/etc/pki/bao-ci-runner.pem";
     deploy.bao.ciRunnerClientKeyFile = "/etc/pki/bao-ci-runner-key.pem";
+    deploy.bao.servicesIssuerClientCertFile = "/etc/pki/bao-services-issuer.pem";
+    deploy.bao.servicesIssuerClientKeyFile = "/etc/pki/bao-services-issuer-key.pem";
   };
   # The old name alone, with no store identity placed and none defaulted.
   ciUnderForgeNoIdentity = hive {
@@ -88,6 +90,8 @@ let
     swarm.forge.ci.package = pkgs.hello;
     deploy.bao.ciRunnerClientCertFile = "/etc/pki/bao-ci-runner.pem";
     deploy.bao.ciRunnerClientKeyFile = "/etc/pki/bao-ci-runner-key.pem";
+    deploy.bao.servicesIssuerClientCertFile = "/etc/pki/bao-services-issuer.pem";
+    deploy.bao.servicesIssuerClientKeyFile = "/etc/pki/bao-services-issuer-key.pem";
   };
 
   runsForge = m: m.services.hyperhive.deploy.forgejo.enable && m.containers ? hive-forge;

nix/module-eval/hive-tls.nix

@@ -41,11 +41,14 @@ let
   };
 
   # Fronting none of the swarm's names, reading a store it does not run,
-  # with the reader identity and the hive's own identity placed by hand.
+  # with the reader identity, the services-issuer pair and the hive's own
+  # identity placed by hand.
   noService = hiveWithController {
     deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
     deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
     deploy.bao.serverCaFile = "/etc/pki/bao-ca.pem";
+    deploy.bao.servicesIssuerClientCertFile = "/etc/pki/bao-services-issuer.pem";
+    deploy.bao.servicesIssuerClientKeyFile = "/etc/pki/bao-services-issuer-key.pem";
     deploy.hive-controller.tls.baoHiveIdentityCertFile = "/etc/pki/bao-hive.pem";
     deploy.hive-controller.tls.baoHiveIdentityKeyFile = "/etc/pki/bao-hive-key.pem";
   };
@@ -55,6 +58,23 @@ let
   noIdentity = hiveWithController { };
   noIdFetch = noIdentity.systemd.services.swarm-services-cert.script;
 
+  # A gateway on a host with no hive and no store, fronting no swarm name,
+  # with the services-issuer pair placed by hand; and the same host without it.
+  gatewayNoHive = hive {
+    gateway.enable = true;
+    deploy.bao.serverCaFile = "/etc/pki/bao-ca.pem";
+    deploy.bao.servicesIssuerClientCertFile = "/etc/pki/bao-services-issuer.pem";
+    deploy.bao.servicesIssuerClientKeyFile = "/etc/pki/bao-services-issuer-key.pem";
+  };
+  gatewayNoHiveNoPair = hive { gateway.enable = true; };
+
+  # The store's host with the pair still named under its old path.
+  renamedPair = hiveWithController {
+    deploy.singleHostSwarm = true;
+    deploy.hive-controller.tls.baoClientCertFile = "/etc/pki/old-issuer.pem";
+    deploy.hive-controller.tls.baoClientKeyFile = "/etc/pki/old-issuer-key.pem";
+  };
+
   # The `want_svc=0` branch of a rendered script, up to the `fi` that closes it (the
   # only one followed by a blank line).
   noSvcBranch =
@@ -190,42 +210,124 @@ let
         && lib.hasInfix ''cat "$svcroot" > "$bundle"'' noService.systemd.services.hive-tls-ca.script;
     }
     {
-      # The hive's own identity, not its reader identity: the reader's policy
+      # The services-issuer pair, not the reader identity, whose policy reads
-      # reads every secret the hive is entitled to.
+      # every secret the hive is entitled to, and not the hive's own.
-      name = "the root fetch reaches the store with the hive's own identity and server CA";
+      name = "the root fetch reaches the store with the services-issuer pair and server CA";
       ok =
         lib.hasInfix "--cacert ${lib.escapeShellArg "/etc/pki/bao-ca.pem"}" noSvcFetch
         && lib.hasInfix (
-          "--cert ${lib.escapeShellArg "/etc/pki/bao-hive.pem"}"
+          "--cert ${lib.escapeShellArg "/etc/pki/bao-services-issuer.pem"}"
-          + " --key ${lib.escapeShellArg "/etc/pki/bao-hive-key.pem"}"
+          + " --key ${lib.escapeShellArg "/etc/pki/bao-services-issuer-key.pem"}"
         ) noSvcFetch
-        && !(lib.hasInfix "/etc/pki/bao-client.pem" (noSvcBranch noSvcFetch));
+        && !(lib.hasInfix "/etc/pki/bao-client.pem" (noSvcBranch noSvcFetch))
+        && !(lib.hasInfix "/etc/pki/bao-hive" (noSvcBranch noSvcFetch));
     }
     {
       # No identity to present: the unit fails, naming both options, rather
       # than leaving the bundle without the root behind a green unit. The
-      # control is the same hive with an identity, whose branch fetches.
+      # control is the same hive with the pair, whose branch fetches.
-      name = "a hive fronting no swarm name with no store identity fails the unit, naming both options";
+      name = "a host fronting no swarm name with no services-issuer pair fails the unit, naming both options";
       ok =
         noIdentity.services.hyperhive.swarm.localServiceDomains == [ ]
-        && noIdentity.services.hyperhive.deploy.hive-controller.tls.baoHiveIdentityCertFile == null
+        && noIdentity.services.hyperhive.deploy.bao.servicesIssuerClientCertFile == null
         && !(lib.hasInfix "/v1/pki/ca/pem" noIdFetch)
-        && lib.hasInfix "baoHiveIdentityCertFile (unset)" (noSvcBranch noIdFetch)
+        && lib.hasInfix "servicesIssuerClientCertFile (unset)" (noSvcBranch noIdFetch)
-        && lib.hasInfix "baoHiveIdentityKeyFile (unset)" (noSvcBranch noIdFetch)
+        && lib.hasInfix "servicesIssuerClientKeyFile (unset)" (noSvcBranch noIdFetch)
         && lib.hasSuffix "exit 1" (lib.removeSuffix "\n" (noSvcBranch noIdFetch))
         && !(lib.hasInfix "exit 0" (noSvcBranch noIdFetch))
         && !(lib.hasInfix "not staged" noIdFetch)
         && lib.hasInfix "/v1/pki/ca/pem" (noSvcBranch noSvcFetch);
     }
     {
-      # An identity named but not on disk fails before curl, naming the paths.
+      # A pair named but not on disk fails before curl, naming the paths.
-      name = "a hive fronting no swarm name whose identity files are absent fails the unit, naming the paths";
+      name = "a host fronting no swarm name whose services-issuer files are absent fails the unit, naming the paths";
       ok =
-        lib.hasInfix "for f in /etc/pki/bao-hive.pem /etc/pki/bao-hive-key.pem; do" (noSvcBranch noSvcFetch)
+        lib.hasInfix "for f in /etc/pki/bao-services-issuer.pem /etc/pki/bao-services-issuer-key.pem; do" (
+          noSvcBranch noSvcFetch
+        )
         && lib.hasInfix ''[ -r "$f" ] || missing="$missing $f"'' (noSvcBranch noSvcFetch)
         && lib.hasInfix "cannot read$missing" (noSvcBranch noSvcFetch)
-        && lib.hasInfix "baoHiveIdentityCertFile (/etc/pki/bao-hive.pem)" (noSvcBranch noSvcFetch)
+        && lib.hasInfix "servicesIssuerClientCertFile (/etc/pki/bao-services-issuer.pem)" (
-        && lib.hasInfix "baoHiveIdentityKeyFile (/etc/pki/bao-hive-key.pem)" (noSvcBranch noSvcFetch);
+          noSvcBranch noSvcFetch
+        )
+        && lib.hasInfix "servicesIssuerClientKeyFile (/etc/pki/bao-services-issuer-key.pem)" (
+          noSvcBranch noSvcFetch
+        );
+    }
+    {
+      # A gateway host with no hive, fronting no swarm name, the pair placed
+      # by hand. The control is the hive host, whose gateway-leaf unit does
+      # present the hive identity, so the scan can find one.
+      name = "on a host without a hive the swarm-services units present no hive identity, and the root fetch runs";
+      ok =
+        let
+          hiveRefs = [
+            "baoHiveIdentity"
+            "hive-leaf"
+            "/etc/pki/bao-hive"
+          ];
+          refsHive =
+            units:
+            lib.any (
+              u: lib.any (r: lib.hasInfix r u.script || lib.hasInfix r (builtins.toJSON u.environment)) hiveRefs
+            ) units;
+          units = gatewayNoHive.systemd.services;
+          s = units.swarm-services-cert;
+        in
+        !gatewayNoHive.services.hyperhive.deploy.hive-controller.enable
+        && gatewayNoHive.services.hyperhive.swarm.localServiceDomains == [ ]
+        && !(refsHive [
+          s
+          units.swarm-services-cert-renew
+        ])
+        && lib.hasInfix "/v1/pki/ca/pem" (noSvcBranch s.script)
+        && lib.hasInfix "--cert ${lib.escapeShellArg "/etc/pki/bao-services-issuer.pem"}" (
+          noSvcBranch s.script
+        )
+        && s.environment.BAO_CLIENT_CERT == "/etc/pki/bao-services-issuer.pem"
+        && refsHive [ noService.systemd.services.hive-tls-resign ];
+    }
+    {
+      # The control is the same host with the pair placed, and the store's
+      # host, where the glue supplies it.
+      name = "a self-signed gateway with no services-issuer pair fails evaluation, naming both options";
+      ok =
+        let
+          refused =
+            m:
+            lib.any (
+              a:
+              !a.assertion
+              && lib.hasInfix "services.hyperhive.deploy.bao.servicesIssuerClientCertFile" a.message
+              && lib.hasInfix "services.hyperhive.deploy.bao.servicesIssuerClientKeyFile" a.message
+            ) m.assertions;
+        in
+        refused gatewayNoHiveNoPair
+        && refused noIdentity
+        && !(refused gatewayNoHive)
+        && !(refused noService)
+        && !(refused allLocal)
+        &&
+          allLocal.services.hyperhive.deploy.bao.servicesIssuerClientCertFile
+          == "/var/lib/swarm-bao-pki/services-issuer.pem";
+    }
+    {
+      # The old path's value reaches the new option over the store glue's
+      # `mkDefault`, and the unit presents it. The control is the same store
+      # host on its defaults.
+      name = "the services-issuer pair set under deploy.hive-controller.tls reaches deploy.bao and the unit";
+      ok =
+        let
+          b = renamedPair.services.hyperhive.deploy.bao;
+          env = renamedPair.systemd.services.swarm-services-cert.environment;
+        in
+        b.servicesIssuerClientCertFile == "/etc/pki/old-issuer.pem"
+        && b.servicesIssuerClientKeyFile == "/etc/pki/old-issuer-key.pem"
+        && env.BAO_CLIENT_CERT == "/etc/pki/old-issuer.pem"
+        && env.BAO_CLIENT_KEY == "/etc/pki/old-issuer-key.pem"
+        &&
+          allLocal.systemd.services.swarm-services-cert.environment.BAO_CLIENT_CERT
+          == "/var/lib/swarm-bao-pki/services-issuer.pem";
     }
     {
       # The fetch retries against a sealed store; a gateway `Requires=` on it
@@ -469,7 +571,9 @@ let
         && leafUnit.environment.BAO_CLIENT_CERT == "/var/lib/swarm-bao-pki/hive-leaf.pem"
         && leafUnit.environment.BAO_CLIENT_KEY == "/var/lib/swarm-bao-pki/hive-leaf-key.pem"
         && t.baoHiveIdentityCertFile == leafUnit.environment.BAO_CLIENT_CERT
-        && leafUnit.environment.BAO_CLIENT_CERT != t.baoClientCertFile
+        &&
+          leafUnit.environment.BAO_CLIENT_CERT
+          != allLocal.services.hyperhive.deploy.bao.servicesIssuerClientCertFile
         && leafUnit.environment.BAO_CLIENT_CERT != allLocal.services.hyperhive.deploy.bao.clientCertFile;
     }
     {

nix/module-eval/nats-tls.nix

@@ -67,14 +67,16 @@ let
   # address set by hand: what every hive but one in a multi-host swarm looks
   # like. The controller is on too, since it may run away from the queue.
   #
-  # The forge's secret and the controller's store identity are what a hive
+  # The forge's secret, the controller's store identity and the gateway's
-  # away from authelia and the store already has to be handed, and none is an
+  # services-issuer pair are what a hive away from authelia and the store
-  # address; without them this hive would fail assertions that have nothing to
+  # already has to be handed, and none is an address; without them this hive
-  # do with the queue.
+  # would fail assertions that have nothing to do with the queue.
   remoteSecrets = {
     deploy.forgejo.sso.clientSecretFile = "/var/lib/forgejo-oidc/by-hand.secret";
     deploy.swarm-controller.baoClientCertFile = "/var/lib/swarm-controller/bao-client.pem";
     deploy.swarm-controller.baoClientKeyFile = "/var/lib/swarm-controller/bao-client-key.pem";
+    deploy.bao.servicesIssuerClientCertFile = "/etc/pki/bao-services-issuer.pem";
+    deploy.bao.servicesIssuerClientKeyFile = "/etc/pki/bao-services-issuer-key.pem";
   };
   remote = hiveWithController (
     lib.recursiveUpdate remoteSecrets { deploy.swarm-controller.enable = true; }

nix/module-eval/swarm-otel-identity.nix

@@ -27,9 +27,12 @@ let
   # hive's telemetry with nothing to push authenticated with, which is the
   # already-supported degrade `haveCollectorSecret` names above the module's
   # `let`. What this fixture is for is checking the reading unit itself does
-  # not render, rather than rendering with an env var nothing filled in.
+  # not render, rather than rendering with an env var nothing filled in. Its
+  # gateway has a certificate of its own: a self-signed one needs the
+  # services-issuer identity.
   otelNoIdentity = hive {
     deploy.swarm-otel.enable = true;
+    gateway.tls.certDir = "/var/lib/acme/t.local";
     swarm.authelia.url = "https://auth.example.invalid";
     deploy.forgejo.sso.clientSecretFile = "/var/lib/forgejo-oidc/by-hand.secret";
   };