From 92e1909caf3bcbf2cff62fc3a368d1c66d883e6d Mon Sep 17 00:00:00 2001 From: atlas Date: Thu, 24 Sep 2026 17:25:34 +0200 Subject: [PATCH] swarm-bao: give the store forwarder's OIDC reader its own bao identity `swarm-bao-forwarder-oidc` fetches the store container's collector secret, one path, and was the last reader still logging in with `deploy.bao.clientCertFile`: the hive's own leaf, whose policy reads every agent's credentials, the hive's tree and every service's OIDC secret. The four-way split gave grafana's and the swarm collector's readers leaves of their own and left this one behind. It now holds `forwarder-oidc.pem`, minted by `swarm-bao-pki`, and logs in under the `swarm-forwarder-oidc` cert-auth role, whose policy reads `secret/data/swarm/services//oidc/client` and nothing else. The role is written by `swarm-bao-forwarder-oidc-policy` from the bootstrap token, which gains the two grants that unit calls, and the reader is ordered after it. The subject is reserved as a hive name. A store host whose pair is null is refused at eval rather than falling back to the hive's leaf. The hive's own role and `client.pem` are untouched; nothing is revoked. --- docs/swarm/secrets.md | 43 ++++--- .../glue-bao-readers-policy-order.nix | 3 + nix/host-modules/glue-bao-tls.nix | 7 ++ .../swarm-bao-bootstrap-policy.hcl | 9 ++ nix/host-modules/swarm-bao.nix | 112 +++++++++++++++--- nix/host-modules/swarm.nix | 1 + nix/module-eval/bao-grants.nix | 83 +++++++++++-- nix/module-eval/bao-otel-collector.nix | 2 +- nix/module-eval/name-guards.nix | 16 +++ 9 files changed, 229 insertions(+), 47 deletions(-) diff --git a/docs/swarm/secrets.md b/docs/swarm/secrets.md index 743242d2..fac7ad76 100644 --- a/docs/swarm/secrets.md +++ b/docs/swarm/secrets.md @@ -98,7 +98,7 @@ collector this unit never reaches. The **secret store's own** collector — the forwarder inside the `swarm-bao` container — needs a delivery step too, and it takes the same route with one principal of its own: `swarm-bao-forwarder-oidc.service` reads -`swarm/services//oidc/client` under this host's certificate and +`swarm/services//oidc/client` under a leaf of its own and lands it in the container's tree, where `LoadCredential` hands it to the collector. The client id is its own (`services.hyperhive.swarm.bao.otel.clientId`), registered by @@ -284,8 +284,10 @@ fourth in both directions. It renders unconditionally, because the export it authenticates has no unauthenticated mode to degrade into — and it orders itself **after** `container@swarm-bao`, because the store it reads runs in the container it delivers into. Nothing circular sits behind that: the identity it -logs in with is this host's static `swarm-bao-pki` leaf, not anything the store -mints. +logs in with is the static `forwarder-oidc.pem` leaf `swarm-bao-pki` signs, not +anything the store mints. With no leaf it has no fallback, so `swarm-bao.nix` +refuses the build and names `deploy.bao.forwarderOidcClientCertFile` / +`forwarderOidcClientKeyFile`. A service's secret is one value for the whole swarm rather than one per hive, so it lives under the `services` prefix, and a hive's read policy grants that prefix @@ -295,9 +297,9 @@ there is nothing to scope the grant to. A hive's **own** leaf can therefore read every swarm service's client secret, and that's stated in `swarm-secret-client`'s `policy` module beside the grant itself. -⚠️ **The readers no longer present it.** Four units used to log in with +⚠️ **The readers no longer present it.** Five units used to log in with `deploy.bao.clientCertFile`, which is the hive's own leaf, and bao identifies a -principal by the subject of the certificate it presents — so four readers behind +principal by the subject of the certificate it presents — so five readers behind one leaf were one principal holding the union of their needs. Each now holds a leaf of its own, and a policy naming only the path that unit reads. See [per-principal identities](#per-principal-identities) below. @@ -326,31 +328,34 @@ else a hive needs does. ### Per-principal identities Bao matches a cert-auth role on the certificate's subject, so a certificate is an -identity and sharing one merges the identities. These four units read one path +identity and sharing one merges the identities. These five units read one path each and each holds a leaf, a role and a policy of its own: -| unit | option pair under `deploy.bao.` | reads | -| ------------------------ | -------------------------------------------------------- | -------------------------------------------------- | -| `swarm-bao-matrix-token` | `matrixTokenClientCertFile` / `matrixTokenClientKeyFile` | `swarm/hives//matrix/appservice-token` | -| `swarm-bao-queue-agent` | `queueAgentClientCertFile` / `queueAgentClientKeyFile` | `swarm/hives//queue/agent` | -| `swarm-bao-grafana-oidc` | `grafanaOidcClientCertFile` / `grafanaOidcClientKeyFile` | `swarm/services//oidc/client` | -| `swarm-bao-otel-oidc` | `otelOidcClientCertFile` / `otelOidcClientKeyFile` | `swarm/services//oidc/client` | +| unit | option pair under `deploy.bao.` | reads | +| -------------------------- | ------------------------------------------------------------ | -------------------------------------------------------- | +| `swarm-bao-matrix-token` | `matrixTokenClientCertFile` / `matrixTokenClientKeyFile` | `swarm/hives//matrix/appservice-token` | +| `swarm-bao-queue-agent` | `queueAgentClientCertFile` / `queueAgentClientKeyFile` | `swarm/hives//queue/agent` | +| `swarm-bao-grafana-oidc` | `grafanaOidcClientCertFile` / `grafanaOidcClientKeyFile` | `swarm/services//oidc/client` | +| `swarm-bao-otel-oidc` | `otelOidcClientCertFile` / `otelOidcClientKeyFile` | `swarm/services//oidc/client` | +| `swarm-bao-forwarder-oidc` | `forwarderOidcClientCertFile` / `forwarderOidcClientKeyFile` | `swarm/services//oidc/client` | The first two exist **per hive**, because the path they read carries a hive name and every hive runs its own reader. Their subjects are `-` and `-`; `swarm.nix` reserves both composed spellings as hive names, so nobody can name a hive into another hive's -role. The other two read a path that names a swarm service rather than a hive, so -one role each is enough and their subjects are the flat -`deploy.bao.grafanaOidcCommonName` and `deploy.bao.otelOidcCommonName`. +role. The other three read a path that names a swarm service rather than a hive, +so one role each is enough and their subjects are the flat +`deploy.bao.grafanaOidcCommonName`, `deploy.bao.otelOidcCommonName` and +`deploy.bao.forwarderOidcCommonName`. -On a host that mints its own PKI, `glue-bao-tls.nix` signs all four and defaults -all eight options, and there is nothing to do. Elsewhere you issue each leaf from +On a host that mints its own PKI, `glue-bao-tls.nix` signs all five and defaults +all ten options, and there is nothing to do. Elsewhere you issue each leaf from that CA out of band and name it here — one file per principal rather than one file -shared by four, which is the whole of what this buys. +shared by five, which is the whole of what this buys. -Forgetting one of the eight elsewhere isn't a quiet degrade. Three of the four +On a host without the store, where the fifth never runs, forgetting one of the +other eight isn't a quiet degrade. Three of the four readers used to render only where their leaf existed, so a hand-configured remote-store hive that named the hive's own `clientCertFile` and missed a principal's pair just lost that unit; `swarm-grafana.nix` was alone in refusing diff --git a/nix/host-modules/glue-bao-readers-policy-order.nix b/nix/host-modules/glue-bao-readers-policy-order.nix index 98deff57..16cfe5d5 100644 --- a/nix/host-modules/glue-bao-readers-policy-order.nix +++ b/nix/host-modules/glue-bao-readers-policy-order.nix @@ -44,6 +44,9 @@ let swarm-bao-otel-oidc = deployCfg.swarm-otel.enable && havePair baoDeploy.otelOidcClientCertFile baoDeploy.otelOidcClientKeyFile; + # ./swarm-bao.nix: its block is gated on the store, which `orderAfterPolicy` + # already checks. + swarm-bao-forwarder-oidc = hyperhiveCfg.enable; # ./swarm-nats.nix: the queue's TLS leaf, not a secret, but the same wait. swarm-bao-nats-tls = deployCfg.nats.enable; }; diff --git a/nix/host-modules/glue-bao-tls.nix b/nix/host-modules/glue-bao-tls.nix index d84f2885..9073c293 100644 --- a/nix/host-modules/glue-bao-tls.nix +++ b/nix/host-modules/glue-bao-tls.nix @@ -103,6 +103,10 @@ in grafanaOidcClientKeyFile = lib.mkDefault "${pkiDir}/grafana-oidc-key.pem"; otelOidcClientCertFile = lib.mkDefault "${pkiDir}/otel-oidc.pem"; otelOidcClientKeyFile = lib.mkDefault "${pkiDir}/otel-oidc-key.pem"; + # The fifth, the store forwarder's: the reader the four-way split left + # on `client.pem`. + forwarderOidcClientCertFile = lib.mkDefault "${pkiDir}/forwarder-oidc.pem"; + forwarderOidcClientKeyFile = lib.mkDefault "${pkiDir}/forwarder-oidc-key.pem"; }; # Idempotent on ABSENCE, never on content. Re-issuing the CA invalidates @@ -219,6 +223,9 @@ in [ -s ${pkiDir}/otel-oidc.pem ] || ${signLeaf} ${pkiDir} otel-oidc \ ${lib.escapeShellArg deployCfg.bao.otelOidcCommonName} "" clientAuth + [ -s ${pkiDir}/forwarder-oidc.pem ] || ${signLeaf} ${pkiDir} forwarder-oidc \ + ${lib.escapeShellArg deployCfg.bao.forwarderOidcCommonName} "" clientAuth + # The identity a hive presents to ask the store's `pki` mount for the # swarm-services certificate its gateway serves. Minted here like the # three above, and the reason is the sharpest of the four: this leaf diff --git a/nix/host-modules/swarm-bao-bootstrap-policy.hcl b/nix/host-modules/swarm-bao-bootstrap-policy.hcl index 21674749..6d572848 100644 --- a/nix/host-modules/swarm-bao-bootstrap-policy.hcl +++ b/nix/host-modules/swarm-bao-bootstrap-policy.hcl @@ -116,6 +116,15 @@ path "auth/cert/certs/swarm-otel-oidc" { capabilities = ["create", "update"] } +# swarm-bao-forwarder-oidc-policy +path "sys/policies/acl/swarm-forwarder-oidc" { + capabilities = ["create", "update"] +} + +path "auth/cert/certs/swarm-forwarder-oidc" { + capabilities = ["create", "update"] +} + # swarm-bao-nats-tls-policy: the queue's own pki role, beside # `swarm-services` above, and its policy and login role. path "pki/roles/swarm-nats" { diff --git a/nix/host-modules/swarm-bao.nix b/nix/host-modules/swarm-bao.nix index c1d29dda..bde146cb 100644 --- a/nix/host-modules/swarm-bao.nix +++ b/nix/host-modules/swarm-bao.nix @@ -471,6 +471,17 @@ let } ]; + # The fifth, and the one the four-way split left on the hive's leaf: the + # store's own forwarder. Its path is `forwarderStoreSecretPath` below with + # KV v2's `data/` inserted. + forwarderOidcReaders = [ + { + name = "swarm-forwarder-oidc"; + cn = baoDeploy.forwarderOidcCommonName; + policyText = readStanza "${credentialMountPath}/data/swarm/services/${cfg.otel.clientId}/oidc/client"; + } + ]; + # The role name IS the policy name, as for the three service principals # above: the role attaches the policy by spelling it identically, and one # string for both objects removes the way they drift apart. @@ -703,15 +714,15 @@ let # The forwarder's own credential, and the three names it takes on the way # in. The shape is ./swarm-otel.nix's `swarm-bao-otel-oidc` — the unit that - # already reads an OIDC client secret out of the store with this host's - # certificate and lands it in a collector's container — because that route - # is proven and there is no second one worth inventing. + # already reads an OIDC client secret out of the store with a leaf of its + # own and lands it in a collector's container — because that route is + # proven and there is no second one worth inventing. # # Where the publisher on authelia's host leaves it. The `services` segment # is `swarm-secret-client`'s `path::Kind::Service`: a swarm service's client # is registered once for the whole swarm, so its secret is one value. The - # prefix is also exactly what a hive certificate's read grant covers, so a - # path outside it answers 403 however correct it looks. + # `swarm-forwarder-oidc` role's grant is this one path, so any other answers + # 403 however correct it looks. forwarderStoreSecretPath = "secret/swarm/services/${cfg.otel.clientId}/oidc/client"; # At rest in the container's tree, written by the host unit below — under # /var/lib rather than /run, because a secret that evaporates on reboot @@ -1213,6 +1224,23 @@ in ''; }; + forwarderOidcCommonName = lib.mkOption { + type = lib.types.str; + default = "swarm-bao-forwarder-oidc"; + example = "swarm-bao-forwarder-oidc.svc"; + description = '' + Subject the store's `swarm-forwarder-oidc` cert-auth role accepts — the + identity `swarm-bao-forwarder-oidc`, the unit that fetches the store's + own forwarder's OIDC client secret, presents. Its grant is one path, + `swarm/services//oidc/client`, and read only. + + A **fifth** identity rather than reuse of + {option}`services.hyperhive.deploy.bao.otelOidcCommonName`: the two + read different clients' secrets, and the store's forwarder is not + entitled to the swarm collector's. + ''; + }; + matrixTokenClientCertFile = lib.mkOption { type = lib.types.nullOr lib.types.str; default = null; @@ -1318,6 +1346,30 @@ in ''; }; + forwarderOidcClientCertFile = lib.mkOption { + type = lib.types.nullOr lib.types.str; + default = null; + example = "/var/lib/swarm-bao-pki/forwarder-oidc.pem"; + description = '' + Certificate the unit that fetches the store's own forwarder's OIDC + client secret presents to the store. Its subject must be + {option}`services.hyperhive.deploy.bao.forwarderOidcCommonName`. + + ⚠️ Not the hive's own leaf, for the reason + {option}`services.hyperhive.deploy.bao.matrixTokenClientCertFile` gives. + ''; + }; + + forwarderOidcClientKeyFile = lib.mkOption { + type = lib.types.nullOr lib.types.str; + default = null; + example = "/var/lib/swarm-bao-pki/forwarder-oidc-key.pem"; + description = '' + Private key for + {option}`services.hyperhive.deploy.bao.forwarderOidcClientCertFile`. + ''; + }; + serverCaFile = lib.mkOption { type = lib.types.nullOr lib.types.str; default = null; @@ -1511,6 +1563,28 @@ in something set these back to null. ''; } + { + # Refused rather than degraded: `swarm-bao-forwarder-oidc` renders + # wherever the store does and has no mode without a secret, and the + # only fallback left would be the hive's leaf — the union grant this + # pair exists to end. + assertion = + baoDeploy.forwarderOidcClientCertFile != null && baoDeploy.forwarderOidcClientKeyFile != null; + message = '' + The swarm secret store runs on this host, so its forwarder needs a + client identity of its own: set both + + services.hyperhive.deploy.bao.forwarderOidcClientCertFile + services.hyperhive.deploy.bao.forwarderOidcClientKeyFile + + swarm-bao-forwarder-oidc.service fetches the store forwarder's OIDC + client secret out of the store with them. A hive that runs the store + normally gets both from ./glue-bao-tls.nix. + + ⚠️ Not deploy.bao.clientCertFile. That one is the hive's, and its + grant reads every secret in the store; this role reads one path. + ''; + } ]; # The name every reader dials, made resolvable where the store runs. @@ -1557,6 +1631,7 @@ in "swarm-bao-queue-agent-policy" "swarm-bao-grafana-oidc-policy" "swarm-bao-otel-oidc-policy" + "swarm-bao-forwarder-oidc-policy" "swarm-bao-services-issuer-policy" "swarm-bao-nats-tls-policy" ]; @@ -1618,14 +1693,14 @@ in # reads a store in the container NEXT DOOR; this one reads the store in # the very container it is delivering into, so "before" is a wait on a # process that cannot start until this finishes. There is no bootstrap - # cycle behind it — the identity used here is this host's static - # `swarm-bao-pki` certificate, not anything the store mints — only an - # ordering one, and the cost is bounded: the file is under /var/lib, so - # it survives reboots and only a FIRST boot has the collector starting - # before it exists. `LoadCredential` refuses to start a unit whose - # source is missing, so that boot is a collector that restarts, says so - # each time, and comes up the moment this lands. Loud and self-healing - # rather than silently exporting without a credential. + # cycle behind it — the identity used here is the static + # `forwarder-oidc.pem` leaf from `swarm-bao-pki`, not anything the store + # issues — only an ordering one, and the cost is bounded: the file is + # under /var/lib, so it survives reboots and only a FIRST boot has the + # collector starting before it exists. `LoadCredential` refuses to start + # a unit whose source is missing, so that boot is a collector that + # restarts, says so each time, and comes up the moment this lands. Loud + # and self-healing rather than silently exporting without a credential. systemd.services.swarm-bao-forwarder-oidc = { description = "fetch the secret store forwarder's OIDC client secret from the store"; after = [ @@ -1656,8 +1731,11 @@ in }; environment = { BAO_ADDR = "https://${cfg.domain}:${toString cfg.port}"; - BAO_CLIENT_CERT = baoDeploy.clientCertFile; - BAO_CLIENT_KEY = baoDeploy.clientKeyFile; + # 🩸 Its OWN leaf, not `clientCertFile`: the hive's grant reads every + # agent's credential and every service's OIDC secret, and this unit + # needs one path. The pair is asserted set above. + BAO_CLIENT_CERT = baoDeploy.forwarderOidcClientCertFile; + BAO_CLIENT_KEY = baoDeploy.forwarderOidcClientKeyFile; } # Absent means the system trust store, which is what a deployment with # a real CA wants and what a self-signed one must not be left with. @@ -1680,7 +1758,7 @@ in # this unit's `path` does not carry — `-token-only` answers on # stdout and skips the helper on both sides. if ! BAO_TOKEN="$(bao login -method=cert -token-only 2>"$err")"; then - echo "could not log in to the swarm secret store with this host's certificate." >&2 + echo "could not log in to the swarm secret store with the forwarder's own certificate." >&2 cat "$err" >&2 exit 1 fi @@ -2206,6 +2284,8 @@ in systemd.services.swarm-bao-otel-oidc-policy = readerPolicyUnit "write the collector's OIDC-secret-reader bao policy and cert-auth role" otelOidcReaders; + systemd.services.swarm-bao-forwarder-oidc-policy = readerPolicyUnit "write the store forwarder's OIDC-secret-reader bao policy and cert-auth role" forwarderOidcReaders; + # A FOURTH sibling, same shape and same reasons as the two above. This # one is what turns `swarm-services-issuer` from a declaration into a # grant: a bao policy reaches nothing until a login role hands it to a diff --git a/nix/host-modules/swarm.nix b/nix/host-modules/swarm.nix index 3c91b3f5..93844741 100644 --- a/nix/host-modules/swarm.nix +++ b/nix/host-modules/swarm.nix @@ -49,6 +49,7 @@ let deployCfg.bao.matrixCtlCommonName deployCfg.bao.grafanaOidcCommonName deployCfg.bao.otelOidcCommonName + deployCfg.bao.forwarderOidcCommonName deployCfg.bao.servicesIssuerCommonName deployCfg.bao.natsCommonName ] diff --git a/nix/module-eval/bao-grants.nix b/nix/module-eval/bao-grants.nix index b014a23e..7726dc2a 100644 --- a/nix/module-eval/bao-grants.nix +++ b/nix/module-eval/bao-grants.nix @@ -67,6 +67,14 @@ let deploy.bao.queueAgentClientKeyFile = lib.mkForce null; }; + # The store with the forwarder's own pair taken away. The forwarder renders + # wherever the store does, so this is the deployment the assertion refuses. + baoNoForwarderIdentity = hive { + deploy.bao.enable = true; + deploy.bao.forwarderOidcClientCertFile = lib.mkForce null; + deploy.bao.forwarderOidcClientKeyFile = lib.mkForce null; + }; + # All four readers against a store they do not run, each with a leaf placed # by hand. The deployment in which there is no local policy unit to wait for. baoRemoteReaders = hive { @@ -454,6 +462,26 @@ let && !(lib.hasInfix "swarm-grafana" s) && !(lib.hasInfix "sys/policies/acl" s); } + { + # The fifth, and the one that stayed on the hive's leaf longest: the + # store's own forwarder. Its client id is `swarm-bao-collector`, so the + # arm naming `swarm-collector/` is the swarm collector's secret, which + # this principal is not entitled to. + name = "the store forwarder's OIDC reader's grant is its own client secret and nothing else"; + ok = + let + s = baoGrantHere.systemd.services.swarm-bao-forwarder-oidc-policy.script; + in + lib.hasInfix "path \"secret/data/swarm/services/swarm-bao-collector/oidc/client\" {" s + && lib.hasInfix "capabilities = [\"read\"]" s + && lib.length (lib.filter lib.isList (builtins.split "path \"" s)) == 1 + && !(lib.hasInfix "secret/data/swarm/agents" s) + && !(lib.hasInfix "secret/data/swarm/hives" s) + && !(lib.hasInfix "secret/data/swarm/services/*" s) + && !(lib.hasInfix "services/swarm-collector/" s) + && !(lib.hasInfix "swarm-grafana" s) + && !(lib.hasInfix "sys/policies/acl" s); + } { # 🩸 The half that makes the policies above bind: a policy grants only # through a token that carries it, and a token is minted by a cert-auth @@ -463,7 +491,7 @@ let # The per-hive subjects carry the hive name because their paths do; the # two service subjects do not, because an OIDC client is registered once # per swarm. Pinned so neither shape is tidied into the other. - name = "each of the four readers logs in under a subject of its own"; + name = "each of the five readers logs in under a subject of its own"; ok = let subjectOf = @@ -481,7 +509,8 @@ let subjectOf "swarm-bao-matrix-token-policy" "swarm-matrix-token-h1" "swarm-bao-matrix-token-h1" && subjectOf "swarm-bao-queue-agent-policy" "swarm-queue-agent-h1" "swarm-bao-queue-agent-h1" && subjectOf "swarm-bao-grafana-oidc-policy" "swarm-grafana-oidc" "swarm-bao-grafana-oidc" - && subjectOf "swarm-bao-otel-oidc-policy" "swarm-otel-oidc" "swarm-bao-otel-oidc"; + && subjectOf "swarm-bao-otel-oidc-policy" "swarm-otel-oidc" "swarm-bao-otel-oidc" + && subjectOf "swarm-bao-forwarder-oidc-policy" "swarm-forwarder-oidc" "swarm-bao-forwarder-oidc"; } { # 🩸 The consuming side, and the arm that would catch the regression that @@ -492,7 +521,7 @@ let # # Each pair is asserted whole: a certificate with no key authenticates # nothing, so a half-set pair is a reader that does not render. - name = "each of the four readers presents its own leaf, never the hive's"; + name = "each of the five readers presents its own leaf, never the hive's"; ok = let b = baoGrantWithConsumers.services.hyperhive.deploy.bao; @@ -509,6 +538,8 @@ let b.grafanaOidcClientKeyFile b.otelOidcClientCertFile b.otelOidcClientKeyFile + b.forwarderOidcClientCertFile + b.forwarderOidcClientKeyFile ]; envOf = unit: baoGrantWithConsumers.systemd.services.${unit}.environment; presents = @@ -521,7 +552,8 @@ let && presents "swarm-bao-matrix-token" b.matrixTokenClientCertFile b.matrixTokenClientKeyFile && presents "swarm-bao-queue-agent" b.queueAgentClientCertFile b.queueAgentClientKeyFile && presents "swarm-bao-grafana-oidc" b.grafanaOidcClientCertFile b.grafanaOidcClientKeyFile - && presents "swarm-bao-otel-oidc" b.otelOidcClientCertFile b.otelOidcClientKeyFile; + && presents "swarm-bao-otel-oidc" b.otelOidcClientCertFile b.otelOidcClientKeyFile + && presents "swarm-bao-forwarder-oidc" b.forwarderOidcClientCertFile b.forwarderOidcClientKeyFile; } { # The minting side of the same claim. A role matching a subject nothing @@ -547,6 +579,8 @@ let "swarm-bao-grafana-oidc \"\" clientAuth" "/otel-oidc.pem ]" "swarm-bao-otel-oidc \"\" clientAuth" + "/forwarder-oidc.pem ]" + "swarm-bao-forwarder-oidc \"\" clientAuth" ]; } { @@ -555,7 +589,7 @@ let # still asserted, exactly as the three service principals above behave in # this deployment. A unit that vanished here would take the policy with # it and leave nothing to diagnose. - name = "with no client CA the four readers get policies but no login roles"; + name = "with no client CA the five readers get policies but no login roles"; ok = let units = [ @@ -563,6 +597,7 @@ let "swarm-bao-queue-agent-policy" "swarm-bao-grafana-oidc-policy" "swarm-bao-otel-oidc-policy" + "swarm-bao-forwarder-oidc-policy" ]; scriptOf = unit: baoGrantNoClientCa.systemd.services.${unit}.script; in @@ -578,7 +613,7 @@ let # client certificate and the host is the side that has one, so a unit # rendered inside the store's container would have neither an identity # nor a route. Plus the ordering that makes the mounts exist first. - name = "the four readers' granting units are ordered after the mounts and rendered on the host"; + name = "the five readers' granting units are ordered after the mounts and rendered on the host"; ok = let units = [ @@ -586,6 +621,7 @@ let "swarm-bao-queue-agent-policy" "swarm-bao-grafana-oidc-policy" "swarm-bao-otel-oidc-policy" + "swarm-bao-forwarder-oidc-policy" ]; in lib.all ( @@ -598,7 +634,11 @@ let # The other end of those units: each reader logs in against the role its # own policy unit writes, so it has to wait for that unit. Ordering and # never a requirement, since the policy unit skips once the token is gone. - name = "each of the four readers is ordered after the unit writing its role"; + # + # The forwarder is listed apart from `policyReaders`: it renders wherever + # the store does, so it is never absent on a store host and never present + # on a remote one, and the two cases below would fail on it for that. + name = "each of the five readers is ordered after the unit writing its role"; ok = let s = baoGrantWithConsumers.systemd.services; @@ -611,7 +651,7 @@ let && lib.elem policy s.${reader}.wants && !(lib.elem policy s.${reader}.requires); in - lib.all waitsFor policyReaders; + lib.all waitsFor (policyReaders ++ [ "swarm-bao-forwarder-oidc" ]); } { # The ordering is set apart from each reader's own definition, so it can @@ -643,7 +683,7 @@ let # `baoGrantNoStore` makes for the controller's, one file over. Without # this arm `lib.mkIf haveBootstrapToken` could be dropped from the shared # builder and every other case here would still pass. - name = "without a bootstrap token none of the four readers' granting units render"; + name = "without a bootstrap token none of the five readers' granting units render"; ok = let s = baoGrantNoStore.systemd.services; @@ -651,7 +691,27 @@ let !(s ? swarm-bao-matrix-token-policy) && !(s ? swarm-bao-queue-agent-policy) && !(s ? swarm-bao-grafana-oidc-policy) - && !(s ? swarm-bao-otel-oidc-policy); + && !(s ? swarm-bao-otel-oidc-policy) + && !(s ? swarm-bao-forwarder-oidc-policy); + } + { + # 🩸 The refusal half of the forwarder's own leaf. It renders wherever the + # store does and has no mode without a secret, so a null pair has one + # fallback left — the hive's leaf and its union grant. Refused at eval, + # with both options named. + name = "a store host without the forwarder's own pair is refused, naming both options"; + ok = + let + refused = lib.filter (a: !a.assertion) baoNoForwarderIdentity.assertions; + names = + a: + lib.hasInfix "services.hyperhive.deploy.bao.forwarderOidcClientCertFile" a.message + && lib.hasInfix "services.hyperhive.deploy.bao.forwarderOidcClientKeyFile" a.message; + in + lib.any names refused + # The control: the same store with the pair in place trips no such + # assertion, so the arm above is not firing on every store host. + && !(lib.any (a: !a.assertion && names a) baoGrantHere.assertions); } { # The policy authorising this route lives in another file, and nothing @@ -760,7 +820,7 @@ let { # What makes the case above mean something: discovery by token path # reaches every unit that uses the token today, and each yields calls. - name = "the bootstrap-policy check sees all nine units that use the token, and parses calls from each"; + name = "the bootstrap-policy check sees all ten units that use the token, and parses calls from each"; ok = lib.all (n: bootstrapUnits ? ${n}) [ "swarm-bao-controller-policy" @@ -770,6 +830,7 @@ let "swarm-bao-queue-agent-policy" "swarm-bao-grafana-oidc-policy" "swarm-bao-otel-oidc-policy" + "swarm-bao-forwarder-oidc-policy" "swarm-bao-services-issuer-policy" "swarm-bao-nats-tls-policy" ] diff --git a/nix/module-eval/bao-otel-collector.nix b/nix/module-eval/bao-otel-collector.nix index 4ec0d472..d852f5de 100644 --- a/nix/module-eval/bao-otel-collector.nix +++ b/nix/module-eval/bao-otel-collector.nix @@ -382,7 +382,7 @@ let } { # Same 403-not-a-miss property the grafana and matrix readers are pinned - # for: the reader's grant covers the `services` prefix, so a secret + # for: the reader's grant is one path under `services`, so a secret # filed under the hive that happens to run the store would be refused # rather than missing, however correct the path reads. name = "the forwarder's secret is read from the prefix the publisher writes"; diff --git a/nix/module-eval/name-guards.nix b/nix/module-eval/name-guards.nix index e035b0f0..9a4f85a9 100644 --- a/nix/module-eval/name-guards.nix +++ b/nix/module-eval/name-guards.nix @@ -76,6 +76,13 @@ let swarm.hives.otctl.domain = "o.t.local"; }; + # The store forwarder's OIDC reader, the fifth fixed subject. + hiveNamedAfterForwarderOidcSubject = hive { + deploy.swarm-otel.enable = false; + deploy.bao.forwarderOidcCommonName = "fwctl"; + swarm.hives.fwctl.domain = "f.t.local"; + }; + # 🩸 A different shape from every fixture above: the matrix-token and # queue-credential roles are written PER HIVE, so the subject a hive must not # be is `-` rather than the prefix itself. Reserving @@ -157,6 +164,15 @@ let a: !a.assertion && lib.hasInfix "'otctl'" a.message ) hiveNamedAfterOtelOidcSubject.assertions; } + { + # And the store forwarder's, for the same reason one element later. + name = "a hive named after the store forwarder's OIDC-reader subject is refused too"; + ok = + equalityGuardFired hiveNamedAfterForwarderOidcSubject + && lib.any ( + a: !a.assertion && lib.hasInfix "'fwctl'" a.message + ) hiveNamedAfterForwarderOidcSubject.assertions; + } { # 🩸 The per-hive half, and the one a prefix-only reservation would miss: # the role is `-`, so the reserved string has to be composed