vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

swarm-bao: give each hive-cert consumer its own bao identity

Browse source 
Four units read one path each out of the store, and all four logged in
holding `deploy.bao.clientCertFile` — the hive's own leaf. Bao identifies
a principal by the subject of the certificate it presents, so four
readers behind one certificate were ONE principal, and the only grant
expressible was the union of what the four need: read on
`swarm/agents/*`, `swarm/hives/<hive>/*` and `swarm/services/*`. The unit
fetching Grafana's OIDC client secret could fetch every agent credential
in the swarm; the one fetching this hive's matrix token could fetch
Grafana's. Least privilege was not misconfigured here, it was
unrepresentable.

Each now holds a leaf, a cert-auth role and a policy of its own, and each
policy is the single `secret/data/…` path that unit's own script names —
spelled to the leaf, not to a prefix, the way matrix-ctl's already is.
Following the four exemplars in-tree rather than building a mechanism:
`signLeaf` mints the leaves, `swarm-bao.nix` writes the roles from the
bootstrap token, the consumers name their own pair.

Two of the four are written PER HIVE and two are not, which is the shape
of the paths rather than a preference. A matrix appservice token and a
queue credential live under `swarm/hives/<name>/` and every hive runs a
reader for its own, so one role for all of them would have to be granted
`hives/*` — letting one hive read another's, a reach no hive has today.
An OIDC client secret lives under `swarm/services/<client-id>/` and a
swarm registers each exactly once, so one role each is enough. The
per-hive subjects are `<prefix>-<hive>` and swarm.nix reserves every
composed spelling as a hive name, so a hive cannot be named into another
hive's role.

The shared leaf stays: hive-c0re still passes it into its container, the
`bao` CLI wrapper still defaults to it, and the three
`glue-*-bao-identity.nix` files derive the PKI directory from it.

module-eval-bao-grants gains a negative arm per principal — each pins the
three stanzas the hive's leaf carried and the two wildcards a later
widening would reach for, so a policy that grows fails here rather than
in a store. Plus the consuming side: repointing a unit back at the hive's
leaf would evaluate, deploy and log in, and silently restore the union.

A hive that reads a store on another machine now places one leaf per
principal instead of one shared by four. That cost is the point, and
docs/swarm/secrets.md lists the pairs.
This commit is contained in:
atlas 2026-09-21 23:44:59 +02:00 • committed by mara
commit f1445b4c8b
14 changed files with 859 additions and 43 deletions
Download patch file Download diff file Expand all files Collapse all files

15
docs/getting-started/setup.md
View file

@ -170,16 +170,19 @@ What else you need depends on
restart, with the keys `init` printed.
The store serves TLS, and on a hive that deploys it you need do nothing: a
first-boot unit mints a CA of the store's own plus the two leaves it signs —
the store's server certificate and this host's client certificate — and points
`deploy.bao.serverCertFile`, `.serverKeyFile` and `.clientCaFile` at the store's
half, `.clientCertFile`, `.clientKeyFile` and `.serverCaFile` at the reader's.
first-boot unit mints a CA of the store's own plus the leaves it signs — the
store's server certificate, this host's client certificate, and one per service
principal — and points `deploy.bao.serverCertFile`, `.serverKeyFile` and
`.clientCaFile` at the store's half, `.clientCertFile`, `.clientKeyFile` and
`.serverCaFile` at the reader's, and each principal's own pair at its own leaf.
Those are `mkDefault`s, so naming your own paths wins. Do that when your
certificates come from a real internal CA; the store has no opinion about
which. A hive that does **not** deploy the store names the reader's three
itself: the operator issues that leaf out of band, and it's the one credential
the store can't hand you, being what opens it. ⚠️ Not the gateway's HTTPS certificates and not the hive CA — this is
itself, plus a pair for every principal it runs — see
[per-principal identities](../swarm/secrets.md#per-principal-identities) for the
list. The operator issues those leaves out of band; they're the credentials the
store can't hand you, being what opens it. ⚠️ Not the gateway's HTTPS certificates and not the hive CA — this is
**mTLS between services and the store**, a separate trust domain, because a
store that took its identity from an authority it itself distributes could
never come up before that authority.

68
docs/swarm/secrets.md
View file

@ -88,7 +88,7 @@ where an absent OIDC block is a container with no way in at all, a collector
with nothing to authenticate with still receives every hive's telemetry —
only its own pushes to the stores go out unauthenticated and get refused
there. `swarm-bao-otel-oidc.service` therefore doesn't render at all without
`deploy.bao.clientCertFile` / `clientKeyFile`, the same choice
`deploy.bao.otelOidcClientCertFile` / `otelOidcClientKeyFile`, the same choice
`glue-matrix-bao-token.nix` and `glue-queue-agent-credential.nix` make for
their own optional readers, rather than the hard assertion `swarm-grafana.nix`
uses for its non-optional one. The operator can still deliver the secret by
@ -249,14 +249,14 @@ straight out of authelia's tree when the two happen to share a host is one
round trip cheaper and costs a second delivery unit, a second way for the file
to be wrong, and a gate deciding between them — and the gate is the expensive
part, because whatever it's wrong about is an outage nobody can read. The store
exists so a host holds **one** out-of-band secret, its client certificate, and
reads everything else with it.
exists so the only secrets a host holds out of band are **certificates**, and
everything else is read with them.
Two things follow, and `swarm-grafana.nix` asserts both rather than degrading:
running Grafana requires `swarm.authelia.url` (this module disables its local
login form unconditionally, so SSO isn't a feature of some topologies — it's the
only way in), and it requires this host's `deploy.bao.clientCertFile` /
`clientKeyFile`. Each refusal names the option to set. Both used to be silent:
only way in), and it requires this host's `deploy.bao.grafanaOidcClientCertFile`
/ `grafanaOidcClientKeyFile`. Each refusal names the option to set. Both used to be silent:
a null URL dropped the OIDC block, a missing leaf produced a warning, and
either one left a Grafana with no SSO and no password box — nothing failed, so
nothing said anything. Registration is separate and stays where authelia is
@ -271,7 +271,8 @@ runs the collector, and registration lives separately in
`glue-swarm-otel-oidc-client.nix` for the same reason Grafana's does. Where it
differs is what an absent credential means: the collector still receives
telemetry with none, so `swarm-otel.nix` doesn't assert
`deploy.bao.clientCertFile` / `clientKeyFile` the way Grafana does — it
`deploy.bao.otelOidcClientCertFile` / `otelOidcClientKeyFile` the way Grafana
does — it
doesn't render the reading unit without them, the shape
`glue-matrix-bao-token.nix` and `glue-queue-agent-credential.nix` use for
their own optional reads.
@ -290,10 +291,18 @@ A service's secret is one value for the whole swarm rather than one per hive, so
it lives under the `services` prefix, and a hive's read policy grants that prefix
whole. That's because nothing in a swarm's configuration records which hive runs
a given service — placement is a `deploy.*` fact, per-host by definition — so
there is nothing to scope the grant to. Every hive can therefore read every swarm
service's client secret; that's the cost of a shape where the service's host has
no store identity of its own to present, and it's stated in
`swarm-secret-client`'s `policy` module beside the grant itself. Absence behaves
there is nothing to scope the grant to. A hive's **own** leaf can therefore read
every swarm service's client secret, and that's stated in
`swarm-secret-client`'s `policy` module beside the grant itself.
⚠️ **The readers no longer present it.** Four units used to log in with
`deploy.bao.clientCertFile`, which is the hive's own leaf, and bao identifies a
principal by the subject of the certificate it presents — so four readers behind
one leaf were one principal holding the union of their needs. Each now holds a
leaf of its own, and a policy naming only the path that unit reads. See
[per-principal identities](#per-principal-identities) below.
Absence behaves
as it does for the other readers: a store that says "not here" leaves the
file alone and says so, since there is no such thing as a locally valid OIDC
client secret to stand in.
@ -303,14 +312,43 @@ credential you must already hold to authenticate from the thing it
authenticates you to, so the store's identity can't come from an authority the store
distributes — which excludes the hive CA and the swarm CA both, and has nothing
to do with the gateway's HTTPS certificates either way. `glue-bao-tls.nix`
mints a CA that signs exactly two things, the store's server certificate and a
reader's client certificate, and distributes nothing. A deployment with a real
mints a CA that signs a fixed, short list of leaves — the store's server
certificate, the hive's own reader certificate, and one per service principal —
and distributes nothing. A deployment with a real
internal CA deletes that file and names its own paths in
`deploy.bao.serverCertFile` / `clientCaFile`; the store itself has no opinion.
A hive that reads from a store on **another** machine names the reader's half —
`clientCertFile`, `clientKeyFile`, `serverCaFile` — and places that leaf by hand.
It's the one credential that can't come out of the store, being what opens it;
everything else a hive needs does.
`clientCertFile`, `clientKeyFile`, `serverCaFile` — and places that leaf by hand,
plus one leaf per principal it runs (the options are listed below). Those are the
credentials that can't come out of the store, being what opens it; everything
else a hive needs does.
### Per-principal identities
Bao matches a cert-auth role on the certificate's subject, so a certificate is an
identity and sharing one merges the identities. These four units read one path
each and each holds a leaf, a role and a policy of its own:
| unit | option pair under `deploy.bao.` | reads |
| ------------------------ | -------------------------------------------------------- | -------------------------------------------------- |
| `swarm-bao-matrix-token` | `matrixTokenClientCertFile` / `matrixTokenClientKeyFile` | `swarm/hives/<hive>/matrix/appservice-token` |
| `swarm-bao-queue-agent` | `queueAgentClientCertFile` / `queueAgentClientKeyFile` | `swarm/hives/<hive>/queue/agent` |
| `swarm-bao-grafana-oidc` | `grafanaOidcClientCertFile` / `grafanaOidcClientKeyFile` | `swarm/services/<grafana client id>/oidc/client` |
| `swarm-bao-otel-oidc` | `otelOidcClientCertFile` / `otelOidcClientKeyFile` | `swarm/services/<collector client id>/oidc/client` |
The first two are written **per hive**, because the path they read carries a hive
name and every hive runs its own reader. Their subjects are
`<deploy.bao.matrixTokenCommonNamePrefix>-<hive>` and
`<deploy.bao.queueAgentCommonNamePrefix>-<hive>`; `swarm.nix` reserves both
composed spellings as hive names, so a hive can't be named into another hive's
role. The other two read a path that names a swarm service rather than a hive, so
one role each is enough and their subjects are the flat
`deploy.bao.grafanaOidcCommonName` and `deploy.bao.otelOidcCommonName`.
On a host that mints its own PKI, `glue-bao-tls.nix` signs all four and defaults
all eight options, and there is nothing to do. Elsewhere each leaf is issued from
that CA out of band and named here — one file per principal rather than one file
shared by four, which is the whole of what this buys.
## How a reader reaches the store

48
nix/host-modules/glue-bao-tls.nix
View file

@ -84,6 +84,25 @@ in
clientCertFile = lib.mkDefault "${pkiDir}/client.pem";
clientKeyFile = lib.mkDefault "${pkiDir}/client-key.pem";
serverCaFile = lib.mkDefault "${pkiDir}/ca.pem";
# 🩸 Four readers that used to present `client.pem` above, each now
# pointed at a leaf of its own. The leaf is what bao sees, so this pairing
# is the whole of what turns "one principal with the union of four grants"
# into four principals with one grant each — see ./swarm-bao.nix's
# `perHiveReaders` block for the grants themselves.
#
# Defaulted here rather than in four `glue-<consumer>-bao-identity.nix`
# files: those exist where the consumer is a CONTAINER with a
# `deploy.<service>.*` namespace of its own to point at. These four are
# host units reading the store, which is the pairing this file already is.
matrixTokenClientCertFile = lib.mkDefault "${pkiDir}/matrix-token.pem";
matrixTokenClientKeyFile = lib.mkDefault "${pkiDir}/matrix-token-key.pem";
queueAgentClientCertFile = lib.mkDefault "${pkiDir}/queue-agent.pem";
queueAgentClientKeyFile = lib.mkDefault "${pkiDir}/queue-agent-key.pem";
grafanaOidcClientCertFile = lib.mkDefault "${pkiDir}/grafana-oidc.pem";
grafanaOidcClientKeyFile = lib.mkDefault "${pkiDir}/grafana-oidc-key.pem";
otelOidcClientCertFile = lib.mkDefault "${pkiDir}/otel-oidc.pem";
otelOidcClientKeyFile = lib.mkDefault "${pkiDir}/otel-oidc-key.pem";
};
# Idempotent on ABSENCE, never on content. Re-issuing the CA invalidates
@ -160,6 +179,35 @@ in
# container an identity instead of lending it the hive's.
[ -s ${pkiDir}/matrix-ctl.pem ] || ${signLeaf} ${pkiDir} matrix-ctl \
${lib.escapeShellArg deployCfg.bao.matrixCtlCommonName} "" clientAuth
# 🩸 The four readers that used to present `client.pem` above. Each
# carries its own subject, which is the entire mechanism: bao matches a
# cert-auth role on the CN, so four units sharing one leaf were one
# principal holding the union of four grants, and the union included
# every agent credential in the swarm.
#
# ⚠️ The first two carry THIS hive's name in the subject, unlike the
# three service leaves above. Their grants name one hive's path, because
# a matrix token and a queue credential live under `swarm/hives/<name>/`
# and every hive runs a reader for its own; ./swarm-bao.nix writes one
# role per hive in the swarm directory to match. The two OIDC readers
# need no such segment — a client is registered once per swarm.
#
# Minted whether or not the consumer runs here, for the reason the three
# leaves above give: on a hive that does not run the store this is the
# file an operator copies, and a leaf that only appears where its
# consumer does is one nobody can copy from anywhere.
[ -s ${pkiDir}/matrix-token.pem ] || ${signLeaf} ${pkiDir} matrix-token \
${lib.escapeShellArg "${deployCfg.bao.matrixTokenCommonNamePrefix}-${clientCn}"} "" clientAuth
[ -s ${pkiDir}/queue-agent.pem ] || ${signLeaf} ${pkiDir} queue-agent \
${lib.escapeShellArg "${deployCfg.bao.queueAgentCommonNamePrefix}-${clientCn}"} "" clientAuth
[ -s ${pkiDir}/grafana-oidc.pem ] || ${signLeaf} ${pkiDir} grafana-oidc \
${lib.escapeShellArg deployCfg.bao.grafanaOidcCommonName} "" clientAuth
[ -s ${pkiDir}/otel-oidc.pem ] || ${signLeaf} ${pkiDir} otel-oidc \
${lib.escapeShellArg deployCfg.bao.otelOidcCommonName} "" clientAuth
'';
};
};

15
nix/host-modules/glue-matrix-bao-token.nix
View file

@ -38,7 +38,16 @@ let
# What decides whether this unit exists at all. A reader is defined by holding
# a certificate the store accepts, and that is true on the store's own host
# and on a hive three networks away for exactly the same reason.
haveClientIdentity = baoDeploy.clientCertFile != null && baoDeploy.clientKeyFile != null;
#
# 🩸 This principal's OWN leaf, not `clientCertFile` — the hive's, which four
# units used to share. Bao matches a cert-auth role on the CN, so one leaf for
# four readers was ONE principal holding the union of four grants: read on
# `swarm/agents/*` AND `swarm/hives/<hive>/*` AND `swarm/services/*`, when
# this unit reads one appservice token and nothing else. Its own leaf carries
# `<matrixTokenCommonNamePrefix>-<hive>` and its role grants the single path
# below.
haveClientIdentity =
baoDeploy.matrixTokenClientCertFile != null && baoDeploy.matrixTokenClientKeyFile != null;
# Where the token lives in the store. A path, not a convention to guess at:
# whoever writes it and whoever reads it must agree, and the agreement
@ -115,8 +124,8 @@ in
};
environment = {
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
BAO_CLIENT_CERT = baoDeploy.clientCertFile;
BAO_CLIENT_KEY = baoDeploy.clientKeyFile;
BAO_CLIENT_CERT = baoDeploy.matrixTokenClientCertFile;
BAO_CLIENT_KEY = baoDeploy.matrixTokenClientKeyFile;
}
# Absent means the system trust store, which is what a deployment with a
# real CA wants and what a self-signed one must not be left with.

15
nix/host-modules/glue-queue-agent-credential.nix
View file

@ -40,7 +40,16 @@ let
# What decides whether this unit exists at all. A reader is defined by holding
# a certificate the store accepts, and that is true on the store's own host
# and on a hive three networks away for exactly the same reason.
haveClientIdentity = baoDeploy.clientCertFile != null && baoDeploy.clientKeyFile != null;
#
# 🩸 This principal's OWN leaf, not `clientCertFile` — the hive's, which four
# units used to share. Bao matches a cert-auth role on the CN, so one leaf for
# four readers was ONE principal holding the union of four grants: read on
# `swarm/agents/*` AND `swarm/hives/<hive>/*` AND `swarm/services/*`, when
# this unit reads one queue credential and nothing else. Its own leaf carries
# `<queueAgentCommonNamePrefix>-<hive>` and its role grants the single path
# below — still this hive's own, so the narrowing costs no reach.
haveClientIdentity =
baoDeploy.queueAgentClientCertFile != null && baoDeploy.queueAgentClientKeyFile != null;
credentialDir = toString deployCfg.hive-controller.queue.agentCredentialDir;
secretFile = "${credentialDir}/secret";
@ -141,8 +150,8 @@ in
};
environment = {
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
BAO_CLIENT_CERT = baoDeploy.clientCertFile;
BAO_CLIENT_KEY = baoDeploy.clientKeyFile;
BAO_CLIENT_CERT = baoDeploy.queueAgentClientCertFile;
BAO_CLIENT_KEY = baoDeploy.queueAgentClientKeyFile;
}
# Absent means the system trust store, which is what a deployment with a
# real CA wants and what a self-signed one must not be left with.

342
nix/host-modules/swarm-bao.nix
View file

@ -340,6 +340,156 @@ let
}
'';
# ── the four principals that used to share the hive's own leaf ─────────────
#
# 🩸 Each of the four reads exactly ONE path in the store, and until this
# split each did it holding `deploy.bao.clientCertFile` — the hive's own leaf,
# whose policy (`swarm-secret-client`'s `policy::render`) grants read on
# `swarm/agents/*`, `swarm/hives/<hive>/*` AND `swarm/services/*`. Four
# principals presenting one certificate are ONE principal to bao, so the union
# of what the four need was the only grant expressible: the unit that fetches
# Grafana's OIDC client secret could fetch every agent credential in the
# swarm, and the unit that fetches this hive's matrix token could fetch
# Grafana's. Least privilege was not misconfigured, it was unrepresentable.
#
# Each now gets its own leaf, its own cert-auth role and a policy holding the
# one `secret/data/…` path that principal's script actually names — spelled to
# the LEAF rather than to a prefix, for the reason `matrixCtlPolicyText` above
# gives for doing the same.
#
# ⚠️ Two of the four are PER HIVE and two are not, and that asymmetry is the
# shape of the paths rather than a preference. A matrix appservice token and a
# queue credential live under `swarm/hives/<name>/`, and every hive runs a
# reader for its own; an OIDC client secret lives under
# `swarm/services/<client-id>/` and a swarm registers each of those exactly
# once. A single role for a per-hive reader would have to name `hives/*` to
# serve every hive — which would let one hive read another's matrix token, a
# reach no hive has today and one this change exists to remove rather than
# create.
readStanza = path: ''
path "${path}" {
capabilities = ["read"]
}
'';
# One reader per hive in the swarm directory. Written from the directory
# rather than from this host's own name because the policy is written where
# the STORE is and the reader runs where its hive is — the same split
# `matrixCtlHiveName` above exists to paper over, answered here by naming
# every hive instead of asking the operator which one.
perHiveReaders =
{ rolePrefix, cnPrefix, ... }@spec:
lib.mapAttrsToList (hiveName: _: {
name = "${rolePrefix}-${hiveName}";
cn = "${cnPrefix}-${hiveName}";
policyText = readStanza (spec.path hiveName);
}) hyperhiveCfg.swarm.hives;
matrixTokenReaders = perHiveReaders {
rolePrefix = "swarm-matrix-token";
cnPrefix = baoDeploy.matrixTokenCommonNamePrefix;
path = h: "${credentialMountPath}/data/swarm/hives/${h}/matrix/appservice-token";
};
queueAgentReaders = perHiveReaders {
rolePrefix = "swarm-queue-agent";
cnPrefix = baoDeploy.queueAgentCommonNamePrefix;
path = h: "${credentialMountPath}/data/swarm/hives/${h}/queue/agent";
};
# Singletons, so a list of one rather than a second shape: the granting unit
# below takes a list either way and the four units stay identical apart from
# the objects they write.
grafanaOidcReaders = [
{
name = "swarm-grafana-oidc";
cn = baoDeploy.grafanaOidcCommonName;
policyText = readStanza "${credentialMountPath}/data/swarm/services/${hyperhiveCfg.swarm.grafana.oidc.clientId}/oidc/client";
}
];
otelOidcReaders = [
{
name = "swarm-otel-oidc";
cn = baoDeploy.otelOidcCommonName;
policyText = readStanza "${credentialMountPath}/data/swarm/services/${hyperhiveCfg.swarm.otel.clientId}/oidc/client";
}
];
# The role name IS the policy name, as for the three service principals
# above: the role attaches the policy by spelling it identically, and one
# string for both objects removes the way they drift apart.
readerPolicyWrite = obj: ''
printf '%s' ${lib.escapeShellArg obj.policyText} |
bao policy write ${lib.escapeShellArg obj.name} -
'';
# ⚠️ Names are `swarm-<principal>[-<hive>]`, outside the `hive-*` namespace the
# controller may rewrite, for the reason the three service principals above
# state. A per-hive CN cannot collide with a hive's own leaf either — a hive's
# CN is its bare name and ../reserved-hive-fragments.nix forbids the substring
# `swarm` in one — and ./swarm.nix reserves the concrete per-hive spellings
# anyway, because the prefixes are operator-settable and an operator may spell
# one without that substring.
readerRoleWrite = obj: ''
bao write auth/cert/certs/${lib.escapeShellArg obj.name} \
certificate=@${tlsDir}/client-ca.pem \
allowed_common_names=${lib.escapeShellArg obj.cn} \
token_policies=${lib.escapeShellArg obj.name} \
display_name=${lib.escapeShellArg obj.cn}
'';
# One unit per PRINCIPAL, not one unit for the four: these names are
# operator-facing strings, the same reason `swarm-bao-secret-publisher-policy`
# above gives for not being more script inside the controller's unit. A hive
# whose matrix token never arrives is diagnosed by the state of the unit named
# after it.
#
# `after` and not `requires`, for the reason the publisher's unit states: the
# controller's unit creates the KV and cert-auth mounts this one writes into,
# but a failed oneshot still counts as finished, so ordering plus this unit's
# own retry is what converges.
#
# The role write is inside the client-CA branch and the policy write is not,
# exactly as the three above: with no CA there is no trust anchor for a login
# role, but the policy it would attach is still worth asserting.
readerPolicyUnit =
description: objects:
lib.mkIf haveBootstrapToken {
inherit description;
after = [
"container@${cfg.machine}.service"
"swarm-bao-controller-policy.service"
];
wantedBy = [ "multi-user.target" ];
path = [
baoCli
pkgs.coreutils
];
unitConfig.ConditionPathExists = baoDeploy.bootstrapTokenFile;
# Same unseal wait as its siblings above, for the reason stated there:
# under `seal = "shamir"` a human unseals by hand.
startLimitBurst = 2880;
startLimitIntervalSec = 90000;
serviceConfig = {
Type = "oneshot";
RemainAfterExit = true;
Restart = "on-failure";
RestartSec = 30;
};
script = ''
set -euo pipefail
BAO_TOKEN="$(cat ${lib.escapeShellArg baoDeploy.bootstrapTokenFile})"
export BAO_TOKEN
''
+ lib.concatMapStrings readerPolicyWrite objects
+ lib.optionalString (baoDeploy.clientCaFile != null) (
"\n" + lib.concatMapStrings readerRoleWrite objects
);
};
# Every listener serves the same identity: they differ in which address
# they answer on, not in who they are. Client verification is separate and
# optional — a store with no `clientCaFile` still serves TLS, it just does
@ -846,6 +996,185 @@ in
'';
};
matrixTokenCommonNamePrefix = lib.mkOption {
type = lib.types.str;
default = "swarm-bao-matrix-token";
example = "swarm-bao-matrix-token.svc";
description = ''
Prefix of the subject the store's per-hive matrix-token roles accept.
The role for hive `h` accepts `<prefix>-h` and nothing else, and grants
read on exactly `swarm/hives/h/matrix/appservice-token`.
A **prefix** rather than one subject because every hive runs its own
reader and the path it reads carries that hive's name. One subject for
all of them would need a `hives/*` grant, which would let one hive read
another hive's appservice token — a reach no hive has today.
⚠️ Reserved as a hive name by ./swarm.nix, per hive, for the reason its
fixed-subject siblings are: cert auth trusts the CA and
`allowed_common_names` is the whole narrowing, so a hive whose own leaf
carried one of these spellings would receive that grant.
'';
};
queueAgentCommonNamePrefix = lib.mkOption {
type = lib.types.str;
default = "swarm-bao-queue-agent";
example = "swarm-bao-queue-agent.svc";
description = ''
Prefix of the subject the store's per-hive queue-credential roles
accept. The role for hive `h` accepts `<prefix>-h` and grants read on
exactly `swarm/hives/h/queue/agent` — the client credential that hive's
agent containers authenticate to the swarm queue with.
Per hive for the same reason
{option}`services.hyperhive.deploy.bao.matrixTokenCommonNamePrefix` is,
and more sharply: every hive in the swarm runs this reader, so a single
subject would have to be granted `hives/*`.
'';
};
grafanaOidcCommonName = lib.mkOption {
type = lib.types.str;
default = "swarm-bao-grafana-oidc";
example = "swarm-bao-grafana-oidc.svc";
description = ''
Subject the store's `swarm-grafana-oidc` cert-auth role accepts — the
identity the unit that fetches Grafana's OIDC client secret presents.
Its grant is one path,
`swarm/services/<grafana client id>/oidc/client`, and read only. Not
per hive, unlike the two prefixes above: an OIDC client is registered
once per swarm, so the path names the service and never a hive.
⚠️ Same collision as its siblings, and the same answer: ./swarm.nix
feeds this value into the guard on
{option}`services.hyperhive.swarm.hives`.
'';
};
otelOidcCommonName = lib.mkOption {
type = lib.types.str;
default = "swarm-bao-otel-oidc";
example = "swarm-bao-otel-oidc.svc";
description = ''
Subject the store's `swarm-otel-oidc` cert-auth role accepts — the
identity the unit that fetches the collector's OIDC client secret
presents. Its grant is one path,
`swarm/services/<collector client id>/oidc/client`, and read only.
A **fourth** identity rather than reuse of
{option}`services.hyperhive.deploy.bao.grafanaOidcCommonName`: the two
read different services' client secrets, and a collector is not
entitled to Grafana's.
'';
};
matrixTokenClientCertFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-bao-pki/matrix-token.pem";
description = ''
Certificate the unit that fetches this hive's matrix appservice token
presents to the store. Its subject must be
`<matrixTokenCommonNamePrefix>-<this hive's name>`; cert auth matches on
the CN and the role accepts nothing else.
⚠️ **Not** {option}`services.hyperhive.deploy.bao.clientCertFile`.
Pointing this at the hive's own leaf would evaluate, deploy and log in —
and hand a unit that reads one token a credential that reads every
secret in the store. That is what this option exists to end.
No default here: ./glue-bao-tls.nix points it at the leaf it mints,
wherever this host mints its own PKI. A deployment that reads a store on
another host names it by hand, alongside the leaf itself — one file per
principal placed out of band instead of one shared by four.
'';
};
matrixTokenClientKeyFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-bao-pki/matrix-token-key.pem";
description = ''
Private key for
{option}`services.hyperhive.deploy.bao.matrixTokenClientCertFile`. Both
or neither — a certificate with no key authenticates nothing.
'';
};
queueAgentClientCertFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-bao-pki/queue-agent.pem";
description = ''
Certificate the unit that fetches this hive's queue client credential
presents to the store. Its subject must be
`<queueAgentCommonNamePrefix>-<this hive's name>`.
⚠️ Not the hive's own leaf, for the reason
{option}`services.hyperhive.deploy.bao.matrixTokenClientCertFile` gives.
'';
};
queueAgentClientKeyFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-bao-pki/queue-agent-key.pem";
description = ''
Private key for
{option}`services.hyperhive.deploy.bao.queueAgentClientCertFile`.
'';
};
grafanaOidcClientCertFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-bao-pki/grafana-oidc.pem";
description = ''
Certificate the unit that fetches Grafana's OIDC client secret presents
to the store. Its subject must be
{option}`services.hyperhive.deploy.bao.grafanaOidcCommonName`.
⚠️ Not the hive's own leaf, for the reason
{option}`services.hyperhive.deploy.bao.matrixTokenClientCertFile` gives.
'';
};
grafanaOidcClientKeyFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-bao-pki/grafana-oidc-key.pem";
description = ''
Private key for
{option}`services.hyperhive.deploy.bao.grafanaOidcClientCertFile`.
'';
};
otelOidcClientCertFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-bao-pki/otel-oidc.pem";
description = ''
Certificate the unit that fetches the collector's OIDC client secret
presents to the store. Its subject must be
{option}`services.hyperhive.deploy.bao.otelOidcCommonName`.
⚠️ Not the hive's own leaf, for the reason
{option}`services.hyperhive.deploy.bao.matrixTokenClientCertFile` gives.
'';
};
otelOidcClientKeyFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-bao-pki/otel-oidc-key.pem";
description = ''
Private key for
{option}`services.hyperhive.deploy.bao.otelOidcClientCertFile`.
'';
};
serverCaFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
@ -1593,6 +1922,19 @@ in
'';
};
# 🩸 The four principals that used to log in as the hive. See the block
# around `perHiveReaders` for what each grant is and why the first two are
# per hive; the four units are identical apart from the objects they
# write, so the shape lives in `readerPolicyUnit` and the difference is
# visible here in one line each.
systemd.services.swarm-bao-matrix-token-policy = readerPolicyUnit "write the per-hive matrix-token bao policies and cert-auth roles" matrixTokenReaders;
systemd.services.swarm-bao-queue-agent-policy = readerPolicyUnit "write the per-hive queue-credential bao policies and cert-auth roles" queueAgentReaders;
systemd.services.swarm-bao-grafana-oidc-policy = readerPolicyUnit "write Grafana's OIDC-secret-reader bao policy and cert-auth role" grafanaOidcReaders;
systemd.services.swarm-bao-otel-oidc-policy = readerPolicyUnit "write the collector's OIDC-secret-reader bao policy and cert-auth role" otelOidcReaders;
# The CA bind source is written at runtime by a host unit, so the
# container has to start after it — otherwise nspawn sets up a mount
# over a file that does not exist yet.

28
nix/host-modules/swarm-grafana.nix
View file

@ -119,7 +119,14 @@ let
# other secret is read with it. A Grafana host without it has not been given
# its identity yet, which is a thing to say out loud rather than to route
# around by reaching into authelia's tree whenever it happens to be local.
haveClientIdentity = baoDeploy.clientCertFile != null && baoDeploy.clientKeyFile != null;
#
# 🩸 Grafana's OWN leaf, not `clientCertFile` — the hive's, which four units
# used to share. Bao matches a cert-auth role on the CN, so one leaf for four
# readers was ONE principal holding the union of four grants: this unit could
# read every agent credential in the swarm and every other service's OIDC
# client secret, when what it needs is the one path `storeSecretPath` names.
haveClientIdentity =
baoDeploy.grafanaOidcClientCertFile != null && baoDeploy.grafanaOidcClientKeyFile != null;
autheliaUrl = toString hyperhiveCfg.swarm.authelia.url;
@ -451,15 +458,20 @@ in
services.hyperhive.deploy.grafana.enable requires this host to hold a
swarm-secret-store client identity: set both
services.hyperhive.deploy.bao.clientCertFile
services.hyperhive.deploy.bao.clientKeyFile
services.hyperhive.deploy.bao.grafanaOidcClientCertFile
services.hyperhive.deploy.bao.grafanaOidcClientKeyFile
Grafana's OIDC client secret is minted by authelia and read out of
the store, on every host that runs Grafana — including the host that
runs authelia. That is one delivery route rather than two, and it is
what the store is for: this certificate is the single credential
placed out of band, and every other secret comes from the store with
it.
what the store is for: a certificate is the credential placed out of
band, and every other secret comes from the store with it.
⚠️ Grafana's OWN leaf, not deploy.bao.clientCertFile. That one is the
hive's, and its grant reads every secret in the store; this role
reads the one path Grafana's client secret lives at. Pointing this
option at the hive's leaf would evaluate, deploy and log in — and
undo the split.
On a hive that runs the store, glue-bao-tls.nix supplies both as
defaults and there is nothing to do. Elsewhere the leaf is issued
@ -610,8 +622,8 @@ in
};
environment = {
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
BAO_CLIENT_CERT = baoDeploy.clientCertFile;
BAO_CLIENT_KEY = baoDeploy.clientKeyFile;
BAO_CLIENT_CERT = baoDeploy.grafanaOidcClientCertFile;
BAO_CLIENT_KEY = baoDeploy.grafanaOidcClientKeyFile;
}
# Absent means the system trust store, which is what a deployment with a
# real CA wants and what a self-signed one must not be left with.

13
nix/host-modules/swarm-otel.nix
View file

@ -248,7 +248,14 @@ let
# their own optional readers, because a collector with no client identity is
# `haveCollectorSecret = false` above, and that is already a supported,
# merely degraded shape rather than a service with no way in at all.
haveClientIdentity = baoDeploy.clientCertFile != null && baoDeploy.clientKeyFile != null;
#
# 🩸 The collector's OWN leaf, not `clientCertFile` — the hive's, which four
# units used to share. Bao matches a cert-auth role on the CN, so one leaf for
# four readers was ONE principal holding the union of four grants: this unit
# could read every agent credential in the swarm and Grafana's OIDC client
# secret, when what it needs is the one path `storeSecretPath` names.
haveClientIdentity =
baoDeploy.otelOidcClientCertFile != null && baoDeploy.otelOidcClientKeyFile != null;
# Where the publisher on authelia's host leaves this client's secret —
# composed from the same swarm-wide `clientId` the registration in
@ -775,8 +782,8 @@ in
};
environment = {
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
BAO_CLIENT_CERT = baoDeploy.clientCertFile;
BAO_CLIENT_KEY = baoDeploy.clientKeyFile;
BAO_CLIENT_CERT = baoDeploy.otelOidcClientCertFile;
BAO_CLIENT_KEY = baoDeploy.otelOidcClientKeyFile;
}
# Absent means the system trust store, which is what a deployment with a
# real CA wants and what a self-signed one must not be left with.

15
nix/host-modules/swarm.nix
View file

@ -47,7 +47,20 @@ let
deployCfg.bao.controllerCommonName
deployCfg.bao.secretPublisherCommonName
deployCfg.bao.matrixCtlCommonName
];
deployCfg.bao.grafanaOidcCommonName
deployCfg.bao.otelOidcCommonName
]
# The two per-hive readers' subjects, spelled out per hive rather than as the
# prefix. The prefix alone would reserve the wrong string: the role for hive
# `h` accepts `<prefix>-h`, so the name a hive must not BE is that composed
# spelling, and a hive named it would present a leaf that role accepts.
#
# Both spellings are reserved for every declared hive, including the one the
# CN belongs to — a hive cannot be named after its own reader either.
++ lib.concatMap (hive: [
"${deployCfg.bao.matrixTokenCommonNamePrefix}-${hive}"
"${deployCfg.bao.queueAgentCommonNamePrefix}-${hive}"
]) hiveNames;
# Public hostnames of the swarm's own services, in declaration order.
# `serviceDomains` below is this set sorted + deduplicated.

256
nix/module-eval/bao-grants.nix
View file

@ -45,6 +45,18 @@ let
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
deploy.bao.clientCaFile = lib.mkForce null;
};
# The store plus every one of the four readers that used to log in as the
# hive. One fixture rather than four: the claim they are four *separate*
# principals is only testable where all four render at once — that is the
# deployment in which two of them sharing a leaf would be invisible.
baoGrantWithConsumers = hive {
deploy.bao.enable = true;
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
deploy.matrix.enable = true;
deploy.grafana.enable = true;
deploy.swarm-otel.enable = true;
};
cases = [
{
# Reads the rendered unit on the HOST, which is where the write happens:
@ -196,6 +208,250 @@ let
)
&& !(baoGrantHere.containers.swarm-bao.config.systemd.services ? swarm-bao-matrix-ctl-policy);
}
# ── the four readers that used to share the hive's own leaf ──────────────
#
# 🩸 Until this split all four presented `deploy.bao.clientCertFile`, whose
# policy grants read on `swarm/agents/*`, `swarm/hives/<hive>/*` AND
# `swarm/services/*`. Four principals behind one certificate are one
# principal to bao, so the only expressible grant was the union: the unit
# fetching Grafana's OIDC secret could fetch every agent credential in the
# swarm.
#
# Every one of these cases carries the same three negative arms, and they
# are the deliverable rather than decoration — a positive arm alone passes
# just as well when the other two stanzas are still there beside it. The
# arms pin what each principal must NOT reach, so a later widening fails
# here instead of being noticed in a store.
{
name = "the matrix-token reader's grant is one hive's appservice token and nothing else";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-matrix-token-policy.script;
in
lib.hasInfix "path \"secret/data/swarm/hives/h1/matrix/appservice-token\" {" s
&& lib.hasInfix "capabilities = [\"read\"]" s
# The three stanzas the hive's own leaf carried, none of which this
# principal needs: every agent's credential, every service's OIDC
# client, and the rest of its own hive's tree — including the queue
# credential its sibling reader fetches.
&& !(lib.hasInfix "secret/data/swarm/agents" s)
&& !(lib.hasInfix "secret/data/swarm/services" s)
&& !(lib.hasInfix "secret/data/swarm/hives/h1/*" s)
&& !(lib.hasInfix "secret/data/swarm/hives/h1/queue" s)
# A `hives/*` wildcard would serve every hive from one role and let any
# hive read any other's token — the reach this split exists to remove,
# not to create.
&& !(lib.hasInfix "secret/data/swarm/hives/*" s)
# Nothing may rewrite the policy constraining it, for the reason the
# controller's own `hive-*` narrowing above gives.
&& !(lib.hasInfix "sys/policies/acl" s);
}
{
name = "the queue-credential reader's grant is one hive's queue credential and nothing else";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-queue-agent-policy.script;
in
lib.hasInfix "path \"secret/data/swarm/hives/h1/queue/agent\" {" s
&& lib.hasInfix "capabilities = [\"read\"]" s
&& !(lib.hasInfix "secret/data/swarm/agents" s)
&& !(lib.hasInfix "secret/data/swarm/services" s)
&& !(lib.hasInfix "secret/data/swarm/hives/h1/*" s)
&& !(lib.hasInfix "secret/data/swarm/hives/h1/matrix" s)
&& !(lib.hasInfix "secret/data/swarm/hives/*" s)
&& !(lib.hasInfix "sys/policies/acl" s);
}
{
# ⚠️ The client id is the path segment, so the negative arm that matters
# for this one is the OTHER service's: `services/*` would have granted
# both, and the two are separate principals precisely because a
# dashboard is not entitled to a collector's credential.
name = "the Grafana OIDC reader's grant is Grafana's own client secret and nothing else";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-grafana-oidc-policy.script;
in
lib.hasInfix "path \"secret/data/swarm/services/swarm-grafana/oidc/client\" {" s
&& lib.hasInfix "capabilities = [\"read\"]" s
&& !(lib.hasInfix "secret/data/swarm/agents" s)
&& !(lib.hasInfix "secret/data/swarm/hives" s)
&& !(lib.hasInfix "secret/data/swarm/services/*" s)
&& !(lib.hasInfix "swarm-collector" s)
&& !(lib.hasInfix "sys/policies/acl" s);
}
{
# The mirror of the case above, and the arm naming `swarm-grafana` is why
# these are two principals rather than one `services/*` grant shared.
name = "the collector OIDC reader's grant is the collector's own client secret and nothing else";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-otel-oidc-policy.script;
in
lib.hasInfix "path \"secret/data/swarm/services/swarm-collector/oidc/client\" {" s
&& lib.hasInfix "capabilities = [\"read\"]" s
&& !(lib.hasInfix "secret/data/swarm/agents" s)
&& !(lib.hasInfix "secret/data/swarm/hives" s)
&& !(lib.hasInfix "secret/data/swarm/services/*" s)
&& !(lib.hasInfix "swarm-grafana" s)
&& !(lib.hasInfix "sys/policies/acl" s);
}
{
# 🩸 The half that makes the policies above bind: a policy grants only
# through a token that carries it, and a token is minted by a cert-auth
# role matching a CN. Four distinct subjects is the whole mechanism — one
# subject for four readers is one principal however the policies read.
#
# The per-hive subjects carry the hive name because their paths do; the
# two service subjects do not, because an OIDC client is registered once
# per swarm. Pinned so neither shape is tidied into the other.
name = "each of the four readers logs in under a subject of its own";
ok =
let
subjectOf =
unit: role: cn:
let
s = baoGrantHere.systemd.services.${unit}.script;
in
lib.hasInfix "auth/cert/certs/${role}" s
&& lib.hasInfix "allowed_common_names=${cn}" s
&& lib.hasInfix "token_policies=${role}" s
# Outside the `hive-*` namespace the controller may rewrite, for
# the reason the three service principals above state.
&& !(lib.hasInfix "auth/cert/certs/hive-" s);
in
subjectOf "swarm-bao-matrix-token-policy" "swarm-matrix-token-h1" "swarm-bao-matrix-token-h1"
&& subjectOf "swarm-bao-queue-agent-policy" "swarm-queue-agent-h1" "swarm-bao-queue-agent-h1"
&& subjectOf "swarm-bao-grafana-oidc-policy" "swarm-grafana-oidc" "swarm-bao-grafana-oidc"
&& subjectOf "swarm-bao-otel-oidc-policy" "swarm-otel-oidc" "swarm-bao-otel-oidc";
}
{
# 🩸 The consuming side, and the arm that would catch the regression that
# costs the most: a unit repointed back at `deploy.bao.clientCertFile`
# evaluates, deploys and logs in — and silently restores the union grant,
# because bao would again see one principal. Nothing about the policies
# above would look wrong.
#
# Each pair is asserted whole: a certificate with no key authenticates
# nothing, so a half-set pair is a reader that does not render.
name = "each of the four readers presents its own leaf, never the hive's";
ok =
let
b = baoGrantWithConsumers.services.hyperhive.deploy.bao;
hiveLeaf = [
b.clientCertFile
b.clientKeyFile
];
own = [
b.matrixTokenClientCertFile
b.matrixTokenClientKeyFile
b.queueAgentClientCertFile
b.queueAgentClientKeyFile
b.grafanaOidcClientCertFile
b.grafanaOidcClientKeyFile
b.otelOidcClientCertFile
b.otelOidcClientKeyFile
];
envOf = unit: baoGrantWithConsumers.systemd.services.${unit}.environment;
presents =
unit: cert: key:
(envOf unit).BAO_CLIENT_CERT == cert && (envOf unit).BAO_CLIENT_KEY == key;
in
lib.all (p: p != null) own
&& !(lib.any (p: lib.elem p hiveLeaf) own)
&& lib.length (lib.unique own) == lib.length own
&& presents "swarm-bao-matrix-token" b.matrixTokenClientCertFile b.matrixTokenClientKeyFile
&& presents "swarm-bao-queue-agent" b.queueAgentClientCertFile b.queueAgentClientKeyFile
&& presents "swarm-bao-grafana-oidc" b.grafanaOidcClientCertFile b.grafanaOidcClientKeyFile
&& presents "swarm-bao-otel-oidc" b.otelOidcClientCertFile b.otelOidcClientKeyFile;
}
{
# The minting side of the same claim. A role matching a subject nothing
# signs is a reader that cannot log in, so the leaves and the roles have
# to be asserted against each other — and the two per-hive leaves carry
# THIS host's hive name, which is what makes one hive's leaf useless
# against another hive's role.
name = "the PKI unit signs a leaf per reader, each under that reader's own subject";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-pki.script;
in
# The basename and the subject are matched separately: `signLeaf` takes
# them as consecutive arguments across a `\` continuation, so one
# literal spanning both would pin this file's line wrapping rather than
# the pairing it means to.
lib.all (lib.flip lib.hasInfix s) [
"/matrix-token.pem ]"
"swarm-bao-matrix-token-h1 \"\" clientAuth"
"/queue-agent.pem ]"
"swarm-bao-queue-agent-h1 \"\" clientAuth"
"/grafana-oidc.pem ]"
"swarm-bao-grafana-oidc \"\" clientAuth"
"/otel-oidc.pem ]"
"swarm-bao-otel-oidc \"\" clientAuth"
];
}
{
# The absence arm: with no client CA there is no trust anchor, so the
# login roles cannot be written — but the policies they would attach are
# still asserted, exactly as the three service principals above behave in
# this deployment. A unit that vanished here would take the policy with
# it and leave nothing to diagnose.
name = "with no client CA the four readers get policies but no login roles";
ok =
let
units = [
"swarm-bao-matrix-token-policy"
"swarm-bao-queue-agent-policy"
"swarm-bao-grafana-oidc-policy"
"swarm-bao-otel-oidc-policy"
];
scriptOf = unit: baoGrantNoClientCa.systemd.services.${unit}.script;
in
lib.all (
unit:
(baoGrantNoClientCa.systemd.services ? ${unit})
&& lib.hasInfix "bao policy write" (scriptOf unit)
&& !(lib.hasInfix "auth/cert/certs" (scriptOf unit))
) units;
}
{
# Same control the three service principals carry: the write needs a
# client certificate and the host is the side that has one, so a unit
# rendered inside the store's container would have neither an identity
# nor a route. Plus the ordering that makes the mounts exist first.
name = "the four readers' granting units are ordered after the mounts and rendered on the host";
ok =
let
units = [
"swarm-bao-matrix-token-policy"
"swarm-bao-queue-agent-policy"
"swarm-bao-grafana-oidc-policy"
"swarm-bao-otel-oidc-policy"
];
in
lib.all (
unit:
lib.elem "swarm-bao-controller-policy.service" baoGrantHere.systemd.services.${unit}.after
&& !(baoGrantHere.containers.swarm-bao.config.systemd.services ? ${unit})
) units;
}
{
# A store host that has not placed a bootstrap token can write no grant at
# all, so none of the four units may exist — the same claim
# `baoGrantNoStore` makes for the controller's, one file over. Without
# this arm `lib.mkIf haveBootstrapToken` could be dropped from the shared
# builder and every other case here would still pass.
name = "without a bootstrap token none of the four readers' granting units render";
ok =
let
s = baoGrantNoStore.systemd.services;
in
!(s ? swarm-bao-matrix-token-policy)
&& !(s ? swarm-bao-queue-agent-policy)
&& !(s ? swarm-bao-grafana-oidc-policy)
&& !(s ? swarm-bao-otel-oidc-policy);
}
{
# The policy authorising this route lives in another file, and nothing
# else relates the grants to the paths the code actually writes.

10
nix/module-eval/bao-matrix-reader.nix
View file

@ -32,10 +32,20 @@ let
# nothing here mints a leaf and the operator names one placed by hand. The
# deployment this pairing exists to serve, and the one that was previously
# inexpressible — the gate asked whether the store was a neighbour.
#
# 🩸 One leaf PER READER, and this fixture is where that cost is visible: the
# two units below identify themselves to the store separately, so an operator
# placing leaves by hand places one for each rather than one for both. Naming
# only `clientCertFile` here would leave neither unit rendered — which is what
# the arms below would then be asserting about.
baoRemoteReader = hive {
deploy.matrix.enable = true;
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
deploy.bao.matrixTokenClientCertFile = "/etc/pki/bao-matrix-token.pem";
deploy.bao.matrixTokenClientKeyFile = "/etc/pki/bao-matrix-token-key.pem";
deploy.bao.queueAgentClientCertFile = "/etc/pki/bao-queue-agent.pem";
deploy.bao.queueAgentClientKeyFile = "/etc/pki/bao-queue-agent-key.pem";
};
# A homeserver on a hive with NO store identity at all — neither a local

12
nix/module-eval/grafana.nix
View file

@ -41,6 +41,8 @@ let
deploy.authelia.enable = true;
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
deploy.bao.grafanaOidcClientCertFile = "/etc/pki/bao-grafana-oidc.pem";
deploy.bao.grafanaOidcClientKeyFile = "/etc/pki/bao-grafana-oidc-key.pem";
};
# The same UI with the IdP on ANOTHER host and a store leaf placed by hand.
@ -54,6 +56,8 @@ let
swarm.authelia.url = "https://auth.example.invalid";
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
deploy.bao.grafanaOidcClientCertFile = "/etc/pki/bao-grafana-oidc.pem";
deploy.bao.grafanaOidcClientKeyFile = "/etc/pki/bao-grafana-oidc-key.pem";
};
# A Grafana host holding no store identity. This used to be the shape the
@ -148,8 +152,8 @@ let
# mean the two assertions had been collapsed into one conjunction.
name = "a grafana host with no store identity is refused, naming the options to set";
ok =
grafanaRefusedFor grafanaNoIdentity "deploy.bao.clientCertFile"
&& grafanaRefusedFor grafanaNoIdentity "deploy.bao.clientKeyFile"
grafanaRefusedFor grafanaNoIdentity "deploy.bao.grafanaOidcClientCertFile"
&& grafanaRefusedFor grafanaNoIdentity "deploy.bao.grafanaOidcClientKeyFile"
&& !(grafanaRefusedFor grafanaNoIdentity "swarm.authelia.url");
}
{
@ -161,9 +165,9 @@ let
name = "neither grafana refusal fires on a correctly configured host, co-located or not";
ok =
!(grafanaRefusedFor grafanaWithAuthelia "services.hyperhive.swarm.authelia.url")
&& !(grafanaRefusedFor grafanaWithAuthelia "deploy.bao.clientCertFile")
&& !(grafanaRefusedFor grafanaWithAuthelia "deploy.bao.grafanaOidcClientCertFile")
&& !(grafanaRefusedFor grafanaRemoteAuthelia "services.hyperhive.swarm.authelia.url")
&& !(grafanaRefusedFor grafanaRemoteAuthelia "deploy.bao.clientCertFile");
&& !(grafanaRefusedFor grafanaRemoteAuthelia "deploy.bao.grafanaOidcClientCertFile");
}
{
# Same 403-not-a-miss reason as the matrix and queue arms below: the

59
nix/module-eval/name-guards.nix
View file

@ -63,6 +63,35 @@ let
swarm.hives.mintctl.domain = "m.t.local";
};
# The two OIDC-secret readers' subjects, fixed strings like the three above.
hiveNamedAfterGrafanaOidcSubject = hive {
deploy.swarm-otel.enable = false;
deploy.bao.grafanaOidcCommonName = "gfctl";
swarm.hives.gfctl.domain = "g.t.local";
};
hiveNamedAfterOtelOidcSubject = hive {
deploy.swarm-otel.enable = false;
deploy.bao.otelOidcCommonName = "otctl";
swarm.hives.otctl.domain = "o.t.local";
};
# 🩸 A different shape from every fixture above: the matrix-token and
# queue-credential roles are written PER HIVE, so the subject a hive must not
# be is `<prefix>-<some hive's name>` rather than the prefix itself. Reserving
# only the prefix would leave the composed spelling free, and a hive taking it
# would present a leaf the other hive's role accepts — which is a hive reading
# another hive's queue credential, the exact widening the split exists to
# avoid.
#
# Two hives here, not one: the collision is with the OTHER hive's role.
hiveNamedAfterPerHiveReaderSubject = hive {
deploy.swarm-otel.enable = false;
deploy.bao.queueAgentCommonNamePrefix = "qr";
swarm.hives.other.domain = "o.t.local";
swarm.hives.qr-other.domain = "q.t.local";
};
hiveNameWithComposedWord = hive {
deploy.swarm-otel.enable = false;
swarm.hives."h1-agent".domain = "a.t.local";
@ -111,6 +140,36 @@ let
a: !a.assertion && lib.hasInfix "'mintctl'" a.message
) hiveNamedAfterMatrixCtlSubject.assertions;
}
{
# The fourth and fifth, for the reason the case above gives: `certAuthCns`
# is where a role added beside the others registers itself, and nothing
# but a case per element notices when one forgets. These two are the
# subjects of the readers that fetch Grafana's and the collector's OIDC
# client secrets.
name = "a hive named after either OIDC-secret reader's subject is refused too";
ok =
equalityGuardFired hiveNamedAfterGrafanaOidcSubject
&& lib.any (
a: !a.assertion && lib.hasInfix "'gfctl'" a.message
) hiveNamedAfterGrafanaOidcSubject.assertions
&& equalityGuardFired hiveNamedAfterOtelOidcSubject
&& lib.any (
a: !a.assertion && lib.hasInfix "'otctl'" a.message
) hiveNamedAfterOtelOidcSubject.assertions;
}
{
# 🩸 The per-hive half, and the one a prefix-only reservation would miss:
# the role is `<prefix>-<hive>`, so the reserved string has to be composed
# against every declared hive. Here hive `qr-other` collides with the role
# written for hive `other` — a leaf that reads a credential belonging to a
# hive that is not it.
name = "a hive named after another hive's per-hive reader subject is refused";
ok =
equalityGuardFired hiveNamedAfterPerHiveReaderSubject
&& lib.any (
a: !a.assertion && lib.hasInfix "'qr-other'" a.message
) hiveNamedAfterPerHiveReaderSubject.assertions;
}
{
# Without this the case above proves nothing: an arm that fires for every
# roster is not a guard, and `hives` is non-empty in both fixtures.

6
nix/module-eval/swarm-otel-core.nix
View file

@ -32,6 +32,8 @@ let
deploy.authelia.enable = true;
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
deploy.bao.otelOidcClientCertFile = "/etc/pki/bao-otel-oidc.pem";
deploy.bao.otelOidcClientKeyFile = "/etc/pki/bao-otel-oidc-key.pem";
deploy.victoriametrics.enable = false;
deploy.victorialogs.enable = false;
};
@ -44,6 +46,8 @@ let
deploy.authelia.enable = true;
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
deploy.bao.otelOidcClientCertFile = "/etc/pki/bao-otel-oidc.pem";
deploy.bao.otelOidcClientKeyFile = "/etc/pki/bao-otel-oidc-key.pem";
};
# The same collector with the IdP on ANOTHER host and a store leaf placed by
@ -54,6 +58,8 @@ let
swarm.authelia.url = "https://auth.example.invalid";
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
deploy.bao.otelOidcClientCertFile = "/etc/pki/bao-otel-oidc.pem";
deploy.bao.otelOidcClientKeyFile = "/etc/pki/bao-otel-oidc-key.pem";
};
cases = [
{