swarm-bao: give each hive-cert consumer its own bao identity
Four units read one path each out of the store, and all four logged in holding `deploy.bao.clientCertFile` — the hive's own leaf. Bao identifies a principal by the subject of the certificate it presents, so four readers behind one certificate were ONE principal, and the only grant expressible was the union of what the four need: read on `swarm/agents/*`, `swarm/hives/<hive>/*` and `swarm/services/*`. The unit fetching Grafana's OIDC client secret could fetch every agent credential in the swarm; the one fetching this hive's matrix token could fetch Grafana's. Least privilege was not misconfigured here, it was unrepresentable. Each now holds a leaf, a cert-auth role and a policy of its own, and each policy is the single `secret/data/…` path that unit's own script names — spelled to the leaf, not to a prefix, the way matrix-ctl's already is. Following the four exemplars in-tree rather than building a mechanism: `signLeaf` mints the leaves, `swarm-bao.nix` writes the roles from the bootstrap token, the consumers name their own pair. Two of the four are written PER HIVE and two are not, which is the shape of the paths rather than a preference. A matrix appservice token and a queue credential live under `swarm/hives/<name>/` and every hive runs a reader for its own, so one role for all of them would have to be granted `hives/*` — letting one hive read another's, a reach no hive has today. An OIDC client secret lives under `swarm/services/<client-id>/` and a swarm registers each exactly once, so one role each is enough. The per-hive subjects are `<prefix>-<hive>` and swarm.nix reserves every composed spelling as a hive name, so a hive cannot be named into another hive's role. The shared leaf stays: hive-c0re still passes it into its container, the `bao` CLI wrapper still defaults to it, and the three `glue-*-bao-identity.nix` files derive the PKI directory from it. module-eval-bao-grants gains a negative arm per principal — each pins the three stanzas the hive's leaf carried and the two wildcards a later widening would reach for, so a policy that grows fails here rather than in a store. Plus the consuming side: repointing a unit back at the hive's leaf would evaluate, deploy and log in, and silently restore the union. A hive that reads a store on another machine now places one leaf per principal instead of one shared by four. That cost is the point, and docs/swarm/secrets.md lists the pairs.
This commit is contained in:
atlas
mara
parent
ded5b08258
commit
f1445b4c8b
14 changed files with 859 additions and 43 deletions
|
|
@ -84,6 +84,25 @@ in
|
|||
clientCertFile = lib.mkDefault "${pkiDir}/client.pem";
|
||||
clientKeyFile = lib.mkDefault "${pkiDir}/client-key.pem";
|
||||
serverCaFile = lib.mkDefault "${pkiDir}/ca.pem";
|
||||
|
||||
# 🩸 Four readers that used to present `client.pem` above, each now
|
||||
# pointed at a leaf of its own. The leaf is what bao sees, so this pairing
|
||||
# is the whole of what turns "one principal with the union of four grants"
|
||||
# into four principals with one grant each — see ./swarm-bao.nix's
|
||||
# `perHiveReaders` block for the grants themselves.
|
||||
#
|
||||
# Defaulted here rather than in four `glue-<consumer>-bao-identity.nix`
|
||||
# files: those exist where the consumer is a CONTAINER with a
|
||||
# `deploy.<service>.*` namespace of its own to point at. These four are
|
||||
# host units reading the store, which is the pairing this file already is.
|
||||
matrixTokenClientCertFile = lib.mkDefault "${pkiDir}/matrix-token.pem";
|
||||
matrixTokenClientKeyFile = lib.mkDefault "${pkiDir}/matrix-token-key.pem";
|
||||
queueAgentClientCertFile = lib.mkDefault "${pkiDir}/queue-agent.pem";
|
||||
queueAgentClientKeyFile = lib.mkDefault "${pkiDir}/queue-agent-key.pem";
|
||||
grafanaOidcClientCertFile = lib.mkDefault "${pkiDir}/grafana-oidc.pem";
|
||||
grafanaOidcClientKeyFile = lib.mkDefault "${pkiDir}/grafana-oidc-key.pem";
|
||||
otelOidcClientCertFile = lib.mkDefault "${pkiDir}/otel-oidc.pem";
|
||||
otelOidcClientKeyFile = lib.mkDefault "${pkiDir}/otel-oidc-key.pem";
|
||||
};
|
||||
|
||||
# Idempotent on ABSENCE, never on content. Re-issuing the CA invalidates
|
||||
|
|
@ -160,6 +179,35 @@ in
|
|||
# container an identity instead of lending it the hive's.
|
||||
[ -s ${pkiDir}/matrix-ctl.pem ] || ${signLeaf} ${pkiDir} matrix-ctl \
|
||||
${lib.escapeShellArg deployCfg.bao.matrixCtlCommonName} "" clientAuth
|
||||
|
||||
# 🩸 The four readers that used to present `client.pem` above. Each
|
||||
# carries its own subject, which is the entire mechanism: bao matches a
|
||||
# cert-auth role on the CN, so four units sharing one leaf were one
|
||||
# principal holding the union of four grants, and the union included
|
||||
# every agent credential in the swarm.
|
||||
#
|
||||
# ⚠️ The first two carry THIS hive's name in the subject, unlike the
|
||||
# three service leaves above. Their grants name one hive's path, because
|
||||
# a matrix token and a queue credential live under `swarm/hives/<name>/`
|
||||
# and every hive runs a reader for its own; ./swarm-bao.nix writes one
|
||||
# role per hive in the swarm directory to match. The two OIDC readers
|
||||
# need no such segment — a client is registered once per swarm.
|
||||
#
|
||||
# Minted whether or not the consumer runs here, for the reason the three
|
||||
# leaves above give: on a hive that does not run the store this is the
|
||||
# file an operator copies, and a leaf that only appears where its
|
||||
# consumer does is one nobody can copy from anywhere.
|
||||
[ -s ${pkiDir}/matrix-token.pem ] || ${signLeaf} ${pkiDir} matrix-token \
|
||||
${lib.escapeShellArg "${deployCfg.bao.matrixTokenCommonNamePrefix}-${clientCn}"} "" clientAuth
|
||||
|
||||
[ -s ${pkiDir}/queue-agent.pem ] || ${signLeaf} ${pkiDir} queue-agent \
|
||||
${lib.escapeShellArg "${deployCfg.bao.queueAgentCommonNamePrefix}-${clientCn}"} "" clientAuth
|
||||
|
||||
[ -s ${pkiDir}/grafana-oidc.pem ] || ${signLeaf} ${pkiDir} grafana-oidc \
|
||||
${lib.escapeShellArg deployCfg.bao.grafanaOidcCommonName} "" clientAuth
|
||||
|
||||
[ -s ${pkiDir}/otel-oidc.pem ] || ${signLeaf} ${pkiDir} otel-oidc \
|
||||
${lib.escapeShellArg deployCfg.bao.otelOidcCommonName} "" clientAuth
|
||||
'';
|
||||
};
|
||||
};
|
||||
|
|
|
|||
|
|
@ -38,7 +38,16 @@ let
|
|||
# What decides whether this unit exists at all. A reader is defined by holding
|
||||
# a certificate the store accepts, and that is true on the store's own host
|
||||
# and on a hive three networks away for exactly the same reason.
|
||||
haveClientIdentity = baoDeploy.clientCertFile != null && baoDeploy.clientKeyFile != null;
|
||||
#
|
||||
# 🩸 This principal's OWN leaf, not `clientCertFile` — the hive's, which four
|
||||
# units used to share. Bao matches a cert-auth role on the CN, so one leaf for
|
||||
# four readers was ONE principal holding the union of four grants: read on
|
||||
# `swarm/agents/*` AND `swarm/hives/<hive>/*` AND `swarm/services/*`, when
|
||||
# this unit reads one appservice token and nothing else. Its own leaf carries
|
||||
# `<matrixTokenCommonNamePrefix>-<hive>` and its role grants the single path
|
||||
# below.
|
||||
haveClientIdentity =
|
||||
baoDeploy.matrixTokenClientCertFile != null && baoDeploy.matrixTokenClientKeyFile != null;
|
||||
|
||||
# Where the token lives in the store. A path, not a convention to guess at:
|
||||
# whoever writes it and whoever reads it must agree, and the agreement
|
||||
|
|
@ -115,8 +124,8 @@ in
|
|||
};
|
||||
environment = {
|
||||
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
|
||||
BAO_CLIENT_CERT = baoDeploy.clientCertFile;
|
||||
BAO_CLIENT_KEY = baoDeploy.clientKeyFile;
|
||||
BAO_CLIENT_CERT = baoDeploy.matrixTokenClientCertFile;
|
||||
BAO_CLIENT_KEY = baoDeploy.matrixTokenClientKeyFile;
|
||||
}
|
||||
# Absent means the system trust store, which is what a deployment with a
|
||||
# real CA wants and what a self-signed one must not be left with.
|
||||
|
|
|
|||
|
|
@ -40,7 +40,16 @@ let
|
|||
# What decides whether this unit exists at all. A reader is defined by holding
|
||||
# a certificate the store accepts, and that is true on the store's own host
|
||||
# and on a hive three networks away for exactly the same reason.
|
||||
haveClientIdentity = baoDeploy.clientCertFile != null && baoDeploy.clientKeyFile != null;
|
||||
#
|
||||
# 🩸 This principal's OWN leaf, not `clientCertFile` — the hive's, which four
|
||||
# units used to share. Bao matches a cert-auth role on the CN, so one leaf for
|
||||
# four readers was ONE principal holding the union of four grants: read on
|
||||
# `swarm/agents/*` AND `swarm/hives/<hive>/*` AND `swarm/services/*`, when
|
||||
# this unit reads one queue credential and nothing else. Its own leaf carries
|
||||
# `<queueAgentCommonNamePrefix>-<hive>` and its role grants the single path
|
||||
# below — still this hive's own, so the narrowing costs no reach.
|
||||
haveClientIdentity =
|
||||
baoDeploy.queueAgentClientCertFile != null && baoDeploy.queueAgentClientKeyFile != null;
|
||||
|
||||
credentialDir = toString deployCfg.hive-controller.queue.agentCredentialDir;
|
||||
secretFile = "${credentialDir}/secret";
|
||||
|
|
@ -141,8 +150,8 @@ in
|
|||
};
|
||||
environment = {
|
||||
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
|
||||
BAO_CLIENT_CERT = baoDeploy.clientCertFile;
|
||||
BAO_CLIENT_KEY = baoDeploy.clientKeyFile;
|
||||
BAO_CLIENT_CERT = baoDeploy.queueAgentClientCertFile;
|
||||
BAO_CLIENT_KEY = baoDeploy.queueAgentClientKeyFile;
|
||||
}
|
||||
# Absent means the system trust store, which is what a deployment with a
|
||||
# real CA wants and what a self-signed one must not be left with.
|
||||
|
|
|
|||
|
|
@ -340,6 +340,156 @@ let
|
|||
}
|
||||
'';
|
||||
|
||||
# ── the four principals that used to share the hive's own leaf ─────────────
|
||||
#
|
||||
# 🩸 Each of the four reads exactly ONE path in the store, and until this
|
||||
# split each did it holding `deploy.bao.clientCertFile` — the hive's own leaf,
|
||||
# whose policy (`swarm-secret-client`'s `policy::render`) grants read on
|
||||
# `swarm/agents/*`, `swarm/hives/<hive>/*` AND `swarm/services/*`. Four
|
||||
# principals presenting one certificate are ONE principal to bao, so the union
|
||||
# of what the four need was the only grant expressible: the unit that fetches
|
||||
# Grafana's OIDC client secret could fetch every agent credential in the
|
||||
# swarm, and the unit that fetches this hive's matrix token could fetch
|
||||
# Grafana's. Least privilege was not misconfigured, it was unrepresentable.
|
||||
#
|
||||
# Each now gets its own leaf, its own cert-auth role and a policy holding the
|
||||
# one `secret/data/…` path that principal's script actually names — spelled to
|
||||
# the LEAF rather than to a prefix, for the reason `matrixCtlPolicyText` above
|
||||
# gives for doing the same.
|
||||
#
|
||||
# ⚠️ Two of the four are PER HIVE and two are not, and that asymmetry is the
|
||||
# shape of the paths rather than a preference. A matrix appservice token and a
|
||||
# queue credential live under `swarm/hives/<name>/`, and every hive runs a
|
||||
# reader for its own; an OIDC client secret lives under
|
||||
# `swarm/services/<client-id>/` and a swarm registers each of those exactly
|
||||
# once. A single role for a per-hive reader would have to name `hives/*` to
|
||||
# serve every hive — which would let one hive read another's matrix token, a
|
||||
# reach no hive has today and one this change exists to remove rather than
|
||||
# create.
|
||||
readStanza = path: ''
|
||||
path "${path}" {
|
||||
capabilities = ["read"]
|
||||
}
|
||||
'';
|
||||
|
||||
# One reader per hive in the swarm directory. Written from the directory
|
||||
# rather than from this host's own name because the policy is written where
|
||||
# the STORE is and the reader runs where its hive is — the same split
|
||||
# `matrixCtlHiveName` above exists to paper over, answered here by naming
|
||||
# every hive instead of asking the operator which one.
|
||||
perHiveReaders =
|
||||
{ rolePrefix, cnPrefix, ... }@spec:
|
||||
lib.mapAttrsToList (hiveName: _: {
|
||||
name = "${rolePrefix}-${hiveName}";
|
||||
cn = "${cnPrefix}-${hiveName}";
|
||||
policyText = readStanza (spec.path hiveName);
|
||||
}) hyperhiveCfg.swarm.hives;
|
||||
|
||||
matrixTokenReaders = perHiveReaders {
|
||||
rolePrefix = "swarm-matrix-token";
|
||||
cnPrefix = baoDeploy.matrixTokenCommonNamePrefix;
|
||||
path = h: "${credentialMountPath}/data/swarm/hives/${h}/matrix/appservice-token";
|
||||
};
|
||||
|
||||
queueAgentReaders = perHiveReaders {
|
||||
rolePrefix = "swarm-queue-agent";
|
||||
cnPrefix = baoDeploy.queueAgentCommonNamePrefix;
|
||||
path = h: "${credentialMountPath}/data/swarm/hives/${h}/queue/agent";
|
||||
};
|
||||
|
||||
# Singletons, so a list of one rather than a second shape: the granting unit
|
||||
# below takes a list either way and the four units stay identical apart from
|
||||
# the objects they write.
|
||||
grafanaOidcReaders = [
|
||||
{
|
||||
name = "swarm-grafana-oidc";
|
||||
cn = baoDeploy.grafanaOidcCommonName;
|
||||
policyText = readStanza "${credentialMountPath}/data/swarm/services/${hyperhiveCfg.swarm.grafana.oidc.clientId}/oidc/client";
|
||||
}
|
||||
];
|
||||
|
||||
otelOidcReaders = [
|
||||
{
|
||||
name = "swarm-otel-oidc";
|
||||
cn = baoDeploy.otelOidcCommonName;
|
||||
policyText = readStanza "${credentialMountPath}/data/swarm/services/${hyperhiveCfg.swarm.otel.clientId}/oidc/client";
|
||||
}
|
||||
];
|
||||
|
||||
# The role name IS the policy name, as for the three service principals
|
||||
# above: the role attaches the policy by spelling it identically, and one
|
||||
# string for both objects removes the way they drift apart.
|
||||
readerPolicyWrite = obj: ''
|
||||
printf '%s' ${lib.escapeShellArg obj.policyText} |
|
||||
bao policy write ${lib.escapeShellArg obj.name} -
|
||||
'';
|
||||
|
||||
# ⚠️ Names are `swarm-<principal>[-<hive>]`, outside the `hive-*` namespace the
|
||||
# controller may rewrite, for the reason the three service principals above
|
||||
# state. A per-hive CN cannot collide with a hive's own leaf either — a hive's
|
||||
# CN is its bare name and ../reserved-hive-fragments.nix forbids the substring
|
||||
# `swarm` in one — and ./swarm.nix reserves the concrete per-hive spellings
|
||||
# anyway, because the prefixes are operator-settable and an operator may spell
|
||||
# one without that substring.
|
||||
readerRoleWrite = obj: ''
|
||||
bao write auth/cert/certs/${lib.escapeShellArg obj.name} \
|
||||
certificate=@${tlsDir}/client-ca.pem \
|
||||
allowed_common_names=${lib.escapeShellArg obj.cn} \
|
||||
token_policies=${lib.escapeShellArg obj.name} \
|
||||
display_name=${lib.escapeShellArg obj.cn}
|
||||
'';
|
||||
|
||||
# One unit per PRINCIPAL, not one unit for the four: these names are
|
||||
# operator-facing strings, the same reason `swarm-bao-secret-publisher-policy`
|
||||
# above gives for not being more script inside the controller's unit. A hive
|
||||
# whose matrix token never arrives is diagnosed by the state of the unit named
|
||||
# after it.
|
||||
#
|
||||
# `after` and not `requires`, for the reason the publisher's unit states: the
|
||||
# controller's unit creates the KV and cert-auth mounts this one writes into,
|
||||
# but a failed oneshot still counts as finished, so ordering plus this unit's
|
||||
# own retry is what converges.
|
||||
#
|
||||
# The role write is inside the client-CA branch and the policy write is not,
|
||||
# exactly as the three above: with no CA there is no trust anchor for a login
|
||||
# role, but the policy it would attach is still worth asserting.
|
||||
readerPolicyUnit =
|
||||
description: objects:
|
||||
lib.mkIf haveBootstrapToken {
|
||||
inherit description;
|
||||
after = [
|
||||
"container@${cfg.machine}.service"
|
||||
"swarm-bao-controller-policy.service"
|
||||
];
|
||||
wantedBy = [ "multi-user.target" ];
|
||||
path = [
|
||||
baoCli
|
||||
pkgs.coreutils
|
||||
];
|
||||
unitConfig.ConditionPathExists = baoDeploy.bootstrapTokenFile;
|
||||
# Same unseal wait as its siblings above, for the reason stated there:
|
||||
# under `seal = "shamir"` a human unseals by hand.
|
||||
startLimitBurst = 2880;
|
||||
startLimitIntervalSec = 90000;
|
||||
serviceConfig = {
|
||||
Type = "oneshot";
|
||||
RemainAfterExit = true;
|
||||
Restart = "on-failure";
|
||||
RestartSec = 30;
|
||||
};
|
||||
script = ''
|
||||
set -euo pipefail
|
||||
|
||||
BAO_TOKEN="$(cat ${lib.escapeShellArg baoDeploy.bootstrapTokenFile})"
|
||||
export BAO_TOKEN
|
||||
|
||||
''
|
||||
+ lib.concatMapStrings readerPolicyWrite objects
|
||||
+ lib.optionalString (baoDeploy.clientCaFile != null) (
|
||||
"\n" + lib.concatMapStrings readerRoleWrite objects
|
||||
);
|
||||
};
|
||||
|
||||
# Every listener serves the same identity: they differ in which address
|
||||
# they answer on, not in who they are. Client verification is separate and
|
||||
# optional — a store with no `clientCaFile` still serves TLS, it just does
|
||||
|
|
@ -846,6 +996,185 @@ in
|
|||
'';
|
||||
};
|
||||
|
||||
matrixTokenCommonNamePrefix = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
default = "swarm-bao-matrix-token";
|
||||
example = "swarm-bao-matrix-token.svc";
|
||||
description = ''
|
||||
Prefix of the subject the store's per-hive matrix-token roles accept.
|
||||
The role for hive `h` accepts `<prefix>-h` and nothing else, and grants
|
||||
read on exactly `swarm/hives/h/matrix/appservice-token`.
|
||||
|
||||
A **prefix** rather than one subject because every hive runs its own
|
||||
reader and the path it reads carries that hive's name. One subject for
|
||||
all of them would need a `hives/*` grant, which would let one hive read
|
||||
another hive's appservice token — a reach no hive has today.
|
||||
|
||||
⚠️ Reserved as a hive name by ./swarm.nix, per hive, for the reason its
|
||||
fixed-subject siblings are: cert auth trusts the CA and
|
||||
`allowed_common_names` is the whole narrowing, so a hive whose own leaf
|
||||
carried one of these spellings would receive that grant.
|
||||
'';
|
||||
};
|
||||
|
||||
queueAgentCommonNamePrefix = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
default = "swarm-bao-queue-agent";
|
||||
example = "swarm-bao-queue-agent.svc";
|
||||
description = ''
|
||||
Prefix of the subject the store's per-hive queue-credential roles
|
||||
accept. The role for hive `h` accepts `<prefix>-h` and grants read on
|
||||
exactly `swarm/hives/h/queue/agent` — the client credential that hive's
|
||||
agent containers authenticate to the swarm queue with.
|
||||
|
||||
Per hive for the same reason
|
||||
{option}`services.hyperhive.deploy.bao.matrixTokenCommonNamePrefix` is,
|
||||
and more sharply: every hive in the swarm runs this reader, so a single
|
||||
subject would have to be granted `hives/*`.
|
||||
'';
|
||||
};
|
||||
|
||||
grafanaOidcCommonName = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
default = "swarm-bao-grafana-oidc";
|
||||
example = "swarm-bao-grafana-oidc.svc";
|
||||
description = ''
|
||||
Subject the store's `swarm-grafana-oidc` cert-auth role accepts — the
|
||||
identity the unit that fetches Grafana's OIDC client secret presents.
|
||||
|
||||
Its grant is one path,
|
||||
`swarm/services/<grafana client id>/oidc/client`, and read only. Not
|
||||
per hive, unlike the two prefixes above: an OIDC client is registered
|
||||
once per swarm, so the path names the service and never a hive.
|
||||
|
||||
⚠️ Same collision as its siblings, and the same answer: ./swarm.nix
|
||||
feeds this value into the guard on
|
||||
{option}`services.hyperhive.swarm.hives`.
|
||||
'';
|
||||
};
|
||||
|
||||
otelOidcCommonName = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
default = "swarm-bao-otel-oidc";
|
||||
example = "swarm-bao-otel-oidc.svc";
|
||||
description = ''
|
||||
Subject the store's `swarm-otel-oidc` cert-auth role accepts — the
|
||||
identity the unit that fetches the collector's OIDC client secret
|
||||
presents. Its grant is one path,
|
||||
`swarm/services/<collector client id>/oidc/client`, and read only.
|
||||
|
||||
A **fourth** identity rather than reuse of
|
||||
{option}`services.hyperhive.deploy.bao.grafanaOidcCommonName`: the two
|
||||
read different services' client secrets, and a collector is not
|
||||
entitled to Grafana's.
|
||||
'';
|
||||
};
|
||||
|
||||
matrixTokenClientCertFile = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
example = "/var/lib/swarm-bao-pki/matrix-token.pem";
|
||||
description = ''
|
||||
Certificate the unit that fetches this hive's matrix appservice token
|
||||
presents to the store. Its subject must be
|
||||
`<matrixTokenCommonNamePrefix>-<this hive's name>`; cert auth matches on
|
||||
the CN and the role accepts nothing else.
|
||||
|
||||
⚠️ **Not** {option}`services.hyperhive.deploy.bao.clientCertFile`.
|
||||
Pointing this at the hive's own leaf would evaluate, deploy and log in —
|
||||
and hand a unit that reads one token a credential that reads every
|
||||
secret in the store. That is what this option exists to end.
|
||||
|
||||
No default here: ./glue-bao-tls.nix points it at the leaf it mints,
|
||||
wherever this host mints its own PKI. A deployment that reads a store on
|
||||
another host names it by hand, alongside the leaf itself — one file per
|
||||
principal placed out of band instead of one shared by four.
|
||||
'';
|
||||
};
|
||||
|
||||
matrixTokenClientKeyFile = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
example = "/var/lib/swarm-bao-pki/matrix-token-key.pem";
|
||||
description = ''
|
||||
Private key for
|
||||
{option}`services.hyperhive.deploy.bao.matrixTokenClientCertFile`. Both
|
||||
or neither — a certificate with no key authenticates nothing.
|
||||
'';
|
||||
};
|
||||
|
||||
queueAgentClientCertFile = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
example = "/var/lib/swarm-bao-pki/queue-agent.pem";
|
||||
description = ''
|
||||
Certificate the unit that fetches this hive's queue client credential
|
||||
presents to the store. Its subject must be
|
||||
`<queueAgentCommonNamePrefix>-<this hive's name>`.
|
||||
|
||||
⚠️ Not the hive's own leaf, for the reason
|
||||
{option}`services.hyperhive.deploy.bao.matrixTokenClientCertFile` gives.
|
||||
'';
|
||||
};
|
||||
|
||||
queueAgentClientKeyFile = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
example = "/var/lib/swarm-bao-pki/queue-agent-key.pem";
|
||||
description = ''
|
||||
Private key for
|
||||
{option}`services.hyperhive.deploy.bao.queueAgentClientCertFile`.
|
||||
'';
|
||||
};
|
||||
|
||||
grafanaOidcClientCertFile = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
example = "/var/lib/swarm-bao-pki/grafana-oidc.pem";
|
||||
description = ''
|
||||
Certificate the unit that fetches Grafana's OIDC client secret presents
|
||||
to the store. Its subject must be
|
||||
{option}`services.hyperhive.deploy.bao.grafanaOidcCommonName`.
|
||||
|
||||
⚠️ Not the hive's own leaf, for the reason
|
||||
{option}`services.hyperhive.deploy.bao.matrixTokenClientCertFile` gives.
|
||||
'';
|
||||
};
|
||||
|
||||
grafanaOidcClientKeyFile = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
example = "/var/lib/swarm-bao-pki/grafana-oidc-key.pem";
|
||||
description = ''
|
||||
Private key for
|
||||
{option}`services.hyperhive.deploy.bao.grafanaOidcClientCertFile`.
|
||||
'';
|
||||
};
|
||||
|
||||
otelOidcClientCertFile = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
example = "/var/lib/swarm-bao-pki/otel-oidc.pem";
|
||||
description = ''
|
||||
Certificate the unit that fetches the collector's OIDC client secret
|
||||
presents to the store. Its subject must be
|
||||
{option}`services.hyperhive.deploy.bao.otelOidcCommonName`.
|
||||
|
||||
⚠️ Not the hive's own leaf, for the reason
|
||||
{option}`services.hyperhive.deploy.bao.matrixTokenClientCertFile` gives.
|
||||
'';
|
||||
};
|
||||
|
||||
otelOidcClientKeyFile = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
example = "/var/lib/swarm-bao-pki/otel-oidc-key.pem";
|
||||
description = ''
|
||||
Private key for
|
||||
{option}`services.hyperhive.deploy.bao.otelOidcClientCertFile`.
|
||||
'';
|
||||
};
|
||||
|
||||
serverCaFile = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
|
|
@ -1593,6 +1922,19 @@ in
|
|||
'';
|
||||
};
|
||||
|
||||
# 🩸 The four principals that used to log in as the hive. See the block
|
||||
# around `perHiveReaders` for what each grant is and why the first two are
|
||||
# per hive; the four units are identical apart from the objects they
|
||||
# write, so the shape lives in `readerPolicyUnit` and the difference is
|
||||
# visible here in one line each.
|
||||
systemd.services.swarm-bao-matrix-token-policy = readerPolicyUnit "write the per-hive matrix-token bao policies and cert-auth roles" matrixTokenReaders;
|
||||
|
||||
systemd.services.swarm-bao-queue-agent-policy = readerPolicyUnit "write the per-hive queue-credential bao policies and cert-auth roles" queueAgentReaders;
|
||||
|
||||
systemd.services.swarm-bao-grafana-oidc-policy = readerPolicyUnit "write Grafana's OIDC-secret-reader bao policy and cert-auth role" grafanaOidcReaders;
|
||||
|
||||
systemd.services.swarm-bao-otel-oidc-policy = readerPolicyUnit "write the collector's OIDC-secret-reader bao policy and cert-auth role" otelOidcReaders;
|
||||
|
||||
# The CA bind source is written at runtime by a host unit, so the
|
||||
# container has to start after it — otherwise nspawn sets up a mount
|
||||
# over a file that does not exist yet.
|
||||
|
|
|
|||
|
|
@ -119,7 +119,14 @@ let
|
|||
# other secret is read with it. A Grafana host without it has not been given
|
||||
# its identity yet, which is a thing to say out loud rather than to route
|
||||
# around by reaching into authelia's tree whenever it happens to be local.
|
||||
haveClientIdentity = baoDeploy.clientCertFile != null && baoDeploy.clientKeyFile != null;
|
||||
#
|
||||
# 🩸 Grafana's OWN leaf, not `clientCertFile` — the hive's, which four units
|
||||
# used to share. Bao matches a cert-auth role on the CN, so one leaf for four
|
||||
# readers was ONE principal holding the union of four grants: this unit could
|
||||
# read every agent credential in the swarm and every other service's OIDC
|
||||
# client secret, when what it needs is the one path `storeSecretPath` names.
|
||||
haveClientIdentity =
|
||||
baoDeploy.grafanaOidcClientCertFile != null && baoDeploy.grafanaOidcClientKeyFile != null;
|
||||
|
||||
autheliaUrl = toString hyperhiveCfg.swarm.authelia.url;
|
||||
|
||||
|
|
@ -451,15 +458,20 @@ in
|
|||
services.hyperhive.deploy.grafana.enable requires this host to hold a
|
||||
swarm-secret-store client identity: set both
|
||||
|
||||
services.hyperhive.deploy.bao.clientCertFile
|
||||
services.hyperhive.deploy.bao.clientKeyFile
|
||||
services.hyperhive.deploy.bao.grafanaOidcClientCertFile
|
||||
services.hyperhive.deploy.bao.grafanaOidcClientKeyFile
|
||||
|
||||
Grafana's OIDC client secret is minted by authelia and read out of
|
||||
the store, on every host that runs Grafana — including the host that
|
||||
runs authelia. That is one delivery route rather than two, and it is
|
||||
what the store is for: this certificate is the single credential
|
||||
placed out of band, and every other secret comes from the store with
|
||||
it.
|
||||
what the store is for: a certificate is the credential placed out of
|
||||
band, and every other secret comes from the store with it.
|
||||
|
||||
⚠️ Grafana's OWN leaf, not deploy.bao.clientCertFile. That one is the
|
||||
hive's, and its grant reads every secret in the store; this role
|
||||
reads the one path Grafana's client secret lives at. Pointing this
|
||||
option at the hive's leaf would evaluate, deploy and log in — and
|
||||
undo the split.
|
||||
|
||||
On a hive that runs the store, glue-bao-tls.nix supplies both as
|
||||
defaults and there is nothing to do. Elsewhere the leaf is issued
|
||||
|
|
@ -610,8 +622,8 @@ in
|
|||
};
|
||||
environment = {
|
||||
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
|
||||
BAO_CLIENT_CERT = baoDeploy.clientCertFile;
|
||||
BAO_CLIENT_KEY = baoDeploy.clientKeyFile;
|
||||
BAO_CLIENT_CERT = baoDeploy.grafanaOidcClientCertFile;
|
||||
BAO_CLIENT_KEY = baoDeploy.grafanaOidcClientKeyFile;
|
||||
}
|
||||
# Absent means the system trust store, which is what a deployment with a
|
||||
# real CA wants and what a self-signed one must not be left with.
|
||||
|
|
|
|||
|
|
@ -248,7 +248,14 @@ let
|
|||
# their own optional readers, because a collector with no client identity is
|
||||
# `haveCollectorSecret = false` above, and that is already a supported,
|
||||
# merely degraded shape rather than a service with no way in at all.
|
||||
haveClientIdentity = baoDeploy.clientCertFile != null && baoDeploy.clientKeyFile != null;
|
||||
#
|
||||
# 🩸 The collector's OWN leaf, not `clientCertFile` — the hive's, which four
|
||||
# units used to share. Bao matches a cert-auth role on the CN, so one leaf for
|
||||
# four readers was ONE principal holding the union of four grants: this unit
|
||||
# could read every agent credential in the swarm and Grafana's OIDC client
|
||||
# secret, when what it needs is the one path `storeSecretPath` names.
|
||||
haveClientIdentity =
|
||||
baoDeploy.otelOidcClientCertFile != null && baoDeploy.otelOidcClientKeyFile != null;
|
||||
|
||||
# Where the publisher on authelia's host leaves this client's secret —
|
||||
# composed from the same swarm-wide `clientId` the registration in
|
||||
|
|
@ -775,8 +782,8 @@ in
|
|||
};
|
||||
environment = {
|
||||
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
|
||||
BAO_CLIENT_CERT = baoDeploy.clientCertFile;
|
||||
BAO_CLIENT_KEY = baoDeploy.clientKeyFile;
|
||||
BAO_CLIENT_CERT = baoDeploy.otelOidcClientCertFile;
|
||||
BAO_CLIENT_KEY = baoDeploy.otelOidcClientKeyFile;
|
||||
}
|
||||
# Absent means the system trust store, which is what a deployment with a
|
||||
# real CA wants and what a self-signed one must not be left with.
|
||||
|
|
|
|||
|
|
@ -47,7 +47,20 @@ let
|
|||
deployCfg.bao.controllerCommonName
|
||||
deployCfg.bao.secretPublisherCommonName
|
||||
deployCfg.bao.matrixCtlCommonName
|
||||
];
|
||||
deployCfg.bao.grafanaOidcCommonName
|
||||
deployCfg.bao.otelOidcCommonName
|
||||
]
|
||||
# The two per-hive readers' subjects, spelled out per hive rather than as the
|
||||
# prefix. The prefix alone would reserve the wrong string: the role for hive
|
||||
# `h` accepts `<prefix>-h`, so the name a hive must not BE is that composed
|
||||
# spelling, and a hive named it would present a leaf that role accepts.
|
||||
#
|
||||
# Both spellings are reserved for every declared hive, including the one the
|
||||
# CN belongs to — a hive cannot be named after its own reader either.
|
||||
++ lib.concatMap (hive: [
|
||||
"${deployCfg.bao.matrixTokenCommonNamePrefix}-${hive}"
|
||||
"${deployCfg.bao.queueAgentCommonNamePrefix}-${hive}"
|
||||
]) hiveNames;
|
||||
|
||||
# Public hostnames of the swarm's own services, in declaration order.
|
||||
# `serviceDomains` below is this set sorted + deduplicated.
|
||||
|
|
|
|||
Loading…
Reference in a new issue