swarm-bao: write every swarm-* grant as a bao granter, not with a 24h token
Every unit that writes a bao policy or cert-auth role ran only while the operator-placed bootstrap token existed, and skipped silently otherwise. The token lives 24h, so on any real swarm a PR adding or changing a grant deployed with its unit skipped, and each one needed a manual token refresh (plus a root `bao policy write` when it added a path). A `bao-granter` principal now writes them. Its leaf is minted by swarm-bao-pki on the store host (0600 root, never copied off it), and its policy covers `swarm-*` policies, `swarm-*` cert-auth roles and `pki/roles/swarm-*` by glob, plus the mount and services-root paths the controller's unit already used. All ten granting units (controller, secret-publisher, matrix-ctl, matrix-token, queue-agent, grafana-oidc, otel-oidc, forwarder-oidc, services-issuer, nats-tls) log in with it instead of reading the token. They keep the 2880 x 30s retry, now require swarm-bao-pki, and when the store refuses the granter they fail and print the one-time step instead of skipping. swarm-bao-granter-role is the one unit left on the token. It enables the auth mounts (moved out of the controller's unit) and writes the granter's own policy and role. The bootstrap policy is renamed `bao-bootstrap` and shrinks to those five stanzas; it is shipped at /etc/hyperhive/bao-bootstrap-policy.hcl. The old name `swarm-bootstrap` matched the granter's own `swarm-*` glob. The granter's CN joins certAuthCns, so no hive can be named into its role. An assertion keeps both pki role names under `swarm-`. With no client CA the granting units no longer render, and a warning says so. module-eval pins the granter's policy stanza by stanza, what it cannot reach, that every call a granting unit makes is granted, and that only swarm-bao-granter-role reads the token. Refs #4704
This commit is contained in:
atlas
mara
parent
19cc1b12e2
commit
e9cec0da21
12 changed files with 898 additions and 443 deletions
|
|
@ -22,7 +22,7 @@ let
|
|||
;
|
||||
|
||||
# The store, plus a placed bootstrap token: the only shape in which the
|
||||
# swarm's first grant can be written at all.
|
||||
# granter's own role can be written at all.
|
||||
baoGrantHere = hive {
|
||||
deploy.bao.enable = true;
|
||||
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
|
||||
|
|
@ -36,10 +36,31 @@ let
|
|||
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
|
||||
};
|
||||
|
||||
# The store with no bootstrap token: the steady state once the granter is set
|
||||
# up, and the state of a store host that has never named one.
|
||||
baoGranterNoToken = hive {
|
||||
deploy.bao.enable = true;
|
||||
};
|
||||
|
||||
# The store with the granter's pair taken away: the deployment that writes
|
||||
# its grants some other way.
|
||||
baoGranterOptOut = hive {
|
||||
deploy.bao.enable = true;
|
||||
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
|
||||
deploy.bao.granterClientCertFile = lib.mkForce null;
|
||||
deploy.bao.granterClientKeyFile = lib.mkForce null;
|
||||
};
|
||||
|
||||
# A pki role the granter's `roles/swarm-*` does not reach.
|
||||
baoGranterOddPkiRole = hive {
|
||||
deploy.bao.enable = true;
|
||||
deploy.bao.natsPkiRoleName = "queue";
|
||||
};
|
||||
|
||||
# The store and the token, with no CA to trust. `mkForce` because the PKI
|
||||
# glue supplies one by default here — this is the deployment that brings its
|
||||
# own certificates and has not named the authority yet, and it separates
|
||||
# "the grant unit runs" from "cert auth can be set up".
|
||||
# own certificates and has not named the authority yet, in which nothing can
|
||||
# log in as the granter.
|
||||
baoGrantNoClientCa = hive {
|
||||
deploy.bao.enable = true;
|
||||
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
|
||||
|
|
@ -102,38 +123,71 @@ let
|
|||
"swarm-bao-otel-oidc"
|
||||
];
|
||||
|
||||
# What the bootstrap token may do, read from the file the operator writes it
|
||||
# from (../../docs/getting-started/setup.md points there), against what the
|
||||
# units holding that token actually call. The units are found by the token
|
||||
# path in their script rather than by name, so a new one is checked without
|
||||
# anyone listing it here.
|
||||
# Two credentials write grants, and each is checked against what the units
|
||||
# holding it actually call. The bootstrap token's policy is read from the
|
||||
# file the operator writes it from (../../docs/getting-started/setup.md
|
||||
# points there); the granter's from the unit that writes it. Units are found
|
||||
# by the credential they read rather than by name, so a new one is checked
|
||||
# without anyone listing it here.
|
||||
bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
|
||||
|
||||
# The READ, not the path: every granting unit prints the path in the
|
||||
# one-time step it shows when the granter is refused.
|
||||
bootstrapUnits = lib.filterAttrs (
|
||||
_: u: lib.hasInfix bootstrapTokenFile u.script
|
||||
_: u: lib.hasInfix "cat ${lib.escapeShellArg bootstrapTokenFile}" u.script
|
||||
) baoGrantWithConsumers.systemd.services;
|
||||
|
||||
# The pair ./glue-bao-tls.nix defaults on a store host.
|
||||
granterCertFile = "/var/lib/swarm-bao-pki/granter.pem";
|
||||
granterKeyFile = "/var/lib/swarm-bao-pki/granter-key.pem";
|
||||
|
||||
granterUnits = lib.filterAttrs (
|
||||
_: u: (u.environment.BAO_CLIENT_CERT or null) == granterCertFile
|
||||
) baoGrantWithConsumers.systemd.services;
|
||||
|
||||
# The ten units that write a `swarm-*` grant, by name, for the discovery
|
||||
# control below.
|
||||
grantingUnitNames = [
|
||||
"swarm-bao-controller-policy"
|
||||
"swarm-bao-secret-publisher-policy"
|
||||
"swarm-bao-matrix-ctl-policy"
|
||||
"swarm-bao-matrix-token-policy"
|
||||
"swarm-bao-queue-agent-policy"
|
||||
"swarm-bao-grafana-oidc-policy"
|
||||
"swarm-bao-otel-oidc-policy"
|
||||
"swarm-bao-forwarder-oidc-policy"
|
||||
"swarm-bao-services-issuer-policy"
|
||||
"swarm-bao-nats-tls-policy"
|
||||
];
|
||||
|
||||
# Comment lines dropped first: both the HCL and the scripts explain
|
||||
# themselves in prose that names paths and `bao` commands.
|
||||
codeLines =
|
||||
text: lib.filter (l: builtins.match "[[:space:]]*#.*" l == null) (lib.splitString "\n" text);
|
||||
|
||||
bootstrapPolicyText = lib.concatStringsSep "\n" (
|
||||
codeLines (builtins.readFile ../host-modules/swarm-bao-bootstrap-policy.hcl)
|
||||
codeLines (builtins.readFile ../host-modules/bao-bootstrap-policy.hcl)
|
||||
);
|
||||
|
||||
# The granter's HCL is the only policy text in the unit that writes it.
|
||||
granterPolicyText = baoGrantHere.systemd.services.swarm-bao-granter-role.script;
|
||||
|
||||
matches = re: text: lib.filter lib.isList (builtins.split re text);
|
||||
|
||||
bootstrapGrants =
|
||||
grantsIn =
|
||||
text:
|
||||
map
|
||||
(m: {
|
||||
path = lib.elemAt m 0;
|
||||
caps = map lib.head (matches ''"([a-z]+)"'' (lib.elemAt m 1));
|
||||
})
|
||||
(
|
||||
matches ''path "([^"]+)"[[:space:]]*[{][[:space:]]*capabilities[[:space:]]*=[[:space:]]*[[]([a-z", ]*)'' bootstrapPolicyText
|
||||
matches ''path "([^"]+)"[[:space:]]*[{][[:space:]]*capabilities[[:space:]]*=[[:space:]]*[[]([a-z", ]*)'' text
|
||||
);
|
||||
|
||||
bootstrapGrants = grantsIn bootstrapPolicyText;
|
||||
granterGrants = grantsIn granterPolicyText;
|
||||
|
||||
# One `bao …` invocation → the path and capabilities it needs, as
|
||||
# `bao <cmd> -output-policy` reports them. Path-specific `sudo` (bao's
|
||||
# root-protected paths, e.g. `pki/root` for a delete) does not follow from
|
||||
|
|
@ -154,7 +208,10 @@ let
|
|||
"update"
|
||||
];
|
||||
in
|
||||
if a 0 == "policy" && a 1 == "write" then
|
||||
# A login and a seal-status check are unauthenticated: no policy grants them.
|
||||
if a 0 == "login" || a 0 == "status" then
|
||||
null
|
||||
else if a 0 == "policy" && a 1 == "write" then
|
||||
need "sys/policies/acl/${a 2}" cu
|
||||
else if a 0 == "secrets" && a 1 == "list" then
|
||||
need "sys/mounts" [ "read" ]
|
||||
|
|
@ -179,25 +236,28 @@ let
|
|||
|
||||
baoCalls =
|
||||
script:
|
||||
map
|
||||
(
|
||||
inv:
|
||||
baoCallNeeds (lib.filter (w: w != "") (lib.splitString " " (lib.replaceStrings [ "'" ] [ "" ] inv)))
|
||||
)
|
||||
(
|
||||
lib.concatMap (l: map (m: lib.elemAt m 1) (matches "(^[[:space:]]*|[$][(]|[)] )bao ([^|;)]*)" l)) (
|
||||
codeLines script
|
||||
lib.filter (n: n != null) (
|
||||
map
|
||||
(
|
||||
inv:
|
||||
baoCallNeeds (lib.filter (w: w != "") (lib.splitString " " (lib.replaceStrings [ "'" ] [ "" ] inv)))
|
||||
)
|
||||
);
|
||||
(
|
||||
lib.concatMap (l: map (m: lib.elemAt m 1) (matches "(^[[:space:]]*|[$][(]|[)] )bao ([^|;)]*)" l)) (
|
||||
codeLines script
|
||||
)
|
||||
)
|
||||
);
|
||||
|
||||
# bao's own rule: an exact path wins, otherwise the longest glob prefix.
|
||||
bootstrapGrantFor =
|
||||
path:
|
||||
# bao's own rule (vault/policy/acl.go): an exact path wins, otherwise the
|
||||
# longest glob prefix, and a trailing `*` is a plain string prefix.
|
||||
grantFor =
|
||||
grants: path:
|
||||
let
|
||||
exact = lib.filter (g: g.path == path) bootstrapGrants;
|
||||
exact = lib.filter (g: g.path == path) grants;
|
||||
globs = lib.filter (
|
||||
g: lib.hasSuffix "*" g.path && lib.hasPrefix (lib.removeSuffix "*" g.path) path
|
||||
) bootstrapGrants;
|
||||
) grants;
|
||||
in
|
||||
if exact != [ ] then
|
||||
lib.head exact
|
||||
|
|
@ -206,33 +266,40 @@ let
|
|||
best: g: if best == null || lib.stringLength g.path > lib.stringLength best.path then g else best
|
||||
) null globs;
|
||||
|
||||
bootstrapUngranted = lib.concatLists (
|
||||
lib.mapAttrsToList (
|
||||
unit: u:
|
||||
map (n: "${unit}: `bao ${n.call}` needs ${n.path} [${toString n.caps}]") (
|
||||
lib.filter (
|
||||
n:
|
||||
let
|
||||
g = bootstrapGrantFor n.path;
|
||||
in
|
||||
g == null || !(lib.all (c: lib.elem c g.caps) n.caps)
|
||||
) (baoCalls u.script)
|
||||
)
|
||||
) bootstrapUnits
|
||||
);
|
||||
ungranted =
|
||||
grants: units:
|
||||
lib.concatLists (
|
||||
lib.mapAttrsToList (
|
||||
unit: u:
|
||||
map (n: "${unit}: `bao ${n.call}` needs ${n.path} [${toString n.caps}]") (
|
||||
lib.filter (
|
||||
n:
|
||||
let
|
||||
g = grantFor grants n.path;
|
||||
in
|
||||
g == null || !(lib.all (c: lib.elem c g.caps) n.caps)
|
||||
) (baoCalls u.script)
|
||||
)
|
||||
) units
|
||||
);
|
||||
|
||||
bootstrapUngranted = ungranted bootstrapGrants bootstrapUnits;
|
||||
granterUngranted = ungranted granterGrants granterUnits;
|
||||
|
||||
cases = [
|
||||
{
|
||||
# Reads the rendered unit on the HOST, which is where the write happens:
|
||||
# every API listener demands a client certificate, and the host is the
|
||||
# side that has one.
|
||||
name = "a store host with a placed bootstrap token renders the granting unit on the host";
|
||||
name = "a store host renders the granting unit on the host, logging in as the granter";
|
||||
ok =
|
||||
let
|
||||
u = baoGrantHere.systemd.services.swarm-bao-controller-policy;
|
||||
in
|
||||
lib.hasInfix "/run/secrets/bao-bootstrap.token" u.script
|
||||
&& u.unitConfig.ConditionPathExists == "/run/secrets/bao-bootstrap.token";
|
||||
u.environment.BAO_CLIENT_CERT == granterCertFile
|
||||
&& u.environment.BAO_CLIENT_KEY == granterKeyFile
|
||||
&& lib.hasInfix "bao login -method=cert -token-only" u.script
|
||||
&& !(u.unitConfig ? ConditionPathExists);
|
||||
}
|
||||
{
|
||||
# The move is the fix, so pin the side it landed on: in the container it
|
||||
|
|
@ -273,13 +340,12 @@ let
|
|||
{
|
||||
# Same host-side reasoning as the controller's granting unit above: the
|
||||
# write needs a client certificate and the host is the side that has one.
|
||||
name = "a store host with a placed bootstrap token renders the publisher's granting unit too";
|
||||
name = "a store host renders the publisher's granting unit too, logging in as the granter";
|
||||
ok =
|
||||
let
|
||||
u = baoGrantHere.systemd.services.swarm-bao-secret-publisher-policy;
|
||||
in
|
||||
u.unitConfig.ConditionPathExists == "/run/secrets/bao-bootstrap.token"
|
||||
&& lib.hasInfix "swarm-secret-publisher" u.script;
|
||||
u.environment.BAO_CLIENT_CERT == granterCertFile && lib.hasInfix "swarm-secret-publisher" u.script;
|
||||
}
|
||||
{
|
||||
# The control for the case above, and the same one the controller's unit
|
||||
|
|
@ -590,29 +656,18 @@ let
|
|||
];
|
||||
}
|
||||
{
|
||||
# The absence arm: with no client CA there is no trust anchor, so the
|
||||
# login roles cannot be written — but the policies they would attach are
|
||||
# still asserted, exactly as the three service principals above behave in
|
||||
# this deployment. A unit that vanished here would take the policy with
|
||||
# it and leave nothing to diagnose.
|
||||
name = "with no client CA the five readers get policies but no login roles";
|
||||
# The absence arm: with no client CA there is no trust anchor, so no
|
||||
# role can be written and nothing can log in as the granter. The units
|
||||
# are gone, so the deployment has to say so itself.
|
||||
name = "with no client CA no granting unit renders, and the deployment warns";
|
||||
ok =
|
||||
let
|
||||
units = [
|
||||
"swarm-bao-matrix-token-policy"
|
||||
"swarm-bao-queue-agent-policy"
|
||||
"swarm-bao-grafana-oidc-policy"
|
||||
"swarm-bao-otel-oidc-policy"
|
||||
"swarm-bao-forwarder-oidc-policy"
|
||||
];
|
||||
scriptOf = unit: baoGrantNoClientCa.systemd.services.${unit}.script;
|
||||
s = baoGrantNoClientCa.systemd.services;
|
||||
in
|
||||
lib.all (
|
||||
unit:
|
||||
(baoGrantNoClientCa.systemd.services ? ${unit})
|
||||
&& lib.hasInfix "bao policy write" (scriptOf unit)
|
||||
&& !(lib.hasInfix "auth/cert/certs" (scriptOf unit))
|
||||
) units;
|
||||
lib.all (unit: !(s ? ${unit})) (grantingUnitNames ++ [ "swarm-bao-granter-role" ])
|
||||
&& lib.any (lib.hasInfix "services.hyperhive.deploy.bao.clientCaFile is null") baoGrantNoClientCa.warnings
|
||||
# The control: a store with a CA does not warn.
|
||||
&& !(lib.any (lib.hasInfix "clientCaFile is null") baoGrantHere.warnings);
|
||||
}
|
||||
{
|
||||
# Same control the three service principals carry: the write needs a
|
||||
|
|
@ -639,7 +694,8 @@ let
|
|||
{
|
||||
# The other end of those units: each reader logs in against the role its
|
||||
# own policy unit writes, so it has to wait for that unit. Ordering and
|
||||
# never a requirement, since the policy unit skips once the token is gone.
|
||||
# never a requirement: a failed policy unit still counts as done, and the
|
||||
# reader's own retries carry it past that.
|
||||
#
|
||||
# The forwarder is listed apart from `policyReaders`: it renders wherever
|
||||
# the store does, so it is never absent on a store host and never present
|
||||
|
|
@ -684,21 +740,237 @@ let
|
|||
lib.all unordered policyReaders;
|
||||
}
|
||||
{
|
||||
# A store host that has not placed a bootstrap token can write no grant at
|
||||
# all, so none of the four units may exist — the same claim
|
||||
# `baoGrantNoStore` makes for the controller's, one file over. Without
|
||||
# this arm `lib.mkIf haveBootstrapToken` could be dropped from the shared
|
||||
# builder and every other case here would still pass.
|
||||
name = "without a bootstrap token none of the five readers' granting units render";
|
||||
# A store host without the granter's pair writes its grants some other
|
||||
# way, so none of the ten units may exist. Without this arm
|
||||
# `lib.mkIf haveGranter` could be dropped from any of them and every other
|
||||
# case here would still pass.
|
||||
name = "without the granter's pair none of the ten granting units render";
|
||||
ok =
|
||||
let
|
||||
s = baoGrantNoStore.systemd.services;
|
||||
s = baoGranterOptOut.systemd.services;
|
||||
in
|
||||
!(s ? swarm-bao-matrix-token-policy)
|
||||
&& !(s ? swarm-bao-queue-agent-policy)
|
||||
&& !(s ? swarm-bao-grafana-oidc-policy)
|
||||
&& !(s ? swarm-bao-otel-oidc-policy)
|
||||
&& !(s ? swarm-bao-forwarder-oidc-policy);
|
||||
lib.all (unit: !(s ? ${unit})) (grantingUnitNames ++ [ "swarm-bao-granter-role" ])
|
||||
# The control: the same store with the pair renders all ten.
|
||||
&& lib.all (unit: baoGrantHere.systemd.services ? ${unit}) grantingUnitNames;
|
||||
}
|
||||
{
|
||||
# 🩸 What replaced the silent skip. With no bootstrap token the ten still
|
||||
# render, and a refused granter fails them with the step that fixes it.
|
||||
# A store host that never named a token is told to name one, since the
|
||||
# unit that sets the granter up renders only where it has.
|
||||
name = "a store host without a bootstrap token renders the ten, each failing loudly with the one-time step";
|
||||
ok =
|
||||
let
|
||||
s = baoGranterNoToken.systemd.services;
|
||||
loud =
|
||||
unit:
|
||||
s ? ${unit}
|
||||
&&
|
||||
lib.hasInfix "bao policy write bao-bootstrap /etc/hyperhive/bao-bootstrap-policy.hcl"
|
||||
s.${unit}.script
|
||||
&& lib.hasInfix "set services.hyperhive.deploy.bao.bootstrapTokenFile" s.${unit}.script
|
||||
&& lib.hasInfix "exit 1" s.${unit}.script;
|
||||
in
|
||||
lib.all loud grantingUnitNames && !(s ? swarm-bao-granter-role);
|
||||
}
|
||||
{
|
||||
# Where the token is named, the step names the file to put it in and the
|
||||
# unit to restart.
|
||||
name = "with a bootstrap token named, the one-time step places it and restarts the granter's unit";
|
||||
ok = lib.all (
|
||||
unit:
|
||||
let
|
||||
sc = baoGrantHere.systemd.services.${unit}.script;
|
||||
in
|
||||
lib.hasInfix "install -D -m 0600 /dev/stdin /run/secrets/bao-bootstrap.token" sc
|
||||
&& lib.hasInfix "systemctl restart swarm-bao-granter-role" sc
|
||||
) grantingUnitNames;
|
||||
}
|
||||
{
|
||||
# Every granting unit retries a sealed or late store for a day, in the
|
||||
# `[Unit]` section systemd reads it from, and waits for the unit that
|
||||
# mints the granter's leaf.
|
||||
name = "each granting unit requires the PKI unit and retries 2880 times at 30s";
|
||||
ok = lib.all (
|
||||
unit:
|
||||
let
|
||||
u = baoGrantHere.systemd.services.${unit};
|
||||
in
|
||||
lib.elem "swarm-bao-pki.service" u.requires
|
||||
&& lib.elem "swarm-bao-pki.service" u.after
|
||||
&& lib.elem "swarm-bao-granter-role.service" u.after
|
||||
&& !(lib.elem "swarm-bao-granter-role.service" (u.requires ++ u.wants))
|
||||
&& toString u.unitConfig.StartLimitBurst == "2880"
|
||||
&& toString u.unitConfig.StartLimitIntervalSec == "90000"
|
||||
&& toString u.serviceConfig.RestartSec == "30"
|
||||
&& u.serviceConfig.Restart == "on-failure"
|
||||
) grantingUnitNames;
|
||||
}
|
||||
{
|
||||
# The only unit left acting with the token, so the only one that may
|
||||
# skip on it.
|
||||
name = "no unit but the granter's role reads the bootstrap token or skips on it";
|
||||
ok =
|
||||
lib.attrNames bootstrapUnits == [ "swarm-bao-granter-role" ]
|
||||
&& lib.all (u: !(u.unitConfig ? ConditionPathExists)) (lib.attrValues granterUnits)
|
||||
&&
|
||||
baoGrantHere.systemd.services.swarm-bao-granter-role.unitConfig.ConditionPathExists
|
||||
== bootstrapTokenFile;
|
||||
}
|
||||
{
|
||||
# The granter's grants, whole. Pinned as the full list, because an added
|
||||
# path or capability is exactly what a presence check misses.
|
||||
name = "the granter's policy is exactly these eleven stanzas";
|
||||
ok =
|
||||
let
|
||||
cu = [
|
||||
"create"
|
||||
"update"
|
||||
];
|
||||
in
|
||||
granterGrants == [
|
||||
{
|
||||
path = "sys/policies/acl/swarm-*";
|
||||
caps = cu;
|
||||
}
|
||||
{
|
||||
path = "auth/cert/certs/swarm-*";
|
||||
caps = cu;
|
||||
}
|
||||
{
|
||||
path = "pki/roles/swarm-*";
|
||||
caps = cu;
|
||||
}
|
||||
{
|
||||
path = "sys/mounts";
|
||||
caps = [ "read" ];
|
||||
}
|
||||
{
|
||||
path = "sys/mounts/secret";
|
||||
caps = cu;
|
||||
}
|
||||
{
|
||||
path = "sys/mounts/pki";
|
||||
caps = cu;
|
||||
}
|
||||
{
|
||||
path = "sys/mounts/pki/tune";
|
||||
caps = cu;
|
||||
}
|
||||
{
|
||||
path = "pki/issuers";
|
||||
caps = [ "list" ];
|
||||
}
|
||||
{
|
||||
path = "pki/cert/ca";
|
||||
caps = [ "read" ];
|
||||
}
|
||||
{
|
||||
path = "pki/root";
|
||||
caps = [
|
||||
"delete"
|
||||
"sudo"
|
||||
];
|
||||
}
|
||||
{
|
||||
path = "pki/root/generate/internal";
|
||||
caps = cu;
|
||||
}
|
||||
];
|
||||
}
|
||||
{
|
||||
# Neither its own policy and role nor the bootstrap policy may be
|
||||
# reachable, or the granter could rewrite what constrains it and what the
|
||||
# next bootstrap token carries.
|
||||
name = "the granter cannot reach the policy or role that constrains it, nor the bootstrap policy";
|
||||
ok = lib.all (p: grantFor granterGrants p == null) [
|
||||
"sys/policies/acl/bao-granter"
|
||||
"auth/cert/certs/bao-granter"
|
||||
"sys/policies/acl/bao-bootstrap"
|
||||
];
|
||||
}
|
||||
{
|
||||
# Outside `swarm-*` and the store's own mounts it holds nothing: no
|
||||
# hive's policy or role, no auth mount, no token, no secret.
|
||||
name = "the granter grants nothing outside swarm-* and the store's own mounts";
|
||||
ok =
|
||||
lib.all (p: grantFor granterGrants p == null) [
|
||||
"sys/policies/acl/hive-x"
|
||||
"auth/cert/certs/hive-x"
|
||||
"sys/auth"
|
||||
"sys/auth/cert"
|
||||
"sys/auth/x"
|
||||
"auth/token/create"
|
||||
"auth/token/create-orphan"
|
||||
"secret/data/x"
|
||||
"secret/data/swarm/agents/x/queue"
|
||||
"sys/policies/acl/x"
|
||||
"sys/policies/acl/root"
|
||||
"pki/issue/swarm-services"
|
||||
"pki/sign/swarm-services"
|
||||
"*"
|
||||
]
|
||||
&& !(lib.any (
|
||||
g:
|
||||
lib.elem g.path [
|
||||
"*"
|
||||
"sys/policies/acl/*"
|
||||
"auth/cert/certs/*"
|
||||
"pki/roles/*"
|
||||
]
|
||||
) granterGrants);
|
||||
}
|
||||
{
|
||||
# Its names sit outside both globs that write grants — its own
|
||||
# `swarm-*` and the controller's `hive-*`.
|
||||
name = "the granter's own names are outside swarm-* and hive-*";
|
||||
ok =
|
||||
let
|
||||
sc = baoGrantHere.systemd.services.swarm-bao-granter-role.script;
|
||||
cn = baoGrantHere.services.hyperhive.deploy.bao.granterCommonName;
|
||||
in
|
||||
lib.hasInfix "bao policy write bao-granter -" sc
|
||||
&& lib.hasInfix "auth/cert/certs/bao-granter" sc
|
||||
&& lib.hasInfix "token_policies=bao-granter" sc
|
||||
&& lib.hasInfix "token_ttl=15m" sc
|
||||
&& !(lib.hasPrefix "swarm-" cn)
|
||||
&& !(lib.hasPrefix "hive-" cn);
|
||||
}
|
||||
{
|
||||
# The other principals are what they were: no unit but the granter's own
|
||||
# hands its policy to a role, and none of them logs in as it.
|
||||
name = "no other principal gains the granter's policy";
|
||||
ok =
|
||||
lib.all (u: !(lib.hasInfix "token_policies=bao-granter" u.script)) (
|
||||
lib.attrValues (lib.removeAttrs baoGrantWithConsumers.systemd.services [ "swarm-bao-granter-role" ])
|
||||
)
|
||||
&& lib.all (u: (u.environment.BAO_CLIENT_CERT or null) != granterCertFile) (
|
||||
lib.attrValues (lib.removeAttrs baoGrantWithConsumers.systemd.services grantingUnitNames)
|
||||
);
|
||||
}
|
||||
{
|
||||
# The minting side: a role matching a subject nothing signs is a
|
||||
# granter that cannot log in.
|
||||
name = "the PKI unit signs the granter's leaf under its own subject";
|
||||
ok =
|
||||
let
|
||||
s = baoGrantHere.systemd.services.swarm-bao-pki.script;
|
||||
in
|
||||
lib.hasInfix "/granter.pem ]" s && lib.hasInfix "bao-granter \"\" clientAuth" s;
|
||||
}
|
||||
{
|
||||
# The granter writes pki roles through `roles/swarm-*` only, so a role
|
||||
# named otherwise is refused at eval rather than 403'd at deploy.
|
||||
name = "a pki role name outside swarm-* is refused, naming both options";
|
||||
ok =
|
||||
let
|
||||
names =
|
||||
a:
|
||||
lib.hasInfix "services.hyperhive.deploy.bao.servicesPkiRoleName" a.message
|
||||
&& lib.hasInfix "services.hyperhive.deploy.bao.natsPkiRoleName" a.message;
|
||||
in
|
||||
lib.any (a: !a.assertion && names a) baoGranterOddPkiRole.assertions
|
||||
&& !(lib.any (a: !a.assertion && names a) baoGrantHere.assertions);
|
||||
}
|
||||
{
|
||||
# 🩸 The refusal half of the forwarder's own leaf. It renders wherever the
|
||||
|
|
@ -762,15 +1034,17 @@ let
|
|||
ok = lib.hasInfix "path \"secret/data/swarm/controller/swarm-controller/matrix/appservice-token\" {\n capabilities = [\"read\"]\n}" baoGrantHere.systemd.services.swarm-bao-controller-policy.script;
|
||||
}
|
||||
{
|
||||
# The policy above grants paths under a mount nothing else creates, so
|
||||
# the unit that writes the policy has to create it too — otherwise every
|
||||
# certificate login fails against a path that is not there.
|
||||
name = "the granting unit creates the cert auth mount and the controller's role";
|
||||
# Every role lives under a mount nothing else creates, and the granter
|
||||
# holds no `sys/auth`, so the token-holding unit creates it — otherwise
|
||||
# every certificate login fails against a path that is not there.
|
||||
name = "the granter's role unit creates the cert auth mount, and the controller's unit writes its role";
|
||||
ok =
|
||||
let
|
||||
s = baoGrantHere.systemd.services.swarm-bao-controller-policy.script;
|
||||
g = baoGrantHere.systemd.services.swarm-bao-granter-role.script;
|
||||
in
|
||||
lib.hasInfix "bao auth enable cert" s
|
||||
lib.hasInfix "bao auth enable cert" g
|
||||
&& !(lib.hasInfix "bao auth enable" s)
|
||||
&& lib.hasInfix "auth/cert/certs/swarm-controller" s
|
||||
&& lib.hasInfix "/var/lib/swarm-bao-tls/client-ca.pem" s;
|
||||
}
|
||||
|
|
@ -790,28 +1064,6 @@ let
|
|||
in
|
||||
lib.hasInfix "bao secrets enable -path=secret kv-v2" s;
|
||||
}
|
||||
{
|
||||
# The arm that makes the one above mean something. A role's trust anchor
|
||||
# is the CA, so with none named there is nothing to write — and the
|
||||
# policy write, which needs no CA, must survive that.
|
||||
#
|
||||
# ⚠️ Matched on the COMMANDS, not on `auth/cert/certs`: the policy text is
|
||||
# embedded in this same script and grants that very path, so the shorter
|
||||
# infix is present either way and the arm could never fail.
|
||||
name = "with no client CA the unit still writes the policy and skips the role";
|
||||
ok =
|
||||
let
|
||||
s = baoGrantNoClientCa.systemd.services.swarm-bao-controller-policy.script;
|
||||
in
|
||||
lib.hasInfix "bao policy write" s
|
||||
&& !(lib.hasInfix "bao auth enable cert" s)
|
||||
&& !(lib.hasInfix "client-ca.pem" s)
|
||||
# The KV mount is NOT part of what a missing client CA switches off:
|
||||
# the controller writes through it whether or not anything can log in
|
||||
# by certificate. Asserted here rather than trusted, because both
|
||||
# steps live in the same script and one indentation level decides it.
|
||||
&& lib.hasInfix "bao secrets enable -path=secret kv-v2" s;
|
||||
}
|
||||
{
|
||||
# What makes the granting-unit cases mean something, and the property
|
||||
# the host-side half depends on: no store here, so no bind mount and no
|
||||
|
|
@ -821,43 +1073,66 @@ let
|
|||
ok = !(baoGrantNoStore.systemd.services ? swarm-bao-bootstrap-dir);
|
||||
}
|
||||
{
|
||||
# The drift this case exists to stop: setup.md's copy of the policy
|
||||
# stayed at the controller's first six grants while seven more units
|
||||
# started using the token. Failing names every ungranted call.
|
||||
# The operator writes this policy by hand, so a call the token-holding
|
||||
# unit makes and the file does not grant is a one-time step that fails.
|
||||
# Failing names every ungranted call.
|
||||
name =
|
||||
"every bao call a bootstrap-token unit makes is granted by swarm-bao-bootstrap-policy.hcl"
|
||||
"every bao call the bootstrap-token unit makes is granted by bao-bootstrap-policy.hcl"
|
||||
+ lib.optionalString (bootstrapUngranted != [ ]) (
|
||||
": " + lib.concatStringsSep "; " bootstrapUngranted
|
||||
);
|
||||
ok = bootstrapUngranted == [ ];
|
||||
}
|
||||
{
|
||||
# What makes the case above mean something: discovery by token path
|
||||
# reaches every unit that uses the token today, and each yields calls.
|
||||
name = "the bootstrap-policy check sees all ten units that use the token, and parses calls from each";
|
||||
# The same check for the granter: a grant a unit writes outside its
|
||||
# globs is a 403 on deploy. Failing names every ungranted call.
|
||||
name =
|
||||
"every bao call a granting unit makes is granted by the granter's policy"
|
||||
+ lib.optionalString (granterUngranted != [ ]) (": " + lib.concatStringsSep "; " granterUngranted);
|
||||
ok = granterUngranted == [ ];
|
||||
}
|
||||
{
|
||||
# What makes the case above mean something: discovery by the granter's
|
||||
# certificate reaches all ten units, and each yields calls.
|
||||
name = "the granter-policy check sees all ten granting units, and parses calls from each";
|
||||
ok =
|
||||
lib.all (n: bootstrapUnits ? ${n}) [
|
||||
"swarm-bao-controller-policy"
|
||||
"swarm-bao-secret-publisher-policy"
|
||||
"swarm-bao-matrix-ctl-policy"
|
||||
"swarm-bao-matrix-token-policy"
|
||||
"swarm-bao-queue-agent-policy"
|
||||
"swarm-bao-grafana-oidc-policy"
|
||||
"swarm-bao-otel-oidc-policy"
|
||||
"swarm-bao-forwarder-oidc-policy"
|
||||
"swarm-bao-services-issuer-policy"
|
||||
"swarm-bao-nats-tls-policy"
|
||||
]
|
||||
lib.sort lib.lessThan (lib.attrNames granterUnits) == lib.sort lib.lessThan grantingUnitNames
|
||||
&& lib.all (u: baoCalls u.script != [ ]) (lib.attrValues granterUnits)
|
||||
&& lib.all (u: baoCalls u.script != [ ]) (lib.attrValues bootstrapUnits);
|
||||
}
|
||||
{
|
||||
# And the grants side: a stanza the parser skipped would read as a
|
||||
# grant that is not there.
|
||||
name = "every path stanza in swarm-bao-bootstrap-policy.hcl parses";
|
||||
name = "every path stanza in bao-bootstrap-policy.hcl and the granter's policy parses";
|
||||
ok =
|
||||
bootstrapGrants != [ ]
|
||||
&& lib.length bootstrapGrants == lib.length (matches ''path "'' bootstrapPolicyText)
|
||||
&& lib.all (g: g.caps != [ ]) bootstrapGrants;
|
||||
lib.all
|
||||
(
|
||||
t:
|
||||
let
|
||||
grants = grantsIn t;
|
||||
in
|
||||
grants != [ ]
|
||||
&& lib.length grants == lib.length (matches ''path "'' t)
|
||||
&& lib.all (g: g.caps != [ ]) grants
|
||||
)
|
||||
[
|
||||
bootstrapPolicyText
|
||||
granterPolicyText
|
||||
];
|
||||
}
|
||||
{
|
||||
# The bootstrap policy, whole: the auth mounts and the granter's own two
|
||||
# objects, and nothing a `swarm-*` grant lives at.
|
||||
name = "the bootstrap policy is exactly the auth mounts and the granter's policy and role";
|
||||
ok =
|
||||
lib.map (g: g.path) bootstrapGrants == [
|
||||
"sys/auth"
|
||||
"sys/auth/cert"
|
||||
"sys/auth/approle"
|
||||
"sys/policies/acl/bao-granter"
|
||||
"auth/cert/certs/bao-granter"
|
||||
]
|
||||
&& grantFor bootstrapGrants "sys/policies/acl/swarm-controller" == null;
|
||||
}
|
||||
];
|
||||
in
|
||||
|
|
|
|||
|
|
@ -23,18 +23,16 @@ let
|
|||
runGroup
|
||||
;
|
||||
|
||||
# Every service on one host, with a bootstrap token so the store's granting
|
||||
# unit renders the role this leaf is issued through.
|
||||
# Every service on one host, so the store's granting unit renders the role
|
||||
# this leaf is issued through.
|
||||
allLocal = hive {
|
||||
deploy.singleHostSwarm = true;
|
||||
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
|
||||
};
|
||||
|
||||
# The same host with the role's lifetime moved, so a threshold that is a
|
||||
# number of its own shows up as one that did not move with it.
|
||||
shortTtl = hive {
|
||||
deploy.singleHostSwarm = true;
|
||||
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
|
||||
deploy.bao.servicesPkiLeafTtlHours = 48;
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -83,6 +83,13 @@ let
|
|||
swarm.hives.fwctl.domain = "f.t.local";
|
||||
};
|
||||
|
||||
# The granter's, the one subject whose role may write every `swarm-*` grant.
|
||||
hiveNamedAfterGranterSubject = hive {
|
||||
deploy.swarm-otel.enable = false;
|
||||
deploy.bao.granterCommonName = "grctl";
|
||||
swarm.hives.grctl.domain = "gr.t.local";
|
||||
};
|
||||
|
||||
# 🩸 A different shape from every fixture above: the matrix-token and
|
||||
# queue-credential roles are written PER HIVE, so the subject a hive must not
|
||||
# be is `<prefix>-<some hive's name>` rather than the prefix itself. Reserving
|
||||
|
|
@ -173,6 +180,16 @@ let
|
|||
a: !a.assertion && lib.hasInfix "'fwctl'" a.message
|
||||
) hiveNamedAfterForwarderOidcSubject.assertions;
|
||||
}
|
||||
{
|
||||
# And the granter's, whose role is root-equivalent: a hive holding a leaf
|
||||
# it accepts could grant itself anything.
|
||||
name = "a hive named after the bao granter's subject is refused too";
|
||||
ok =
|
||||
equalityGuardFired hiveNamedAfterGranterSubject
|
||||
&& lib.any (
|
||||
a: !a.assertion && lib.hasInfix "'grctl'" a.message
|
||||
) hiveNamedAfterGranterSubject.assertions;
|
||||
}
|
||||
{
|
||||
# 🩸 The per-hive half, and the one a prefix-only reservation would miss:
|
||||
# the role is `<prefix>-<hive>`, so the reserved string has to be composed
|
||||
|
|
|
|||
|
|
@ -26,12 +26,10 @@ let
|
|||
natsName = "nats.t.local";
|
||||
natsUrl = "tls://${natsName}:4222";
|
||||
|
||||
# Every service on one host, with a bootstrap token so the store's granting
|
||||
# units render. The queue, the store and every in-tree client of the queue
|
||||
# Every service on one host. The queue, the store and every in-tree client of the queue
|
||||
# are all here, so the scan below reads each of them.
|
||||
allLocal = hive {
|
||||
deploy.singleHostSwarm = true;
|
||||
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
|
||||
};
|
||||
|
||||
# The same host on the mesh.
|
||||
|
|
@ -232,8 +230,8 @@ let
|
|||
&& !(lib.elem 4222 allLocal.networking.firewall.allowedTCPPorts);
|
||||
}
|
||||
{
|
||||
# Ordering, never a requirement: the policy unit skips once the bootstrap
|
||||
# token is gone, and a skipped unit counts as done.
|
||||
# Ordering, never a requirement: a policy unit that failed still counts
|
||||
# as done, and the leaf unit's own retries carry it past that.
|
||||
name = "the leaf unit is ordered after its policy unit, with no requires";
|
||||
ok =
|
||||
let
|
||||
|
|
|
|||
|
|
@ -80,6 +80,7 @@ let
|
|||
"hive-tls-ca"
|
||||
"swarm-services-cert"
|
||||
"hive-gateway-self-signed-cert"
|
||||
"swarm-bao-granter-role"
|
||||
"swarm-bao-controller-policy"
|
||||
"swarm-bao-secret-publisher-policy"
|
||||
"swarm-bao-matrix-ctl-policy"
|
||||
|
|
|
|||
Loading…
Reference in a new issue