hyperhive/nix/module-eval/bao-grants.nix
atlas 67ba28448f swarm-matrix-ctl: one control binary for the matrix container, not one per job
Renames `swarm-matrix-minter` and reshapes it around subcommands. Minting
is now `swarm-matrix-ctl mint`.

Running rust inside `containers.hive-matrix` is not free: it needs its own
store identity, its own cert role and its own bind mounts, and every one of
those is per-*container*, not per-task. A second single-purpose crate would
have had to duplicate that plumbing to add one action, so the next thing
that has to run in there should be a verb here rather than a new crate.
The old name guaranteed the opposite.

`main.rs` is clap dispatch; the minting logic moves to `mint.rs` unchanged.
A bare invocation is refused: `mint` writes a credential, so "no verb"
defaulting to it would make a typo in the unit mint rather than fail.

The environment prefix moves with it, `MATRIX_MINTER_*` → `MATRIX_MINT_*`.
Scoped to the verb and not to the binary, because a binary-scoped prefix is
one the next verb has to share or widen, and a widened one never narrows
again. A test asserts every variable carries the verb's prefix.

The principal renames too. The cert role, bao policy, granting unit, leaf
filename and `certAuthCns` entry all have to spell one string the same way,
so leaving them as `swarm-matrix-minter` would have rebuilt the naming
split this branch exists to remove. Renaming the nix options alongside is
free here: every one of them is introduced by this PR and has never been
released, so no operator config names them yet.

`ExecStart` now names the verb, which is a contract between a nix string
and a clap enum that fails at deploy time with no local signal. Both ends
assert it: `mint_is_spelled_the_way_the_unit_invokes_it` in the crate, and
a new module-eval arm reading the rendered `ExecStart`.

docs/getting-started/setup.md drops the sender token from its "live on the
host" list: setup does not touch this credential, so a setup guide has no
reason to name it.
2026-09-20 22:07:16 +02:00

286 lines
13 KiB
Nix

# `checks.module-eval-bao-grants` — see ./lib.nix for the shared
# rationale (why this suite exists, naming convention, "evaluates
# not executes").
{
pkgs,
lib,
self,
nixosSystem,
}:
let
inherit
(import ./lib.nix {
inherit
pkgs
lib
self
nixosSystem
;
})
hive
runGroup
;
# The store, plus a placed bootstrap token: the only shape in which the
# swarm's first grant can be written at all.
baoGrantHere = hive {
deploy.bao.enable = true;
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
};
# The credential without the store. Writing the first grant is a store-side
# operation, so a host holding only the token has nothing to do — and this
# is the arm that separates "an operator placed a token" from "this box can
# act on it".
baoGrantNoStore = hive {
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
};
# The store and the token, with no CA to trust. `mkForce` because the PKI
# glue supplies one by default here — this is the deployment that brings its
# own certificates and has not named the authority yet, and it separates
# "the grant unit runs" from "cert auth can be set up".
baoGrantNoClientCa = hive {
deploy.bao.enable = true;
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
deploy.bao.clientCaFile = lib.mkForce null;
};
cases = [
{
# Reads the rendered unit on the HOST, which is where the write happens:
# every API listener demands a client certificate, and the host is the
# side that has one.
name = "a store host with a placed bootstrap token renders the granting unit on the host";
ok =
let
u = baoGrantHere.systemd.services.swarm-bao-controller-policy;
in
lib.hasInfix "/run/secrets/bao-bootstrap.token" u.script
&& u.unitConfig.ConditionPathExists == "/run/secrets/bao-bootstrap.token";
}
{
# The move is the fix, so pin the side it landed on: in the container it
# had no identity to open a connection with, and no address that resolved
# to the store from its own netns.
name = "the granting unit is not rendered inside the store's container";
ok = !(baoGrantHere.containers.swarm-bao.config.systemd.services ? swarm-bao-controller-policy);
}
{
# `StartLimit*` are `[Unit]` settings that systemd ignores under
# `[Service]`, so a bound written into `serviceConfig` renders, deploys
# and does nothing. Asserted where nixpkgs puts it rather than where it
# was written. The values are pinned because they are the bound: under
# `shamir` a human unseals by hand, and anything shorter than a day gives
# up first — `start-limit-hit` does not self-heal.
name = "the granting unit's start limit lands in [Unit], not [Service]";
ok =
let
u = baoGrantHere.systemd.services.swarm-bao-controller-policy;
in
toString u.unitConfig.StartLimitBurst == "2880"
&& toString u.unitConfig.StartLimitIntervalSec == "90000"
&& !(u.serviceConfig ? StartLimitBurst);
}
{
# The grants themselves, and the `hive-` prefix is the whole point:
# without it the controller can rewrite the policy that constrains it,
# which is a privilege escalation that renders, deploys and looks fine.
# Readable here only because the HCL is piped as an argument rather than
# written to a store path.
name = "the controller's bao grants cannot reach the policy that constrains it";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-controller-policy.script;
in
lib.hasInfix "sys/policies/acl/hive-*" s && !(lib.hasInfix "sys/policies/acl/*" s);
}
{
# Same host-side reasoning as the controller's granting unit above: the
# write needs a client certificate and the host is the side that has one.
name = "a store host with a placed bootstrap token renders the publisher's granting unit too";
ok =
let
u = baoGrantHere.systemd.services.swarm-bao-secret-publisher-policy;
in
u.unitConfig.ConditionPathExists == "/run/secrets/bao-bootstrap.token"
&& lib.hasInfix "swarm-secret-publisher" u.script;
}
{
# The control for the case above, and the same one the controller's unit
# has: rendered on the host means NOT rendered in the container, where it
# would have neither an identity nor a route to the store.
name = "the publisher's granting unit is not rendered inside the store's container";
ok =
!(baoGrantHere.containers.swarm-bao.config.systemd.services ? swarm-bao-secret-publisher-policy);
}
{
# The whole point of a second principal. The two prefixes it publishes to
# and not `swarm/`, so it cannot touch an agent's credentials; and no
# `read`, so a unit whose job is copying a file cannot recover what is
# already there. Pinned as the full capability list per prefix, because an
# added capability is exactly what a presence check misses.
name = "the publisher's grant is write-only and reaches the hive and service prefixes alone";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-secret-publisher-policy.script;
in
lib.hasInfix "path \"secret/data/swarm/hives/*\" {\n capabilities = [\"create\", \"update\"]" s
&& lib.hasInfix "path \"secret/data/swarm/services/*\" {\n capabilities = [\"create\", \"update\"]" s
&& !(lib.hasInfix "secret/data/swarm/agents" s)
&& !(lib.hasInfix "secret/data/swarm/*" s)
&& !(lib.hasInfix "sys/policies/acl" s);
}
{
# The ordering is load-bearing and invisible at runtime: the controller's
# unit creates the KV and cert-auth mounts this one writes into, so
# without it a cold boot races and fails with "route entry not found",
# which names neither unit.
name = "the publisher's granting unit is ordered after the one that creates the mounts";
ok = lib.elem "swarm-bao-controller-policy.service" (
baoGrantHere.systemd.services.swarm-bao-secret-publisher-policy.after
);
}
{
# The third principal's grant, and the narrowest of the three: ONE path,
# spelled to the leaf. The negative arms are the property — a homeserver
# is not entitled to overwrite Grafana's OIDC client, so widening this to
# the `services/` prefix the publisher holds would be a real loss even
# though it would read as tidier.
#
# ⚠️ `services` is PLURAL, because the path segment comes from
# `Kind::Service`'s `#[strum(serialize = "services")]` and not from
# `Kind::label`, which renders the singular for error text. The singular
# spelling evaluates, deploys, and 403s every read with "permission
# denied" and nothing else.
name = "matrix-ctl's grant is the sender token's path and nothing else";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-matrix-ctl-policy.script;
in
lib.hasInfix "path \"secret/data/swarm/services/matrix/sender-token\" {" s
&& !(lib.hasInfix "secret/data/swarm/services/*" s)
&& !(lib.hasInfix "secret/data/swarm/agents" s)
&& !(lib.hasInfix "secret/data/swarm/hives" s)
&& !(lib.hasInfix "sys/policies/acl" s);
}
{
# 🩸 `read` is load-bearing here and is the one capability neither
# sibling has. matrix-ctl's first act is to read this path back and stop
# if something is there — that read IS "and only once", so without the
# capability every container restart would mint a second access token and
# invalidate the hive's.
name = "matrix-ctl may read back the one path it writes";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-matrix-ctl-policy.script;
in
lib.hasInfix "capabilities = [\"create\", \"update\", \"read\"]" s
&& lib.hasInfix "auth/cert/certs/swarm-matrix-ctl" s
&& lib.hasInfix "allowed_common_names=swarm-matrix-ctl" s;
}
{
# Same two controls its siblings carry: ordered after the unit that makes
# the mounts it writes into, and rendered on the HOST rather than inside
# the store's container, where it would have neither an identity nor a
# route to the store.
name = "matrix-ctl's granting unit is ordered after the mounts and rendered on the host";
ok =
lib.elem "swarm-bao-controller-policy.service" (
baoGrantHere.systemd.services.swarm-bao-matrix-ctl-policy.after
)
&& !(baoGrantHere.containers.swarm-bao.config.systemd.services ? swarm-bao-matrix-ctl-policy);
}
{
# The policy authorising this route lives in another file, and nothing
# else relates the grants to the paths the code actually writes.
#
# `secret/data/` is KV v2's ACL prefix; `swarm` is
# `swarm_secret_client::path::ROOT` and `agents` is
# `Kind::Agent.as_str()`, both of which that crate pins in its own test.
#
# The grant is still the agent kind alone because nothing writes another
# one yet. It widens when a path outside `agents/` gains a writer, not
# when the kinds are declared.
name = "the controller may write agent credentials, and only under the agent prefix";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-controller-policy.script;
in
lib.hasInfix "secret/data/swarm/agents/*" s
&& !(lib.hasInfix "secret/data/*" s)
&& !(lib.hasInfix "path \"secret/*\"" s);
}
{
# Write-only is the property, not an accident of how it was typed: a
# `read` here would let the controller recover every agent's credentials
# instead of only replacing them. Pinned as the whole capability list,
# because an added capability is exactly what a presence check misses.
name = "the controller's grant on agent credentials is write-only";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-controller-policy.script;
in
lib.hasInfix "path \"secret/data/swarm/agents/*\" {\n capabilities = [\"create\", \"update\"]" s;
}
{
# The policy above grants paths under a mount nothing else creates, so
# the unit that writes the policy has to create it too — otherwise every
# certificate login fails against a path that is not there.
name = "the granting unit creates the cert auth mount and the controller's role";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-controller-policy.script;
in
lib.hasInfix "bao auth enable cert" s
&& lib.hasInfix "auth/cert/certs/swarm-controller" s
&& lib.hasInfix "/var/lib/swarm-bao-tls/client-ca.pem" s;
}
{
# Same shape as the cert mount above, for the engine the controller
# writes credentials through: a fresh store has no `secret/`, so the
# grant would name a mount nobody created and the first write would 404.
#
# ⚠️ Matched on the COMMAND, for the reason the no-client-CA case below
# spells out: the policy text is embedded in this same script and grants
# `secret/data/...`, so any arm keyed on the *path* is satisfied either
# way and could never fail.
name = "the granting unit creates the KV mount the controller writes through";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-controller-policy.script;
in
lib.hasInfix "bao secrets enable -path=secret kv-v2" s;
}
{
# The arm that makes the one above mean something. A role's trust anchor
# is the CA, so with none named there is nothing to write — and the
# policy write, which needs no CA, must survive that.
#
# ⚠️ Matched on the COMMANDS, not on `auth/cert/certs`: the policy text is
# embedded in this same script and grants that very path, so the shorter
# infix is present either way and the arm could never fail.
name = "with no client CA the unit still writes the policy and skips the role";
ok =
let
s = baoGrantNoClientCa.systemd.services.swarm-bao-controller-policy.script;
in
lib.hasInfix "bao policy write" s
&& !(lib.hasInfix "bao auth enable cert" s)
&& !(lib.hasInfix "client-ca.pem" s)
# The KV mount is NOT part of what a missing client CA switches off:
# the controller writes through it whether or not anything can log in
# by certificate. Asserted here rather than trusted, because both
# steps live in the same script and one indentation level decides it.
&& lib.hasInfix "bao secrets enable -path=secret kv-v2" s;
}
{
# What makes the granting-unit cases mean something, and the property
# the host-side half depends on: no store here, so no bind mount and no
# unit. Without it a hive that merely names a token would drag the
# store's container config into its evaluation.
name = "a bootstrap token on a host that runs no store grants nothing";
ok = !(baoGrantNoStore.systemd.services ? swarm-bao-bootstrap-dir);
}
];
in
runGroup "bao-grants" cases