swarm-bao: let the controller write agent credentials, and gate that it can

Closes #4124.

The controller's policy granted only the bootstrap paths -- hive cert-auth
roles and hive ACLs. #4113 then made it a secret WRITER, and nothing related
the grants to the paths the code writes, so every matrix token provision
answered 403. The two halves landed on different issues and neither looked
wrong on its own.

`secret/data/` is KV v2's ACL prefix and is absent from the path the code
passes, so matching `swarm-secret-client`'s spelling literally would have
granted nothing. Write-only: the controller mints these and never reads one
back, and a read capability would let it recover every agent's credentials
rather than only replace them.

The gate is the point. Two module-eval arms -- the grant exists and is not
a broader wildcard, and its capability list is pinned whole, because an
ADDED capability is what a presence check misses -- plus a test in path.rs
pinning MOUNT/AGENT_PREFIX and naming the nix file, since renaming either
constant is a silent 403 rather than a compile error.

setup.md carried two warnings this makes false: that nothing in the tree had
ever authenticated to the store, and that no deployment shape mints a leaf
whose CN reads swarm-controller. glue-bao-tls.nix has minted one since #3726
item 1.
This commit is contained in:
atlas 2026-09-09 00:46:01 +02:00
commit cc8fb0ee44
4 changed files with 62 additions and 8 deletions

View file

@ -94,14 +94,12 @@ Put the token's value at `services.hyperhive.deploy.bao.bootstrapTokenFile`
store's container reads it, writes the `swarm-controller` policy, enables the
cert auth method, and creates the `swarm-controller` role that attaches the two.
⚠️ **None of this has been run against a live store.** Nothing in the tree has
ever authenticated to OpenBao, so treat the block above as derived rather than
exercised — the grants come from `-output-policy`, not from a swarm that came
up on them.
⚠️ **The role it creates has nothing to present a certificate for yet.** No
deployment shape mints a leaf whose CN reads `swarm-controller`, so until one
does, the role stays provisioning that waits for a consumer.
⚠️ **This has been exercised once, and it did not go all the way through.** The
first real provision against a live store returned 403 (hyperhive#4124), so
treat the block above as derived-then-partly-tested rather than proven: the
grants still come from `-output-policy`, not from a swarm that came up on them.
What that 403 does _not_ tell you is whether the login or the write was
refused — `bao read auth/cert/certs/swarm-controller` separates the two.
**Delete the file once that has run.** The unit skips when it's absent, so a
host that has finished bootstrapping stops carrying the credential — and the

View file

@ -168,6 +168,19 @@ let
# `writeText`: a store path puts the grants behind a hash where
# ../module-eval.nix cannot read them, and a heredoc would make the HCL's
# indentation a function of this file's.
#
# The last grant is a different kind from the others: they let the controller
# bootstrap hives, this lets it write an agent's credentials. Two things about
# it do not read as they look.
#
# `secret/data/` is KV v2's ACL prefix, not part of the path the code passes:
# `swarm-secret-client` writes `swarm/agents/<agent>/...` under mount
# `secret`, and the engine inserts `data/`. Matching the code's spelling
# literally would grant nothing.
#
# Write-only on purpose. The controller mints these; nothing in its job reads
# one back, and a read capability here would let it recover every agent's
# credentials rather than merely replace them.
controllerPolicyText = ''
path "auth/cert/certs/hive-*" {
capabilities = ["create", "update", "read", "delete"]
@ -184,6 +197,10 @@ let
path "sys/policies/acl" {
capabilities = ["list"]
}
path "secret/data/swarm/agents/*" {
capabilities = ["create", "update"]
}
'';
# Every listener serves the same identity: they differ in which address

View file

@ -528,6 +528,34 @@ let
in
lib.hasInfix "sys/policies/acl/hive-*" s && !(lib.hasInfix "sys/policies/acl/*" s);
}
{
# The policy authorising this route lives in another file, and nothing
# else relates the grants to the paths the code actually writes.
#
# `secret/data/` is KV v2's ACL prefix; `swarm/agents` is
# `swarm_secret_client::path::AGENT_PREFIX`, whose value that crate
# pins in its own test.
name = "the controller may write agent credentials, and only under the agent prefix";
ok =
let
s = baoGrantHere.containers.swarm-bao.config.systemd.services.swarm-bao-controller-policy.script;
in
lib.hasInfix "secret/data/swarm/agents/*" s
&& !(lib.hasInfix "secret/data/*" s)
&& !(lib.hasInfix "path \"secret/*\"" s);
}
{
# Write-only is the property, not an accident of how it was typed: a
# `read` here would let the controller recover every agent's credentials
# instead of only replacing them. Pinned as the whole capability list,
# because an added capability is exactly what a presence check misses.
name = "the controller's grant on agent credentials is write-only";
ok =
let
s = baoGrantHere.containers.swarm-bao.config.systemd.services.swarm-bao-controller-policy.script;
in
lib.hasInfix "path \"secret/data/swarm/agents/*\" {\n capabilities = [\"create\", \"update\"]" s;
}
{
# The policy above grants paths under a mount nothing else creates, so
# the unit that writes the policy has to create it too — otherwise every

View file

@ -98,4 +98,15 @@ mod tests {
"got {e:?}"
);
}
#[test]
fn the_bao_policy_grants_exactly_these_two_values() {
// Renaming either constant is a silent 403 at provision time, not a
// compile error: the controller's grant spells them out in
// `nix/host-modules/swarm-bao.nix` (`controllerPolicyText`), which no
// Rust change can reach. Editing here means editing there, and
// `nix/module-eval.nix` asserts the other side of the same pair.
assert_eq!(MOUNT, "secret");
assert_eq!(AGENT_PREFIX, "swarm/agents");
}
}