vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

swarm-bao: say "run the bootstrap step" when the bootstrap token is dead, not "sealed"

Browse source 
swarm-bao-granter-role used its token for `bao auth list` with no check,
so an expired, revoked or policy-less token in bootstrapTokenFile exited
2 with a raw 403 and no hint. It now checks whether the store is up when
that call fails: if it is, the token is at fault, and the unit prints the
one-time bootstrap step and exits 4. A missing token file is still a
ConditionPathExists skip, so the two read differently in the journal.

The twelve granterLogin units printed "the store is sealed or
unreachable" on a healthy store, because `bao status` exited 1 there:
the CLI resolves a token helper under $HOME before asking, systemd sets
no HOME for a unit without User=, and the fallback shells out to
`getent`/`sh`, neither of which is on the unit's PATH ("failed to get
token helper: error expanding config path "": exec: "sh": executable
file not found in $PATH"). The check now runs with HOME=/var/empty and
keeps its stderr, so a genuinely unreachable store says why. When the
store is up and the login is refused, the units now name
swarm-bao-granter-role as the unit that writes the missing role.

setup.md's post-step restart used 'swarm-bao-*-policy.service', which
misses swarm-bao-agent-pki. It now names that unit too, and a
module-eval case fails when the restart misses any unit that logs in as
the granter.

Refs #4704
This commit is contained in:
atlas 2026-09-28 11:03:08 +02:00
commit a2b4acfb6a
3 changed files with 80 additions and 8 deletions
Download patch file Download diff file Expand all files Collapse all files

35
nix/host-modules/swarm-bao.nix
View file

@ -285,17 +285,24 @@ let
]
);
# Exit 0 only on a reachable, unsealed store; its error, if any, is appended
# to `$err`. `status` needs no token, but the CLI resolves a token helper
# under `$HOME` before it asks, and systemd sets no `HOME` for a unit without
# `User=`: there it exits 1 on a healthy store. An empty home holds no helper
# config and no token.
baoStoreUp = ''HOME=/var/empty bao status >/dev/null 2>>"$err"'';
# The login every granting unit starts with. It FAILS rather than skips: a
# grant that was not written is otherwise invisible until whatever needs it
# fails somewhere else. `bao status` exits 0 only on a reachable, unsealed
# store, which separates "the granter is not set up" from "retry later";
# either way bao's own message follows.
# fails somewhere else. `baoStoreUp` separates "the granter is not set up"
# from "retry later"; either way bao's own message follows.
granterLogin = ''
err="$(mktemp)"
trap 'rm -f "$err"' EXIT
if ! BAO_TOKEN="$(bao login -method=cert -token-only 2>"$err")"; then
if bao status >/dev/null 2>&1; then
if ${baoStoreUp}; then
echo ${lib.escapeShellArg "the store is unsealed but refused the granter's certificate (CN ${granterCn}): the ${granterPolicyName} role is not set up."} >&2
${lib.optionalString haveBootstrapToken ''echo "swarm-bao-granter-role writes that role; journalctl -u swarm-bao-granter-role says why it has not." >&2''}
echo "one-time step, as root on this host (docs/getting-started/setup.md):" >&2
${lib.concatMapStringsSep "\n" (l: "echo ${lib.escapeShellArg " ${l}"} >&2") granterSetupSteps}
else
@ -2379,13 +2386,31 @@ in
BAO_TOKEN="$(cat ${lib.escapeShellArg baoDeploy.bootstrapTokenFile})"
export BAO_TOKEN
err="$(mktemp)"
trap 'rm -f "$err"' EXIT
# Every cert-auth role in this file lives under `auth/cert/`, and
# nothing else creates that mount.
#
# Asked rather than attempted: `auth enable` errors on a mount
# that already exists, and recognising that would tie a rebuild
# to an error string we have never seen this store emit.
mounted="$(bao auth list -format=json)"
#
# The token's first use, and the bootstrap policy grants this read,
# so a refusal from a store that is up is the token's: expired,
# revoked, or minted without that policy.
if ! mounted="$(bao auth list -format=json 2>"$err")"; then
if ${baoStoreUp}; then
echo ${lib.escapeShellArg "the store is unsealed but refused the bootstrap token in ${baoDeploy.bootstrapTokenFile}: it has expired, been revoked, or does not carry the bao-bootstrap policy."} >&2
echo "one-time step, as root on this host (docs/getting-started/setup.md):" >&2
${lib.concatMapStringsSep "\n" (l: "echo ${lib.escapeShellArg " ${l}"} >&2") granterSetupSteps}
cat "$err" >&2
exit 4
fi
echo "the store is sealed or unreachable; retrying." >&2
cat "$err" >&2
exit 1
fi
case "$mounted" in
*'"cert/"'*) ;;
*) bao auth enable cert ;;