Compare commits
3 changed files with 16 additions and 96 deletions
|
|
@ -75,7 +75,7 @@ authenticate until some role exists — this token is what breaks that cycle,
|
||||||
and it's the only step that needs the root token.
|
and it's the only step that needs the root token.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Exactly the six grants the bootstrap unit needs, and nothing else. Each was
|
# Exactly the four grants the bootstrap unit needs, and nothing else. Each was
|
||||||
# derived with `bao <cmd> -output-policy`, which prints what a command requires
|
# derived with `bao <cmd> -output-policy`, which prints what a command requires
|
||||||
# without running it.
|
# without running it.
|
||||||
bao policy write swarm-bootstrap - <<'EOF'
|
bao policy write swarm-bootstrap - <<'EOF'
|
||||||
|
|
@ -96,17 +96,6 @@ path "sys/auth/cert" {
|
||||||
path "auth/cert/certs/swarm-controller" {
|
path "auth/cert/certs/swarm-controller" {
|
||||||
capabilities = ["create", "update"]
|
capabilities = ["create", "update"]
|
||||||
}
|
}
|
||||||
|
|
||||||
# The KV engine the controller writes agent credentials through — also absent
|
|
||||||
# on a fresh store, and checked the same way. No `sudo` here, unlike the auth
|
|
||||||
# mount above: enabling a secrets engine does not ask for it.
|
|
||||||
path "sys/mounts" {
|
|
||||||
capabilities = ["read"]
|
|
||||||
}
|
|
||||||
|
|
||||||
path "sys/mounts/secret" {
|
|
||||||
capabilities = ["create", "update"]
|
|
||||||
}
|
|
||||||
EOF
|
EOF
|
||||||
|
|
||||||
# A token holding it. `-orphan` so it outlives the session that made it.
|
# A token holding it. `-orphan` so it outlives the session that made it.
|
||||||
|
|
@ -116,47 +105,23 @@ bao token create -policy=swarm-bootstrap -ttl=24h -orphan -display-name=swarm-bo
|
||||||
Put the token's value at `services.hyperhive.deploy.bao.bootstrapTokenFile`
|
Put the token's value at `services.hyperhive.deploy.bao.bootstrapTokenFile`
|
||||||
(all-local names that path for you), then rebuild. A one-shot unit **on the
|
(all-local names that path for you), then rebuild. A one-shot unit **on the
|
||||||
host** reads it, writes the `swarm-controller` policy, enables the cert auth
|
host** reads it, writes the `swarm-controller` policy, enables the cert auth
|
||||||
method, mounts the KV engine the controller stores credentials in, and creates
|
method, and creates the `swarm-controller` role that attaches the two. It runs
|
||||||
the `swarm-controller` role that attaches policy to certificate. It runs there
|
there because every API listener demands a client certificate, and the host is
|
||||||
because every API listener demands a client certificate, and the host is the
|
the side that has one.
|
||||||
side that has one.
|
|
||||||
|
|
||||||
⏱️ **Expect the first attempt to fail if you rebuilt into this.** A rebuild
|
⚠️ **The grants above are derived, not proven.** They come from
|
||||||
restarts the store, and the unit races it — the store answers `local node not
|
`-output-policy`, not from a swarm that came up on them. The unit has run twice
|
||||||
active` until it finishes coming up. It retries every 30s and the second
|
against a live store and failed both times for reasons of its own — it could
|
||||||
attempt is the one that usually lands. Nothing to do.
|
not reach the store from where it then ran — so neither run exercised these
|
||||||
|
grants at all.
|
||||||
|
|
||||||
**Confirm with `systemctl status swarm-bao-controller-policy`**, which wants no
|
**Delete the file only once `bao read auth/cert/certs/swarm-controller` returns
|
||||||
token — a successful run logs `Uploaded policy`, `Enabled cert auth method` and
|
the role.** The unit skips when the token is absent, so a host that has finished
|
||||||
`Data written to: auth/cert/certs/swarm-controller`. ⚠️ Do _not_ reach for `bao
|
bootstrapping stops carrying the credential — but deleting it before the role
|
||||||
read auth/cert/…` to check: the host's `bao` wrapper carries an address, a CA
|
|
||||||
and a client certificate but deliberately **no token**, so that read answers
|
|
||||||
`403` whether or not the role exists.
|
|
||||||
|
|
||||||
**Delete the token file only once that unit has succeeded.** It skips when the
|
|
||||||
token is absent, so a host that has finished bootstrapping stops carrying the
|
|
||||||
credential — but deleting it before the role
|
|
||||||
exists leaves the unit skipping forever with nothing to show for it, and looks
|
exists leaves the unit skipping forever with nothing to show for it, and looks
|
||||||
exactly like a store that was never bootstrapped. The TTL above means a
|
exactly like a store that was never bootstrapped. The TTL above means a
|
||||||
forgotten one expires rather than lingering.
|
forgotten one expires rather than lingering.
|
||||||
|
|
||||||
<details><summary>Already bootstrapped before the KV mount existed?</summary>
|
|
||||||
|
|
||||||
A store bootstrapped by an earlier version has the policy, the auth method and
|
|
||||||
the role, but no `secret/` engine — the controller's first credential write
|
|
||||||
answers `no handler for route "secret/data/…"`. The bootstrap token can't fix it
|
|
||||||
either: the policy it was minted from names nothing under `sys/mounts`. Mount it
|
|
||||||
once with the root token from `init`:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo bash -c 'BAO_TOKEN="<root token>" bao secrets enable -path=secret kv-v2'
|
|
||||||
```
|
|
||||||
|
|
||||||
No rebuild needed — the unit's own check finds the mount on its next run and
|
|
||||||
leaves it alone.
|
|
||||||
|
|
||||||
</details>
|
|
||||||
|
|
||||||
Whether anything more is needed depends on
|
Whether anything more is needed depends on
|
||||||
`services.hyperhive.deploy.bao.seal`:
|
`services.hyperhive.deploy.bao.seal`:
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -198,18 +198,11 @@ let
|
||||||
capabilities = ["list"]
|
capabilities = ["list"]
|
||||||
}
|
}
|
||||||
|
|
||||||
path "${credentialMountPath}/data/swarm/agents/*" {
|
path "secret/data/swarm/agents/*" {
|
||||||
capabilities = ["create", "update"]
|
capabilities = ["create", "update"]
|
||||||
}
|
}
|
||||||
'';
|
'';
|
||||||
|
|
||||||
# The KV v2 engine the controller writes agent credentials through. Named
|
|
||||||
# once because the grant above and the `secrets enable` in the bootstrap unit
|
|
||||||
# have to agree: a policy pointing at a mount nobody created is precisely the
|
|
||||||
# failure this pair exists to avoid. `swarm-secret-client` pins the same
|
|
||||||
# value on the reading side.
|
|
||||||
credentialMountPath = "secret";
|
|
||||||
|
|
||||||
# Every listener serves the same identity: they differ in which address
|
# Every listener serves the same identity: they differ in which address
|
||||||
# they answer on, not in who they are. Client verification is separate and
|
# they answer on, not in who they are. Client verification is separate and
|
||||||
# optional — a store with no `clientCaFile` still serves TLS, it just does
|
# optional — a store with no `clientCaFile` still serves TLS, it just does
|
||||||
|
|
@ -911,23 +904,6 @@ in
|
||||||
# than failing on one that already exists.
|
# than failing on one that already exists.
|
||||||
printf '%s' ${lib.escapeShellArg controllerPolicyText} |
|
printf '%s' ${lib.escapeShellArg controllerPolicyText} |
|
||||||
bao policy write ${lib.escapeShellArg controllerPolicyName} -
|
bao policy write ${lib.escapeShellArg controllerPolicyName} -
|
||||||
|
|
||||||
# The policy grants paths under `${credentialMountPath}/`, and a
|
|
||||||
# fresh store has no such engine — only a dev-mode one does. Writing
|
|
||||||
# to an unmounted path answers "route entry not found", so the
|
|
||||||
# controller's first credential write fails against a grant that
|
|
||||||
# reads as correct.
|
|
||||||
#
|
|
||||||
# Outside the client-CA block below on purpose: this mount is what
|
|
||||||
# the controller writes *through*, independent of who may log in.
|
|
||||||
#
|
|
||||||
# Asked rather than attempted, same as the auth mount: `secrets
|
|
||||||
# enable` errors on a path already in use.
|
|
||||||
mounts="$(bao secrets list -format=json)"
|
|
||||||
case "$mounts" in
|
|
||||||
*'"${credentialMountPath}/"'*) ;;
|
|
||||||
*) bao secrets enable -path=${credentialMountPath} kv-v2 ;;
|
|
||||||
esac
|
|
||||||
''
|
''
|
||||||
+ lib.optionalString (baoDeploy.clientCaFile != null) ''
|
+ lib.optionalString (baoDeploy.clientCaFile != null) ''
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -613,22 +613,6 @@ let
|
||||||
&& lib.hasInfix "auth/cert/certs/swarm-controller" s
|
&& lib.hasInfix "auth/cert/certs/swarm-controller" s
|
||||||
&& lib.hasInfix "/var/lib/swarm-bao-tls/client-ca.pem" s;
|
&& lib.hasInfix "/var/lib/swarm-bao-tls/client-ca.pem" s;
|
||||||
}
|
}
|
||||||
{
|
|
||||||
# Same shape as the cert mount above, for the engine the controller
|
|
||||||
# writes credentials through: a fresh store has no `secret/`, so the
|
|
||||||
# grant would name a mount nobody created and the first write would 404.
|
|
||||||
#
|
|
||||||
# ⚠️ Matched on the COMMAND, for the reason the no-client-CA case below
|
|
||||||
# spells out: the policy text is embedded in this same script and grants
|
|
||||||
# `secret/data/...`, so any arm keyed on the *path* is satisfied either
|
|
||||||
# way and could never fail.
|
|
||||||
name = "the granting unit creates the KV mount the controller writes through";
|
|
||||||
ok =
|
|
||||||
let
|
|
||||||
s = baoGrantHere.systemd.services.swarm-bao-controller-policy.script;
|
|
||||||
in
|
|
||||||
lib.hasInfix "bao secrets enable -path=secret kv-v2" s;
|
|
||||||
}
|
|
||||||
{
|
{
|
||||||
# Nothing asserted the PKI script before this, so a third leaf could be
|
# Nothing asserted the PKI script before this, so a third leaf could be
|
||||||
# added to it and every case still passed — measured, not assumed: the
|
# added to it and every case still passed — measured, not assumed: the
|
||||||
|
|
@ -796,12 +780,7 @@ let
|
||||||
in
|
in
|
||||||
lib.hasInfix "bao policy write" s
|
lib.hasInfix "bao policy write" s
|
||||||
&& !(lib.hasInfix "bao auth enable cert" s)
|
&& !(lib.hasInfix "bao auth enable cert" s)
|
||||||
&& !(lib.hasInfix "client-ca.pem" s)
|
&& !(lib.hasInfix "client-ca.pem" s);
|
||||||
# The KV mount is NOT part of what a missing client CA switches off:
|
|
||||||
# the controller writes through it whether or not anything can log in
|
|
||||||
# by certificate. Asserted here rather than trusted, because both
|
|
||||||
# steps live in the same script and one indentation level decides it.
|
|
||||||
&& lib.hasInfix "bao secrets enable -path=secret kv-v2" s;
|
|
||||||
}
|
}
|
||||||
{
|
{
|
||||||
# Reads the vhost's rendered `root`, not the option: the UI is served
|
# Reads the vhost's rendered `root`, not the option: the UI is served
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue