bao: enable cert auth and give the controller a role for its policy
The policy the granting unit already writes grants paths under `auth/cert/certs/*`, and nothing in the tree creates that mount. Every certificate login therefore fails against a path that is not there -- the controller's own, and the per-hive ones it is meant to issue against the same mount. Same unit, same bootstrap token: check whether cert auth is mounted, enable it if not, then write a role binding CN `swarm-controller` to the `swarm-controller` policy. Idempotency is a read rather than a tolerated error. `auth enable` fails on an existing mount, and recognising that would tie a rebuild to an error string no run of this store has ever produced, so the unit asks `bao auth list` and mounts only on absence. That read is why the token policy in setup.md gains `sys/auth`. Each grant came from `bao <cmd> -output-policy`, which prints what a command requires without running it -- the same way controllerPolicyText was derived. Enabling an auth method needs `sudo` on `sys/auth/cert`, which the documented token did not have. Gated on `clientCaFile`, not on the token alone: `swarm-bao-certs` installs `client-ca.pem` only under that condition, and a role's `certificate=` has to name a real CA. The policy write, which needs no CA, is unchanged in that case. Nothing can present a certificate for this role yet -- the only client leaf the tree mints carries CN = the hive's name -- and none of this has been run against a live store. Both are stated in setup.md.
This commit is contained in:
parent
df18d3d4d7
commit
8bd41fd84f
3 changed files with 107 additions and 5 deletions
|
|
@ -62,11 +62,27 @@ authenticate until some role exists — this token is what breaks that cycle,
|
||||||
and it's the only step that needs the root token.
|
and it's the only step that needs the root token.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# A policy that can write exactly one policy, and nothing else.
|
# Exactly the four grants the bootstrap unit needs, and nothing else. Each was
|
||||||
|
# derived with `bao <cmd> -output-policy`, which prints what a command requires
|
||||||
|
# without running it.
|
||||||
bao policy write swarm-bootstrap - <<'EOF'
|
bao policy write swarm-bootstrap - <<'EOF'
|
||||||
path "sys/policies/acl/swarm-controller" {
|
path "sys/policies/acl/swarm-controller" {
|
||||||
capabilities = ["create", "update"]
|
capabilities = ["create", "update"]
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Cert auth is a mount, and nothing has created it yet: reading `sys/auth` is
|
||||||
|
# how the unit checks, and `sudo` is what enabling one costs.
|
||||||
|
path "sys/auth" {
|
||||||
|
capabilities = ["read"]
|
||||||
|
}
|
||||||
|
|
||||||
|
path "sys/auth/cert" {
|
||||||
|
capabilities = ["create", "update", "sudo"]
|
||||||
|
}
|
||||||
|
|
||||||
|
path "auth/cert/certs/swarm-controller" {
|
||||||
|
capabilities = ["create", "update"]
|
||||||
|
}
|
||||||
EOF
|
EOF
|
||||||
|
|
||||||
# A token holding it. `-orphan` so it outlives the session that made it.
|
# A token holding it. `-orphan` so it outlives the session that made it.
|
||||||
|
|
@ -75,7 +91,17 @@ bao token create -policy=swarm-bootstrap -ttl=24h -orphan -display-name=swarm-bo
|
||||||
|
|
||||||
Put the token's value at `services.hyperhive.deploy.bao.bootstrapTokenFile`
|
Put the token's value at `services.hyperhive.deploy.bao.bootstrapTokenFile`
|
||||||
(all-local names that path for you), then rebuild. A one-shot unit inside the
|
(all-local names that path for you), then rebuild. A one-shot unit inside the
|
||||||
store's container reads it and writes the `swarm-controller` policy.
|
store's container reads it, writes the `swarm-controller` policy, enables the
|
||||||
|
cert auth method, and creates the `swarm-controller` role that attaches the two.
|
||||||
|
|
||||||
|
⚠️ **None of this has been run against a live store.** Nothing in the tree has
|
||||||
|
ever authenticated to OpenBao, so treat the block above as derived rather than
|
||||||
|
exercised — the grants come from `-output-policy`, not from a swarm that came
|
||||||
|
up on them.
|
||||||
|
|
||||||
|
⚠️ **The role it creates has nothing to present a certificate for yet.** Minting
|
||||||
|
a leaf whose CN is `swarm-controller` is not wired up in any deployment shape;
|
||||||
|
until it is, the role is provisioning waiting for a consumer.
|
||||||
|
|
||||||
**Delete the file once that has run.** The unit skips when it's absent, so a
|
**Delete the file once that has run.** The unit skips when it's absent, so a
|
||||||
host that has finished bootstrapping stops carrying the credential — and the
|
host that has finished bootstrapping stops carrying the credential — and the
|
||||||
|
|
|
||||||
|
|
@ -150,10 +150,15 @@ let
|
||||||
bootstrapTokenDir =
|
bootstrapTokenDir =
|
||||||
if haveBootstrapToken then builtins.dirOf baoDeploy.bootstrapTokenFile else null;
|
if haveBootstrapToken then builtins.dirOf baoDeploy.bootstrapTokenFile else null;
|
||||||
|
|
||||||
# The name both ends must agree on: whoever later gives the controller a
|
# The name both ends must agree on: the cert-auth role below attaches this
|
||||||
# cert-auth role attaches this policy by spelling it the same way.
|
# policy by spelling it the same way, and is itself named after it.
|
||||||
controllerPolicyName = "swarm-controller";
|
controllerPolicyName = "swarm-controller";
|
||||||
|
|
||||||
|
# The subject the controller's certificate must carry. Cert auth matches on
|
||||||
|
# the CN, so this is an interface rather than a label: a leaf signed by the
|
||||||
|
# right CA but minted with any other subject cannot authenticate.
|
||||||
|
controllerCn = "swarm-controller";
|
||||||
|
|
||||||
# Derived with `bao write -output-policy`, which short-circuits the request
|
# Derived with `bao write -output-policy`, which short-circuits the request
|
||||||
# and needs no server, rather than written from memory.
|
# and needs no server, rather than written from memory.
|
||||||
#
|
#
|
||||||
|
|
@ -999,7 +1004,7 @@ in
|
||||||
# where the store is reachable without a client certificate — which
|
# where the store is reachable without a client certificate — which
|
||||||
# is the point, since no role exists yet to issue one against.
|
# is the point, since no role exists yet to issue one against.
|
||||||
systemd.services.swarm-bao-controller-policy = lib.mkIf haveBootstrapToken {
|
systemd.services.swarm-bao-controller-policy = lib.mkIf haveBootstrapToken {
|
||||||
description = "write the swarm controller's bao policy";
|
description = "write the swarm controller's bao policy and cert-auth role";
|
||||||
after = [ "openbao.service" ];
|
after = [ "openbao.service" ];
|
||||||
wantedBy = [ "multi-user.target" ];
|
wantedBy = [ "multi-user.target" ];
|
||||||
path = [
|
path = [
|
||||||
|
|
@ -1034,6 +1039,37 @@ in
|
||||||
# than failing on one that already exists.
|
# than failing on one that already exists.
|
||||||
printf '%s' ${lib.escapeShellArg controllerPolicyText} |
|
printf '%s' ${lib.escapeShellArg controllerPolicyText} |
|
||||||
bao policy write ${lib.escapeShellArg controllerPolicyName} -
|
bao policy write ${lib.escapeShellArg controllerPolicyName} -
|
||||||
|
''
|
||||||
|
+ lib.optionalString (baoDeploy.clientCaFile != null) ''
|
||||||
|
|
||||||
|
# The policy above grants paths under `auth/cert/`, and nothing
|
||||||
|
# in this tree creates that mount. Without this, the grant names
|
||||||
|
# a location that does not exist and every certificate login
|
||||||
|
# fails — the controller's own, and the per-hive ones it later
|
||||||
|
# issues against the same mount.
|
||||||
|
#
|
||||||
|
# Asked rather than attempted: `auth enable` errors on a mount
|
||||||
|
# that already exists, and recognising that would tie a rebuild
|
||||||
|
# to an error string we have never seen this store emit.
|
||||||
|
mounted="$(bao auth list -format=json)"
|
||||||
|
case "$mounted" in
|
||||||
|
*'"cert/"'*) ;;
|
||||||
|
*) bao auth enable cert ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
# `certificate=` is the CA, so this role trusts every leaf that
|
||||||
|
# CA signed and `allowed_common_names` is the whole of what
|
||||||
|
# narrows it to one identity. ⚠️ The same CA signs each hive's
|
||||||
|
# reader leaf with CN = the hive's name, so a hive named
|
||||||
|
# `${controllerCn}` would satisfy this role.
|
||||||
|
#
|
||||||
|
# Named outside the `hive-*` namespace the policy grants, so the
|
||||||
|
# controller cannot rewrite the role that constrains it.
|
||||||
|
bao write auth/cert/certs/${lib.escapeShellArg controllerPolicyName} \
|
||||||
|
certificate=@${tlsDir}/client-ca.pem \
|
||||||
|
allowed_common_names=${lib.escapeShellArg controllerCn} \
|
||||||
|
token_policies=${lib.escapeShellArg controllerPolicyName} \
|
||||||
|
display_name=${lib.escapeShellArg controllerCn}
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
|
||||||
|
|
@ -254,6 +254,15 @@ let
|
||||||
baoGrantNoStore = hive {
|
baoGrantNoStore = hive {
|
||||||
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
|
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
|
||||||
};
|
};
|
||||||
|
# The store and the token, with no CA to trust. `mkForce` because the PKI
|
||||||
|
# glue supplies one by default here — this is the deployment that brings its
|
||||||
|
# own certificates and has not named the authority yet, and it separates
|
||||||
|
# "the grant unit runs" from "cert auth can be set up".
|
||||||
|
baoGrantNoClientCa = hive {
|
||||||
|
deploy.bao.enable = true;
|
||||||
|
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
|
||||||
|
deploy.bao.clientCaFile = lib.mkForce null;
|
||||||
|
};
|
||||||
|
|
||||||
baoNames = machine: machine.services.hyperhive.gateway.localNames;
|
baoNames = machine: machine.services.hyperhive.gateway.localNames;
|
||||||
|
|
||||||
|
|
@ -505,6 +514,37 @@ let
|
||||||
in
|
in
|
||||||
lib.hasInfix "sys/policies/acl/hive-*" s && !(lib.hasInfix "sys/policies/acl/*" s);
|
lib.hasInfix "sys/policies/acl/hive-*" s && !(lib.hasInfix "sys/policies/acl/*" s);
|
||||||
}
|
}
|
||||||
|
{
|
||||||
|
# The policy above grants paths under a mount nothing else creates, so
|
||||||
|
# the unit that writes the policy has to create it too — otherwise every
|
||||||
|
# certificate login fails against a path that is not there.
|
||||||
|
name = "the granting unit creates the cert auth mount and the controller's role";
|
||||||
|
ok =
|
||||||
|
let
|
||||||
|
s = baoGrantHere.containers.swarm-bao.config.systemd.services.swarm-bao-controller-policy.script;
|
||||||
|
in
|
||||||
|
lib.hasInfix "bao auth enable cert" s
|
||||||
|
&& lib.hasInfix "auth/cert/certs/swarm-controller" s
|
||||||
|
&& lib.hasInfix "/var/lib/swarm-bao-tls/client-ca.pem" s;
|
||||||
|
}
|
||||||
|
{
|
||||||
|
# The arm that makes the one above mean something. A role's trust anchor
|
||||||
|
# is the CA, so with none named there is nothing to write — and the
|
||||||
|
# policy write, which needs no CA, must survive that.
|
||||||
|
#
|
||||||
|
# ⚠️ Matched on the COMMANDS, not on `auth/cert/certs`: the policy text is
|
||||||
|
# embedded in this same script and grants that very path, so the shorter
|
||||||
|
# infix is present either way and the arm could never fail.
|
||||||
|
name = "with no client CA the unit still writes the policy and skips the role";
|
||||||
|
ok =
|
||||||
|
let
|
||||||
|
s =
|
||||||
|
baoGrantNoClientCa.containers.swarm-bao.config.systemd.services.swarm-bao-controller-policy.script;
|
||||||
|
in
|
||||||
|
lib.hasInfix "bao policy write" s
|
||||||
|
&& !(lib.hasInfix "bao auth enable cert" s)
|
||||||
|
&& !(lib.hasInfix "client-ca.pem" s);
|
||||||
|
}
|
||||||
{
|
{
|
||||||
# The arm that makes the one above mean something, and the property the
|
# The arm that makes the one above mean something, and the property the
|
||||||
# host-side half depends on: no store here, so no bind mount and no unit.
|
# host-side half depends on: no store here, so no bind mount and no unit.
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue