vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

docs: callout xkey needs the store at every responder start; split-host order; replacing it

Browse source 
secrets.md now says that the responder reads the seed at every start, so
a store outage plus a responder restart refuses every client; that on a
split deployment the store's host deploys first; and that replacing the
key takes `bao kv metadata delete` on both paths, since a soft delete
keeps the metadata and the create is then refused. It also drops the
claim that every token reads `swarm/public/`.
This commit is contained in:
atlas 2026-10-05 22:13:59 +02:00 • committed by mara
commit 1ac34a4ad6
1 changed files with 37 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

49
docs/swarm/secrets.md
View file

@ -161,12 +161,12 @@ account or on the wire can read the request or the user JWT minted for it.
You configure nothing about the key: it lives in the swarm secret store and You configure nothing about the key: it lives in the swarm secret store and
comes into being on the queue's first boot. comes into being on the queue's first boot.
- **Seed** at `secret/swarm/services/swarm-nats/callout/xkey-seed`. Only the - **Seed** at `secret/swarm/services/swarm-nats/callout/xkey-seed`.
`swarm-nats-auth` role reads it, and it may create the path but never - **Public half** at `secret/swarm/public/swarm-nats/callout-xkey`.
replace it.
- **Public half** at `secret/swarm/public/swarm-nats/callout-xkey`. Every token Only the `swarm-nats-auth` role reads either path, and it may create each but
the store issues may read `swarm/public/`; see never replace it. The policy of `swarm-secret-publisher`, which writes the rest
[`bao.md`](bao.md#what-every-token-may-read). of `swarm/services/`, denies it the seed's path.
On the queue's host, `swarm-bao-nats-callout-xkey` runs before the On the queue's host, `swarm-bao-nats-callout-xkey` runs before the
`swarm-nats` container, under the responder's store leaf: `swarm-nats` container, under the responder's store leaf:
@ -182,15 +182,40 @@ On the queue's host, `swarm-bao-nats-callout-xkey` runs before the
A responder that can't read it doesn't start, rather than answering A responder that can't read it doesn't start, rather than answering
requests it can't open. requests it can't open.
When the store is down at boot, the fetch unit fails and retries for a day. The store has to be reachable each time the responder starts. When it's down,
The server starts with the key fetched last time, or not at all on a first the fetch unit fails and retries for a day. The server starts with the key
boot. The retry that lands the file starts or restarts the server, and the fetched last time, or not at all on a first boot. A responder that starts
responder comes up with it. while the store is down exits and retries every 5 seconds, and until one
start reads the seed the queue refuses every client, OIDC ones included. The
retry that lands the file starts or restarts the server, and the responder
comes up with it.
Where the queue and the store run on different hosts, deploy the store's host
first. `swarm-bao-nats-auth-policy` there writes the role's grants on the two
paths. A queue host deployed before it can't read or create the key, so its
fetch fails and the server stays down until you deploy the store's host. If
that takes longer than the fetch's day of retries, run
`systemctl restart swarm-bao-nats-callout-xkey` on the queue's host.
Two starts that create the key at once can't both win: each write is a Two starts that create the key at once can't both win: each write is a
create that fails once the path holds anything, and the start that lost reads create that fails once the path holds anything, and the start that lost reads
the winner's key back. To replace the key, delete both paths, then restart the winner's key back.
`swarm-bao-nats-callout-xkey`.
To replace the key, remove both paths with their metadata and every version,
then restart the fetch unit on the queue's host. Its restart of `nats.service`
restarts the responder too, which then loads the new seed:
```sh
bao kv metadata delete secret/swarm/services/swarm-nats/callout/xkey-seed
bao kv metadata delete secret/swarm/public/swarm-nats/callout-xkey
systemctl restart swarm-bao-nats-callout-xkey
```
`bao kv delete` isn't enough. It deletes the latest version but keeps the
path's metadata, so the store still counts the path as existing and refuses
the create that makes a new key. No principal hyperhive configures
may delete under `secret/metadata/`, so run the two deletes with a token
that may, such as the root token.
## Hive-level — one of each per hive ## Hive-level — one of each per hive