From c21ec7719da9d0891fc94117eede7edf3969b218 Mon Sep 17 00:00:00 2001 From: atlas Date: Wed, 23 Sep 2026 12:14:58 +0200 Subject: [PATCH] swarm: tests and docs for the queue-credential revocation Pins the three properties the revocation rests on and cannot check against a store: that only `Destroyed` revokes (a revocation on `Offline` or `Paused` would give an agent that stops and never restarts), that a 404 is absence while a 403 stays a failure, and that the delete addresses `secret/metadata/` -- the path that takes every version, which is the string the grant has to match. docs/swarm/credentials.md gains the revocation section and its table cell stops describing the deletion as something an operator does by hand. --- docs/swarm/credentials.md | 48 ++++++++++++++++++-------- swarm-controller/src/agent_identity.rs | 33 ++++++++++++++++++ swarm-controller/src/main.rs | 26 ++++++++++++++ swarm-secret-client/src/client.rs | 10 ++++-- 4 files changed, 101 insertions(+), 16 deletions(-) diff --git a/docs/swarm/credentials.md b/docs/swarm/credentials.md index 306f5b57..bc1bc7fa 100644 --- a/docs/swarm/credentials.md +++ b/docs/swarm/credentials.md @@ -59,20 +59,20 @@ of the cell says how. -| store path | minter | reader — pulls at runtime, holds in memory | automatic re-mint | automatic re-pull | -| ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `swarm/agents//matrix/main` | `swarm-controller`, with the swarm's appservice token, at agent creation and in a five-minute pass | the agent container itself, under the certificate its hive passed in | ✅ the pass re-mints when the stored token is missing, unknown to the homeserver, or someone else's | ✅ `hive-matrix-daemon` exits when the homeserver rejects its token, and a five-minute timer restarts it, which reads the store again | -| `swarm/agents//matrix/` | `swarm-controller` | the agent container itself, under the certificate its hive passed in | must be stated | must be stated | -| `swarm/controller/swarm-controller/matrix/appservice-token` | `swarm-matrix-ctl`, inside the `hive-matrix` container, once | `swarm-controller`, under its own certificate | ❌ `swarm-matrix-ctl` mints it once; the container keeps its copy and republishes it when the store's differs | ✅ the controller reads it on every five-minute matrix pass | -| `swarm/controller/swarm-controller/oidc/client` | authelia, at its first boot, where the controller registers its client; `swarm-secret-publish` copies it in | `swarm-controller`, under its own certificate, once at start | ❌ authelia mints it once. A re-mint is republished by `swarm-secret-publish`'s path unit | ❌ read once at start; the controller holds the old value until it restarts | -| `swarm/agents//bao-mtls` | the store's agent PKI mount (`deploy.bao.agentPkiMountPath`), which generates the key, at `swarm-controller`'s request at agent creation | `hive-c0re`, under the hive's own certificate, when it writes the agent's container config | ✅ `swarm-controller`'s five-minute pass re-issues a live agent's leaf once it's past half its validity (45 of 90 days, read from the certificate itself) | ❌ `hive-c0re` reads it when it writes the container config, so the agent presents a new leaf from its next start; the old leaf stays valid until it expires | -| `swarm/agents//queue` | `swarm-controller`, at agent creation | the agent container itself, under its own certificate — the identity it presents to the swarm queue, naming that one agent rather than its hive | ✅ `swarm-controller`'s five-minute pass re-mints a live agent's secret once it's 45 days old by `minted_at` on the stored object; a secret with no `minted_at` gets one stamped, value unchanged | ❌ fetched when the container starts. The queue checks the secret only at connect, so an open connection survives a re-mint, but a reconnect before the next restart is denied | -| `swarm/agents//forge-token` | `swarm-controller`, at agent creation and in a pass every 5 minutes over every agent with a store identity | the agent container itself, under its own certificate, fetched to `/run/hive-agent-forge-token/token` | ✅ the controller re-mints when the stored token is missing or no longer matches the forge (last eight characters and scopes) | ✅ the agent re-fetches on a 10-minute timer | -| `swarm/hives//matrix/appservice-token` | one minter, on the authelia host | the hive process that presents the token to its homeserver, under the hive's own certificate | must be stated | must be stated | -| `swarm/hives//matrix/sender-token` | `swarm-matrix-ctl`, in the `hive-matrix` container | `swarm-matrix-ctl` itself, under its own certificate, before it decides whether to mint, and hive-c0re's `stored_sender_token()`, under the hive's own certificate | must be stated | must be stated | -| `swarm/hives//queue/agent` | authelia | `swarm-bao-queue-agent` on the hive's host, under its own per-hive certificate; no agent's policy reaches it | must be stated | must be stated | -| `swarm/services//oidc/client` | authelia | the service process that presents the client secret, under the certificate of the host it runs on | must be stated | must be stated | -| _(not in the store)_ a hive's mTLS leaf | the store's own PKI, or an operator placing it by hand | its own client, off disk — the exception above, because it's what makes every other row's pull possible | must be stated | must be stated | +| store path | minter | reader — pulls at runtime, holds in memory | automatic re-mint | automatic re-pull | +| ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `swarm/agents//matrix/main` | `swarm-controller`, with the swarm's appservice token, at agent creation and in a five-minute pass | the agent container itself, under the certificate its hive passed in | ✅ the pass re-mints when the stored token is missing, unknown to the homeserver, or someone else's | ✅ `hive-matrix-daemon` exits when the homeserver rejects its token, and a five-minute timer restarts it, which reads the store again | +| `swarm/agents//matrix/` | `swarm-controller` | the agent container itself, under the certificate its hive passed in | must be stated | must be stated | +| `swarm/controller/swarm-controller/matrix/appservice-token` | `swarm-matrix-ctl`, inside the `hive-matrix` container, once | `swarm-controller`, under its own certificate | ❌ `swarm-matrix-ctl` mints it once; the container keeps its copy and republishes it when the store's differs | ✅ the controller reads it on every five-minute matrix pass | +| `swarm/controller/swarm-controller/oidc/client` | authelia, at its first boot, where the controller registers its client; `swarm-secret-publish` copies it in | `swarm-controller`, under its own certificate, once at start | ❌ authelia mints it once. A re-mint is republished by `swarm-secret-publish`'s path unit | ❌ read once at start; the controller holds the old value until it restarts | +| `swarm/agents//bao-mtls` | the store's agent PKI mount (`deploy.bao.agentPkiMountPath`), which generates the key, at `swarm-controller`'s request at agent creation | `hive-c0re`, under the hive's own certificate, when it writes the agent's container config | ✅ `swarm-controller`'s five-minute pass re-issues a live agent's leaf once it's past half its validity (45 of 90 days, read from the certificate itself) | ❌ `hive-c0re` reads it when it writes the container config, so the agent presents a new leaf from its next start; the old leaf stays valid until it expires | +| `swarm/agents//queue` | `swarm-controller`, at agent creation | the agent container itself, under its own certificate — the identity it presents to the swarm queue, naming that one agent rather than its hive | ✅ `swarm-controller`'s five-minute pass re-mints a live agent's secret once it's 45 days old by `minted_at` on the stored object; a secret with no `minted_at` gets one stamped, value unchanged. The pass skips agents declared `Destroyed` — declaring an agent destroyed deletes every version of the path instead, the undo of the mint rather than another one | ❌ fetched when the container starts. The queue checks the secret only at connect, so an open connection survives a re-mint, but a reconnect before the next restart is denied — including a reconnect after the credential was revoked | +| `swarm/agents//forge-token` | `swarm-controller`, at agent creation and in a pass every 5 minutes over every agent with a store identity | the agent container itself, under its own certificate, fetched to `/run/hive-agent-forge-token/token` | ✅ the controller re-mints when the stored token is missing or no longer matches the forge (last eight characters and scopes) | ✅ the agent re-fetches on a 10-minute timer | +| `swarm/hives//matrix/appservice-token` | one minter, on the authelia host | the hive process that presents the token to its homeserver, under the hive's own certificate | must be stated | must be stated | +| `swarm/hives//matrix/sender-token` | `swarm-matrix-ctl`, in the `hive-matrix` container | `swarm-matrix-ctl` itself, under its own certificate, before it decides whether to mint, and hive-c0re's `stored_sender_token()`, under the hive's own certificate | must be stated | must be stated | +| `swarm/hives//queue/agent` | authelia | `swarm-bao-queue-agent` on the hive's host, under its own per-hive certificate; no agent's policy reaches it | must be stated | must be stated | +| `swarm/services//oidc/client` | authelia | the service process that presents the client secret, under the certificate of the host it runs on | must be stated | must be stated | +| _(not in the store)_ a hive's mTLS leaf | the store's own PKI, or an operator placing it by hand | its own client, off disk — the exception above, because it's what makes every other row's pull possible | must be stated | must be stated | @@ -137,6 +137,26 @@ that makes a container require a credential it may not have.** A container whose credential is absent doesn't start — that's deliberate, and it's what makes the backfill a step rather than a suggestion. +**Revoking an agent's queue credential.** Declaring an agent `destroyed` — +`PUT /api/hives/{hive}/agents/{agent}/state` — deletes +`swarm/agents//queue` and every version it ever held, right after the +declaration lands. The credential is a bearer secret, so a soft delete would +leave the same value readable at an older version number; the controller +removes the path's metadata, which takes the versions with it. + +An agent recreated under the same name draws a fresh secret, because the +mint keeps an existing value only when it finds one at that path +(`agent_identity::mint_and_verify`, step 3) and the revocation left nothing +to find. + +Two things this deliberately doesn't do. It doesn't touch the agent's mTLS +leaf, its ACL document or its cert-auth role — the mint rewrites all three on +every run, so a re-created agent gets new ones regardless. And it doesn't +fail the destroy: the declaration is already published by the time the +revocation runs, so a store that refuses the delete gets a `swarm-controller` +log line at `error` naming the agent, and the teardown continues. Declare the +agent destroyed again to re-run the revocation. + **Who reads that row, and what happens to it.** `hive-c0re` reads it every time it writes an agent's container configuration (`lifecycle::agent_identity`), stages the certificate and its key `0600` diff --git a/swarm-controller/src/agent_identity.rs b/swarm-controller/src/agent_identity.rs index 3928fbe9..c36f1e6b 100644 --- a/swarm-controller/src/agent_identity.rs +++ b/swarm-controller/src/agent_identity.rs @@ -364,6 +364,39 @@ mod tests { assert!(!a.contains('='), "{a}"); } + /// Revocation undoes step 3 of the mint and nothing else, and both ends + /// spell the path through one function — the property that keeps a + /// revocation from missing the object it is meant to remove. + /// + /// The second half is what stops this being a tautology: the agent's + /// other published object, the mTLS leaf, is at a different path and is + /// deliberately left alone. Revoking both would take away the identity a + /// re-created agent is re-minted under, for a credential problem that is + /// only about the queue. + #[test] + fn revocation_names_the_path_the_mint_published_and_not_the_leaf_beside_it() { + use swarm_secret_client::{mtls, queue}; + + assert_eq!( + queue::agent_queue_path("atlas").expect("a plain name is legal"), + "swarm/agents/atlas/queue" + ); + assert_ne!( + queue::agent_queue_path("atlas").expect("legal"), + mtls::identity_path("atlas").expect("legal"), + ); + } + + /// A name the store must never be asked to delete under. `revoke_queue_credential` + /// builds its path with the same validating function the mint does, so a + /// traversal is refused before a request is made rather than addressing + /// some other principal's secret. + #[test] + fn a_traversal_never_becomes_a_revocation() { + swarm_secret_client::queue::agent_queue_path("../pr1ma") + .expect_err("a traversal is not a legal agent name"); + } + fn both(k: &str) -> Option { match k { ENV_AGENT_PKI_MOUNT => Some("pki-agents".to_owned()), diff --git a/swarm-controller/src/main.rs b/swarm-controller/src/main.rs index 32996cc7..445de332 100644 --- a/swarm-controller/src/main.rs +++ b/swarm-controller/src/main.rs @@ -3308,6 +3308,32 @@ mod tests { ); } + /// `Destroyed` is the one declaration that ends an agent's life, so it is + /// the one that revokes. The credential is a bearer secret: left behind, + /// anything that recovers it still authenticates as an agent that no + /// longer exists. + #[test] + fn declaring_an_agent_destroyed_revokes_its_queue_credential() { + assert!(crate::revokes_queue_credential( + swarm_queue_client::wanted::AgentState::Destroyed + )); + } + + /// The control, and the more expensive half to get wrong: a revocation on + /// `Offline` or `Paused` would give an agent that can be stopped and never + /// restarted. Every non-terminal state is listed rather than sampled, so + /// a state added later shows up here as well as in the match itself. + #[test] + fn no_state_an_agent_comes_back_from_revokes() { + use swarm_queue_client::wanted::AgentState; + for state in [AgentState::Up, AgentState::Offline, AgentState::Paused] { + assert!( + !crate::revokes_queue_credential(state), + "{state:?} is a state an agent returns from" + ); + } + } + /// The whole point of the node: a freshly created agent is declared /// `Paused`, so it does not start driving turns the moment its hive /// brings it up. Any other state here is the bug this fixes. diff --git a/swarm-secret-client/src/client.rs b/swarm-secret-client/src/client.rs index 98f2002b..7a80b89b 100644 --- a/swarm-secret-client/src/client.rs +++ b/swarm-secret-client/src/client.rs @@ -633,8 +633,14 @@ mod tests { /// /// The distinction is the whole point of the verb: the data path's delete /// leaves earlier versions readable, so revoking a secret that was ever - /// rewritten there would leave the old value recoverable. It is also a - /// separately-ACL'd path, which is why the grant had to gain a stanza. + /// rewritten there would leave the old value recoverable. + /// + /// Asserted on [`vaultrs`]'s own request type rather than on a call, since + /// there is no store to call: it pins the endpoint + /// `kv2::delete_metadata` targets, which is the string + /// `swarm-bao.nix`'s grant has to match. A dependency bump that moved it + /// would otherwise surface as a 403 at the first revocation, far from + /// here. #[test] fn a_revocation_targets_every_version_and_not_just_the_newest() { use rustify::endpoint::Endpoint as _;