vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

Compare commits

base: vinzenz:642be5767897e9788cfe995d1dbb8657cc116969
Branches Tags
vinzenz:main
..
compare: vinzenz:88c386c96c143599c3300b9235650866082120e6
Branches Tags
vinzenz:main
6 changed files with 26 additions and 333 deletions
Download patch file Download diff file Expand all files Collapse all files

24
docs/swarm/credentials.md
View file

@ -60,13 +60,13 @@ of the cell says how.
<!-- vale write-good.Passive = NO -->
| store path | minter | reader — pulls at runtime, holds in memory | automatic re-mint | automatic re-pull |
| ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| ----------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `swarm/agents/<agent>/matrix/main` | `swarm-controller`, with the swarm's appservice token, at agent creation and in a five-minute pass | the agent container itself, under the certificate its hive passed in | ✅ the pass re-mints when the stored token is missing, unknown to the homeserver, or someone else's | ✅ `hive-matrix-daemon` exits when the homeserver rejects its token, and a five-minute timer restarts it, which reads the store again |
| `swarm/agents/<agent>/matrix/<account>` | `swarm-controller` | the agent container itself, under the certificate its hive passed in | must be stated | must be stated |
| `swarm/controller/swarm-controller/matrix/appservice-token` | `swarm-matrix-ctl`, inside the `hive-matrix` container, once | `swarm-controller`, under its own certificate | ❌ `swarm-matrix-ctl` mints it once; the container keeps its copy and republishes it when the store's differs | ✅ the controller reads it on every five-minute matrix pass |
| `swarm/controller/swarm-controller/oidc/client` | authelia, at its first boot, where the controller registers its client; `swarm-secret-publish` copies it in | `swarm-controller`, under its own certificate, once at start | ❌ authelia mints it once. A re-mint is republished by `swarm-secret-publish`'s path unit | ❌ read once at start; the controller holds the old value until it restarts |
| `swarm/agents/<agent>/bao-mtls` | the store's agent PKI mount (`deploy.bao.agentPkiMountPath`), which generates the key, at `swarm-controller`'s request at agent creation | `hive-c0re`, under the hive's own certificate, when it writes the agent's container config | ✅ `swarm-controller`'s five-minute pass re-issues a live agent's leaf once it's past half its validity (45 of 90 days, read from the certificate itself) | ❌ `hive-c0re` reads it when it writes the container config, so the agent presents a new leaf from its next start; the old leaf stays valid until it expires |
| `swarm/agents/<agent>/queue` | `swarm-controller`, at agent creation | the agent container itself, under its own certificate — the identity it presents to the swarm queue, naming that one agent rather than its hive | ✅ `swarm-controller`'s five-minute pass re-mints a live agent's secret once it's 45 days old by `minted_at` on the stored object; a secret with no `minted_at` gets one stamped, value unchanged. The pass skips agents declared `Destroyed` — declaring an agent destroyed deletes every version of the path instead, the undo of the mint rather than another one | ❌ fetched when the container starts. The queue checks the secret only at connect, so an open connection survives a re-mint, but a reconnect before the next restart is denied — including a reconnect after the credential was revoked |
| `swarm/agents/<agent>/queue` | `swarm-controller`, at agent creation | the agent container itself, under its own certificate — the identity it presents to the swarm queue, naming that one agent rather than its hive | ✅ `swarm-controller`'s five-minute pass re-mints a live agent's secret once it's 45 days old by `minted_at` on the stored object; a secret with no `minted_at` gets one stamped, value unchanged | ❌ fetched when the container starts. The queue checks the secret only at connect, so an open connection survives a re-mint, but a reconnect before the next restart is denied |
| `swarm/agents/<agent>/forge-token` | `swarm-controller`, at agent creation and in a pass every 5 minutes over every agent with a store identity | the agent container itself, under its own certificate, fetched to `/run/hive-agent-forge-token/token` | ✅ the controller re-mints when the stored token is missing or no longer matches the forge (last eight characters and scopes) | ✅ the agent re-fetches on a 10-minute timer |
| `swarm/hives/<hive>/matrix/appservice-token` | one minter, on the authelia host | the hive process that presents the token to its homeserver, under the hive's own certificate | must be stated | must be stated |
| `swarm/hives/<hive>/matrix/sender-token` | `swarm-matrix-ctl`, in the `hive-matrix` container | `swarm-matrix-ctl` itself, under its own certificate, before it decides whether to mint, and hive-c0re's `stored_sender_token()`, under the hive's own certificate | must be stated | must be stated |
@ -137,26 +137,6 @@ that makes a container require a credential it may not have.** A container
whose credential is absent doesn't start — that's deliberate, and it's what
makes the backfill a step rather than a suggestion.
**Revoking an agent's queue credential.** Declaring an agent `destroyed` —
`PUT /api/hives/{hive}/agents/{agent}/state` — deletes
`swarm/agents/<agent>/queue` and every version it ever held, right after the
declaration lands. The credential is a bearer secret, so a soft delete would
leave the same value readable at an older version number; the controller
removes the path's metadata, which takes the versions with it.
An agent recreated under the same name draws a fresh secret, because the
mint keeps an existing value only when it finds one at that path
(`agent_identity::mint_and_verify`, step 3) and the revocation left nothing
to find.
Two things this deliberately doesn't do. It doesn't touch the agent's mTLS
leaf, its ACL document or its cert-auth role — the mint rewrites all three on
every run, so a re-created agent gets new ones regardless. And it doesn't
fail the destroy: the declaration is already published by the time the
revocation runs, so a store that refuses the delete gets a `swarm-controller`
log line at `error` naming the agent, and the teardown continues. Declare the
agent destroyed again to re-run the revocation.
**Who reads that row, and what happens to it.** `hive-c0re` reads it every
time it writes an agent's container configuration
(`lifecycle::agent_identity`), stages the certificate and its key `0600`

26
nix/host-modules/swarm-bao.nix
View file

@ -362,16 +362,18 @@ let
# ../module-eval.nix cannot read them, and a heredoc would make the HCL's
# indentation a function of this file's.
#
# The credential-write grant below (unlike the bootstrap ones above it) has
# three things about its paths that do not read as written. `secret/data/` is
# KV v2's ACL prefix, not part of the path the code passes:
# `swarm-secret-client` writes `swarm/agents/<agent>/...` under mount `secret`,
# and the engine inserts `data/` — matching the code's spelling literally would
# grant nothing. `read` is required too: `mint_and_verify` reads a credential
# back before writing so a re-run keeps the value a live agent already holds
# instead of rotating it. `metadata/` is the revocation half: `delete` on
# `data/` only soft-deletes the newest version, and `+` being one path segment
# keeps this to the queue leaf alone.
# The last grant is a different kind from the others: they let the controller
# bootstrap hives, this lets it write an agent's credentials. Two things about
# it do not read as they look.
#
# `secret/data/` is KV v2's ACL prefix, not part of the path the code passes:
# `swarm-secret-client` writes `swarm/agents/<agent>/...` under mount
# `secret`, and the engine inserts `data/`. Matching the code's spelling
# literally would grant nothing.
#
# `read` too: `mint_and_verify` reads a credential back before writing so a
# re-run keeps the value a live agent already holds instead of rotating it —
# the read is required, not incidental.
#
# The swarm appservice token and its own OIDC client secret, read-only: it
# uses both and writes neither. matrix-ctl publishes the token
@ -400,10 +402,6 @@ let
capabilities = ["create", "read", "update"]
}
path "${credentialMountPath}/metadata/swarm/agents/+/queue" {
capabilities = ["delete"]
}
path "${credentialMountPath}/data/${swarmAppserviceTokenLeaf}" {
capabilities = ["read"]
}

23
nix/module-eval/bao-grants.nix
View file

@ -1147,29 +1147,6 @@ let
in
lib.hasInfix "path \"secret/data/swarm/agents/*\" {\n capabilities = [\"create\", \"read\", \"update\"]" s;
}
{
# Revocation, and the reason it is a stanza of its own: `delete` on the
# `data/` path soft-deletes the newest version and leaves earlier ones
# readable, so a credential the mint had ever rewritten would survive it.
# `metadata/` is the path that removes every version, and the store ACLs
# it separately — without this grant the revocation is a 403 and a
# destroyed agent's credential stays valid.
#
# `+` is one path segment, so this reaches `swarm/agents/<agent>/queue`
# and nothing else an agent holds; `agents/*` would reach every object
# under the prefix, which `revoke_queue_credential` never asks the store
# to delete. Pinned as the whole capability list too: `read` or `list`
# here would let the controller read back the queue-secret version
# history it is meant only to delete.
name = "the controller may revoke an agent's queue credential, and only that one";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-controller-policy.script;
in
lib.hasInfix "path \"secret/metadata/swarm/agents/+/queue\" {\n capabilities = [\"delete\"]" s
&& !(lib.hasInfix "secret/metadata/swarm/agents/*" s)
&& !(lib.hasInfix "secret/metadata/*" s);
}
{
# The swarm appservice token is a homeserver-admin credential. The
# controller mints agents' accounts with it and has no business replacing

81
swarm-controller/src/agent_identity.rs
View file

@ -220,54 +220,6 @@ pub async fn mint_and_verify(agent: &str) -> Result<()> {
Ok(())
}
/// Revoke `agent`'s queue credential: delete the path
/// [`mint_and_verify`]'s step 3 published, and everything ever written at it.
///
/// The undo of that one step and of no other. The leaf, the ACL document and
/// the cert-auth role that make up the rest of an agent's identity stay where
/// they are — they are what a *hive* uses to collect an agent's secrets, they
/// are minted afresh on every run of the mint, and tearing them down is not
/// what the queue credential outliving its holder is about.
///
/// **Deletes every version, not the newest one.** The mint rewrites this path
/// whenever the principal it names has to be corrected, so a soft delete would
/// leave the identical secret sitting in version history, readable at
/// `?version=N` by anything that can read the path at all — a value still
/// recoverable has not been revoked. See
/// [`SecretStore::delete_all_versions`][swarm_secret_client::SecretStore::delete_all_versions].
///
/// **Idempotent**: revoking an agent that never had a credential, or one
/// already revoked, succeeds and says so. A teardown that runs twice is
/// ordinary, and a second run that failed would be a worse fault than the one
/// this exists to fix.
///
/// # Errors
/// When the store cannot be reached or refuses the delete. The caller decides
/// what that costs — `set_agent_state` logs it and lets the destroy proceed,
/// since a credential that is still live is a smaller harm than an agent that
/// cannot be torn down.
pub async fn revoke_queue_credential(agent: &str) -> Result<()> {
let queue_path = queue::agent_queue_path(agent)?;
let store = crate::store::connect()
.await
.context("logging in to the swarm secret store")?;
store
.delete_all_versions(&queue_path)
.await
.with_context(|| format!("revoking the agent queue credential at {queue_path}"))?;
// At `info` and unconditional: an unlogged revocation is indistinguishable
// from a leak, and this line is the only record an operator has that the
// credential stopped being usable. It cannot say whether one was there —
// the controller's grant on these paths is write-only by design, so it
// deletes blind.
tracing::info!(
agent,
%queue_path,
"agent queue credential revoked: every version of the path deleted"
);
Ok(())
}
/// The consumer of everything [`mint_and_verify`] wrote: log in **as the
/// agent**, with the leaf just issued, and read back both paths just
/// published.
@ -364,39 +316,6 @@ mod tests {
assert!(!a.contains('='), "{a}");
}
/// Revocation undoes step 3 of the mint and nothing else, and both ends
/// spell the path through one function — the property that keeps a
/// revocation from missing the object it is meant to remove.
///
/// The second half is what stops this being a tautology: the agent's
/// other published object, the mTLS leaf, is at a different path and is
/// deliberately left alone. Revoking both would take away the identity a
/// re-created agent is re-minted under, for a credential problem that is
/// only about the queue.
#[test]
fn revocation_names_the_path_the_mint_published_and_not_the_leaf_beside_it() {
use swarm_secret_client::{mtls, queue};
assert_eq!(
queue::agent_queue_path("atlas").expect("a plain name is legal"),
"swarm/agents/atlas/queue"
);
assert_ne!(
queue::agent_queue_path("atlas").expect("legal"),
mtls::identity_path("atlas").expect("legal"),
);
}
/// A name the store must never be asked to delete under. `revoke_queue_credential`
/// builds its path with the same validating function the mint does, so a
/// traversal is refused before a request is made rather than addressing
/// some other principal's secret.
#[test]
fn a_traversal_never_becomes_a_revocation() {
swarm_secret_client::queue::agent_queue_path("../pr1ma")
.expect_err("a traversal is not a legal agent name");
}
fn both(k: &str) -> Option<String> {
match k {
ENV_AGENT_PKI_MOUNT => Some("pki-agents".to_owned()),

73
swarm-controller/src/main.rs
View file

@ -1050,56 +1050,9 @@ async fn set_agent_state(
tracing::warn!(hive = %hive, agent = %agent, error = %format!("{e:#}"), "declaring agent state failed");
error_problem(wanted_error_status(&e), &format!("{e:#}"))
})?;
// After the declaration and never before it: the write above is the
// destroy order, and until it lands the agent is not being torn down, so
// its credential has to keep working. Sequenced rather than spawned so
// the revocation attempt is finished — and its outcome logged — by the
// time the operator's call returns.
if revokes_queue_credential(req.state) {
revoke_or_complain(&hive, &agent).await;
}
Ok(Json(render(&declaration)))
}
/// Whether declaring an agent into `state` ends its queue credential's life.
///
/// Exhaustive on purpose, like every other match on
/// [`AgentState`][swarm_queue_client::wanted::AgentState] in this
/// tree: a state added later has to say whether it revokes rather than
/// inheriting "no" from a catch-all. `Offline` and `Paused` deliberately do
/// not — both are states an agent comes back from, and revoking on either
/// would mean an agent that could be stopped but never restarted.
fn revokes_queue_credential(state: swarm_queue_client::wanted::AgentState) -> bool {
use swarm_queue_client::wanted::AgentState;
match state {
AgentState::Destroyed => true,
AgentState::Up | AgentState::Offline | AgentState::Paused => false,
}
}
/// Revoke the credential, and make a failure impossible to miss without
/// letting it stop the teardown.
///
/// **The destroy is not blocked by this.** The declaration is already
/// published, the hive will converge on it, and refusing the operator's call
/// at this point would leave them with an agent they cannot destroy — the
/// larger of the two harms. What is left is that the failure must not be
/// quiet, or the outcome is the silent orphan this whole path exists to
/// remove: hence `error`, naming the agent, with the cause chain attached.
async fn revoke_or_complain(hive: &str, agent: &str) {
if let Err(e) = agent_identity::revoke_queue_credential(agent).await {
tracing::error!(
hive,
agent,
error = %format!("{e:#}"),
"revoking this agent's queue credential FAILED; it was destroyed anyway, so \
the credential outlives it and may still authenticate. Declaring the agent \
destroyed again re-runs this revocation."
);
}
}
/// What this swarm currently declares for a hive.
///
/// Read back from the bucket rather than from a second copy kept here: the
@ -3309,32 +3262,6 @@ mod tests {
);
}
/// `Destroyed` is the one declaration that ends an agent's life, so it is
/// the one that revokes. The credential is a bearer secret: left behind,
/// anything that recovers it still authenticates as an agent that no
/// longer exists.
#[test]
fn declaring_an_agent_destroyed_revokes_its_queue_credential() {
assert!(crate::revokes_queue_credential(
swarm_queue_client::wanted::AgentState::Destroyed
));
}
/// The control, and the more expensive half to get wrong: a revocation on
/// `Offline` or `Paused` would give an agent that can be stopped and never
/// restarted. Every non-terminal state is listed rather than sampled, so
/// a state added later shows up here as well as in the match itself.
#[test]
fn no_state_an_agent_comes_back_from_revokes() {
use swarm_queue_client::wanted::AgentState;
for state in [AgentState::Up, AgentState::Offline, AgentState::Paused] {
assert!(
!crate::revokes_queue_credential(state),
"{state:?} is a state an agent returns from"
);
}
}
/// The whole point of the node: a freshly created agent is declared
/// `Paused`, so it does not start driving turns the moment its hive
/// brings it up. Any other state here is the bug this fixes.

108
swarm-secret-client/src/client.rs
View file

@ -211,41 +211,6 @@ impl SecretStore {
Ok(())
}
/// Delete `path` outright: every version of it, and the metadata that
/// lists them.
///
/// The undo of [`write`][Self::write], and deliberately **not** KV v2's
/// plain delete. That one soft-deletes the newest version only, leaving
/// every earlier one readable at `?version=N` and the newest one
/// recoverable by `undelete` — so a caller revoking a bearer secret that
/// [`write`][Self::write] has ever replaced would leave the old value
/// sitting in version history, still usable. `metadata` is the one verb
/// that removes the lot.
///
/// Addresses `secret/metadata/<path>`, which the store ACLs separately
/// from the `secret/data/<path>` the other methods here use: a token
/// granted write on the data path is refused at this one until its policy
/// names the metadata path too.
///
/// **Idempotent.** Deleting a path that holds nothing succeeds, because
/// the caller is tearing something down and a teardown that runs twice —
/// or on a principal that never got a secret — must not fail the second
/// time. Only a 404 is absence, exactly as in
/// [`read_optional`][Self::read_optional]: a 403 stays an error, or a
/// principal whose grant does not cover the path would report a
/// revocation it never performed.
///
/// # Errors
/// [`Error::Vault`] for anything that is not a 404 — a denial, or a store
/// that could not be reached.
pub async fn delete_all_versions(&self, path: &str) -> Result<(), Error> {
match vaultrs::kv2::delete_metadata(&self.inner, MOUNT, path).await {
Ok(()) => Ok(()),
Err(e) if is_absent(&e) => Ok(()),
Err(e) => Err(e.into()),
}
}
/// Replace the ACL policy named `name` with `policy`.
///
/// A whole-document write, not a merge: the store has no other verb, and
@ -343,17 +308,6 @@ impl SecretStore {
}
}
/// Whether a failed store call means the path holds nothing — the one status
/// [`SecretStore::delete_all_versions`] is allowed to treat as success.
///
/// The same rule [`SecretStore::read_optional`] documents, named here because
/// the delete side has to be able to state it in a test: a 403 is a principal
/// whose grant misses the path, and swallowing that would let a caller report
/// a revocation the store refused to perform.
fn is_absent(e: &vaultrs::error::ClientError) -> bool {
matches!(e, vaultrs::error::ClientError::APIError { code: 404, .. })
}
/// Write an ACL policy, spelled out here because [`vaultrs`] does not have it:
/// its `sys::policy` module targets `sys/policy/<name>`, the deprecated alias,
/// and the store ACLs that path separately from `sys/policies/acl/<name>`. A
@ -600,66 +554,4 @@ mod tests {
assert!(!e.to_string().contains("SECRET-KEY-BYTES"), "{e}");
}
}
/// What makes [`SecretStore::delete_all_versions`] safe to run twice: the
/// second run finds nothing and must still succeed. A teardown that
/// hard-errors on an already-revoked credential is a worse fault than the
/// credential outliving its holder.
#[test]
fn a_path_that_holds_nothing_is_not_a_failed_delete() {
assert!(is_absent(&vaultrs::error::ClientError::APIError {
code: 404,
errors: vec![],
}));
}
/// The other half, and the one that matters for a revocation: a token
/// whose policy misses the path is refused with a 403, and reading that
/// as "nothing was there" would log a revocation the store never made.
#[test]
fn a_denial_is_not_absence() {
for code in [400, 403, 500, 503] {
assert!(
!is_absent(&vaultrs::error::ClientError::APIError {
code,
errors: vec![],
}),
"{code} must stay a failure"
);
}
}
/// Revocation addresses `secret/metadata/…`, not `secret/data/…`.
///
/// The distinction is the whole point of the verb: the data path's delete
/// leaves earlier versions readable, so revoking a secret that was ever
/// rewritten there would leave the old value recoverable.
///
/// Asserted on [`vaultrs`]'s own request type rather than on a call, since
/// there is no store to call: it pins the endpoint
/// `kv2::delete_metadata` targets, which is the string
/// `swarm-bao.nix`'s grant has to match. A dependency bump that moved it
/// would otherwise surface as a 403 at the first revocation, far from
/// here.
#[test]
fn a_revocation_targets_every_version_and_not_just_the_newest() {
use rustify::endpoint::Endpoint as _;
let request = vaultrs::api::kv2::requests::DeleteSecretMetadataRequest::builder()
.mount(MOUNT)
.path("swarm/agents/atlas/queue")
.build()
.expect("the request builds");
assert_eq!(request.path(), "secret/metadata/swarm/agents/atlas/queue");
// The control: the verb deliberately not used, which addresses the
// data path and soft-deletes exactly one version of it.
let soft = vaultrs::api::kv2::requests::DeleteLatestSecretVersionRequest::builder()
.mount(MOUNT)
.path("swarm/agents/atlas/queue")
.build()
.expect("the request builds");
assert_eq!(soft.path(), "secret/data/swarm/agents/atlas/queue");
assert_ne!(request.path(), soft.path());
}
}