swarm: mint a per-agent queue credential beside the agent's store identity
Every agent on a hive authenticates to the swarm queue with the same hive-scoped secret today, so at the auth callout one agent is indistinguishable from its co-hived neighbours and no subject can be scoped to one of them. Mint a secret per agent instead, at swarm level, into secret/swarm/agents/<agent>/queue -- inside the stanza every agent's ACL document already grants, so no policy changes and no existing agent's document is rewritten. It is written by the same node that already mints the agent's certificate, and read back under the agent's own token before that node reports success. The secret is not derived from the agent's mTLS identity: the two credentials answer different questions and coupling their lifetimes would mean renewing either implied renewing the other. Nothing here rotates a queue secret -- a re-run keeps the existing value and only corrects the principal it names, because this function is re-run deliberately against agents that are already connected. Revoking one means deleting the path. Nothing reads the new credential yet; this is the minting half.
This commit is contained in:
atlas
mara
parent
11097ed336
commit
ffd5018b18
6 changed files with 359 additions and 19 deletions
|
|
@ -5,10 +5,17 @@
|
|||
//! The swarm mints the agent's certificate so that no hive ever needs the
|
||||
//! capability to mint one; the hive only carries it down. The controller is
|
||||
//! the swarm-level service that does it because it already logs in to the
|
||||
//! store, and its grant already covers exactly the three objects written here
|
||||
//! store, and its grant already covers exactly the objects written here
|
||||
//! (`swarm-bao.nix`'s `controllerPolicyText`: `create/update` on
|
||||
//! `secret/data/swarm/agents/*`, on `sys/policies/acl/hive-*`, and on
|
||||
//! `auth/cert/certs/hive-*`). No new authority is asked for anywhere.
|
||||
//! `auth/cert/certs/hive-*`). No new authority is asked for anywhere — the
|
||||
//! agent's queue secret lives under the same `swarm/agents/<agent>` prefix the
|
||||
//! certificate does, which is why adding it costs no grant on either side.
|
||||
//!
|
||||
//! **Two credentials, deliberately unrelated.** The certificate is how the
|
||||
//! agent reaches the store; the queue secret is how it identifies itself to the
|
||||
//! swarm queue. The second is not derived from the first, so renewing either is
|
||||
//! a question that can be answered without reference to the other.
|
||||
//!
|
||||
//! Four separate strings have to agree before an agent can authenticate: the
|
||||
//! policy's name, the cert-auth role's name, the certificate's common name,
|
||||
|
|
@ -30,7 +37,7 @@ use anyhow::{Context, Result, bail};
|
|||
use swarm_secret_client::{
|
||||
SecretStore,
|
||||
client::{DEFAULT_CERT_MOUNT, Settings},
|
||||
mtls, policy,
|
||||
mtls, policy, queue,
|
||||
};
|
||||
use time::OffsetDateTime;
|
||||
|
||||
|
|
@ -191,22 +198,64 @@ fn validity(now: SystemTime, lifetime: Duration) -> Result<(OffsetDateTime, Offs
|
|||
Ok((not_before, not_after))
|
||||
}
|
||||
|
||||
/// How many bytes of kernel randomness a queue secret is before encoding.
|
||||
///
|
||||
/// Thirty-two because the secret is a bearer token compared for equality and
|
||||
/// nothing else — there is no work factor and no rate limit behind it, so the
|
||||
/// only defence is that guessing is not worth attempting. Encoded it is 43
|
||||
/// characters.
|
||||
const QUEUE_SECRET_BYTES: usize = 32;
|
||||
|
||||
/// Generate a queue secret: [`QUEUE_SECRET_BYTES`] from the kernel's CSPRNG,
|
||||
/// base64url without padding.
|
||||
///
|
||||
/// The alphabet matters and is the reason for `URL_SAFE_NO_PAD` rather than
|
||||
/// the standard engine: the secret is destined to be carried in a token the
|
||||
/// verifying end splits on `.`, so it must not be able to contain one. This
|
||||
/// alphabet is `[A-Za-z0-9_-]`, and `=` padding is dropped as well so the
|
||||
/// value survives anything that treats it as a word.
|
||||
///
|
||||
/// # Errors
|
||||
/// When the kernel will not supply randomness. Bubbled rather than panicked
|
||||
/// on: the caller is a job node that reports a named failure, and a secret
|
||||
/// from a degraded source is worse than no secret.
|
||||
fn generate_queue_secret() -> Result<String> {
|
||||
let mut bytes = [0u8; QUEUE_SECRET_BYTES];
|
||||
getrandom::fill(&mut bytes).context("drawing a queue secret from the kernel's CSPRNG")?;
|
||||
Ok(base64::Engine::encode(
|
||||
&base64::engine::general_purpose::URL_SAFE_NO_PAD,
|
||||
bytes,
|
||||
))
|
||||
}
|
||||
|
||||
/// Give `agent` an identity at the store, and prove it works.
|
||||
///
|
||||
/// Four store writes' worth of agreement, then the login that checks it:
|
||||
/// Five store writes' worth of agreement, then the login that checks it:
|
||||
///
|
||||
/// 1. mint a leaf whose common name is [`policy::agent_object_name`];
|
||||
/// 2. publish it at [`mtls::identity_path`], where the agent's hive collects
|
||||
/// it under the hive's own certificate;
|
||||
/// 3. write the ACL document [`policy::render_agent_with_queue`] renders —
|
||||
/// 3. publish a queue secret at [`queue::agent_queue_path`] — the agent's own
|
||||
/// identity at the swarm queue, minted here so that the credential an agent
|
||||
/// presents names *it* rather than its hive;
|
||||
/// 4. write the ACL document [`policy::render_agent_with_queue`] renders —
|
||||
/// read on this one agent's paths, plus the hive-shared queue credential
|
||||
/// every agent container on `hive` already receives out of band;
|
||||
/// 4. write the cert-auth role that ties the three together.
|
||||
/// 5. write the cert-auth role that ties the three together.
|
||||
///
|
||||
/// Policy before role, for the reason `read_policy::provision` gives: the role
|
||||
/// names the policy, so the other order leaves a window in which it points at
|
||||
/// nothing.
|
||||
///
|
||||
/// ⚠️ **Step 3 is idempotent and step 2 is not.** Re-running this function
|
||||
/// re-mints the agent's certificate — a fresh leaf the agent picks up on its
|
||||
/// next boot — but leaves an existing queue secret exactly as it is. The
|
||||
/// asymmetry is deliberate: an agent holds its queue secret in a live
|
||||
/// connection, so replacing it would drop that agent off the queue until it
|
||||
/// reconnected, and this function is re-run deliberately (by the backfill
|
||||
/// route) against agents that are already running. Nothing here rotates a
|
||||
/// queue secret; revoking one means deleting the path.
|
||||
///
|
||||
/// # Errors
|
||||
/// Anything that stops one of those five steps, with the step named. A
|
||||
/// failure here fails the job node and nothing else — the agent is still
|
||||
|
|
@ -214,6 +263,7 @@ fn validity(now: SystemTime, lifetime: Duration) -> Result<(OffsetDateTime, Offs
|
|||
pub async fn mint_and_verify(authority: &Authority, agent: &str, hive: &str) -> Result<()> {
|
||||
let name = policy::agent_object_name(agent)?;
|
||||
let path = mtls::identity_path(agent)?;
|
||||
let queue_path = queue::agent_queue_path(agent)?;
|
||||
|
||||
let (cert, key) = authority.mint_leaf(&name)?;
|
||||
let credential = mtls::Credential {
|
||||
|
|
@ -229,6 +279,50 @@ pub async fn mint_and_verify(authority: &Authority, agent: &str, hive: &str) ->
|
|||
.write(&path, &credential)
|
||||
.await
|
||||
.with_context(|| format!("publishing the agent identity at {path}"))?;
|
||||
|
||||
// Read before write, and `read_optional` rather than `read`, so that only
|
||||
// a genuine 404 leads to a new secret — see that method for why every
|
||||
// other failure has to stay a failure here.
|
||||
let existing: Option<queue::AgentCredential> = store
|
||||
.read_optional(&queue_path)
|
||||
.await
|
||||
.with_context(|| format!("checking whether {queue_path} already holds a credential"))?;
|
||||
// The secret survives a re-run; the principal it names does not get to.
|
||||
// An object whose `hive` disagrees with the hive this node was invoked
|
||||
// with would grant its holder subjects on the wrong hive, so it is
|
||||
// corrected — but by rewriting the two name fields around the *same*
|
||||
// `value`, which is a correction no live connection notices.
|
||||
let wanted = queue::AgentCredential {
|
||||
value: match &existing {
|
||||
Some(existing) => existing.value.clone(),
|
||||
None => generate_queue_secret()?,
|
||||
},
|
||||
agent: agent.to_owned(),
|
||||
hive: hive.to_owned(),
|
||||
};
|
||||
if existing.as_ref() == Some(&wanted) {
|
||||
tracing::info!(
|
||||
agent,
|
||||
%queue_path,
|
||||
"agent queue credential already published; left as it is"
|
||||
);
|
||||
} else {
|
||||
store
|
||||
.write(&queue_path, &wanted)
|
||||
.await
|
||||
.with_context(|| format!("publishing the agent queue credential at {queue_path}"))?;
|
||||
tracing::info!(
|
||||
agent,
|
||||
hive,
|
||||
%queue_path,
|
||||
// Never "rotated": the secret is the same one, only the names
|
||||
// around it moved.
|
||||
corrected = existing.is_some(),
|
||||
"agent queue credential published"
|
||||
);
|
||||
}
|
||||
let queue_credential = wanted;
|
||||
|
||||
store
|
||||
.write_policy(&name, &policy::render_agent_with_queue(agent, hive)?)
|
||||
.await
|
||||
|
|
@ -239,29 +333,42 @@ pub async fn mint_and_verify(authority: &Authority, agent: &str, hive: &str) ->
|
|||
.with_context(|| format!("writing the cert-auth role {name}"))?;
|
||||
tracing::info!(agent, %path, role = %name, "agent store identity published");
|
||||
|
||||
read_back_as_agent(&credential, &name, &path).await?;
|
||||
read_back_as_agent(&credential, &name, &path, &queue_path, &queue_credential).await?;
|
||||
tracing::info!(
|
||||
agent,
|
||||
role = %name,
|
||||
"agent store identity verified: the minted leaf logged in and read its own path"
|
||||
"agent store identity verified: the minted leaf logged in and read both its own paths"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// The consumer of everything [`mint_and_verify`] wrote: log in **as the
|
||||
/// agent**, with the leaf just minted, and read back the path just published.
|
||||
/// agent**, with the leaf just minted, and read back both paths just
|
||||
/// published.
|
||||
///
|
||||
/// The address and the store's CA come from this process's own `BAO_*`
|
||||
/// environment; the *identity* deliberately does not — see
|
||||
/// [`SecretStore::connect_with_identity`]. The freshly minted private key
|
||||
/// never touches a filesystem.
|
||||
///
|
||||
/// Both paths, not just the certificate's, for the reason this function
|
||||
/// exists at all: the queue credential is readable only because it sits inside
|
||||
/// the stanza [`policy::render_agent`] already grants, and a policy that
|
||||
/// drifted from that path would otherwise fail at the agent's first connection
|
||||
/// — far from here, as a denial naming neither the policy nor the path.
|
||||
///
|
||||
/// # Errors
|
||||
/// When the store refuses the login (the role, the authority or the common
|
||||
/// name disagree), when the read is denied (the policy does not cover the
|
||||
/// name disagree), when either read is denied (the policy does not cover the
|
||||
/// path, or the role attached the wrong policy), or when what comes back is
|
||||
/// not what went in.
|
||||
async fn read_back_as_agent(credential: &mtls::Credential, role: &str, path: &str) -> Result<()> {
|
||||
async fn read_back_as_agent(
|
||||
credential: &mtls::Credential,
|
||||
role: &str,
|
||||
path: &str,
|
||||
queue_path: &str,
|
||||
queue_credential: &queue::AgentCredential,
|
||||
) -> Result<()> {
|
||||
let settings = Settings::from_env()
|
||||
.context("reading this daemon's own store settings for the read-back")?;
|
||||
|
||||
|
|
@ -287,12 +394,25 @@ async fn read_back_as_agent(credential: &mtls::Credential, role: &str, path: &st
|
|||
if read_back != *credential {
|
||||
bail!("the store returned a different object at {path} than the one just published");
|
||||
}
|
||||
|
||||
let read_back: queue::AgentCredential = as_agent
|
||||
.read(queue_path)
|
||||
.await
|
||||
.with_context(|| format!("reading {queue_path} back under {role}'s own token"))?;
|
||||
// The two name fields are compared as well as the secret: they are what
|
||||
// the verifying end will grant subjects from, so a mismatch here is the
|
||||
// same class of fault as an unreadable path.
|
||||
if read_back != *queue_credential {
|
||||
bail!("the store returned a different object at {queue_path} than the one just published");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{Authority, CLOCK_SKEW, LEAF_LIFETIME, validity};
|
||||
use super::{
|
||||
Authority, CLOCK_SKEW, LEAF_LIFETIME, QUEUE_SECRET_BYTES, generate_queue_secret, validity,
|
||||
};
|
||||
use std::time::{Duration, SystemTime, UNIX_EPOCH};
|
||||
|
||||
/// A throwaway CA, minted in-process so no test needs a fixture file.
|
||||
|
|
@ -378,6 +498,28 @@ mod tests {
|
|||
reqwest::Identity::from_pem(&identity).expect("the leaf and its key form a TLS identity");
|
||||
}
|
||||
|
||||
/// The alphabet claim the token format rests on: the secret is carried in
|
||||
/// a composite the verifying end splits on `.`, so a secret that could
|
||||
/// contain one would make that split ambiguous. Also the entropy claim —
|
||||
/// a generator that silently returned a short or constant value would pass
|
||||
/// every other test in this file.
|
||||
#[test]
|
||||
fn a_queue_secret_is_high_entropy_and_carries_no_separator() {
|
||||
let a = generate_queue_secret().expect("the kernel supplies randomness");
|
||||
let b = generate_queue_secret().expect("twice");
|
||||
assert_ne!(a, b, "two draws must not agree");
|
||||
|
||||
// base64 without padding: one character per six bits, rounded up.
|
||||
assert_eq!(a.len(), (QUEUE_SECRET_BYTES * 8).div_ceil(6));
|
||||
assert!(
|
||||
a.chars()
|
||||
.all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_'),
|
||||
"{a}"
|
||||
);
|
||||
assert!(!a.contains('.'), "{a}");
|
||||
assert!(!a.contains('='), "{a}");
|
||||
}
|
||||
|
||||
/// A misconfiguration that would otherwise look like "not configured":
|
||||
/// half an authority has to be an error, not a silent `None`.
|
||||
///
|
||||
|
|
|
|||
Loading…
Reference in a new issue