vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

swarm-nats-auth: read its own OIDC client secret from the store

Browse source 
The responder reads its OIDC client secret at
`swarm/services/<client-id>/oidc/client` with the store login it already
opens for the callout keys, and holds it in memory. `--client-secret-file`
is removed.

When introspection answers non-2xx, the responder re-reads the secret from
the store once and, if it changed, retries the introspection once. Both fit
inside the request's 1.5s introspection budget, under the server's 2s
callout timeout: the budget is now one deadline for the whole request. A
re-read that does not finish by then keeps running in the background, the
request is denied, and the next request introspects with what it read. One
re-read runs at a time; a refusal while one runs waits on it.

In swarm-nats.nix the `swarm-bao-nats-auth-oidc` fetch unit, the flag and
the `oidc-client.secret` LoadCredential entry are gone, along with the
unit's policy ordering in glue-bao-readers-policy-order.nix. A tmpfiles `r`
rule removes the copy a container still holds at
/var/lib/nixos-containers/swarm-nats/var/lib/swarm-nats-auth/oidc-client.secret;
nothing reads it.

No policy change: the `swarm-nats-auth` role already reads this path
(`natsAuthReaders` in swarm-bao.nix), which is the role the removed fetch
unit logged in as.

Deploy order: none, and no operator step. The store's grant already exists,
so the queue host can be deployed before, after or without the store's host
and authelia's. Rolling back re-creates the fetch unit and the file; the
store leaf is untouched.
This commit is contained in:
atlas 2026-10-06 13:43:14 +02:00
commit 6646329da5
12 changed files with 557 additions and 253 deletions
Download patch file Download diff file Expand all files Collapse all files

3
swarm-nats-auth/Cargo.toml
View file

@ -48,6 +48,9 @@ tracing-subscriber.workspace = true
# builds a user token both ways and requires byte equality, on the one shape
# nats-jwt does model.
nats-jwt.workspace = true
# `test-util` for `#[tokio::test(start_paused = true)]`: `oidc_secret`'s tests
# run a re-read slower than the introspection budget on paused time.
tokio = { workspace = true, features = ["test-util"] }
[lints]
workspace = true

7
swarm-nats-auth/README.md
View file

@ -64,10 +64,15 @@ paths and the boot order.
- **Every credential is a path, never a value.** A value in nix config lands
in the world-readable store; a value in `argv` is readable via
`/proc/<pid>/cmdline`, which is `0444`. Paths are not secrets, so passing
them as flags is fine.
them as flags is fine. The secrets themselves, the callout seeds and the
responder's own OIDC client secret, are read from the secret store and held
in memory.
- **The introspection timeout is pinned below the server's `authorization.
timeout`**, with a test asserting the relation — a responder that answers
after the server gave up is indistinguishable from one that never answered.
It is one request's whole budget: when the `IdP` refuses the responder's own
secret, the re-read from the store and the one retry fit inside it, or the
request is denied and the re-read lands for the next one.
## What the tests do and do not cover

39
swarm-nats-auth/src/introspect.rs
View file

@ -33,12 +33,26 @@ use std::time::Duration;
use anyhow::{Context, Result};
use serde::Deserialize;
use tokio::time::Instant;
/// Kept under `swarm-nats.nix`'s `authorization.timeout = "2s"` — see the
/// module docs. A slower answer is not a late success, it is a denial that
/// already happened.
/// already happened. It bounds a whole request, the re-read of this
/// responder's own secret and the retry after it included, which is why
/// [`identify_caller`] takes a deadline rather than a duration.
pub const INTROSPECTION_TIMEOUT: Duration = Duration::from_millis(1500);
/// What one introspection call said.
#[derive(Debug, PartialEq, Eq)]
pub enum Answer {
/// A 2xx: the caller, or `None` for every shape that is not an admission.
Caller(Option<String>),
/// A non-2xx: the `IdP` did not answer about the token at all, which is
/// also how it refuses this responder's own credential. `oidc_secret`
/// re-reads that credential and retries once before this denies.
Refused,
}
/// The two fields this responder acts on — one per decision. Authelia returns
/// more (`sub`, `scope`, `exp`); modelling them would imply we check them.
#[derive(Debug, Deserialize)]
@ -75,10 +89,12 @@ impl IntrospectionResponse {
/// Ask the `IdP` whether `token` is currently valid, and as whom.
///
/// Returns `Ok(Some(client_id))` **only** on a 2xx whose body says `active:
/// true` and names the client. Every other outcome is `Ok(None)` with the
/// reason logged, or `Err` when the call could not be made at all — callers
/// must treat both as a denial.
/// Returns `Ok(Answer::Caller(Some(client_id)))` **only** on a 2xx whose
/// body says `active: true` and names the client. Every other 2xx is
/// `Ok(Answer::Caller(None))` with the reason logged, a non-2xx is
/// `Ok(Answer::Refused)`, and `Err` is a call that could not be made at all
/// or did not finish by `deadline` — callers must treat all but the first as
/// a denial.
///
/// `own_client_id` / `own_client_secret` are *this responder's* credential for
/// the introspection endpoint. They are not the caller's identity and must
@ -93,26 +109,27 @@ pub async fn identify_caller(
own_client_id: &str,
own_client_secret: &str,
token: &str,
) -> Result<Option<String>> {
deadline: Instant,
) -> Result<Answer> {
let resp = http
.post(url)
.basic_auth(own_client_id, Some(own_client_secret))
.form(&[("token", token)])
.timeout(INTROSPECTION_TIMEOUT)
.timeout(deadline.saturating_duration_since(Instant::now()))
.send()
.await
.context("introspection request")?;
let status = resp.status();
if !status.is_success() {
tracing::warn!(%status, "introspection returned non-2xx; denying");
return Ok(None);
tracing::warn!(%status, "introspection returned non-2xx");
return Ok(Answer::Refused);
}
let body: IntrospectionResponse = match resp.json().await {
Ok(b) => b,
Err(e) => {
tracing::warn!(error = ?e, "introspection body did not parse; denying");
return Ok(None);
return Ok(Answer::Caller(None));
}
};
if body.active && body.caller().is_none() {
@ -121,7 +138,7 @@ pub async fn identify_caller(
// rejected for no visible reason.
tracing::warn!("introspection said active but named no client; denying");
}
Ok(body.caller().map(str::to_owned))
Ok(Answer::Caller(body.caller().map(str::to_owned)))
}
#[cfg(test)]

133
swarm-nats-auth/src/main.rs
View file

@ -22,12 +22,12 @@
//!
//! # Secrets
//!
//! Every credential is taken as a **path**, never a value. Two reasons, both
//! previously learned the hard way here: a value in nix config is rendered into
//! the world-readable store, and a value in `argv` is readable by anyone via
//! `/proc/<pid>/cmdline`, which is `0444`. Paths are not secrets, so passing
//! them as flags is fine. The callout's three seeds are not files: they are
//! read from the secret store, at the paths `xkey` and `nkey` name.
//! Every credential is taken as a **path**, never a value: a value in nix
//! config is rendered into the world-readable store, and a value in `argv` is
//! readable by anyone via `/proc/<pid>/cmdline`, which is `0444`. Paths are
//! not secrets, so passing them as flags is fine. The callout's three seeds
//! and the responder's own OIDC client secret are not files: they are read
//! from the secret store, at the paths `xkey`, `nkey` and `oidc_secret` name.
use std::path::PathBuf;
@ -39,6 +39,7 @@ mod agent_token;
mod introspect;
mod keystore;
mod nkey;
mod oidc_secret;
mod policy;
mod request;
mod respond;
@ -95,7 +96,8 @@ struct Args {
#[arg(long)]
introspection_url: String,
/// `OAuth2` client id this responder introspects as. Must match
/// `OAuth2` client id this responder introspects as, and whose secret it
/// reads from the store. Must match
/// `services.hyperhive.swarm.nats.clientId`, whose default this mirrors.
#[arg(long, default_value = "swarm-nats")]
client_id: String,
@ -107,10 +109,6 @@ struct Args {
#[arg(long, default_value = "APP")]
account: String,
/// Path to this responder's own OIDC client secret.
#[arg(long)]
client_secret_file: PathBuf,
/// Client-id prefix that marks a hive. `swarm-authelia.nix` mints one
/// machine client per roster entry as `hive-<name>`, while the KV key is
/// the bare `<name>` — this is the contract between the two, declared
@ -160,27 +158,14 @@ struct Args {
agent_token_publish_subjects: Vec<String>,
/// Role on the secret store's `cert` auth mount this responder logs in
/// as to read the callout keys and agent credentials. The store is found
/// as to read the callout keys, its own OIDC client secret and agent
/// credentials. The store is found
/// through the `BAO_*` environment, which the responder does not start
/// without.
#[arg(long, default_value = "swarm-nats-auth")]
store_cert_role: String,
}
/// Read a secret file and strip surrounding whitespace.
///
/// The trim matters: an `echo`-created file ends in a newline, which is not
/// part of the secret. The value is never logged, and the error deliberately
/// names only the path.
fn read_secret(path: &std::path::Path) -> anyhow::Result<String> {
let raw = std::fs::read_to_string(path).with_context(|| format!("read {}", path.display()))?;
let trimmed = raw.trim();
if trimmed.is_empty() {
anyhow::bail!("{} is empty", path.display());
}
Ok(trimmed.to_owned())
}
#[tokio::main]
async fn main() -> anyhow::Result<()> {
swarm_queue_client::install_crypto_provider();
@ -219,7 +204,6 @@ async fn main() -> anyhow::Result<()> {
/// The responder: answers every authorization request until its
/// subscription ends.
async fn serve(args: &Args) -> anyhow::Result<()> {
let client_secret = read_secret(&args.client_secret_file)?;
// The bucket is NOT a flag. `swarm_queue_client::status`'s own docs say
// why: reader and writer must name the same bucket, and an option is a
// way for two deployments to disagree about which one that is. This
@ -241,9 +225,17 @@ async fn serve(args: &Args) -> anyhow::Result<()> {
let settings = swarm_secret_client::client::Settings::from_env()
.context("reading the secret store's address and this responder's identity")?;
// Before the connect: a responder that cannot open a sealed request, sign
// a grant or authenticate to the queue must not start answering.
let (callout_xkey, issuer, user_seed) =
load_callout_keys(&settings, &args.store_cert_role).await?;
// a grant, authenticate to the queue or introspect must not start
// answering.
let keys = swarm_secret_client::SecretStore::connect(
&settings,
&args.store_cert_role,
swarm_secret_client::client::DEFAULT_CERT_MOUNT,
)
.await
.context("logging in to the secret store for the callout keys and the OIDC client secret")?;
let (callout_xkey, issuer, user_seed) = load_callout_keys(&keys).await?;
let client_secret = load_client_secret(&keys, args).await?;
let store = agent_token::Store::new(settings, args.store_cert_role.clone());
let client = async_nats::ConnectOptions::with_nkey(user_seed)
.name("swarm-nats-auth")
@ -368,21 +360,13 @@ fn open_request(
/// The callout xkey, the issuer key, and the user key's seed, read from the
/// store and never created here; see `xkey` and `nkey`.
async fn load_callout_keys(
settings: &swarm_secret_client::client::Settings,
store_cert_role: &str,
keys: &impl keystore::KeyStore,
) -> anyhow::Result<(nkeys::XKey, nkeys::KeyPair, String)> {
let keys = swarm_secret_client::SecretStore::connect(
settings,
store_cert_role,
swarm_secret_client::client::DEFAULT_CERT_MOUNT,
)
.await
.context("logging in to the secret store for the callout keys")?;
let callout_xkey = xkey::load(&keys).await?;
let callout_xkey = xkey::load(keys).await?;
tracing::info!(public_key = %callout_xkey.public_key(), "loaded the callout xkey");
let issuer = nkey::load(&keys, &nkey::ISSUER).await?;
let issuer = nkey::load(keys, &nkey::ISSUER).await?;
tracing::info!(public_key = %issuer.public_key(), "loaded the callout issuer key");
let user = nkey::load(&keys, &nkey::USER).await?;
let user = nkey::load(keys, &nkey::USER).await?;
tracing::info!(public_key = %user.public_key(), "loaded the callout user key");
let user_seed = user
.seed()
@ -390,6 +374,23 @@ async fn load_callout_keys(
Ok((callout_xkey, issuer, user_seed))
}
/// This responder's own OIDC client secret, read from `keys`, and re-read
/// later as `args.store_cert_role`; see `oidc_secret`.
async fn load_client_secret(
keys: &impl keystore::KeyStore,
args: &Args,
) -> anyhow::Result<oidc_secret::OidcSecret<oidc_secret::Store>> {
let path = oidc_secret::path(&args.client_id)?;
let secret = oidc_secret::read(keys, &path).await?;
tracing::info!(%path, "loaded the OIDC client secret");
let source = oidc_secret::Store::new(
swarm_secret_client::client::Settings::from_env()?,
args.store_cert_role.clone(),
path,
);
Ok(oidc_secret::OidcSecret::new(source, secret))
}
/// The `ensure-callout-keys` verb: see [`xkey::ensure`] and [`nkey::ensure`].
async fn ensure_callout_keys(
store_cert_role: &str,
@ -429,24 +430,30 @@ async fn introspected(
policy: &policy::Policy,
http: &reqwest::Client,
args: &Args,
client_secret: &str,
client_secret: &oidc_secret::OidcSecret<impl oidc_secret::Source>,
token: &str,
) -> (Option<String>, Option<policy::Permissions>) {
let caller = introspect::identify_caller(
http,
&args.introspection_url,
&args.client_id,
client_secret,
token,
)
.await
// An introspection that could not be *made* is a denial too. The
// failure modes of an HTTP call are exactly the conditions under
// which an attacker would most like this to fall open.
.unwrap_or_else(|e| {
tracing::warn!(error = ?e, "introspection failed; denying");
None
});
let deadline = tokio::time::Instant::now() + introspect::INTROSPECTION_TIMEOUT;
let caller = client_secret
.introspect(deadline, |secret| async move {
introspect::identify_caller(
http,
&args.introspection_url,
&args.client_id,
&secret,
token,
deadline,
)
.await
})
.await
// An introspection that could not be *made* is a denial too. The
// failure modes of an HTTP call are exactly the conditions under
// which an attacker would most like this to fall open.
.unwrap_or_else(|e| {
tracing::warn!(error = ?e, "introspection failed; denying");
None
});
// Admission said who; the policy says what. A caller the `IdP`
// vouches for but no rule matches is denied — see `policy`'s module
// docs for why that is deny and not "connect with nothing".
@ -484,8 +491,6 @@ mod tests {
fn the_units_own_argument_vector_parses() {
let args = responder([
"swarm-nats-auth",
"--client-secret-file",
"/run/credentials/oidc-client.secret",
"--introspection-url",
"https://auth.example/api/oidc/introspection",
"--hive-client-prefix",
@ -543,8 +548,6 @@ mod tests {
fn an_agent_suffix_without_a_leading_dash_parses_too() {
let args = responder([
"swarm-nats-auth",
"--client-secret-file",
"/run/credentials/oidc-client.secret",
"--introspection-url",
"https://auth.example/api/oidc/introspection",
"--agent-client-suffix",
@ -560,8 +563,6 @@ mod tests {
fn the_default_suffix_is_what_the_module_passes() {
let args = responder([
"swarm-nats-auth",
"--client-secret-file",
"/run/credentials/oidc-client.secret",
"--introspection-url",
"https://auth.example/api/oidc/introspection",
])
@ -619,8 +620,8 @@ mod tests {
fn the_verb_does_not_take_the_responders_flags() {
Cli::try_parse_from([
"swarm-nats-auth",
"--client-secret-file",
"/run/credentials/oidc-client.secret",
"--introspection-url",
"https://auth.example/api/oidc/introspection",
"ensure-callout-keys",
])
.expect_err("a verb and the responder's flags conflict");

397
swarm-nats-auth/src/oidc_secret.rs Normal file
View file

@ -0,0 +1,397 @@
//! This responder's own OIDC client secret, the credential it introspects
//! with. Read from the secret store as the responder starts and held in
//! memory, never in a file, `argv` or the environment.
//!
//! The secret can change under a running responder: authelia's host
//! re-publishes it when it is re-minted. So when the `IdP` refuses an
//! introspection (a non-2xx), the secret is read from the store again and,
//! when it changed, the introspection is tried once more. Both fit in the one
//! request's [`INTROSPECTION_TIMEOUT`](crate::introspect::INTROSPECTION_TIMEOUT).
//! A re-read that does not is left running: that request is denied, and the
//! next one introspects with what the re-read landed. One re-read runs at a
//! time; a refusal while one is running waits for it rather than starting
//! another.
use std::future::Future;
use std::sync::{Arc, Mutex, RwLock};
use anyhow::{Context, Result};
use futures_util::FutureExt;
use futures_util::future::{BoxFuture, Shared};
use swarm_secret_client::client::{DEFAULT_CERT_MOUNT, Settings};
use swarm_secret_client::path::{Kind, principal_prefix};
use tokio::task::AbortHandle;
use tokio::time::Instant;
use crate::introspect::Answer;
use crate::keystore::KeyStore;
/// The path under the store's KV mount holding `client_id`'s secret.
/// `swarm-secret-publisher.nix` writes it, and `natsAuthReaders` in
/// `swarm-bao.nix` grants this responder's role the read.
///
/// # Errors
/// When `client_id` is not a single path segment.
pub fn path(client_id: &str) -> Result<String> {
let prefix = principal_prefix(Kind::Service, client_id)?;
Ok(format!("{prefix}/oidc/client"))
}
/// The secret at `path`, with surrounding whitespace trimmed: the publisher
/// stores the minted file as is, and a trailing newline is not part of the
/// secret. Errors name the path, never the value.
///
/// # Errors
/// When the store cannot be read, or holds nothing or only whitespace there.
pub async fn read(store: &impl KeyStore, path: &str) -> Result<String> {
let stored = store.read(path).await?.with_context(|| {
format!("nothing is stored at {path}; swarm-secret-publish on authelia's host writes it")
})?;
let trimmed = stored.trim();
if trimmed.is_empty() {
anyhow::bail!("{path} holds an empty secret");
}
Ok(trimmed.to_owned())
}
/// Where a re-read of the secret comes from.
pub trait Source: Send + Sync + 'static {
/// The secret as the store holds it now.
fn read(&self) -> impl Future<Output = Result<String>> + Send;
}
/// The secret store, logged in to per read: a re-read can come long after
/// the start's token has expired.
pub struct Store {
settings: Settings,
cert_role: String,
path: String,
}
impl Store {
/// The secret at `path`, read as `cert_role` from the store at `settings`.
pub fn new(settings: Settings, cert_role: String, path: String) -> Self {
Self {
settings,
cert_role,
path,
}
}
}
impl Source for Store {
async fn read(&self) -> Result<String> {
let store = swarm_secret_client::SecretStore::connect(
&self.settings,
&self.cert_role,
DEFAULT_CERT_MOUNT,
)
.await
.context("logging in to the secret store")?;
read(&store, &self.path).await
}
}
/// A re-read in progress, shared by every request waiting on it.
type Reread = Shared<BoxFuture<'static, ()>>;
/// The secret in use, and the re-read that replaces it. No `Debug`: it holds
/// the secret.
pub struct OidcSecret<S> {
source: Arc<S>,
current: Arc<RwLock<String>>,
reread: Mutex<Option<(AbortHandle, Reread)>>,
}
impl<S: Source> OidcSecret<S> {
/// `initial`, re-read from `source` when the `IdP` refuses it.
pub fn new(source: S, initial: String) -> Self {
Self {
source: Arc::new(source),
current: Arc::new(RwLock::new(initial)),
reread: Mutex::new(None),
}
}
fn current(&self) -> String {
self.current.read().expect("unpoisoned").clone()
}
/// The caller `attempt` names, introspecting with the current secret.
/// When the `IdP` refuses that, the secret is re-read and, if it changed
/// and `deadline` has not passed, `attempt` runs once more with the new
/// one. `attempt` bounds itself by the same `deadline`.
///
/// # Errors
/// When an attempt could not be made at all; the caller denies.
pub async fn introspect<F, Fut>(&self, deadline: Instant, attempt: F) -> Result<Option<String>>
where
F: Fn(String) -> Fut,
Fut: Future<Output = Result<Answer>>,
{
let used = self.current();
if let Answer::Caller(caller) = attempt(used.clone()).await? {
return Ok(caller);
}
if tokio::time::timeout_at(deadline, self.reread())
.await
.is_err()
{
tracing::warn!(
"re-reading the OIDC client secret did not finish inside the introspection budget; \
denying, and the next request introspects with what it reads"
);
return Ok(None);
}
let fresh = self.current();
if fresh == used {
tracing::warn!("the stored OIDC client secret is the one the IdP refused; denying");
return Ok(None);
}
tracing::info!("re-read a changed OIDC client secret; introspecting again");
match attempt(fresh).await? {
Answer::Caller(caller) => Ok(caller),
Answer::Refused => Ok(None),
}
}
/// The running re-read, or a new one when none is running. Spawned, so it
/// finishes even when every request waiting on it has given up.
fn reread(&self) -> Reread {
let mut slot = self.reread.lock().expect("unpoisoned");
if let Some((handle, running)) = slot.as_ref()
&& !handle.is_finished()
{
return running.clone();
}
let source = Arc::clone(&self.source);
let current = Arc::clone(&self.current);
let task = tokio::spawn(async move {
match source.read().await {
Ok(secret) => *current.write().expect("unpoisoned") = secret,
Err(e) => tracing::warn!(
error = format!("{e:#}"),
"could not re-read the OIDC client secret; keeping the one in use"
),
}
});
let handle = task.abort_handle();
let shared = task.map(|_| ()).boxed().shared();
*slot = Some((handle, shared.clone()));
shared
}
}
#[cfg(test)]
mod tests {
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::Duration;
use super::*;
use crate::introspect::INTROSPECTION_TIMEOUT;
use crate::keystore::tests::Fake as FakeKeys;
/// A store holding `secret`, or failing, that takes `delay` to answer.
struct Fake {
secret: Option<&'static str>,
delay: Duration,
reads: Arc<AtomicUsize>,
}
impl Fake {
fn holding(secret: &'static str) -> Self {
Self {
secret: Some(secret),
delay: Duration::ZERO,
reads: Arc::default(),
}
}
}
impl Source for Fake {
async fn read(&self) -> Result<String> {
self.reads.fetch_add(1, Ordering::SeqCst);
tokio::time::sleep(self.delay).await;
self.secret.map(str::to_owned).context("store unreachable")
}
}
/// An `IdP` that admits as `hive-alpha` only when introspected with
/// `registered`, and refuses every other secret, counting its calls.
fn idp<'a>(
registered: &'static str,
calls: &'a AtomicUsize,
) -> impl Fn(String) -> std::future::Ready<Result<Answer>> + 'a {
move |secret| {
calls.fetch_add(1, Ordering::SeqCst);
std::future::ready(Ok(if secret == registered {
Answer::Caller(Some("hive-alpha".to_owned()))
} else {
Answer::Refused
}))
}
}
fn budget() -> Instant {
Instant::now() + INTROSPECTION_TIMEOUT
}
#[test]
fn the_path_is_the_one_the_publisher_writes() {
assert_eq!(
path("swarm-nats").expect("path"),
"swarm/services/swarm-nats/oidc/client"
);
assert!(path("../swarm-grafana").is_err());
}
#[tokio::test]
async fn the_stored_secret_is_trimmed_and_an_empty_or_absent_one_refused() {
let at = "swarm/services/swarm-nats/oidc/client";
let keys = FakeKeys::holding(&[(at, "s3cret\n")]);
assert_eq!(read(&keys, at).await.expect("stored"), "s3cret");
keys.put(at, " \n");
assert!(read(&keys, at).await.is_err());
assert!(read(&FakeKeys::default(), at).await.is_err());
}
#[tokio::test]
async fn an_answered_introspection_reads_nothing() {
let store = Fake::holding("new");
let reads = Arc::clone(&store.reads);
let secret = OidcSecret::new(store, "old".to_owned());
let calls = AtomicUsize::new(0);
let caller = secret
.introspect(budget(), idp("old", &calls))
.await
.expect("answered");
assert_eq!(caller.as_deref(), Some("hive-alpha"));
assert_eq!(calls.load(Ordering::SeqCst), 1);
assert_eq!(reads.load(Ordering::SeqCst), 0);
}
/// The rotation the re-read exists for: the `IdP` refuses the secret held
/// since start, the store holds the new one, and the same request is
/// admitted with it.
#[tokio::test]
async fn a_refused_secret_is_re_read_and_the_request_retried_once() {
let store = Fake::holding("new");
let reads = Arc::clone(&store.reads);
let secret = OidcSecret::new(store, "old".to_owned());
let calls = AtomicUsize::new(0);
let caller = secret
.introspect(budget(), idp("new", &calls))
.await
.expect("answered");
assert_eq!(caller.as_deref(), Some("hive-alpha"));
assert_eq!(calls.load(Ordering::SeqCst), 2);
assert_eq!(reads.load(Ordering::SeqCst), 1);
}
#[tokio::test]
async fn a_refusal_after_the_re_read_is_a_denial_with_no_third_attempt() {
let store = Fake::holding("newer");
let reads = Arc::clone(&store.reads);
let secret = OidcSecret::new(store, "old".to_owned());
let calls = AtomicUsize::new(0);
let caller = secret
.introspect(budget(), idp("unknown", &calls))
.await
.expect("answered");
assert_eq!(caller, None);
assert_eq!(calls.load(Ordering::SeqCst), 2);
assert_eq!(reads.load(Ordering::SeqCst), 1);
}
#[tokio::test]
async fn an_unchanged_secret_is_not_retried() {
let secret = OidcSecret::new(Fake::holding("old"), "old".to_owned());
let calls = AtomicUsize::new(0);
let caller = secret
.introspect(budget(), idp("unknown", &calls))
.await
.expect("answered");
assert_eq!(caller, None);
assert_eq!(calls.load(Ordering::SeqCst), 1);
}
#[tokio::test]
async fn a_failed_re_read_denies_and_keeps_the_secret_in_use() {
let store = Fake {
secret: None,
delay: Duration::ZERO,
reads: Arc::default(),
};
let secret = OidcSecret::new(store, "old".to_owned());
let calls = AtomicUsize::new(0);
let caller = secret
.introspect(budget(), idp("new", &calls))
.await
.expect("answered");
assert_eq!(caller, None);
assert_eq!(calls.load(Ordering::SeqCst), 1);
assert_eq!(secret.current(), "old");
}
/// A store slower than the budget: the request is denied at its
/// deadline, the re-read finishes behind it, and the next request is
/// admitted on its first attempt without reading again.
#[tokio::test(start_paused = true)]
async fn a_re_read_past_the_budget_serves_the_next_request() {
let store = Fake {
secret: Some("new"),
delay: Duration::from_secs(3),
reads: Arc::default(),
};
let reads = Arc::clone(&store.reads);
let secret = OidcSecret::new(store, "old".to_owned());
let calls = AtomicUsize::new(0);
let started = Instant::now();
let deadline = started + INTROSPECTION_TIMEOUT;
let first = secret
.introspect(deadline, idp("new", &calls))
.await
.expect("answered");
assert_eq!(first, None);
assert_eq!(Instant::now(), deadline);
assert_eq!(calls.load(Ordering::SeqCst), 1);
tokio::time::sleep(Duration::from_secs(2)).await;
let next = secret
.introspect(budget(), idp("new", &calls))
.await
.expect("answered");
assert_eq!(next.as_deref(), Some("hive-alpha"));
assert_eq!(calls.load(Ordering::SeqCst), 2);
assert_eq!(reads.load(Ordering::SeqCst), 1);
}
/// A refusal while a re-read is still running waits on that one, and is
/// admitted when it lands inside this request's budget.
#[tokio::test(start_paused = true)]
async fn a_refusal_during_a_running_re_read_joins_it() {
let store = Fake {
secret: Some("new"),
delay: Duration::from_secs(2),
reads: Arc::default(),
};
let reads = Arc::clone(&store.reads);
let secret = OidcSecret::new(store, "old".to_owned());
let calls = AtomicUsize::new(0);
let first = secret
.introspect(budget(), idp("new", &calls))
.await
.expect("answered");
assert_eq!(first, None);
let second = secret
.introspect(budget(), idp("new", &calls))
.await
.expect("answered");
assert_eq!(second.as_deref(), Some("hive-alpha"));
assert_eq!(calls.load(Ordering::SeqCst), 3);
assert_eq!(reads.load(Ordering::SeqCst), 1);
}
}