vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

swarm-nats-auth: read its own OIDC client secret from the store

Browse source 
The responder reads its OIDC client secret at
`swarm/services/<client-id>/oidc/client` with the store login it already
opens for the callout keys, and holds it in memory. `--client-secret-file`
is removed.

When introspection answers non-2xx, the responder re-reads the secret from
the store once and, if it changed, retries the introspection once. Both fit
inside the request's 1.5s introspection budget, under the server's 2s
callout timeout: the budget is now one deadline for the whole request. A
re-read that does not finish by then keeps running in the background, the
request is denied, and the next request introspects with what it read. One
re-read runs at a time; a refusal while one runs waits on it.

In swarm-nats.nix the `swarm-bao-nats-auth-oidc` fetch unit, the flag and
the `oidc-client.secret` LoadCredential entry are gone, along with the
unit's policy ordering in glue-bao-readers-policy-order.nix. A tmpfiles `r`
rule removes the copy a container still holds at
/var/lib/nixos-containers/swarm-nats/var/lib/swarm-nats-auth/oidc-client.secret;
nothing reads it.

No policy change: the `swarm-nats-auth` role already reads this path
(`natsAuthReaders` in swarm-bao.nix), which is the role the removed fetch
unit logged in as.

Deploy order: none, and no operator step. The store's grant already exists,
so the queue host can be deployed before, after or without the store's host
and authelia's. Rolling back re-creates the fetch unit and the file; the
store leaf is untouched.
This commit is contained in:
atlas 2026-10-06 13:43:14 +02:00
commit 6646329da5
12 changed files with 557 additions and 253 deletions
Download patch file Download diff file Expand all files Collapse all files

39
swarm-nats-auth/src/introspect.rs
View file

@ -33,12 +33,26 @@ use std::time::Duration;
use anyhow::{Context, Result};
use serde::Deserialize;
use tokio::time::Instant;
/// Kept under `swarm-nats.nix`'s `authorization.timeout = "2s"` — see the
/// module docs. A slower answer is not a late success, it is a denial that
/// already happened.
/// already happened. It bounds a whole request, the re-read of this
/// responder's own secret and the retry after it included, which is why
/// [`identify_caller`] takes a deadline rather than a duration.
pub const INTROSPECTION_TIMEOUT: Duration = Duration::from_millis(1500);
/// What one introspection call said.
#[derive(Debug, PartialEq, Eq)]
pub enum Answer {
/// A 2xx: the caller, or `None` for every shape that is not an admission.
Caller(Option<String>),
/// A non-2xx: the `IdP` did not answer about the token at all, which is
/// also how it refuses this responder's own credential. `oidc_secret`
/// re-reads that credential and retries once before this denies.
Refused,
}
/// The two fields this responder acts on — one per decision. Authelia returns
/// more (`sub`, `scope`, `exp`); modelling them would imply we check them.
#[derive(Debug, Deserialize)]
@ -75,10 +89,12 @@ impl IntrospectionResponse {
/// Ask the `IdP` whether `token` is currently valid, and as whom.
///
/// Returns `Ok(Some(client_id))` **only** on a 2xx whose body says `active:
/// true` and names the client. Every other outcome is `Ok(None)` with the
/// reason logged, or `Err` when the call could not be made at all — callers
/// must treat both as a denial.
/// Returns `Ok(Answer::Caller(Some(client_id)))` **only** on a 2xx whose
/// body says `active: true` and names the client. Every other 2xx is
/// `Ok(Answer::Caller(None))` with the reason logged, a non-2xx is
/// `Ok(Answer::Refused)`, and `Err` is a call that could not be made at all
/// or did not finish by `deadline` — callers must treat all but the first as
/// a denial.
///
/// `own_client_id` / `own_client_secret` are *this responder's* credential for
/// the introspection endpoint. They are not the caller's identity and must
@ -93,26 +109,27 @@ pub async fn identify_caller(
own_client_id: &str,
own_client_secret: &str,
token: &str,
) -> Result<Option<String>> {
deadline: Instant,
) -> Result<Answer> {
let resp = http
.post(url)
.basic_auth(own_client_id, Some(own_client_secret))
.form(&[("token", token)])
.timeout(INTROSPECTION_TIMEOUT)
.timeout(deadline.saturating_duration_since(Instant::now()))
.send()
.await
.context("introspection request")?;
let status = resp.status();
if !status.is_success() {
tracing::warn!(%status, "introspection returned non-2xx; denying");
return Ok(None);
tracing::warn!(%status, "introspection returned non-2xx");
return Ok(Answer::Refused);
}
let body: IntrospectionResponse = match resp.json().await {
Ok(b) => b,
Err(e) => {
tracing::warn!(error = ?e, "introspection body did not parse; denying");
return Ok(None);
return Ok(Answer::Caller(None));
}
};
if body.active && body.caller().is_none() {
@ -121,7 +138,7 @@ pub async fn identify_caller(
// rejected for no visible reason.
tracing::warn!("introspection said active but named no client; denying");
}
Ok(body.caller().map(str::to_owned))
Ok(Answer::Caller(body.caller().map(str::to_owned)))
}
#[cfg(test)]

133
swarm-nats-auth/src/main.rs
View file

@ -22,12 +22,12 @@
//!
//! # Secrets
//!
//! Every credential is taken as a **path**, never a value. Two reasons, both
//! previously learned the hard way here: a value in nix config is rendered into
//! the world-readable store, and a value in `argv` is readable by anyone via
//! `/proc/<pid>/cmdline`, which is `0444`. Paths are not secrets, so passing
//! them as flags is fine. The callout's three seeds are not files: they are
//! read from the secret store, at the paths `xkey` and `nkey` name.
//! Every credential is taken as a **path**, never a value: a value in nix
//! config is rendered into the world-readable store, and a value in `argv` is
//! readable by anyone via `/proc/<pid>/cmdline`, which is `0444`. Paths are
//! not secrets, so passing them as flags is fine. The callout's three seeds
//! and the responder's own OIDC client secret are not files: they are read
//! from the secret store, at the paths `xkey`, `nkey` and `oidc_secret` name.
use std::path::PathBuf;
@ -39,6 +39,7 @@ mod agent_token;
mod introspect;
mod keystore;
mod nkey;
mod oidc_secret;
mod policy;
mod request;
mod respond;
@ -95,7 +96,8 @@ struct Args {
#[arg(long)]
introspection_url: String,
/// `OAuth2` client id this responder introspects as. Must match
/// `OAuth2` client id this responder introspects as, and whose secret it
/// reads from the store. Must match
/// `services.hyperhive.swarm.nats.clientId`, whose default this mirrors.
#[arg(long, default_value = "swarm-nats")]
client_id: String,
@ -107,10 +109,6 @@ struct Args {
#[arg(long, default_value = "APP")]
account: String,
/// Path to this responder's own OIDC client secret.
#[arg(long)]
client_secret_file: PathBuf,
/// Client-id prefix that marks a hive. `swarm-authelia.nix` mints one
/// machine client per roster entry as `hive-<name>`, while the KV key is
/// the bare `<name>` — this is the contract between the two, declared
@ -160,27 +158,14 @@ struct Args {
agent_token_publish_subjects: Vec<String>,
/// Role on the secret store's `cert` auth mount this responder logs in
/// as to read the callout keys and agent credentials. The store is found
/// as to read the callout keys, its own OIDC client secret and agent
/// credentials. The store is found
/// through the `BAO_*` environment, which the responder does not start
/// without.
#[arg(long, default_value = "swarm-nats-auth")]
store_cert_role: String,
}
/// Read a secret file and strip surrounding whitespace.
///
/// The trim matters: an `echo`-created file ends in a newline, which is not
/// part of the secret. The value is never logged, and the error deliberately
/// names only the path.
fn read_secret(path: &std::path::Path) -> anyhow::Result<String> {
let raw = std::fs::read_to_string(path).with_context(|| format!("read {}", path.display()))?;
let trimmed = raw.trim();
if trimmed.is_empty() {
anyhow::bail!("{} is empty", path.display());
}
Ok(trimmed.to_owned())
}
#[tokio::main]
async fn main() -> anyhow::Result<()> {
swarm_queue_client::install_crypto_provider();
@ -219,7 +204,6 @@ async fn main() -> anyhow::Result<()> {
/// The responder: answers every authorization request until its
/// subscription ends.
async fn serve(args: &Args) -> anyhow::Result<()> {
let client_secret = read_secret(&args.client_secret_file)?;
// The bucket is NOT a flag. `swarm_queue_client::status`'s own docs say
// why: reader and writer must name the same bucket, and an option is a
// way for two deployments to disagree about which one that is. This
@ -241,9 +225,17 @@ async fn serve(args: &Args) -> anyhow::Result<()> {
let settings = swarm_secret_client::client::Settings::from_env()
.context("reading the secret store's address and this responder's identity")?;
// Before the connect: a responder that cannot open a sealed request, sign
// a grant or authenticate to the queue must not start answering.
let (callout_xkey, issuer, user_seed) =
load_callout_keys(&settings, &args.store_cert_role).await?;
// a grant, authenticate to the queue or introspect must not start
// answering.
let keys = swarm_secret_client::SecretStore::connect(
&settings,
&args.store_cert_role,
swarm_secret_client::client::DEFAULT_CERT_MOUNT,
)
.await
.context("logging in to the secret store for the callout keys and the OIDC client secret")?;
let (callout_xkey, issuer, user_seed) = load_callout_keys(&keys).await?;
let client_secret = load_client_secret(&keys, args).await?;
let store = agent_token::Store::new(settings, args.store_cert_role.clone());
let client = async_nats::ConnectOptions::with_nkey(user_seed)
.name("swarm-nats-auth")
@ -368,21 +360,13 @@ fn open_request(
/// The callout xkey, the issuer key, and the user key's seed, read from the
/// store and never created here; see `xkey` and `nkey`.
async fn load_callout_keys(
settings: &swarm_secret_client::client::Settings,
store_cert_role: &str,
keys: &impl keystore::KeyStore,
) -> anyhow::Result<(nkeys::XKey, nkeys::KeyPair, String)> {
let keys = swarm_secret_client::SecretStore::connect(
settings,
store_cert_role,
swarm_secret_client::client::DEFAULT_CERT_MOUNT,
)
.await
.context("logging in to the secret store for the callout keys")?;
let callout_xkey = xkey::load(&keys).await?;
let callout_xkey = xkey::load(keys).await?;
tracing::info!(public_key = %callout_xkey.public_key(), "loaded the callout xkey");
let issuer = nkey::load(&keys, &nkey::ISSUER).await?;
let issuer = nkey::load(keys, &nkey::ISSUER).await?;
tracing::info!(public_key = %issuer.public_key(), "loaded the callout issuer key");
let user = nkey::load(&keys, &nkey::USER).await?;
let user = nkey::load(keys, &nkey::USER).await?;
tracing::info!(public_key = %user.public_key(), "loaded the callout user key");
let user_seed = user
.seed()
@ -390,6 +374,23 @@ async fn load_callout_keys(
Ok((callout_xkey, issuer, user_seed))
}
/// This responder's own OIDC client secret, read from `keys`, and re-read
/// later as `args.store_cert_role`; see `oidc_secret`.
async fn load_client_secret(
keys: &impl keystore::KeyStore,
args: &Args,
) -> anyhow::Result<oidc_secret::OidcSecret<oidc_secret::Store>> {
let path = oidc_secret::path(&args.client_id)?;
let secret = oidc_secret::read(keys, &path).await?;
tracing::info!(%path, "loaded the OIDC client secret");
let source = oidc_secret::Store::new(
swarm_secret_client::client::Settings::from_env()?,
args.store_cert_role.clone(),
path,
);
Ok(oidc_secret::OidcSecret::new(source, secret))
}
/// The `ensure-callout-keys` verb: see [`xkey::ensure`] and [`nkey::ensure`].
async fn ensure_callout_keys(
store_cert_role: &str,
@ -429,24 +430,30 @@ async fn introspected(
policy: &policy::Policy,
http: &reqwest::Client,
args: &Args,
client_secret: &str,
client_secret: &oidc_secret::OidcSecret<impl oidc_secret::Source>,
token: &str,
) -> (Option<String>, Option<policy::Permissions>) {
let caller = introspect::identify_caller(
http,
&args.introspection_url,
&args.client_id,
client_secret,
token,
)
.await
// An introspection that could not be *made* is a denial too. The
// failure modes of an HTTP call are exactly the conditions under
// which an attacker would most like this to fall open.
.unwrap_or_else(|e| {
tracing::warn!(error = ?e, "introspection failed; denying");
None
});
let deadline = tokio::time::Instant::now() + introspect::INTROSPECTION_TIMEOUT;
let caller = client_secret
.introspect(deadline, |secret| async move {
introspect::identify_caller(
http,
&args.introspection_url,
&args.client_id,
&secret,
token,
deadline,
)
.await
})
.await
// An introspection that could not be *made* is a denial too. The
// failure modes of an HTTP call are exactly the conditions under
// which an attacker would most like this to fall open.
.unwrap_or_else(|e| {
tracing::warn!(error = ?e, "introspection failed; denying");
None
});
// Admission said who; the policy says what. A caller the `IdP`
// vouches for but no rule matches is denied — see `policy`'s module
// docs for why that is deny and not "connect with nothing".
@ -484,8 +491,6 @@ mod tests {
fn the_units_own_argument_vector_parses() {
let args = responder([
"swarm-nats-auth",
"--client-secret-file",
"/run/credentials/oidc-client.secret",
"--introspection-url",
"https://auth.example/api/oidc/introspection",
"--hive-client-prefix",
@ -543,8 +548,6 @@ mod tests {
fn an_agent_suffix_without_a_leading_dash_parses_too() {
let args = responder([
"swarm-nats-auth",
"--client-secret-file",
"/run/credentials/oidc-client.secret",
"--introspection-url",
"https://auth.example/api/oidc/introspection",
"--agent-client-suffix",
@ -560,8 +563,6 @@ mod tests {
fn the_default_suffix_is_what_the_module_passes() {
let args = responder([
"swarm-nats-auth",
"--client-secret-file",
"/run/credentials/oidc-client.secret",
"--introspection-url",
"https://auth.example/api/oidc/introspection",
])
@ -619,8 +620,8 @@ mod tests {
fn the_verb_does_not_take_the_responders_flags() {
Cli::try_parse_from([
"swarm-nats-auth",
"--client-secret-file",
"/run/credentials/oidc-client.secret",
"--introspection-url",
"https://auth.example/api/oidc/introspection",
"ensure-callout-keys",
])
.expect_err("a verb and the responder's flags conflict");

397
swarm-nats-auth/src/oidc_secret.rs Normal file
View file

@ -0,0 +1,397 @@
//! This responder's own OIDC client secret, the credential it introspects
//! with. Read from the secret store as the responder starts and held in
//! memory, never in a file, `argv` or the environment.
//!
//! The secret can change under a running responder: authelia's host
//! re-publishes it when it is re-minted. So when the `IdP` refuses an
//! introspection (a non-2xx), the secret is read from the store again and,
//! when it changed, the introspection is tried once more. Both fit in the one
//! request's [`INTROSPECTION_TIMEOUT`](crate::introspect::INTROSPECTION_TIMEOUT).
//! A re-read that does not is left running: that request is denied, and the
//! next one introspects with what the re-read landed. One re-read runs at a
//! time; a refusal while one is running waits for it rather than starting
//! another.
use std::future::Future;
use std::sync::{Arc, Mutex, RwLock};
use anyhow::{Context, Result};
use futures_util::FutureExt;
use futures_util::future::{BoxFuture, Shared};
use swarm_secret_client::client::{DEFAULT_CERT_MOUNT, Settings};
use swarm_secret_client::path::{Kind, principal_prefix};
use tokio::task::AbortHandle;
use tokio::time::Instant;
use crate::introspect::Answer;
use crate::keystore::KeyStore;
/// The path under the store's KV mount holding `client_id`'s secret.
/// `swarm-secret-publisher.nix` writes it, and `natsAuthReaders` in
/// `swarm-bao.nix` grants this responder's role the read.
///
/// # Errors
/// When `client_id` is not a single path segment.
pub fn path(client_id: &str) -> Result<String> {
let prefix = principal_prefix(Kind::Service, client_id)?;
Ok(format!("{prefix}/oidc/client"))
}
/// The secret at `path`, with surrounding whitespace trimmed: the publisher
/// stores the minted file as is, and a trailing newline is not part of the
/// secret. Errors name the path, never the value.
///
/// # Errors
/// When the store cannot be read, or holds nothing or only whitespace there.
pub async fn read(store: &impl KeyStore, path: &str) -> Result<String> {
let stored = store.read(path).await?.with_context(|| {
format!("nothing is stored at {path}; swarm-secret-publish on authelia's host writes it")
})?;
let trimmed = stored.trim();
if trimmed.is_empty() {
anyhow::bail!("{path} holds an empty secret");
}
Ok(trimmed.to_owned())
}
/// Where a re-read of the secret comes from.
pub trait Source: Send + Sync + 'static {
/// The secret as the store holds it now.
fn read(&self) -> impl Future<Output = Result<String>> + Send;
}
/// The secret store, logged in to per read: a re-read can come long after
/// the start's token has expired.
pub struct Store {
settings: Settings,
cert_role: String,
path: String,
}
impl Store {
/// The secret at `path`, read as `cert_role` from the store at `settings`.
pub fn new(settings: Settings, cert_role: String, path: String) -> Self {
Self {
settings,
cert_role,
path,
}
}
}
impl Source for Store {
async fn read(&self) -> Result<String> {
let store = swarm_secret_client::SecretStore::connect(
&self.settings,
&self.cert_role,
DEFAULT_CERT_MOUNT,
)
.await
.context("logging in to the secret store")?;
read(&store, &self.path).await
}
}
/// A re-read in progress, shared by every request waiting on it.
type Reread = Shared<BoxFuture<'static, ()>>;
/// The secret in use, and the re-read that replaces it. No `Debug`: it holds
/// the secret.
pub struct OidcSecret<S> {
source: Arc<S>,
current: Arc<RwLock<String>>,
reread: Mutex<Option<(AbortHandle, Reread)>>,
}
impl<S: Source> OidcSecret<S> {
/// `initial`, re-read from `source` when the `IdP` refuses it.
pub fn new(source: S, initial: String) -> Self {
Self {
source: Arc::new(source),
current: Arc::new(RwLock::new(initial)),
reread: Mutex::new(None),
}
}
fn current(&self) -> String {
self.current.read().expect("unpoisoned").clone()
}
/// The caller `attempt` names, introspecting with the current secret.
/// When the `IdP` refuses that, the secret is re-read and, if it changed
/// and `deadline` has not passed, `attempt` runs once more with the new
/// one. `attempt` bounds itself by the same `deadline`.
///
/// # Errors
/// When an attempt could not be made at all; the caller denies.
pub async fn introspect<F, Fut>(&self, deadline: Instant, attempt: F) -> Result<Option<String>>
where
F: Fn(String) -> Fut,
Fut: Future<Output = Result<Answer>>,
{
let used = self.current();
if let Answer::Caller(caller) = attempt(used.clone()).await? {
return Ok(caller);
}
if tokio::time::timeout_at(deadline, self.reread())
.await
.is_err()
{
tracing::warn!(
"re-reading the OIDC client secret did not finish inside the introspection budget; \
denying, and the next request introspects with what it reads"
);
return Ok(None);
}
let fresh = self.current();
if fresh == used {
tracing::warn!("the stored OIDC client secret is the one the IdP refused; denying");
return Ok(None);
}
tracing::info!("re-read a changed OIDC client secret; introspecting again");
match attempt(fresh).await? {
Answer::Caller(caller) => Ok(caller),
Answer::Refused => Ok(None),
}
}
/// The running re-read, or a new one when none is running. Spawned, so it
/// finishes even when every request waiting on it has given up.
fn reread(&self) -> Reread {
let mut slot = self.reread.lock().expect("unpoisoned");
if let Some((handle, running)) = slot.as_ref()
&& !handle.is_finished()
{
return running.clone();
}
let source = Arc::clone(&self.source);
let current = Arc::clone(&self.current);
let task = tokio::spawn(async move {
match source.read().await {
Ok(secret) => *current.write().expect("unpoisoned") = secret,
Err(e) => tracing::warn!(
error = format!("{e:#}"),
"could not re-read the OIDC client secret; keeping the one in use"
),
}
});
let handle = task.abort_handle();
let shared = task.map(|_| ()).boxed().shared();
*slot = Some((handle, shared.clone()));
shared
}
}
#[cfg(test)]
mod tests {
use std::sync::atomic::{AtomicUsize, Ordering};
use std::time::Duration;
use super::*;
use crate::introspect::INTROSPECTION_TIMEOUT;
use crate::keystore::tests::Fake as FakeKeys;
/// A store holding `secret`, or failing, that takes `delay` to answer.
struct Fake {
secret: Option<&'static str>,
delay: Duration,
reads: Arc<AtomicUsize>,
}
impl Fake {
fn holding(secret: &'static str) -> Self {
Self {
secret: Some(secret),
delay: Duration::ZERO,
reads: Arc::default(),
}
}
}
impl Source for Fake {
async fn read(&self) -> Result<String> {
self.reads.fetch_add(1, Ordering::SeqCst);
tokio::time::sleep(self.delay).await;
self.secret.map(str::to_owned).context("store unreachable")
}
}
/// An `IdP` that admits as `hive-alpha` only when introspected with
/// `registered`, and refuses every other secret, counting its calls.
fn idp<'a>(
registered: &'static str,
calls: &'a AtomicUsize,
) -> impl Fn(String) -> std::future::Ready<Result<Answer>> + 'a {
move |secret| {
calls.fetch_add(1, Ordering::SeqCst);
std::future::ready(Ok(if secret == registered {
Answer::Caller(Some("hive-alpha".to_owned()))
} else {
Answer::Refused
}))
}
}
fn budget() -> Instant {
Instant::now() + INTROSPECTION_TIMEOUT
}
#[test]
fn the_path_is_the_one_the_publisher_writes() {
assert_eq!(
path("swarm-nats").expect("path"),
"swarm/services/swarm-nats/oidc/client"
);
assert!(path("../swarm-grafana").is_err());
}
#[tokio::test]
async fn the_stored_secret_is_trimmed_and_an_empty_or_absent_one_refused() {
let at = "swarm/services/swarm-nats/oidc/client";
let keys = FakeKeys::holding(&[(at, "s3cret\n")]);
assert_eq!(read(&keys, at).await.expect("stored"), "s3cret");
keys.put(at, " \n");
assert!(read(&keys, at).await.is_err());
assert!(read(&FakeKeys::default(), at).await.is_err());
}
#[tokio::test]
async fn an_answered_introspection_reads_nothing() {
let store = Fake::holding("new");
let reads = Arc::clone(&store.reads);
let secret = OidcSecret::new(store, "old".to_owned());
let calls = AtomicUsize::new(0);
let caller = secret
.introspect(budget(), idp("old", &calls))
.await
.expect("answered");
assert_eq!(caller.as_deref(), Some("hive-alpha"));
assert_eq!(calls.load(Ordering::SeqCst), 1);
assert_eq!(reads.load(Ordering::SeqCst), 0);
}
/// The rotation the re-read exists for: the `IdP` refuses the secret held
/// since start, the store holds the new one, and the same request is
/// admitted with it.
#[tokio::test]
async fn a_refused_secret_is_re_read_and_the_request_retried_once() {
let store = Fake::holding("new");
let reads = Arc::clone(&store.reads);
let secret = OidcSecret::new(store, "old".to_owned());
let calls = AtomicUsize::new(0);
let caller = secret
.introspect(budget(), idp("new", &calls))
.await
.expect("answered");
assert_eq!(caller.as_deref(), Some("hive-alpha"));
assert_eq!(calls.load(Ordering::SeqCst), 2);
assert_eq!(reads.load(Ordering::SeqCst), 1);
}
#[tokio::test]
async fn a_refusal_after_the_re_read_is_a_denial_with_no_third_attempt() {
let store = Fake::holding("newer");
let reads = Arc::clone(&store.reads);
let secret = OidcSecret::new(store, "old".to_owned());
let calls = AtomicUsize::new(0);
let caller = secret
.introspect(budget(), idp("unknown", &calls))
.await
.expect("answered");
assert_eq!(caller, None);
assert_eq!(calls.load(Ordering::SeqCst), 2);
assert_eq!(reads.load(Ordering::SeqCst), 1);
}
#[tokio::test]
async fn an_unchanged_secret_is_not_retried() {
let secret = OidcSecret::new(Fake::holding("old"), "old".to_owned());
let calls = AtomicUsize::new(0);
let caller = secret
.introspect(budget(), idp("unknown", &calls))
.await
.expect("answered");
assert_eq!(caller, None);
assert_eq!(calls.load(Ordering::SeqCst), 1);
}
#[tokio::test]
async fn a_failed_re_read_denies_and_keeps_the_secret_in_use() {
let store = Fake {
secret: None,
delay: Duration::ZERO,
reads: Arc::default(),
};
let secret = OidcSecret::new(store, "old".to_owned());
let calls = AtomicUsize::new(0);
let caller = secret
.introspect(budget(), idp("new", &calls))
.await
.expect("answered");
assert_eq!(caller, None);
assert_eq!(calls.load(Ordering::SeqCst), 1);
assert_eq!(secret.current(), "old");
}
/// A store slower than the budget: the request is denied at its
/// deadline, the re-read finishes behind it, and the next request is
/// admitted on its first attempt without reading again.
#[tokio::test(start_paused = true)]
async fn a_re_read_past_the_budget_serves_the_next_request() {
let store = Fake {
secret: Some("new"),
delay: Duration::from_secs(3),
reads: Arc::default(),
};
let reads = Arc::clone(&store.reads);
let secret = OidcSecret::new(store, "old".to_owned());
let calls = AtomicUsize::new(0);
let started = Instant::now();
let deadline = started + INTROSPECTION_TIMEOUT;
let first = secret
.introspect(deadline, idp("new", &calls))
.await
.expect("answered");
assert_eq!(first, None);
assert_eq!(Instant::now(), deadline);
assert_eq!(calls.load(Ordering::SeqCst), 1);
tokio::time::sleep(Duration::from_secs(2)).await;
let next = secret
.introspect(budget(), idp("new", &calls))
.await
.expect("answered");
assert_eq!(next.as_deref(), Some("hive-alpha"));
assert_eq!(calls.load(Ordering::SeqCst), 2);
assert_eq!(reads.load(Ordering::SeqCst), 1);
}
/// A refusal while a re-read is still running waits on that one, and is
/// admitted when it lands inside this request's budget.
#[tokio::test(start_paused = true)]
async fn a_refusal_during_a_running_re_read_joins_it() {
let store = Fake {
secret: Some("new"),
delay: Duration::from_secs(2),
reads: Arc::default(),
};
let reads = Arc::clone(&store.reads);
let secret = OidcSecret::new(store, "old".to_owned());
let calls = AtomicUsize::new(0);
let first = secret
.introspect(budget(), idp("new", &calls))
.await
.expect("answered");
assert_eq!(first, None);
let second = secret
.introspect(budget(), idp("new", &calls))
.await
.expect("answered");
assert_eq!(second.as_deref(), Some("hive-alpha"));
assert_eq!(calls.load(Ordering::SeqCst), 3);
assert_eq!(reads.load(Ordering::SeqCst), 1);
}
}