swarm-queue-client: re-read a held OIDC client secret when the IdP refuses it
hive-c0re (`hive-<name>`), swarm-controller and swarm-review-classifier read their queue client secret from the store once and held it as `ClientSecret::Value` for the life of the process. After authelia re-minted a secret and swarm-secret-publish stored the new one, every later token mint presented the old value and was refused until the daemon restarted. `ClientSecret::Stored` replaces `Value`: a `StoredSecret` holds the secret in memory with a caller-supplied read callback, so the crate takes no store dependency. When the token endpoint answers 400 or 401, `mint_token` (and its blocking sibling, used by the controller's OTLP exporters) re-reads it and, if the store holds a different secret, mints once more with it. One re-read runs at a time and is spawned so it outlives a caller that gives up. A re-read that fails or finds the refused value delays the next one, 5 s doubling to 5 min, so a secret that stays wrong costs one store read per delay. Logs name the client id and never the secret. swarm-nats-auth's `oidc_secret` was the template; its re-read machinery now sits on `StoredSecret`, so the tree has one copy. The responder gains the same re-read backoff. The controller keeps its `IDENTITY` OnceLock: it now holds a `QueueConfig` whose secret is re-read, and the lazily built vcs-metrics exporter reads the identity from it. agent_renewal.rs's module doc said a re-minted agent queue secret only reaches the agent at its next start; hive-agent reads the store on every connection attempt after its first, so the doc now says that. Part of #5017 (slice Q).
This commit is contained in:
atlas
mara
parent
be9b2475a9
commit
a6935e57c3
11 changed files with 819 additions and 376 deletions
|
|
@ -385,16 +385,17 @@ async fn load_callout_keys(
|
|||
async fn load_client_secret(
|
||||
keys: &impl keystore::KeyStore,
|
||||
args: &Args,
|
||||
) -> anyhow::Result<oidc_secret::OidcSecret<oidc_secret::Store>> {
|
||||
) -> anyhow::Result<oidc_secret::OidcSecret> {
|
||||
let path = oidc_secret::path(&args.client_id)?;
|
||||
let secret = oidc_secret::read(keys, &path).await?;
|
||||
tracing::info!(%path, "loaded the OIDC client secret");
|
||||
let source = oidc_secret::Store::new(
|
||||
Ok(oidc_secret::OidcSecret::from_store(
|
||||
&args.client_id,
|
||||
secret,
|
||||
swarm_secret_client::client::Settings::from_env()?,
|
||||
args.store_cert_role.clone(),
|
||||
path,
|
||||
);
|
||||
Ok(oidc_secret::OidcSecret::new(source, secret))
|
||||
))
|
||||
}
|
||||
|
||||
/// The `ensure-callout-keys` verb: see [`xkey::ensure`] and [`nkey::ensure`].
|
||||
|
|
@ -436,7 +437,7 @@ async fn introspected(
|
|||
policy: &policy::Policy,
|
||||
http: &reqwest::Client,
|
||||
args: &Args,
|
||||
client_secret: &oidc_secret::OidcSecret<impl oidc_secret::Source>,
|
||||
client_secret: &oidc_secret::OidcSecret,
|
||||
token: &str,
|
||||
) -> (Option<String>, Option<policy::Permissions>) {
|
||||
let deadline = tokio::time::Instant::now() + introspect::INTROSPECTION_TIMEOUT;
|
||||
|
|
|
|||
|
|
@ -4,24 +4,20 @@
|
|||
//!
|
||||
//! The secret can change under a running responder: authelia's host
|
||||
//! re-publishes it when it is re-minted. So when the `IdP` refuses an
|
||||
//! introspection (a non-2xx), the secret is read from the store again and,
|
||||
//! when it changed, the introspection is tried once more. Both fit in the one
|
||||
//! request's [`INTROSPECTION_TIMEOUT`](crate::introspect::INTROSPECTION_TIMEOUT).
|
||||
//! A re-read that does not is left running: that request is denied, and the
|
||||
//! next one introspects with what the re-read landed. One re-read runs at a
|
||||
//! time; a refusal while one is running waits for it rather than starting
|
||||
//! another.
|
||||
//! introspection (a non-2xx), the secret is read from the store again
|
||||
//! ([`StoredSecret::after_refusal`]) and, when it changed, the introspection
|
||||
//! is tried once more. Both fit in the one request's
|
||||
//! [`INTROSPECTION_TIMEOUT`](crate::introspect::INTROSPECTION_TIMEOUT). A
|
||||
//! re-read that does not is left running: that request is denied, and the
|
||||
//! next one introspects with what the re-read landed.
|
||||
|
||||
use std::future::Future;
|
||||
use std::sync::{Arc, Mutex, RwLock};
|
||||
use std::time::Duration;
|
||||
use std::sync::Arc;
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
use futures_util::FutureExt;
|
||||
use futures_util::future::{BoxFuture, Shared};
|
||||
use swarm_queue_client::stored_secret::StoredSecret;
|
||||
use swarm_secret_client::client::{DEFAULT_CERT_MOUNT, Settings};
|
||||
use swarm_secret_client::path::{Kind, principal_prefix};
|
||||
use tokio::task::AbortHandle;
|
||||
use tokio::time::Instant;
|
||||
|
||||
use crate::introspect::Answer;
|
||||
|
|
@ -55,73 +51,41 @@ pub async fn read(store: &impl KeyStore, path: &str) -> Result<String> {
|
|||
Ok(trimmed.to_owned())
|
||||
}
|
||||
|
||||
/// Where a re-read of the secret comes from.
|
||||
pub trait Source: Send + Sync + 'static {
|
||||
/// The secret as the store holds it now.
|
||||
fn read(&self) -> impl Future<Output = Result<String>> + Send;
|
||||
/// The secret in use, and how it is re-read.
|
||||
pub struct OidcSecret {
|
||||
secret: StoredSecret,
|
||||
}
|
||||
|
||||
/// The secret store, logged in to per read: a re-read can come long after
|
||||
/// the start's token has expired.
|
||||
pub struct Store {
|
||||
settings: Settings,
|
||||
cert_role: String,
|
||||
path: String,
|
||||
}
|
||||
|
||||
impl Store {
|
||||
/// The secret at `path`, read as `cert_role` from the store at `settings`.
|
||||
pub fn new(settings: Settings, cert_role: String, path: String) -> Self {
|
||||
Self {
|
||||
settings,
|
||||
cert_role,
|
||||
path,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Source for Store {
|
||||
async fn read(&self) -> Result<String> {
|
||||
let store = swarm_secret_client::SecretStore::connect(
|
||||
&self.settings,
|
||||
&self.cert_role,
|
||||
DEFAULT_CERT_MOUNT,
|
||||
)
|
||||
.await
|
||||
.context("logging in to the secret store")?;
|
||||
read(&store, &self.path).await
|
||||
}
|
||||
}
|
||||
|
||||
/// The longest one re-read runs. The store client sets no timeout of its
|
||||
/// own, and while a re-read runs every refusal waits on it instead of
|
||||
/// starting another.
|
||||
const REREAD_TIMEOUT: Duration = Duration::from_secs(10);
|
||||
|
||||
/// A re-read in progress, shared by every request waiting on it. It yields
|
||||
/// whether the store was read; a failed or timed-out read logs for itself.
|
||||
type Reread = Shared<BoxFuture<'static, bool>>;
|
||||
|
||||
/// The secret in use, and the re-read that replaces it. No `Debug`: it holds
|
||||
/// the secret.
|
||||
pub struct OidcSecret<S> {
|
||||
source: Arc<S>,
|
||||
current: Arc<RwLock<String>>,
|
||||
reread: Mutex<Option<(AbortHandle, Reread)>>,
|
||||
}
|
||||
|
||||
impl<S: Source> OidcSecret<S> {
|
||||
/// `initial`, re-read from `source` when the `IdP` refuses it.
|
||||
pub fn new(source: S, initial: String) -> Self {
|
||||
Self {
|
||||
source: Arc::new(source),
|
||||
current: Arc::new(RwLock::new(initial)),
|
||||
reread: Mutex::new(None),
|
||||
}
|
||||
impl OidcSecret {
|
||||
/// `client_id`'s secret, `initial` until a re-read replaces it. A re-read
|
||||
/// logs in to the store at `settings` as `cert_role` and reads `path`: it
|
||||
/// can come long after the start's token has expired.
|
||||
pub fn from_store(
|
||||
client_id: &str,
|
||||
initial: String,
|
||||
settings: Settings,
|
||||
cert_role: String,
|
||||
path: String,
|
||||
) -> Self {
|
||||
let source = Arc::new((settings, cert_role, path));
|
||||
Self::new(StoredSecret::new(client_id, initial, move || {
|
||||
let source = Arc::clone(&source);
|
||||
async move {
|
||||
let (settings, cert_role, path) = &*source;
|
||||
let store = swarm_secret_client::SecretStore::connect(
|
||||
settings,
|
||||
cert_role,
|
||||
DEFAULT_CERT_MOUNT,
|
||||
)
|
||||
.await
|
||||
.context("logging in to the secret store")?;
|
||||
Ok(read(&store, path).await?)
|
||||
}
|
||||
}))
|
||||
}
|
||||
|
||||
fn current(&self) -> String {
|
||||
self.current.read().expect("unpoisoned").clone()
|
||||
fn new(secret: StoredSecret) -> Self {
|
||||
Self { secret }
|
||||
}
|
||||
|
||||
/// The caller `attempt` names, introspecting with the current secret.
|
||||
|
|
@ -136,82 +100,26 @@ impl<S: Source> OidcSecret<S> {
|
|||
F: Fn(String) -> Fut,
|
||||
Fut: Future<Output = Result<Answer>>,
|
||||
{
|
||||
let used = self.current();
|
||||
let used = self.secret.current();
|
||||
if let Answer::Caller(caller) = attempt(used.clone()).await? {
|
||||
return Ok(caller);
|
||||
}
|
||||
match tokio::time::timeout_at(deadline, self.reread()).await {
|
||||
Err(_) => {
|
||||
tracing::warn!(
|
||||
"re-reading the OIDC client secret did not finish inside the introspection budget; \
|
||||
denying, and the next request introspects with what it reads"
|
||||
);
|
||||
return Ok(None);
|
||||
}
|
||||
Ok(false) => return Ok(None),
|
||||
Ok(true) => {}
|
||||
}
|
||||
let fresh = self.current();
|
||||
if fresh == used {
|
||||
tracing::warn!("the stored OIDC client secret is the one the IdP refused; denying");
|
||||
let Ok(fresh) = tokio::time::timeout_at(deadline, self.secret.after_refusal(&used)).await
|
||||
else {
|
||||
tracing::warn!(
|
||||
"re-reading the OIDC client secret did not finish inside the introspection budget; \
|
||||
denying, and the next request introspects with what it reads"
|
||||
);
|
||||
return Ok(None);
|
||||
}
|
||||
tracing::info!("re-read a changed OIDC client secret; introspecting again");
|
||||
};
|
||||
let Some(fresh) = fresh else {
|
||||
return Ok(None);
|
||||
};
|
||||
match attempt(fresh).await? {
|
||||
Answer::Caller(caller) => Ok(caller),
|
||||
Answer::Refused => Ok(None),
|
||||
}
|
||||
}
|
||||
|
||||
/// The running re-read, or a new one when none is running. Spawned, so it
|
||||
/// finishes even when every request waiting on it has given up.
|
||||
fn reread(&self) -> Reread {
|
||||
let mut slot = self.reread.lock().expect("unpoisoned");
|
||||
if let Some((handle, running)) = slot.as_ref()
|
||||
&& !handle.is_finished()
|
||||
{
|
||||
return running.clone();
|
||||
}
|
||||
let source = Arc::clone(&self.source);
|
||||
let current = Arc::clone(&self.current);
|
||||
let task = tokio::spawn(async move {
|
||||
match tokio::time::timeout(REREAD_TIMEOUT, source.read()).await {
|
||||
Ok(Ok(secret)) => {
|
||||
*current.write().expect("unpoisoned") = secret;
|
||||
true
|
||||
}
|
||||
Ok(Err(e)) => {
|
||||
tracing::warn!(
|
||||
error = format!("{e:#}"),
|
||||
"could not re-read the OIDC client secret; keeping the one in use"
|
||||
);
|
||||
false
|
||||
}
|
||||
Err(_) => {
|
||||
tracing::warn!(
|
||||
timeout = ?REREAD_TIMEOUT,
|
||||
"re-reading the OIDC client secret timed out; keeping the one in use"
|
||||
);
|
||||
false
|
||||
}
|
||||
}
|
||||
});
|
||||
let handle = task.abort_handle();
|
||||
let shared = task
|
||||
.map(|read| {
|
||||
read.unwrap_or_else(|e| {
|
||||
tracing::warn!(
|
||||
error = %e,
|
||||
"the OIDC client secret re-read task died; keeping the one in use"
|
||||
);
|
||||
false
|
||||
})
|
||||
})
|
||||
.boxed()
|
||||
.shared();
|
||||
*slot = Some((handle, shared.clone()));
|
||||
shared
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
|
@ -223,29 +131,23 @@ mod tests {
|
|||
use crate::introspect::INTROSPECTION_TIMEOUT;
|
||||
use crate::keystore::tests::Fake as FakeKeys;
|
||||
|
||||
/// A store holding `secret`, or failing, that takes `delay` to answer.
|
||||
struct Fake {
|
||||
secret: Option<&'static str>,
|
||||
/// `initial` in use, re-read from a store holding `stored` that takes
|
||||
/// `delay` to answer; and the count of its reads.
|
||||
fn secret(
|
||||
initial: &str,
|
||||
stored: &'static str,
|
||||
delay: Duration,
|
||||
reads: Arc<AtomicUsize>,
|
||||
}
|
||||
|
||||
impl Fake {
|
||||
fn holding(secret: &'static str) -> Self {
|
||||
Self {
|
||||
secret: Some(secret),
|
||||
delay: Duration::ZERO,
|
||||
reads: Arc::default(),
|
||||
) -> (OidcSecret, Arc<AtomicUsize>) {
|
||||
let reads = Arc::new(AtomicUsize::new(0));
|
||||
let counted = Arc::clone(&reads);
|
||||
let secret = StoredSecret::new("swarm-nats", initial.to_owned(), move || {
|
||||
counted.fetch_add(1, Ordering::SeqCst);
|
||||
async move {
|
||||
tokio::time::sleep(delay).await;
|
||||
Ok(stored.to_owned())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl Source for Fake {
|
||||
async fn read(&self) -> Result<String> {
|
||||
self.reads.fetch_add(1, Ordering::SeqCst);
|
||||
tokio::time::sleep(self.delay).await;
|
||||
self.secret.map(str::to_owned).context("store unreachable")
|
||||
}
|
||||
});
|
||||
(OidcSecret::new(secret), reads)
|
||||
}
|
||||
|
||||
/// An `IdP` that admits as `hive-alpha` only when introspected with
|
||||
|
|
@ -289,9 +191,7 @@ mod tests {
|
|||
|
||||
#[tokio::test]
|
||||
async fn an_answered_introspection_reads_nothing() {
|
||||
let store = Fake::holding("new");
|
||||
let reads = Arc::clone(&store.reads);
|
||||
let secret = OidcSecret::new(store, "old".to_owned());
|
||||
let (secret, reads) = secret("old", "new", Duration::ZERO);
|
||||
let calls = AtomicUsize::new(0);
|
||||
let caller = secret
|
||||
.introspect(budget(), idp("old", &calls))
|
||||
|
|
@ -307,9 +207,7 @@ mod tests {
|
|||
/// admitted with it.
|
||||
#[tokio::test]
|
||||
async fn a_refused_secret_is_re_read_and_the_request_retried_once() {
|
||||
let store = Fake::holding("new");
|
||||
let reads = Arc::clone(&store.reads);
|
||||
let secret = OidcSecret::new(store, "old".to_owned());
|
||||
let (secret, reads) = secret("old", "new", Duration::ZERO);
|
||||
let calls = AtomicUsize::new(0);
|
||||
let caller = secret
|
||||
.introspect(budget(), idp("new", &calls))
|
||||
|
|
@ -322,9 +220,7 @@ mod tests {
|
|||
|
||||
#[tokio::test]
|
||||
async fn a_refusal_after_the_re_read_is_a_denial_with_no_third_attempt() {
|
||||
let store = Fake::holding("newer");
|
||||
let reads = Arc::clone(&store.reads);
|
||||
let secret = OidcSecret::new(store, "old".to_owned());
|
||||
let (secret, reads) = secret("old", "newer", Duration::ZERO);
|
||||
let calls = AtomicUsize::new(0);
|
||||
let caller = secret
|
||||
.introspect(budget(), idp("unknown", &calls))
|
||||
|
|
@ -337,7 +233,7 @@ mod tests {
|
|||
|
||||
#[tokio::test]
|
||||
async fn an_unchanged_secret_is_not_retried() {
|
||||
let secret = OidcSecret::new(Fake::holding("old"), "old".to_owned());
|
||||
let (secret, _) = secret("old", "old", Duration::ZERO);
|
||||
let calls = AtomicUsize::new(0);
|
||||
let caller = secret
|
||||
.introspect(budget(), idp("unknown", &calls))
|
||||
|
|
@ -347,99 +243,12 @@ mod tests {
|
|||
assert_eq!(calls.load(Ordering::SeqCst), 1);
|
||||
}
|
||||
|
||||
thread_local! {
|
||||
static LOG: std::cell::RefCell<Vec<u8>> = const { std::cell::RefCell::new(Vec::new()) };
|
||||
}
|
||||
|
||||
/// What `tracing` records on the calling thread. The subscriber is
|
||||
/// process-wide, because a scoped one loses to the callsite cache when
|
||||
/// another test reaches the same event first; the buffer is per thread,
|
||||
/// which is where a current-thread test runtime runs the spawned re-read.
|
||||
struct Logs;
|
||||
|
||||
impl std::io::Write for Logs {
|
||||
fn write(&mut self, buf: &[u8]) -> std::io::Result<usize> {
|
||||
LOG.with(|log| log.borrow_mut().extend_from_slice(buf));
|
||||
Ok(buf.len())
|
||||
}
|
||||
|
||||
fn flush(&mut self) -> std::io::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl Logs {
|
||||
/// Starts this thread's capture empty.
|
||||
fn capture() {
|
||||
static INSTALLED: std::sync::Once = std::sync::Once::new();
|
||||
INSTALLED.call_once(|| {
|
||||
tracing_subscriber::fmt()
|
||||
.with_writer(|| Logs)
|
||||
.with_ansi(false)
|
||||
.init();
|
||||
});
|
||||
LOG.with(|log| log.borrow_mut().clear());
|
||||
}
|
||||
|
||||
fn text() -> String {
|
||||
LOG.with(|log| String::from_utf8(log.borrow().clone()).expect("utf-8"))
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_failed_re_read_denies_and_keeps_the_secret_in_use() {
|
||||
Logs::capture();
|
||||
let store = Fake {
|
||||
secret: None,
|
||||
delay: Duration::ZERO,
|
||||
reads: Arc::default(),
|
||||
};
|
||||
let secret = OidcSecret::new(store, "old".to_owned());
|
||||
let calls = AtomicUsize::new(0);
|
||||
let caller = secret
|
||||
.introspect(budget(), idp("new", &calls))
|
||||
.await
|
||||
.expect("answered");
|
||||
assert_eq!(caller, None);
|
||||
assert_eq!(calls.load(Ordering::SeqCst), 1);
|
||||
assert_eq!(secret.current(), "old");
|
||||
|
||||
let text = Logs::text();
|
||||
assert!(text.contains("could not re-read the OIDC client secret"));
|
||||
assert!(!text.contains("is the one the IdP refused"), "{text}");
|
||||
assert!(!text.contains("old"), "no secret in the log: {text}");
|
||||
}
|
||||
|
||||
/// The store was read and holds what the `IdP` already refused: the one
|
||||
/// case that says so.
|
||||
#[tokio::test]
|
||||
async fn the_unchanged_secret_is_logged_as_the_one_the_idp_refused() {
|
||||
Logs::capture();
|
||||
let secret = OidcSecret::new(Fake::holding("old"), "old".to_owned());
|
||||
let calls = AtomicUsize::new(0);
|
||||
let caller = secret
|
||||
.introspect(budget(), idp("new", &calls))
|
||||
.await
|
||||
.expect("answered");
|
||||
assert_eq!(caller, None);
|
||||
|
||||
let text = Logs::text();
|
||||
assert!(text.contains("is the one the IdP refused"), "{text}");
|
||||
assert!(!text.contains("could not re-read"), "{text}");
|
||||
}
|
||||
|
||||
/// A store slower than the budget: the request is denied at its
|
||||
/// deadline, the re-read finishes behind it, and the next request is
|
||||
/// admitted on its first attempt without reading again.
|
||||
#[tokio::test(start_paused = true)]
|
||||
async fn a_re_read_past_the_budget_serves_the_next_request() {
|
||||
let store = Fake {
|
||||
secret: Some("new"),
|
||||
delay: Duration::from_secs(3),
|
||||
reads: Arc::default(),
|
||||
};
|
||||
let reads = Arc::clone(&store.reads);
|
||||
let secret = OidcSecret::new(store, "old".to_owned());
|
||||
let (secret, reads) = secret("old", "new", Duration::from_secs(3));
|
||||
let calls = AtomicUsize::new(0);
|
||||
|
||||
let started = Instant::now();
|
||||
|
|
@ -462,64 +271,6 @@ mod tests {
|
|||
assert_eq!(reads.load(Ordering::SeqCst), 1);
|
||||
}
|
||||
|
||||
/// A refusal while a re-read is still running waits on that one, and is
|
||||
/// admitted when it lands inside this request's budget.
|
||||
#[tokio::test(start_paused = true)]
|
||||
async fn a_refusal_during_a_running_re_read_joins_it() {
|
||||
let store = Fake {
|
||||
secret: Some("new"),
|
||||
delay: Duration::from_secs(2),
|
||||
reads: Arc::default(),
|
||||
};
|
||||
let reads = Arc::clone(&store.reads);
|
||||
let secret = OidcSecret::new(store, "old".to_owned());
|
||||
let calls = AtomicUsize::new(0);
|
||||
|
||||
let first = secret
|
||||
.introspect(budget(), idp("new", &calls))
|
||||
.await
|
||||
.expect("answered");
|
||||
assert_eq!(first, None);
|
||||
|
||||
let second = secret
|
||||
.introspect(budget(), idp("new", &calls))
|
||||
.await
|
||||
.expect("answered");
|
||||
assert_eq!(second.as_deref(), Some("hive-alpha"));
|
||||
assert_eq!(calls.load(Ordering::SeqCst), 3);
|
||||
assert_eq!(reads.load(Ordering::SeqCst), 1);
|
||||
}
|
||||
|
||||
/// A re-read that stalls is given up at [`REREAD_TIMEOUT`]: the secret in
|
||||
/// use stays, and the next refusal starts a fresh re-read rather than
|
||||
/// waiting on the stalled one.
|
||||
#[tokio::test(start_paused = true)]
|
||||
async fn a_stalled_re_read_is_dropped_at_its_timeout() {
|
||||
let store = Fake {
|
||||
secret: Some("new"),
|
||||
delay: REREAD_TIMEOUT * 6,
|
||||
reads: Arc::default(),
|
||||
};
|
||||
let reads = Arc::clone(&store.reads);
|
||||
let secret = OidcSecret::new(store, "old".to_owned());
|
||||
let calls = AtomicUsize::new(0);
|
||||
|
||||
let first = secret
|
||||
.introspect(budget(), idp("new", &calls))
|
||||
.await
|
||||
.expect("answered");
|
||||
assert_eq!(first, None);
|
||||
|
||||
tokio::time::sleep(REREAD_TIMEOUT).await;
|
||||
assert_eq!(secret.current(), "old");
|
||||
let second = secret
|
||||
.introspect(budget(), idp("new", &calls))
|
||||
.await
|
||||
.expect("answered");
|
||||
assert_eq!(second, None);
|
||||
assert_eq!(reads.load(Ordering::SeqCst), 2);
|
||||
}
|
||||
|
||||
/// An `IdP` on a loopback port that admits as `hive-alpha` a request
|
||||
/// whose basic auth is `swarm-nats:{registered}`, and answers any other
|
||||
/// with a 401.
|
||||
|
|
@ -558,9 +309,7 @@ mod tests {
|
|||
.no_proxy()
|
||||
.build()
|
||||
.expect("client");
|
||||
let store = Fake::holding("new");
|
||||
let reads = Arc::clone(&store.reads);
|
||||
let secret = OidcSecret::new(store, "old".to_owned());
|
||||
let (secret, reads) = secret("old", "new", Duration::ZERO);
|
||||
let deadline = budget();
|
||||
let caller = secret
|
||||
.introspect(deadline, |own| {
|
||||
|
|
|
|||
Loading…
Reference in a new issue