swarm: keep the webhook secret on read errors; surface hidden KV/auth/forge failures
- swarm-controller webhook: regenerate the secret only when the file is absent. Any other read error is returned with the path; the caller already logs "webhook secret unavailable" and the endpoint answers 503. Overwriting meant every registered hook kept the old secret and every delivery failed HMAC, with nothing re-registering them. - swarm-queue-client wanted: open_read_only/watch return Result; only STREAM_NOT_FOUND on the bucket maps to None. hive-c0re's wanted pull fails the DAG node with the cause instead of logging "nothing has been declared"; the watch warns with the error chain. - swarm-nats-auth: warn when the OIDC secret re-read task dies. - swarm-controller forge: listing a repo's config PRs failing is warn, not debug. Closes #5056
This commit is contained in:
parent
d11c9537b6
commit
7129ee0015
6 changed files with 147 additions and 30 deletions
|
|
@ -118,6 +118,22 @@ pub enum Error {
|
|||
source: async_nats::jetstream::context::CreateKeyValueError,
|
||||
},
|
||||
|
||||
#[cfg(feature = "kv")]
|
||||
#[error("opening the {bucket} bucket")]
|
||||
OpenBucket {
|
||||
bucket: String,
|
||||
#[source]
|
||||
source: async_nats::jetstream::context::KeyValueError,
|
||||
},
|
||||
|
||||
#[cfg(feature = "kv")]
|
||||
#[error("watching key {key}")]
|
||||
WatchKey {
|
||||
key: String,
|
||||
#[source]
|
||||
source: async_nats::jetstream::kv::WatchError,
|
||||
},
|
||||
|
||||
#[cfg(feature = "notices")]
|
||||
#[error("creating the {stream} stream")]
|
||||
CreateStream {
|
||||
|
|
|
|||
|
|
@ -134,34 +134,76 @@ pub async fn open_or_create(
|
|||
/// **Hive-side.** `None` is the ordinary pre-publication state, not a failure:
|
||||
/// a hive holds no grant to create this bucket and must not treat its absence
|
||||
/// as a reason to converge to an empty agent set — see the module docs.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// [`Error::OpenBucket`] for every failure other than the bucket's stream not
|
||||
/// existing: a refused grant, a timeout, `JetStream` unavailable.
|
||||
#[cfg(feature = "kv")]
|
||||
pub async fn open_read_only(
|
||||
client: &async_nats::Client,
|
||||
hive: &str,
|
||||
) -> Option<async_nats::jetstream::kv::Store> {
|
||||
) -> Result<Option<async_nats::jetstream::kv::Store>, Error> {
|
||||
let js = async_nats::jetstream::new(client.clone());
|
||||
js.get_key_value(bucket(hive)).await.ok()
|
||||
let bucket = bucket(hive);
|
||||
match js.get_key_value(&bucket).await {
|
||||
Ok(store) => Ok(Some(store)),
|
||||
Err(e) if is_bucket_not_found(&e) => Ok(None),
|
||||
Err(source) => Err(Error::OpenBucket { bucket, source }),
|
||||
}
|
||||
}
|
||||
|
||||
/// `get_key_value` reports a missing bucket as `GetBucket` wrapping the
|
||||
/// stream lookup's `STREAM_NOT_FOUND`; every other kind is a real failure.
|
||||
#[cfg(feature = "kv")]
|
||||
fn is_bucket_not_found(e: &async_nats::jetstream::context::KeyValueError) -> bool {
|
||||
use async_nats::jetstream::context::{GetStreamError, GetStreamErrorKind, KeyValueErrorKind};
|
||||
|
||||
e.kind() == KeyValueErrorKind::GetBucket
|
||||
&& std::error::Error::source(e)
|
||||
.and_then(|s| s.downcast_ref::<GetStreamError>())
|
||||
.is_some_and(|s| {
|
||||
matches!(
|
||||
s.kind(),
|
||||
GetStreamErrorKind::JetStream(js)
|
||||
if js.error_code() == async_nats::jetstream::ErrorCode::STREAM_NOT_FOUND
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
/// Watch this hive's own declaration for changes.
|
||||
///
|
||||
/// **Hive-side.** `None` on the same terms as [`open_read_only`] — no bucket
|
||||
/// yet — plus one more: a watch is a `JetStream` *consumer*, so it needs a grant
|
||||
/// a plain `get` does not. A hive missing `CONSUMER.CREATE` on its own stream
|
||||
/// gets `None` here while `get` keeps working, which is why the caller must
|
||||
/// treat this as "not watching yet" and retry rather than as a dead end.
|
||||
/// **Hive-side.** `None` on the same terms as [`open_read_only`]: no bucket
|
||||
/// yet. A watch is a `JetStream` *consumer*, so it needs a grant a plain `get`
|
||||
/// does not; a hive missing `CONSUMER.CREATE` on its own stream gets
|
||||
/// [`Error::WatchKey`] here while `get` keeps working.
|
||||
///
|
||||
/// Updates only, deliberately: the boot-time read already has the current
|
||||
/// value, and a watch that replayed history would re-converge the whole
|
||||
/// declaration on every reconnect for nothing.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// [`Error::OpenBucket`] as for [`open_read_only`], and [`Error::WatchKey`]
|
||||
/// when the watch itself is refused or fails.
|
||||
#[cfg(feature = "kv")]
|
||||
pub async fn watch(
|
||||
client: &async_nats::Client,
|
||||
hive: &str,
|
||||
) -> Option<async_nats::jetstream::kv::Watch> {
|
||||
) -> Result<Option<async_nats::jetstream::kv::Watch>, Error> {
|
||||
let Some(store) = open_read_only(client, hive).await? else {
|
||||
return Ok(None);
|
||||
};
|
||||
// The bucket holds exactly one key, named for the hive — same key
|
||||
// `open_read_only`'s caller reads, so both paths address one declaration.
|
||||
open_read_only(client, hive).await?.watch(hive).await.ok()
|
||||
store
|
||||
.watch(hive)
|
||||
.await
|
||||
.map(Some)
|
||||
.map_err(|source| Error::WatchKey {
|
||||
key: hive.to_owned(),
|
||||
source,
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
|
|
|||
Loading…
Reference in a new issue