A hive reads its own declaration today and that scopes cleanly: DIRECT.GET carries the key in the subject, so the grant can name it. A *watch* cannot be scoped that way — a consumer's filter travels in the request payload, so $JS.API.CONSUMER.CREATE.<stream> grants the whole stream. With every hive in one bucket, letting a hive watch its own declaration would let it read every other hive's. One bucket per hive (hive-wanted-<hive>) makes the stream a hive may hold exactly as wide as what it is allowed to see, which is what #4006's live-watch needs. That watch is a separate change; this only moves the boundary. mara's calls, both on #4006: one stream per hive rather than teaching the auth responder a hive roster, and a wildcard for the controller — "its okay if swarm controller can theoretically override hive". A bucket name is a single subject token with no prefix matching, so no wildcard narrower than * covers N per-hive buckets; the controller's grant is account-wide by consequence, and documented as chosen rather than left to look accidental. The reader arm of #4005's key-layout guard asserted the opposite of that ruling, so it is replaced rather than deleted: the hive arm survives as no_hive_may_write_another_role_s_agent_status (with a positive control), and the_readers_grant_is_deliberately_account_wide pins the decision and names the ruling, so the width reads as chosen to whoever finds it next. Two pre-existing negative assertions were silently defanged by the rename -- they matched hive-wanted.beta and $KV.hive-wanted.alpha, strings nothing produces any more, and kept passing. Both now match current names. swarm-controller resolves the store per hive per call instead of caching one in a OnceCell: there is no single handle that serves N buckets, and declarations change on operator action rather than per tick.
194 lines
7.7 KiB
Rust
194 lines
7.7 KiB
Rust
//! The per-agent status KV bucket: one key per `(hive, agent)` pair.
|
|
//!
|
|
//! Deliberately **not** a field inside [`crate::status::BUCKET`]'s hive
|
|
//! document, even though a hive already publishes its own status there.
|
|
//! Three reasons, none of which show up until the second write:
|
|
//!
|
|
//! - **Cardinality and write rate.** Hive status is one value per hive.
|
|
//! Agent status is N values per hive, changing independently and far
|
|
//! more often — folded into the hive document, every agent's change
|
|
//! rewrites the whole thing, and two agents changing at once race on
|
|
//! the same key.
|
|
//! - **A KV key is the unit of update and of watch.** A consumer that
|
|
//! cares about one agent wants a per-key read with its own revision,
|
|
//! not a hive-wide document to diff.
|
|
//! - **Staleness means different things.** A hive can be healthy while
|
|
//! one agent is wedged, and the reverse; one timestamp cannot answer
|
|
//! both questions honestly.
|
|
//!
|
|
//! The transport *is* shared — this rides the same [`async_nats::Client`]
|
|
//! and the same auth callout as [`crate::status`] and [`crate::wanted`],
|
|
//! just a different bucket.
|
|
|
|
#[cfg(feature = "kv")]
|
|
use crate::Error;
|
|
|
|
/// The KV bucket per-agent status snapshots are published into, keyed by
|
|
/// [`key`].
|
|
///
|
|
/// A constant and not an option, for the reason [`crate::status::BUCKET`]
|
|
/// gives: writer (a hive) and reader (the controller) must name the same
|
|
/// bucket, and an option is a way for the two to disagree about which one
|
|
/// that is.
|
|
pub const BUCKET: &str = "agent-status";
|
|
|
|
/// The bucket key for one agent on one hive.
|
|
///
|
|
/// `.`-joined, and the separator is load-bearing rather than cosmetic: a KV
|
|
/// entry is published to `$KV.<bucket>.<key>`, and NATS wildcards match
|
|
/// whole `.`-delimited tokens. Two tokens make a hive's write grant
|
|
/// expressible as `$KV.agent-status.<hive>.*` — every agent of one hive and
|
|
/// nothing else. Joined by any character that is not `.`, the key is a
|
|
/// single token, and the only grants available are one exact subject per
|
|
/// agent or a bucket-wide wildcard that lets any hive overwrite any other
|
|
/// hive's agents.
|
|
///
|
|
/// Neither name can be empty or contain a `.` (`hive_types::Ident` is
|
|
/// `[a-z0-9-]`, non-empty), so the join stays unambiguous to split back
|
|
/// apart — and a leading or trailing `.` would be rejected outright by the
|
|
/// client's own KV key validation.
|
|
///
|
|
/// Plain code span above, deliberately not an intra-doc link — this crate
|
|
/// doesn't depend on `hive_types`, and a bracketed reference to its `Ident`
|
|
/// type fails `cargo doc` with "no item named `hive_types` in scope" under
|
|
/// this workspace's `docs-rustdoc` CI job, which treats broken doc links as
|
|
/// errors.
|
|
#[must_use]
|
|
pub fn key(hive: &str, agent: &str) -> String {
|
|
format!("{hive}.{agent}")
|
|
}
|
|
|
|
/// The inverse of [`key`]: split a bucket key back into `(hive, agent)`.
|
|
///
|
|
/// `None` for a key with zero or more than one `.` — a hive publishing
|
|
/// under [`key`] never produces one, so a malformed key means something
|
|
/// else wrote this bucket. Splitting on the *first* `.` would be equally
|
|
/// valid today (neither name can contain one), but this rejects rather
|
|
/// than guesses, so a future name-charset change can't silently start
|
|
/// misreading old keys.
|
|
#[must_use]
|
|
pub fn split_key(key: &str) -> Option<(&str, &str)> {
|
|
let mut parts = key.splitn(3, '.');
|
|
let hive = parts.next()?;
|
|
let agent = parts.next()?;
|
|
if parts.next().is_some() {
|
|
return None;
|
|
}
|
|
Some((hive, agent))
|
|
}
|
|
|
|
/// One agent's status snapshot, as published under [`key`].
|
|
///
|
|
/// Mirrors the tuple `container_view::read_agent_status_live` returns on
|
|
/// the publishing side — this is that value, not a recomputation of it.
|
|
/// Nothing here stamps a time: same rule as [`crate::status`], freshness is
|
|
/// derived by the reader from when the value landed in the bucket, so a
|
|
/// hive with a wrong clock only skews its own payload.
|
|
#[derive(Debug, Clone, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
|
pub struct AgentStatus {
|
|
/// The agent's free-text status (`set_status`), or `None` if it has
|
|
/// never set one, or if the container isn't running.
|
|
pub status_text: Option<String>,
|
|
/// Unix timestamp the status was last set. `None` alongside
|
|
/// `status_text: None` — the two are never independently absent.
|
|
pub status_set_at: Option<i64>,
|
|
/// Whether the container is currently running. `false` forces the two
|
|
/// fields above to `None` even if an on-disk status file is stale from
|
|
/// before the container stopped — the same "stopped containers have
|
|
/// stale state" rule `read_agent_status_live` already enforces.
|
|
pub running: bool,
|
|
}
|
|
|
|
/// Open the agent-status bucket, creating it if nothing has yet.
|
|
///
|
|
/// `history: 1`, same rationale as [`crate::status::open_or_create`]: a
|
|
/// consumer wants the last thing each agent said, not a log of everything
|
|
/// it has ever said.
|
|
#[cfg(feature = "kv")]
|
|
pub async fn open_or_create(
|
|
client: &async_nats::Client,
|
|
) -> Result<async_nats::jetstream::kv::Store, Error> {
|
|
let js = async_nats::jetstream::new(client.clone());
|
|
match js.get_key_value(BUCKET).await {
|
|
Ok(store) => Ok(store),
|
|
Err(e) => {
|
|
tracing::info!(
|
|
bucket = BUCKET,
|
|
reason = %e,
|
|
"agent-status bucket not available, creating it"
|
|
);
|
|
js.create_key_value(async_nats::jetstream::kv::Config {
|
|
bucket: BUCKET.to_owned(),
|
|
description: "Last status snapshot offered by each agent".to_owned(),
|
|
history: 1,
|
|
..Default::default()
|
|
})
|
|
.await
|
|
.map_err(|source| Error::CreateBucket {
|
|
bucket: BUCKET.to_owned(),
|
|
source,
|
|
})
|
|
}
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::{AgentStatus, BUCKET, key, split_key};
|
|
|
|
#[test]
|
|
fn key_joins_hive_and_agent_with_a_dot() {
|
|
assert_eq!(key("prod", "iris"), "prod.iris");
|
|
}
|
|
|
|
#[test]
|
|
fn the_published_subject_carries_the_hive_as_its_own_token() {
|
|
// This is what the hive's write grant `$KV.agent-status.<hive>.*`
|
|
// matches on, so the separator is pinned here rather than left as a
|
|
// property of `key`'s formatting string.
|
|
assert_eq!(
|
|
format!("$KV.{BUCKET}.{}", key("prod", "iris")),
|
|
"$KV.agent-status.prod.iris"
|
|
);
|
|
// Control: a single-token join lands in the same bucket and would
|
|
// pass any assertion that only checked the prefix — it is the hive
|
|
// token that the grant needs, and only the `.` produces one.
|
|
assert!(!format!("$KV.{BUCKET}.prod/iris").starts_with("$KV.agent-status.prod."));
|
|
}
|
|
|
|
#[test]
|
|
fn split_key_is_the_inverse_of_key() {
|
|
assert_eq!(split_key(&key("prod", "iris")), Some(("prod", "iris")));
|
|
}
|
|
|
|
#[test]
|
|
fn split_key_rejects_a_key_with_no_dot() {
|
|
assert_eq!(split_key("iris"), None);
|
|
}
|
|
|
|
#[test]
|
|
fn split_key_rejects_a_key_with_two_dots() {
|
|
assert_eq!(split_key("prod.iris.extra"), None);
|
|
}
|
|
|
|
#[test]
|
|
fn agent_status_round_trips() {
|
|
let status = AgentStatus {
|
|
status_text: Some("reviewing a pull request".to_owned()),
|
|
status_set_at: Some(1_725_000_000),
|
|
running: true,
|
|
};
|
|
let json = serde_json::to_string(&status).expect("serialises");
|
|
let decoded: AgentStatus = serde_json::from_str(&json).expect("decodes");
|
|
assert_eq!(decoded, status);
|
|
}
|
|
|
|
#[test]
|
|
fn a_stopped_agent_serialises_with_no_status() {
|
|
let status = AgentStatus::default();
|
|
assert_eq!(
|
|
serde_json::to_string(&status).expect("serialises"),
|
|
r#"{"status_text":null,"status_set_at":null,"running":false}"#
|
|
);
|
|
}
|
|
}
|