agent capabilities live in a swarm KV record, set from swarmctl
An agent's capabilities move from the hive's meta repo
(`meta/capabilities.json`, injected as `HIVE_CAPABILITIES`) into the
`capabilities` field of its key in a new `agent-permissions` KV bucket,
which swarm-controller alone writes.
- swarm-queue-client: `agent_permissions` (bucket, key, `AgentPermissions`,
direct-get read, the import subject and its wire types).
- swarm-controller: creates the bucket at start; `GET|PUT|DELETE
/api/agents/{name}/permissions` (deny_unknown_fields and unknown
capability ⇒ 422, duplicate ⇒ 400, agent must be declared by a hive);
answers each hive's one-shot import, writing a key only where there is
none and only for agents that hive's wanted state declares.
- swarmctl: `agent permissions get|set|clear`.
- swarm-nats-auth: a hive reads every key of the bucket and sends its
import on its own subject; the controller subscribes to the imports and
may answer them (`responses`).
- hive-c0re: reads each of its agents' keys every 30s into
/var/lib/hyperhive/swarm-agent-permissions.json and restarts an agent,
gracefully, when its capabilities change. Unset is the compiled-in
default (ruth: manage_root_agent; everyone else: none). A failed read
keeps the stored value. On first boot it seeds that file from
meta/capabilities.json, sends the grants to the controller, and deletes
the file once the controller answers.
Deleted: the capabilities.json reader/writer, `HIVE_CAPABILITIES` in the
rendered meta flake, ruth's capability seed, the capabilities half of the
permissions rebuild job (`PermPayload::Capabilities`/`Combined`), the
dashboard's capabilities table and `capabilities_changed` event, and the
now-unused `Capability::description`.
Part of #5170.
This commit is contained in:
parent
fcda1f4a04
commit
f862354e43
47 changed files with 1943 additions and 1146 deletions
255
swarm-queue-client/src/agent_permissions.rs
Normal file
255
swarm-queue-client/src/agent_permissions.rs
Normal file
|
|
@ -0,0 +1,255 @@
|
|||
//! The per-agent permissions KV bucket: one key per agent, holding the
|
||||
//! capabilities the swarm grants it.
|
||||
//!
|
||||
//! **swarm-controller is the only writer**, and creates the bucket when it
|
||||
//! starts, so a bucket that does not answer means the queue is not serving it,
|
||||
//! never "nothing set". A field left unset leaves the agent on its hive's
|
||||
//! compiled-in default; an empty list grants nothing.
|
||||
//!
|
||||
//! **The hive running the agent reads the key**, not the agent: capabilities
|
||||
//! are bind mounts the host sets up at container start. A hive reads with one
|
||||
//! `DIRECT.GET` per agent
|
||||
//! ([`crate::agent_permissions::direct_get_subject`]), and its grant covers
|
||||
//! every key of this bucket, read-only, for the reason [`crate::agent_limits`]
|
||||
//! gives.
|
||||
//!
|
||||
//! Keyed by the agent alone, like [`crate::agent_limits`]: an agent can move
|
||||
//! between hives, and its permissions go with it.
|
||||
//!
|
||||
//! Also carries the one-shot import a hive sends of the grants its meta repo
|
||||
//! held ([`crate::agent_permissions::import_subject`]), answered by the
|
||||
//! controller.
|
||||
|
||||
/// The KV bucket permissions are published into, keyed by agent name.
|
||||
pub const BUCKET: &str = "agent-permissions";
|
||||
|
||||
/// The stream backing [`BUCKET`]. `create_key_value` prefixes `KV_`.
|
||||
pub const STREAM: &str = "KV_agent-permissions";
|
||||
|
||||
/// The subject `agent`'s entry is stored under.
|
||||
#[must_use]
|
||||
pub fn key_subject(agent: &str) -> String {
|
||||
format!("$KV.{BUCKET}.{agent}")
|
||||
}
|
||||
|
||||
/// The request subject that reads `agent`'s current entry.
|
||||
#[must_use]
|
||||
pub fn direct_get_subject(agent: &str) -> String {
|
||||
format!("$JS.API.DIRECT.GET.{STREAM}.{}", key_subject(agent))
|
||||
}
|
||||
|
||||
const IMPORT_SUBJECT_PREFIX: &str = "$SWARM.permissions-import";
|
||||
|
||||
/// The subject `hive` sends its import on. One per hive, so the hive's grant
|
||||
/// names only its own, and the controller takes the hive from the subject
|
||||
/// rather than from the payload.
|
||||
#[must_use]
|
||||
pub fn import_subject(hive: &str) -> String {
|
||||
format!("{IMPORT_SUBJECT_PREFIX}.{hive}")
|
||||
}
|
||||
|
||||
/// Every [`import_subject`] at once, for the controller's subscription.
|
||||
#[must_use]
|
||||
pub fn import_subject_wildcard() -> String {
|
||||
import_subject("*")
|
||||
}
|
||||
|
||||
/// The hive an [`import_subject`] names, or `None` for any other subject.
|
||||
#[must_use]
|
||||
pub fn hive_from_import_subject(subject: &str) -> Option<&str> {
|
||||
let hive = subject
|
||||
.strip_prefix(IMPORT_SUBJECT_PREFIX)?
|
||||
.strip_prefix('.')?;
|
||||
(!hive.is_empty() && !hive.contains('.')).then_some(hive)
|
||||
}
|
||||
|
||||
/// How long a hive waits for the controller's answer to an import, and the
|
||||
/// queue lets the controller answer.
|
||||
pub const IMPORT_REPLY_DEADLINE: std::time::Duration = std::time::Duration::from_secs(30);
|
||||
|
||||
/// One agent's permissions: the value under `BUCKET`/`<agent>`. `None` is
|
||||
/// "not set by the swarm", which a hive reads as its default.
|
||||
///
|
||||
/// No `deny_unknown_fields`: a field a newer controller adds must not cost an
|
||||
/// older hive the fields it knows.
|
||||
#[cfg(feature = "agent-permissions")]
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||
pub struct AgentPermissions {
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub capabilities: Option<Vec<hive_sh4re::permissions::Capability>>,
|
||||
}
|
||||
|
||||
/// What a hive sends once: the capabilities its meta repo held, by agent.
|
||||
#[cfg(feature = "agent-permissions")]
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||
pub struct Import {
|
||||
pub capabilities: std::collections::BTreeMap<String, Vec<hive_sh4re::permissions::Capability>>,
|
||||
}
|
||||
|
||||
/// The controller's answer to an [`Import`].
|
||||
#[cfg(feature = "agent-permissions")]
|
||||
#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||
#[serde(tag = "state", rename_all = "kebab-case")]
|
||||
pub enum ImportReply {
|
||||
/// Every agent was handled. `written` had no key and got the hive's value;
|
||||
/// `kept` already had a key, which was left alone; `skipped` is not an
|
||||
/// agent the sending hive's wanted state declares.
|
||||
Imported {
|
||||
written: Vec<String>,
|
||||
kept: Vec<String>,
|
||||
skipped: Vec<String>,
|
||||
},
|
||||
/// Nothing is known to be written; the hive sends again later.
|
||||
Failed { error: String },
|
||||
}
|
||||
|
||||
/// Open the permissions bucket, creating it if nothing has yet.
|
||||
///
|
||||
/// **Controller-side only.** `history: 1`: a hive wants the current value.
|
||||
#[cfg(feature = "kv")]
|
||||
pub async fn open_or_create(
|
||||
client: &async_nats::Client,
|
||||
) -> Result<async_nats::jetstream::kv::Store, crate::Error> {
|
||||
let js = async_nats::jetstream::new(client.clone());
|
||||
crate::kv::open_or_create(
|
||||
&js,
|
||||
BUCKET,
|
||||
"agent-permissions",
|
||||
"Permissions the swarm controller sets for each agent".to_owned(),
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
/// What reading one agent's key found.
|
||||
#[cfg(feature = "agent-permissions")]
|
||||
#[derive(Debug)]
|
||||
pub enum Read {
|
||||
/// The bucket answered; an absent or deleted key has every field unset.
|
||||
Current(AgentPermissions),
|
||||
/// Nothing answers for the bucket's stream. Not an unset value: it says
|
||||
/// nothing about the agent's permissions.
|
||||
NoBucket,
|
||||
}
|
||||
|
||||
/// Read `agent`'s current permissions with one direct get.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// A failed request or an undecodable value. A request the queue refused for
|
||||
/// permissions reaches the client as a timeout, not as a refusal.
|
||||
#[cfg(feature = "agent-permissions")]
|
||||
pub async fn read(
|
||||
client: &async_nats::Client,
|
||||
agent: &str,
|
||||
) -> Result<Read, Box<dyn std::error::Error + Send + Sync>> {
|
||||
use async_nats::jetstream::stream::DirectGetErrorKind;
|
||||
|
||||
let js = async_nats::jetstream::new(client.clone());
|
||||
let stream = js.get_stream_no_info(STREAM).await?;
|
||||
let message = match stream.direct_get_last_for_subject(key_subject(agent)).await {
|
||||
Ok(message) => message,
|
||||
Err(e) if matches!(e.kind(), DirectGetErrorKind::NotFound) => {
|
||||
return Ok(Read::Current(AgentPermissions::default()));
|
||||
}
|
||||
Err(e)
|
||||
if matches!(
|
||||
e.kind(),
|
||||
DirectGetErrorKind::ErrorResponse(status, _)
|
||||
if status == async_nats::StatusCode::NO_RESPONDERS
|
||||
) =>
|
||||
{
|
||||
return Ok(Read::NoBucket);
|
||||
}
|
||||
Err(e) => return Err(e.into()),
|
||||
};
|
||||
Ok(Read::Current(decode(
|
||||
message
|
||||
.headers
|
||||
.get("KV-Operation")
|
||||
.map(async_nats::HeaderValue::as_str),
|
||||
&message.payload,
|
||||
)?))
|
||||
}
|
||||
|
||||
/// One stored message as a value: a KV delete or purge is stored as an empty
|
||||
/// marker carrying the `KV-Operation` header, and reads as every field unset.
|
||||
#[cfg(feature = "agent-permissions")]
|
||||
fn decode(operation: Option<&str>, payload: &[u8]) -> serde_json::Result<AgentPermissions> {
|
||||
if matches!(operation, Some("DEL" | "PURGE")) {
|
||||
return Ok(AgentPermissions::default());
|
||||
}
|
||||
serde_json::from_slice(payload)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// The names the grants in `swarm-nats-auth` are built from.
|
||||
#[test]
|
||||
fn names_match_the_queue_grant() {
|
||||
assert_eq!(STREAM, format!("KV_{BUCKET}"));
|
||||
assert_eq!(key_subject("iris"), "$KV.agent-permissions.iris");
|
||||
assert_eq!(
|
||||
direct_get_subject("iris"),
|
||||
"$JS.API.DIRECT.GET.KV_agent-permissions.$KV.agent-permissions.iris"
|
||||
);
|
||||
assert_eq!(import_subject("pr1ma"), "$SWARM.permissions-import.pr1ma");
|
||||
assert_eq!(import_subject_wildcard(), "$SWARM.permissions-import.*");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_hive_is_read_back_from_its_import_subject_only() {
|
||||
assert_eq!(
|
||||
hive_from_import_subject(&import_subject("pr1ma")),
|
||||
Some("pr1ma")
|
||||
);
|
||||
for other in [
|
||||
"$SWARM.permissions-import",
|
||||
"$SWARM.permissions-import.",
|
||||
"$SWARM.permissions-import.a.b",
|
||||
"$SWARM.permissions-importx.pr1ma",
|
||||
"$SWARM.deploy.pr1ma",
|
||||
] {
|
||||
assert_eq!(hive_from_import_subject(other), None, "{other}");
|
||||
}
|
||||
}
|
||||
|
||||
/// Unset is left out on the wire, `{}` is unset, and an empty list stays
|
||||
/// an empty list: "grants nothing" is not "not set".
|
||||
#[cfg(feature = "agent-permissions")]
|
||||
#[test]
|
||||
fn unset_and_empty_are_different_values() {
|
||||
use hive_sh4re::permissions::Capability;
|
||||
assert_eq!(
|
||||
serde_json::to_string(&AgentPermissions::default()).unwrap(),
|
||||
"{}"
|
||||
);
|
||||
let none: AgentPermissions = serde_json::from_str(r#"{"capabilities":[]}"#).unwrap();
|
||||
assert_eq!(none.capabilities, Some(Vec::new()));
|
||||
let ruth: AgentPermissions =
|
||||
serde_json::from_str(r#"{"capabilities":["manage_root_agent"]}"#).unwrap();
|
||||
assert_eq!(ruth.capabilities, Some(vec![Capability::ManageRootAgent]));
|
||||
}
|
||||
|
||||
/// Fields a newer controller adds must not cost an older hive the ones it
|
||||
/// knows; a deleted key is unset.
|
||||
#[cfg(feature = "agent-permissions")]
|
||||
#[test]
|
||||
fn decoding_tolerates_unknown_fields_and_reads_a_delete_as_unset() {
|
||||
let decoded = decode(
|
||||
None,
|
||||
br#"{"capabilities":["manage_root_agent"],"tool_groups":["meta"]}"#,
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(decoded.capabilities.map(|c| c.len()), Some(1));
|
||||
assert_eq!(
|
||||
decode(Some("DEL"), b"").unwrap(),
|
||||
AgentPermissions::default()
|
||||
);
|
||||
assert_eq!(
|
||||
decode(Some("PURGE"), b"").unwrap(),
|
||||
AgentPermissions::default()
|
||||
);
|
||||
}
|
||||
}
|
||||
Loading…
Reference in a new issue