vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

agent capabilities live in a swarm KV record, set from swarmctl

Browse source 
An agent's capabilities move from the hive's meta repo
(`meta/capabilities.json`, injected as `HIVE_CAPABILITIES`) into the
`capabilities` field of its key in a new `agent-permissions` KV bucket,
which swarm-controller alone writes.

- swarm-queue-client: `agent_permissions` (bucket, key, `AgentPermissions`,
  direct-get read, the import subject and its wire types).
- swarm-controller: creates the bucket at start; `GET|PUT|DELETE
  /api/agents/{name}/permissions` (deny_unknown_fields and unknown
  capability ⇒ 422, duplicate ⇒ 400, agent must be declared by a hive);
  answers each hive's one-shot import, writing a key only where there is
  none and only for agents that hive's wanted state declares.
- swarmctl: `agent permissions get|set|clear`.
- swarm-nats-auth: a hive reads every key of the bucket and sends its
  import on its own subject; the controller subscribes to the imports and
  may answer them (`responses`).
- hive-c0re: reads each of its agents' keys every 30s into
  /var/lib/hyperhive/swarm-agent-permissions.json and restarts an agent,
  gracefully, when its capabilities change. Unset is the compiled-in
  default (ruth: manage_root_agent; everyone else: none). A failed read
  keeps the stored value. On first boot it seeds that file from
  meta/capabilities.json, sends the grants to the controller, and deletes
  the file once the controller answers.

Deleted: the capabilities.json reader/writer, `HIVE_CAPABILITIES` in the
rendered meta flake, ruth's capability seed, the capabilities half of the
permissions rebuild job (`PermPayload::Capabilities`/`Combined`), the
dashboard's capabilities table and `capabilities_changed` event, and the
now-unused `Capability::description`.

Part of #5170.
This commit is contained in:
atlas 2026-10-15 00:24:41 +02:00 • committed by mara
commit f862354e43
47 changed files with 1943 additions and 1146 deletions
Download patch file Download diff file Expand all files Collapse all files

1
hive-c0re/Cargo.toml
View file

@ -57,6 +57,7 @@ swarm-queue-client = { workspace = true, features = [
"kv",
"notices",
"agent-limits",
"agent-permissions",
] }
swarm-secret-client.workspace = true
tokio.workspace = true

313
hive-c0re/src/agent_config/capabilities.rs
View file

@ -1,262 +1,109 @@
//! Per-agent capability configuration. Stored at
//! `/var/lib/hyperhive/meta/capabilities.json` alongside `topology.json`
//! and `tool-groups.json`.
//! The capabilities the swarm grants this hive's agents, as last read from the
//! `agent-permissions` bucket by `workers::swarm_permissions`. Stored at
//! `/var/lib/hyperhive/swarm-agent-permissions.json`, outside the meta repo:
//! the swarm is the record, this file is the hive's last read of it.
//!
//! Format: a JSON object mapping agent name to an array of
//! `hive_sh4re::permissions::Capability` `snake_case` strings:
//! Format: agent name → the stored
//! [`swarm_queue_client::agent_permissions::AgentPermissions`]. An agent with
//! no entry, or an entry with `capabilities` unset, holds its default
//! ([`default_for`]); `"capabilities": []` holds none.
//!
//! ```json
//! {
//! "atlas": ["manage_root_agent"]
//! }
//! ```
//!
//! An absent entry (or an absent file) means "no extra capabilities".
//! `render_flake` in `meta.rs` reads this file and injects
//! `HIVE_CAPABILITIES` into each agent's systemd service env; absent
//! entries emit no env var so agents without capabilities don't trigger
//! a spurious rebuild.
//!
//! Write path: `set_caps` is called from the dashboard action handler
//! that the operator uses to grant/revoke capabilities per agent.
//! Read path: `lifecycle::host_config::set_nspawn_flags`, on every container
//! start, which is why a change takes a restart.
use hive_sh4re::permissions::Capability;
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
const CAPABILITIES_FILE: &str = "capabilities.json";
use hive_sh4re::permissions::Capability;
use swarm_queue_client::agent_permissions::AgentPermissions;
const FILE: &str = "swarm-agent-permissions.json";
pub type Stored = BTreeMap<String, AgentPermissions>;
#[must_use]
pub fn capabilities_path() -> PathBuf {
crate::paths::meta_root().join(CAPABILITIES_FILE)
pub fn path() -> PathBuf {
crate::paths::state_root().join(FILE)
}
/// True if `name` matches a recognised [`Capability`], case-insensitive
/// (same comparison [`has_cap`] uses).
/// Read the stored permissions. An absent file is the empty map: every agent
/// on its default. A file that exists but can't be read or parsed is an error.
pub fn read_at(path: &Path) -> std::io::Result<Stored> {
super::read_map(path)
}
/// Persist the permissions atomically.
pub fn write_at(path: &Path, stored: &Stored) -> std::io::Result<()> {
super::write_map(path, stored)
}
/// The capabilities an agent holds when the swarm sets none: `ruth` recovers
/// the other agents, every other agent holds nothing.
#[must_use]
fn is_known(name: &str) -> bool {
Capability::ALL
.iter()
.any(|cap| name.eq_ignore_ascii_case(<&str>::from(*cap)))
}
/// Drop any capability string in `map` that isn't a recognised
/// [`Capability`], warning per dropped string so a typo'd or stale
/// grant doesn't look like it worked. An agent left with no
/// capabilities is removed entirely, matching [`set_caps`]'s
/// empty-vec behaviour. Returns whether anything was dropped.
fn prune_unknown(map: &mut BTreeMap<String, Vec<String>>) -> bool {
let mut dropped_any = false;
map.retain(|agent, caps| {
caps.retain(|cap| {
let known = is_known(cap);
if !known {
dropped_any = true;
tracing::warn!(agent, capability = %cap, "dropping unrecognised capability");
}
known
});
!caps.is_empty()
});
dropped_any
}
/// Read the per-agent capability map. An absent file is the empty map —
/// callers treat a missing entry as "no extra capabilities". A file
/// that exists but can't be read or parsed is an error. Any entry that
/// isn't a recognised [`Capability`] is dropped (with a `warn!`) from
/// what's returned — a stale or typo'd name is never honoured — but
/// `read` itself never writes. Unknown names stay on disk until the next
/// write: `set_caps` always rewrites the file, `remove_agent` only when
/// the agent had an entry.
pub fn read() -> std::io::Result<BTreeMap<String, Vec<String>>> {
read_from(&capabilities_path())
}
fn read_from(path: &Path) -> std::io::Result<BTreeMap<String, Vec<String>>> {
let mut map = super::read_map(path)?;
prune_unknown(&mut map);
Ok(map)
}
/// Look up the configured capabilities for one agent in the file at
/// `path` (normally [`capabilities_path`]). Returns an empty vec when
/// the agent has no entry. Errors as [`read`] does.
pub fn caps_for(path: &Path, name: &str) -> std::io::Result<Vec<String>> {
Ok(read_from(path)?.get(name).cloned().unwrap_or_default())
}
/// Check whether an agent holds a specific capability, per the file at
/// `path` (normally [`capabilities_path`]). Errors as [`read`] does.
pub fn has_cap(
path: &Path,
name: &str,
cap: hive_sh4re::permissions::Capability,
) -> std::io::Result<bool> {
Ok(caps_for(path, name)?
.iter()
.any(|s| s.eq_ignore_ascii_case(<&str>::from(cap))))
}
/// Filter `caps` down to recognised [`Capability`] names (warning per
/// drop) and apply the result to `current` for `name`: an empty
/// result — whether `caps` started empty or every name in it was
/// unknown — removes the entry, matching `read`'s empty-entry
/// convention. Split out from [`set_caps`] so the decision logic is
/// testable without touching the real capabilities file.
fn apply_known_caps(current: &mut BTreeMap<String, Vec<String>>, name: &str, caps: &[String]) {
let known: Vec<String> = caps
.iter()
.filter(|cap| {
let ok = is_known(cap);
if !ok {
tracing::warn!(agent = name, capability = %cap, "dropping unrecognised capability");
}
ok
})
.cloned()
.collect();
if known.is_empty() {
current.remove(name);
pub fn default_for(agent: &str) -> Vec<Capability> {
if agent == crate::lifecycle::MANAGER_NAME {
vec![Capability::ManageRootAgent]
} else {
current.insert(name.to_owned(), known);
Vec::new()
}
}
/// Set the capabilities for one agent and persist the map. Any name
/// that isn't a recognised [`Capability`] is dropped (with a `warn!`)
/// rather than written — an unknown grant should never look like it
/// took effect. An empty `caps` vec, or one that becomes empty after
/// dropping unknown names, removes the entry (agent has no
/// capabilities). Fails without writing if the existing file can't be
/// read or parsed.
pub fn set_caps(name: &str, caps: &[String]) -> std::io::Result<()> {
set_caps_at(&capabilities_path(), name, caps)
/// The capabilities `agent` holds under `record`.
#[must_use]
pub fn effective(agent: &str, record: Option<&AgentPermissions>) -> Vec<Capability> {
record
.and_then(|r| r.capabilities.clone())
.unwrap_or_else(|| default_for(agent))
}
fn set_caps_at(path: &Path, name: &str, caps: &[String]) -> std::io::Result<()> {
let mut current = read_from(path)?;
apply_known_caps(&mut current, name, caps);
super::write_map(path, &current)
}
/// Remove an agent from the capability map entirely. Called by
/// `meta::sync_agents` when an agent is deprovisioned so stale entries
/// don't accumulate. No-op if the agent has no entry. Fails without
/// writing if the existing file can't be read or parsed.
pub fn remove_agent(name: &str) -> std::io::Result<()> {
remove_agent_at(&capabilities_path(), name)
}
fn remove_agent_at(path: &Path, name: &str) -> std::io::Result<()> {
let mut current = read_from(path)?;
if current.remove(name).is_some() {
super::write_map(path, &current)?;
}
Ok(())
/// Whether `agent` holds `cap` per the file at `path` (normally [`path`]).
///
/// # Errors
///
/// As [`read_at`].
pub fn holds(path: &Path, agent: &str, cap: Capability) -> std::io::Result<bool> {
Ok(effective(agent, read_at(path)?.get(agent)).contains(&cap))
}
#[cfg(test)]
mod tests {
use super::*;
// `read`/`set_caps`/`remove_agent` call `crate::paths::meta_root`,
// which is hardcoded to `/var/lib/hyperhive` (no test override) — so
// these tests pin the pure decision logic (`prune_unknown`,
// `apply_known_caps`) against in-memory maps, and the file handling
// through the path-taking `*_at` / `read_from` variants.
const TRUNCATED: &str = "{\n \"atlas\": [\"manage_root_ag";
fn corrupt_file() -> (tempfile::TempDir, std::path::PathBuf) {
let dir = tempfile::tempdir().expect("tempdir");
let path = dir.path().join(CAPABILITIES_FILE);
std::fs::write(&path, TRUNCATED).expect("seed");
(dir, path)
fn record(json: &str) -> AgentPermissions {
serde_json::from_str(json).expect("decodes")
}
#[test]
fn read_of_a_corrupt_file_is_an_error() {
let (_dir, path) = corrupt_file();
read_from(&path).expect_err("truncated JSON must not read as a map");
}
#[test]
fn set_caps_leaves_a_corrupt_file_untouched() {
let (_dir, path) = corrupt_file();
let err = set_caps_at(&path, "ruth", &["manage_root_agent".to_owned()])
.expect_err("a corrupt file must not be overwritten");
assert_eq!(err.kind(), std::io::ErrorKind::InvalidData);
assert_eq!(std::fs::read(&path).expect("read"), TRUNCATED.as_bytes());
}
#[test]
fn remove_agent_leaves_a_corrupt_file_untouched() {
let (_dir, path) = corrupt_file();
let err =
remove_agent_at(&path, "atlas").expect_err("a corrupt file must not be overwritten");
assert_eq!(err.kind(), std::io::ErrorKind::InvalidData);
assert_eq!(std::fs::read(&path).expect("read"), TRUNCATED.as_bytes());
}
#[test]
fn set_caps_round_trips_through_a_missing_file() {
let dir = tempfile::tempdir().expect("tempdir");
let path = dir.path().join(CAPABILITIES_FILE);
assert!(read_from(&path).expect("missing file is empty").is_empty());
set_caps_at(&path, "ruth", &["manage_root_agent".to_owned()]).expect("set");
let map = read_from(&path).expect("read");
assert_eq!(map["ruth"], vec!["manage_root_agent".to_owned()]);
}
#[test]
fn prune_unknown_keeps_known_name() {
let mut map = BTreeMap::from([("atlas".to_owned(), vec!["manage_root_agent".to_owned()])]);
assert!(!prune_unknown(&mut map));
assert_eq!(map["atlas"], vec!["manage_root_agent".to_owned()]);
}
#[test]
fn prune_unknown_drops_unrecognised_name() {
let mut map = BTreeMap::from([("atlas".to_owned(), vec!["fly_to_the_moon".to_owned()])]);
assert!(prune_unknown(&mut map));
assert!(!map.contains_key("atlas"));
}
#[test]
fn prune_unknown_keeps_known_and_drops_unknown_in_same_entry() {
let mut map = BTreeMap::from([(
"atlas".to_owned(),
vec!["manage_root_agent".to_owned(), "fly_to_the_moon".to_owned()],
)]);
assert!(prune_unknown(&mut map));
assert_eq!(map["atlas"], vec!["manage_root_agent".to_owned()]);
}
#[test]
fn apply_known_caps_removes_entry_on_empty_input() {
let mut current =
BTreeMap::from([("atlas".to_owned(), vec!["manage_root_agent".to_owned()])]);
apply_known_caps(&mut current, "atlas", &[]);
assert!(!current.contains_key("atlas"));
}
#[test]
fn apply_known_caps_removes_entry_when_only_unknown_names_given() {
let mut current =
BTreeMap::from([("atlas".to_owned(), vec!["manage_root_agent".to_owned()])]);
apply_known_caps(&mut current, "atlas", &["fly_to_the_moon".to_owned()]);
assert!(!current.contains_key("atlas"));
}
#[test]
fn apply_known_caps_keeps_only_the_known_name() {
let mut current = BTreeMap::new();
apply_known_caps(
&mut current,
"atlas",
&["manage_root_agent".to_owned(), "fly_to_the_moon".to_owned()],
fn unset_is_the_default_and_an_empty_list_is_none() {
assert_eq!(effective("ruth", None), vec![Capability::ManageRootAgent]);
assert_eq!(
effective("ruth", Some(&record("{}"))),
vec![Capability::ManageRootAgent]
);
assert_eq!(current["atlas"], vec!["manage_root_agent".to_owned()]);
assert!(effective("ruth", Some(&record(r#"{"capabilities":[]}"#))).is_empty());
assert!(effective("atlas", None).is_empty());
assert_eq!(
effective(
"atlas",
Some(&record(r#"{"capabilities":["manage_root_agent"]}"#))
),
vec![Capability::ManageRootAgent]
);
}
#[test]
fn a_missing_file_holds_the_defaults() {
let dir = tempfile::tempdir().expect("tempdir");
let path = dir.path().join(FILE);
assert!(holds(&path, "ruth", Capability::ManageRootAgent).expect("missing is empty"));
assert!(!holds(&path, "atlas", Capability::ManageRootAgent).expect("missing is empty"));
}
#[test]
fn a_corrupt_file_is_an_error() {
let dir = tempfile::tempdir().expect("tempdir");
let path = dir.path().join(FILE);
std::fs::write(&path, "{\n \"ruth\": {\"capabilities\": [\"manage_root_ag").expect("seed");
holds(&path, "ruth", Capability::ManageRootAgent).expect_err("truncated JSON");
}
}

6
hive-c0re/src/agent_config/mod.rs
View file

@ -1,6 +1,6 @@
//! Per-agent configuration registries: tool groups, capabilities,
//! resource limits, topology (all JSON files under
//! `/var/lib/hyperhive/meta/`) and the shared wire-protocol size
//! Per-agent configuration registries: tool groups, resource limits,
//! topology (JSON files under `/var/lib/hyperhive/meta/`), the capabilities
//! the swarm grants (outside the meta repo) and the shared wire-protocol size
//! limits. Each submodule is re-exported at the crate root, so
//! `crate::topology::…` etc. keep working unchanged.
//!

2
hive-c0re/src/agent_config/resource_limits.rs
View file

@ -3,7 +3,7 @@
//!
//! The swarm caps are stored at
//! `/var/lib/hyperhive/meta/swarm-resource-limits.json` alongside
//! `topology.json`, `tool-groups.json` and `capabilities.json`.
//! `topology.json` and `tool-groups.json`.
//!
//! Format: a JSON object mapping agent name to an object with optional
//! `cpu_quota` / `memory_max` strings, passed verbatim to systemd's

3
hive-c0re/src/agent_config/topology.rs
View file

@ -13,7 +13,8 @@
//! bind-mounted ([`all_agents`]).
//!
//! Broader-than-your-own bind-mount grants are **not** stored here: they
//! hang off the `ManageRootAgent` capability in `capabilities.json`. See
//! hang off the `ManageRootAgent` capability the swarm grants
//! (`swarm_permissions`). See
//! `lifecycle::set_nspawn_flags`.
use std::collections::BTreeSet;

35
hive-c0re/src/coordinator.rs
View file

@ -565,41 +565,6 @@ impl Coordinator {
.map(|m| m.keys().cloned().collect())
}
/// Emit a `CapabilitiesChanged` snapshot event. Called from the
/// rebuild-queue worker after a `PermChange` / Capabilities entry
/// commits the JSON file, so the P3RM1SS10NS tab updates live.
pub fn emit_capabilities_snapshot(self: &Arc<Self>) {
use hive_sh4re::permissions::Capability;
let caps = Capability::ALL.iter().map(|c| <&str>::from(*c)).collect();
let descriptions = Capability::ALL
.iter()
.map(|c| (<&str>::from(*c), c.description()))
.collect();
// An empty snapshot would show every agent as having lost its
// capabilities; emit nothing and let the HTTP refetch report it.
let assignments = match crate::capabilities::read() {
Ok(map) => map,
Err(e) => {
tracing::warn!(error = ?e, "capabilities snapshot skipped");
return;
}
};
// Best-effort roster (sync path); on a contended cache miss we
// emit explicit keys only — the HTTP refetch fills the rest in.
let roster = self.live_container_names_blocking().unwrap_or_default();
// Capability default is "no extra caps" — empty default slice.
let (agents, effective) =
crate::dashboard::permissions::roster_and_effective(roster, &assignments, &[]);
self.emit_dashboard_event(DashboardEvent::CapabilitiesChanged {
seq: self.next_seq(),
caps,
descriptions,
assignments,
agents,
effective,
});
}
/// Emit a `ToolGroupsChanged` snapshot event. Called from the
/// rebuild-queue worker after a `PermChange` / `ToolGroups` entry
/// commits the JSON file, so the P3RM1SS10NS tab updates live.

6
hive-c0re/src/dashboard/mod.rs
View file

@ -37,7 +37,7 @@ use crate::lifecycle;
(name = "lifecycle_ops", description = "agent container lifecycle: rebuild/restart/start/stop/pause/limits"),
(name = "meta_inputs", description = "bulk flake-input update for the meta flake"),
(name = "misc_api", description = "mark-all-read, compose, spawn-request, hive stats"),
(name = "permissions", description = "tool-group + capability assignment for agents"),
(name = "permissions", description = "tool-group assignment for agents"),
(name = "rebuild_queue", description = "rebuild-queue cancel"),
(name = "state_files", description = "proxied reads of allow-listed per-agent state files"),
(name = "state_snapshot", description = "cold-load dashboard snapshot"),
@ -126,8 +126,6 @@ fn app(coord: Arc<Coordinator>) -> axum::Router {
.routes(routes!(build_logs::get_build_log_raw))
.routes(routes!(permissions::get_tool_groups))
.routes(routes!(permissions::post_tool_groups))
.routes(routes!(permissions::get_capabilities))
.routes(routes!(permissions::post_capabilities))
.routes(routes!(permissions::post_permissions))
.routes(routes!(permissions::get_stale_permissions))
.routes(routes!(permissions::delete_agent_permissions))
@ -380,8 +378,6 @@ mod router_build_probe {
.routes(routes!(build_logs::get_build_log_raw))
.routes(routes!(permissions::get_tool_groups))
.routes(routes!(permissions::post_tool_groups))
.routes(routes!(permissions::get_capabilities))
.routes(routes!(permissions::post_capabilities))
.routes(routes!(permissions::post_permissions))
.routes(routes!(permissions::get_stale_permissions))
.routes(routes!(permissions::delete_agent_permissions))

238
hive-c0re/src/dashboard/permissions.rs
View file

@ -1,6 +1,6 @@
//! Tool-group + capability permission endpoints for the dashboard.
//! Tool-group permission endpoints for the dashboard.
//!
//! Read endpoints return the full set of known groups/capabilities plus
//! Read endpoints return the full set of known groups plus
//! descriptions and the per-agent assignment map (the UI never hard-codes
//! the lists). Write endpoints validate, then enqueue a `PermChange` so the
//! JSON file write is serialised through the FIFO rebuild worker.
@ -20,7 +20,7 @@ use super::{AppState, Ident, guard_agent_name, strip_container_prefix};
#[derive(Serialize, ToSchema)]
pub(super) struct ToolGroupsSnapshot {
/// Ordered list of all known tool-group names. Drives the column
/// headers in the capabilities table — the UI does not hard-code them.
/// headers in the tool-groups table — the UI does not hard-code them.
groups: Vec<&'static str>,
/// Short description for each group name. Keys match `groups` entries.
descriptions: std::collections::BTreeMap<&'static str, &'static str>,
@ -100,8 +100,8 @@ pub(crate) fn tool_group_default_names() -> Vec<&'static str> {
/// `agents` is the sorted union of the live roster and any agent that
/// already has an explicit entry; `effective[agent]` is the explicit
/// assignment when present, else `default` (the role fallback). Shared by
/// the capabilities + tool-groups snapshots on both the HTTP and SSE
/// paths so the two surfaces never drift.
/// the tool-groups snapshots on both the HTTP and SSE paths so the two
/// surfaces never drift.
#[must_use]
pub(crate) fn roster_and_effective(
roster: impl IntoIterator<Item = String>,
@ -184,135 +184,15 @@ pub(super) async fn post_tool_groups(
Ok((StatusCode::OK, "ok").into_response())
}
#[derive(Serialize, ToSchema)]
pub(super) struct CapabilitiesSnapshot {
/// Ordered list of all known capability names. Drives the column
/// headers in the capabilities table — the UI does not hard-code them.
caps: Vec<&'static str>,
/// Short description for each capability name. Keys match `caps` entries.
descriptions: std::collections::BTreeMap<&'static str, &'static str>,
/// Per-agent *explicit* capability grant map. Absent agents have no
/// extra caps — the UI uses presence here to badge "(default)".
assignments: std::collections::BTreeMap<String, Vec<String>>,
/// Every agent the operator can manage (live roster ∪ explicit keys),
/// sorted — so default agents always list.
agents: Vec<String>,
/// Per-agent *effective* caps: explicit entry when present, else the
/// default (no caps). Keeps the snapshot shape symmetric with
/// tool-groups; the default here is always empty.
effective: std::collections::BTreeMap<String, Vec<String>>,
}
/// Every known capability name + description,
/// plus the per-agent explicit/effective grant maps.
#[utoipa::path(
get,
path = "/api/capabilities",
responses(
(status = 200, description = "capability catalogue + assignments", body = CapabilitiesSnapshot),
(status = 500, description = "capabilities file unreadable"),
),
tag = "permissions"
)]
pub(super) async fn get_capabilities(
State(state): State<AppState>,
) -> Result<axum::Json<CapabilitiesSnapshot>, ProblemDetails> {
use hive_sh4re::permissions::Capability;
let caps = Capability::ALL.iter().map(|c| <&str>::from(*c)).collect();
let descriptions = Capability::ALL
.iter()
.map(|c| (<&str>::from(*c), c.description()))
.collect();
let assignments = crate::capabilities::read().map_err(|e| unreadable(&e))?;
let roster = state
.coord
.containers_snapshot()
.await
.into_iter()
.map(|c| c.name);
// Capability default is "no extra caps" — empty default slice.
let (agents, effective) = roster_and_effective(roster, &assignments, &[]);
Ok(axum::Json(CapabilitiesSnapshot {
caps,
descriptions,
assignments,
agents,
effective,
}))
}
#[derive(Deserialize, ToSchema)]
pub(super) struct SetCapabilitiesBody {
caps: Vec<String>,
}
/// Replace `agent`'s explicit capability
/// grant set (JSON body `{"caps": [...]}`).
#[utoipa::path(
post,
path = "/api/capabilities/{agent}",
params(("agent" = String, Path, description = "agent name")),
request_body = SetCapabilitiesBody,
responses(
(status = 200, description = "capabilities queued for write", body = String),
(status = 400, description = "unknown capability name"),
(status = 404, description = "no such agent"),
),
tag = "permissions"
)]
pub(super) async fn post_capabilities(
State(state): State<AppState>,
AxumPath(name): AxumPath<String>,
axum::Json(body): axum::Json<SetCapabilitiesBody>,
) -> Result<Response, ProblemDetails> {
let logical = strip_container_prefix(&name);
if let Some(reject) = guard_agent_name(&state, &logical).await {
return Ok(reject);
}
let known: Vec<&str> = hive_sh4re::permissions::Capability::ALL
.iter()
.map(|c| <&str>::from(*c))
.collect();
for cap in &body.caps {
if !known.contains(&cap.as_str()) {
return Err(ProblemDetails::from_status_code(StatusCode::BAD_REQUEST)
.with_detail(format!("unknown capability: {cap}")));
}
}
// Submit a PermChange DAG: the JSON file write commits under
// META_LOCK inside the WritePermFile node, so concurrent
// batch-apply actions for different agents never race on the
// shared capabilities.json.
state
.coord
.job_queue
.insert_job(|b| {
crate::job_queue::templates::perm_change(
b,
&logical,
crate::job_queue::PermPayload::Capabilities {
caps: body.caps.clone(),
},
);
Vec::new()
})
.expect("template-declared shapes are acyclic");
state.coord.emit_rebuild_queue_snapshot();
tracing::info!(agent = %logical, caps = ?body.caps, "operator: set capabilities via dashboard");
Ok((StatusCode::OK, "ok").into_response())
}
/// One agent's slice of a batch permission change. Sparse: an omitted
/// field leaves that perm-type untouched, an empty array clears it, a
/// One agent's slice of a batch permission change. An omitted
/// `tool_groups` leaves the agent untouched, an empty array clears it, a
/// populated array fully replaces it (same replace semantics as the
/// per-agent endpoints).
/// per-agent endpoint).
#[derive(Deserialize, ToSchema)]
pub(super) struct PermChangeBody {
agent: String,
#[serde(default)]
tool_groups: Option<Vec<String>>,
#[serde(default)]
capabilities: Option<Vec<String>>,
}
#[derive(Deserialize, ToSchema)]
@ -320,25 +200,19 @@ pub(super) struct BatchPermsBody {
changes: Vec<PermChangeBody>,
}
/// A validated, non-empty change staged for enqueue:
/// `(logical agent, new groups?, new caps?)`.
type StagedPerm = (String, Option<Vec<String>>, Option<Vec<String>>);
/// Batch permission apply — `POST /api/permissions`.
///
/// The save-all permissions UI sends only the perm-types that actually
/// changed per agent; each affected agent gets ONE combined
/// `PermChange`, so the dedup key collapses to `(kind, agent)` and an
/// agent whose caps AND groups both changed rebuilds once, not twice.
/// The whole batch is atomic: every change is validated up front and
/// on any validation error nothing is written or enqueued.
/// The save-all permissions UI sends only the agents whose groups
/// changed; each gets one `PermChange`. The whole batch is atomic: every
/// change is validated up front and on any validation error nothing is
/// written or enqueued.
#[utoipa::path(
post,
path = "/api/permissions",
request_body = BatchPermsBody,
responses(
(status = 200, description = "batch permission change queued", body = String),
(status = 400, description = "unknown tool-group/capability name, or no such agent"),
(status = 400, description = "unknown tool-group name, or no such agent"),
(status = 404, description = "no such agent"),
),
tag = "permissions"
@ -347,43 +221,25 @@ pub(super) async fn post_permissions(
State(state): State<AppState>,
axum::Json(body): axum::Json<BatchPermsBody>,
) -> Result<Response, ProblemDetails> {
let known_caps: Vec<&str> = hive_sh4re::permissions::Capability::ALL
.iter()
.map(|c| <&str>::from(*c))
.collect();
// Phase 1 — validate everything before touching any file or the
// queue, so a bad entry fails the whole POST with zero side effects.
// No-op rows (both fields omitted) are skipped, not errors.
let mut staged: Vec<StagedPerm> = Vec::new();
// No-op rows (field omitted) are skipped, not errors.
let mut staged: Vec<(String, Vec<String>)> = Vec::new();
for change in &body.changes {
let logical = strip_container_prefix(&change.agent);
if let Some(reject) = guard_agent_name(&state, &logical).await {
return Ok(reject);
}
if let Some(groups) = &change.tool_groups
&& let Err(e) = crate::tool_groups::validate_groups(groups)
{
return Err(ProblemDetails::from_status_code(StatusCode::BAD_REQUEST)
.with_detail(format!("invalid tool-groups for {logical}: {e}")));
}
if let Some(caps) = &change.capabilities {
for cap in caps {
if !known_caps.contains(&cap.as_str()) {
return Err(ProblemDetails::from_status_code(StatusCode::BAD_REQUEST)
.with_detail(format!("unknown capability for {logical}: {cap}")));
}
if let Some(groups) = &change.tool_groups {
if let Err(e) = crate::tool_groups::validate_groups(groups) {
return Err(ProblemDetails::from_status_code(StatusCode::BAD_REQUEST)
.with_detail(format!("invalid tool-groups for {logical}: {e}")));
}
}
if change.tool_groups.is_some() || change.capabilities.is_some() {
staged.push((
logical,
change.tool_groups.clone(),
change.capabilities.clone(),
));
staged.push((logical, groups.clone()));
}
}
// Phase 2 — submit one combined PermChange DAG per affected agent.
for (logical, groups, caps) in staged {
// Phase 2 — submit one PermChange DAG per affected agent.
for (logical, groups) in staged {
state
.coord
.job_queue
@ -391,7 +247,7 @@ pub(super) async fn post_permissions(
crate::job_queue::templates::perm_change(
b,
&logical,
crate::job_queue::PermPayload::Combined { groups, caps },
crate::job_queue::PermPayload::ToolGroups { groups },
);
Vec::new()
})
@ -402,7 +258,7 @@ pub(super) async fn post_permissions(
Ok((StatusCode::OK, "ok").into_response())
}
/// Agent names that have explicit capability/tool-group entries but are
/// Agent names that have explicit tool-group entries but are
/// not in the live container roster AND not in the kept-state directory
/// list (i.e. truly gone — renamed or destroyed agents whose JSON entries
/// persisted). The client uses this to drive the "stale permission entries"
@ -421,7 +277,7 @@ pub(super) struct StalePermsResponse {
path = "/api/permissions/stale",
responses(
(status = 200, description = "ghost agent names with stale permission entries", body = StalePermsResponse),
(status = 500, description = "tool-groups/capabilities file unreadable"),
(status = 500, description = "tool-groups file unreadable"),
),
tag = "permissions"
)]
@ -447,18 +303,9 @@ pub(super) async fn get_stale_permissions(
.collect();
// Known = live roster ∪ kept-state names.
let known: std::collections::HashSet<&String> = live.iter().chain(kept.iter()).collect();
// Explicit entries in either JSON file.
let caps = crate::capabilities::read().map_err(|e| unreadable(&e))?;
let tgs = crate::tool_groups::read().map_err(|e| unreadable(&e))?;
let mut ghost_names: Vec<String> = caps
.keys()
.chain(tgs.keys())
.filter(|n| !known.contains(n))
.cloned()
.collect::<std::collections::HashSet<_>>()
.into_iter()
.collect();
ghost_names.sort();
// `tgs` is a `BTreeMap`, so its keys are already sorted.
let ghost_names: Vec<String> = tgs.keys().filter(|n| !known.contains(n)).cloned().collect();
Ok(axum::Json(StalePermsResponse { stale: ghost_names }))
}
@ -466,15 +313,15 @@ pub(super) async fn get_stale_permissions(
/// requiring it to exist in the live roster.
///
/// Used by the P3RM1SS10NS tab's "remove" button for agents that have
/// stale explicit entries in `tool-groups.json` / `capabilities.json`
/// but are no longer running (e.g. an agent that was renamed or
/// destroyed while its JSON entries persisted).
/// stale explicit entries in `tool-groups.json` but are no longer
/// running (e.g. an agent that was renamed or destroyed while its JSON
/// entries persisted).
///
/// Bypasses `guard_agent_name`'s live-roster check intentionally —
/// the whole point is to remove entries for non-roster agents. Only
/// the format check ([`Ident::parse`]) is applied. No rebuild is
/// enqueued (the agent doesn't exist to rebuild); the SSE snapshots
/// update the P3RM1SS10NS tab live.
/// enqueued (the agent doesn't exist to rebuild); the SSE snapshot
/// updates the P3RM1SS10NS tab live.
#[utoipa::path(
delete,
path = "/api/permissions/{agent}",
@ -482,7 +329,7 @@ pub(super) async fn get_stale_permissions(
responses(
(status = 200, description = "stale permission entries cleared", body = String),
(status = 400, description = "bad agent name"),
(status = 500, description = "tool-groups/capabilities file write failed"),
(status = 500, description = "tool-groups file write failed"),
),
tag = "permissions"
)]
@ -496,24 +343,13 @@ pub(super) async fn delete_agent_permissions(
return (StatusCode::BAD_REQUEST, format!("bad agent name: {reason}")).into_response();
}
};
// Run both removals regardless so we clean up as much as possible
// even on partial I/O errors. Collect errors to surface below.
let tg_err = crate::tool_groups::remove_agent(logical.as_str()).err();
if let Some(ref e) = tg_err {
tracing::warn!(agent = %logical, error = ?e, "failed to remove tool-groups entry");
}
let cap_err = crate::capabilities::remove_agent(logical.as_str()).err();
if let Some(ref e) = cap_err {
tracing::warn!(agent = %logical, error = ?e, "failed to remove capabilities entry");
}
// Emit live snapshots even on partial failure so the UI stays as
// accurate as possible — the surviving table gets updated immediately.
let removed = crate::tool_groups::remove_agent(logical.as_str());
state.coord.emit_tool_groups_snapshot();
state.coord.emit_capabilities_snapshot();
// Surface any I/O error as 500 so the frontend's `!resp.ok` path
// Surface an I/O error as 500 so the frontend's `!resp.ok` path
// fires and the operator sees a meaningful message rather than a
// silent "success" followed by the row reappearing unchanged.
if let Some(e) = tg_err.or(cap_err) {
if let Err(e) = removed {
tracing::warn!(agent = %logical, error = ?e, "failed to remove tool-groups entry");
return (
StatusCode::INTERNAL_SERVER_ERROR,
format!("failed to remove permission entries for {logical}: {e}"),

27
hive-c0re/src/dashboard_events.rs
View file

@ -141,24 +141,6 @@ pub enum DashboardEvent {
/// (transients, container state), and a poll loop would be
/// slower for the same information.
RebuildQueueChanged { seq: u64 },
/// Full snapshot of capability grants (per-agent `Vec<cap_name>`).
/// Emitted from the rebuild-queue worker after a `PermChange`
/// `Capabilities` entry commits the JSON file. Lets the P3RM1SS10NS
/// tab update live when the worker applies a queued change.
CapabilitiesChanged {
seq: u64,
/// Ordered list of all known capability names (column headers).
caps: Vec<&'static str>,
/// Short description for each capability name (tooltip).
descriptions: std::collections::BTreeMap<&'static str, &'static str>,
/// Per-agent *explicit* capability grant map; absent agents have
/// no extra caps (the UI badges those "(default)").
assignments: std::collections::BTreeMap<String, Vec<String>>,
/// Sorted roster the operator can manage (live ∪ explicit keys).
agents: Vec<String>,
/// Per-agent *effective* caps (explicit-or-default; default empty).
effective: std::collections::BTreeMap<String, Vec<String>>,
},
/// Full snapshot of tool-group assignments (per-agent `Vec<group_name>`).
/// Emitted from the rebuild-queue worker after a `PermChange`
/// `ToolGroups` entry commits the JSON file. Lets the P3RM1SS10NS
@ -203,7 +185,6 @@ impl DashboardEvent {
DashboardEvent::MetaInputsChanged { .. } => "meta_inputs_changed",
DashboardEvent::MetaUpdateRunning { .. } => "meta_update_running",
DashboardEvent::RebuildQueueChanged { .. } => "rebuild_queue_changed",
DashboardEvent::CapabilitiesChanged { .. } => "capabilities_changed",
DashboardEvent::ToolGroupsChanged { .. } => "tool_groups_changed",
}
}
@ -279,14 +260,6 @@ mod tests {
running: false,
},
DashboardEvent::RebuildQueueChanged { seq: 1 },
DashboardEvent::CapabilitiesChanged {
seq: 1,
caps: Vec::new(),
descriptions: std::collections::BTreeMap::new(),
assignments: std::collections::BTreeMap::new(),
agents: Vec::new(),
effective: std::collections::BTreeMap::new(),
},
DashboardEvent::ToolGroupsChanged {
seq: 1,
groups: Vec::new(),

17
hive-c0re/src/job_queue/exec.rs
View file

@ -810,23 +810,6 @@ async fn run_write_perm_file(
.with_context(|| format!("commit tool-groups for {name}"))?;
coord.emit_tool_groups_snapshot();
}
PermPayload::Capabilities { caps } => {
crate::meta::commit_capabilities(name, caps)
.await
.with_context(|| format!("commit capabilities for {name}"))?;
coord.emit_capabilities_snapshot();
}
PermPayload::Combined { groups, caps } => {
crate::meta::commit_perms(name, groups.as_deref(), caps.as_deref())
.await
.with_context(|| format!("commit perms for {name}"))?;
if groups.is_some() {
coord.emit_tool_groups_snapshot();
}
if caps.is_some() {
coord.emit_capabilities_snapshot();
}
}
}
Ok(())
}

2
hive-c0re/src/job_queue/model.rs
View file

@ -98,7 +98,7 @@ pub enum NodeKind {
PauseDrain { agent: String },
/// `set_nspawn_flags` + `set_resource_limits` + daemon-reload.
WriteDropin { agent: String },
/// Commit `tool-groups.json` / `capabilities.json` per its `payload`.
/// Commit `tool-groups.json` per its `payload`.
WritePermFile { agent: String, payload: PermPayload },
/// Group root of a rebuild subtree — the **brace**, scoped to one
/// agent. Why braces exist and what they cost: _Braces_.

4
hive-c0re/src/job_queue/templates.rs
View file

@ -403,8 +403,8 @@ pub fn destroy(builder: &JobBuilder, agent: &str, purge: bool) {
.after_ok(last);
}
/// Perm change: commit the JSON file(s), then the rebuild subgraph so
/// the updated `HIVE_TOOL_GROUPS` / `HIVE_CAPABILITIES` env var takes
/// Perm change: commit the JSON file, then the rebuild subgraph so
/// the updated `HIVE_TOOL_GROUPS` env var takes
/// effect in the container. Group-roots are `WritePermFile` plus the rebuild
/// subgraph's `MetaSync` / `Prebuild` / `Reconcile`, so the `EmitRebuilt` tail
/// edges all four.

5
hive-c0re/src/job_queue/tests.rs
View file

@ -1522,10 +1522,7 @@ fn perm_change_shape_prefixes_rebuild_chain() {
templates::perm_change(
builder,
"agent-a",
PermPayload::Combined {
groups: Some(vec![]),
caps: None,
},
PermPayload::ToolGroups { groups: vec![] },
);
});
assert_eq!(

48
hive-c0re/src/lifecycle/host_config.rs
View file

@ -381,22 +381,20 @@ fn bind_own_agent_dirs(agent_name: &str, notes_dir: &Path, binds: &mut Vec<BindM
}
}
/// Whether `agent_name` holds `ManageRootAgent` per the capabilities file
/// at `caps_path`. An unreadable file denies: the error is logged and the
/// Whether `agent_name` holds `ManageRootAgent` per the swarm permissions
/// stored at `path`. An unreadable file denies: the error is logged and the
/// agent starts without the cross-agent mounts, so one corrupt file never
/// blocks any agent's start, spawn or swap.
fn holds_manage_root_agent(agent_name: &str, caps_path: &Path) -> bool {
crate::capabilities::has_cap(caps_path, agent_name, Capability::ManageRootAgent).unwrap_or_else(
|e| {
tracing::error!(
agent = %agent_name,
path = %caps_path.display(),
error = ?e,
"capabilities unreadable — no ManageRootAgent mounts"
);
false
},
)
fn holds_manage_root_agent(agent_name: &str, path: &Path) -> bool {
crate::capabilities::holds(path, agent_name, Capability::ManageRootAgent).unwrap_or_else(|e| {
tracing::error!(
agent = %agent_name,
path = %path.display(),
error = ?e,
"swarm permissions unreadable — no ManageRootAgent mounts"
);
false
})
}
/// Env var naming the host directory `swarm-bao-queue-agent.service` lands
@ -587,7 +585,7 @@ async fn set_nspawn_flags(
// parent field took with it the unconditional grant every
// agent used to get over its own direct children — so an agent with
// no capability now sees its own dirs and nothing else.
if holds_manage_root_agent(agent_name, &crate::capabilities::capabilities_path()) {
if holds_manage_root_agent(agent_name, &crate::capabilities::path()) {
// Skipping self is a no-op, not a narrowing: `agent_notes_dir` is
// `agent_state_dir/state` and `config_bind_source` is shared, so
// binding the holder as its own virtual child reproduced the two
@ -818,22 +816,30 @@ mod tests {
}
#[test]
fn a_corrupt_capabilities_file_denies_without_failing() {
fn corrupt_swarm_permissions_deny_without_failing() {
let dir = tempfile::tempdir().expect("tempdir");
let path = dir.path().join("capabilities.json");
let truncated = "{\n \"ruth\": [\"manage_root_ag";
let path = dir.path().join("swarm-agent-permissions.json");
let truncated = "{\n \"ruth\": {\"capabilities\": [\"manage_root_ag";
std::fs::write(&path, truncated).expect("seed");
assert!(!holds_manage_root_agent("ruth", &path));
assert_eq!(std::fs::read(&path).expect("read"), truncated.as_bytes());
}
/// Unset is the default, which gives ruth the recovery mounts and nobody
/// else; a grant or an explicit empty list replaces it.
#[test]
fn a_readable_grant_is_honoured() {
fn the_swarm_grant_or_the_default_decides_the_recovery_mounts() {
let dir = tempfile::tempdir().expect("tempdir");
let path = dir.path().join("capabilities.json");
std::fs::write(&path, r#"{"ruth": ["manage_root_agent"]}"#).expect("seed");
let path = dir.path().join("swarm-agent-permissions.json");
assert!(holds_manage_root_agent("ruth", &path));
assert!(!holds_manage_root_agent("alice", &path));
std::fs::write(
&path,
r#"{"ruth": {"capabilities": []}, "alice": {"capabilities": ["manage_root_agent"]}}"#,
)
.expect("seed");
assert!(!holds_manage_root_agent("ruth", &path));
assert!(holds_manage_root_agent("alice", &path));
}
const TRUNCATED_LIMITS: &str = "{\n \"sock\": { \"cpu_quota\": \"50%\", \"mem";

3
hive-c0re/src/main.rs
View file

@ -277,6 +277,9 @@ async fn cmd_serve(
if let Err(e) = migrate::run(&coord).await {
tracing::warn!(error = ?e, "startup migration failed");
}
// Before any agent starts, so the bind mounts it gets from the stored
// permissions are the meta repo's grants until the swarm answers.
crate::workers::permissions_import::seed_local();
// Auto-create the root agent container if it isn't there yet. Block
// on this — without root the system has no manager harness.
// Failures are logged but allowed: a broken auto-spawn shouldn't

99
hive-c0re/src/meta.rs
View file

@ -47,8 +47,8 @@ pub struct AgentSpec {
/// Stage every generated meta JSON file that exists: topology.json is
/// regenerated by `reconcile` whenever the agent set changed;
/// tool-groups/capabilities are created lazily on
/// first write (`set_groups`/`set_caps`) — absent means
/// tool-groups is created lazily on
/// first write (`set_groups`) — absent means
/// every agent is on defaults, no file needed. Without staging, an
/// existing-but-untracked file shows up as untracked in the meta repo,
/// which can confuse nix's dirty-tree fetch. `git add` is a no-op when
@ -57,10 +57,6 @@ async fn stage_generated_meta_files(dir: &std::path::Path) -> Result<()> {
for (path, name) in [
(crate::topology::topology_path(), "topology.json"),
(crate::tool_groups::tool_groups_path(), "tool-groups.json"),
(
crate::capabilities::capabilities_path(),
"capabilities.json",
),
] {
if path.exists() {
git(dir, &["add", name]).await?;
@ -197,7 +193,6 @@ pub async fn sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<()> {
// Only ever a REMOVAL now; see `is_embedded_ca_name`.
f if f.starts_with("peer-ca-") && has_pem_ext(f) => Some("peer-ca"),
"topology.json" => Some("topology"),
"capabilities.json" => Some("capabilities"),
"tool-groups.json" => Some("tool-groups"),
_ => None,
})
@ -326,68 +321,28 @@ pub async fn commit_tool_groups(agent: &str, groups: &[String]) -> Result<()> {
Ok(())
}
/// Write the capabilities file for `agent` and commit it atomically
/// under `META_LOCK`. Same rationale as `commit_tool_groups`.
pub async fn commit_capabilities(agent: &str, caps: &[String]) -> Result<()> {
let _guard = META_LOCK.lock().await;
crate::capabilities::set_caps(agent, caps)
.map_err(|e| anyhow::anyhow!("set capabilities for {agent}: {e}"))?;
let dir = crate::paths::meta_root();
if crate::capabilities::capabilities_path().exists() {
git(&dir, &["add", "capabilities.json"]).await?;
}
if paths_dirty(&dir, &["capabilities.json"]).await? {
git_commit_paths(
&dir,
&format!("set capabilities for {agent}"),
&["capabilities.json"],
)
.await?;
}
Ok(())
}
/// Write both perm files for `agent` (whichever are `Some`) and commit
/// them in a SINGLE git commit under `META_LOCK` — the batch
/// `POST /api/permissions` path. A `None` field leaves that file
/// untouched. One commit + (caller does) one rebuild means changing an
/// agent's caps and tool-groups together no longer triggers two
/// rebuilds. Mirrors the staging discipline of `commit_tool_groups` /
/// `commit_capabilities`.
/// Delete `capabilities.json` from the meta repo and commit the removal under
/// `META_LOCK`. Its grants live in the swarm's `agent-permissions` bucket
/// once `workers::permissions_import` has sent them.
///
/// # Errors
///
/// Returns an error if writing either JSON file fails, a capability name
/// is invalid (`set_caps`), or a git stage/commit step fails.
pub async fn commit_perms(
agent: &str,
groups: Option<&[String]>,
caps: Option<&[String]>,
) -> Result<()> {
/// Removing the file or a git stage/commit step fails.
pub async fn remove_capabilities_file() -> Result<()> {
const FILE: &str = crate::workers::permissions_import::META_FILE;
let _guard = META_LOCK.lock().await;
let dir = crate::paths::meta_root();
let mut parts: Vec<&str> = Vec::new();
if let Some(groups) = groups {
crate::tool_groups::set_groups(agent, groups)?;
if crate::tool_groups::tool_groups_path().exists() {
git(&dir, &["add", "tool-groups.json"]).await?;
}
parts.push("tool-groups");
match std::fs::remove_file(dir.join(FILE)) {
Ok(()) => {}
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
Err(e) => return Err(e).with_context(|| format!("removing {FILE}")),
}
if let Some(caps) = caps {
crate::capabilities::set_caps(agent, caps)
.map_err(|e| anyhow::anyhow!("set capabilities for {agent}: {e}"))?;
if crate::capabilities::capabilities_path().exists() {
git(&dir, &["add", "capabilities.json"]).await?;
}
parts.push("capabilities");
}
let paths = ["tool-groups.json", "capabilities.json"];
if paths_dirty(&dir, &paths).await? {
git(&dir, &["rm", "--cached", "--ignore-unmatch", "-q", FILE]).await?;
if paths_dirty(&dir, &[FILE]).await? {
git_commit_paths(
&dir,
&format!("set {} for {agent}", parts.join(" + ")),
&paths,
"capabilities moved to the swarm's agent-permissions",
&[FILE],
)
.await?;
}
@ -905,7 +860,7 @@ where
let pronouns_escaped = operator_pronouns.replace('\\', "\\\\").replace('"', "\\\"");
let _ = writeln!(
out,
" dashboardPort = {dashboard_port};\n operatorPronouns = \"{pronouns_escaped}\";\n mkAgent = {{ name, isManager, port, toolGroups ? null, capabilities ? null }}:"
" dashboardPort = {dashboard_port};\n operatorPronouns = \"{pronouns_escaped}\";\n mkAgent = {{ name, isManager, port, toolGroups ? null }}:"
);
out.push_str(
r#" let
@ -915,7 +870,6 @@ where
input = inputs."agent-${name}";
service = "hive-agent";
toolGroupsEnv = if toolGroups == null then {} else { HIVE_TOOL_GROUPS = toolGroups; };
capabilitiesEnv = if capabilities == null then {} else { HIVE_CAPABILITIES = capabilities; };
in
base.extendModules {
modules = [
@ -1092,7 +1046,7 @@ where
}
out.push_str(
r" };
systemd.services.${service}.environment = toolGroupsEnv // capabilitiesEnv // {
systemd.services.${service}.environment = toolGroupsEnv // {
HIVE_PORT = toString port;
HIVE_LABEL = name;
HIVE_DASHBOARD_PORT = toString dashboardPort;
@ -1140,7 +1094,6 @@ where
// renders `toolGroups = null` and gets `AGENT_DEFAULT`, which fails
// open for any agent whose explicit entry is narrower.
let tool_groups_map = crate::tool_groups::read()?;
let capabilities_map = crate::capabilities::read()?;
for spec in agents {
// Emit `toolGroups = "group1,group2"` when the operator has
// explicitly configured groups for this agent. Absent entry = null
@ -1153,28 +1106,14 @@ where
let joined = groups.join(",");
format!("\"{joined}\"")
};
// Emit `capabilities = "cap1,cap2"` when the operator has
// granted capabilities to this agent. Absent entry = null = no
// capability env var injected, capability-gated tools hidden.
let caps = capabilities_map
.get(&spec.name)
.cloned()
.unwrap_or_default();
let capabilities_attr = if caps.is_empty() {
"null".to_owned()
} else {
let joined = caps.join(",");
format!("\"{joined}\"")
};
let _ = writeln!(
out,
" {} = mkAgent {{ name = \"{}\"; isManager = {}; port = {}; toolGroups = {}; capabilities = {}; }};",
" {} = mkAgent {{ name = \"{}\"; isManager = {}; port = {}; toolGroups = {}; }};",
spec.name,
spec.name,
if spec.is_manager { "true" } else { "false" },
spec.port,
tool_groups_attr,
capabilities_attr,
);
}
out.push_str(" };\n };\n}\n");

10
hive-c0re/src/swarm_status.rs
View file

@ -98,7 +98,7 @@ pub fn spawn(
// reason, and is a third consumer rather than a second connection.
tokio::spawn(crate::workers::wanted::watch_declarations(
client.clone(),
coord,
std::sync::Arc::clone(&coord),
hive.clone(),
shutdown.clone(),
));
@ -109,6 +109,14 @@ pub fn spawn(
shutdown.clone(),
));
// The capabilities the swarm grants, and the import of the meta repo's.
tokio::spawn(crate::workers::swarm_permissions::follow(
client.clone(),
std::sync::Arc::clone(&coord),
hive.clone(),
shutdown.clone(),
));
let mut health = SweepHealth::new("swarm_status_publish", "warn", FAILURES_BEFORE_BANNER);
loop {
match publish(&client, &hive).await {

110
hive-c0re/src/workers/auto_update.rs
View file

@ -124,10 +124,6 @@ pub async fn ensure_root_agent(coord: &Arc<Coordinator>) -> Result<()> {
);
return Ok(());
}
// Before the create/start branch, not inside it: on a hive whose root
// container already exists this is the only run that can still seed the
// grant, and it has to land before her next rebuild bakes the binds.
seed_manager_capabilities();
let list_result = lifecycle::list().await;
if let Err(e) = &list_result {
tracing::warn!(
@ -257,68 +253,6 @@ fn seed_manager_tool_groups() {
}
}
/// The capability the root agent is seeded with, as the `snake_case` string
/// `capabilities.json` stores and `capabilities::has_cap` compares against.
/// Taken from the enum rather than written out, so the seed cannot drift into
/// a name `capabilities::prune_unknown` would silently drop.
fn manager_seed_caps() -> Vec<String> {
vec![<&str>::from(hive_sh4re::permissions::Capability::ManageRootAgent).to_owned()]
}
/// Whether the root agent's default capability grant should be written.
///
/// `store_written` is "`capabilities.json` exists" — see
/// [`seed_manager_capabilities`] for why that, and not the absence of an
/// entry for the manager, is the condition.
fn should_seed_manager_caps(store_written: bool) -> bool {
!store_written
}
/// Give ruth the `ManageRootAgent` capability on the path that deploys her.
///
/// This is the capability-store half of the grant `roles.json` used to make:
/// `topology::reconcile_roles` seeded `can_manage_top_level_agents` onto
/// `MANAGER_NAME` on every meta sync, and that role is what put the other
/// agents' state/config dirs, `/applied` and `/meta` into her nspawn binds.
/// Collapsing the role into the capability means that without a seed here she
/// loses those recovery mounts at her next container rebuild — silently, and
/// only then, because nspawn bakes bind flags at container start.
///
/// **Seeded once, not re-ensured on every boot.** The role kept an empty-list
/// tombstone so an explicit revoke stuck; the capability store *deletes* an
/// entry that has been emptied rather than tombstoning it, so "the manager
/// has no entry" cannot tell a fresh hive apart from a deliberate revoke.
/// File existence can: every grant and revoke goes through
/// `meta::commit_capabilities` → `capabilities::set_caps`, which
/// writes the file even when the result is an empty `{}`. So while the file
/// is absent nobody has ever had a say, and once it exists this is inert
/// forever — including on the destroy+recreate path, matching
/// [`seed_manager_tool_groups`]'s refusal to reset an operator's choice.
///
/// Written with plain `set_caps` rather than `meta::commit_capabilities`: the
/// role's own seed wrote the file and left it for `stage_generated_meta_files`
/// to `git add` into the next deploy commit, and taking `META_LOCK` on the
/// boot path would be a new ordering constraint for no gain.
fn seed_manager_capabilities() {
if !should_seed_manager_caps(crate::capabilities::capabilities_path().exists()) {
tracing::debug!(
"capabilities.json already written — leaving the manager's capabilities as-is"
);
return;
}
let caps = manager_seed_caps();
match crate::capabilities::set_caps(MANAGER_NAME, &caps) {
Ok(()) => tracing::info!(
caps = ?caps,
"seeded ruth's default capabilities (recovery bind mounts for every agent)"
),
Err(e) => tracing::warn!(
error = ?e,
"failed to seed ruth's capabilities — she will come up without the recovery mounts"
),
}
}
/// Boot reconcile (see the module doc): classify every agent by rev
/// freshness + persisted `wanted` intent, submit one `Boot` DAG
/// (hyperhive lock bump growing an in-DAG rebuild subgraph per stale
@ -572,10 +506,7 @@ fn submit_boot_tree(
#[cfg(test)]
mod tests {
use super::{
BootAction, MANAGER_NAME, RootAgentPlan, boot_action, manager_seed_caps, plan_root_agent,
should_seed_manager_caps,
};
use super::{BootAction, MANAGER_NAME, RootAgentPlan, boot_action, plan_root_agent};
use crate::lifecycle::AGENT_PREFIX;
use crate::power::Wanted;
@ -604,45 +535,6 @@ mod tests {
assert_eq!(plan_root_agent(result), RootAgentPlan::Present);
}
// -----------------------------------------------------------------------
// Root-agent capability seed. `capabilities_path()` resolves under
// `paths::meta_root()`, which is hardcoded to `/var/lib/hyperhive` with no
// test override, so — same as `capabilities.rs`'s own tests — these pin
// the pure decision (`should_seed_manager_caps`, `manager_seed_caps`)
// rather than round-tripping the real file. Together they are the
// replacement for the deleted `reconcile_roles_in_seeds_root_when_absent`
// and `reconcile_roles_in_does_not_reseed_after_explicit_revoke`.
#[test]
fn the_manager_is_seeded_while_the_capability_store_has_never_been_written() {
assert!(should_seed_manager_caps(false));
assert_eq!(manager_seed_caps(), vec!["manage_root_agent".to_owned()]);
}
/// The revoke-sticks half, and the reason the condition is the file rather
/// than the manager's entry: the store deletes an emptied entry instead of
/// keeping a tombstone, but a revoke still writes the file — so once it
/// exists the seed must never fire again.
#[test]
fn a_written_capability_store_is_never_reseeded() {
assert!(!should_seed_manager_caps(true));
}
/// A seed the store would drop as unrecognised grants nothing while
/// logging success, so pin that what we write is a name `prune_unknown`
/// keeps — i.e. one of `Capability::ALL`'s `snake_case` spellings.
#[test]
fn the_seeded_name_is_a_capability_the_store_recognises() {
for cap in manager_seed_caps() {
assert!(
hive_sh4re::permissions::Capability::ALL
.iter()
.any(|known| <&str>::from(*known) == cap),
"{cap} is not a known capability"
);
}
}
// The regression the `Unreadable` variant exists for: the arm used to
// substitute `from_running(running)`, which is `Noop` against BOTH
// observations — so no combination of inputs could reach a reconcile.

4
hive-c0re/src/workers/mod.rs
View file

@ -1,7 +1,7 @@
//! Background tasks and periodic sweeps: crash/login watcher, boot-time
//! auto-update reconcile, the agent-sockets.json writer loop, the MCP socket listener
//! reconcile loop, knowledge-repo sync, the swarm wanted-state pull, and the
//! swarm resource-cap read.
//! swarm resource-cap and permissions reads.
//! Each submodule is re-exported
//! at the crate root, so `crate::crash_watch::…` etc. keep working unchanged.
@ -10,5 +10,7 @@ pub mod auto_update;
pub mod crash_watch;
pub mod knowledge;
pub mod mcp_sockets;
pub mod permissions_import;
pub mod swarm_limits;
pub mod swarm_permissions;
pub mod wanted;

197
hive-c0re/src/workers/permissions_import.rs Normal file
View file

@ -0,0 +1,197 @@
//! The one-shot move of the capabilities this hive's meta repo held
//! (`meta/capabilities.json`) into the swarm's `agent-permissions` bucket.
//!
//! At boot, before any agent starts, [`seed_local`] copies the file's grants
//! into [`crate::capabilities`] when nothing is stored there yet, so agents
//! keep their mounts until the swarm answers. [`send_pending`] then sends the
//! grants to swarm-controller, which writes each agent's key only where there
//! is none, and on its answer deletes the file from the meta repo. Nothing
//! reads the file after that.
use std::collections::BTreeMap;
use std::path::{Path, PathBuf};
use anyhow::{Context, Result};
use hive_sh4re::permissions::Capability;
use swarm_queue_client::agent_permissions::{
self, AgentPermissions, IMPORT_REPLY_DEADLINE, Import, ImportReply,
};
/// The meta repo file the grants are moved out of.
pub const META_FILE: &str = "capabilities.json";
fn meta_path() -> PathBuf {
crate::paths::meta_root().join(META_FILE)
}
/// The grants the meta file holds, by agent, as the swarm stores them. A name
/// that is not a [`Capability`] is dropped, with a warning.
///
/// `ruth` with no entry in a file that exists had her grant revoked: the file
/// was created holding it, and a revoke removed the entry. She is sent an
/// empty list, since unset would give her the default back.
fn grants(file: BTreeMap<String, Vec<String>>) -> BTreeMap<String, Vec<Capability>> {
let mut out: BTreeMap<String, Vec<Capability>> = file
.into_iter()
.map(|(agent, names)| {
let caps = names
.iter()
.filter_map(|name| {
let cap = Capability::ALL
.iter()
.copied()
.find(|c| name.eq_ignore_ascii_case(<&str>::from(*c)));
if cap.is_none() {
tracing::warn!(%agent, capability = %name, "import: dropping unrecognised capability");
}
cap
})
.collect();
(agent, caps)
})
.collect();
out.entry(crate::lifecycle::MANAGER_NAME.to_owned())
.or_default();
out
}
fn read_grants(path: &Path) -> Result<Option<BTreeMap<String, Vec<Capability>>>> {
if !path.exists() {
return Ok(None);
}
let file: BTreeMap<String, Vec<String>> = serde_json::from_slice(
&std::fs::read(path).with_context(|| format!("reading {}", path.display()))?,
)
.with_context(|| format!("parsing {}", path.display()))?;
Ok(Some(grants(file)))
}
/// Store the meta file's grants as the last-read swarm permissions, when the
/// file is still there and nothing has been stored yet.
pub fn seed_local() {
if let Err(e) = seed_local_at(&meta_path(), &crate::capabilities::path()) {
tracing::warn!(error = ?e, "import: seeding the stored permissions from the meta repo failed");
}
}
fn seed_local_at(meta: &Path, stored: &Path) -> Result<()> {
if stored.exists() {
return Ok(());
}
let Some(grants) = read_grants(meta)? else {
return Ok(());
};
let records = grants
.into_iter()
.map(|(agent, caps)| {
(
agent,
AgentPermissions {
capabilities: Some(caps),
},
)
})
.collect();
crate::capabilities::write_at(stored, &records).context("storing the seeded permissions")?;
tracing::info!("import: seeded the stored permissions from the meta repo's capabilities");
Ok(())
}
/// Send the meta file's grants to swarm-controller while the file is there,
/// and delete it once the controller has answered that it handled them.
///
/// # Errors
///
/// The file is still there: the import failed, or deleting the file did. The
/// next pass sends again; the controller keeps a key it already has.
pub async fn send_pending(client: &async_nats::Client, hive: &str) -> Result<()> {
send_at(client, hive, &meta_path())
.await
.context("importing the meta repo's capabilities")
}
async fn send_at(client: &async_nats::Client, hive: &str, path: &Path) -> Result<()> {
let Some(capabilities) = read_grants(path)? else {
return Ok(());
};
swarm_queue_client::ensure_connected(client)?;
let payload = serde_json::to_vec(&Import { capabilities })?;
let reply = tokio::time::timeout(
IMPORT_REPLY_DEADLINE,
client.request(agent_permissions::import_subject(hive), payload.into()),
)
.await
.context("the controller did not answer the import")?
.context("sending the import")?;
match serde_json::from_slice(&reply.payload).context("decoding the controller's answer")? {
ImportReply::Imported {
written,
kept,
skipped,
} => {
tracing::info!(
?written,
?kept,
?skipped,
"import: the controller took the meta repo's capabilities"
);
crate::meta::remove_capabilities_file()
.await
.context("deleting the imported capabilities file")
}
ImportReply::Failed { error } => {
anyhow::bail!("the controller refused the import: {error}")
}
}
}
#[cfg(test)]
mod tests {
use std::collections::BTreeMap;
use super::{grants, seed_local_at};
use hive_sh4re::permissions::Capability;
fn file(json: &str) -> BTreeMap<String, Vec<String>> {
serde_json::from_str(json).expect("decodes")
}
#[test]
fn grants_keep_known_names_and_a_revoked_ruth_stays_revoked() {
let out = grants(file(
r#"{"atlas": ["manage_root_agent", "fly_to_the_moon"], "iris": []}"#,
));
assert_eq!(out["atlas"], vec![Capability::ManageRootAgent]);
assert_eq!(out["iris"], Vec::<Capability>::new());
assert_eq!(out["ruth"], Vec::<Capability>::new());
let out = grants(file(r#"{"ruth": ["manage_root_agent"]}"#));
assert_eq!(out["ruth"], vec![Capability::ManageRootAgent]);
}
/// The seed writes once, from the file, and never over a stored read.
#[test]
fn the_seed_copies_the_file_only_when_nothing_is_stored() {
let dir = tempfile::tempdir().expect("tempdir");
let meta = dir.path().join("capabilities.json");
let stored = dir.path().join("stored.json");
seed_local_at(&meta, &stored).expect("no file is no seed");
assert!(!stored.exists());
std::fs::write(&meta, r#"{"atlas": ["manage_root_agent"]}"#).expect("seed");
seed_local_at(&meta, &stored).expect("seeded");
assert!(
crate::capabilities::holds(&stored, "atlas", Capability::ManageRootAgent)
.expect("read")
);
assert!(
!crate::capabilities::holds(&stored, "ruth", Capability::ManageRootAgent)
.expect("read")
);
std::fs::write(&stored, "{}").expect("a later read");
seed_local_at(&meta, &stored).expect("left alone");
assert_eq!(std::fs::read_to_string(&stored).expect("read"), "{}");
}
}

295
hive-c0re/src/workers/swarm_permissions.rs Normal file
View file

@ -0,0 +1,295 @@
//! Reading the capabilities the swarm grants this hive's agents.
//!
//! swarm-controller keeps one key per agent in the `agent-permissions` bucket
//! ([`swarm_queue_client::agent_permissions`]). Every [`READ_INTERVAL`] this
//! reads the key of each agent this hive has and stores what it finds with
//! [`crate::capabilities`], where the container's bind mounts are worked out.
//! Those are fixed when the container starts, so an agent whose capabilities
//! changed is restarted, after its current turn.
//!
//! A failed read, or a bucket that does not answer, leaves the stored
//! permissions as they are and restarts nothing: dropping them would put an
//! agent back on its default, which can be wider than what the swarm set.
use std::path::Path;
use std::sync::Arc;
use std::time::Duration;
use anyhow::{Context, Result};
use swarm_queue_client::agent_permissions::{self, AgentPermissions, Read};
use crate::capabilities::{self, Stored};
use crate::coordinator::Coordinator;
use crate::stats::sweep_health::{self, SweepHealth};
/// How often the permissions are read.
pub const READ_INTERVAL: Duration = Duration::from_secs(30);
/// At [`READ_INTERVAL`] this is ~90 seconds of failing reads.
const FAILURES_BEFORE_BANNER: u32 = 3;
/// Send the meta repo's grants once, then read the permissions now and every
/// [`READ_INTERVAL`] until shutdown.
pub async fn follow(
client: async_nats::Client,
coord: Arc<Coordinator>,
hive: String,
mut shutdown: tokio::sync::watch::Receiver<bool>,
) {
let mut health = SweepHealth::new("swarm_permissions_read", "warn", FAILURES_BEFORE_BANNER);
loop {
// While the meta repo's grants are not in the swarm yet, an unset key
// is not the swarm's answer, so nothing is read until they are.
let pass = match super::permissions_import::send_pending(&client, &hive).await {
Ok(()) => sync(&client).await,
Err(e) => Err(e),
};
match pass {
Ok(changed) => {
health.record_ok();
restart(&coord, changed).await;
}
Err(e) => {
tracing::warn!(error = ?e, "swarm permissions: the pass failed");
let err = format!("{e:#}");
health.record_err(|ctx| {
let age = ctx.since_last_ok.map_or_else(
|| "no success this session".to_owned(),
|d| format!("last ok {} ago", sweep_health::fmt_age(d)),
);
format!(
"syncing the swarm's agent permissions is failing ({} consecutive, \
{age}) — agents keep the permissions last read: {err}",
ctx.consecutive
)
});
}
}
tokio::select! {
() = tokio::time::sleep(READ_INTERVAL) => {}
_ = shutdown.changed() => {
tracing::info!("swarm permissions: shutdown signal received");
return;
}
}
}
}
/// Read every agent's key and store the result. Returns the agents whose
/// capabilities changed.
async fn sync(client: &async_nats::Client) -> Result<Vec<String>> {
// An unconnected client does not fail a JetStream request, it hangs on it.
swarm_queue_client::ensure_connected(client)?;
let agents: Vec<String> = crate::lifecycle::agents_for_meta_listing()
.await
.context("enumerating this hive's agents")?
.into_iter()
.map(|spec| spec.name)
.collect();
sync_with(&agents, &capabilities::path(), |agent| async move {
agent_permissions::read(client, &agent)
.await
.map_err(|e| anyhow::anyhow!(e))
.with_context(|| format!("reading the swarm's permissions for {agent}"))
})
.await
}
/// [`sync`] with the read and the file given. Nothing is written unless every
/// read succeeds.
async fn sync_with<F, Fut>(agents: &[String], path: &Path, read: F) -> Result<Vec<String>>
where
F: Fn(String) -> Fut,
Fut: std::future::Future<Output = Result<Read>>,
{
let mut read_now = Stored::new();
for agent in agents {
match read(agent.clone()).await? {
Read::Current(record) => {
if record != AgentPermissions::default() {
read_now.insert(agent.clone(), record);
}
}
Read::NoBucket => {
tracing::debug!("swarm permissions: the agent-permissions bucket does not answer");
return Ok(Vec::new());
}
}
}
let stored = capabilities::read_at(path).context("reading the stored swarm permissions")?;
if read_now == stored {
return Ok(Vec::new());
}
capabilities::write_at(path, &read_now).context("storing the swarm permissions")?;
tracing::info!(permissions = ?read_now, "swarm permissions: stored the swarm's permissions");
Ok(agents
.iter()
.filter(|a| {
capabilities::effective(a, stored.get(*a))
!= capabilities::effective(a, read_now.get(*a))
})
.cloned()
.collect())
}
/// Restart each running agent in `changed` once its current turn is done, so
/// its container starts with the bind mounts its new capabilities give. A
/// stopped agent gets them at its next start.
async fn restart(coord: &Arc<Coordinator>, changed: Vec<String>) {
let mut running = Vec::new();
for agent in changed {
if crate::lifecycle::is_running(&agent).await {
running.push(agent);
}
}
if running.is_empty() {
return;
}
tracing::info!(agents = ?running, "swarm permissions: capabilities changed, restarting");
if let Err(e) = crate::job_queue::power::restart_many(coord, &running, true).await {
tracing::error!(agents = ?running, error = ?e, "swarm permissions: restart insert failed");
}
}
#[cfg(test)]
mod tests {
use std::path::Path;
use super::{Read, sync_with};
use swarm_queue_client::agent_permissions::AgentPermissions;
fn record(json: &str) -> AgentPermissions {
serde_json::from_str(json).expect("decodes")
}
fn agents() -> Vec<String> {
vec!["atlas".to_owned(), "ruth".to_owned()]
}
/// Sync `agents()` against `path` with each agent read as `answer` gives.
async fn sync(
path: &Path,
answer: impl Fn(&str) -> anyhow::Result<Read>,
) -> anyhow::Result<Vec<String>> {
sync_with(&agents(), path, |agent| {
let out = answer(&agent);
async move { out }
})
.await
}
/// No key anywhere is everyone on the default: nothing stored, and no
/// restart, since that is what the agents already run with.
#[tokio::test]
async fn missing_keys_are_the_default_and_restart_nothing() {
let dir = tempfile::tempdir().expect("tempdir");
let path = dir.path().join("p.json");
let changed = sync(&path, |_| Ok(Read::Current(AgentPermissions::default())))
.await
.expect("read");
assert!(changed.is_empty());
assert!(
crate::capabilities::read_at(&path)
.expect("read")
.is_empty()
);
assert!(
crate::capabilities::holds(
&path,
"ruth",
hive_sh4re::permissions::Capability::ManageRootAgent
)
.expect("read")
);
}
/// Revoking ruth's default, and granting atlas, restarts both; clearing
/// them again restarts both back.
#[tokio::test]
async fn a_changed_capability_asks_for_a_restart() {
let dir = tempfile::tempdir().expect("tempdir");
let path = dir.path().join("p.json");
let changed = sync(&path, |agent| {
Ok(Read::Current(record(if agent == "ruth" {
r#"{"capabilities":[]}"#
} else {
r#"{"capabilities":["manage_root_agent"]}"#
})))
})
.await
.expect("read");
assert_eq!(changed, agents());
// The same answer again changes nothing.
let unchanged = sync(&path, |agent| {
Ok(Read::Current(record(if agent == "ruth" {
r#"{"capabilities":[]}"#
} else {
r#"{"capabilities":["manage_root_agent"]}"#
})))
})
.await
.expect("read");
assert!(unchanged.is_empty());
let cleared = sync(&path, |_| Ok(Read::Current(AgentPermissions::default())))
.await
.expect("read");
assert_eq!(cleared, agents());
}
/// Setting ruth's default explicitly is a different record but the same
/// mounts, so it is stored and restarts nothing.
#[tokio::test]
async fn an_explicit_default_is_stored_without_a_restart() {
let dir = tempfile::tempdir().expect("tempdir");
let path = dir.path().join("p.json");
let changed = sync(&path, |agent| {
Ok(Read::Current(if agent == "ruth" {
record(r#"{"capabilities":["manage_root_agent"]}"#)
} else {
AgentPermissions::default()
}))
})
.await
.expect("read");
assert!(changed.is_empty());
assert!(
crate::capabilities::read_at(&path)
.expect("read")
.contains_key("ruth")
);
}
/// One failed read, or a bucket that does not answer, keeps what was
/// stored and restarts nothing: never a fall back to the default.
#[tokio::test]
async fn a_failed_read_keeps_the_last_value() {
let dir = tempfile::tempdir().expect("tempdir");
let path = dir.path().join("p.json");
let revoked = || Ok(Read::Current(record(r#"{"capabilities":[]}"#)));
// An empty list is atlas's default already, so only ruth changes.
assert_eq!(
sync(&path, |_| revoked()).await.expect("read"),
vec!["ruth".to_owned()]
);
let before = std::fs::read(&path).expect("stored");
sync(&path, |agent| {
if agent == "ruth" {
Err(anyhow::anyhow!("timed out"))
} else {
Ok(Read::Current(AgentPermissions::default()))
}
})
.await
.expect_err("a failed read fails the pass");
assert_eq!(std::fs::read(&path).expect("stored"), before);
let changed = sync(&path, |_| Ok(Read::NoBucket))
.await
.expect("no bucket");
assert!(changed.is_empty());
assert_eq!(std::fs::read(&path).expect("stored"), before);
}
}