agent permissions: drop hive-c0re's swarm-agent-permissions.json fallback, fail closed before the first read
Removes the HiveCopy hand-off: hive-c0re no longer writes each agent's entry from swarm-agent-permissions.json into its runtime dir as swarm-permissions.json, and the harness and subagent daemon no longer read it. hive-c0re keeps the file as its own last read of the swarm for the manage_root_agent mounts. Also removes permissions_import::pending(), its only caller being the hand-off. No nix provisioned the copy, so no nix option is removed and no tombstone is needed. Rule after this change: until an agent's first successful read of its key it holds no permissions (no execution, so no bash), and a subagent it starts holds the same. After a read a set key wins, [] holds none and an unset key is the default. A failed read keeps the last value, so an unreachable queue at start keeps the agent closed until a read succeeds. The harness logs once at warn that it starts closed. Fixes #5315
This commit is contained in:
parent
6e7b5f84e9
commit
2f00f9041f
11 changed files with 121 additions and 283 deletions
|
|
@ -430,9 +430,7 @@ holds the permissions the swarm grants this hive's agents, as
|
||||||
`workers::swarm_permissions` last read them from the swarm's
|
`workers::swarm_permissions` last read them from the swarm's
|
||||||
`agent-permissions` bucket (every 30 seconds): a copy, not the
|
`agent-permissions` bucket (every 30 seconds): a copy, not the
|
||||||
record, which `swarmctl agent permissions` sets. `set_nspawn_flags`
|
record, which `swarmctl agent permissions` sets. `set_nspawn_flags`
|
||||||
reads it at every container start for `manage_root_agent`. Each agent gets
|
reads it at every container start for `manage_root_agent`.
|
||||||
its own entry from it as `swarm-permissions.json` in its runtime dir, which
|
|
||||||
it holds until its own first read of its key succeeds.
|
|
||||||
|
|
||||||
`/var/lib/hyperhive/swarm-agent-archetypes.json` holds the archetype each of
|
`/var/lib/hyperhive/swarm-agent-archetypes.json` holds the archetype each of
|
||||||
this hive's archetype agents runs, as last read from the swarm's
|
this hive's archetype agents runs, as last read from the swarm's
|
||||||
|
|
|
||||||
|
|
@ -322,12 +322,11 @@ none.
|
||||||
`/var/lib/hyperhive/swarm-agent-permissions.json`, outside the meta repo.
|
`/var/lib/hyperhive/swarm-agent-permissions.json`, outside the meta repo.
|
||||||
nspawn fixes the bind mounts at container start, so when an agent's
|
nspawn fixes the bind mounts at container start, so when an agent's
|
||||||
`manage_root_agent` changes hive-c0re restarts it after its current turn.
|
`manage_root_agent` changes hive-c0re restarts it after its current turn.
|
||||||
- hive-c0re copies each agent's entry from that file into the agent's
|
- Until its first read succeeds, an agent holds no permissions: no
|
||||||
runtime dir (`/run/hive/swarm-permissions.json` in the container) at every
|
`execution` (so no `bash`), only the always-on tools. A subagent it starts
|
||||||
start and after every 30-second pass, with whether the import below is
|
holds the same. After a read, an unset key means the default. A failed read
|
||||||
still pending. The harness and the subagent daemon hold that copy until
|
keeps the last value read, so an unreachable queue at start keeps the agent
|
||||||
their own first read succeeds, so an agent whose queue is down starts on
|
closed until a read succeeds.
|
||||||
its last stored grant, not the default.
|
|
||||||
|
|
||||||
Every reader keeps its last good value on a failed read.
|
Every reader keeps its last good value on a failed read.
|
||||||
|
|
||||||
|
|
@ -338,8 +337,10 @@ sends them to swarm-controller once as one list per agent (without
|
||||||
only where there is none, and only for agents that hive's wanted state
|
only where there is none, and only for agents that hive's wanted state
|
||||||
declares), and deletes
|
declares), and deletes
|
||||||
both files from the meta repo when the controller answers
|
both files from the meta repo when the controller answers
|
||||||
(`workers/permissions_import.rs`). Until then hive-c0re reads no keys, and an
|
(`workers/permissions_import.rs`). Until then hive-c0re reads no keys. An
|
||||||
agent that reads its own key unset keeps the grant in its copy.
|
agent reads a key the controller hasn't yet written as unset, the default,
|
||||||
|
so an agent whose meta grant differs from the default holds the default
|
||||||
|
until the import lands.
|
||||||
|
|
||||||
**Not configurable from `agent.nix`** — that file changes through config PRs
|
**Not configurable from `agent.nix`** — that file changes through config PRs
|
||||||
an agent can open, so letting it declare its own permissions would let an
|
an agent can open, so letting it declare its own permissions would let an
|
||||||
|
|
|
||||||
|
|
@ -7,18 +7,16 @@
|
||||||
//! loop asks [`current`] for every turn's `--tools`, `--allowedTools` and MCP
|
//! loop asks [`current`] for every turn's `--tools`, `--allowedTools` and MCP
|
||||||
//! config, so a change reaches the next turn.
|
//! config, so a change reaches the next turn.
|
||||||
//!
|
//!
|
||||||
//! Until the first successful read after a start, the agent holds what
|
//! Until the first successful read after a start the agent holds nothing
|
||||||
//! hive-c0re stored for it ([`HiveCopy::before_read`]), so a queue that is down
|
//! ([`BEFORE_FIRST_READ`]), so a queue that is down at start never leaves it
|
||||||
//! or slow leaves it on its last known grant, not the default. A read is
|
//! wider than the swarm grants. After one, a set key wins, `[]` holds none and
|
||||||
//! judged by [`HiveCopy::after_read`]: an unset key is the default, except
|
//! an unset key is the default. A failed read keeps the last value.
|
||||||
//! while the hive's import is pending. A failed read keeps the last value.
|
|
||||||
|
|
||||||
use std::path::Path;
|
|
||||||
use std::sync::{PoisonError, RwLock};
|
use std::sync::{PoisonError, RwLock};
|
||||||
use std::time::Duration;
|
use std::time::Duration;
|
||||||
|
|
||||||
use hive_types::Permission;
|
use hive_types::Permission;
|
||||||
use swarm_queue_client::agent_permissions::{HIVE_COPY_PATH, HiveCopy, Read, read};
|
use swarm_queue_client::agent_permissions::{BEFORE_FIRST_READ, Read, held, read};
|
||||||
|
|
||||||
/// How often the key is read. Also the longest a change can take to reach
|
/// How often the key is read. Also the longest a change can take to reach
|
||||||
/// the agent's next turn.
|
/// the agent's next turn.
|
||||||
|
|
@ -34,40 +32,50 @@ pub fn current() -> Vec<Permission> {
|
||||||
.read()
|
.read()
|
||||||
.unwrap_or_else(PoisonError::into_inner)
|
.unwrap_or_else(PoisonError::into_inner)
|
||||||
.clone();
|
.clone();
|
||||||
current_with(last, Path::new(HIVE_COPY_PATH))
|
held_now(last)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// [`current`] with the last read and the hive's copy given.
|
/// What the agent holds given the last value read: that value, or nothing
|
||||||
fn current_with(last: Option<Vec<Permission>>, copy: &Path) -> Vec<Permission> {
|
/// before the first read has succeeded.
|
||||||
last.unwrap_or_else(|| HiveCopy::read_at(copy).before_read())
|
fn held_now(last: Option<Vec<Permission>>) -> Vec<Permission> {
|
||||||
|
last.unwrap_or_else(|| BEFORE_FIRST_READ.to_vec())
|
||||||
}
|
}
|
||||||
|
|
||||||
/// What one read of the key leaves the agent holding, judged against the
|
/// What one read of the key leaves the agent holding; `None` (keep the last
|
||||||
/// hive's copy at `copy`; `None` (keep the last value) when the read failed
|
/// value) when the read failed or the bucket did not answer.
|
||||||
/// or the bucket did not answer.
|
fn after(read: Option<Read>) -> Option<Vec<Permission>> {
|
||||||
fn after(copy: &Path, read: Option<Read>) -> Option<Vec<Permission>> {
|
|
||||||
match read? {
|
match read? {
|
||||||
Read::Current(record) => Some(HiveCopy::read_at(copy).after_read(&record)),
|
Read::Current(record) => Some(held(&record)),
|
||||||
Read::NoBucket => None,
|
Read::NoBucket => None,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The last value read once `read` has been applied to `last`.
|
||||||
|
fn settle(last: Option<Vec<Permission>>, read: Option<Read>) -> Option<Vec<Permission>> {
|
||||||
|
after(read).or(last)
|
||||||
|
}
|
||||||
|
|
||||||
/// Run the poller for the harness's lifetime, if this agent has a label to
|
/// Run the poller for the harness's lifetime, if this agent has a label to
|
||||||
/// name its key with.
|
/// name its key with.
|
||||||
pub async fn run() {
|
pub async fn run() {
|
||||||
let agent = crate::identity::label();
|
let agent = crate::identity::label();
|
||||||
if agent.is_empty() {
|
if agent.is_empty() {
|
||||||
tracing::warn!("this agent has no label; not reading its swarm permissions");
|
tracing::warn!("this agent has no label; it holds no swarm permissions");
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
let client = crate::swarm_queue::client().await;
|
let client = crate::swarm_queue::client().await;
|
||||||
tracing::info!(%agent, "reading swarm permissions from the swarm queue");
|
tracing::warn!(
|
||||||
|
%agent,
|
||||||
|
"permissions: holding none (no bash) until the first read of this agent's key succeeds"
|
||||||
|
);
|
||||||
loop {
|
loop {
|
||||||
let read = read_once(&client, &agent).await;
|
let read = read_once(&client, &agent).await;
|
||||||
if let Some(perms) = after(Path::new(HIVE_COPY_PATH), read) {
|
{
|
||||||
let mut current = CURRENT.write().unwrap_or_else(PoisonError::into_inner);
|
let mut current = CURRENT.write().unwrap_or_else(PoisonError::into_inner);
|
||||||
if current.as_ref() != Some(&perms) {
|
let next = settle(current.clone(), read);
|
||||||
tracing::info!(?perms, "permissions: the swarm's set changed");
|
if *current != next {
|
||||||
*current = Some(perms);
|
tracing::info!(perms = ?next, "permissions: the swarm's set changed");
|
||||||
|
*current = next;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
tokio::time::sleep(POLL_INTERVAL).await;
|
tokio::time::sleep(POLL_INTERVAL).await;
|
||||||
|
|
@ -95,77 +103,53 @@ mod tests {
|
||||||
Read::Current(serde_json::from_str(json).expect("decodes"))
|
Read::Current(serde_json::from_str(json).expect("decodes"))
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A hive's copy holding `json`, in a tempdir kept alive by the guard.
|
/// What the agent holds after each read in turn, from a fresh start.
|
||||||
fn copy(json: &str) -> (tempfile::TempDir, std::path::PathBuf) {
|
fn held_after(reads: Vec<Option<Read>>) -> Vec<Permission> {
|
||||||
let dir = tempfile::tempdir().expect("tempdir");
|
held_now(reads.into_iter().fold(None, settle))
|
||||||
let path = dir.path().join("swarm-permissions.json");
|
|
||||||
std::fs::write(&path, json).expect("write the copy");
|
|
||||||
(dir, path)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn a_read_sets_the_list_or_the_default_and_a_failure_keeps_the_last() {
|
fn before_the_first_read_execution_is_not_held() {
|
||||||
let none = Path::new("/nonexistent/swarm-permissions.json");
|
let held = held_after(Vec::new());
|
||||||
assert_eq!(
|
assert!(held.is_empty(), "held {held:?} before any read");
|
||||||
after(none, Some(record(r#"{"permissions":["web_tools"]}"#))),
|
assert!(!held.contains(&Permission::Execution));
|
||||||
Some(vec![Permission::WebTools])
|
assert!(
|
||||||
|
Permission::DEFAULT.contains(&Permission::Execution),
|
||||||
|
"control: the default does hold it"
|
||||||
);
|
);
|
||||||
assert_eq!(
|
for failed in [None, Some(Read::NoBucket)] {
|
||||||
after(none, Some(record(r#"{"permissions":[]}"#))),
|
assert!(
|
||||||
Some(Vec::new())
|
held_after(vec![failed]).is_empty(),
|
||||||
|
"a read that fails leaves the agent closed"
|
||||||
);
|
);
|
||||||
assert_eq!(
|
}
|
||||||
after(none, Some(record("{}"))),
|
|
||||||
Some(Permission::DEFAULT.to_vec())
|
|
||||||
);
|
|
||||||
assert_eq!(after(none, Some(Read::NoBucket)), None);
|
|
||||||
assert_eq!(after(none, None), None);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn before_the_first_read_the_stored_grant_applies() {
|
fn a_read_with_the_key_unset_holds_the_default() {
|
||||||
let (_dir, path) = copy(r#"{"permissions":["web_tools"]}"#);
|
let held = held_after(vec![Some(record("{}"))]);
|
||||||
assert_eq!(current_with(None, &path), vec![Permission::WebTools]);
|
assert_eq!(held, Permission::DEFAULT.to_vec());
|
||||||
let (_dir, path) = copy(r#"{"permissions":[]}"#);
|
assert!(held.contains(&Permission::Execution));
|
||||||
assert_eq!(current_with(None, &path), Vec::new());
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_read_with_a_restricted_set_holds_exactly_that_set() {
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
current_with(Some(vec![Permission::Meta]), &path),
|
held_after(vec![Some(record(r#"{"permissions":["web_tools"]}"#))]),
|
||||||
vec![Permission::Meta],
|
vec![Permission::WebTools]
|
||||||
"a read outranks the copy"
|
);
|
||||||
|
assert_eq!(
|
||||||
|
held_after(vec![Some(record(r#"{"permissions":[]}"#))]),
|
||||||
|
Vec::new()
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn while_the_import_is_pending_an_unset_key_keeps_the_stored_grant() {
|
fn a_failed_read_after_a_success_keeps_the_last_set() {
|
||||||
let (_dir, path) = copy(r#"{"permissions":["web_tools"],"import_pending":true}"#);
|
let web = Some(record(r#"{"permissions":["web_tools"]}"#));
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
after(&path, Some(record("{}"))),
|
held_after(vec![web, None, Some(Read::NoBucket)]),
|
||||||
Some(vec![Permission::WebTools])
|
vec![Permission::WebTools]
|
||||||
);
|
);
|
||||||
assert_eq!(
|
|
||||||
after(&path, Some(record(r#"{"permissions":["meta"]}"#))),
|
|
||||||
Some(vec![Permission::Meta]),
|
|
||||||
"a set key still wins"
|
|
||||||
);
|
|
||||||
let (_dir, path) = copy(r#"{"permissions":["web_tools"]}"#);
|
|
||||||
assert_eq!(
|
|
||||||
after(&path, Some(record("{}"))),
|
|
||||||
Some(Permission::DEFAULT.to_vec()),
|
|
||||||
"once imported, unset is the default"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
/// No read ever succeeds: the last value stays unset and the agent holds
|
|
||||||
/// the hive's stored grant.
|
|
||||||
#[test]
|
|
||||||
fn with_the_queue_down_the_agent_holds_the_stored_grant() {
|
|
||||||
let (_dir, path) = copy(r#"{"permissions":["messaging"],"import_pending":true}"#);
|
|
||||||
let mut last = None;
|
|
||||||
for read in [None, Some(Read::NoBucket), None] {
|
|
||||||
if let Some(perms) = after(&path, read) {
|
|
||||||
last = Some(perms);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
assert_eq!(current_with(last, &path), vec![Permission::Messaging]);
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -11,14 +11,12 @@
|
||||||
//! hive-c0re enforces one member, [`Permission::ManageRootAgent`]: read by
|
//! hive-c0re enforces one member, [`Permission::ManageRootAgent`]: read by
|
||||||
//! `lifecycle::host_config::set_nspawn_flags` on every container start, which
|
//! `lifecycle::host_config::set_nspawn_flags` on every container start, which
|
||||||
//! is why a change to it takes a restart. The agent reads the tool members
|
//! is why a change to it takes a restart. The agent reads the tool members
|
||||||
//! itself, and holds its entry here until its own first read succeeds:
|
//! itself.
|
||||||
//! [`hand_at`] copies the entry into the agent's runtime dir.
|
|
||||||
|
|
||||||
use std::collections::BTreeMap;
|
use std::collections::BTreeMap;
|
||||||
use std::path::{Path, PathBuf};
|
use std::path::{Path, PathBuf};
|
||||||
|
|
||||||
use hive_types::Permission;
|
use hive_types::Permission;
|
||||||
use swarm_queue_client::agent_permissions::{HIVE_COPY_FILE, HiveCopy};
|
|
||||||
use swarm_wire::agent_permissions::AgentPermissions;
|
use swarm_wire::agent_permissions::AgentPermissions;
|
||||||
|
|
||||||
const FILE: &str = "swarm-agent-permissions.json";
|
const FILE: &str = "swarm-agent-permissions.json";
|
||||||
|
|
@ -58,30 +56,6 @@ pub fn holds(path: &Path, agent: &str, perm: Permission) -> std::io::Result<bool
|
||||||
Ok(effective(read_at(path)?.get(agent)).contains(&perm))
|
Ok(effective(read_at(path)?.get(agent)).contains(&perm))
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Write `agent`'s [`HiveCopy`] into `runtime_dir`: its entry in the file at
|
|
||||||
/// `stored` (normally [`path`]) and `import_pending`. Left alone when the
|
|
||||||
/// copy there already says the same.
|
|
||||||
///
|
|
||||||
/// # Errors
|
|
||||||
///
|
|
||||||
/// As [`read_at`], or the write failed.
|
|
||||||
pub fn hand_at(
|
|
||||||
stored: &Path,
|
|
||||||
import_pending: bool,
|
|
||||||
agent: &str,
|
|
||||||
runtime_dir: &Path,
|
|
||||||
) -> std::io::Result<()> {
|
|
||||||
let copy = HiveCopy {
|
|
||||||
stored: read_at(stored)?.remove(agent).unwrap_or_default(),
|
|
||||||
import_pending,
|
|
||||||
};
|
|
||||||
let dest = runtime_dir.join(HIVE_COPY_FILE);
|
|
||||||
if dest.exists() && HiveCopy::read_at(&dest) == copy {
|
|
||||||
return Ok(());
|
|
||||||
}
|
|
||||||
super::write_map(&dest, ©)
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
@ -129,32 +103,6 @@ mod tests {
|
||||||
assert!(!holds(&path, "atlas", Permission::ManageRootAgent).expect("read"));
|
assert!(!holds(&path, "atlas", Permission::ManageRootAgent).expect("read"));
|
||||||
}
|
}
|
||||||
|
|
||||||
/// With the queue down nothing is read, so the agent starts on the copy
|
|
||||||
/// written here: the stored grant, not the default.
|
|
||||||
#[test]
|
|
||||||
fn the_agent_is_handed_its_stored_grant_and_whether_the_import_is_pending() {
|
|
||||||
let dir = tempfile::tempdir().expect("tempdir");
|
|
||||||
let stored = dir.path().join(FILE);
|
|
||||||
let runtime = dir.path().join("run");
|
|
||||||
write_at(
|
|
||||||
&stored,
|
|
||||||
&Stored::from([(
|
|
||||||
"atlas".to_owned(),
|
|
||||||
record(r#"{"permissions":["web_tools"]}"#),
|
|
||||||
)]),
|
|
||||||
)
|
|
||||||
.expect("seed");
|
|
||||||
|
|
||||||
hand_at(&stored, true, "atlas", &runtime).expect("hand");
|
|
||||||
let copy = HiveCopy::read_at(&runtime.join(HIVE_COPY_FILE));
|
|
||||||
assert!(copy.import_pending);
|
|
||||||
assert_eq!(copy.before_read(), vec![Permission::WebTools]);
|
|
||||||
|
|
||||||
hand_at(&stored, false, "iris", &runtime).expect("hand");
|
|
||||||
let copy = HiveCopy::read_at(&runtime.join(HIVE_COPY_FILE));
|
|
||||||
assert_eq!(copy, HiveCopy::default(), "no entry is the default");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn a_corrupt_file_is_an_error() {
|
fn a_corrupt_file_is_an_error() {
|
||||||
let dir = tempfile::tempdir().expect("tempdir");
|
let dir = tempfile::tempdir().expect("tempdir");
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
//! Per-container host-side config: the nspawn conf rewrite (bind mounts,
|
//! Per-container host-side config: the nspawn conf rewrite (bind mounts,
|
||||||
//! network isolation, forwarded credentials), the systemd resource-limits
|
//! network isolation, forwarded credentials), the systemd resource-limits
|
||||||
//! drop-in, the agent's copies of its memory cap and stored permissions, and
|
//! drop-in, the agent's copy of its memory cap, and
|
||||||
//! the `write_dropins` verb that re-applies them.
|
//! the `write_dropins` verb that re-applies them.
|
||||||
|
|
||||||
use std::path::{Path, PathBuf};
|
use std::path::{Path, PathBuf};
|
||||||
|
|
@ -17,8 +17,8 @@ use super::{
|
||||||
};
|
};
|
||||||
|
|
||||||
/// Re-apply the per-container host-side config: nspawn flags (bind
|
/// Re-apply the per-container host-side config: nspawn flags (bind
|
||||||
/// mounts etc.), the systemd resource-limits drop-in, the agent's copies of
|
/// mounts etc.), the systemd resource-limits drop-in, the agent's copy of
|
||||||
/// its memory cap and stored permissions, and a daemon reload so all of it
|
/// its memory cap, and a daemon reload so all of it
|
||||||
/// takes effect on the next unit (re)start. Idempotent — the job queue's
|
/// takes effect on the next unit (re)start. Idempotent — the job queue's
|
||||||
/// `WriteDropin` node, also folded into every `Swap` (rebuild is the
|
/// `WriteDropin` node, also folded into every `Swap` (rebuild is the
|
||||||
/// reconcile verb).
|
/// reconcile verb).
|
||||||
|
|
@ -26,7 +26,6 @@ pub async fn write_dropins(name: &str, hive: &HiveEnv, paths: &AgentPaths) -> Re
|
||||||
validate(name)?;
|
validate(name)?;
|
||||||
let container = container_name(name);
|
let container = container_name(name);
|
||||||
set_nspawn_flags(&container, hive, &paths.agent, &paths.claude, &paths.notes).await?;
|
set_nspawn_flags(&container, hive, &paths.agent, &paths.claude, &paths.notes).await?;
|
||||||
hand_permissions(name, &paths.agent);
|
|
||||||
if let Some((cpu_quota, memory_max)) = stage_limits(
|
if let Some((cpu_quota, memory_max)) = stage_limits(
|
||||||
name,
|
name,
|
||||||
&crate::resource_limits::swarm_limits_path(),
|
&crate::resource_limits::swarm_limits_path(),
|
||||||
|
|
@ -401,22 +400,6 @@ fn holds_manage_root_agent(agent_name: &str, path: &Path) -> bool {
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Copy `agent`'s stored permissions into its runtime dir, which it holds
|
|
||||||
/// until its own first read of the swarm's key succeeds. A failure is logged
|
|
||||||
/// and the agent still starts, on the default.
|
|
||||||
pub fn hand_permissions(agent: &str, runtime_dir: &Path) {
|
|
||||||
let pending = crate::workers::permissions_import::pending();
|
|
||||||
if let Err(e) =
|
|
||||||
crate::permissions::hand_at(&crate::permissions::path(), pending, agent, runtime_dir)
|
|
||||||
{
|
|
||||||
tracing::error!(
|
|
||||||
agent = %agent,
|
|
||||||
error = ?e,
|
|
||||||
"could not hand the agent its stored permissions; it holds the default until it reads its key"
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Idempotently rewrite the lines in `/etc/nixos-containers/<container>.conf`
|
/// Idempotently rewrite the lines in `/etc/nixos-containers/<container>.conf`
|
||||||
/// that hive-c0re owns: `PRIVATE_NETWORK` (always 1), `HOST_ADDRESS` (the
|
/// that hive-c0re owns: `PRIVATE_NETWORK` (always 1), `HOST_ADDRESS` (the
|
||||||
/// bridge gateway IP) and `EXTRA_NSPAWN_FLAGS` (the runtime-dir bind). What
|
/// bridge gateway IP) and `EXTRA_NSPAWN_FLAGS` (the runtime-dir bind). What
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@ pub use git::{
|
||||||
git_is_ancestor, git_read_tree_reset, git_rev_parse, git_show_file, git_tag, git_update_ref,
|
git_is_ancestor, git_read_tree_reset, git_rev_parse, git_show_file, git_tag, git_update_ref,
|
||||||
git_update_ref_cas, run_with_deadline,
|
git_update_ref_cas, run_with_deadline,
|
||||||
};
|
};
|
||||||
pub use host_config::{hand_permissions, write_dropins};
|
pub use host_config::write_dropins;
|
||||||
pub use setup::{
|
pub use setup::{
|
||||||
ensure_agent_state_subvolume, ensure_claude_dir, ensure_state_dir, initial_flake_nix,
|
ensure_agent_state_subvolume, ensure_claude_dir, ensure_state_dir, initial_flake_nix,
|
||||||
setup_applied, setup_proposed,
|
setup_applied, setup_proposed,
|
||||||
|
|
|
||||||
|
|
@ -120,13 +120,6 @@ fn meta_dir() -> PathBuf {
|
||||||
crate::paths::meta_root()
|
crate::paths::meta_root()
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Whether a meta repo file is still waiting to be imported.
|
|
||||||
#[must_use]
|
|
||||||
pub fn pending() -> bool {
|
|
||||||
let meta = meta_dir();
|
|
||||||
META_FILES.iter().any(|f| meta.join(f).exists())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Store the files' grants as the last-read swarm permissions, when a file
|
/// Store the files' grants as the last-read swarm permissions, when a file
|
||||||
/// is still there and nothing has been stored yet.
|
/// is still there and nothing has been stored yet.
|
||||||
pub fn seed_local() {
|
pub fn seed_local() {
|
||||||
|
|
|
||||||
|
|
@ -6,9 +6,7 @@
|
||||||
//! [`crate::permissions`], where the container's bind mounts are worked out.
|
//! [`crate::permissions`], where the container's bind mounts are worked out.
|
||||||
//! Those are fixed when the container starts, so an agent whose
|
//! Those are fixed when the container starts, so an agent whose
|
||||||
//! `manage_root_agent` changed is restarted, after its current turn. The
|
//! `manage_root_agent` changed is restarted, after its current turn. The
|
||||||
//! agent applies the rest of its permissions itself; after every pass each
|
//! agent applies the rest of its permissions itself.
|
||||||
//! agent's copy of its stored entry is refreshed
|
|
||||||
//! ([`crate::permissions::hand_at`]), which it holds until it reads its key.
|
|
||||||
//!
|
//!
|
||||||
//! A failed read, or a bucket that does not answer, leaves the stored
|
//! A failed read, or a bucket that does not answer, leaves the stored
|
||||||
//! permissions as they are and restarts nothing.
|
//! permissions as they are and restarts nothing.
|
||||||
|
|
@ -71,7 +69,6 @@ pub async fn follow(
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
hand_all().await;
|
|
||||||
tokio::select! {
|
tokio::select! {
|
||||||
() = tokio::time::sleep(READ_INTERVAL) => {}
|
() = tokio::time::sleep(READ_INTERVAL) => {}
|
||||||
_ = shutdown.changed() => {
|
_ = shutdown.changed() => {
|
||||||
|
|
@ -136,24 +133,6 @@ where
|
||||||
.collect())
|
.collect())
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Refresh the copy of its stored permissions each agent with a runtime dir
|
|
||||||
/// holds, so it follows this pass's reads and the import's progress.
|
|
||||||
async fn hand_all() {
|
|
||||||
let agents = match crate::lifecycle::agents_for_meta_listing().await {
|
|
||||||
Ok(agents) => agents,
|
|
||||||
Err(e) => {
|
|
||||||
tracing::warn!(error = ?e, "swarm permissions: enumerating agents to hand their copies failed");
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
for spec in agents {
|
|
||||||
let dir = crate::paths::agent_runtime_dir(&spec.name);
|
|
||||||
if dir.is_dir() {
|
|
||||||
crate::lifecycle::hand_permissions(&spec.name, &dir);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Whether `agent` holds the one permission hive-c0re enforces under `stored`.
|
/// Whether `agent` holds the one permission hive-c0re enforces under `stored`.
|
||||||
fn manages(agent: &str, stored: &Stored) -> bool {
|
fn manages(agent: &str, stored: &Stored) -> bool {
|
||||||
permissions::effective(stored.get(agent)).contains(&Permission::ManageRootAgent)
|
permissions::effective(stored.get(agent)).contains(&Permission::ManageRootAgent)
|
||||||
|
|
|
||||||
|
|
@ -20,7 +20,6 @@
|
||||||
//! [`Read::Failed`]: the caller checks against `HIVE_AVAILABLE_MODELS` and
|
//! [`Read::Failed`]: the caller checks against `HIVE_AVAILABLE_MODELS` and
|
||||||
//! applies no cap.
|
//! applies no cap.
|
||||||
|
|
||||||
use std::path::Path;
|
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
use std::sync::atomic::{AtomicUsize, Ordering};
|
use std::sync::atomic::{AtomicUsize, Ordering};
|
||||||
use std::sync::{Arc, OnceLock};
|
use std::sync::{Arc, OnceLock};
|
||||||
|
|
@ -28,7 +27,7 @@ use std::time::Duration;
|
||||||
|
|
||||||
use hive_types::Permission;
|
use hive_types::Permission;
|
||||||
use swarm_queue_client::agent_inference::{self, read_own};
|
use swarm_queue_client::agent_inference::{self, read_own};
|
||||||
use swarm_queue_client::agent_permissions::{self, HIVE_COPY_PATH, HiveCopy};
|
use swarm_queue_client::agent_permissions::{self, BEFORE_FIRST_READ, held};
|
||||||
use swarm_wire::agent_inference::AgentInference;
|
use swarm_wire::agent_inference::AgentInference;
|
||||||
|
|
||||||
use crate::swarm_term::{Target, connect};
|
use crate::swarm_term::{Target, connect};
|
||||||
|
|
@ -38,15 +37,15 @@ use crate::swarm_term::{Target, connect};
|
||||||
static LAST_PERMISSIONS: std::sync::RwLock<Option<Vec<Permission>>> = std::sync::RwLock::new(None);
|
static LAST_PERMISSIONS: std::sync::RwLock<Option<Vec<Permission>>> = std::sync::RwLock::new(None);
|
||||||
|
|
||||||
/// The permissions this agent holds now, for the subagent being spawned: the
|
/// The permissions this agent holds now, for the subagent being spawned: the
|
||||||
/// last set read, or what hive-c0re stored for the agent before any read
|
/// last set read, or nothing before any read succeeds
|
||||||
/// succeeds ([`HiveCopy::before_read`]).
|
/// ([`BEFORE_FIRST_READ`]).
|
||||||
#[must_use]
|
#[must_use]
|
||||||
pub fn permissions_now() -> Vec<Permission> {
|
pub fn permissions_now() -> Vec<Permission> {
|
||||||
let last = LAST_PERMISSIONS
|
let last = LAST_PERMISSIONS
|
||||||
.read()
|
.read()
|
||||||
.unwrap_or_else(std::sync::PoisonError::into_inner)
|
.unwrap_or_else(std::sync::PoisonError::into_inner)
|
||||||
.clone();
|
.clone();
|
||||||
last.unwrap_or_else(|| HiveCopy::read_at(Path::new(HIVE_COPY_PATH)).before_read())
|
last.unwrap_or_else(|| BEFORE_FIRST_READ.to_vec())
|
||||||
}
|
}
|
||||||
|
|
||||||
/// How long after a failed connect the task tries again.
|
/// How long after a failed connect the task tries again.
|
||||||
|
|
@ -150,8 +149,8 @@ impl Reader {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Read the permissions the swarm grants this agent and keep them for
|
/// Read the permissions the swarm grants this agent and keep them for
|
||||||
/// [`permissions_now`], judged by [`HiveCopy::after_read`]. A read that
|
/// [`permissions_now`], with an unset key the default. A read that fails keeps the
|
||||||
/// fails keeps the last value.
|
/// last value.
|
||||||
pub async fn refresh_permissions(&self) {
|
pub async fn refresh_permissions(&self) {
|
||||||
let read = match &self.source {
|
let read = match &self.source {
|
||||||
Source::Queue { agent, client } => read_permissions(agent, client).await,
|
Source::Queue { agent, client } => read_permissions(agent, client).await,
|
||||||
|
|
@ -173,9 +172,7 @@ async fn read_permissions(
|
||||||
let client = client.get()?;
|
let client = client.get()?;
|
||||||
swarm_queue_client::ensure_connected(client).ok()?;
|
swarm_queue_client::ensure_connected(client).ok()?;
|
||||||
match tokio::time::timeout(READ_TIMEOUT, agent_permissions::read(client, agent)).await {
|
match tokio::time::timeout(READ_TIMEOUT, agent_permissions::read(client, agent)).await {
|
||||||
Ok(Ok(agent_permissions::Read::Current(record))) => {
|
Ok(Ok(agent_permissions::Read::Current(record))) => Some(held(&record)),
|
||||||
Some(HiveCopy::read_at(Path::new(HIVE_COPY_PATH)).after_read(&record))
|
|
||||||
}
|
|
||||||
Ok(Ok(agent_permissions::Read::NoBucket)) => None,
|
Ok(Ok(agent_permissions::Read::NoBucket)) => None,
|
||||||
Ok(Err(e)) => {
|
Ok(Err(e)) => {
|
||||||
tracing::warn!(error = %e, "subagent permissions: reading this agent's key failed");
|
tracing::warn!(error = %e, "subagent permissions: reading this agent's key failed");
|
||||||
|
|
@ -238,4 +235,16 @@ mod tests {
|
||||||
let client = OnceLock::new();
|
let client = OnceLock::new();
|
||||||
assert!(matches!(read_queue("iris", &client).await, Read::Failed(_)));
|
assert!(matches!(read_queue("iris", &client).await, Read::Failed(_)));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// No test sets `LAST_PERMISSIONS` (a fixed reader never reads the
|
||||||
|
/// permissions), so this is the state before the first read.
|
||||||
|
#[test]
|
||||||
|
fn before_the_first_read_a_subagent_gets_no_execution() {
|
||||||
|
let held = permissions_now();
|
||||||
|
assert!(held.is_empty(), "held {held:?} before any read");
|
||||||
|
assert!(
|
||||||
|
Permission::DEFAULT.contains(&Permission::Execution),
|
||||||
|
"control: the default does hold it"
|
||||||
|
);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -51,7 +51,7 @@ operator-profiles = ["async-nats/jetstream"]
|
||||||
# `agent_limits::read`, for hive-c0re, which reads the key of each agent it
|
# `agent_limits::read`, for hive-c0re, which reads the key of each agent it
|
||||||
# runs the same way. Explicit for the same reason as `notices`.
|
# runs the same way. Explicit for the same reason as `notices`.
|
||||||
agent-limits = ["async-nats/jetstream"]
|
agent-limits = ["async-nats/jetstream"]
|
||||||
# `agent_permissions::read` and `HiveCopy`, for hive-c0re, the agent
|
# `agent_permissions::read` and `held`, for hive-c0re, the agent
|
||||||
# harness, the subagent daemon and swarm-controller. Pulls `hive-types` for the
|
# harness, the subagent daemon and swarm-controller. Pulls `hive-types` for the
|
||||||
# `Permission` names, so the crates that only need the bucket name do not link
|
# `Permission` names, so the crates that only need the bucket name do not link
|
||||||
# it.
|
# it.
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,4 @@
|
||||||
//! Opening and reading the per-agent permissions KV bucket, and the copy
|
//! Opening and reading the per-agent permissions KV bucket. The names, the
|
||||||
//! hive-c0re keeps of a record for the agent's own processes. The names, the
|
|
||||||
//! stored type and the import messages are `swarm_wire::agent_permissions`;
|
//! stored type and the import messages are `swarm_wire::agent_permissions`;
|
||||||
//! reading needs the `agent-permissions` feature and writing needs `kv`.
|
//! reading needs the `agent-permissions` feature and writing needs `kv`.
|
||||||
|
|
||||||
|
|
@ -34,72 +33,21 @@ pub enum Read {
|
||||||
NoBucket,
|
NoBucket,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The file hive-c0re writes an agent's [`HiveCopy`] to in that agent's
|
/// What an agent holds until its first read of its key succeeds: nothing, the
|
||||||
/// runtime dir.
|
/// narrowest set. Not the default, which includes `Execution`; a read of an
|
||||||
|
/// unset key is what makes an agent hold the default.
|
||||||
#[cfg(feature = "agent-permissions")]
|
#[cfg(feature = "agent-permissions")]
|
||||||
pub const HIVE_COPY_FILE: &str = "swarm-permissions.json";
|
pub const BEFORE_FIRST_READ: &[hive_types::Permission] = &[];
|
||||||
|
|
||||||
/// Where the agent's own processes read [`HIVE_COPY_FILE`]: the runtime dir
|
/// The permissions held under `record`: its list when set (`[]` holds none),
|
||||||
/// is mounted at `/run/hive` in the container.
|
/// the default when unset.
|
||||||
#[cfg(feature = "agent-permissions")]
|
#[cfg(feature = "agent-permissions")]
|
||||||
pub const HIVE_COPY_PATH: &str = "/run/hive/swarm-permissions.json";
|
#[must_use]
|
||||||
|
pub fn held(record: &AgentPermissions) -> Vec<hive_types::Permission> {
|
||||||
/// What hive-c0re holds for one agent: its record as the hive last stored it
|
record
|
||||||
/// (read from this bucket, or taken from the meta repo before the import),
|
|
||||||
/// and whether the meta repo's import is still pending. The agent's harness
|
|
||||||
/// and subagent daemon hold [`HiveCopy::before_read`] until their own first
|
|
||||||
/// read of the key succeeds, and judge every read with
|
|
||||||
/// [`HiveCopy::after_read`].
|
|
||||||
#[cfg(feature = "agent-permissions")]
|
|
||||||
#[derive(Debug, Clone, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
|
||||||
pub struct HiveCopy {
|
|
||||||
#[serde(flatten)]
|
|
||||||
pub stored: AgentPermissions,
|
|
||||||
#[serde(default)]
|
|
||||||
pub import_pending: bool,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(feature = "agent-permissions")]
|
|
||||||
impl HiveCopy {
|
|
||||||
/// The copy at `path`. A missing or unreadable file is the empty copy,
|
|
||||||
/// which holds the default and has no import pending.
|
|
||||||
#[must_use]
|
|
||||||
pub fn read_at(path: &std::path::Path) -> Self {
|
|
||||||
let bytes = match std::fs::read(path) {
|
|
||||||
Ok(bytes) => bytes,
|
|
||||||
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Self::default(),
|
|
||||||
Err(e) => {
|
|
||||||
tracing::warn!(path = %path.display(), error = %e, "permissions: the hive's copy is unreadable");
|
|
||||||
return Self::default();
|
|
||||||
}
|
|
||||||
};
|
|
||||||
serde_json::from_slice(&bytes).unwrap_or_else(|e| {
|
|
||||||
tracing::warn!(path = %path.display(), error = %e, "permissions: the hive's copy does not decode");
|
|
||||||
Self::default()
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
/// What the agent holds before any read of its key has succeeded: the
|
|
||||||
/// stored list, or the default when the hive stored none.
|
|
||||||
#[must_use]
|
|
||||||
pub fn before_read(&self) -> Vec<hive_types::Permission> {
|
|
||||||
self.stored
|
|
||||||
.permissions
|
.permissions
|
||||||
.clone()
|
.clone()
|
||||||
.unwrap_or_else(|| hive_types::Permission::DEFAULT.to_vec())
|
.unwrap_or_else(|| hive_types::Permission::DEFAULT.to_vec())
|
||||||
}
|
|
||||||
|
|
||||||
/// What the agent holds after reading `read` from its key. An unset key
|
|
||||||
/// while the import is pending may only mean the grant is not imported
|
|
||||||
/// yet, so the stored list stands; otherwise unset is the default.
|
|
||||||
#[must_use]
|
|
||||||
pub fn after_read(&self, read: &AgentPermissions) -> Vec<hive_types::Permission> {
|
|
||||||
match &read.permissions {
|
|
||||||
Some(list) => list.clone(),
|
|
||||||
None if self.import_pending => self.before_read(),
|
|
||||||
None => hive_types::Permission::DEFAULT.to_vec(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Read `agent`'s current permissions with one direct get: the one subject
|
/// Read `agent`'s current permissions with one direct get: the one subject
|
||||||
|
|
@ -162,28 +110,23 @@ fn decode(operation: Option<&str>, payload: &[u8]) -> serde_json::Result<AgentPe
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
/// Before a read the stored list stands; after one, a set key wins and an
|
/// A set key wins, `[]` holds none, and an unset key is the default.
|
||||||
/// unset key is the default unless the import is pending.
|
|
||||||
#[cfg(feature = "agent-permissions")]
|
|
||||||
#[test]
|
#[test]
|
||||||
fn the_hive_copy_stands_until_read_and_while_the_import_is_pending() {
|
fn an_unset_key_holds_the_default() {
|
||||||
use hive_types::Permission;
|
use hive_types::Permission;
|
||||||
let record = |json: &str| -> AgentPermissions { serde_json::from_str(json).unwrap() };
|
let record = |json: &str| -> AgentPermissions { serde_json::from_str(json).unwrap() };
|
||||||
let copy = |json: &str| -> HiveCopy { serde_json::from_str(json).unwrap() };
|
|
||||||
let web = vec![Permission::WebTools];
|
|
||||||
let default = Permission::DEFAULT.to_vec();
|
|
||||||
|
|
||||||
assert_eq!(copy(r#"{"permissions":["web_tools"]}"#).before_read(), web);
|
|
||||||
assert_eq!(copy("{}").before_read(), default);
|
|
||||||
|
|
||||||
let pending = copy(r#"{"permissions":["web_tools"],"import_pending":true}"#);
|
|
||||||
assert_eq!(pending.after_read(&record("{}")), web);
|
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
pending.after_read(&record(r#"{"permissions":[]}"#)),
|
held(&record(r#"{"permissions":["web_tools"]}"#)),
|
||||||
Vec::new()
|
vec![Permission::WebTools]
|
||||||
|
);
|
||||||
|
assert_eq!(held(&record(r#"{"permissions":[]}"#)), Vec::new());
|
||||||
|
assert_eq!(held(&record("{}")), Permission::DEFAULT.to_vec());
|
||||||
|
assert_ne!(
|
||||||
|
Permission::DEFAULT.to_vec(),
|
||||||
|
Vec::new(),
|
||||||
|
"control: the default is not the empty list"
|
||||||
);
|
);
|
||||||
let settled = copy(r#"{"permissions":["web_tools"]}"#);
|
|
||||||
assert_eq!(settled.after_read(&record("{}")), default);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Fields a newer controller adds must not cost an older hive the ones it
|
/// Fields a newer controller adds must not cost an older hive the ones it
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue