Watch
0
0
Fork
You've already forked hyperhive
0

remove the /shared bind from agent containers

This commit is contained in:
atlas 2026-10-26 21:33:04 +01:00
commit 7dd5d30da8
8 changed files with 52 additions and 71 deletions

View file

@ -145,9 +145,7 @@ would require opening a netns anyway.
see [`docs/trust-boundary/security.md`](security.md#hive-c0re-privilege-separation) for see [`docs/trust-boundary/security.md`](security.md#hive-c0re-privilege-separation) for
how `PrivRequest::ReloadGatewayNginx`'s containment works. how `PrivRequest::ReloadGatewayNginx`'s containment works.
⚠️ Contrast `/shared`, which _is_ sticky world-writable (`1777`): it has ⚠️ This dir has exactly one writer, so it needs no world write at all.
many legitimate writers, so sticky is the best available answer there.
This dir has exactly one writer, so it needs no world write at all.
### host admin socket access (`hivectl`) ### host admin socket access (`hivectl`)

View file

@ -37,7 +37,7 @@ Durable knowledge:
Claude session (OAuth credentials) lives at `/root/.claude/` and persists across restarts. Claude session (OAuth credentials) lives at `/root/.claude/` and persists across restarts.
**Shared space**: `/shared` is accessible to all agents (read/write). Only put things here you're willing to lose — other agents may delete them. Use for explicit cross-agent communication or shared artifacts when appropriate. **Files between agents**: there is no shared directory. Cross-agent files go through the forge (attach them to an issue or PR); small files for the operator go over matrix (`send_file`).
**Hive knowledge**: `/knowledge` is a read-only bind-mount (inside your container, at the literal path `/knowledge`) of the `internal/knowledge` repo on the forge — hive-wide reference docs shared across all agents: conventions, runbooks, shared notes. **Start at `/knowledge/README.md`**, its table of contents, then read whatever's relevant for context. The README also covers how to contribute a doc. **Hive knowledge**: `/knowledge` is a read-only bind-mount (inside your container, at the literal path `/knowledge`) of the `internal/knowledge` repo on the forge — hive-wide reference docs shared across all agents: conventions, runbooks, shared notes. **Start at `/knowledge/README.md`**, its table of contents, then read whatever's relevant for context. The README also covers how to contribute a doc.
@ -45,7 +45,7 @@ Claude session (OAuth credentials) lives at `/root/.claude/` and persists across
The `hive-forge` CLI is the supported interface to the Forgejo — issues, PRs, comments, labels, reviews, CI status, attachments, triage (`lint`). **Discover the verb list and each verb's full signature with `hive-forge --help` and `hive-forge <verb> --help`** rather than memorising them. The active repo is whichever git checkout you're standing in (inferred from the `origin` remote), or pass `-r <repo>` (global flag, works before or after the verb) to target a different one explicitly — there's no single hardcoded default repo, so don't assume one. A few conventions `--help` won't surface: **never `curl` the forge** — the CLI handles auth and is the only supported path; to check a PR's CI + mergeability use `hive-forge pr status <n>` (or `--sha <commit>` for a CI-only fast path; exit code is a merge-readiness verdict), not curl. `--body-file -` reads the body from stdin, so a HEREDOC works for multi-line comments/issues: `hive-forge comment <num> --body-file - <<EOF ... EOF`. `hive-forge pr create --title "..." --head <branch> [--push]` opens a PR and prints its URL; `--push` `git push`es the head branch first (default remote `forge`). Forge notifications are delivered via the internal message daemon (sender `forge`), not polling. A `forge` notification stays unread **on the forge** until you actually read its thread — viewing the referenced issue/PR with `hive-forge comments <n>` or `view <n>` marks that notification read (it's the forge's own read-state, not a local mirror). So when a `forge` message points you at a thread, read the thread to clear the notification instead of letting the same one linger and re-surface. (`hive-forge comment` does the opposite — it _refuses_ to post to a thread with unread activity until you've read it, so read first, then comment.) The `hive-forge` CLI is the supported interface to the Forgejo — issues, PRs, comments, labels, reviews, CI status, attachments, triage (`lint`). **Discover the verb list and each verb's full signature with `hive-forge --help` and `hive-forge <verb> --help`** rather than memorising them. The active repo is whichever git checkout you're standing in (inferred from the `origin` remote), or pass `-r <repo>` (global flag, works before or after the verb) to target a different one explicitly — there's no single hardcoded default repo, so don't assume one. A few conventions `--help` won't surface: **never `curl` the forge** — the CLI handles auth and is the only supported path; to check a PR's CI + mergeability use `hive-forge pr status <n>` (or `--sha <commit>` for a CI-only fast path; exit code is a merge-readiness verdict), not curl. `--body-file -` reads the body from stdin, so a HEREDOC works for multi-line comments/issues: `hive-forge comment <num> --body-file - <<EOF ... EOF`. `hive-forge pr create --title "..." --head <branch> [--push]` opens a PR and prints its URL; `--push` `git push`es the head branch first (default remote `forge`). Forge notifications are delivered via the internal message daemon (sender `forge`), not polling. A `forge` notification stays unread **on the forge** until you actually read its thread — viewing the referenced issue/PR with `hive-forge comments <n>` or `view <n>` marks that notification read (it's the forge's own read-state, not a local mirror). So when a `forge` message points you at a thread, read the thread to clear the notification instead of letting the same one linger and re-surface. (`hive-forge comment` does the opposite — it _refuses_ to post to a thread with unread activity until you've read it, so read first, then comment.)
Keep messages short — a few sentences each. For anything big (file listings, long diffs, transcripts, analysis), don't hand anyone a filesystem path — neither your state dir nor `/shared`; the operator can't open either and peers can't read your state. If it's about an issue or PR, attach it there (`hive-forge attach-issue` / `attach-comment`) or inline it in the comment. If it's a non-forge matter for the operator, send it over matrix (inline it; `send_file` for attachments). For a peer, inline it in the `send` or put it on the relevant forge thread, then keep the message to a short headline ("cluster audit attached to the PR, headline: 3 nodes over 80% mem"). Name any issue or PR to the operator as a full clickable URL, never a bare number. Keep messages short — a few sentences each. For anything big (file listings, long diffs, transcripts, analysis), don't hand anyone a filesystem path — not your state dir; the operator can't open it and peers can't read your state. If it's about an issue or PR, attach it there (`hive-forge attach-issue` / `attach-comment`) or inline it in the comment. If it's a non-forge matter for the operator, send it over matrix (inline it; `send_file` for attachments). For a peer, inline it in the `send` or put it on the relevant forge thread, then keep the message to a short headline ("cluster audit attached to the PR, headline: 3 nodes over 80% mem"). Name any issue or PR to the operator as a full clickable URL, never a bare number.
When your inbox has a message, handle it and stop. Don't narrate intent — act. When your inbox has a message, handle it and stop. Don't narrate intent — act.

View file

@ -249,7 +249,7 @@ fn big_dirs() -> Vec<PathBuf> {
let mut big = Vec::new(); let mut big = Vec::new();
for root in roots() { for root in roots() {
// Stay on the filesystem we measured: bind mounts (`/nix`, // Stay on the filesystem we measured: bind mounts (`/nix`,
// `/shared`, `/knowledge`) aren't this agent's to clean up. // `/knowledge`) aren't this agent's to clean up.
let Ok(md) = std::fs::metadata(&root) else { let Ok(md) = std::fs::metadata(&root) else {
continue; continue;
}; };

View file

@ -12,8 +12,7 @@ use hive_types::Permission;
use crate::coordinator::{AgentPaths, HiveEnv}; use crate::coordinator::{AgentPaths, HiveEnv};
use super::{ use super::{
AGENT_PREFIX, CONTAINER_RUNTIME_MOUNT, CONTAINER_SHARED_MOUNT, container_claude_mount, AGENT_PREFIX, CONTAINER_RUNTIME_MOUNT, container_claude_mount, container_name, validate,
container_name, validate,
}; };
/// Re-apply the per-container host-side config: nspawn flags (bind /// Re-apply the per-container host-side config: nspawn flags (bind
@ -420,26 +419,6 @@ async fn set_nspawn_flags(
claude_dir: &Path, claude_dir: &Path,
notes_dir: &Path, notes_dir: &Path,
) -> Result<()> { ) -> Result<()> {
// Ensure /shared directory exists before binding. systemd-nspawn requires the bind source to exist.
let shared_root = crate::paths::shared_root();
std::fs::create_dir_all(&shared_root)
.with_context(|| format!("create {}", shared_root.display()))?;
// Make /shared writable by every agent. Containers share host uids (no
// PrivateUsers), but each agent is a distinct unix user, so a root-owned
// 0755 dir leaves them unable to write — the documented "read/write for
// all agents" contract was broken. A setgid group would need a
// pinned GID declared in every container plus all agent users joined to
// it (cross-container coordination + a rebuild cascade); instead we use
// the /tmp model — sticky world-writable (1777). The sticky bit lets any
// agent create files while protecting each agent's entries from deletion
// by the others, and matches /shared's documented "free-for-all, may be
// deleted/lost" semantics without touching any per-agent config.
{
use std::os::unix::fs::PermissionsExt as _;
let perms = std::fs::Permissions::from_mode(0o1777);
std::fs::set_permissions(&shared_root, perms)
.with_context(|| format!("chmod 1777 {}", shared_root.display()))?;
}
// Ensure /knowledge dir exists. It may be empty until forge seeds it; // Ensure /knowledge dir exists. It may be empty until forge seeds it;
// nspawn refuses to start if the bind source is missing entirely. // nspawn refuses to start if the bind source is missing entirely.
std::fs::create_dir_all(crate::knowledge::LOCAL_DIR) std::fs::create_dir_all(crate::knowledge::LOCAL_DIR)
@ -454,7 +433,6 @@ async fn set_nspawn_flags(
// harness service's environment sets `HOME` to the same path // harness service's environment sets `HOME` to the same path
// (`agent.nix` template), so no `--setenv` plumbing // (`agent.nix` template), so no `--setenv` plumbing
// is needed here — the bind alone is enough. // is needed here — the bind alone is enough.
let claude_mount = container_claude_mount(agent_name);
// The agent's own identity *at* the swarm secret store, collected under // The agent's own identity *at* the swarm secret store, collected under
// this hive's certificate. Staged here rather than at spawn because this // this hive's certificate. Staged here rather than at spawn because this
@ -462,28 +440,7 @@ async fn set_nspawn_flags(
// `super::agent_identity` for the whole hop. // `super::agent_identity` for the whole hop.
let load_creds = super::agent_identity::stage(agent_name).await; let load_creds = super::agent_identity::stage(agent_name).await;
let mut binds: Vec<BindMount> = vec![ let mut binds = base_binds(runtime_dir, claude_dir, container_claude_mount(agent_name));
BindMount {
host_path: runtime_dir.to_string_lossy().into_owned(),
container_path: CONTAINER_RUNTIME_MOUNT.to_owned(),
read_only: false,
},
BindMount {
host_path: claude_dir.to_string_lossy().into_owned(),
container_path: claude_mount,
read_only: false,
},
BindMount {
host_path: shared_root.to_string_lossy().into_owned(),
container_path: CONTAINER_SHARED_MOUNT.to_owned(),
read_only: false,
},
BindMount {
host_path: crate::knowledge::LOCAL_DIR.to_owned(),
container_path: crate::knowledge::CONTAINER_MOUNT.to_owned(),
read_only: true,
},
];
// Own state, harness, work, and config dirs — same for every agent // Own state, harness, work, and config dirs — same for every agent
// including the manager. Config is RO: an agent must not edit its own // including the manager. Config is RO: an agent must not edit its own
@ -587,14 +544,55 @@ async fn set_nspawn_flags(
.await .await
} }
/// The binds every agent container gets, before its own state, config and
/// role-specific dirs.
fn base_binds(runtime_dir: &Path, claude_dir: &Path, claude_mount: String) -> Vec<BindMount> {
vec![
BindMount {
host_path: runtime_dir.to_string_lossy().into_owned(),
container_path: CONTAINER_RUNTIME_MOUNT.to_owned(),
read_only: false,
},
BindMount {
host_path: claude_dir.to_string_lossy().into_owned(),
container_path: claude_mount,
read_only: false,
},
BindMount {
host_path: crate::knowledge::LOCAL_DIR.to_owned(),
container_path: crate::knowledge::CONTAINER_MOUNT.to_owned(),
read_only: true,
},
]
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::{ use super::{
BindMount, HostMemory, MEMORY_CAP_FILE, bind_child_agent_dirs, bind_own_agent_dirs, BindMount, HostMemory, MEMORY_CAP_FILE, base_binds, bind_child_agent_dirs,
holds_manage_root_agent, limits_to_apply, memory_cap_value, stage_limits, bind_own_agent_dirs, holds_manage_root_agent, limits_to_apply, memory_cap_value,
stage_limits,
}; };
use crate::coordinator::HiveEnv; use crate::coordinator::HiveEnv;
/// No agent container gets a cross-agent scratch dir; `/knowledge` is the
/// control that the lookup finds a bind that is there.
#[test]
fn base_binds_have_no_shared_dir() {
let binds = base_binds(
std::path::Path::new("/run/x"),
std::path::Path::new("/var/lib/x/claude"),
"/home/a/.claude".to_owned(),
);
let targets: Vec<&str> = binds.iter().map(|b| b.container_path.as_str()).collect();
assert!(targets.contains(&"/knowledge"), "control: {targets:?}");
assert!(!targets.contains(&"/shared"), "{targets:?}");
assert!(
binds.iter().all(|b| !b.host_path.ends_with("/shared")),
"{binds:?}"
);
}
fn child_binds() -> Vec<BindMount> { fn child_binds() -> Vec<BindMount> {
let mut binds = Vec::new(); let mut binds = Vec::new();
bind_child_agent_dirs("kiddo", &mut binds); bind_child_agent_dirs("kiddo", &mut binds);

View file

@ -51,11 +51,6 @@ pub fn container_claude_mount(name: &str) -> String {
format!("/home/{name}/.claude") format!("/home/{name}/.claude")
} }
/// Mount point of the shared directory accessible to all agents.
/// All agents can read/write here; agents should only put things they're
/// willing to lose (other agents may delete them).
pub const CONTAINER_SHARED_MOUNT: &str = "/shared";
pub use hive_meta_flake::agent_web_port; pub use hive_meta_flake::agent_web_port;
/// Extract the bridge gateway IP from `HIVE_NETWORK_SUBNET`. /// Extract the bridge gateway IP from `HIVE_NETWORK_SUBNET`.

View file

@ -6,7 +6,7 @@
//! …). This module is the **single Rust-side source** for every host //! …). This module is the **single Rust-side source** for every host
//! path — the strictly host-side ones grouped into subdirs (`db/`, //! path — the strictly host-side ones grouped into subdirs (`db/`,
//! `matrix/`, `run/`), plus the **nix-coupled** roots //! `matrix/`, `run/`), plus the **nix-coupled** roots
//! (`agents/`, `applied/`, `meta/`, `shared/`, `gateway/`, `knowledge/`, //! (`agents/`, `applied/`, `meta/`, `gateway/`, `knowledge/`,
//! the `/run` runtime dirs). //! the `/run` runtime dirs).
//! //!
//! Nix-coupled paths are NOT excluded (the old stance) — they're moved //! Nix-coupled paths are NOT excluded (the old stance) — they're moved
@ -192,16 +192,6 @@ pub fn meta_rev_dir() -> PathBuf {
state_root().join("meta-rev") state_root().join("meta-rev")
} }
/// `shared/` — the cross-agent `/shared` scratch space. A `&str`, so it
/// stays a const; [`shared_root`] wraps it.
// nix: bind-mounted into every agent container as `/shared` (the harness nix modules) — must match.
pub const SHARED_ROOT: &str = "/var/lib/hyperhive/shared";
#[must_use]
pub fn shared_root() -> PathBuf {
PathBuf::from(SHARED_ROOT)
}
/// `knowledge/` — local checkout of the `internal/knowledge` repo. A /// `knowledge/` — local checkout of the `internal/knowledge` repo. A
/// `&str` (used as a git `-C` arg / clone target throughout the knowledge /// `&str` (used as a git `-C` arg / clone target throughout the knowledge
/// worker), so it stays a const rather than a `PathBuf` fn. /// worker), so it stays a const rather than a `PathBuf` fn.

View file

@ -16,7 +16,7 @@
//! artifact is served as a zip; the default output path reflects that. //! artifact is served as a zip; the default output path reflects that.
//! //!
//! Use case: an agent pulling a CI-built artifact (e.g. a paper PDF) into //! Use case: an agent pulling a CI-built artifact (e.g. a paper PDF) into
//! `/shared` for delivery or review without host access. //! `work/` for delivery or review without host access.
use std::io::Write as _; use std::io::Write as _;
use std::path::PathBuf; use std::path::PathBuf;

View file

@ -539,7 +539,7 @@ mod tests {
fn non_repo_mounts_are_left_alone() { fn non_repo_mounts_are_left_alone() {
let flags = git_overlay_flags(&[ let flags = git_overlay_flags(&[
bind("/agents/atlas/state"), bind("/agents/atlas/state"),
bind("/shared"), bind("/knowledge"),
bind("/applied"), bind("/applied"),
bind("/agents/atlas/config-notes"), bind("/agents/atlas/config-notes"),
]); ]);