vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

agents: read the memory cap at boot, not from the build

Browse source 
An agent's memory cap was rendered into its flake (`memoryMaxBytes` →
`services.hyperhive.agent.claudeMemoryMaxBytes`), so the closure depended
on it and a changed cap reached the JSC heap ceiling and the subagent
`MemoryHigh=` only after a rebuild.

hive-c0re now writes the effective `MemoryMax=` in bytes to the agent's
runtime dir (`/run/hive/memory-max-bytes` inside) in `write_dropins`,
from the same value the drop-in gets. The file is removed for
`infinity` or a RAM percentage, and left alone when the drop-in is kept.

Inside the container, `nix/agent-modules/memory-cap.nix` reads it at
boot: an environment generator sets `BUN_JSC_forceRAMSize` (75%) for
every unit and `machinectl shell`, `/etc/set-environment` exports it in
login shells, and a unit generator writes `MemoryHigh=` (2/3) as a
drop-in for `hive-subagent-daemon`.

The render drops `memoryMaxBytes`, the agent option is removed (both
spellings fail eval with a removal message), and
`effective_memory_bytes_from` goes with its only caller.

Part of #5081 (slice S0-2).
This commit is contained in:
atlas 2026-10-12 09:00:45 +02:00
commit 0d619e215e
15 changed files with 416 additions and 287 deletions
Download patch file Download diff file Expand all files Collapse all files

8
docs/agent-lifecycle/persistence.md
View file

@ -403,7 +403,8 @@ Contents:
Written by `resource_limits::set_limits`; read where
`lifecycle::write_dropins` generates the systemd drop-in, **not** injected
into the container — these are host-side caps on the container, so
the capped party never sees or sets them. Fallback is per *field*:
the capped party never sets them. It sees only the resulting memory cap,
as a byte count in its runtime dir (below). Fallback is per *field*:
an absent file, absent agent, or absent field falls back to the
hive-wide `services.hyperhive.c0re.agentCpuQuota` / `agentMemoryMax`,
so an agent can override only its memory and still track the hive
@ -508,6 +509,11 @@ containers can't start.
- `/run/hyperhive/host.sock` — admin socket (host-side CLI).
- `/run/hyperhive/agents/<name>/mcp.sock` — per-agent socket
(bind-mounted into the container as `/run/hive/mcp.sock`).
- `/run/hyperhive/agents/<name>/memory-max-bytes` — the agent's effective
`MemoryMax=` in bytes, written by `lifecycle::write_dropins` before every
start (absent when the cap is `infinity` or a RAM percentage). The
container reads it at boot as `/run/hive/memory-max-bytes`
(`nix/agent-modules/memory-cap.nix`).
On startup, `Coordinator::register_agent` drops any prior socket
task before rebinding — idempotent so a hive-c0re restart followed

8
docs/scheduler/coordinator.md
View file

@ -452,6 +452,14 @@ regardless.
rebuild, so changes take effect on the next lifecycle op without
requiring a host rebuild.
The container can't read its own cap, so `write_dropins` also writes the
effective `MemoryMax=` in bytes into the agent's runtime dir
(`/run/hive/memory-max-bytes` inside). At boot the container derives two
values from it: `BUN_JSC_forceRAMSize` (75%, claude-code's heap ceiling)
and `MemoryHigh=` on `hive-subagent-daemon` (two thirds). Neither is in the
agent's build, so a changed cap reaches them at the next container start
without a rebuild. A cap of `infinity` or a RAM percentage sets neither.
The same drop-in carries `CPUWeight=` / `IOWeight=` from
`agentCpuWeight` / `agentIoWeight`. Those are a different kind of
setting: the quota and the memory max are **hard caps** that throttle

96
hive-c0re/src/agent_config/resource_limits.rs
View file

@ -122,55 +122,6 @@ fn resolve(limits: &AgentLimits, hive_cpu_quota: &str, hive_memory_max: &str) ->
(cpu, mem)
}
/// Effective `MemoryMax=` string for `name` against an already-loaded
/// map — the memory-only half of [`effective_from`], split out because
/// [`effective_memory_bytes_from`] doesn't have a CPU quota to pass.
/// No single-agent convenience wrapper (unlike [`effective`] /
/// [`effective_from`]): every current caller already has a loaded map
/// in hand, so one would just be dead code.
#[must_use]
fn effective_memory_max_from(
limits: &BTreeMap<String, AgentLimits>,
name: &str,
hive_memory_max: &str,
) -> String {
limits
.get(name)
.and_then(|l| l.memory_max.clone())
.unwrap_or_else(|| hive_memory_max.to_owned())
}
/// Effective `MemoryMax=` for `name`, as a raw byte count, against an
/// already-loaded map — the form `render_flake_with_lookup` uses so it
/// doesn't re-read `resource-limits.json` once per agent (same
/// reasoning as [`effective_from`] / `container_view::build_all`). No
/// single-agent convenience wrapper, for the same reason as
/// [`effective_memory_max_from`] — every current caller already has a
/// loaded map in hand.
///
/// Logs a `warn!` naming `name` when the effective value is a RAM
/// percentage: unlike `"infinity"`, a percentage IS a real, resolvable
/// cap — resolving one just needs host `MemTotal`, which this module
/// doesn't track — so today it degrades to `None` (no derived heap
/// ceiling) rather than silently guessing wrong against `MemTotal`.
#[must_use]
pub fn effective_memory_bytes_from(
limits: &BTreeMap<String, AgentLimits>,
name: &str,
hive_memory_max: &str,
) -> Option<u64> {
let mem = effective_memory_max_from(limits, name, hive_memory_max);
if mem.ends_with('%') {
tracing::warn!(
agent = name,
memory_max = %mem,
"effective MemoryMax= is a RAM percentage; can't derive a JSC heap ceiling from it \
without host MemTotal — leaving BUN_JSC_forceRAMSize unset for this agent"
);
}
parse_bytes(&mem)
}
/// Parse a systemd `MemoryMax=`-style byte-size value (`"4G"`, `"512M"`,
/// a bare byte count, optionally with a decimal like `"1.5G"`) into a
/// raw byte count. systemd's `K`/`M`/`G`/`T` suffixes are IEC binary
@ -179,9 +130,8 @@ pub fn effective_memory_bytes_from(
/// value is always a small non-negative decimal (enforced by
/// [`is_plain_number`] upstream in [`validate_memory_max`]), so floats
/// would only add rounding / sign-loss risk for no benefit. Returns
/// `None` for `"infinity"` and percentages; see
/// [`effective_memory_bytes_from`] for why those can't be turned into a
/// byte count here.
/// `None` for `"infinity"` and for percentages, which need the host's
/// `MemTotal` to resolve.
#[must_use]
pub fn parse_bytes(value: &str) -> Option<u64> {
if value == "infinity" || value.ends_with('%') {
@ -532,46 +482,4 @@ mod tests {
assert_eq!(parse_bytes(v), None, "{v} should not parse");
}
}
#[test]
fn effective_memory_bytes_falls_back_to_hive_default() {
// Empty map, i.e. no per-agent override — exercises the
// "use the hive-wide default" arm end to end.
let empty = BTreeMap::new();
assert_eq!(
effective_memory_bytes_from(&empty, "nobody-configured-this-agent", "4G"),
Some(4 * 1024 * 1024 * 1024)
);
assert_eq!(
effective_memory_bytes_from(&empty, "nobody-configured-this-agent", "infinity"),
None
);
}
#[test]
fn effective_memory_bytes_from_prefers_per_agent_override() {
let map = BTreeMap::from([("sock".to_owned(), limits(None, Some("8G")))]);
assert_eq!(
effective_memory_bytes_from(&map, "sock", HIVE_MEM),
Some(8 * 1024 * 1024 * 1024)
);
// A different agent not in the map still falls back to the
// hive-wide default from the same loaded map (no re-read).
assert_eq!(
effective_memory_bytes_from(&map, "iris", HIVE_MEM),
Some(4 * 1024 * 1024 * 1024)
);
}
/// A percentage cap is real and resolvable in principle, but this
/// module has no host `MemTotal` to resolve it against — must
/// degrade to `None` (no derived heap ceiling) rather than silently
/// treating it as unbounded or guessing a number. `warn!` firing is
/// exercised for coverage but not asserted on (no tracing test
/// subscriber wired up here) — the `None` return is the contract.
#[test]
fn effective_memory_bytes_from_returns_none_for_percentage() {
let map = BTreeMap::from([("sock".to_owned(), limits(None, Some("50%")))]);
assert_eq!(effective_memory_bytes_from(&map, "sock", HIVE_MEM), None);
}
}

137
hive-c0re/src/lifecycle/host_config.rs
View file

@ -1,6 +1,7 @@
//! Per-container host-side config: the nspawn conf rewrite (bind mounts,
//! network isolation, forwarded credentials), the systemd resource-limits
//! drop-in, and the `write_dropins` verb that re-applies both.
//! drop-in, the agent's copy of its memory cap, and the `write_dropins` verb
//! that re-applies them.
use std::path::{Path, PathBuf};
@ -16,20 +17,21 @@ use super::{
};
/// Re-apply the per-container host-side config: nspawn flags (bind
/// mounts etc.), the systemd resource-limits drop-in, and a daemon
/// reload so both take effect on the next unit (re)start. Idempotent —
/// the job queue's `WriteDropin` node, also folded into every `Swap`
/// (rebuild is the reconcile verb).
/// mounts etc.), the systemd resource-limits drop-in, the agent's copy of
/// its memory cap, and a daemon reload so all of it takes effect on the next
/// unit (re)start. Idempotent — the job queue's `WriteDropin` node, also
/// folded into every `Swap` (rebuild is the reconcile verb).
pub async fn write_dropins(name: &str, hive: &HiveEnv, paths: &AgentPaths) -> Result<()> {
validate(name)?;
let container = container_name(name);
set_nspawn_flags(&container, &paths.agent, &paths.claude, &paths.notes).await?;
if let Some((cpu_quota, memory_max)) = limits_to_apply(
if let Some((cpu_quota, memory_max)) = stage_limits(
name,
&crate::resource_limits::resource_limits_path(),
&limits_dropin_path(&container),
&paths.agent,
hive,
) {
)? {
set_resource_limits(
&container,
&cpu_quota,
@ -51,6 +53,60 @@ fn limits_dropin_path(container: &str) -> PathBuf {
))
}
/// File in the agent's runtime dir, `/run/hive/` in the container, holding the
/// byte count of the `MemoryMax=` its drop-in carries.
/// `nix/agent-modules/memory-cap.nix` reads it at boot under this name.
const MEMORY_CAP_FILE: &str = "memory-max-bytes";
/// [`limits_to_apply`], plus handing the memory cap to the agent through
/// [`MEMORY_CAP_FILE`] in `runtime_dir`. The container cannot read the cap
/// systemd enforces on it, so this file is how it learns the value it sizes
/// its heap and its subagents against. When the drop-in is kept as it is, so is
/// the file.
fn stage_limits(
agent_name: &str,
limits_path: &Path,
dropin_path: &Path,
runtime_dir: &Path,
hive: &HiveEnv,
) -> Result<Option<(String, String)>> {
let limits = limits_to_apply(agent_name, limits_path, dropin_path, hive);
if let Some((_, memory_max)) = &limits {
write_memory_cap(runtime_dir, agent_name, memory_max)?;
}
Ok(limits)
}
/// Write `memory_max` into [`MEMORY_CAP_FILE`] as a byte count, or remove the
/// file when the value has none: `infinity`, or a RAM percentage, which needs
/// the host's `MemTotal` to resolve. With no file the agent sets no heap or
/// subagent ceiling.
fn write_memory_cap(runtime_dir: &Path, agent_name: &str, memory_max: &str) -> Result<()> {
let path = runtime_dir.join(MEMORY_CAP_FILE);
let Some(bytes) = crate::resource_limits::parse_bytes(memory_max) else {
if memory_max.ends_with('%') {
tracing::warn!(
agent = agent_name,
memory_max,
"effective MemoryMax= is a RAM percentage; the agent gets no byte count, so \
no JSC heap ceiling and no subagent MemoryHigh="
);
}
return match std::fs::remove_file(&path) {
Err(e) if e.kind() != std::io::ErrorKind::NotFound => {
Err(e).with_context(|| format!("remove {}", path.display()))
}
_ => Ok(()),
};
};
std::fs::create_dir_all(runtime_dir)
.with_context(|| format!("create agent runtime dir {}", runtime_dir.display()))?;
let tmp = runtime_dir.join(format!(".{MEMORY_CAP_FILE}.tmp"));
std::fs::write(&tmp, format!("{bytes}\n"))
.with_context(|| format!("write {}", tmp.display()))?;
std::fs::rename(&tmp, &path).with_context(|| format!("rename onto {}", path.display()))
}
/// The `(CPUQuota, MemoryMax)` to write into `agent_name`'s limits
/// drop-in, or `None` to leave the drop-in at `dropin_path` as it is.
///
@ -489,8 +545,9 @@ async fn set_nspawn_flags(
#[cfg(test)]
mod tests {
use super::{
BindMount, QUEUE_CLIENT_ID_CREDENTIAL, QUEUE_SECRET_CREDENTIAL, bind_child_agent_dirs,
bind_own_agent_dirs, holds_manage_root_agent, limits_to_apply, queue_agent_credentials,
BindMount, MEMORY_CAP_FILE, QUEUE_CLIENT_ID_CREDENTIAL, QUEUE_SECRET_CREDENTIAL,
bind_child_agent_dirs, bind_own_agent_dirs, holds_manage_root_agent, limits_to_apply,
queue_agent_credentials, stage_limits,
};
use crate::coordinator::HiveEnv;
@ -707,4 +764,66 @@ mod tests {
let applied = limits_to_apply("sock", &limits, &dropin, &HiveEnv::default());
assert_eq!(applied, Some(("200%".to_owned(), "1G".to_owned())));
}
/// Runs [`stage_limits`] against the files in `dir` and returns the
/// drop-in's `MemoryMax=` and the agent's copy of it, `None` when absent.
fn stage(dir: &tempfile::TempDir) -> (Option<String>, Option<String>) {
let runtime = dir.path().join("runtime");
let applied = stage_limits(
"sock",
&dir.path().join("resource-limits.json"),
&dir.path().join("hyperhive-limits.conf"),
&runtime,
&HiveEnv::default(),
)
.expect("stage limits");
let copy = std::fs::read_to_string(runtime.join(MEMORY_CAP_FILE)).ok();
(applied.map(|(_, mem)| mem), copy)
}
#[test]
fn the_agent_reads_the_cap_its_dropin_carries() {
let dir = tempfile::tempdir().expect("tempdir");
let limits = dir.path().join("resource-limits.json");
// No overrides anywhere: the hive-wide default, 4G.
assert_eq!(
stage(&dir),
(Some("4G".to_owned()), Some("4294967296\n".to_owned()))
);
std::fs::write(&limits, r#"{"sock": {"memory_max": "16G"}}"#).expect("seed limits");
assert_eq!(
stage(&dir),
(Some("16G".to_owned()), Some("17179869184\n".to_owned()))
);
}
#[test]
fn a_cap_with_no_byte_count_removes_the_agents_copy() {
let dir = tempfile::tempdir().expect("tempdir");
let limits = dir.path().join("resource-limits.json");
assert_eq!(stage(&dir).1, Some("4294967296\n".to_owned()));
std::fs::write(&limits, r#"{"sock": {"memory_max": "infinity"}}"#).expect("seed limits");
assert_eq!(stage(&dir), (Some("infinity".to_owned()), None));
assert_eq!(stage(&dir).1, None, "an absent copy stays absent");
std::fs::write(&limits, r#"{"sock": {"memory_max": "50%"}}"#).expect("seed limits");
assert_eq!(stage(&dir), (Some("50%".to_owned()), None));
}
#[test]
fn a_kept_dropin_keeps_the_agents_copy() {
let dir = tempfile::tempdir().expect("tempdir");
let limits = dir.path().join("resource-limits.json");
std::fs::write(&limits, r#"{"sock": {"memory_max": "1G"}}"#).expect("seed limits");
assert_eq!(stage(&dir).1, Some("1073741824\n".to_owned()));
std::fs::write(&limits, TRUNCATED_LIMITS).expect("seed limits");
std::fs::write(dir.path().join("hyperhive-limits.conf"), "MemoryMax=1G\n")
.expect("seed drop-in");
assert_eq!(stage(&dir), (None, Some("1073741824\n".to_owned())));
}
}

107
hive-c0re/src/meta.rs
View file

@ -88,17 +88,7 @@ pub async fn sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<()> {
let dir = crate::paths::meta_root();
std::fs::create_dir_all(&dir).with_context(|| format!("create {}", dir.display()))?;
let new_flake = render_flake(
&hive.hyperhive_flake,
&hive.hyperhive_docs_flake,
&hive.nixpkgs_flake,
hive.claude_code_path.as_deref(),
hive.dashboard_port,
&hive.operator_pronouns,
&hive.context_window_tokens,
&hive.agent_memory_max,
agents,
)?;
let new_flake = render_hive_flake(hive, agents)?;
let flake_path = dir.join("flake.nix");
let on_disk = std::fs::read_to_string(&flake_path).unwrap_or_default();
let initial = !dir.join(".git").exists();
@ -444,6 +434,21 @@ pub async fn commit_perms(
Ok(())
}
/// [`render_flake`] with its inputs taken from `hive`, as [`sync_agents`]
/// renders it.
fn render_hive_flake(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<String> {
render_flake(
&hive.hyperhive_flake,
&hive.hyperhive_docs_flake,
&hive.nixpkgs_flake,
hive.claude_code_path.as_deref(),
hive.dashboard_port,
&hive.operator_pronouns,
&hive.context_window_tokens,
agents,
)
}
#[allow(
clippy::too_many_arguments,
reason = "many genuine flake inputs (source flakes, port, pronouns, tokens, \
@ -457,7 +462,6 @@ fn render_flake(
dashboard_port: u16,
operator_pronouns: &str,
context_window_tokens: &std::collections::HashMap<String, u64>,
hive_memory_max: &str,
agents: &[AgentSpec],
) -> Result<String> {
render_flake_with_lookup(
@ -468,7 +472,6 @@ fn render_flake(
dashboard_port,
operator_pronouns,
context_window_tokens,
hive_memory_max,
agents,
agent_canonical_inputs,
)
@ -861,7 +864,6 @@ fn render_flake_with_lookup<F>(
dashboard_port: u16,
operator_pronouns: &str,
context_window_tokens: &std::collections::HashMap<String, u64>,
hive_memory_max: &str,
agents: &[AgentSpec],
lookup: F,
) -> Result<String>
@ -943,7 +945,7 @@ where
let pronouns_escaped = operator_pronouns.replace('\\', "\\\\").replace('"', "\\\"");
let _ = writeln!(
out,
" dashboardPort = {dashboard_port};\n operatorPronouns = \"{pronouns_escaped}\";\n mkAgent = {{ name, isManager, port, toolGroups ? null, capabilities ? null, memoryMaxBytes ? null }}:"
" dashboardPort = {dashboard_port};\n operatorPronouns = \"{pronouns_escaped}\";\n mkAgent = {{ name, isManager, port, toolGroups ? null, capabilities ? null }}:"
);
out.push_str(
r#" let
@ -1094,7 +1096,6 @@ where
# nspawn itself already call this container, so logs join to
# both without a prefix transform.
networking.hostName = "h-${name}";
services.hyperhive.agent.claudeMemoryMaxBytes = memoryMaxBytes;
programs.git.config.user = {
name = name;
email = "${name}@hyperhive.local";
@ -1177,12 +1178,9 @@ where
);
// Propagated, never read as empty: an agent with no tool-groups entry
// renders `toolGroups = null` and gets `AGENT_DEFAULT`, which fails
// open for any agent whose explicit entry is narrower. Likewise an
// agent with no resource-limits entry gets the hive `memoryMaxBytes`,
// a larger heap ceiling than a tighter override.
// open for any agent whose explicit entry is narrower.
let tool_groups_map = crate::tool_groups::read()?;
let capabilities_map = crate::capabilities::read()?;
let resource_limits_map = crate::resource_limits::read()?;
for spec in agents {
// Emit `toolGroups = "group1,group2"` when the operator has
// explicitly configured groups for this agent. Absent entry = null
@ -1208,29 +1206,15 @@ where
let joined = caps.join(",");
format!("\"{joined}\"")
};
// Effective `MemoryMax=` for this agent (per-agent override, else
// the hive-wide default), turned into a raw byte count so
// the agent's claude-settings module can derive a JSC heap ceiling
// (see `services.hyperhive.agent.claudeMemoryMaxBytes`). `null` when the
// effective value is `"infinity"` or a RAM percentage — no
// byte count to derive, dependent env var stays unset, same as
// today's no-cap behavior.
let memory_max_attr = crate::resource_limits::effective_memory_bytes_from(
&resource_limits_map,
&spec.name,
hive_memory_max,
)
.map_or_else(|| "null".to_owned(), |b| b.to_string());
let _ = writeln!(
out,
" {} = mkAgent {{ name = \"{}\"; isManager = {}; port = {}; toolGroups = {}; capabilities = {}; memoryMaxBytes = {}; }};",
" {} = mkAgent {{ name = \"{}\"; isManager = {}; port = {}; toolGroups = {}; capabilities = {}; }};",
spec.name,
spec.name,
if spec.is_manager { "true" } else { "false" },
spec.port,
tool_groups_attr,
capabilities_attr,
memory_max_attr,
);
}
out.push_str(" };\n };\n}\n");
@ -1468,7 +1452,6 @@ mod tests {
8000,
"she/her",
&std::collections::HashMap::new(),
"4G",
&[sample_spec("alice", false, 9001)],
)
.expect("render flake");
@ -1516,7 +1499,6 @@ mod tests {
8000,
"she/her",
&std::collections::HashMap::new(),
"4G",
&[sample_spec("alice", false, 9001)],
)
.expect("render flake");
@ -1536,7 +1518,6 @@ mod tests {
8000,
"she/her",
&std::collections::HashMap::new(),
"4G",
&[sample_spec("alice", false, 9001)],
)
.expect("render flake");
@ -1578,7 +1559,6 @@ mod tests {
8000,
"she/her",
&std::collections::HashMap::new(),
"4G",
&[sample_spec("alice", false, 9001)],
)
.expect("render flake");
@ -1608,7 +1588,6 @@ mod tests {
8000,
"she/her",
&std::collections::HashMap::new(),
"4G",
&[sample_spec("alice", false, 9001)],
)
.expect("render flake");
@ -1630,7 +1609,6 @@ mod tests {
8000,
"she/her",
&std::collections::HashMap::new(),
"4G",
&[sample_spec("alice", false, 9001)],
)
.expect("render flake");
@ -1662,7 +1640,6 @@ mod tests {
8000,
"she/her",
&std::collections::HashMap::new(),
"4G",
&[
sample_spec("argus", false, 9001),
sample_spec("bitburner", false, 9002),
@ -1697,7 +1674,6 @@ mod tests {
8000,
"she/her",
&std::collections::HashMap::new(),
"4G",
&[sample_spec("alice", false, 9001)],
|_| Vec::new(),
)
@ -1737,7 +1713,6 @@ mod tests {
8000,
"she/her",
&std::collections::HashMap::new(),
"4G",
&[sample_spec("alice", false, 9001)],
)
.expect("render flake");
@ -1784,7 +1759,6 @@ mod tests {
8000,
"she/her",
&std::collections::HashMap::new(),
"4G",
&[sample_spec("alice", false, 9001)],
)
.expect("render flake")
@ -1966,7 +1940,6 @@ mod tests {
8000,
"she/her",
&std::collections::HashMap::new(),
"4G",
&[sample_spec("alice", false, 9001)],
)
.expect("render flake");
@ -2004,7 +1977,6 @@ mod tests {
8000,
"she/her",
&std::collections::HashMap::new(),
"4G",
&[sample_spec("alice", false, 9001)],
)
.expect("render flake");
@ -2050,7 +2022,6 @@ mod tests {
8000,
"she/her",
&std::collections::HashMap::new(),
"4G",
&[sample_spec("alice", false, 9001)],
)
.expect("render flake")
@ -2144,7 +2115,6 @@ mod tests {
8000,
"she/her",
&std::collections::HashMap::new(),
"4G",
&[sample_spec("alice", false, 9001)],
)
.expect("render flake")
@ -2220,7 +2190,6 @@ mod tests {
8000,
"she/her",
&std::collections::HashMap::new(),
"4G",
&[sample_spec("alice", false, 9001)],
)
.expect("render flake")
@ -2246,32 +2215,20 @@ mod tests {
);
}
/// The JSC-heap-ceiling fix needs the effective per-agent memory cap
/// threaded into the flake as raw bytes, since nspawn hides the real
/// cgroup cap from inside the container. An
/// agent with no `resource-limits.json` override falls back to the
/// hive-wide default passed to `render_flake` (`"4G"` in every test
/// in this module) — this locks in the byte-count conversion end to
/// end through the actual render path (not just `parse_bytes`
/// in isolation).
/// The agent reads its memory cap at boot from the copy
/// `lifecycle::write_dropins` hands it, so the cap is no build input.
#[test]
fn render_flake_derives_memory_max_bytes_from_hive_default() {
let out = render_flake(
"github:example/hyperhive",
"path:/nix/store/bbbb-hyperhive-docs-source",
"path:/nix/store/aaaa-nixpkgs-source",
None,
8000,
"she/her",
&std::collections::HashMap::new(),
"4G",
&[sample_spec("alice", false, 9001)],
)
.expect("render flake");
let want_bytes = 4u64 * 1024 * 1024 * 1024;
assert!(
out.contains(&format!("memoryMaxBytes = {want_bytes};")),
"memoryMaxBytes must reflect the hive-wide default in bytes:\n{out}"
fn render_hive_flake_is_identical_across_memory_caps() {
let hive = |cap: &str| HiveEnv {
agent_memory_max: cap.to_owned(),
..HiveEnv::default()
};
let agents = [sample_spec("alice", false, 9001)];
let small = render_hive_flake(&hive("4G"), &agents).expect("render 4G");
let large = render_hive_flake(&hive("16G"), &agents).expect("render 16G");
assert_eq!(
small, large,
"the rendered flake must not depend on the memory cap"
);
}
}

7
hive-c0re/src/server.rs
View file

@ -433,10 +433,9 @@ async fn handle_set_resource_limits(
&hive.agent_memory_max,
)?;
Ok(HostResponse::messages(vec![format!(
"{name}: CPUQuota={cpu} MemoryMax={mem} — the cgroup cap itself is live now (restart \
the container if it's running and needs the new cap immediately), but the derived \
Claude/JSC heap ceiling is baked in at build time, so it needs a REBUILD \
(`hivectl agent {name} rebuild`) to actually track this change"
"{name}: CPUQuota={cpu} MemoryMax={mem} — the cgroup cap itself is live now; the \
Claude/JSC heap ceiling and the subagent MemoryHigh= derived from it follow at the \
container's next start (restart it if it's running and needs them now)"
)]))
}

47
nix/agent-modules/claude-settings.nix
View file

@ -19,11 +19,6 @@ let
# meta.rs `otel_config`). Options declared in `otel.nix`, which also
# exports the generic OTLP environment this container's producers read.
otelCfg = config.services.hyperhive.agent.otel;
# Effective per-agent MemoryMax=, in bytes, injected by meta.rs's
# per-agent flake render (`services.hyperhive.agent.claudeMemoryMaxBytes`). `null`
# when the effective cap is unbounded ("infinity") or a RAM
# percentage — see `resource_limits::effective_memory_bytes`.
memoryMaxBytes = config.services.hyperhive.agent.claudeMemoryMaxBytes;
# Base claude-code environment applied to every agent regardless of OTEL.
# Shipped via the managed settings `env` block so claude and `hivectl
# choom` both inherit them without a launch wrapper.
@ -47,19 +42,6 @@ let
CLAUDE_CODE_RESUME_INTERRUPTED_TURN = "1";
# Use the simpler system prompt variant suited to headless operation.
CLAUDE_CODE_SIMPLE_SYSTEM_PROMPT = "1";
}
# Bun/JavaScriptCore (the runtime under claude-code's `.claude-wrapped`
# binary) sizes its default heap ceiling off the container's visible
# `/proc/meminfo`, which under nspawn is the *host's* physical RAM, not
# the systemd `MemoryMax=` cgroup cap actually enforced on this
# container — the cap lives on the host's outer cgroup and is invisible
# from inside (confirmed: `/sys/fs/cgroup/memory.max` reads `max` at
# every level from in here). That mismatch lets JSC's heap grow well
# past the real wall before GC kicks in hard, causing choom sessions to
# hang. Pin JSC's ceiling to 75% of the *actual* effective cap instead,
# once it's known at build time.
// lib.optionalAttrs (memoryMaxBytes != null) {
BUN_JSC_forceRAMSize = toString (memoryMaxBytes * 75 / 100);
};
# Claude Code's own telemetry switches — what it emits, and whether it
# emits at all. Everything an OTEL SDK reads generically (endpoint,
@ -99,31 +81,6 @@ in
'';
};
# Build-time implementation surface for the JSC-heap-ceiling fix:
# meta.rs's per-agent flake render injects this from the effective
# `MemoryMax=` (per-agent `resource-limits.json` override, else the
# hive-wide `services.hyperhive.agentMemoryMax`) — see
# `resource_limits::effective_memory_bytes_from`. Not meant to be set
# directly in an agent.nix, same convention as `services.hyperhive.agent.otel.*`
# above; the host option (or `hivectl agent <name> set-limits`) is
# the real operator knob, and this only reflects the value baked in at
# the agent's *last rebuild* — `set-limits` still applies the
# cgroup cap live via a drop-in reload, but this derived heap ceiling
# needs a rebuild to pick up a new value.
options.services.hyperhive.agent.claudeMemoryMaxBytes = lib.mkOption {
type = lib.types.nullOr lib.types.ints.positive;
default = null;
internal = true;
description = ''
Effective per-agent memory cap in bytes, when it's a plain
byte-size value (null for an unbounded or percentage-based cap).
This is the whole container's cap, not claude's share of it —
the name records its first consumer, `BUN_JSC_forceRAMSize` in
`baseClaudeEnv`. `mcp.nix` reads it too, to size the subagent
daemon's `MemoryHigh=` against the container it runs in.
'';
};
options.services.hyperhive.agent.claudeMarketplaces = lib.mkOption {
type = lib.types.listOf lib.types.str;
default = [
@ -257,9 +214,7 @@ in
environment.etc."claude-code/managed-settings.json".source =
let
baseSettings = "${config.services.hyperhive.agent.packages.assets}/share/hyperhive/prompts/claude-settings.json";
# Merge base env (always) with OTEL env (when enabled). jq is always
# run — `baseClaudeEnv` can carry per-agent values (e.g.
# BUN_JSC_forceRAMSize) that can't live in the static store asset.
# Merge base env (always) with OTEL env (when enabled).
allEnv =
baseClaudeEnv
// lib.optionalAttrs otelCfg.enable otelSettingsEnv

13
nix/agent-modules/default.nix
View file

@ -41,6 +41,11 @@ let
which the hive writes into every container, so no agent build sets it.
Remove the setting.
'';
memoryMaxBytesRemoved = ''
An agent reads its memory cap at boot from /run/hive/memory-max-bytes,
which the hive writes before every start, so no agent build sets it.
Remove the setting.
'';
in
{
imports = [
@ -60,6 +65,7 @@ in
./logs.nix
./matrix.nix
./mcp.nix
./memory-cap.nix
./network.nix
./otel.nix
./packages.nix
@ -94,6 +100,13 @@ in
# Both spellings, for the reason given above.
(lib.mkRemovedOptionModule [ "hyperhive" "hiveName" ] hiveNameRemoved)
(lib.mkRemovedOptionModule [ "services" "hyperhive" "agent" "hiveName" ] hiveNameRemoved)
(lib.mkRemovedOptionModule [ "hyperhive" "claudeMemoryMaxBytes" ] memoryMaxBytesRemoved)
(lib.mkRemovedOptionModule [
"services"
"hyperhive"
"agent"
"claudeMemoryMaxBytes"
] memoryMaxBytesRemoved)
];
options.services.hyperhive.agent.icon = lib.mkOption {

24
nix/agent-modules/mcp.nix
View file

@ -11,25 +11,6 @@
}:
let
userName = config.services.hyperhive.agent.user.name;
# This container's own effective `MemoryMax=` in bytes, baked in per
# agent by meta.rs's flake render — see
# `services.hyperhive.agent.claudeMemoryMaxBytes` in ./claude-settings.nix. `null`
# when the cap is `infinity` or a RAM percentage, i.e. when the module
# has no byte count to size anything against.
containerMemoryMaxBytes = config.services.hyperhive.agent.claudeMemoryMaxBytes;
# Two thirds of the container's cap, as the soft ceiling on everything
# the subagent daemon runs. The daemon spawns nested `claude` sessions
# as plain children, so its cgroup already *is* the "all subagents"
# cgroup and a unit-level ceiling bounds the set without a slice.
#
# `MemoryHigh=` and not `MemoryMax=`: this throttles rather than walls.
# Past it the kernel reclaims aggressively and the cgroup stalls, so a
# subagent that overshoots gets visibly slow and the remaining third
# stays available for the agent's own turn — but a subagent that
# genuinely needs more than two thirds still gets it when the container
# has the memory to spare, which is what keeps overprovisioning
# (several agents that rarely compile at the same time) working.
subagentMemoryHigh = containerMemoryMaxBytes * 2 / 3;
# Set on the harness only for an ACP agent on the opencode preset
# (./agent-service.nix).
acpApiKeyEnv = config.systemd.services.hive-agent.environment.HIVE_ACP_API_KEY_ENV or null;
@ -475,12 +456,11 @@ in
# actually chose and leaves the daemon serving, which is also what
# lets it report the kill rather than vanish and be restarted.
OOMPolicy = "continue";
# `MemoryHigh=` is a drop-in from ./memory-cap.nix, sized at boot from
# the container's cap.
User = userName;
Group = userName;
}
// lib.optionalAttrs (containerMemoryMaxBytes != null) {
MemoryHigh = toString subagentMemoryHigh;
}
// lib.optionalAttrs subagentReadsStore {
# Bare id: inherits the credential the container manager passed in,
# the same form ./queue-identity.nix uses.

68
nix/agent-modules/memory-cap.nix Normal file
View file

@ -0,0 +1,68 @@
# The container's memory cap, read at boot rather than built in, so a changed
# cap needs no rebuild.
#
# The cap is `MemoryMax=` on the host's `container@` unit, which nothing in
# here can read: `/sys/fs/cgroup/memory.max` reads `max` at every level from
# inside. hive-c0re writes its byte count to `/run/hive/memory-max-bytes`
# before every start (`lifecycle::host_config`), and removes the file when the
# cap is `infinity` or a RAM percentage. Two values are derived from it, and
# neither is set without the file:
#
# - `BUN_JSC_forceRAMSize`, 75% of the cap. Bun/JavaScriptCore, the runtime
# under claude-code, sizes its heap ceiling off `/proc/meminfo`, which under
# nspawn is the host's RAM, so without it the heap grows well past the real
# wall before GC kicks in hard and choom sessions hang. An environment
# generator hands it to every unit of the container's systemd, `machinectl
# shell` sessions (`hivectl choom`) included; `/etc/set-environment` exports
# it in login shells that no unit starts (`nixos-container root-login`).
# - `MemoryHigh=` on `hive-subagent-daemon`, two thirds of the cap, as a
# generator drop-in. The daemon spawns nested `claude` sessions as plain
# children, so its cgroup already is the "all subagents" cgroup. `High`, not
# `Max`: past it the kernel reclaims hard and the subagents slow down, so the
# last third stays free for the agent's own turn, but a subagent that needs
# more still gets it when the container has memory to spare.
#
# Tested by `checks.script-test-agent-memory-cap`.
{ pkgs, ... }:
let
# Sets `bytes`, or exits when there is no file or its first line is not a
# plain byte count. A leading zero would read as octal, and 16 digits or
# more could overflow `bytes * 75`. Shell builtins only.
readCap = ''
bytes=
[ -r /run/hive/memory-max-bytes ] && read -r bytes < /run/hive/memory-max-bytes
case "$bytes" in
"" | 0* | *[!0-9]* | ????????????????*) exit 0 ;;
esac
'';
envGenerator = pkgs.writeShellScript "hyperhive-memory-cap-env" ''
${readCap}
printf 'BUN_JSC_forceRAMSize=%s\n' "$((bytes * 75 / 100))"
'';
# `$1` is the generator's normal-priority output dir. A drop-in there merges
# with the unit's own file in /etc.
unitGenerator = pkgs.writeShellScript "hyperhive-memory-cap" ''
${readCap}
dir="$1/hive-subagent-daemon.service.d"
${pkgs.coreutils}/bin/mkdir -p "$dir"
printf '[Service]\nMemoryHigh=%s\n' "$((bytes * 2 / 3))" > "$dir/50-memory-high.conf"
'';
# Sourced by every shell's `/etc/set-environment`, so POSIX sh only.
profile = pkgs.writeText "hyperhive-memory-cap.sh" ''
__hyperhive_cap=$(${envGenerator})
case "$__hyperhive_cap" in
BUN_JSC_forceRAMSize=?*) export "$__hyperhive_cap" ;;
esac
unset __hyperhive_cap
'';
in
{
environment.etc."systemd/system-environment-generators/50-hyperhive-memory-cap".source =
envGenerator;
systemd.generators.hyperhive-memory-cap = unitGenerator;
environment.etc."hyperhive/memory-cap.sh".source = profile;
environment.extraInit = ". /etc/hyperhive/memory-cap.sh";
}

4
nix/agent-modules/renamed-options.nix
View file

@ -45,10 +45,6 @@
[ "hyperhive" "claudeMarketplaces" ]
[ "services" "hyperhive" "agent" "claudeMarketplaces" ]
)
(lib.mkRenamedOptionModule
[ "hyperhive" "claudeMemoryMaxBytes" ]
[ "services" "hyperhive" "agent" "claudeMemoryMaxBytes" ]
)
(lib.mkRenamedOptionModule
[ "hyperhive" "claudePlugins" ]
[ "services" "hyperhive" "agent" "claudePlugins" ]

6
nix/checks.nix
View file

@ -138,6 +138,12 @@ in
inherit pkgs self nixosSystem;
inherit (pkgs) lib;
};
# Runs ./agent-modules/memory-cap.nix's generators and login-shell snippet
# against sample cap files.
script-test-agent-memory-cap = import ./script-tests/agent-memory-cap.nix {
inherit pkgs self nixosSystem;
inherit (pkgs) lib;
};
module-eval-agent-github-bao = import ./module-eval/agent-github-bao.nix {
inherit pkgs self nixosSystem;
inherit (pkgs) lib;

42
nix/module-eval/agent-memory.nix
View file

@ -22,39 +22,17 @@ let
agentHarness
;
# The memory-pressure pair. `claudeMemoryMaxBytes` is the container's own
# cap, rendered per agent by meta.rs — the capped arm is the one every
# real deploy gets, the uncapped arm is a hive that set `infinity` or a
# RAM percentage and so hands the module no byte count to size against.
agentCapped = agent { claudeMemoryMaxBytes = 8589934592; };
agentUncapped = agent { };
agentDefault = agent { };
agentBashDaemon = machine: machine.systemd.services.hive-bash-daemon;
agentSubagentDaemon = machine: machine.systemd.services.hive-subagent-daemon;
cases = [
{
# Two thirds of the container's cap, and a SOFT ceiling: the daemon's
# cgroup holds every nested claude, so `MemoryHigh=` throttles the
# subagent set as a whole before the kernel picks a victim, while the
# absent `MemoryMax=` is what still lets one subagent use more than
# its share on a container that has the memory free. A hard cap here
# would trade the silent kill for a guaranteed wall, which is the
# shape this deliberately does not have.
name = "the subagent daemon throttles at two thirds of the container's memory";
ok =
let
c = (agentSubagentDaemon agentCapped).serviceConfig;
in
c.MemoryHigh == "5726623061" && !(c ? MemoryMax);
}
{
# What makes the case above able to fail. With no byte count for the
# container there is no fraction to take, and a hardcoded fallback
# would be a number about some other hive's machine — so the unit
# renders no ceiling at all rather than a fabricated one.
name = "an agent with no byte-valued memory cap renders no subagent ceiling";
ok = !((agentSubagentDaemon agentUncapped).serviceConfig ? MemoryHigh);
# The ceiling is sized at boot from the container's cap
# (../agent-modules/memory-cap.nix). One in the unit would be the cap at
# the last rebuild, and every agent would carry its own hive's number.
name = "the subagent daemon's memory ceiling is not built into its unit";
ok = !((agentSubagentDaemon agentDefault).serviceConfig ? MemoryHigh);
}
{
# The sign is the whole property, and it is easy to write backwards:
@ -68,8 +46,8 @@ let
name = "the OOM killer prefers a subagent over the agent's own session";
ok =
let
sub = (agentSubagentDaemon agentUncapped).serviceConfig.OOMScoreAdjust;
own = (agentHarness agentUncapped).serviceConfig.OOMScoreAdjust;
sub = (agentSubagentDaemon agentDefault).serviceConfig.OOMScoreAdjust;
own = (agentHarness agentDefault).serviceConfig.OOMScoreAdjust;
in
sub > 0 && own < 0 && sub > own;
}
@ -81,13 +59,13 @@ let
# chose, which is the blast radius that made the preference a bad
# trade in the first place.
name = "one subagent losing the OOM draw does not stop the daemon";
ok = (agentSubagentDaemon agentUncapped).serviceConfig.OOMPolicy == "continue";
ok = (agentSubagentDaemon agentDefault).serviceConfig.OOMPolicy == "continue";
}
{
# Each bash task is a process in the daemon's cgroup; under `stop` one
# OOM-killed task would end every running sibling task.
name = "one bash task losing the OOM draw does not stop the daemon";
ok = (agentBashDaemon agentUncapped).serviceConfig.OOMPolicy or null == "continue";
ok = (agentBashDaemon agentDefault).serviceConfig.OOMPolicy or null == "continue";
}
];
in

42
nix/script-tests/agent-memory-cap.nix Normal file
View file

@ -0,0 +1,42 @@
# `checks.script-test-agent-memory-cap` — runs the three readers of
# `/run/hive/memory-max-bytes` in ../agent-modules/memory-cap.nix: the
# environment generator, the unit generator, and the login-shell snippet
# `/etc/set-environment` sources. The cases are in ./agent-memory-cap.sh.
#
# What runs is the rendered scripts. The one edit is the cap file's path, moved
# under the build directory, and the snippet's reference to the environment
# generator, pointed at that edited copy.
{
pkgs,
lib,
self,
nixosSystem,
}:
let
inherit
(import ../module-eval/lib.nix {
inherit
pkgs
lib
self
nixosSystem
;
})
agentWith
;
machine = agentWith { };
in
pkgs.runCommand "hyperhive-script-test-agent-memory-cap"
{
ENV_GENERATOR =
machine.environment.etc."systemd/system-environment-generators/50-hyperhive-memory-cap".source;
UNIT_GENERATOR = machine.systemd.generators.hyperhive-memory-cap;
PROFILE = machine.environment.etc."hyperhive/memory-cap.sh".source;
EXTRA_INIT = machine.environment.extraInit;
SHELLS = "${pkgs.dash}/bin/dash ${pkgs.bash}/bin/sh";
}
''
${pkgs.bash}/bin/bash ${./agent-memory-cap.sh}
touch "$out"
''

94
nix/script-tests/agent-memory-cap.sh Normal file
View file

@ -0,0 +1,94 @@
# Cases for ./agent-memory-cap.nix. Each case writes a cap file (or none), runs
# both generators with a clean environment and compares what they produced,
# then sources the login-shell snippet under each POSIX shell and compares what
# a child process sees.
set -uo pipefail
failures=0
count=0
fail() {
echo "FAILED: $case: $*" >&2
failures=$((failures + 1))
}
dir=$(mktemp -d)
cap=$dir/memory-max-bytes
sed "s|/run/hive/memory-max-bytes|$cap|g" "$ENV_GENERATOR" >"$dir/env-generator"
sed "s|/run/hive/memory-max-bytes|$cap|g" "$UNIT_GENERATOR" >"$dir/unit-generator"
chmod +x "$dir/env-generator" "$dir/unit-generator"
sed "s|$ENV_GENERATOR|$dir/env-generator|g" "$PROFILE" >"$dir/profile"
envbin=$(command -v env)
case="setup"
for g in env-generator unit-generator; do
if grep -q '/run/hive/memory-max-bytes' "$dir/$g"; then fail "$g still names /run/hive after the rewrite"; fi
if ! grep -qF "$cap" "$dir/$g"; then fail "the rewrite did not reach $g's read"; fi
done
if ! grep -qF "$dir/env-generator" "$dir/profile"; then fail "the snippet does not call the rewritten generator"; fi
if ! printf '%s\n' "$EXTRA_INIT" | grep -qxF '. /etc/hyperhive/memory-cap.sh'; then
fail "/etc/set-environment does not source the snippet"
fi
# check <description> <expected BUN_JSC_forceRAMSize or ""> <expected MemoryHigh or "">
check() {
case=$1
count=$((count + 1))
local want_env=${2:+BUN_JSC_forceRAMSize=$2} want_high=$3 got
got=$(env -i "$dir/env-generator")
if [ "$got" != "$want_env" ]; then
fail "environment generator printed:"$'\n'"$got"$'\n'"wanted:"$'\n'"$want_env"
fi
local out=$dir/generator-out
rm -rf "$out"
mkdir -p "$out/normal" "$out/early" "$out/late"
env -i "$dir/unit-generator" "$out/normal" "$out/early" "$out/late"
local dropin=$out/normal/hive-subagent-daemon.service.d/50-memory-high.conf
if [ -z "$want_high" ]; then
got=$(find "$out" -type f)
if [ -n "$got" ]; then fail "unit generator wrote:"$'\n'"$got"; fi
else
got=$(cat "$dropin" 2>&1)
if [ "$got" != "[Service]"$'\n'"MemoryHigh=$want_high" ]; then
fail "unit generator's drop-in:"$'\n'"$got"
fi
fi
local sh
for sh in $SHELLS; do
# Only what the snippet exported reaches the child's environment. No PATH:
# the snippet must not need one.
got=$(env -i "$sh" -c ". '$dir/profile'; exec '$envbin'" | grep -E '^(BUN_|LD_PRELOAD=|__hyperhive)')
if [ "$got" != "$want_env" ]; then
fail "$sh: a child of the sourcing shell saw:"$'\n'"$got"
fi
done
}
printf '8589934592\n' >"$cap"
check "an 8G cap" 6442450944 5726623061
printf '4294967296' >"$cap"
check "a line without a newline is kept" 3221225472 2863311530
printf '4294967296\nLD_PRELOAD=/x\n' >"$cap"
check "only the first line is read" 3221225472 2863311530
printf '8G\n' >"$cap"
check "a size with a suffix is ignored" "" ""
printf '0123\n' >"$cap"
check "a leading zero is ignored" "" ""
printf '1234567890123456\n' >"$cap"
check "sixteen digits are ignored" "" ""
: >"$cap"
check "an empty file" "" ""
rm -f "$cap"
check "no file" "" ""
echo "$count cases, $failures failed"
[ "$failures" -eq 0 ]