vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

feat: per-agent CPU and memory limits

Browse source 
The hive applies one `agentCpuQuota` / `agentMemoryMax` to every
container. That's the right default and the wrong ceiling: a build-heavy
agent needs headroom the other twelve don't, and raising the hive-wide
value to suit it hands that headroom to everyone.

Adds a per-agent override, persisted host-side and resolved per-field
against the hive defaults.

Follows the existing `meta/*.json` pattern (`capabilities.json`,
`tool-groups.json`): a host-side map read by `hive-c0re`, staged and
committed in the meta repo so every change lands in the audit trail.

```json
{ "sock": { "cpu_quota": "400%", "memory_max": "8G" } }
```

Fallback is **per field**, not per agent: an entry with only
`memory_max` leaves that agent on the hive-wide CPU quota. Absent file,
absent agent and absent field all resolve to the hive default, so the
feature is inert until someone opts an agent in.

Unlike the other meta files this one is **not** injected into the
container — a limit is something done *to* an agent, not something it
reads about itself.

```
hivectl agents set-limits sock --cpu-quota 400% --memory-max 8G
hivectl agents set-limits sock --reset
```

Values are validated before they're persisted: they go into a systemd
drop-in verbatim, and a typo there makes the unit fail to *start* —
turning a fat-fingered quota into a container that won't come back.

The command is declarative: each call replaces the agent's whole entry.
That makes a forgotten flag a silent revert, so a bare `set-limits
<name>` is rejected at the clap layer and clearing needs an explicit
`--reset`.

`ContainerView` gains `cpu_quota` / `memory_max`, both always populated:
there's no "unset" state to render, only "same as everyone else". They
reflect what the drop-in *says* — what the next start will enforce — not
a live cgroup reading.

The write goes through `meta::commit_resource_limits` rather than the
bare setter, so it's staged and committed under `META_LOCK`. Writing
without committing would leave the meta working tree dirty for the next
`prepare_deploy` to trip over.

Docs: `persistence.md` (the new meta file, and why it isn't injected),
`tools/hivectl.md` (the prose guide), `tools/hivectl-cli.md`
(regenerated clap dump).

Closes: internal/requests issue 25
This commit is contained in:
atlas 2026-07-26 14:12:58 +02:00
commit a6dc980700
15 changed files with 594 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

72
hive-c0re/src/server.rs
View file

@ -162,7 +162,7 @@ async fn dispatch(req: &HostRequest, coord: Arc<Coordinator>) -> HostResponse {
HostResponse::dags(dags)
}
HostRequest::List => HostResponse::list(lifecycle::list().await?),
HostRequest::AgentStatus => handle_agent_status().await,
HostRequest::AgentStatus => handle_agent_status(&coord).await,
// The hive domain + per-surface public URLs are injected into
// c0re's service env by hive-c0re.nix; surface them so the
// operator CLI can fill in this hive's own identity (the
@ -194,6 +194,19 @@ async fn dispatch(req: &HostRequest, coord: Arc<Coordinator>) -> HostResponse {
.map_err(anyhow::Error::msg)?;
HostResponse::success()
}
HostRequest::SetResourceLimits {
name,
cpu_quota,
memory_max,
} => {
handle_set_resource_limits(
&coord,
name,
cpu_quota.as_deref(),
memory_max.as_deref(),
)
.await?
}
HostRequest::MatrixCreateUser { name, password } => {
handle_matrix_create_user(name, password.as_deref()).await?
}
@ -324,8 +337,8 @@ async fn handle_set_paused(
}
/// Collect per-agent status rows for `hivectl status` and the dashboard.
async fn handle_agent_status() -> HostResponse {
let rows = crate::container_view::build_all()
async fn handle_agent_status(coord: &Arc<Coordinator>) -> HostResponse {
let rows = crate::container_view::build_all(&coord.hive_env())
.await
.into_iter()
.map(|v| hive_sh4re::AgentStatusRow {
@ -376,6 +389,59 @@ fn agent_exists(name: &hive_types::Ident) -> Result<bool> {
.with_context(|| format!("check agent state dir for {name}"))
}
/// Validate + persist an agent's CPU/memory overrides, then re-apply the
/// drop-in so the change lands without waiting for a rebuild.
///
/// Validation is here rather than only in `hivectl` because the values
/// are written verbatim into the systemd drop-in: a malformed
/// `CPUQuota=` makes systemd reject the unit, and the container stops
/// starting. Every client (CLI, dashboard, anything later) goes through
/// this path, so the guard belongs on this side of the socket.
///
/// `None`/`None` removes the agent's entry, returning it to the
/// hive-wide defaults.
async fn handle_set_resource_limits(
coord: &Arc<Coordinator>,
name: &hive_types::Ident,
cpu_quota: Option<&str>,
memory_max: Option<&str>,
) -> Result<HostResponse> {
if let Some(value) = cpu_quota {
crate::resource_limits::validate_cpu_quota(value).map_err(anyhow::Error::msg)?;
}
if let Some(value) = memory_max {
crate::resource_limits::validate_memory_max(value).map_err(anyhow::Error::msg)?;
}
tracing::info!(%name, ?cpu_quota, ?memory_max, "set_resource_limits");
let limits = crate::resource_limits::AgentLimits {
cpu_quota: cpu_quota.map(ToOwned::to_owned),
memory_max: memory_max.map(ToOwned::to_owned),
};
// Goes through `meta::commit_resource_limits`, not the bare
// `resource_limits::set_limits`: the write has to be staged +
// committed under `META_LOCK` or it leaves the meta working tree
// dirty for the next `prepare_deploy` / `sync_agents` to trip over.
crate::meta::commit_resource_limits(name.as_str(), &limits).await?;
// Re-apply the drop-in straight away — same three lines as the job
// queue's `WriteDropin` node. Without this the new values would sit
// in the JSON until the agent's next spawn or rebuild.
let agent_dir = crate::paths::agent_runtime_dir(name.as_str());
let hive = coord.hive_env();
let paths = Coordinator::agent_paths(name.as_str(), agent_dir);
crate::lifecycle::write_dropins(name.as_str(), &hive, &paths).await?;
let (cpu, mem) = crate::resource_limits::effective(
name.as_str(),
&hive.agent_cpu_quota,
&hive.agent_memory_max,
);
Ok(HostResponse::messages(vec![format!(
"{name}: CPUQuota={cpu} MemoryMax={mem} (restart the container if it is running \
and the new caps need to take effect immediately)"
)]))
}
/// Guard: matrix provisioning needs the homeserver container running.
async fn require_matrix_present() -> Result<()> {
if crate::matrix::is_present().await {