docs(#2754): document the container weights in coordinator/security/persistence

The PR added CPUWeight=/IOWeight= to the drop-in but left the prose
docs describing a two-setting file. Covers the cap-vs-share
distinction, the hive-wide-only scope (no resource-limits.json
override), and the iocost/BFQ caveat that makes IOWeight= inert on
most hosts.
This commit is contained in:
atlas 2026-07-27 11:01:36 +02:00
commit f28a1e33d3
3 changed files with 30 additions and 2 deletions

View file

@ -423,10 +423,36 @@ regardless.
rebuild, so changes take effect on the next lifecycle op without rebuild, so changes take effect on the next lifecycle op without
requiring a host rebuild. requiring a host rebuild.
The same drop-in carries `CPUWeight=` / `IOWeight=` from
`agentCpuWeight` / `agentIoWeight`. Those are a different kind of
setting: the quota and the memory max are **hard caps** that throttle
an agent even on a completely idle host, while the weights are cgroup
v2 **relative shares** that only decide who yields *under contention*.
A low-weight container still gets the whole machine when nothing else
wants it.
| Option | Default | Description | | Option | Default | Description |
| ---------------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------ | | ---------------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| `services.hyperhive.c0re.agentCpuQuota` | `"200%"` | CPU cap per agent, as a percentage of one core (`"200%"` = 2 cores). Raise if agents hit CPU limits during builds or heavy tool use. | | `services.hyperhive.c0re.agentCpuQuota` | `"200%"` | CPU cap per agent, as a percentage of one core (`"200%"` = 2 cores). Raise if agents hit CPU limits during builds or heavy tool use. |
| `services.hyperhive.c0re.agentMemoryMax` | `"4G"` | Memory cap per agent. Raise for agents that run large nix builds or hold big in-memory data. | | `services.hyperhive.c0re.agentMemoryMax` | `"4G"` | Memory cap per agent. Raise for agents that run large nix builds or hold big in-memory data. |
| `services.hyperhive.c0re.agentCpuWeight` | `80` | `cpu.weight` share per agent, `1``10000` or `null` to omit the setting. Kernel default is `100`, so `80` makes agents yield. |
| `services.hyperhive.c0re.agentIoWeight` | `80` | `io.weight` share per agent, same range and `null` handling. See the caveat below — it is a no-op on many hosts. |
Two things to know about the weights:
- They are **hive-wide** — unlike the caps there is no per-agent
override in `meta/resource-limits.json`, so every agent carries the
same value and the weight does *not* rank agents against each other.
What `80` buys is that agents yield to everything **not** on this
drop-in path: host services and the infra containers (`hive-ci`,
`hive-forge`, `hive-gateway`, `hive-matrix`), which stay at the
kernel default of `100`.
- `IOWeight=` is only honoured when the backing device runs the BFQ
scheduler or has blk-iocost QoS enabled. On a host using
`none`/`mq-deadline`/`kyber` without iocost, systemd writes the value
and the kernel ignores it — harmless, but it will measure as nothing.
Check with `cat /sys/fs/cgroup/io.cost.qos`, and set the option to
`null` if you would rather not write a setting nothing reads.
For a hive-wide cap across all containers together, set For a hive-wide cap across all containers together, set
`systemd.slices.machine.serviceConfig.CPUQuota` in your NixOS `systemd.slices.machine.serviceConfig.CPUQuota` in your NixOS

View file

@ -318,7 +318,9 @@ Contents:
an absent file, absent agent, or absent field falls back to the an absent file, absent agent, or absent field falls back to the
hive-wide `services.hyperhive.agentCpuQuota` / `agentMemoryMax`, hive-wide `services.hyperhive.agentCpuQuota` / `agentMemoryMax`,
so an agent can override only its memory and still track the hive so an agent can override only its memory and still track the hive
default for CPU. default for CPU. The `CPUWeight=` / `IOWeight=` shares in the same
drop-in have **no** per-agent override — they are hive-wide only and
come straight off `HiveEnv`, so this file has no field for them.
The root agent has the meta dir RO-mounted at `/meta/`. The root agent has the meta dir RO-mounted at `/meta/`.

View file

@ -209,7 +209,7 @@ known operations; there is no arbitrary command pass-through:
| `ReadContainerJournal` | `journalctl -M <container> -n <n> [filters...]` | | `ReadContainerJournal` | `journalctl -M <container> -n <n> [filters...]` |
| `ReloadGatewayNginx` | `systemctl -M hive-gateway reload/start/reset-failed nginx` | | `ReloadGatewayNginx` | `systemctl -M hive-gateway reload/start/reset-failed nginx` |
| `WriteNspawnFlags` | write `/etc/nixos-containers/<container>.conf` (bind-mount list + network isolation vars) | | `WriteNspawnFlags` | write `/etc/nixos-containers/<container>.conf` (bind-mount list + network isolation vars) |
| `WriteResourceLimits` | write `CPUQuota=`/`MemoryMax=` systemd drop-in for agent container | | `WriteResourceLimits` | write `CPUQuota=`/`MemoryMax=`/`CPUWeight=`/`IOWeight=` systemd drop-in for agent container |
| `RemoveServiceDropin` | remove `container@<name>.service.d/` drop-in on destroy | | `RemoveServiceDropin` | remove `container@<name>.service.d/` drop-in on destroy |
| `DaemonReload` | `systemctl daemon-reload` | | `DaemonReload` | `systemctl daemon-reload` |
| `ChownSocketDir` / `ChmodSocketDir` | chown/chmod `/run/hive-agent/<name>/` socket directory | | `ChownSocketDir` / `ChmodSocketDir` | chown/chmod `/run/hive-agent/<name>/` socket directory |