eaca3c4eb3
The cap file was removed for a cap of `infinity` or a RAM percentage, so the boot check warned on every start of such an agent, for a legitimate config. The file is now written on every start: - a size: its byte count, as before; - `infinity`: the literal `infinity`. The generators set no JSC heap ceiling and no subagent MemoryHigh=, and the boot check is quiet; - a percentage: resolved on the host the way systemd resolves MemoryMax=N% for the container@ unit. That is permyriad (at most two decimals, at most 100%) of sysconf(_SC_PHYS_PAGES) * sysconf(_SC_PAGESIZE), rounded down to a page; - a value systemd ignores (zero, over 100%, more than two decimals, unparseable) leaves the unit uncapped, so it is `infinity`. The file is missing only when hive-c0re can't tell the cap: the limits file and the kept drop-in are both unreadable, or a percentage meets an unknown host memory size. The boot check now warns only for a missing file or a first line that is neither a byte count nor `infinity`. Part of #5081 (slice S0-2).
98 lines
4.4 KiB
Nix
98 lines
4.4 KiB
Nix
# The container's memory cap, read at boot rather than built in, so a changed
|
|
# cap needs no rebuild.
|
|
#
|
|
# The cap is `MemoryMax=` on the host's `container@` unit, which nothing in
|
|
# here can read: `/sys/fs/cgroup/memory.max` reads `max` at every level from
|
|
# inside. Before every start hive-c0re (`lifecycle::host_config`) writes it to
|
|
# `/run/hive/memory-max-bytes` as a byte count, a percentage resolved against
|
|
# the host's RAM as systemd resolves it, or `infinity` for no cap. Two values
|
|
# are derived from a byte count, and neither is set for `infinity`. A missing
|
|
# or malformed file sets neither too, and `hyperhive-memory-cap-check` logs a
|
|
# warning at boot:
|
|
#
|
|
# - `BUN_JSC_forceRAMSize`, 75% of the cap. Bun/JavaScriptCore, the runtime
|
|
# under claude-code, sizes its heap ceiling off `/proc/meminfo`, which under
|
|
# nspawn is the host's RAM, so without it the heap grows well past the real
|
|
# wall before GC kicks in hard and choom sessions hang. An environment
|
|
# generator hands it to every unit of the container's systemd, `machinectl
|
|
# shell` sessions (`hivectl choom`) included; `/etc/set-environment` exports
|
|
# it in login shells that no unit starts (`nixos-container root-login`).
|
|
# `hive-agent` and `hive-subagent-daemon` spawn `claude` as a plain child,
|
|
# which inherits it, so neither unit may set or unset it.
|
|
# - `MemoryHigh=` on `hive-subagent-daemon`, two thirds of the cap, as a
|
|
# generator drop-in. The daemon spawns nested `claude` sessions as plain
|
|
# children, so its cgroup already is the "all subagents" cgroup. `High`, not
|
|
# `Max`: past it the kernel reclaims hard and the subagents slow down, so the
|
|
# last third stays free for the agent's own turn, but a subagent that needs
|
|
# more still gets it when the container has memory to spare.
|
|
#
|
|
# Tested by `checks.script-test-agent-memory-cap` and
|
|
# `checks.module-eval-agent-memory`.
|
|
{ pkgs, ... }:
|
|
let
|
|
# Sets `bytes`. Exits for `infinity`, and runs `noCap` when there is no file
|
|
# or its first line is neither `infinity` nor a plain byte count. A leading
|
|
# zero would read as octal, and 16 digits or more could overflow
|
|
# `bytes * 75`. Shell builtins only.
|
|
readCap = noCap: ''
|
|
bytes=
|
|
[ -r /run/hive/memory-max-bytes ] && read -r bytes < /run/hive/memory-max-bytes
|
|
case "$bytes" in
|
|
infinity) exit 0 ;;
|
|
"" | 0* | *[!0-9]* | ????????????????*) ${noCap} ;;
|
|
esac
|
|
'';
|
|
|
|
envGenerator = pkgs.writeShellScript "hyperhive-memory-cap-env" ''
|
|
${readCap "exit 0"}
|
|
printf 'BUN_JSC_forceRAMSize=%s\n' "$((bytes * 75 / 100))"
|
|
'';
|
|
|
|
# `$1` is the generator's normal-priority output dir. A drop-in there merges
|
|
# with the unit's own file in /etc.
|
|
unitGenerator = pkgs.writeShellScript "hyperhive-memory-cap" ''
|
|
${readCap "exit 0"}
|
|
dir="$1/hive-subagent-daemon.service.d"
|
|
${pkgs.coreutils}/bin/mkdir -p "$dir"
|
|
printf '[Service]\nMemoryHigh=%s\n' "$((bytes * 2 / 3))" > "$dir/50-memory-high.conf"
|
|
'';
|
|
|
|
# The generators run before the container's journal starts and an nspawn
|
|
# container has no /dev/kmsg, so the boot warning for a missing or malformed
|
|
# cap file comes from a unit. `<4>` is the journal's warning priority.
|
|
check = pkgs.writeShellScript "hyperhive-memory-cap-check" ''
|
|
${readCap ''
|
|
if [ -e /run/hive/memory-max-bytes ]; then
|
|
why="holds neither a byte count nor infinity"
|
|
else
|
|
why="is missing"
|
|
fi
|
|
echo "<4>/run/hive/memory-max-bytes $why: no BUN_JSC_forceRAMSize heap ceiling and no MemoryHigh= on hive-subagent-daemon; the host's MemoryMax= on the container still applies." >&2
|
|
exit 0
|
|
''}
|
|
'';
|
|
|
|
# Sourced by every shell's `/etc/set-environment`, so POSIX sh only.
|
|
profile = pkgs.writeText "hyperhive-memory-cap.sh" ''
|
|
__hyperhive_cap=$(${envGenerator})
|
|
case "$__hyperhive_cap" in
|
|
BUN_JSC_forceRAMSize=?*) export "$__hyperhive_cap" ;;
|
|
esac
|
|
unset __hyperhive_cap
|
|
'';
|
|
in
|
|
{
|
|
environment.etc."systemd/system-environment-generators/50-hyperhive-memory-cap".source =
|
|
envGenerator;
|
|
systemd.generators.hyperhive-memory-cap = unitGenerator;
|
|
systemd.services.hyperhive-memory-cap-check = {
|
|
description = "Warn when the container's memory cap is unusable";
|
|
wantedBy = [ "multi-user.target" ];
|
|
serviceConfig = {
|
|
Type = "oneshot";
|
|
ExecStart = check;
|
|
};
|
|
};
|
|
environment.etc."hyperhive/memory-cap.sh".source = profile;
|
|
environment.extraInit = ". /etc/hyperhive/memory-cap.sh";
|
|
}
|