vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

check every container's root before its start, through the container@ template

Browse source
This commit is contained in:
atlas 2026-10-28 23:15:03 +01:00
commit c191975aab
1 changed files with 179 additions and 339 deletions
Download patch file Download diff file Expand all files Collapse all files

37
docs/agent-lifecycle/persistence.md
View file

@ -385,35 +385,32 @@ never data.
**Checked before every start.** nixpkgs' start script runs as host root on
the root template, and its `mkdir -p`, `chmod`, `touch` and
`cp --remove-destination` follow a link the container planted at the paths
they touch. The container's limits drop-in
(`container@h-<name>.service.d/hyperhive-limits.conf`) therefore carries an
`ExecStartPre=` that runs `hive-priv check-rootfs h-<name>` ahead of that
script, on systemd's own restarts too. It looks at each path with `lstat`,
never following a link, and fails the start with one line on stderr naming the
path and its type when:
they touch. The host's nix config therefore adds a drop-in to the `container@`
template (`container@.service.d/`, from
`nix/host-modules/container-rootfs-check.nix`) with an `ExecStartPre=` that
runs `hive-priv check-rootfs %i`. It applies to every container instance, the
agents and the swarm's service containers alike, and runs on systemd's own
restarts too. systemd reads a drop-in after the unit file and runs the
`ExecStartPre=` lines in that order, all of them before `ExecStart=`, so the
check runs before the start script touches the root. The unit file names the
`hive-priv` store path, so the system closure keeps that build alive.
The check looks at each path with `lstat`, never following a link, and fails
the start with one line on stderr naming the path and its type when:
- `usr`, `usr/bin`, `etc`, `var`, `var/lib`, `var/lib/private` or `root`
exists and isn't a real directory;
- `etc/machine-id` or `etc/resolv.conf` exists and isn't a regular file
(`etc/os-release` may also be a link, which the script skips and a booted
NixOS guest creates);
- the profile's `system` (`/nix/var/nix/profiles/per-container/h-<name>/system`)
- the profile's `system` (`/nix/var/nix/profiles/per-container/<name>/system`)
exists and isn't a link that resolves into `/nix/store`, or the `system`
file inside the resolved directory isn't a regular file.
Absent paths pass, so a first start works. The check neither deletes nor
repairs: the operator removes the offending entry, and the next start goes
through.
The drop-in names the `hive-priv` binary that wrote it, so a drop-in never
points at a build without the sub-command. A drop-in written by an older
points at a build without the sub-command. That build's store path has a GC
root, `/nix/var/nix/gcroots/hyperhive-check-rootfs/h-<name>`, written before the
drop-in and replaced on each write, so a garbage collection can't remove the
binary a restart would run. The directory sits beside `gcroots/per-container`,
which is the only gcroots path bound into a container, so only root writes it.
Destroying the container removes the drop-in and then its root; the build stays
until the next collection. A drop-in written by an older hive-priv has no
`ExecStartPre=` and no root, and the container starts as before.
The name `%i` selects both paths, so the check refuses a name that isn't
lowercase ascii, digits and hyphens. Absent paths pass, so a first start works.
The check neither deletes nor repairs: the operator removes the offending
entry, and the next start goes through.
**Disk use.** The copy shares blocks with the old root where it can: hive-priv
clones each file (`FICLONE`), which on btrfs or xfs allocates metadata only, and