hive-priv: create agent socket dirs on start; drop hyperhive-agents.conf
/etc/tmpfiles.d/hyperhive-agents.conf was a boot-time backstop (#2290) that pre-created every agent's bind sources. The start preamble already creates them for every c0re-driven start, and on this host only hive-c0re starts agent containers. The file was also the reason the socket dir's owner had to be declared there, which is how it spent its life at `0777 root root` whenever the uid could not be resolved (#4742). - hive-priv gains `EnsureAgentSocketDir { name }`, called from `set_nspawn_flags` in every start path. It creates `/run/hive-agent/<name>` `0751 root:root` with mkdirat relative to an O_DIRECTORY|O_NOFOLLOW fd for the parent. An existing entry has to be a directory (fstatat AT_SYMLINK_NOFOLLOW); anything else is refused, and a directory is left alone. hive-c0re's own create_dir_all went: its /run is read-only under ProtectSystem=strict. - The container's `hive-agent-user-migrate` activation chowns that dir to the agent user and sets 0751, the same way it already handles state/ and harness/. It refuses a symlink or non-directory there, since `test -d` and chmod follow links. No host-side passwd parse, and no window where the dir is world-writable. - `/run/hyperhive/agents/<name>` stays created by hive-c0re itself (`ensure_agent_runtime_dir`). It holds the `mcp.sock` that hive-c0re binds as hive-core, so it must not become root- or agent-owned. - The `/run/hive-agent` parent is declared in hive-priv.nix, `0755 root:root`, instead of hive-gateway's hive-core rule. hive-priv is its only writer now, and hive-priv's ReadWritePaths needs it to exist. - The manager start in `ensure_root_agent` now goes through `converge_start_preamble` + `start_with_fallback`. It was a bare start, so after a reboot the manager's bind sources existed only because of the tmpfiles file, and its limits drop-in did not exist at all. - Removed: `sync_tmpfiles`, `agent_uid_gid` / `parse_passwd_uid_gid`, `priv_client::sync_agent_tmpfiles`, `AgentTmpfilesEntry`, the tmpfiles body builder and their tests, plus the three call sites. - Legacy: hive-priv unlinks the file at every start, ignoring ENOENT. `SyncAgentTmpfiles` stays one release as a payload-ignoring variant that does the same unlink and returns Ok, for an older hive-c0re. Salvaged from #4752: the boundary.md correction that nginx only dials, because ProtectSystem=strict makes its /run read-only. Behaviour change: a manual `nixos-container start h-<name>` right after a reboot, before hive-c0re has started that agent, now fails on a missing bind source instead of starting. Closes #4742
This commit is contained in:
parent
e7456a49ff
commit
2252c55df8
21 changed files with 369 additions and 421 deletions
|
|
@ -174,9 +174,10 @@ pub const META_DIR: &str = "/var/lib/hyperhive/meta";
|
|||
/// boundary prevents importing across the crate.
|
||||
pub const AGENT_STATE_ROOT: &str = "/var/lib/hyperhive/agents";
|
||||
|
||||
/// Root of per-agent runtime directories on the host (regenerated each boot
|
||||
/// by `hive-priv` tmpfiles.d; not persistent). Used by `hive-priv` when
|
||||
/// creating per-agent subdirs via `nsenter` / tmpfiles.
|
||||
/// Root of per-agent runtime directories on the host (tmpfs, not
|
||||
/// persistent). hive-c0re creates each `<name>` subdir itself in the start
|
||||
/// preamble; `hive-priv` only names it in the limits drop-in's
|
||||
/// `ConditionPathIsDirectory=`.
|
||||
/// Must stay in sync with `hive-c0re::paths::agent_runtime_root()`
|
||||
/// (`RUNTIME_ROOT + "/agents"`); the privsep boundary prevents importing
|
||||
/// across the crate.
|
||||
|
|
@ -382,6 +383,16 @@ pub enum PrivRequest {
|
|||
load_credentials: Vec<CredentialMount>,
|
||||
},
|
||||
|
||||
/// Create the agent's socket dir `/run/hive-agent/<name>` as `0751
|
||||
/// root:root` if it is missing; an existing directory is left as it is.
|
||||
/// It is a bind source, so it has to exist before every container start,
|
||||
/// and `/run` is a tmpfs. The container's own activation hands it to the
|
||||
/// agent user. Called from `lifecycle::set_nspawn_flags`.
|
||||
EnsureAgentSocketDir {
|
||||
/// Logical agent name (validated by `validate_agent_name`).
|
||||
name: String,
|
||||
},
|
||||
|
||||
/// Write `/run/systemd/system/container@<container>.service.d/hyperhive-limits.conf`
|
||||
/// with `[Service]` carrying `MemoryMax=` / `CPUQuota=` (hard caps) and
|
||||
/// `CPUWeight=` / `IOWeight=` (cgroup v2 relative shares, contention-only).
|
||||
|
|
@ -773,42 +784,11 @@ pub enum PrivRequest {
|
|||
parent_snapshot_name: Option<String>,
|
||||
},
|
||||
|
||||
/// Write `/etc/tmpfiles.d/hyperhive-agents.conf` for the given agent set
|
||||
/// and immediately apply it with `systemd-tmpfiles --create`. Each entry
|
||||
/// declares the per-agent runtime dirs (`/run/hyperhive/agents/<name>` and
|
||||
/// `/run/hive-agent/<name>`) so systemd recreates them at every boot before
|
||||
/// any container units start — preventing bind-mount source missing errors
|
||||
/// when container@h-* units race hive-c0re after a reboot.
|
||||
///
|
||||
/// Called at hive-c0re startup and after every agent spawn / destroy.
|
||||
/// Agents are logical names (validated by `validate_agent_name`).
|
||||
SyncAgentTmpfiles {
|
||||
/// One entry per live agent. hive-priv validates each name before
|
||||
/// writing any path component derived from it.
|
||||
agents: Vec<AgentTmpfilesEntry>,
|
||||
},
|
||||
}
|
||||
|
||||
/// One agent's runtime-dir declaration for `SyncAgentTmpfiles`.
|
||||
///
|
||||
/// Carries the container uid/gid so the tmpfiles entry can *declare* who owns
|
||||
/// `/run/hive-agent/<name>` instead of having it corrected afterwards by a
|
||||
/// privileged chown. The two mechanisms used to fight: the tmpfiles line wrote
|
||||
/// `0777 root root` and a follow-up `ChownSocketDir` narrowed it, but any
|
||||
/// later spawn or destroy re-applied the file and reset *every* agent's dir
|
||||
/// back to world-writable.
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct AgentTmpfilesEntry {
|
||||
/// Logical agent name (e.g. `"atlas"`, `"ruth"`).
|
||||
pub name: String,
|
||||
/// Container uid/gid of the agent user, when known.
|
||||
///
|
||||
/// `None` only before the container's `/etc/passwd` has been rendered
|
||||
/// (first boot). The dir must stay writable by the not-yet-identifiable
|
||||
/// harness in that window, so hive-priv falls back to the historical
|
||||
/// permissive mode for that one agent; the next sync tightens it.
|
||||
pub uid: Option<u32>,
|
||||
pub gid: Option<u32>,
|
||||
/// Legacy: an older hive-c0re still sends this. hive-priv only unlinks
|
||||
/// `/etc/tmpfiles.d/hyperhive-agents.conf` and returns `Ok`; the `agents`
|
||||
/// payload that request carries is ignored on decode. Remove once no
|
||||
/// deployed hive-c0re sends it, one release after `EnsureAgentSocketDir`.
|
||||
SyncAgentTmpfiles,
|
||||
}
|
||||
|
||||
/// Response from the privileged helper.
|
||||
|
|
|
|||
Loading…
Reference in a new issue