vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

hive-c0re: hivectl subvol upgrade — migrate an agent state dir to a btrfs subvolume

Browse source 
New agents get a btrfs subvolume state root automatically when the host
FS is btrfs, but agents that predate that migration are left on plain
dirs and miss the subvolume feature set (snapshots, per-subvol
usage/quota, send/receive migration). Add an opt-in operator verb to
convert an existing plain-dir agent in place.

btrfs cannot promote a directory to a subvolume in place, so the new
privileged op stages a sibling subvolume mirroring the dir (create +
`cp -a --reflink=auto` preserving ownership/permissions/xattrs + match
the root's owner and mode), then atomically renames the original aside
and the subvolume into place, then removes the original. Any failure
before the swap leaves the original untouched; idempotent (no-op if
already a subvolume) and btrfs-gated.

The `hivectl subvol upgrade <agent> --yes` verb composes it client-side
like `restart`: stop the agent so its state bind-mount is released, run
the migration via hive-priv, then restart it — the restart is attempted
regardless of the migration outcome so a failed migration never leaves
the agent down.

- hive-sh4re: UpgradeAgentSubvolume priv request variant.
- hive-priv: the migration handler plus stage/cleanup helpers.
- hive-c0re: priv_client wrapper and the hivectl verb; regen CLI docs.
This commit is contained in:
atlas 2026-06-21 14:12:43 +02:00 • committed by mara
commit 6b1dbebe5a
5 changed files with 351 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

171
hive-priv/src/main.rs
View file

@ -297,6 +297,11 @@ async fn exec(req: PrivRequest, writer: &mut OwnedWriteHalf) -> Result<(String,
validate_agent_name(agent_name)?;
set_subvolume_quota(agent_name, limit_bytes).await
}
PrivRequest::UpgradeAgentSubvolume { ref agent_name } => {
validate_agent_name(agent_name)?;
upgrade_agent_subvolume(agent_name).await
}
}
}
@ -700,6 +705,172 @@ async fn read_subvolume_usage(agent_name: &str) -> Result<(String, String)> {
))
}
/// Best-effort removal of a leftover migration path from a prior aborted
/// upgrade: try `btrfs subvolume delete` (in case it's a half-created
/// subvolume) then a plain recursive remove. Both failures are ignored —
/// the path may simply not exist.
async fn cleanup_stale_path(path: &Path) {
if path.exists() {
let _ = Command::new("btrfs")
.args(["subvolume", "delete"])
.arg(path)
.output()
.await;
let _ = std::fs::remove_dir_all(path);
}
}
/// Stage a populated subvolume at `tmp` mirroring `agent_root`: create the
/// subvolume, copy `agent_root`'s contents into it preserving
/// ownership/permissions/xattrs, then match the subvolume root's owner + mode
/// to the original. On any failure the partially-staged `tmp` is cleaned up
/// (so the caller can bail with the original dir still untouched).
async fn stage_upgrade_subvolume(agent_root: &Path, tmp: &Path) -> Result<()> {
use std::os::unix::fs::{MetadataExt as _, PermissionsExt as _};
// Fresh subvolume to receive the copy.
let out = Command::new("btrfs")
.args(["subvolume", "create"])
.arg(tmp)
.output()
.await
.with_context(|| format!("spawn btrfs subvolume create {}", tmp.display()))?;
if !out.status.success() {
bail!(
"btrfs subvolume create {} failed: {}",
tmp.display(),
String::from_utf8_lossy(&out.stderr).trim()
);
}
// Copy contents preserving everything (`-a` = --preserve=all → mode,
// ownership, timestamps, links, xattrs); reflink for fast CoW clones on the
// same btrfs. `<src>/.` copies the directory's contents (incl. dotfiles)
// into the subvolume rather than nesting it.
let copy = Command::new("cp")
.arg("-a")
.arg("--reflink=auto")
.arg(format!("{}/.", agent_root.display()))
.arg(tmp)
.output()
.await
.with_context(|| format!("spawn cp into {}", tmp.display()))?;
if !copy.status.success() {
cleanup_stale_path(tmp).await;
bail!(
"copy {} -> {} failed (original left untouched): {}",
agent_root.display(),
tmp.display(),
String::from_utf8_lossy(&copy.stderr).trim()
);
}
// Match the new subvolume root's ownership + mode to the original dir.
// `cp -a <src>/.` copies the *contents* but the subvolume root keeps its
// create-time root ownership, so set it explicitly — the swapped-in
// subvolume must be indistinguishable from the original to hive-c0re.
let apply = std::fs::metadata(agent_root)
.with_context(|| format!("stat {} for ownership", agent_root.display()))
.and_then(|m| {
std::os::unix::fs::chown(tmp, Some(m.uid()), Some(m.gid()))
.with_context(|| format!("chown {} to match original", tmp.display()))?;
std::fs::set_permissions(tmp, std::fs::Permissions::from_mode(m.mode()))
.with_context(|| format!("chmod {} to match original", tmp.display()))?;
Ok(())
});
if let Err(e) = apply {
cleanup_stale_path(tmp).await;
return Err(e.context("upgrade aborted before swap; original left untouched"));
}
Ok(())
}
/// `UpgradeAgentSubvolume` — convert an existing plain-dir agent state root
/// into a btrfs subvolume in place. Operator opt-in; the caller (hivectl)
/// stops the agent first and restarts it after. See the wire doc.
///
/// Migration: stage a sibling subvolume mirroring the dir
/// ([`stage_upgrade_subvolume`]), then rename the original aside and the
/// subvolume into place, then remove the original. Any failure before the
/// rename-swap leaves the original dir untouched.
async fn upgrade_agent_subvolume(agent_name: &str) -> Result<(String, String)> {
let root = PathBuf::from(AGENT_STATE_ROOT);
let agent_root = root.join(agent_name);
if !agent_root.exists() {
bail!(
"no state dir to upgrade at {} — nothing to do",
agent_root.display()
);
}
// Idempotent: already a subvolume → nothing to do.
if is_btrfs_subvolume(&agent_root) {
return Ok((String::new(), String::new()));
}
if !is_on_btrfs(&root)? {
bail!(
"{} is not on btrfs — subvolumes are unsupported, cannot upgrade",
root.display()
);
}
// Sibling temp paths on the same filesystem (so the copy can reflink and
// the swap renames are atomic). Leading dots keep them out of the agent
// namespace (`validate_agent_name` rejects dot-prefixed names).
let tmp = root.join(format!(".{agent_name}.migrating"));
let old = root.join(format!(".{agent_name}.old"));
// Clear any debris from a previously interrupted run before starting.
cleanup_stale_path(&tmp).await;
cleanup_stale_path(&old).await;
stage_upgrade_subvolume(&agent_root, &tmp).await?;
// Swap. `rename` is atomic within a filesystem. The window between the
// two renames is the only unsafe point: a crash there leaves the agent
// root missing but both `.old` (original) and the new subvolume present
// — recoverable by hand, hence the loud logging.
if let Err(e) = std::fs::rename(&agent_root, &old) {
cleanup_stale_path(&tmp).await;
return Err(anyhow::Error::new(e).context(format!(
"rename {} -> {} failed; original left untouched",
agent_root.display(),
old.display()
)));
}
if let Err(e) = std::fs::rename(&tmp, &agent_root) {
// Restore the original from its renamed-aside copy.
let restored = std::fs::rename(&old, &agent_root).is_ok();
cleanup_stale_path(&tmp).await;
return Err(anyhow::Error::new(e).context(format!(
"rename {} -> {} failed; original {}",
tmp.display(),
agent_root.display(),
if restored {
"restored"
} else {
"COULD NOT BE RESTORED — manual recovery needed"
}
)));
}
// 5. Success: drop the original (a plain dir) and report.
if let Err(e) = std::fs::remove_dir_all(&old) {
// The migration succeeded; a leftover `.old` is cosmetic. Warn only.
tracing::warn!(
agent = %agent_name, path = %old.display(),
"upgraded subvolume but failed to remove old dir: {e}"
);
}
tracing::info!(
agent = %agent_name, path = %agent_root.display(),
"upgraded agent state dir to btrfs subvolume"
);
Ok((
format!("upgraded {} to a btrfs subvolume", agent_root.display()),
String::new(),
))
}
/// `SetSubvolumeQuota` — set or clear a qgroup size limit on an agent
/// subvolume (`btrfs qgroup limit <bytes|none> <…/agent_name>`). See the
/// wire doc.