Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2c079afd65 | ||
|
|
720ac81235 | ||
|
|
8f8076b8ed | ||
|
|
7799e0762a |
5 changed files with 314 additions and 5 deletions
|
|
@ -38,6 +38,9 @@ This document contains the help content for the `hivectl` command-line program.
|
|||
* [`hivectl quota limit`↴](#hivectl-quota-limit)
|
||||
* [`hivectl subvol`↴](#hivectl-subvol)
|
||||
* [`hivectl subvol upgrade`↴](#hivectl-subvol-upgrade)
|
||||
* [`hivectl subvol snapshot`↴](#hivectl-subvol-snapshot)
|
||||
* [`hivectl subvol snapshot create`↴](#hivectl-subvol-snapshot-create)
|
||||
* [`hivectl subvol snapshot delete`↴](#hivectl-subvol-snapshot-delete)
|
||||
* [`hivectl open`↴](#hivectl-open)
|
||||
* [`hivectl completions`↴](#hivectl-completions)
|
||||
|
||||
|
|
@ -548,6 +551,7 @@ New agents get a btrfs subvolume state root automatically (when the host FS is b
|
|||
###### **Subcommands:**
|
||||
|
||||
* `upgrade` — Convert an existing plain-dir agent state root into a btrfs subvolume in place. Stops the agent (so its state bind-mount is released), migrates `…/agents/<name>/` to a subvolume preserving ownership/permissions/xattrs, then restarts it. Idempotent (no-op if already a subvolume) and safe (the original dir is left untouched on any failure before the final swap). Requires `--yes` since it bounces the agent and moves its state
|
||||
* `snapshot` — Read-only snapshots of an agent's state subvolume — the first step of the (in-progress) inter-hive migration path, or a manual point-in-time backup
|
||||
|
||||
|
||||
|
||||
|
|
@ -567,6 +571,48 @@ Convert an existing plain-dir agent state root into a btrfs subvolume in place.
|
|||
|
||||
|
||||
|
||||
## `hivectl subvol snapshot`
|
||||
|
||||
Read-only snapshots of an agent's state subvolume — the first step of the (in-progress) inter-hive migration path, or a manual point-in-time backup
|
||||
|
||||
**Usage:** `hivectl subvol snapshot <COMMAND>`
|
||||
|
||||
###### **Subcommands:**
|
||||
|
||||
* `create` — Create a read-only snapshot. Agent must already be a subvolume (`subvol upgrade` first). Prints the snapshot's host path
|
||||
* `delete` — Delete a snapshot created by `subvol snapshot create`
|
||||
|
||||
|
||||
|
||||
## `hivectl subvol snapshot create`
|
||||
|
||||
Create a read-only snapshot. Agent must already be a subvolume (`subvol upgrade` first). Prints the snapshot's host path
|
||||
|
||||
**Usage:** `hivectl subvol snapshot create --label <LABEL> <NAME>`
|
||||
|
||||
###### **Arguments:**
|
||||
|
||||
* `<NAME>` — Agent name (e.g. `damocles`, `iris`)
|
||||
|
||||
###### **Options:**
|
||||
|
||||
* `--label <LABEL>` — Snapshot label. Mandatory, and must start with `hive-` — the prefix doubles as an allow-list hive-priv checks so only hivectl-issued snapshot names can reach the `btrfs subvolume snapshot` shellout
|
||||
|
||||
|
||||
|
||||
## `hivectl subvol snapshot delete`
|
||||
|
||||
Delete a snapshot created by `subvol snapshot create`
|
||||
|
||||
**Usage:** `hivectl subvol snapshot delete <NAME> <LABEL>`
|
||||
|
||||
###### **Arguments:**
|
||||
|
||||
* `<NAME>` — Agent name the snapshot belongs to
|
||||
* `<LABEL>` — Snapshot label passed to `subvol snapshot create --label`
|
||||
|
||||
|
||||
|
||||
## `hivectl open`
|
||||
|
||||
Print (and best-effort open in a browser) a hive web surface URL.
|
||||
|
|
|
|||
|
|
@ -620,6 +620,36 @@ enum SubvolCmd {
|
|||
#[arg(long)]
|
||||
yes: bool,
|
||||
},
|
||||
/// Read-only snapshots of an agent's state subvolume — the first step
|
||||
/// of the (in-progress) inter-hive migration path, or a manual
|
||||
/// point-in-time backup.
|
||||
Snapshot {
|
||||
#[command(subcommand)]
|
||||
cmd: SnapshotCmd,
|
||||
},
|
||||
}
|
||||
|
||||
#[derive(Subcommand)]
|
||||
enum SnapshotCmd {
|
||||
/// Create a read-only snapshot. Agent must already be a subvolume
|
||||
/// (`subvol upgrade` first). Prints the snapshot's host path.
|
||||
Create {
|
||||
/// Agent name (e.g. `damocles`, `iris`).
|
||||
name: String,
|
||||
/// Snapshot label. Mandatory, and must start with `hive-` — the
|
||||
/// prefix doubles as an allow-list hive-priv checks so only
|
||||
/// hivectl-issued snapshot names can reach the `btrfs subvolume
|
||||
/// snapshot` shellout.
|
||||
#[arg(long)]
|
||||
label: String,
|
||||
},
|
||||
/// Delete a snapshot created by `subvol snapshot create`.
|
||||
Delete {
|
||||
/// Agent name the snapshot belongs to.
|
||||
name: String,
|
||||
/// Snapshot label passed to `subvol snapshot create --label`.
|
||||
label: String,
|
||||
},
|
||||
}
|
||||
|
||||
#[tokio::main]
|
||||
|
|
@ -693,6 +723,10 @@ async fn main() -> Result<()> {
|
|||
Cmd::Restart { scope, graceful } => restart(&socket, scope.to_scope(), graceful).await,
|
||||
Cmd::Subvol { cmd } => match cmd {
|
||||
SubvolCmd::Upgrade { name, yes } => subvol_upgrade(&socket, &name, yes).await,
|
||||
SubvolCmd::Snapshot { cmd } => match cmd {
|
||||
SnapshotCmd::Create { name, label } => subvol_snapshot_create(&name, label).await,
|
||||
SnapshotCmd::Delete { name, label } => subvol_snapshot_delete(&name, &label).await,
|
||||
},
|
||||
},
|
||||
Cmd::Choom {
|
||||
name,
|
||||
|
|
@ -1703,6 +1737,46 @@ async fn subvol_upgrade(socket: &Path, name: &str, yes: bool) -> Result<()> {
|
|||
Ok(())
|
||||
}
|
||||
|
||||
/// `subvol snapshot create <agent> --label <label>` — create a read-only
|
||||
/// btrfs snapshot of an agent's state subvolume. Unlike `upgrade`, this does
|
||||
/// NOT stop the agent: btrfs snapshots are atomic + consistent to take
|
||||
/// against a live subvolume. `label` is mandatory, must start with
|
||||
/// `hive-`, and is otherwise restricted to `[A-Za-z0-9_-]` (no `.` at all
|
||||
/// — mara: "we are making up the rules here, lets go strict"). hive-priv
|
||||
/// enforces the same rules server-side, so this check is
|
||||
/// belt-and-suspenders (fail fast client-side with a clear message).
|
||||
async fn subvol_snapshot_create(name: &str, label: String) -> Result<()> {
|
||||
if !agent_exists(name)? {
|
||||
bail!("no agent named {name:?} (no state dir under the agents root)");
|
||||
}
|
||||
if !label.starts_with("hive-") {
|
||||
bail!("snapshot label {label:?} must start with \"hive-\"");
|
||||
}
|
||||
if !label
|
||||
.bytes()
|
||||
.all(|b| b.is_ascii_alphanumeric() || matches!(b, b'_' | b'-'))
|
||||
{
|
||||
bail!(
|
||||
"snapshot label {label:?} must be [A-Za-z0-9_-] only (no \".\" — hive-priv rejects it)"
|
||||
);
|
||||
}
|
||||
let path = hive_c0re::priv_client::snapshot_agent_subvolume(name, &label)
|
||||
.await
|
||||
.with_context(|| format!("snapshot {name} state subvolume (label {label:?})"))?;
|
||||
println!("{path}");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// `subvol snapshot delete <agent> <label>` — remove a snapshot created by
|
||||
/// `subvol snapshot create`.
|
||||
async fn subvol_snapshot_delete(name: &str, label: &str) -> Result<()> {
|
||||
hive_c0re::priv_client::delete_agent_snapshot(name, label)
|
||||
.await
|
||||
.with_context(|| format!("delete {name} snapshot (label {label:?})"))?;
|
||||
println!("deleted snapshot {label:?} for {name}");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Render a hive-wide stop/start response: one `<verb>: <name>` line per
|
||||
/// touched container, then surface any aggregated per-target failure as a
|
||||
/// non-zero exit. `verb` is the past-tense word printed per item
|
||||
|
|
|
|||
|
|
@ -439,6 +439,41 @@ pub async fn upgrade_agent_subvolume(agent_name: &str) -> Result<()> {
|
|||
.await?)
|
||||
}
|
||||
|
||||
/// Create a read-only btrfs snapshot of an agent's state subvolume (via
|
||||
/// hive-priv as root) — the first step of `hivectl migrate`'s send/receive
|
||||
/// path. Returns the snapshot's absolute host path. Fails if the agent's
|
||||
/// state root isn't a subvolume yet, or a snapshot with the same
|
||||
/// `snapshot_name` already exists.
|
||||
///
|
||||
/// # Errors
|
||||
/// Returns an error if the hive-priv call fails, the state dir isn't a
|
||||
/// btrfs subvolume, or the snapshot already exists.
|
||||
pub async fn snapshot_agent_subvolume(agent_name: &str, snapshot_name: &str) -> Result<String> {
|
||||
let (stdout, _) = check(
|
||||
call(&PrivRequest::SnapshotAgentSubvolume {
|
||||
agent_name: agent_name.to_owned(),
|
||||
snapshot_name: snapshot_name.to_owned(),
|
||||
})
|
||||
.await?,
|
||||
)?;
|
||||
Ok(stdout)
|
||||
}
|
||||
|
||||
/// Delete a previously-created read-only agent-state snapshot (cleanup
|
||||
/// counterpart to [`snapshot_agent_subvolume`]). No-op if the snapshot
|
||||
/// doesn't exist. Via hive-priv as root.
|
||||
///
|
||||
/// # Errors
|
||||
/// Returns an error if the hive-priv call fails or the underlying
|
||||
/// `btrfs subvolume delete` fails.
|
||||
pub async fn delete_agent_snapshot(agent_name: &str, snapshot_name: &str) -> Result<()> {
|
||||
ok(call(&PrivRequest::DeleteAgentSnapshot {
|
||||
agent_name: agent_name.to_owned(),
|
||||
snapshot_name: snapshot_name.to_owned(),
|
||||
})
|
||||
.await?)
|
||||
}
|
||||
|
||||
/// Write `/etc/tmpfiles.d/hyperhive-agents.conf` for `agents` (logical names,
|
||||
/// e.g. `"atlas"`) and immediately apply it with `systemd-tmpfiles --create`.
|
||||
/// See [`PrivRequest::SyncAgentTmpfiles`] for the full semantics.
|
||||
|
|
|
|||
|
|
@ -365,6 +365,24 @@ async fn exec(req: PrivRequest, writer: &mut OwnedWriteHalf) -> Result<(String,
|
|||
upgrade_agent_subvolume(agent_name).await
|
||||
}
|
||||
|
||||
PrivRequest::SnapshotAgentSubvolume {
|
||||
ref agent_name,
|
||||
ref snapshot_name,
|
||||
} => {
|
||||
validate_agent_name(agent_name)?;
|
||||
validate_snapshot_name(snapshot_name)?;
|
||||
snapshot_agent_subvolume(agent_name, snapshot_name).await
|
||||
}
|
||||
|
||||
PrivRequest::DeleteAgentSnapshot {
|
||||
ref agent_name,
|
||||
ref snapshot_name,
|
||||
} => {
|
||||
validate_agent_name(agent_name)?;
|
||||
validate_snapshot_name(snapshot_name)?;
|
||||
delete_agent_snapshot(agent_name, snapshot_name).await
|
||||
}
|
||||
|
||||
PrivRequest::SyncAgentTmpfiles { ref agents } => sync_agent_tmpfiles(agents).await,
|
||||
}
|
||||
}
|
||||
|
|
@ -412,16 +430,36 @@ fn handle_write_nspawn_flags(
|
|||
Ok((String::new(), String::new()))
|
||||
}
|
||||
|
||||
/// A btrfs snapshot label must start with `hive-` — this doubles as an
|
||||
/// allow-list: only names hivectl itself constructs (or an operator who
|
||||
/// knows the convention) can reach the `btrfs subvolume snapshot`/`delete`
|
||||
/// shellouts, so an arbitrary caller can't use the snapshot ops to probe or
|
||||
/// churn unrelated paths under `AGENT_STATE_ROOT`. Beyond the prefix, the
|
||||
/// same charset restriction as [`validate_credential_name`] applies (it's
|
||||
/// interpolated straight into a filesystem path).
|
||||
fn validate_snapshot_name(name: &str) -> Result<()> {
|
||||
if !name.starts_with("hive-") {
|
||||
bail!("invalid snapshot label {name:?}: must start with \"hive-\"");
|
||||
}
|
||||
validate_credential_name(name)
|
||||
}
|
||||
|
||||
/// A systemd credential id must be a short token — restrict to
|
||||
/// `[A-Za-z0-9_.-]` so it can't inject extra `--load-credential` argv or
|
||||
/// break the `name:path` shape.
|
||||
/// `[A-Za-z0-9_-]` (no `.`) so it can't inject extra `--load-credential`
|
||||
/// argv or break the `name:path` shape. `.` is deliberately excluded, not
|
||||
/// just a bare `..`: this name gets interpolated into filesystem paths
|
||||
/// (snapshot labels) and there's no legitimate need for a dot in either a
|
||||
/// systemd credential id or a `hive-`-prefixed snapshot label — we're
|
||||
/// defining this token format from scratch, so keep it maximally strict
|
||||
/// rather than allow-then-patch each traversal-adjacent character
|
||||
/// (mara: "we are making up the rules here, lets go strict").
|
||||
fn validate_credential_name(name: &str) -> Result<()> {
|
||||
if name.is_empty()
|
||||
|| !name
|
||||
.bytes()
|
||||
.all(|b| b.is_ascii_alphanumeric() || matches!(b, b'_' | b'.' | b'-'))
|
||||
.all(|b| b.is_ascii_alphanumeric() || matches!(b, b'_' | b'-'))
|
||||
{
|
||||
bail!("invalid credential name {name:?}: must be non-empty [A-Za-z0-9_.-]");
|
||||
bail!("invalid credential name {name:?}: must be non-empty [A-Za-z0-9_-]");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
|
@ -1059,6 +1097,86 @@ async fn upgrade_agent_subvolume(agent_name: &str) -> Result<(String, String)> {
|
|||
))
|
||||
}
|
||||
|
||||
/// Derive a snapshot's path from the agent name + label: a dot-prefixed
|
||||
/// sibling of the agent's state root so it can never collide with a real
|
||||
/// agent directory (`validate_agent_name` rejects dot-prefixed names).
|
||||
fn snapshot_path(agent_name: &str, snapshot_name: &str) -> PathBuf {
|
||||
PathBuf::from(AGENT_STATE_ROOT).join(format!(".{agent_name}.snapshot.{snapshot_name}"))
|
||||
}
|
||||
|
||||
/// `SnapshotAgentSubvolume` — create a read-only btrfs snapshot of an
|
||||
/// agent's state subvolume, for `btrfs send` to stream from during
|
||||
/// inter-hive migration. See the wire doc.
|
||||
async fn snapshot_agent_subvolume(
|
||||
agent_name: &str,
|
||||
snapshot_name: &str,
|
||||
) -> Result<(String, String)> {
|
||||
let agent_root = PathBuf::from(AGENT_STATE_ROOT).join(agent_name);
|
||||
if !is_btrfs_subvolume(&agent_root) {
|
||||
bail!(
|
||||
"{} is not a btrfs subvolume — nothing to snapshot (run `hivectl subvol upgrade` first)",
|
||||
agent_root.display()
|
||||
);
|
||||
}
|
||||
let snap = snapshot_path(agent_name, snapshot_name);
|
||||
if snap.exists() {
|
||||
bail!(
|
||||
"snapshot {} already exists — delete it first or pick a different name",
|
||||
snap.display()
|
||||
);
|
||||
}
|
||||
let out = Command::new("btrfs")
|
||||
.args(["subvolume", "snapshot", "-r"])
|
||||
.arg(&agent_root)
|
||||
.arg(&snap)
|
||||
.output()
|
||||
.await
|
||||
.with_context(|| {
|
||||
format!(
|
||||
"spawn btrfs subvolume snapshot -r {} {}",
|
||||
agent_root.display(),
|
||||
snap.display()
|
||||
)
|
||||
})?;
|
||||
if !out.status.success() {
|
||||
bail!(
|
||||
"btrfs subvolume snapshot -r {} {} failed: {}",
|
||||
agent_root.display(),
|
||||
snap.display(),
|
||||
String::from_utf8_lossy(&out.stderr).trim()
|
||||
);
|
||||
}
|
||||
tracing::info!(
|
||||
agent = %agent_name, snapshot = %snap.display(),
|
||||
"created read-only agent state snapshot"
|
||||
);
|
||||
Ok((snap.display().to_string(), String::new()))
|
||||
}
|
||||
|
||||
/// `DeleteAgentSnapshot` — delete a previously-created read-only snapshot.
|
||||
/// No-op if the path doesn't exist. See the wire doc.
|
||||
async fn delete_agent_snapshot(agent_name: &str, snapshot_name: &str) -> Result<(String, String)> {
|
||||
let snap = snapshot_path(agent_name, snapshot_name);
|
||||
if !snap.exists() {
|
||||
return Ok((String::new(), String::new()));
|
||||
}
|
||||
let out = Command::new("btrfs")
|
||||
.args(["subvolume", "delete"])
|
||||
.arg(&snap)
|
||||
.output()
|
||||
.await
|
||||
.with_context(|| format!("spawn btrfs subvolume delete {}", snap.display()))?;
|
||||
if !out.status.success() {
|
||||
bail!(
|
||||
"btrfs subvolume delete {} failed: {}",
|
||||
snap.display(),
|
||||
String::from_utf8_lossy(&out.stderr).trim()
|
||||
);
|
||||
}
|
||||
tracing::info!(agent = %agent_name, snapshot = %snap.display(), "deleted agent state snapshot");
|
||||
Ok((String::new(), String::new()))
|
||||
}
|
||||
|
||||
/// `SetSubvolumeQuota` — set or clear a qgroup size limit on an agent
|
||||
/// subvolume (`btrfs qgroup limit <bytes|none> <…/agent_name>`). See the
|
||||
/// wire doc.
|
||||
|
|
|
|||
|
|
@ -199,7 +199,7 @@ pub struct BindMount {
|
|||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct CredentialMount {
|
||||
/// systemd credential id (e.g. `otel-headers`); inner units inherit
|
||||
/// it by this name. Restricted to `[A-Za-z0-9_.-]` by hive-priv.
|
||||
/// it by this name. Restricted to `[A-Za-z0-9_-]` (no `.`) by hive-priv.
|
||||
pub name: String,
|
||||
/// Host path to the secret file, forwarded via nspawn
|
||||
/// `--load-credential=<name>:<host_path>`.
|
||||
|
|
@ -580,6 +580,42 @@ pub enum PrivRequest {
|
|||
agent_name: String,
|
||||
},
|
||||
|
||||
/// Create a read-only snapshot of an agent's state subvolume
|
||||
/// (`btrfs subvolume snapshot -r <agent_root> <snapshot_path>`). Used as
|
||||
/// the first step of inter-hive migration (`hivectl migrate`): freezing
|
||||
/// a consistent point-in-time copy that `btrfs send` can stream from
|
||||
/// while the source subvolume keeps running underneath the live agent.
|
||||
///
|
||||
/// The snapshot is created as a sibling of the agent's state root
|
||||
/// (`<AGENT_STATE_ROOT>/.<agent_name>.snapshot.<snapshot_name>`, dot-prefixed
|
||||
/// so it never collides with a real agent name) and its path is returned
|
||||
/// verbatim in the response's `stdout`. Fails if the agent's state root
|
||||
/// isn't a btrfs subvolume (nothing to snapshot) or a snapshot with the
|
||||
/// same name already exists. Requires root.
|
||||
SnapshotAgentSubvolume {
|
||||
/// Logical agent name (validated by `validate_agent_name`).
|
||||
agent_name: String,
|
||||
/// Snapshot label. Must start with `hive-` — the prefix doubles as
|
||||
/// an allow-list hive-priv enforces so only hivectl-issued names
|
||||
/// can reach the `btrfs subvolume snapshot` shellout — and
|
||||
/// otherwise follows the same charset as a credential name
|
||||
/// (non-empty `[A-Za-z0-9_-]`, no `.`); becomes part of the
|
||||
/// snapshot path.
|
||||
snapshot_name: String,
|
||||
},
|
||||
|
||||
/// Delete a previously-created read-only snapshot
|
||||
/// (`btrfs subvolume delete <snapshot_path>`). Cleanup counterpart to
|
||||
/// [`PrivRequest::SnapshotAgentSubvolume`] — called once a migration's
|
||||
/// `btrfs send` has completed (or aborted) and the frozen copy is no
|
||||
/// longer needed. No-op if the snapshot path doesn't exist. Requires root.
|
||||
DeleteAgentSnapshot {
|
||||
/// Logical agent name (validated by `validate_agent_name`).
|
||||
agent_name: String,
|
||||
/// Snapshot label, same validation as `SnapshotAgentSubvolume`.
|
||||
snapshot_name: String,
|
||||
},
|
||||
|
||||
/// Write `/etc/tmpfiles.d/hyperhive-agents.conf` for the given agent set
|
||||
/// and immediately apply it with `systemd-tmpfiles --create`. Each entry
|
||||
/// declares the per-agent runtime dirs (`/run/hyperhive/agents/<name>` and
|
||||
|
|
|
|||
Loading…
Reference in a new issue