feat(#2862): push a snapshot to a peer hive's store over the mesh
Adds the caller the fd-passing machinery existed for: hivectl agent <name> subvol snapshot push --peer <hive> resolves the peer, connects to its snapshot store, writes the agent header, and hands the connected socket to hive-priv, which runs btrfs send straight into it. The split keeps the root helper ignorant. Everything that involves knowing where a peer is, what the wire protocol looks like, and which hive to trust happens in the unprivileged daemon; hive-priv only ever receives an already-open descriptor. Once btrfs send starts, neither process is in the data path, so a multi-gigabyte transfer costs no per-byte work and survives a hive-c0re restart. call_with_fd takes the descriptor by value and closes it as soon as the kernel has it. A socket stays open until every copy closes, so holding one back would leave the receiver waiting for an EOF that never comes: btrfs receive blocks and this side reports success for a transfer the peer never committed. Ownership makes that unrepresentable. The peer's store port is a new swarm.peers.<domain>.snapshotStorePort option rather than a constant matching the module default. A pushing hive cannot read the receiver's configuration, so assuming 51821 would push at a port nobody promised to listen on; absent, the push fails naming the option. swarm_peers parses the mesh address the host module has always rendered into HYPERHIVE_PEERS but nothing read.
This commit is contained in:
parent
51f352f0ca
commit
282bbc3709
10 changed files with 546 additions and 9 deletions
89
hive-c0re/src/snapshot_push.rs
Normal file
89
hive-c0re/src/snapshot_push.rs
Normal file
|
|
@ -0,0 +1,89 @@
|
|||
//! Push an agent snapshot to a peer hive's snapshot store.
|
||||
//!
|
||||
//! This is the c0re half of the transport: it resolves the peer, opens
|
||||
//! the connection, writes the protocol header, and hands the connected
|
||||
//! socket to `hive-priv`, which runs `btrfs send` straight into it.
|
||||
//!
|
||||
//! The split is the point. Everything that requires knowing *where* the
|
||||
//! peer is and *what* the wire protocol looks like happens here, in the
|
||||
//! unprivileged daemon. hive-priv only ever receives an already-open
|
||||
//! descriptor, so the root helper never learns an address, never parses
|
||||
//! a peer list, and never decides who to trust — and once `btrfs send`
|
||||
//! starts, neither process is in the data path.
|
||||
|
||||
use anyhow::{Context as _, Result, bail};
|
||||
use hive_priv_sock::PrivRequest;
|
||||
use tokio::io::AsyncWriteExt as _;
|
||||
use tokio::net::TcpStream;
|
||||
|
||||
use crate::priv_client;
|
||||
use crate::swarm_peers;
|
||||
|
||||
/// Send `snapshot` of `agent` to `peer_domain`'s snapshot store,
|
||||
/// optionally as an incremental against `parent`.
|
||||
///
|
||||
/// # Errors
|
||||
///
|
||||
/// Fails when the peer is unknown, declares no mesh address or store
|
||||
/// port, is unreachable, or when hive-priv reports the `btrfs send`
|
||||
/// failed.
|
||||
pub async fn push_agent_snapshot(
|
||||
agent: &str,
|
||||
snapshot: &str,
|
||||
parent: Option<&str>,
|
||||
peer_domain: &str,
|
||||
) -> Result<()> {
|
||||
let peer = swarm_peers::peer(peer_domain)?;
|
||||
let addr = peer.snapshot_store_addr()?;
|
||||
|
||||
let mut sock = TcpStream::connect(&addr)
|
||||
.await
|
||||
.with_context(|| format!("connect to {peer_domain} snapshot store at {addr}"))?;
|
||||
|
||||
// The header names which agent this stream belongs to: a btrfs
|
||||
// stream carries the sender's own subvolume name, not the hive's
|
||||
// notion of the agent, so the receiver cannot infer it. One line,
|
||||
// because the receiver reads it with `read` and must not buffer
|
||||
// past the newline into the byte stream.
|
||||
sock.write_all(format!("agent {agent}\n").as_bytes())
|
||||
.await
|
||||
.with_context(|| format!("send header to {peer_domain} snapshot store"))?;
|
||||
sock.flush()
|
||||
.await
|
||||
.with_context(|| format!("flush header to {peer_domain} snapshot store"))?;
|
||||
|
||||
// Hand the connected socket over. `call_with_fd` takes ownership and
|
||||
// closes our copy as soon as the kernel has it, so the receiver sees
|
||||
// EOF when `btrfs send` finishes rather than hanging on a descriptor
|
||||
// this process is still holding.
|
||||
let fd = sock
|
||||
.into_std()
|
||||
.context("detach snapshot-store socket for hand-off")?
|
||||
.into();
|
||||
|
||||
let resp = priv_client::call_with_fd(
|
||||
&PrivRequest::SendAgentSnapshotToFd {
|
||||
agent_name: agent.to_owned(),
|
||||
snapshot_name: snapshot.to_owned(),
|
||||
parent_snapshot_name: parent.map(str::to_owned),
|
||||
},
|
||||
fd,
|
||||
)
|
||||
.await
|
||||
.with_context(|| format!("stream {agent}/{snapshot} to {peer_domain}"))?;
|
||||
|
||||
if !resp.ok {
|
||||
bail!(
|
||||
"push of {agent}/{snapshot} to {peer_domain} failed: {}",
|
||||
resp.error.unwrap_or_else(|| resp.stderr.clone())
|
||||
);
|
||||
}
|
||||
tracing::info!(
|
||||
agent,
|
||||
snapshot,
|
||||
parent,
|
||||
peer = peer_domain,
|
||||
"pushed agent snapshot to peer snapshot store"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
Loading…
Reference in a new issue