feat(#2862): push a snapshot to a peer hive's store over the mesh

Adds the caller the fd-passing machinery existed for: hivectl agent
<name> subvol snapshot push --peer <hive> resolves the peer, connects
to its snapshot store, writes the agent header, and hands the connected
socket to hive-priv, which runs btrfs send straight into it.

The split keeps the root helper ignorant. Everything that involves
knowing where a peer is, what the wire protocol looks like, and which
hive to trust happens in the unprivileged daemon; hive-priv only ever
receives an already-open descriptor. Once btrfs send starts, neither
process is in the data path, so a multi-gigabyte transfer costs no
per-byte work and survives a hive-c0re restart.

call_with_fd takes the descriptor by value and closes it as soon as the
kernel has it. A socket stays open until every copy closes, so holding
one back would leave the receiver waiting for an EOF that never comes:
btrfs receive blocks and this side reports success for a transfer the
peer never committed. Ownership makes that unrepresentable.

The peer's store port is a new swarm.peers.<domain>.snapshotStorePort
option rather than a constant matching the module default. A pushing
hive cannot read the receiver's configuration, so assuming 51821 would
push at a port nobody promised to listen on; absent, the push fails
naming the option. swarm_peers parses the mesh address the host module
has always rendered into HYPERHIVE_PEERS but nothing read.
This commit is contained in:
atlas 2026-07-31 21:40:34 +02:00 committed by mara
commit 282bbc3709
10 changed files with 546 additions and 9 deletions

View file

@ -0,0 +1,89 @@
//! Push an agent snapshot to a peer hive's snapshot store.
//!
//! This is the c0re half of the transport: it resolves the peer, opens
//! the connection, writes the protocol header, and hands the connected
//! socket to `hive-priv`, which runs `btrfs send` straight into it.
//!
//! The split is the point. Everything that requires knowing *where* the
//! peer is and *what* the wire protocol looks like happens here, in the
//! unprivileged daemon. hive-priv only ever receives an already-open
//! descriptor, so the root helper never learns an address, never parses
//! a peer list, and never decides who to trust — and once `btrfs send`
//! starts, neither process is in the data path.
use anyhow::{Context as _, Result, bail};
use hive_priv_sock::PrivRequest;
use tokio::io::AsyncWriteExt as _;
use tokio::net::TcpStream;
use crate::priv_client;
use crate::swarm_peers;
/// Send `snapshot` of `agent` to `peer_domain`'s snapshot store,
/// optionally as an incremental against `parent`.
///
/// # Errors
///
/// Fails when the peer is unknown, declares no mesh address or store
/// port, is unreachable, or when hive-priv reports the `btrfs send`
/// failed.
pub async fn push_agent_snapshot(
agent: &str,
snapshot: &str,
parent: Option<&str>,
peer_domain: &str,
) -> Result<()> {
let peer = swarm_peers::peer(peer_domain)?;
let addr = peer.snapshot_store_addr()?;
let mut sock = TcpStream::connect(&addr)
.await
.with_context(|| format!("connect to {peer_domain} snapshot store at {addr}"))?;
// The header names which agent this stream belongs to: a btrfs
// stream carries the sender's own subvolume name, not the hive's
// notion of the agent, so the receiver cannot infer it. One line,
// because the receiver reads it with `read` and must not buffer
// past the newline into the byte stream.
sock.write_all(format!("agent {agent}\n").as_bytes())
.await
.with_context(|| format!("send header to {peer_domain} snapshot store"))?;
sock.flush()
.await
.with_context(|| format!("flush header to {peer_domain} snapshot store"))?;
// Hand the connected socket over. `call_with_fd` takes ownership and
// closes our copy as soon as the kernel has it, so the receiver sees
// EOF when `btrfs send` finishes rather than hanging on a descriptor
// this process is still holding.
let fd = sock
.into_std()
.context("detach snapshot-store socket for hand-off")?
.into();
let resp = priv_client::call_with_fd(
&PrivRequest::SendAgentSnapshotToFd {
agent_name: agent.to_owned(),
snapshot_name: snapshot.to_owned(),
parent_snapshot_name: parent.map(str::to_owned),
},
fd,
)
.await
.with_context(|| format!("stream {agent}/{snapshot} to {peer_domain}"))?;
if !resp.ok {
bail!(
"push of {agent}/{snapshot} to {peer_domain} failed: {}",
resp.error.unwrap_or_else(|| resp.stderr.clone())
);
}
tracing::info!(
agent,
snapshot,
parent,
peer = peer_domain,
"pushed agent snapshot to peer snapshot store"
);
Ok(())
}