vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

snapshot store: bound the transfer on both ends

Browse source 
The receiver blocked in read until a peer sent a header, and a stalled
btrfs receive was never reaped, so a silent or stuck connection held a
root service instance indefinitely. The receiver now gives the header 30 s
and the unit has RuntimeMaxSec=6h.

The sender had no deadline on the TCP connect or the header write, so an
unreachable mesh address held the admin-socket handler until the kernel
gave up. Both now time out after 15 s and report it like any other
failure. The stream body stays unbounded on the sender: the receiver's
RuntimeMaxSec already caps it, and btrfs send writes it outside this
process.
This commit is contained in:
atlas 2026-10-16 10:08:23 +02:00
commit b03549fde6
3 changed files with 67 additions and 17 deletions
Download patch file Download diff file Expand all files Collapse all files

23
nix/host-modules/swarm-snapshot-store.nix
View file

@ -33,6 +33,11 @@ let
# address must be the bare IP, so strip it.
meshAddress = lib.head (lib.splitString "/" wgCfg.address);
# How long a connected peer may stay silent before sending the header
# line. The sender writes it immediately after connecting, so this only
# ever expires for a peer that is not a hive.
headerTimeoutSeconds = 30;
# The receiver. Socket-activated with Accept=yes, so stdin IS the
# accepted connection and systemd hands us the peer address in
# $REMOTE_ADDR --- which, on this interface, is a cryptographically
@ -57,8 +62,15 @@ let
# Read exactly the header line, leaving the byte stream untouched
# for btrfs receive. `read` stops at the newline and does not
# buffer ahead, which is why the header is a line and not a
# fixed-width record.
if ! read -r keyword agent; then
# fixed-width record. `-t` bounds the wait: a peer that connects
# and stays silent would otherwise hold this root instance forever.
# `read` exits above 128 on timeout and 1 on EOF.
rc=0
read -r -t ${toString headerTimeoutSeconds} keyword agent || rc=$?
if [ "$rc" -gt 128 ]; then
echo "swarm-snapshot-store: peer ''${REMOTE_ADDR:-?} sent no header within ${toString headerTimeoutSeconds}s" >&2
exit 1
elif [ "$rc" -ne 0 ]; then
echo "swarm-snapshot-store: peer ''${REMOTE_ADDR:-?} closed before sending a header" >&2
exit 1
fi
@ -236,6 +248,13 @@ in
StandardInput = "socket";
StandardError = "journal";
User = "root";
# Upper bound on one whole transfer, header to the last byte of
# `btrfs receive`. Snapshots are whole agent state roots and run to
# multiple gigabytes (hive-priv `send_agent_snapshot_to_fd`); no
# tighter bound is documented, so this is sized for ~100 GB at
# 5 MB/s (about 5.6 h), not measured. A stalled receive is killed
# here instead of holding a root instance indefinitely.
RuntimeMaxSec = "6h";
PrivateTmp = true;
ProtectHome = true;
ProtectSystem = "strict";