vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix(network): wire default route + bridge DNS for isolated containers

Browse source 
When isolateContainers=true, claude (and all egress) broke in every
container: agents came up with an IP but no way off the bridge subnet.

Two container-side gaps, both confirmed against nixpkgs
nixos-containers.nix:

1. No default route. hive-priv wrote HOST_ADDRESS= empty in the nspawn
   conf. nixos-container's container-side setup only installs
   `ip route add default via $HOST_ADDRESS` when HOST_ADDRESS is
   non-empty, so the container had an address but no gateway -> nothing
   off-subnet (incl. api.anthropic.com) was reachable. Fix: write
   HOST_ADDRESS=<bridge-ip>. In bridge mode the host-side address/route
   setup is skipped, so this only affects the container's default route.

2. No usable resolver. nixos-container copies the host's /etc/resolv.conf
   into the container at every start; the host resolver (e.g. 127.0.0.53)
   is unreachable from a private netns and isn't authoritative for the
   hive's own zones. Fix: hive-priv drops a marker carrying the gateway
   IP only when isolated, and a new harness-base oneshot
   (hyperhive-isolated-dns) rewrites resolv.conf to point at the bridge
   dnsmasq. Inert in shared-netns mode (no marker), so the shared
   container toplevel does the right thing in both modes.

The gateway IP is the address part of HIVE_NETWORK_SUBNET (the bridge IP
verbatim, honouring a non-.1 operator override), via a new validated
bridge_gateway_ip() helper with unit tests.

Unblocks defaulting isolation on.
This commit is contained in:
atlas 2026-06-10 20:52:12 +02:00 • committed by mara
commit d993ad2c47
4 changed files with 177 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

40
nix/templates/harness-base.nix
View file

@ -1025,6 +1025,46 @@ in
# One-shot: tea config.yml from the seeded forge token. Shape
# contract (always exit 0, no set -e, skip-silently, re-runnable):
# docs/conventions.md::Best-effort oneshot services.
# Point resolv.conf at the hive bridge resolver when the container is
# network-isolated. nixos-container copies the *host's* /etc/resolv.conf
# into the container at every start — but the host resolver (e.g.
# 127.0.0.53) is unreachable from a private netns and isn't
# authoritative for the hive's own zones (forge.<domain> etc.). The
# bridge dnsmasq (gateway IP) is. hive-priv drops the marker
# `/etc/hyperhive-bridge-dns` (containing the gateway IP) only when
# isolation is on, so this oneshot is inert in shared-netns mode — the
# same shared container toplevel does the right thing in both modes.
# Ordered before the first DNS consumer (tea-login) and the network
# targets so name resolution works for the very first turn.
systemd.services.hyperhive-isolated-dns = {
description = "point resolv.conf at the hive bridge resolver (isolated containers)";
wantedBy = [ "multi-user.target" ];
after = [ "local-fs.target" ];
before = [
"network-online.target"
"tea-login.service"
];
unitConfig.ConditionPathExists = "/etc/hyperhive-bridge-dns";
serviceConfig = {
Type = "oneshot";
RemainAfterExit = true;
};
path = [ pkgs.coreutils ];
script = ''
set -eu
gw=$(tr -d '[:space:]' < /etc/hyperhive-bridge-dns)
if [ -z "$gw" ]; then
echo "hyperhive-isolated-dns: empty marker; leaving resolv.conf as-is"
exit 0
fi
# resolv.conf is a regular file copied from the host by
# nixos-container; replace it (rm first in case it's a symlink).
rm -f /etc/resolv.conf
printf 'nameserver %s\n' "$gw" > /etc/resolv.conf
echo "hyperhive-isolated-dns: resolv.conf -> nameserver $gw"
'';
};
systemd.services.tea-login = {
description = "configure tea CLI from hive-forge token (best-effort)";
wantedBy = [ "multi-user.target" ];