feat(#2363): DHCP pool for bridge service containers

Add a DHCP pool to the gateway's dnsmasq so bridge-attached service
containers (hive-ci and future equivalents) get their addresses from
a proper DHCP server instead of a brittle static derivation.

gateway (hive-gateway.nix):
- Add IPv4 arithmetic helpers (ipToInt, intToIp, pow2) to compute the
  DHCP pool range at nix eval time from bridgeIp + bridgePrefixLength.
- Reserve the last dhcpPoolSize (14) usable host addresses as the DHCP
  pool (e.g. .241-.254 on a /24 with 10.42.0.0 network).
- Add dhcp-range and dhcp-leasefile to the dnsmasq settings block.
  The pool is active whenever services.hyperhive.network.enable is true.

hive-ci (hive-ci.nix):
- Remove the ciBridgeIp / ciBridgeOctets static derivation and the
  brittle top-of-/24 comment block.
- Switch networking.interfaces.eth0 to useDHCP = true so hive-ci gets
  its address from the gateway DHCP pool.

lifecycle (mod.rs, tests.rs):
- Add DHCP_POOL_SIZE = 14 constant (must stay in sync with
  dhcpPoolSize in hive-gateway.nix).
- Remap agents whose FNV-1a hash falls in the DHCP pool into the
  agent-only window [2, dhcp_start - 1]. Only the rare agent whose
  name hashes into the pool is affected; all others keep their IPs.
- Update and extend tests: agent range is now .2-.240 on /24;
  add agent_network_ip_never_in_dhcp_pool covering 18 agent names.
This commit is contained in:
atlas 2026-07-13 10:31:44 +02:00 committed by mara
commit a3796890f5
4 changed files with 114 additions and 36 deletions

View file

@ -11,19 +11,6 @@ let
networkCfg = config.services.hyperhive.network;
tlsCfg = config.services.hyperhive.tls;
# Static bridge address for the hive-ci container. The hive bridge has
# NO DHCP server: agent containers get deterministic static IPs
# (`lifecycle::agent_network_ip` hashes each name across .2..broadcast-1)
# and dnsmasq on the bridge is DNS-only. hive-ci is the one service
# container on the bridge, so it needs a static address too — `useDHCP`
# here only hangs the boot waiting for a lease nothing serves. Reserve the
# top host address of the (default /24) subnet; a clash with an agent that
# happens to hash here is the same rename-to-resolve case as any
# agent/agent IP collision. Operators on a non-/24 bridge (or with
# `bridgeIp` set to the top address) should pick a free host address.
ciBridgeOctets = lib.splitString "." networkCfg.bridgeIp;
ciBridgeIp = "${lib.elemAt ciBridgeOctets 0}.${lib.elemAt ciBridgeOctets 1}.${lib.elemAt ciBridgeOctets 2}.254";
# Self-signed TLS is the gateway default (no operator cert / ACME). When
# active, forgejo's ROOT_URL is `https://forge.<domain>` and the leaf is
# signed by the host hive CA — so the runner's Node-based actions (e.g.
@ -422,22 +409,11 @@ in
# registries, etc.). The bridge→loopback DROP rule does not
# affect traffic destined for the bridge IP itself.
networking.nameservers = [ networkCfg.bridgeIp ];
# With privateNetwork=true + hostBridge the container's veth
# (eth0) is bridge-attached. There is no DHCP server on the hive
# bridge (dnsmasq is DNS-only; agents use static IPs), so assign a
# static address + default route via the bridge gateway rather than
# DHCP — `useDHCP` here just hangs boot on a lease that never
# arrives. See `ciBridgeIp` above.
networking.interfaces.eth0.ipv4.addresses = [
{
address = ciBridgeIp;
prefixLength = networkCfg.bridgePrefixLength;
}
];
networking.defaultGateway = {
address = networkCfg.bridgeIp;
interface = "eth0";
};
# Bridge-attached via privateNetwork=true + hostBridge. The
# gateway's dnsmasq now serves a DHCP pool for service containers
# (see dhcp-range in hive-gateway.nix). hive-ci gets its address
# from that pool; no static address needed.
networking.interfaces.eth0.useDHCP = true;
# nspawn containers can't create user-namespaces, so nix
# sandboxing always fails. Fall back to unsandboxed builds.

View file

@ -11,6 +11,37 @@ let
forgeCfg = config.services.hyperhive.forge;
networkCfg = config.services.hyperhive.network;
# DHCP pool for bridge-attached service containers (hive-ci, etc.).
# Occupies the last dhcpPoolSize usable addresses of the subnet
# (e.g. .241-.254 on a /24). Agent containers use deterministic static
# IPs (lifecycle::agent_network_ip) and are excluded from this range
# by a remap in the Rust code. Must stay in sync with DHCP_POOL_SIZE
# in hive-c0re/src/lifecycle/mod.rs.
dhcpPoolSize = 14;
# IPv4 helpers — nix integers are 64-bit so all /0-/32 values are safe.
ipToInt =
ip:
builtins.foldl' (acc: x: acc * 256 + x) 0 (
map lib.strings.toIntBase10 (lib.strings.splitString "." ip)
);
intToIp =
n:
let
a = n / 16777216;
b = (n - a * 16777216) / 65536;
c = (n - a * 16777216 - b * 65536) / 256;
d = n - a * 16777216 - b * 65536 - c * 256;
in
"${toString a}.${toString b}.${toString c}.${toString d}";
# 2^n via recursion (nix has no pow builtin).
pow2 = n: if n == 0 then 1 else 2 * (pow2 (n - 1));
hostCount = pow2 (32 - networkCfg.bridgePrefixLength);
# Mask off host bits to get the network base address.
networkBase = builtins.bitAnd (ipToInt networkCfg.bridgeIp) (4294967295 - hostCount + 1);
# DHCP range: last dhcpPoolSize usable host addresses (broadcast - 1 down).
dhcpEnd = intToIp (networkBase + hostCount - 2); # last usable = broadcast - 1
dhcpStart = intToIp (networkBase + hostCount - 1 - dhcpPoolSize); # dhcpEnd - poolSize + 1
# Dashboard SPA dist, static-served by nginx below. Read in OUTER scope so
# `config` is the host's (inside the container block it'd be the container's).
dashboardDist = "${config.services.hyperhive.c0re.servedFrontend}/dashboard";
@ -999,6 +1030,12 @@ in
++ lib.optional (
matrixCfg.enable && matrixCfg.gatewayHost != null
) "/${matrixCfg.gatewayHost}/${networkCfg.bridgeIp}";
# DHCP pool for bridge-attached service containers (hive-ci, etc.).
# Range is computed from the bridgeIp/bridgePrefixLength at eval
# time; the last dhcpPoolSize usable host addresses are reserved.
# Agent containers are excluded by agent_network_ip's DHCP remap.
dhcp-range = "${dhcpStart},${dhcpEnd},1h";
dhcp-leasefile = "/var/lib/dnsmasq/dnsmasq.leases";
};
};
};