vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

nix/hive-network: bridge + dnsmasq resolver, opt-in (#805 v1)

Browse source 
Stand up the host-side bridge interface + per-agent DNS resolver
ahead of #14 (netns isolation). Mara on #805#11541: "we need it
before #14 so nothing breaks when we switch over". v1 ships the
endpoint live but containers stay on shared host netns — when #14
flips them to private netns the DNS contract is already there.

Shape:

- new `nix/modules/hive-network.nix` with `services.hyperhive.network.*`
  options (enable + bridgeName + bridgeIp + bridgePrefixLength +
  upstreamDns). Default off. Imported from `hive-c0re.nix`.
- bridge interface via `networking.bridges` (no slave NICs at v1;
  per-agent veth pairs attach once #14 lands).
- bridge IP assigned via `networking.interfaces`.
- `networking.firewall.interfaces.<bridge>.allowed{UDP,TCP}Ports =
  [ 53 ]` opens the resolver on the bridge interface only —
  other interfaces stay closed.
- dnsmasq config added to the existing `hive-gateway` container
  (mara on #805:10957: "put the resolver into the gateway container").
  Listens only on `bridgeName` + `lo`; authoritative for
  `<hive-domain>`, `forge.<hive>`, `matrix.<hive>` answering with
  the bridge IP; forwards everything else to upstream.
  `resolveLocalQueries = false` keeps the gateway container's own
  resolver untouched.

Asserts `services.hyperhive.domain != null` + `gateway.enable =
true` — both required for the resolver to be meaningful.

Docs: new `docs/network.md` covering v1 vs v2 split, container shape
rationale, default addressing, resolver behaviour, firewall posture.

`nix flake check` clean.
This commit is contained in:
atlas 2026-05-31 16:50:54 +02:00 • committed by mara
commit ed3b9d853e
4 changed files with 301 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

113
docs/network.md Normal file
View file

@ -0,0 +1,113 @@
# hive-network
Host-side bridge + per-agent DNS resolver — the foundation that
makes [`#14` container netns isolation](http://localhost:3000/hyperhive/hyperhive/issues/14)
safe to land. Configured via `services.hyperhive.network.*`; off by
default during rollout.
## Why ship before #14
Mara on #805#issuecomment-11541: "we need it before #14 so nothing
breaks when we switch over". If netns isolation lands first, agent
containers lose `/etc/resolv.conf` propagation from the host and DNS
breaks until a separate resolver is up. Inverting the sequence —
bridge + dnsmasq first, netns flip second — makes the flag day
boring: the resolver endpoint is already live, agents just discover
it via veth instead of shared netns.
## v1 vs v2
| feature | v1 (this PR) | v2 (after #14) |
|---|---|---|
| bridge interface | created on host, no slave NICs | per-agent veth pairs attach |
| dnsmasq binding | bridge IP (reachable via host loopback in shared netns) | bridge IP (reachable via veth in private netns) |
| agent container netns | shared host | private |
| agent `/etc/resolv.conf` | unchanged (host DNS) | `nameserver <bridge-ip>` |
| `address` rules target | `<bridge-ip>` (works in both modes) | unchanged from v1 |
The `address` rules ship pointing at the bridge IP from v1 so the
DNS contract is fixed before any container actually depends on it
— minimises the things that flip on #14 day.
## Container shape (where dnsmasq lives)
Co-located in the existing `hive-gateway` container per mara on
#805:10957 — single front-door for both DNS and HTTP, saves a
sibling container, single systemd-unit / state surface to monitor.
The gateway shares host netns (`privateNetwork = false`) so
dnsmasq's `bind-interfaces` listener on `bridgeIp` works without
any veth gymnastics today; when #14 flips agent containers to
private netns the binding doesn't change (it's still on the host's
bridge interface).
## Configuration
```nix
{
services.hyperhive = {
enable = true;
domain = "darkest.space";
network.enable = true; # opt in to bridge + DNS
network.bridgeIp = "10.42.0.1"; # default
network.upstreamDns = [ # default Cloudflare + Quad9
"1.1.1.1"
"9.9.9.9"
];
};
}
```
Asserts `services.hyperhive.domain != null` (resolver needs a domain
to be authoritative for) + `services.hyperhive.gateway.enable =
true` (resolver lives in the gateway container).
## Bridge addressing
Default subnet is `10.42.0.0/24`, host-side gateway at `10.42.0.1`.
RFC 1918 space, unlikely to clash with operator's existing setup;
override `bridgeIp` + `bridgePrefixLength` if a different range is
already in use. `/24` gives 254 usable per-agent addresses — enough
for any single-host hive; bigger swarms or tighter addressing
schemes pick their own.
## Resolver behaviour
dnsmasq is **authoritative** for the hive's own zones — answers
`<hive-domain>`, `forge.<hive-domain>`, `matrix.<hive-domain>`
queries with the bridge IP (where nginx is reachable). Everything
else gets forwarded to `upstreamDns`. Containers don't need to know
the upstream — they query the bridge IP and dnsmasq does the right
thing per-name.
`bind-interfaces` + `interface = [ bridgeName "lo" ]` means the
listener only accepts queries from the bridge interface (plus lo for
container health-checks). External hosts can't reach it — no
DNS-amplification surface even when the operator opens port 80 for
gateway HTTP.
`resolveLocalQueries = false` keeps dnsmasq out of the host's own
resolution stack — the host's resolver (systemd-resolved, plain
glibc nss, dnscrypt-proxy, etc.) keeps doing whatever the operator
configured. The hive resolver is purely for inbound queries from
agent containers.
## Firewall posture
`networking.firewall.interfaces.<bridge>.allowedUDPPorts = [ 53 ]`
+ `allowedTCPPorts = [ 53 ]` opens the resolver on the bridge
interface only. Other interfaces stay closed. The hive resolver
isn't an external-facing service.
## Sequencing history
- mara on #805 (`comment-10957`): "can we put the resolver into the
gateway container?" — yes, this v1 does that.
- mara on #805 (`comment-11541`): "we need it before #14 so nothing
breaks when we switch over" — flipped the dependency direction;
v1 ships now, #14 flips containers later.
## Cross-references
- Issue [#805](http://localhost:3000/hyperhive/hyperhive/issues/805) — DNS resolver tracking
- Issue [#14](http://localhost:3000/hyperhive/hyperhive/issues/14) — netns isolation (downstream consumer)
- `docs/gateway.md` — vhost map + the gateway container's other duties

1
nix/modules/hive-c0re.nix
View file

@ -25,6 +25,7 @@ in
./hive-forge.nix
./hive-gateway.nix
./hive-matrix.nix
./hive-network.nix
];
# Top-level hyperhive enable flag. When true, automatically enables

47
nix/modules/hive-gateway.nix
View file

@ -9,6 +9,7 @@ let
hyperhiveDomain = config.services.hyperhive.domain;
matrixCfg = config.services.hyperhive.matrix;
forgeCfg = config.services.hyperhive.forge;
networkCfg = config.services.hyperhive.network;
# Per-agent port table for `/agent/<name>/` routing. C0re writes
# this JSON on every topology change; gateway reads at deploy time.
@ -551,6 +552,52 @@ in
};
};
};
# Hive-internal DNS resolver (#805 v1). Co-located in the
# gateway container per mara's call (#805:10957) — single
# front-door for both DNS and HTTP, saves a sibling
# container. Listens on the bridge interface from
# `services.hyperhive.network`; authoritative for the hive
# domain + sub-domains, forwards everything else upstream.
# No-op when `network.enable = false`.
services.dnsmasq = lib.mkIf networkCfg.enable {
enable = true;
# Don't substitute the container's /etc/resolv.conf —
# the gateway uses the host's resolver for its own
# outbound traffic; dnsmasq is purely for incoming
# queries from agent containers.
resolveLocalQueries = false;
settings = {
# Bind only on the bridge interface (and lo for
# health-checks). Outside hosts can't even see the
# listener.
interface = [
networkCfg.bridgeName
"lo"
];
bind-interfaces = true;
port = 53;
# Don't read /etc/resolv.conf — we control upstream
# explicitly to dodge dependency on the gateway
# container's own resolver state.
no-resolv = true;
server = networkCfg.upstreamDns;
# Hive authoritative records — answer queries for the
# hive domain + its sub-domains with the bridge IP
# (where nginx is reachable from container netns once
# #14 lands; today it's the host loopback alias and
# works in either shape).
address = [
"/${hyperhiveDomain}/${networkCfg.bridgeIp}"
]
++ lib.optional (
(forgeCfg.enable or false) && (forgeCfg.behindGateway or false)
) "/${forgeCfg.domain}/${networkCfg.bridgeIp}"
++ lib.optional (
matrixCfg.enable && matrixCfg.gatewayHost != null
) "/${matrixCfg.gatewayHost}/${networkCfg.bridgeIp}";
};
};
};
};

140
nix/modules/hive-network.nix Normal file
View file

@ -0,0 +1,140 @@
{
lib,
config,
...
}:
let
cfg = config.services.hyperhive.network;
in
{
# Hive-internal network — host-side bridge + per-agent DNS resolver
# (#805, prereq for #14 netns isolation). Containers stay on shared
# host netns at v1 — this module just stands the bridge + resolver
# up so the endpoint is in place before #14 flips containers to
# private netns (mara on #805: "we need it before #14 so nothing
# breaks when we switch over"). Full design: docs/network.md.
options.services.hyperhive.network = {
enable = lib.mkOption {
type = lib.types.bool;
default = false;
example = true;
description = ''
Stand up the hive-internal bridge + dnsmasq resolver.
Off by default while v1 phases in. When enabled:
a bridge interface (`bridgeName`) appears on the host with
`bridgeIp` assigned, and the hive-gateway container runs a
dnsmasq listening on that IP for `<hive-domain>` +
sub-domains. Agent containers still default to shared host
netns at v1 — the endpoint is up but only used once #14
lands and flips containers to a private netns + veth peer
on this bridge.
'';
};
bridgeName = lib.mkOption {
type = lib.types.str;
default = "hive-br0";
example = "h0";
description = ''
Name of the host-side bridge interface the hive uses for
inter-container traffic. Kept short so it survives the
IFNAMSIZ (15-char) cap, and prefixed so it's obviously
hive-managed in `ip link` output.
'';
};
bridgeIp = lib.mkOption {
type = lib.types.str;
default = "10.42.0.1";
example = "172.30.0.1";
description = ''
IPv4 address assigned to the bridge interface on the host
side. Becomes the DNS server address agents point at (and
the upstream the gateway proxies to once netns isolation
lands). Default `10.42.0.1` is in RFC 1918 space and
unlikely to clash with operator's existing setup; override
if a different range is already in use.
'';
};
bridgePrefixLength = lib.mkOption {
type = lib.types.int;
default = 24;
example = 16;
description = ''
Netmask prefix length for the bridge subnet. Default `/24`
gives 254 usable per-agent addresses, enough for any
single-host hive. Operator with a larger swarm or a tighter
addressing scheme overrides.
'';
};
upstreamDns = lib.mkOption {
type = lib.types.listOf lib.types.str;
default = [
"1.1.1.1"
"9.9.9.9"
];
example = [
"192.168.1.1"
"8.8.8.8"
];
description = ''
Upstream DNS servers dnsmasq forwards non-hive queries to.
Defaults to Cloudflare + Quad9. Override for operators on
private networks who need a specific resolver (corporate
DNS, pi-hole, etc.). The hive resolver itself stays
authoritative for `<hive-domain>` and its sub-domains
regardless of upstream choice.
'';
};
};
config = lib.mkIf cfg.enable {
assertions = [
{
assertion = config.services.hyperhive.domain != null;
message = ''
services.hyperhive.network.enable = true requires
services.hyperhive.domain to be set — the resolver needs a
domain to be authoritative for. Either pin a hostname
(`services.hyperhive.domain = "example.com";`) or leave
`network.enable` at its default of false.
'';
}
{
assertion = config.services.hyperhive.gateway.enable;
message = ''
services.hyperhive.network.enable = true requires
services.hyperhive.gateway.enable = true — the dnsmasq
resolver runs inside the hive-gateway container (single
front-door for both DNS and HTTP). Enable the gateway or
leave `network.enable` at its default of false.
'';
}
];
# Bridge interface on the host. Empty interfaces list = purely
# virtual bridge (no slave NICs attached). Per-agent veth pairs
# will join this bridge once #14 lands; at v1 it stands alone.
networking.bridges.${cfg.bridgeName}.interfaces = [ ];
# Host-side IP assignment on the bridge. This is what dnsmasq
# (inside the gateway container, shared host netns) binds on.
networking.interfaces.${cfg.bridgeName}.ipv4.addresses = [
{
address = cfg.bridgeIp;
prefixLength = cfg.bridgePrefixLength;
}
];
# Open the resolver port in the host firewall for traffic from
# the bridge subnet only. Other interfaces stay closed —
# external DNS-amplification surface is not exposed.
networking.firewall.interfaces.${cfg.bridgeName} = {
allowedUDPPorts = [ 53 ];
allowedTCPPorts = [ 53 ];
};
};
}