2af8c2d17d
The gateway container's /etc/resolv.conf is a one-shot copy: nixos-container cps it in from the host in its start script, and nspawn's --resolv-conf=auto copies (not binds) for a writable host-netns container. systemd-nspawn(1) states the consequence outright — "no further propagation of configuration is generally done after the one-time early initialization (this is because the file is usually updated through copying and renaming)". dnsmasq has no explicit upstream and follows that file, so a host network change strands it on a resolver that no longer answers and every non-hive lookup from every agent hangs. Agents' own resolvers point at the static bridge IP and never go stale, which is why the symptom presents as "the gateway needs a kick". Add a host-side hive-gateway-resolv path unit watching /etc/resolv.conf. On change it machinectl copy-to's the file into the container and reloads dnsmasq — ExecReload is kill -HUP, so upstreams are re-read and the cache flushed without dropping anything; nginx never notices. - watched from the HOST: a rename on the host doesn't cross the nspawn mount namespace, so an in-container path unit can't see it (same reason c0re reloads nginx from the host side) - copy, not a file bind-mount: openresolv renames over the file, so a bind would pin the first inode forever — strictly worse than today - machinectl copy-to writes through the container's own mount namespace, so this holds regardless of how the container assembles /etc - armed Before=network-pre.target so the boot's first DHCP write is caught, and re-run on gateway start for changes made while it was down - a host file with no nameserver line is skipped, not pushed, so a mid-rewrite snapshot can't blank hive DNS - deliberately no fallback server=: dnsmasq queries all known upstreams in parallel, so a hardcoded public resolver would take a share of normal traffic rather than only covering the gap
67 lines
3.1 KiB
Nix
67 lines
3.1 KiB
Nix
# Hive-internal DNS resolver + DHCP, co-located in the gateway
|
|
# container — single front-door for both DNS and HTTP, saves a
|
|
# sibling container. Listens on the bridge interface from
|
|
# `services.hyperhive.network`; authoritative for the hive domain +
|
|
# sub-domains, forwards everything else upstream. Returns the
|
|
# `services.dnsmasq` value for the container config (see
|
|
# ./default.nix); the DHCP pool bounds are computed by hive-network.
|
|
{
|
|
lib,
|
|
networkCfg,
|
|
forgeCfg,
|
|
matrixCfg,
|
|
hyperhiveDomain,
|
|
}:
|
|
{
|
|
enable = true;
|
|
# Don't substitute the container's /etc/resolv.conf — the gateway
|
|
# uses the host's resolver for its own outbound traffic; dnsmasq is
|
|
# purely for incoming queries from agent containers.
|
|
resolveLocalQueries = false;
|
|
settings = {
|
|
# Bind only on the bridge interface (and lo for health-checks).
|
|
# Outside hosts can't even see the listener.
|
|
interface = [
|
|
networkCfg.bridgeName
|
|
"lo"
|
|
];
|
|
bind-interfaces = true;
|
|
port = 53;
|
|
# Hive authoritative records — answer queries for the hive domain
|
|
# + its sub-domains with the bridge IP, where nginx is reachable
|
|
# from every container netns.
|
|
#
|
|
# The forge / matrix entries are redundant in the common case
|
|
# where `forge.domain` / `matrix.gatewayHost` are sub-domains of
|
|
# `hyperhive.domain` — dnsmasq's `/<domain>/` rule already matches
|
|
# sub-domains. Kept explicit because operators can override either
|
|
# to a cross-domain hostname (e.g. `forge.domain =
|
|
# "git.example.com"`); listing them explicitly keeps that case
|
|
# routed without needing an extra config block.
|
|
address = [
|
|
"/${hyperhiveDomain}/${networkCfg.bridgeIp}"
|
|
]
|
|
++ lib.optional ((forgeCfg.behindGateway or false)) "/${forgeCfg.domain}/${networkCfg.bridgeIp}"
|
|
++ lib.optional (
|
|
matrixCfg.enable && matrixCfg.gatewayHost != null
|
|
) "/${matrixCfg.gatewayHost}/${networkCfg.bridgeIp}";
|
|
# DHCP pool covering all usable host addresses on the bridge
|
|
# subnet — bounds computed by hive-network.nix from
|
|
# bridgeIp/bridgePrefixLength. All containers (agents and service
|
|
# containers such as hive-ci) receive their IPs dynamically.
|
|
dhcp-range = "${networkCfg.dhcpRangeStart},${networkCfg.dhcpRangeEnd},1h";
|
|
dhcp-leasefile = "/var/lib/dnsmasq/dnsmasq.leases";
|
|
# No explicit upstream: non-hive queries follow dnsmasq's
|
|
# resolv.conf default — the gateway container's `/etc/resolv.conf`,
|
|
# which nixos-container copies from the host at every start, so the
|
|
# hive always uses the host's resolvers. resolvconf is disabled in
|
|
# the container (see ./default.nix) so nothing regenerates that
|
|
# copy; the host-side `hive-gateway-resolv` path unit (also in
|
|
# ./default.nix) pushes in a fresh copy and reloads dnsmasq whenever
|
|
# the host's resolvers change, so the copy can't go stale under a
|
|
# network switch. Deliberately no fallback `server=`: dnsmasq queries
|
|
# all known upstreams in parallel, so a hardcoded public resolver
|
|
# would take a share of *normal* traffic, not just fill in when the
|
|
# host file is empty.
|
|
};
|
|
}
|