fix(gateway): resync the gateway's resolv.conf when the host's changes
The gateway container's /etc/resolv.conf is a one-shot copy: nixos-container cps it in from the host in its start script, and nspawn's --resolv-conf=auto copies (not binds) for a writable host-netns container. systemd-nspawn(1) states the consequence outright — "no further propagation of configuration is generally done after the one-time early initialization (this is because the file is usually updated through copying and renaming)". dnsmasq has no explicit upstream and follows that file, so a host network change strands it on a resolver that no longer answers and every non-hive lookup from every agent hangs. Agents' own resolvers point at the static bridge IP and never go stale, which is why the symptom presents as "the gateway needs a kick". Add a host-side hive-gateway-resolv path unit watching /etc/resolv.conf. On change it machinectl copy-to's the file into the container and reloads dnsmasq — ExecReload is kill -HUP, so upstreams are re-read and the cache flushed without dropping anything; nginx never notices. - watched from the HOST: a rename on the host doesn't cross the nspawn mount namespace, so an in-container path unit can't see it (same reason c0re reloads nginx from the host side) - copy, not a file bind-mount: openresolv renames over the file, so a bind would pin the first inode forever — strictly worse than today - machinectl copy-to writes through the container's own mount namespace, so this holds regardless of how the container assembles /etc - armed Before=network-pre.target so the boot's first DHCP write is caught, and re-run on gateway start for changes made while it was down - a host file with no nameserver line is skipped, not pushed, so a mid-rewrite snapshot can't blank hive DNS - deliberately no fallback server=: dnsmasq queries all known upstreams in parallel, so a hardcoded public resolver would take a share of normal traffic rather than only covering the gap
This commit is contained in:
parent
7320e2ba3d
commit
2af8c2d17d
3 changed files with 157 additions and 5 deletions
|
|
@ -112,11 +112,48 @@ dnsmasq is **authoritative** for the hive's own zones — answers
|
|||
queries with the bridge IP (where nginx is reachable). Everything
|
||||
else is forwarded to the host's own resolvers: dnsmasq reads the
|
||||
gateway container's `/etc/resolv.conf`, the host copy nixos-container
|
||||
makes at each container start — a host resolver change is picked up
|
||||
on the next gateway restart. Containers don't need to know the
|
||||
makes at each container start. Containers don't need to know the
|
||||
upstream — they query the bridge IP and dnsmasq does the right thing
|
||||
per-name.
|
||||
|
||||
That copy is one-shot — systemd-nspawn(1) is explicit that nothing
|
||||
propagates into it after early init, because resolv.conf is normally
|
||||
updated by rename rather than in place. Left alone, a host network
|
||||
change (new router, new lease, laptop moving networks) would strand
|
||||
dnsmasq on a resolver that no longer answers, and every non-hive
|
||||
lookup from every agent would hang until someone restarted the
|
||||
gateway. The host-side **`hive-gateway-resolv`** path unit closes
|
||||
that: it watches `/etc/resolv.conf`, `machinectl copy-to`s it into
|
||||
the container, and reloads dnsmasq (`SIGHUP` — re-read upstreams +
|
||||
flush cache, nothing dropped). The watch is armed before
|
||||
`network-pre.target` so the boot's first DHCP write is caught as well,
|
||||
and the sync also runs once per gateway start to pick up a resolver
|
||||
change that happened while the container was down. A host file with no
|
||||
`nameserver` line is
|
||||
skipped rather than pushed, so a mid-rewrite snapshot can't blank the
|
||||
hive's DNS. There is deliberately no fallback `server=`: dnsmasq
|
||||
queries all known upstreams in parallel, so a hardcoded public
|
||||
resolver would take a share of normal traffic, not just cover the gap.
|
||||
|
||||
Two alternatives that look simpler and aren't:
|
||||
|
||||
- **A path unit inside the container.** The host replaces
|
||||
`/etc/resolv.conf` by rename, and that `IN_MOVED_TO` doesn't cross
|
||||
the nspawn mount namespace — the same reason hive-c0re reloads nginx
|
||||
from the host side after each `agents.conf` write.
|
||||
- **Bind-mounting the host's `/etc/resolv.conf` into the container.**
|
||||
openresolv writes a temp file and renames over the target, so the
|
||||
bind mount would pin the *first* inode for the container's whole
|
||||
lifetime — strictly worse than the copy, which at least a restart
|
||||
clears. (Reachability is not the problem here: the gateway runs with
|
||||
`privateNetwork = false`, so it shares the host's netns and can reach
|
||||
anything the host can.)
|
||||
|
||||
`machinectl copy-to` is used rather than writing to the container's
|
||||
rootfs from the host, so the push goes through the container's own
|
||||
mount namespace and stays correct if `/etc` is ever assembled
|
||||
differently (e.g. `system.etc.overlay`).
|
||||
|
||||
`bind-interfaces` + `interface = [ bridgeName "lo" ]` means the
|
||||
listener only accepts queries from the bridge interface (plus lo for
|
||||
container health-checks). External hosts can't reach it — no
|
||||
|
|
|
|||
Loading…
Reference in a new issue