hive-forge: resolve through the hive's dnsmasq
The forge container used the host's resolvers, where the swarm domain has no records — so every outbound call to a swarm name failed with "no such host". Forgejo's webhook deliveries died there: knowledge, config-pr and vcs-activity alike, which is why the swarm-controller's receiver has never logged a verified delivery and knowledge propagation has been running on its hourly fallback. An earlier fix taught the container authelia's name with a `networking.hosts` entry. That covered one name and left the rest, so use the resolver every other service container already uses and drop the override — hive-matrix records why a second answer that can disagree with the first is worse than none. Closes #3824
This commit is contained in:
parent
36082f991f
commit
d6f3af60ae
3 changed files with 36 additions and 33 deletions
|
|
@ -55,9 +55,12 @@ The flows, end to end:
|
||||||
|
|
||||||
- **DHCP** — agent `dhcpcd` broadcasts on `eth0` → veth → bridge →
|
- **DHCP** — agent `dhcpcd` broadcasts on `eth0` → veth → bridge →
|
||||||
host firewall (udp 67 hole) → dnsmasq pool → lease + router option.
|
host firewall (udp 67 hole) → dnsmasq pool → lease + router option.
|
||||||
- **DNS** — agents query `bridgeIp:53`; hive zones are answered
|
- **DNS** — agents and the service containers query `bridgeIp:53`; hive
|
||||||
authoritatively with the bridge IP, everything else forwards to the
|
zones are answered authoritatively with the bridge IP, everything else
|
||||||
host's resolvers (see *Resolver behaviour* below).
|
forwards to the host's resolvers (see *Resolver behaviour* below). Each
|
||||||
|
container points its own `resolv.conf` there, and one that instead
|
||||||
|
inherits the host's resolves no swarm name at all — those records exist
|
||||||
|
only on the bridge.
|
||||||
- **HTTP** — `forge.` and `chat.` (under `swarm.domain`) plus the hive's
|
- **HTTP** — `forge.` and `chat.` (under `swarm.domain`) plus the hive's
|
||||||
own dashboard name resolve to the bridge IP, land on nginx
|
own dashboard name resolve to the bridge IP, land on nginx
|
||||||
`:80`/`:443`, and proxy to forgejo
|
`:80`/`:443`, and proxy to forgejo
|
||||||
|
|
|
||||||
|
|
@ -10,6 +10,7 @@ let
|
||||||
hyperhiveDomain = config.services.hyperhive.domain;
|
hyperhiveDomain = config.services.hyperhive.domain;
|
||||||
swarmDomain = config.services.hyperhive.swarm.domain;
|
swarmDomain = config.services.hyperhive.swarm.domain;
|
||||||
tlsCfg = config.services.hyperhive.deploy.hive-controller.tls;
|
tlsCfg = config.services.hyperhive.deploy.hive-controller.tls;
|
||||||
|
networkCfg = config.services.hyperhive.network;
|
||||||
|
|
||||||
# Forgejo's name for the login source. A constant, not an option: it
|
# Forgejo's name for the login source. A constant, not an option: it
|
||||||
# is the key this module's own idempotency check looks up, so making
|
# is the key this module's own idempotency check looks up, so making
|
||||||
|
|
@ -669,6 +670,24 @@ in
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
imports = [
|
imports = [
|
||||||
|
# Ask the hive's dnsmasq, not the host's resolvers. The swarm
|
||||||
|
# names — the controller's webhook endpoint, the SSO issuer —
|
||||||
|
# resolve to the bridge and have no public records, so a
|
||||||
|
# container using the host's resolvers gets "no such host" for
|
||||||
|
# every one of them while the same name resolves fine one
|
||||||
|
# container over.
|
||||||
|
#
|
||||||
|
# Both units that make an outbound call are consumers, for the
|
||||||
|
# same reason the trust bundle below names both: an unordered
|
||||||
|
# resolver write is a race that only shows up on a cold boot.
|
||||||
|
(import ./../swarm-container-resolver.nix {
|
||||||
|
inherit (networkCfg) bridgeIp;
|
||||||
|
dnsConsumers = [
|
||||||
|
"forgejo.service"
|
||||||
|
]
|
||||||
|
++ lib.optional ssoLocal "forgejo-sso-source.service";
|
||||||
|
})
|
||||||
|
|
||||||
# Forgejo is Go, and `SSL_CERT_FILE` *replaces* the default store
|
# Forgejo is Go, and `SSL_CERT_FILE` *replaces* the default store
|
||||||
# rather than adding to it — so it needs the system CAs and the
|
# rather than adding to it — so it needs the system CAs and the
|
||||||
# hive CA concatenated, not the CA alone, or public mirror fetches
|
# hive CA concatenated, not the CA alone, or public mirror fetches
|
||||||
|
|
@ -699,28 +718,11 @@ in
|
||||||
# bridge DHCP/DNS holes and agent NAT. The host firewall owns
|
# bridge DHCP/DNS holes and agent NAT. The host firewall owns
|
||||||
# all filtering; never run one in here.
|
# all filtering; never run one in here.
|
||||||
networking.firewall.enable = false;
|
networking.firewall.enable = false;
|
||||||
|
# resolvconf stays off because the resolver unit imported above
|
||||||
# Teach this container the SSO name, because nothing else will.
|
# owns /etc/resolv.conf. Leaving it on would let host-tracking
|
||||||
#
|
# regenerate the file empty, since the host's copy doesn't cross
|
||||||
# The hive's dnsmasq is authoritative for the swarm service
|
# the boundary after start.
|
||||||
# names, but only containers whose resolv.conf points at the
|
networking.resolvconf.enable = lib.mkForce false;
|
||||||
# bridge ask it — agent containers do, by an explicit unit
|
|
||||||
# (`nix/agent-modules/network.nix`) written for exactly this
|
|
||||||
# reason. This container resolves through the host's resolvers
|
|
||||||
# instead, and the swarm domain has no public records, so
|
|
||||||
# `admin auth add-oauth` fails at discovery with "no such
|
|
||||||
# host" while the same name resolves fine one container over.
|
|
||||||
#
|
|
||||||
# `127.0.0.1` rather than the bridge IP: sharing the host netns
|
|
||||||
# means loopback IS the host, where nginx serves this vhost.
|
|
||||||
# TLS still validates — the CA trust bundle is bind-mounted
|
|
||||||
# above, and the leaf covers this name.
|
|
||||||
#
|
|
||||||
# Only when THIS host runs authelia. With a remote provider the
|
|
||||||
# name belongs to another machine and must resolve normally.
|
|
||||||
networking.hosts = lib.mkIf ssoLocal {
|
|
||||||
"127.0.0.1" = [ autheliaCfg.domain ];
|
|
||||||
};
|
|
||||||
|
|
||||||
services.forgejo = {
|
services.forgejo = {
|
||||||
enable = true;
|
enable = true;
|
||||||
|
|
|
||||||
|
|
@ -752,14 +752,12 @@ in
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|
||||||
# ⚠️ Deliberately NO `networking.hosts` entry for authelia's name, and
|
# ⚠️ Deliberately NO `networking.hosts` entry for authelia's name. This
|
||||||
# the difference from hive-forge (which needs one) is worth stating:
|
# container resolves through the hive's dnsmasq at `bridgeIp` (see the
|
||||||
# that container resolves through the host's resolvers, where the swarm
|
# static resolv.conf below), and every `gateway.localNames` entry —
|
||||||
# domain has no records. This one resolves through the hive's dnsmasq
|
# authelia's domain among them — is already mapped there. Adding a
|
||||||
# at `bridgeIp` (see the static resolv.conf below), and every
|
# loopback override would only create a second answer that can
|
||||||
# `gateway.localNames` entry — authelia's domain among them — is
|
# disagree with the first.
|
||||||
# already mapped there. Adding a loopback override would only create a
|
|
||||||
# second answer that can disagree with the first.
|
|
||||||
|
|
||||||
# Activation-time token generation — without this the bind-mount
|
# Activation-time token generation — without this the bind-mount
|
||||||
# would hand tuwunel an empty file on first boot and break every
|
# would hand tuwunel an empty file on first boot and break every
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue