nix: give the gateway, resolver and bridge their own enable
`services.hyperhive.gateway.enable`, `gateway.dns.enable` and `network.enable` replace the `hyperhive.enable` gate on all three. Each defaults to false; the modules that need one assert it with `mkDefault true` from inside the guard their own deployment already carries, and `swarm-required-services.nix` — the module that owns what the swarm-services toggle implies — asserts all three explicitly. hive-c0re asserts all three unconditionally, so an ordinary hive keeps getting them with no opt-in: it is the host's only knowledge that agent containers exist. The resolver moves to its own `hive-gateway/dns.nix` so it can be gated without reindenting the nginx half of the module. Reinstates `network.enable`, dropping its `mkRemovedOptionModule` shim. A config still carrying `network.enable = false` from before the removal now switches the bridge off instead of failing eval. Also deletes a duplicate `centralToggleOff` fixture in nix/module-eval.nix. Two sibling slices added it independently (c5f60fd5,ce3b3d94); the merge was textually clean and left `main` failing to evaluate at all, so this file could not be gated without removing one.
This commit is contained in:
parent
a4e4016214
commit
d8e26a17bb
21 changed files with 249 additions and 75 deletions
|
|
@ -1,6 +1,6 @@
|
||||||
# hive-gateway
|
# hive-gateway
|
||||||
|
|
||||||
Single nginx in front of every hyperhive web surface. Runs on the **host**, next to hive-c0re, rather than in its own container: it shares the host netns anyway (see [Vhost map](#vhost-map) below), so containerizing it would buy no network isolation while costing a resolv.conf sync, a machine-bus reload, and three bind mounts. System-config (not meta-flake managed). Configured via `services.hyperhive.gateway.*` + per-subsystem opt-in flags in `services.hyperhive.{forge,matrix,...}`.
|
Single nginx in front of every hyperhive web surface. Runs on the **host**, next to hive-c0re, rather than in its own container: it shares the host netns anyway (see [Vhost map](#vhost-map) below), so containerizing it would buy no network isolation while costing a resolv.conf sync, a machine-bus reload, and three bind mounts. System-config (not meta-flake managed). Configured via `services.hyperhive.gateway.*` + per-subsystem opt-in flags in `services.hyperhive.{forge,matrix,...}`. `gateway.enable` and `gateway.dns.enable` are asserted by the modules that need them, so a host serving a vhost or resolving hive names gets them without an opt-in.
|
||||||
|
|
||||||
## Vhost map
|
## Vhost map
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,12 +1,12 @@
|
||||||
# hive-network
|
# hive-network
|
||||||
|
|
||||||
Host-side bridge + per-agent private-netns isolation — always on
|
Host-side bridge + per-agent private-netns isolation, up on a host
|
||||||
whenever hyperhive is enabled. Configured via
|
where something attaches to it (`services.hyperhive.network.enable`,
|
||||||
`services.hyperhive.network.*`.
|
asserted by the modules that need it rather than set by hand).
|
||||||
|
Configured via `services.hyperhive.network.*`.
|
||||||
|
|
||||||
> Isolation is the only mode — there is no shared-netns fallback. The
|
> Isolation is the only mode — there is no shared-netns fallback. The
|
||||||
> former `services.hyperhive.network.enable`,
|
> former `services.hyperhive.network.isolateContainers` and
|
||||||
> `services.hyperhive.network.isolateContainers` and
|
|
||||||
> `services.hyperhive.network.upstreamDns` options were removed; a
|
> `services.hyperhive.network.upstreamDns` options were removed; a
|
||||||
> config that still sets one fails eval with a removal message.
|
> config that still sets one fails eval with a removal message.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -158,6 +158,13 @@ in
|
||||||
];
|
];
|
||||||
|
|
||||||
config = lib.mkIf cfg.enable {
|
config = lib.mkIf cfg.enable {
|
||||||
|
# This module is the host's only knowledge that agent containers
|
||||||
|
# exist: they hang off the bridge, resolve through dnsmasq, and their
|
||||||
|
# UIs — plus the operator dashboard — are served by the gateway.
|
||||||
|
services.hyperhive.gateway.enable = lib.mkDefault true;
|
||||||
|
services.hyperhive.gateway.dns.enable = lib.mkDefault true;
|
||||||
|
services.hyperhive.network.enable = lib.mkDefault true;
|
||||||
|
|
||||||
# The daemon that owns every container on this hive, and the helper it
|
# The daemon that owns every container on this hive, and the helper it
|
||||||
# delegates its root operations to. An agent asking why a container did
|
# delegates its root operations to. An agent asking why a container did
|
||||||
# not come up is asking about one of these two.
|
# not come up is asking about one of these two.
|
||||||
|
|
|
||||||
|
|
@ -177,6 +177,13 @@ in
|
||||||
# name follows that attr rather than `cfg.name`.
|
# name follows that attr rather than `cfg.name`.
|
||||||
services.hyperhive.swarm.otel.journaldUnits = [ "gitea-runner-hive" ];
|
services.hyperhive.swarm.otel.journaldUnits = [ "gitea-runner-hive" ];
|
||||||
|
|
||||||
|
# The runner container has a private netns of its own, so it is
|
||||||
|
# attached to the bridge and reaches the forge by name through the
|
||||||
|
# gateway vhost the assertion above already insists on.
|
||||||
|
services.hyperhive.network.enable = lib.mkDefault true;
|
||||||
|
services.hyperhive.gateway.enable = lib.mkDefault true;
|
||||||
|
services.hyperhive.gateway.dns.enable = lib.mkDefault true;
|
||||||
|
|
||||||
# Create /run/hive-ci/ on the host and seed runner-token with a
|
# Create /run/hive-ci/ on the host and seed runner-token with a
|
||||||
# placeholder. The container bind-mounts this file read-only; hive-c0re
|
# placeholder. The container bind-mounts this file read-only; hive-c0re
|
||||||
# (via hive-priv's RegisterCiRunner) overwrites it with the real
|
# (via hive-priv's RegisterCiRunner) overwrites it with the real
|
||||||
|
|
|
||||||
|
|
@ -455,6 +455,11 @@ in
|
||||||
# fronts forgejo themselves, so this hive must neither claim the
|
# fronts forgejo themselves, so this hive must neither claim the
|
||||||
# vhost nor answer DNS for it.
|
# vhost nor answer DNS for it.
|
||||||
services.hyperhive.gateway.localNames = lib.optional deployCfg.forgejo.behindGateway cfg.domain;
|
services.hyperhive.gateway.localNames = lib.optional deployCfg.forgejo.behindGateway cfg.domain;
|
||||||
|
services.hyperhive.gateway.enable = lib.mkIf deployCfg.forgejo.behindGateway (lib.mkDefault true);
|
||||||
|
|
||||||
|
# Not conditional on `behindGateway`: the forge container resolves the
|
||||||
|
# rest of the hive through dnsmasq whoever fronts it.
|
||||||
|
services.hyperhive.gateway.dns.enable = lib.mkDefault true;
|
||||||
|
|
||||||
# This swarm-ui quick-links entry, same `behindGateway` guard as the
|
# This swarm-ui quick-links entry, same `behindGateway` guard as the
|
||||||
# vhost/DNS name above — with it off, this host doesn't actually
|
# vhost/DNS name above — with it off, this host doesn't actually
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,9 @@
|
||||||
# Full vhost map + discovery flow + design rationale in
|
# Full vhost map + discovery flow + design rationale in
|
||||||
# `docs/networking/gateway.md`. Layout: ./options.nix (option declarations),
|
# `docs/networking/gateway.md`. Layout: ./options.nix (option declarations),
|
||||||
# ./vhosts.nix (the nginx virtual-host tree), ./error-pages.nix
|
# ./vhosts.nix (the nginx virtual-host tree), ./error-pages.nix
|
||||||
# (styled static pages), ./dnsmasq.nix (resolver + DHCP config).
|
# (styled static pages), ./dns.nix + ./dnsmasq.nix (the resolver, which
|
||||||
|
# has an enable of its own because a host can need hive names to resolve
|
||||||
|
# without serving a vhost).
|
||||||
{
|
{
|
||||||
pkgs,
|
pkgs,
|
||||||
lib,
|
lib,
|
||||||
|
|
@ -23,7 +25,6 @@ let
|
||||||
swarmServiceDomains = config.services.hyperhive.swarm.serviceDomains;
|
swarmServiceDomains = config.services.hyperhive.swarm.serviceDomains;
|
||||||
matrixCfg = config.services.hyperhive.swarm.matrix;
|
matrixCfg = config.services.hyperhive.swarm.matrix;
|
||||||
matrixDeployCfg = config.services.hyperhive.deploy.matrix;
|
matrixDeployCfg = config.services.hyperhive.deploy.matrix;
|
||||||
networkCfg = config.services.hyperhive.network;
|
|
||||||
|
|
||||||
# Every vhost claiming `default_server`, ours and the operator's
|
# Every vhost claiming `default_server`, ours and the operator's
|
||||||
# alike. Computed once so the assertion below and the message it
|
# alike. Computed once so the assertion below and the message it
|
||||||
|
|
@ -113,9 +114,12 @@ let
|
||||||
};
|
};
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
imports = [ ./options.nix ];
|
imports = [
|
||||||
|
./options.nix
|
||||||
|
./dns.nix
|
||||||
|
];
|
||||||
|
|
||||||
config = lib.mkIf config.services.hyperhive.enable {
|
config = lib.mkIf cfg.enable {
|
||||||
# Publish the kit. Defined here rather than as an option `default`
|
# Publish the kit. Defined here rather than as an option `default`
|
||||||
# so it stays a plain value computed once from resolved cert paths —
|
# so it stays a plain value computed once from resolved cert paths —
|
||||||
# `tlsFor` closes over `svcCert`/`svcKey`, which are derived in this
|
# `tlsFor` closes over `svcCert`/`svcKey`, which are derived in this
|
||||||
|
|
@ -126,13 +130,7 @@ in
|
||||||
# is the one unit that can say a service was unreachable rather than
|
# is the one unit that can say a service was unreachable rather than
|
||||||
# merely quiet. Named even on hives that run no swarm collector: the
|
# merely quiet. Named even on hives that run no swarm collector: the
|
||||||
# option is inert unless one is collecting on this host.
|
# option is inert unless one is collecting on this host.
|
||||||
#
|
services.hyperhive.swarm.otel.journaldUnits = [ "nginx" ];
|
||||||
# dnsmasq alongside it for the same reason one level down: a name that
|
|
||||||
# stops resolving presents as every client timing out at once.
|
|
||||||
services.hyperhive.swarm.otel.journaldUnits = [
|
|
||||||
"nginx"
|
|
||||||
"dnsmasq"
|
|
||||||
];
|
|
||||||
|
|
||||||
assertions = [
|
assertions = [
|
||||||
{
|
{
|
||||||
|
|
@ -255,32 +253,6 @@ in
|
||||||
"f /var/lib/hive-gateway/conf/gateway.htpasswd 0644 hive-core hive-core - -"
|
"f /var/lib/hive-gateway/conf/gateway.htpasswd 0644 hive-core hive-core - -"
|
||||||
];
|
];
|
||||||
|
|
||||||
# The host asks the hive's own resolver, at the BRIDGE IP.
|
|
||||||
#
|
|
||||||
# Every container inherits a COPY of this host's `/etc/resolv.conf`
|
|
||||||
# at start (`nixos-containers.nix`: `cp --remove-destination`, one
|
|
||||||
# shot, not a bind-mount) — so whatever address is written here is
|
|
||||||
# the address every container will try, in its own netns.
|
|
||||||
#
|
|
||||||
# 🚨 That is why this is the bridge IP and not `127.0.0.1`, and the
|
|
||||||
# distinction is load-bearing rather than stylistic:
|
|
||||||
#
|
|
||||||
# value host host-netns containers bridged containers
|
|
||||||
# 127.0.0.1 ok ok THEIR OWN loopback
|
|
||||||
# bridge IP ok ok ok
|
|
||||||
#
|
|
||||||
# dnsmasq binds both `lo` and the bridge (./dnsmasq.nix), so the
|
|
||||||
# bridge IP is reachable from the host too — it is the only value
|
|
||||||
# correct on both sides of a netns boundary. `resolveLocalQueries`
|
|
||||||
# publishes loopback by default, hence both overrides here; the
|
|
||||||
# flag stays on for its `resolv-file` plumbing, which is what keeps
|
|
||||||
# dnsmasq's own upstreams out of the file we are pointing at it.
|
|
||||||
#
|
|
||||||
# Cost, stated because it is real: the host's DNS now depends on
|
|
||||||
# dnsmasq being up. Every container already did.
|
|
||||||
networking.nameservers = lib.mkForce [ networkCfg.bridgeIp ];
|
|
||||||
networking.resolvconf.useLocalResolver = lib.mkForce false;
|
|
||||||
|
|
||||||
# ACME (Let's Encrypt) integration. nginx vhosts set
|
# ACME (Let's Encrypt) integration. nginx vhosts set
|
||||||
# `enableACME = true` via the vhost builder; this provides the
|
# `enableACME = true` via the vhost builder; this provides the
|
||||||
# shared ACME config (acceptTerms + email).
|
# shared ACME config (acceptTerms + email).
|
||||||
|
|
@ -423,18 +395,6 @@ in
|
||||||
# dir removes the need and the exposure together. Re-adding this line
|
# dir removes the need and the exposure together. Re-adding this line
|
||||||
# would restore both.
|
# would restore both.
|
||||||
|
|
||||||
# dnsmasq is a host service alongside nginx, so it reads the host's
|
|
||||||
# /etc/resolv.conf directly and picks up network changes as they
|
|
||||||
# happen — no copy to keep in sync.
|
|
||||||
services.dnsmasq = import ./dnsmasq.nix {
|
|
||||||
inherit
|
|
||||||
lib
|
|
||||||
cfg
|
|
||||||
networkCfg
|
|
||||||
hyperhiveDomain
|
|
||||||
;
|
|
||||||
};
|
|
||||||
|
|
||||||
networking.firewall = lib.mkIf cfg.openFirewall {
|
networking.firewall = lib.mkIf cfg.openFirewall {
|
||||||
allowedTCPPorts = [
|
allowedTCPPorts = [
|
||||||
cfg.port
|
cfg.port
|
||||||
|
|
|
||||||
63
nix/host-modules/hive-gateway/dns.nix
Normal file
63
nix/host-modules/hive-gateway/dns.nix
Normal file
|
|
@ -0,0 +1,63 @@
|
||||||
|
# The hive's dnsmasq resolver. Split out of ./default.nix because it has
|
||||||
|
# its own enable: a host can need hive names to resolve without serving a
|
||||||
|
# single vhost. Option declared in ./options.nix, config rendered by
|
||||||
|
# ./dnsmasq.nix.
|
||||||
|
{
|
||||||
|
lib,
|
||||||
|
config,
|
||||||
|
...
|
||||||
|
}:
|
||||||
|
let
|
||||||
|
cfg = config.services.hyperhive.gateway;
|
||||||
|
networkCfg = config.services.hyperhive.network;
|
||||||
|
hyperhiveDomain = config.services.hyperhive.domain;
|
||||||
|
in
|
||||||
|
{
|
||||||
|
config = lib.mkIf cfg.dns.enable {
|
||||||
|
# dnsmasq is a host service alongside nginx, so it reads the host's
|
||||||
|
# /etc/resolv.conf directly and picks up network changes as they
|
||||||
|
# happen — no copy to keep in sync.
|
||||||
|
services.dnsmasq = import ./dnsmasq.nix {
|
||||||
|
inherit
|
||||||
|
lib
|
||||||
|
cfg
|
||||||
|
networkCfg
|
||||||
|
hyperhiveDomain
|
||||||
|
;
|
||||||
|
};
|
||||||
|
|
||||||
|
# dnsmasq binds the bridge interface and answers with the bridge
|
||||||
|
# address, so the resolver is itself a consumer one layer down.
|
||||||
|
services.hyperhive.network.enable = lib.mkDefault true;
|
||||||
|
|
||||||
|
# A name that stops resolving presents as every client timing out at
|
||||||
|
# once, so this unit's journal is worth reading swarm-wide.
|
||||||
|
services.hyperhive.swarm.otel.journaldUnits = [ "dnsmasq" ];
|
||||||
|
|
||||||
|
# The host asks the hive's own resolver, at the BRIDGE IP.
|
||||||
|
#
|
||||||
|
# Every container inherits a COPY of this host's `/etc/resolv.conf`
|
||||||
|
# at start (`nixos-containers.nix`: `cp --remove-destination`, one
|
||||||
|
# shot, not a bind-mount) — so whatever address is written here is
|
||||||
|
# the address every container will try, in its own netns.
|
||||||
|
#
|
||||||
|
# 🚨 That is why this is the bridge IP and not `127.0.0.1`, and the
|
||||||
|
# distinction is load-bearing rather than stylistic:
|
||||||
|
#
|
||||||
|
# value host host-netns containers bridged containers
|
||||||
|
# 127.0.0.1 ok ok THEIR OWN loopback
|
||||||
|
# bridge IP ok ok ok
|
||||||
|
#
|
||||||
|
# dnsmasq binds both `lo` and the bridge (./dnsmasq.nix), so the
|
||||||
|
# bridge IP is reachable from the host too — it is the only value
|
||||||
|
# correct on both sides of a netns boundary. `resolveLocalQueries`
|
||||||
|
# publishes loopback by default, hence both overrides here; the
|
||||||
|
# flag stays on for its `resolv-file` plumbing, which is what keeps
|
||||||
|
# dnsmasq's own upstreams out of the file we are pointing at it.
|
||||||
|
#
|
||||||
|
# Cost, stated because it is real: the host's DNS now depends on
|
||||||
|
# dnsmasq being up. Every container already did.
|
||||||
|
networking.nameservers = lib.mkForce [ networkCfg.bridgeIp ];
|
||||||
|
networking.resolvconf.useLocalResolver = lib.mkForce false;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
@ -1,8 +1,7 @@
|
||||||
# Option declarations for `services.hyperhive.gateway.*`. The gateway
|
# Option declarations for `services.hyperhive.gateway.*`. The gateway is
|
||||||
# is always run alongside hyperhive (it's the single nginx in front of
|
# the single nginx in front of every surface and the only thing exposed
|
||||||
# every surface and the only thing exposed to the outside); there is
|
# to the outside. An operator who wants their own reverse proxy in front
|
||||||
# no enable flag. An operator who wants their own reverse proxy in
|
# points it at the gateway's `port`.
|
||||||
# front points it at the gateway's `port`.
|
|
||||||
{
|
{
|
||||||
lib,
|
lib,
|
||||||
config,
|
config,
|
||||||
|
|
@ -22,6 +21,18 @@ in
|
||||||
];
|
];
|
||||||
|
|
||||||
options.services.hyperhive.gateway = {
|
options.services.hyperhive.gateway = {
|
||||||
|
enable = lib.mkOption {
|
||||||
|
type = lib.types.bool;
|
||||||
|
default = false;
|
||||||
|
description = "Run the hive's reverse proxy (nginx) on this host.";
|
||||||
|
};
|
||||||
|
|
||||||
|
dns.enable = lib.mkOption {
|
||||||
|
type = lib.types.bool;
|
||||||
|
default = false;
|
||||||
|
description = "Run the hive's resolver (dnsmasq) on this host.";
|
||||||
|
};
|
||||||
|
|
||||||
port = lib.mkOption {
|
port = lib.mkOption {
|
||||||
type = lib.types.port;
|
type = lib.types.port;
|
||||||
default = 80;
|
default = 80;
|
||||||
|
|
|
||||||
|
|
@ -705,6 +705,11 @@ in
|
||||||
# than fronted, so there is no name to claim and no vhost to serve —
|
# than fronted, so there is no name to claim and no vhost to serve —
|
||||||
# every clause below carries that guard.
|
# every clause below carries that guard.
|
||||||
services.hyperhive.gateway.localNames = lib.optional (cfg.gatewayHost != null) cfg.gatewayHost;
|
services.hyperhive.gateway.localNames = lib.optional (cfg.gatewayHost != null) cfg.gatewayHost;
|
||||||
|
services.hyperhive.gateway.enable = lib.mkIf (cfg.gatewayHost != null) (lib.mkDefault true);
|
||||||
|
|
||||||
|
# Unguarded, unlike the two above: the homeserver container resolves
|
||||||
|
# through the hive's dnsmasq whether or not the gateway fronts it.
|
||||||
|
services.hyperhive.gateway.dns.enable = lib.mkDefault true;
|
||||||
|
|
||||||
# The homeserver's own journal (`tuwunel` is the unit name inside the
|
# The homeserver's own journal (`tuwunel` is the unit name inside the
|
||||||
# container, whatever the nixpkgs option is called), plus the host-side
|
# container, whatever the nixpkgs option is called), plus the host-side
|
||||||
|
|
|
||||||
|
|
@ -29,16 +29,10 @@ let
|
||||||
networkBase = builtins.bitAnd (ipToInt cfg.bridgeIp) (4294967295 - hostCount + 1);
|
networkBase = builtins.bitAnd (ipToInt cfg.bridgeIp) (4294967295 - hostCount + 1);
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
# Hive-internal network — host-side bridge + per-agent DNS resolver.
|
# Hive-internal network — the host-side bridge every container in a
|
||||||
# Always active when hyperhive is enabled: agent containers run in
|
# private netns hangs off. Full design: docs/networking/network.md.
|
||||||
# private netns behind the bridge. Full design: docs/networking/network.md.
|
|
||||||
|
|
||||||
imports = [
|
imports = [
|
||||||
(lib.mkRemovedOptionModule [ "services" "hyperhive" "network" "enable" ] ''
|
|
||||||
The hive network (bridge + dnsmasq resolver + private-netns
|
|
||||||
isolation) is always on whenever hyperhive is enabled. Remove the
|
|
||||||
setting.
|
|
||||||
'')
|
|
||||||
(lib.mkRemovedOptionModule [ "services" "hyperhive" "network" "isolateContainers" ] ''
|
(lib.mkRemovedOptionModule [ "services" "hyperhive" "network" "isolateContainers" ] ''
|
||||||
Network isolation is the only mode and is always on whenever
|
Network isolation is the only mode and is always on whenever
|
||||||
hyperhive is enabled; the shared-netns path was removed. Remove
|
hyperhive is enabled; the shared-netns path was removed. Remove
|
||||||
|
|
@ -52,6 +46,12 @@ in
|
||||||
];
|
];
|
||||||
|
|
||||||
options.services.hyperhive.network = {
|
options.services.hyperhive.network = {
|
||||||
|
enable = lib.mkOption {
|
||||||
|
type = lib.types.bool;
|
||||||
|
default = false;
|
||||||
|
description = "Bring up the hive bridge, its NAT and its firewall rules on this host.";
|
||||||
|
};
|
||||||
|
|
||||||
bridgeName = lib.mkOption {
|
bridgeName = lib.mkOption {
|
||||||
type = lib.types.str;
|
type = lib.types.str;
|
||||||
default = "hive-br0";
|
default = "hive-br0";
|
||||||
|
|
@ -148,9 +148,10 @@ in
|
||||||
};
|
};
|
||||||
|
|
||||||
config = lib.mkMerge [
|
config = lib.mkMerge [
|
||||||
# The hive network + container isolation are unconditional whenever
|
# Identity checks belong to hyperhive being on, not to the bridge
|
||||||
# hyperhive is enabled: the shared-netns mode was removed, so there
|
# being up: a hive with no domain is misconfigured either way, and
|
||||||
# is one mode (private netns behind the bridge).
|
# moving them under the bridge gate would hide them on exactly the
|
||||||
|
# hosts that are hardest to debug.
|
||||||
(lib.mkIf config.services.hyperhive.enable {
|
(lib.mkIf config.services.hyperhive.enable {
|
||||||
# This message is only useful if an operator can actually reach
|
# This message is only useful if an operator can actually reach
|
||||||
# it, and an assertion competes with every eager default that
|
# it, and an assertion competes with every eager default that
|
||||||
|
|
@ -206,7 +207,11 @@ in
|
||||||
'';
|
'';
|
||||||
}
|
}
|
||||||
];
|
];
|
||||||
|
})
|
||||||
|
|
||||||
|
# The bridge itself, up only where something hangs off it. Private
|
||||||
|
# netns is still the only container mode.
|
||||||
|
(lib.mkIf cfg.enable {
|
||||||
# Virtual bridge — each agent container attaches a veth pair (isolation
|
# Virtual bridge — each agent container attaches a veth pair (isolation
|
||||||
# is unconditional now).
|
# is unconditional now).
|
||||||
networking.bridges.${cfg.bridgeName}.interfaces = [ ];
|
networking.bridges.${cfg.bridgeName}.interfaces = [ ];
|
||||||
|
|
@ -233,9 +238,10 @@ in
|
||||||
};
|
};
|
||||||
})
|
})
|
||||||
|
|
||||||
# Container isolation overlay — now unconditional (the shared-netns
|
# Container isolation overlay — the routing, NAT and firewall half of
|
||||||
# mode was removed). See docs/networking/network.md#container-isolation.
|
# the same bridge, so it rides the same gate.
|
||||||
(lib.mkIf config.services.hyperhive.enable {
|
# See docs/networking/network.md#container-isolation.
|
||||||
|
(lib.mkIf cfg.enable {
|
||||||
|
|
||||||
# Agents route internet traffic via the bridge; NAT masquerades their RFC-1918 IPs.
|
# Agents route internet traffic via the bridge; NAT masquerades their RFC-1918 IPs.
|
||||||
boot.kernel.sysctl."net.ipv4.ip_forward" = 1;
|
boot.kernel.sysctl."net.ipv4.ip_forward" = 1;
|
||||||
|
|
|
||||||
|
|
@ -336,6 +336,10 @@ in
|
||||||
# the port on the bridge interface only.
|
# the port on the bridge interface only.
|
||||||
services.hyperhive.network.exposeHostPorts = [ otel.collector.port ];
|
services.hyperhive.network.exposeHostPorts = [ otel.collector.port ];
|
||||||
|
|
||||||
|
# …and the receiver binds that bridge address, so the interface has
|
||||||
|
# to exist. `exposeHostPorts` is a firewall hole, not a demand.
|
||||||
|
services.hyperhive.network.enable = lib.mkDefault true;
|
||||||
|
|
||||||
# The collector scrapes itself. Its own counters — refused, failed,
|
# The collector scrapes itself. Its own counters — refused, failed,
|
||||||
# queue depth — are the only signal that says telemetry is being
|
# queue depth — are the only signal that says telemetry is being
|
||||||
# dropped, and they reach no store unless something reads them.
|
# dropped, and they reach no store unless something reads them.
|
||||||
|
|
|
||||||
|
|
@ -1096,6 +1096,11 @@ in
|
||||||
# bridge at that wrong answer.
|
# bridge at that wrong answer.
|
||||||
services.hyperhive.gateway.localNames = [ cfg.domain ];
|
services.hyperhive.gateway.localNames = [ cfg.domain ];
|
||||||
|
|
||||||
|
# This host serves the vhost, and the container behind it resolves
|
||||||
|
# through the hive's dnsmasq.
|
||||||
|
services.hyperhive.gateway.enable = lib.mkDefault true;
|
||||||
|
services.hyperhive.gateway.dns.enable = lib.mkDefault true;
|
||||||
|
|
||||||
# The bridge as well as authelia: it is the half that writes the identity
|
# The bridge as well as authelia: it is the half that writes the identity
|
||||||
# store, and its refusals are returned to callers as a bare 401.
|
# store, and its refusals are returned to callers as a bare 401.
|
||||||
#
|
#
|
||||||
|
|
|
||||||
|
|
@ -819,6 +819,13 @@ in
|
||||||
# this file's header. `localNames` is the one half of the sibling
|
# this file's header. `localNames` is the one half of the sibling
|
||||||
# pattern that applies.
|
# pattern that applies.
|
||||||
services.hyperhive.gateway.localNames = [ cfg.domain ];
|
services.hyperhive.gateway.localNames = [ cfg.domain ];
|
||||||
|
|
||||||
|
# All three, and no vhost among them: the `ssl_preread` stream
|
||||||
|
# server below needs the nginx process without asking it to serve
|
||||||
|
# anything, and it listens on the bridge address.
|
||||||
|
services.hyperhive.gateway.enable = lib.mkDefault true;
|
||||||
|
services.hyperhive.gateway.dns.enable = lib.mkDefault true;
|
||||||
|
services.hyperhive.network.enable = lib.mkDefault true;
|
||||||
})
|
})
|
||||||
|
|
||||||
(lib.mkIf (hyperhiveCfg.enable && deployCfg.bao.enable) {
|
(lib.mkIf (hyperhiveCfg.enable && deployCfg.bao.enable) {
|
||||||
|
|
|
||||||
|
|
@ -385,6 +385,11 @@ in
|
||||||
# hive declaring the vhost would answer for a service it does not have.
|
# hive declaring the vhost would answer for a service it does not have.
|
||||||
services.hyperhive.gateway.localNames = [ cfg.domain ];
|
services.hyperhive.gateway.localNames = [ cfg.domain ];
|
||||||
|
|
||||||
|
# This host serves the vhost, and the container behind it resolves
|
||||||
|
# through the hive's dnsmasq.
|
||||||
|
services.hyperhive.gateway.enable = lib.mkDefault true;
|
||||||
|
services.hyperhive.gateway.dns.enable = lib.mkDefault true;
|
||||||
|
|
||||||
# The secret oneshots as well as grafana itself: each runs before it and
|
# The secret oneshots as well as grafana itself: each runs before it and
|
||||||
# fails in ways grafana then reports only as a login that does not work.
|
# fails in ways grafana then reports only as a login that does not work.
|
||||||
#
|
#
|
||||||
|
|
|
||||||
|
|
@ -439,6 +439,10 @@ in
|
||||||
"swarm-nats-auth"
|
"swarm-nats-auth"
|
||||||
];
|
];
|
||||||
|
|
||||||
|
# Resolver only: NATS speaks its own protocol, so nginx fronts
|
||||||
|
# nothing here — but the auth responder introspects authelia by name.
|
||||||
|
services.hyperhive.gateway.dns.enable = lib.mkDefault true;
|
||||||
|
|
||||||
assertions = [
|
assertions = [
|
||||||
{
|
{
|
||||||
# Fail at EVAL, not at boot: a queue that comes up unable to
|
# Fail at EVAL, not at boot: a queue that comes up unable to
|
||||||
|
|
|
||||||
|
|
@ -570,6 +570,11 @@ in
|
||||||
# vhost would answer for a service it does not have.
|
# vhost would answer for a service it does not have.
|
||||||
services.hyperhive.gateway.localNames = [ cfg.domain ];
|
services.hyperhive.gateway.localNames = [ cfg.domain ];
|
||||||
|
|
||||||
|
# This host serves the vhost, and the container behind it resolves
|
||||||
|
# through the hive's dnsmasq.
|
||||||
|
services.hyperhive.gateway.enable = lib.mkDefault true;
|
||||||
|
services.hyperhive.gateway.dns.enable = lib.mkDefault true;
|
||||||
|
|
||||||
# This collector reads its own journal, so a pipeline that stops
|
# This collector reads its own journal, so a pipeline that stops
|
||||||
# delivering says so in the store it stopped delivering to. That is
|
# delivering says so in the store it stopped delivering to. That is
|
||||||
# less circular than it sounds: the failure that matters here is a
|
# less circular than it sounds: the failure that matters here is a
|
||||||
|
|
|
||||||
|
|
@ -87,6 +87,18 @@ in
|
||||||
# hive that is not the service host is a *client* of it, not a second one.
|
# hive that is not the service host is a *client* of it, not a second one.
|
||||||
config.services.hyperhive.deploy.victorialogs.enable = lib.mkDefault deployCfg.allSwarmServices;
|
config.services.hyperhive.deploy.victorialogs.enable = lib.mkDefault deployCfg.allSwarmServices;
|
||||||
|
|
||||||
|
# The plumbing those services are reached over: every one of them is
|
||||||
|
# fronted by the gateway, resolved through the hive's dnsmasq, and runs
|
||||||
|
# in a container hanging off the bridge. Written as `mkIf … mkDefault`
|
||||||
|
# rather than `mkDefault allSwarmServices` because the modules that need
|
||||||
|
# these also assert them — a `false` from here would collide with their
|
||||||
|
# `true` instead of losing to it.
|
||||||
|
config.services.hyperhive.gateway.enable = lib.mkIf deployCfg.allSwarmServices (lib.mkDefault true);
|
||||||
|
config.services.hyperhive.gateway.dns.enable = lib.mkIf deployCfg.allSwarmServices (
|
||||||
|
lib.mkDefault true
|
||||||
|
);
|
||||||
|
config.services.hyperhive.network.enable = lib.mkIf deployCfg.allSwarmServices (lib.mkDefault true);
|
||||||
|
|
||||||
# The secret store. Once per swarm and optional, so it belongs to the
|
# The secret store. Once per swarm and optional, so it belongs to the
|
||||||
# same switch: a hive that does not run it is a *client*, reading its
|
# same switch: a hive that does not run it is a *client*, reading its
|
||||||
# own secrets from whoever does. `mkDefault` is what keeps the store
|
# own secrets from whoever does. `mkDefault` is what keeps the store
|
||||||
|
|
|
||||||
|
|
@ -132,6 +132,10 @@ in
|
||||||
# makes the name resolve at all.
|
# makes the name resolve at all.
|
||||||
services.hyperhive.gateway.localNames = [ apex ];
|
services.hyperhive.gateway.localNames = [ apex ];
|
||||||
|
|
||||||
|
# Gateway only — static files served by nginx, no container of its
|
||||||
|
# own and so nothing to resolve.
|
||||||
|
services.hyperhive.gateway.enable = lib.mkDefault true;
|
||||||
|
|
||||||
# This UI's own swagger docs, always same-origin (`/api/docs/` below)
|
# This UI's own swagger docs, always same-origin (`/api/docs/` below)
|
||||||
# so — unlike authelia/matrix/forge's entries — this one needs no
|
# so — unlike authelia/matrix/forge's entries — this one needs no
|
||||||
# host name and is never conditional on anything but this module
|
# host name and is never conditional on anything but this module
|
||||||
|
|
|
||||||
|
|
@ -153,6 +153,11 @@ in
|
||||||
# `swarm-victoriametrics.nix`).
|
# `swarm-victoriametrics.nix`).
|
||||||
services.hyperhive.gateway.localNames = [ cfg.domain ];
|
services.hyperhive.gateway.localNames = [ cfg.domain ];
|
||||||
|
|
||||||
|
# This host serves the vhost, and the container behind it resolves
|
||||||
|
# through the hive's dnsmasq.
|
||||||
|
services.hyperhive.gateway.enable = lib.mkDefault true;
|
||||||
|
services.hyperhive.gateway.dns.enable = lib.mkDefault true;
|
||||||
|
|
||||||
# Declared here rather than in the collector, so this store's logs are
|
# Declared here rather than in the collector, so this store's logs are
|
||||||
# collected because it runs, not because a list elsewhere remembered it.
|
# collected because it runs, not because a list elsewhere remembered it.
|
||||||
services.hyperhive.swarm.otel.journaldUnits = [ "victorialogs" ];
|
services.hyperhive.swarm.otel.journaldUnits = [ "victorialogs" ];
|
||||||
|
|
|
||||||
|
|
@ -103,6 +103,11 @@ in
|
||||||
# have.
|
# have.
|
||||||
services.hyperhive.gateway.localNames = [ cfg.domain ];
|
services.hyperhive.gateway.localNames = [ cfg.domain ];
|
||||||
|
|
||||||
|
# This host serves the vhost, and the container behind it resolves
|
||||||
|
# through the hive's dnsmasq.
|
||||||
|
services.hyperhive.gateway.enable = lib.mkDefault true;
|
||||||
|
services.hyperhive.gateway.dns.enable = lib.mkDefault true;
|
||||||
|
|
||||||
# Declared here rather than in the collector, so this store's logs are
|
# Declared here rather than in the collector, so this store's logs are
|
||||||
# collected because it runs, not because a list elsewhere remembered it.
|
# collected because it runs, not because a list elsewhere remembered it.
|
||||||
services.hyperhive.swarm.otel.journaldUnits = [ "victoriametrics" ];
|
services.hyperhive.swarm.otel.journaldUnits = [ "victoriametrics" ];
|
||||||
|
|
|
||||||
|
|
@ -104,8 +104,20 @@ let
|
||||||
# that quietly re-introduces the dependency — or that changes what the
|
# that quietly re-introduces the dependency — or that changes what the
|
||||||
# default renders for a hive with the toggle on — fails here. Reading an
|
# default renders for a hive with the toggle on — fails here. Reading an
|
||||||
# option off this fixture forces that option only, not the config, so the
|
# option off this fixture forces that option only, not the config, so the
|
||||||
# toggle being off costs nothing.
|
# toggle being off costs nothing. Also the "installs the modules and turns
|
||||||
|
# nothing on" host the swarm-service absences below read: none of the
|
||||||
|
# per-service deployment toggles derives from the hive being on, so it
|
||||||
|
# renders the same absences `bare` does.
|
||||||
centralToggleOff = hive { enable = false; };
|
centralToggleOff = hive { enable = false; };
|
||||||
|
|
||||||
|
# The swarm-services toggle with the central one off, so the only thing
|
||||||
|
# that can enable the gateway/resolver/bridge here is that toggle's own
|
||||||
|
# module — every other module that asserts them is behind `enable`.
|
||||||
|
swarmServicesOnly = hive {
|
||||||
|
enable = false;
|
||||||
|
deploy.allSwarmServices = true;
|
||||||
|
};
|
||||||
|
|
||||||
withCi = hive { deploy.forgejo.ci.enable = true; };
|
withCi = hive { deploy.forgejo.ci.enable = true; };
|
||||||
|
|
||||||
# A host configured against the pre-rename option path. `mkRenamedOptionModule`
|
# A host configured against the pre-rename option path. `mkRenamedOptionModule`
|
||||||
|
|
@ -3237,6 +3249,48 @@ let
|
||||||
name = "the swarm UI claims the swarm apex where this host serves it";
|
name = "the swarm UI claims the swarm apex where this host serves it";
|
||||||
ok = swarmUiHere.services.nginx.virtualHosts ? "t.local";
|
ok = swarmUiHere.services.nginx.virtualHosts ? "t.local";
|
||||||
}
|
}
|
||||||
|
{
|
||||||
|
# The three infrastructure toggles are off by default and asserted by
|
||||||
|
# whoever needs them. With nothing on the host needing them, none of
|
||||||
|
# the three renders — which is also the control for the arm below.
|
||||||
|
name = "the gateway, resolver and bridge are absent where nothing on the host needs them";
|
||||||
|
ok =
|
||||||
|
!centralToggleOff.services.hyperhive.gateway.enable
|
||||||
|
&& !centralToggleOff.services.hyperhive.gateway.dns.enable
|
||||||
|
&& !centralToggleOff.services.hyperhive.network.enable
|
||||||
|
&& !(centralToggleOff.services.nginx.enable or false)
|
||||||
|
&& !(centralToggleOff.services.dnsmasq.enable or false)
|
||||||
|
&& !(centralToggleOff.networking.bridges ? hive-br0);
|
||||||
|
}
|
||||||
|
{
|
||||||
|
# hive-c0re asserts all three, and it follows the central toggle — so
|
||||||
|
# an ordinary hive keeps getting them with no opt-in, which is what
|
||||||
|
# this change must not break.
|
||||||
|
name = "an ordinary hive runs the gateway, resolver and bridge because its coordinator needs them";
|
||||||
|
ok =
|
||||||
|
bare.services.hyperhive.gateway.enable
|
||||||
|
&& bare.services.hyperhive.gateway.dns.enable
|
||||||
|
&& bare.services.hyperhive.network.enable
|
||||||
|
&& bare.services.nginx.enable
|
||||||
|
&& bare.services.dnsmasq.enable
|
||||||
|
&& bare.networking.bridges ? hive-br0;
|
||||||
|
}
|
||||||
|
{
|
||||||
|
# The swarm-services toggle enables them explicitly, from its own
|
||||||
|
# module rather than from any of their defaults.
|
||||||
|
name = "the swarm-services toggle turns on the gateway, resolver and bridge by itself";
|
||||||
|
ok =
|
||||||
|
swarmServicesOnly.services.hyperhive.gateway.enable
|
||||||
|
&& swarmServicesOnly.services.hyperhive.gateway.dns.enable
|
||||||
|
&& swarmServicesOnly.services.hyperhive.network.enable;
|
||||||
|
}
|
||||||
|
{
|
||||||
|
# An operator's explicit `false` beats every `mkDefault` assertion,
|
||||||
|
# which is what keeps "asserted by whoever needs it" from being a
|
||||||
|
# setting the operator cannot turn off.
|
||||||
|
name = "an explicit gateway.enable = false wins over the modules asserting it";
|
||||||
|
ok = !(hive { gateway.enable = false; }).services.nginx.enable;
|
||||||
|
}
|
||||||
];
|
];
|
||||||
|
|
||||||
bad = builtins.filter (c: !c.ok) cases;
|
bad = builtins.filter (c: !c.ok) cases;
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue