diff --git a/CLAUDE.md b/CLAUDE.md index 9b29051f..768e5e4f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -142,7 +142,7 @@ hand-maintained per-file tree drifts out of sync with the code. - **`hive-metric/`** — small CLI to push a single labeled metric to the OTEL collector via the OpenTelemetry Rust SDK / OTLP HTTP exporter. - **`swarm-controller/`** — swarm-level daemon, opt-in per host - (`services.hyperhive.swarm.controller.enable`). Where `hive-c0re` owns + (`services.hyperhive.deploy.swarm-controller.enable`). Where `hive-c0re` owns the agents on **one** host, this owns what is true **across** hives; a swarm runs one of them, so most hives leave it off. Serves HTTP over a unix socket the gateway's nginx proxies to — ⚠️ **the socket's diff --git a/docs/gateway.md b/docs/gateway.md index e465cce7..6d306c7f 100644 --- a/docs/gateway.md +++ b/docs/gateway.md @@ -14,8 +14,8 @@ Single nginx in front of every hyperhive web surface. Runs on the **host**, next | `chat./_matrix/*` | `chat.` | tuwunel (`8008`) | `matrix.gatewayHost != null` | | `chat./` | `chat.` | fluffychat-web static | `matrix.gui.enable` | | `chat./config.json` | `chat.` | inline JSON (FluffyChat boot config) | `matrix.gui.enable && domain != null` | -| `auth./` | `auth.` | authelia (`9091`) | `swarm.authelia.enable` | -| `/` | `` | swarm-ui dist (static), behind an authelia subrequest | `swarm.ui.enable` | +| `auth./` | `auth.` | authelia (`9091`) | `deploy.authelia` | +| `/` | `` | swarm-ui dist (static), behind an authelia subrequest | `deploy.swarm-ui` | The authelia vhost is declared only by the host that **runs** authelia, not by every hive that uses it — a client hive knows the swarm's `authelia.url` but must not answer for a name it doesn't serve. Its server name is exactly `swarm.authelia.domain`: authelia validates `authelia_url ⊂ session cookie domain` at startup, so a near-miss is a container that refuses to boot. It carries no `auth_basic` — the login page must not sit behind the login mechanism it replaces — and sets the four `X-Forwarded-{Proto,Host,Uri,For}` headers, since authelia decides by the *original* request rather than the hop it sees. @@ -64,7 +64,7 @@ Each location carries a duplicated `auth_basic` block (separate locations don't - `` → `127.0.0.1` - `forge.` → `127.0.0.1` (when forge.behindGateway) - `chat.` → `127.0.0.1` (when matrix.gatewayHost set) -- `auth.` → `127.0.0.1` (when swarm.authelia.enable) +- `auth.` → `127.0.0.1` (when deploy.authelia) `lib.unique` de-dupes if any sub-domain happens to equal another entry. Operators with real DNS leave it off. diff --git a/docs/observability.md b/docs/observability.md index 35f22bf1..cb2f2b21 100644 --- a/docs/observability.md +++ b/docs/observability.md @@ -18,7 +18,7 @@ services.hyperhive.otel = { `enable` is the single gate. `endpoint` is where telemetry ends up after it leaves the swarm — optional, because the swarm's own metrics store -(`swarm.victoriametrics.enable`) is a destination in its own right. With both, +(`deploy.victoriametrics`) is a destination in its own right. With both, telemetry goes to both. See [`swarm/services.md`](swarm/services.md#metrics-victoriametrics--grafana). @@ -80,7 +80,7 @@ the host is: | | runs where | receives from | does | |---|---|---|---| | **hive tier** — `otel.enable` | every hive with agents | that hive's agents, on the bridge | forwards to the swarm tier. Holds no credential, picks no destination | -| **swarm tier** — `swarm.otel.enable` | once per swarm | every hive's collector | writes the swarm's store and exports upstream | +| **swarm tier** — `deploy.swarm-otel` | once per swarm | every hive's collector | writes the swarm's store and exports upstream | An all-local host runs both, and needs nothing said about the hop between them. diff --git a/docs/persistence.md b/docs/persistence.md index b7f3ae00..b95291d9 100644 --- a/docs/persistence.md +++ b/docs/persistence.md @@ -472,7 +472,7 @@ snapshot contains and how the store authenticates a sender. ## `/var/lib/swarm-controller/` (swarm-controller host only) Only present on the one host running -`services.hyperhive.swarm.controller.enable`. systemd `StateDirectory=`, +`services.hyperhive.deploy.swarm-controller.enable`. systemd `StateDirectory=`, so it survives restarts and redeploys. - `webhook-secret` — the HMAC key the swarm's forge webhooks are signed diff --git a/docs/setup.md b/docs/setup.md index cdebe400..4af6a569 100644 --- a/docs/setup.md +++ b/docs/setup.md @@ -44,7 +44,7 @@ echo "hunter2" | hivectl gateway create-user mara --password-stdin hivectl gateway list-users ``` -### 3 · Swarm SSO (only when `swarm.authelia.enable`) +### 3 · Swarm SSO (only when `deploy.authelia`) ⚠️ **Required to finish the install, not optional.** Authelia treats an empty user store as a fatal startup error, so until this runs the @@ -70,7 +70,7 @@ than amend — adding the group afterwards is `swarmctl user update mara Detail, including what the password is and why this stays manual: [`swarm/sso.md`](swarm/sso.md). -### 4 · Swarm UI (only when `swarm.ui.enable`, on by default with the controller) +### 4 · Swarm UI (only when `deploy.swarm-ui`, on by default with the controller) Nothing to run — it is served on the swarm apex (`https:///`) as soon as the host rebuilds. Two things diff --git a/docs/swarm/README.md b/docs/swarm/README.md index 853a44cc..3ca5c4c4 100644 --- a/docs/swarm/README.md +++ b/docs/swarm/README.md @@ -276,7 +276,7 @@ migrating agent keeps one unbroken incremental chain. See ## Swarm controller -`services.hyperhive.swarm.controller.enable` runs the `swarm-controller` +`services.hyperhive.deploy.swarm-controller.enable` runs the `swarm-controller` daemon on this host. **Off by default and deliberately not derived from `services.hyperhive.enable`**: a swarm has one controller, so enabling it is a statement about swarm topology, not about whether hyperhive is diff --git a/docs/swarm/services.md b/docs/swarm/services.md index 56764f15..c3de4e08 100644 --- a/docs/swarm/services.md +++ b/docs/swarm/services.md @@ -34,7 +34,7 @@ One authelia per swarm, in a `swarm-authelia` container, at provider, differentiated by roles and claims rather than by mechanism — there is one IdP and one auth path. -- **`swarm.authelia.enable`** — run the container here. Defaults from +- **`deploy.authelia`** — run the container here. Defaults from `swarm.enableRequiredServices`. - **`swarm.authelia.url`** — where clients are sent to authenticate. Present on **every** hive, defaulting to this host's own instance only @@ -74,8 +74,8 @@ together: a store with no UI is unreadable and a UI with no store is empty. To run one without the other, set it directly: ```nix -services.hyperhive.swarm.victoriametrics.enable = true; -services.hyperhive.swarm.grafana.enable = false; +services.hyperhive.deploy.victoriametrics.enable = true; +services.hyperhive.deploy.grafana.enable = false; ``` ⚠️ **This starts a database that grows for as long as the swarm runs.** diff --git a/docs/swarm/sso.md b/docs/swarm/sso.md index 53aad373..26d5c7ea 100644 --- a/docs/swarm/sso.md +++ b/docs/swarm/sso.md @@ -14,7 +14,7 @@ separate `enable` would be a second fact free to disagree with the first. authelia binds loopback only. The **gateway** on the host running it publishes it as `auth.` — vhost, dnsmasq record and TLS -name all follow `swarm.authelia.enable`, so there is nothing to turn on +name all follow `deploy.authelia`, so there is nothing to turn on separately. (Details, including why a client hive must not declare that vhost: [`../gateway.md`](../gateway.md).) diff --git a/docs/swarm/ui.md b/docs/swarm/ui.md index f292993e..10bd72ee 100644 --- a/docs/swarm/ui.md +++ b/docs/swarm/ui.md @@ -9,7 +9,7 @@ answers for one host. This one is the view *across* hives. ## Enabling ```nix -services.hyperhive.swarm.ui.enable = true; # defaults to swarm.controller.enable +services.hyperhive.deploy.swarm-ui.enable = true; # defaults to deploy.swarm-controller.enable ``` Derived from the controller rather than from `enableRequiredServices`: diff --git a/nix/host-modules/default.nix b/nix/host-modules/default.nix index da0225c9..8c180f4f 100644 --- a/nix/host-modules/default.nix +++ b/nix/host-modules/default.nix @@ -12,6 +12,7 @@ { imports = [ ./hyperhive.nix + ./deploy.nix ./local-defaults.nix ./hive-c0re ./hive-ci.nix diff --git a/nix/host-modules/deploy.nix b/nix/host-modules/deploy.nix new file mode 100644 index 00000000..8ae9ae71 --- /dev/null +++ b/nix/host-modules/deploy.nix @@ -0,0 +1,200 @@ +# "What does THIS host deploy?" +# +# Separated from `services.hyperhive.swarm.*` because those are two +# different kinds of fact and only one of them varies per machine: +# +# swarm.* — swarm-wide truth. The swarm's name, domain, hives, peers, +# CA, and where each service lives. **Identical on every +# host**, byte for byte; a hive needs all of it to be a +# *client* of the swarm. +# deploy.* — this machine's deployment decisions. Necessarily different +# on every host, because that is what a deployment is. +# +# The `enable` toggles used to live under `swarm.*`, which made the +# namespace that is supposed to be identical everywhere carry the one +# thing that must differ. +# +# Flat and named for the thing deployed — `deploy.forgejo`, not +# `deploy.swarmServices.forgejo`: grouping by "swarm service" re-encodes +# the service-side taxonomy into a layer that does not care about it. +# +# ⚠️ Each entry is an attrset with an `enable`, not a bare bool, so a +# service that grows a second *deployment* decision has somewhere to put +# it — `deploy.forgejo = { enable; ci; }` is then an ordinary addition +# rather than a migration. `ci` ("does this host run the runner too") is +# exactly that shape, and a bare bool leaves it unrepresentable. +# +# ⚠️ The renames below are deliberately in this one file rather than +# spread across the service modules, so the whole move has a single home +# and a single file to delete when the deprecation window closes — the +# shape ./swarm-peers-removed.nix already uses. +{ lib, config, ... }: +let + deployCfg = config.services.hyperhive.deploy; +in +{ + imports = [ + # Same type, same meaning, new path — so a rename carries it exactly + # and existing configs keep evaluating with one warning naming both + # paths. Precedent: ./hive-forge/default.nix, ./hive-matrix.nix. + (lib.mkRenamedOptionModule + [ "services" "hyperhive" "swarm" "grafana" "enable" ] + [ "services" "hyperhive" "deploy" "grafana" "enable" ] + ) + (lib.mkRenamedOptionModule + [ "services" "hyperhive" "swarm" "victoriametrics" "enable" ] + [ "services" "hyperhive" "deploy" "victoriametrics" "enable" ] + ) + (lib.mkRenamedOptionModule + [ "services" "hyperhive" "swarm" "victorialogs" "enable" ] + [ "services" "hyperhive" "deploy" "victorialogs" "enable" ] + ) + (lib.mkRenamedOptionModule + [ "services" "hyperhive" "swarm" "controller" "enable" ] + [ "services" "hyperhive" "deploy" "swarm-controller" "enable" ] + ) + (lib.mkRenamedOptionModule + [ "services" "hyperhive" "swarm" "ui" "enable" ] + [ "services" "hyperhive" "deploy" "swarm-ui" "enable" ] + ) + (lib.mkRenamedOptionModule + [ "services" "hyperhive" "swarm" "authelia" "enable" ] + [ "services" "hyperhive" "deploy" "authelia" "enable" ] + ) + (lib.mkRenamedOptionModule + [ "services" "hyperhive" "swarm" "nats" "enable" ] + [ "services" "hyperhive" "deploy" "nats" "enable" ] + ) + (lib.mkRenamedOptionModule + [ "services" "hyperhive" "swarm" "otel" "enable" ] + [ "services" "hyperhive" "deploy" "swarm-otel" "enable" ] + ) + ]; + + options.services.hyperhive.deploy = { + grafana.enable = lib.mkOption { + type = lib.types.bool; + default = false; + description = '' + Run the swarm's metrics UI on this host. + + Off by default and not derived from + {option}`services.hyperhive.enable`: a swarm has one Grafana, so + running it is a decision about this host rather than about + whether hyperhive is installed. + ''; + }; + + victoriametrics.enable = lib.mkOption { + type = lib.types.bool; + default = false; + description = '' + Run the swarm's metrics store on this host. + + Derives from + {option}`services.hyperhive.swarm.enableRequiredServices` together + with {option}`services.hyperhive.deploy.grafana.enable`: a store + with no UI is unreadable and a UI with no store is empty, so there + is no sensible deployment that takes one and not the other from + that switch. Set either directly to run exactly one. + ''; + }; + + victorialogs.enable = lib.mkOption { + type = lib.types.bool; + default = false; + description = '' + Run the swarm's log store on this host. + + Derives from + {option}`services.hyperhive.swarm.enableRequiredServices` for the + same reason as the metrics pair above: a hive that is not the + service host is a *client* of this store, not a second one. + ''; + }; + + authelia.enable = lib.mkOption { + type = lib.types.bool; + default = false; + example = true; + description = '' + Run the swarm's authelia in a `swarm-authelia` container on this + host. {option}`services.hyperhive.swarm.enableRequiredServices` + turns this on — a swarm has one SSO provider, and that says it + lives here. + + With it off, this hive is a *client*: + {option}`services.hyperhive.swarm.authelia.url` still points at + whoever runs it, and no container is created. That asymmetry is + why the two live in different namespaces — every hive needs the + client half, only one runs the server half. + ''; + }; + + swarm-otel.enable = lib.mkOption { + type = lib.types.bool; + default = false; + description = '' + Run the **swarm's** telemetry collector on this host. + + Derives from + {option}`services.hyperhive.swarm.enableRequiredServices` with the + metrics pair it feeds: a swarm has one of these, and it belongs + wherever the shared services live rather than on every hive. + + Named `swarm-otel` rather than `otel` because there are two + collectors and the tier is the whole distinction: + {option}`services.hyperhive.otel.enable` is the **hive-tier** one, + which every hive runs. A bare `deploy.otel` would not say which + it meant. A hive that does not run the swarm collector still runs + its own, and reaches this one by name at + {option}`services.hyperhive.swarm.otel.domain`. + ''; + }; + + nats.enable = lib.mkOption { + type = lib.types.bool; + default = false; + description = '' + Run the swarm's message queue in a `swarm-nats` container on this + host. A swarm has one queue, so this belongs on the same host as + the rest of the shared services. + + Off by default, and off means *absent*: no container is created + and nothing else in the evaluated config changes. + ''; + }; + + swarm-controller.enable = lib.mkOption { + type = lib.types.bool; + default = false; + description = '' + Run the swarm-controller daemon on this host. + + Off by default and deliberately not derived from + {option}`services.hyperhive.enable`: a swarm has one controller, + so running it is a decision about this host rather than about + whether hyperhive is installed. + ''; + }; + + swarm-ui.enable = lib.mkOption { + type = lib.types.bool; + default = deployCfg.swarm-controller.enable; + defaultText = lib.literalExpression "services.hyperhive.deploy.swarm-controller.enable"; + example = true; + description = '' + Serve the swarm UI from this host. + + Derived from + {option}`services.hyperhive.deploy.swarm-controller.enable` rather + than from + {option}`services.hyperhive.swarm.enableRequiredServices`: the UI + is a view onto the controller's state and reaches it over that + daemon's unix socket, so the host that runs the controller is the + host that can serve the UI. A hive that merely *uses* a swarm has + nothing to serve here. + ''; + }; + }; +} diff --git a/nix/host-modules/hive-forge/default.nix b/nix/host-modules/hive-forge/default.nix index d0fca0f6..8c0dc724 100644 --- a/nix/host-modules/hive-forge/default.nix +++ b/nix/host-modules/hive-forge/default.nix @@ -22,6 +22,7 @@ let # assertion below turns into an eval failure rather than a discovery # request to `null/.well-known/...`. autheliaCfg = config.services.hyperhive.swarm.authelia; + deployCfg = config.services.hyperhive.deploy; autheliaUrl = autheliaCfg.url; autheliaDiscoveryUrl = "${toString autheliaUrl}/.well-known/openid-configuration"; @@ -29,7 +30,7 @@ let # authelia, so the secret can be moved without an operator. The other # two cases (swarm side via swarmctl, remote hive) leave # `clientSecretFile` to be set explicitly — see docs/swarm/. - ssoLocal = autheliaCfg.enable; + ssoLocal = deployCfg.authelia.enable; # Where the plaintext lands inside the forge container. Under # /var/lib rather than /run: the forge may start before the delivery diff --git a/nix/host-modules/hive-matrix.nix b/nix/host-modules/hive-matrix.nix index 3b7aa588..8084f380 100644 --- a/nix/host-modules/hive-matrix.nix +++ b/nix/host-modules/hive-matrix.nix @@ -25,13 +25,14 @@ let # below turns into an eval failure rather than a discovery request to # `null/.well-known/…`. autheliaCfg = config.services.hyperhive.swarm.authelia; + deployCfg = config.services.hyperhive.deploy; autheliaUrl = autheliaCfg.url; # The all-local case: this host runs BOTH the homeserver and the swarm's # authelia, so the secret can be moved without an operator. The other # two cases (swarm side, remote hive) leave `clientSecretFile` to be set # explicitly — same split the forge module documents. - ssoLocal = autheliaCfg.enable; + ssoLocal = deployCfg.authelia.enable; # Where the plaintext lands inside the matrix container. Under /var/lib # rather than /run: the homeserver may start before the delivery unit on diff --git a/nix/host-modules/hive-tls.nix b/nix/host-modules/hive-tls.nix index 34b3b109..b8f1010d 100644 --- a/nix/host-modules/hive-tls.nix +++ b/nix/host-modules/hive-tls.nix @@ -669,6 +669,6 @@ in # in review on this PR; it evaluates and builds clean either way, which # is exactly why it needed a reviewer rather than a check. systemd.services.swarm-controller.environment.SWARM_CONTROLLER_OIDC_CA_FILE = - lib.mkIf hyperhiveCfg.swarm.controller.enable "${cfg.stateDir}/trust-bundle.pem"; + lib.mkIf hyperhiveCfg.deploy.swarm-controller.enable "${cfg.stateDir}/trust-bundle.pem"; }; } diff --git a/nix/host-modules/local-defaults.nix b/nix/host-modules/local-defaults.nix index aab9b180..5034eee5 100644 --- a/nix/host-modules/local-defaults.nix +++ b/nix/host-modules/local-defaults.nix @@ -33,7 +33,7 @@ in shared services (`services.hyperhive.swarm.enableRequiredServices`), the swarm CA (`services.hyperhive.swarm.ca.autoConfigure`), the swarm - controller (`services.hyperhive.swarm.controller.enable`), and the + controller (`services.hyperhive.deploy.swarm-controller.enable`), and the host's `/etc/hosts` entries for the names this hive serves (`services.hyperhive.gateway.localHostsEntry`) — with no real DNS for those names, the operator is browsing them from the same box @@ -80,22 +80,13 @@ in # reach whoever runs the responder, and minting them here would move # that hand-off somewhere less visible rather than removing it. nats.autoGenerateCallout = lib.mkDefault cfg.enableAllLocalDefaults; - # The controller is asserted by the MODE and by nothing else. Its own - # option stays `default = false` precisely because running it is a - # statement about swarm topology — but "this box is the whole - # deployment" IS that statement, and it is the one shape where the - # answer isn't ambiguous. Deriving it from `enableRequiredServices` - # instead would be wrong: a hive in a larger swarm can legitimately - # want the shared services without being the host that controls them. - controller.enable = lib.mkDefault cfg.enableAllLocalDefaults; - # The controller's queue coordinates. Co-location is what makes these # derivable at all — loopback only reaches the queue when the queue is # here, and the minted client secret only exists on the host authelia # ran its first boot on — so they belong to the mode that asserts this # box is the whole deployment, not to the options' own `default`. # - # Deriving them from `swarm.nats.enable` / `swarm.authelia.enable` + # Deriving them from `deploy.nats` / `deploy.authelia` # inside those defaults is the mixing this file exists to prevent: the # option would be describing a deployment shape instead of describing # itself, and "what does all-local turn on?" would stop having one @@ -117,4 +108,19 @@ in lib.mkDefault "${config.services.hyperhive.swarm.authelia.hostClientSecretDir}/swarm-controller.secret" ); }; + + # The controller is asserted by the MODE and by nothing else. Its own + # option stays `default = false` precisely because running it is a + # statement about swarm topology — but "this box is the whole + # deployment" IS that statement, and it is the one shape where the + # answer isn't ambiguous. Deriving it from `enableRequiredServices` + # instead would be wrong: a hive in a larger swarm can legitimately + # want the shared services without being the host that controls them. + # + # Sits outside the `swarm` attrset above because it is a `deploy.*` + # option (./deploy.nix): "does THIS host run the controller" is exactly + # the per-host fact `swarm.*` may not carry. The ⚠️ collision note above + # does not apply here — that one is about two definitions of `swarm` + # itself, and this is a different top-level path. + config.services.hyperhive.deploy.swarm-controller.enable = lib.mkDefault cfg.enableAllLocalDefaults; } diff --git a/nix/host-modules/otel.nix b/nix/host-modules/otel.nix index 40b9e1b8..19a44f6a 100644 --- a/nix/host-modules/otel.nix +++ b/nix/host-modules/otel.nix @@ -61,7 +61,7 @@ in their own containers can reach. That collector forwards to the swarm's - ({option}`services.hyperhive.swarm.otel.enable`), which holds the + ({option}`services.hyperhive.deploy.swarm-otel.enable`), which holds the upstream credential and writes the swarm's store. So an agent never sees the credential, and neither does this tier. @@ -81,13 +81,13 @@ in has left the swarm. Read by the swarm's collector - ({option}`services.hyperhive.swarm.otel.enable`), which is the only + ({option}`services.hyperhive.deploy.swarm-otel.enable`), which is the only tier that holds the upstream credential. An agent is handed the *first* hop instead — this hive's own collector — so this value is never given to a container. Optional. Leave it empty and the swarm's own metrics store - ({option}`services.hyperhive.swarm.victoriametrics.enable`) is the + ({option}`services.hyperhive.deploy.victoriametrics.enable`) is the destination; that is a complete deployment, not a degraded one. Set both and telemetry goes to both. ''; @@ -232,7 +232,7 @@ in type = lib.types.nullOr lib.types.str; default = if - config.services.hyperhive.swarm.authelia.enable && config.services.hyperhive.hiveName != null + config.services.hyperhive.deploy.authelia.enable && config.services.hyperhive.hiveName != null then "${config.services.hyperhive.swarm.authelia.hostClientSecretDir}/" + "${config.services.hyperhive.swarm.authelia.hiveClientPrefix}${config.services.hyperhive.hiveName}.secret" @@ -284,6 +284,7 @@ in let otel = config.services.hyperhive.otel; autheliaCfg = config.services.hyperhive.swarm.authelia; + deployCfg = config.services.hyperhive.deploy; swarmOtelCfg = config.services.hyperhive.swarm.otel; hiveName = config.services.hyperhive.hiveName; listen = "${config.services.hyperhive.network.bridgeIp}:${toString otel.collector.port}"; @@ -580,8 +581,8 @@ in # Only when the minting container is on THIS host. Elsewhere the file # is operator-provided and there is no local unit to order against — # naming one that does not exist orders nothing, silently. - after = lib.optional autheliaCfg.enable "container@${autheliaCfg.machine}.service"; - requires = lib.optional autheliaCfg.enable "container@${autheliaCfg.machine}.service"; + after = lib.optional deployCfg.authelia.enable "container@${autheliaCfg.machine}.service"; + requires = lib.optional deployCfg.authelia.enable "container@${autheliaCfg.machine}.service"; serviceConfig = { Type = "oneshot"; RemainAfterExit = true; diff --git a/nix/host-modules/swarm-authelia.nix b/nix/host-modules/swarm-authelia.nix index dfbb51ab..4c8ba9e7 100644 --- a/nix/host-modules/swarm-authelia.nix +++ b/nix/host-modules/swarm-authelia.nix @@ -42,6 +42,7 @@ let hyperhiveDomain = hyperhiveCfg.domain; swarmDomain = hyperhiveCfg.swarm.domain; uiCfg = hyperhiveCfg.swarm.ui; + deployCfg = hyperhiveCfg.deploy; forgeCfg = hyperhiveCfg.swarm.forge; # Group an account must hold to reach operator-only surfaces. Named @@ -344,22 +345,12 @@ let ''; in { + # `enable` moved to `services.hyperhive.deploy.authelia.enable` — see + # ./deploy.nix. Whether this host runs the swarm's SSO provider is a + # deployment decision; what stays here is what authelia IS, including + # `url` and the OIDC client registry every hive needs as a *client* + # whether or not it runs the container. options.services.hyperhive.swarm.authelia = { - enable = lib.mkOption { - type = lib.types.bool; - default = false; - example = true; - description = '' - Run the swarm's authelia in a `swarm-authelia` container on this - host. `services.hyperhive.swarm.enableRequiredServices` turns - this on — a swarm has one SSO provider, and that says it lives - here. - - With it off, this hive is a *client*: `url` below still points - at whoever runs it, and no container is created. - ''; - }; - package = lib.mkOption { type = lib.types.package; default = pkgs.authelia; @@ -436,7 +427,7 @@ in url = lib.mkOption { type = lib.types.nullOr lib.types.str; - default = if cfg.enable then "https://${cfg.domain}" else null; + default = if deployCfg.authelia.enable then "https://${cfg.domain}" else null; defaultText = lib.literalExpression ''if enable then "https://''${domain}" else null''; example = "https://auth.example.com"; description = '' @@ -476,8 +467,8 @@ in oidc.hiveIdentities = lib.mkOption { type = lib.types.bool; - default = hyperhiveCfg.swarm.nats.enable; - defaultText = lib.literalExpression "services.hyperhive.swarm.nats.enable"; + default = deployCfg.nats.enable; + defaultText = lib.literalExpression "services.hyperhive.deploy.nats.enable"; description = '' Mint one machine client per hive in {option}`services.hyperhive.swarm.hives`, so each hive can @@ -819,7 +810,7 @@ in bridgeUrl = lib.mkOption { type = lib.types.nullOr lib.types.str; readOnly = true; - default = if cfg.enable then "http://127.0.0.1:${toString cfg.bridgePort}" else null; + default = if deployCfg.authelia.enable then "http://127.0.0.1:${toString cfg.bridgePort}" else null; defaultText = lib.literalExpression ''if enable then "http://127.0.0.1:''${bridgePort}" else null''; description = '' Where `swarm-authelia-bridge` answers, **as seen from this @@ -838,7 +829,7 @@ in }; }; - config = lib.mkIf (hyperhiveCfg.enable && cfg.enable) { + config = lib.mkIf (hyperhiveCfg.enable && deployCfg.authelia.enable) { # The derived half of the client list, declared the same way an # operator declares one. Everything downstream then reads a single # uniformly-typed `cfg.oidc.clients` and cannot tell the parts apart — @@ -973,7 +964,7 @@ in # Authelia's own gateway surface: the vhost that fronts it and the # name the hive resolver answers for. Both live here rather than in - # the gateway, and both are inside `cfg.enable` — that guard is the + # the gateway, and both are inside `deployCfg.authelia.enable` — that guard is the # load-bearing part. # # ⚠️ Every hive in a swarm knows `authelia.url`, but only the host @@ -1018,9 +1009,11 @@ in # Written out in full rather than through a `let` binding so the gate and # the option it gates are visibly the same path: gating the wrong one is # not a build error, it is a target that is always declared or never is. - services.hyperhive.swarm.otel.scrapeTargets = lib.mkIf config.services.hyperhive.swarm.otel.enable { - authelia = "127.0.0.1:${toString cfg.metricsPort}"; - }; + services.hyperhive.swarm.otel.scrapeTargets = + lib.mkIf config.services.hyperhive.deploy.swarm-otel.enable + { + authelia = "127.0.0.1:${toString cfg.metricsPort}"; + }; # This swarm-ui quick-links entry, same guard as the vhost/DNS name # above (only the host actually running the container claims it — @@ -1406,7 +1399,7 @@ in # collector is registered — see `metricsRule` above, # which is where the reasoning for both halves lives. lib.optional forgeCfg.behindGateway metricsRule - ++ lib.optional uiCfg.enable { + ++ lib.optional deployCfg.swarm-ui.enable { domain = uiCfg.domain; subject = [ "group:${operatorGroup}" ]; policy = "one_factor"; diff --git a/nix/host-modules/swarm-controller.nix b/nix/host-modules/swarm-controller.nix index cd100f3e..fbc5a3f3 100644 --- a/nix/host-modules/swarm-controller.nix +++ b/nix/host-modules/swarm-controller.nix @@ -15,6 +15,7 @@ }: let cfg = config.services.hyperhive.swarm.controller; + deployCfg = config.services.hyperhive.deploy; autheliaCfg = config.services.hyperhive.swarm.authelia; # What `swarmctl` needs in order to act on authelia from the host. @@ -24,7 +25,7 @@ let # the right behaviour is for `swarmctl user add` to fail saying the # value is unset. A guessed path would resolve cleanly and write a file # nothing reads, which is the failure mode that costs an afternoon. - autheliaEnv = lib.optionalAttrs autheliaCfg.enable { + autheliaEnv = lib.optionalAttrs deployCfg.authelia.enable { # The CONFIGURED authelia, not whatever is on PATH: the argon2 # parameters baked into a hash have to match the verifier's. SWARMCTL_AUTHELIA_BIN = "${autheliaCfg.package}/bin/authelia"; @@ -119,7 +120,7 @@ let # ⚠️ Correct ONLY while the vhost and this daemon share a host, and they do # by construction: the UI's `/api/` location proxies # `http://unix:`, a path that resolves nowhere else, and this - # daemon binds no TCP address at all (see the header). `swarm.ui.enable` + # daemon binds no TCP address at all (see the header). `deploy.swarm-ui` # therefore is not a guess about *some* host publishing the endpoint — it is # the flag that declares that vhost, on the box holding the socket. # @@ -129,7 +130,7 @@ let # the moment to add an explicit `publicUrl` option — not before, because # until then there is exactly one derivable answer and an option would only # be a second place to get it wrong. - webhookEnv = lib.optionalAttrs uiCfg.enable { + webhookEnv = lib.optionalAttrs deployCfg.swarm-ui.enable { SWARM_CONTROLLER_PUBLIC_URL = "https://${uiCfg.domain}"; }; @@ -173,14 +174,14 @@ let # so the two ends agree on exactly one path, `/v1/metrics`, arrived at # from opposite directions. # - # Gated on `otelSwarmCfg.enable` alone (a swarm-wide fact, not "enabled + # Gated on `deployCfg.swarm-otel.enable` alone (a swarm-wide fact, not "enabled # HERE"): the collector is reachable by name wherever it runs, so a # controller not co-located with it now exports over https instead of # exporting nothing — the graceful-absence case left is "no swarm-otel # anywhere in this swarm at all," which `forgeEnv` already models the # same way. otelSwarmCfg = config.services.hyperhive.swarm.otel; - otelEnv = lib.optionalAttrs otelSwarmCfg.enable { + otelEnv = lib.optionalAttrs deployCfg.swarm-otel.enable { OTEL_EXPORTER_OTLP_ENDPOINT = "https://${otelSwarmCfg.domain}/${otelSwarmCfg.producerName}"; # Checked by `swarm-otel.nix`'s `oidc/${producerName}` authenticator # against exactly this controller's own registered audience (see the @@ -214,7 +215,7 @@ in name = "swarm-controller"; consumers = [ "swarm-controller" ]; hostUnit = true; - enable = cfg.enable; + enable = deployCfg.swarm-controller.enable; }) ]; @@ -238,22 +239,12 @@ in ''; }; - enable = lib.mkOption { - type = lib.types.bool; - default = false; - description = '' - Run the swarm-controller daemon on this host. Off by default and - deliberately not derived from `services.hyperhive.enable`: a swarm - has one controller, so enabling it per hive is a decision about - swarm topology, not about whether hyperhive is installed. - - `services.hyperhive.enableAllLocalDefaults` does assert it, and - that is not an exception to the rule above — it is the rule - applied. That mode says "this box is the whole deployment", which - answers the topology question outright, where - `services.hyperhive.enable` alone never can. - ''; - }; + # `enable` moved to `services.hyperhive.deploy.swarm-controller.enable` — see + # ./deploy.nix. `services.hyperhive.enableAllLocalDefaults` still + # asserts it, and that was never an exception to "not derived from + # services.hyperhive.enable": that mode says "this box is the whole + # deployment", which answers the topology question outright, where + # `enable` alone never can. What stays here is what the daemon IS. package = lib.mkOption { type = lib.types.package; @@ -460,7 +451,7 @@ in authBridgeUrl = lib.mkOption { type = lib.types.nullOr lib.types.str; - default = if autheliaCfg.enable then autheliaCfg.bridgeUrl else null; + default = if deployCfg.authelia.enable then autheliaCfg.bridgeUrl else null; defaultText = lib.literalExpression '' authelia's own `bridgeUrl` when this host also runs `swarm-authelia`, else null @@ -483,7 +474,7 @@ in }; }; - config = lib.mkIf (config.services.hyperhive.enable && cfg.enable) { + config = lib.mkIf (config.services.hyperhive.enable && deployCfg.swarm-controller.enable) { # The daemon and the oneshot that mints its credential — the second one # failing leaves the first running and unable to authenticate anywhere. services.hyperhive.swarm.otel.journaldUnits = [ @@ -508,7 +499,7 @@ in # client list would be a second source of truth for a string whose # mismatch is an opaque 401 from the token endpoint. Same shape as the # queue's own client declaration. - services.hyperhive.swarm.authelia.oidc.clients = lib.mkIf autheliaCfg.enable [ + services.hyperhive.swarm.authelia.oidc.clients = lib.mkIf deployCfg.authelia.enable [ { id = queueClientId; description = "HyperHive swarm controller"; @@ -548,7 +539,7 @@ in services.hyperhive.swarm.controller.queue.natsUrl is unset. It defaults to loopback only when this host also runs the queue - (`services.hyperhive.swarm.nats.enable`). A controller on its + (`services.hyperhive.deploy.nats.enable`). A controller on its own host has to be told where the queue is. ''; } diff --git a/nix/host-modules/swarm-grafana.nix b/nix/host-modules/swarm-grafana.nix index 1e9fde3f..4c4b159b 100644 --- a/nix/host-modules/swarm-grafana.nix +++ b/nix/host-modules/swarm-grafana.nix @@ -16,6 +16,7 @@ }: let cfg = config.services.hyperhive.swarm.grafana; + deployCfg = config.services.hyperhive.deploy; networkCfg = config.services.hyperhive.network; hyperhiveCfg = config.services.hyperhive; gatewayCfg = hyperhiveCfg.gateway; @@ -76,7 +77,7 @@ let # The all-local case: this host runs BOTH Grafana and the swarm's authelia, # so the minted secret can be moved without an operator. Same split the # forge and matrix modules document. - ssoLocal = cfg.enable && autheliaCfg.enable; + ssoLocal = deployCfg.grafana.enable && deployCfg.authelia.enable; autheliaUrl = toString autheliaCfg.url; # Where the plaintext lands inside the container. Under /var/lib rather @@ -106,18 +107,12 @@ let in { + # `enable` moved to `services.hyperhive.deploy.grafana.enable` — see + # ./deploy.nix. Whether this host runs the swarm's Grafana is a + # deployment decision, and `swarm.*` has to be identical on every host. + # What stays here is what the service IS: its package, domain, and + # wiring. options.services.hyperhive.swarm.grafana = { - enable = lib.mkOption { - type = lib.types.bool; - default = false; - description = '' - Run the swarm's metrics UI on this host. Off by default and not - derived from {option}`services.hyperhive.enable`: a swarm has one - Grafana, so enabling it is a decision about swarm topology rather - than about whether hyperhive is installed. - ''; - }; - package = lib.mkOption { type = lib.types.package; default = pkgs.grafana; @@ -312,8 +307,8 @@ in }; }; - config = lib.mkIf (hyperhiveCfg.enable && cfg.enable) { - # The gateway name and the quick-link, both inside `cfg.enable` — that + config = lib.mkIf (hyperhiveCfg.enable && deployCfg.grafana.enable) { + # The gateway name and the quick-link, both inside `deploy.grafana` — that # guard is the load-bearing part. Every hive in a swarm may know this UI # exists, but only the host that RUNS it may claim the name; a client # hive declaring the vhost would answer for a service it does not have. @@ -413,9 +408,11 @@ in # Declared here rather than in the collector's module, per that option's # rule: an entry exists where the service that named it runs. - services.hyperhive.swarm.otel.scrapeTargets = lib.mkIf config.services.hyperhive.swarm.otel.enable { - grafana = "127.0.0.1:${toString cfg.metricsPort}"; - }; + services.hyperhive.swarm.otel.scrapeTargets = + lib.mkIf config.services.hyperhive.deploy.swarm-otel.enable + { + grafana = "127.0.0.1:${toString cfg.metricsPort}"; + }; # Order the container after the host CA service so the bind source below # exists before nspawn sets the mount up. diff --git a/nix/host-modules/swarm-nats.nix b/nix/host-modules/swarm-nats.nix index 1e4d0b2b..8e98cac2 100644 --- a/nix/host-modules/swarm-nats.nix +++ b/nix/host-modules/swarm-nats.nix @@ -7,6 +7,7 @@ let cfg = config.services.hyperhive.swarm.nats; autheliaCfg = config.services.hyperhive.swarm.authelia; + deployCfg = config.services.hyperhive.deploy; autheliaUrl = autheliaCfg.url; networkCfg = config.services.hyperhive.network; # Read even when the controller runs on a different host: what is needed @@ -48,7 +49,7 @@ let hostPath = name: "${secretDir}/${name}"; # The responder needs all three credentials. Gating on them rather than - # on `cfg.enable` keeps a half-configured hive at "queue up, denying + # on `deployCfg.nats.enable` keeps a half-configured hive at "queue up, denying # everyone" instead of "unit crash-looping on a missing file". # # In auto mode the seeds are minted on this host before the container @@ -209,18 +210,9 @@ in # per-service vhost pattern do not apply. options.services.hyperhive.swarm.nats = { - enable = lib.mkOption { - type = lib.types.bool; - default = false; - description = '' - Run the swarm's message queue in a `swarm-nats` container on this - host. A swarm has one queue, so this belongs on the same host as - the rest of the shared services. - - Off by default, and off means *absent*: no container is created - and nothing else in the evaluated config changes. - ''; - }; + # `enable` moved to `services.hyperhive.deploy.nats.enable` — see + # ./deploy.nix. What stays here is what the queue IS: its domain, + # ports, accounts and callout wiring. # ⚠️ Deliberately NO `package` option, unlike this module's siblings. # `services.nats` upstream does not expose one — it resolves @@ -407,7 +399,7 @@ in }; }; - config = lib.mkIf cfg.enable { + config = lib.mkIf deployCfg.nats.enable { # The responder as well as the server: a denial reaches the client as a # timeout, so the server's own log is the only place it is an error. services.hyperhive.swarm.otel.journaldUnits = [ @@ -422,7 +414,7 @@ in # several layers from "the operator never set the issuer". assertion = cfg.autoGenerateCallout || cfg.calloutIssuerPublicKey != ""; message = '' - services.hyperhive.swarm.nats.enable requires + services.hyperhive.deploy.nats.enable requires nats.calloutIssuerPublicKey — the public half of the account nkey that signs user JWTs for this queue. @@ -439,7 +431,7 @@ in # somebody connects. assertion = cfg.autoGenerateCallout || cfg.calloutUserPublicKey != ""; message = '' - services.hyperhive.swarm.nats.enable requires + services.hyperhive.deploy.nats.enable requires nats.calloutUserPublicKey — the public half of the user nkey the auth-callout responder authenticates as. @@ -484,7 +476,7 @@ in { assertion = autheliaUrl != null; message = '' - services.hyperhive.swarm.nats.enable requires + services.hyperhive.deploy.nats.enable requires services.hyperhive.swarm.authelia.url — the queue authenticates clients by validating tokens that authelia issued. @@ -514,7 +506,7 @@ in # authorization grant 'client_credentials'`. Introspection — which # is all the responder needs today — worked throughout, which is why # nothing was visibly broken while the comment was untrue. - services.hyperhive.swarm.authelia.oidc.clients = lib.mkIf autheliaCfg.enable [ + services.hyperhive.swarm.authelia.oidc.clients = lib.mkIf deployCfg.authelia.enable [ { id = cfg.clientId; description = "HyperHive swarm queue"; @@ -543,9 +535,11 @@ in # ⚠️ `swarm.otel`, not `hyperhive.otel` — two collectors one word apart, # and only this one reads `scrapeTargets`. Written in full so the gate # and the option it gates are visibly the same path. - services.hyperhive.swarm.otel.scrapeTargets = lib.mkIf config.services.hyperhive.swarm.otel.enable { - nats = "127.0.0.1:${toString cfg.metricsPort}"; - }; + services.hyperhive.swarm.otel.scrapeTargets = + lib.mkIf config.services.hyperhive.deploy.swarm-otel.enable + { + nats = "127.0.0.1:${toString cfg.metricsPort}"; + }; containers.swarm-nats = { autoStart = true; @@ -683,7 +677,7 @@ in # door nobody can open, which is the deliberate interim state. # # ⚠️ It is gated on the seeds being configured rather than on - # `cfg.enable`, so a half-configured hive gets a running, + # `deployCfg.nats.enable`, so a half-configured hive gets a running, # refusing queue instead of a unit that crash-loops on a # missing file. A queue that denies everyone is a legible # failure; a restart loop is not. @@ -767,7 +761,7 @@ in # being up says nothing about whether its in-container secrets unit # has finished. The wait in the script is what actually closes it; # this only stops us spinning for the full timeout on every boot. - ++ lib.optional autheliaCfg.enable "container@${autheliaCfg.machine}.service"; + ++ lib.optional deployCfg.authelia.enable "container@${autheliaCfg.machine}.service"; requires = lib.optional cfg.autoGenerateCallout "swarm-nats-callout-keys.service"; serviceConfig = { Type = "oneshot"; diff --git a/nix/host-modules/swarm-otel.nix b/nix/host-modules/swarm-otel.nix index b39e66bd..baf64d13 100644 --- a/nix/host-modules/swarm-otel.nix +++ b/nix/host-modules/swarm-otel.nix @@ -29,6 +29,7 @@ }: let cfg = config.services.hyperhive.swarm.otel; + deployCfg = config.services.hyperhive.deploy; swarmCfg = config.services.hyperhive.swarm; otelCfg = config.services.hyperhive.otel; vmCfg = config.services.hyperhive.swarm.victoriametrics; @@ -162,13 +163,15 @@ let # exactly the same destinations as the single pipeline they replace. # Written once because "which exporters" is a property of this tier, not # of which hive a sample came from. - exporterNames = upstreamExporters ++ lib.optional vmCfg.enable "otlphttp/victoriametrics"; + exporterNames = + upstreamExporters ++ lib.optional deployCfg.victoriametrics.enable "otlphttp/victoriametrics"; # The same fan-out for logs, and the local store is only ONE of its # destinations. A deployment that turns the swarm's log store off and keeps # an upstream endpoint still collects — the store is where logs may be kept, # not the reason to read the journal at all. - logExporterNames = upstreamExporters ++ lib.optional vlCfg.enable "otlphttp/victorialogs"; + logExporterNames = + upstreamExporters ++ lib.optional deployCfg.victorialogs.enable "otlphttp/victorialogs"; # Collect when there is anywhere to send it, and only then. A pipeline with # an empty exporter list is not a quiet no-op — the collector rejects it — @@ -177,23 +180,11 @@ let collectLogs = logExporterNames != [ ]; in { + # `enable` moved to `services.hyperhive.deploy.swarm-otel.enable` — see ./deploy.nix. + # ⚠️ That is the SWARM collector. The per-hive one keeps its own + # `services.hyperhive.otel.enable` (./otel.nix) and is a different + # option entirely — every hive runs that one. options.services.hyperhive.swarm.otel = { - enable = lib.mkOption { - type = lib.types.bool; - default = false; - description = '' - Run the swarm's telemetry collector on this host. - - Asserted from `swarm.enableRequiredServices` in - ./swarm-required-services.nix, with the metrics pair this - collector feeds: a swarm has one of these, and it belongs - wherever the shared services live rather than on every hive. - - A hive that does not run it still runs its own hive-tier collector - (`services.hyperhive.otel.enable`) and reaches this one by name, at - {option}`services.hyperhive.swarm.otel.domain`. - ''; - }; machine = lib.mkOption { type = lib.types.str; @@ -444,8 +435,8 @@ in }; }; - config = lib.mkIf (config.services.hyperhive.enable && cfg.enable) { - # The gateway name, inside `cfg.enable` — that guard is the load-bearing + config = lib.mkIf (config.services.hyperhive.enable && deployCfg.swarm-otel.enable) { + # The gateway name, inside `deployCfg.swarm-otel.enable` — that guard is the load-bearing # part. Every hive in a swarm may know this collector exists, but only # the host that RUNS it may claim the name; a client hive declaring the # vhost would answer for a service it does not have. @@ -515,7 +506,7 @@ in # declaration would break every hive that runs a collector and # publishes nothing. services.hyperhive.swarm.authelia.oidc.clients = - lib.mkIf (autheliaCfg.enable && cfg.publishedScrapeTargets != { }) + lib.mkIf (deployCfg.authelia.enable && cfg.publishedScrapeTargets != { }) [ { id = cfg.clientId; @@ -553,7 +544,7 @@ in # after it. On a fresh swarm that is a permanent stall presenting as # "metrics are broken", several layers from its cause. systemd.services.swarm-otel-oidc-secret = - lib.mkIf (autheliaCfg.enable && cfg.publishedScrapeTargets != { }) + lib.mkIf (deployCfg.authelia.enable && cfg.publishedScrapeTargets != { }) { description = "deliver the swarm collector's OIDC client secret from authelia"; after = [ "container@${autheliaCfg.machine}.service" ]; @@ -608,12 +599,12 @@ in # The tier exists to hold the upstream credential and to write the # swarm's store. With neither, it is a process that receives # samples and drops them — which looks healthy and loses data. - assertion = otelCfg.endpoint != "" || vmCfg.enable; + assertion = otelCfg.endpoint != "" || deployCfg.victoriametrics.enable; message = '' - services.hyperhive.swarm.otel.enable is true but this collector + services.hyperhive.deploy.swarm-otel.enable is true but this collector has nowhere to send what it receives: services.hyperhive.otel.endpoint is empty and - services.hyperhive.swarm.victoriametrics.enable is false. + services.hyperhive.deploy.victoriametrics.enable is false. Set the endpoint to export upstream, or enable the swarm's metrics store. @@ -642,7 +633,7 @@ in # collector would listen on nothing while looking configured. assertion = hyperhiveCfg.swarm.hives != { }; message = '' - services.hyperhive.swarm.otel.enable is true but + services.hyperhive.deploy.swarm-otel.enable is true but services.hyperhive.swarm.hives is empty: ingest is authenticated per hive, so an empty roster means this collector accepts nothing from anyone. @@ -754,7 +745,7 @@ in # issuer URL evaluates cleanly and refuses every hive at runtime. assertion = autheliaCfg.url != null; message = '' - services.hyperhive.swarm.otel.enable is true but + services.hyperhive.deploy.swarm-otel.enable is true but services.hyperhive.swarm.authelia.url is null: every hive authenticates to this collector as itself, and the token comes from the swarm's identity provider. @@ -780,7 +771,7 @@ in cfg.producerPort otelCfg.collector.port ] - ++ lib.optional vmCfg.enable vmCfg.port; + ++ lib.optional deployCfg.victoriametrics.enable vmCfg.port; all = derived ++ others; in lib.length (lib.unique all) == lib.length all; @@ -1015,7 +1006,7 @@ in }; exporters = - lib.optionalAttrs vmCfg.enable { + lib.optionalAttrs deployCfg.victoriametrics.enable { # `metrics_endpoint`, NOT `endpoint`: the latter is a # base that otlphttp appends `/v1/metrics` to, while # VictoriaMetrics serves OTLP at @@ -1038,7 +1029,7 @@ in } // lib.optionalAttrs (otelCfg.protocol == "http/json") { encoding = "json"; }; } - // lib.optionalAttrs vlCfg.enable { + // lib.optionalAttrs deployCfg.victorialogs.enable { # `logs_endpoint`, NOT `endpoint`, for exactly the reason the # metrics exporter above spells out — and the trap is worse # here, because the two stores' OTLP routes differ. `endpoint` diff --git a/nix/host-modules/swarm-required-services.nix b/nix/host-modules/swarm-required-services.nix index a40b7e56..8e8e5fc1 100644 --- a/nix/host-modules/swarm-required-services.nix +++ b/nix/host-modules/swarm-required-services.nix @@ -24,9 +24,13 @@ in description = '' Host the swarm's shared services on this hive. The services that exist once per swarm rather than once per hive and are *optional* - — the matrix homeserver, the SSO provider — have their `enable` - asserted from this, so a swarm's service host is declared in one - place. + — the matrix homeserver, the SSO provider, the queue, the metrics + and log stores — have their toggle asserted from this, so a + swarm's service host is declared in one place. + + Those toggles live in two namespaces and the split is deliberate: + {option}`services.hyperhive.deploy.*` for "does THIS host run it", + `swarm.*.enable` for the ones not yet moved. See ./deploy.nix. The forge is swarm-wide too but has nothing to assert: it is the canonical store for the meta flake and every agent's config repo, @@ -45,47 +49,49 @@ in # Same precedence reasoning as ./local-defaults.nix: fills in for an # operator who hasn't spoken, yields to one who has. config.services.hyperhive.swarm = { + # The last one still spelled `swarm.*.enable`. Everything else that + # used to derive here — the queue, the SSO provider, the collector, + # the metrics pair, the log store — now derives below under + # `deploy.*`, because "does THIS host run it" is a per-host decision + # and `swarm.*` has to be identical on every host. Same switch, same + # rule, different attribute path. matrix.enable = lib.mkDefault swarmCfg.enableRequiredServices; - authelia.enable = lib.mkDefault swarmCfg.enableRequiredServices; - # The queue. Added later than the two above and missed at the time — - # this file predates the `swarm-nats` container by nine days and had - # not been revisited since, so its absence was sequence rather than - # intent. It meets the rule in the option's own description exactly: - # once per swarm, and optional. - # - # The tell that it was an omission: `local-defaults.nix` already - # derives `nats.autoGenerateCallout` from the all-local mode, so that - # mode was minting the queue's callout nkeys and then never starting - # the queue they authenticate against. - nats.enable = lib.mkDefault swarmCfg.enableRequiredServices; - - # The metrics pair. Once per swarm and optional, so they meet the rule - # in the option's description the same way the three above do — a hive - # that is not the service host is a *client* of this Grafana, not a - # second one. - # - # They derive together on purpose: a store with no UI is unreadable and - # a UI with no store is empty, so there is no sensible deployment that - # takes one and not the other from this switch. An operator who wants - # exactly one still sets it directly, which `mkDefault` allows. - victoriametrics.enable = lib.mkDefault swarmCfg.enableRequiredServices; - grafana.enable = lib.mkDefault swarmCfg.enableRequiredServices; - - # The log store, deriving from the same switch for the same reason — - # and deliberately in the same commit as the collector pipeline that - # writes to it, never before it. A store nothing writes to is worse - # than no store: it starts, answers queries, and returns nothing, so - # the first person to look concludes there were no logs. - victorialogs.enable = lib.mkDefault swarmCfg.enableRequiredServices; - - # The collector that feeds the pair above, and the only tier holding - # the upstream credential. Same rule as the rest: once per swarm, - # optional, and a hive that is not the service host is a *client* of - # it (by name, `swarm.otel.domain`) rather than a second one. - otel.enable = lib.mkDefault swarmCfg.enableRequiredServices; }; # The collector that feeds the pair above (note: no `swarm.` prefix, # this is ./otel.nix's existing per-hive option). config.services.hyperhive.otel.enable = lib.mkDefault swarmCfg.enableRequiredServices; + + # The rest of the shared services, deriving from the same switch as the + # `swarm.*` ones above. They read differently only because "does THIS + # host run it" lives in `deploy.*` (./deploy.nix) — `swarm.*` has to be + # identical on every host, and these are exactly the values that must + # differ. + # + # authelia: a swarm has one SSO provider, and this says it lives here. + # With it off the hive is a *client* — `swarm.authelia.url` still points + # at whoever runs it. + config.services.hyperhive.deploy.authelia.enable = lib.mkDefault swarmCfg.enableRequiredServices; + + # The queue. Same rule: once per swarm, optional. + config.services.hyperhive.deploy.nats.enable = lib.mkDefault swarmCfg.enableRequiredServices; + + # The swarm collector that feeds the metrics pair, and the only tier + # holding the upstream credential. ⚠️ NOT the per-hive collector below, + # which every hive runs. + config.services.hyperhive.deploy.swarm-otel.enable = lib.mkDefault swarmCfg.enableRequiredServices; + + # The metrics pair, deriving together on purpose: a store with no UI is + # unreadable and a UI with no store is empty, so there is no sensible + # deployment that takes one and not the other from this switch. An + # operator who wants exactly one still sets it directly, which + # `mkDefault` allows. + config.services.hyperhive.deploy.victoriametrics.enable = + lib.mkDefault swarmCfg.enableRequiredServices; + config.services.hyperhive.deploy.grafana.enable = lib.mkDefault swarmCfg.enableRequiredServices; + + # The log store, from the same switch for the same reason as the rest: a + # hive that is not the service host is a *client* of it, not a second one. + config.services.hyperhive.deploy.victorialogs.enable = + lib.mkDefault swarmCfg.enableRequiredServices; } diff --git a/nix/host-modules/swarm-ui.nix b/nix/host-modules/swarm-ui.nix index e92b5cec..1cc72ec2 100644 --- a/nix/host-modules/swarm-ui.nix +++ b/nix/host-modules/swarm-ui.nix @@ -14,6 +14,7 @@ }: let cfg = config.services.hyperhive.swarm.ui; + deployCfg = config.services.hyperhive.deploy; gatewayCfg = config.services.hyperhive.gateway; autheliaCfg = config.services.hyperhive.swarm.authelia; controllerCfg = config.services.hyperhive.swarm.controller; @@ -56,23 +57,11 @@ let hiveDomain = config.services.hyperhive.domain; in { + # `enable` moved to `services.hyperhive.deploy.swarm-ui.enable` — see + # ./deploy.nix, where it still derives from the controller's own deploy + # toggle for the same reason. What stays here is what the UI IS: its + # domain and wiring. options.services.hyperhive.swarm.ui = { - enable = lib.mkOption { - type = lib.types.bool; - default = swarmCfg.controller.enable; - defaultText = lib.literalExpression "services.hyperhive.swarm.controller.enable"; - example = true; - description = '' - Serve the swarm UI from this host. - - Derived from `swarm.controller.enable` rather than from - `enableRequiredServices`: the UI is a view onto the controller's - state and reaches it over that daemon's unix socket, so the host - that runs the controller is the host that can serve the UI. A - hive that merely *uses* a swarm has nothing to serve here. - ''; - }; - domain = lib.mkOption { type = lib.types.str; default = if swarmCfg.domain == null then "swarm.invalid" else swarmCfg.domain; @@ -110,7 +99,7 @@ in }; }; - config = lib.mkIf (config.services.hyperhive.enable && cfg.enable) { + config = lib.mkIf (config.services.hyperhive.enable && deployCfg.swarm-ui.enable) { assertions = [ { # The `_` default server already answers for the hive domain diff --git a/nix/host-modules/swarm-victorialogs.nix b/nix/host-modules/swarm-victorialogs.nix index 68f87a1f..aadd8ec3 100644 --- a/nix/host-modules/swarm-victorialogs.nix +++ b/nix/host-modules/swarm-victorialogs.nix @@ -29,6 +29,7 @@ }: let cfg = config.services.hyperhive.swarm.victorialogs; + deployCfg = config.services.hyperhive.deploy; networkCfg = config.services.hyperhive.network; hyperhiveCfg = config.services.hyperhive; gatewayCfg = hyperhiveCfg.gateway; @@ -52,18 +53,10 @@ let ''; in { + # `enable` moved to `services.hyperhive.deploy.victorialogs.enable` — see + # ./deploy.nix. What stays here is what the store IS: its package, + # domain, retention and wiring. options.services.hyperhive.swarm.victorialogs = { - enable = lib.mkOption { - type = lib.types.bool; - default = false; - description = '' - Run the swarm's log store on this host. Off by default and not - derived from {option}`services.hyperhive.enable`: a swarm has one - log store, so enabling it is a decision about swarm topology - rather than about whether hyperhive is installed. - ''; - }; - package = lib.mkOption { type = lib.types.package; default = pkgs.victorialogs; @@ -128,7 +121,7 @@ in }; }; - config = lib.mkIf (hyperhiveCfg.enable && cfg.enable) { + config = lib.mkIf (hyperhiveCfg.enable && deployCfg.victorialogs.enable) { # This store publishes its own health as prometheus metrics on the same # listener it serves queries on, so the swarm's collector scrapes it with # no exporter and no extra port — same arrangement as the metrics store. @@ -141,7 +134,7 @@ in # hosts are separate evaluations. services.hyperhive.swarm.otel.scrapeTargets.victorialogs = "127.0.0.1:${toString cfg.port}"; - # The gateway name and the quick-link, both inside `cfg.enable` — same + # The gateway name and the quick-link, both inside `deployCfg.victorialogs.enable` — same # "only the host that runs the service may claim the name" guard every # sibling swarm-service module uses (`swarm-grafana.nix`, # `swarm-victoriametrics.nix`). diff --git a/nix/host-modules/swarm-victoriametrics.nix b/nix/host-modules/swarm-victoriametrics.nix index a991f989..3fa29305 100644 --- a/nix/host-modules/swarm-victoriametrics.nix +++ b/nix/host-modules/swarm-victoriametrics.nix @@ -17,6 +17,7 @@ }: let cfg = config.services.hyperhive.swarm.victoriametrics; + deployCfg = config.services.hyperhive.deploy; networkCfg = config.services.hyperhive.network; hyperhiveCfg = config.services.hyperhive; gatewayCfg = hyperhiveCfg.gateway; @@ -28,18 +29,10 @@ let domainBase = if swarmDomain == null then "invalid" else swarmDomain; in { + # `enable` moved to `services.hyperhive.deploy.victoriametrics.enable` — see + # ./deploy.nix. What stays here is what the store IS: its package, + # domain, retention and wiring. options.services.hyperhive.swarm.victoriametrics = { - enable = lib.mkOption { - type = lib.types.bool; - default = false; - description = '' - Run the swarm's metrics store on this host. Off by default and not - derived from {option}`services.hyperhive.enable`: a swarm has one - metrics store, so enabling it is a decision about swarm topology - rather than about whether hyperhive is installed. - ''; - }; - package = lib.mkOption { type = lib.types.package; default = pkgs.victoriametrics; @@ -97,8 +90,8 @@ in }; }; - config = lib.mkIf (hyperhiveCfg.enable && cfg.enable) { - # The gateway name and the quick-link, both inside `cfg.enable` — that + config = lib.mkIf (hyperhiveCfg.enable && deployCfg.victoriametrics.enable) { + # The gateway name and the quick-link, both inside `deployCfg.victoriametrics.enable` — that # guard is the load-bearing part. Every hive in a swarm may know this # store exists, but only the host that RUNS it may claim the name; a # client hive declaring the vhost would answer for a service it does not diff --git a/nix/host-modules/swarm.nix b/nix/host-modules/swarm.nix index aad8691a..9bedc776 100644 --- a/nix/host-modules/swarm.nix +++ b/nix/host-modules/swarm.nix @@ -23,6 +23,7 @@ let cfg = config.services.hyperhive; swarmCfg = cfg.swarm; + deployCfg = cfg.deploy; # Public hostnames of the swarm's own services, in declaration order. # `serviceDomains` below is this set sorted + deduplicated. @@ -45,7 +46,7 @@ let # `forge.` is, and no CA in the hierarchy issues for it # implicitly. Left out, its vhost falls back to the hive leaf and the # swarm's front page opens with a name mismatch. - ++ lib.optional swarmCfg.ui.enable swarmCfg.ui.domain + ++ lib.optional deployCfg.swarm-ui.enable swarmCfg.ui.domain # Every swarm service that claims a gateway name belongs here, and # these three were missing it. Membership is what `gateway.lib.tlsFor` # consults to pick the services leaf over the hive one, so a name @@ -57,16 +58,16 @@ let # metrics UI and store looked fine for as long as only people opened # them; the collector's exporter — same defect, no human in the loop — # failed every POST and dropped the samples. - ++ lib.optional swarmCfg.grafana.enable swarmCfg.grafana.domain - ++ lib.optional swarmCfg.victoriametrics.enable swarmCfg.victoriametrics.domain - ++ lib.optional swarmCfg.otel.enable swarmCfg.otel.domain + ++ lib.optional deployCfg.grafana.enable swarmCfg.grafana.domain + ++ lib.optional deployCfg.victoriametrics.enable swarmCfg.victoriametrics.domain + ++ lib.optional deployCfg.swarm-otel.enable swarmCfg.otel.domain # VictoriaLogs' vhost is new (was previously unpublished entirely — see # swarm-victorialogs.nix's file-top comment) and needs the same # membership every gateway-published swarm service needs: absent from # this list, `gateway.lib.tlsFor` falls back to the hive leaf, which # cannot cover a name under a different apex — see the ⚠️ above this # list for what that looked like the last time a name was missed here. - ++ lib.optional swarmCfg.victorialogs.enable swarmCfg.victorialogs.domain; + ++ lib.optional deployCfg.victorialogs.enable swarmCfg.victorialogs.domain; # Hives whose entry still carries the removed `certFingerprint`. Scanned # here, at top level, because that is the only place an assertion about a @@ -79,18 +80,23 @@ let # ONE condition for all three of them, deliberately. # # 🩸 They were three independent conditions first, and that was wrong in - # a way only an eval gate finds: `enableRequiredServices` turns on - # matrix and authelia but NOT nats (nats has no mode that enables it — - # see the auto-deploy question on the swarm-queue issue), so an ordinary - # all-local hive resolved authelia's two coordinates and not the queue - # URL. Two of three set is exactly what the assertion below rejects, so - # every `enableAllLocalDefaults` hive would have stopped evaluating. + # a way only an eval gate finds: at the time `enableRequiredServices` + # turned on matrix and authelia but NOT nats, so an ordinary all-local + # hive resolved authelia's two coordinates and not the queue URL. Two of + # three set is exactly what the assertion below rejects, so every + # `enableAllLocalDefaults` hive would have stopped evaluating. + # + # (The queue does derive from that switch now — ./swarm-required-services.nix + # — so the original asymmetry is gone. The reasoning below is why the + # single predicate stays anyway: it makes the partial state + # unrepresentable rather than merely detected, which does not depend on + # which services happen to derive together this month.) # # Deriving all three from one predicate makes the partial state # unrepresentable rather than merely detected: a default set is all or # nothing, and the assertion is then only ever about what an operator # typed. - queueLocal = swarmCfg.nats.enable && swarmCfg.authelia.enable && cfg.hiveName != null; + queueLocal = deployCfg.nats.enable && deployCfg.authelia.enable && cfg.hiveName != null; in { options.services.hyperhive.swarm.hives = lib.mkOption { diff --git a/nix/module-eval.nix b/nix/module-eval.nix index 29d70fa2..9f2e6435 100644 --- a/nix/module-eval.nix +++ b/nix/module-eval.nix @@ -90,11 +90,11 @@ let cases = [ { name = "a hive that has not opted into all-local runs no swarm controller"; - ok = !bare.services.hyperhive.swarm.controller.enable; + ok = !bare.services.hyperhive.deploy.swarm-controller.enable; } { name = "the all-local mode turns the swarm controller on"; - ok = allLocal.services.hyperhive.swarm.controller.enable; + ok = allLocal.services.hyperhive.deploy.swarm-controller.enable; } { # The gateway's per-name issuer choice. If this ever collapses to a diff --git a/swarm-controller/README.md b/swarm-controller/README.md index fe96e5f2..da5fd893 100644 --- a/swarm-controller/README.md +++ b/swarm-controller/README.md @@ -4,7 +4,7 @@ The **swarm-level** daemon. Where `hive-c0re` owns the agents on one host, this owns what is true *across* hives — so a swarm runs one of them and most hives leave it off. -Opt-in per host via `services.hyperhive.swarm.controller.enable`, which is +Opt-in per host via `services.hyperhive.deploy.swarm-controller.enable`, which is deliberately **not** derived from `services.hyperhive.enable`: turning it on is a statement about swarm topology, not about whether hyperhive is installed. diff --git a/swarm-controller/src/main.rs b/swarm-controller/src/main.rs index ea09dfc9..c058c696 100644 --- a/swarm-controller/src/main.rs +++ b/swarm-controller/src/main.rs @@ -1,6 +1,6 @@ //! Swarm-level controller daemon. Runs as the unprivileged //! `swarm-controller` user on whichever host the operator flips -//! `services.hyperhive.swarm.controller.enable` on, and serves HTTP over a +//! `services.hyperhive.deploy.swarm-controller.enable` on, and serves HTTP over a //! unix socket that the hive-gateway's nginx proxies to. //! //! Configuration is read-only and loaded once at startup from env vars the