Watch
0
0
Fork
You've already forked hyperhive
0

nix: run the forge on one host per swarm (deploy.forgejo.enable)

Every hive with hyperhive enabled ran its own hive-forge container, and
its gateway answered forge.<swarm> with its own bridge IP, so on a
multi-host swarm each hive talked to its own forge.

deploy.forgejo.enable defaults to false and allSwarmServices sets it with
mkDefault, like authelia and bao; singleHostSwarm gets it through that.
The forge's OIDC client moves to a glue module gated on authelia, so a
split authelia/forge swarm still registers it. CI now requires the forge
on the same host, and the controller's forgeTokenFile defaults to null
where the forge is not.

Closes #4705
Refs #3782
This commit is contained in:
atlas 2026-09-24 19:44:04 +02:00 • committed by mara
commit 978164dc53
15 changed files with 346 additions and 84 deletions

View file

@ -457,8 +457,9 @@ reconcile) DAGs use the same queue but skip the approval plumbing.
### Forge mirror ### Forge mirror
The bundled `hive-forge` container is mandatory (it deploys with The bundled `hive-forge` container runs on the swarm's forge host
hyperhive), and hive-c0re mirrors every agent's applied repo into a (`deploy.forgejo.enable`, see [`../swarm/services.md`](../swarm/services.md)),
and hive-c0re mirrors every agent's applied repo into a
private `agent-configs` Forgejo org. `forge::push_config(<name>)` pushes `applied/main` plus private `agent-configs` Forgejo org. `forge::push_config(<name>)` pushes `applied/main` plus
every tag to `agent-configs/<name>` after each ref mutation: every tag to `agent-configs/<name>` after each ref mutation:
the spawn that seeds `deployed/0`, every successful deploy (which the spawn that seeds `deployed/0`, every successful deploy (which

View file

@ -346,6 +346,11 @@ State lives at `/var/lib/nixos-containers/hive-forge/var/lib/forgejo/`
and survives container restart / host reboot. To wipe, destroy the and survives container restart / host reboot. To wipe, destroy the
container. container.
The container runs only on the swarm's forge host
(`services.hyperhive.deploy.forgejo.enable`, see
[`../swarm/services.md`](../swarm/services.md)). Every other hive
reaches that host's gateway by `forge.<swarm-domain>`.
### Network and port configuration ### Network and port configuration
```nix ```nix

View file

@ -105,8 +105,10 @@ slow); run those manually before pushing Rust changes.
## Configuration reference ## Configuration reference
The internal forge is always present (mandatory), so the runner always has a The runner registers against the forge on its own host, so enable it on the
hive-forge instance to register against — nothing extra to enable beyond swarm's forge host (`services.hyperhive.deploy.forgejo.enable`, see
[`../swarm/services.md`](../swarm/services.md)). Evaluation fails on any other
host. There, nothing extra to enable beyond
`services.hyperhive.deploy.forgejo.ci.enable = true` (see _For operators_ above). `services.hyperhive.deploy.forgejo.ci.enable = true` (see _For operators_ above).
Optional tuning: `services.hyperhive.deploy.forgejo.ci.name` (runner name in forge Optional tuning: `services.hyperhive.deploy.forgejo.ci.name` (runner name in forge

View file

@ -10,8 +10,7 @@ services.hyperhive.deploy.allSwarmServices = true;
``` ```
**`deploy.allSwarmServices` is what "the swarm's shared services run **`deploy.allSwarmServices` is what "the swarm's shared services run
here" means: every once-per-swarm service that's _optional_ takes its here" means: every once-per-swarm service takes its `enable` from it.** That's the whole rule, stated once — the per-service
`enable` from it.** That's the whole rule, stated once — the per-service
sections below don't repeat it, so a service that stops deriving is a sections below don't repeat it, so a service that stops deriving is a
visible difference rather than one more paragraph saying the same thing. visible difference rather than one more paragraph saying the same thing.
@ -26,10 +25,24 @@ operator saying so rather than something inferred. With them off, a hive
is a _client_ of those services — it configures how to reach them and is a _client_ of those services — it configures how to reach them and
runs none of them. runs none of them.
The forge is the exception, and not because it's per-hive: it's That includes the forge (`deploy.forgejo.enable`). Every swarm needs
swarm-wide but **not optional**, being the canonical store for the meta one, being the canonical store for the meta flake and every agent's
flake and every agent's config repo, so it deploys with hyperhive itself config repo, so **exactly one host must turn it on**: `singleHostSwarm`,
and has no `enable` to derive from anything. `allSwarmServices`, or `deploy.forgejo.enable = true` set by hand. A
host that enables its services one by one without any of those runs no
forge.
A hive that runs none of these reaches each one by name — the forge at
`forge.<swarm-domain>`, for example. Only the host running a service
answers its name from its own resolver, so on a swarm spread over
more than one host, the operator's DNS has to resolve those names to that
host.
A hive that stops running the forge keeps the old container's state at
`/var/lib/nixos-containers/hive-forge/`. Nothing moves it to the swarm's
forge: push anything worth keeping there by hand. Its
`/var/lib/hyperhive/forge-core-token` came from that old forge and
fails against the swarm's one.
## Deployment shapes ## Deployment shapes

View file

@ -51,6 +51,10 @@ in
inherit pkgs self nixosSystem; inherit pkgs self nixosSystem;
inherit (pkgs) lib; inherit (pkgs) lib;
}; };
module-eval-forge-placement = import ./module-eval/forge-placement.nix {
inherit pkgs self nixosSystem;
inherit (pkgs) lib;
};
module-eval-bao-basics = import ./module-eval/bao-basics.nix { module-eval-bao-basics = import ./module-eval/bao-basics.nix {
inherit pkgs self nixosSystem; inherit pkgs self nixosSystem;
inherit (pkgs) lib; inherit (pkgs) lib;

View file

@ -3,10 +3,11 @@
# the package/source wiring; see flake.nix). One import covers # the package/source wiring; see flake.nix). One import covers
# everything; `services.hyperhive.enable = true` turns the stack on. # everything; `services.hyperhive.enable = true` turns the stack on.
# #
# The forge is mandatory — hive-c0re mirrors every agent's applied # Every swarm needs the forge — hive-c0re mirrors every agent's applied
# config repo into it and it's the canonical store for the meta flake # config repo into it and it's the canonical store for the meta flake
# + `internal/*` repos, so there's no enable toggle; it deploys with # + `internal/*` repos — but only the host with `deploy.forgejo.enable`
# hyperhive itself. hive-matrix is opt-in (off by default). All # runs it (derived from `deploy.allSwarmServices`, see
# ./swarm-required-services.nix). hive-matrix is opt-in (off by default). All
# subsystems rely on `services.hyperhive.domain`, which is required # subsystems rely on `services.hyperhive.domain`, which is required
# (asserted in hive-network.nix) whenever hyperhive is enabled. # (asserted in hive-network.nix) whenever hyperhive is enabled.
{ {
@ -26,6 +27,7 @@
./glue-bao-readers-policy-order.nix ./glue-bao-readers-policy-order.nix
./glue-bao-tls.nix ./glue-bao-tls.nix
./glue-controller-bao-identity.nix ./glue-controller-bao-identity.nix
./glue-forge-oidc-client.nix
./glue-grafana-oidc-client.nix ./glue-grafana-oidc-client.nix
./glue-matrix-bao-token.nix ./glue-matrix-bao-token.nix
./glue-matrix-ctl-bao-identity.nix ./glue-matrix-ctl-bao-identity.nix

View file

@ -418,14 +418,33 @@ in
) )
]; ];
# ⚠️ `deploy.forgejo` is declared in ./hive-ci.nix, not here, and it is the # ⚠️ Only `deploy.forgejo.enable` is declared here. The rest of
# one entry with no `enable`: the forge is not optional — it is the canonical # `deploy.forgejo` is in ./hive-forge/default.nix and ./hive-ci.nix: the
# store for the meta flake and every agent's config repo, so it deploys with # knobs live with the module that reads them; this file stays the registry
# hyperhive itself. Running the CI runner is the only *deployment* decision # of toggles.
# it has, which is exactly the `{ enable; ci; }` shape the header describes,
# minus the half that does not apply. The knobs live with the module that
# reads them; this file stays the registry of toggles.
options.services.hyperhive.deploy = { options.services.hyperhive.deploy = {
forgejo.enable = lib.mkOption {
type = lib.types.bool;
default = false;
example = true;
description = ''
Run the swarm's forge in a `hive-forge` container on this host. A
swarm has one forge, so one host turns this on. It is the
canonical store for the meta flake and every agent's config repo,
so the swarm needs it, but *this* host running it is a decision
like any other shared service's.
With it off, this hive is a *client*: it reaches the swarm's
forge at {option}`services.hyperhive.swarm.forge.domain`, which
the operator's DNS must resolve to the forge's host. No container
is created, and an existing one's state stays on disk under
`/var/lib/nixos-containers/hive-forge/`.
{option}`services.hyperhive.deploy.allSwarmServices` turns it on,
and `singleHostSwarm` through that.
'';
};
grafana.enable = lib.mkOption { grafana.enable = lib.mkOption {
type = lib.types.bool; type = lib.types.bool;
default = false; default = false;

View file

@ -0,0 +1,40 @@
# Glue: register the swarm's forge as an OIDC client wherever authelia runs.
#
# ONE PAIRING PER FILE — forge ← authelia, and nothing else. Deleting this
# leaves a swarm whose forge is not a client authelia has ever heard of, so
# its "sign in with" button can never complete a login and nothing minted its
# secret either.
#
# ⚠️ Gated on authelia being HERE, and deliberately NOT on this host running
# the forge. A client is a row in THIS host's provider config, so it can only be
# declared where that config is rendered — and ./hive-forge/default.nix's whole
# `config` block hangs off `deploy.forgejo.enable`, so a swarm with the forge
# and authelia on different hosts would register the client nowhere at all.
# Same shape as ./glue-grafana-oidc-client.nix, for the same reason.
#
# Unlike Grafana's, this client is never unused: every swarm runs a forge.
{
lib,
config,
...
}:
let
hyperhiveCfg = config.services.hyperhive;
deployCfg = hyperhiveCfg.deploy;
forgeCfg = hyperhiveCfg.swarm.forge;
in
{
config = lib.mkIf (hyperhiveCfg.enable && deployCfg.authelia.enable) {
# One declaration, two readers. The forge knows its own callback URL;
# making the operator restate it in authelia's client list would be a
# second source of truth for a string whose mismatch is a silent
# rejected login.
services.hyperhive.swarm.authelia.oidc.clients = [
{
id = forgeCfg.sso.clientId;
description = "HyperHive forge";
redirectUris = [ forgeCfg.sso.redirectUri ];
}
];
};
}

View file

@ -9,9 +9,8 @@
# declared where that config is rendered — and ./swarm-grafana.nix's whole # declared where that config is rendered — and ./swarm-grafana.nix's whole
# `config` block hangs off `deploy.grafana.enable`, so a swarm with Grafana and # `config` block hangs off `deploy.grafana.enable`, so a swarm with Grafana and
# authelia on different hosts registered the client nowhere at all. # authelia on different hosts registered the client nowhere at all.
# ./hive-forge/default.nix is already on the right side of that line: its module # ./glue-forge-oidc-client.nix does the same for the forge. This file puts
# is gated on `hyperhive.enable` and only the registration asks about authelia. # Grafana there without moving the rest of its module.
# This file puts Grafana there without moving the rest of its module.
# #
# ⚠️ Registered whether or not the swarm has a Grafana, because nothing in # ⚠️ Registered whether or not the swarm has a Grafana, because nothing in
# `swarm.*` records that — `deploy.grafana.enable` answers "does THIS host run # `swarm.*` records that — `deploy.grafana.enable` answers "does THIS host run

View file

@ -82,8 +82,9 @@ in
Run a Forgejo Actions runner in a `hive-ci` nixos-container. Run a Forgejo Actions runner in a `hive-ci` nixos-container.
Grouped under `services.hyperhive.deploy.forgejo` because the runner Grouped under `services.hyperhive.deploy.forgejo` because the runner
is tightly coupled to the forge instance it registers against. is tightly coupled to the forge instance it registers against.
Disabled by default; the internal forge it registers against is Disabled by default. It registers against the forge on this same
always present (mandatory), so enabling this is all that's needed. host, so it requires
{option}`services.hyperhive.deploy.forgejo.enable` here.
On first start the container auto-registers against hive-forge using On first start the container auto-registers against hive-forge using
hive-c0re's admin token — no manual token provisioning needed. hive-c0re's admin token — no manual token provisioning needed.
@ -170,6 +171,20 @@ in
Set behindGateway = true (it is the default). Set behindGateway = true (it is the default).
''; '';
} }
{
# The runner reaches the forge through THIS host's gateway, and
# hive-c0re registers it through the local forge container; neither
# exists on a host that does not run the forge. A runner on another
# host, registered by the swarm instead, is a separate design.
assertion = forgeDeployCfg.enable;
message = ''
services.hyperhive.deploy.forgejo.ci.enable requires
services.hyperhive.deploy.forgejo.enable on the same host.
The CI runner reaches the forge through this host's gateway and
is registered through the local forge container, so it can only
run where the forge does. Enable CI on the swarm's forge host.
'';
}
]; ];
# The runner's journal, named as the unit is *inside* the container — # The runner's journal, named as the unit is *inside* the container —

View file

@ -105,8 +105,9 @@ let
in in
{ {
# Private Forgejo in a `hive-forge` nixos-container, shared host # Private Forgejo in a `hive-forge` nixos-container, shared host
# netns. Agents reach it at `forge.<domain>` via the gateway. State # netns, on the one host with `deploy.forgejo.enable`. Agents reach it
# at `/var/lib/nixos-containers/hive-forge/var/lib/forgejo/` survives # at `forge.<domain>` via the gateway. State at
# `/var/lib/nixos-containers/hive-forge/var/lib/forgejo/` survives
# restart. See `docs/networking/gateway.md::hive-forge container shape`. # restart. See `docs/networking/gateway.md::hive-forge container shape`.
# External Forgejo/Gitea/Codeberg-compatible forges (beyond the mandatory # External Forgejo/Gitea/Codeberg-compatible forges (beyond the mandatory
@ -138,10 +139,10 @@ in
'') '')
]; ];
# The internal forge is mandatory — it's the canonical store for the # Every swarm needs this forge — it's the canonical store for the meta
# meta flake + every agent's config repo (and the `internal/*` repos), # flake + every agent's config repo (and the `internal/*` repos) — but
# so there is no enable/disable toggle. It deploys whenever hyperhive # only one host runs it: `deploy.forgejo.enable`, in ../deploy.nix. What
# itself is enabled (`services.hyperhive.enable`). # follows is what every hive needs to reach it, wherever it runs.
options.services.hyperhive.swarm.forge = { options.services.hyperhive.swarm.forge = {
httpPort = lib.mkOption { httpPort = lib.mkOption {
type = lib.types.port; type = lib.types.port;
@ -252,8 +253,8 @@ in
}; };
# The swarm's authelia is always registered as an OpenID Connect # The swarm's authelia is always registered as an OpenID Connect
# login source here — there is no toggle, for the same reason the # login source here — there is no toggle: a forge without it has no
# forge itself has none. # way to log a person in through the swarm's SSO.
# #
# **Additive, never exclusive.** Forgejo keeps its local password # **Additive, never exclusive.** Forgejo keeps its local password
# database and gains an extra "sign in with" button; this does not # database and gains an extra "sign in with" button; this does not
@ -270,6 +271,29 @@ in
`services.hyperhive.swarm.authelia.oidc.clients`. `services.hyperhive.swarm.authelia.oidc.clients`.
''; '';
}; };
redirectUri = lib.mkOption {
type = lib.types.str;
readOnly = true;
default = ssoRedirectUri;
defaultText = lib.literalExpression ''"''${ROOT_URL}user/oauth2/authelia/callback"'';
description = ''
OAuth2 callback authelia sends the browser back to, and the URI
it matches **exactly**.
Read-only: forgejo derives it from its own `ROOT_URL` and the
login source's name, so it is a fact other modules read rather
than a knob. The glue that registers this client wherever
authelia runs reads it from here instead of restating the
format.
⚠️ With {option}`services.hyperhive.swarm.forge.rootUrl` unset,
`ROOT_URL` follows
{option}`services.hyperhive.deploy.forgejo.behindGateway` and
the gateway's `httpsPort`, which are per-host. An authelia host
that is not the forge's host renders the forge's callback only
if the two agree on them; set `rootUrl` if they do not.
'';
};
# The secret half is a path on the host that runs the forge, so it # The secret half is a path on the host that runs the forge, so it
# lives under `deploy.forgejo.sso` — see the block below. # lives under `deploy.forgejo.sso` — see the block below.
}; };
@ -435,7 +459,7 @@ in
}; };
}; };
config = lib.mkIf config.services.hyperhive.enable { config = lib.mkIf (config.services.hyperhive.enable && deployCfg.forgejo.enable) {
# Same principle as the vhost below — this service's own surface lives # Same principle as the vhost below — this service's own surface lives
# with the service. The SSO source is named alongside forgejo because # with the service. The SSO source is named alongside forgejo because
# its failure mode is a login that silently falls back, not an error. # its failure mode is a login that silently falls back, not an error.
@ -1194,20 +1218,10 @@ in
}; };
}; };
# One declaration, two readers. The forge knows its own callback URL; # The client this login source authenticates as is registered by
# making the operator restate it in authelia's client list would be a # ../glue-forge-oidc-client.nix, wherever authelia runs. When authelia
# second source of truth for a string whose mismatch is a silent # is here too, this host minted the secret, so it can say where the
# rejected login. # forge will find it.
services.hyperhive.swarm.authelia.oidc.clients = lib.mkIf ssoLocal [
{
id = cfg.sso.clientId;
description = "HyperHive forge";
redirectUris = [ ssoRedirectUri ];
}
];
# Same case, same reasoning: this host minted the secret, so it can
# say where the forge will find it.
services.hyperhive.deploy.forgejo.sso.clientSecretFile = lib.mkIf ssoLocal ( services.hyperhive.deploy.forgejo.sso.clientSecretFile = lib.mkIf ssoLocal (
lib.mkDefault forgeSecretPath lib.mkDefault forgeSecretPath
); );

View file

@ -502,16 +502,17 @@ in
forgeTokenFile = lib.mkOption { forgeTokenFile = lib.mkOption {
type = lib.types.nullOr lib.types.str; type = lib.types.nullOr lib.types.str;
# The forge has no `enable` of its own to condition this on — # Forge's own delivery path when the forge runs here, since nothing
# neither half of its split namespace carries one — and it deploys # writes that path anywhere else. `null` otherwise, so a controller
# unconditionally wherever the rest of the stack does, so its own # away from the forge logs that it has no forge access instead of
# delivery path is simply the right default. The controller's units # waiting on a file that never appears. The controller's units are
# are the thing that decides whether the file is ever read: they only # the thing that decides whether the file is ever read: they only
# exist under `deploy.swarm-controller.enable`, and a host whose forge # exist under `deploy.swarm-controller.enable`.
# lives elsewhere overrides this (or sets `null`) explicitly. default = if deployCfg.forgejo.enable then deployCfg.forgejo.hostSwarmControllerTokenFile else null;
default = deployCfg.forgejo.hostSwarmControllerTokenFile;
defaultText = lib.literalExpression '' defaultText = lib.literalExpression ''
config.services.hyperhive.deploy.forgejo.hostSwarmControllerTokenFile if config.services.hyperhive.deploy.forgejo.enable
then config.services.hyperhive.deploy.forgejo.hostSwarmControllerTokenFile
else null
''; '';
example = "/var/lib/secrets/swarm-controller-forge.token"; example = "/var/lib/secrets/swarm-controller-forge.token";
description = '' description = ''
@ -520,11 +521,10 @@ in
`forgejo-swarm-controller-account` + `hive-forge-swarm-controller-token` `forgejo-swarm-controller-account` + `hive-forge-swarm-controller-token`
units, which mint and collect it onto forge's own host). units, which mint and collect it onto forge's own host).
Defaults to forge's own delivery path (forge deploys Defaults to forge's own delivery path when
unconditionally alongside the rest of the stack — see {option}`services.hyperhive.deploy.forgejo.enable` is set on this
`hive-forge/default.nix`, it has no `enable` of its own). host, and to `null` otherwise. Set it explicitly when the forge
Override explicitly if forge's actual token file ends up runs on another host — copy the token out of forge's
somewhere else — copy it out of forge's
{option}`services.hyperhive.deploy.forgejo.hostSwarmControllerTokenFile` {option}`services.hyperhive.deploy.forgejo.hostSwarmControllerTokenFile`
with whatever secret management this deployment already uses, with whatever secret management this deployment already uses,
the same shape `swarm.nix`'s `clientSecretFile` documents for the same shape `swarm.nix`'s `clientSecretFile` documents for

View file

@ -4,10 +4,8 @@
# where they live, and asserts the per-service `enable`s that follow — # where they live, and asserts the per-service `enable`s that follow —
# the same mode-not-default shape as ./local-defaults.nix, one tier down. # the same mode-not-default shape as ./local-defaults.nix, one tier down.
# #
# Only the *optional* services derive. The forge has no `enable` to # The forge derives from here like the rest: every swarm needs one, but
# assert, because it is not optional — it is the canonical store for the # only one host in it runs it.
# meta flake and every agent's config repo, so it deploys with hyperhive
# itself.
{ {
lib, lib,
config, config,
@ -23,20 +21,16 @@ in
example = true; example = true;
description = '' description = ''
Host the swarm's shared services on this hive. The services that Host the swarm's shared services on this hive. The services that
exist once per swarm rather than once per hive and are *optional* exist once per swarm rather than once per hive — the forge, the
— the matrix homeserver, the SSO provider, the queue, the metrics matrix homeserver, the SSO provider, the queue, the metrics and log
and log stores — have their toggle asserted from this, so a stores — have their toggle asserted from this, so a swarm's service
swarm's service host is declared in one place. host is declared in one place.
Every toggle it asserts is a {option}`services.hyperhive.deploy.*` Every toggle it asserts is a {option}`services.hyperhive.deploy.*`
one, because "does THIS host run it" is a per-host decision — which one, because "does THIS host run it" is a per-host decision — which
is the same reason this option is a `deploy.*` one itself. See is the same reason this option is a `deploy.*` one itself. See
./deploy.nix. ./deploy.nix.
The forge is swarm-wide too but has nothing to assert: it is the
canonical store for the meta flake and every agent's config repo,
so it deploys with hyperhive itself and is not optional.
`services.hyperhive.deploy.singleHostSwarm` turns this on as part `services.hyperhive.deploy.singleHostSwarm` turns this on as part
of the all-on-one-box mode. Set it directly to run the swarm's of the all-on-one-box mode. Set it directly to run the swarm's
services on a host that is not otherwise all-local — a dedicated services on a host that is not otherwise all-local — a dedicated
@ -105,4 +99,10 @@ in
# placeable on a host of its own — it can be set directly here and # placeable on a host of its own — it can be set directly here and
# turned off wherever this switch happens to be on. # turned off wherever this switch happens to be on.
config.services.hyperhive.deploy.bao.enable = lib.mkDefault deployCfg.allSwarmServices; config.services.hyperhive.deploy.bao.enable = lib.mkDefault deployCfg.allSwarmServices;
# The forge. Every swarm needs it, but one host runs it: a hive that does
# not is a *client*, reaching it at `swarm.forge.domain`. Without this a
# `singleHostSwarm` box, which gets here through `allSwarmServices`, would
# have no forge at all.
config.services.hyperhive.deploy.forgejo.enable = lib.mkDefault deployCfg.allSwarmServices;
} }

View file

@ -46,7 +46,11 @@ let
# only hive without one is a hive told to have none. # only hive without one is a hive told to have none.
noAgentQueue = hive { deploy.hive-controller.queue.agentNatsUrl = null; }; noAgentQueue = hive { deploy.hive-controller.queue.agentNatsUrl = null; };
withCi = hive { deploy.forgejo.ci.enable = true; }; # On the forge's host: the runner is refused anywhere else.
withCi = hive {
deploy.forgejo.enable = true;
deploy.forgejo.ci.enable = true;
};
# A priority collision is a property of the *option*, not # A priority collision is a property of the *option*, not
# of the merged value's interior — nix throws the moment the value is # of the merged value's interior — nix throws the moment the value is
@ -90,16 +94,16 @@ let
(hive { deploy.forgejo.behindGateway = false; }).services.hyperhive.swarm.forge.publicUrl == null; (hive { deploy.forgejo.behindGateway = false; }).services.hyperhive.swarm.forge.publicUrl == null;
} }
{ {
# The controller's token path defaulted to forge's delivery path only on # The controller's token path follows where the forge runs
# a host with the central toggle on, and to `null` otherwise. Forge # (./forge-placement.nix), never the central toggle: a forge host with
# deploys unconditionally, so the path is now unconditional too. # the toggle off still renders the path.
name = "the swarm controller's forgeTokenFile defaults to forge's delivery path regardless of the central toggle"; name = "the swarm controller's forgeTokenFile default does not consult the central toggle";
ok = ok =
let (hive {
forgePath = "/var/lib/hyperhive-forge/swarm-controller.token"; enable = false;
in deploy.forgejo.enable = true;
bare.services.hyperhive.deploy.swarm-controller.forgeTokenFile == forgePath }).services.hyperhive.deploy.swarm-controller.forgeTokenFile
&& centralToggleOff.services.hyperhive.deploy.swarm-controller.forgeTokenFile == forgePath; == "/var/lib/hyperhive-forge/swarm-controller.token";
} }
{ {
name = "a hive that does not host the swarm's shared services runs none of them"; name = "a hive that does not host the swarm's shared services runs none of them";

View file

@ -0,0 +1,144 @@
# `checks.module-eval-forge-placement` — see ./lib.nix for the shared
# rationale (why this suite exists, naming convention, "evaluates
# not executes").
#
# Where the forge runs. A swarm has one, on the host with
# `deploy.forgejo.enable`; every other hive is a client of it, and the parts
# of a split deployment that used to lean on every host running a forge
# (the OIDC client, the controller's token, the CI runner) still have to
# hold.
{
pkgs,
lib,
self,
nixosSystem,
}:
let
inherit
(import ./lib.nix {
inherit
pkgs
lib
self
nixosSystem
;
})
hive
runGroup
;
bare = hive { };
allLocal = hive { deploy.singleHostSwarm = true; };
servicesHere = hive { deploy.allSwarmServices = true; };
forgeHere = hive { deploy.forgejo.enable = true; };
# The shared services here, the forge somewhere else. Every derivation in
# ../host-modules/swarm-required-services.nix is `mkDefault`, so this stays
# expressible — and it is also the authelia-without-forge host the OIDC
# client case needs.
servicesForgeElsewhere = hive {
deploy.allSwarmServices = true;
deploy.forgejo.enable = false;
};
# The forge without authelia: the other half of the split.
forgeNoAuthelia = hive {
deploy.forgejo.enable = true;
deploy.forgejo.sso.clientSecretFile = "/var/lib/forgejo-oidc/by-hand.secret";
};
ciNoForge = hive { deploy.forgejo.ci.enable = true; };
ciWithForge = hive {
deploy.forgejo.enable = true;
deploy.forgejo.ci.enable = true;
};
runsForge = m: m.services.hyperhive.deploy.forgejo.enable && m.containers ? hive-forge;
forgeClientIds =
m:
map (c: c.id) (
lib.filter (
c: c.id == m.services.hyperhive.swarm.forge.sso.clientId
) m.services.hyperhive.swarm.authelia.oidc.clients
);
# Matched on the option the message names, same reasoning as
# ./grafana.nix's `grafanaRefusedFor`.
refusedOver = m: needle: lib.any (a: !a.assertion && lib.hasInfix needle a.message) m.assertions;
tokenFile = m: m.services.hyperhive.deploy.swarm-controller.forgeTokenFile;
forgePath = "/var/lib/hyperhive-forge/swarm-controller.token";
cases = [
{
# The absence the whole option exists for: a second forge in a swarm
# is a split brain nobody notices, so a hive that has not been told it
# is the forge's host runs none of its surface.
name = "a hive that is not the forge's host runs no forge";
ok =
!bare.services.hyperhive.deploy.forgejo.enable
&& !(bare.containers ? hive-forge)
&& !(bare.systemd.services ? hive-forge-swarm-controller-token)
&& !(lib.elem "forge.t.local" bare.services.hyperhive.gateway.localNames)
&& !(refusedOver bare "deploy.forgejo.sso.clientSecretFile");
}
{
name = "hosting the swarm's shared services runs the forge";
ok = runsForge servicesHere;
}
{
name = "the all-local mode runs the forge";
ok = runsForge allLocal && lib.elem "forge.t.local" allLocal.services.hyperhive.gateway.localNames;
}
{
name = "an explicit deploy.forgejo.enable runs the forge on its own";
ok = runsForge forgeHere && !forgeHere.services.hyperhive.deploy.allSwarmServices;
}
{
# `mkDefault`, not a plain assignment: the forge stays placeable on a
# host of its own. `nats` is the control, so the case cannot pass on a
# fixture where nothing came on.
name = "placing the forge elsewhere survives the switch that would enable it";
ok =
!(servicesForgeElsewhere.containers ? hive-forge)
&& servicesForgeElsewhere.services.hyperhive.deploy.nats.enable;
}
{
# A client is a row in authelia's config, so it is declared where
# authelia runs. With the forge's module gated, registering it from
# there would leave a split swarm's forge unknown to its IdP.
name = "the forge's OIDC client is registered wherever authelia runs, and only there";
ok =
forgeClientIds servicesForgeElsewhere == [ "forgejo" ]
&& forgeClientIds allLocal == [ "forgejo" ]
&& forgeClientIds forgeNoAuthelia == [ ];
}
{
name = "the forge's OIDC callback is the one forgejo sends";
ok =
servicesForgeElsewhere.services.hyperhive.swarm.forge.sso.redirectUri
== "https://forge.t.local/user/oauth2/authelia/callback";
}
{
# The runner reaches the forge through this host's gateway and is
# registered through the local container. The second arm is the
# control: a refusal that fires everywhere is not a check.
name = "CI is refused on a host that does not run the forge";
ok =
refusedOver ciNoForge "deploy.forgejo.enable on the same host"
&& !(refusedOver ciWithForge "deploy.forgejo.enable on the same host");
}
{
# Nothing writes the delivery path away from the forge, and a
# `LoadCredential=` naming a missing path is fatal to the unit.
name = "the controller's forge token defaults to the delivery path only where the forge runs";
ok =
tokenFile bare == null
&& tokenFile servicesForgeElsewhere == null
&& tokenFile forgeHere == forgePath
&& tokenFile allLocal == forgePath;
}
];
in
runGroup "forge-placement" cases