Compare commits
15 changed files with 107 additions and 112 deletions
|
|
@ -24,7 +24,7 @@ the root, so setting the swarm CA up is an operator action rather than
|
||||||
something a host assumes. Turn it on for an all-on-one-host deployment
|
something a host assumes. Turn it on for an all-on-one-host deployment
|
||||||
and the hierarchy costs no configuration.
|
and the hierarchy costs no configuration.
|
||||||
|
|
||||||
It defaults from `services.hyperhive.deploy.singleHostSwarm`, the single
|
It defaults from `services.hyperhive.enableAllLocalDefaults`, the single
|
||||||
switch that says "this box is the whole deployment".
|
switch that says "this box is the whole deployment".
|
||||||
|
|
||||||
**A hive given neither artifact keeps the self-signed CA it has always
|
**A hive given neither artifact keeps the self-signed CA it has always
|
||||||
|
|
|
||||||
|
|
@ -86,7 +86,7 @@ scraped; otherwise no secret is needed and none is placed.
|
||||||
### Minting the queue's callout nkeys
|
### Minting the queue's callout nkeys
|
||||||
|
|
||||||
`nats.autoGenerateCallout` mints both keypairs on the host before the queue
|
`nats.autoGenerateCallout` mints both keypairs on the host before the queue
|
||||||
starts. It is on by default only under `singleHostSwarm` — the one
|
starts. It is on by default only under `enableAllLocalDefaults` — the one
|
||||||
topology where the queue, its responder and the operator are the same person. On
|
topology where the queue, its responder and the operator are the same person. On
|
||||||
every other topology, mint them yourself:
|
every other topology, mint them yourself:
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,24 +1,20 @@
|
||||||
# Swarm-wide services
|
# Swarm-wide services
|
||||||
|
|
||||||
Some things exist once per **swarm** rather than once per hive. Two
|
Some things exist once per **swarm**, not once per hive: the forge, the
|
||||||
options say where the optional ones live, and everything else derives:
|
matrix homeserver, the SSO provider, the CA. Two options say where the
|
||||||
|
optional ones live, and everything else derives:
|
||||||
|
|
||||||
```nix
|
```nix
|
||||||
services.hyperhive.deploy.singleHostSwarm = true; # everything on this box
|
services.hyperhive.enableAllLocalDefaults = true; # everything on this box
|
||||||
# or, for a dedicated services host with hives elsewhere:
|
# or, for a dedicated services host with hives elsewhere:
|
||||||
services.hyperhive.deploy.allSwarmServices = true;
|
services.hyperhive.swarm.enableRequiredServices = true;
|
||||||
```
|
```
|
||||||
|
|
||||||
**`deploy.allSwarmServices` is what "the swarm's shared services run
|
`enableAllLocalDefaults` is the all-on-one-box switch: it defaults both
|
||||||
here" means: every once-per-swarm service that is *optional* takes its
|
`swarm.enableRequiredServices` (the shared services run here) and
|
||||||
`enable` from it.** That is the whole rule, stated once — the per-service
|
`swarm.ca.autoConfigure` (the swarm CA is generated here). Each derived
|
||||||
sections below do not repeat it, so a service that stops deriving is a
|
toggle can still be set on its own, so "all local except X" needs no
|
||||||
visible difference rather than one more paragraph saying the same thing.
|
further option.
|
||||||
|
|
||||||
`singleHostSwarm` is the all-on-one-box switch above it: it defaults
|
|
||||||
both `deploy.allSwarmServices` and `swarm.ca.autoConfigure` (the swarm CA
|
|
||||||
is generated here). Each derived toggle can still be set on its own,
|
|
||||||
which wins, so "all local except X" needs no further option.
|
|
||||||
|
|
||||||
**Both default to off**, and that is deliberate: a host cannot tell
|
**Both default to off**, and that is deliberate: a host cannot tell
|
||||||
whether it is meant to be the swarm's service host, so this is an
|
whether it is meant to be the swarm's service host, so this is an
|
||||||
|
|
@ -37,10 +33,10 @@ Those two options are what makes the difference between deployments, so
|
||||||
the shapes worth naming are the ones they produce:
|
the shapes worth naming are the ones they produce:
|
||||||
|
|
||||||
- **All-local.** Everything on one machine:
|
- **All-local.** Everything on one machine:
|
||||||
`singleHostSwarm = true`. Setup is automatic apart from
|
`enableAllLocalDefaults = true`. Setup is automatic apart from
|
||||||
choosing a domain and creating the first user.
|
choosing a domain and creating the first user.
|
||||||
- **Services on the swarm controller host.**
|
- **Services on the swarm controller host.**
|
||||||
`deploy.allSwarmServices = true` there; the required services
|
`swarm.enableRequiredServices = true` there; the required services
|
||||||
deploy together on that host, with hives elsewhere.
|
deploy together on that host, with hives elsewhere.
|
||||||
- **Fully spread out.** One container / VM / machine per service,
|
- **Fully spread out.** One container / VM / machine per service,
|
||||||
somewhere.
|
somewhere.
|
||||||
|
|
@ -68,7 +64,8 @@ One authelia per swarm, in a `swarm-authelia` container, at
|
||||||
provider, differentiated by roles and claims rather than by mechanism —
|
provider, differentiated by roles and claims rather than by mechanism —
|
||||||
there is one IdP and one auth path.
|
there is one IdP and one auth path.
|
||||||
|
|
||||||
- **`deploy.authelia`** — run the container here.
|
- **`deploy.authelia`** — run the container here. Defaults from
|
||||||
|
`swarm.enableRequiredServices`.
|
||||||
- **`swarm.authelia.url`** — where clients are sent to authenticate.
|
- **`swarm.authelia.url`** — where clients are sent to authenticate.
|
||||||
Present on **every** hive, defaulting to this host's own instance only
|
Present on **every** hive, defaulting to this host's own instance only
|
||||||
when this module is the thing running it; otherwise `null`, and a hive
|
when this module is the thing running it; otherwise `null`, and a hive
|
||||||
|
|
@ -101,8 +98,10 @@ one Grafana, in two containers at `metrics.<swarm-domain>` and
|
||||||
`grafana.<swarm-domain>`. Two containers rather than one so Grafana can
|
`grafana.<swarm-domain>`. Two containers rather than one so Grafana can
|
||||||
be restarted or broken without taking the time-series database with it.
|
be restarted or broken without taking the time-series database with it.
|
||||||
|
|
||||||
They derive together: a store with no UI is unreadable and a UI with no
|
Both follow `swarm.enableRequiredServices` like authelia and matrix, so
|
||||||
store is empty. To run one without the other, set it directly:
|
the swarm's service host gets them with everything else. They derive
|
||||||
|
together: a store with no UI is unreadable and a UI with no store is
|
||||||
|
empty. To run one without the other, set it directly:
|
||||||
|
|
||||||
```nix
|
```nix
|
||||||
services.hyperhive.deploy.victoriametrics.enable = true;
|
services.hyperhive.deploy.victoriametrics.enable = true;
|
||||||
|
|
@ -133,8 +132,9 @@ the collector is the only intended writer.
|
||||||
### Logs (VictoriaLogs)
|
### Logs (VictoriaLogs)
|
||||||
|
|
||||||
The swarm's service containers ship their journals to one VictoriaLogs at
|
The swarm's service containers ship their journals to one VictoriaLogs at
|
||||||
`logs.<swarm-domain>`, behind the same SSO as everything else. The
|
`logs.<swarm-domain>`, behind the same SSO as everything else. It follows
|
||||||
collector below is what writes to it.
|
`swarm.enableRequiredServices` too, and the collector below is what writes
|
||||||
|
to it.
|
||||||
|
|
||||||
**Reading them.** Open Grafana, pick **Explore**, and choose the
|
**Reading them.** Open Grafana, pick **Explore**, and choose the
|
||||||
`VictoriaLogs` datasource — it is provisioned for you. Grafana's *Logs
|
`VictoriaLogs` datasource — it is provisioned for you. Grafana's *Logs
|
||||||
|
|
@ -160,8 +160,9 @@ and exports to `otel.endpoint`, doing both when both are configured. It
|
||||||
also holds the upstream credential, which is why no hive and no agent
|
also holds the upstream credential, which is why no hive and no agent
|
||||||
needs one.
|
needs one.
|
||||||
|
|
||||||
It runs in a `swarm-otel` container. Its `swarm.otel.port` defaults to
|
It follows `swarm.enableRequiredServices` like the services above, in a
|
||||||
`4319` rather than OTLP's usual `4318`, which the hive tier uses — swarm
|
`swarm-otel` container. Its `swarm.otel.port` defaults to `4319` rather
|
||||||
|
than OTLP's usual `4318`, which the hive tier already uses — swarm
|
||||||
containers share the host's network namespace, so two collectors on one
|
containers share the host's network namespace, so two collectors on one
|
||||||
port is a coin toss at runtime rather than an error at build time.
|
port is a coin toss at runtime rather than an error at build time.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@ answers for one host. This one is the view *across* hives.
|
||||||
services.hyperhive.deploy.swarm-ui.enable = true; # defaults to deploy.swarm-controller.enable
|
services.hyperhive.deploy.swarm-ui.enable = true; # defaults to deploy.swarm-controller.enable
|
||||||
```
|
```
|
||||||
|
|
||||||
Derived from the controller rather than from `allSwarmServices`:
|
Derived from the controller rather than from `enableRequiredServices`:
|
||||||
the UI is a view onto the controller's state and reaches it over that
|
the UI is a view onto the controller's state and reaches it over that
|
||||||
daemon's socket, so the host that runs the controller is the host that
|
daemon's socket, so the host that runs the controller is the host that
|
||||||
can serve the UI. A hive that merely *uses* a swarm has nothing to serve.
|
can serve the UI. A hive that merely *uses* a swarm has nothing to serve.
|
||||||
|
|
|
||||||
|
|
@ -110,25 +110,6 @@ in
|
||||||
[ "services" "hyperhive" "swarm" "victorialogs" "retentionPeriod" ]
|
[ "services" "hyperhive" "swarm" "victorialogs" "retentionPeriod" ]
|
||||||
[ "services" "hyperhive" "deploy" "victorialogs" "retentionPeriod" ]
|
[ "services" "hyperhive" "deploy" "victorialogs" "retentionPeriod" ]
|
||||||
)
|
)
|
||||||
|
|
||||||
# The switch over all of the above, and the name changes with the path
|
|
||||||
# because the old one described the wrong subject: those services are
|
|
||||||
# required of the SWARM, while the option says whether THIS host runs
|
|
||||||
# them. `allSwarmServices` is mara's own phrasing of what it means.
|
|
||||||
(lib.mkRenamedOptionModule
|
|
||||||
[ "services" "hyperhive" "swarm" "enableRequiredServices" ]
|
|
||||||
[ "services" "hyperhive" "deploy" "allSwarmServices" ]
|
|
||||||
)
|
|
||||||
|
|
||||||
# The mode above that one. It sat at the TOP of `services.hyperhive`,
|
|
||||||
# which is the same defect one tier up: that namespace is everything
|
|
||||||
# about hyperhive, not the settings of a single hive. The new name says
|
|
||||||
# what the mode asserts — the whole swarm runs on this host — instead of
|
|
||||||
# naming its mechanism.
|
|
||||||
(lib.mkRenamedOptionModule
|
|
||||||
[ "services" "hyperhive" "enableAllLocalDefaults" ]
|
|
||||||
[ "services" "hyperhive" "deploy" "singleHostSwarm" ]
|
|
||||||
)
|
|
||||||
];
|
];
|
||||||
|
|
||||||
# ⚠️ `deploy.forgejo` is declared in ./hive-ci.nix, not here, and it is the
|
# ⚠️ `deploy.forgejo` is declared in ./hive-ci.nix, not here, and it is the
|
||||||
|
|
@ -158,8 +139,9 @@ in
|
||||||
description = ''
|
description = ''
|
||||||
Run the swarm's metrics store on this host.
|
Run the swarm's metrics store on this host.
|
||||||
|
|
||||||
Paired with
|
Derives from
|
||||||
{option}`services.hyperhive.deploy.grafana.enable`: a store
|
{option}`services.hyperhive.swarm.enableRequiredServices` together
|
||||||
|
with {option}`services.hyperhive.deploy.grafana.enable`: a store
|
||||||
with no UI is unreadable and a UI with no store is empty, so there
|
with no UI is unreadable and a UI with no store is empty, so there
|
||||||
is no sensible deployment that takes one and not the other from
|
is no sensible deployment that takes one and not the other from
|
||||||
that switch. Set either directly to run exactly one.
|
that switch. Set either directly to run exactly one.
|
||||||
|
|
@ -172,8 +154,10 @@ in
|
||||||
description = ''
|
description = ''
|
||||||
Run the swarm's log store on this host.
|
Run the swarm's log store on this host.
|
||||||
|
|
||||||
A hive that is not the service host is a *client* of this store,
|
Derives from
|
||||||
not a second one.
|
{option}`services.hyperhive.swarm.enableRequiredServices` for the
|
||||||
|
same reason as the metrics pair above: a hive that is not the
|
||||||
|
service host is a *client* of this store, not a second one.
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
@ -183,10 +167,14 @@ in
|
||||||
example = true;
|
example = true;
|
||||||
description = ''
|
description = ''
|
||||||
Run the swarm's secret store in a `swarm-bao` container on this
|
Run the swarm's secret store in a `swarm-bao` container on this
|
||||||
host. A swarm has one store and it has to exist somewhere.
|
host. A swarm has one store and it has to exist somewhere, so
|
||||||
|
this is asserted from
|
||||||
|
{option}`services.hyperhive.swarm.enableRequiredServices`
|
||||||
|
alongside the other once-per-swarm services.
|
||||||
|
|
||||||
*Where* it runs is a separate question from *that* it runs: set
|
That assertion is a `mkDefault`, which is what keeps *where* the
|
||||||
this directly to put the store on a host of its own, and clients
|
store runs a separate question from *that* it runs: set this
|
||||||
|
directly to put the store on a host of its own, and clients
|
||||||
still reach it by name at
|
still reach it by name at
|
||||||
{option}`services.hyperhive.swarm.bao.domain` rather than at a
|
{option}`services.hyperhive.swarm.bao.domain` rather than at a
|
||||||
local address.
|
local address.
|
||||||
|
|
@ -205,7 +193,9 @@ in
|
||||||
example = true;
|
example = true;
|
||||||
description = ''
|
description = ''
|
||||||
Run the swarm's authelia in a `swarm-authelia` container on this
|
Run the swarm's authelia in a `swarm-authelia` container on this
|
||||||
host. A swarm has one SSO provider, and this says it lives here.
|
host. {option}`services.hyperhive.swarm.enableRequiredServices`
|
||||||
|
turns this on — a swarm has one SSO provider, and that says it
|
||||||
|
lives here.
|
||||||
|
|
||||||
With it off, this hive is a *client*:
|
With it off, this hive is a *client*:
|
||||||
{option}`services.hyperhive.swarm.authelia.url` still points at
|
{option}`services.hyperhive.swarm.authelia.url` still points at
|
||||||
|
|
@ -221,8 +211,10 @@ in
|
||||||
description = ''
|
description = ''
|
||||||
Run the **swarm's** telemetry collector on this host.
|
Run the **swarm's** telemetry collector on this host.
|
||||||
|
|
||||||
A swarm has one of these, and it belongs wherever the shared
|
Derives from
|
||||||
services live rather than on every hive.
|
{option}`services.hyperhive.swarm.enableRequiredServices` with the
|
||||||
|
metrics pair it feeds: a swarm has one of these, and it belongs
|
||||||
|
wherever the shared services live rather than on every hive.
|
||||||
|
|
||||||
Named `swarm-otel` rather than `otel` because there are two
|
Named `swarm-otel` rather than `otel` because there are two
|
||||||
collectors and the tier is the whole distinction:
|
collectors and the tier is the whole distinction:
|
||||||
|
|
@ -241,8 +233,10 @@ in
|
||||||
Run the swarm's matrix homeserver — matrix-tuwunel, in a
|
Run the swarm's matrix homeserver — matrix-tuwunel, in a
|
||||||
`hive-matrix` container — on this host.
|
`hive-matrix` container — on this host.
|
||||||
|
|
||||||
Set it directly to put the homeserver somewhere other than the
|
Derives from
|
||||||
host holding the rest of the swarm's services.
|
{option}`services.hyperhive.swarm.enableRequiredServices` with
|
||||||
|
the other once-per-swarm services. Set it here directly to put
|
||||||
|
the homeserver somewhere other than the host holding the rest.
|
||||||
|
|
||||||
Client-side settings stay in
|
Client-side settings stay in
|
||||||
{option}`services.hyperhive.swarm.matrix.*`, which every hive
|
{option}`services.hyperhive.swarm.matrix.*`, which every hive
|
||||||
|
|
@ -286,8 +280,10 @@ in
|
||||||
Serve the swarm UI from this host.
|
Serve the swarm UI from this host.
|
||||||
|
|
||||||
Derived from
|
Derived from
|
||||||
{option}`services.hyperhive.deploy.swarm-controller.enable`: the
|
{option}`services.hyperhive.deploy.swarm-controller.enable` rather
|
||||||
UI is a view onto the controller's state and reaches it over that
|
than from
|
||||||
|
{option}`services.hyperhive.swarm.enableRequiredServices`: the UI
|
||||||
|
is a view onto the controller's state and reaches it over that
|
||||||
daemon's unix socket, so the host that runs the controller is the
|
daemon's unix socket, so the host that runs the controller is the
|
||||||
host that can serve the UI. A hive that merely *uses* a swarm has
|
host that can serve the UI. A hive that merely *uses* a swarm has
|
||||||
nothing to serve here.
|
nothing to serve here.
|
||||||
|
|
|
||||||
|
|
@ -92,7 +92,7 @@ in
|
||||||
DNS shouldn't have a stale `/etc/hosts` entry sticking
|
DNS shouldn't have a stale `/etc/hosts` entry sticking
|
||||||
around. Requires `services.hyperhive.domain` to be set.
|
around. Requires `services.hyperhive.domain` to be set.
|
||||||
|
|
||||||
`services.hyperhive.deploy.singleHostSwarm` turns this on as
|
`services.hyperhive.enableAllLocalDefaults` turns this on as
|
||||||
part of saying "this box is the whole deployment": that mode
|
part of saying "this box is the whole deployment": that mode
|
||||||
means there is no real DNS for these names and the operator is
|
means there is no real DNS for these names and the operator is
|
||||||
browsing them from the host itself. Set it here explicitly to
|
browsing them from the host itself. Set it here explicitly to
|
||||||
|
|
|
||||||
|
|
@ -170,7 +170,7 @@ in
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|
||||||
# `singleHostSwarm` is declared in ./local-defaults.nix, with
|
# `enableAllLocalDefaults` is declared in ./local-defaults.nix, with
|
||||||
# the values it asserts. It is a deployment mode rather than a setting
|
# the values it asserts. It is a deployment mode rather than a setting
|
||||||
# this module's options read, so it lives with its consequences.
|
# this module's options read, so it lives with its consequences.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
# The all-local deployment mode.
|
# The all-local deployment mode.
|
||||||
#
|
#
|
||||||
# `singleHostSwarm` is a *mode*, not a default other options read:
|
# `enableAllLocalDefaults` is a *mode*, not a default other options read:
|
||||||
# it says "this box is the whole deployment" and then asserts the values
|
# it says "this box is the whole deployment" and then asserts the values
|
||||||
# that follow from that. mara, on the issue: it is "more of a deployment
|
# that follow from that. mara, on the issue: it is "more of a deployment
|
||||||
# mode via settings set, less a default setting".
|
# mode via settings set, less a default setting".
|
||||||
|
|
@ -23,7 +23,7 @@ let
|
||||||
cfg = config.services.hyperhive;
|
cfg = config.services.hyperhive;
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
options.services.hyperhive.deploy.singleHostSwarm = lib.mkOption {
|
options.services.hyperhive.enableAllLocalDefaults = lib.mkOption {
|
||||||
type = lib.types.bool;
|
type = lib.types.bool;
|
||||||
default = false;
|
default = false;
|
||||||
example = true;
|
example = true;
|
||||||
|
|
@ -31,7 +31,7 @@ in
|
||||||
Run the whole swarm on this host. Turning this on asserts the
|
Run the whole swarm on this host. Turning this on asserts the
|
||||||
toggles that an all-on-one-box deployment implies: the swarm's
|
toggles that an all-on-one-box deployment implies: the swarm's
|
||||||
shared services
|
shared services
|
||||||
(`services.hyperhive.deploy.allSwarmServices`), the swarm
|
(`services.hyperhive.swarm.enableRequiredServices`), the swarm
|
||||||
CA (`services.hyperhive.swarm.ca.autoConfigure`), the swarm
|
CA (`services.hyperhive.swarm.ca.autoConfigure`), the swarm
|
||||||
controller (`services.hyperhive.deploy.swarm-controller.enable`), and the
|
controller (`services.hyperhive.deploy.swarm-controller.enable`), and the
|
||||||
host's `/etc/hosts` entries for the names this hive serves
|
host's `/etc/hosts` entries for the names this hive serves
|
||||||
|
|
@ -69,23 +69,17 @@ in
|
||||||
# every agent at its own netns. That guard already existing is what
|
# every agent at its own netns. That guard already existing is what
|
||||||
# makes turning this on by default safe; without it this line would
|
# makes turning this on by default safe; without it this line would
|
||||||
# break every agent's access to the forge.
|
# break every agent's access to the forge.
|
||||||
config.services.hyperhive.gateway.localHostsEntry = lib.mkDefault cfg.deploy.singleHostSwarm;
|
config.services.hyperhive.gateway.localHostsEntry = lib.mkDefault cfg.enableAllLocalDefaults;
|
||||||
|
|
||||||
# Out of the `swarm` attrset below, because it is a `deploy.*` option now
|
|
||||||
# (./deploy.nix): "does THIS host run the swarm's services" is a per-host
|
|
||||||
# decision. Written as a path rather than folded into a second
|
|
||||||
# `config.services.hyperhive.deploy = { … }` attrset, for the same reason
|
|
||||||
# the ⚠️ below gives about `swarm`.
|
|
||||||
config.services.hyperhive.deploy.allSwarmServices = lib.mkDefault cfg.deploy.singleHostSwarm;
|
|
||||||
|
|
||||||
config.services.hyperhive.swarm = {
|
config.services.hyperhive.swarm = {
|
||||||
ca.autoConfigure = lib.mkDefault cfg.deploy.singleHostSwarm;
|
enableRequiredServices = lib.mkDefault cfg.enableAllLocalDefaults;
|
||||||
|
ca.autoConfigure = lib.mkDefault cfg.enableAllLocalDefaults;
|
||||||
# The queue's auth-callout nkeys. Generating them is safe exactly
|
# The queue's auth-callout nkeys. Generating them is safe exactly
|
||||||
# when one operator owns both the queue and its responder, which is
|
# when one operator owns both the queue and its responder, which is
|
||||||
# what this mode asserts. On any other topology the seeds have to
|
# what this mode asserts. On any other topology the seeds have to
|
||||||
# reach whoever runs the responder, and minting them here would move
|
# reach whoever runs the responder, and minting them here would move
|
||||||
# that hand-off somewhere less visible rather than removing it.
|
# that hand-off somewhere less visible rather than removing it.
|
||||||
nats.autoGenerateCallout = lib.mkDefault cfg.deploy.singleHostSwarm;
|
nats.autoGenerateCallout = lib.mkDefault cfg.enableAllLocalDefaults;
|
||||||
# The controller's queue coordinates. Co-location is what makes these
|
# The controller's queue coordinates. Co-location is what makes these
|
||||||
# derivable at all — loopback only reaches the queue when the queue is
|
# derivable at all — loopback only reaches the queue when the queue is
|
||||||
# here, and the minted client secret only exists on the host authelia
|
# here, and the minted client secret only exists on the host authelia
|
||||||
|
|
@ -107,10 +101,10 @@ in
|
||||||
# The *requirement* stays in `swarm-controller.nix` as an assertion:
|
# The *requirement* stays in `swarm-controller.nix` as an assertion:
|
||||||
# needing a queue is the controller's own property in every topology,
|
# needing a queue is the controller's own property in every topology,
|
||||||
# and only the convenience is local.
|
# and only the convenience is local.
|
||||||
controller.queue.natsUrl = lib.mkIf cfg.deploy.singleHostSwarm (
|
controller.queue.natsUrl = lib.mkIf cfg.enableAllLocalDefaults (
|
||||||
lib.mkDefault "nats://127.0.0.1:${toString config.services.hyperhive.swarm.nats.port}"
|
lib.mkDefault "nats://127.0.0.1:${toString config.services.hyperhive.swarm.nats.port}"
|
||||||
);
|
);
|
||||||
controller.queue.clientSecretFile = lib.mkIf cfg.deploy.singleHostSwarm (
|
controller.queue.clientSecretFile = lib.mkIf cfg.enableAllLocalDefaults (
|
||||||
lib.mkDefault "${config.services.hyperhive.swarm.authelia.hostClientSecretDir}/swarm-controller.secret"
|
lib.mkDefault "${config.services.hyperhive.swarm.authelia.hostClientSecretDir}/swarm-controller.secret"
|
||||||
);
|
);
|
||||||
};
|
};
|
||||||
|
|
@ -119,7 +113,7 @@ in
|
||||||
# option stays `default = false` precisely because running it is a
|
# option stays `default = false` precisely because running it is a
|
||||||
# statement about swarm topology — but "this box is the whole
|
# statement about swarm topology — but "this box is the whole
|
||||||
# deployment" IS that statement, and it is the one shape where the
|
# deployment" IS that statement, and it is the one shape where the
|
||||||
# answer isn't ambiguous. Deriving it from `allSwarmServices`
|
# answer isn't ambiguous. Deriving it from `enableRequiredServices`
|
||||||
# instead would be wrong: a hive in a larger swarm can legitimately
|
# instead would be wrong: a hive in a larger swarm can legitimately
|
||||||
# want the shared services without being the host that controls them.
|
# want the shared services without being the host that controls them.
|
||||||
#
|
#
|
||||||
|
|
@ -128,5 +122,5 @@ in
|
||||||
# the per-host fact `swarm.*` may not carry. The ⚠️ collision note above
|
# the per-host fact `swarm.*` may not carry. The ⚠️ collision note above
|
||||||
# does not apply here — that one is about two definitions of `swarm`
|
# does not apply here — that one is about two definitions of `swarm`
|
||||||
# itself, and this is a different top-level path.
|
# itself, and this is a different top-level path.
|
||||||
config.services.hyperhive.deploy.swarm-controller.enable = lib.mkDefault cfg.deploy.singleHostSwarm;
|
config.services.hyperhive.deploy.swarm-controller.enable = lib.mkDefault cfg.enableAllLocalDefaults;
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -6,7 +6,7 @@
|
||||||
# - the CLIENT pointer (`url`) exists on every hive, because a hive
|
# - the CLIENT pointer (`url`) exists on every hive, because a hive
|
||||||
# that doesn't run authelia still has to know where to send people.
|
# that doesn't run authelia still has to know where to send people.
|
||||||
# - the CONTAINER only exists where the swarm's shared services live.
|
# - the CONTAINER only exists where the swarm's shared services live.
|
||||||
# `deploy.allSwarmServices` asserts this module's `enable`
|
# `swarm.enableRequiredServices` asserts this module's `enable`
|
||||||
# (see ./swarm-required-services.nix); a hive is a client by default.
|
# (see ./swarm-required-services.nix); a hive is a client by default.
|
||||||
#
|
#
|
||||||
# Operator and agents are both subjects of the same provider,
|
# Operator and agents are both subjects of the same provider,
|
||||||
|
|
@ -992,7 +992,7 @@ in
|
||||||
#
|
#
|
||||||
# Gated on the collector's `enable` as well, and that second condition is
|
# Gated on the collector's `enable` as well, and that second condition is
|
||||||
# what makes the loopback address honest. Both services default from
|
# what makes the loopback address honest. Both services default from
|
||||||
# `deploy.allSwarmServices` — but `mkDefault` is an invitation to
|
# `swarm.enableRequiredServices` — but `mkDefault` is an invitation to
|
||||||
# override, not a guarantee, so "they are on the same host" is a property
|
# override, not a guarantee, so "they are on the same host" is a property
|
||||||
# of the auto-deployed topology rather than of the module. Without this
|
# of the auto-deployed topology rather than of the module. Without this
|
||||||
# gate, a host running authelia and no collector would declare a target
|
# gate, a host running authelia and no collector would declare a target
|
||||||
|
|
|
||||||
|
|
@ -11,7 +11,7 @@
|
||||||
# Two provisioning modes, ONE structure — what differs is who puts the
|
# Two provisioning modes, ONE structure — what differs is who puts the
|
||||||
# artifacts on disk, never what the artifacts are:
|
# artifacts on disk, never what the artifacts are:
|
||||||
#
|
#
|
||||||
# - autoconfigured (`services.hyperhive.deploy.singleHostSwarm`, or
|
# - autoconfigured (`services.hyperhive.enableAllLocalDefaults`, or
|
||||||
# `autoConfigure` set directly): the unit below generates the root
|
# `autoConfigure` set directly): the unit below generates the root
|
||||||
# here on first boot.
|
# here on first boot.
|
||||||
# - operator-provided (multi-host): the operator installs the root cert
|
# - operator-provided (multi-host): the operator installs the root cert
|
||||||
|
|
@ -68,7 +68,7 @@ in
|
||||||
Run the whole swarm CA on this one host: generate the swarm
|
Run the whole swarm CA on this one host: generate the swarm
|
||||||
root when it is missing, and issue this hive's CA under it.
|
root when it is missing, and issue this hive's CA under it.
|
||||||
|
|
||||||
`services.hyperhive.deploy.singleHostSwarm` turns this on as
|
`services.hyperhive.enableAllLocalDefaults` turns this on as
|
||||||
part of the all-on-one-box mode. Set it here directly to run the
|
part of the all-on-one-box mode. Set it here directly to run the
|
||||||
CA on a host that is not otherwise all-local.
|
CA on a host that is not otherwise all-local.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -240,7 +240,7 @@ in
|
||||||
};
|
};
|
||||||
|
|
||||||
# `enable` moved to `services.hyperhive.deploy.swarm-controller.enable` — see
|
# `enable` moved to `services.hyperhive.deploy.swarm-controller.enable` — see
|
||||||
# ./deploy.nix. `services.hyperhive.deploy.singleHostSwarm` still
|
# ./deploy.nix. `services.hyperhive.enableAllLocalDefaults` still
|
||||||
# asserts it, and that was never an exception to "not derived from
|
# asserts it, and that was never an exception to "not derived from
|
||||||
# services.hyperhive.enable": that mode says "this box is the whole
|
# services.hyperhive.enable": that mode says "this box is the whole
|
||||||
# deployment", which answers the topology question outright, where
|
# deployment", which answers the topology question outright, where
|
||||||
|
|
@ -320,7 +320,7 @@ in
|
||||||
Empty means unset, which the assertion below refuses — a
|
Empty means unset, which the assertion below refuses — a
|
||||||
controller with no queue is not a lighter controller.
|
controller with no queue is not a lighter controller.
|
||||||
|
|
||||||
`singleHostSwarm` fills this in with loopback, because
|
`enableAllLocalDefaults` fills this in with loopback, because
|
||||||
that address is only correct when the queue is on this host:
|
that address is only correct when the queue is on this host:
|
||||||
its container shares the host netns. That derivation lives with
|
its container shares the host netns. That derivation lives with
|
||||||
the mode rather than here, so this option describes itself
|
the mode rather than here, so this option describes itself
|
||||||
|
|
@ -354,7 +354,7 @@ in
|
||||||
The controller cannot mint its own — minting happens inside
|
The controller cannot mint its own — minting happens inside
|
||||||
authelia's state directory during its first boot — so away from
|
authelia's state directory during its first boot — so away from
|
||||||
that host the operator places the secret and names it here.
|
that host the operator places the secret and names it here.
|
||||||
`singleHostSwarm` points this at the minted file, which
|
`enableAllLocalDefaults` points this at the minted file, which
|
||||||
is exactly the case where one exists locally.
|
is exactly the case where one exists locally.
|
||||||
|
|
||||||
Read by `LoadCredential`, so it needs to be readable by root at
|
Read by `LoadCredential`, so it needs to be readable by root at
|
||||||
|
|
|
||||||
|
|
@ -297,7 +297,7 @@ in
|
||||||
Leave it off wherever the queue and its clients are not the same
|
Leave it off wherever the queue and its clients are not the same
|
||||||
operator's problem: the seeds must reach whoever runs the
|
operator's problem: the seeds must reach whoever runs the
|
||||||
responder, and minting them here only moves that distribution
|
responder, and minting them here only moves that distribution
|
||||||
somewhere less visible. `singleHostSwarm` turns it on.
|
somewhere less visible. `enableAllLocalDefaults` turns it on.
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -4,20 +4,20 @@
|
||||||
# where they live, and asserts the per-service `enable`s that follow —
|
# where they live, and asserts the per-service `enable`s that follow —
|
||||||
# the same mode-not-default shape as ./local-defaults.nix, one tier down.
|
# the same mode-not-default shape as ./local-defaults.nix, one tier down.
|
||||||
#
|
#
|
||||||
# Only the *optional* services derive. The forge has no `enable` to
|
# Only the *optional* services derive: matrix and authelia. The forge has
|
||||||
# assert, because it is not optional — it is the canonical store for the
|
# no `enable` to assert, because it is not optional — it is the canonical
|
||||||
# meta flake and every agent's config repo, so it deploys with hyperhive
|
# store for the meta flake and every agent's config repo, so it deploys
|
||||||
# itself.
|
# with hyperhive itself.
|
||||||
{
|
{
|
||||||
lib,
|
lib,
|
||||||
config,
|
config,
|
||||||
...
|
...
|
||||||
}:
|
}:
|
||||||
let
|
let
|
||||||
deployCfg = config.services.hyperhive.deploy;
|
swarmCfg = config.services.hyperhive.swarm;
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
options.services.hyperhive.deploy.allSwarmServices = lib.mkOption {
|
options.services.hyperhive.swarm.enableRequiredServices = lib.mkOption {
|
||||||
type = lib.types.bool;
|
type = lib.types.bool;
|
||||||
default = false;
|
default = false;
|
||||||
example = true;
|
example = true;
|
||||||
|
|
@ -28,16 +28,15 @@ in
|
||||||
and log stores — have their toggle asserted from this, so a
|
and log stores — have their toggle asserted from this, so a
|
||||||
swarm's service host is declared in one place.
|
swarm's service host is declared in one place.
|
||||||
|
|
||||||
Every toggle it asserts is a {option}`services.hyperhive.deploy.*`
|
Those toggles live in two namespaces and the split is deliberate:
|
||||||
one, because "does THIS host run it" is a per-host decision — which
|
{option}`services.hyperhive.deploy.*` for "does THIS host run it",
|
||||||
is the same reason this option is a `deploy.*` one itself. See
|
`swarm.*.enable` for the ones not yet moved. See ./deploy.nix.
|
||||||
./deploy.nix.
|
|
||||||
|
|
||||||
The forge is swarm-wide too but has nothing to assert: it is the
|
The forge is swarm-wide too but has nothing to assert: it is the
|
||||||
canonical store for the meta flake and every agent's config repo,
|
canonical store for the meta flake and every agent's config repo,
|
||||||
so it deploys with hyperhive itself and is not optional.
|
so it deploys with hyperhive itself and is not optional.
|
||||||
|
|
||||||
`services.hyperhive.deploy.singleHostSwarm` turns this on as part
|
`services.hyperhive.enableAllLocalDefaults` turns this on as part
|
||||||
of the all-on-one-box mode. Set it directly to run the swarm's
|
of the all-on-one-box mode. Set it directly to run the swarm's
|
||||||
services on a host that is not otherwise all-local — a dedicated
|
services on a host that is not otherwise all-local — a dedicated
|
||||||
services box with hives elsewhere is exactly that shape.
|
services box with hives elsewhere is exactly that shape.
|
||||||
|
|
@ -53,44 +52,49 @@ in
|
||||||
# Everything derives under `deploy.*` now, because "does THIS host run
|
# Everything derives under `deploy.*` now, because "does THIS host run
|
||||||
# it" is a per-host decision and `swarm.*` has to be identical on every
|
# it" is a per-host decision and `swarm.*` has to be identical on every
|
||||||
# host. Same switch, same rule, one attribute path.
|
# host. Same switch, same rule, one attribute path.
|
||||||
config.services.hyperhive.deploy.matrix.enable = lib.mkDefault deployCfg.allSwarmServices;
|
config.services.hyperhive.deploy.matrix.enable = lib.mkDefault swarmCfg.enableRequiredServices;
|
||||||
|
|
||||||
# The collector that feeds the pair above (note: no `swarm.` prefix,
|
# The collector that feeds the pair above (note: no `swarm.` prefix,
|
||||||
# this is ./otel.nix's existing per-hive option).
|
# this is ./otel.nix's existing per-hive option).
|
||||||
config.services.hyperhive.otel.enable = lib.mkDefault deployCfg.allSwarmServices;
|
config.services.hyperhive.otel.enable = lib.mkDefault swarmCfg.enableRequiredServices;
|
||||||
|
|
||||||
# The rest of the shared services, from the same switch and for the same
|
# The rest of the shared services, deriving from the same switch as the
|
||||||
# reason.
|
# `swarm.*` ones above. They read differently only because "does THIS
|
||||||
|
# host run it" lives in `deploy.*` (./deploy.nix) — `swarm.*` has to be
|
||||||
|
# identical on every host, and these are exactly the values that must
|
||||||
|
# differ.
|
||||||
#
|
#
|
||||||
# authelia: a swarm has one SSO provider, and this says it lives here.
|
# authelia: a swarm has one SSO provider, and this says it lives here.
|
||||||
# With it off the hive is a *client* — `swarm.authelia.url` still points
|
# With it off the hive is a *client* — `swarm.authelia.url` still points
|
||||||
# at whoever runs it.
|
# at whoever runs it.
|
||||||
config.services.hyperhive.deploy.authelia.enable = lib.mkDefault deployCfg.allSwarmServices;
|
config.services.hyperhive.deploy.authelia.enable = lib.mkDefault swarmCfg.enableRequiredServices;
|
||||||
|
|
||||||
# The queue. Same rule: once per swarm, optional.
|
# The queue. Same rule: once per swarm, optional.
|
||||||
config.services.hyperhive.deploy.nats.enable = lib.mkDefault deployCfg.allSwarmServices;
|
config.services.hyperhive.deploy.nats.enable = lib.mkDefault swarmCfg.enableRequiredServices;
|
||||||
|
|
||||||
# The swarm collector that feeds the metrics pair, and the only tier
|
# The swarm collector that feeds the metrics pair, and the only tier
|
||||||
# holding the upstream credential. ⚠️ NOT the per-hive collector below,
|
# holding the upstream credential. ⚠️ NOT the per-hive collector below,
|
||||||
# which every hive runs.
|
# which every hive runs.
|
||||||
config.services.hyperhive.deploy.swarm-otel.enable = lib.mkDefault deployCfg.allSwarmServices;
|
config.services.hyperhive.deploy.swarm-otel.enable = lib.mkDefault swarmCfg.enableRequiredServices;
|
||||||
|
|
||||||
# The metrics pair, deriving together on purpose: a store with no UI is
|
# The metrics pair, deriving together on purpose: a store with no UI is
|
||||||
# unreadable and a UI with no store is empty, so there is no sensible
|
# unreadable and a UI with no store is empty, so there is no sensible
|
||||||
# deployment that takes one and not the other from this switch. An
|
# deployment that takes one and not the other from this switch. An
|
||||||
# operator who wants exactly one still sets it directly, which
|
# operator who wants exactly one still sets it directly, which
|
||||||
# `mkDefault` allows.
|
# `mkDefault` allows.
|
||||||
config.services.hyperhive.deploy.victoriametrics.enable = lib.mkDefault deployCfg.allSwarmServices;
|
config.services.hyperhive.deploy.victoriametrics.enable =
|
||||||
config.services.hyperhive.deploy.grafana.enable = lib.mkDefault deployCfg.allSwarmServices;
|
lib.mkDefault swarmCfg.enableRequiredServices;
|
||||||
|
config.services.hyperhive.deploy.grafana.enable = lib.mkDefault swarmCfg.enableRequiredServices;
|
||||||
|
|
||||||
# The log store, from the same switch for the same reason as the rest: a
|
# The log store, from the same switch for the same reason as the rest: a
|
||||||
# hive that is not the service host is a *client* of it, not a second one.
|
# hive that is not the service host is a *client* of it, not a second one.
|
||||||
config.services.hyperhive.deploy.victorialogs.enable = lib.mkDefault deployCfg.allSwarmServices;
|
config.services.hyperhive.deploy.victorialogs.enable =
|
||||||
|
lib.mkDefault swarmCfg.enableRequiredServices;
|
||||||
|
|
||||||
# The secret store. Once per swarm and optional, so it belongs to the
|
# The secret store. Once per swarm and optional, so it belongs to the
|
||||||
# same switch: a hive that does not run it is a *client*, reading its
|
# same switch: a hive that does not run it is a *client*, reading its
|
||||||
# own secrets from whoever does. `mkDefault` is what keeps the store
|
# own secrets from whoever does. `mkDefault` is what keeps the store
|
||||||
# placeable on a host of its own — it can be set directly here and
|
# placeable on a host of its own — it can be set directly here and
|
||||||
# turned off wherever this switch happens to be on.
|
# turned off wherever this switch happens to be on.
|
||||||
config.services.hyperhive.deploy.bao.enable = lib.mkDefault deployCfg.allSwarmServices;
|
config.services.hyperhive.deploy.bao.enable = lib.mkDefault swarmCfg.enableRequiredServices;
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -80,11 +80,11 @@ let
|
||||||
# ONE condition for all three of them, deliberately.
|
# ONE condition for all three of them, deliberately.
|
||||||
#
|
#
|
||||||
# 🩸 They were three independent conditions first, and that was wrong in
|
# 🩸 They were three independent conditions first, and that was wrong in
|
||||||
# a way only an eval gate finds: at the time `allSwarmServices`
|
# a way only an eval gate finds: at the time `enableRequiredServices`
|
||||||
# turned on matrix and authelia but NOT nats, so an ordinary all-local
|
# turned on matrix and authelia but NOT nats, so an ordinary all-local
|
||||||
# hive resolved authelia's two coordinates and not the queue URL. Two of
|
# hive resolved authelia's two coordinates and not the queue URL. Two of
|
||||||
# three set is exactly what the assertion below rejects, so every
|
# three set is exactly what the assertion below rejects, so every
|
||||||
# `singleHostSwarm` hive would have stopped evaluating.
|
# `enableAllLocalDefaults` hive would have stopped evaluating.
|
||||||
#
|
#
|
||||||
# (The queue does derive from that switch now — ./swarm-required-services.nix
|
# (The queue does derive from that switch now — ./swarm-required-services.nix
|
||||||
# — so the original asymmetry is gone. The reasoning below is why the
|
# — so the original asymmetry is gone. The reasoning below is why the
|
||||||
|
|
@ -382,7 +382,7 @@ in
|
||||||
];
|
];
|
||||||
};
|
};
|
||||||
|
|
||||||
# `allSwarmServices` is declared in ./swarm-required-services.nix
|
# `enableRequiredServices` is declared in ./swarm-required-services.nix
|
||||||
# together with the per-service `enable`s it asserts — it is a
|
# together with the per-service `enable`s it asserts — it is a
|
||||||
# deployment-shape switch rather than swarm bookkeeping, so it lives
|
# deployment-shape switch rather than swarm bookkeeping, so it lives
|
||||||
# with its consequences instead of here.
|
# with its consequences instead of here.
|
||||||
|
|
|
||||||
|
|
@ -67,7 +67,7 @@ let
|
||||||
];
|
];
|
||||||
}).config;
|
}).config;
|
||||||
|
|
||||||
allLocal = hive { deploy.singleHostSwarm = true; };
|
allLocal = hive { enableAllLocalDefaults = true; };
|
||||||
bare = hive { };
|
bare = hive { };
|
||||||
withCi = hive { deploy.forgejo.ci.enable = true; };
|
withCi = hive { deploy.forgejo.ci.enable = true; };
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue