vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

Compare commits

base: vinzenz:9a019062753cacebf8ad73df04de6fc9e9233995
Branches Tags
vinzenz:main
...
compare: vinzenz:09fe94a8ca18d3e00510099604c60a7cb52e8018
Branches Tags
vinzenz:main
Author SHA1 Message Date
atlas
09fe94a8ca deploy: name the swarm collector swarm-otel, not otel 
Per review: `deploy.otel` does not imply swarm level, and there is a
hive-tier collector too -- `services.hyperhive.otel.enable`, which every
hive runs. The tier is the entire distinction between the two, so the
name has to carry it, matching `deploy.swarm-controller` and
`deploy.swarm-ui`.

15 swarm-tier references renamed across 8 files. The hive-tier
collector's 16 references are deliberately untouched, verified as a
control on the same command.

Three spellings needed three different patterns, all inside this one
rename: the dotted path; the `mkRenamedOptionModule` target written as a
nix LIST (`[ ... "deploy" "otel" "enable" ]`), which no dotted grep can
match; and prose in docs/observability.md spelled `deploy.otel` with
neither `.enable` nor a leading dot. Unanchored `deploy\.otel\b` is the
only pattern that finds all three.
 2026-08-30 04:23:22 +02:00
atlas
13b1b41172 local-defaults: assert the controller through deploy, not the old alias 
Round-3 review catch: the all-local mode still set
`services.hyperhive.swarm.controller.enable` -- the exact path this
branch's own `mkRenamedOptionModule` deprecates. It forwards correctly,
so nothing broke, but every eval with `enableAllLocalDefaults = true`
printed a deprecation warning and the tree depended on the shim it is
retiring.

It survived the rewrite because the path is SPLIT ACROSS NESTING:
`swarm = { controller.enable = ...; }`. No grep for a dotted path can
match text that is not contiguous -- not the prefix-anchored pattern the
rewrite used, and not the suffix-anchored one added after that missed
hive-tls.nix. What finds it is the bare tail with no prefix at all
(`^\s*controller\.enable\s*=`), which reports exactly one hit outside
deploy.nix's declarations.

The assignment moves out of the `swarm` attrset rather than staying in
it: `deploy.*` is a different top-level path, so the collision the
neighbouring warning describes -- two definitions of `swarm`, the nested
one silently lost -- does not apply.
 2026-08-30 04:23:22 +02:00
atlas
d3b40da1c8 deploy: give every option an enable, and name the controller one 
Two corrections from review, applied forward on this branch rather than
by rewriting it.

`deploy.<service>` was a bare bool, which makes
`deploy.forgejo = { enable; ci; }` unrepresentable -- the nested
CI-runner sub-option this namespace was designed around. Every entry is
now an attrset with an `enable`, so a second per-host deployment
decision becomes an ordinary addition rather than a migration.

`deploy.controller` is now `deploy.swarm-controller`, consistent with
`deploy.swarm-ui`, which was introduced in the same commit.

89 references rewritten across 24 files -- nix, Rust, docs, and the
repo's own CLAUDE.md.

The prefix-anchored sweep missed exactly one, and it was live code:
hive-tls.nix spells it `hyperhiveCfg.deploy.controller` -- the only
`hyperhiveCfg` prefix among 45 references. A suffix grep
(`\.deploy\.<name>`) finds it; a path-anchored one cannot, because the
head of a reference is whatever alias the reading file happens to bind.
 2026-08-30 04:23:22 +02:00
atlas
ba5ab313b3 deploy: move the swarm collector toggle 
The one move where the suffix grep is the wrong instrument. There are two
otel options a word apart — `swarm.otel.enable` (one per swarm) and
`otel.enable` (one per hive, every hive runs it) — so `\.otel\.enable`
matches twenty-five references of which most must not change. The module
already carries a comment warning about exactly this, on a line that
names `swarm.otel` in full rather than through a binding.

Triaged by hand and confirmed the other way round: after the move, a grep
for the per-hive option still finds it in the files that should keep it.

Also worth recording what the alias sweep cannot do. `swarm.nix` reads
this as `swarmCfg.otel.enable`, where `swarmCfg = cfg.swarm` and `cfg =
config.services.hyperhive` — an alias bound to an alias, two hops from
the option path. No syntactic sweep resolves that chain, which is why the
module system's own evaluation is the only complete check here and the
static sweeps are a way to narrow the work, not to finish it.
 2026-08-30 04:23:22 +02:00
atlas
64e8c62f18 deploy: move the queue toggle 
Same shape as the others. Three of the eight references were prose
rather than code: two assertion messages naming the option to an
operator, and a `defaultText` in the authelia module, whose
`oidc.hiveIdentities` derives from whether the queue runs here.

Also corrects a comment in swarm.nix that this commit's own hunk sits
directly beneath. It said `enableRequiredServices` turns on matrix and
authelia "but NOT nats (nats has no mode that enables it)", which stopped
being true when the queue was added to that switch — before this change,
not by it. The reasoning it introduces is still right and still worth
keeping; only the premise had expired.
 2026-08-30 04:23:22 +02:00
atlas
37ca7676d6 deploy: move the SSO provider toggle 
The largest of these moves: sixteen references spelled through `let`
aliases across eight modules, plus eight more spelled as a path, plus
five documentation pages.

authelia is also the clearest case for why the two namespaces exist.
`swarm.authelia.url` is needed by *every* hive in the swarm — it says
where to send a browser to authenticate — while running the container is
the business of exactly one host. The client half and the server half
were sharing a namespace whose whole contract is "identical everywhere",
and only one of them could honour it.

`swarm.authelia.oidc.clients` stays where it is for the same reason:
several modules register a client there, gated on authelia running here,
and the registry itself is what the service *is* rather than a decision
about this machine.

One sweep note worth recording: a grep for `swarm.authelia.enable` misses
`swarmCfg.authelia.enable`, because the prefix is whatever the reading
file bound. Grepping the suffix `.authelia.enable` finds both, and found
a reference in swarm.nix that the path-shaped pattern did not.
 2026-08-30 04:23:22 +02:00
atlas
0b7357d4b8 deploy: move the controller and swarm-ui toggles 
One commit rather than two because they are not independent: the UI's
`enable` had the controller's as its literal default, so moving the
controller alone would leave the UI's default naming an option that no
longer exists.

The UI keeps that derivation in its new home — it is a view onto the
controller's state and reaches it over that daemon's unix socket, so the
host running the controller is the host that can serve it.

Three spellings had to move together for the UI, not one: the `default`,
the `defaultText` shown in the options doc, and the description prose
that names the old path in words. A grep for the option path finds the
first two.

The sweep also reached outside nix: `swarm-controller`'s crate README and
its `//!` module doc both named the option, as did this repo's own
CLAUDE.md and four pages under docs/. An option's name is API, and its
documentation lives wherever someone thought to write it down.
 2026-08-30 04:23:22 +02:00
atlas
29647f963e deploy: move the metrics and log store toggles 
Same move as grafana, and the three belong together: they derive from one
switch and a store with no UI is as useless as a UI with no store.

`victorialogs` is the case that shows why the option-path sweep is not
enough on its own. It has **zero** references spelled
`swarm.victorialogs.enable` anywhere in the tree, and four spelled
through `let` aliases (`vlCfg.enable` in the collector, `cfg.enable` in
its own module). A sweep for the path would have reported nothing to do
and left every reader broken.

Prose moved with the code rather than being left behind: the comments in
swarm-required-services.nix that explained why the pair derives together
now sit above the assignments that do it, instead of above the gap where
they used to be.
 2026-08-30 04:23:22 +02:00
atlas
cf1f2f99cf deploy: fix a reference the option-path sweep could not see 
`ssoLocal` in the grafana module read `cfg.enable`, where `cfg` is the
local binding for `services.hyperhive.swarm.grafana` — the option set the
previous commit removed `enable` from. Evaluation fails on the missing
attribute.

The sweep that missed it looked for the option *path*
(`grafana.enable`). This reference is spelled through a `let` alias, so
no grep for the path can find it, and a parse check cannot either: it is
syntactically valid and only wrong once the module system resolves the
attribute.

The check that does find it: for each file, extract the local bindings
that point at the option set being changed, then grep for
`<alias>.<removed-attr>`. Run against this tree it reports nothing for
grafana and sixteen references for authelia, which is the next module to
move.
 2026-08-30 04:23:22 +02:00
atlas
b422367942 deploy: separate "what this host deploys" from swarm-wide truth 
`services.hyperhive.swarm.*` is meant to be identical on every host in a
swarm — it describes the swarm, and every hive needs all of it to be a
client. But it also carried the `enable` toggles, which are precisely the
values that must differ per machine. The namespace that should be the
same everywhere held the one thing that cannot be.

Adds `services.hyperhive.deploy.*` for a host's deployment decisions, and
moves the first of them (`swarm.grafana.enable` -> `deploy.grafana`) as
the pattern for the rest. Flat and named for the thing deployed rather
than grouped under a "swarm services" attribute: from the deploy side it
does not matter what kind of thing each one is, and a grouping by service
kind would re-encode the service-side taxonomy into a layer that does not
care about it.

Behaviour is unchanged. The move is a rename in the strict sense — same
type, same meaning, new path — so `mkRenamedOptionModule` carries it and
existing configs keep evaluating with one warning naming both paths. The
renames live in the new module rather than the service modules, so the
whole migration has a single home and a single file to delete when the
deprecation window closes.
 2026-08-30 04:23:22 +02:00
29 changed files with 418 additions and 255 deletions
Download patch file Download diff file Expand all files Collapse all files

2
CLAUDE.md
View file

@ -142,7 +142,7 @@ hand-maintained per-file tree drifts out of sync with the code.
- **`hive-metric/`** — small CLI to push a single labeled metric to the
OTEL collector via the OpenTelemetry Rust SDK / OTLP HTTP exporter.
- **`swarm-controller/`** — swarm-level daemon, opt-in per host
(`services.hyperhive.swarm.controller.enable`). Where `hive-c0re` owns
(`services.hyperhive.deploy.swarm-controller.enable`). Where `hive-c0re` owns
the agents on **one** host, this owns what is true **across** hives; a
swarm runs one of them, so most hives leave it off. Serves HTTP over a
unix socket the gateway's nginx proxies to — ⚠️ **the socket's

6
docs/gateway.md
View file

@ -14,8 +14,8 @@ Single nginx in front of every hyperhive web surface. Runs on the **host**, next
| `chat.<swarm>/_matrix/*` | `chat.<swarm>` | tuwunel (`8008`) | `matrix.gatewayHost != null` |
| `chat.<swarm>/` | `chat.<swarm>` | fluffychat-web static | `matrix.gui.enable` |
| `chat.<swarm>/config.json` | `chat.<swarm>` | inline JSON (FluffyChat boot config) | `matrix.gui.enable && domain != null` |
| `auth.<swarm>/` | `auth.<swarm>` | authelia (`9091`) | `swarm.authelia.enable` |
| `<swarm>/` | `<swarm>` | swarm-ui dist (static), behind an authelia subrequest | `swarm.ui.enable` |
| `auth.<swarm>/` | `auth.<swarm>` | authelia (`9091`) | `deploy.authelia` |
| `<swarm>/` | `<swarm>` | swarm-ui dist (static), behind an authelia subrequest | `deploy.swarm-ui` |
The authelia vhost is declared only by the host that **runs** authelia, not by every hive that uses it — a client hive knows the swarm's `authelia.url` but must not answer for a name it doesn't serve. Its server name is exactly `swarm.authelia.domain`: authelia validates `authelia_url ⊂ session cookie domain` at startup, so a near-miss is a container that refuses to boot. It carries no `auth_basic` — the login page must not sit behind the login mechanism it replaces — and sets the four `X-Forwarded-{Proto,Host,Uri,For}` headers, since authelia decides by the *original* request rather than the hop it sees.
@ -64,7 +64,7 @@ Each location carries a duplicated `auth_basic` block (separate locations don't
- `<hive-domain>` → `127.0.0.1`
- `forge.<swarm>` → `127.0.0.1` (when forge.behindGateway)
- `chat.<swarm>` → `127.0.0.1` (when matrix.gatewayHost set)
- `auth.<swarm>` → `127.0.0.1` (when swarm.authelia.enable)
- `auth.<swarm>` → `127.0.0.1` (when deploy.authelia)
`lib.unique` de-dupes if any sub-domain happens to equal another entry. Operators with real DNS leave it off.

4
docs/observability.md
View file

@ -18,7 +18,7 @@ services.hyperhive.otel = {
`enable` is the single gate. `endpoint` is where telemetry ends up after it
leaves the swarm — optional, because the swarm's own metrics store
(`swarm.victoriametrics.enable`) is a destination in its own right. With both,
(`deploy.victoriametrics`) is a destination in its own right. With both,
telemetry goes to both. See
[`swarm/services.md`](swarm/services.md#metrics-victoriametrics--grafana).
@ -80,7 +80,7 @@ the host is:
| | runs where | receives from | does |
|---|---|---|---|
| **hive tier** — `otel.enable` | every hive with agents | that hive's agents, on the bridge | forwards to the swarm tier. Holds no credential, picks no destination |
| **swarm tier** — `swarm.otel.enable` | once per swarm | every hive's collector | writes the swarm's store and exports upstream |
| **swarm tier** — `deploy.swarm-otel` | once per swarm | every hive's collector | writes the swarm's store and exports upstream |
An all-local host runs both, and needs nothing said about the hop between them.

2
docs/persistence.md
View file

@ -472,7 +472,7 @@ snapshot contains and how the store authenticates a sender.
## `/var/lib/swarm-controller/` (swarm-controller host only)
Only present on the one host running
`services.hyperhive.swarm.controller.enable`. systemd `StateDirectory=`,
`services.hyperhive.deploy.swarm-controller.enable`. systemd `StateDirectory=`,
so it survives restarts and redeploys.
- `webhook-secret` — the HMAC key the swarm's forge webhooks are signed

4
docs/setup.md
View file

@ -44,7 +44,7 @@ echo "hunter2" | hivectl gateway create-user mara --password-stdin
hivectl gateway list-users
```
### 3 · Swarm SSO (only when `swarm.authelia.enable`)
### 3 · Swarm SSO (only when `deploy.authelia`)
⚠️ **Required to finish the install, not optional.** Authelia treats an
empty user store as a fatal startup error, so until this runs the
@ -70,7 +70,7 @@ than amend — adding the group afterwards is `swarmctl user update mara
Detail, including what the password is and why this stays manual:
[`swarm/sso.md`](swarm/sso.md).
### 4 · Swarm UI (only when `swarm.ui.enable`, on by default with the controller)
### 4 · Swarm UI (only when `deploy.swarm-ui`, on by default with the controller)
Nothing to run — it is served on the swarm apex
(`https://<swarm.domain>/`) as soon as the host rebuilds. Two things

2
docs/swarm/README.md
View file

@ -276,7 +276,7 @@ migrating agent keeps one unbroken incremental chain. See
## Swarm controller
`services.hyperhive.swarm.controller.enable` runs the `swarm-controller`
`services.hyperhive.deploy.swarm-controller.enable` runs the `swarm-controller`
daemon on this host. **Off by default and deliberately not derived from
`services.hyperhive.enable`**: a swarm has one controller, so enabling it
is a statement about swarm topology, not about whether hyperhive is

6
docs/swarm/services.md
View file

@ -34,7 +34,7 @@ One authelia per swarm, in a `swarm-authelia` container, at
provider, differentiated by roles and claims rather than by mechanism —
there is one IdP and one auth path.
- **`swarm.authelia.enable`** — run the container here. Defaults from
- **`deploy.authelia`** — run the container here. Defaults from
`swarm.enableRequiredServices`.
- **`swarm.authelia.url`** — where clients are sent to authenticate.
Present on **every** hive, defaulting to this host's own instance only
@ -74,8 +74,8 @@ together: a store with no UI is unreadable and a UI with no store is
empty. To run one without the other, set it directly:
```nix
services.hyperhive.swarm.victoriametrics.enable = true;
services.hyperhive.swarm.grafana.enable = false;
services.hyperhive.deploy.victoriametrics.enable = true;
services.hyperhive.deploy.grafana.enable = false;
```
⚠️ **This starts a database that grows for as long as the swarm runs.**

2
docs/swarm/sso.md
View file

@ -14,7 +14,7 @@ separate `enable` would be a second fact free to disagree with the first.
authelia binds loopback only. The **gateway** on the host running it
publishes it as `auth.<swarm.domain>` — vhost, dnsmasq record and TLS
name all follow `swarm.authelia.enable`, so there is nothing to turn on
name all follow `deploy.authelia`, so there is nothing to turn on
separately. (Details, including why a client hive must not declare that
vhost: [`../gateway.md`](../gateway.md).)

2
docs/swarm/ui.md
View file

@ -9,7 +9,7 @@ answers for one host. This one is the view *across* hives.
## Enabling
```nix
services.hyperhive.swarm.ui.enable = true; # defaults to swarm.controller.enable
services.hyperhive.deploy.swarm-ui.enable = true; # defaults to deploy.swarm-controller.enable
```
Derived from the controller rather than from `enableRequiredServices`:

1
nix/host-modules/default.nix
View file

@ -12,6 +12,7 @@
{
imports = [
./hyperhive.nix
./deploy.nix
./local-defaults.nix
./hive-c0re
./hive-ci.nix

200
nix/host-modules/deploy.nix Normal file
View file

@ -0,0 +1,200 @@
# "What does THIS host deploy?"
#
# Separated from `services.hyperhive.swarm.*` because those are two
# different kinds of fact and only one of them varies per machine:
#
# swarm.* — swarm-wide truth. The swarm's name, domain, hives, peers,
# CA, and where each service lives. **Identical on every
# host**, byte for byte; a hive needs all of it to be a
# *client* of the swarm.
# deploy.* — this machine's deployment decisions. Necessarily different
# on every host, because that is what a deployment is.
#
# The `enable` toggles used to live under `swarm.*`, which made the
# namespace that is supposed to be identical everywhere carry the one
# thing that must differ.
#
# Flat and named for the thing deployed — `deploy.forgejo`, not
# `deploy.swarmServices.forgejo`: grouping by "swarm service" re-encodes
# the service-side taxonomy into a layer that does not care about it.
#
# ⚠️ Each entry is an attrset with an `enable`, not a bare bool, so a
# service that grows a second *deployment* decision has somewhere to put
# it — `deploy.forgejo = { enable; ci; }` is then an ordinary addition
# rather than a migration. `ci` ("does this host run the runner too") is
# exactly that shape, and a bare bool leaves it unrepresentable.
#
# ⚠️ The renames below are deliberately in this one file rather than
# spread across the service modules, so the whole move has a single home
# and a single file to delete when the deprecation window closes — the
# shape ./swarm-peers-removed.nix already uses.
{ lib, config, ... }:
let
deployCfg = config.services.hyperhive.deploy;
in
{
imports = [
# Same type, same meaning, new path — so a rename carries it exactly
# and existing configs keep evaluating with one warning naming both
# paths. Precedent: ./hive-forge/default.nix, ./hive-matrix.nix.
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "grafana" "enable" ]
[ "services" "hyperhive" "deploy" "grafana" "enable" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "victoriametrics" "enable" ]
[ "services" "hyperhive" "deploy" "victoriametrics" "enable" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "victorialogs" "enable" ]
[ "services" "hyperhive" "deploy" "victorialogs" "enable" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "controller" "enable" ]
[ "services" "hyperhive" "deploy" "swarm-controller" "enable" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "ui" "enable" ]
[ "services" "hyperhive" "deploy" "swarm-ui" "enable" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "authelia" "enable" ]
[ "services" "hyperhive" "deploy" "authelia" "enable" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "nats" "enable" ]
[ "services" "hyperhive" "deploy" "nats" "enable" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "otel" "enable" ]
[ "services" "hyperhive" "deploy" "swarm-otel" "enable" ]
)
];
options.services.hyperhive.deploy = {
grafana.enable = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
Run the swarm's metrics UI on this host.
Off by default and not derived from
{option}`services.hyperhive.enable`: a swarm has one Grafana, so
running it is a decision about this host rather than about
whether hyperhive is installed.
'';
};
victoriametrics.enable = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
Run the swarm's metrics store on this host.
Derives from
{option}`services.hyperhive.swarm.enableRequiredServices` together
with {option}`services.hyperhive.deploy.grafana.enable`: a store
with no UI is unreadable and a UI with no store is empty, so there
is no sensible deployment that takes one and not the other from
that switch. Set either directly to run exactly one.
'';
};
victorialogs.enable = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
Run the swarm's log store on this host.
Derives from
{option}`services.hyperhive.swarm.enableRequiredServices` for the
same reason as the metrics pair above: a hive that is not the
service host is a *client* of this store, not a second one.
'';
};
authelia.enable = lib.mkOption {
type = lib.types.bool;
default = false;
example = true;
description = ''
Run the swarm's authelia in a `swarm-authelia` container on this
host. {option}`services.hyperhive.swarm.enableRequiredServices`
turns this on — a swarm has one SSO provider, and that says it
lives here.
With it off, this hive is a *client*:
{option}`services.hyperhive.swarm.authelia.url` still points at
whoever runs it, and no container is created. That asymmetry is
why the two live in different namespaces — every hive needs the
client half, only one runs the server half.
'';
};
swarm-otel.enable = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
Run the **swarm's** telemetry collector on this host.
Derives from
{option}`services.hyperhive.swarm.enableRequiredServices` with the
metrics pair it feeds: a swarm has one of these, and it belongs
wherever the shared services live rather than on every hive.
Named `swarm-otel` rather than `otel` because there are two
collectors and the tier is the whole distinction:
{option}`services.hyperhive.otel.enable` is the **hive-tier** one,
which every hive runs. A bare `deploy.otel` would not say which
it meant. A hive that does not run the swarm collector still runs
its own, and reaches this one by name at
{option}`services.hyperhive.swarm.otel.domain`.
'';
};
nats.enable = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
Run the swarm's message queue in a `swarm-nats` container on this
host. A swarm has one queue, so this belongs on the same host as
the rest of the shared services.
Off by default, and off means *absent*: no container is created
and nothing else in the evaluated config changes.
'';
};
swarm-controller.enable = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
Run the swarm-controller daemon on this host.
Off by default and deliberately not derived from
{option}`services.hyperhive.enable`: a swarm has one controller,
so running it is a decision about this host rather than about
whether hyperhive is installed.
'';
};
swarm-ui.enable = lib.mkOption {
type = lib.types.bool;
default = deployCfg.swarm-controller.enable;
defaultText = lib.literalExpression "services.hyperhive.deploy.swarm-controller.enable";
example = true;
description = ''
Serve the swarm UI from this host.
Derived from
{option}`services.hyperhive.deploy.swarm-controller.enable` rather
than from
{option}`services.hyperhive.swarm.enableRequiredServices`: the UI
is a view onto the controller's state and reaches it over that
daemon's unix socket, so the host that runs the controller is the
host that can serve the UI. A hive that merely *uses* a swarm has
nothing to serve here.
'';
};
};
}

3
nix/host-modules/hive-forge/default.nix
View file

@ -22,6 +22,7 @@ let
# assertion below turns into an eval failure rather than a discovery
# request to `null/.well-known/...`.
autheliaCfg = config.services.hyperhive.swarm.authelia;
deployCfg = config.services.hyperhive.deploy;
autheliaUrl = autheliaCfg.url;
autheliaDiscoveryUrl = "${toString autheliaUrl}/.well-known/openid-configuration";
@ -29,7 +30,7 @@ let
# authelia, so the secret can be moved without an operator. The other
# two cases (swarm side via swarmctl, remote hive) leave
# `clientSecretFile` to be set explicitly — see docs/swarm/.
ssoLocal = autheliaCfg.enable;
ssoLocal = deployCfg.authelia.enable;
# Where the plaintext lands inside the forge container. Under
# /var/lib rather than /run: the forge may start before the delivery

3
nix/host-modules/hive-matrix.nix
View file

@ -25,13 +25,14 @@ let
# below turns into an eval failure rather than a discovery request to
# `null/.well-known/…`.
autheliaCfg = config.services.hyperhive.swarm.authelia;
deployCfg = config.services.hyperhive.deploy;
autheliaUrl = autheliaCfg.url;
# The all-local case: this host runs BOTH the homeserver and the swarm's
# authelia, so the secret can be moved without an operator. The other
# two cases (swarm side, remote hive) leave `clientSecretFile` to be set
# explicitly — same split the forge module documents.
ssoLocal = autheliaCfg.enable;
ssoLocal = deployCfg.authelia.enable;
# Where the plaintext lands inside the matrix container. Under /var/lib
# rather than /run: the homeserver may start before the delivery unit on

2
nix/host-modules/hive-tls.nix
View file

@ -669,6 +669,6 @@ in
# in review on this PR; it evaluates and builds clean either way, which
# is exactly why it needed a reviewer rather than a check.
systemd.services.swarm-controller.environment.SWARM_CONTROLLER_OIDC_CA_FILE =
lib.mkIf hyperhiveCfg.swarm.controller.enable "${cfg.stateDir}/trust-bundle.pem";
lib.mkIf hyperhiveCfg.deploy.swarm-controller.enable "${cfg.stateDir}/trust-bundle.pem";
};
}

28
nix/host-modules/local-defaults.nix
View file

@ -33,7 +33,7 @@ in
shared services
(`services.hyperhive.swarm.enableRequiredServices`), the swarm
CA (`services.hyperhive.swarm.ca.autoConfigure`), the swarm
controller (`services.hyperhive.swarm.controller.enable`), and the
controller (`services.hyperhive.deploy.swarm-controller.enable`), and the
host's `/etc/hosts` entries for the names this hive serves
(`services.hyperhive.gateway.localHostsEntry`) — with no real DNS
for those names, the operator is browsing them from the same box
@ -80,22 +80,13 @@ in
# reach whoever runs the responder, and minting them here would move
# that hand-off somewhere less visible rather than removing it.
nats.autoGenerateCallout = lib.mkDefault cfg.enableAllLocalDefaults;
# The controller is asserted by the MODE and by nothing else. Its own
# option stays `default = false` precisely because running it is a
# statement about swarm topology — but "this box is the whole
# deployment" IS that statement, and it is the one shape where the
# answer isn't ambiguous. Deriving it from `enableRequiredServices`
# instead would be wrong: a hive in a larger swarm can legitimately
# want the shared services without being the host that controls them.
controller.enable = lib.mkDefault cfg.enableAllLocalDefaults;
# The controller's queue coordinates. Co-location is what makes these
# derivable at all — loopback only reaches the queue when the queue is
# here, and the minted client secret only exists on the host authelia
# ran its first boot on — so they belong to the mode that asserts this
# box is the whole deployment, not to the options' own `default`.
#
# Deriving them from `swarm.nats.enable` / `swarm.authelia.enable`
# Deriving them from `deploy.nats` / `deploy.authelia`
# inside those defaults is the mixing this file exists to prevent: the
# option would be describing a deployment shape instead of describing
# itself, and "what does all-local turn on?" would stop having one
@ -117,4 +108,19 @@ in
lib.mkDefault "${config.services.hyperhive.swarm.authelia.hostClientSecretDir}/swarm-controller.secret"
);
};
# The controller is asserted by the MODE and by nothing else. Its own
# option stays `default = false` precisely because running it is a
# statement about swarm topology — but "this box is the whole
# deployment" IS that statement, and it is the one shape where the
# answer isn't ambiguous. Deriving it from `enableRequiredServices`
# instead would be wrong: a hive in a larger swarm can legitimately
# want the shared services without being the host that controls them.
#
# Sits outside the `swarm` attrset above because it is a `deploy.*`
# option (./deploy.nix): "does THIS host run the controller" is exactly
# the per-host fact `swarm.*` may not carry. The ⚠️ collision note above
# does not apply here — that one is about two definitions of `swarm`
# itself, and this is a different top-level path.
config.services.hyperhive.deploy.swarm-controller.enable = lib.mkDefault cfg.enableAllLocalDefaults;
}

13
nix/host-modules/otel.nix
View file

@ -61,7 +61,7 @@ in
their own containers can reach.
That collector forwards to the swarm's
({option}`services.hyperhive.swarm.otel.enable`), which holds the
({option}`services.hyperhive.deploy.swarm-otel.enable`), which holds the
upstream credential and writes the swarm's store. So an agent never
sees the credential, and neither does this tier.
@ -81,13 +81,13 @@ in
has left the swarm.
Read by the swarm's collector
({option}`services.hyperhive.swarm.otel.enable`), which is the only
({option}`services.hyperhive.deploy.swarm-otel.enable`), which is the only
tier that holds the upstream credential. An agent is handed the
*first* hop instead — this hive's own collector — so this value is
never given to a container.
Optional. Leave it empty and the swarm's own metrics store
({option}`services.hyperhive.swarm.victoriametrics.enable`) is the
({option}`services.hyperhive.deploy.victoriametrics.enable`) is the
destination; that is a complete deployment, not a degraded one.
Set both and telemetry goes to both.
'';
@ -232,7 +232,7 @@ in
type = lib.types.nullOr lib.types.str;
default =
if
config.services.hyperhive.swarm.authelia.enable && config.services.hyperhive.hiveName != null
config.services.hyperhive.deploy.authelia.enable && config.services.hyperhive.hiveName != null
then
"${config.services.hyperhive.swarm.authelia.hostClientSecretDir}/"
+ "${config.services.hyperhive.swarm.authelia.hiveClientPrefix}${config.services.hyperhive.hiveName}.secret"
@ -284,6 +284,7 @@ in
let
otel = config.services.hyperhive.otel;
autheliaCfg = config.services.hyperhive.swarm.authelia;
deployCfg = config.services.hyperhive.deploy;
swarmOtelCfg = config.services.hyperhive.swarm.otel;
hiveName = config.services.hyperhive.hiveName;
listen = "${config.services.hyperhive.network.bridgeIp}:${toString otel.collector.port}";
@ -580,8 +581,8 @@ in
# Only when the minting container is on THIS host. Elsewhere the file
# is operator-provided and there is no local unit to order against —
# naming one that does not exist orders nothing, silently.
after = lib.optional autheliaCfg.enable "container@${autheliaCfg.machine}.service";
requires = lib.optional autheliaCfg.enable "container@${autheliaCfg.machine}.service";
after = lib.optional deployCfg.authelia.enable "container@${autheliaCfg.machine}.service";
requires = lib.optional deployCfg.authelia.enable "container@${autheliaCfg.machine}.service";
serviceConfig = {
Type = "oneshot";
RemainAfterExit = true;

43
nix/host-modules/swarm-authelia.nix
View file

@ -42,6 +42,7 @@ let
hyperhiveDomain = hyperhiveCfg.domain;
swarmDomain = hyperhiveCfg.swarm.domain;
uiCfg = hyperhiveCfg.swarm.ui;
deployCfg = hyperhiveCfg.deploy;
forgeCfg = hyperhiveCfg.swarm.forge;
# Group an account must hold to reach operator-only surfaces. Named
@ -344,22 +345,12 @@ let
'';
in
{
# `enable` moved to `services.hyperhive.deploy.authelia.enable` — see
# ./deploy.nix. Whether this host runs the swarm's SSO provider is a
# deployment decision; what stays here is what authelia IS, including
# `url` and the OIDC client registry every hive needs as a *client*
# whether or not it runs the container.
options.services.hyperhive.swarm.authelia = {
enable = lib.mkOption {
type = lib.types.bool;
default = false;
example = true;
description = ''
Run the swarm's authelia in a `swarm-authelia` container on this
host. `services.hyperhive.swarm.enableRequiredServices` turns
this on — a swarm has one SSO provider, and that says it lives
here.
With it off, this hive is a *client*: `url` below still points
at whoever runs it, and no container is created.
'';
};
package = lib.mkOption {
type = lib.types.package;
default = pkgs.authelia;
@ -436,7 +427,7 @@ in
url = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = if cfg.enable then "https://${cfg.domain}" else null;
default = if deployCfg.authelia.enable then "https://${cfg.domain}" else null;
defaultText = lib.literalExpression ''if enable then "https://''${domain}" else null'';
example = "https://auth.example.com";
description = ''
@ -476,8 +467,8 @@ in
oidc.hiveIdentities = lib.mkOption {
type = lib.types.bool;
default = hyperhiveCfg.swarm.nats.enable;
defaultText = lib.literalExpression "services.hyperhive.swarm.nats.enable";
default = deployCfg.nats.enable;
defaultText = lib.literalExpression "services.hyperhive.deploy.nats.enable";
description = ''
Mint one machine client per hive in
{option}`services.hyperhive.swarm.hives`, so each hive can
@ -819,7 +810,7 @@ in
bridgeUrl = lib.mkOption {
type = lib.types.nullOr lib.types.str;
readOnly = true;
default = if cfg.enable then "http://127.0.0.1:${toString cfg.bridgePort}" else null;
default = if deployCfg.authelia.enable then "http://127.0.0.1:${toString cfg.bridgePort}" else null;
defaultText = lib.literalExpression ''if enable then "http://127.0.0.1:''${bridgePort}" else null'';
description = ''
Where `swarm-authelia-bridge` answers, **as seen from this
@ -838,7 +829,7 @@ in
};
};
config = lib.mkIf (hyperhiveCfg.enable && cfg.enable) {
config = lib.mkIf (hyperhiveCfg.enable && deployCfg.authelia.enable) {
# The derived half of the client list, declared the same way an
# operator declares one. Everything downstream then reads a single
# uniformly-typed `cfg.oidc.clients` and cannot tell the parts apart —
@ -973,7 +964,7 @@ in
# Authelia's own gateway surface: the vhost that fronts it and the
# name the hive resolver answers for. Both live here rather than in
# the gateway, and both are inside `cfg.enable` — that guard is the
# the gateway, and both are inside `deployCfg.authelia.enable` — that guard is the
# load-bearing part.
#
# ⚠️ Every hive in a swarm knows `authelia.url`, but only the host
@ -1018,9 +1009,11 @@ in
# Written out in full rather than through a `let` binding so the gate and
# the option it gates are visibly the same path: gating the wrong one is
# not a build error, it is a target that is always declared or never is.
services.hyperhive.swarm.otel.scrapeTargets = lib.mkIf config.services.hyperhive.swarm.otel.enable {
authelia = "127.0.0.1:${toString cfg.metricsPort}";
};
services.hyperhive.swarm.otel.scrapeTargets =
lib.mkIf config.services.hyperhive.deploy.swarm-otel.enable
{
authelia = "127.0.0.1:${toString cfg.metricsPort}";
};
# This swarm-ui quick-links entry, same guard as the vhost/DNS name
# above (only the host actually running the container claims it —
@ -1406,7 +1399,7 @@ in
# collector is registered — see `metricsRule` above,
# which is where the reasoning for both halves lives.
lib.optional forgeCfg.behindGateway metricsRule
++ lib.optional uiCfg.enable {
++ lib.optional deployCfg.swarm-ui.enable {
domain = uiCfg.domain;
subject = [ "group:${operatorGroup}" ];
policy = "one_factor";

43
nix/host-modules/swarm-controller.nix
View file

@ -15,6 +15,7 @@
}:
let
cfg = config.services.hyperhive.swarm.controller;
deployCfg = config.services.hyperhive.deploy;
autheliaCfg = config.services.hyperhive.swarm.authelia;
# What `swarmctl` needs in order to act on authelia from the host.
@ -24,7 +25,7 @@ let
# the right behaviour is for `swarmctl user add` to fail saying the
# value is unset. A guessed path would resolve cleanly and write a file
# nothing reads, which is the failure mode that costs an afternoon.
autheliaEnv = lib.optionalAttrs autheliaCfg.enable {
autheliaEnv = lib.optionalAttrs deployCfg.authelia.enable {
# The CONFIGURED authelia, not whatever is on PATH: the argon2
# parameters baked into a hash have to match the verifier's.
SWARMCTL_AUTHELIA_BIN = "${autheliaCfg.package}/bin/authelia";
@ -119,7 +120,7 @@ let
# ⚠️ Correct ONLY while the vhost and this daemon share a host, and they do
# by construction: the UI's `/api/` location proxies
# `http://unix:<socketPath>`, a path that resolves nowhere else, and this
# daemon binds no TCP address at all (see the header). `swarm.ui.enable`
# daemon binds no TCP address at all (see the header). `deploy.swarm-ui`
# therefore is not a guess about *some* host publishing the endpoint — it is
# the flag that declares that vhost, on the box holding the socket.
#
@ -129,7 +130,7 @@ let
# the moment to add an explicit `publicUrl` option — not before, because
# until then there is exactly one derivable answer and an option would only
# be a second place to get it wrong.
webhookEnv = lib.optionalAttrs uiCfg.enable {
webhookEnv = lib.optionalAttrs deployCfg.swarm-ui.enable {
SWARM_CONTROLLER_PUBLIC_URL = "https://${uiCfg.domain}";
};
@ -173,14 +174,14 @@ let
# so the two ends agree on exactly one path, `/v1/metrics`, arrived at
# from opposite directions.
#
# Gated on `otelSwarmCfg.enable` alone (a swarm-wide fact, not "enabled
# Gated on `deployCfg.swarm-otel.enable` alone (a swarm-wide fact, not "enabled
# HERE"): the collector is reachable by name wherever it runs, so a
# controller not co-located with it now exports over https instead of
# exporting nothing — the graceful-absence case left is "no swarm-otel
# anywhere in this swarm at all," which `forgeEnv` already models the
# same way.
otelSwarmCfg = config.services.hyperhive.swarm.otel;
otelEnv = lib.optionalAttrs otelSwarmCfg.enable {
otelEnv = lib.optionalAttrs deployCfg.swarm-otel.enable {
OTEL_EXPORTER_OTLP_ENDPOINT = "https://${otelSwarmCfg.domain}/${otelSwarmCfg.producerName}";
# Checked by `swarm-otel.nix`'s `oidc/${producerName}` authenticator
# against exactly this controller's own registered audience (see the
@ -214,7 +215,7 @@ in
name = "swarm-controller";
consumers = [ "swarm-controller" ];
hostUnit = true;
enable = cfg.enable;
enable = deployCfg.swarm-controller.enable;
})
];
@ -238,22 +239,12 @@ in
'';
};
enable = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
Run the swarm-controller daemon on this host. Off by default and
deliberately not derived from `services.hyperhive.enable`: a swarm
has one controller, so enabling it per hive is a decision about
swarm topology, not about whether hyperhive is installed.
`services.hyperhive.enableAllLocalDefaults` does assert it, and
that is not an exception to the rule above — it is the rule
applied. That mode says "this box is the whole deployment", which
answers the topology question outright, where
`services.hyperhive.enable` alone never can.
'';
};
# `enable` moved to `services.hyperhive.deploy.swarm-controller.enable` — see
# ./deploy.nix. `services.hyperhive.enableAllLocalDefaults` still
# asserts it, and that was never an exception to "not derived from
# services.hyperhive.enable": that mode says "this box is the whole
# deployment", which answers the topology question outright, where
# `enable` alone never can. What stays here is what the daemon IS.
package = lib.mkOption {
type = lib.types.package;
@ -460,7 +451,7 @@ in
authBridgeUrl = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = if autheliaCfg.enable then autheliaCfg.bridgeUrl else null;
default = if deployCfg.authelia.enable then autheliaCfg.bridgeUrl else null;
defaultText = lib.literalExpression ''
authelia's own `bridgeUrl` when this host also runs
`swarm-authelia`, else null
@ -483,7 +474,7 @@ in
};
};
config = lib.mkIf (config.services.hyperhive.enable && cfg.enable) {
config = lib.mkIf (config.services.hyperhive.enable && deployCfg.swarm-controller.enable) {
# The daemon and the oneshot that mints its credential — the second one
# failing leaves the first running and unable to authenticate anywhere.
services.hyperhive.swarm.otel.journaldUnits = [
@ -508,7 +499,7 @@ in
# client list would be a second source of truth for a string whose
# mismatch is an opaque 401 from the token endpoint. Same shape as the
# queue's own client declaration.
services.hyperhive.swarm.authelia.oidc.clients = lib.mkIf autheliaCfg.enable [
services.hyperhive.swarm.authelia.oidc.clients = lib.mkIf deployCfg.authelia.enable [
{
id = queueClientId;
description = "HyperHive swarm controller";
@ -548,7 +539,7 @@ in
services.hyperhive.swarm.controller.queue.natsUrl is unset.
It defaults to loopback only when this host also runs the queue
(`services.hyperhive.swarm.nats.enable`). A controller on its
(`services.hyperhive.deploy.nats.enable`). A controller on its
own host has to be told where the queue is.
'';
}

31
nix/host-modules/swarm-grafana.nix
View file

@ -16,6 +16,7 @@
}:
let
cfg = config.services.hyperhive.swarm.grafana;
deployCfg = config.services.hyperhive.deploy;
networkCfg = config.services.hyperhive.network;
hyperhiveCfg = config.services.hyperhive;
gatewayCfg = hyperhiveCfg.gateway;
@ -76,7 +77,7 @@ let
# The all-local case: this host runs BOTH Grafana and the swarm's authelia,
# so the minted secret can be moved without an operator. Same split the
# forge and matrix modules document.
ssoLocal = cfg.enable && autheliaCfg.enable;
ssoLocal = deployCfg.grafana.enable && deployCfg.authelia.enable;
autheliaUrl = toString autheliaCfg.url;
# Where the plaintext lands inside the container. Under /var/lib rather
@ -106,18 +107,12 @@ let
in
{
# `enable` moved to `services.hyperhive.deploy.grafana.enable` — see
# ./deploy.nix. Whether this host runs the swarm's Grafana is a
# deployment decision, and `swarm.*` has to be identical on every host.
# What stays here is what the service IS: its package, domain, and
# wiring.
options.services.hyperhive.swarm.grafana = {
enable = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
Run the swarm's metrics UI on this host. Off by default and not
derived from {option}`services.hyperhive.enable`: a swarm has one
Grafana, so enabling it is a decision about swarm topology rather
than about whether hyperhive is installed.
'';
};
package = lib.mkOption {
type = lib.types.package;
default = pkgs.grafana;
@ -312,8 +307,8 @@ in
};
};
config = lib.mkIf (hyperhiveCfg.enable && cfg.enable) {
# The gateway name and the quick-link, both inside `cfg.enable` — that
config = lib.mkIf (hyperhiveCfg.enable && deployCfg.grafana.enable) {
# The gateway name and the quick-link, both inside `deploy.grafana` — that
# guard is the load-bearing part. Every hive in a swarm may know this UI
# exists, but only the host that RUNS it may claim the name; a client
# hive declaring the vhost would answer for a service it does not have.
@ -413,9 +408,11 @@ in
# Declared here rather than in the collector's module, per that option's
# rule: an entry exists where the service that named it runs.
services.hyperhive.swarm.otel.scrapeTargets = lib.mkIf config.services.hyperhive.swarm.otel.enable {
grafana = "127.0.0.1:${toString cfg.metricsPort}";
};
services.hyperhive.swarm.otel.scrapeTargets =
lib.mkIf config.services.hyperhive.deploy.swarm-otel.enable
{
grafana = "127.0.0.1:${toString cfg.metricsPort}";
};
# Order the container after the host CA service so the bind source below
# exists before nspawn sets the mount up.

40
nix/host-modules/swarm-nats.nix
View file

@ -7,6 +7,7 @@
let
cfg = config.services.hyperhive.swarm.nats;
autheliaCfg = config.services.hyperhive.swarm.authelia;
deployCfg = config.services.hyperhive.deploy;
autheliaUrl = autheliaCfg.url;
networkCfg = config.services.hyperhive.network;
# Read even when the controller runs on a different host: what is needed
@ -48,7 +49,7 @@ let
hostPath = name: "${secretDir}/${name}";
# The responder needs all three credentials. Gating on them rather than
# on `cfg.enable` keeps a half-configured hive at "queue up, denying
# on `deployCfg.nats.enable` keeps a half-configured hive at "queue up, denying
# everyone" instead of "unit crash-looping on a missing file".
#
# In auto mode the seeds are minted on this host before the container
@ -209,18 +210,9 @@ in
# per-service vhost pattern do not apply.
options.services.hyperhive.swarm.nats = {
enable = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
Run the swarm's message queue in a `swarm-nats` container on this
host. A swarm has one queue, so this belongs on the same host as
the rest of the shared services.
Off by default, and off means *absent*: no container is created
and nothing else in the evaluated config changes.
'';
};
# `enable` moved to `services.hyperhive.deploy.nats.enable` — see
# ./deploy.nix. What stays here is what the queue IS: its domain,
# ports, accounts and callout wiring.
# ⚠️ Deliberately NO `package` option, unlike this module's siblings.
# `services.nats` upstream does not expose one — it resolves
@ -407,7 +399,7 @@ in
};
};
config = lib.mkIf cfg.enable {
config = lib.mkIf deployCfg.nats.enable {
# The responder as well as the server: a denial reaches the client as a
# timeout, so the server's own log is the only place it is an error.
services.hyperhive.swarm.otel.journaldUnits = [
@ -422,7 +414,7 @@ in
# several layers from "the operator never set the issuer".
assertion = cfg.autoGenerateCallout || cfg.calloutIssuerPublicKey != "";
message = ''
services.hyperhive.swarm.nats.enable requires
services.hyperhive.deploy.nats.enable requires
nats.calloutIssuerPublicKey — the public half of the account
nkey that signs user JWTs for this queue.
@ -439,7 +431,7 @@ in
# somebody connects.
assertion = cfg.autoGenerateCallout || cfg.calloutUserPublicKey != "";
message = ''
services.hyperhive.swarm.nats.enable requires
services.hyperhive.deploy.nats.enable requires
nats.calloutUserPublicKey — the public half of the user nkey
the auth-callout responder authenticates as.
@ -484,7 +476,7 @@ in
{
assertion = autheliaUrl != null;
message = ''
services.hyperhive.swarm.nats.enable requires
services.hyperhive.deploy.nats.enable requires
services.hyperhive.swarm.authelia.url — the queue authenticates
clients by validating tokens that authelia issued.
@ -514,7 +506,7 @@ in
# authorization grant 'client_credentials'`. Introspection — which
# is all the responder needs today — worked throughout, which is why
# nothing was visibly broken while the comment was untrue.
services.hyperhive.swarm.authelia.oidc.clients = lib.mkIf autheliaCfg.enable [
services.hyperhive.swarm.authelia.oidc.clients = lib.mkIf deployCfg.authelia.enable [
{
id = cfg.clientId;
description = "HyperHive swarm queue";
@ -543,9 +535,11 @@ in
# ⚠️ `swarm.otel`, not `hyperhive.otel` — two collectors one word apart,
# and only this one reads `scrapeTargets`. Written in full so the gate
# and the option it gates are visibly the same path.
services.hyperhive.swarm.otel.scrapeTargets = lib.mkIf config.services.hyperhive.swarm.otel.enable {
nats = "127.0.0.1:${toString cfg.metricsPort}";
};
services.hyperhive.swarm.otel.scrapeTargets =
lib.mkIf config.services.hyperhive.deploy.swarm-otel.enable
{
nats = "127.0.0.1:${toString cfg.metricsPort}";
};
containers.swarm-nats = {
autoStart = true;
@ -683,7 +677,7 @@ in
# door nobody can open, which is the deliberate interim state.
#
# ⚠️ It is gated on the seeds being configured rather than on
# `cfg.enable`, so a half-configured hive gets a running,
# `deployCfg.nats.enable`, so a half-configured hive gets a running,
# refusing queue instead of a unit that crash-loops on a
# missing file. A queue that denies everyone is a legible
# failure; a restart loop is not.
@ -767,7 +761,7 @@ in
# being up says nothing about whether its in-container secrets unit
# has finished. The wait in the script is what actually closes it;
# this only stops us spinning for the full timeout on every boot.
++ lib.optional autheliaCfg.enable "container@${autheliaCfg.machine}.service";
++ lib.optional deployCfg.authelia.enable "container@${autheliaCfg.machine}.service";
requires = lib.optional cfg.autoGenerateCallout "swarm-nats-callout-keys.service";
serviceConfig = {
Type = "oneshot";

Some files were not shown because too many files have changed in this diff Show more