c364d262e5
Review feedback: the metrics pair had nothing writing into it, and it sat outside the switch that turns on every other swarm-wide service. The collector now exports to VictoriaMetrics as well as upstream — a fan-out, not a choice: a local store is for looking at this swarm, an upstream is for whoever aggregates across swarms. That makes a local store a complete destination on its own, so `otel.endpoint` is no longer required when it runs here; a hive with neither is still refused. The assertion only ever relaxes, so every config that evaluated before still does. `enableRequiredServices` now derives both halves, alongside matrix, authelia and nats. They derive together because a store with no UI is unreadable and a UI with no store is empty.
107 lines
5.4 KiB
Markdown
107 lines
5.4 KiB
Markdown
# Swarm-wide services
|
|
|
|
Some things exist once per **swarm**, not once per hive: the forge, the
|
|
matrix homeserver, the SSO provider, the CA. Two options say where the
|
|
optional ones live, and everything else derives:
|
|
|
|
```nix
|
|
services.hyperhive.enableAllLocalDefaults = true; # everything on this box
|
|
# or, for a dedicated services host with hives elsewhere:
|
|
services.hyperhive.swarm.enableRequiredServices = true;
|
|
```
|
|
|
|
`enableAllLocalDefaults` is the all-on-one-box switch: it defaults both
|
|
`swarm.enableRequiredServices` (the shared services run here) and
|
|
`swarm.ca.autoConfigure` (the swarm CA is generated here). Each derived
|
|
toggle can still be set on its own, so "all local except X" needs no
|
|
further option.
|
|
|
|
**Both default to off**, and that is deliberate: a host cannot tell
|
|
whether it is meant to be the swarm's service host, so this is an
|
|
operator saying so rather than something inferred. With them off, a hive
|
|
is a *client* of those services — it configures how to reach them and
|
|
runs none of them.
|
|
|
|
The forge is the exception, and not because it is per-hive: it is
|
|
swarm-wide but **not optional**, being the canonical store for the meta
|
|
flake and every agent's config repo, so it deploys with hyperhive itself
|
|
and has no `enable` to derive from anything.
|
|
|
|
### SSO (authelia)
|
|
|
|
One authelia per swarm, in a `swarm-authelia` container, at
|
|
`auth.<swarm-domain>`. Operator and agents are both subjects of the same
|
|
provider, differentiated by roles and claims rather than by mechanism —
|
|
there is one IdP and one auth path.
|
|
|
|
- **`swarm.authelia.enable`** — run the container here. Defaults from
|
|
`swarm.enableRequiredServices`.
|
|
- **`swarm.authelia.url`** — where clients are sent to authenticate.
|
|
Present on **every** hive, defaulting to this host's own instance only
|
|
when this module is the thing running it; otherwise `null`, and a hive
|
|
joining someone else's swarm sets it explicitly. Null means "no SSO
|
|
configured", and consumers say so rather than guessing an address.
|
|
|
|
The users database is written by swarm-controller, not by hand: agents
|
|
are created and destroyed continuously, so the subject set is dynamic.
|
|
This module only guarantees the file exists and parses, so authelia
|
|
starts with nobody in it rather than failing to start — a provider with
|
|
no subjects yet is the correct state before anything has provisioned
|
|
them. Session and storage keys are generated in the container on first
|
|
boot and never rotated automatically; replacing one invalidates data
|
|
already written (sessions, the encrypted store), so that is an operator
|
|
action.
|
|
|
|
Storage is local sqlite and the notifier writes to a file. Both are
|
|
small-deployment choices, and the scope is the justification: redis
|
|
buys shared session state across replicas and there is one instance;
|
|
SMTP exists to mail humans, and provisioning here is programmatic.
|
|
|
|
See [`sso.md`](sso.md) for bootstrapping the first user and the OIDC
|
|
relying-party flow, and [`secrets.md`](secrets.md) for where each of
|
|
authelia's keys is generated and read.
|
|
|
|
### Metrics (VictoriaMetrics + Grafana)
|
|
|
|
The swarm's telemetry lands in one VictoriaMetrics and is read through
|
|
one Grafana, in two containers at `metrics.<swarm-domain>` and
|
|
`grafana.<swarm-domain>`. Two containers rather than one so Grafana can
|
|
be restarted or broken without taking the time-series database with it.
|
|
|
|
Both follow `swarm.enableRequiredServices` like authelia and matrix, so
|
|
the swarm's service host gets them with everything else. They derive
|
|
together: a store with no UI is unreadable and a UI with no store is
|
|
empty. To run one without the other, set it directly:
|
|
|
|
```nix
|
|
services.hyperhive.swarm.victoriametrics.enable = true;
|
|
services.hyperhive.swarm.grafana.enable = false;
|
|
```
|
|
|
|
⚠️ **This starts a database that grows for as long as the swarm runs.**
|
|
See `retentionPeriod` below before leaving it at its default.
|
|
|
|
| Option | When you'd touch it |
|
|
|---|---|
|
|
| `swarm.victoriametrics.retentionPeriod` | Default `5y`. Lower it once you have measured how fast this swarm actually fills a disk — the default is deliberately generous because too-short silently discards history you cannot get back. |
|
|
| `swarm.grafana.oidc.role` | Default `Admin` for everyone who logs in. Lower to `Viewer`/`Editor` if the swarm grows operators who should not be able to reconfigure Grafana. |
|
|
| `swarm.grafana.datasourceUrl` | Only if you front VictoriaMetrics with something else. It defaults to the store on this host, which is the only thing it can reach. |
|
|
|
|
**Logging in.** Grafana is behind swarm SSO, so the accounts are the
|
|
authelia ones — there is no separate Grafana password, and the local
|
|
login form is switched off whenever SSO is configured. If you enable
|
|
Grafana on a host with no authelia, the form stays on and Grafana's
|
|
default `admin`/`admin` applies; change it before exposing that host.
|
|
|
|
**Where the data comes from.** With `otel.enable` on, the hive's OTEL
|
|
collector writes into this store as well as to any upstream endpoint —
|
|
both, not one or the other, since a local store is for looking at this
|
|
swarm and an upstream is for whoever aggregates across swarms. That also
|
|
means `otel.endpoint` is no longer required when the store runs here: a
|
|
hive with a local store already has somewhere for telemetry to go. See
|
|
[`../observability.md`](../observability.md).
|
|
|
|
Neither container is reachable except through the gateway: both bind
|
|
loopback, and VictoriaMetrics' write endpoint takes no credential, so
|
|
the collector is the only intended writer.
|
|
|