834ea88420
argus's optional note. Nothing in the doc was inaccurate, but an operator who sees the activation-time warning has no thread back to the page that explains why pinning matters -- and this section is the explanation. States the one property they need from it: it never fails the rebuild, so acting on it is theirs to do before the ids are minted.
285 lines
13 KiB
Markdown
285 lines
13 KiB
Markdown
# hive-matrix
|
|
|
|
Private Matrix homeserver (matrix-tuwunel — the conduwuit
|
|
successor) wrapped in a nixos-container, plus optional fluffychat-web
|
|
client at `chat.<swarm-domain>/` (the `gatewayHost` vhost). Configured via
|
|
`services.hyperhive.swarm.matrix.*`; vhost routing lives in
|
|
[`gateway.md`](gateway.md).
|
|
|
|
## Container shape
|
|
|
|
Same shape as [`gateway.md::hive-forge container shape`](gateway.md):
|
|
|
|
- Container name `hive-matrix` (not `h-*`) so c0re's lifecycle
|
|
scanner ignores it; operator manages via the standard
|
|
`nixos-container` CLI.
|
|
- Keeps hive-matrix from fighting any `services.matrix-*` the
|
|
operator already runs on the host — separate systemd namespace,
|
|
separate state dir.
|
|
- Container shares the host network namespace
|
|
(`privateNetwork = false`) for state + systemd-unit isolation. Agents
|
|
reach the homeserver at `chat.<swarm-domain>` via the gateway (agents
|
|
run in private netns and can't access host loopback directly).
|
|
- Persistent state at
|
|
`/var/lib/nixos-containers/hive-matrix/var/lib/matrix-tuwunel/`
|
|
survives container restart / host reboot. To wipe, destroy the
|
|
container.
|
|
|
|
## Identity vs API listener: `serverName` vs `gatewayHost`
|
|
|
|
Two distinct hostnames:
|
|
|
|
- **`serverName`** — matrix-spec `server_name`, embedded
|
|
*irrevocably* in every `@user:<server_name>` and `!room:<server_name>`
|
|
identifier minted on this homeserver. Cannot be changed later
|
|
without abandoning every account and chat history. Defaults to the
|
|
bare `services.hyperhive.swarm.domain`; clients auto-discover the
|
|
actual API endpoint via the `.well-known/matrix/{client,server}`
|
|
routes the gateway serves at that domain.
|
|
- **`gatewayHost`** — the API listener hostname, where the gateway's
|
|
matrix vhost proxies `/_matrix/*` to tuwunel. Defaults to
|
|
`chat.<services.hyperhive.swarm.domain>`. Set to `null` to skip the
|
|
gateway vhost (tuwunel stays direct on `httpPort`).
|
|
|
|
Both now default under the **swarm** domain, because a swarm runs one
|
|
homeserver: tying its identity to a single hive's domain would make
|
|
relocating the container between hives look like a different
|
|
homeserver.
|
|
|
|
⚠️ **They are still not interchangeable, and the difference is the
|
|
cost of changing one.** `gatewayHost` is a routing detail clients
|
|
rediscover through `.well-known`, so it is safe to move on a running
|
|
deployment. `serverName` is baked into every user and room id, so
|
|
adopting a new one does **not** rename the existing users and rooms —
|
|
it strands them, because their ids still name a homeserver that no
|
|
longer answers.
|
|
|
|
**Breaking change — pin `serverName` before rebuilding.** Its default
|
|
has now moved twice: from `matrix.${services.hyperhive.domain}`, to
|
|
the bare hive domain, and now to the swarm domain. Any homeserver that
|
|
has already minted ids must name the value it minted them under:
|
|
|
|
```nix
|
|
services.hyperhive.swarm.matrix = {
|
|
# whichever this deployment already uses
|
|
serverName = config.services.hyperhive.domain;
|
|
gatewayHost = "matrix.${config.services.hyperhive.domain}";
|
|
};
|
|
```
|
|
|
|
A rebuild on a host that already has a homeserver prints a
|
|
`hive-matrix: WARNING — … serverName is unset` line when this is missing,
|
|
naming the value it is about to default to. That warning is why this
|
|
section exists; it never fails the rebuild, so it is on you to act on it
|
|
before the ids are minted.
|
|
|
|
## Default-closed firewall
|
|
|
|
`openFirewall` defaults to `false` (secure-by-default): the host
|
|
reaches the homeserver on loopback, and agent containers reach it
|
|
at `chat.<swarm-domain>` via the gateway — so the firewall hole only
|
|
matters for access from *outside* the host. Flip to `true` when
|
|
announcing the homeserver to other hives or when an external matrix
|
|
client needs to reach the client-server API directly.
|
|
|
|
**Breaking change**: used to default to `true`. Operators relying on
|
|
external reach must add
|
|
`services.hyperhive.swarm.matrix.openFirewall = true;` before rebuilding.
|
|
|
|
Federation port 8448 is intentionally not opened here — tuwunel
|
|
serves the federation API on the same `httpPort` as client-server
|
|
by default. Reaching it on 8448 needs either an explicit tuwunel
|
|
bind to that port OR a reverse-proxy + `.well-known/matrix/server`
|
|
delegation (the latter lives in `gateway.md::Discovery flow`).
|
|
|
|
## Provisioning flow (registration token)
|
|
|
|
Token-gated registration: hive-c0re holds the token, agents never
|
|
see it. The agent only receives the resulting `access_token`.
|
|
|
|
1. **System activation** writes a 32-byte random hex token (64
|
|
chars) to `cfg.registrationTokenFile`
|
|
(`/var/lib/hyperhive/matrix-register-token` by default), mode
|
|
`0600 root:root`, before any container start. Idempotent — only
|
|
writes when the file is missing or empty; always re-applies 0600
|
|
(normalises any 0640 / world-readable carry-over from
|
|
pre-LoadCredential deployments). This runs at activation time
|
|
(not first container start) to dodge a race where nspawn creates
|
|
an empty file when the bind-mount target is missing and tuwunel
|
|
reads `registration_token_file=""`, rejecting every registration
|
|
until next restart.
|
|
2. **Read-only bind-mount** maps the host file into the tuwunel
|
|
container at the same path.
|
|
3. **systemd `LoadCredential=`** inside the container copies the
|
|
bind-mounted file into
|
|
`/run/credentials/tuwunel.service/registration_token`, owned by
|
|
tuwunel's dynamic user with mode `0400`, at service start. The
|
|
host file stays `root:root 0600` — no `chown :tuwunel` /
|
|
`chmod 0640` / GID-pin gymnastics required. Keeps
|
|
`DynamicUser = true` + `PrivateUsers = true` intact.
|
|
4. tuwunel's `registration_token_file` points at the credentials
|
|
path, not the original bind-mount path.
|
|
5. **hive-c0re** uses the token to register each agent account via
|
|
the matrix-spec UIAA registration flow, persists the returned
|
|
`access_token` to `<agent-state>/matrix-token`. The agent's
|
|
matrix MCP client authenticates with that access_token and
|
|
never touches the shared registration token.
|
|
6. **hive-c0re restarts `hive-matrix-daemon`** for the agent
|
|
immediately after writing the token so the daemon picks up the
|
|
new credential without waiting for a full container restart. If
|
|
the restart fails (e.g. daemon not yet running on first boot) the
|
|
error is logged as a warning and the `.path`-trigger sibling
|
|
(`hive-matrix-daemon.path` watching for `matrix-token` appearance)
|
|
brings the daemon up on the same boot cycle anyway.
|
|
|
|
Initial rollout settings:
|
|
|
|
- `allow_federation = true` at the protocol level so swarms can be
|
|
wired up later by extending `trustedServers` without a homeserver
|
|
restart. `trusted_servers = []` keeps it effectively closed
|
|
until peers are listed.
|
|
- `allow_registration = true` (required for the token flow to
|
|
engage). The absent
|
|
`yes_i_am_very_very_sure_…_open_registration_…` flag keeps the
|
|
server closed to anyone without the token.
|
|
- `allow_encryption` — server-side E2EE switch, sourced from
|
|
`services.hyperhive.swarm.matrix.allowEncryption` (**default `false`**, opt-in).
|
|
Off by default because on the hive-internal homeserver the operator
|
|
already controls the transport; turn it on for encrypted rooms on
|
|
external / federated homeservers or to keep contents opaque to the
|
|
homeserver admin. **The agent matrix client always supports decryption
|
|
regardless of this flag** — it uses the `e2e-encryption` feature of
|
|
`matrix-sdk` so it can read encrypted rooms it is invited to even when
|
|
this homeserver doesn't permit room encryption. Crypto keys are stored
|
|
in the per-agent sqlite store under the state dir and persist across
|
|
restarts (lost on `--purge`). `read_room` decrypts via
|
|
`room.messages()` — UTD events surface as `event_type =
|
|
"m.room.encrypted"` with `body = "[unable to decrypt]"`.
|
|
Cross-signing and automatic key backup are not enabled for the first
|
|
pass: static bearer-token bot accounts can't bootstrap cross-signing
|
|
without MSC3967.
|
|
|
|
## Hive Matrix Space
|
|
|
|
On first boot (after all agent accounts are provisioned), hive-c0re
|
|
creates a private **Matrix Space** named `"hive"` using the admin
|
|
account (`@hive:<server_name>`) and invites every provisioned agent
|
|
into it. This gives the operator a single Space in FluffyChat or any
|
|
Matrix client that groups all agent-to-agent + operator rooms in one
|
|
place.
|
|
|
|
The sweep also provisions a default **`hive-chat` room** as an
|
|
`m.space.child` of the Space. Joining a Space doesn't auto-join
|
|
child rooms — the explicit room entry ensures the operator and every
|
|
agent can find a common chat room without manual setup. Room join is
|
|
restricted (any Space member including the operator can join; agents
|
|
are explicitly invited). Room version pinned to 10 for the restricted
|
|
join floor.
|
|
|
|
**State**: both room IDs are persisted to `/var/lib/hyperhive/matrix/`
|
|
(mode `0600`, owned by the hive-c0re service user):
|
|
- `space-room-id` — the Space itself
|
|
- `chat-room-id` — the `hive-chat` room
|
|
|
|
These paths are **outside** every agent state dir and are NOT deleted by
|
|
`nixos-container destroy --purge` — both survive full agent purges and
|
|
are reused on re-provision.
|
|
|
|
**Idempotent**: if the files exist and are non-empty, the Space and
|
|
room are considered already created. Delete the files to force
|
|
re-creation (e.g. after a homeserver wipe).
|
|
|
|
## Configuration tuning
|
|
|
|
```nix
|
|
services.hyperhive.swarm.matrix = {
|
|
trustedServers = [ "matrix.org" "example.com" ]; # default: []
|
|
maxRequestSize = 20000000; # default: 20 MB
|
|
};
|
|
```
|
|
|
|
**`trustedServers`** (default `[]`) — list of peer homeserver names
|
|
whose signing keys tuwunel will fetch and trust. Federation is enabled
|
|
at the protocol level from first boot (`allow_federation = true`) but
|
|
no remote homeserver is trusted until listed here. For a closed
|
|
single-hive deployment the default empty list is correct — add peer
|
|
hive domains here when connecting hives into a swarm (see
|
|
[`docs/swarm/`](swarm/README.md)).
|
|
|
|
**`maxRequestSize`** (default `20_000_000` bytes = 20 MB) — maximum
|
|
size of a single matrix client request body. Matches the matrix-spec
|
|
recommendation for media uploads and the upstream tuwunel default.
|
|
Raise for deployments that need large file transfers; lower for
|
|
resource-constrained hosts where a 20 MB request is unexpectedly large.
|
|
|
|
## Assertion rationale
|
|
|
|
`config.assertions` in this module fail eval early rather than ship
|
|
surprising behaviour:
|
|
|
|
- **`cfg.gatewayHost != ""`** — same footgun as `forge.domain`:
|
|
empty string renders `.<hive>`-shaped garbage in both nginx
|
|
`server_name` (treated as wildcard catch-all, surprising) and
|
|
`/etc/hosts` (invalid entry). `null` is the right opt-out shape;
|
|
empty string is rejected explicitly.
|
|
- **`sso.enable` requires `sso.clientSecretFile`** — fails at eval,
|
|
not at boot: tuwunel reads its identity providers from the config
|
|
file, so a half-configured one can stop the homeserver from
|
|
starting outright rather than merely hiding a login button.
|
|
- **`sso.enable` requires `swarm.authelia.url`** — without a
|
|
provider URL there is nothing to discover against.
|
|
- **`sso.enable` requires `gatewayHost != null`** — the SSO callback
|
|
URL is format-locked to `<homeserver>/_matrix/client/unstable/login/sso/callback/<client_id>`,
|
|
and the identity provider needs a public name to redirect the
|
|
browser to.
|
|
|
|
`server_name`'s own bogus-value guard lives in `hive-network.nix`
|
|
(`services.hyperhive.domain != null`), not here — see
|
|
[`docs/network.md`](network.md).
|
|
|
|
## fluffychat-web build fixes
|
|
|
|
`pkgs.fluffychat-web` ships from `flutter341.buildFlutterApplication`,
|
|
which has two upstream gaps for fluffychat's web target:
|
|
|
|
- The dart web-worker entry point (`web/native_executor.dart`) isn't
|
|
compiled — `buildFlutterApplication` only runs `flutter build web`
|
|
on the main entry.
|
|
- `native_imaging`'s C source isn't built — emscripten isn't a
|
|
flutter-builder native build input.
|
|
|
|
Both fixed in `nix/host-modules/hive-matrix.nix` via two derivations:
|
|
|
|
- **`fluffychat-web-imaging`** builds `Imaging.{js,wasm}` from the
|
|
`native_imaging` C source via `pkgs.emscripten`. Source comes
|
|
from `pkgs.fluffychat-web.passthru.pubspecLock.dependencySources.native_imaging`
|
|
— already in the build closure of the flutter app, so no parallel
|
|
hash pin and version auto-syncs with nixpkgs bumps. Build closure
|
|
is ~3.6 GiB (emscripten LLVM); runtime closure is just the two
|
|
output files. `dontConfigure = true` because cmake runs inside
|
|
`js/Makefile` via `emcmake cmake`, not at the package root. The
|
|
build script needs `HOME` + `EM_CACHE` writable for emscripten's
|
|
on-demand sysroot build (libc, libc++ → wasm).
|
|
- **`fluffychat-web-fixed`** is `pkgs.fluffychat-web` plus a
|
|
`postInstall` patch that (a) compiles `web/native_executor.dart`
|
|
via `dart compile js` (dart from the flutter341 closure, no
|
|
incremental cost) and (b) installs `fluffychat-web-imaging`'s
|
|
outputs into `$out`.
|
|
|
|
Two non-obvious details worth knowing before touching either derivation:
|
|
|
|
- **`make -C js`** instead of `cd js; make` — keeps the build-phase
|
|
pwd at the source root so `installPhase` doesn't have to know
|
|
about the cd. Robust against future reorders / `dontBuild`.
|
|
- **`web/native_executor.dart`** as a build-CWD-relative path,
|
|
*not* `$src/web/...` — `dart`'s `package_config.json` walk-up
|
|
needs to hit `buildFlutterApplication`'s pub-get output
|
|
(`.dart_tool/` in the build CWD). Walking up from a read-only
|
|
`$src/` store path finds no `.dart_tool/` and errors with
|
|
"Couldn't resolve the package 'matrix'".
|
|
|
|
Drop both derivations when nixpkgs's flutter builder grows worker
|
|
+ emcc support upstream.
|
|
|
|
Mount point is `chat.<swarm-domain>/` (the `gatewayHost` vhost);
|
|
upstream `--base-href "/"` is correct at sub-domain root, no override.
|