vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
hyperhive/nix/modules/hive-gateway.nix
atlas 01cc664d76 nix/hive-{gateway,matrix}: matrix vhost at matrix.<hive-domain> + .well-known delegation (#747) 
mara on #747:9722: "this still seems to be an issue in current version"
(after #751 closed without merge). Mirroring the forge sub-domain
pattern just merged as #754 for matrix per mara's #749:9609 verdict
(sub-domain over sub-path for forge + matrix, "not user-visible for
matrix because the .well-known/matrix/{client,server} redirect routes
clients through automatically").

## Mechanics

**New `services.hyperhive.matrix.gatewayHost`** — nullable str, defaults
to `matrix.<services.hyperhive.domain>` when hive-domain set, else
null. Full hostname (`matrix.darkest.space`, `homeserver.internal.lan`)
for bespoke shapes per mara's #754:9684 "specify full domain in
options instead" pattern.

**Gateway:** new `server { server_name = matrixCfg.gatewayHost; }`
block proxying `/_matrix/...` → `http://127.0.0.1:<httpPort>/_matrix/...`
with matrix-spec CORS + tuned for long-poll `/sync` (1h timeout) +
typical media uploads (50M body cap). `/` returns 404 — nothing
else lives at the matrix vhost. Matches the forge vhost shape from #754.

**`.well-known/matrix/{client,server}`** (already served at bare hive-
domain since #660): now points at `matrixCfg.gatewayHost` (no port
suffix when gateway is on the canonical port 80) instead of the
direct `<hive-domain>:<httpPort>` shape. Falls back to direct shape
when `gatewayHost = null` (no hive-domain, or operator nulled it).

**`localHostsEntry` extension**: `/etc/hosts` (when set) now adds the
matrix sub-domain → 127.0.0.1 alongside hive-domain + forge.domain.
`lib.unique` collapses any duplicate (edge case if operator sets
gatewayHost equal to hive-domain).

## Verified via `nix eval`

```
vhosts:         ["_", "forge.test.local", "matrix.test.local"]
gatewayHost:    "matrix.test.local"
client wellknown: m.homeserver.base_url = "http://matrix.test.local"
server wellknown: m.server = "matrix.test.local"
/etc/hosts:     ["test.local", "forge.test.local", "matrix.test.local"]
```

## What this fixes for #747

mara's HAR showed `GET /.well-known/matrix/client` and
`GET /_matrix/client/versions` both failing on `pr1ma.darkest.space`:

1. **`.well-known/matrix/client`** was advertising
   `http://pr1ma.darkest.space:8008` — that URL only works if tuwunel's
   port 8008 is firewall-open to the operator's browser (it isn't by
   default — `services.hyperhive.matrix.openFirewall` defaults to false
   since #651). Now advertises `http://matrix.pr1ma.darkest.space/`
   which goes through the gateway on the (already-open) port 80.

2. **`/_matrix/client/versions`** was hitting the bare-domain `"_"`
   vhost, which has no `/_matrix/` location — fell through to `/` →
   c0re's dashboard upstream → 404. Now hits the new `matrix.<hive>`
   vhost which proxies the request to tuwunel cleanly.

server_name + serverName unaffected — matrix identifiers (`@alice:<hive>`)
still embed the bare hive-domain per #660; only the wire-level transport
URL moves to the sub-domain.

## Risk

Medium. Existing matrix tokens / sessions stay valid because:
- `serverName` (the identifier domain) doesn't change
- tuwunel's `/_matrix/` endpoints serve the same requests, just reached
  via the new sub-domain instead of the direct port

Operators with `services.hyperhive.matrix.openFirewall = true` and
external clients reaching `:8008` directly keep working too — the
sub-domain vhost is additive, doesn't take away the direct port.

## Sequencing

This is a parallel matrix-side mirror of #754 (forge). Both follow
the same mara-verdict pattern; once both have soaked, the gateway-
behind-everything story is done for v0.

Closes #747.
2026-05-31 14:08:40 +02:00

558 lines
26 KiB
Nix
Raw Blame History

{
pkgs,
lib,
config,
...
}:
let
cfg = config.services.hyperhive.gateway;
hyperhiveDomain = config.services.hyperhive.domain;
matrixCfg = config.services.hyperhive.matrix;
forgeCfg = config.services.hyperhive.forge;
# Per-agent port table for `/agent/<name>/` routing (#15 v0). Single-
# sourced from `cfg.agentPortsFile` (default
# `/var/lib/hyperhive/agent-ports.json`), written by hive-c0re on every
# topology change in shape `{ "<name>": <port>, ... }`.
#
# Read at deploy time via `builtins.fromJSON (builtins.readFile ...)`
# — pure eval (the file lives outside the nix store; nix copies the
# content into the store as a fixed-output dep). When the file is
# missing (fresh install before c0re has had a chance to write it),
# default to an empty map → no per-agent routes generated → gateway
# falls back to its pre-#15 shape. The container rebuilds on every
# `hivectl gateway-sync` (operator-initiated) or on the next
# `nixos-rebuild switch`, picking up whatever c0re has written
# since the last build.
#
# mara on #740 (comment 9295) + #15 (comment 9270): the gateway
# nginx container lives in system config (not meta), so it can't
# auto-rebuild from meta-flake events — the JSON file is what
# bridges the host's nix eval to the agent-lifecycle data c0re owns.
agentPortsTable =
if cfg.agentPortsFile == null || !builtins.pathExists cfg.agentPortsFile then
{ }
else
builtins.fromJSON (builtins.readFile cfg.agentPortsFile);
in
{
# Single nginx in front of every hyperhive surface (#609 / #15 v0).
# Lives in its own nixos-container (like hive-forge / hive-matrix) so
# the operator can opt out without touching the host's own nginx, and
# so the static-serve responsibility for the matrix GUI moves off
# hive-c0re's axum router. Shares host netns so `localhost`
# upstream resolution works without any port-forward dance.
#
# Routes (v0):
# `location /matrix/` → static-serve fluffychat-web dist (when
# `services.hyperhive.matrix.gui.enable` is true)
# `location /` → proxy_pass to hive-c0re's dashboard upstream
#
# Container name `hive-gateway` keeps hive-c0re's lifecycle scanner
# (which only sees `h-*`) out of the picture. State-free — nginx
# config lives in the nix store, no runtime persistence to manage.
options.services.hyperhive.gateway = {
enable = lib.mkOption {
type = lib.types.bool;
default = true;
description = ''
Run hive-gateway — a single nginx in front of every hyperhive
surface. On by default: the gateway hosts the matrix GUI static
dist (when `services.hyperhive.matrix.gui.enable` is true) and
proxies everything else to hive-c0re's dashboard upstream. Set
`services.hyperhive.gateway.enable = false` to bypass nginx
entirely and reach hive-c0re directly on its dashboard port
(7000 by default).
v0 is HTTP-only; TLS / public-domain shape is tracked
separately.
'';
};
port = lib.mkOption {
type = lib.types.port;
default = 80;
example = 8080;
description = ''
TCP port the gateway listens on. Default 80 (canonical web
port). nginx inside the container binds <1024 because the
container's init runs as root; if 80 is already taken on the
host (existing nginx, traefik, etc.) override to an unused
port like 8080 or move the conflicting service.
'';
};
upstreamHost = lib.mkOption {
type = lib.types.str;
default = "127.0.0.1";
description = ''
Host the gateway proxies non-static requests to. Defaults to
`127.0.0.1` because the gateway container shares the host
netns, so loopback resolves directly to hive-c0re.
'';
};
upstreamPort = lib.mkOption {
type = lib.types.port;
default = 7000;
description = ''
TCP port the gateway proxies non-static requests to. Defaults
to `7000` (hive-c0re's out-of-the-box dashboard port). Operators
who change `services.hyperhive.c0re.dashboardPort` should set
`upstreamPort` to match — kept as a hardcoded default rather
than a cross-reference to keep this module's options eval
independent of c0re's option tree shape.
'';
};
openFirewall = lib.mkOption {
type = lib.types.bool;
default = false;
example = true;
description = ''
Open `port` in the host firewall. Off by default (#651,
secure-by-default). Flip to `true` to expose the gateway to
the operator's browser / external clients — required for any
out-of-host reach, since the agents themselves talk to
hive-c0re via the per-agent unix sockets and don't need the
nginx vhost. Leave off when running behind another reverse
proxy (e.g. caddy / traefik on the host) that handles TLS
termination + forwards to `port`.
**Breaking change as of #651**: this used to default to
`true`. If you relied on the old default for external reach
(the common case — the gateway is the operator's primary
entry point), add `services.hyperhive.gateway.openFirewall = true;`
to your host config before rebuilding.
'';
};
localHostsEntry = lib.mkOption {
type = lib.types.bool;
default = false;
example = true;
description = ''
Add an `/etc/hosts` entry mapping `services.hyperhive.domain`
to `127.0.0.1` on the host. Useful for local deployments +
tests where there's no real DNS for `services.hyperhive.domain`
but the operator (or browser-based tests) want to hit
`http://''${services.hyperhive.domain}` to exercise the
gateway shape. Off by default — operators running with real
DNS shouldn't have a stale `/etc/hosts` entry sticking
around. Requires `services.hyperhive.domain` to be set.
'';
};
agentPortsFile = lib.mkOption {
type = lib.types.nullOr lib.types.path;
default = "/var/lib/hyperhive/agent-ports.json";
example = "/var/lib/hyperhive/agent-ports.json";
description = ''
Path to a JSON file mapping sub-agent names to their web ports
for `/agent/<name>/` routing through the gateway (#15 v0).
Shape: `{ "<name>": <port>, ... }`. Written by hive-c0re on
every topology change (the rust side knows the canonical port
allocation via `lifecycle::agent_web_port`; the gateway just
reads what it's told).
For each `<name>: <port>` entry, the gateway adds a
`location /agent/<name>/` block that `proxy_pass`es to
`http://127.0.0.1:<port>/`. Empty / missing file → no
per-agent routes generated → gateway falls back to its pre-#15
shape (just `/` + matrix surfaces).
**Purely additive**: the old `http://<host>:<port>/` direct
reach keeps working in parallel; this just gives the operator
a single-origin route. Manager isn't included in the map (no
per-agent prefix needed; manager already gets the `/` route
via the c0re upstream block).
Set to `null` to disable per-agent routing entirely without
creating the file. Set to a custom path if the operator's c0re
writes the table elsewhere.
**Rebuild trigger**: the gateway container picks up new entries
on the next `nixos-rebuild switch` (or `hivectl gateway-sync`
if that helper lands). c0re writes are not auto-applied to a
running gateway — see the follow-up in #15 for runtime nginx
include + reload + eventual per-agent unix sockets.
'';
};
};
config = lib.mkIf cfg.enable {
assertions = [
{
assertion = !cfg.localHostsEntry || hyperhiveDomain != null;
message = ''
services.hyperhive.gateway.localHostsEntry = true requires
services.hyperhive.domain to be set. Either pin a hostname
or leave `localHostsEntry` at its default of false.
'';
}
];
containers.hive-gateway = {
autoStart = true;
ephemeral = false;
# Share host netns — nginx then binds host-level ports directly,
# `localhost` upstream resolution reaches hive-c0re without any
# port-forward dance, and the firewall config below is the only
# layer that matters.
privateNetwork = false;
config =
{ pkgs, ... }:
{
system.stateVersion = "26.05";
services.nginx = {
enable = true;
recommendedProxySettings = true;
recommendedOptimisation = true;
# SPA-fallback target keyed on the `Accept` request header
# (#686, mara + damocles on PR #729). This decides whether a
# `/matrix/...` miss falls through to `index.html` (route
# navigation) or returns a clean 404 (asset miss) — see the
# `/matrix/` location comment below for the full rationale.
#
# Top-frame browser navigations always send
# `Accept: text/html,...` (chrome/firefox/safari are
# consistent on this). Asset fetches from script tags / img
# / fetch() / XHR send asset-typed Accepts (`image/*`,
# `application/javascript`, `*/*`) without `text/html`.
# Mapping is purely on the header → no extension allowlist
# to keep in sync with whatever the SPA ships, no regex
# heuristic to false-positive on dot-segment routes.
#
# `$matrix_spa_target` defaults to a sentinel nonexistent
# path so `try_files` falls through to the trailing `=404`
# for asset misses. Browser navigations route to
# `/matrix/index.html` where the SPA's client-side router
# takes over.
#
# Only emitted when the matrix GUI is on (saves a no-op
# `map` directive otherwise).
appendHttpConfig = lib.optionalString (matrixCfg.enable && matrixCfg.gui.enable) ''
map $http_accept $matrix_spa_target {
default "/__matrix_spa_no_html_fallback";
"~*text/html" "/matrix/index.html";
}
'';
virtualHosts = {
"_" = {
listen = [
{
addr = "0.0.0.0";
port = cfg.port;
}
];
locations =
# Matrix GUI: when the operator has flipped both
# `services.hyperhive.matrix.enable` and `matrix.gui.enable`
# on, nginx serves fluffychat-web (or whatever override)
# as a static dist at `/matrix/`.
#
# SPA fallback (iris/#643, rewritten in #686 per mara
# + damocles on PR #729): the original
# `try_files $uri $uri/ /matrix/index.html;` shape
# silently masked missing assets — flutter's bootstrap
# requesting e.g. `/matrix/native_executor.js` got
# `index.html` (Content-Type: text/html, status 200)
# when the file was absent from the dist, so the JS
# runtime never loaded and `/matrix/` rendered blank
# without any visible error.
#
# The followup #729 narrowed it with an extension
# allowlist; this version uses `$matrix_spa_target`
# (defined in the `appendHttpConfig` above, keyed on
# the `Accept` header) so the decision lives in HTTP
# semantics rather than a maintained extension list.
# Navigations (Accept: text/html) fall to index.html;
# asset fetches (Accept: */*, image/*, etc.) get a
# clean 404 via the trailing `=404`.
lib.optionalAttrs (matrixCfg.enable && matrixCfg.gui.enable) {
"/matrix/" = {
alias = "${matrixCfg.gui.package}/";
extraConfig = ''
try_files $uri $uri/ $matrix_spa_target =404;
'';
};
# FluffyChat fetches `/matrix/config.json` directly on
# boot for its own branding + default-homeserver
# bootstrap, BEFORE asking the user to pick a server
# (#736). The upstream `pkgs.fluffychat-web` dist
# ships without one, so the fetch 404s and the user
# sees the empty "enter homeserver" prompt. Serve a
# minimal config that pre-fills `defaultHomeserver`
# with the operator's hive domain — the matrix
# client then runs `.well-known/matrix/client` against
# that domain (already served by the
# `= /.well-known/matrix/client` block below) and
# discovers the actual tuwunel endpoint.
#
# Only the `defaultHomeserver` field is overridden —
# everything else (branding, audio defaults, etc.)
# falls back to fluffychat's hardcoded defaults so
# we don't pin against upstream config-schema drift.
# No-op when `services.hyperhive.domain` is unset
# (the location block is omitted entirely; the SPA
# then falls back to its empty form, same as before
# #736).
} // lib.optionalAttrs (
matrixCfg.enable && matrixCfg.gui.enable && hyperhiveDomain != null
) {
"= /matrix/config.json" = {
extraConfig = ''
default_type application/json;
return 200 '{"defaultHomeserver":"${hyperhiveDomain}"}';
'';
};
}
//
# `.well-known/matrix/*` auto-discovery (#660): when
# `services.hyperhive.matrix.enable` is on and the
# operator's set a hive domain, the gateway serves
# the matrix-spec discovery JSON at the canonical
# location so clients pointed at `${hyperhive.domain}`
# resolve through to the actual tuwunel endpoint
# without needing a `matrix.` subdomain.
#
# `m.homeserver.base_url` advertises the client-server
# API. `m.server` advertises the federation
# `host:port` (tuwunel serves both client + federation
# on the same `httpPort` — see hive-matrix.nix).
#
# CORS `*` on the client endpoint per the matrix spec
# (https://spec.matrix.org/v1.15/client-server-api/#getwell-knownmatrixclient).
# No-op until the operator turns matrix on; until then
# there's no homeserver to advertise.
lib.optionalAttrs (matrixCfg.enable && hyperhiveDomain != null) (
let
# `.well-known/matrix/{client,server}` advertise where
# the actual matrix API lives. When `matrixCfg.gatewayHost`
# is set (default `matrix.<hive-domain>`, #747), point
# at the sub-domain — no port suffix when the gateway
# is on the canonical port 80, transparent to clients
# (mara on #749:9609 sub-domain verdict, "not user-
# visible because the .well-known redirect routes
# clients through automatically"). When `gatewayHost`
# is unset (no hive-domain, or operator nulled it),
# fall back to the direct `host:port` shape — clients
# reach tuwunel without going through the gateway,
# no sub-domain delegation.
portSuffix = if cfg.port == 80 then "" else ":${toString cfg.port}";
clientBaseUrl =
if matrixCfg.gatewayHost != null then
"http://${matrixCfg.gatewayHost}${portSuffix}"
else
"http://${hyperhiveDomain}:${toString matrixCfg.httpPort}";
serverHostPort =
if matrixCfg.gatewayHost != null then
"${matrixCfg.gatewayHost}${portSuffix}"
else
"${hyperhiveDomain}:${toString matrixCfg.httpPort}";
in
{
"= /.well-known/matrix/client" = {
extraConfig = ''
default_type application/json;
add_header Access-Control-Allow-Origin *;
return 200 '{"m.homeserver":{"base_url":"${clientBaseUrl}"}}';
'';
};
"= /.well-known/matrix/server" = {
extraConfig = ''
default_type application/json;
return 200 '{"m.server":"${serverHostPort}"}';
'';
};
}
)
//
# Per-agent UIs (#15 v0). One `/agent/<name>/`
# block per `<name>: <port>` entry in
# `agentPortsTable` (loaded from `cfg.agentPortsFile`
# — `/var/lib/hyperhive/agent-ports.json` by default,
# written by hive-c0re on every topology change).
#
# Trailing-slash pair (`/agent/<name>/` + `proxy_pass
# http://...:<port>/`) strips the `/agent/<name>`
# prefix on the upstream side, so the agent server
# receives `GET /` for the SPA root, `GET /api/state`
# for the API, `GET /screen/ws` for the websocket, etc.
# The agent's emitted asset URLs are document-relative
# (iris's #731) so they round-trip back through the
# gateway under the same prefix without the harness
# needing prefix-awareness.
#
# `X-Forwarded-Prefix` set so the harness can build
# correct absolute URLs for any case where relative
# isn't enough (server-emitted redirects, OG meta
# tags, etc.).
#
# SSE / websocket support via `proxyWebsockets = true`
# (same as the c0re `/` block below).
#
# Empty / missing `cfg.agentPortsFile` → empty table
# → no per-agent blocks; old `<host>:<port>/` direct
# reach still works.
lib.mapAttrs' (name: port: {
name = "/agent/${name}/";
value = {
proxyPass = "http://127.0.0.1:${toString port}/";
proxyWebsockets = true;
extraConfig = ''
proxy_set_header X-Forwarded-Prefix /agent/${name};
proxy_buffering off;
proxy_read_timeout 1d;
'';
};
}) agentPortsTable
// {
# Everything else proxies to hive-c0re. Upgrade
# headers stay set so SSE (`/dashboard/stream`,
# `/events/stream`) + websocket (`/screen/ws`)
# endpoints keep working transparently.
"/" = {
proxyPass = "http://${cfg.upstreamHost}:${toString cfg.upstreamPort}";
proxyWebsockets = true;
extraConfig = ''
proxy_buffering off;
proxy_read_timeout 1d;
'';
};
};
};
}
//
# Forge vhost (#749, mara verdict at issue:9609 —
# sub-domain over sub-path). When forgejo runs behind the
# gateway (`forge.behindGateway = true`), it gets its own
# `server { server_name = forge.domain; }` block. The
# block proxies all `/` → `http://127.0.0.1:<forge.httpPort>/`
# so forgejo handles requests at root (default deploy shape
# — no `ROOT_URL`-prefix translation needed).
#
# `forge.domain` is the full hostname (e.g.
# `forge.darkest.space`, `git.example.com`) — single source
# of truth for both the forgejo `DOMAIN` setting and the
# gateway vhost name (mara on #754:9684 — "specify full
# forge domain in options instead").
#
# `client_max_body_size 1G` — git pushes + LFS uploads can
# be large; nginx's default 1M would 413 most real commits.
#
# Long timeouts for big repo operations: a fresh clone of a
# multi-GB repo can take minutes; the default 60s
# `proxy_read_timeout` would abort mid-stream.
#
# `proxyWebsockets = true` keeps forgejo's live-update
# endpoints (`/api/v1/events`) + any future websocket
# endpoints working transparently. SSH stays direct on
# `forge.sshPort` (separate listener protocol, not HTTP).
lib.optionalAttrs (forgeCfg.enable or false && forgeCfg.behindGateway or false) {
"${forgeCfg.domain}" = {
listen = [
{
addr = "0.0.0.0";
port = cfg.port;
}
];
locations."/" = {
proxyPass = "http://127.0.0.1:${toString forgeCfg.httpPort}/";
proxyWebsockets = true;
extraConfig = ''
proxy_buffering off;
client_max_body_size 1G;
proxy_read_timeout 1h;
proxy_send_timeout 1h;
'';
};
};
}
//
# Matrix homeserver vhost (#747, mara verdict on #749:9609 —
# sub-domain over sub-path for matrix; "not user-visible"
# because clients discover the sub-domain via the
# `.well-known/matrix/{client,server}` delegation served
# above on the bare hive-domain).
#
# `server { server_name = matrixCfg.gatewayHost; }` proxies
# `/_matrix/...` → `http://127.0.0.1:''${matrixCfg.httpPort}/_matrix/...`.
# Tuwunel listens on `:''${httpPort}` (default 8008); the
# gateway terminates on `:''${cfg.port}` (80) so external
# clients speak matrix over the canonical web port without
# operators having to open the tuwunel port through firewalls.
#
# `/` returns 404 — nothing else lives at the matrix vhost;
# the matrix client-server API is entirely under `/_matrix/`,
# and federation under `/_matrix/federation/...`.
#
# CORS `*` on the matrix vhost per the matrix spec —
# federation + client requests come from any origin.
#
# `client_max_body_size 50M` covers typical media uploads
# (matrix-spec media size cap default); operators with bigger
# uploads override via the matrix module's own cap when that
# lands.
#
# `proxy_read_timeout 1h` for long-poll `/sync`; the default
# 60s would abort `/sync?timeout=30000` legitimately when
# tuwunel's keepalive exceeds that.
lib.optionalAttrs (matrixCfg.enable && matrixCfg.gatewayHost != null) {
"${matrixCfg.gatewayHost}" = {
listen = [
{
addr = "0.0.0.0";
port = cfg.port;
}
];
locations = {
"/_matrix/" = {
proxyPass = "http://127.0.0.1:${toString matrixCfg.httpPort}";
proxyWebsockets = true;
extraConfig = ''
proxy_buffering off;
client_max_body_size 50M;
proxy_read_timeout 1h;
proxy_send_timeout 1h;
add_header Access-Control-Allow-Origin *;
'';
};
"/" = {
return = "404";
};
};
};
};
};
};
};
networking.firewall = lib.mkIf cfg.openFirewall {
allowedTCPPorts = [ cfg.port ];
};
# `/etc/hosts` entries for local dev: the bare hive domain plus
# any sub-domain modules (forge via #749/#754, matrix via #747)
# that are on. All map to `127.0.0.1` since the gateway shares
# host netns. Operators with real DNS leave `localHostsEntry =
# false`; this is the dev-loop shortcut for `http://<hive-domain>/`
# + `http://forge.<hive-domain>/` + `http://matrix.<hive-domain>/`
# resolving locally.
#
# `lib.unique` collapses any duplicate (e.g. if forge.domain
# happens to equal hyperhiveDomain or matrixCfg.gatewayHost) so
# `/etc/hosts` doesn't carry the same entry twice.
networking.hosts = lib.mkIf (cfg.localHostsEntry && hyperhiveDomain != null) {
"127.0.0.1" = lib.unique (
[ hyperhiveDomain ]
++ lib.optional (
(config.services.hyperhive.forge.enable or false)
&& (config.services.hyperhive.forge.behindGateway or false)
) config.services.hyperhive.forge.domain
++ lib.optional (matrixCfg.enable && matrixCfg.gatewayHost != null) matrixCfg.gatewayHost
);
};
};
}
Reference in a new issue View git blame Copy permalink