hive-matrix: run the homeserver in its own netns, behind a unix socket
tuwunel bound `0.0.0.0:<httpPort>` in the host netns, so the homeserver
port was open on every host address, and the gateway proxied to it on
host loopback.
The container now sets `privateNetwork = true` and hangs off the hive
bridge with a DHCP lease, the same attach the CI runner, builder and
review classifier use (resolvconf on, `networking.nameservers` the bridge
IP). tuwunel listens only on `/run/hive-matrix/tuwunel.sock`, a host
directory (root:nginx 0770) bind-mounted into the container; its primary
group is nginx's gid and the socket is 0660. The gateway's `/_matrix/`
location proxies to the socket.
The static resolv.conf goes: it existed because resolvconf's host
tracking came up empty in the shared netns.
`swarm.matrix.httpPort` and `{swarm,deploy}.matrix.openFirewall` are
removed with a message: there is no host port to choose or open. The
`.well-known/matrix/*` fallback to `<hive>:<httpPort>` is dropped with
them; it only rendered with `gatewayHost = null`, which the homeserver's
SSO assertion already rejects.
Part of #4815.
This commit is contained in:
atlas
mara
parent
e455964cb3
commit
34e142bb5c
11 changed files with 191 additions and 163 deletions
|
|
@ -16,7 +16,7 @@ You rarely switch it on yourself. `gateway.enable` defaults to off, and every mo
|
|||
| `auth.<swarm>/` | authelia (`9091`) | `swarm-authelia.nix`, `deploy.authelia.enable` |
|
||||
| `auth.<swarm>/bridge/*` | swarm-authelia-bridge (`9092`), prefix stripped; answers swarm-controller's client only | `swarm-authelia.nix`, `deploy.authelia.enable` |
|
||||
| `forge.<swarm>/` | forgejo (`3000`) | `hive-forge/`, `deploy.forgejo.enable` |
|
||||
| `chat.<swarm>/_matrix/*` | tuwunel (`8008`) | `hive-matrix.nix`, `swarm.matrix.gatewayHost != null` |
|
||||
| `chat.<swarm>/_matrix/*` | tuwunel (unix socket `/run/hive-matrix/tuwunel.sock`) | `hive-matrix.nix`, `swarm.matrix.gatewayHost != null` |
|
||||
| `chat.<swarm>/` | fluffychat-web static | `hive-matrix.nix`, `swarm.matrix.gatewayHost != null` |
|
||||
| `chat.<swarm>/config.json` | inline JSON (FluffyChat boot config) | `hive-matrix.nix`, `swarm.matrix.gatewayHost != null` |
|
||||
| `<queue domain>/metrics` | the queue exporter's `/metrics` on loopback, behind an authelia `auth_request`; every other path, `/` included, returns `403`, so `/varz` and `/connz` are never reachable | `swarm-nats.nix`, the queue's host |
|
||||
|
|
@ -221,7 +221,7 @@ The operator points a client at `<hive>`:
|
|||
|
||||
1. The client fetches `https://<hive>/.well-known/matrix/client` → `{"m.homeserver":{"base_url":"https://chat.<swarm>"}}` (no port suffix when `httpsPort` is 443).
|
||||
2. It connects to `chat.<swarm>/_matrix/client/...`.
|
||||
3. The gateway proxies `/_matrix/*` to tuwunel at `127.0.0.1:8008`.
|
||||
3. The gateway proxies `/_matrix/*` to tuwunel over its unix socket, `/run/hive-matrix/tuwunel.sock`.
|
||||
|
||||
matrix-dart-sdk (FluffyChat and others) always fetches the well-known over `https`, which is why [self-signed TLS](#self-signed-tls-default) is on by default.
|
||||
|
||||
|
|
|
|||
|
|
@ -22,12 +22,11 @@ that touches it.
|
|||
│ uplink NIC — NAT MASQUERADE for the
|
||||
│ bridge subnet (10.42.0.0/24 default)
|
||||
┌──────────────────────────┴─────────────────────────────────────────┐
|
||||
│ host netns — the host itself plus gateway / forge / matrix │
|
||||
│ host netns — the host itself plus gateway / forge │
|
||||
│ │
|
||||
│ nginx :80/:443 [hive-gateway] │
|
||||
│ dnsmasq 10.42.0.1:53 (DNS) + :67 (DHCP) [hive-gateway] │
|
||||
│ forgejo :3000 http, :2222 git-ssh [hive-forge] │
|
||||
│ tuwunel :8008 client API [hive-matrix] │
|
||||
│ hive-c0re dashboard 127.0.0.1:7000 (host service) │
|
||||
│ wg-hive :51820/udp — swarm mesh, when enabled (host iface) │
|
||||
│ │
|
||||
|
|
@ -46,7 +45,7 @@ that touches it.
|
|||
| ------------------------- | ----------------------- | -------------- | ----------------------------------------------------------------------------------------------- |
|
||||
| gateway (host) | host | host addresses | nginx `:80`/`:443` (every vhost); dnsmasq `bridgeIp:53` + DHCP `:67` on the bridge |
|
||||
| `hive-forge` | host (shared) | host addresses | forgejo `:3000` http, `:2222` git-ssh; fronted by the `forge.<swarm-domain>` vhost |
|
||||
| `hive-matrix` | host (shared) | host addresses | tuwunel `:8008` (+ optional federation port); fronted by the matrix vhost |
|
||||
| `hive-matrix` | private, veth on bridge | DHCP pool | tuwunel on UDS `/run/hive-matrix/tuwunel.sock` → the matrix vhost; out to store, SSO, peers |
|
||||
| `swarm-ci-runner` | private, veth on bridge | DHCP pool | outbound only (runner → forge); no inbound surface |
|
||||
| `swarm-review-classifier` | private, veth on bridge | DHCP pool | outbound only (classifier → queue, secret store, hive collector); no inbound surface |
|
||||
| `h-<agent>` | private, veth on bridge | DHCP pool | web UI via UDS `/run/hive-agent/<name>` → nginx sub-path; in-container UI port hashed 8100–8999 |
|
||||
|
|
@ -64,7 +63,7 @@ The flows, end to end:
|
|||
- **HTTP** — `forge.` and `chat.` (under `swarm.domain`) plus the hive's
|
||||
own dashboard name resolve to the bridge IP, land on nginx
|
||||
`:80`/`:443`, and proxy to forgejo
|
||||
`:3000`, tuwunel `:8008`, hive-c0re `127.0.0.1:7000`, or a per-agent
|
||||
`:3000`, tuwunel's unix socket, hive-c0re `127.0.0.1:7000`, or a per-agent
|
||||
UI unix socket.
|
||||
- **Internet egress** — agent default route points at the bridge IP;
|
||||
the host forwards + masquerades out its uplink.
|
||||
|
|
|
|||
Loading…
Reference in a new issue