hive-gateway: keep matrix discovery on the hive domain of a homeserver host without a hive
The hive-domain vhost renders where a hive or the homeserver runs. Its matrix .well-known JSON and /matrix/ redirect are served wherever the homeserver runs; the dashboard, swagger UI, agent routes and the agents.conf include only where a hive runs, since they proxy to hive-c0re. The gateway doc's vhost map says which rows exist without a hive, and the trust-bundle text describes ca.pem by what it is rather than as retired. Refs #4588
This commit is contained in:
atlas
mara
parent
60268b70b0
commit
3e1ac4e678
5 changed files with 91 additions and 51 deletions
|
|
@ -21,14 +21,14 @@ You rarely switch it on yourself. `gateway.enable` defaults to off, and every mo
|
|||
| `chat.<swarm>/config.json` | inline JSON (FluffyChat boot config) | `hive-matrix.nix`, `swarm.matrix.gatewayHost != null` |
|
||||
| `grafana.<swarm>`, `metrics.<swarm>`, `logs.<swarm>`, `otel.<swarm>`, `bao.<swarm>` | the matching swarm service | that service's module → [`swarm/services.md`](../swarm/services.md) |
|
||||
|
||||
**The hive's own vhost**, named for the hive domain:
|
||||
**The hive domain's own vhost**, named for the hive domain. It renders where a hive runs (`deploy.hive-controller.enable`) or the homeserver does (`deploy.matrix.enable`); on a host with neither there is no such vhost. On a host without a hive only the two matrix rows exist:
|
||||
|
||||
| URL | upstream | when |
|
||||
| --- | --- | --- |
|
||||
| `<hive>/` | dashboard dist (static, from `servedFrontend`) | always |
|
||||
| `<hive>/api/`, `/health/` | hive-c0re (`7000`) | always |
|
||||
| `<hive>/api/docs/` | themed Swagger UI dist (static) | always |
|
||||
| `<hive>/agent/<name>/` | per-agent harness over its unix socket | `agents.conf` (runtime-generated) |
|
||||
| `<hive>/` | dashboard dist (static, from `servedFrontend`) | `deploy.hive-controller.enable` |
|
||||
| `<hive>/api/`, `/health/` | hive-c0re (`7000`) | `deploy.hive-controller.enable` |
|
||||
| `<hive>/api/docs/` | themed Swagger UI dist (static) | `deploy.hive-controller.enable` |
|
||||
| `<hive>/agent/<name>/` | per-agent harness over its unix socket | `deploy.hive-controller.enable`; `agents.conf` (runtime-generated) |
|
||||
| `<hive>/.well-known/matrix/{client,server}` | inline JSON | `deploy.matrix.enable` |
|
||||
| `<hive>/matrix/` | 301 → `chat.<swarm>/` | `deploy.matrix.enable` and `gatewayHost` set |
|
||||
|
||||
|
|
@ -80,7 +80,7 @@ The issuer is the **secret store's `pki` mount**, not a bare self-signed leaf. `
|
|||
|
||||
**Why an issuer, not a bare leaf**: a bare self-signed leaf is its own trust anchor, so every regeneration is a new anchor every consumer must re-trust, and an agent's build sets its trust store once, before any leaf exists. Agents and federation peers trust the store's root once; leaf rotation never breaks them.
|
||||
|
||||
**What consumers trust**: `trust-bundle.pem` in the same state dir. It carries the store's `pki` root, plus the retired hive CA's public `ca.pem` and the swarm root where they exist ([`swarm/ca.md`](../swarm/ca.md#the-trust-bundle)). nginx serves the leaf alone, since its issuer is a self-signed root. Agents (via `security.pki.certificateFiles`), the CI and forge containers and federating peers all read the bundle.
|
||||
**What consumers trust**: `trust-bundle.pem` in the same state dir. It carries the store's `pki` root, plus a hive CA's public `ca.pem` and the swarm root where they exist ([`swarm/ca.md`](../swarm/ca.md#the-trust-bundle)). nginx serves the leaf alone, since its issuer is a self-signed root. Agents (via `security.pki.certificateFiles`), the CI and forge containers and federating peers all read the bundle.
|
||||
|
||||
**Why on by default**: matrix-dart-sdk (FluffyChat's SDK) fetches `https://<host>/.well-known/matrix/client` and never falls back to plain http, so without TLS the browser client can't bootstrap.
|
||||
|
||||
|
|
|
|||
|
|
@ -49,8 +49,8 @@ identity a swarm service uses.
|
|||
`hive-tls-resign.timer` once a day. It asks the store for a new leaf when
|
||||
the current one is missing, past half the role's 720-hour lifetime, missing
|
||||
a configured name, doesn't match the key beside it, or doesn't chain to the
|
||||
store's root (`swarm-services-root.pem`). That last check is what replaces a
|
||||
leaf the retired hive CA signed, on the first run that reaches the store.
|
||||
store's root (`swarm-services-root.pem`). The first run that reaches the
|
||||
store replaces a leaf that doesn't chain to that root, whoever signed it.
|
||||
|
||||
**The gateway doesn't wait for the store.** Nothing orders the gateway after
|
||||
`hive-tls-resign.service`, so a hive holding a valid leaf starts its gateway
|
||||
|
|
@ -65,8 +65,9 @@ self-signed root a consumer already holds in its trust bundle.
|
|||
|
||||
**A gateway host that runs no hive** (`deploy.hive-controller.enable` off,
|
||||
such as a dedicated swarm-services or secret-store host) has no hive leaf, no
|
||||
`hive-tls-resign.service`, no hive identity and no hive-domain vhost. Its
|
||||
gateway serves the swarm-services leaf as its default certificate too. Where it
|
||||
`hive-tls-resign.service`, no hive identity and no dashboard or agent routes;
|
||||
its hive-domain vhost exists only to serve matrix discovery where the
|
||||
homeserver runs there. Its gateway serves the swarm-services leaf as its default certificate too. Where it
|
||||
fronts other swarm names it waits for `swarm-services-cert.service` at start.
|
||||
Where the store's UI is its only swarm name it starts on a throwaway
|
||||
self-signed certificate and switches to the services leaf once the store
|
||||
|
|
@ -79,16 +80,16 @@ the file every consumer anchors on. It holds whichever of these exist:
|
|||
|
||||
- `swarm-services-root.pem`, the store's `pki` root, which the gateway leaf
|
||||
and the swarm-services leaf both chain to.
|
||||
- `ca.pem` and `ca-previous.pem`, the retired hive CA's public certificates,
|
||||
so a leaf it signed keeps verifying until it's replaced. Removing them is a
|
||||
deliberate later step.
|
||||
- `ca.pem` and `ca-previous.pem`: a hive CA's public certificates. Nothing
|
||||
issues under that CA; they stay in the bundle so a leaf it signed verifies
|
||||
until `hive-tls-resign` replaces it. Removing them is a deliberate later
|
||||
step.
|
||||
- the swarm root, `<swarm.ca.stateDir>/root.pem`, on the host that generates
|
||||
it.
|
||||
|
||||
No hive generates a CA or signs with a CA key. A
|
||||
`<deploy.hive-controller.tls.stateDir>/ca-key.pem` an earlier generation left
|
||||
stays on disk, unread: removing it comes with dropping `ca.pem` from the
|
||||
bundle.
|
||||
`<deploy.hive-controller.tls.stateDir>/ca-key.pem` stays on disk, unread:
|
||||
removing it comes with dropping `ca.pem` from the bundle.
|
||||
|
||||
⚠️ **A consumer reads its trust when it starts.** An agent or service
|
||||
container that last started before the store's root reached the bundle
|
||||
|
|
|
|||
|
|
@ -248,31 +248,34 @@ in
|
|||
'';
|
||||
};
|
||||
}
|
||||
// lib.optionalAttrs runsHive {
|
||||
# The hive's own surface, reachable by NAME. Named rather than left
|
||||
# to `_`, so the catch-all can reject everything else. Only where a
|
||||
# hive runs: it proxies to hive-c0re, which runs only with the hive
|
||||
# controller.
|
||||
// lib.optionalAttrs (runsHive || matrixDeployCfg.enable) {
|
||||
# The hive domain's own vhost, reachable by NAME, so the catch-all can
|
||||
# reject everything else. It exists where it has something to serve:
|
||||
# matrix discovery and the `/matrix/` redirect where the homeserver runs
|
||||
# here, and the dashboard, agent routes and swagger UI where a hive runs
|
||||
# (they proxy to hive-c0re, which runs only with the hive controller).
|
||||
${hyperhiveDomain} = (vhostTlsFor hyperhiveDomain) // {
|
||||
listen = vhostListen;
|
||||
locations =
|
||||
matrixRedirectLocations
|
||||
// wellKnownLocations
|
||||
// agentLocations
|
||||
// dashboardProxyLocation
|
||||
// swaggerUiLocations
|
||||
// lib.optionalAttrs cfg.auth.enable {
|
||||
# Internal-only target for the 401 error_page above.
|
||||
# `internal` prevents direct client access; `alias` serves
|
||||
# the pre-built HTML from the Nix store.
|
||||
"= /__hive_auth_unauthorized" = {
|
||||
extraConfig = ''
|
||||
internal;
|
||||
alias ${errorPages.unauthorized};
|
||||
default_type text/html;
|
||||
'';
|
||||
};
|
||||
};
|
||||
// lib.optionalAttrs runsHive (
|
||||
agentLocations
|
||||
// dashboardProxyLocation
|
||||
// swaggerUiLocations
|
||||
// lib.optionalAttrs cfg.auth.enable {
|
||||
# Internal-only target for the dashboard's 401 error_page.
|
||||
# `internal` prevents direct client access; `alias` serves
|
||||
# the pre-built HTML from the Nix store.
|
||||
"= /__hive_auth_unauthorized" = {
|
||||
extraConfig = ''
|
||||
internal;
|
||||
alias ${errorPages.unauthorized};
|
||||
default_type text/html;
|
||||
'';
|
||||
};
|
||||
}
|
||||
);
|
||||
# Per-agent location blocks, generated at runtime by
|
||||
# hive-c0re and written to /var/lib/hive-gateway/conf/agents.conf
|
||||
# on the host — the same machine nginx runs on. nginx parses
|
||||
|
|
@ -281,9 +284,11 @@ in
|
|||
# agents without a nixos-rebuild. nginx's longest-prefix-
|
||||
# match rule ensures `/agent/<name>/` from this file beats
|
||||
# the `/agent/` catch-all above.
|
||||
extraConfig = securityHeaders + ''
|
||||
include /var/lib/hive-gateway/conf/agents.conf;
|
||||
'';
|
||||
extraConfig =
|
||||
securityHeaders
|
||||
+ lib.optionalString runsHive ''
|
||||
include /var/lib/hive-gateway/conf/agents.conf;
|
||||
'';
|
||||
};
|
||||
};
|
||||
}
|
||||
|
|
|
|||
|
|
@ -251,9 +251,8 @@ in
|
|||
# generates or signs with a CA key. See
|
||||
# `docs/networking/gateway.md` ("Self-signed TLS") and docs/swarm/ca.md.
|
||||
#
|
||||
# The trust bundle still carries the retired hive CA's public `ca.pem`
|
||||
# where one exists, so a leaf it signed keeps verifying until it is
|
||||
# replaced.
|
||||
# The trust bundle also carries a hive CA's public `ca.pem` where one
|
||||
# exists, so a leaf it signed verifies until it is replaced.
|
||||
|
||||
options.services.hyperhive.deploy.hive-controller.tls = {
|
||||
stateDir = lib.mkOption {
|
||||
|
|
@ -264,9 +263,9 @@ in
|
|||
self-signed gateway mode: `gateway.pem` / `gateway-key.pem` (the
|
||||
leaf the store issues and nginx serves) and `trust-bundle.pem` (the
|
||||
anchors agents and federation peers trust). `ca.pem` and
|
||||
`ca-previous.pem`, where present, are the retired hive CA's public
|
||||
certificates and stay in the bundle. A `ca-key.pem` an earlier
|
||||
generation left is neither read nor removed.
|
||||
`ca-previous.pem`, where present, are a hive CA's public
|
||||
certificates and stay in the bundle. A `ca-key.pem` here is neither
|
||||
read nor removed.
|
||||
'';
|
||||
};
|
||||
|
||||
|
|
@ -421,8 +420,8 @@ in
|
|||
#
|
||||
# `swarm-services-root.pem` is the store's `pki` root, which both
|
||||
# the gateway leaf and the swarm-services leaf chain to. `ca.pem`
|
||||
# and `ca-previous.pem` are the retired hive CA's public
|
||||
# certificates, kept so a leaf they signed verifies until replaced.
|
||||
# and `ca-previous.pem` are a hive CA's public certificates, kept
|
||||
# so a leaf they signed verifies until replaced.
|
||||
bundle="$d/trust-bundle.pem"
|
||||
anchors=()
|
||||
if [ -s "$ca" ]; then anchors+=("$ca"); fi
|
||||
|
|
@ -528,8 +527,8 @@ in
|
|||
{ [ -s "$leaf" ] && [ -s "$leafk" ]; } || reissue=1
|
||||
openssl x509 -in "$leaf" -noout -checkend "$renewat" >/dev/null 2>&1 || reissue=1
|
||||
leavesCoverNames || reissue=1
|
||||
# A leaf the retired hive CA signed, or one under a store root since
|
||||
# replaced, fails this and is re-issued.
|
||||
# A leaf that does not chain to the store's current root, whoever
|
||||
# signed it, fails this and is re-issued.
|
||||
{ [ -s "$svcroot" ] && openssl verify -CAfile "$svcroot" "$leaf" >/dev/null 2>&1; } || reissue=1
|
||||
# The leaf and the key are two renames, so an interrupted rotation can
|
||||
# leave a certificate beside a key it does not belong to.
|
||||
|
|
|
|||
|
|
@ -45,6 +45,13 @@ let
|
|||
deploy.bao.enable = true;
|
||||
};
|
||||
|
||||
# The homeserver on a host without a hive: its hive-domain vhost serves
|
||||
# matrix discovery and nothing of the hive's.
|
||||
matrixNoHive = hive {
|
||||
deploy.hive-controller.enable = false;
|
||||
deploy.matrix.enable = true;
|
||||
};
|
||||
|
||||
# The same switch with every service it owns placed elsewhere, so the only
|
||||
# thing that can enable the gateway/resolver/bridge here is that toggle's
|
||||
# own module — each service module asserts them from its own `enable`.
|
||||
|
|
@ -232,7 +239,25 @@ let
|
|||
in
|
||||
!(lib.any (t: lib.any (r: lib.hasInfix r t) hiveLeafRefs) texts)
|
||||
&& m.services.hyperhive.deploy.hive-controller.tls.baoHiveIdentityCertFile == null
|
||||
&& !(m.services.nginx.virtualHosts ? "h1.t.local");
|
||||
&& noHiveSurface m;
|
||||
# The hive-domain vhost, where it renders at all, carries nothing
|
||||
# that proxies to hive-c0re.
|
||||
noHiveSurface =
|
||||
m:
|
||||
let
|
||||
vh = m.services.nginx.virtualHosts;
|
||||
v = vh."h1.t.local";
|
||||
in
|
||||
!(vh ? "h1.t.local")
|
||||
|| (
|
||||
!(lib.any (l: v.locations ? ${l}) [
|
||||
"/"
|
||||
"/api/"
|
||||
"/agent/"
|
||||
"/api/docs/"
|
||||
])
|
||||
&& !(lib.hasInfix "agents.conf" (v.extraConfig or ""))
|
||||
);
|
||||
noHive =
|
||||
m:
|
||||
let
|
||||
|
|
@ -277,13 +302,23 @@ let
|
|||
store.services.hyperhive.swarm.localServiceDomains
|
||||
== [ store.services.hyperhive.swarm.bao.ui.domain ]
|
||||
&& !(waitsForLeaf store)
|
||||
# The homeserver without a hive: the hive-domain vhost serves matrix
|
||||
# discovery and the redirect, and no hive surface.
|
||||
&& noHive matrixNoHive
|
||||
&& (
|
||||
let
|
||||
l = matrixNoHive.services.nginx.virtualHosts."h1.t.local".locations;
|
||||
in
|
||||
l ? "= /.well-known/matrix/client" && l ? "= /.well-known/matrix/server"
|
||||
)
|
||||
&& !(store.services.nginx.virtualHosts ? "h1.t.local")
|
||||
&& lib.hasInfix "want_svc=1" store.systemd.services.swarm-services-cert.script
|
||||
# Control: a hive host with the same services keeps the hive leaf,
|
||||
# its unit and identity, and the hive-domain vhost.
|
||||
&& hs ? hive-tls-resign
|
||||
&& lib.hasInfix "install -m 0644 /var/lib/hive-tls/gateway.pem /var/lib/hive-gateway/tls/cert.pem" hs.hive-gateway-self-signed-cert.script
|
||||
&& swarmServicesHere.services.hyperhive.deploy.hive-controller.tls.baoHiveIdentityCertFile != null
|
||||
&& swarmServicesHere.services.nginx.virtualHosts ? "h1.t.local"
|
||||
&& swarmServicesHere.services.nginx.virtualHosts."h1.t.local".locations ? "/api/"
|
||||
&& !(noHiveLeafAnywhere swarmServicesHere);
|
||||
}
|
||||
{
|
||||
|
|
|
|||
Loading…
Reference in a new issue