hive-gateway: no hive-domain vhost on a host without a hive
The hive-domain vhost renders only where a hive runs. On a host without one the only certificate is the swarm-services leaf, which does not name the hive domain, so a homeserver host without a hive serves matrix on the homeserver's own name only; apex matrix discovery is #4878. Comments and test names describe the code's current behaviour. Refs #4588
This commit is contained in:
parent
3e1ac4e678
commit
01ec29da48
6 changed files with 68 additions and 76 deletions
|
|
@ -21,14 +21,14 @@ You rarely switch it on yourself. `gateway.enable` defaults to off, and every mo
|
||||||
| `chat.<swarm>/config.json` | inline JSON (FluffyChat boot config) | `hive-matrix.nix`, `swarm.matrix.gatewayHost != null` |
|
| `chat.<swarm>/config.json` | inline JSON (FluffyChat boot config) | `hive-matrix.nix`, `swarm.matrix.gatewayHost != null` |
|
||||||
| `grafana.<swarm>`, `metrics.<swarm>`, `logs.<swarm>`, `otel.<swarm>`, `bao.<swarm>` | the matching swarm service | that service's module → [`swarm/services.md`](../swarm/services.md) |
|
| `grafana.<swarm>`, `metrics.<swarm>`, `logs.<swarm>`, `otel.<swarm>`, `bao.<swarm>` | the matching swarm service | that service's module → [`swarm/services.md`](../swarm/services.md) |
|
||||||
|
|
||||||
**The hive domain's own vhost**, named for the hive domain. It renders where a hive runs (`deploy.hive-controller.enable`) or the homeserver does (`deploy.matrix.enable`); on a host with neither there is no such vhost. On a host without a hive only the two matrix rows exist:
|
**The hive's own vhost**, named for the hive domain. It exists only on a host that runs a hive (`deploy.hive-controller.enable`); a host without a hive has none of these rows:
|
||||||
|
|
||||||
| URL | upstream | when |
|
| URL | upstream | when |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| `<hive>/` | dashboard dist (static, from `servedFrontend`) | `deploy.hive-controller.enable` |
|
| `<hive>/` | dashboard dist (static, from `servedFrontend`) | always |
|
||||||
| `<hive>/api/`, `/health/` | hive-c0re (`7000`) | `deploy.hive-controller.enable` |
|
| `<hive>/api/`, `/health/` | hive-c0re (`7000`) | always |
|
||||||
| `<hive>/api/docs/` | themed Swagger UI dist (static) | `deploy.hive-controller.enable` |
|
| `<hive>/api/docs/` | themed Swagger UI dist (static) | always |
|
||||||
| `<hive>/agent/<name>/` | per-agent harness over its unix socket | `deploy.hive-controller.enable`; `agents.conf` (runtime-generated) |
|
| `<hive>/agent/<name>/` | per-agent harness over its unix socket | `agents.conf` (runtime-generated) |
|
||||||
| `<hive>/.well-known/matrix/{client,server}` | inline JSON | `deploy.matrix.enable` |
|
| `<hive>/.well-known/matrix/{client,server}` | inline JSON | `deploy.matrix.enable` |
|
||||||
| `<hive>/matrix/` | 301 → `chat.<swarm>/` | `deploy.matrix.enable` and `gatewayHost` set |
|
| `<hive>/matrix/` | 301 → `chat.<swarm>/` | `deploy.matrix.enable` and `gatewayHost` set |
|
||||||
|
|
||||||
|
|
@ -86,7 +86,7 @@ The issuer is the **secret store's `pki` mount**, not a bare self-signed leaf. `
|
||||||
|
|
||||||
**Cert shape**: the leaf's CN is the bare hive domain; its SANs are `<hive>` and `*.<hive>`. The hive's role allows nothing outside `<hive>`, so a swarm-service name can't go on this leaf. Those names get the swarm-services leaf instead ([`swarm/ca.md`](../swarm/ca.md)).
|
**Cert shape**: the leaf's CN is the bare hive domain; its SANs are `<hive>` and `*.<hive>`. The hive's role allows nothing outside `<hive>`, so a swarm-service name can't go on this leaf. Those names get the swarm-services leaf instead ([`swarm/ca.md`](../swarm/ca.md)).
|
||||||
|
|
||||||
**Rotation**: `hive-tls-resign.service` runs at boot, on every deploy and once a day, and asks the store for a new leaf when the current one is missing, past half its 720-hour lifetime, no longer covers the configured names, doesn't match its key, or doesn't chain to the store's root. A leaf already on disk lets the gateway start while the store can't answer; a failed renewal leaves it in place. To force a rotation, delete `gateway.pem` under the state dir and restart the unit; it re-imports the leaf and reloads `nginx` itself.
|
**Rotation**: `hive-tls-resign.service` runs at boot, on every deploy and once a day, and asks the store for a new leaf when the current one is missing, past half its 720-hour lifetime, doesn't cover the configured names, doesn't match its key, or doesn't chain to the store's root. A leaf already on disk lets the gateway start while the store can't answer; a failed renewal leaves it in place. To force a rotation, delete `gateway.pem` under the state dir and restart the unit; it re-imports the leaf and reloads `nginx` itself.
|
||||||
|
|
||||||
**Cert prompts**: browsers warn once per host until you add the hive's `trust-bundle.pem` (the anchor, not the leaf) to the browser or OS trust store. A container that needs to trust it for its own outbound TLS gets the bundle a different way, bind-mounted read-only at `/run/hive-ca/trust-bundle.pem` (`nix/host-modules/lib/hive-ca-trust.nix`).
|
**Cert prompts**: browsers warn once per host until you add the hive's `trust-bundle.pem` (the anchor, not the leaf) to the browser or OS trust store. A container that needs to trust it for its own outbound TLS gets the bundle a different way, bind-mounted read-only at `/run/hive-ca/trust-bundle.pem` (`nix/host-modules/lib/hive-ca-trust.nix`).
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -65,9 +65,8 @@ self-signed root a consumer already holds in its trust bundle.
|
||||||
|
|
||||||
**A gateway host that runs no hive** (`deploy.hive-controller.enable` off,
|
**A gateway host that runs no hive** (`deploy.hive-controller.enable` off,
|
||||||
such as a dedicated swarm-services or secret-store host) has no hive leaf, no
|
such as a dedicated swarm-services or secret-store host) has no hive leaf, no
|
||||||
`hive-tls-resign.service`, no hive identity and no dashboard or agent routes;
|
`hive-tls-resign.service`, no hive identity and no hive-domain vhost. Its
|
||||||
its hive-domain vhost exists only to serve matrix discovery where the
|
gateway serves the swarm-services leaf as its default certificate too. Where it
|
||||||
homeserver runs there. Its gateway serves the swarm-services leaf as its default certificate too. Where it
|
|
||||||
fronts other swarm names it waits for `swarm-services-cert.service` at start.
|
fronts other swarm names it waits for `swarm-services-cert.service` at start.
|
||||||
Where the store's UI is its only swarm name it starts on a throwaway
|
Where the store's UI is its only swarm name it starts on a throwaway
|
||||||
self-signed certificate and switches to the services leaf once the store
|
self-signed certificate and switches to the services leaf once the store
|
||||||
|
|
|
||||||
|
|
@ -248,19 +248,18 @@ in
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
// lib.optionalAttrs (runsHive || matrixDeployCfg.enable) {
|
// lib.optionalAttrs runsHive {
|
||||||
# The hive domain's own vhost, reachable by NAME, so the catch-all can
|
# The hive domain's own vhost, reachable by NAME, so the catch-all can
|
||||||
# reject everything else. It exists where it has something to serve:
|
# reject everything else. Only where a hive runs: the dashboard, agent
|
||||||
# matrix discovery and the `/matrix/` redirect where the homeserver runs
|
# routes and swagger UI proxy to hive-c0re, which runs only with the hive
|
||||||
# here, and the dashboard, agent routes and swagger UI where a hive runs
|
# controller, and the certificate naming the hive domain is the hive
|
||||||
# (they proxy to hive-c0re, which runs only with the hive controller).
|
# leaf, which exists only there.
|
||||||
${hyperhiveDomain} = (vhostTlsFor hyperhiveDomain) // {
|
${hyperhiveDomain} = (vhostTlsFor hyperhiveDomain) // {
|
||||||
listen = vhostListen;
|
listen = vhostListen;
|
||||||
locations =
|
locations =
|
||||||
matrixRedirectLocations
|
matrixRedirectLocations
|
||||||
// wellKnownLocations
|
// wellKnownLocations
|
||||||
// lib.optionalAttrs runsHive (
|
// agentLocations
|
||||||
agentLocations
|
|
||||||
// dashboardProxyLocation
|
// dashboardProxyLocation
|
||||||
// swaggerUiLocations
|
// swaggerUiLocations
|
||||||
// lib.optionalAttrs cfg.auth.enable {
|
// lib.optionalAttrs cfg.auth.enable {
|
||||||
|
|
@ -274,8 +273,7 @@ in
|
||||||
default_type text/html;
|
default_type text/html;
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
}
|
};
|
||||||
);
|
|
||||||
# Per-agent location blocks, generated at runtime by
|
# Per-agent location blocks, generated at runtime by
|
||||||
# hive-c0re and written to /var/lib/hive-gateway/conf/agents.conf
|
# hive-c0re and written to /var/lib/hive-gateway/conf/agents.conf
|
||||||
# on the host — the same machine nginx runs on. nginx parses
|
# on the host — the same machine nginx runs on. nginx parses
|
||||||
|
|
@ -284,9 +282,7 @@ in
|
||||||
# agents without a nixos-rebuild. nginx's longest-prefix-
|
# agents without a nixos-rebuild. nginx's longest-prefix-
|
||||||
# match rule ensures `/agent/<name>/` from this file beats
|
# match rule ensures `/agent/<name>/` from this file beats
|
||||||
# the `/agent/` catch-all above.
|
# the `/agent/` catch-all above.
|
||||||
extraConfig =
|
extraConfig = securityHeaders + ''
|
||||||
securityHeaders
|
|
||||||
+ lib.optionalString runsHive ''
|
|
||||||
include /var/lib/hive-gateway/conf/agents.conf;
|
include /var/lib/hive-gateway/conf/agents.conf;
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|
|
||||||
|
|
@ -594,15 +594,10 @@ in
|
||||||
|
|
||||||
# The swarm-services leaf, issued by the secret store's `pki` mount.
|
# The swarm-services leaf, issued by the secret store's `pki` mount.
|
||||||
#
|
#
|
||||||
# ⚠️ This unit replaces a chain, not a command. What used to happen
|
# The leaf chains straight to the mount's root, with no intermediate:
|
||||||
# was two openssl hops on whichever host held the swarm root key: a
|
# the store's `pki/roles/swarm-services` narrows the names with an
|
||||||
# name-constrained `swarm-services` sub-CA (../swarm-ca.nix), then a
|
# `allowed_domains` list enforced at issue time, so there is no x509
|
||||||
# leaf signed under it here. Both are gone. The store's
|
# `nameConstraints` for an intermediate to carry.
|
||||||
# `pki/roles/swarm-services` carries the same narrowing as an
|
|
||||||
# `allowed_domains` list enforced at issue time, so the intermediate
|
|
||||||
# that used to encode it in x509 `nameConstraints` has nothing left
|
|
||||||
# to express — and with the root inside the mount there is nothing
|
|
||||||
# for it to be an intermediate OF.
|
|
||||||
#
|
#
|
||||||
# The names it may carry are not this unit's to assert: the role
|
# The names it may carry are not this unit's to assert: the role
|
||||||
# refuses anything outside `allowed_domains`, read from the
|
# refuses anything outside `allowed_domains`, read from the
|
||||||
|
|
@ -613,14 +608,10 @@ in
|
||||||
systemd.services.swarm-services-cert = {
|
systemd.services.swarm-services-cert = {
|
||||||
description = "Issue the swarm-services TLS leaf from the secret store's PKI";
|
description = "Issue the swarm-services TLS leaf from the secret store's PKI";
|
||||||
wantedBy = [ "multi-user.target" ];
|
wantedBy = [ "multi-user.target" ];
|
||||||
# ⚠️ The ordering the old shape got by ACCIDENT, written down.
|
# ⚠️ Where the gateway waits for this leaf, the store being up is a
|
||||||
# Nothing used to declare "no TLS until the store is up" — it held
|
# precondition of the gateway's certificate, and `requiredBy` is
|
||||||
# because `swarm-services-ca.service` happened to block the gateway.
|
# what makes a store that never comes up an outage that says so
|
||||||
# Now it is a real edge: the gateway's certificate comes out of the
|
# instead of a gateway serving the wrong name.
|
||||||
# store, so the store being up is a precondition rather than a
|
|
||||||
# coincidence, and `requiredBy` is what makes a store that never
|
|
||||||
# comes up an outage that says so instead of a gateway serving the
|
|
||||||
# wrong name.
|
|
||||||
#
|
#
|
||||||
# 🩸 NOT `Before=hive-tls-ca.service`, however much the trust bundle
|
# 🩸 NOT `Before=hive-tls-ca.service`, however much the trust bundle
|
||||||
# wants it. That edge closed a cycle — the store's container is
|
# wants it. That edge closed a cycle — the store's container is
|
||||||
|
|
|
||||||
|
|
@ -231,10 +231,10 @@ let
|
||||||
&& lib.hasInfix "root-key.pem" rootScript;
|
&& lib.hasInfix "root-key.pem" rootScript;
|
||||||
}
|
}
|
||||||
{
|
{
|
||||||
# Left for PR 3, beside dropping `ca.pem`, so a rollback finds the old
|
# Nothing reads or removes a `ca-key.pem`, so a rollback finds the
|
||||||
# key and does not mint a new anchor. The control is the bundle reading
|
# key and does not mint a new anchor. The control is the bundle
|
||||||
# `ca.pem` in the same script.
|
# reading `ca.pem` in the same script.
|
||||||
name = "a retired hive CA key is left in place, and its public certificate stays in the trust bundle";
|
name = "a hive CA key is neither read nor removed, and its public certificate stays in the trust bundle";
|
||||||
ok =
|
ok =
|
||||||
!(lib.hasInfix "ca-key.pem" caUnit.script)
|
!(lib.hasInfix "ca-key.pem" caUnit.script)
|
||||||
&& !(lib.hasInfix "ca-key.pem" leafUnit.script)
|
&& !(lib.hasInfix "ca-key.pem" leafUnit.script)
|
||||||
|
|
@ -281,8 +281,6 @@ let
|
||||||
&& !(lib.hasInfix "(unset)" ok');
|
&& !(lib.hasInfix "(unset)" ok');
|
||||||
}
|
}
|
||||||
{
|
{
|
||||||
# A leaf the retired hive CA signed is re-issued from the store on the
|
|
||||||
# first run that reaches it; one that renews in time never is.
|
|
||||||
# A cert from one issuance beside the key of another, left by an
|
# A cert from one issuance beside the key of another, left by an
|
||||||
# interrupted pair of renames, is re-issued rather than left for
|
# interrupted pair of renames, is re-issued rather than left for
|
||||||
# `nginx -t` to refuse. Checked on the decision side of the exit.
|
# `nginx -t` to refuse. Checked on the decision side of the exit.
|
||||||
|
|
|
||||||
|
|
@ -45,8 +45,7 @@ let
|
||||||
deploy.bao.enable = true;
|
deploy.bao.enable = true;
|
||||||
};
|
};
|
||||||
|
|
||||||
# The homeserver on a host without a hive: its hive-domain vhost serves
|
# The homeserver on a host without a hive.
|
||||||
# matrix discovery and nothing of the hive's.
|
|
||||||
matrixNoHive = hive {
|
matrixNoHive = hive {
|
||||||
deploy.hive-controller.enable = false;
|
deploy.hive-controller.enable = false;
|
||||||
deploy.matrix.enable = true;
|
deploy.matrix.enable = true;
|
||||||
|
|
@ -240,24 +239,27 @@ let
|
||||||
!(lib.any (t: lib.any (r: lib.hasInfix r t) hiveLeafRefs) texts)
|
!(lib.any (t: lib.any (r: lib.hasInfix r t) hiveLeafRefs) texts)
|
||||||
&& m.services.hyperhive.deploy.hive-controller.tls.baoHiveIdentityCertFile == null
|
&& m.services.hyperhive.deploy.hive-controller.tls.baoHiveIdentityCertFile == null
|
||||||
&& noHiveSurface m;
|
&& noHiveSurface m;
|
||||||
# The hive-domain vhost, where it renders at all, carries nothing
|
# No hive-domain vhost, and no vhost on the host carrying the
|
||||||
# that proxies to hive-c0re.
|
# hive's locations: c0re's `/health/`, the agent routes, swagger,
|
||||||
|
# or the hive domain's matrix discovery and `/matrix/` redirect.
|
||||||
noHiveSurface =
|
noHiveSurface =
|
||||||
m:
|
m:
|
||||||
let
|
let
|
||||||
vh = m.services.nginx.virtualHosts;
|
vh = m.services.nginx.virtualHosts;
|
||||||
v = vh."h1.t.local";
|
|
||||||
in
|
in
|
||||||
!(vh ? "h1.t.local")
|
!(vh ? "h1.t.local")
|
||||||
|| (
|
&& !(lib.any (
|
||||||
!(lib.any (l: v.locations ? ${l}) [
|
v:
|
||||||
"/"
|
lib.any (l: (v.locations or { }) ? ${l}) [
|
||||||
"/api/"
|
"/health/"
|
||||||
"/agent/"
|
"/agent/"
|
||||||
"/api/docs/"
|
"/api/docs/"
|
||||||
])
|
"/matrix/"
|
||||||
&& !(lib.hasInfix "agents.conf" (v.extraConfig or ""))
|
"= /.well-known/matrix/client"
|
||||||
);
|
"= /.well-known/matrix/server"
|
||||||
|
]
|
||||||
|
|| lib.hasInfix "agents.conf" (v.extraConfig or "")
|
||||||
|
) (lib.attrValues vh));
|
||||||
noHive =
|
noHive =
|
||||||
m:
|
m:
|
||||||
let
|
let
|
||||||
|
|
@ -302,14 +304,15 @@ let
|
||||||
store.services.hyperhive.swarm.localServiceDomains
|
store.services.hyperhive.swarm.localServiceDomains
|
||||||
== [ store.services.hyperhive.swarm.bao.ui.domain ]
|
== [ store.services.hyperhive.swarm.bao.ui.domain ]
|
||||||
&& !(waitsForLeaf store)
|
&& !(waitsForLeaf store)
|
||||||
# The homeserver without a hive: the hive-domain vhost serves matrix
|
# The homeserver without a hive: no hive-domain vhost at all, and the
|
||||||
# discovery and the redirect, and no hive surface.
|
# homeserver's own vhost still proxies `/_matrix/`.
|
||||||
&& noHive matrixNoHive
|
&& noHive matrixNoHive
|
||||||
&& (
|
&& (
|
||||||
let
|
let
|
||||||
l = matrixNoHive.services.nginx.virtualHosts."h1.t.local".locations;
|
gh = matrixNoHive.services.hyperhive.swarm.matrix.gatewayHost;
|
||||||
|
vh = matrixNoHive.services.nginx.virtualHosts;
|
||||||
in
|
in
|
||||||
l ? "= /.well-known/matrix/client" && l ? "= /.well-known/matrix/server"
|
gh != null && vh ? ${gh} && lib.any (lib.hasPrefix "/_matrix") (lib.attrNames vh.${gh}.locations)
|
||||||
)
|
)
|
||||||
&& !(store.services.nginx.virtualHosts ? "h1.t.local")
|
&& !(store.services.nginx.virtualHosts ? "h1.t.local")
|
||||||
&& lib.hasInfix "want_svc=1" store.systemd.services.swarm-services-cert.script
|
&& lib.hasInfix "want_svc=1" store.systemd.services.swarm-services-cert.script
|
||||||
|
|
@ -318,7 +321,12 @@ let
|
||||||
&& hs ? hive-tls-resign
|
&& hs ? hive-tls-resign
|
||||||
&& lib.hasInfix "install -m 0644 /var/lib/hive-tls/gateway.pem /var/lib/hive-gateway/tls/cert.pem" hs.hive-gateway-self-signed-cert.script
|
&& lib.hasInfix "install -m 0644 /var/lib/hive-tls/gateway.pem /var/lib/hive-gateway/tls/cert.pem" hs.hive-gateway-self-signed-cert.script
|
||||||
&& swarmServicesHere.services.hyperhive.deploy.hive-controller.tls.baoHiveIdentityCertFile != null
|
&& swarmServicesHere.services.hyperhive.deploy.hive-controller.tls.baoHiveIdentityCertFile != null
|
||||||
&& swarmServicesHere.services.nginx.virtualHosts."h1.t.local".locations ? "/api/"
|
&& (
|
||||||
|
let
|
||||||
|
l = swarmServicesHere.services.nginx.virtualHosts."h1.t.local".locations;
|
||||||
|
in
|
||||||
|
l ? "/api/" && l ? "/health/" && l ? "= /.well-known/matrix/client"
|
||||||
|
)
|
||||||
&& !(noHiveLeafAnywhere swarmServicesHere);
|
&& !(noHiveLeafAnywhere swarmServicesHere);
|
||||||
}
|
}
|
||||||
{
|
{
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue