bao: serve the browser UI to admins via a loopback-only listener
openbao gains a second listener, `ui`, on 127.0.0.1:<deploy.bao.uiPort> (default 8204) with TLS off and no client-certificate requirement, and `ui = true`. The existing listeners are unchanged. An nginx inside the store's container, on 127.0.0.1:<deploy.bao.uiProxyPort> (default 8206), forwards only /ui/ and /v1/ to it, redirects / to /ui/, answers 403 on sys/unseal, sys/seal, sys/step-down, sys/rekey* and sys/generate-root*, and 404 on everything else. The gateway on the store's host serves `swarm.bao.ui.domain` (default bao-ui.<swarm>) behind the authelia auth_request subrequest, proxying to that nginx; the name joins serviceDomains and localNames like every other gateway-published swarm service. authelia gets an access_control rule restricting that name to group:admins, rendered wherever authelia runs, since the default policy admits any session. Trade-off, ruled by the operator on the parent issue: the UI listener asks for no client certificate, so on that door a bao token alone is the credential. Three comments and a doc line claimed every API listener demands a client certificate; they now except the loopback UI listener. The module-eval case counting declared listeners excludes `ui` by name, as it already did `metrics`. On a self-signed gateway, the UI's name is a swarm service name, so its host requests the services leaf from the store. `swarm-services-cert` sits Before= and RequiredBy= the gateway's cert import, which nginx Requires=. On a host whose only swarm name is the UI, that would hold nginx, and with it the stream passthrough every reader dials, on a login to a store that may be sealed. hive-tls drops those two edges exactly when the UI is the only local swarm name: nginx starts on the existing hive-leaf fallback, and the script's existing re-import reloads nginx once the leaf issues. Every other host keeps both edges.
This commit is contained in:
parent
a86379eac5
commit
3898ca33c7
10 changed files with 376 additions and 29 deletions
|
|
@ -118,8 +118,8 @@ which accepts the leaf `/var/lib/swarm-bao-pki/granter.pem`. Every
|
||||||
`swarm-bao-*-policy` unit then logs in with that leaf. The controller's unit
|
`swarm-bao-*-policy` unit then logs in with that leaf. The controller's unit
|
||||||
mounts the KV and pki engines and writes the `swarm-controller` role, and each
|
mounts the KV and pki engines and writes the `swarm-controller` role, and each
|
||||||
sibling unit writes its own principal's policy and role. Every one runs on the
|
sibling unit writes its own principal's policy and role. Every one runs on the
|
||||||
host, because every API listener demands a client certificate and the host is
|
host, because every API listener but the loopback UI one demands a client
|
||||||
the side that has one.
|
certificate and the host is the side that has one.
|
||||||
|
|
||||||
**Confirm with `systemctl status swarm-bao-granter-role`**, which should log
|
**Confirm with `systemctl status swarm-bao-granter-role`**, which should log
|
||||||
`Uploaded policy: bao-granter` and `Data written to: auth/cert/certs/bao-granter`.
|
`Uploaded policy: bao-granter` and `Data written to: auth/cert/certs/bao-granter`.
|
||||||
|
|
|
||||||
|
|
@ -67,7 +67,9 @@ leaf it signed and folded into `trust-bundle.pem`. On the host running
|
||||||
the store it's also at
|
the store it's also at
|
||||||
`/var/lib/swarm-bao-services-pki/services-root.pem`. That's the file to
|
`/var/lib/swarm-bao-services-pki/services-root.pem`. That's the file to
|
||||||
hand a browser, and reading it needs no store login — which matters,
|
hand a browser, and reading it needs no store login — which matters,
|
||||||
because every store listener demands a client certificate.
|
because every store listener but the loopback UI one demands a client
|
||||||
|
certificate, and that one is gated behind authelia to the `admins`
|
||||||
|
group, not open to an anonymous browser fetching a trust anchor.
|
||||||
|
|
||||||
**The granting unit generates the root once, and never again.** It asks
|
**The granting unit generates the root once, and never again.** It asks
|
||||||
the mount whether it already has an issuer (`bao list pki/issuers`)
|
the mount whether it already has an issuer (`bao list pki/issuers`)
|
||||||
|
|
|
||||||
|
|
@ -428,6 +428,25 @@ agent CA. The listener reads both from `listener-client-ca.pem`, which no
|
||||||
cert-auth role pins. The passthrough carries
|
cert-auth role pins. The passthrough carries
|
||||||
whichever certificate the reader presents, unchanged.
|
whichever certificate the reader presents, unchanged.
|
||||||
|
|
||||||
|
## The browser UI
|
||||||
|
|
||||||
|
OpenBao's built-in web UI is at `https://bao-ui.<swarm domain>/`
|
||||||
|
(`swarm.bao.ui.domain`), for members of authelia's `admins` group only. Log in
|
||||||
|
with a bao token. The gateway vhost checks the session with authelia and
|
||||||
|
proxies to an nginx inside the store's container on
|
||||||
|
`127.0.0.1:<deploy.bao.uiProxyPort>`. That nginx forwards `/ui/` and `/v1/` to
|
||||||
|
a second openbao listener on `127.0.0.1:<deploy.bao.uiPort>`, answers 403 on
|
||||||
|
the unseal, seal, step-down, rekey and generate-root endpoints, and 404 on
|
||||||
|
everything else. ⚠️ That listener asks for **no client certificate**, because
|
||||||
|
a browser has none to present. So on this one door a bao token is the whole
|
||||||
|
credential. Anything on the store's host that can dial loopback, and any
|
||||||
|
`admins` session through the vhost, needs only a token to use the API. Unseal
|
||||||
|
from the host's `bao` CLI; the UI's unseal form gets a 403. On a self-signed
|
||||||
|
gateway where the UI is the only swarm name its host fronts, that host's nginx
|
||||||
|
doesn't wait for the store: it serves the hive certificate on the UI's name (a
|
||||||
|
browser warning) until the unsealed store issues the services leaf, so the
|
||||||
|
stream passthrough readers use on that host comes up with the store sealed.
|
||||||
|
|
||||||
## The constraint that decides where the root lives
|
## The constraint that decides where the root lives
|
||||||
|
|
||||||
A hive CA carries `nameConstraints=permitted;DNS:<hive domain>`, and **a swarm
|
A hive CA carries `nameConstraints=permitted;DNS:<hive domain>`, and **a swarm
|
||||||
|
|
|
||||||
|
|
@ -35,6 +35,15 @@ let
|
||||||
builtins.head localServiceDomains
|
builtins.head localServiceDomains
|
||||||
);
|
);
|
||||||
|
|
||||||
|
# Whether the gateway's start waits for the services leaf. Not when the
|
||||||
|
# store's own browser UI is the only swarm name this host fronts: that
|
||||||
|
# host's nginx also carries the stream passthrough every reader dials to
|
||||||
|
# reach the store, and waiting on a login to that store would take that
|
||||||
|
# path down whenever it is sealed. There the gateway starts on the hive
|
||||||
|
# leaf fallback, and the issuing script re-imports the services leaf into
|
||||||
|
# the running nginx once it lands.
|
||||||
|
gatewayWaitsForServicesLeaf = localServiceDomains != [ baoCfg.ui.domain ];
|
||||||
|
|
||||||
# The host-managed hive CA is the trust anchor for self-signed mode.
|
# The host-managed hive CA is the trust anchor for self-signed mode.
|
||||||
# It is only stood up when the gateway actually serves a self-signed
|
# It is only stood up when the gateway actually serves a self-signed
|
||||||
# cert: the gateway must run here and be in self-signed mode. The
|
# cert: the gateway must run here and be in self-signed mode. The
|
||||||
|
|
@ -723,8 +732,8 @@ in
|
||||||
# bundle reaches the root the other way round, at the end of the script
|
# bundle reaches the root the other way round, at the end of the script
|
||||||
# below: it restarts `hive-tls-ca` when the root CHANGED, which is the
|
# below: it restarts `hive-tls-ca` when the root CHANGED, which is the
|
||||||
# same path that already covered a store coming up hours late.
|
# same path that already covered a store coming up hours late.
|
||||||
before = [ "hive-gateway-self-signed-cert.service" ];
|
before = lib.optional gatewayWaitsForServicesLeaf "hive-gateway-self-signed-cert.service";
|
||||||
requiredBy = [ "hive-gateway-self-signed-cert.service" ];
|
requiredBy = lib.optional gatewayWaitsForServicesLeaf "hive-gateway-self-signed-cert.service";
|
||||||
# The store's container, where it runs here. On a hive that reads a
|
# The store's container, where it runs here. On a hive that reads a
|
||||||
# store hosted elsewhere no such unit exists and systemd ignores
|
# store hosted elsewhere no such unit exists and systemd ignores
|
||||||
# the name, which is the correct behaviour rather than a gap: what
|
# the name, which is the correct behaviour rather than a gap: what
|
||||||
|
|
|
||||||
|
|
@ -1508,6 +1508,14 @@ in
|
||||||
domain = swarmDomain;
|
domain = swarmDomain;
|
||||||
subject = [ "group:${operatorGroup}" ];
|
subject = [ "group:${operatorGroup}" ];
|
||||||
policy = "one_factor";
|
policy = "one_factor";
|
||||||
|
}
|
||||||
|
# The store's browser UI. Not keyed to a deploy flag: its
|
||||||
|
# vhost is on the store's host, which need not be this
|
||||||
|
# one, and without this rule any session passes it.
|
||||||
|
++ lib.optional (swarmDomain != null) {
|
||||||
|
domain = hyperhiveCfg.swarm.bao.ui.domain;
|
||||||
|
subject = [ "group:${operatorGroup}" ];
|
||||||
|
policy = "one_factor";
|
||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -19,6 +19,7 @@
|
||||||
# `deploy.bao.extraListenAddresses` names, and — for readers inside an agent
|
# `deploy.bao.extraListenAddresses` names, and — for readers inside an agent
|
||||||
# container — the nginx *stream* passthrough below, which routes on the SNI
|
# container — the nginx *stream* passthrough below, which routes on the SNI
|
||||||
# without decrypting and so leaves the client certificate intact.
|
# without decrypting and so leaves the client certificate intact.
|
||||||
|
# The browser UI's vhost fronts a different listener; see `uiListener`.
|
||||||
#
|
#
|
||||||
# ⚠️ THIS MODULE HAS NO OPINION ABOUT WHERE THE STORE'S IDENTITY COMES FROM.
|
# ⚠️ THIS MODULE HAS NO OPINION ABOUT WHERE THE STORE'S IDENTITY COMES FROM.
|
||||||
# A store must not take its certificates from an authority it will itself
|
# A store must not take its certificates from an authority it will itself
|
||||||
|
|
@ -942,7 +943,8 @@ let
|
||||||
// listenerTls;
|
// listenerTls;
|
||||||
}
|
}
|
||||||
// extraListeners
|
// extraListeners
|
||||||
// metricsListener;
|
// metricsListener
|
||||||
|
// uiListener;
|
||||||
|
|
||||||
# Metrics get their own listener rather than a flag on the one above, and
|
# Metrics get their own listener rather than a flag on the one above, and
|
||||||
# that follows from what a scraper can express: a prometheus scrape config
|
# that follows from what a scraper can express: a prometheus scrape config
|
||||||
|
|
@ -979,6 +981,63 @@ let
|
||||||
# above.
|
# above.
|
||||||
metricsScrapeTarget = "127.0.0.1:${toString baoDeploy.metricsPort}";
|
metricsScrapeTarget = "127.0.0.1:${toString baoDeploy.metricsPort}";
|
||||||
|
|
||||||
|
# The browser UI's listener. No client certificate, because a browser has
|
||||||
|
# none to present, so anything that can dial this address is one bao token
|
||||||
|
# away from the API. That is why it is loopback only, and why the only
|
||||||
|
# thing proxying to it is the nginx below, which browsers reach only
|
||||||
|
# through the gateway, past authelia's `admins` rule. Plain HTTP for the
|
||||||
|
# metrics listener's reason: the hop never leaves this netns.
|
||||||
|
uiListener = {
|
||||||
|
ui = {
|
||||||
|
type = "tcp";
|
||||||
|
address = "127.0.0.1:${toString baoDeploy.uiPort}";
|
||||||
|
tls_disable = true;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
# Every port this container binds in the host's netns, each listener's
|
||||||
|
# derived cluster port included.
|
||||||
|
containerPorts = [
|
||||||
|
cfg.port
|
||||||
|
(cfg.port + 1)
|
||||||
|
baoDeploy.metricsPort
|
||||||
|
(baoDeploy.metricsPort + 1)
|
||||||
|
baoDeploy.uiPort
|
||||||
|
(baoDeploy.uiPort + 1)
|
||||||
|
baoDeploy.uiProxyPort
|
||||||
|
cfg.otel.telemetryPort
|
||||||
|
];
|
||||||
|
|
||||||
|
# Paths a browser session has no business sending to the store: they unseal,
|
||||||
|
# seal, rekey or mint a root token, and the unauthenticated ones among them
|
||||||
|
# can start or cancel an attempt with no token at all. `seal-status` must
|
||||||
|
# stay reachable, since the UI polls it, which is why the first pattern is
|
||||||
|
# anchored at the end.
|
||||||
|
uiDeniedPaths = [
|
||||||
|
"~* ^/v1/sys/(unseal|seal|step-down)/?$"
|
||||||
|
"~* ^/v1/sys/(rekey|generate-root)"
|
||||||
|
];
|
||||||
|
|
||||||
|
# Passes on what the gateway already set for the browser's request. bao
|
||||||
|
# reads none of these unless `x_forwarded_for_authorized_addrs` is set on
|
||||||
|
# the listener, which it is not, so its audit log names 127.0.0.1.
|
||||||
|
uiProxyHeaders = ''
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;
|
||||||
|
proxy_set_header X-Forwarded-Host $http_x_forwarded_host;
|
||||||
|
'';
|
||||||
|
|
||||||
|
# The gateway vhost's gate, copied from ./swarm-victorialogs.nix for that
|
||||||
|
# file's reason: one string does not justify importing ./swarm-ui.nix.
|
||||||
|
# `auth_request` does not inherit across sibling locations, so every
|
||||||
|
# location but the subrequest itself repeats it.
|
||||||
|
swarmAuthRequest = ''
|
||||||
|
auth_request /__hive_authelia;
|
||||||
|
auth_request_set $target_url $scheme://$http_host$request_uri;
|
||||||
|
error_page 401 =302 https://${autheliaCfg.domain}/?rd=$target_url;
|
||||||
|
'';
|
||||||
|
|
||||||
# ⚠️ Neither the listener above nor the forwarder below has a condition of
|
# ⚠️ Neither the listener above nor the forwarder below has a condition of
|
||||||
# its own, and they lost theirs for the same reason at different times.
|
# its own, and they lost theirs for the same reason at different times.
|
||||||
#
|
#
|
||||||
|
|
@ -1812,6 +1871,33 @@ in
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|
||||||
|
uiPort = lib.mkOption {
|
||||||
|
type = lib.types.port;
|
||||||
|
default = 8204;
|
||||||
|
description = ''
|
||||||
|
Loopback port of the listener that serves the store's browser UI.
|
||||||
|
It requires **no client certificate**, so only the nginx in the
|
||||||
|
store's container proxies to it; see
|
||||||
|
{option}`services.hyperhive.deploy.bao.uiProxyPort`.
|
||||||
|
|
||||||
|
⚠️ openbao also binds this port **+ 1** as the listener's cluster
|
||||||
|
address, so neither may coincide with another port the store or its
|
||||||
|
container binds. An assertion checks this.
|
||||||
|
'';
|
||||||
|
};
|
||||||
|
|
||||||
|
uiProxyPort = lib.mkOption {
|
||||||
|
type = lib.types.port;
|
||||||
|
default = 8206;
|
||||||
|
description = ''
|
||||||
|
Loopback port of the nginx inside the store's container that fronts
|
||||||
|
{option}`services.hyperhive.deploy.bao.uiPort`. The gateway's
|
||||||
|
{option}`services.hyperhive.swarm.bao.ui.domain` vhost proxies here.
|
||||||
|
This nginx forwards only `/ui/` and `/v1/`, and refuses the
|
||||||
|
unseal, seal, step-down, rekey and generate-root endpoints.
|
||||||
|
'';
|
||||||
|
};
|
||||||
|
|
||||||
extraListenAddresses = lib.mkOption {
|
extraListenAddresses = lib.mkOption {
|
||||||
type = lib.types.listOf lib.types.str;
|
type = lib.types.listOf lib.types.str;
|
||||||
default = [ ];
|
default = [ ];
|
||||||
|
|
@ -1858,6 +1944,21 @@ in
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|
||||||
|
ui.domain = lib.mkOption {
|
||||||
|
type = lib.types.str;
|
||||||
|
default = "bao-ui.${domainBase}";
|
||||||
|
defaultText = lib.literalExpression ''"bao-ui.''${services.hyperhive.swarm.domain}"'';
|
||||||
|
description = ''
|
||||||
|
Name the gateway serves the store's browser UI on, to members of
|
||||||
|
authelia's `admins` group only. Swarm-wide because authelia's host
|
||||||
|
writes the access rule for it and the store's host serves it.
|
||||||
|
|
||||||
|
Must differ from {option}`services.hyperhive.swarm.bao.domain`: that
|
||||||
|
name is the mutual-TLS endpoint every reader dials, and it has no
|
||||||
|
vhost.
|
||||||
|
'';
|
||||||
|
};
|
||||||
|
|
||||||
port = lib.mkOption {
|
port = lib.mkOption {
|
||||||
type = lib.types.port;
|
type = lib.types.port;
|
||||||
default = 8200;
|
default = 8200;
|
||||||
|
|
@ -1953,6 +2054,26 @@ in
|
||||||
about it. Pick any other free port.
|
about it. Pick any other free port.
|
||||||
'';
|
'';
|
||||||
}
|
}
|
||||||
|
{
|
||||||
|
assertion = lib.length (lib.unique containerPorts) == lib.length containerPorts;
|
||||||
|
message = ''
|
||||||
|
Two ports the store's container binds in the host's netns coincide:
|
||||||
|
${lib.concatMapStringsSep ", " toString containerPorts}. They are
|
||||||
|
services.hyperhive.swarm.bao.port and +1, deploy.bao.metricsPort and
|
||||||
|
+1, deploy.bao.uiPort and +1 (openbao binds each listener's port + 1
|
||||||
|
as its cluster address), deploy.bao.uiProxyPort, and
|
||||||
|
swarm.bao.otel.telemetryPort. Pick free ports.
|
||||||
|
'';
|
||||||
|
}
|
||||||
|
{
|
||||||
|
assertion = cfg.ui.domain != cfg.domain;
|
||||||
|
message = ''
|
||||||
|
services.hyperhive.swarm.bao.ui.domain equals
|
||||||
|
services.hyperhive.swarm.bao.domain (${cfg.domain}). That name is the
|
||||||
|
store's mutual-TLS endpoint; a vhost on it would terminate the TLS
|
||||||
|
every reader authenticates with. Give the UI a name of its own.
|
||||||
|
'';
|
||||||
|
}
|
||||||
{
|
{
|
||||||
assertion = haveServerTls;
|
assertion = haveServerTls;
|
||||||
message = ''
|
message = ''
|
||||||
|
|
@ -2024,16 +2145,18 @@ in
|
||||||
# to varies, and a multi-host swarm is the operator's upstream DNS. This
|
# to varies, and a multi-host swarm is the operator's upstream DNS. This
|
||||||
# covers the case that has no upstream record to configure.
|
# covers the case that has no upstream record to configure.
|
||||||
#
|
#
|
||||||
# ⚠️ DNS only. Bao is deliberately absent from `swarm.serviceDomains`
|
# ⚠️ DNS only for `cfg.domain`. It is deliberately absent from
|
||||||
# and gets no vhost: nginx terminating TLS would strip the client
|
# `swarm.serviceDomains` and gets no vhost: nginx terminating TLS would
|
||||||
# certificate, which is how the store authenticates every hive — see
|
# strip the client certificate, which is how the store authenticates
|
||||||
# this file's header. `localNames` is the one half of the sibling
|
# every hive — see this file's header. The UI's name is an ordinary
|
||||||
# pattern that applies.
|
# sibling service name and gets the whole pattern.
|
||||||
services.hyperhive.gateway.localNames = [ cfg.domain ];
|
services.hyperhive.gateway.localNames = [
|
||||||
|
cfg.domain
|
||||||
|
cfg.ui.domain
|
||||||
|
];
|
||||||
|
|
||||||
# All three, and no vhost among them: the `ssl_preread` stream
|
# The `ssl_preread` stream server below needs the nginx process and
|
||||||
# server below needs the nginx process without asking it to serve
|
# listens on the bridge address; the UI's vhost is the one vhost.
|
||||||
# anything, and it listens on the bridge address.
|
|
||||||
services.hyperhive.gateway.enable = lib.mkDefault true;
|
services.hyperhive.gateway.enable = lib.mkDefault true;
|
||||||
services.hyperhive.gateway.dns.enable = lib.mkDefault true;
|
services.hyperhive.gateway.dns.enable = lib.mkDefault true;
|
||||||
services.hyperhive.network.enable = lib.mkDefault true;
|
services.hyperhive.network.enable = lib.mkDefault true;
|
||||||
|
|
@ -2125,6 +2248,46 @@ in
|
||||||
# certificate its CA signed.
|
# certificate its CA signed.
|
||||||
services.hyperhive.network.exposeHostPorts = [ cfg.port ];
|
services.hyperhive.network.exposeHostPorts = [ cfg.port ];
|
||||||
|
|
||||||
|
# The operators' door to the UI, shaped like ./swarm-victorialogs.nix's
|
||||||
|
# vhost. It proxies to the container's nginx over loopback (shared
|
||||||
|
# netns), so no port opens anywhere. `auth_request` only proves a
|
||||||
|
# session; restricting it to `admins` is ./swarm-authelia.nix's rule
|
||||||
|
# for this name. `forceSSL` because authelia answers a plain-http
|
||||||
|
# subrequest with 400, and `removeAttrs` because nixos refuses
|
||||||
|
# `addSSL` beside it.
|
||||||
|
services.nginx.virtualHosts.${cfg.ui.domain} =
|
||||||
|
(builtins.removeAttrs (hyperhiveCfg.gateway.lib.tlsFor cfg.ui.domain) [ "addSSL" ])
|
||||||
|
// {
|
||||||
|
forceSSL = true;
|
||||||
|
listen = hyperhiveCfg.gateway.lib.listen;
|
||||||
|
extraConfig = hyperhiveCfg.gateway.lib.securityHeaders;
|
||||||
|
locations = {
|
||||||
|
"/" = {
|
||||||
|
proxyPass = "http://127.0.0.1:${toString baoDeploy.uiProxyPort}";
|
||||||
|
extraConfig = swarmAuthRequest;
|
||||||
|
};
|
||||||
|
|
||||||
|
"= /__hive_authelia" = {
|
||||||
|
proxyPass = "https://${autheliaCfg.domain}/api/authz/auth-request";
|
||||||
|
# nixpkgs appends its OWN `Host $host` after extraConfig,
|
||||||
|
# which would override verifiedProxyTo's — see the comment
|
||||||
|
# on verifiedProxyTo in hive-gateway/vhost-lib.nix.
|
||||||
|
recommendedProxySettings = false;
|
||||||
|
extraConfig = ''
|
||||||
|
internal;
|
||||||
|
${hyperhiveCfg.gateway.lib.verifiedProxyTo autheliaCfg.domain}
|
||||||
|
proxy_pass_request_body off;
|
||||||
|
proxy_set_header Content-Length "";
|
||||||
|
proxy_set_header X-Original-Method $request_method;
|
||||||
|
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
|
||||||
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
|
proxy_set_header X-Forwarded-Host $http_host;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
'';
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
# THE delivery unit for the forwarder's own OIDC client secret, copied
|
# THE delivery unit for the forwarder's own OIDC client secret, copied
|
||||||
# in shape from ./swarm-otel.nix's `swarm-bao-otel-oidc`: a cert login
|
# in shape from ./swarm-otel.nix's `swarm-bao-otel-oidc`: a cert login
|
||||||
# that fails LOUDLY, since every state it fails on is one a retry fixes,
|
# that fails LOUDLY, since every state it fails on is one a retry fixes,
|
||||||
|
|
@ -2450,9 +2613,10 @@ in
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|
||||||
# The swarm's first grant, written from the HOST. Every API listener sets
|
# The swarm's first grant, written from the HOST. Every API listener but
|
||||||
# `tls_require_and_verify_client_cert`, so a client needs an identity
|
# the loopback UI one sets `tls_require_and_verify_client_cert`, so a
|
||||||
# wherever it runs — and only the host has one: the granter's leaf.
|
# client needs an identity wherever it runs — and only the host has one:
|
||||||
|
# the granter's leaf.
|
||||||
systemd.services.swarm-bao-controller-policy = lib.mkIf haveGranter {
|
systemd.services.swarm-bao-controller-policy = lib.mkIf haveGranter {
|
||||||
description = "write the swarm controller's bao policy and cert-auth role";
|
description = "write the swarm controller's bao policy and cert-auth role";
|
||||||
after = [ "container@${cfg.machine}.service" ] ++ granterAfter;
|
after = [ "container@${cfg.machine}.service" ] ++ granterAfter;
|
||||||
|
|
@ -3281,12 +3445,52 @@ in
|
||||||
settings = {
|
settings = {
|
||||||
listener = listeners;
|
listener = listeners;
|
||||||
storage.raft.path = stateDir;
|
storage.raft.path = stateDir;
|
||||||
|
# Served on every listener, but a browser can only use the
|
||||||
|
# one that asks for no client certificate: `ui`, via the
|
||||||
|
# nginx below.
|
||||||
|
ui = true;
|
||||||
}
|
}
|
||||||
// advertise
|
// advertise
|
||||||
// telemetry
|
// telemetry
|
||||||
// sealSettings;
|
// sealSettings;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
# The only client of the `ui` listener. Loopback in the host's
|
||||||
|
# netns, where the gateway's vhost for `swarm.bao.ui.domain` dials
|
||||||
|
# it. Everything but the UI and its API is 404, and the key-share
|
||||||
|
# and seal endpoints are 403 whatever token is presented.
|
||||||
|
# `absolute_redirect off` keeps the `/` redirect relative;
|
||||||
|
# otherwise its Location names this listener, not the gateway.
|
||||||
|
services.nginx = {
|
||||||
|
enable = true;
|
||||||
|
virtualHosts.bao-ui = {
|
||||||
|
listen = [
|
||||||
|
{
|
||||||
|
addr = "127.0.0.1";
|
||||||
|
port = baoDeploy.uiProxyPort;
|
||||||
|
}
|
||||||
|
];
|
||||||
|
extraConfig = ''
|
||||||
|
absolute_redirect off;
|
||||||
|
'';
|
||||||
|
locations = {
|
||||||
|
"= /".return = "302 /ui/";
|
||||||
|
"/".return = "404";
|
||||||
|
"/ui/" = {
|
||||||
|
proxyPass = "http://127.0.0.1:${toString baoDeploy.uiPort}";
|
||||||
|
extraConfig = uiProxyHeaders;
|
||||||
|
};
|
||||||
|
"/v1/" = {
|
||||||
|
proxyPass = "http://127.0.0.1:${toString baoDeploy.uiPort}";
|
||||||
|
extraConfig = uiProxyHeaders;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
// lib.genAttrs uiDeniedPaths (_: {
|
||||||
|
return = "403";
|
||||||
|
});
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
# The private key crosses the user boundary here, not on the mount.
|
# The private key crosses the user boundary here, not on the mount.
|
||||||
# `swarm-bao-certs` installs it 0600 root-owned, and the unit runs
|
# `swarm-bao-certs` installs it 0600 root-owned, and the unit runs
|
||||||
# as a `DynamicUser`, so the bind-mounted file is unreadable to it —
|
# as a `DynamicUser`, so the bind-mounted file is unreadable to it —
|
||||||
|
|
@ -3348,9 +3552,9 @@ in
|
||||||
# ⚠️ NO `units` allowlist here, and that is the point rather than a
|
# ⚠️ NO `units` allowlist here, and that is the point rather than a
|
||||||
# simplification. A shared collector needs one because the journal
|
# simplification. A shared collector needs one because the journal
|
||||||
# it reads holds six containers' units plus the host's own; this
|
# it reads holds six containers' units plus the host's own; this
|
||||||
# one reads only openbao and the two oneshots beside it, so there
|
# one reads only openbao, the UI's nginx and the two oneshots, so
|
||||||
# is nothing foreign to separate out — and a list of unit names is
|
# there is nothing foreign to separate out — and a list of unit names
|
||||||
# a thing to get wrong, which ships nothing while looking healthy.
|
# is a thing to get wrong, which ships nothing while looking healthy.
|
||||||
assertions = [
|
assertions = [
|
||||||
{
|
{
|
||||||
# Sibling of the agent forwarder's identical assertion, and it
|
# Sibling of the agent forwarder's identical assertion, and it
|
||||||
|
|
|
||||||
|
|
@ -109,7 +109,10 @@ let
|
||||||
# this list, `gateway.lib.tlsFor` falls back to the hive leaf, which
|
# this list, `gateway.lib.tlsFor` falls back to the hive leaf, which
|
||||||
# cannot cover a name under a different apex — see the ⚠️ above this
|
# cannot cover a name under a different apex — see the ⚠️ above this
|
||||||
# list for what that looked like the last time a name was missed here.
|
# list for what that looked like the last time a name was missed here.
|
||||||
++ lib.optional deployCfg.victorialogs.enable swarmCfg.victorialogs.domain;
|
++ lib.optional deployCfg.victorialogs.enable swarmCfg.victorialogs.domain
|
||||||
|
# The store's browser UI, not the store: `swarm.bao.domain` is mTLS and
|
||||||
|
# never gets a gateway certificate.
|
||||||
|
++ lib.optional deployCfg.bao.enable swarmCfg.bao.ui.domain;
|
||||||
|
|
||||||
# Hives whose entry still carries the removed `certFingerprint`. Scanned
|
# Hives whose entry still carries the removed `certFingerprint`. Scanned
|
||||||
# here, at top level, because that is the only place an assertion about a
|
# here, at top level, because that is the only place an assertion about a
|
||||||
|
|
|
||||||
|
|
@ -41,6 +41,13 @@ let
|
||||||
deploy.bao.serverKeyFile = "/etc/pki/bao-key.pem";
|
deploy.bao.serverKeyFile = "/etc/pki/bao-key.pem";
|
||||||
};
|
};
|
||||||
|
|
||||||
|
autheliaOnly = hive { deploy.authelia.enable = true; };
|
||||||
|
|
||||||
|
baoWithForge = hive {
|
||||||
|
deploy.bao.enable = true;
|
||||||
|
deploy.forgejo.enable = true;
|
||||||
|
};
|
||||||
|
|
||||||
# The store's units live inside its container, so the gates below have to
|
# The store's units live inside its container, so the gates below have to
|
||||||
# look there rather than at the host's service set.
|
# look there rather than at the host's service set.
|
||||||
baoUnits = machine: machine.containers.swarm-bao.config.systemd.services;
|
baoUnits = machine: machine.containers.swarm-bao.config.systemd.services;
|
||||||
|
|
@ -281,6 +288,95 @@ let
|
||||||
&& !(lib.hasInfix restart sh)
|
&& !(lib.hasInfix restart sh)
|
||||||
&& lib.hasInfix "# then unseal" sh;
|
&& lib.hasInfix "# then unseal" sh;
|
||||||
}
|
}
|
||||||
|
{
|
||||||
|
# The browser UI's listener must not loosen the one every reader dials.
|
||||||
|
# `loopback` is named so an empty filter cannot pass.
|
||||||
|
name = "every listener but metrics and the UI still requires a client certificate";
|
||||||
|
ok =
|
||||||
|
let
|
||||||
|
l = (baoSettings baoPkcs11).listener;
|
||||||
|
api = builtins.removeAttrs l [
|
||||||
|
"metrics"
|
||||||
|
"ui"
|
||||||
|
];
|
||||||
|
in
|
||||||
|
api ? loopback
|
||||||
|
&& lib.all (x: (x.tls_require_and_verify_client_cert or false) == true) (lib.attrValues api);
|
||||||
|
}
|
||||||
|
{
|
||||||
|
# No client certificate on this one, so loopback is all that keeps it
|
||||||
|
# away from everything but this netns.
|
||||||
|
name = "the UI listener is loopback-only and the UI is served";
|
||||||
|
ok =
|
||||||
|
let
|
||||||
|
s = baoSettings baoPkcs11;
|
||||||
|
in
|
||||||
|
(s.ui or false) == true
|
||||||
|
&& lib.hasPrefix "127.0.0.1:" (s.listener.ui.address or "")
|
||||||
|
&& !(s.listener.ui ? tls_require_and_verify_client_cert);
|
||||||
|
}
|
||||||
|
{
|
||||||
|
# The nginx in front of it is the listener's only client and must bind
|
||||||
|
# loopback too, and it refuses the endpoints a browser never needs.
|
||||||
|
name = "the UI's nginx binds loopback and refuses the unseal and root-generation paths";
|
||||||
|
ok =
|
||||||
|
let
|
||||||
|
v = baoPkcs11.containers.swarm-bao.config.services.nginx.virtualHosts.bao-ui;
|
||||||
|
in
|
||||||
|
lib.all (x: x.addr == "127.0.0.1") v.listen
|
||||||
|
&& (v.locations."~* ^/v1/sys/(unseal|seal|step-down)/?$".return or null) == "403"
|
||||||
|
&& (v.locations."~* ^/v1/sys/(rekey|generate-root)".return or null) == "403";
|
||||||
|
}
|
||||||
|
{
|
||||||
|
# A new vhost gets no group gate from `auth_request`; authelia's
|
||||||
|
# default policy admits any session. Evaluated on a host that runs
|
||||||
|
# authelia and no store, because that is where the rule has to render.
|
||||||
|
name = "authelia restricts the store's UI name to admins wherever authelia runs";
|
||||||
|
ok =
|
||||||
|
let
|
||||||
|
rules =
|
||||||
|
autheliaOnly.containers.swarm-authelia.config.services.authelia.instances.swarm.settings.access_control.rules;
|
||||||
|
in
|
||||||
|
!autheliaOnly.services.hyperhive.deploy.bao.enable
|
||||||
|
&& builtins.elem {
|
||||||
|
domain = "bao-ui.t.local";
|
||||||
|
subject = [ "group:admins" ];
|
||||||
|
policy = "one_factor";
|
||||||
|
} rules;
|
||||||
|
}
|
||||||
|
{
|
||||||
|
# The other half of that rule: the vhost exists, and asks authelia.
|
||||||
|
name = "the store's host serves the UI name behind the authelia subrequest";
|
||||||
|
ok =
|
||||||
|
let
|
||||||
|
v = baoPkcs11.services.nginx.virtualHosts."bao-ui.t.local";
|
||||||
|
in
|
||||||
|
lib.hasInfix "auth_request /__hive_authelia;" (v.locations."/".extraConfig or "")
|
||||||
|
&& (v.locations."/".proxyPass or "") == "http://127.0.0.1:8206";
|
||||||
|
}
|
||||||
|
{
|
||||||
|
# The stream passthrough every reader dials rides the same nginx as the
|
||||||
|
# UI's vhost. On a host whose only swarm name is the UI, that nginx must
|
||||||
|
# not wait on the services leaf, which needs a login to this very store.
|
||||||
|
# The control is a store host that also fronts forge: its gateway waits,
|
||||||
|
# exactly as it did before the UI existed.
|
||||||
|
name = "a store host fronting only the UI does not hold nginx on a store login";
|
||||||
|
ok =
|
||||||
|
let
|
||||||
|
waits =
|
||||||
|
m:
|
||||||
|
builtins.elem "hive-gateway-self-signed-cert.service" m.systemd.services.swarm-services-cert.requiredBy;
|
||||||
|
orders =
|
||||||
|
m:
|
||||||
|
builtins.elem "hive-gateway-self-signed-cert.service" m.systemd.services.swarm-services-cert.before;
|
||||||
|
in
|
||||||
|
baoPkcs11.services.hyperhive.swarm.localServiceDomains == [ "bao-ui.t.local" ]
|
||||||
|
&& baoPkcs11.systemd.services ? hive-gateway-self-signed-cert
|
||||||
|
&& !(waits baoPkcs11)
|
||||||
|
&& !(orders baoPkcs11)
|
||||||
|
&& waits baoWithForge
|
||||||
|
&& orders baoWithForge;
|
||||||
|
}
|
||||||
];
|
];
|
||||||
in
|
in
|
||||||
runGroup "bao-basics" cases
|
runGroup "bao-basics" cases
|
||||||
|
|
|
||||||
|
|
@ -297,8 +297,8 @@ let
|
||||||
cases = [
|
cases = [
|
||||||
{
|
{
|
||||||
# Reads the rendered unit on the HOST, which is where the write happens:
|
# Reads the rendered unit on the HOST, which is where the write happens:
|
||||||
# every API listener demands a client certificate, and the host is the
|
# every API listener but the loopback UI one demands a client
|
||||||
# side that has one.
|
# certificate, and the host is the side that has one.
|
||||||
name = "a store host renders the granting unit on the host, logging in as the granter";
|
name = "a store host renders the granting unit on the host, logging in as the granter";
|
||||||
ok =
|
ok =
|
||||||
let
|
let
|
||||||
|
|
|
||||||
|
|
@ -115,13 +115,19 @@ let
|
||||||
}
|
}
|
||||||
{
|
{
|
||||||
# Control for the case above: these settings are rendered per deployment,
|
# Control for the case above: these settings are rendered per deployment,
|
||||||
# not constants a passing case could be indifferent to. The metrics
|
# not constants a passing case could be indifferent to. The metrics and
|
||||||
# listener is excluded by name — it answers on a port of its own and is
|
# UI listeners are excluded by name — each answers on a loopback port of
|
||||||
# not one of the API addresses this case is counting.
|
# its own and is not one of the declared addresses this case is counting.
|
||||||
name = "a declared extra address renders a second listener beside loopback";
|
name = "a declared extra address renders a second listener beside loopback";
|
||||||
ok =
|
ok =
|
||||||
builtins.length (
|
builtins.length (
|
||||||
builtins.filter (n: n != "metrics") (builtins.attrNames (baoSettings baoTwoAddresses).listener)
|
builtins.filter (
|
||||||
|
n:
|
||||||
|
!(builtins.elem n [
|
||||||
|
"metrics"
|
||||||
|
"ui"
|
||||||
|
])
|
||||||
|
) (builtins.attrNames (baoSettings baoTwoAddresses).listener)
|
||||||
) == 2;
|
) == 2;
|
||||||
}
|
}
|
||||||
{
|
{
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue