Watch
0
0
Fork
You've already forked hyperhive
0

bao: serve the browser UI to admins via a loopback-only listener

openbao gains a second listener, `ui`, on 127.0.0.1:<deploy.bao.uiPort>
(default 8204) with TLS off and no client-certificate requirement, and
`ui = true`. The existing listeners are unchanged. An nginx inside the
store's container, on 127.0.0.1:<deploy.bao.uiProxyPort> (default 8206),
forwards only /ui/ and /v1/ to it, redirects / to /ui/, answers 403 on
sys/unseal, sys/seal, sys/step-down, sys/rekey* and sys/generate-root*,
and 404 on everything else.

The gateway on the store's host serves `swarm.bao.ui.domain` (default
bao-ui.<swarm>) behind the authelia auth_request subrequest, proxying to
that nginx; the name joins serviceDomains and localNames like every
other gateway-published swarm service. authelia gets an access_control
rule restricting that name to group:admins, rendered wherever authelia
runs, since the default policy admits any session.

Trade-off, ruled by the operator on the parent issue: the UI listener
asks for no client certificate, so on that door a bao token alone is the
credential.

Three comments and a doc line claimed every API listener demands a
client certificate; they now except the loopback UI listener. The
module-eval case counting declared listeners excludes `ui` by name, as
it already did `metrics`.

On a self-signed gateway, the UI's name is a swarm service name, so its
host requests the services leaf from the store. `swarm-services-cert`
sits Before= and RequiredBy= the gateway's cert import, which nginx
Requires=. On a host whose only swarm name is the UI, that would hold
nginx, and with it the stream passthrough every reader dials, on a login
to a store that may be sealed. hive-tls drops those two edges exactly
when the UI is the only local swarm name: nginx starts on the existing
hive-leaf fallback, and the script's existing re-import reloads nginx
once the leaf issues. Every other host keeps both edges.
This commit is contained in:
atlas 2026-09-28 17:28:59 +02:00 • committed by mara
commit 3898ca33c7
10 changed files with 376 additions and 29 deletions

View file

@ -118,8 +118,8 @@ which accepts the leaf `/var/lib/swarm-bao-pki/granter.pem`. Every
`swarm-bao-*-policy` unit then logs in with that leaf. The controller's unit
mounts the KV and pki engines and writes the `swarm-controller` role, and each
sibling unit writes its own principal's policy and role. Every one runs on the
host, because every API listener demands a client certificate and the host is
the side that has one.
host, because every API listener but the loopback UI one demands a client
certificate and the host is the side that has one.
**Confirm with `systemctl status swarm-bao-granter-role`**, which should log
`Uploaded policy: bao-granter` and `Data written to: auth/cert/certs/bao-granter`.

View file

@ -67,7 +67,9 @@ leaf it signed and folded into `trust-bundle.pem`. On the host running
the store it's also at
`/var/lib/swarm-bao-services-pki/services-root.pem`. That's the file to
hand a browser, and reading it needs no store login — which matters,
because every store listener demands a client certificate.
because every store listener but the loopback UI one demands a client
certificate, and that one is gated behind authelia to the `admins`
group, not open to an anonymous browser fetching a trust anchor.
**The granting unit generates the root once, and never again.** It asks
the mount whether it already has an issuer (`bao list pki/issuers`)

View file

@ -428,6 +428,25 @@ agent CA. The listener reads both from `listener-client-ca.pem`, which no
cert-auth role pins. The passthrough carries
whichever certificate the reader presents, unchanged.
## The browser UI
OpenBao's built-in web UI is at `https://bao-ui.<swarm domain>/`
(`swarm.bao.ui.domain`), for members of authelia's `admins` group only. Log in
with a bao token. The gateway vhost checks the session with authelia and
proxies to an nginx inside the store's container on
`127.0.0.1:<deploy.bao.uiProxyPort>`. That nginx forwards `/ui/` and `/v1/` to
a second openbao listener on `127.0.0.1:<deploy.bao.uiPort>`, answers 403 on
the unseal, seal, step-down, rekey and generate-root endpoints, and 404 on
everything else. ⚠️ That listener asks for **no client certificate**, because
a browser has none to present. So on this one door a bao token is the whole
credential. Anything on the store's host that can dial loopback, and any
`admins` session through the vhost, needs only a token to use the API. Unseal
from the host's `bao` CLI; the UI's unseal form gets a 403. On a self-signed
gateway where the UI is the only swarm name its host fronts, that host's nginx
doesn't wait for the store: it serves the hive certificate on the UI's name (a
browser warning) until the unsealed store issues the services leaf, so the
stream passthrough readers use on that host comes up with the store sealed.
## The constraint that decides where the root lives
A hive CA carries `nameConstraints=permitted;DNS:<hive domain>`, and **a swarm

View file

@ -35,6 +35,15 @@ let
builtins.head localServiceDomains
);
# Whether the gateway's start waits for the services leaf. Not when the
# store's own browser UI is the only swarm name this host fronts: that
# host's nginx also carries the stream passthrough every reader dials to
# reach the store, and waiting on a login to that store would take that
# path down whenever it is sealed. There the gateway starts on the hive
# leaf fallback, and the issuing script re-imports the services leaf into
# the running nginx once it lands.
gatewayWaitsForServicesLeaf = localServiceDomains != [ baoCfg.ui.domain ];
# The host-managed hive CA is the trust anchor for self-signed mode.
# It is only stood up when the gateway actually serves a self-signed
# cert: the gateway must run here and be in self-signed mode. The
@ -723,8 +732,8 @@ in
# bundle reaches the root the other way round, at the end of the script
# below: it restarts `hive-tls-ca` when the root CHANGED, which is the
# same path that already covered a store coming up hours late.
before = [ "hive-gateway-self-signed-cert.service" ];
requiredBy = [ "hive-gateway-self-signed-cert.service" ];
before = lib.optional gatewayWaitsForServicesLeaf "hive-gateway-self-signed-cert.service";
requiredBy = lib.optional gatewayWaitsForServicesLeaf "hive-gateway-self-signed-cert.service";
# The store's container, where it runs here. On a hive that reads a
# store hosted elsewhere no such unit exists and systemd ignores
# the name, which is the correct behaviour rather than a gap: what

View file

@ -1508,6 +1508,14 @@ in
domain = swarmDomain;
subject = [ "group:${operatorGroup}" ];
policy = "one_factor";
}
# The store's browser UI. Not keyed to a deploy flag: its
# vhost is on the store's host, which need not be this
# one, and without this rule any session passes it.
++ lib.optional (swarmDomain != null) {
domain = hyperhiveCfg.swarm.bao.ui.domain;
subject = [ "group:${operatorGroup}" ];
policy = "one_factor";
};
};

View file

@ -19,6 +19,7 @@
# `deploy.bao.extraListenAddresses` names, and — for readers inside an agent
# container — the nginx *stream* passthrough below, which routes on the SNI
# without decrypting and so leaves the client certificate intact.
# The browser UI's vhost fronts a different listener; see `uiListener`.
#
# ⚠️ THIS MODULE HAS NO OPINION ABOUT WHERE THE STORE'S IDENTITY COMES FROM.
# A store must not take its certificates from an authority it will itself
@ -942,7 +943,8 @@ let
// listenerTls;
}
// extraListeners
// metricsListener;
// metricsListener
// uiListener;
# Metrics get their own listener rather than a flag on the one above, and
# that follows from what a scraper can express: a prometheus scrape config
@ -979,6 +981,63 @@ let
# above.
metricsScrapeTarget = "127.0.0.1:${toString baoDeploy.metricsPort}";
# The browser UI's listener. No client certificate, because a browser has
# none to present, so anything that can dial this address is one bao token
# away from the API. That is why it is loopback only, and why the only
# thing proxying to it is the nginx below, which browsers reach only
# through the gateway, past authelia's `admins` rule. Plain HTTP for the
# metrics listener's reason: the hop never leaves this netns.
uiListener = {
ui = {
type = "tcp";
address = "127.0.0.1:${toString baoDeploy.uiPort}";
tls_disable = true;
};
};
# Every port this container binds in the host's netns, each listener's
# derived cluster port included.
containerPorts = [
cfg.port
(cfg.port + 1)
baoDeploy.metricsPort
(baoDeploy.metricsPort + 1)
baoDeploy.uiPort
(baoDeploy.uiPort + 1)
baoDeploy.uiProxyPort
cfg.otel.telemetryPort
];
# Paths a browser session has no business sending to the store: they unseal,
# seal, rekey or mint a root token, and the unauthenticated ones among them
# can start or cancel an attempt with no token at all. `seal-status` must
# stay reachable, since the UI polls it, which is why the first pattern is
# anchored at the end.
uiDeniedPaths = [
"~* ^/v1/sys/(unseal|seal|step-down)/?$"
"~* ^/v1/sys/(rekey|generate-root)"
];
# Passes on what the gateway already set for the browser's request. bao
# reads none of these unless `x_forwarded_for_authorized_addrs` is set on
# the listener, which it is not, so its audit log names 127.0.0.1.
uiProxyHeaders = ''
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;
proxy_set_header X-Forwarded-Host $http_x_forwarded_host;
'';
# The gateway vhost's gate, copied from ./swarm-victorialogs.nix for that
# file's reason: one string does not justify importing ./swarm-ui.nix.
# `auth_request` does not inherit across sibling locations, so every
# location but the subrequest itself repeats it.
swarmAuthRequest = ''
auth_request /__hive_authelia;
auth_request_set $target_url $scheme://$http_host$request_uri;
error_page 401 =302 https://${autheliaCfg.domain}/?rd=$target_url;
'';
# ⚠️ Neither the listener above nor the forwarder below has a condition of
# its own, and they lost theirs for the same reason at different times.
#
@ -1812,6 +1871,33 @@ in
'';
};
uiPort = lib.mkOption {
type = lib.types.port;
default = 8204;
description = ''
Loopback port of the listener that serves the store's browser UI.
It requires **no client certificate**, so only the nginx in the
store's container proxies to it; see
{option}`services.hyperhive.deploy.bao.uiProxyPort`.
⚠️ openbao also binds this port **+ 1** as the listener's cluster
address, so neither may coincide with another port the store or its
container binds. An assertion checks this.
'';
};
uiProxyPort = lib.mkOption {
type = lib.types.port;
default = 8206;
description = ''
Loopback port of the nginx inside the store's container that fronts
{option}`services.hyperhive.deploy.bao.uiPort`. The gateway's
{option}`services.hyperhive.swarm.bao.ui.domain` vhost proxies here.
This nginx forwards only `/ui/` and `/v1/`, and refuses the
unseal, seal, step-down, rekey and generate-root endpoints.
'';
};
extraListenAddresses = lib.mkOption {
type = lib.types.listOf lib.types.str;
default = [ ];
@ -1858,6 +1944,21 @@ in
'';
};
ui.domain = lib.mkOption {
type = lib.types.str;
default = "bao-ui.${domainBase}";
defaultText = lib.literalExpression ''"bao-ui.''${services.hyperhive.swarm.domain}"'';
description = ''
Name the gateway serves the store's browser UI on, to members of
authelia's `admins` group only. Swarm-wide because authelia's host
writes the access rule for it and the store's host serves it.
Must differ from {option}`services.hyperhive.swarm.bao.domain`: that
name is the mutual-TLS endpoint every reader dials, and it has no
vhost.
'';
};
port = lib.mkOption {
type = lib.types.port;
default = 8200;
@ -1953,6 +2054,26 @@ in
about it. Pick any other free port.
'';
}
{
assertion = lib.length (lib.unique containerPorts) == lib.length containerPorts;
message = ''
Two ports the store's container binds in the host's netns coincide:
${lib.concatMapStringsSep ", " toString containerPorts}. They are
services.hyperhive.swarm.bao.port and +1, deploy.bao.metricsPort and
+1, deploy.bao.uiPort and +1 (openbao binds each listener's port + 1
as its cluster address), deploy.bao.uiProxyPort, and
swarm.bao.otel.telemetryPort. Pick free ports.
'';
}
{
assertion = cfg.ui.domain != cfg.domain;
message = ''
services.hyperhive.swarm.bao.ui.domain equals
services.hyperhive.swarm.bao.domain (${cfg.domain}). That name is the
store's mutual-TLS endpoint; a vhost on it would terminate the TLS
every reader authenticates with. Give the UI a name of its own.
'';
}
{
assertion = haveServerTls;
message = ''
@ -2024,16 +2145,18 @@ in
# to varies, and a multi-host swarm is the operator's upstream DNS. This
# covers the case that has no upstream record to configure.
#
# ⚠️ DNS only. Bao is deliberately absent from `swarm.serviceDomains`
# and gets no vhost: nginx terminating TLS would strip the client
# certificate, which is how the store authenticates every hive — see
# this file's header. `localNames` is the one half of the sibling
# pattern that applies.
services.hyperhive.gateway.localNames = [ cfg.domain ];
# ⚠️ DNS only for `cfg.domain`. It is deliberately absent from
# `swarm.serviceDomains` and gets no vhost: nginx terminating TLS would
# strip the client certificate, which is how the store authenticates
# every hive — see this file's header. The UI's name is an ordinary
# sibling service name and gets the whole pattern.
services.hyperhive.gateway.localNames = [
cfg.domain
cfg.ui.domain
];
# All three, and no vhost among them: the `ssl_preread` stream
# server below needs the nginx process without asking it to serve
# anything, and it listens on the bridge address.
# The `ssl_preread` stream server below needs the nginx process and
# listens on the bridge address; the UI's vhost is the one vhost.
services.hyperhive.gateway.enable = lib.mkDefault true;
services.hyperhive.gateway.dns.enable = lib.mkDefault true;
services.hyperhive.network.enable = lib.mkDefault true;
@ -2125,6 +2248,46 @@ in
# certificate its CA signed.
services.hyperhive.network.exposeHostPorts = [ cfg.port ];
# The operators' door to the UI, shaped like ./swarm-victorialogs.nix's
# vhost. It proxies to the container's nginx over loopback (shared
# netns), so no port opens anywhere. `auth_request` only proves a
# session; restricting it to `admins` is ./swarm-authelia.nix's rule
# for this name. `forceSSL` because authelia answers a plain-http
# subrequest with 400, and `removeAttrs` because nixos refuses
# `addSSL` beside it.
services.nginx.virtualHosts.${cfg.ui.domain} =
(builtins.removeAttrs (hyperhiveCfg.gateway.lib.tlsFor cfg.ui.domain) [ "addSSL" ])
// {
forceSSL = true;
listen = hyperhiveCfg.gateway.lib.listen;
extraConfig = hyperhiveCfg.gateway.lib.securityHeaders;
locations = {
"/" = {
proxyPass = "http://127.0.0.1:${toString baoDeploy.uiProxyPort}";
extraConfig = swarmAuthRequest;
};
"= /__hive_authelia" = {
proxyPass = "https://${autheliaCfg.domain}/api/authz/auth-request";
# nixpkgs appends its OWN `Host $host` after extraConfig,
# which would override verifiedProxyTo's — see the comment
# on verifiedProxyTo in hive-gateway/vhost-lib.nix.
recommendedProxySettings = false;
extraConfig = ''
internal;
${hyperhiveCfg.gateway.lib.verifiedProxyTo autheliaCfg.domain}
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header X-Original-Method $request_method;
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
'';
};
};
};
# THE delivery unit for the forwarder's own OIDC client secret, copied
# in shape from ./swarm-otel.nix's `swarm-bao-otel-oidc`: a cert login
# that fails LOUDLY, since every state it fails on is one a retry fixes,
@ -2450,9 +2613,10 @@ in
'';
};
# The swarm's first grant, written from the HOST. Every API listener sets
# `tls_require_and_verify_client_cert`, so a client needs an identity
# wherever it runs — and only the host has one: the granter's leaf.
# The swarm's first grant, written from the HOST. Every API listener but
# the loopback UI one sets `tls_require_and_verify_client_cert`, so a
# client needs an identity wherever it runs — and only the host has one:
# the granter's leaf.
systemd.services.swarm-bao-controller-policy = lib.mkIf haveGranter {
description = "write the swarm controller's bao policy and cert-auth role";
after = [ "container@${cfg.machine}.service" ] ++ granterAfter;
@ -3281,12 +3445,52 @@ in
settings = {
listener = listeners;
storage.raft.path = stateDir;
# Served on every listener, but a browser can only use the
# one that asks for no client certificate: `ui`, via the
# nginx below.
ui = true;
}
// advertise
// telemetry
// sealSettings;
};
# The only client of the `ui` listener. Loopback in the host's
# netns, where the gateway's vhost for `swarm.bao.ui.domain` dials
# it. Everything but the UI and its API is 404, and the key-share
# and seal endpoints are 403 whatever token is presented.
# `absolute_redirect off` keeps the `/` redirect relative;
# otherwise its Location names this listener, not the gateway.
services.nginx = {
enable = true;
virtualHosts.bao-ui = {
listen = [
{
addr = "127.0.0.1";
port = baoDeploy.uiProxyPort;
}
];
extraConfig = ''
absolute_redirect off;
'';
locations = {
"= /".return = "302 /ui/";
"/".return = "404";
"/ui/" = {
proxyPass = "http://127.0.0.1:${toString baoDeploy.uiPort}";
extraConfig = uiProxyHeaders;
};
"/v1/" = {
proxyPass = "http://127.0.0.1:${toString baoDeploy.uiPort}";
extraConfig = uiProxyHeaders;
};
}
// lib.genAttrs uiDeniedPaths (_: {
return = "403";
});
};
};
# The private key crosses the user boundary here, not on the mount.
# `swarm-bao-certs` installs it 0600 root-owned, and the unit runs
# as a `DynamicUser`, so the bind-mounted file is unreadable to it —
@ -3348,9 +3552,9 @@ in
# ⚠️ NO `units` allowlist here, and that is the point rather than a
# simplification. A shared collector needs one because the journal
# it reads holds six containers' units plus the host's own; this
# one reads only openbao and the two oneshots beside it, so there
# is nothing foreign to separate out — and a list of unit names is
# a thing to get wrong, which ships nothing while looking healthy.
# one reads only openbao, the UI's nginx and the two oneshots, so
# there is nothing foreign to separate out — and a list of unit names
# is a thing to get wrong, which ships nothing while looking healthy.
assertions = [
{
# Sibling of the agent forwarder's identical assertion, and it

View file

@ -109,7 +109,10 @@ let
# this list, `gateway.lib.tlsFor` falls back to the hive leaf, which
# cannot cover a name under a different apex — see the ⚠️ above this
# list for what that looked like the last time a name was missed here.
++ lib.optional deployCfg.victorialogs.enable swarmCfg.victorialogs.domain;
++ lib.optional deployCfg.victorialogs.enable swarmCfg.victorialogs.domain
# The store's browser UI, not the store: `swarm.bao.domain` is mTLS and
# never gets a gateway certificate.
++ lib.optional deployCfg.bao.enable swarmCfg.bao.ui.domain;
# Hives whose entry still carries the removed `certFingerprint`. Scanned
# here, at top level, because that is the only place an assertion about a

View file

@ -41,6 +41,13 @@ let
deploy.bao.serverKeyFile = "/etc/pki/bao-key.pem";
};
autheliaOnly = hive { deploy.authelia.enable = true; };
baoWithForge = hive {
deploy.bao.enable = true;
deploy.forgejo.enable = true;
};
# The store's units live inside its container, so the gates below have to
# look there rather than at the host's service set.
baoUnits = machine: machine.containers.swarm-bao.config.systemd.services;
@ -281,6 +288,95 @@ let
&& !(lib.hasInfix restart sh)
&& lib.hasInfix "# then unseal" sh;
}
{
# The browser UI's listener must not loosen the one every reader dials.
# `loopback` is named so an empty filter cannot pass.
name = "every listener but metrics and the UI still requires a client certificate";
ok =
let
l = (baoSettings baoPkcs11).listener;
api = builtins.removeAttrs l [
"metrics"
"ui"
];
in
api ? loopback
&& lib.all (x: (x.tls_require_and_verify_client_cert or false) == true) (lib.attrValues api);
}
{
# No client certificate on this one, so loopback is all that keeps it
# away from everything but this netns.
name = "the UI listener is loopback-only and the UI is served";
ok =
let
s = baoSettings baoPkcs11;
in
(s.ui or false) == true
&& lib.hasPrefix "127.0.0.1:" (s.listener.ui.address or "")
&& !(s.listener.ui ? tls_require_and_verify_client_cert);
}
{
# The nginx in front of it is the listener's only client and must bind
# loopback too, and it refuses the endpoints a browser never needs.
name = "the UI's nginx binds loopback and refuses the unseal and root-generation paths";
ok =
let
v = baoPkcs11.containers.swarm-bao.config.services.nginx.virtualHosts.bao-ui;
in
lib.all (x: x.addr == "127.0.0.1") v.listen
&& (v.locations."~* ^/v1/sys/(unseal|seal|step-down)/?$".return or null) == "403"
&& (v.locations."~* ^/v1/sys/(rekey|generate-root)".return or null) == "403";
}
{
# A new vhost gets no group gate from `auth_request`; authelia's
# default policy admits any session. Evaluated on a host that runs
# authelia and no store, because that is where the rule has to render.
name = "authelia restricts the store's UI name to admins wherever authelia runs";
ok =
let
rules =
autheliaOnly.containers.swarm-authelia.config.services.authelia.instances.swarm.settings.access_control.rules;
in
!autheliaOnly.services.hyperhive.deploy.bao.enable
&& builtins.elem {
domain = "bao-ui.t.local";
subject = [ "group:admins" ];
policy = "one_factor";
} rules;
}
{
# The other half of that rule: the vhost exists, and asks authelia.
name = "the store's host serves the UI name behind the authelia subrequest";
ok =
let
v = baoPkcs11.services.nginx.virtualHosts."bao-ui.t.local";
in
lib.hasInfix "auth_request /__hive_authelia;" (v.locations."/".extraConfig or "")
&& (v.locations."/".proxyPass or "") == "http://127.0.0.1:8206";
}
{
# The stream passthrough every reader dials rides the same nginx as the
# UI's vhost. On a host whose only swarm name is the UI, that nginx must
# not wait on the services leaf, which needs a login to this very store.
# The control is a store host that also fronts forge: its gateway waits,
# exactly as it did before the UI existed.
name = "a store host fronting only the UI does not hold nginx on a store login";
ok =
let
waits =
m:
builtins.elem "hive-gateway-self-signed-cert.service" m.systemd.services.swarm-services-cert.requiredBy;
orders =
m:
builtins.elem "hive-gateway-self-signed-cert.service" m.systemd.services.swarm-services-cert.before;
in
baoPkcs11.services.hyperhive.swarm.localServiceDomains == [ "bao-ui.t.local" ]
&& baoPkcs11.systemd.services ? hive-gateway-self-signed-cert
&& !(waits baoPkcs11)
&& !(orders baoPkcs11)
&& waits baoWithForge
&& orders baoWithForge;
}
];
in
runGroup "bao-basics" cases

View file

@ -297,8 +297,8 @@ let
cases = [
{
# Reads the rendered unit on the HOST, which is where the write happens:
# every API listener demands a client certificate, and the host is the
# side that has one.
# every API listener but the loopback UI one demands a client
# certificate, and the host is the side that has one.
name = "a store host renders the granting unit on the host, logging in as the granter";
ok =
let

View file

@ -115,13 +115,19 @@ let
}
{
# Control for the case above: these settings are rendered per deployment,
# not constants a passing case could be indifferent to. The metrics
# listener is excluded by name — it answers on a port of its own and is
# not one of the API addresses this case is counting.
# not constants a passing case could be indifferent to. The metrics and
# UI listeners are excluded by name — each answers on a loopback port of
# its own and is not one of the declared addresses this case is counting.
name = "a declared extra address renders a second listener beside loopback";
ok =
builtins.length (
builtins.filter (n: n != "metrics") (builtins.attrNames (baoSettings baoTwoAddresses).listener)
builtins.filter (
n:
!(builtins.elem n [
"metrics"
"ui"
])
) (builtins.attrNames (baoSettings baoTwoAddresses).listener)
) == 2;
}
{