vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

bao: serve the browser UI to admins via a loopback-only listener

Browse source 
openbao gains a second listener, `ui`, on 127.0.0.1:<deploy.bao.uiPort>
(default 8204) with TLS off and no client-certificate requirement, and
`ui = true`. The existing listeners are unchanged. An nginx inside the
store's container, on 127.0.0.1:<deploy.bao.uiProxyPort> (default 8206),
forwards only /ui/ and /v1/ to it, redirects / to /ui/, answers 403 on
sys/unseal, sys/seal, sys/step-down, sys/rekey* and sys/generate-root*,
and 404 on everything else.

The gateway on the store's host serves `swarm.bao.ui.domain` (default
bao-ui.<swarm>) behind the authelia auth_request subrequest, proxying to
that nginx; the name joins serviceDomains and localNames like every
other gateway-published swarm service. authelia gets an access_control
rule restricting that name to group:admins, rendered wherever authelia
runs, since the default policy admits any session.

Trade-off, ruled by the operator on the parent issue: the UI listener
asks for no client certificate, so on that door a bao token alone is the
credential.

Three comments and a doc line claimed every API listener demands a
client certificate; they now except the loopback UI listener. The
module-eval case counting declared listeners excludes `ui` by name, as
it already did `metrics`.

On a self-signed gateway, the UI's name is a swarm service name, so its
host requests the services leaf from the store. `swarm-services-cert`
sits Before= and RequiredBy= the gateway's cert import, which nginx
Requires=. On a host whose only swarm name is the UI, that would hold
nginx, and with it the stream passthrough every reader dials, on a login
to a store that may be sealed. hive-tls drops those two edges exactly
when the UI is the only local swarm name: nginx starts on the existing
hive-leaf fallback, and the script's existing re-import reloads nginx
once the leaf issues. Every other host keeps both edges.
This commit is contained in:
atlas 2026-09-28 17:28:59 +02:00 • committed by mara
commit 3898ca33c7
10 changed files with 376 additions and 29 deletions
Download patch file Download diff file Expand all files Collapse all files

236
nix/host-modules/swarm-bao.nix
View file

@ -19,6 +19,7 @@
# `deploy.bao.extraListenAddresses` names, and — for readers inside an agent
# container — the nginx *stream* passthrough below, which routes on the SNI
# without decrypting and so leaves the client certificate intact.
# The browser UI's vhost fronts a different listener; see `uiListener`.
#
# ⚠️ THIS MODULE HAS NO OPINION ABOUT WHERE THE STORE'S IDENTITY COMES FROM.
# A store must not take its certificates from an authority it will itself
@ -942,7 +943,8 @@ let
// listenerTls;
}
// extraListeners
// metricsListener;
// metricsListener
// uiListener;
# Metrics get their own listener rather than a flag on the one above, and
# that follows from what a scraper can express: a prometheus scrape config
@ -979,6 +981,63 @@ let
# above.
metricsScrapeTarget = "127.0.0.1:${toString baoDeploy.metricsPort}";
# The browser UI's listener. No client certificate, because a browser has
# none to present, so anything that can dial this address is one bao token
# away from the API. That is why it is loopback only, and why the only
# thing proxying to it is the nginx below, which browsers reach only
# through the gateway, past authelia's `admins` rule. Plain HTTP for the
# metrics listener's reason: the hop never leaves this netns.
uiListener = {
ui = {
type = "tcp";
address = "127.0.0.1:${toString baoDeploy.uiPort}";
tls_disable = true;
};
};
# Every port this container binds in the host's netns, each listener's
# derived cluster port included.
containerPorts = [
cfg.port
(cfg.port + 1)
baoDeploy.metricsPort
(baoDeploy.metricsPort + 1)
baoDeploy.uiPort
(baoDeploy.uiPort + 1)
baoDeploy.uiProxyPort
cfg.otel.telemetryPort
];
# Paths a browser session has no business sending to the store: they unseal,
# seal, rekey or mint a root token, and the unauthenticated ones among them
# can start or cancel an attempt with no token at all. `seal-status` must
# stay reachable, since the UI polls it, which is why the first pattern is
# anchored at the end.
uiDeniedPaths = [
"~* ^/v1/sys/(unseal|seal|step-down)/?$"
"~* ^/v1/sys/(rekey|generate-root)"
];
# Passes on what the gateway already set for the browser's request. bao
# reads none of these unless `x_forwarded_for_authorized_addrs` is set on
# the listener, which it is not, so its audit log names 127.0.0.1.
uiProxyHeaders = ''
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;
proxy_set_header X-Forwarded-Host $http_x_forwarded_host;
'';
# The gateway vhost's gate, copied from ./swarm-victorialogs.nix for that
# file's reason: one string does not justify importing ./swarm-ui.nix.
# `auth_request` does not inherit across sibling locations, so every
# location but the subrequest itself repeats it.
swarmAuthRequest = ''
auth_request /__hive_authelia;
auth_request_set $target_url $scheme://$http_host$request_uri;
error_page 401 =302 https://${autheliaCfg.domain}/?rd=$target_url;
'';
# ⚠️ Neither the listener above nor the forwarder below has a condition of
# its own, and they lost theirs for the same reason at different times.
#
@ -1812,6 +1871,33 @@ in
'';
};
uiPort = lib.mkOption {
type = lib.types.port;
default = 8204;
description = ''
Loopback port of the listener that serves the store's browser UI.
It requires **no client certificate**, so only the nginx in the
store's container proxies to it; see
{option}`services.hyperhive.deploy.bao.uiProxyPort`.
⚠️ openbao also binds this port **+ 1** as the listener's cluster
address, so neither may coincide with another port the store or its
container binds. An assertion checks this.
'';
};
uiProxyPort = lib.mkOption {
type = lib.types.port;
default = 8206;
description = ''
Loopback port of the nginx inside the store's container that fronts
{option}`services.hyperhive.deploy.bao.uiPort`. The gateway's
{option}`services.hyperhive.swarm.bao.ui.domain` vhost proxies here.
This nginx forwards only `/ui/` and `/v1/`, and refuses the
unseal, seal, step-down, rekey and generate-root endpoints.
'';
};
extraListenAddresses = lib.mkOption {
type = lib.types.listOf lib.types.str;
default = [ ];
@ -1858,6 +1944,21 @@ in
'';
};
ui.domain = lib.mkOption {
type = lib.types.str;
default = "bao-ui.${domainBase}";
defaultText = lib.literalExpression ''"bao-ui.''${services.hyperhive.swarm.domain}"'';
description = ''
Name the gateway serves the store's browser UI on, to members of
authelia's `admins` group only. Swarm-wide because authelia's host
writes the access rule for it and the store's host serves it.
Must differ from {option}`services.hyperhive.swarm.bao.domain`: that
name is the mutual-TLS endpoint every reader dials, and it has no
vhost.
'';
};
port = lib.mkOption {
type = lib.types.port;
default = 8200;
@ -1953,6 +2054,26 @@ in
about it. Pick any other free port.
'';
}
{
assertion = lib.length (lib.unique containerPorts) == lib.length containerPorts;
message = ''
Two ports the store's container binds in the host's netns coincide:
${lib.concatMapStringsSep ", " toString containerPorts}. They are
services.hyperhive.swarm.bao.port and +1, deploy.bao.metricsPort and
+1, deploy.bao.uiPort and +1 (openbao binds each listener's port + 1
as its cluster address), deploy.bao.uiProxyPort, and
swarm.bao.otel.telemetryPort. Pick free ports.
'';
}
{
assertion = cfg.ui.domain != cfg.domain;
message = ''
services.hyperhive.swarm.bao.ui.domain equals
services.hyperhive.swarm.bao.domain (${cfg.domain}). That name is the
store's mutual-TLS endpoint; a vhost on it would terminate the TLS
every reader authenticates with. Give the UI a name of its own.
'';
}
{
assertion = haveServerTls;
message = ''
@ -2024,16 +2145,18 @@ in
# to varies, and a multi-host swarm is the operator's upstream DNS. This
# covers the case that has no upstream record to configure.
#
# ⚠️ DNS only. Bao is deliberately absent from `swarm.serviceDomains`
# and gets no vhost: nginx terminating TLS would strip the client
# certificate, which is how the store authenticates every hive — see
# this file's header. `localNames` is the one half of the sibling
# pattern that applies.
services.hyperhive.gateway.localNames = [ cfg.domain ];
# ⚠️ DNS only for `cfg.domain`. It is deliberately absent from
# `swarm.serviceDomains` and gets no vhost: nginx terminating TLS would
# strip the client certificate, which is how the store authenticates
# every hive — see this file's header. The UI's name is an ordinary
# sibling service name and gets the whole pattern.
services.hyperhive.gateway.localNames = [
cfg.domain
cfg.ui.domain
];
# All three, and no vhost among them: the `ssl_preread` stream
# server below needs the nginx process without asking it to serve
# anything, and it listens on the bridge address.
# The `ssl_preread` stream server below needs the nginx process and
# listens on the bridge address; the UI's vhost is the one vhost.
services.hyperhive.gateway.enable = lib.mkDefault true;
services.hyperhive.gateway.dns.enable = lib.mkDefault true;
services.hyperhive.network.enable = lib.mkDefault true;
@ -2125,6 +2248,46 @@ in
# certificate its CA signed.
services.hyperhive.network.exposeHostPorts = [ cfg.port ];
# The operators' door to the UI, shaped like ./swarm-victorialogs.nix's
# vhost. It proxies to the container's nginx over loopback (shared
# netns), so no port opens anywhere. `auth_request` only proves a
# session; restricting it to `admins` is ./swarm-authelia.nix's rule
# for this name. `forceSSL` because authelia answers a plain-http
# subrequest with 400, and `removeAttrs` because nixos refuses
# `addSSL` beside it.
services.nginx.virtualHosts.${cfg.ui.domain} =
(builtins.removeAttrs (hyperhiveCfg.gateway.lib.tlsFor cfg.ui.domain) [ "addSSL" ])
// {
forceSSL = true;
listen = hyperhiveCfg.gateway.lib.listen;
extraConfig = hyperhiveCfg.gateway.lib.securityHeaders;
locations = {
"/" = {
proxyPass = "http://127.0.0.1:${toString baoDeploy.uiProxyPort}";
extraConfig = swarmAuthRequest;
};
"= /__hive_authelia" = {
proxyPass = "https://${autheliaCfg.domain}/api/authz/auth-request";
# nixpkgs appends its OWN `Host $host` after extraConfig,
# which would override verifiedProxyTo's — see the comment
# on verifiedProxyTo in hive-gateway/vhost-lib.nix.
recommendedProxySettings = false;
extraConfig = ''
internal;
${hyperhiveCfg.gateway.lib.verifiedProxyTo autheliaCfg.domain}
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header X-Original-Method $request_method;
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
'';
};
};
};
# THE delivery unit for the forwarder's own OIDC client secret, copied
# in shape from ./swarm-otel.nix's `swarm-bao-otel-oidc`: a cert login
# that fails LOUDLY, since every state it fails on is one a retry fixes,
@ -2450,9 +2613,10 @@ in
'';
};
# The swarm's first grant, written from the HOST. Every API listener sets
# `tls_require_and_verify_client_cert`, so a client needs an identity
# wherever it runs — and only the host has one: the granter's leaf.
# The swarm's first grant, written from the HOST. Every API listener but
# the loopback UI one sets `tls_require_and_verify_client_cert`, so a
# client needs an identity wherever it runs — and only the host has one:
# the granter's leaf.
systemd.services.swarm-bao-controller-policy = lib.mkIf haveGranter {
description = "write the swarm controller's bao policy and cert-auth role";
after = [ "container@${cfg.machine}.service" ] ++ granterAfter;
@ -3281,12 +3445,52 @@ in
settings = {
listener = listeners;
storage.raft.path = stateDir;
# Served on every listener, but a browser can only use the
# one that asks for no client certificate: `ui`, via the
# nginx below.
ui = true;
}
// advertise
// telemetry
// sealSettings;
};
# The only client of the `ui` listener. Loopback in the host's
# netns, where the gateway's vhost for `swarm.bao.ui.domain` dials
# it. Everything but the UI and its API is 404, and the key-share
# and seal endpoints are 403 whatever token is presented.
# `absolute_redirect off` keeps the `/` redirect relative;
# otherwise its Location names this listener, not the gateway.
services.nginx = {
enable = true;
virtualHosts.bao-ui = {
listen = [
{
addr = "127.0.0.1";
port = baoDeploy.uiProxyPort;
}
];
extraConfig = ''
absolute_redirect off;
'';
locations = {
"= /".return = "302 /ui/";
"/".return = "404";
"/ui/" = {
proxyPass = "http://127.0.0.1:${toString baoDeploy.uiPort}";
extraConfig = uiProxyHeaders;
};
"/v1/" = {
proxyPass = "http://127.0.0.1:${toString baoDeploy.uiPort}";
extraConfig = uiProxyHeaders;
};
}
// lib.genAttrs uiDeniedPaths (_: {
return = "403";
});
};
};
# The private key crosses the user boundary here, not on the mount.
# `swarm-bao-certs` installs it 0600 root-owned, and the unit runs
# as a `DynamicUser`, so the bind-mounted file is unreadable to it —
@ -3348,9 +3552,9 @@ in
# ⚠️ NO `units` allowlist here, and that is the point rather than a
# simplification. A shared collector needs one because the journal
# it reads holds six containers' units plus the host's own; this
# one reads only openbao and the two oneshots beside it, so there
# is nothing foreign to separate out — and a list of unit names is
# a thing to get wrong, which ships nothing while looking healthy.
# one reads only openbao, the UI's nginx and the two oneshots, so
# there is nothing foreign to separate out — and a list of unit names
# is a thing to get wrong, which ships nothing while looking healthy.
assertions = [
{
# Sibling of the agent forwarder's identical assertion, and it