swarm-bao: reach the store through a TLS passthrough, not a vhost
An agent container cannot dial the store's loopback listener: the bridge to-loopback DROP rule is there precisely to stop that, and the store authenticates every reader by client certificate, so the usual answer — a gateway vhost — is the one shape that cannot work. A terminating proxy strips the certificate and bao sees nginx as the client for every hive. nginx's stream module does not terminate. `ssl_preread` reads the SNI off the ClientHello and splices the rest of the connection through byte for byte, so bao completes the handshake itself and authenticates the client it actually has. That is the no-vhost rule kept, not bent. The listener binds the bridge IP rather than every address, because bao already holds `127.0.0.1:<port>` in the same netns and a wildcard bind there is EADDRINUSE — nginx would fail to start, taking the gateway with it. Nothing moves as a result: the name already resolves two ways, so a host-side reader still goes straight to loopback and an agent goes through the passthrough, both on one `BAO_ADDR`. Renders only inside the store's own `deploy.bao.enable` region; a host that runs no store grows no listener and opens no port. Per-agent certificates and per-agent policy are separate work. Refs #4386
This commit is contained in:
parent
cab6910cbb
commit
ef2dfbfb31
3 changed files with 145 additions and 2 deletions
|
|
@ -15,8 +15,10 @@
|
|||
# a non-HTTP protocol. It speaks HTTPS, so nginx *could* front it: **the client
|
||||
# certificate IS the authentication**, and a terminating proxy strips it,
|
||||
# leaving bao seeing nginx as the client for every hive in the swarm — one
|
||||
# identity where there must be many. Reach is loopback plus whatever
|
||||
# `deploy.bao.extraListenAddresses` names.
|
||||
# identity where there must be many. Reach is loopback, whatever
|
||||
# `deploy.bao.extraListenAddresses` names, and — for readers inside an agent
|
||||
# container — the nginx *stream* passthrough below, which routes on the SNI
|
||||
# without decrypting and so leaves the client certificate intact.
|
||||
#
|
||||
# ⚠️ THIS MODULE HAS NO OPINION ABOUT WHERE THE STORE'S IDENTITY COMES FROM.
|
||||
# A store must not take its certificates from an authority it will itself
|
||||
|
|
@ -795,6 +797,39 @@ in
|
|||
# dead exporter rather than a wrong address.
|
||||
bao = "127.0.0.1:${toString baoDeploy.metricsPort}/v1/sys/metrics?format=prometheus";
|
||||
};
|
||||
|
||||
# ⚠️ The one nginx exception to this file's header, and it is one because
|
||||
# it never terminates. `ssl_preread` reads the SNI off the ClientHello
|
||||
# and splices the rest through byte for byte, so bao completes the
|
||||
# handshake itself and sees the *client's* certificate — the exact
|
||||
# property the no-vhost rule protects. A vhost would decrypt here and
|
||||
# hand the store one identity for every hive in the swarm.
|
||||
#
|
||||
# Bridge IP, never `0.0.0.0`: the store shares this netns and already
|
||||
# holds `127.0.0.1:<port>`, so a wildcard bind on that port is
|
||||
# `EADDRINUSE` and nginx would not start at all. Nothing is lost to the
|
||||
# split, because the name already resolves two ways — `/etc/hosts` sends
|
||||
# a host-side reader straight to loopback, dnsmasq answers an agent's
|
||||
# query with the bridge IP. One name, one port, one `BAO_ADDR`; only
|
||||
# which netns asked decides whether the passthrough is in the path.
|
||||
services.nginx.streamConfig = ''
|
||||
map $ssl_preread_server_name $swarm_bao_backend {
|
||||
${cfg.domain} 127.0.0.1:${toString cfg.port};
|
||||
default "";
|
||||
}
|
||||
|
||||
server {
|
||||
listen ${networkCfg.bridgeIp}:${toString cfg.port};
|
||||
ssl_preread on;
|
||||
proxy_pass $swarm_bao_backend;
|
||||
}
|
||||
'';
|
||||
|
||||
# Firewall only: the listener above is already the bridge-IP bind this
|
||||
# option's own documentation asks a host service for. Reaching the port
|
||||
# still proves nothing — bao answers nothing without a client
|
||||
# certificate its CA signed.
|
||||
services.hyperhive.network.exposeHostPorts = [ cfg.port ];
|
||||
})
|
||||
|
||||
(lib.mkIf (hyperhiveCfg.enable && deployCfg.bao.enable && haveServerTls) {
|
||||
|
|
|
|||
Loading…
Reference in a new issue