swarm-nats: give the queue a name, a bao-issued leaf, and require TLS

The queue listened in plaintext on 4222, reached by bridge IP or loopback,
and nothing in-tree opened it to another hive. It now has a name, serves a
certificate for that name alone, and refuses clients that do not speak TLS.

- `swarm.nats.domain`, default `nats.<swarm.domain>`, a sibling name like
  `swarm.bao.domain`. The queue host answers it via `gateway.localNames`;
  every other hive resolves it through the operator's DNS, as for bao.
- `pki/roles/swarm-nats` allows that one name (bare domain, no subdomains,
  IPs or localhost, server flag). A `swarm-nats` cert-auth role and policy
  may only `update` `pki/issue/swarm-nats`, written by
  `swarm-bao-nats-tls-policy`. The login leaf is minted by glue-bao-tls and
  paired by glue-nats-bao-identity. `deploy.bao.natsCommonName` is reserved
  as a hive name.
- `swarm-bao-nats-tls` issues the leaf into a directory bound read-only into
  the container, restarts nats when it rotates, and re-runs daily.
  It joins glue-bao-readers-policy-order, so it is ordered after its policy
  unit (`after` and `wants`, never `requires`) where the store is on the
  same host. The policy unit joins the store's journald list.
- nats gets `tls {}`, with the key via `LoadCredential`, and no
  `allow_non_tls`. `validateConfig` is now off in every mode, because the
  build-time check loads a leaf that only exists at runtime.
- 4222 is also open on `wg-hive` when the host is on the mesh, never
  host-wide.
- `statusPublish.natsUrl`, `queue.agentNatsUrl`, the controller's URL under
  `singleHostSwarm`, and the auth responder all dial
  `tls://<swarm.nats.domain>:<port>`. swarm-queue-client hands its CA file
  to the NATS connection too, so hive-c0re and the controller trust the
  leaf's root.
- docs/swarm/README.md: the queue URL and the one DNS record a multi-host
  swarm needs.

module-eval-nats-tls pins the role, the policy, the served leaf, the
firewall, the ordering, and a scan of every `*_NATS_URL` and the
responder's URL across the host and its containers.

Closes #4626
This commit is contained in:
atlas 2026-09-24 15:43:25 +02:00 • committed by mara
commit 1d261b3fed
17 changed files with 748 additions and 108 deletions

View file

@ -50,6 +50,7 @@ let
deployCfg.bao.grafanaOidcCommonName
deployCfg.bao.otelOidcCommonName
deployCfg.bao.servicesIssuerCommonName
deployCfg.bao.natsCommonName
]
# The two per-hive readers' subjects, spelled out per hive rather than as the
# prefix. The prefix alone would reserve the wrong string: the role for hive
@ -542,17 +543,17 @@ in
options.services.hyperhive.deploy.hive-controller.statusPublish = {
natsUrl = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = if queueLocal then "nats://127.0.0.1:${toString swarmCfg.nats.port}" else null;
defaultText = lib.literalExpression ''"nats://127.0.0.1:''${swarm.nats.port}" when this host runs the queue and the IdP, else null'';
example = "nats://10.100.0.1:4222";
default =
if queueLocal then "tls://${swarmCfg.nats.domain}:${toString swarmCfg.nats.port}" else null;
defaultText = lib.literalExpression ''"tls://''${swarm.nats.domain}:''${swarm.nats.port}" when this host runs the queue and the IdP, else null'';
example = "tls://nats.example.com:4222";
description = ''
Where the swarm queue listens, as seen from *this* hive.
Defaults to loopback when this host runs the queue container
itself (it shares the host netns, so loopback is correct there
and is not the "localhost means the wrong thing" trap that
applies inside agent containers). A hive that is not the swarm
host has to name the swarm's mesh address.
Defaults to the queue's name when this host runs the queue and the
IdP. A hive that is not the swarm host sets the same URL: the name
resolves through the operator's DNS there. TLS only, and by name,
since the queue's certificate carries the name and no address.
Null disables status publishing: this hive computes its own
readiness as always, and simply offers it to nobody. The swarm
@ -589,9 +590,8 @@ in
};
# The same queue, reached from one layer further in. An agent container has
# its own network namespace, so it needs an address of its own rather than
# the one beside it in `statusPublish.natsUrl` — sharing that option would
# hand every agent a loopback address that resolves to the agent.
# its own network namespace and resolver, so this is its own option, even
# though by name it defaults to the same URL as `statusPublish.natsUrl`.
#
# Only the address lives here. The credential does not: it is published per
# hive and read out of the store by ./glue-queue-agent-credential.nix, which
@ -599,18 +599,18 @@ in
options.services.hyperhive.deploy.hive-controller.queue.agentNatsUrl = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default =
if queueLocal then "nats://${cfg.network.bridgeIp}:${toString swarmCfg.nats.port}" else null;
defaultText = lib.literalExpression ''"nats://''${network.bridgeIp}:''${swarm.nats.port}" when this host runs the queue and the IdP, else null'';
example = "nats://10.100.0.1:4222";
if queueLocal then "tls://${swarmCfg.nats.domain}:${toString swarmCfg.nats.port}" else null;
defaultText = lib.literalExpression ''"tls://''${swarm.nats.domain}:''${swarm.nats.port}" when this host runs the queue and the IdP, else null'';
example = "tls://nats.example.com:4222";
description = ''
Where the swarm queue listens, as an agent *container* on this host
reaches it.
Defaults to the bridge address when this host runs the queue, because
that is the only address it is reachable at from a container:
{option}`services.hyperhive.swarm.nats.port` is opened on the bridge
interface alone. ⚠️ Never a loopback address — inside an agent's network
namespace `127.0.0.1` is the agent, not this host.
Defaults to the queue's name when this host runs the queue. The agent
resolves it through the bridge, where dnsmasq answers it with the
bridge address. ⚠️ Never a loopback address: inside an agent's network
namespace `127.0.0.1` is the agent, not this host, and the queue's
certificate names no address anyway.
Null means this hive's agents have not been given the queue's address.
Together with