swarm-nats: give the queue a name, a bao-issued leaf, and require TLS
The queue listened in plaintext on 4222, reached by bridge IP or loopback,
and nothing in-tree opened it to another hive. It now has a name, serves a
certificate for that name alone, and refuses clients that do not speak TLS.
- `swarm.nats.domain`, default `nats.<swarm.domain>`, a sibling name like
`swarm.bao.domain`. The queue host answers it via `gateway.localNames`;
every other hive resolves it through the operator's DNS, as for bao.
- `pki/roles/swarm-nats` allows that one name (bare domain, no subdomains,
IPs or localhost, server flag). A `swarm-nats` cert-auth role and policy
may only `update` `pki/issue/swarm-nats`, written by
`swarm-bao-nats-tls-policy`. The login leaf is minted by glue-bao-tls and
paired by glue-nats-bao-identity. `deploy.bao.natsCommonName` is reserved
as a hive name.
- `swarm-bao-nats-tls` issues the leaf into a directory bound read-only into
the container, restarts nats when it rotates, and re-runs daily.
It joins glue-bao-readers-policy-order, so it is ordered after its policy
unit (`after` and `wants`, never `requires`) where the store is on the
same host. The policy unit joins the store's journald list.
- nats gets `tls {}`, with the key via `LoadCredential`, and no
`allow_non_tls`. `validateConfig` is now off in every mode, because the
build-time check loads a leaf that only exists at runtime.
- 4222 is also open on `wg-hive` when the host is on the mesh, never
host-wide.
- `statusPublish.natsUrl`, `queue.agentNatsUrl`, the controller's URL under
`singleHostSwarm`, and the auth responder all dial
`tls://<swarm.nats.domain>:<port>`. swarm-queue-client hands its CA file
to the NATS connection too, so hive-c0re and the controller trust the
leaf's root.
- docs/swarm/README.md: the queue URL and the one DNS record a multi-host
swarm needs.
module-eval-nats-tls pins the role, the policy, the served leaf, the
firewall, the ordering, and a scan of every `*_NATS_URL` and the
responder's URL across the host and its containers.
Closes #4626
This commit is contained in:
parent
244db367c4
commit
1d261b3fed
17 changed files with 748 additions and 108 deletions
|
|
@ -50,6 +50,7 @@ let
|
|||
deployCfg.bao.grafanaOidcCommonName
|
||||
deployCfg.bao.otelOidcCommonName
|
||||
deployCfg.bao.servicesIssuerCommonName
|
||||
deployCfg.bao.natsCommonName
|
||||
]
|
||||
# The two per-hive readers' subjects, spelled out per hive rather than as the
|
||||
# prefix. The prefix alone would reserve the wrong string: the role for hive
|
||||
|
|
@ -542,17 +543,17 @@ in
|
|||
options.services.hyperhive.deploy.hive-controller.statusPublish = {
|
||||
natsUrl = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = if queueLocal then "nats://127.0.0.1:${toString swarmCfg.nats.port}" else null;
|
||||
defaultText = lib.literalExpression ''"nats://127.0.0.1:''${swarm.nats.port}" when this host runs the queue and the IdP, else null'';
|
||||
example = "nats://10.100.0.1:4222";
|
||||
default =
|
||||
if queueLocal then "tls://${swarmCfg.nats.domain}:${toString swarmCfg.nats.port}" else null;
|
||||
defaultText = lib.literalExpression ''"tls://''${swarm.nats.domain}:''${swarm.nats.port}" when this host runs the queue and the IdP, else null'';
|
||||
example = "tls://nats.example.com:4222";
|
||||
description = ''
|
||||
Where the swarm queue listens, as seen from *this* hive.
|
||||
|
||||
Defaults to loopback when this host runs the queue container
|
||||
itself (it shares the host netns, so loopback is correct there
|
||||
and is not the "localhost means the wrong thing" trap that
|
||||
applies inside agent containers). A hive that is not the swarm
|
||||
host has to name the swarm's mesh address.
|
||||
Defaults to the queue's name when this host runs the queue and the
|
||||
IdP. A hive that is not the swarm host sets the same URL: the name
|
||||
resolves through the operator's DNS there. TLS only, and by name,
|
||||
since the queue's certificate carries the name and no address.
|
||||
|
||||
Null disables status publishing: this hive computes its own
|
||||
readiness as always, and simply offers it to nobody. The swarm
|
||||
|
|
@ -589,9 +590,8 @@ in
|
|||
};
|
||||
|
||||
# The same queue, reached from one layer further in. An agent container has
|
||||
# its own network namespace, so it needs an address of its own rather than
|
||||
# the one beside it in `statusPublish.natsUrl` — sharing that option would
|
||||
# hand every agent a loopback address that resolves to the agent.
|
||||
# its own network namespace and resolver, so this is its own option, even
|
||||
# though by name it defaults to the same URL as `statusPublish.natsUrl`.
|
||||
#
|
||||
# Only the address lives here. The credential does not: it is published per
|
||||
# hive and read out of the store by ./glue-queue-agent-credential.nix, which
|
||||
|
|
@ -599,18 +599,18 @@ in
|
|||
options.services.hyperhive.deploy.hive-controller.queue.agentNatsUrl = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default =
|
||||
if queueLocal then "nats://${cfg.network.bridgeIp}:${toString swarmCfg.nats.port}" else null;
|
||||
defaultText = lib.literalExpression ''"nats://''${network.bridgeIp}:''${swarm.nats.port}" when this host runs the queue and the IdP, else null'';
|
||||
example = "nats://10.100.0.1:4222";
|
||||
if queueLocal then "tls://${swarmCfg.nats.domain}:${toString swarmCfg.nats.port}" else null;
|
||||
defaultText = lib.literalExpression ''"tls://''${swarm.nats.domain}:''${swarm.nats.port}" when this host runs the queue and the IdP, else null'';
|
||||
example = "tls://nats.example.com:4222";
|
||||
description = ''
|
||||
Where the swarm queue listens, as an agent *container* on this host
|
||||
reaches it.
|
||||
|
||||
Defaults to the bridge address when this host runs the queue, because
|
||||
that is the only address it is reachable at from a container:
|
||||
{option}`services.hyperhive.swarm.nats.port` is opened on the bridge
|
||||
interface alone. ⚠️ Never a loopback address — inside an agent's network
|
||||
namespace `127.0.0.1` is the agent, not this host.
|
||||
Defaults to the queue's name when this host runs the queue. The agent
|
||||
resolves it through the bridge, where dnsmasq answers it with the
|
||||
bridge address. ⚠️ Never a loopback address: inside an agent's network
|
||||
namespace `127.0.0.1` is the agent, not this host, and the queue's
|
||||
certificate names no address anyway.
|
||||
|
||||
Null means this hive's agents have not been given the queue's address.
|
||||
Together with
|
||||
|
|
|
|||
Loading…
Reference in a new issue