1d261b3fed
The queue listened in plaintext on 4222, reached by bridge IP or loopback,
and nothing in-tree opened it to another hive. It now has a name, serves a
certificate for that name alone, and refuses clients that do not speak TLS.
- `swarm.nats.domain`, default `nats.<swarm.domain>`, a sibling name like
`swarm.bao.domain`. The queue host answers it via `gateway.localNames`;
every other hive resolves it through the operator's DNS, as for bao.
- `pki/roles/swarm-nats` allows that one name (bare domain, no subdomains,
IPs or localhost, server flag). A `swarm-nats` cert-auth role and policy
may only `update` `pki/issue/swarm-nats`, written by
`swarm-bao-nats-tls-policy`. The login leaf is minted by glue-bao-tls and
paired by glue-nats-bao-identity. `deploy.bao.natsCommonName` is reserved
as a hive name.
- `swarm-bao-nats-tls` issues the leaf into a directory bound read-only into
the container, restarts nats when it rotates, and re-runs daily.
It joins glue-bao-readers-policy-order, so it is ordered after its policy
unit (`after` and `wants`, never `requires`) where the store is on the
same host. The policy unit joins the store's journald list.
- nats gets `tls {}`, with the key via `LoadCredential`, and no
`allow_non_tls`. `validateConfig` is now off in every mode, because the
build-time check loads a leaf that only exists at runtime.
- 4222 is also open on `wg-hive` when the host is on the mesh, never
host-wide.
- `statusPublish.natsUrl`, `queue.agentNatsUrl`, the controller's URL under
`singleHostSwarm`, and the auth responder all dial
`tls://<swarm.nats.domain>:<port>`. swarm-queue-client hands its CA file
to the NATS connection too, so hive-c0re and the controller trust the
leaf's root.
- docs/swarm/README.md: the queue URL and the one DNS record a multi-host
swarm needs.
module-eval-nats-tls pins the role, the policy, the served leaf, the
firewall, the ordering, and a scan of every `*_NATS_URL` and the
responder's URL across the host and its containers.
Closes #4626
159 lines
8.1 KiB
Nix
159 lines
8.1 KiB
Nix
# The all-local deployment mode.
|
|
#
|
|
# `singleHostSwarm` is a *mode*, not a default other options read:
|
|
# it says "this box is the whole deployment" and then asserts the values
|
|
# that follow from that. mara, on the issue: it is "more of a deployment
|
|
# mode via settings set, less a default setting".
|
|
#
|
|
# That distinction is why the derivations live here as `mkDefault` in a
|
|
# `config` block rather than as `default =` inside each option. An option
|
|
# declares what IT is and what it is when nobody asks; a mode declares
|
|
# what a deployment shape implies. Written the other way round, every
|
|
# service option had to name a flag it has no relationship to, and the
|
|
# answer to "what does all-local turn on?" was spread across five files.
|
|
#
|
|
# Adding an autoconfigurable thing later means one line here — not a
|
|
# `default =` in the new module pointing back at this flag.
|
|
{
|
|
lib,
|
|
config,
|
|
...
|
|
}:
|
|
let
|
|
cfg = config.services.hyperhive;
|
|
in
|
|
{
|
|
options.services.hyperhive.deploy.singleHostSwarm = lib.mkOption {
|
|
type = lib.types.bool;
|
|
default = false;
|
|
example = true;
|
|
description = ''
|
|
Run the whole swarm on this host. Turning this on asserts the
|
|
toggles that an all-on-one-box deployment implies: the swarm's
|
|
shared services
|
|
(`services.hyperhive.deploy.allSwarmServices`), the swarm
|
|
CA (`services.hyperhive.swarm.ca.autoConfigure`), the swarm
|
|
controller (`services.hyperhive.deploy.swarm-controller.enable`), and the
|
|
host's `/etc/hosts` entries for the names this hive serves
|
|
(`services.hyperhive.gateway.localHostsEntry`) — with no real DNS
|
|
for those names, the operator is browsing them from the same box
|
|
that answers for them.
|
|
|
|
**Off by default, and that is the load-bearing part.** A swarm's
|
|
services and its hives can live on different hosts, and a host has
|
|
no way to tell which ones it is meant to be — so this is an
|
|
operator saying "this is that box", never something inferred.
|
|
Turn it on for a dev box or a single-hive swarm and get a working
|
|
deployment with no further configuration; leave it off and every
|
|
swarm-level artifact is operator-provided.
|
|
|
|
Each toggle it asserts can still be set explicitly, which wins —
|
|
so "all local except X" needs no new option.
|
|
'';
|
|
};
|
|
|
|
# What the mode asserts. `mkDefault` (priority 1000) beats an option's
|
|
# own `default` (1500) and loses to any explicit definition, which is
|
|
# exactly the precedence a deployment mode wants: it fills in for an
|
|
# operator who hasn't spoken, and never argues with one who has.
|
|
# The gateway's own all-local bit. `localHostsEntry` maps every name
|
|
# this hive answers for to 127.0.0.1 in the HOST's /etc/hosts, which is
|
|
# exactly what "this box is the whole deployment" implies: there is no
|
|
# real DNS for these names, and the operator is browsing them from the
|
|
# same machine that serves them.
|
|
#
|
|
# ⚠️ It does NOT affect what containers resolve. dnsmasq sets
|
|
# `no-hosts = true` unconditionally (see hive-gateway/dnsmasq.nix), so
|
|
# agents keep getting the bridge IP from the authoritative `address=`
|
|
# rules rather than the host's 127.0.0.1 — an entry that would point
|
|
# every agent at its own netns. That guard already existing is what
|
|
# makes turning this on by default safe; without it this line would
|
|
# break every agent's access to the forge.
|
|
config.services.hyperhive.gateway.localHostsEntry = lib.mkDefault cfg.deploy.singleHostSwarm;
|
|
|
|
# Out of the `swarm` attrset below, because it is a `deploy.*` option now
|
|
# (./deploy.nix): "does THIS host run the swarm's services" is a per-host
|
|
# decision. Written as a path rather than folded into a second
|
|
# `config.services.hyperhive.deploy = { … }` attrset, for the same reason
|
|
# the ⚠️ below gives about `swarm`.
|
|
config.services.hyperhive.deploy.allSwarmServices = lib.mkDefault cfg.deploy.singleHostSwarm;
|
|
|
|
# The queue's auth-callout nkeys. Generating them is safe exactly
|
|
# when one operator owns both the queue and its responder, which is
|
|
# what this mode asserts. On any other topology the seeds have to
|
|
# reach whoever runs the responder, and minting them here would move
|
|
# that hand-off somewhere less visible rather than removing it.
|
|
config.services.hyperhive.deploy.nats.autoGenerateCallout =
|
|
lib.mkDefault cfg.deploy.singleHostSwarm;
|
|
|
|
config.services.hyperhive.swarm = {
|
|
ca.autoConfigure = lib.mkDefault cfg.deploy.singleHostSwarm;
|
|
# The controller's queue coordinates. Kept with the mode, not in the
|
|
# options' own `default`, because the controller's minted client secret
|
|
# only exists on the host authelia ran its first boot on — so they belong
|
|
# to the mode that asserts this box is the whole deployment.
|
|
#
|
|
# Deriving them from `deploy.nats` / `deploy.authelia`
|
|
# inside those defaults is the mixing this file exists to prevent: the
|
|
# option would be describing a deployment shape instead of describing
|
|
# itself, and "what does all-local turn on?" would stop having one
|
|
# answer.
|
|
#
|
|
# ⚠️ Must live INSIDE this attrset, not as a second
|
|
# `config.services.hyperhive.swarm.…` path beside it — written that
|
|
# way the two definitions of `swarm` collide and the nested one is
|
|
# silently lost. The gate caught exactly that: mode on, `natsUrl`
|
|
# still "".
|
|
#
|
|
# The *requirement* stays in `swarm-controller.nix` as an assertion:
|
|
# needing a queue is the controller's own property in every topology,
|
|
# and only the convenience is local.
|
|
controller.queue.natsUrl = lib.mkIf cfg.deploy.singleHostSwarm (
|
|
lib.mkDefault "tls://${config.services.hyperhive.swarm.nats.domain}:${toString config.services.hyperhive.swarm.nats.port}"
|
|
);
|
|
};
|
|
|
|
# The controller is asserted by the MODE and by nothing else. Its own
|
|
# option stays `default = false` precisely because running it is a
|
|
# statement about swarm topology — but "this box is the whole
|
|
# deployment" IS that statement, and it is the one shape where the
|
|
# answer isn't ambiguous. Deriving it from `allSwarmServices`
|
|
# instead would be wrong: a hive in a larger swarm can legitimately
|
|
# want the shared services without being the host that controls them.
|
|
#
|
|
# Sits outside the `swarm` attrset above because it is a `deploy.*`
|
|
# option (./deploy.nix): "does THIS host run the controller" is exactly
|
|
# the per-host fact `swarm.*` may not carry. The ⚠️ collision note above
|
|
# does not apply here — that one is about two definitions of `swarm`
|
|
# itself, and this is a different top-level path.
|
|
config.services.hyperhive.deploy.swarm-controller.enable = lib.mkDefault cfg.deploy.singleHostSwarm;
|
|
|
|
# Where the operator drops the token that writes the swarm's first bao
|
|
# grants. All-local supplies the path, never the file: `bao operator init`
|
|
# stays a human step in every shape (see ./swarm-bao.nix), so what
|
|
# co-location makes derivable is only *where* it goes. The granting unit
|
|
# skips until the file appears, so naming the path early costs a boot
|
|
# nothing.
|
|
#
|
|
# ⚠️ Deliberately NOT under `/var/lib/swarm-bao-token`, which already holds
|
|
# the PKCS11 seal material. Those are both "a bao token" and neither can
|
|
# substitute for the other: one unseals the store, this one authorises the
|
|
# first write.
|
|
#
|
|
# Outside the `swarm` attrset above for the same reason the controller's
|
|
# options are — a path on this host is a `deploy.*` fact.
|
|
config.services.hyperhive.deploy.bao.bootstrapTokenFile = lib.mkIf cfg.deploy.singleHostSwarm (
|
|
lib.mkDefault "/var/lib/swarm-bao-bootstrap/grant.token"
|
|
);
|
|
|
|
# Same reason, one option later: the queue secret is a path on THIS host,
|
|
# so it moved to `deploy.*` with the rest of the controller's credentials.
|
|
# It has to sit out here rather than in the `swarm` attrset above — a bare
|
|
# `controller.` prefix in there means `swarm.controller`, which is now only
|
|
# a rename shim, so the definition would still resolve and warn on every
|
|
# evaluation of a single-host swarm.
|
|
config.services.hyperhive.deploy.swarm-controller.queue.clientSecretFile =
|
|
lib.mkIf cfg.deploy.singleHostSwarm (
|
|
lib.mkDefault "${config.services.hyperhive.deploy.authelia.hostClientSecretDir}/swarm-controller.secret"
|
|
);
|
|
}
|