2f7d3e02e9
Cert auth answers a role, so nothing in the swarm can authenticate to the store until some role exists. Creating the first one therefore cannot itself use a certificate — the credential has to come from outside that cycle, and an operator places it. `deploy.bao.bootstrapTokenFile` names that token. A one-shot unit inside the store's container reads it and writes the `swarm-controller` policy; `local-defaults.nix` supplies the path (never the file) on an all-local deploy, since co-location makes only the location derivable. `bao operator init` stays an operator action in every shape. The unit runs in the container rather than on the host because writing the first grant is a store-side operation: it reaches the store locally and needs no client certificate. Gating it on the store being here is therefore not the co-location assumption glue-matrix-bao-token.nix warns about — a reader has to work from anywhere, the first write never does. Policy only, deliberately: a cert-auth role binds a certificate and the controller has no bao identity yet — it holds no leaf and contains no bao code at all. The two certificates that do exist are both wrong to bind. `clientCertFile` is the store host's own reader leaf rather than the controller's, and the CA that signed it also signs every other reader's, so binding that would let any reader authenticate as the controller. Whoever gives the controller an identity writes the role. The grants are scoped to `hive-*`. Without the prefix the controller could rewrite the policy that constrains it. Two things the module-eval arms pin: the unit renders inside the container with the token path in both its script and its ConditionPathExists, and a host that names a token while running no store gets neither the unit nor the host-side directory. The capabilities were derived with `bao write -output-policy` rather than written from memory, and the setup.md commands were run against a real binary for shape before being written down. Refs #3726.
160 lines
8.2 KiB
Nix
160 lines
8.2 KiB
Nix
# The all-local deployment mode.
|
|
#
|
|
# `singleHostSwarm` is a *mode*, not a default other options read:
|
|
# it says "this box is the whole deployment" and then asserts the values
|
|
# that follow from that. mara, on the issue: it is "more of a deployment
|
|
# mode via settings set, less a default setting".
|
|
#
|
|
# That distinction is why the derivations live here as `mkDefault` in a
|
|
# `config` block rather than as `default =` inside each option. An option
|
|
# declares what IT is and what it is when nobody asks; a mode declares
|
|
# what a deployment shape implies. Written the other way round, every
|
|
# service option had to name a flag it has no relationship to, and the
|
|
# answer to "what does all-local turn on?" was spread across five files.
|
|
#
|
|
# Adding an autoconfigurable thing later means one line here — not a
|
|
# `default =` in the new module pointing back at this flag.
|
|
{
|
|
lib,
|
|
config,
|
|
...
|
|
}:
|
|
let
|
|
cfg = config.services.hyperhive;
|
|
in
|
|
{
|
|
options.services.hyperhive.deploy.singleHostSwarm = lib.mkOption {
|
|
type = lib.types.bool;
|
|
default = false;
|
|
example = true;
|
|
description = ''
|
|
Run the whole swarm on this host. Turning this on asserts the
|
|
toggles that an all-on-one-box deployment implies: the swarm's
|
|
shared services
|
|
(`services.hyperhive.deploy.allSwarmServices`), the swarm
|
|
CA (`services.hyperhive.swarm.ca.autoConfigure`), the swarm
|
|
controller (`services.hyperhive.deploy.swarm-controller.enable`), and the
|
|
host's `/etc/hosts` entries for the names this hive serves
|
|
(`services.hyperhive.gateway.localHostsEntry`) — with no real DNS
|
|
for those names, the operator is browsing them from the same box
|
|
that answers for them.
|
|
|
|
**Off by default, and that is the load-bearing part.** A swarm's
|
|
services and its hives can live on different hosts, and a host has
|
|
no way to tell which ones it is meant to be — so this is an
|
|
operator saying "this is that box", never something inferred.
|
|
Turn it on for a dev box or a single-hive swarm and get a working
|
|
deployment with no further configuration; leave it off and every
|
|
swarm-level artifact is operator-provided.
|
|
|
|
Each toggle it asserts can still be set explicitly, which wins —
|
|
so "all local except X" needs no new option.
|
|
'';
|
|
};
|
|
|
|
# What the mode asserts. `mkDefault` (priority 1000) beats an option's
|
|
# own `default` (1500) and loses to any explicit definition, which is
|
|
# exactly the precedence a deployment mode wants: it fills in for an
|
|
# operator who hasn't spoken, and never argues with one who has.
|
|
# The gateway's own all-local bit. `localHostsEntry` maps every name
|
|
# this hive answers for to 127.0.0.1 in the HOST's /etc/hosts, which is
|
|
# exactly what "this box is the whole deployment" implies: there is no
|
|
# real DNS for these names, and the operator is browsing them from the
|
|
# same machine that serves them.
|
|
#
|
|
# ⚠️ It does NOT affect what containers resolve. dnsmasq sets
|
|
# `no-hosts = true` unconditionally (see hive-gateway/dnsmasq.nix), so
|
|
# agents keep getting the bridge IP from the authoritative `address=`
|
|
# rules rather than the host's 127.0.0.1 — an entry that would point
|
|
# every agent at its own netns. That guard already existing is what
|
|
# makes turning this on by default safe; without it this line would
|
|
# break every agent's access to the forge.
|
|
config.services.hyperhive.gateway.localHostsEntry = lib.mkDefault cfg.deploy.singleHostSwarm;
|
|
|
|
# Out of the `swarm` attrset below, because it is a `deploy.*` option now
|
|
# (./deploy.nix): "does THIS host run the swarm's services" is a per-host
|
|
# decision. Written as a path rather than folded into a second
|
|
# `config.services.hyperhive.deploy = { … }` attrset, for the same reason
|
|
# the ⚠️ below gives about `swarm`.
|
|
config.services.hyperhive.deploy.allSwarmServices = lib.mkDefault cfg.deploy.singleHostSwarm;
|
|
|
|
# The queue's auth-callout nkeys. Generating them is safe exactly
|
|
# when one operator owns both the queue and its responder, which is
|
|
# what this mode asserts. On any other topology the seeds have to
|
|
# reach whoever runs the responder, and minting them here would move
|
|
# that hand-off somewhere less visible rather than removing it.
|
|
config.services.hyperhive.deploy.nats.autoGenerateCallout =
|
|
lib.mkDefault cfg.deploy.singleHostSwarm;
|
|
|
|
config.services.hyperhive.swarm = {
|
|
ca.autoConfigure = lib.mkDefault cfg.deploy.singleHostSwarm;
|
|
# The controller's queue coordinates. Co-location is what makes these
|
|
# derivable at all — loopback only reaches the queue when the queue is
|
|
# here, and the minted client secret only exists on the host authelia
|
|
# ran its first boot on — so they belong to the mode that asserts this
|
|
# box is the whole deployment, not to the options' own `default`.
|
|
#
|
|
# Deriving them from `deploy.nats` / `deploy.authelia`
|
|
# inside those defaults is the mixing this file exists to prevent: the
|
|
# option would be describing a deployment shape instead of describing
|
|
# itself, and "what does all-local turn on?" would stop having one
|
|
# answer.
|
|
#
|
|
# ⚠️ Must live INSIDE this attrset, not as a second
|
|
# `config.services.hyperhive.swarm.…` path beside it — written that
|
|
# way the two definitions of `swarm` collide and the nested one is
|
|
# silently lost. The gate caught exactly that: mode on, `natsUrl`
|
|
# still "".
|
|
#
|
|
# The *requirement* stays in `swarm-controller.nix` as an assertion:
|
|
# needing a queue is the controller's own property in every topology,
|
|
# and only the convenience is local.
|
|
controller.queue.natsUrl = lib.mkIf cfg.deploy.singleHostSwarm (
|
|
lib.mkDefault "nats://127.0.0.1:${toString config.services.hyperhive.swarm.nats.port}"
|
|
);
|
|
};
|
|
|
|
# The controller is asserted by the MODE and by nothing else. Its own
|
|
# option stays `default = false` precisely because running it is a
|
|
# statement about swarm topology — but "this box is the whole
|
|
# deployment" IS that statement, and it is the one shape where the
|
|
# answer isn't ambiguous. Deriving it from `allSwarmServices`
|
|
# instead would be wrong: a hive in a larger swarm can legitimately
|
|
# want the shared services without being the host that controls them.
|
|
#
|
|
# Sits outside the `swarm` attrset above because it is a `deploy.*`
|
|
# option (./deploy.nix): "does THIS host run the controller" is exactly
|
|
# the per-host fact `swarm.*` may not carry. The ⚠️ collision note above
|
|
# does not apply here — that one is about two definitions of `swarm`
|
|
# itself, and this is a different top-level path.
|
|
config.services.hyperhive.deploy.swarm-controller.enable = lib.mkDefault cfg.deploy.singleHostSwarm;
|
|
|
|
# Where the operator drops the token that writes the swarm's first bao
|
|
# grants. All-local supplies the path, never the file: `bao operator init`
|
|
# stays a human step in every shape (see ./swarm-bao.nix), so what
|
|
# co-location makes derivable is only *where* it goes. The granting unit
|
|
# skips until the file appears, so naming the path early costs a boot
|
|
# nothing.
|
|
#
|
|
# ⚠️ Deliberately NOT under `/var/lib/swarm-bao-token`, which already holds
|
|
# the PKCS11 seal material. Those are both "a bao token" and neither can
|
|
# substitute for the other: one unseals the store, this one authorises the
|
|
# first write.
|
|
#
|
|
# Outside the `swarm` attrset above for the same reason the controller's
|
|
# options are — a path on this host is a `deploy.*` fact.
|
|
config.services.hyperhive.deploy.bao.bootstrapTokenFile = lib.mkIf cfg.deploy.singleHostSwarm (
|
|
lib.mkDefault "/var/lib/swarm-bao-bootstrap/grant.token"
|
|
);
|
|
|
|
# Same reason, one option later: the queue secret is a path on THIS host,
|
|
# so it moved to `deploy.*` with the rest of the controller's credentials.
|
|
# It has to sit out here rather than in the `swarm` attrset above — a bare
|
|
# `controller.` prefix in there means `swarm.controller`, which is now only
|
|
# a rename shim, so the definition would still resolve and warn on every
|
|
# evaluation of a single-host swarm.
|
|
config.services.hyperhive.deploy.swarm-controller.queue.clientSecretFile =
|
|
lib.mkIf cfg.deploy.singleHostSwarm (
|
|
lib.mkDefault "${config.services.hyperhive.deploy.authelia.hostClientSecretDir}/swarm-controller.secret"
|
|
);
|
|
}
|