vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

deploy: move the queue's callout identity out of swarm.nats

Browse source 
`swarm.*` is what a hive needs to be a *client* of the swarm. For the
queue that is the ports it answers on, the client id it is registered
under, and the responder package. Whether this host mints its own
auth-callout keypairs and where the seeds sit are decisions of the
machine running it, so autoGenerateCallout, both seed files and both
public keys move to `deploy.nats.*`.

The two PUBLIC keys are the non-obvious half, so the reasoning is here
rather than in a comment nobody re-reads. A public key looks swarm-wide,
and docs/swarm/secrets.md says the user key "is published to every
client that connects" — which is true and does not make it swarm
config: peers RECEIVE it over the wire at connect time, they never
configure it. What decides the placement is that its seed is
unambiguously host-side, and splitting a keypair across two namespaces
is worse than either placement.

local-defaults.nix set `nats.autoGenerateCallout` from INSIDE
`config.services.hyperhive.swarm = { ... }`, so the bare prefix meant
`swarm.nats` and no full-path grep could see it. It moves out of that
block into a `deploy` statement rather than being rewritten in place.

swarm-controller.nix bound `natsCfg` and never used it — one mention,
no bare pass, while its sibling bindings have 5 and 10. Pre-existing
dead code, found by this slice's own alias sweep, removed with it.

Four assertion messages and five doc lines named moved options by a
short form (`nats.calloutUserSeedFile`) that is ambiguous now that both
`swarm.nats` and `deploy.nats` exist; one opened with
"services.hyperhive.swarm.nats has callout public keys", which the split
makes false. Prose that names a VALUE rather than a path — the `nk`
pipeline's `# -> calloutUserPublicKey` — is left bare on purpose.

module-eval configures a hive through all five OLD paths and asserts the
responder's delivery unit exists and interpolates the seed path.
`autoGenerateCallout` is deliberately FALSE there: that is what makes
the seed options decide `responderConfigured`, so the assertion is about
the seeds rather than about the auto-mint branch.
This commit is contained in:
atlas 2026-09-07 12:19:06 +02:00 • committed by mara
commit 81b9ddd189
7 changed files with 129 additions and 53 deletions
Download patch file Download diff file Expand all files Collapse all files

31
nix/host-modules/deploy.nix
View file

@ -98,6 +98,37 @@ in
[ "services" "hyperhive" "deploy" "forgejo" "ci" "package" ]
)
# The queue's callout identity. What stays under `swarm.nats` is what the
# queue IS to every hive — the ports it answers on, the client id it is
# registered under, the responder package; these five are what the host
# running it decides.
#
# ⚠️ The two PUBLIC keys move with their seeds. A public key looks
# swarm-wide, and the user key genuinely is published to every client that
# connects — but peers RECEIVE it over the wire, they never configure it,
# and splitting a keypair across two namespaces is worse than either
# placement. `enable` moved earlier; its shim is the one above.
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "nats" "autoGenerateCallout" ]
[ "services" "hyperhive" "deploy" "nats" "autoGenerateCallout" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "nats" "calloutUserPublicKey" ]
[ "services" "hyperhive" "deploy" "nats" "calloutUserPublicKey" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "nats" "calloutIssuerPublicKey" ]
[ "services" "hyperhive" "deploy" "nats" "calloutIssuerPublicKey" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "nats" "calloutUserSeedFile" ]
[ "services" "hyperhive" "deploy" "nats" "calloutUserSeedFile" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "nats" "calloutIssuerSeedFile" ]
[ "services" "hyperhive" "deploy" "nats" "calloutIssuerSeedFile" ]
)
# The rest of the forge split. What stays under `swarm.forge` is what the
# forge IS from any hive's point of view — its package, the names and ports
# it answers on, the URLs it advertises, the client id it is registered

14
nix/host-modules/local-defaults.nix
View file

@ -78,14 +78,16 @@ in
# the ⚠️ below gives about `swarm`.
config.services.hyperhive.deploy.allSwarmServices = lib.mkDefault cfg.deploy.singleHostSwarm;
# The queue's auth-callout nkeys. Generating them is safe exactly
# when one operator owns both the queue and its responder, which is
# what this mode asserts. On any other topology the seeds have to
# reach whoever runs the responder, and minting them here would move
# that hand-off somewhere less visible rather than removing it.
config.services.hyperhive.deploy.nats.autoGenerateCallout =
lib.mkDefault cfg.deploy.singleHostSwarm;
config.services.hyperhive.swarm = {
ca.autoConfigure = lib.mkDefault cfg.deploy.singleHostSwarm;
# The queue's auth-callout nkeys. Generating them is safe exactly
# when one operator owns both the queue and its responder, which is
# what this mode asserts. On any other topology the seeds have to
# reach whoever runs the responder, and minting them here would move
# that hand-off somewhere less visible rather than removing it.
nats.autoGenerateCallout = lib.mkDefault cfg.deploy.singleHostSwarm;
# The controller's queue coordinates. Co-location is what makes these
# derivable at all — loopback only reaches the queue when the queue is
# here, and the minted client secret only exists on the host authelia

1
nix/host-modules/swarm-controller.nix
View file

@ -32,7 +32,6 @@ let
SWARMCTL_AUTHELIA_USERS_FILE = autheliaCfg.hostUsersFile;
};
natsCfg = config.services.hyperhive.swarm.nats;
forgeCfg = config.services.hyperhive.swarm.forge;
swarmDomain = config.services.hyperhive.swarm.domain;

95
nix/host-modules/swarm-nats.nix
View file

@ -55,14 +55,17 @@ let
# In auto mode the seeds are minted on this host before the container
# starts, so they are configured by construction.
responderConfigured =
cfg.autoGenerateCallout || (cfg.calloutUserSeedFile != "" && cfg.calloutIssuerSeedFile != "");
deployCfg.nats.autoGenerateCallout
|| (deployCfg.nats.calloutUserSeedFile != "" && deployCfg.nats.calloutIssuerSeedFile != "");
clientSecretSource = "${autheliaCfg.hostClientSecretDir}/${cfg.clientId}.secret";
introspectionUrl = "${toString autheliaUrl}/api/oidc/introspection";
# Where the responder's seeds actually come from. One name for two
# origins, so everything downstream stops caring which mode it is in.
userSeedFile = if cfg.autoGenerateCallout then autoUserSeed else cfg.calloutUserSeedFile;
issuerSeedFile = if cfg.autoGenerateCallout then autoIssuerSeed else cfg.calloutIssuerSeedFile;
userSeedFile =
if deployCfg.nats.autoGenerateCallout then autoUserSeed else deployCfg.nats.calloutUserSeedFile;
issuerSeedFile =
if deployCfg.nats.autoGenerateCallout then autoIssuerSeed else deployCfg.nats.calloutIssuerSeedFile;
# Seeds stay on the host at 0600 and never enter the container or the
# store: only the responder needs them, and it reads them by
@ -281,6 +284,32 @@ in
'';
};
authPackage = lib.mkOption {
type = lib.types.package;
defaultText = lib.literalExpression "hyperhive.packages.\${system}.swarm-nats-auth";
description = ''
The auth-callout responder package.
⚠️ Named `authPackage`, not `package`, on purpose: this module
deliberately has **no** `package` option for the server itself
(see the note above — upstream's `services.nats` resolves
`pkgs.nats-server` on its own), so a bare `package` here would
read as "the NATS package" and mean something else entirely.
'';
};
};
# What stays above is what the queue IS to every hive: the ports it answers
# on, the client id it is registered under, the responder package. The
# callout identity below is what the host running it decides — whether it
# mints its own keypairs, and where the seeds sit. `enable` already lives in
# ./deploy.nix, which also carries the renames.
#
# ⚠️ The two PUBLIC keys move with their seeds rather than staying: peers
# receive the user key over the wire when they connect (docs/swarm/secrets.md),
# they never configure it, and splitting a keypair across two namespaces is
# worse than either placement.
options.services.hyperhive.deploy.nats = {
autoGenerateCallout = lib.mkOption {
type = lib.types.bool;
default = false;
@ -324,7 +353,7 @@ in
— which is what makes a hive with no responder genuinely closed
rather than merely gated.
Required when `enable` is set.
Required when `deploy.nats.enable` is set.
'';
};
@ -342,27 +371,13 @@ in
and its whole job is to be widely known; the matching *seed* is
the secret, is never named here, and reaches only the responder.
Required when `enable` is set. Without it the server has no
issuer to trust and no client can be authorized — which is the
fail-closed state described below, but arrived at by accident
Required when `deploy.nats.enable` is set. Without it the server
has no issuer to trust and no client can be authorized — which is
the fail-closed state described below, but arrived at by accident
rather than on purpose, so it fails at eval instead.
'';
};
authPackage = lib.mkOption {
type = lib.types.package;
defaultText = lib.literalExpression "hyperhive.packages.\${system}.swarm-nats-auth";
description = ''
The auth-callout responder package.
⚠️ Named `authPackage`, not `package`, on purpose: this module
deliberately has **no** `package` option for the server itself
(see the note above — upstream's `services.nats` resolves
`pkgs.nats-server` on its own), so a bare `package` here would
read as "the NATS package" and mean something else entirely.
'';
};
calloutUserSeedFile = lib.mkOption {
type = lib.types.str;
default = "";
@ -412,11 +427,12 @@ in
# Fail at EVAL, not at boot: a queue that comes up unable to
# authenticate anyone presents as every client hanging, which is
# several layers from "the operator never set the issuer".
assertion = cfg.autoGenerateCallout || cfg.calloutIssuerPublicKey != "";
assertion = deployCfg.nats.autoGenerateCallout || deployCfg.nats.calloutIssuerPublicKey != "";
message = ''
services.hyperhive.deploy.nats.enable requires
nats.calloutIssuerPublicKey — the public half of the account
nkey that signs user JWTs for this queue.
services.hyperhive.deploy.nats.calloutIssuerPublicKey — the
public half of the account nkey that signs user JWTs for this
queue.
It is public and belongs in config; the matching seed is a
secret and is delivered to the callout responder instead. See
@ -429,11 +445,12 @@ in
# failure is the only place to catch that: the rendered config is
# valid, the server starts, and the hole is invisible until
# somebody connects.
assertion = cfg.autoGenerateCallout || cfg.calloutUserPublicKey != "";
assertion = deployCfg.nats.autoGenerateCallout || deployCfg.nats.calloutUserPublicKey != "";
message = ''
services.hyperhive.deploy.nats.enable requires
nats.calloutUserPublicKey — the public half of the user nkey
the auth-callout responder authenticates as.
services.hyperhive.deploy.nats.calloutUserPublicKey — the
public half of the user nkey the auth-callout responder
authenticates as.
It is exempt from callout approval by design, which is exactly
why it needs its own credential: a `users` entry with a name
@ -457,11 +474,11 @@ in
# boolean is two places for a future edit to land in only one.
assertion = responderConfigured;
message = ''
services.hyperhive.swarm.nats has callout public keys but no
services.hyperhive.deploy.nats has callout public keys but no
seed files:
nats.calloutUserSeedFile = "${cfg.calloutUserSeedFile}"
nats.calloutIssuerSeedFile = "${cfg.calloutIssuerSeedFile}"
deploy.nats.calloutUserSeedFile = "${deployCfg.nats.calloutUserSeedFile}"
deploy.nats.calloutIssuerSeedFile = "${deployCfg.nats.calloutIssuerSeedFile}"
Each seed is the private half of the public key already set
here — the server verifies with the public half, the responder
@ -469,8 +486,8 @@ in
this queue with an auth-callout nobody answers, which refuses
every client rather than degrading.
Set both seed paths, or set nats.autoGenerateCallout = true to
have this host mint all four.
Set both seed paths, or set deploy.nats.autoGenerateCallout = true
to have this host mint all four.
'';
}
{
@ -608,12 +625,12 @@ in
# on fails the BUILD of every all-local hive. Upstream's own
# description names the case: disable it when the config
# includes other files. The check moves to server start.
validateConfig = !cfg.autoGenerateCallout;
validateConfig = !deployCfg.nats.autoGenerateCallout;
settings =
calloutBlocks {
userKey = cfg.calloutUserPublicKey;
issuerKey = cfg.calloutIssuerPublicKey;
userKey = deployCfg.nats.calloutUserPublicKey;
issuerKey = deployCfg.nats.calloutIssuerPublicKey;
}
// {
# The monitoring endpoint, which is what the exporter below
@ -671,7 +688,7 @@ in
#
# `mkForce`: upstream defines ExecStart inside an `mkMerge`, so a
# plain override conflicts rather than wins.
systemd.services.nats.serviceConfig.ExecStart = lib.mkIf cfg.autoGenerateCallout (
systemd.services.nats.serviceConfig.ExecStart = lib.mkIf deployCfg.nats.autoGenerateCallout (
lib.mkForce "${pkgs.nats-server}/bin/nats-server -c ${runtimeWrapper}"
);
@ -757,7 +774,7 @@ in
# nothing to deliver, and a copy that silently succeeds with a stale
# or absent seed is worse than not running.
after =
lib.optional cfg.autoGenerateCallout "swarm-nats-callout-keys.service"
lib.optional deployCfg.nats.autoGenerateCallout "swarm-nats-callout-keys.service"
# The third credential does not come from the generator above — it is
# minted by authelia's FIRST BOOT, inside its own container. Ordering
# after that container is necessary and NOT sufficient: the container
@ -765,7 +782,7 @@ in
# has finished. The wait in the script is what actually closes it;
# this only stops us spinning for the full timeout on every boot.
++ lib.optional deployCfg.authelia.enable "container@${autheliaCfg.machine}.service";
requires = lib.optional cfg.autoGenerateCallout "swarm-nats-callout-keys.service";
requires = lib.optional deployCfg.nats.autoGenerateCallout "swarm-nats-callout-keys.service";
serviceConfig = {
Type = "oneshot";
RemainAfterExit = true;
@ -820,7 +837,7 @@ in
# a separate unit that needs the user seed: generating it inside would
# trap it there and require a secret-export path back out — the exact
# mechanism this is meant to avoid inventing. Only public halves cross.
systemd.services.swarm-nats-callout-keys = lib.mkIf cfg.autoGenerateCallout {
systemd.services.swarm-nats-callout-keys = lib.mkIf deployCfg.nats.autoGenerateCallout {
description = "mint the swarm queue's auth-callout nkeys";
before = [ "container@swarm-nats.service" ];
wantedBy = [ "container@swarm-nats.service" ];