swarm-nats: cut the comments back to what the code cannot say

Review: high comment-to-code ratio. Measured — 197 of 337 added lines
were comments, and net of the block that only moved it was still +135
comment against +117 code.

Kept the constraints, the rejected alternatives and the costs; dropped
everything that restated what the code plainly does, and the
slash-stripping explanation that was duplicated in the generator.
This commit is contained in:
atlas 2026-08-16 16:19:30 +02:00
commit 7c3154e3f4

View file

@ -49,57 +49,38 @@ let
autoUserSeed = "${autoSeedDir}/callout-user.seed"; autoUserSeed = "${autoSeedDir}/callout-user.seed";
autoIssuerSeed = "${autoSeedDir}/issuer.seed"; autoIssuerSeed = "${autoSeedDir}/issuer.seed";
# The runtime config directory. World-readable is correct — everything in # The runtime config directory: wrapper, settings symlink and fragment.
# it is public (the server publishes these keys to every client that # World-readable is correct (everything in it is public) and it is not in
# connects) — and it deliberately does not live in the 0700 responder # the 0700 responder secret dir, which the `nats` user cannot traverse.
# secret dir, which the `nats` user cannot traverse.
# #
# ⚠️ ALL THREE FILES LIVE IN ONE DIRECTORY, AND THAT IS FORCED, NOT # ⚠️ ONE DIRECTORY IS FORCED, NOT TIDINESS. NATS resolves an `include` with
# TIDINESS. `include` paths are resolved with Go's `filepath.Join` against # `filepath.Join(configDir, path)`, which strips a leading slash, so an
# the config file's own directory, which **strips a leading slash** — so an # absolute include silently becomes relative and is never found. The
# absolute include silently becomes a relative one: # includes must therefore be bare filenames, i.e. siblings. Invisible to
# # eval: the wrapper renders perfectly and the server refuses to start.
# include "/nix/store/…-nats.conf"
# → open agents/atlas/…/nix/store/…-nats.conf: no such file
#
# Measured, and it is invisible to every eval-level check: the wrapper
# renders perfectly and the server refuses to start. ⇒ the includes must be
# bare filenames, which means the wrapper, the settings and the fragment
# have to be siblings. (Also measured: resolution is relative to the CONFIG
# FILE, not the process CWD, so an absolute `-c` path is fine.)
runtimeDir = "/var/lib/nats-callout"; runtimeDir = "/var/lib/nats-callout";
runtimeWrapper = "${runtimeDir}/nats.conf"; runtimeWrapper = "${runtimeDir}/nats.conf";
hostRuntimeDir = "/var/lib/nixos-containers/${machine}${runtimeDir}"; hostRuntimeDir = "/var/lib/nixos-containers/${machine}${runtimeDir}";
natsFormat = pkgs.formats.json { }; natsFormat = pkgs.formats.json { };
# Upstream's rendered settings, read back out of the container's own # Upstream's rendered settings, re-rendered with the same generator on the
# evaluated config and re-rendered with the same generator — so this is the # same value — the artifact `services.nats` would have used, not a
# artifact `services.nats` would have used, not a transcription of it. # transcription. ⚠️ This reference is also the only thing keeping it alive:
# Referencing it here is also what keeps it alive: with `ExecStart` pointed # `ExecStart` names a runtime path, so nothing else in the closure names
# at a runtime path, nothing else in the closure names it, and the string # the store file, and its string context is what stops it being
# context of this reference is what stops it being garbage-collected out # garbage-collected out from under a running server.
# from under a running server.
#
# ⚠️ It should be the very same store path as upstream's (same function,
# same value), but nothing depends on that being true: if it ever diverged,
# this is still a correct rendering of the same settings.
renderedSettings = natsFormat.generate "nats.conf" ( renderedSettings = natsFormat.generate "nats.conf" (
config.containers.${machine}.config.services.nats.settings config.containers.${machine}.config.services.nats.settings
); );
# The two blocks whose contents depend on the callout keys, defined ONCE # Defined once, rendered twice: into `settings` with the operator's values,
# and rendered TWICE: into `services.nats.settings` with the operator's # and into the runtime fragment with placeholders the generator fills in.
# values, and into the runtime fragment with placeholders the generator
# substitutes.
# #
# 🪤 Do NOT inline these into `settings` and hand-write the fragment. Both # 🪤 Do not inline these and hand-write the fragment instead. Both files are
# files are loaded, and the fragment is the LATER definition, so it WINS # loaded and the fragment is the LATER definition, so it wins — a
# (measured — state/probe-3112-override.sh). A hand-written copy would # hand-written copy would make a future edit to `settings` silently
# mean a future edit to `settings` alone is silently ignored on exactly # ineffective on exactly the hives that use auto mode.
# the hives that use auto mode: the config looks changed and behaves
# unchanged. One definition is what makes that impossible rather than
# merely unlikely.
calloutBlocks = calloutBlocks =
{ {
userKey, userKey,
@ -132,11 +113,10 @@ let
# hole. So the only way to get a live server here is a real key # hole. So the only way to get a live server here is a real key
# whose seed nobody but the responder holds. # whose seed nobody but the responder holds.
# #
# 🔒 That last property is what makes auto mode safe. In auto mode # 🔒 That property is what makes auto mode safe: it renders as ""
# this renders as "" — which the server refuses to start on — and # there, so any field the fragment fails to override keeps a value
# the fragment overrides it at runtime. Every field the merge # the server refuses to start on. An incomplete merge cannot leave
# fails to reach therefore keeps a value that FAILS CLOSED LOUDLY, # a walk-in-able server.
# so an incomplete merge can never leave a walk-in-able server.
${calloutAccount}.users = [ { nkey = userKey; } ]; ${calloutAccount}.users = [ { nkey = userKey; } ];
# ⚠️ `services.nats.jetstream = true` gives the SERVER JetStream; # ⚠️ `services.nats.jetstream = true` gives the SERVER JetStream;
# an account gets it only from its own grant. Measured against a # an account gets it only from its own grant. Measured against a
@ -153,11 +133,9 @@ let
# responder mints credentials; it has no business holding stream # responder mints credentials; it has no business holding stream
# state. # state.
# #
# ⚠️ This is also why the fragment renders the COMPLETE accounts # ⚠️ Also why the fragment renders the COMPLETE accounts block: if a
# block rather than only the AUTH entry: if a later definition # later definition replaced rather than merged, a partial one would
# replaced the block instead of merging into it, a partial fragment # drop this grant and every KV op would fail on a healthy server.
# would drop this grant, and the result is a healthy-looking server
# whose every KV op fails at runtime.
${clientAccount} = { ${clientAccount} = {
jetstream = "enabled"; jetstream = "enabled";
}; };
@ -259,21 +237,15 @@ in
Generate the auth-callout nkeys on this host instead of taking Generate the auth-callout nkeys on this host instead of taking
them from `calloutUserPublicKey` / `calloutIssuerPublicKey`. them from `calloutUserPublicKey` / `calloutIssuerPublicKey`.
A first-boot unit mints both keypairs if they are absent, keeps A first-boot unit mints both keypairs if absent, keeps the seeds
the seeds host-side at `0600`, and writes only the public halves host-side at `0600`, and writes only the public halves into a
into a fragment the server reads. Nothing secret is evaluated, fragment the server reads. Nothing secret is evaluated, so
so nothing secret reaches the nix store. nothing secret reaches the nix store.
Off by default: an option says what it is, not what a deployment
shape implies. The all-local mode turns this on see
`services.hyperhive.enableAllLocalDefaults` in
./local-defaults.nix, which is the one place that decides what
"the whole swarm is this box" means.
Leave it off wherever the queue and its clients are not the same Leave it off wherever the queue and its clients are not the same
operator's problem: the seeds have to be distributed to whoever operator's problem: the seeds must reach whoever runs the
runs the responder, and generating them here would just move responder, and minting them here only moves that distribution
that distribution somewhere less visible. somewhere less visible. `enableAllLocalDefaults` turns it on.
''; '';
}; };
@ -492,52 +464,28 @@ in
serverName = "swarm-nats"; serverName = "swarm-nats";
port = cfg.port; port = cfg.port;
# ⚠️ Build-time validation cannot survive auto mode, and this # In auto mode the keys are empty until the generator runs,
# is measured rather than assumed. In auto mode the keys do # and `nats-server -t` rejects that ("Expected callout user to
# not exist until the generator runs, so this renders with # be a valid public account nkey, got \"\""), so leaving this
# empty strings, and `nats-server -t` rejects exactly that: # on fails the BUILD of every all-local hive. Upstream's own
# # description names the case: disable it when the config
# User entry requires a user # includes other files. The check moves to server start.
# Expected callout user to be a valid public account nkey,
# got ""
#
# ⇒ leaving it on would fail the BUILD of every all-local
# hive. Upstream's own description of this option names the
# case: disable it "when the config includes other files".
#
# The check is not lost, it MOVES: the server parses both
# halves at start, with the fragment present, and refuses to
# come up if either is malformed. Later rather than at build
# time — say so in the docs; do not paper over it with an
# assertion of our own, which would be a second system
# deciding what counts as a valid config.
validateConfig = !cfg.autoGenerateCallout; validateConfig = !cfg.autoGenerateCallout;
# One definition, rendered here with the operator's values and
# into the runtime fragment with placeholders. See
# `calloutBlocks` for why it is not written out twice.
settings = calloutBlocks { settings = calloutBlocks {
userKey = cfg.calloutUserPublicKey; userKey = cfg.calloutUserPublicKey;
issuerKey = cfg.calloutIssuerPublicKey; issuerKey = cfg.calloutIssuerPublicKey;
}; };
}; };
# Auto mode only: point the server at a wrapper that includes # A wrapper that includes upstream's rendered settings verbatim
# upstream's rendered settings *verbatim* plus the runtime # plus the runtime fragment; rendering the config ourselves
# fragment. The alternative — teaching this module to render the # instead would throw away upstream's `settings`, where the
# config itself — would throw away upstream's `settings` # reviewed reasoning lives. The generator writes it, because the
# rendering, which is where all the reviewed reasoning lives. # includes must be siblings of the fragment (see `runtimeDir`).
# #
# The wrapper is written by the generator rather than being a # `mkForce`: upstream defines ExecStart inside an `mkMerge`, so a
# store file, because its includes must be siblings of the # plain override conflicts rather than wins.
# fragment (see `runtimeDir`). A server started before the
# generator has run therefore finds no config at all and refuses,
# which is the same fail-closed direction as every other failure
# mode here.
#
# `mkForce` because `services.nats` defines ExecStart inside an
# `mkMerge`; a plain override conflicts rather than winning
# (measured — state/probe-3112-mkforce.sh).
systemd.services.nats.serviceConfig.ExecStart = lib.mkIf cfg.autoGenerateCallout ( systemd.services.nats.serviceConfig.ExecStart = lib.mkIf cfg.autoGenerateCallout (
lib.mkForce "${pkgs.nats-server}/bin/nats-server -c ${runtimeWrapper}" lib.mkForce "${pkgs.nats-server}/bin/nats-server -c ${runtimeWrapper}"
); );
@ -629,16 +577,10 @@ in
''; '';
}; };
# Auto mode: mint the callout keypairs on this host before anything # ⚠️ Minted on the HOST, not in the container, because the responder is
# that needs them exists. # a separate unit that needs the user seed: generating it inside would
# # trap it there and require a secret-export path back out — the exact
# ⚠️ ON THE HOST, not inside the container, and that is the decision # mechanism this is meant to avoid inventing. Only public halves cross.
# the container boundary forces. The responder is a separate unit that
# needs the user SEED; generating inside the container would trap the
# seed there and require a way back out — a new secret-export path,
# which is precisely the mechanism this issue exists to avoid
# inventing. Here, the seeds never leave a 0700 host directory and only
# the public halves cross the boundary.
systemd.services.swarm-nats-callout-keys = lib.mkIf cfg.autoGenerateCallout { systemd.services.swarm-nats-callout-keys = lib.mkIf cfg.autoGenerateCallout {
description = "mint the swarm queue's auth-callout nkeys"; description = "mint the swarm queue's auth-callout nkeys";
before = [ "container@swarm-nats.service" ]; before = [ "container@swarm-nats.service" ];
@ -694,12 +636,9 @@ in
# would silently run yesterday's config. # would silently run yesterday's config.
ln -sfn ${renderedSettings} ${lib.escapeShellArg hostRuntimeDir}/settings.conf ln -sfn ${renderedSettings} ${lib.escapeShellArg hostRuntimeDir}/settings.conf
# The wrapper. Bare filenames on purpose: NATS resolves an include # The wrapper. Bare filenames — see `runtimeDir`. printf rather than
# against the config file's own directory and strips a leading # a heredoc, whose terminator would depend on nix's indentation
# slash, so an absolute path here would silently fail to open. # stripping and break the next time `nix fmt` touched this block.
# printf, not a heredoc: this lives inside a nix indented string, so
# a heredoc's terminator depends on nix's indentation stripping and
# would break the moment `nix fmt` re-indented this block.
printf 'include "settings.conf"\ninclude "callout.conf"\n' \ printf 'include "settings.conf"\ninclude "callout.conf"\n' \
> ${lib.escapeShellArg hostRuntimeDir}/nats.conf.tmp > ${lib.escapeShellArg hostRuntimeDir}/nats.conf.tmp
chmod 0444 ${lib.escapeShellArg hostRuntimeDir}/nats.conf.tmp chmod 0444 ${lib.escapeShellArg hostRuntimeDir}/nats.conf.tmp