vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

deploy: move the controller's socket and credentials out of swarm.controller

Browse source 
`swarm.*` is what a hive needs to be a *client* of the swarm. For the
controller that is the quick-links it publishes, the client id it is
registered under, how stale a snapshot may read, and its packages. Where
its socket lives and where the three credentials it reads sit are
decisions of the machine running it, so socketPath, forgeTokenFile,
authBridgeUrl and queue.clientSecretFile move to
`deploy.swarm-controller.*`.

⚠️ `deploy.swarm-controller`, not `deploy.hive-controller`. Both exist on
main — the latter is hive-c0re's namespace, where `tls.*` lives — and a
definition on the wrong one lands on a live unrelated option instead of
failing. The shim table is anchored on both sides for that reason.

`queue.clientSecretFile` is a nested field, not a top-level option: it
moves out of `queue` and leaves natsUrl and tokenEndpoint behind, so the
far side needs `deploy.swarm-controller.queue.clientSecretFile`. That is
the `forge.sso` split shape, applied again rather than reinvented — the
endpoints a client dials are swarm-wide, the secret beside them is a path
on one host.

local-defaults.nix set that secret from INSIDE
`config.services.hyperhive.swarm = { ... }`, where a bare `controller.`
prefix means `swarm.controller`. Left there it would still resolve —
through the rename — and warn on every evaluation of a single-host swarm,
which is the same defect an earlier slice fixed for the matrix module. It
moves out into its own `deploy` statement beside the controller's
`enable`. `queue.natsUrl` stays bare: it is a stayer.

swarm-ui.nix read `socketPath` through its own `controllerCfg` alias.
With that repointed the binding had no reader left, so it goes. Unlike
the dead `natsCfg` an earlier slice removed, this one was live until this
commit — the move orphaned it. `deployCfg` was already bound there.

Prose that named a moved option by its full path is requalified: the
assertion message for the queue secret (operator-facing, and inside
`config` where an options-block sweep would miss it) and
`swarm-controller/README.md`'s socket-path line. Left bare on purpose:
`docs/swarm/README.md` and the README's own "never point `socketPath` at
a directory that carries anything else" both name the FEATURE, not a
path.

module-eval configures a hive through all four OLD paths and asserts a
rendered effect for EACH of them — the unit's socket env, its auth-bridge
env, and both credential paths in `LoadCredential`. Asserting all four
rather than one means a rename that resolves but stops reaching the
module is caught per-option, not only where a single assertion happens to
look.
This commit is contained in:
atlas 2026-09-07 13:31:34 +02:00 • committed by mara
commit b7c5067f79
6 changed files with 126 additions and 62 deletions
Download patch file Download diff file Expand all files Collapse all files

16
nix/host-modules/deploy.nix
View file

@ -69,6 +69,22 @@ in
[ "services" "hyperhive" "swarm" "controller" "enable" ]
[ "services" "hyperhive" "deploy" "swarm-controller" "enable" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "controller" "socketPath" ]
[ "services" "hyperhive" "deploy" "swarm-controller" "socketPath" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "controller" "forgeTokenFile" ]
[ "services" "hyperhive" "deploy" "swarm-controller" "forgeTokenFile" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "controller" "authBridgeUrl" ]
[ "services" "hyperhive" "deploy" "swarm-controller" "authBridgeUrl" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "controller" "queue" "clientSecretFile" ]
[ "services" "hyperhive" "deploy" "swarm-controller" "queue" "clientSecretFile" ]
)
(lib.mkRenamedOptionModule
[ "services" "hyperhive" "swarm" "ui" "enable" ]
[ "services" "hyperhive" "deploy" "swarm-ui" "enable" ]

14
nix/host-modules/local-defaults.nix
View file

@ -112,9 +112,6 @@ in
controller.queue.natsUrl = lib.mkIf cfg.deploy.singleHostSwarm (
lib.mkDefault "nats://127.0.0.1:${toString config.services.hyperhive.swarm.nats.port}"
);
controller.queue.clientSecretFile = lib.mkIf cfg.deploy.singleHostSwarm (
lib.mkDefault "${config.services.hyperhive.swarm.authelia.hostClientSecretDir}/swarm-controller.secret"
);
};
# The controller is asserted by the MODE and by nothing else. Its own
@ -131,4 +128,15 @@ in
# does not apply here — that one is about two definitions of `swarm`
# itself, and this is a different top-level path.
config.services.hyperhive.deploy.swarm-controller.enable = lib.mkDefault cfg.deploy.singleHostSwarm;
# Same reason, one option later: the queue secret is a path on THIS host,
# so it moved to `deploy.*` with the rest of the controller's credentials.
# It has to sit out here rather than in the `swarm` attrset above — a bare
# `controller.` prefix in there means `swarm.controller`, which is now only
# a rename shim, so the definition would still resolve and warn on every
# evaluation of a single-host swarm.
config.services.hyperhive.deploy.swarm-controller.queue.clientSecretFile =
lib.mkIf cfg.deploy.singleHostSwarm (
lib.mkDefault "${config.services.hyperhive.swarm.authelia.hostClientSecretDir}/swarm-controller.secret"
);
}

128
nix/host-modules/swarm-controller.nix
View file

@ -91,15 +91,15 @@ let
# (`queueLocal`) is gone — its coordinates are options with assertions,
# so a split-host controller fails at eval instead of silently losing
# its identity. Forge access still has nothing to do with where authelia
# and NATS run, only with whether `cfg.forgeTokenFile` resolves to a real
# file — which `forgeTokenFile`'s own default already handles
# and NATS run, only with whether `forgeTokenFile` resolves to a real
# file — which that option's own default already handles
# (forge-local vs. operator-copied). Gating a second time here would
# just repeat that option's own logic under a different name.
#
# ⚠️ The asymmetry is deliberate: the queue is REQUIRED for a controller
# and the forge is not, so absence is an error for one and a supported
# shape for the other.
forgeEnv = lib.optionalAttrs (cfg.forgeTokenFile != null) {
forgeEnv = lib.optionalAttrs (deployCfg.swarm-controller.forgeTokenFile != null) {
SWARM_CONTROLLER_FORGE_URL = "https://${forgeCfg.domain}";
# Same `%d` shape as the queue secret above — root reads the plaintext
# at unit start, the daemon's own user sees a 0400 copy.
@ -137,8 +137,8 @@ let
# is satisfied by THIS daemon's existing queue OIDC identity
# (`queueEnv` above): "one identity per principal" already covers this,
# so there is nothing new to mint or copy, just the bridge's address.
authBridgeEnv = lib.optionalAttrs (cfg.authBridgeUrl != null) {
SWARM_CONTROLLER_AUTH_BRIDGE_URL = cfg.authBridgeUrl;
authBridgeEnv = lib.optionalAttrs (deployCfg.swarm-controller.authBridgeUrl != null) {
SWARM_CONTROLLER_AUTH_BRIDGE_URL = deployCfg.swarm-controller.authBridgeUrl;
};
# The swarm's own display name, for `GET /api/swarm` (swarm-ui's chrome).
@ -268,27 +268,6 @@ in
'';
};
socketPath = lib.mkOption {
type = lib.types.str;
default = "/run/swarm-controller/controller.sock";
description = ''
Unix socket the daemon serves on, and the path the gateway's nginx
proxies to.
The **directory** is the access control here, not the socket mode:
the socket itself is `0666` (nginx runs as another user, and
`connect(2)` needs write), exactly as hive-c0re publishes the
per-agent sockets. What keeps that safe is that the directory holds
exactly one socket and is traverse-only (`0751`) for everyone else.
Moving this path under a directory that carries anything else —
`/run/hyperhive`, which holds the host admin socket, above all —
exposes whatever else lives there to the same reachability.
Changing this therefore means re-checking what else lives in the
new directory, not just the daemon.
'';
};
staleAfterSeconds = lib.mkOption {
type = lib.types.ints.positive;
default = 120;
@ -340,27 +319,6 @@ in
correct for a remote provider as long as that URL is set.
'';
};
clientSecretFile = lib.mkOption {
type = lib.types.str;
default = "";
example = "/var/lib/secrets/swarm-controller-queue.secret";
description = ''
Path on **this** host holding the plaintext of the controller's
OAuth2 client secret. A path, never a value: the secret would
otherwise land in the world-readable nix store.
The controller cannot mint its own — minting happens inside
authelia's state directory during its first boot — so away from
that host the operator places the secret and names it here.
`singleHostSwarm` points this at the minted file, which
is exactly the case where one exists locally.
Read by `LoadCredential`, so it needs to be readable by root at
unit start and nothing more; the daemon's own user never sees
the original path.
'';
};
};
links = lib.mkOption {
@ -410,6 +368,40 @@ in
'';
};
};
# What stays above is what the controller IS to the swarm: the quick-links
# it publishes, the client id it is registered under, how stale a snapshot
# may read, and its packages. What the host running it decides is here —
# where its socket lives, and the three credentials it reads off local
# disk. `enable` already lives in ./deploy.nix, which also carries the
# renames.
#
# ⚠️ `deploy.swarm-controller`, NOT `deploy.hive-controller`. Both exist:
# the latter is hive-c0re's namespace, where `tls.*` lives. A definition on
# the wrong one lands on a live, unrelated option rather than failing.
options.services.hyperhive.deploy.swarm-controller = {
socketPath = lib.mkOption {
type = lib.types.str;
default = "/run/swarm-controller/controller.sock";
description = ''
Unix socket the daemon serves on, and the path the gateway's nginx
proxies to.
The **directory** is the access control here, not the socket mode:
the socket itself is `0666` (nginx runs as another user, and
`connect(2)` needs write), exactly as hive-c0re publishes the
per-agent sockets. What keeps that safe is that the directory holds
exactly one socket and is traverse-only (`0751`) for everyone else.
Moving this path under a directory that carries anything else —
`/run/hyperhive`, which holds the host admin socket, above all —
exposes whatever else lives there to the same reachability.
Changing this therefore means re-checking what else lives in the
new directory, not just the daemon.
'';
};
forgeTokenFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
# The forge has no `enable` of its own to check — neither half of
@ -465,13 +457,37 @@ in
No new credential to configure: the bearer token presented to
the bridge is minted from THIS daemon's own existing queue OIDC
identity (`queue.*` above) — "one identity per principal"
identity — its endpoints stay in `swarm.controller.queue`, its secret is
`queue.clientSecretFile` below — "one identity per principal"
already covers it. `null` means no agent-identity support:
`CreateIdentity` jobs fail with a clear "no auth bridge
configured here" error rather than the daemon refusing to
start, the same graceful-absence shape `forgeTokenFile` uses.
'';
};
queue = {
clientSecretFile = lib.mkOption {
type = lib.types.str;
default = "";
example = "/var/lib/secrets/swarm-controller-queue.secret";
description = ''
Path on **this** host holding the plaintext of the controller's
OAuth2 client secret. A path, never a value: the secret would
otherwise land in the world-readable nix store.
The controller cannot mint its own — minting happens inside
authelia's state directory during its first boot — so away from
that host the operator places the secret and names it here.
`singleHostSwarm` points this at the minted file, which
is exactly the case where one exists locally.
Read by `LoadCredential`, so it needs to be readable by root at
unit start and nothing more; the daemon's own user never sees
the original path.
'';
};
};
};
config = lib.mkIf (config.services.hyperhive.enable && deployCfg.swarm-controller.enable) {
@ -555,10 +571,10 @@ in
'';
}
{
assertion = cfg.queue.clientSecretFile != "";
assertion = deployCfg.swarm-controller.queue.clientSecretFile != "";
message = ''
services.hyperhive.swarm.controller.queue.clientSecretFile is
unset.
services.hyperhive.deploy.swarm-controller.queue.clientSecretFile
is unset.
It defaults to the file authelia's first-boot generator mints,
which only exists when authelia runs on this host. Elsewhere the
@ -593,9 +609,11 @@ in
# that race on a real boot — two seconds more and it would have hit
# `start-limit-hit`, which does not self-heal.
LoadCredential = [
"queue-client.secret:${cfg.queue.clientSecretFile}"
"queue-client.secret:${deployCfg.swarm-controller.queue.clientSecretFile}"
]
++ lib.optional (cfg.forgeTokenFile != null) "forge-token:${cfg.forgeTokenFile}";
++ lib.optional (
deployCfg.swarm-controller.forgeTokenFile != null
) "forge-token:${deployCfg.swarm-controller.forgeTokenFile}";
# The placeholder default that makes the above non-fatal.
# `LoadCredential=` takes priority over `SetCredential=`, so this is
@ -685,7 +703,7 @@ in
# broken rather than lighter; forge's and the auth bridge's are
# genuinely optional and stay gated on their own option resolving.
environment = {
SWARM_CONTROLLER_SOCKET = cfg.socketPath;
SWARM_CONTROLLER_SOCKET = deployCfg.swarm-controller.socketPath;
# The swarm's hive directory, JSON-encoded — the full directory
# (this daemon has no "self" hive to exclude, unlike
# `swarm.peerHives`, `swarm.hives` minus this hive) rather than
@ -750,7 +768,7 @@ in
description = "watch the swarm controller's queue credential";
wantedBy = [ "multi-user.target" ];
pathConfig = {
PathChanged = cfg.queue.clientSecretFile;
PathChanged = deployCfg.swarm-controller.queue.clientSecretFile;
Unit = "swarm-controller-credential.service";
};
};

5
nix/host-modules/swarm-ui.nix
View file

@ -17,7 +17,6 @@ let
deployCfg = config.services.hyperhive.deploy;
gatewayCfg = config.services.hyperhive.gateway;
autheliaCfg = config.services.hyperhive.swarm.authelia;
controllerCfg = config.services.hyperhive.swarm.controller;
# Same stylix auto-theming `hive-c0re/theme.nix` gives the dashboard —
# this UI was left out of that overlay entirely (an operator with a
@ -211,7 +210,7 @@ in
# path swarm-controller registered a route at is the path
# nginx forwards, no prefix-stripping to keep in sync by hand.
"/api/" = {
proxyPass = "http://unix:${controllerCfg.socketPath}:";
proxyPass = "http://unix:${deployCfg.swarm-controller.socketPath}:";
extraConfig = swarmAuthRequest;
};
# ⚠️ THE ONE LOCATION ON THIS VHOST WITH NO `swarmAuthRequest`,
@ -233,7 +232,7 @@ in
# a future `/webhook/<something-else>` does not inherit the
# bypass by living under a shared prefix.
"/webhook/forge/" = {
proxyPass = "http://unix:${controllerCfg.socketPath}:";
proxyPass = "http://unix:${deployCfg.swarm-controller.socketPath}:";
};
# Swagger UI: same "nginx hosts the themed dist straight from
# the store, only /api/openapi.json is dynamic" shape as the

23
nix/module-eval.nix
View file

@ -118,6 +118,14 @@ let
swarm.nats.calloutIssuerSeedFile = "/run/secrets/nats-issuer.seed";
};
controllerOldPath = hive {
deploy.swarm-controller.enable = true;
swarm.controller.socketPath = "/run/test-ctrl/ctrl.sock";
swarm.controller.forgeTokenFile = "/run/secrets/ctrl-forge.token";
swarm.controller.authBridgeUrl = "http://127.0.0.1:19097";
swarm.controller.queue.clientSecretFile = "/run/secrets/ctrl-queue.secret";
};
grafanaOldPath = hive {
deploy.grafana.enable = true;
swarm.grafana.socketDir = "/run/test-grafana-sock";
@ -308,6 +316,21 @@ let
# proxying to a path nothing creates. All four old paths are defined in
# the fixture, so removing any single shim fails the eval rather than
# only the one this assertion reads.
# All FOUR movers are asserted as rendered effects rather than as option
# values, so a rename that resolved but stopped reaching the module is
# caught per-option instead of only where one assertion happens to look.
name = "a config written against the pre-rename swarm-controller paths still reaches the unit";
ok =
let
u = controllerOldPath.systemd.services.swarm-controller;
creds = u.serviceConfig.LoadCredential;
in
u.environment.SWARM_CONTROLLER_SOCKET == "/run/test-ctrl/ctrl.sock"
&& u.environment.SWARM_CONTROLLER_AUTH_BRIDGE_URL == "http://127.0.0.1:19097"
&& lib.any (c: lib.hasInfix "/run/secrets/ctrl-queue.secret" c) creds
&& lib.any (c: lib.hasInfix "/run/secrets/ctrl-forge.token" c) creds;
}
{
name = "a config written against the pre-rename grafana paths still creates the socket directory";
ok = lib.any (
rule: lib.hasInfix "/run/test-grafana-sock" rule

2
swarm-controller/README.md
View file

@ -24,7 +24,7 @@ The hive-gateway's nginx is the only intended client and reaches the socket
through a bind-mount. A listener that is never bound to an address cannot be
reached from off-host by mistake.
The socket path is `services.hyperhive.swarm.controller.socketPath`, default
The socket path is `services.hyperhive.deploy.swarm-controller.socketPath`, default
`/run/swarm-controller/controller.sock`, exported to the process as
`SWARM_CONTROLLER_SOCKET`.