vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(swarm): wire a hive's queue coordinates for status publishing

Browse source 
Three options, all three derived from ONE predicate — this host runs both
the queue and the IdP — so a defaulted set is all or nothing. Deriving
them per-service looks equivalent and is not: `enableRequiredServices`
turns on matrix and authelia but not nats, so an ordinary all-local hive
would resolve two of three and trip the assertion below. Making the
partial state unrepresentable is what keeps that assertion honest.

Deliberately not the shape swarm-controller uses. That module emits its
queue coordinates only when authelia and NATS are local, which is right
for a service that *is* a swarm-host service — but a hive is the one thing
in a swarm that routinely is not on the swarm host, so the same rule would
make status publishing work on exactly the deployment that needs it least.

There is no `enable`: three coordinates that are all set is the enable. An
extra flag would allow configured-but-off, which is one more state to
explain and one more way to be silently quiet.

A half-set trio is an eval error rather than a silent no-op, because its
runtime failure mode is the expensive kind — the daemon comes up fine,
never connects, and the hive reads never_reported on a dashboard nobody is
watching yet. With the defaults all-or-nothing, the assertion only ever
judges what an operator typed by hand.

The secret arrives by LoadCredential, not a copy: hive-c0re is a host
unit, so systemd hands it the file directly and the secret never gains a
second on-disk copy. The client id is not chosen here either — it is
`hive-<hiveName>`, the identity swarm-authelia.nix already declares for
every entry in the roster.
This commit is contained in:
atlas 2026-08-15 22:50:51 +02:00
commit 48f69fcdea
3 changed files with 167 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

17
nix/host-modules/hive-c0re/default.nix
View file

@ -251,9 +251,20 @@ in
# path. Same secret the agent containers get (forwarded there via
# nspawn --load-credential); this just also hands it to c0re itself.
# Empty list (no credential) when otel is off or no header is set.
LoadCredential = lib.optional (
config.services.hyperhive.otel.enable && config.services.hyperhive.otel.headersCredential != null
) "otel-headers:${config.services.hyperhive.otel.headersCredential}";
LoadCredential =
lib.optional (
config.services.hyperhive.otel.enable && config.services.hyperhive.otel.headersCredential != null
) "otel-headers:${config.services.hyperhive.otel.headersCredential}"
# The swarm-queue client secret this hive authenticates with to
# publish its own status. `LoadCredential` and not a copy: root
# reads the plaintext at unit start and hive-core sees it 0400
# under `%d`, so the secret never gains a second on-disk copy
# and the daemon never needs read access to wherever it lives.
# (The callout responder copies instead only because it
# delivers into a container, across a filesystem boundary.)
++ lib.optional (
config.services.hyperhive.swarm.statusPublish.clientSecretFile != null
) "swarm-status-client.secret:${config.services.hyperhive.swarm.statusPublish.clientSecretFile}";
# Sandboxing. hive-c0re is unprivileged (runs as hive-core, never
# setuid), makes HTTP requests to forge/matrix/Anthropic (keeps INET),
# and delegates all privileged ops to hive-priv via a Unix socket.

19
nix/host-modules/hive-c0re/environment.nix
View file

@ -209,3 +209,22 @@ in
in
"${s.address}:${toString s.port}";
}
//
# Swarm-queue coordinates for offering this hive's status upward
# (hive-c0re::swarm_status). All four together or none: a half-set
# environment is a deployment bug the daemon refuses to treat as
# "no queue configured", because the failure it would otherwise
# produce is a hive that comes up fine and silently never reports.
# The three-option version of that same rule is asserted at eval in
# ./../swarm.nix, so this can only ever emit a complete set.
lib.optionalAttrs (config.services.hyperhive.swarm.statusPublish.natsUrl != null) {
HIVE_C0RE_NATS_URL = config.services.hyperhive.swarm.statusPublish.natsUrl;
HIVE_C0RE_OIDC_TOKEN_ENDPOINT = config.services.hyperhive.swarm.statusPublish.tokenEndpoint;
# The identity swarm-authelia.nix already declares for every entry in
# `swarm.hives` — the hive does not choose its own name here, it uses
# the one the roster gave it.
HIVE_C0RE_OIDC_CLIENT_ID = "hive-${config.services.hyperhive.hiveName}";
# `%d` is systemd's credentials directory — see the LoadCredential in
# ./default.nix. The daemon reads a path, never a value.
HIVE_C0RE_OIDC_CLIENT_SECRET_FILE = "%d/swarm-status-client.secret";
}

134
nix/host-modules/swarm.nix
View file

@ -53,6 +53,23 @@ let
pinnedHives = lib.attrNames (
lib.filterAttrs (_: hive: hive.certFingerprint != null) swarmCfg.hives
);
# Whether this host can derive its own status-publishing coordinates —
# ONE condition for all three of them, deliberately.
#
# 🩸 They were three independent conditions first, and that was wrong in
# a way only an eval gate finds: `enableRequiredServices` turns on
# matrix and authelia but NOT nats (nats has no mode that enables it —
# see the auto-deploy question on the swarm-queue issue), so an ordinary
# all-local hive resolved authelia's two coordinates and not the queue
# URL. Two of three set is exactly what the assertion below rejects, so
# every `enableAllLocalDefaults` hive would have stopped evaluating.
#
# Deriving all three from one predicate makes the partial state
# unrepresentable rather than merely detected: a default set is all or
# nothing, and the assertion is then only ever about what an operator
# typed.
queueLocal = swarmCfg.nats.enable && swarmCfg.authelia.enable && cfg.hiveName != null;
in
{
options.services.hyperhive.swarm.hives = lib.mkOption {
@ -298,6 +315,43 @@ in
designed for it.
'';
}
{
# Deliberately an assertion and not a silent "then publish
# nothing": a half-set trio is a config an operator believes is
# working, and its runtime failure mode is the expensive one —
# the daemon comes up fine, never connects, and the hive reads
# `never_reported` on a dashboard nobody is watching yet.
#
# Safe to add to an existing deployment: every `statusPublish`
# default is either all-local or all-null, so no config that
# evaluates today can be caught by this. It also encodes a
# property of the code rather than an intended shape — hive-c0re
# genuinely cannot publish with two of three coordinates — which
# is the distinction the `serviceDomains'` comment at the top of
# this file was written about.
assertion =
let
set = lib.filter (v: v != null) [
swarmCfg.statusPublish.natsUrl
swarmCfg.statusPublish.tokenEndpoint
swarmCfg.statusPublish.clientSecretFile
];
in
builtins.length set == 0 || builtins.length set == 3;
message = ''
services.hyperhive.swarm.statusPublish needs natsUrl,
tokenEndpoint and clientSecretFile set together or not at all
— this hive has only some of them.
Currently:
natsUrl = ${toString swarmCfg.statusPublish.natsUrl}
tokenEndpoint = ${toString swarmCfg.statusPublish.tokenEndpoint}
clientSecretFile = ${toString swarmCfg.statusPublish.clientSecretFile}
Set the missing ones to publish this hive's status to the
swarm, or set all three to null to turn publishing off.
'';
}
];
};
@ -343,4 +397,84 @@ in
};
};
# How this hive reaches the swarm queue to offer its own status
# (hive-c0re's `swarm_status`). Three coordinates, defaulted from the
# local swarm services when this host runs them, and set by hand
# otherwise — one code path for both deployments.
#
# The alternative was the shape swarm-controller uses: emit the
# coordinates only when authelia and NATS are local, and nothing
# otherwise. That is right for the controller, which *is* a swarm-host
# service — but a hive is the one thing in a swarm that routinely is
# not on the swarm host, so the same rule would mean status publishing
# works on exactly the deployment that needs it least.
#
# There is no `enable`: three coordinates that are all set is the
# enable. An extra flag would let a hive be configured-but-off, which
# is one more state to explain and one more way to be silently quiet.
options.services.hyperhive.swarm.statusPublish = {
natsUrl = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = if queueLocal then "nats://127.0.0.1:${toString swarmCfg.nats.port}" else null;
defaultText = lib.literalExpression ''"nats://127.0.0.1:''${swarm.nats.port}" when this host runs the queue and the IdP, else null'';
example = "nats://10.100.0.1:4222";
description = ''
Where the swarm queue listens, as seen from *this* hive.
Defaults to loopback when this host runs the queue container
itself (it shares the host netns, so loopback is correct there
and is not the "localhost means the wrong thing" trap that
applies inside agent containers). A hive that is not the swarm
host has to name the swarm's mesh address.
Null disables status publishing: this hive computes its own
readiness as always, and simply offers it to nobody. The swarm
controller then reports it `never_reported`, which is the honest
reading.
'';
};
tokenEndpoint = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default =
if queueLocal && swarmCfg.authelia.url != null then
"${swarmCfg.authelia.url}/api/oidc/token"
else
null;
defaultText = lib.literalExpression ''"''${swarm.authelia.url}/api/oidc/token" when this host runs both the queue and the IdP, else null'';
example = "https://auth.example.com/api/oidc/token";
description = ''
The swarm IdP's OAuth2 token endpoint. This hive mints a
`client_credentials` access token there and presents it when
connecting to the queue, which authenticates it as
`hive-<hiveName>` — the client
{file}`nix/host-modules/swarm-authelia.nix` already declares for
every entry in {option}`services.hyperhive.swarm.hives`.
'';
};
clientSecretFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default =
if queueLocal then "${swarmCfg.authelia.hostClientSecretDir}/hive-${cfg.hiveName}.secret" else null;
defaultText = lib.literalExpression ''"''${swarm.authelia.hostClientSecretDir}/hive-''${hiveName}.secret" when this host runs both the queue and the IdP, else null'';
example = "/var/lib/secrets/swarm-queue-client.secret";
description = ''
Path to a file holding the plaintext client secret for this
hive's `hive-<hiveName>` identity.
A path and not a value: a secret in the Nix store is world
readable, and one in the environment is readable by anything
that can open {file}`/proc/<pid>/environ`.
Defaults to authelia's own minted secret when the IdP runs on
this host. On any other hive the secret has to get here somehow,
and the swarm does not distribute it — copy it out of the swarm
host's
{option}`services.hyperhive.swarm.authelia.hostClientSecretDir`
with whatever secret management this deployment already uses.
'';
};
};
}