vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

nix: derive hive identities and the token endpoint from the swarm, not this host

Browse source 
Two swarm-wide facts were being read off this machine's deploy set, so the
answer differed between two hosts of one swarm:

  - `swarm.authelia.oidc.hiveIdentities` defaulted to `deploy.nats.enable`,
    so whether a hive gets an identity at all depended on whether the IdP
    host happened to also run the queue. It is on by default now: a swarm's
    hives have identities, and the clients are inert until used.

  - `swarm.statusPublish.tokenEndpoint` defaulted through `queueLocal`
    (`deploy.nats.enable && deploy.authelia.enable`), so a hive that was not
    the swarm host had no token endpoint even when the swarm's IdP was
    reachable and named. It follows `swarm.authelia.url` now — the same
    derivation `swarm-controller.nix`'s own `queue.tokenEndpoint` already
    uses, which is correct for a remote provider.

`deploy.nix:1-30` is what makes this a rule rather than a preference:
`swarm.*` is "identical on every host, byte for byte" and `deploy.*` is
"necessarily different on every host". A swarm value derived from a deploy
value cannot satisfy both.

The all-or-nothing status-publish assertion follows: the token endpoint is
no longer one of the coordinates that says this hive publishes — every hive
in a swarm with an IdP has one — so the two per-host coordinates are what
must agree, and they now require the endpoint rather than being counted
beside it.

`queueLocal` itself stays for the three remaining host-local addresses
(`natsUrl`, `clientSecretFile`, `agentNatsUrl`): each of those is a
`deploy.*` value that genuinely differs per host.

Closes #4048
This commit is contained in:
atlas 2026-09-19 14:14:54 +02:00 • committed by mara
commit 97cde357e5
4 changed files with 48 additions and 50 deletions
Download patch file Download diff file Expand all files Collapse all files

14
nix/host-modules/swarm-authelia.nix
View file

@ -479,8 +479,7 @@ in
oidc.hiveIdentities = lib.mkOption {
type = lib.types.bool;
default = deployCfg.nats.enable;
defaultText = lib.literalExpression "services.hyperhive.deploy.nats.enable";
default = true;
description = ''
Mint machine clients per hive in
{option}`services.hyperhive.swarm.hives`, so each hive can
@ -498,12 +497,11 @@ in
is that agents on a hive are indistinguishable from each other,
tracked as a follow-up rather than papered over.
Defaults to whether the swarm message queue is enabled, because
that is the first service that needs a hive to prove who it is.
It is an option rather than a hard-coded condition so a second
consumer — the swarm telemetry collector — can turn it on
without the queue, and so a swarm that wants the identities
provisioned ahead of either can say so.
On by default: a swarm's hives have identities, and that is a
fact about the swarm rather than about any one host. It used to
default to whether the queue ran on *this* machine, which made
the answer differ between two hosts of one swarm — set it false
for a swarm whose hives authenticate to nothing.
The clients are inert until something authenticates with them:
each is a client id and a secret sitting on this host. What

7
nix/host-modules/swarm-otel.nix
View file

@ -626,10 +626,9 @@ in
};
};
# Turning on the identities this tier authenticates against, which the
# option exists to allow: its own description names this module as the
# second consumer, so the queue is not a prerequisite for authenticated
# telemetry.
# The identities this tier authenticates against. Stated rather than
# assumed: the option is an operator's to turn off, and this tier does
# not work without it.
services.hyperhive.swarm.authelia.oidc.hiveIdentities = true;
# The collector's own identity, for the other direction: the hive

62
nix/host-modules/swarm.nix
View file

@ -99,26 +99,16 @@ let
lib.filterAttrs (_: hive: hive.certFingerprint != null) swarmCfg.hives
);
# Whether this host can derive its own status-publishing coordinates —
# ONE condition for all three of them, deliberately.
# Whether the queue and its minted secret are on THIS host — the two
# coordinates that are a statement about this machine's disk and
# netns, from one condition so a partial set is unrepresentable rather
# than merely detected.
#
# 🩸 They were three independent conditions first, and that was wrong in
# a way only an eval gate finds: at the time `allSwarmServices`
# turned on matrix and authelia but NOT nats, so an ordinary all-local
# hive resolved authelia's two coordinates and not the queue URL. Two of
# three set is exactly what the assertion below rejects, so every
# `singleHostSwarm` hive would have stopped evaluating.
#
# (The queue does derive from that switch now — ./swarm-required-services.nix
# — so the original asymmetry is gone. The reasoning below is why the
# single predicate stays anyway: it makes the partial state
# unrepresentable rather than merely detected, which does not depend on
# which services happen to derive together this month.)
#
# Deriving all three from one predicate makes the partial state
# unrepresentable rather than merely detected: a default set is all or
# nothing, and the assertion is then only ever about what an operator
# typed.
# ⚠️ The token endpoint is NOT one of them any more. It is the swarm's
# one address, derived from `swarm.authelia.url` like
# ./swarm-controller.nix's own `queue.tokenEndpoint` already is, so it
# is correct for a remote provider and does not ask where anything
# runs.
queueLocal = deployCfg.nats.enable && deployCfg.authelia.enable && cfg.hiveName != null;
in
{
@ -385,15 +375,21 @@ in
# machine's. The message spells all three paths out, because an
# operator told only the option names would look for them under
# one prefix and find one of them.
#
# ⚠️ Asymmetric on purpose. A token endpoint on its own is the
# normal state of every hive in a swarm that has an IdP — it says
# where the IdP is, not that this hive publishes. Only the two
# per-host coordinates are a claim to be publishing, and they
# need the endpoint to mean anything.
assertion =
let
set = lib.filter (v: v != null) [
local = lib.filter (v: v != null) [
deployCfg.hive-controller.statusPublish.natsUrl
swarmCfg.statusPublish.tokenEndpoint
deployCfg.hive-controller.statusPublish.clientSecretFile
];
in
builtins.length set == 0 || builtins.length set == 3;
builtins.length local == 0
|| (builtins.length local == 2 && swarmCfg.statusPublish.tokenEndpoint != null);
message = ''
This hive's status-publishing coordinates have to be set
together or not at all — it has only some of them.
@ -492,16 +488,10 @@ in
};
# How this hive reaches the swarm queue to offer its own status
# (hive-c0re's `swarm_status`). Three coordinates, defaulted from the
# local swarm services when this host runs them, and set by hand
# otherwise — one code path for both deployments.
#
# The alternative was the shape swarm-controller uses: emit the
# coordinates only when authelia and NATS are local, and nothing
# otherwise. That is right for the controller, which *is* a swarm-host
# service — but a hive is the one thing in a swarm that routinely is
# not on the swarm host, so the same rule would mean status publishing
# works on exactly the deployment that needs it least.
# (hive-c0re's `swarm_status`). Three coordinates: the token endpoint
# below follows the swarm's IdP address wherever that is, and the two
# under `deploy.hive-controller` default from the local swarm services
# when this host runs them and are set by hand otherwise.
#
# There is no `enable`: three coordinates that are all set is the
# enable. An extra flag would let a hive be configured-but-off, which
@ -509,12 +499,8 @@ in
options.services.hyperhive.swarm.statusPublish = {
tokenEndpoint = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default =
if queueLocal && swarmCfg.authelia.url != null then
"${swarmCfg.authelia.url}/api/oidc/token"
else
null;
defaultText = lib.literalExpression ''"''${swarm.authelia.url}/api/oidc/token" when this host runs both the queue and the IdP, else null'';
default = if swarmCfg.authelia.url != null then "${swarmCfg.authelia.url}/api/oidc/token" else null;
defaultText = lib.literalExpression ''"''${swarm.authelia.url}/api/oidc/token" when that URL is set, else null'';
example = "https://auth.example.com/api/oidc/token";
description = ''
The swarm IdP's OAuth2 token endpoint. This hive mints a

15
nix/module-eval.nix
View file

@ -1266,6 +1266,21 @@ let
secretPublisherHere.systemd.services.swarm-secret-publish.script
);
}
{
# Both halves of the co-location assumption, which was one host's
# `deploy.*` answering a question about the whole swarm: the identities
# were minted only where the queue happened to run, and the token
# endpoint was known only where the IdP happened to run.
name = "hive identities and the token endpoint do not depend on which host runs what";
ok =
let
autheliaNoQueue = hive { deploy.authelia.enable = true; };
in
lib.elem "hive-h1" (map (c: c.id) autheliaNoQueue.services.hyperhive.swarm.authelia.oidc.clients)
&&
grafanaRemoteAuthelia.services.hyperhive.swarm.statusPublish.tokenEndpoint
== "https://auth.example.invalid/api/oidc/token";
}
{
# Registering the client cannot live where the rest of grafana's module
# lives: that block is gated on this host RUNNING grafana, so on the split