vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

swarm: give agent containers their own queue principal

Browse source 
Agents have authelia *users*; they had no machine identity at all, so an
agent could not authenticate to the swarm queue as anything. This mints
one `agent-<hive>` OIDC client per hive beside the existing
`hive-<hive>` one, teaches the auth-callout responder an agent arm, and
opens the queue's client port on the bridge so a container can reach it.

One client per HIVE, not per agent: agents are created at runtime, and a
per-agent client would make creating one a config change plus an
authelia reload. The cost is that agents on a hive are indistinguishable
to the broker, which is deliberate and tracked separately.

The agent grant is deny-by-default twice over. An agent id matches no
hive rule, so it gets a hive's status-key grant from neither; and with
no agent subject configured the responder returns no grant at all rather
than an empty publish list, which would be a denial wearing a grant's
shape. What an agent may publish is a deployment's decision, taken
through `--agent-publish-subject` the same way `--hive-publish-subject`
already works.

`Policy::new` now refuses two prefixes where one contains the other. The
arms are tried in order, so that overlap does not error at match time -
it silently hands one principal the other's grant.

Not shipped here, and neither is reachable without it: no subject is
configured for agents anywhere in nix, and nothing yet delivers
`agent-<hive>.secret` into an agent container. Both belong to the stream
that will be the first consumer.
This commit is contained in:
atlas 2026-08-31 17:30:00 +02:00 • committed by mara
commit fe9417ae52
5 changed files with 301 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

66
nix/host-modules/swarm-authelia.nix
View file

@ -143,6 +143,33 @@ let
accessTokenSignedResponseAlg = "RS256";
}) hyperhiveCfg.swarm.hives;
# The identity an agent container presents to the swarm's queue. One per
# HIVE, not per agent, and that is the load-bearing choice rather than a
# shortcut: agents are created at **runtime**, so anything minted per agent
# would make creating one a config change plus an authelia reload. Keyed on
# the hive, this list's length tracks the roster above it — deploy-time, like
# every other entry here.
#
# ⚠️ The cost, stated rather than discovered later: every agent on a hive
# presents the SAME client id, so the broker can tell *hives* apart and not
# *agents*. Deliberate and deferred — closing it requires the client list to
# stop being static, which is the same problem the users-database writer
# already solves for identities.
#
# Mirrors `hiveClients` field for field so the two stay comparable. The
# signing algorithm is not load-bearing here — this client's only consumer is
# the queue's auth-callout responder, which *introspects* rather than
# verifying offline — but it matches its sibling rather than inventing a
# second answer to a question nobody asked.
agentClients = lib.mapAttrsToList (name: _: {
id = "${cfg.agentClientPrefix}${name}";
description = "HyperHive agents on hive ${name}";
kind = "machine";
redirectUris = [ ];
audience = [ "${cfg.agentClientPrefix}${name}" ];
accessTokenSignedResponseAlg = "RS256";
}) hyperhiveCfg.swarm.hives;
# `swarm-authelia-bridge`'s own identity — distinct from
# `swarm-controller`'s (`swarm-controller.nix`'s `queueClientId`). A
# resource server introspecting a token proves its OWN identity to the
@ -416,9 +443,21 @@ in
default = deployCfg.nats.enable;
defaultText = lib.literalExpression "services.hyperhive.deploy.nats.enable";
description = ''
Mint one machine client per hive in
Mint machine clients per hive in
{option}`services.hyperhive.swarm.hives`, so each hive can
authenticate to swarm services as itself.
authenticate to swarm services as itself: `hive-<name>` for the
hive's own daemons, and `agent-<name>` for the agent containers
running on it.
Two clients rather than one because they are not the same
principal — a hive's daemons run on the host and an agent runs
in a container the host hands a credential to, so a swarm
service has to be able to grant them different things. It is
one client per *hive* on the agent side, not per agent: agents
are created at runtime, and a per-agent client would make
creating one a config change plus an authelia reload. The cost
is that agents on a hive are indistinguishable from each other,
tracked as a follow-up rather than papered over.
Defaults to whether the swarm message queue is enabled, because
that is the first service that needs a hive to prove who it is.
@ -667,6 +706,27 @@ in
'';
};
agentClientPrefix = lib.mkOption {
type = lib.types.str;
readOnly = true;
default = "agent-";
description = ''
Prefix of the OAuth2 client id minted for the *agents* of each hive
in `services.hyperhive.swarm.hives` — agents on hive `alpha` all
present
`${config.services.hyperhive.swarm.authelia.agentClientPrefix}alpha`.
Read-only for the same reason as `hiveClientPrefix`, and read by the
same consumer with the same failure mode: a split spelling denies
every agent as a timeout.
One id per hive rather than per agent, because agents are created at
runtime and a per-agent client would make creating one a config
change plus a reload. The consequence is that this identity says
*which hive* an agent belongs to and never *which agent* — the
broker cannot tell two agents on one hive apart.
'';
};
machine = lib.mkOption {
type = lib.types.str;
readOnly = true;
@ -828,7 +888,7 @@ in
# wherever this module is, so `oidc.clients` is never actually empty
# — see `oidcEnabled`'s comment above.
services.hyperhive.swarm.authelia.oidc.clients =
lib.optionals cfg.oidc.hiveIdentities hiveClients
lib.optionals cfg.oidc.hiveIdentities (hiveClients ++ agentClients)
++ [ bridgeClient ];
# A redirect URI on a machine client is not harmless-but-unused: it

36
nix/host-modules/swarm-nats.nix
View file

@ -233,6 +233,17 @@ in
TCP port the queue listens on. 4222 is upstream's default and
sits outside hyperhive's claimed ranges (dashboard 7000, forge
3000, matrix 8008, every agent in 8100..8999 via FNV-1a hash).
Contributed to
`services.hyperhive.network.exposeHostPorts`, which opens it on
the bridge interface only — so it is reachable from agent
containers and not from the outside world. An agent connects to
`nats://<bridgeIp>:<port>`; loopback inside a container is the
agent itself, not this host.
Unlike {option}`monitorPort` and {option}`metricsPort`, the
address is not what bounds who may use this port: the queue's
`auth_callout` refuses every client it cannot identify.
'';
};
@ -564,6 +575,26 @@ in
nats = "127.0.0.1:${toString cfg.metricsPort}";
};
# Open the client port on the bridge, so agent containers can reach
# the queue. `privateNetwork = false` below means the server binds in
# the host netns, which is the precondition this option names — but
# sharing a netns is not reachability: the bridge interface is
# default-deny, so without this an agent's connect attempt is dropped
# by the firewall and looks exactly like every other NATS failure,
# a timeout.
#
# ⚠️ Bridge interface only — never the world. What makes it safe to
# open at all is that the queue is fail-closed: `auth_callout` admits
# nobody until the responder above answers for them, so an agent that
# reaches this port still has to present a token authelia vouches for.
#
# An agent connects to `nats://<bridgeIp>:<port>`, NOT to
# `nats://127.0.0.1:<port>` — inside a container loopback is the
# *agent*. Every url in this repo today is the loopback one and each
# is correct for its reader, because those readers share the host
# netns; an agent does not.
services.hyperhive.network.exposeHostPorts = [ cfg.port ];
containers.swarm-nats = {
autoStart = true;
ephemeral = false;
@ -730,14 +761,15 @@ in
# be the same string — which is why both come from one let.
"--account ${lib.escapeShellArg clientAccount}"
"--introspection-url ${lib.escapeShellArg introspectionUrl}"
# Both of these name a principal some OTHER module mints,
# so both are read out of that module rather than spelled
# Each of these names a principal some OTHER module mints,
# so each is read out of that module rather than spelled
# again here — same argument as `--account` above, one
# level wider. The responder denies a client id it does
# not recognise, and a NATS denial arrives as a timeout,
# so a drift here is silent at the point of change and
# misattributed at the point of failure.
"--hive-client-prefix ${lib.escapeShellArg autheliaCfg.hiveClientPrefix}"
"--agent-client-prefix ${lib.escapeShellArg autheliaCfg.agentClientPrefix}"
"--reader-client ${lib.escapeShellArg controllerCfg.queueClientId}"
];
# Every credential arrives by `LoadCredential` and is named