vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

drop the shared hive agent login

Browse source 
Every agent now connects to the queue and reads logs with its own secret
from the store (#5181), so the per-hive `hive-<h>-agent` OIDC client has
no reader left. Remove it end to end:

- hive-c0re no longer forwards `hive-queue-agent-{secret,client-id}` into
  containers, and no longer forwards HIVE_AGENT_OIDC_TOKEN_ENDPOINT; the
  agent module that loaded them (nix/agent-modules/queue.nix) is gone, its
  options refused with a removal message.
- The host reader (glue-queue-agent-credential.nix, swarm-bao-queue-agent),
  its store role and policy unit (swarm-bao-queue-agent-policy), its PKI
  leaf and the deploy.bao.queueAgent* / queue.agentCredentialDir options
  are gone; the options are refused with a removal message.
- authelia mints no `hive-<h>-agent` client, and swarm-secret-publish no
  longer writes swarm/hives/<h>/queue/agent.
- swarm-secret-client drops queue::agent_client_path / queue::Credential.
- HIVE_AGENT_NATS_URL is now gated on the agents' queue address alone.

The log store's authelia check stays for operators. swarm-nats-auth still
refuses a `-agent` client id; that refusal and `agentClientSuffix` are left
for a follow-up.

Refs #4459
Refs #4765
This commit is contained in:
atlas 2026-10-13 09:49:32 +02:00 • committed by mara
commit 868c6cf96c
41 changed files with 286 additions and 1410 deletions
Download patch file Download diff file Expand all files Collapse all files

184
nix/module-eval/bao-matrix-reader.nix
View file

@ -31,18 +31,10 @@ let
# nothing here mints a leaf and the operator names one placed by hand. The
# deployment this pairing exists to serve, and the one that was previously
# inexpressible — the gate asked whether the store was a neighbour.
#
# 🩸 One leaf PER READER, and this fixture is where that cost is visible: the
# queue-credential unit identifies itself to the store separately from the
# hive, so an operator placing leaves by hand places one for it as well as
# the hive's own. Naming only `clientCertFile` here would leave the unit
# unrendered — which is what the arms below would then be asserting about.
baoRemoteReader = hiveWithController {
deploy.matrix.enable = true;
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
deploy.bao.queueAgentClientCertFile = "/etc/pki/bao-queue-agent.pem";
deploy.bao.queueAgentClientKeyFile = "/etc/pki/bao-queue-agent-key.pem";
};
# A homeserver on a hive with NO store identity at all — neither a local
@ -51,24 +43,15 @@ let
# fixture set its own, as ./lib.nix asks.
inherit (shared) matrixNoBaoIdentity;
# 🩸 The hand-configured remote reader that named the hive's leaf and stopped.
# `baoRemoteReader` above is the same deployment done right; this one holds
# the hive's own leaf, so it demonstrably reads the store, and is missing the
# queue reader's pair. Without the refusal it would render no queue reader
# and say nothing — which is what the refusal arms below are about.
# 🩸 The hand-configured remote reader that named the hive's leaf and stopped:
# it holds the hive's own leaf, so it demonstrably reads the store, and is
# missing matrix-ctl's pair.
baoRemoteReaderMissingLeaves = hiveWithController {
deploy.matrix.enable = true;
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
};
# The same omission on a hive that does NOT run a homeserver: the queue
# refusal checks only the hive's own toggle.
remoteReaderNoMatrixMissingLeaves = hiveWithController {
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
};
# The store's host with the hive's own reader pointed somewhere else.
matrixOnStoreHostReaderMoved = hiveWithController {
deploy.bao.enable = true;
@ -77,122 +60,15 @@ let
deploy.bao.clientKeyFile = "/etc/pki/h1-key.pem";
};
# Did a module refuse this host, and over which option. An assertion is a
# config VALUE until something forces it — `.config` never throws — so a
# fixture in a state the module refuses stays evaluable and the refusal reads
# back as data. Matched on the option name the message names rather than on
# its prose, because the option name is the part an operator has to act on
# and a message that stopped naming it would be the actual defect. Same shape
# as ./grafana.nix's `grafanaRefusedFor`, which reads `swarm-grafana.nix`'s
# own refusal.
refusedOver = m: option: lib.any (a: !a.assertion && lib.hasInfix option a.message) m.assertions;
cases = [
{
# A login failure is the store being unreachable, sealed, or not yet
# holding this host's role — all of which a retry fixes. A read that
# answers "nothing there" is not, so only the first is allowed to fail
# the unit.
name = "the queue credential reader retries a failed login and still degrades on an empty read";
ok =
let
u = baoWithMatrix.systemd.services.swarm-bao-queue-agent;
# Everything between the login's failure branch and the read's, which
# is where the exit that decides "retry or give up" lives.
afterLogin = lib.last (lib.splitString "bao login" u.script);
loginBranch = lib.head (lib.splitString "bao kv get" afterLogin);
in
u.serviceConfig.Restart or null == "on-failure"
&& u.startLimitBurst or 0 > 0
# The window has to outlast every attempt, or the burst is unreachable.
&& u.startLimitIntervalSec or 0 > (u.serviceConfig.RestartSec or 0) * (u.startLimitBurst or 0)
&& lib.hasInfix "exit 1" loginBranch
&& lib.hasInfix "exit 0" (lib.last (lib.splitString "bao kv get" u.script));
}
{
# The reader's own grant covers `swarm/hives/<this hive>/*`; a path
# outside it answers 403, not "no such key". This is the path
# `swarm_secret_client::queue::agent_client_path` builds from the same
# pieces. The negative arm is the rename this one is exposed to: a
# credential named for the queue rather than for the hive that presents
# it reads as correct and is refused on every boot.
name = "the agent queue credential path sits inside the prefix the reader is granted";
ok =
let
s = baoRemoteReader.systemd.services.swarm-bao-queue-agent.script;
in
lib.hasInfix "secret/swarm/hives/h1/queue/agent" s && !(lib.hasInfix "secret/swarm/queue/" s);
}
{
# The unit's output is the option's value, not a literal that agrees with
# it today: an operator moving the directory has to move both files. The
# prefix is asserted too because `hasInfix ""` is true — an option
# renamed out from under this arm would otherwise read empty and pass.
name = "the queue credential reader writes both files under the directory its option names";
ok =
let
m = baoRemoteReader;
dir = toString m.services.hyperhive.deploy.hive-controller.queue.agentCredentialDir;
s = m.systemd.services.swarm-bao-queue-agent.script;
in
lib.hasPrefix "/var/lib/" dir
&& lib.hasInfix "${dir}/secret" s
&& lib.hasInfix "${dir}/client_id" s;
}
{
# No agent container may render before this unit has had its attempts,
# and the edge that guarantees it must delay hive-c0re rather than sink
# it: an unreachable store is this unit's `Restart=on-failure` window,
# not a reason for the daemon that renders every agent to fail its own
# start.
name = "the queue credential reader orders before hive-c0re and is wanted, not required, by it";
ok =
let
u = baoRemoteReader.systemd.services.swarm-bao-queue-agent;
in
builtins.elem "hive-c0re.service" (u.before or [ ])
&& builtins.elem "hive-c0re.service" (u.wantedBy or [ ])
&& !(builtins.elem "hive-c0re.service" (u.requiredBy or [ ]))
&& !(builtins.elem "hive-c0re.service" (u.requires or [ ]));
}
{
# Nothing local mints certificates off-host, so this orders against
# nothing there.
name = "an off-host reader names no unit the store's host would have provided";
# Membership first, then the value: indexing a missing unit throws, and a
# table that reports which property broke must not be the thing that dies.
ok =
let
s = baoRemoteReader.systemd.services;
u = s.swarm-bao-queue-agent;
in
s ? swarm-bao-queue-agent && !(lib.elem "swarm-bao-pki.service" (u.after ++ u.wants ++ u.requires));
}
{
# Presence control for the case above: the list is conditional, not gone.
# Wanted, never required: a refused pki start fails a requirer's job with
# `dependency`, which its `Restart=on-failure` never retries.
name = "a co-located reader still orders after and wants the local pki unit, and does not require it";
ok =
let
s = baoWithMatrix.systemd.services;
u = s.swarm-bao-queue-agent;
in
s ? swarm-bao-queue-agent
&& lib.elem "swarm-bao-pki.service" u.after
&& lib.elem "swarm-bao-pki.service" u.wants
&& u.requires == [ ];
}
{
# Nothing asserted this script before, which is how it kept a branch that
# named three states and threw away the only thing telling them apart. A
# missing value, a refused identity and an unreachable host all end in the
# same degraded mode here, correctly — what must survive is which one.
name = "the queue credential reader carries the store's own diagnostic into the journal";
ok =
let
s = baoWithMatrix.systemd.services.swarm-bao-queue-agent.script;
in
!(lib.hasInfix "2>/dev/null" s) && lib.hasInfix ''cat "''$err"'' s;
# A hive holds no shared queue credential for its agents, wherever the
# store runs. The control: hive-c0re renders on both hosts.
name = "no hive renders a reader for a hive-shared agent queue credential";
ok = lib.all (m: !(m.systemd.services ? swarm-bao-queue-agent) && m.systemd.services ? hive-c0re) [
baoWithMatrix
baoRemoteReader
];
}
{
# hive-c0re runs as hive-core and the client key is `0600` root-owned
@ -404,46 +280,6 @@ let
&& !(c.systemd.services ? swarm-matrix-appservice-publish)
&& !(lib.any (lib.hasPrefix "swarm-appservice.yaml") c.systemd.services.tuwunel.serviceConfig.LoadCredential);
}
{
# 🩸 The arm the whole refusal exists for. The reader is gated on its own
# leaf, so the way this regresses is the build going green and the unit
# not rendering — which no presence check on a rendered unit can see,
# because the unit that is missing is the evidence. Read as a refusal
# naming each option, so an operator acts on the message without opening
# the nix.
name = "a remote reader missing the queue reader's leaf is refused, naming both options";
ok =
refusedOver baoRemoteReaderMissingLeaves "queueAgentClientCertFile"
&& refusedOver baoRemoteReaderMissingLeaves "queueAgentClientKeyFile";
}
{
name = "the queue refusal needs no homeserver";
ok = refusedOver remoteReaderNoMatrixMissingLeaves "queueAgentClientCertFile";
}
{
# ⚠️ The arm that keeps the refusal from being worse than the silence it
# replaced. A hive with NO store identity is the supported no-store
# deployment and also the state an operator passes through bringing a hive
# up — neither may fail to evaluate. Asserted as "no refusal names either
# option", not as "this one fixture is fine", because the way this breaks
# is a gate widened to the principal's leaf alone.
name = "a hive with no store identity at all is refused over none of the per-principal leaves";
ok = lib.all (option: !(refusedOver matrixNoBaoIdentity option)) [
"queueAgentClientCertFile"
"queueAgentClientKeyFile"
];
}
{
# The other half of the same guard, and the one an operator meets far more
# often: on the store's own host ./host-modules/glue-bao-tls.nix mkDefaults
# every pair, so there is nothing to name and nothing to refuse. Paired with
# the fully-named remote reader, which is the same deployment done by hand.
name = "neither a store host nor a correctly-named remote reader is refused";
ok = lib.all (m: !(refusedOver m "queueAgentClientCertFile")) [
baoWithMatrix
baoRemoteReader
];
}
];
in
runGroup "bao-matrix-reader" cases