vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

matrix: drop the per-hive appservice and its unused token

Browse source 
Every hive's homeserver loaded a per-hive `hyperhive` appservice
registration, and the swarm minted and delivered an as_token for it
through bao. No client presented that token: swarm-controller creates
every `@hive-<hive>` sender and every agent account with the swarm
appservice token (hive_sender.rs, agent_token.rs), and no Rust code read
the per-hive one.

Removed, with everything that existed only to serve it:
- hive-matrix.nix: the `hyperhive` registration, its activation-time
  render, the host bind mount of /var/lib/hyperhive/matrix-appservice,
  its LoadCredential entry, and the internal options
  `deploy.matrix.appserviceTokenFile` / `appserviceRegistrationScript`.
  tuwunel now loads only the swarm registration.
- glue-matrix-bao-token.nix and its `swarm-bao-matrix-token` unit.
- glue-bao-tls.nix: the matrix-token leaf and its defaults.
- swarm-bao.nix: the `swarm-matrix-token-<hive>` roles and policies
  (`swarm-bao-matrix-token-policy`), and the options
  `deploy.bao.matrixTokenClient{Cert,Key}File` and
  `matrixTokenCommonNamePrefix`, now mkRemovedOptionModule.
- swarm-secret-publisher.nix: the per-hive token mint and its state dir.
- swarm.nix: the reserved `<matrixTokenCommonNamePrefix>-<hive>` CN.
- swarm-secret-client: `matrix::appservice_token_path`.

The swarm registration (swarm-matrix-ctl, swarm-controller) is untouched.
Comments and docs that pointed at the removed glue file now point at
glue-queue-agent-credential.nix, which has the same reader shape.

Refs #4428
This commit is contained in:
atlas 2026-10-04 22:52:22 +02:00 • committed by mara
commit 48067d07e8
31 changed files with 362 additions and 1233 deletions
Download patch file Download diff file Expand all files Collapse all files

112
nix/module-eval/bao-grants.nix
View file

@ -87,20 +87,19 @@ let
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
deploy.bao.clientCaFile = lib.mkForce null;
};
# The store plus every one of the four readers that used to log in as the
# hive. One fixture rather than four: the claim they are four *separate*
# principals is only testable where all four render at once — that is the
# deployment in which two of them sharing a leaf would be invisible.
# The store plus every reader that used to log in as the hive. One fixture
# rather than one per reader: the claim they are *separate* principals is
# only testable where all of them render at once — that is the deployment in
# which two of them sharing a leaf would be invisible.
baoGrantWithConsumers = hive {
deploy.bao.enable = true;
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
deploy.matrix.enable = true;
deploy.grafana.enable = true;
deploy.swarm-otel.enable = true;
};
# The store with none of the four readers beside it. Grafana, the collector and
# the homeserver are simply off; the queue reader renders on any host holding
# The store with none of the readers beside it. Grafana and the collector are
# simply off; the queue reader renders on any host holding
# its leaf, which ./glue-bao-tls.nix mints here, so that leaf is taken away.
baoGrantNoReaders = hive {
deploy.bao.enable = true;
@ -117,16 +116,13 @@ let
deploy.bao.forwarderOidcClientKeyFile = lib.mkForce null;
};
# All four readers against a store they do not run, each with a leaf placed
# by hand. The deployment in which there is no local policy unit to wait for.
# Every reader against a store it does not run, each with a leaf placed by
# hand. The deployment in which there is no local policy unit to wait for.
baoRemoteReaders = hive {
deploy.matrix.enable = true;
deploy.grafana.enable = true;
deploy.swarm-otel.enable = true;
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
deploy.bao.matrixTokenClientCertFile = "/etc/pki/bao-matrix-token.pem";
deploy.bao.matrixTokenClientKeyFile = "/etc/pki/bao-matrix-token-key.pem";
deploy.bao.queueAgentClientCertFile = "/etc/pki/bao-queue-agent.pem";
deploy.bao.queueAgentClientKeyFile = "/etc/pki/bao-queue-agent-key.pem";
deploy.bao.grafanaOidcClientCertFile = "/etc/pki/bao-grafana-oidc.pem";
@ -135,10 +131,9 @@ let
deploy.bao.otelOidcClientKeyFile = "/etc/pki/bao-otel-oidc-key.pem";
};
# The four readers ./glue-bao-readers-policy-order.nix orders after their
# policy units.
# The readers ./glue-bao-readers-policy-order.nix orders after their policy
# units.
policyReaders = [
"swarm-bao-matrix-token"
"swarm-bao-queue-agent"
"swarm-bao-grafana-oidc"
"swarm-bao-otel-oidc"
@ -166,13 +161,12 @@ let
_: u: (u.environment.BAO_CLIENT_CERT or null) == granterCertFile
) baoGrantWithConsumers.systemd.services;
# The fourteen units that write a `swarm-*` grant, by name, for the discovery
# The thirteen units that write a `swarm-*` grant, by name, for the discovery
# control below.
grantingUnitNames = [
"swarm-bao-controller-policy"
"swarm-bao-secret-publisher-policy"
"swarm-bao-matrix-ctl-policy"
"swarm-bao-matrix-token-policy"
"swarm-bao-queue-agent-policy"
"swarm-bao-grafana-oidc-policy"
"swarm-bao-otel-oidc-policy"
@ -479,11 +473,11 @@ let
&& !(baoGrantHere.containers.swarm-bao.config.systemd.services ? swarm-bao-matrix-ctl-policy);
}
# ── the four readers that used to share the hive's own leaf ──────────────
# ── the readers that used to share the hive's own leaf ───────────────────
#
# 🩸 Until this split all four presented `deploy.bao.clientCertFile`, whose
# policy granted read on `swarm/agents/*`, `swarm/hives/<hive>/*` AND
# `swarm/services/*`. Four principals behind one certificate are one
# 🩸 Until this split all of them presented `deploy.bao.clientCertFile`,
# whose policy granted read on `swarm/agents/*`, `swarm/hives/<hive>/*` AND
# `swarm/services/*`. Principals behind one certificate are one
# principal to bao, so the only expressible grant was the union: the unit
# fetching Grafana's OIDC secret could fetch every agent credential in the
# swarm.
@ -493,30 +487,6 @@ let
# just as well when the other two stanzas are still there beside it. The
# arms pin what each principal must NOT reach, so a later widening fails
# here instead of being noticed in a store.
{
name = "the matrix-token reader's grant is one hive's appservice token and nothing else";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-matrix-token-policy.script;
in
lib.hasInfix "path \"secret/data/swarm/hives/h1/matrix/appservice-token\" {" s
&& lib.hasInfix "capabilities = [\"read\"]" s
# The three stanzas the hive's own leaf carried, none of which this
# principal needs: every agent's credential, every service's OIDC
# client, and the rest of its own hive's tree — including the queue
# credential its sibling reader fetches.
&& !(lib.hasInfix "secret/data/swarm/agents" s)
&& !(lib.hasInfix "secret/data/swarm/services" s)
&& !(lib.hasInfix "secret/data/swarm/hives/h1/*" s)
&& !(lib.hasInfix "secret/data/swarm/hives/h1/queue" s)
# A `hives/*` wildcard would serve every hive from one role and let any
# hive read any other's token — the reach this split exists to remove,
# not to create.
&& !(lib.hasInfix "secret/data/swarm/hives/*" s)
# Nothing may rewrite the policy constraining it, for the reason the
# controller's own `hive-*` narrowing above gives.
&& !(lib.hasInfix "sys/policies/acl" s);
}
{
name = "the queue-credential reader's grant is one hive's queue credential and nothing else";
ok =
@ -616,13 +586,13 @@ let
{
# 🩸 The half that makes the policies above bind: a policy grants only
# through a token that carries it, and a token is minted by a cert-auth
# role matching a CN. Four distinct subjects is the whole mechanism — one
# subject for four readers is one principal however the policies read.
# role matching a CN. Distinct subjects is the whole mechanism — one
# subject for every reader is one principal however the policies read.
#
# The per-hive subjects carry the hive name because their paths do; the
# two service subjects do not, because an OIDC client is registered once
# The per-hive subject carries the hive name because its path does; the
# service subjects do not, because an OIDC client is registered once
# per swarm. Pinned so neither shape is tidied into the other.
name = "each of the five readers logs in under a subject of its own";
name = "each of the four readers logs in under a subject of its own";
ok =
let
subjectOf =
@ -637,8 +607,7 @@ let
# the reason the three service principals above state.
&& !(lib.hasInfix "auth/cert/certs/hive-" s);
in
subjectOf "swarm-bao-matrix-token-policy" "swarm-matrix-token-h1" "swarm-bao-matrix-token-h1"
&& subjectOf "swarm-bao-queue-agent-policy" "swarm-queue-agent-h1" "swarm-bao-queue-agent-h1"
subjectOf "swarm-bao-queue-agent-policy" "swarm-queue-agent-h1" "swarm-bao-queue-agent-h1"
&& subjectOf "swarm-bao-grafana-oidc-policy" "swarm-grafana-oidc" "swarm-bao-grafana-oidc"
&& subjectOf "swarm-bao-otel-oidc-policy" "swarm-otel-oidc" "swarm-bao-otel-oidc"
&& subjectOf "swarm-bao-forwarder-oidc-policy" "swarm-forwarder-oidc" "swarm-bao-forwarder-oidc";
@ -652,7 +621,7 @@ let
#
# Each pair is asserted whole: a certificate with no key authenticates
# nothing, so a half-set pair is a reader that does not render.
name = "each of the five readers presents its own leaf, never the hive's";
name = "each of the four readers presents its own leaf, never the hive's";
ok =
let
b = baoGrantWithConsumers.services.hyperhive.deploy.bao;
@ -661,8 +630,6 @@ let
b.clientKeyFile
];
own = [
b.matrixTokenClientCertFile
b.matrixTokenClientKeyFile
b.queueAgentClientCertFile
b.queueAgentClientKeyFile
b.grafanaOidcClientCertFile
@ -680,7 +647,6 @@ let
lib.all (p: p != null) own
&& !(lib.any (p: lib.elem p hiveLeaf) own)
&& lib.length (lib.unique own) == lib.length own
&& presents "swarm-bao-matrix-token" b.matrixTokenClientCertFile b.matrixTokenClientKeyFile
&& presents "swarm-bao-queue-agent" b.queueAgentClientCertFile b.queueAgentClientKeyFile
&& presents "swarm-bao-grafana-oidc" b.grafanaOidcClientCertFile b.grafanaOidcClientKeyFile
&& presents "swarm-bao-otel-oidc" b.otelOidcClientCertFile b.otelOidcClientKeyFile
@ -689,7 +655,7 @@ let
{
# The minting side of the same claim. A role matching a subject nothing
# signs is a reader that cannot log in, so the leaves and the roles have
# to be asserted against each other — and the two per-hive leaves carry
# to be asserted against each other — and the per-hive leaf carries
# THIS host's hive name, which is what makes one hive's leaf useless
# against another hive's role.
name = "the PKI unit signs a leaf per reader, each under that reader's own subject";
@ -702,8 +668,6 @@ let
# literal spanning both would pin this file's line wrapping rather than
# the pairing it means to.
lib.all (lib.flip lib.hasInfix s) [
"/matrix-token.pem ]"
"swarm-bao-matrix-token-h1 \"\" clientAuth"
"/queue-agent.pem ]"
"swarm-bao-queue-agent-h1 \"\" clientAuth"
"/grafana-oidc.pem ]"
@ -733,11 +697,10 @@ let
# client certificate and the host is the side that has one, so a unit
# rendered inside the store's container would have neither an identity
# nor a route. Plus the ordering that makes the mounts exist first.
name = "the five readers' granting units are ordered after the mounts and rendered on the host";
name = "the four readers' granting units are ordered after the mounts and rendered on the host";
ok =
let
units = [
"swarm-bao-matrix-token-policy"
"swarm-bao-queue-agent-policy"
"swarm-bao-grafana-oidc-policy"
"swarm-bao-otel-oidc-policy"
@ -759,7 +722,7 @@ let
# The forwarder is listed apart from `policyReaders`: it renders wherever
# the store does, so it is never absent on a store host and never present
# on a remote one, and the two cases below would fail on it for that.
name = "each of the five readers is ordered after the unit writing its role";
name = "each of the four readers is ordered after the unit writing its role";
ok =
let
s = baoGrantWithConsumers.systemd.services;
@ -778,7 +741,7 @@ let
# The ordering is set apart from each reader's own definition, so it can
# define a reader by itself: `after` on a unit nothing else declares is a
# unit with no ExecStart. On the store's host without the readers, none
# of the four may exist.
# of them may exist.
name = "a store host without the readers gains no reader unit from their ordering";
ok = lib.all (reader: !(baoGrantNoReaders.systemd.services ? ${reader})) policyReaders;
}
@ -800,24 +763,24 @@ let
}
{
# A store host without the granter's pair writes its grants some other
# way, so none of the fourteen units may exist. Without this arm
# way, so none of the thirteen units may exist. Without this arm
# `lib.mkIf haveGranter` could be dropped from any of them and every other
# case here would still pass.
name = "without the granter's pair none of the fourteen granting units render";
name = "without the granter's pair none of the thirteen granting units render";
ok =
let
s = baoGranterOptOut.systemd.services;
in
lib.all (unit: !(s ? ${unit})) (grantingUnitNames ++ [ "swarm-bao-granter-role" ])
# The control: the same store with the pair renders all fourteen.
# The control: the same store with the pair renders all thirteen.
&& lib.all (unit: baoGrantHere.systemd.services ? ${unit}) grantingUnitNames;
}
{
# 🩸 What replaced the silent skip. With no bootstrap token the fourteen still
# 🩸 What replaced the silent skip. With no bootstrap token the thirteen still
# render, and a refused granter fails them with the step that fixes it.
# A store host that never named a token is told to name one, since the
# unit that sets the granter up renders only where it has.
name = "a store host without a bootstrap token renders the fourteen, each failing loudly with the one-time step";
name = "a store host without a bootstrap token renders the thirteen, each failing loudly with the one-time step";
ok =
let
s = baoGranterNoToken.systemd.services;
@ -1247,8 +1210,8 @@ let
}
{
# `matrix_account::hive_sender` writes every hive's sender token, and
# nothing else under `hives/`: a hive's appservice token sits beside
# it. `+` is one segment, which keeps this to the one leaf; a `*` is a
# nothing else under `hives/`, where each hive's queue credential also
# lives. `+` is one segment, which keeps this to the one leaf; a `*` is a
# glob only at the end of a path. Pinned as the whole stanza, so an
# added capability fails.
name = "the controller writes each hive's sender token and nothing else under hives";
@ -1392,8 +1355,8 @@ let
}
{
# What makes the case above mean something: discovery by the granter's
# certificate reaches all fourteen units, and each yields calls.
name = "the granter-policy check sees all fourteen granting units, and parses calls from each";
# certificate reaches all thirteen units, and each yields calls.
name = "the granter-policy check sees all thirteen granting units, and parses calls from each";
ok =
lib.sort lib.lessThan (lib.attrNames granterUnits) == lib.sort lib.lessThan grantingUnitNames
&& lib.all (u: baoCalls u.script != [ ]) (lib.attrValues granterUnits)
@ -1723,11 +1686,6 @@ let
}
)
[
{
fetch = "swarm-bao-matrix-token";
machine = "hive-matrix";
consumer = "tuwunel.service";
}
{
fetch = "swarm-bao-otel-oidc";
machine = baoGrantWithConsumers.services.hyperhive.swarm.otel.machine;