vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

drop the shared hive agent login

Browse source 
Every agent now connects to the queue and reads logs with its own secret
from the store (#5181), so the per-hive `hive-<h>-agent` OIDC client has
no reader left. Remove it end to end:

- hive-c0re no longer forwards `hive-queue-agent-{secret,client-id}` into
  containers, and no longer forwards HIVE_AGENT_OIDC_TOKEN_ENDPOINT; the
  agent module that loaded them (nix/agent-modules/queue.nix) is gone, its
  options refused with a removal message.
- The host reader (glue-queue-agent-credential.nix, swarm-bao-queue-agent),
  its store role and policy unit (swarm-bao-queue-agent-policy), its PKI
  leaf and the deploy.bao.queueAgent* / queue.agentCredentialDir options
  are gone; the options are refused with a removal message.
- authelia mints no `hive-<h>-agent` client, and swarm-secret-publish no
  longer writes swarm/hives/<h>/queue/agent.
- swarm-secret-client drops queue::agent_client_path / queue::Credential.
- HIVE_AGENT_NATS_URL is now gated on the agents' queue address alone.

The log store's authelia check stays for operators. swarm-nats-auth still
refuses a `-agent` client id; that refusal and `agentClientSuffix` are left
for a follow-up.

Refs #4459
Refs #4765
This commit is contained in:
atlas 2026-10-13 09:49:32 +02:00 • committed by mara
commit 868c6cf96c
41 changed files with 286 additions and 1410 deletions
Download patch file Download diff file Expand all files Collapse all files

3
docs/swarm/README.md
View file

@ -303,8 +303,7 @@ subjects it publishes to, the credential may answer the one request it was just
sent, which is how a command gets its reply. The harness and the subagent daemon
connect with `_INBOX.agent.<agent>` as their inbox prefix, so the replies to the
icon write and the schedule read arrive. An agent can't read another agent's
terminal, state, input, commands or replies. The queue refuses the hive's shared
agent client outright.
terminal, state, input, commands or replies.
Hives and the swarm controller get a subscribe allow-list too, each with its
own reply inbox `_INBOX.client.<client-id>.>`, which `swarm_queue_client::connect`

11
docs/swarm/bao.md
View file

@ -121,13 +121,12 @@ them and names them in config. The service principals' leaves are at the top of
`/var/lib/swarm-bao-pki/`, named after the principal.
For every hive in `swarm.hives` other than its own, the store's host signs
that hive's three leaves into `/var/lib/swarm-bao-pki/hives/<hive>/`:
that hive's two leaves into `/var/lib/swarm-bao-pki/hives/<hive>/`:
| file | subject | name it in |
| ---------------------------------------- | ------------------------------------------------ | ------------------------------------------------------------------------------- |
| `client.pem`, `client-key.pem` | `<hive>` | `deploy.bao.clientCertFile`, `.clientKeyFile` |
| `queue-agent.pem`, `queue-agent-key.pem` | `<deploy.bao.queueAgentCommonNamePrefix>-<hive>` | `deploy.bao.queueAgentClientCertFile`, `.queueAgentClientKeyFile` |
| `hive-leaf.pem`, `hive-leaf-key.pem` | `<deploy.bao.hiveLeafCommonNamePrefix>-<hive>` | `deploy.hive-controller.tls.baoHiveIdentityCertFile`, `.baoHiveIdentityKeyFile` |
| file | subject | name it in |
| ------------------------------------ | ---------------------------------------------- | ------------------------------------------------------------------------------- |
| `client.pem`, `client-key.pem` | `<hive>` | `deploy.bao.clientCertFile`, `.clientKeyFile` |
| `hive-leaf.pem`, `hive-leaf-key.pem` | `<deploy.bao.hiveLeafCommonNamePrefix>-<hive>` | `deploy.hive-controller.tls.baoHiveIdentityCertFile`, `.baoHiveIdentityKeyFile` |
Copy `/var/lib/swarm-bao-pki/ca.pem` with them and name it in
`deploy.bao.serverCaFile`. A hive added to `swarm.hives` gets its directory on

1
docs/swarm/credentials.md
View file

@ -43,7 +43,6 @@ the rest of the cell says how.
| `swarm/agents/<agent>/claude-credentials` | `swarm-claude-login`, when you log the agent in from its card; one version per login | the agent, while it waits for a login | ❌ log in again | ✅ every 30 seconds while waiting |
| `swarm/hives/<hive>/matrix/sender-token` | `swarm-controller`, every 5 minutes | `swarm-controller` and the hive | ✅ when missing or rejected by the homeserver | ✅ the hive re-reads on every sweep |
| `swarm/hives/<hive>/forge/token` | `swarm-controller`, every 5 minutes | `swarm-controller` and the hive | ✅ when missing or no longer matching the forge | ✅ read on every forge call |
| `swarm/hives/<hive>/queue/agent` | authelia | `swarm-bao-queue-agent` on the hive's host | must be stated | must be stated |
| `swarm/hives/<hive>/oidc/client` | authelia, first boot; `swarm-secret-publish` copies it in | the hive | ❌ minted once; `swarm-secret-publish` republishes a re-mint | ✅ re-read when authelia refuses it |
| `swarm/services/<clientId>/oidc/client` | authelia | that service | must be stated | must be stated |
| `swarm/services/ci-runner/registration-token` | `swarm-controller`, from the forge, at start and every 5 minutes | `swarm-bao-ci-runner` on the runner's host | ❌ the forge keeps one token until an admin resets it; the controller then writes the new one within 5 minutes | ✅ every 10 minutes; restarts the runner on a change |

66
docs/swarm/secrets.md
View file

@ -107,9 +107,8 @@ with nothing to authenticate with still receives every hive's telemetry —
its pushes to the stores go out unauthenticated and get refused there, and its
scrape jobs for the published targets read a credential file nothing
loads, so they can't get a token. `swarm-bao-otel-oidc.service` therefore doesn't render at all without
`deploy.bao.otelOidcClientCertFile` / `otelOidcClientKeyFile`, the same choice
`glue-queue-agent-credential.nix` makes for its own reader, rather than the hard assertion `swarm-grafana.nix`
uses for its non-optional one. The operator can still deliver the secret by
`deploy.bao.otelOidcClientCertFile` / `otelOidcClientKeyFile`, rather than the hard assertion
`swarm-grafana.nix` uses for its non-optional one. The operator can still deliver the secret by
hand, at `services.hyperhive.deploy.swarm-otel.clientSecretFile`, on a
collector this unit never reaches.
@ -259,7 +258,6 @@ such as the root token.
| the hive's own store identity | `glue-bao-tls.nix` on the store's host; placed by hand on every other host ([`ca.md`](ca.md#the-gateway-leaf-comes-from-the-secret-store)) | `deploy.hive-controller.tls.baoHiveIdentityCertFile` / `baoHiveIdentityKeyFile` |
| the forge's copy of its OIDC secret | `swarm-bao-forge-oidc.service` reads it out of the swarm secret store under the forge publisher's leaf, wherever authelia runs | `/run/hive-forge-oidc/<id>.secret` on the host (tmpfs), bound into the forge container, handed to `forgejo-sso-source` through `LoadCredential` |
| the homeserver's copy of its OIDC secret | `swarm-bao-matrix-oidc.service` reads it out of the swarm secret store under matrix-ctl's leaf, wherever authelia runs | `/run/hive-matrix-oidc/<id>.secret` on the host (tmpfs), bound into the homeserver container, handed to tuwunel through `LoadCredential` |
| the agent containers' queue credential | authelia, published to the store by `swarm-secret-publish` | `<deploy.hive-controller.queue.agentCredentialDir>/secret` (`0600`) and `/client_id` (`0644`) |
The forge and the homeserver read their copies out of the store, the way
Grafana does below: the forge under the forge publisher's leaf, the
@ -268,34 +266,8 @@ publisher in authelia's container has written the secret, rather than skipping.
A silent skip produces a service whose login button always fails, which is a
symptom many layers from its cause.
The store's **first reader** is the agent containers' queue credential:
`glue-queue-agent-credential.nix` lands it as two files, the client secret and
the client id it authenticates, because that's how a queue client reads them —
the secret by path, the id by value. No local fallback exists, and none is
possible, so absent files mean an agent built before `swarm-logs` read its own
queue secret has no working `swarm-logs`, which is what a swarm looks like
before the publisher in authelia's container has run.
The reader runs before `hive-c0re.service`, which wants it (doesn't require
it), so an agent container never renders ahead of the credential; an
unreachable store delays the daemon's start rather than failing it.
That credential still has one hop left, because the reader of it lives inside
an agent container. **It crosses as a systemd credential, not as a bind
mount**, and the mode decides that: the secret is `root:0600` and a harness
runs as its own unprivileged agent user, so a bind would deliver a file that
user can't open. `hive-c0re` stats the two files — the directory is `0755`, so
it needs no read access to either — and hands them to
`systemd-nspawn --load-credential` through `hive-priv`, which runs as root.
Inside, `hive-agent.service` names the same two ids in `LoadCredential=`, which
lands them in its own `$CREDENTIALS_DIRECTORY`, owned by the agent user and by
nobody else, where a `swarm-logs` built before it read the agent's own queue
secret reads them. The harness itself doesn't: it connects to the queue with the agent's own credential. `hive-c0re`
never reads the bytes at any point: it runs as `hive-core`, which is the reason
a copy wasn't an option either. When the files aren't there the daemon forwards
nothing and says so in its journal, and the container starts without them.
The **second reader** is Grafana's OIDC client secret, and it's the first one
that's a _swarm service's_ own credential rather than a hive's. It has exactly
The store's **first reader** is Grafana's OIDC client secret, a _swarm
service's_ own credential rather than a hive's. It has exactly
one delivery route: `swarm-bao-grafana-oidc.service` reads
`swarm/services/<client-id>/oidc/client` out of the store, on **every** host
that runs Grafana. Where authelia is doesn't enter into it — the publisher in
@ -338,7 +310,7 @@ the password, and until it has, the store refuses `swarm-bao-grafana-admin` on
the Grafana host, which retries, while Grafana stays down because its config
reads that file.
The **third reader** is the swarm collector's own OIDC client secret, the
The **second reader** is the swarm collector's own OIDC client secret, the
same shape one level down: `swarm-bao-otel-oidc.service` reads
`swarm/services/<client-id>/oidc/client` out of the store on every host that
runs the collector, and registration lives separately in
@ -347,13 +319,12 @@ differs is what an absent credential means: the collector still receives
telemetry with none, so `swarm-otel.nix` doesn't assert
`deploy.bao.otelOidcClientCertFile` / `otelOidcClientKeyFile` the way Grafana
does — it
doesn't render the reading unit without them, the shape
`glue-queue-agent-credential.nix` uses for its own read.
doesn't render the reading unit without them.
The **fourth reader** is the secret store forwarder's OIDC client secret:
The **third reader** is the secret store forwarder's OIDC client secret:
`swarm-bao-forwarder-oidc.service`, on the store's own host, registered by
`glue-swarm-bao-otel-oidc-client.nix` on authelia's. It differs from the
third in both directions. It renders unconditionally, because the export it
second in both directions. It renders unconditionally, because the export it
authenticates has no unauthenticated mode to degrade into — and it orders
itself **after** `container@swarm-bao`, because the store it reads runs in the
container it delivers into. Nothing circular sits behind that: the identity it
@ -412,14 +383,13 @@ next controller restart.
### Per-principal identities
Bao matches a cert-auth role on the certificate's subject, so a certificate is an
identity and sharing one merges the identities. Each of these eleven leaves has
identity and sharing one merges the identities. Each of these ten leaves has
a role and a policy of its own, and reaches one path, except the review
classifier's, the CI runner's, the builder's, the CI run poller's, the claude
login minter's, the forge host's and Grafana's, which reach more than one:
| unit | option pair under `deploy.bao.` | reads, or writes |
| ------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `swarm-bao-queue-agent` | `queueAgentClientCertFile` / `queueAgentClientKeyFile` | `swarm/hives/<hive>/queue/agent` |
| `swarm-bao-grafana-oidc`, `swarm-bao-grafana-admin`, and Grafana's journal forwarder in its container | `grafanaOidcClientCertFile` / `grafanaOidcClientKeyFile` | reads `swarm/services/<grafana client id>/oidc/client` and `swarm/services/<grafana forwarder client id>/oidc/client`; creates and reads `swarm/services/<grafana client id>/admin/password` |
| `swarm-bao-otel-oidc` | `otelOidcClientCertFile` / `otelOidcClientKeyFile` | `swarm/services/<collector client id>/oidc/client` |
| `swarm-bao-forwarder-oidc` | `forwarderOidcClientCertFile` / `forwarderOidcClientKeyFile` | `swarm/services/<store forwarder client id>/oidc/client` |
@ -431,12 +401,8 @@ login minter's, the forge host's and Grafana's, which reach more than one:
| `swarm-ci-metrics` | `ciMetricsClientCertFile` / `ciMetricsClientKeyFile` | `swarm/services/ci-metrics/forge/token` and `swarm/services/<poller client id>/oidc/client` |
| `swarm-bao-forge-oidc` and the forge's controller-token mint and its journal forwarder in its container | `forgePublisherClientCertFile` / `forgePublisherClientKeyFile` | writes `swarm/controller/swarm-controller/forge/token`; reads `swarm/services/<forge client id>/oidc/client` and `swarm/services/<forge forwarder client id>/oidc/client` |
The first exists **per hive**, because the path it reads carries a hive name
and every hive runs its own reader. Its subject is
`<deploy.bao.queueAgentCommonNamePrefix>-<hive>`; `swarm.nix` reserves that
composed spelling as a hive name, so nobody can name a hive into another hive's
role. The other nine reach a path that names a swarm service rather than a hive,
so one role each is enough and their subjects are the flat
Each reaches a path that names a swarm service rather than a hive, so one role
each is enough and their subjects are the flat
`deploy.bao.grafanaOidcCommonName`, `deploy.bao.otelOidcCommonName`,
`deploy.bao.forwarderOidcCommonName`, `deploy.bao.ciRunnerCommonName`,
`deploy.bao.reviewClassifierCommonName`, `deploy.bao.builderCommonName`,
@ -449,7 +415,7 @@ OIDC client secret it reads.
On a host that mints its own PKI, `glue-bao-tls.nix` signs all nine and defaults
all eighteen options, and there is nothing to do. Elsewhere you copy each leaf from
the store's host and name it here: a service principal's from the top of
`/var/lib/swarm-bao-pki/`, another hive's queue-agent pair from
`/var/lib/swarm-bao-pki/`, another hive's leaves from
`/var/lib/swarm-bao-pki/hives/<hive>/` ([bao.md § TLS](bao.md#tls) lists that
directory). That's one file per principal rather than one file
shared by nine, which is the whole of what this buys. `swarm-ci-runner.nix`
@ -461,7 +427,7 @@ naming both options, and the controller runs without forge access and the forge
without a login source until an operator places the leaf.
On a host without the store, where the forwarder never runs, forgetting one of
the other nine isn't a quiet degrade. A hand-configured remote-store hive that
the others isn't a quiet degrade. A hand-configured remote-store hive that
names the hive's own `clientCertFile` and misses a principal's pair has the
build refused, by the module of the reader it missed, naming that pair — see
[what a missing leaf costs](#what-a-missing-leaf-costs) below.
@ -475,7 +441,7 @@ their readers hold a handle for longer than five minutes.
### What a missing leaf costs
Each of the three modules asserts its own pair, on the same condition: **this host
Each of the four modules asserts its own pair, on the same condition: **this host
already reads the store** — it holds `deploy.bao.clientCertFile` and
`clientKeyFile` — **and the consumer is on, and this principal's pair is null**.
The refusal names the two options, so an operator acts on the message without
@ -492,9 +458,7 @@ that matters on its own:
- **A host that runs the store** gets every pair from `glue-bao-tls.nix` as
defaults, so the third clause is false. Nothing to set, nothing to refuse.
- **A consumer that's off** — no collector, no Grafana — has no
unit to skip, so the second clause is false. Only the queue-credential reader
has no toggle of its own to check: every hive runs one for its own agents, so
reading the store at all is what asks for it.
unit to skip, so the second clause is false.
The collector keeps a separate degrade underneath this, unchanged: a host with no
store identity runs a collector with no OIDC client secret and still receives

2
docs/swarm/sso.md
View file

@ -225,7 +225,7 @@ Four consequences worth stating plainly:
Everything above is a person in a browser. A swarm also has callers that
hold no session and follow no redirect: the telemetry collector, the
queue's auth-callout responder, and each hive's own agents.
queue's auth-callout responder, and each hive.
**One client per hive, not one per service.** A hive's identity belongs to
the directory rather than to whichever service happens to consume it, so a

12
hive-c0re/src/lifecycle/agent_identity.rs
View file

@ -8,10 +8,9 @@
//! asked to authenticate in order to obtain the thing it authenticates with:
//! by the time it runs, its identity is already inside it.
//!
//! **A credential and not a bind mount**, the same answer and the same reason
//! as the queue secret beside it in [`super::host_config`]: the files are
//! `0600` to `hive-core` and the units that use them run as the unprivileged
//! agent user, so a bind would deliver a private key that user cannot open.
//! **A credential and not a bind mount**: the files are `0600` to `hive-core`
//! and the units that use them run as the unprivileged agent user, so a bind
//! would deliver a private key that user cannot open.
//! The container manager reads a `--load-credential` source as root and
//! re-exposes it under the consuming unit's own `User=`.
//!
@ -60,8 +59,7 @@ const SERVER_CA_CREDENTIAL: &str = "hive-agent-bao-server-ca";
///
/// Already-staged files survive a failed collection. A store that is briefly
/// unreachable then delivers the identity from the last successful read rather
/// than none at all — the same call as the queue reader's, which leaves its
/// files alone when the store says nothing.
/// than none at all.
pub async fn stage(agent: &str) -> Vec<CredentialMount> {
let dir = crate::paths::agent_identity_dir(agent);
match collect(agent, &dir).await {
@ -246,7 +244,7 @@ mod tests {
/// Half a delivery is worse than none: the container's check would fail
/// on the missing key and report a login problem instead of a delivery
/// one. Same both-or-neither rule the queue credential states.
/// one.
#[test]
fn a_certificate_without_its_key_forwards_nothing() {
let dir = tempfile::tempdir().expect("tempdir");

130
hive-c0re/src/lifecycle/host_config.rs
View file

@ -6,7 +6,7 @@
use std::path::{Path, PathBuf};
use anyhow::{Context, Result};
use hive_priv_sock::{BindMount, CredentialMount};
use hive_priv_sock::BindMount;
use hive_types::Permission;
use crate::coordinator::{AgentPaths, HiveEnv};
@ -417,70 +417,6 @@ pub fn hand_permissions(agent: &str, runtime_dir: &Path) {
}
}
/// Env var naming the host directory `swarm-bao-queue-agent.service` lands
/// this hive's agent queue credential in. Set by the hive-c0re NixOS module
/// from `deploy.hive-controller.queue.agentCredentialDir`; absent means this
/// daemon runs outside its unit, which is the same "no queue credential"
/// answer as an empty directory.
const QUEUE_CREDENTIAL_DIR_ENV: &str = "HIVE_C0RE_AGENT_QUEUE_CREDENTIAL_DIR";
/// systemd credential ids the two files arrive under inside the container.
/// `nix/agent-modules/queue.nix` spells the same two names in the harness
/// unit's `LoadCredential=`; neither side can discover the other's, so a
/// rename here is a rename there.
const QUEUE_SECRET_CREDENTIAL: &str = "hive-queue-agent-secret";
const QUEUE_CLIENT_ID_CREDENTIAL: &str = "hive-queue-agent-client-id";
/// Forward this hive's agent queue credential into `agent_name`'s container
/// as two systemd credentials, or nothing when the publisher has not landed
/// it yet.
///
/// **A credential and not a bind mount, because of the mode.** The secret is
/// `root:0600` on the host and the harness runs as the unprivileged agent
/// user, so a bind would deliver a file that user cannot open. nspawn's
/// `--load-credential` is read by the container manager as root and
/// re-exposed under the consuming unit's own `User=`, which is the whole
/// difference. hive-c0re never reads the bytes either way — it runs as
/// `hive-core` and only ever needs to know the file is *there*, which a
/// `0755` directory permits.
///
/// **Absent files are legal.** Authelia mints the secret on its first boot
/// and a publisher on that host puts it in the store, so "nothing at that
/// path" is the ordinary early state of a swarm rather than a fault. Saying
/// so at `info!` is what keeps it from being invisible: without a line here,
/// an agent that never connects looks identical to one that was never
/// configured.
fn queue_agent_credentials(agent_name: &str, dir: Option<&Path>) -> Vec<CredentialMount> {
let Some(dir) = dir else {
tracing::info!(
%agent_name,
"no {QUEUE_CREDENTIAL_DIR_ENV} in this daemon's environment — agent gets no swarm queue credential"
);
return Vec::new();
};
let secret = dir.join("secret");
let client_id = dir.join("client_id");
// Both or neither: a client holding one of the two comes up fine and
// never connects.
if !secret.is_file() || !client_id.is_file() {
tracing::info!(
%agent_name, dir = %dir.display(),
"swarm queue credential not published yet — agent gets no swarm queue credential"
);
return Vec::new();
}
vec![
CredentialMount {
name: QUEUE_SECRET_CREDENTIAL.to_owned(),
host_path: secret.to_string_lossy().into_owned(),
},
CredentialMount {
name: QUEUE_CLIENT_ID_CREDENTIAL.to_owned(),
host_path: client_id.to_string_lossy().into_owned(),
},
]
}
/// Idempotently rewrite the lines in `/etc/nixos-containers/<container>.conf`
/// that hive-c0re owns: `PRIVATE_NETWORK` (always 1), `HOST_ADDRESS` (the
/// bridge gateway IP) and `EXTRA_NSPAWN_FLAGS` (the runtime-dir bind). What
@ -537,18 +473,11 @@ async fn set_nspawn_flags(
// is needed here — the bind alone is enough.
let claude_mount = container_claude_mount(agent_name);
// The agent's own swarm-queue credential — the one thing forwarded in
// here, and forwarded *as a credential* rather than a bind for the
// reason `queue_agent_credentials` records.
let queue_credential_dir = std::env::var_os(QUEUE_CREDENTIAL_DIR_ENV).map(PathBuf::from);
let mut load_creds = queue_agent_credentials(agent_name, queue_credential_dir.as_deref());
// The agent's own identity *at* the swarm secret store, collected under
// this hive's certificate and forwarded the same way and for the same
// reason. Staged here rather than at spawn because this is the one place
// that writes the container's credential list — see
// this hive's certificate. Staged here rather than at spawn because this
// is the one place that writes the container's credential list — see
// `super::agent_identity` for the whole hop.
load_creds.extend(super::agent_identity::stage(agent_name).await);
let load_creds = super::agent_identity::stage(agent_name).await;
let mut binds: Vec<BindMount> = vec![
BindMount {
@ -678,10 +607,8 @@ async fn set_nspawn_flags(
#[cfg(test)]
mod tests {
use super::{
BindMount, HostMemory, MEMORY_CAP_FILE, QUEUE_CLIENT_ID_CREDENTIAL,
QUEUE_SECRET_CREDENTIAL, bind_child_agent_dirs, bind_own_agent_dirs,
holds_manage_root_agent, limits_to_apply, memory_cap_value, queue_agent_credentials,
stage_limits,
BindMount, HostMemory, MEMORY_CAP_FILE, bind_child_agent_dirs, bind_own_agent_dirs,
holds_manage_root_agent, limits_to_apply, memory_cap_value, stage_limits,
};
use crate::coordinator::HiveEnv;
@ -791,51 +718,6 @@ mod tests {
assert!(binds.is_empty());
}
/// The ordinary state of a swarm before the publisher on the authelia
/// host has run: the directory is named and empty. Forwarding a
/// credential whose source does not exist would make every agent
/// container refuse to start, so this has to be silence-with-a-log
/// rather than a partial list.
#[test]
fn an_unpublished_queue_credential_forwards_nothing() {
let dir = tempfile::tempdir().expect("tempdir");
assert!(queue_agent_credentials("iris", None).is_empty());
assert!(queue_agent_credentials("iris", Some(dir.path())).is_empty());
std::fs::write(dir.path().join("secret"), "s").expect("write secret");
assert!(
queue_agent_credentials("iris", Some(dir.path())).is_empty(),
"a secret with no client id is not a usable credential"
);
}
/// Both files present: two credentials, named the same two ids the
/// harness unit imports, pointing at the two files the reader unit
/// wrote.
#[test]
fn a_published_queue_credential_forwards_both_files() {
let dir = tempfile::tempdir().expect("tempdir");
std::fs::write(dir.path().join("secret"), "s").expect("write secret");
std::fs::write(dir.path().join("client_id"), "hive-h1-agent").expect("write client_id");
let creds = queue_agent_credentials("iris", Some(dir.path()));
let named: Vec<(String, String)> = creds
.into_iter()
.map(|c| (c.name, c.host_path))
.collect::<Vec<_>>();
assert_eq!(
named,
[
(
QUEUE_SECRET_CREDENTIAL.to_owned(),
dir.path().join("secret").to_string_lossy().into_owned(),
),
(
QUEUE_CLIENT_ID_CREDENTIAL.to_owned(),
dir.path().join("client_id").to_string_lossy().into_owned(),
),
]
);
}
#[test]
fn corrupt_swarm_permissions_deny_without_failing() {
let dir = tempfile::tempdir().expect("tempdir");

59
hive-c0re/src/meta.rs
View file

@ -455,12 +455,10 @@ const CANONICAL_INPUTS: &[&str] = &["nixpkgs"];
/// trivial to stub from tests (which build their own slice instead of
/// touching process-wide env).
const FORWARDED_VARS: &[&str] = &[
// Where the swarm queue is and where its tokens are minted, as an agent
// container reaches them. Both are addresses the *host* computes — the
// queue's from the bridge IP an agent routes over, never a loopback
// address, which inside a container is the agent itself.
// Where the swarm queue is, as an agent container reaches it. An address
// the *host* computes, never a loopback address, which inside a container
// is the agent itself.
"HIVE_AGENT_NATS_URL",
"HIVE_AGENT_OIDC_TOKEN_ENDPOINT",
// Where the swarm secret store listens. The hive's own `BAO_ADDR` value,
// forwarded under a separate name so the container never inherits the
// hive's store environment wholesale — the address is the only part of it
@ -470,7 +468,7 @@ const FORWARDED_VARS: &[&str] = &[
// fetches to trust the swarm. Forwarded with the address, under the same gate.
"HIVE_AGENT_BAO_SERVICES_PKI_MOUNT",
// Where an agent's `swarm-logs` queries, computed by the host for the
// same reason as the two above: it is a gateway address, and a container
// same reason as the queue's: it is a gateway address, and a container
// cannot derive it from anything it holds.
"HIVE_AGENT_LOGS_QUERY_URL",
// The metrics store's origin, for `swarm-metrics`, for the same reason.
@ -494,18 +492,6 @@ const FORWARDED_VARS: &[&str] = &[
/// name. Half-wiring this map is a value that evaluates fine and is silently
/// wrong.
const FORWARDED_VAR_OPTIONS: &[(&str, &str)] = &[
// Read at build time to decide whether the harness unit imports the hive's
// queue credential at all — see `nix/agent-modules/queue.nix`. Both halves
// wired for this map's own reason: an agent whose option says no queue and
// whose env says otherwise imports no credential for `swarm-logs`.
(
"HIVE_AGENT_NATS_URL",
"services.hyperhive.agent.queue.natsUrl",
),
(
"HIVE_AGENT_OIDC_TOKEN_ENDPOINT",
"services.hyperhive.agent.queue.tokenEndpoint",
),
// Read at build time to decide whether the in-container identity check is
// generated at all — see `nix/agent-modules/bao.nix`. Both halves wired
// for this map's own reason: the unit bakes the address into its own
@ -1561,35 +1547,22 @@ mod tests {
);
}
/// The queue coordinates travel as a pair and are wired through both
/// halves of the forwarded-var machinery: the env var the harness reads at
/// runtime and the option the harness *unit* is built from. An agent that
/// got only the env var would declare no credential and then report a
/// partial config — visible, but a rebuild away from working.
/// The queue address reaches the harness as env only. The agent module
/// declares no `services.hyperhive.agent.queue.*` option, so rendering one
/// into the meta flake fails every agent's evaluation.
#[test]
fn queue_coordinates_are_forwarded_as_both_env_and_option() {
for var in ["HIVE_AGENT_NATS_URL", "HIVE_AGENT_OIDC_TOKEN_ENDPOINT"] {
fn queue_address_is_forwarded_as_env_and_no_option() {
assert!(FORWARDED_VARS.contains(&"HIVE_AGENT_NATS_URL"));
assert!(
!FORWARDED_VARS.contains(&"HIVE_AGENT_OIDC_TOKEN_ENDPOINT"),
"no agent reads a token endpoint"
);
for (var, option) in FORWARDED_VAR_OPTIONS {
assert!(
FORWARDED_VARS.contains(&var),
"{var} must be forwarded into every agent's env"
!option.starts_with("services.hyperhive.agent.queue."),
"{var} renders {option}, which no agent module declares"
);
}
let mut out = String::new();
push_forwarded_var_options(
&mut out,
&[
("HIVE_AGENT_NATS_URL", "nats://10.42.0.1:4222".to_string()),
(
"HIVE_AGENT_OIDC_TOKEN_ENDPOINT",
"https://auth.t.local/api/oidc/token".to_string(),
),
],
);
assert_eq!(
out,
" services.hyperhive.agent.queue.natsUrl = \"nats://10.42.0.1:4222\";\n\
\x20 services.hyperhive.agent.queue.tokenEndpoint = \"https://auth.t.local/api/oidc/token\";\n"
);
}
/// `swarm-metrics` is installed only when the option is set, and reads

10
nix/agent-modules/bao.nix
View file

@ -14,8 +14,7 @@
# is why: the host file is `0600` to the hive daemon and this unit runs as the
# unprivileged agent user. nspawn's `--load-credential` is read by the
# container manager as root and re-exposed under this unit's own `User=`; a
# bind would deliver a private key this user cannot open. Same answer, same
# reason, as ./queue.nix's credential pair.
# bind would deliver a private key this user cannot open.
#
# This is the courier's other end: the hive-side collector has no purpose
# without it, so the two ship together and neither is reachable alone.
@ -63,12 +62,11 @@ let
# Where the identity lives in the store, spelled from the same pieces the
# publisher uses. `swarm_secret_client::mtls::identity_path` builds
# `swarm/agents/<agent>/bao-mtls` and `path::MOUNT` is `secret`; this
# literal is the nix half of that one agreement, exactly as
# ../host-modules/glue-queue-agent-credential.nix spells its own.
# literal is the nix half of that one agreement.
identityPath = "secret/swarm/agents/${agentName}/bao-mtls";
# The address is the whole switch — no separate `enable`, the same shape
# ./queue.nix and ./logs.nix gate themselves with. A hive that has a store
# ./logs.nix gates itself with. A hive that has a store
# forwards its address and every agent on it gets the check; a hive that has
# none forwards nothing and no agent does. An `enable` beside it would be a
# knob whose only correct setting is whatever the address already says, and
@ -153,7 +151,7 @@ in
UMask = "0377";
# Bare ids, no paths: the terse `LoadCredential=` form that inherits a
# credential the service *manager* received, which is what the
# container manager passed in. ./queue.nix states the same shape.
# container manager passed in.
LoadCredential = [
certCredential
keyCredential

9
nix/agent-modules/default.nix
View file

@ -34,6 +34,12 @@ let
sets it. Pin it host-side with
services.hyperhive.deploy.hive-controller.claudeCodePackage.
'';
queueRemoved = ''
The harness reads the queue's address at runtime from HIVE_AGENT_NATS_URL,
which the hive forwards into every container, and authenticates with the
agent's own secret from the swarm secret store, so no agent build sets a
queue option. Remove the setting.
'';
# `claude` on PATH. A hive that pins a claude-code
# (`services.hyperhive.deploy.hive-controller.claudeCodePackage`) hands its
@ -81,7 +87,6 @@ in
./otel.nix
./otel-removed.nix
./packages.nix
./queue.nix
./queue-identity.nix
./user.nix
./screen.nix
@ -101,6 +106,8 @@ in
"agent"
"claudeCodePath"
] claudeCodePathRemoved)
(lib.mkRemovedOptionModule [ "services" "hyperhive" "agent" "queue" "natsUrl" ] queueRemoved)
(lib.mkRemovedOptionModule [ "services" "hyperhive" "agent" "queue" "tokenEndpoint" ] queueRemoved)
];
options.services.hyperhive.agent.icon = lib.mkOption {

20
nix/agent-modules/queue-identity.nix
View file

@ -1,13 +1,11 @@
# This agent's own credential for the swarm queue, read from the store by the
# harness itself.
#
# The credential beside it in ./queue.nix is keyed per **hive**: one OIDC
# client minted at deploy time and handed to every agent container on the
# hive, so at the queue's auth callout one agent is indistinguishable from its
# co-hived neighbours. This one is keyed per **agent** — `swarm-controller`
# mints it at agent creation into `swarm/agents/<agent>/queue`
# (`swarm_secret_client::queue::agent_queue_path`), at swarm level, with no
# hive anywhere in the chain.
# Keyed per **agent**, so at the queue's auth callout one agent is
# distinguishable from its co-hived neighbours: `swarm-controller` mints it at
# agent creation into `swarm/agents/<agent>/queue`
# (`swarm_secret_client::queue::agent_queue_path`), at swarm level, with no hive
# anywhere in the chain.
#
# The agent authenticates to the store as itself, with the certificate
# ./bao.nix already proves it can log in with, and reads its own path. A hive
@ -42,11 +40,7 @@ let
# too.
serverCaCredential = "hive-agent-bao-server-ca";
# The store's address is the whole switch, exactly as in ./bao.nix — and
# deliberately *not* the queue coordinates in ./queue.nix. Those are the
# hive's, and gating a swarm-minted per-agent credential on a hive-level
# option would put the hive back in a path whose entire purpose is not
# having one.
# The store's address is the whole switch, exactly as in ./bao.nix.
configured = cfg.addr != null;
in
{
@ -54,7 +48,7 @@ in
systemd.services.hive-agent = {
# A bare id, no path: the terse `LoadCredential=` form that inherits a
# credential the service *manager* received, which is what the container
# manager passed in. ./bao.nix and ./queue.nix state the same shape.
# manager passed in. ./bao.nix states the same shape.
serviceConfig.LoadCredential = [ serverCaCredential ];
# The live certificate pair exists once ./bao.nix's unit has run.
after = [ "hive-agent-bao-identity.service" ];

117
nix/agent-modules/queue.nix
View file

@ -1,117 +0,0 @@
# The hive's shared swarm-queue client credential, imported into the harness
# unit's credentials directory for `swarm-logs` (./logs.nix) to authenticate
# with. The harness itself connects to the queue with the agent's own
# credential (./queue-identity.nix) and reads neither file.
#
# The credential is two files, the secret and the client id beside it, so a
# reader never spells `hive-<name>-agent` a second time.
#
# ⚠️ The credential arrives as a systemd credential and NOT as a bind mount,
# and the mode is why: the host file is `root:0600` and the reader runs as the
# unprivileged agent user. nspawn's `--load-credential` (written by
# `hive_c0re::lifecycle::host_config`) is read by the container manager as
# root and re-exposed under this unit's own `User=`; a bind would deliver a
# file the agent user cannot open.
{
lib,
config,
...
}:
let
cfg = config.services.hyperhive.agent.queue;
configured = cfg.natsUrl != null && cfg.tokenEndpoint != null;
# The two ids `hive_c0re::lifecycle::host_config` forwards under. Neither
# side can discover the other's spelling, so a rename is a rename there too.
secretCredential = "hive-queue-agent-secret";
clientIdCredential = "hive-queue-agent-client-id";
# Where systemd materialises this unit's credentials. `%d` only expands
# *inside* a unit, so `swarm-logs` (./logs.nix), which is not one, needs the
# literal; deriving it from the unit name here is what keeps the two from
# disagreeing about which unit's credentials they mean.
credentialsDir = "/run/credentials/hive-agent.service";
in
{
options.services.hyperhive.agent.queue = {
natsUrl = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "tls://nats.example.com:4222";
description = ''
Where the swarm queue listens, as this container reaches it.
Set by the generated meta flake from the host's
{option}`services.hyperhive.deploy.hive-controller.queue.agentNatsUrl`,
which names the queue rather than a loopback address — inside this
container `127.0.0.1` is the agent itself.
`null` means this hive has not been given the queue's address for its
agents: the harness unit imports no credential, and the harness fails
at boot, since it cannot run without a queue connection. It is
deliberately not defaulted to anything: a guessed address builds fine
and talks to the wrong machine.
'';
};
tokenEndpoint = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "https://auth.example.com/api/oidc/token";
description = ''
The swarm IdP's OAuth2 token endpoint, where `swarm-logs` mints a
`client_credentials` token for the client named in the delivered
credential.
The harness unit imports that credential only when this and
{option}`services.hyperhive.agent.queue.natsUrl` are both set.
'';
};
# One declaration, two readers. The credential ids are this module's —
# `hive_c0re::lifecycle::host_config` and the `LoadCredential=` below are
# the pair that has to agree on them — and a second consumer that spelled
# them again would be a second source of truth for a string whose
# mismatch is a file that is simply not there.
clientIdFile = lib.mkOption {
type = lib.types.str;
readOnly = true;
default = "${credentialsDir}/${clientIdCredential}";
description = ''
Path the agent's OIDC client id is delivered at, for a consumer
outside the harness unit. Read-only: it is a fact about where the
credential lands, not a knob — see {option}`services.hyperhive.agent.logs.queryUrl`
for the consumer this exists for.
'';
};
clientSecretFile = lib.mkOption {
type = lib.types.str;
readOnly = true;
default = "${credentialsDir}/${secretCredential}";
description = ''
Path the agent's OIDC client secret is delivered at. Read-only for
the same reason as {option}`services.hyperhive.agent.queue.clientIdFile`.
🩸 A PATH and never a value. The file is `0400` to the agent user and
is read at the moment of a token request; nothing in this tree puts
its contents in an environment variable, where `/proc/<pid>/environ`
would publish them to every process in the container.
'';
};
};
config = lib.mkIf configured {
systemd.services.hive-agent = {
# Bare ids, no paths: this is the terse `LoadCredential=` form that
# inherits a credential the service *manager* received, which is what
# the container manager passed in. `man systemd.exec` also makes that
# form non-fatal when the credential is absent, which is exactly the
# behaviour a hive whose publisher has not run yet needs.
serviceConfig.LoadCredential = [
secretCredential
clientIdCredential
];
};
};
}

14
nix/container-modules/swarm-secret-publisher.nix
View file

@ -1,6 +1,6 @@
# The unit that puts swarm-level secrets into the swarm's secret store, so
# whoever needs one can read it there: a hive its own and its agents'
# credentials, a service or the controller its own. It mints nothing: every
# whoever needs one can read it there: a hive, a service or the controller its
# own. It mints nothing: every
# value is an OIDC client secret authelia minted, and this unit copies it.
#
# Imported inside authelia's container, beside the mint: it reads each secret
@ -61,14 +61,6 @@ in
example = "secret/swarm/services/swarm-grafana/oidc/client";
description = "KV path its reader fetches the value from.";
};
withClientId = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
Also store the client id beside the value, as `client_id`, for
a reader that has to reconstruct it from the store alone.
'';
};
};
}
);
@ -170,7 +162,7 @@ in
# `value="$(cat "$src")"` would publish it to /proc for anyone in
# the container to read.
bao kv put ${lib.escapeShellArg s.path} \
value=@"$src"${lib.optionalString s.withClientId " client_id=${lib.escapeShellArg s.clientId}"}
value=@"$src"
published=$((published + 1))
else
# Not an error: authelia mints on its FIRST BOOT, so an absent file

1
nix/host-modules/default.nix
View file

@ -47,7 +47,6 @@
./glue-nats-bao-identity.nix
./glue-nats-oidc-client.nix
./glue-nats-wireguard.nix
./glue-queue-agent-credential.nix
./glue-builder-oidc-client.nix
./glue-ci-metrics-otel-oidc-client.nix
./glue-ci-runner-otel-oidc-client.nix

37
nix/host-modules/deploy.nix
View file

@ -53,6 +53,14 @@ let
puts it there. Remove this definition; the file it named is read by
nothing now and can be deleted.
'';
# Shared by the removed options of the hive's agent queue credential reader.
queueAgentCredentialRemoved = ''
A hive no longer fetches a shared queue credential for its agents: each
agent reads its own secret from the swarm's secret store under its own
certificate. Remove this definition; the leaf or directory it named is read
by nothing now and can be deleted.
'';
in
{
imports = [
@ -95,6 +103,35 @@ in
"statusPublish"
"clientSecretFile"
] statusSecretRemoved)
(lib.mkRemovedOptionModule [
"services"
"hyperhive"
"deploy"
"bao"
"queueAgentCommonNamePrefix"
] queueAgentCredentialRemoved)
(lib.mkRemovedOptionModule [
"services"
"hyperhive"
"deploy"
"bao"
"queueAgentClientCertFile"
] queueAgentCredentialRemoved)
(lib.mkRemovedOptionModule [
"services"
"hyperhive"
"deploy"
"bao"
"queueAgentClientKeyFile"
] queueAgentCredentialRemoved)
(lib.mkRemovedOptionModule [
"services"
"hyperhive"
"deploy"
"hive-controller"
"queue"
"agentCredentialDir"
] queueAgentCredentialRemoved)
# Same type, same meaning, new path — so a rename carries it exactly
# and existing configs keep evaluating with one warning naming both

4
nix/host-modules/glue-bao-readers-policy-order.nix
View file

@ -31,10 +31,6 @@ let
# Reader unit → the gate its own module defines it under.
readersHere = {
# ./glue-queue-agent-credential.nix
swarm-bao-queue-agent =
deployCfg.hive-controller.enable
&& havePair baoDeploy.queueAgentClientCertFile baoDeploy.queueAgentClientKeyFile;
# ./swarm-grafana.nix
swarm-bao-grafana-oidc = deployCfg.grafana.enable;
# ./swarm-otel.nix

26
nix/host-modules/glue-bao-tls.nix
View file

@ -95,15 +95,13 @@ in
# 🩸 The readers below each present a leaf of their own rather than
# `client.pem` above. The leaf is what bao sees, so this pairing is the
# whole of what turns "one principal with the union of their grants" into
# one principal per grant — see ./swarm-bao.nix's `perHiveReaders` block
# for the grants themselves.
# one principal per grant — see ./swarm-bao.nix's `readStanza` block for
# the grants themselves.
#
# Defaulted here rather than in `glue-<consumer>-bao-identity.nix` files:
# those exist where the consumer is a CONTAINER with a
# `deploy.<service>.*` namespace of its own to point at. These are host
# units reading the store, which is the pairing this file already is.
queueAgentClientCertFile = lib.mkDefault "${pkiDir}/queue-agent.pem";
queueAgentClientKeyFile = lib.mkDefault "${pkiDir}/queue-agent-key.pem";
grafanaOidcClientCertFile = lib.mkDefault "${pkiDir}/grafana-oidc.pem";
grafanaOidcClientKeyFile = lib.mkDefault "${pkiDir}/grafana-oidc-key.pem";
otelOidcClientCertFile = lib.mkDefault "${pkiDir}/otel-oidc.pem";
@ -223,20 +221,10 @@ in
# principal holding the union of their grants, and the union would
# include every agent credential in the swarm.
#
# ⚠️ The first carries THIS hive's name in the subject, unlike the three
# service leaves above. Its grant names one hive's path, because a queue
# credential lives under `swarm/hives/<name>/` and every hive runs a
# reader for its own; ./swarm-bao.nix writes one role per hive in the
# swarm directory to match. The OIDC readers need no such segment — a
# client is registered once per swarm.
#
# Minted whether or not the consumer runs here, for the reason the three
# leaves above give: on a hive that does not run the store this is the
# file an operator copies, and a leaf that only appears where its
# consumer does is one nobody can copy from anywhere.
[ -s ${pkiDir}/queue-agent.pem ] || ${signLeaf} ${pkiDir} queue-agent \
${lib.escapeShellArg "${deployCfg.bao.queueAgentCommonNamePrefix}-${clientCn}"} "" clientAuth
[ -s ${pkiDir}/grafana-oidc.pem ] || ${signLeaf} ${pkiDir} grafana-oidc \
${lib.escapeShellArg deployCfg.bao.grafanaOidcCommonName} "" clientAuth
@ -344,14 +332,14 @@ in
${lib.optionalString deployCfg.hive-controller.enable ''
# This hive's gateway-leaf identity, which opens
# `pki/issue/<hiveLeafPkiRolePrefix>-<hive>` and nothing else. Its
# subject carries this hive's name, like the queue-agent leaf
# above, because ./swarm-bao.nix writes one role
# per hive. Only where a hive runs; the other hives' are below.
# subject carries this hive's name, because ./swarm-bao.nix writes
# one role per hive. Only where a hive runs; the other hives' are
# below.
[ -s ${pkiDir}/hive-leaf.pem ] || ${signLeaf} ${pkiDir} hive-leaf \
${lib.escapeShellArg "${deployCfg.bao.hiveLeafCommonNamePrefix}-${clientCn}"} "" clientAuth
''}
# The reader, queue-agent and gateway-leaf leaves of every other hive in
# The reader and gateway-leaf leaves of every other hive in
# the swarm directory: the files an operator copies to that hive's
# host. One directory per hive, with the basenames this hive's leaves
# use, so a hive's set is one directory to copy.
@ -364,8 +352,6 @@ in
install -d -m 0700 ${d}
[ -s ${d}/client.pem ] || ${signLeaf} ${d} client \
${lib.escapeShellArg h} "" clientAuth
[ -s ${d}/queue-agent.pem ] || ${signLeaf} ${d} queue-agent \
${lib.escapeShellArg "${deployCfg.bao.queueAgentCommonNamePrefix}-${h}"} "" clientAuth
[ -s ${d}/hive-leaf.pem ] || ${signLeaf} ${d} hive-leaf \
${lib.escapeShellArg "${deployCfg.bao.hiveLeafCommonNamePrefix}-${h}"} "" clientAuth
''

284
nix/host-modules/glue-queue-agent-credential.nix
View file

@ -1,284 +0,0 @@
# Glue: this hive's agent queue credential comes out of the secret store.
#
# A cert login that fails LOUDLY because every state it fails on is one a
# retry fixes, then a read that degrades QUIETLY because no retry turns "no
# value there" into a value.
#
# ⚠️ Two files, not one, and that is the consumer's shape rather than a
# preference. `swarm-logs` inside the agent reads the secret and the client id
# each from its own file (`<prefix>_OIDC_CLIENT_SECRET_FILE`,
# `<prefix>_OIDC_CLIENT_ID_FILE`), so splitting them here is what lets
# hive-c0re hand both to an agent without parsing anything.
#
# ⚠️ An ABSENT file means this hive's agents do not connect to the queue, and
# that is correct rather than degraded. The publisher runs on the authelia
# host and authelia mints on its first boot, so "nothing at that path yet" is
# the ordinary early state of a swarm. Nothing here writes a local stand-in:
# there is no such thing as a locally valid OIDC client secret, so a placeholder would turn a hive that cannot connect
# into one that is refused, which reaches the agent as a timeout.
#
# 📌 This runs wherever a client identity is configured, NOT only where the
# store is. `deploy.bao.enable` would have been the co-location assumption
# itself; the reader needs a certificate, not a neighbour. On the store's own
# host ./glue-bao-tls.nix supplies one as a `mkDefault`; elsewhere an operator
# places the leaf and names it, and the same unit works. It also needs this
# host to run a hive (`deploy.hive-controller.enable`): the credential is for
# the agents hive-c0re creates here at runtime, and the unit writes under
# hive-c0re's own state directory.
{
pkgs,
lib,
config,
...
}:
let
hyperhiveCfg = config.services.hyperhive;
deployCfg = hyperhiveCfg.deploy;
baoCfg = hyperhiveCfg.swarm.bao;
baoDeploy = deployCfg.bao;
# What decides whether this unit exists at all. A reader is defined by holding
# a certificate the store accepts, and that is true on the store's own host
# and on a hive three networks away for exactly the same reason.
#
# 🩸 This principal's OWN leaf, not `clientCertFile` — the hive's. Bao
# matches a cert-auth role on the CN, so one leaf shared by four readers
# would be ONE principal holding the union of four grants: read on
# `swarm/agents/*` AND `swarm/hives/<hive>/*`, when this unit reads one
# queue credential and nothing else. Its own leaf carries
# `<queueAgentCommonNamePrefix>-<hive>` and its role grants the single path
# below — still this hive's own, so the narrowing costs no reach.
haveClientIdentity =
baoDeploy.queueAgentClientCertFile != null && baoDeploy.queueAgentClientKeyFile != null;
# Does this host read the store at all — the hive's own leaf, which is the
# one thing a remote-store deployment has to place by hand. Only used to
# decide whether a missing per-principal leaf is a mistake or a deployment
# that has no store: a host holding neither is the supported no-store
# shape, and one holding this pair but not the pair above reads the store
# without a reader identity of its own.
hiveReaderIdentity = baoDeploy.clientCertFile != null && baoDeploy.clientKeyFile != null;
credentialDir = toString deployCfg.hive-controller.queue.agentCredentialDir;
secretFile = "${credentialDir}/secret";
clientIdFile = "${credentialDir}/client_id";
# Where the credential lives in the store, spelled from the same pieces the
# writer uses. Whoever writes it and whoever reads it must agree, and the
# agreement belongs in one visible place.
#
# ⚠️ The `hives/<name>` segment is not decoration — it is what the reader's
# own grant covers, so a path outside it is a 403 rather than a miss, however
# correct it looks. `swarm-secret-client`'s `queue::agent_client_path` builds
# the same string from the same pieces; this literal is the nix half of that
# one agreement.
#
# `hiveName` has no fallback here for the reason ./glue-bao-tls.nix gives at
# its own use of it: it is asserted set wherever
# `deploy.hive-controller.enable` is.
credentialPath = "secret/swarm/hives/${hyperhiveCfg.hiveName}/queue/agent";
atomicWriteSecret = import ./lib/atomic-write-secret.nix { };
storeRetry = import ./lib/store-retry.nix { };
in
{
options.services.hyperhive.deploy.hive-controller.queue = {
agentCredentialDir = lib.mkOption {
type = lib.types.path;
default = "/var/lib/hyperhive/queue-agent";
description = ''
Host directory holding this hive's agent queue credential: `secret`
(the OIDC client secret, `0600`) and `client_id` (the client that
secret authenticates, `0644` — it is sent to the token endpoint on
every connection and is not itself a secret).
Both files are written by `swarm-bao-queue-agent.service`, which also
creates this directory. Named here because nothing else creates it:
the parent `/var/lib/hyperhive` belongs to `hive-c0re.service`'s
`StateDirectory=`, which re-applies its own mode on every start, and a
second declaration of *that* path is the two-mechanisms-one-path trap
./hive-gateway/default.nix records — so this directory is the unit's
to make and the parent stays c0re's.
Absent files mean this hive has no queue credential yet, which is what
a swarm looks like before the publisher in authelia's container has run.
'';
};
};
config = lib.mkMerge [
# ⚠️ A SEPARATE arm from the unit below, and that separation is the whole
# mechanism: the unit's arm is gated on `haveClientIdentity`, so an
# assertion written inside it could never be reached in the state it
# exists to report.
#
# Shaped after ./swarm-grafana.nix's `haveClientIdentity` assertion — the
# same refusal, named to this principal's own pair. Where Grafana's gate
# is `deploy.grafana.enable`, this reader's is the hive's own: every hive
# runs one for its own agents, so on a hive, reading the store at all is
# what asks for it. Hence `hiveReaderIdentity` — a host holding no hive
# leaf has no store to read and nothing is missing.
(lib.mkIf (deployCfg.hive-controller.enable && hiveReaderIdentity) {
assertions = [
{
assertion = haveClientIdentity;
message = ''
This host reads the swarm secret store (services.hyperhive.deploy.bao.clientCertFile
is set), so it needs the agent queue credential reader's own client
identity: set both
services.hyperhive.deploy.bao.queueAgentClientCertFile
services.hyperhive.deploy.bao.queueAgentClientKeyFile
swarm-bao-queue-agent.service fetches this hive's agent queue
credential out of the store, and without these it is not rendered
at all — leaving hive-c0re with no queue credential and no unit
that would ever have written one.
Every hive runs this reader for its own agents, so unlike the other
three principals there is no per-service toggle that turns it off:
a hive that reads the store at all is a hive that needs this pair.
⚠️ This reader's OWN leaf, not deploy.bao.clientCertFile. That one
is the hive's, and its grant reads every agent credential in the
swarm and this hive's own secrets; this role reads the one
queue-credential path. Pointing this option at the hive's leaf
would evaluate, deploy and log in — and undo the split.
On a hive that runs the store, glue-bao-tls.nix supplies both as
defaults and there is nothing to do. Elsewhere copy the pair from
/var/lib/swarm-bao-pki/hives/<hive>/ on the store's host and name it
here — see docs/swarm/bao.md.
'';
}
];
})
(lib.mkIf (deployCfg.hive-controller.enable && haveClientIdentity) {
systemd.services.swarm-bao-queue-agent = {
description = "fetch this hive's agent queue credential from the swarm secret store";
# Every one of these names a unit that exists only where the store runs,
# so the ordering is conditional even though the read is not: off-host
# there is nothing local to wait for, and the timeout below is what
# bounds the attempt instead. `Wants=` pki, for
# ./glue-bao-readers-store-order.nix's reason.
after = lib.optionals baoDeploy.enable [
"swarm-bao-pki.service"
"container@${baoCfg.machine}.service"
];
wants = lib.optionals baoDeploy.enable [
"swarm-bao-pki.service"
"container@${baoCfg.machine}.service"
];
# Ordered before hive-c0re, so no agent container renders ahead of an
# attempt at its credential. `Wants=`, not `Requires=`: a store this
# unit can't reach delays hive-c0re's start by one attempt
# (`TimeoutStartSec` below) rather than failing it — hive-c0re starts
# after that attempt with whatever credential is on disk, while
# ./lib/store-retry.nix keeps retrying in the background.
before = [ "hive-c0re.service" ];
wantedBy = [
"multi-user.target"
"hive-c0re.service"
];
path = [
baoDeploy.package
pkgs.coreutils
];
# ./lib/store-retry.nix.
inherit (storeRetry) startLimitBurst startLimitIntervalSec;
serviceConfig = storeRetry.serviceConfig // {
Type = "oneshot";
RemainAfterExit = true;
# What actually bounds each attempt at the reads below. Stated here
# rather than left to systemd's default, so the number a boot waits on
# is in the file that waits.
TimeoutStartSec = 30;
};
environment = {
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
BAO_CLIENT_CERT = baoDeploy.queueAgentClientCertFile;
BAO_CLIENT_KEY = baoDeploy.queueAgentClientKeyFile;
}
# Absent means the system trust store, which is what a deployment with a
# real CA wants and what a self-signed one must not be left with.
// lib.optionalAttrs (baoDeploy.serverCaFile != null) {
BAO_CACERT = baoDeploy.serverCaFile;
};
script = ''
set -euo pipefail
${atomicWriteSecret}
# `bao`'s own message is the only thing separating a missing value from
# a refused identity from an unreachable host. This unit's degraded
# mode is correct for all three, so it reports which one rather than
# asserting all three in a sentence of ours.
err="$(mktemp)"
trap 'rm -f "$err"' EXIT
# Cert auth is a login, not a transport setting. The `BAO_CLIENT_*`
# variables above only decide which certificate the TLS handshake
# presents; without a token `bao` asks its token helper instead, and
# that is a `sh` this unit's `path` does not carry. `-token-only`
# answers on stdout and skips the helper on both sides.
#
# Fails LOUDLY, unlike the reads below: the three states a login
# failure covers — store not up, sealed, role not written yet — are all
# things a retry fixes, and `Restart=on-failure` above is what retries.
if ! BAO_TOKEN="$(bao login -method=cert -token-only 2>"$err")"; then
echo "could not log in to swarm-bao with this host's certificate; leaving the queue credential in ${credentialDir} as it is." >&2
if [ -s "$err" ]; then
cat "$err" >&2
else
echo "bao failed without writing a diagnostic." >&2
fi
exit 1
fi
export BAO_TOKEN
# Two reads of one object rather than one `-format=json` parsed with
# `jq`: no sibling unit carries `jq` on its `path`, and the pair cannot
# actually disagree — the client id is derived from this hive's name, so
# a rotation landing between these two calls changes the secret and
# rewrites the same id.
if ! secret="$(bao kv get -field=value ${lib.escapeShellArg credentialPath} 2>"$err")"; then
echo "swarm-bao did not return ${credentialPath}; this hive's agents have no queue credential yet." >&2
if [ -s "$err" ]; then
cat "$err" >&2
else
echo "bao failed without writing a diagnostic." >&2
fi
exit 0
fi
if ! client_id="$(bao kv get -field=client_id ${lib.escapeShellArg credentialPath} 2>"$err")"; then
echo "${credentialPath} holds no client_id; the secret alone is not a usable credential, so nothing is written." >&2
if [ -s "$err" ]; then
cat "$err" >&2
else
echo "bao failed without writing a diagnostic." >&2
fi
exit 0
fi
# Both or neither: a client that finds one of the two comes up "fine"
# and never connects.
if [ -z "$secret" ] || [ -z "$client_id" ]; then
echo "swarm-bao returned an empty field of ${credentialPath}; leaving the files as they are." >&2
exit 0
fi
install -d -m 0755 ${lib.escapeShellArg credentialDir}
atomic_write_secret 0600 "" ${lib.escapeShellArg secretFile} "$secret"
# `0644` on purpose: an OIDC client id is presented to the token
# endpoint on every connection and is public by construction.
atomic_write_secret 0644 "" ${lib.escapeShellArg clientIdFile} "$client_id"
'';
};
})
];
}

3
nix/host-modules/hive-c0re/default.nix
View file

@ -333,8 +333,7 @@ in
# from an unprivileged daemon without widening either.
# The gate is the identity, never `deploy.bao.enable`: a hive that
# reads a store on another machine holds one of these and runs no
# store. ./glue-queue-agent-credential.nix gates its own reader the
# same way.
# store. ../swarm-otel.nix gates its own reader the same way.
lib.optionals baoClientIdentity [
"bao-client.pem:${baoDeploy.clientCertFile}"
"bao-client-key.pem:${baoDeploy.clientKeyFile}"

29
nix/host-modules/hive-c0re/environment.nix
View file

@ -11,9 +11,8 @@ let
cfg = config.services.hyperhive.c0re;
baoDeploy = config.services.hyperhive.deploy.bao;
baoCfg = config.services.hyperhive.swarm.bao;
# The same test ../glue-queue-agent-credential.nix applies, and for the same
# reason: a reader is defined by holding a certificate the store accepts,
# not by sharing a host with the store. Deriving this from
# A reader is defined by holding a certificate the store accepts, not by
# sharing a host with the store. Deriving this from
# `deploy.bao.enable` would be the co-location assumption itself.
haveBaoClientIdentity = baoDeploy.clientCertFile != null && baoDeploy.clientKeyFile != null;
in
@ -222,31 +221,15 @@ in
# the one the roster gave it.
HIVE_C0RE_OIDC_CLIENT_ID = "hive-${config.services.hyperhive.hiveName}";
}
// {
# Where ../glue-queue-agent-credential.nix lands the AGENTS' queue
# credential. Read by `hive_c0re::lifecycle::host_config`, which stats the
# two files and forwards them into each container as systemd credentials.
# Never read for its contents here: the secret is `0600` root-owned and
# this daemon is `hive-core`, which is exactly why the transport is a
# credential rather than a bind mount.
HIVE_C0RE_AGENT_QUEUE_CREDENTIAL_DIR = toString config.services.hyperhive.deploy.hive-controller.queue.agentCredentialDir;
}
//
# The agents' half of the same queue, forwarded by `hive_c0re::meta` into
# every container. A pair, gated together, because an agent that got one of
# them would report a half-configured queue instead of none.
# every container.
#
# ⚠️ By name, like `HIVE_C0RE_NATS_URL` above, and never a loopback
# address: inside a container `127.0.0.1` is the agent itself.
lib.optionalAttrs
(
config.services.hyperhive.deploy.hive-controller.queue.agentNatsUrl != null
&& config.services.hyperhive.swarm.statusPublish.tokenEndpoint != null
)
{
HIVE_AGENT_NATS_URL = config.services.hyperhive.deploy.hive-controller.queue.agentNatsUrl;
HIVE_AGENT_OIDC_TOKEN_ENDPOINT = config.services.hyperhive.swarm.statusPublish.tokenEndpoint;
}
lib.optionalAttrs (config.services.hyperhive.deploy.hive-controller.queue.agentNatsUrl != null) {
HIVE_AGENT_NATS_URL = config.services.hyperhive.deploy.hive-controller.queue.agentNatsUrl;
}
//
# Where an agent reads the swarm's logs. Gated on the store identity below,
# because `swarm-logs` authenticates with the agent's own queue secret, read

22
nix/host-modules/swarm-authelia-service.nix
View file

@ -352,22 +352,14 @@ in
readOnly = true;
default = "-agent";
description = ''
Suffix appended to a hive's own client id to name the client its
*agent containers* present — agents on hive `alpha` all present
`${config.services.hyperhive.swarm.authelia.hiveClientPrefix}alpha${config.services.hyperhive.swarm.authelia.agentClientSuffix}`.
Read-only for the same reason as `hiveClientPrefix`, and read by the
same consumer with the same failure mode: a split spelling denies
every agent as a timeout.
Suffix the queue's auth-callout responder refuses on a hive's client
id: `${config.services.hyperhive.swarm.authelia.hiveClientPrefix}alpha${config.services.hyperhive.swarm.authelia.agentClientSuffix}`
is denied, never read as hive `alpha-agent`. No client is minted with
it; an agent authenticates to the queue with its own secret.
A suffix on the hive's id rather than a prefix of its own, because
`agent-alpha` reads as *the agent named alpha* — which is precisely
what this identity does not say.
One id per hive rather than per agent, because agents are created at
runtime and a per-agent client would make creating one a config
change plus a reload. The consequence is that this identity says
*which hive* an agent belongs to and never *which agent* — the
broker cannot tell two agents on one hive apart.
Read-only for the same reason as `hiveClientPrefix`. A hive named with
this suffix is refused at evaluation, since the queue would refuse its
own client id.
'';
};

91
nix/host-modules/swarm-authelia.nix
View file

@ -230,8 +230,8 @@ let
# The secret publisher runs in this container, beside the mint whose secrets
# it copies. It needs a store identity, and a reader is defined by holding a
# certificate the store accepts, never by standing next to it — the rule
# ./glue-queue-agent-credential.nix states in full. `deploy.bao.enable` here
# would be the co-location assumption itself.
# ./swarm-bao.nix's `deploy.bao` options block states in full.
# `deploy.bao.enable` here would be the co-location assumption itself.
publisherCfg = deployCfg.swarm-secret-publisher;
publishes =
publisherCfg.enable
@ -300,71 +300,6 @@ let
accessTokenSignedResponseAlg = "RS256";
}) hyperhiveCfg.swarm.hives;
# The one URL an agent's token has to be authorised at beyond the queue:
# the log store's machine query route. Read off `swarm.victorialogs.domain`
# — an unconditional option, declared whether or not this host runs the
# store — rather than restated, because it is also the string `swarm-logs`
# requests as its audience and the string `hive-c0re/environment.nix` hands
# an agent as its endpoint. Three readers, one formula; two spellings
# present as a valid token refused at the store.
logsQueryUrl = "https://${hyperhiveCfg.swarm.victorialogs.domain}/select/logsql/query";
# The identity an agent container presents to the swarm's queue. One per
# HIVE, not per agent, and that is the load-bearing choice rather than a
# shortcut: agents are created at **runtime**, so anything minted per agent
# would make creating one a config change plus an authelia reload. Keyed on
# the hive, this list's length tracks the roster above it — deploy-time, like
# every other entry here.
#
# ⚠️ The cost, stated rather than discovered later: every agent on a hive
# presents the SAME client id, so the broker can tell *hives* apart and not
# *agents*. Deliberate and deferred — closing it requires the client list to
# stop being static, which is the same problem the users-database writer
# already solves for identities.
#
# 🏷️ The id EXTENDS the hive's own (`hive-<name>-agent`) rather than taking a
# prefix of its own (`agent-<name>`), because that reads as *the agent called
# `<name>`* — which is the one thing this identity does not carry. The cost is
# that the two ids are not distinguishable by prefix, so the responder's
# agent rule must be tried BEFORE its hive rule; `policy.rs` says so at the
# match site, and the assertion below covers the case that ordering cannot.
#
# Mirrors `hiveClients` field for field so the two stay comparable. The
# signing algorithm is not load-bearing here — this client's consumers are
# the queue's auth-callout responder, which *introspects* rather than
# verifying offline, and authelia's own authz endpoint — but it matches its
# sibling rather than inventing a second answer to a question nobody asked.
agentClients = lib.mapAttrsToList (name: _: {
id = "${cfg.hiveClientPrefix}${name}${cfg.agentClientSuffix}";
description = "HyperHive agents on hive ${name}";
kind = "machine";
redirectUris = [ ];
# ⚠️ **Two** audiences, where `hiveClients` has one, because an agent
# presents this token at two different kinds of consumer. The queue
# introspects it and reads the client id; the log store's gateway
# location runs `auth_request` against authelia, which authorises a
# bearer token **by the URL being requested** and refuses one whose
# audience does not name it. Measured, not assumed: without the URL
# registered here the token endpoint answers `invalid_target` for it,
# and a token minted with no audience at all is answered by the gateway
# with a bare 401 that names no cause.
audience = [
"${cfg.hiveClientPrefix}${name}${cfg.agentClientSuffix}"
logsQueryUrl
];
# Without it the authz endpoint refuses an otherwise valid token and
# blames the token rather than the missing grant — the same note
# `glue-swarm-otel-oidc-client.nix` carries over the collector's client.
bearerAuthz = true;
# Stated rather than left on authelia's default, for
# `glue-swarm-otel-oidc-client.nix`'s reason: authelia permits only
# basic / JWT methods for a confidential client holding this scope, and
# enforces it in the startup validator. `swarm-queue-client` sends the
# credential as basic auth, so this is also what the minter already does.
tokenEndpointAuthMethod = "client_secret_basic";
accessTokenSignedResponseAlg = "RS256";
}) hyperhiveCfg.swarm.hives;
# `swarm-authelia-bridge`'s own identity — distinct from
# `swarm-controller`'s (`swarm-controller.nix`'s `queueClientId`). A
# resource server introspecting a token proves its OWN identity to the
@ -706,7 +641,7 @@ in
# wherever this module is, so `oidc.clients` is never actually empty
# — see `oidcEnabled`'s comment above.
services.hyperhive.swarm.authelia.oidc.clients =
lib.optionals cfg.oidc.hiveIdentities (hiveClients ++ agentClients)
lib.optionals cfg.oidc.hiveIdentities hiveClients
++ [
bridgeClient
forwarderClient
@ -749,18 +684,11 @@ in
}
{
# A hive whose name ENDS with the agent suffix mints an id the
# queue's responder reads as somebody else's agents:
# queue's responder reads as a hive's agent client, which it refuses:
# `hive-foo-agent` parses as *the agents of hive foo* before it
# parses as *the hive foo-agent*, because the agent rule is the
# more specific one and therefore runs first.
#
# ⚠️ NOT covered by the duplicate-id assertion above, and the gap is
# the interesting half. That one fires only when BOTH `foo` and
# `foo-agent` are on the roster, because only then are two clients
# actually named the same string. With `foo-agent` alone there is no
# duplicate and nothing to see — the hive simply receives its
# agents' grant instead of its own, and a NATS denial arrives as a
# timeout, so the symptom names nothing.
# more specific one and therefore runs first. A NATS denial arrives
# as a timeout, so the symptom names nothing.
#
# Checkable here and nowhere downstream: the responder runs in a
# container with no view of the roster (`policy.rs`'s `hive_name`
@ -774,10 +702,9 @@ in
+ lib.concatMapStringsSep ", " (h: "'${h}'") (
lib.filter (h: lib.hasSuffix cfg.agentClientSuffix h) (lib.attrNames hyperhiveCfg.swarm.hives)
)
+ " — names ending with '${cfg.agentClientSuffix}', the suffix that "
+ "marks a hive's agent containers. Such a hive's own client id is "
+ "indistinguishable from another hive's agent client, and the "
+ "queue resolves it as the agents. Rename the hive.";
+ " — names ending with '${cfg.agentClientSuffix}', the suffix the "
+ "queue refuses as a hive's agent client. Such a hive's own client "
+ "id reads as one, and the queue refuses it. Rename the hive.";
}
# The two obligations `authelia.bearer.authz` carries. Authelia
# enforces both itself — but in its `preStart` validator, so a

130
nix/host-modules/swarm-bao.nix
View file

@ -808,9 +808,9 @@ let
}
'';
# One gateway-leaf issuer per hive in the swarm directory, for the reason
# `perHiveReaders` below is per hive. One name for the pki role, the policy
# and the cert-auth role, as for the readers.
# One gateway-leaf issuer per hive in the swarm directory: one subject for
# every hive could ask for any hive's certificate. One name for the pki role,
# the policy and the cert-auth role, as for the readers below.
hiveLeafIssuers = lib.mapAttrsToList (hiveName: hive: {
name = "${baoDeploy.hiveLeafPkiRolePrefix}-${hiveName}";
cn = "${baoDeploy.hiveLeafCommonNamePrefix}-${hiveName}";
@ -897,48 +897,21 @@ let
# `swarm/agents/*` AND `swarm/hives/<hive>/*`, and nothing under
# `swarm/services/`.
# Principals presenting one certificate are ONE principal to bao, so sharing
# the hive's leaf would give each of them everything that leaf reads: the
# unit that fetches this hive's queue credential could fetch every agent
# credential in the swarm, and a unit that fetches a service's OIDC client
# secret could not fetch it at all, as that path is not the hive's to read.
# the hive's leaf would give each of them everything that leaf reads: a unit
# that fetches a service's OIDC client secret could fetch every agent
# credential in the swarm, and could not fetch its own secret at all, as
# that path is not the hive's to read.
#
# Each gets its own leaf, its own cert-auth role and a policy holding the
# one `secret/data/…` path that principal's script actually names — spelled
# to the LEAF rather than to a prefix, for the reason `matrixCtlPolicyText`
# above gives for doing the same.
#
# ⚠️ The queue-credential reader is PER HIVE and the OIDC readers are not,
# and that asymmetry is the shape of the paths rather than a preference. A
# queue credential lives under `swarm/hives/<name>/`, and every hive runs a
# reader for its own; an OIDC client secret lives under
# `swarm/services/<client-id>/` and a swarm registers each of those exactly
# once. A single role for a per-hive reader would have to name `hives/*` to
# serve every hive — which would let one hive read another's queue
# credential, a reach no hive needs.
readStanza = path: ''
path "${path}" {
capabilities = ["read"]
}
'';
# One reader per hive in the swarm directory. Written from the directory
# rather than from this host's own name because the policy is written where
# the STORE is and the reader runs where its hive is, so this host's name is
# not the reader's.
perHiveReaders =
{ rolePrefix, cnPrefix, ... }@spec:
lib.mapAttrsToList (hiveName: _: {
name = "${rolePrefix}-${hiveName}";
cn = "${cnPrefix}-${hiveName}";
policyText = readStanza (spec.path hiveName);
}) hyperhiveCfg.swarm.hives;
queueAgentReaders = perHiveReaders {
rolePrefix = "swarm-queue-agent";
cnPrefix = baoDeploy.queueAgentCommonNamePrefix;
path = h: "${credentialMountPath}/data/swarm/hives/${h}/queue/agent";
};
# Singletons, so a list of one rather than a second shape: the granting unit
# below takes a list either way and the units stay identical apart from
# the objects they write.
@ -1257,9 +1230,9 @@ let
# One unit per PRINCIPAL, not one unit for all of them: these names are
# operator-facing strings, the same reason `swarm-bao-secret-publisher-policy`
# below gives for not being more script inside the controller's unit. A hive
# whose queue credential never arrives is diagnosed by the state of the unit named
# after it.
# below gives for not being more script inside the controller's unit. A
# reader whose secret never arrives is diagnosed by the state of the unit
# named after it.
readerPolicyUnit =
description: objects:
granterPolicyUnit description (
@ -1589,7 +1562,7 @@ in
# `serverCaFile` live here too, and they are set by whoever authenticates TO
# the store — which includes a host that runs none of it. A reader is defined
# by holding a certificate the store accepts, never by sharing a host with
# one; ./glue-queue-agent-credential.nix gates on exactly that and not on
# one; ./swarm-otel.nix's store reader gates on exactly that and not on
# `enable`.
# Reading this block as store-runner-only is what makes an off-host reader
# look inexpressible when it is already supported.
@ -2102,28 +2075,6 @@ in
'';
};
queueAgentCommonNamePrefix = lib.mkOption {
type = lib.types.str;
default = "swarm-bao-queue-agent";
example = "swarm-bao-queue-agent.svc";
description = ''
Prefix of the subject the store's per-hive queue-credential roles
accept. The role for hive `h` accepts `<prefix>-h` and grants read on
exactly `swarm/hives/h/queue/agent` — the client credential that hive's
agent containers authenticate to the swarm queue with.
A **prefix** rather than one subject because every hive runs its own
reader and the path it reads carries that hive's name. One subject for
all of them would need a `hives/*` grant, which would let one hive read
another hive's queue credential.
⚠️ Reserved as a hive name by ./swarm.nix, per hive, for the reason its
fixed-subject siblings are: cert auth trusts the CA and
`allowed_common_names` is the whole narrowing, so a hive whose own leaf
carried one of these spellings would receive that grant.
'';
};
hiveLeafCommonNamePrefix = lib.mkOption {
type = lib.types.str;
default = "swarm-bao-hive-leaf";
@ -2134,9 +2085,8 @@ in
`<servicesPkiMountPath>/issue/<hiveLeafPkiRolePrefix>-h`, which issues
for `h`'s domain and `*.` that domain and nothing else.
Per hive for the reason
{option}`services.hyperhive.deploy.bao.queueAgentCommonNamePrefix` is:
one subject for every hive could ask for any hive's certificate.
Per hive because one subject for every hive could ask for any hive's
certificate.
⚠️ Reserved as a hive name by ./swarm.nix, per hive, like its siblings.
'';
@ -2163,7 +2113,7 @@ in
Its grant is one path,
`swarm/services/<grafana client id>/oidc/client`, and read only. Not
per hive, unlike the two prefixes above: an OIDC client is registered
per hive, unlike the prefix above: an OIDC client is registered
once per swarm, so the path names the service and never a hive.
⚠️ Same collision as its siblings, and the same answer: ./swarm.nix
@ -2206,37 +2156,6 @@ in
'';
};
queueAgentClientCertFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-bao-pki/queue-agent.pem";
description = ''
Certificate the unit that fetches this hive's queue client credential
presents to the store. Its subject must be
`<queueAgentCommonNamePrefix>-<this hive's name>`; cert auth matches on
the CN and the role accepts nothing else.
⚠️ **Not** {option}`services.hyperhive.deploy.bao.clientCertFile`.
Pointing this at the hive's own leaf would evaluate, deploy and log in —
and hand a unit that reads one credential a certificate that reads every
agent credential in the swarm and this hive's own secrets.
No default here: ./glue-bao-tls.nix points it at the leaf it mints,
wherever this host mints its own PKI. A deployment that reads a store on
another host names it by hand, alongside the leaf itself.
'';
};
queueAgentClientKeyFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-bao-pki/queue-agent-key.pem";
description = ''
Private key for
{option}`services.hyperhive.deploy.bao.queueAgentClientCertFile`.
'';
};
grafanaOidcClientCertFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
@ -2246,8 +2165,10 @@ in
to the store. Its subject must be
{option}`services.hyperhive.deploy.bao.grafanaOidcCommonName`.
⚠️ Not the hive's own leaf, for the reason
{option}`services.hyperhive.deploy.bao.queueAgentClientCertFile` gives.
⚠️ **Not** {option}`services.hyperhive.deploy.bao.clientCertFile`.
Pointing this at the hive's own leaf would evaluate, deploy and log in —
and hand a unit that reads one secret a certificate that reads every
agent credential in the swarm and this hive's own secrets.
'';
};
@ -2271,7 +2192,7 @@ in
{option}`services.hyperhive.deploy.bao.otelOidcCommonName`.
⚠️ Not the hive's own leaf, for the reason
{option}`services.hyperhive.deploy.bao.queueAgentClientCertFile` gives.
{option}`services.hyperhive.deploy.bao.grafanaOidcClientCertFile` gives.
'';
};
@ -2295,7 +2216,7 @@ in
{option}`services.hyperhive.deploy.bao.forwarderOidcCommonName`.
⚠️ Not the hive's own leaf, for the reason
{option}`services.hyperhive.deploy.bao.queueAgentClientCertFile` gives.
{option}`services.hyperhive.deploy.bao.grafanaOidcClientCertFile` gives.
'';
};
@ -3543,12 +3464,10 @@ in
'';
# 🩸 The principals that each log in with a leaf of their own, not as the
# hive. See the block around `perHiveReaders` for what each grant is and
# why the first is per hive; the units are identical apart from the
# objects they write, so the shape lives in `readerPolicyUnit` and the
# difference is visible here in one line each.
systemd.services.swarm-bao-queue-agent-policy = readerPolicyUnit "write the per-hive queue-credential bao policies and cert-auth roles" queueAgentReaders;
# hive. See the block around `readStanza` for what each grant is; the
# units are identical apart from the objects they write, so the shape
# lives in `readerPolicyUnit` and the difference is visible here in one
# line each.
systemd.services.swarm-bao-grafana-oidc-policy = readerPolicyUnit "write Grafana's OIDC-secret-reader bao policy and cert-auth role" grafanaOidcReaders;
systemd.services.swarm-bao-otel-oidc-policy = readerPolicyUnit "write the collector's OIDC-secret-reader bao policy and cert-auth role" otelOidcReaders;
@ -3693,7 +3612,6 @@ in
"swarm-bao-controller-policy.service"
"swarm-bao-secret-publisher-policy.service"
"swarm-bao-matrix-ctl-policy.service"
"swarm-bao-queue-agent-policy.service"
"swarm-bao-grafana-oidc-policy.service"
"swarm-bao-otel-oidc-policy.service"
"swarm-bao-forwarder-oidc-policy.service"

12
nix/host-modules/swarm-grafana.nix
View file

@ -282,8 +282,8 @@ let
);
# A reader of the store is defined by holding a certificate the store
# accepts, never by standing next to it — the rule
# ./glue-queue-agent-credential.nix states in full.
# accepts, never by standing next to it — the rule ./swarm-bao.nix's
# `deploy.bao` options block states in full.
#
# Also asserted rather than gating anything, and that is the ruling this
# module's second delivery route was deleted under: the store exists so that
@ -722,10 +722,10 @@ in
# to be wrong, and a gate whose false arm is a Grafana with no login.
# Recorded in docs/swarm/secrets.md.
#
# Shaped after ./glue-queue-agent-credential.nix, because it is a store
# reader and this is what the store's readers do: a cert login that fails
# LOUDLY, since every state it fails on is one a retry fixes, then a read
# that degrades QUIETLY, since no retry turns "no value there" into a value.
# It is a store reader, and this is what the store's readers do: a cert
# login that fails LOUDLY, since every state it fails on is one a retry
# fixes, then a read that degrades QUIETLY, since no retry turns "no value
# there" into a value.
#
# ⚠️ An ABSENT secret is "not published yet", not a fault of this host's.
# The publisher runs in authelia's container and authelia mints on its

2
nix/host-modules/swarm-nats.nix
View file

@ -873,7 +873,7 @@ in
# terminal rows and its turn-state header
# (`hive-agent::swarm_agent_state`), two subject families so a
# subscriber following one agent's terminal is not also handed
# every header. The hive's shared agent client gets nothing.
# every header.
#
# `$$`, not `$`: systemd substitutes `$NAME` in `ExecStart`
# whether or not it is quoted, so a single dollar reaches the

11
nix/host-modules/swarm-otel.nix
View file

@ -232,12 +232,11 @@ let
haveCollectorSecret = deployCfg.swarm-otel.clientSecretFile != null;
# A reader of the store is defined by holding a certificate the store
# accepts, never by standing next to it — the rule
# ./glue-queue-agent-credential.nix states in full. Unlike Grafana's identical-
# accepts, never by standing next to it — the rule ./swarm-bao.nix's
# `deploy.bao` options block states in full. Unlike Grafana's identical-
# looking flag, this one does not gate an assertion on its own: the
# store-reading unit below simply does not render without it, the same choice
# ./glue-queue-agent-credential.nix makes for its own reader, because a collector with no client identity is
# `haveCollectorSecret = false` above, and that is already a supported,
# store-reading unit below simply does not render without it, because a
# collector with no client identity is `haveCollectorSecret = false` above, and that is already a supported,
# merely degraded shape rather than a service with no way in at all. What
# IS asserted is narrower and lives in `assertions` below — see
# `hiveReaderIdentity`.
@ -539,7 +538,7 @@ in
# exists so a host holds ONE out-of-band secret — its client certificate —
# and reads everything else with it. Recorded in docs/swarm/secrets.md.
#
# Shaped after ./glue-queue-agent-credential.nix: a cert login that fails
# Shaped after ./swarm-grafana.nix's store reader: a cert login that fails
# LOUDLY, since every state it fails on is one a retry fixes, then a read
# that degrades QUIETLY, since no retry turns "no value there" into a
# value.

39
nix/host-modules/swarm-secret-publisher.nix
View file

@ -12,15 +12,9 @@ let
hyperhiveCfg = config.services.hyperhive;
hiveNames = lib.attrNames hyperhiveCfg.swarm.hives;
# The client id agent containers present, per hive — composed exactly as
# ./swarm-authelia.nix composes it, from the same two read-only options, so a
# rename there cannot leave this spelling behind.
agentClientId =
hive:
"${hyperhiveCfg.swarm.authelia.hiveClientPrefix}${hive}${hyperhiveCfg.swarm.authelia.agentClientSuffix}";
# The hive's own client id, which hive-c0re publishes the hive's status with.
# Composed as ./swarm-authelia.nix composes it, for the reason above.
# Composed exactly as ./swarm-authelia.nix composes it, from the same
# read-only option, so a rename there cannot leave this spelling behind.
hiveClientId = hive: "${hyperhiveCfg.swarm.authelia.hiveClientPrefix}${hive}";
# The swarm's own services, as opposed to its hives. One client for the whole
@ -83,22 +77,14 @@ let
controllerSecretPath = "secret/swarm/controller/swarm-controller/oidc/client";
secrets =
# `client_id` beside the value: an agent's id is derived per hive and has
# to be reconstructable from the store alone.
map (hive: {
clientId = agentClientId hive;
path = "secret/swarm/hives/${hive}/queue/agent";
withClientId = true;
}) hiveNames
# Each hive's own client, which that hive's hive-c0re reads under its own
# certificate. The nix half of
# `swarm_secret_client::queue::hive_client_path`.
++ map (hive: {
map (hive: {
clientId = hiveClientId hive;
path = "secret/swarm/hives/${hive}/oidc/client";
}) hiveNames
# No `client_id`: a service's client id is the swarm-wide option both
# ends already read.
# A service's client id is the swarm-wide option both ends already read.
++ map (id: {
clientId = id;
path = "secret/swarm/services/${id}/oidc/client";
@ -120,9 +106,8 @@ in
default = false;
description = ''
Publish the OIDC client secrets this host mints into the swarm's
secret store, so a hive that does not run authelia can read its
own and its agents' credentials and a swarm service or the
controller can read its own — from wherever it runs, this host
secret store, so a hive that does not run authelia, a swarm service
or the controller can read its own — from wherever it runs, this host
included. The unit, `swarm-secret-publish`, runs inside authelia's
container, so it exists only where authelia runs.
@ -137,10 +122,9 @@ in
publishing nothing. Whether the wiring is complete is the client
identity's job (see `baoClientCertFile`), not this option's.
Turning it off leaves every hive but this one without its agents'
credential, every hive unable to publish its status, the swarm
controller unable to start, the swarm's Grafana without any login at
all, its collector pushing unauthenticated, the queue's responder
Turning it off leaves every hive unable to publish its status, the
swarm controller unable to start, the swarm's Grafana without any login
at all, its collector pushing unauthenticated, the queue's responder
unable to start, the homeserver unable to start, and the forge's login
source without its secret.
Each secret has exactly one route and this is the producer's
@ -185,9 +169,8 @@ in
internal = true;
default = secrets;
description = ''
Every secret the publisher copies: the minted client's id, the store
path its reader fetches, and whether `client_id` is stored beside the
value. Derived from `swarm.*` alone, so it is the same list on every
Every secret the publisher copies: the minted client's id and the store
path its reader fetches. Derived from `swarm.*` alone, so it is the same list on every
host; ./swarm-authelia.nix hands it to the unit in its container.
'';
};

20
nix/host-modules/swarm.nix
View file

@ -64,17 +64,14 @@ let
deployCfg.bao.notifyRouterCommonName
deployCfg.bao.forgePublisherCommonName
]
# The per-hive subjects, spelled out per hive rather than as the prefix.
# The per-hive subject, spelled out per hive rather than as the prefix.
# The prefix alone would reserve the wrong string: the role for hive
# `h` accepts `<prefix>-h`, so the name a hive must not BE is that composed
# spelling, and a hive named it would present a leaf that role accepts.
#
# Every spelling is reserved for every declared hive, including the one the
# CN belongs to — a hive cannot be named after its own reader either.
++ lib.concatMap (hive: [
"${deployCfg.bao.queueAgentCommonNamePrefix}-${hive}"
"${deployCfg.bao.hiveLeafCommonNamePrefix}-${hive}"
]) hiveNames;
# CN belongs to — a hive cannot be named after its own leaf issuer either.
++ map (hive: "${deployCfg.bao.hiveLeafCommonNamePrefix}-${hive}") hiveNames;
# Public hostnames of the swarm's own services, in declaration order.
# `serviceDomains` below is this set sorted + deduplicated.
@ -477,9 +474,8 @@ in
# its own network namespace and resolver, so this is its own option, even
# though by name it defaults to the same URL as `statusPublish.natsUrl`.
#
# Only the address lives here. The credential does not: it is published per
# hive and read out of the store by ./glue-queue-agent-credential.nix, which
# owns `queue.agentCredentialDir` in the same namespace.
# Only the address lives here. Each agent reads its own credential from the
# store (`nix/agent-modules/queue-identity.nix`).
options.services.hyperhive.deploy.hive-controller.queue.agentNatsUrl = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = "tls://${swarmCfg.nats.domain}:${toString swarmCfg.nats.port}";
@ -496,10 +492,8 @@ in
agent's network namespace `127.0.0.1` is the agent, not this host,
and the queue's certificate names no address anyway.
Null means this hive's agents have not been given the queue's address.
Together with
{option}`services.hyperhive.swarm.statusPublish.tokenEndpoint` it is
what decides whether the harness is handed queue coordinates at all.
Null means this hive's agents have not been given the queue's address,
and the harness is handed none.
'';
};

30
nix/module-eval/agent-queue-bao.nix
View file

@ -24,16 +24,13 @@ let
agentHarness
;
# The agent side of the swarm queue. Both coordinates set is the only state
# in which the harness unit declares a credential at all, so the pair and
# the empty fixture beside it are the two arms worth having.
# An `agent.nix` still setting the queue coordinates the agent module no
# longer declares.
agentQueue = agent {
queue.natsUrl = "nats://10.42.0.1:4222";
queue.tokenEndpoint = "https://auth.t.local/api/oidc/token";
};
agentNoQueue = shared.agentBare;
# The agent side of the swarm secret store. The address is the whole switch —
# it is both what generates the login check and what that check points at —
# so it and the empty fixture beside it are the two arms worth having.
@ -106,21 +103,24 @@ let
ok = swarmMetricsIn agentLogsNoBao == null && swarmMetricsIn agentLogs != null;
}
{
# Both ids or neither: the secret authenticates nobody without the id it
# belongs to, and `swarm-logs` reads both.
name = "an agent with queue coordinates imports both halves of its credential";
# The hive forwards no shared queue credential, so the harness unit
# names none, whatever the agent's config says.
name = "an agent imports no hive-shared queue credential";
ok =
let
c = (agentHarness agentQueue).serviceConfig.LoadCredential;
credentials = m: (agentHarness m).serviceConfig.LoadCredential or [ ];
in
builtins.elem "hive-queue-agent-secret" c && builtins.elem "hive-queue-agent-client-id" c;
!lib.any (lib.hasPrefix "hive-queue-agent-") (credentials agentQueue ++ credentials agentBao)
# Control: the lookup finds the store identity's credential.
&& lib.elem "hive-agent-bao-server-ca" (credentials agentBao);
}
{
# An agent built before its hive was handed the queue's address. It
# must declare nothing rather than name a credential that never
# arrives.
name = "an agent with no queue coordinates declares no credential";
ok = !((agentHarness agentNoQueue).serviceConfig ? LoadCredential);
# Both options are refused, naming where the address
# comes from now, rather than dropped silently.
name = "an agent setting the removed queue options is refused";
ok =
lib.count (a: !a.assertion && lib.hasInfix "HIVE_AGENT_NATS_URL" a.message) agentQueue.assertions
== 2;
}
{
# The harness exits when it cannot reach the queue. It has to come back

60
nix/module-eval/bao-grants.nix
View file

@ -106,14 +106,11 @@ let
deploy.matrix.enable = true;
};
# The store with none of the readers beside it. Grafana and the collector are
# simply off; the queue reader renders on any host holding
# its leaf, which ./glue-bao-tls.nix mints here, so that leaf is taken away.
# The store with none of the readers beside it: Grafana and the collector are
# off.
baoGrantNoReaders = hiveWithController {
deploy.bao.enable = true;
deploy.bao.bootstrapTokenFile = "/run/secrets/bao-bootstrap.token";
deploy.bao.queueAgentClientCertFile = lib.mkForce null;
deploy.bao.queueAgentClientKeyFile = lib.mkForce null;
};
# The store with the forwarder's own pair taken away. The forwarder renders
@ -131,8 +128,6 @@ let
deploy.swarm-otel.enable = true;
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
deploy.bao.queueAgentClientCertFile = "/etc/pki/bao-queue-agent.pem";
deploy.bao.queueAgentClientKeyFile = "/etc/pki/bao-queue-agent-key.pem";
deploy.bao.grafanaOidcClientCertFile = "/etc/pki/bao-grafana-oidc.pem";
deploy.bao.grafanaOidcClientKeyFile = "/etc/pki/bao-grafana-oidc-key.pem";
deploy.bao.otelOidcClientCertFile = "/etc/pki/bao-otel-oidc.pem";
@ -142,7 +137,6 @@ let
# The readers ./glue-bao-readers-policy-order.nix orders after their policy
# units.
policyReaders = [
"swarm-bao-queue-agent"
"swarm-bao-grafana-oidc"
"swarm-bao-otel-oidc"
];
@ -175,7 +169,6 @@ let
"swarm-bao-controller-policy"
"swarm-bao-secret-publisher-policy"
"swarm-bao-matrix-ctl-policy"
"swarm-bao-queue-agent-policy"
"swarm-bao-grafana-oidc-policy"
"swarm-bao-otel-oidc-policy"
"swarm-bao-forwarder-oidc-policy"
@ -599,19 +592,18 @@ let
# arms pin what each principal must NOT reach, so a later widening fails
# here instead of being noticed in a store.
{
name = "the queue-credential reader's grant is one hive's queue credential and nothing else";
# A hive holds no shared queue credential for its agents: no unit writes
# a role or policy for one, none reads one, and no granting unit names
# its path. The control: the same scan finds a reader's grant.
name = "no unit grants, reads or writes a hive-shared agent queue credential";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-queue-agent-policy.script;
scripts = lib.mapAttrsToList (_: u: u.script or "") baoGrantWithConsumers.systemd.services;
in
lib.hasInfix "path \"secret/data/swarm/hives/h1/queue/agent\" {" s
&& lib.hasInfix "capabilities = [\"read\"]" s
&& !(lib.hasInfix "secret/data/swarm/agents" s)
&& !(lib.hasInfix "secret/data/swarm/services" s)
&& !(lib.hasInfix "secret/data/swarm/hives/h1/*" s)
&& !(lib.hasInfix "secret/data/swarm/hives/h1/matrix" s)
&& !(lib.hasInfix "secret/data/swarm/hives/*" s)
&& !(lib.hasInfix "sys/policies/acl" s);
!(baoGrantWithConsumers.systemd.services ? swarm-bao-queue-agent-policy)
&& !(baoGrantWithConsumers.systemd.services ? swarm-bao-queue-agent)
&& !(lib.any (lib.hasInfix "/queue/agent") scripts)
&& lib.any (lib.hasInfix "/oidc/client") scripts;
}
{
# ⚠️ The client id is the path segment, so the negative arm that matters
@ -856,11 +848,7 @@ let
# through a token that carries it, and a token is minted by a cert-auth
# role matching a CN. Distinct subjects is the whole mechanism — one
# subject for every reader is one principal however the policies read.
#
# The per-hive subject carries the hive name because its path does; the
# service subjects do not, because an OIDC client is registered once
# per swarm. Pinned so neither shape is tidied into the other.
name = "each of the four readers logs in under a subject of its own";
name = "each of the three readers logs in under a subject of its own";
ok =
let
subjectOf =
@ -875,8 +863,7 @@ let
# the reason the three service principals above state.
&& !(lib.hasInfix "auth/cert/certs/hive-" s);
in
subjectOf "swarm-bao-queue-agent-policy" "swarm-queue-agent-h1" "swarm-bao-queue-agent-h1"
&& subjectOf "swarm-bao-grafana-oidc-policy" "swarm-grafana-oidc" "swarm-bao-grafana-oidc"
subjectOf "swarm-bao-grafana-oidc-policy" "swarm-grafana-oidc" "swarm-bao-grafana-oidc"
&& subjectOf "swarm-bao-otel-oidc-policy" "swarm-otel-oidc" "swarm-bao-otel-oidc"
&& subjectOf "swarm-bao-forwarder-oidc-policy" "swarm-forwarder-oidc" "swarm-bao-forwarder-oidc";
}
@ -889,7 +876,7 @@ let
#
# Each pair is asserted whole: a certificate with no key authenticates
# nothing, so a half-set pair is a reader that does not render.
name = "each of the four readers presents its own leaf, never the hive's";
name = "each of the three readers presents its own leaf, never the hive's";
ok =
let
b = baoGrantWithConsumers.services.hyperhive.deploy.bao;
@ -898,8 +885,6 @@ let
b.clientKeyFile
];
own = [
b.queueAgentClientCertFile
b.queueAgentClientKeyFile
b.grafanaOidcClientCertFile
b.grafanaOidcClientKeyFile
b.otelOidcClientCertFile
@ -915,7 +900,6 @@ let
lib.all (p: p != null) own
&& !(lib.any (p: lib.elem p hiveLeaf) own)
&& lib.length (lib.unique own) == lib.length own
&& presents "swarm-bao-queue-agent" b.queueAgentClientCertFile b.queueAgentClientKeyFile
&& presents "swarm-bao-grafana-oidc" b.grafanaOidcClientCertFile b.grafanaOidcClientKeyFile
&& presents "swarm-bao-otel-oidc" b.otelOidcClientCertFile b.otelOidcClientKeyFile
&& presents "swarm-bao-forwarder-oidc" b.forwarderOidcClientCertFile b.forwarderOidcClientKeyFile;
@ -923,9 +907,7 @@ let
{
# The minting side of the same claim. A role matching a subject nothing
# signs is a reader that cannot log in, so the leaves and the roles have
# to be asserted against each other — and the per-hive leaf carries
# THIS host's hive name, which is what makes one hive's leaf useless
# against another hive's role.
# to be asserted against each other.
name = "the PKI unit signs a leaf per reader, each under that reader's own subject";
ok =
let
@ -936,8 +918,6 @@ let
# literal spanning both would pin this file's line wrapping rather than
# the pairing it means to.
lib.all (lib.flip lib.hasInfix s) [
"/queue-agent.pem ]"
"swarm-bao-queue-agent-h1 \"\" clientAuth"
"/grafana-oidc.pem ]"
"swarm-bao-grafana-oidc \"\" clientAuth"
"/otel-oidc.pem ]"
@ -965,11 +945,10 @@ let
# client certificate and the host is the side that has one, so a unit
# rendered inside the store's container would have neither an identity
# nor a route. Plus the ordering that makes the mounts exist first.
name = "the four readers' granting units are ordered after the mounts and rendered on the host";
name = "the three readers' granting units are ordered after the mounts and rendered on the host";
ok =
let
units = [
"swarm-bao-queue-agent-policy"
"swarm-bao-grafana-oidc-policy"
"swarm-bao-otel-oidc-policy"
"swarm-bao-forwarder-oidc-policy"
@ -990,7 +969,7 @@ let
# The forwarder is listed apart from `policyReaders`: it renders wherever
# the store does, so it is never absent on a store host and never present
# on a remote one, and the two cases below would fail on it for that.
name = "each of the four readers is ordered after the unit writing its role";
name = "each of the three readers is ordered after the unit writing its role";
ok =
let
s = baoGrantWithConsumers.systemd.services;
@ -1483,9 +1462,9 @@ let
}
{
# The other hives' side: every hive in the directory but this one gets
# the same three leaves, under its own name, in a directory of its own
# the same two leaves, under its own name, in a directory of its own
# that the operator copies to its host.
name = "the PKI unit signs every other hive's reader, queue-agent and gateway-leaf identities";
name = "the PKI unit signs every other hive's reader and gateway-leaf identities";
ok =
let
s = baoGrantTwoHives.systemd.services.swarm-bao-pki.script;
@ -1494,7 +1473,6 @@ let
file: cn: lib.hasInfix "/hives/h2/${file}.pem ]" s && lib.hasInfix " ${cn} \"\" clientAuth" s;
in
signs "client" "h2"
&& signs "queue-agent" "${b.queueAgentCommonNamePrefix}-h2"
&& signs "hive-leaf" "${b.hiveLeafCommonNamePrefix}-h2"
# This hive's own leaves stay at the top of the directory.
&& !(lib.hasInfix "/hives/h1/" s);

184
nix/module-eval/bao-matrix-reader.nix
View file

@ -31,18 +31,10 @@ let
# nothing here mints a leaf and the operator names one placed by hand. The
# deployment this pairing exists to serve, and the one that was previously
# inexpressible — the gate asked whether the store was a neighbour.
#
# 🩸 One leaf PER READER, and this fixture is where that cost is visible: the
# queue-credential unit identifies itself to the store separately from the
# hive, so an operator placing leaves by hand places one for it as well as
# the hive's own. Naming only `clientCertFile` here would leave the unit
# unrendered — which is what the arms below would then be asserting about.
baoRemoteReader = hiveWithController {
deploy.matrix.enable = true;
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
deploy.bao.queueAgentClientCertFile = "/etc/pki/bao-queue-agent.pem";
deploy.bao.queueAgentClientKeyFile = "/etc/pki/bao-queue-agent-key.pem";
};
# A homeserver on a hive with NO store identity at all — neither a local
@ -51,24 +43,15 @@ let
# fixture set its own, as ./lib.nix asks.
inherit (shared) matrixNoBaoIdentity;
# 🩸 The hand-configured remote reader that named the hive's leaf and stopped.
# `baoRemoteReader` above is the same deployment done right; this one holds
# the hive's own leaf, so it demonstrably reads the store, and is missing the
# queue reader's pair. Without the refusal it would render no queue reader
# and say nothing — which is what the refusal arms below are about.
# 🩸 The hand-configured remote reader that named the hive's leaf and stopped:
# it holds the hive's own leaf, so it demonstrably reads the store, and is
# missing matrix-ctl's pair.
baoRemoteReaderMissingLeaves = hiveWithController {
deploy.matrix.enable = true;
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
};
# The same omission on a hive that does NOT run a homeserver: the queue
# refusal checks only the hive's own toggle.
remoteReaderNoMatrixMissingLeaves = hiveWithController {
deploy.bao.clientCertFile = "/etc/pki/bao-client.pem";
deploy.bao.clientKeyFile = "/etc/pki/bao-client-key.pem";
};
# The store's host with the hive's own reader pointed somewhere else.
matrixOnStoreHostReaderMoved = hiveWithController {
deploy.bao.enable = true;
@ -77,122 +60,15 @@ let
deploy.bao.clientKeyFile = "/etc/pki/h1-key.pem";
};
# Did a module refuse this host, and over which option. An assertion is a
# config VALUE until something forces it — `.config` never throws — so a
# fixture in a state the module refuses stays evaluable and the refusal reads
# back as data. Matched on the option name the message names rather than on
# its prose, because the option name is the part an operator has to act on
# and a message that stopped naming it would be the actual defect. Same shape
# as ./grafana.nix's `grafanaRefusedFor`, which reads `swarm-grafana.nix`'s
# own refusal.
refusedOver = m: option: lib.any (a: !a.assertion && lib.hasInfix option a.message) m.assertions;
cases = [
{
# A login failure is the store being unreachable, sealed, or not yet
# holding this host's role — all of which a retry fixes. A read that
# answers "nothing there" is not, so only the first is allowed to fail
# the unit.
name = "the queue credential reader retries a failed login and still degrades on an empty read";
ok =
let
u = baoWithMatrix.systemd.services.swarm-bao-queue-agent;
# Everything between the login's failure branch and the read's, which
# is where the exit that decides "retry or give up" lives.
afterLogin = lib.last (lib.splitString "bao login" u.script);
loginBranch = lib.head (lib.splitString "bao kv get" afterLogin);
in
u.serviceConfig.Restart or null == "on-failure"
&& u.startLimitBurst or 0 > 0
# The window has to outlast every attempt, or the burst is unreachable.
&& u.startLimitIntervalSec or 0 > (u.serviceConfig.RestartSec or 0) * (u.startLimitBurst or 0)
&& lib.hasInfix "exit 1" loginBranch
&& lib.hasInfix "exit 0" (lib.last (lib.splitString "bao kv get" u.script));
}
{
# The reader's own grant covers `swarm/hives/<this hive>/*`; a path
# outside it answers 403, not "no such key". This is the path
# `swarm_secret_client::queue::agent_client_path` builds from the same
# pieces. The negative arm is the rename this one is exposed to: a
# credential named for the queue rather than for the hive that presents
# it reads as correct and is refused on every boot.
name = "the agent queue credential path sits inside the prefix the reader is granted";
ok =
let
s = baoRemoteReader.systemd.services.swarm-bao-queue-agent.script;
in
lib.hasInfix "secret/swarm/hives/h1/queue/agent" s && !(lib.hasInfix "secret/swarm/queue/" s);
}
{
# The unit's output is the option's value, not a literal that agrees with
# it today: an operator moving the directory has to move both files. The
# prefix is asserted too because `hasInfix ""` is true — an option
# renamed out from under this arm would otherwise read empty and pass.
name = "the queue credential reader writes both files under the directory its option names";
ok =
let
m = baoRemoteReader;
dir = toString m.services.hyperhive.deploy.hive-controller.queue.agentCredentialDir;
s = m.systemd.services.swarm-bao-queue-agent.script;
in
lib.hasPrefix "/var/lib/" dir
&& lib.hasInfix "${dir}/secret" s
&& lib.hasInfix "${dir}/client_id" s;
}
{
# No agent container may render before this unit has had its attempts,
# and the edge that guarantees it must delay hive-c0re rather than sink
# it: an unreachable store is this unit's `Restart=on-failure` window,
# not a reason for the daemon that renders every agent to fail its own
# start.
name = "the queue credential reader orders before hive-c0re and is wanted, not required, by it";
ok =
let
u = baoRemoteReader.systemd.services.swarm-bao-queue-agent;
in
builtins.elem "hive-c0re.service" (u.before or [ ])
&& builtins.elem "hive-c0re.service" (u.wantedBy or [ ])
&& !(builtins.elem "hive-c0re.service" (u.requiredBy or [ ]))
&& !(builtins.elem "hive-c0re.service" (u.requires or [ ]));
}
{
# Nothing local mints certificates off-host, so this orders against
# nothing there.
name = "an off-host reader names no unit the store's host would have provided";
# Membership first, then the value: indexing a missing unit throws, and a
# table that reports which property broke must not be the thing that dies.
ok =
let
s = baoRemoteReader.systemd.services;
u = s.swarm-bao-queue-agent;
in
s ? swarm-bao-queue-agent && !(lib.elem "swarm-bao-pki.service" (u.after ++ u.wants ++ u.requires));
}
{
# Presence control for the case above: the list is conditional, not gone.
# Wanted, never required: a refused pki start fails a requirer's job with
# `dependency`, which its `Restart=on-failure` never retries.
name = "a co-located reader still orders after and wants the local pki unit, and does not require it";
ok =
let
s = baoWithMatrix.systemd.services;
u = s.swarm-bao-queue-agent;
in
s ? swarm-bao-queue-agent
&& lib.elem "swarm-bao-pki.service" u.after
&& lib.elem "swarm-bao-pki.service" u.wants
&& u.requires == [ ];
}
{
# Nothing asserted this script before, which is how it kept a branch that
# named three states and threw away the only thing telling them apart. A
# missing value, a refused identity and an unreachable host all end in the
# same degraded mode here, correctly — what must survive is which one.
name = "the queue credential reader carries the store's own diagnostic into the journal";
ok =
let
s = baoWithMatrix.systemd.services.swarm-bao-queue-agent.script;
in
!(lib.hasInfix "2>/dev/null" s) && lib.hasInfix ''cat "''$err"'' s;
# A hive holds no shared queue credential for its agents, wherever the
# store runs. The control: hive-c0re renders on both hosts.
name = "no hive renders a reader for a hive-shared agent queue credential";
ok = lib.all (m: !(m.systemd.services ? swarm-bao-queue-agent) && m.systemd.services ? hive-c0re) [
baoWithMatrix
baoRemoteReader
];
}
{
# hive-c0re runs as hive-core and the client key is `0600` root-owned
@ -404,46 +280,6 @@ let
&& !(c.systemd.services ? swarm-matrix-appservice-publish)
&& !(lib.any (lib.hasPrefix "swarm-appservice.yaml") c.systemd.services.tuwunel.serviceConfig.LoadCredential);
}
{
# 🩸 The arm the whole refusal exists for. The reader is gated on its own
# leaf, so the way this regresses is the build going green and the unit
# not rendering — which no presence check on a rendered unit can see,
# because the unit that is missing is the evidence. Read as a refusal
# naming each option, so an operator acts on the message without opening
# the nix.
name = "a remote reader missing the queue reader's leaf is refused, naming both options";
ok =
refusedOver baoRemoteReaderMissingLeaves "queueAgentClientCertFile"
&& refusedOver baoRemoteReaderMissingLeaves "queueAgentClientKeyFile";
}
{
name = "the queue refusal needs no homeserver";
ok = refusedOver remoteReaderNoMatrixMissingLeaves "queueAgentClientCertFile";
}
{
# ⚠️ The arm that keeps the refusal from being worse than the silence it
# replaced. A hive with NO store identity is the supported no-store
# deployment and also the state an operator passes through bringing a hive
# up — neither may fail to evaluate. Asserted as "no refusal names either
# option", not as "this one fixture is fine", because the way this breaks
# is a gate widened to the principal's leaf alone.
name = "a hive with no store identity at all is refused over none of the per-principal leaves";
ok = lib.all (option: !(refusedOver matrixNoBaoIdentity option)) [
"queueAgentClientCertFile"
"queueAgentClientKeyFile"
];
}
{
# The other half of the same guard, and the one an operator meets far more
# often: on the store's own host ./host-modules/glue-bao-tls.nix mkDefaults
# every pair, so there is nothing to name and nothing to refuse. Paired with
# the fully-named remote reader, which is the same deployment done by hand.
name = "neither a store host nor a correctly-named remote reader is refused";
ok = lib.all (m: !(refusedOver m "queueAgentClientCertFile")) [
baoWithMatrix
baoRemoteReader
];
}
];
in
runGroup "bao-matrix-reader" cases

42
nix/module-eval/core-toggle.nix
View file

@ -216,13 +216,17 @@ let
&& env shared.controllerAlone == { };
}
{
# Where the reader puts the files and where the daemon looks for them is
# one agreement spanning two modules. Asserted against the option rather
# than the literal so moving the directory moves both ends.
name = "hive-c0re is told where the agents' queue credential lands";
# An agent connects to the queue with its own secret, so the hive hands
# its agents no shared credential and no token endpoint to mint one at.
# The control: the same environment carries the agents' queue address.
name = "hive-c0re forwards no shared agent queue credential";
ok =
allLocal.systemd.services.hive-c0re.environment.HIVE_C0RE_AGENT_QUEUE_CREDENTIAL_DIR
== toString allLocal.services.hyperhive.deploy.hive-controller.queue.agentCredentialDir;
let
e = allLocal.systemd.services.hive-c0re.environment;
in
!(e ? HIVE_C0RE_AGENT_QUEUE_CREDENTIAL_DIR)
&& !(e ? HIVE_AGENT_OIDC_TOKEN_ENDPOINT)
&& e ? HIVE_AGENT_NATS_URL;
}
{
# Inside an agent container loopback is the agent itself, so the agents'
@ -256,28 +260,10 @@ let
&& lib.elem "/hive-collector.hive/${bridge}" allLocal.services.dnsmasq.settings.address;
}
{
# The agents mint against the swarm's IdP, the same endpoint the hive's
# own client uses — a hive-local guess would produce a token the queue
# would not accept.
name = "the agents' token endpoint is the swarm IdP's";
ok =
let
e = allLocal.systemd.services.hive-c0re.environment;
in
lib.hasSuffix "/api/oidc/token" e.HIVE_AGENT_OIDC_TOKEN_ENDPOINT
&& e.HIVE_AGENT_OIDC_TOKEN_ENDPOINT == e.HIVE_C0RE_OIDC_TOKEN_ENDPOINT;
}
{
# The absence arm, and what makes the two above able to fail: a hive
# with no queue address for its agents must forward neither coordinate,
# because half a pair reaches the harness as a partial configuration
# rather than as none.
name = "a hive with no agent queue address forwards no agent queue coordinates";
ok =
let
e = noAgentQueue.systemd.services.hive-c0re.environment;
in
!(e ? HIVE_AGENT_NATS_URL) && !(e ? HIVE_AGENT_OIDC_TOKEN_ENDPOINT);
# The absence arm, and what makes the agents' queue address cases above able
# to fail.
name = "a hive with no agent queue address forwards none";
ok = !(noAgentQueue.systemd.services.hive-c0re.environment ? HIVE_AGENT_NATS_URL);
}
{
# Control: `bare` sets none of them, and its serve.json carries the

7
nix/module-eval/matrix-core.nix
View file

@ -37,13 +37,6 @@ let
# homeserver across the network.
matrixElsewhere = shared.bareWithController;
cases = [
{
# Absence arm, and what makes the one above able to fail: with no leaf
# this unit would fail a TLS handshake on every boot, so it must not
# exist at all rather than retry its way through the start limit.
name = "a hive with no store identity renders no queue credential reader";
ok = !(matrixNoBaoIdentity.systemd.services ? swarm-bao-queue-agent);
}
{
# Two limits governed a matrix upload and nothing kept them in agreement,
# so raising the documented one past the gateway's hardcoded cap changed

29
nix/module-eval/name-guards.nix
View file

@ -43,14 +43,12 @@ let
# leaves, not a whole prefix, so a hive that inherited a dead entry there
# would silently gain the ability to overwrite the swarm's matrix credential.
#
# 🩸 The two per-hive subjects have a different shape: those roles are
# written PER HIVE, so the subject a hive must not be is
# `<prefix>-<some hive's name>` rather than the prefix itself. Reserving only
# the prefix would leave the composed spelling free, and a hive taking it
# would present a leaf the other hive's role accepts — a hive reading another
# hive's queue credential, the exact widening the split exists to avoid. The
# collision is with the OTHER hive's role, hence `other` beside `qr-other`
# and `hl-other`.
# 🩸 The per-hive subject has a different shape: that role is written PER
# HIVE, so the subject a hive must not be is `<prefix>-<some hive's name>`
# rather than the prefix itself. Reserving only the prefix would leave the
# composed spelling free, and a hive taking it would present a leaf the other
# hive's role accepts. The collision is with the OTHER hive's role, hence
# `other` beside `hl-other`.
hiveNamedAfterEverySubject = hive {
deploy.swarm-otel.enable = false;
deploy.bao = {
@ -62,7 +60,6 @@ let
forwarderOidcCommonName = "fwctl";
forgePublisherCommonName = "fpctl";
granterCommonName = "grctl";
queueAgentCommonNamePrefix = "qr";
hiveLeafCommonNamePrefix = "hl";
};
swarm.hives = {
@ -75,7 +72,6 @@ let
fpctl.domain = "fp.t.local";
grctl.domain = "gr.t.local";
other.domain = "other.t.local";
qr-other.domain = "q.t.local";
hl-other.domain = "hl.t.local";
};
};
@ -216,16 +212,9 @@ let
{
# 🩸 The per-hive half, and the one a prefix-only reservation would miss:
# the role is `<prefix>-<hive>`, so the reserved string has to be composed
# against every declared hive. Here hive `qr-other` collides with the role
# written for hive `other` — a leaf that reads a credential belonging to a
# hive that is not it.
name = "a hive named after another hive's per-hive reader subject is refused";
ok = equalityGuardRefuses hiveNamedAfterEverySubject "qr-other";
}
{
# A hive taking this spelling would present a leaf hive `other`'s
# gateway-leaf role accepts, and could ask the store for that hive's
# certificate. The control is the legal roster below.
# against every declared hive. A hive taking this spelling would present
# a leaf hive `other`'s gateway-leaf role accepts, and could ask the store
# for that hive's certificate. The control is the legal roster below.
name = "a hive named after another hive's gateway-leaf subject is refused";
ok = equalityGuardRefuses hiveNamedAfterEverySubject "hl-other";
}

5
nix/module-eval/nats-tls.nix
View file

@ -77,13 +77,10 @@ let
# The same hive given its own store leaf by hand, which is what turns
# publishing on away from the IdP's host: hive-c0re reads the status secret
# from the store under it. The queue-credential reader's pair comes with it,
# because a hive that reads the store is refused at eval without one.
# from the store under it.
hiveLeaf = {
deploy.bao.clientCertFile = "/etc/pki/h1.pem";
deploy.bao.clientKeyFile = "/etc/pki/h1-key.pem";
deploy.bao.queueAgentClientCertFile = "/etc/pki/h1-queue-agent.pem";
deploy.bao.queueAgentClientKeyFile = "/etc/pki/h1-queue-agent-key.pem";
};
remotePublishing = hiveWithController (lib.recursiveUpdate remoteSecrets hiveLeaf);

20
nix/module-eval/secret-publisher.nix
View file

@ -80,7 +80,6 @@ let
# Each hive's hive-c0re reads this exact path
# (`swarm_secret_client::queue::hive_client_path`, pinned by that crate's
# own test), for every hive in the roster, not only the one this host is.
# The agents' client beside it stays a separate path.
name = "the publisher writes every hive's own client under that hive's prefix";
ok =
let
@ -91,8 +90,7 @@ let
[
"h1"
"h2"
]
&& lib.hasInfix "secret/swarm/hives/h1/queue/agent" s;
];
}
{
# The controller reads this exact path
@ -333,17 +331,17 @@ let
lib.hasInfix "value=@" code && !(lib.hasInfix "$(cat" code);
}
{
# Two ends of a wire nothing at eval time carries end to end: this is the
# path `swarm_secret_client::queue` resolves for the reader. Both hives
# are asserted, so a publisher that knew one name rather than the roster
# fails here rather than on the second hive ever added to a swarm.
name = "the publisher writes every hive in the roster to that hive's own queue path";
# No hive holds a shared queue credential for its agents, so authelia
# mints no agent client and the publisher writes no such path. The
# control: the same script writes each hive's own client.
name = "the publisher writes no hive-shared agent queue credential";
ok =
let
s = (inAuthelia secretPublisherHere).systemd.services.swarm-secret-publish.script;
in
lib.hasInfix "secret/swarm/hives/h1/queue/agent" s
&& lib.hasInfix "secret/swarm/hives/h2/queue/agent" s;
!(lib.hasInfix "/queue/agent" s)
&& !(lib.hasInfix "-agent.secret" s)
&& lib.hasInfix "secret/swarm/hives/h1/oidc/client" s;
}
{
# A property of the SET, not of one unit: both of these authenticate by
@ -373,7 +371,7 @@ let
&& !(lib.hasInfix "bao kv" (lib.head (lib.splitString "bao login" c)));
in
holdsTokenFirst (inAuthelia secretPublisherHere).systemd.services.swarm-secret-publish.script
&& holdsTokenFirst baoWithMatrix.systemd.services.swarm-bao-queue-agent.script
&& holdsTokenFirst baoWithMatrix.systemd.services.swarm-bao-matrix-oidc.script
# Controls, so a clean verdict above means something. In order: a bare
# read is refused, a read placed before the login is refused, and a
# login that exists only in a comment is refused — that last one is the

4
swarm-secret-client/src/matrix.rs
View file

@ -95,7 +95,7 @@ pub fn hive_localpart(hive: &str) -> String {
/// The path holding `hive`'s matrix sender account's homeserver access token.
///
/// Keyed per **hive**, like [`crate::queue::agent_client_path`]. Under
/// Keyed per **hive**, like [`crate::queue::hive_client_path`]. Under
/// [`Kind::Hive`] the grant that reaches it is the hive's **own** stanza,
/// `secret/data/swarm/hives/<this hive>/*`, which interpolates the name: hive
/// `alpha` reads its own token and gets a 403 on `beta`'s.
@ -137,7 +137,7 @@ pub fn swarm_appservice_token_path() -> Result<String, Error> {
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct Credential {
/// Named `value` like [`crate::forge::Credential::value`] and
/// [`crate::queue::Credential::value`], so a nix-side reader spells
/// [`crate::queue::AgentCredential::value`], so a nix-side reader spells
/// `bao kv get -field=value` for every kind.
pub value: String,

7
swarm-secret-client/src/mtls.rs
View file

@ -40,10 +40,9 @@ pub fn identity_path(agent: &str) -> Result<String, Error> {
/// was issued from.
///
/// All three, because a reader has to reconstruct a usable identity from the
/// store alone — the same requirement [`crate::queue::Credential`] states for
/// carrying its client id. The authority rides along so the cert-auth role
/// and the certificate cannot be delivered from two different sources and
/// silently disagree; it is public material, unlike the other two fields.
/// store alone. The authority rides along so the cert-auth role and the
/// certificate cannot be delivered from two different sources and silently
/// disagree; it is public material, unlike the other two fields.
///
/// ⚠️ **No `Debug` derive.** See the hand-written impl below: one of these
/// fields is a private key, and a derived `Debug` would put it in any log line

2
swarm-secret-client/src/path.rs
View file

@ -11,7 +11,7 @@ use crate::Error;
/// The KV v2 mount every swarm secret lives under.
///
/// A literal because the nix-side readers spell the same one, as in
/// `glue-queue-agent-credential.nix`; an option nobody sets would be two ways
/// `nix/agent-modules/bao.nix`; an option nobody sets would be two ways
/// to say one thing.
pub const MOUNT: &str = "secret";

2
swarm-secret-client/src/policy.rs
View file

@ -231,7 +231,7 @@ mod tests {
assert!(
stanza_paths(&p)
.iter()
.any(|g| covers(g, "swarm/hives/pr1ma/queue/agent"))
.any(|g| covers(g, "swarm/hives/pr1ma/oidc/client"))
);
}

137
swarm-secret-client/src/queue.rs
View file

@ -1,21 +1,10 @@
//! The queue agreements: where a queue credential lives in the store, and what
//! the object at that path holds.
//!
//! **Two kinds live here, and which is which is the thing to get right.**
//!
//! - [`agent_client_path`] / [`Credential`] are keyed per **hive**: one OIDC
//! client minted per hive at deploy time, shared by every agent container on
//! it. It says which hive a caller belongs to, never which agent.
//! - [`agent_queue_path`] / [`AgentCredential`] are keyed per **agent**: a
//! secret the swarm mints for one agent, at swarm level, with no hive in the
//! chain. It is what makes one agent distinguishable from its co-hived
//! neighbours.
//!
//! They are separate objects rather than one with a nullable field because they
//! are minted by different principals on different events — the hive-scoped one
//! by authelia at deploy time, the per-agent one by `swarm-controller` when an
//! agent is created — and an object whose shape depends on who wrote it is a
//! reader that has to guess.
//! An agent's credential, [`agent_queue_path`] / [`AgentCredential`], is keyed
//! per **agent**: a secret the swarm mints for one agent, at swarm level, with
//! no hive in the chain. It is what makes one agent distinguishable from its
//! co-hived neighbours.
//!
//! The per-agent secret is deliberately **not** derived from the agent's mTLS
//! identity (the leaf at [`crate::mtls::identity_path`]). That leaf is for
@ -33,17 +22,6 @@ use crate::{
path::{Kind, principal_prefix},
};
/// The path holding the client secret that agent containers on `hive` present
/// to the swarm queue.
///
/// # Errors
/// [`Error::PathSegment`] when `hive` contains anything but `[A-Za-z0-9_-]`,
/// which is what keeps one hive's name from addressing another hive's secret.
pub fn agent_client_path(hive: &str) -> Result<String, Error> {
let prefix = principal_prefix(Kind::Hive, hive)?;
Ok(format!("{prefix}/queue/agent"))
}
/// The path holding the secret **one agent** presents to the swarm queue.
///
/// Under the agent's own principal prefix, and that is the whole reason for
@ -89,8 +67,7 @@ pub struct ControllerCredential {
/// The path holding `hive`'s own OIDC client secret: the `hive-<hive>`
/// identity hive-c0re publishes the hive's status to the swarm queue with.
///
/// Not [`agent_client_path`], which is the agents' client on the same hive.
/// Both sit under the hive's own prefix, which the hive's policy reads
/// Under the hive's own prefix, which the hive's policy reads
/// ([`crate::policy::render`]).
///
/// The nix half is `swarm-secret-publisher.nix`, which writes it.
@ -117,8 +94,8 @@ pub struct HiveClientCredential {
/// field still decode, because unknown fields are ignored.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct AgentCredential {
/// The secret itself. Named to match [`Credential::value`] and
/// [`crate::matrix::Credential::value`] so a nix-side reader spells
/// The secret itself. Named to match [`crate::matrix::Credential::value`]
/// so a nix-side reader spells
/// `bao kv get -field=value` for every kind.
pub value: String,
@ -136,79 +113,10 @@ pub struct AgentCredential {
pub minted_at: Option<i64>,
}
/// What the path holds: the client secret, plus the client id it belongs to.
///
/// The id rides with the secret for the same reason the homeserver rides with
/// a matrix token — a credential has to be reconstructable from the store
/// alone. Deriving it on the reading side instead would mean spelling
/// `hive-<name>-agent` in a second place, and the authelia module's own option
/// says what a split spelling costs: every agent is denied as a timeout.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct Credential {
/// The secret itself. Named to match [`crate::matrix::Credential::value`]
/// so a nix-side reader spells `bao kv get -field=value` for either kind.
pub value: String,
/// The OIDC client id the secret authenticates.
///
/// Required, unlike [`crate::matrix::Credential::homeserver`]: that field
/// is optional because it was added to objects already in the store, and
/// this path is new. An object without an id is not a usable credential,
/// so failing to decode is the wanted behaviour — the alternative is a
/// reader holding `None` whose only recourse is to spell the id itself.
pub client_id: String,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_hive_name_lands_under_its_own_principal_prefix() {
assert_eq!(
agent_client_path("alpha").expect("a plain name is legal"),
"swarm/hives/alpha/queue/agent"
);
}
#[test]
fn a_traversal_in_the_hive_name_is_refused() {
let e = agent_client_path("../beta").expect_err("a traversal is not");
assert!(matches!(e, Error::PathSegment { kind: "hive", .. }), "{e}");
}
#[test]
fn two_hives_never_share_a_path() {
assert_ne!(
agent_client_path("alpha").expect("legal"),
agent_client_path("beta").expect("legal")
);
}
#[test]
fn the_object_round_trips_through_the_store_representation() {
let c = Credential {
value: "s3cr3t".to_owned(),
client_id: "hive-alpha-agent".to_owned(),
};
let json = serde_json::to_string(&c).expect("serialises");
assert_eq!(
serde_json::from_str::<Credential>(&json).expect("deserialises"),
c
);
}
#[test]
fn the_field_names_the_nix_reader_asks_for_are_the_ones_written() {
let json = serde_json::to_value(Credential {
value: "s3cr3t".to_owned(),
client_id: "hive-alpha-agent".to_owned(),
})
.expect("serialises");
assert_eq!(json["value"], "s3cr3t");
assert_eq!(json["client_id"], "hive-alpha-agent");
}
/// Spelled out in full because `swarm-secret-publisher.nix` and
/// `swarm-bao.nix` write the same string by hand.
#[test]
@ -231,9 +139,9 @@ mod tests {
let reads = |path: &str| granted.iter().any(|g| path.starts_with(g));
let path = controller_client_path().expect("a constant segment");
assert!(!reads(&path), "{path} is readable under {granted:?}");
// The control: the hive's own agent client IS under a hive stanza, so
// the assertion above can fail at all.
let own = agent_client_path("alpha").expect("legal");
// The control: the hive's own client IS under a hive stanza, so the
// assertion above can fail at all.
let own = hive_client_path("alpha").expect("legal");
assert!(reads(&own), "{own} should be readable under {granted:?}");
}
@ -381,29 +289,4 @@ mod tests {
serde_json::from_str::<AgentCredential>(r#"{"value":"s","hive":"alpha"}"#).is_err()
);
}
/// decodes as the other — the property that keeps a reader from picking up
/// a hive-shared credential where a per-agent one was meant.
#[test]
fn the_hive_credential_and_the_agent_credential_are_not_interchangeable() {
let hive_json = serde_json::to_string(&Credential {
value: "s".to_owned(),
client_id: "hive-alpha-agent".to_owned(),
})
.expect("serialises");
assert!(serde_json::from_str::<AgentCredential>(&hive_json).is_err());
let agent_json = serde_json::to_string(&AgentCredential {
value: "s".to_owned(),
agent: "atlas".to_owned(),
minted_at: None,
})
.expect("serialises");
assert!(serde_json::from_str::<Credential>(&agent_json).is_err());
assert_ne!(
agent_queue_path("atlas").expect("legal"),
agent_client_path("atlas").expect("legal"),
);
}
}