vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

hive status publishing: read the hive's OIDC secret from the store, retry until it is there

Browse source 
hive-c0re publishes this hive's status to the swarm queue as the OIDC
client `hive-<name>`. Its secret came from
`deploy.hive-controller.statusPublish.clientSecretFile`, which defaulted
to authelia's minted file only on a host running both authelia and the
queue. Every other hive had no secret and never published, so the
controller showed it `never_reported`.

Now `swarm-secret-publish`, beside authelia, also copies each hive's
`hive-<name>` secret to `swarm/hives/<hive>/oidc/client`. Its policy
already allows `swarm/hives/*`. hive-c0re reads that path under the
hive's own store leaf, which the existing `swarm/hives/<hive>/*` grant
covers, and holds the value in memory.

The shared queue client no longer remembers a failed first connect for
the life of the process. A failed attempt is retried with a backoff
from 5s, doubling to a cap of 5 minutes; the dashboard banner stays up
until an attempt succeeds. The two status publishers wait for the
client instead of giving up on the first `None`, so a secret published
after boot is picked up without a restart.

Removed: the `statusPublish.clientSecretFile` option (both spellings,
now removed options with a message), its same-host default, its
LoadCredential, the assertion tying it to the other two coordinates,
and `swarm_queue_client::QueueConfig::from_env`, whose only caller was
this read. Publishing is now on wherever the queue URL and token
endpoint are set and the hive holds a store leaf.

Refs #5015
This commit is contained in:
atlas 2026-10-09 23:12:28 +02:00 • committed by mara
commit c2c61b6146
19 changed files with 643 additions and 394 deletions
Download patch file Download diff file Expand all files Collapse all files

27
swarm-queue-client/README.md
View file

@ -32,24 +32,21 @@ nothing says so until someone reads a dashboard.
## Configuration
`QueueConfig::from_env(prefix)` reads `<prefix>_NATS_URL`,
`<prefix>_OIDC_TOKEN_ENDPOINT`, `<prefix>_OIDC_CLIENT_ID` and
`<prefix>_OIDC_CLIENT_SECRET_FILE`.
The prefix is a parameter because the variables belong to the consuming unit —
a NixOS module sets them alongside its other options. What is shared is the
rule, not the spelling: **all four together or none at all.** A half-set
environment is a hard error, because the failure it would otherwise produce is
the expensive kind — the process comes up "fine", never connects, and the data
it was supposed to move silently stops.
A caller builds a `QueueConfig` itself from its own environment: the queue's
URL, the token endpoint and the client id, plus an optional CA file. The
variables belong to the consuming unit, so each daemon reads them under its own
prefix. What is shared is the rule, not the spelling: **the three coordinates
together or none at all.** A half-set environment is a hard error, because the
failure it would otherwise produce is the expensive kind — the process comes up
"fine", never connects, and the data it was supposed to move silently stops.
The client secret is never in the environment, which would publish it to
anything that can read `/proc/<pid>/environ`. `from_env` takes it as a
anything that can read `/proc/<pid>/environ`. A caller passes it either as a
**path** (`ClientSecret::File`), read per token request rather than cached, so
a rotation the operator believes took effect actually did. A caller that
fetched the secret from somewhere else builds the config itself with
`ClientSecret::Value`, held in memory; `swarm-controller` does this with the
secret it reads from the swarm's secret store.
a rotation the operator believes took effect actually did, or as a value it
fetched itself (`ClientSecret::Value`), held in memory. `swarm-controller` and
`hive-c0re` both do the latter with the secret they read from the swarm's
secret store.
## Token request shape: HTTP Basic, `audience`, `scope`

105
swarm-queue-client/src/lib.rs
View file

@ -37,20 +37,13 @@ use std::path::PathBuf;
/// `anyhow` — `?` converts for free, so nothing downstream is more verbose
/// for it. The same split `hive-claude` uses.
///
/// The variants are the failures an operator acts on differently: a
/// half-configured environment is a deployment bug, a refused token is an
/// The variants are the failures an operator acts on differently: an
/// unreadable secret file is a deployment bug, a refused token is an
/// identity-provider config problem, an unreachable queue is a network
/// one. Collapsing them into one string would make that distinction a
/// matter of reading prose.
#[derive(Debug, thiserror::Error)]
pub enum Error {
#[error(
"swarm queue is half-configured: {prefix}_NATS_URL, \
{prefix}_OIDC_TOKEN_ENDPOINT, {prefix}_OIDC_CLIENT_ID and \
{prefix}_OIDC_CLIENT_SECRET_FILE must be set together or not at all"
)]
PartialConfig { prefix: String },
#[error("reading the queue client secret from {path}")]
ClientSecret {
path: String,
@ -519,56 +512,6 @@ impl ClientSecret {
}
}
impl QueueConfig {
/// Read the config from `<prefix>_NATS_URL`, `<prefix>_OIDC_TOKEN_ENDPOINT`,
/// `<prefix>_OIDC_CLIENT_ID` and `<prefix>_OIDC_CLIENT_SECRET_FILE`, or
/// `None` when the queue was not wired up for this deployment.
///
/// The prefix is a parameter rather than a constant because the variables
/// belong to the *consuming unit* — a NixOS module sets them alongside its
/// other options, and two daemons sharing one name would be a worse
/// coupling than passing four characters. What is shared is the RULE
/// below, not the spelling.
///
/// `None` rather than an error on purpose: a daemon serves its other
/// surfaces on hosts where the queue is not enabled, and refusing to start
/// there would trade a missing feature for a dead process. What must NOT
/// happen is a *half* configuration silently behaving like an absent one —
/// hence the explicit partial check below.
pub fn from_env(prefix: &str) -> Result<Option<Self>, Error> {
let url = std::env::var(format!("{prefix}_NATS_URL")).ok();
let token_endpoint = std::env::var(format!("{prefix}_OIDC_TOKEN_ENDPOINT")).ok();
let client_id = std::env::var(format!("{prefix}_OIDC_CLIENT_ID")).ok();
let secret = std::env::var(format!("{prefix}_OIDC_CLIENT_SECRET_FILE")).ok();
// Read outside the match on purpose: this one is INDEPENDENT of the
// all-or-none rule, so it must not participate in the tuple that
// decides whether the queue is configured at all. A CA path with no
// queue is meaningless rather than half-configured.
let ca_file = std::env::var(format!("{prefix}_OIDC_CA_FILE"))
.ok()
.map(PathBuf::from);
match (url, token_endpoint, client_id, secret) {
(None, None, None, None) => Ok(None),
(Some(url), Some(token_endpoint), Some(client_id), Some(secret)) => Ok(Some(Self {
url,
token_endpoint,
client_id,
client_secret: ClientSecret::File(PathBuf::from(secret)),
ca_file,
})),
// A partially-set environment is a deployment bug, and the failure
// it would otherwise produce is the expensive kind: the process
// comes up "fine", never connects, and the data it was supposed to
// move silently stops moving. Naming the variables costs one line.
_ => Err(Error::PartialConfig {
prefix: prefix.to_owned(),
}),
}
}
}
/// Mint a fresh access token for the controller's own client.
///
/// `client_credentials`, because there is no user here: the controller
@ -998,50 +941,6 @@ mod tests {
.with_no_client_auth();
}
/// The all-unset case is the common one — most hosts do not run the queue.
///
/// Uses a prefix no deployment sets, so it cannot pass vacuously by
/// running inside a configured environment (the guard below covers the
/// same ground, and both are cheap).
#[test]
fn an_absent_environment_is_not_an_error() {
for k in [
"SWARM_QUEUE_TEST_NATS_URL",
"SWARM_QUEUE_TEST_OIDC_TOKEN_ENDPOINT",
"SWARM_QUEUE_TEST_OIDC_CLIENT_ID",
"SWARM_QUEUE_TEST_OIDC_CLIENT_SECRET_FILE",
] {
assert!(std::env::var(k).is_err(), "{k} must be unset for this test");
}
assert!(
QueueConfig::from_env("SWARM_QUEUE_TEST")
.expect("absent is not an error")
.is_none()
);
}
/// The half-set case is the one the rule exists for: a deployment bug that
/// would otherwise look exactly like "no queue configured".
///
/// SAFETY: single-threaded mutation of a process env var under a prefix no
/// other test or deployment uses; removed before returning.
#[test]
fn a_half_set_environment_is_a_hard_error() {
unsafe {
std::env::set_var("SWARM_QUEUE_HALF_NATS_URL", "nats://127.0.0.1:4222");
}
let err = QueueConfig::from_env("SWARM_QUEUE_HALF")
.expect_err("a partial set must not read as absent");
let msg = format!("{err}");
assert!(
msg.contains("SWARM_QUEUE_HALF_OIDC_CLIENT_ID"),
"the error must name the missing variables, got: {msg}"
);
unsafe {
std::env::remove_var("SWARM_QUEUE_HALF_NATS_URL");
}
}
#[test]
fn a_held_secret_is_read_back_as_itself() {
let secret = ClientSecret::Value("s3cret".to_owned());