vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

swarm-logs: authenticate as the agent with its own queue secret

Browse source 
swarm-logs reads `swarm/agents/<agent>/queue` from the swarm secret
store with the agent's own store certificate (the read hive-agent makes
for its queue connection) and presents it as
`Bearer swarm-agent.<agent>.<secret>`, which the gateway's
swarm-logs-auth checks. On a 401 it reads the secret again and retries
once, so a secret re-minted between the read and the check does not
fail the query.

The OIDC mint against the hive's shared `hive-<h>-agent` client is gone
from swarm-logs, along with the unused `HIVE_AGENT_OIDC_CA_FILE` anchor.
logs.nix now gates on the store address instead of the queue's token
endpoint and wraps the binary with the paths queue-identity.nix gives
the harness; the host forwards the query URL wherever it forwards a
store identity.

The shared client is still delivered into containers; removing that is

Part of #4459.
This commit is contained in:
atlas 2026-10-12 22:48:40 +02:00 • committed by mara
commit 0f53e9be6c
12 changed files with 260 additions and 331 deletions
Download patch file Download diff file Expand all files Collapse all files

2
Cargo.lock generated
View file

@ -4882,6 +4882,8 @@ dependencies = [
"reqwest",
"serde_json",
"swarm-queue-client",
"swarm-secret-client",
"tokio",
]
[[package]]

5
docs/swarm/README.md
View file

@ -182,8 +182,9 @@ the credential from the store again for each attempt.
The hive's shared queue client is still delivered into the harness unit's
credentials directory, and `swarm.statusPublish.tokenEndpoint` still reaches
the container, but only for `swarm-logs`: the harness never connects with
them.
the container, but only for a `swarm-logs` built before it read the agent's
own queue secret: the harness never connects with them, and a current
`swarm-logs` presents that secret to the log store instead.
<details><summary>What an agent publishes over the queue</summary>

4
docs/swarm/credentials.md
View file

@ -56,14 +56,14 @@ of the cell says how.
<!-- vale write-good.Passive = NO -->
| store path | minter | reader — pulls at runtime, holds in memory | automatic re-mint | automatic re-pull |
| ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `swarm/agents/<agent>/matrix/main` | `swarm-controller`, with the swarm's appservice token, at agent creation and in a five-minute pass | the agent container itself, under the certificate its hive passed in | ✅ the pass re-mints when the stored token is missing, unknown to the homeserver, or someone else's | ✅ `hive-matrix-daemon` exits when the homeserver rejects its token, and a five-minute timer restarts it, which reads the store again |
| `swarm/agents/<agent>/matrix/<account>` | `swarm-controller` | the agent container itself, under the certificate its hive passed in | must be stated | must be stated |
| `swarm/controller/swarm-controller/matrix/appservice-token` | `swarm-matrix-ctl`, inside the `hive-matrix` container, once | `swarm-controller`, under its own certificate | ❌ `swarm-matrix-ctl` mints it once; the container keeps its copy and republishes it when the store's differs | ✅ the controller reads it on every five-minute matrix pass |
| `swarm/controller/swarm-controller/oidc/client` | authelia, at its first boot; `swarm-secret-publish` copies it in | `swarm-controller`, under its own certificate, once at start | ❌ authelia mints it once. A re-mint is republished by `swarm-secret-publish`'s path unit | ❌ read once at start; the controller holds the old value until it restarts |
| `swarm/controller/swarm-controller/forge/token` | the forge, inside the `hive-forge` container, once; `swarm-bao-forge-publisher` on the forge's host copies it in under its own certificate, with a write-only grant | `swarm-controller`, under its own certificate, at start | ❌ the forge mints it once and keeps its copy in the container; the publisher writes that copy each time the forge container starts and each time the mint writes the file | ❌ the controller reads it at start, and in the background with backoff until it first finds it; a replaced token reaches it with the next restart |
| `swarm/agents/<agent>/bao-mtls` | the store's agent PKI mount (`deploy.bao.agentPkiMountPath`), which generates the key, at `swarm-controller`'s request at agent creation | `hive-c0re`, under the hive's own certificate, when it writes the agent's container config; then `hive-agent-bao-identity` in the agent container, under the leaf it holds | ✅ `swarm-controller`'s five-minute pass re-issues a live agent's leaf once it's past half its validity (45 of 90 days, read from the certificate itself) | ✅ `hive-agent-bao-identity` re-reads it every 10 minutes and swaps a new leaf into `/run/hive-agent-bao-identity/`, which every store login in the container presents. It reads under the leaf it holds, so it renews only while that leaf is valid |
| `swarm/agents/<agent>/queue` | `swarm-controller`, at agent creation | `hive-agent` in the agent container, under the agent's own certificate, held in memory — the identity it presents to the swarm queue, naming that one agent rather than its hive | ✅ `swarm-controller`'s five-minute pass re-mints a live agent's secret once it's 45 days old by `minted_at` on the stored object; a secret with no `minted_at` gets one stamped, value unchanged. The pass skips agents declared `Destroyed` — declaring an agent destroyed deletes every version of the path instead, the undo of the mint rather than another one | ✅ `hive-agent` reads the path before its first connect and again on every reconnect attempt, so a reconnect after a re-mint presents the new secret. An open connection keeps the secret it connected with; after a revocation the agent keeps retrying under the queue client's backoff |
| `swarm/agents/<agent>/queue` | `swarm-controller`, at agent creation | `hive-agent` in the agent container, under the agent's own certificate, held in memory — the identity it presents to the swarm queue, naming that one agent rather than its hive; `swarm-logs` in the agent container, under the same certificate, on every query, as the bearer it presents at the log store; `swarm-nats-auth` and `swarm-logs-auth`, each under its own certificate, to check a presented secret | ✅ `swarm-controller`'s five-minute pass re-mints a live agent's secret once it's 45 days old by `minted_at` on the stored object; a secret with no `minted_at` gets one stamped, value unchanged. The pass skips agents declared `Destroyed` — declaring an agent destroyed deletes every version of the path instead, the undo of the mint rather than another one | ✅ `hive-agent` reads the path before its first connect and again on every reconnect attempt, so a reconnect after a re-mint presents the new secret. An open connection keeps the secret it connected with; after a revocation the agent keeps retrying under the queue client's backoff. `swarm-logs` reads it on every query and once more after a 401 |
| `swarm/agents/<agent>/forge-token` | `swarm-controller`, at agent creation and in a pass every 5 minutes over every agent with a store identity | the agent container itself, under its own certificate, fetched to `/run/hive-agent-forge-token/token` | ✅ the controller re-mints when the stored token is missing or no longer matches the forge (last eight characters and scopes) | ✅ the agent re-fetches on a 10-minute timer |
| `swarm/agents/<agent>/forge/<label>` | `swarm-controller`, when an operator links an external forge account in the swarm UI | `hive-agent-forge-accounts` in the agent container, under the agent's own certificate, into `<state>/forge-<label>-token` and `forge-<label>.json` | ❌ an operator's token; replaced only by deleting it and linking the label again | ✅ the agent re-fetches on a 2-minute timer |
| `swarm/agents/<agent>/github-token` | `swarm-controller`, when an operator links a GitHub account in the swarm UI | `hive-agent-github-token` in the agent container, under the agent's own certificate, into `<state>/github-token` | ❌ an operator's token; replaced only by deleting it and linking it again | ✅ the agent re-fetches on a 2-minute timer, and `hive-github-notify` re-reads the file on every poll |

9
docs/swarm/secrets.md
View file

@ -259,8 +259,9 @@ The store's **first reader** is the agent containers' queue credential:
`glue-queue-agent-credential.nix` lands it as two files, the client secret and
the client id it authenticates, because that's how a queue client reads them —
the secret by path, the id by value. No local fallback exists, and none is
possible, so absent files mean this hive's agents have no `swarm-logs`, which
is what a swarm looks like before the publisher on the authelia host has run.
possible, so absent files mean an agent built before `swarm-logs` read its own
queue secret has no working `swarm-logs`, which is what a swarm looks like
before the publisher on the authelia host has run.
The reader runs before `hive-c0re.service`, which wants it (doesn't require
it), so an agent container never renders ahead of the credential; an
unreachable store delays the daemon's start rather than failing it.
@ -274,8 +275,8 @@ it needs no read access to either — and hands them to
`systemd-nspawn --load-credential` through `hive-priv`, which runs as root.
Inside, `hive-agent.service` names the same two ids in `LoadCredential=`, which
lands them in its own `$CREDENTIALS_DIRECTORY`, owned by the agent user and by
nobody else, where the agent's `swarm-logs` reads them. The harness itself
doesn't: it connects to the queue with the agent's own credential. `hive-c0re`
nobody else, where a `swarm-logs` built before it read the agent's own queue
secret reads them. The harness itself doesn't: it connects to the queue with the agent's own credential. `hive-c0re`
never reads the bytes at any point: it runs as `hive-core`, which is the reason
a copy wasn't an option either. When the files aren't there the daemon forwards
nothing and says so in its journal, and the container starts without them.

42
nix/agent-modules/logs.nix
View file

@ -7,13 +7,11 @@
# and a default inside the binary would be an address we hope points at
# something.
#
# ⚠️ Gated on the queue being configured as well as on the query URL, because
# the identity this CLI presents IS the queue's: one per-hive machine client,
# one credential pair, delivered once. A swarm with a log store and no agent
# credential has nothing for this to authenticate as, and installing a binary
# that can only fail would be worse than leaving it off PATH — an agent would
# read the 401 as "no logs", which is the exact confusion the store's machine
# route was added to end.
# ⚠️ Gated on the secret store's address as well as on the query URL, because
# the identity this CLI presents is the agent's own queue secret, read from the
# store with the certificate ./bao.nix keeps. Without a store there is nothing
# to authenticate as, and a binary that can only fail would read its 401 as
# "no logs".
{
lib,
pkgs,
@ -22,22 +20,22 @@
}:
let
cfg = config.services.hyperhive.agent.logs;
queueCfg = config.services.hyperhive.agent.queue;
configured = cfg.queryUrl != null && queueCfg.tokenEndpoint != null;
baoCfg = config.services.hyperhive.agent.bao;
configured = cfg.queryUrl != null && baoCfg.addr != null;
# The four coordinates `swarm-logs` reads, all-or-none on its side. Bound
# here as one attrset so the wrapper below cannot set three of them: a
# half-set environment is the failure the binary reports as a deployment
# bug, and it should not be reachable from the module that sets it.
# The same store coordinates ./queue-identity.nix gives the harness, except
# the CA: `%d` expands only inside a unit, so it is spelled as the harness
# unit's credentials directory, which the agent user's shells can read.
#
# 🩸 Two of these are credential **paths**. Neither the id nor the secret
# is read here — a value in `--set` lands in the wrapper script, which is
# in the world-readable nix store.
# 🩸 Paths and a name only. A value in `--set` lands in the wrapper script,
# which is in the world-readable nix store.
wrapperEnv = {
HIVE_AGENT_LOGS_QUERY_URL = cfg.queryUrl;
HIVE_AGENT_OIDC_TOKEN_ENDPOINT = queueCfg.tokenEndpoint;
HIVE_AGENT_OIDC_CLIENT_ID_FILE = queueCfg.clientIdFile;
HIVE_AGENT_OIDC_CLIENT_SECRET_FILE = queueCfg.clientSecretFile;
HIVE_AGENT_NAME = config.services.hyperhive.agent.user.name;
BAO_ADDR = baoCfg.addr;
BAO_CLIENT_CERT = baoCfg.clientCertFile;
BAO_CLIENT_KEY = baoCfg.clientKeyFile;
BAO_CACERT = "/run/credentials/hive-agent.service/hive-agent-bao-server-ca";
};
# Same shape `swarm-controller.nix` wraps `swarmctl` in — `symlinkJoin` +
@ -71,12 +69,6 @@ in
Set by the generated meta flake from the host's own
{option}`services.hyperhive.swarm.victorialogs.domain`.
⚠️ The **whole URL**, not a host to build one from, because this same
string is also the audience `swarm-logs` mints its token for. Two
spellings of one address present as a valid token refused at the
store — the rule `swarm-otel.nix` already states over its own push
targets.
`null` means this hive has not been given the log store's address for
its agents, and `swarm-logs` is then not installed at all. Deliberately
not defaulted: a guessed address resolves cleanly to the wrong place.

22
nix/host-modules/hive-c0re/environment.nix
View file

@ -271,27 +271,17 @@ in
HIVE_AGENT_OIDC_TOKEN_ENDPOINT = config.services.hyperhive.swarm.statusPublish.tokenEndpoint;
}
//
# Where an agent reads the swarm's logs, forwarded the same way and gated on
# the same credential: `swarm-logs` authenticates as the queue's own per-hive
# machine client, so an address with no credential behind it would put a
# binary on PATH that can only ever answer 401 — which an agent reads as "no
# logs", the exact confusion the store's machine route was added to end.
# Where an agent reads the swarm's logs. Gated on the store identity below,
# because `swarm-logs` authenticates with the agent's own queue secret, read
# from the store: an address with nothing to authenticate as would put a
# binary on PATH that can only ever answer 401, which an agent reads as "no
# logs".
#
# The **machine** route (`^~ /select/logsql/`), not the browser one at `/`:
# that one ends in `error_page 401 =302` and hands a machine caller
# authelia's login page as a 200 with an HTML body. See
# `swarm-victorialogs.nix`'s location for the whole reasoning.
#
# ⚠️ The full URL rather than the domain, because the same string is also
# the audience the token is minted for — `swarm-otel.nix` states that rule
# over its own push targets, and two spellings present as a valid token
# refused at the store.
lib.optionalAttrs
(
config.services.hyperhive.deploy.hive-controller.queue.agentNatsUrl != null
&& config.services.hyperhive.swarm.statusPublish.tokenEndpoint != null
)
{
lib.optionalAttrs haveBaoClientIdentity {
HIVE_AGENT_LOGS_QUERY_URL = "https://${config.services.hyperhive.swarm.victorialogs.domain}/select/logsql/query";
}
//

37
nix/module-eval/agent-queue-bao.nix
View file

@ -40,8 +40,45 @@ let
agentNoBao = shared.agentBare;
# `swarm-logs` authenticates with the agent's queue secret from the store, so
# the store address gates it beside the query URL.
logsUrl = "https://logs.t.local/select/logsql/query";
agentLogs = agentWith {
services.hyperhive.agent.bao.addr = "https://bao.t.local:8200";
services.hyperhive.agent.logs.queryUrl = logsUrl;
};
agentLogsNoBao = agentWith { services.hyperhive.agent.logs.queryUrl = logsUrl; };
swarmLogsIn =
machine:
lib.findFirst (
p: (p.name or "") == "swarm-logs-configured"
) null machine.environment.systemPackages;
agentBaoIdentity = machine: machine.systemd.services.hive-agent-bao-identity;
cases = [
{
# The wrapper names the store identity ./queue-identity.nix gives the
# harness, by path, and nothing of the hive's shared OIDC client.
name = "swarm-logs is wrapped with the agent's own store identity";
ok =
let
w = (swarmLogsIn agentLogs).buildCommand;
a = agentLogs.services.hyperhive.agent;
set = k: v: lib.hasInfix "--set ${k} ${lib.escapeShellArg v}" w;
in
set "HIVE_AGENT_LOGS_QUERY_URL" logsUrl
&& set "HIVE_AGENT_NAME" a.user.name
&& set "BAO_ADDR" "https://bao.t.local:8200"
&& set "BAO_CLIENT_CERT" a.bao.clientCertFile
&& set "BAO_CLIENT_KEY" a.bao.clientKeyFile
&& set "BAO_CACERT" "/run/credentials/hive-agent.service/hive-agent-bao-server-ca"
&& !(lib.hasInfix "OIDC" w);
}
{
# A query URL with nothing to authenticate as installs nothing.
name = "an agent with a log store but no secret store gets no swarm-logs";
ok = swarmLogsIn agentLogsNoBao == null && swarmLogsIn agentLogs != null;
}
{
# Both ids or neither: the secret authenticates nobody without the id it
# belongs to, and `swarm-logs` reads both.

14
swarm-logs/Cargo.toml
View file

@ -12,13 +12,15 @@ path = "src/main.rs"
anyhow.workspace = true
clap.workspace = true
clap-markdown = "0.1"
# The whole point of the crate's dependency list: minting the access token
# is `swarm_queue_client::mint_token_for_blocking`, not a second copy of
# the `client_credentials` request. See `src/auth.rs`.
# The agent-token spelling the gateway's responder parses, and the rustls
# provider the store client needs.
swarm-queue-client.workspace = true
# Blocking, not async: this binary makes exactly two requests and exits, so
# a reactor would exist only to be torn down. It also matches the token
# minter reached for above, which is `swarm-queue-client`'s blocking one.
# The read of this agent's own queue secret.
swarm-secret-client.workspace = true
# Only to drive that one async read; the query itself is blocking.
tokio.workspace = true
# Blocking, not async: this binary makes one request (two on a 401) and
# exits, so a reactor would exist only to be torn down.
reqwest = { workspace = true, features = ["blocking"] }
serde_json.workspace = true

63
swarm-logs/README.md
View file

@ -20,40 +20,19 @@ attach at the gateway location, not in this binary.
## How it authenticates
The agent container already holds one identity, the per-hive machine client
`hive-<name>-agent`, delivered as a systemd credential pair. This binary
presents a `client_credentials` access token minted from it as
`Authorization: Bearer`.
As the agent itself. On each query it reads the agent's own queue secret,
`swarm/agents/<agent>/queue`, from the swarm secret store with the agent's
store certificate (`nix/agent-modules/bao.nix`), the same read `hive-agent`
makes for its queue connection. It sends the secret as
`Authorization: Bearer swarm-agent.<agent>.<secret>`, spelled by
`swarm_queue_client::agent_token`.
Minting is **not** implemented here — `swarm_queue_client::mint_token_for_blocking`
already owns the request, its error type and its CA handling, and a
token-endpoint fix has to be findable in one place. What this crate adds is
the agent-shaped half: the client id arrives as a _file_ beside the secret, so
nothing outside `nix/agent-modules/queue.nix` spells `hive-<name>-agent`
twice. That is the same problem `hive-agent/src/swarm_queue.rs` solves, and
`src/auth.rs` is its `decide` restated over this binary's inputs.
At the gateway, nginx hands a bearer of that shape to `swarm-logs-auth`, which
compares it with the stored secret (`swarm-agent-verify`, the check the queue's
auth callout also uses). A 401 makes `swarm-logs` read the secret again and
retry once, which covers a secret re-minted between the read and the check.
🩸 Every credential here is a **path**. The secret's contents are read at the
moment of the request, inside `swarm-queue-client`, and are never bound to a
name in this crate, logged, or rendered into an error.
### The audience is the URL
`swarm-logs` mints its token with the query URL as the requested audience,
because that is what authelia checks at the authz endpoint the gateway's
`auth_request` calls. The same rule `swarm-otel.nix` states over its own push
targets: one binding for the address and the audience, since two spellings of
one address present as a valid token refused at the store.
⚠️ That requires the agent client to be _registered_ for that audience and to
hold `authelia.bearer.authz`. Both are set in `swarm-authelia.nix`'s
`agentClients`; without them authelia answers `invalid_target` at the token
endpoint, or the gateway answers 401 with no explanation.
⚠️ Registration is not issuance, so the token request _asks_ for both: the
audience and the scope are named in the `client_credentials` form, because a
client that is registered for a scope it does not request is handed a token
carrying none, and the gateway refuses that with the same bare 401.
🩸 The secret is never logged or rendered into an error.
## Configuration
@ -63,18 +42,16 @@ derived from an option that module owns, so a default here would be an address
we _hope_ points at something.
| variable | what |
| ------------------------------------ | ---------------------------------------------------- |
| `HIVE_AGENT_LOGS_QUERY_URL` | the LogsQL query endpoint; also the token's audience |
| `HIVE_AGENT_OIDC_TOKEN_ENDPOINT` | authelia's token endpoint |
| `HIVE_AGENT_OIDC_CLIENT_ID_FILE` | path of the delivered client-id credential |
| `HIVE_AGENT_OIDC_CLIENT_SECRET_FILE` | path of the delivered secret credential |
| `HIVE_AGENT_OIDC_CA_FILE` | optional extra trust anchor; unset in this tree |
| --------------------------- | ------------------------------------------------ |
| `HIVE_AGENT_LOGS_QUERY_URL` | the LogsQL query endpoint |
| `HIVE_AGENT_NAME` | this agent's name; the store path and role |
| `BAO_ADDR` | the secret store's address |
| `BAO_CLIENT_CERT` | path of the agent's live store certificate |
| `BAO_CLIENT_KEY` | path of its key |
| `BAO_CACERT` | path of the store's CA; missing or empty is fine |
`HIVE_AGENT`, the same prefix the harness reads, because it is the same
identity — an agent authenticates as its hive's agent client whether the
caller is the harness or a CLI the agent typed. The first four are all-or-none:
a half-set environment is a deployment bug, and this binary says so by name
rather than behaving like an unconfigured one.
With no `HIVE_AGENT_LOGS_QUERY_URL` the binary says there is no log store;
with the URL and no store address it reports a deployment bug by name.
## Output

260
swarm-logs/src/auth.rs
View file

@ -1,187 +1,149 @@
//! The access token this CLI presents to the log store's gateway route.
//! The credential this CLI presents to the log store's gateway route: the
//! agent's own queue secret, read from the swarm secret store with the agent's
//! store certificate.
//!
//! Minting it is **not** implemented here. `swarm-queue-client` already owns
//! the `client_credentials` request against authelia — the basic-auth form,
//! the per-call secret read, the CA-trust builder and the error type that
//! surfaces authelia's own `error_description` — and this crate calls
//! [`swarm_queue_client::mint_token_for_blocking`] rather than writing a
//! second copy of it. A token-endpoint fix has to be findable in one place.
//! The read is `hive-agent`'s `swarm_queue` one: the same variables, the same
//! path (`swarm/agents/<agent>/queue`) and the same cert-auth role. The secret
//! goes out as `Bearer swarm-agent.<agent>.<secret>`, spelled by
//! `swarm_queue_client::agent_token`; the gateway's `swarm-logs-auth`
//! responder checks it against the store.
//!
//! What *is* here is the shape of an agent's identity, which differs from the
//! daemons that crate was written for in exactly one way: **the client id
//! arrives as a file, not as a value.** It rides in the same systemd
//! credential pair as the secret so that nothing outside `queue.nix` ever
//! spells `hive-<name>-agent` again. That is the same problem
//! `hive-agent/src/swarm_queue.rs` solves, and [`Config::from_env`] below is
//! that module's `decide` restated over this binary's inputs — including its
//! rule that a *half*-set environment is a deployment bug rather than an
//! absent integration.
//!
//! 🩸 Every credential in here is a **path**. The secret's contents are read
//! by `swarm-queue-client` at the moment of the request and are never bound to
//! a name in this crate, never logged and never rendered into an error.
use std::path::{Path, PathBuf};
//! 🩸 The secret is never logged and never rendered into an error.
use anyhow::{Context as _, Result, bail};
use swarm_queue_client::{ClientSecret, QueueConfig};
use swarm_secret_client::client::{DEFAULT_CERT_MOUNT, Settings};
use swarm_secret_client::queue::{self, AgentCredential};
use swarm_secret_client::{SecretStore, policy};
/// Variable prefix for this binary's coordinates.
///
/// `HIVE_AGENT`, deliberately the **same** prefix the harness reads, because
/// it is the same identity: an agent authenticates as its hive's agent client
/// whether the caller is the harness or a CLI the agent typed. A prefix of its
/// own would be a second set of variables carrying one fact, which is the
/// disagreement `queue.nix`'s credential pair exists to prevent.
const ENV_PREFIX: &str = "HIVE_AGENT";
/// The full URL of the LogsQL query endpoint, gateway side.
const ENV_QUERY_URL: &str = "HIVE_AGENT_LOGS_QUERY_URL";
/// This agent's name, which the store path and the role are built from.
const ENV_AGENT_NAME: &str = "HIVE_AGENT_NAME";
/// The store's address; the rest of its coordinates are `Settings`'s.
const ENV_ADDR: &str = "BAO_ADDR";
/// The store's CA bundle, which may name an undelivered credential.
const ENV_CACERT: &str = "BAO_CACERT";
/// Where this CLI queries, and who it queries as.
pub struct Config {
/// Full URL of the LogsQL query endpoint, gateway side.
///
/// The whole URL rather than a host to build one from, because this same
/// string is **also the audience** the token is minted for — the rule
/// `swarm-otel.nix` states over its own push targets ("one binding
/// because the same string is both the address requested and the audience
/// the token is minted for — two spellings present as a valid token
/// refused at the store"). Splitting it would reintroduce exactly that.
pub query_url: String,
/// Everything needed to mint a token, in the form the minter wants it.
pub queue: QueueConfig,
agent: String,
settings: Settings,
role: String,
path: String,
}
impl Config {
/// Read the configuration out of the environment the wrapper sets.
///
/// All four coordinates or none, for the reason `swarm-queue-client`'s
/// README gives and `hive-agent`'s `swarm_queue` repeats: a half-set
/// environment produces a process that looks configured and never
/// authenticates. Unlike the harness, this binary *errors* on the absent
/// case rather than degrading — a CLI whose only
/// job is the query has nothing left to do without it, and an operator
/// running it wants to be told why rather than handed an empty result
/// they will read as "no logs matched".
/// Errors rather than degrading: a CLI whose only job is the query has
/// nothing to do without it, and an empty result would read as "no logs
/// matched".
pub fn from_env() -> Result<Self> {
let var = |suffix: &str| std::env::var(format!("{ENV_PREFIX}_{suffix}")).ok();
Self::from_lookup(|k| std::env::var(k).ok())
}
let query_url = var("LOGS_QUERY_URL");
let token_endpoint = var("OIDC_TOKEN_ENDPOINT");
let client_id_file = var("OIDC_CLIENT_ID_FILE");
let client_secret_file = var("OIDC_CLIENT_SECRET_FILE");
// Outside the all-or-none group on purpose: a swarm behind a
// publicly-trusted certificate needs no extra anchor, so a CA path
// with no endpoint is meaningless rather than half-configured. Nothing in this tree sets
// it for an agent — a container already trusts the store's root, which
// `hive_c0re::meta` embeds at build time.
let ca_file = var("OIDC_CA_FILE").map(PathBuf::from);
let (query_url, token_endpoint, client_id_file, client_secret_file) = match (
query_url,
token_endpoint,
client_id_file,
client_secret_file,
) {
(Some(u), Some(t), Some(i), Some(s)) => (u, t, i, s),
(None, None, None, None) => bail!(
"this agent has no swarm log store configured: \
{ENV_PREFIX}_LOGS_QUERY_URL, {ENV_PREFIX}_OIDC_TOKEN_ENDPOINT, \
{ENV_PREFIX}_OIDC_CLIENT_ID_FILE and \
{ENV_PREFIX}_OIDC_CLIENT_SECRET_FILE are all unset"
),
_ => bail!(
"swarm log store half-configured: {ENV_PREFIX}_LOGS_QUERY_URL, \
{ENV_PREFIX}_OIDC_TOKEN_ENDPOINT, {ENV_PREFIX}_OIDC_CLIENT_ID_FILE \
and {ENV_PREFIX}_OIDC_CLIENT_SECRET_FILE are set together or not \
at all — this is a deployment bug, not a missing feature"
),
fn from_lookup(get: impl Fn(&str) -> Option<String>) -> Result<Self> {
let set = |k: &str| get(k).filter(|v| !v.is_empty());
let Some(query_url) = set(ENV_QUERY_URL) else {
bail!("this agent has no swarm log store configured: {ENV_QUERY_URL} is unset");
};
let client_id = read_client_id(Path::new(&client_id_file))?;
if set(ENV_ADDR).is_none() {
bail!(
"{ENV_QUERY_URL} is set but {ENV_ADDR} is not, so there is no secret store \
to read this agent's credential from — this is a deployment bug"
);
}
let agent =
set(ENV_AGENT_NAME).with_context(|| format!("{ENV_AGENT_NAME} is unset or empty"))?;
let settings =
Settings::from_lookup(|k| set(k).filter(|v| k != ENV_CACERT || ca_is_usable(v)))
.context("reading the swarm secret store's coordinates from the environment")?;
Ok(Self {
query_url,
queue: QueueConfig {
// 🩸 Unused by the token mint and deliberately not read from
// the environment. `QueueConfig` is the queue's connect
// config as well as its token config, and this binary needs
// only the second half — taking `HIVE_AGENT_NATS_URL` for it
// would make a broker address a prerequisite for reading
// logs, which is a coupling with no cause. Named so a reader
// of a stack trace is not left wondering.
url: String::from("unused: swarm-logs mints a token and never connects"),
token_endpoint,
client_id,
client_secret: ClientSecret::File(PathBuf::from(client_secret_file)),
ca_file,
},
role: policy::agent_object_name(&agent)?,
path: queue::agent_queue_path(&agent)?,
agent,
settings,
})
}
/// Read this agent's secret from the store and spell it as the bearer.
///
/// Read on every call, so a caller that was refused can call again and
/// present a secret re-minted since.
pub fn bearer(&self) -> Result<String> {
let runtime = tokio::runtime::Builder::new_current_thread()
.enable_all()
.build()
.context("starting the runtime for the secret store read")?;
let stored: Option<AgentCredential> = runtime.block_on(async {
SecretStore::connect(&self.settings, &self.role, DEFAULT_CERT_MOUNT)
.await
.context("logging in to the swarm secret store with this agent's certificate")?
.read_optional(&self.path)
.await
.with_context(|| format!("reading {}", self.path))
})?;
let stored = stored.with_context(|| format!("nothing is stored at {}", self.path))?;
swarm_queue_client::agent_token::format_agent_token(&self.agent, &stored.value)
.with_context(|| format!("spelling the credential stored at {}", self.path))
}
}
/// Read the OIDC client id out of the file the systemd credential landed at.
///
/// Trailing newline stripped and an empty file refused, both copied from
/// `hive-agent`'s `read_client_id`: the writing unit ends the file with a
/// newline, and an id carrying one authenticates as nobody. Where the harness
/// treats absence as the ordinary state of a hive whose credential has not
/// been published yet and carries on, this errors — see [`Config::from_env`]
/// for why a CLI has nothing to carry on with.
fn read_client_id(path: &Path) -> Result<String> {
let raw = std::fs::read_to_string(path)
.with_context(|| format!("reading the OIDC client id from {}", path.display()))?;
let id = raw.trim();
if id.is_empty() {
bail!(
"the OIDC client id at {} is empty — the queue credential has not \
been published to this hive yet",
path.display()
);
}
Ok(id.to_owned())
/// Whether the CA bundle at `path` is a file with bytes in it. `BAO_CACERT`
/// names a systemd credential that may not have been delivered; absent means
/// the container's own trust store.
fn ca_is_usable(path: &str) -> bool {
std::fs::metadata(path).is_ok_and(|m| m.len() > 0)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn client_id_loses_its_trailing_newline() {
let dir = std::env::temp_dir().join(format!("swarm-logs-id-{}", std::process::id()));
std::fs::create_dir_all(&dir).expect("temp dir");
let path = dir.join("client-id");
std::fs::write(&path, "hive-alpha-agent\n").expect("write");
assert_eq!(read_client_id(&path).expect("read"), "hive-alpha-agent");
std::fs::remove_dir_all(&dir).expect("cleanup");
fn env(pairs: &[(&str, &str)]) -> impl Fn(&str) -> Option<String> {
let pairs: Vec<(String, String)> = pairs
.iter()
.map(|(k, v)| ((*k).to_owned(), (*v).to_owned()))
.collect();
move |k| pairs.iter().find(|(n, _)| n == k).map(|(_, v)| v.clone())
}
/// An empty credential file is the shape a hive has before its secret
/// store has published anything, and it must not become a token request
/// that authenticates as the empty string.
const FULL: &[(&str, &str)] = &[
(ENV_QUERY_URL, "https://logs.example/select/logsql/query"),
(ENV_AGENT_NAME, "atlas"),
(ENV_ADDR, "https://bao.example:8200"),
("BAO_CLIENT_CERT", "/nonexistent/cert"),
("BAO_CLIENT_KEY", "/nonexistent/key"),
];
#[test]
fn an_empty_client_id_is_refused_by_name() {
let dir = std::env::temp_dir().join(format!("swarm-logs-empty-{}", std::process::id()));
std::fs::create_dir_all(&dir).expect("temp dir");
let path = dir.join("client-id");
std::fs::write(&path, "\n").expect("write");
let err = read_client_id(&path).expect_err("empty id must not be accepted");
assert!(
err.to_string().contains("is empty"),
"error should say the id is empty, got: {err}"
);
std::fs::remove_dir_all(&dir).expect("cleanup");
fn no_query_url_says_there_is_no_log_store() {
let err = Config::from_lookup(env(&[])).err().expect("refused");
assert!(err.to_string().contains("no swarm log store"), "{err}");
}
/// A missing file names the path, because the operator's next move is to
/// look at whether the credential was delivered at all.
#[test]
fn a_missing_client_id_file_names_its_path() {
let path = Path::new("/nonexistent/swarm-logs/client-id");
let err = read_client_id(path).expect_err("missing file must not be accepted");
assert!(
err.to_string()
.contains("/nonexistent/swarm-logs/client-id"),
"error should name the path, got: {err}"
);
fn a_query_url_without_a_store_is_a_deployment_bug() {
let err = Config::from_lookup(env(&FULL[..2])).err().expect("refused");
assert!(err.to_string().contains("deployment bug"), "{err}");
}
#[test]
fn a_missing_agent_name_is_named() {
let mut pairs = FULL.to_vec();
pairs.retain(|(k, _)| *k != ENV_AGENT_NAME);
let err = Config::from_lookup(env(&pairs)).err().expect("refused");
assert!(err.to_string().contains(ENV_AGENT_NAME), "{err}");
}
/// The path and role are the agent's own, the ones its policy grants.
#[test]
fn the_read_is_the_agents_own_queue_secret() {
let cfg = Config::from_lookup(env(FULL)).expect("configured");
assert_eq!(cfg.path, "swarm/agents/atlas/queue");
assert_eq!(cfg.role, policy::agent_object_name("atlas").expect("valid"));
}
}

1
swarm-logs/src/main.rs
View file

@ -85,6 +85,7 @@ fn main() -> Result<()> {
// keep working on a machine that has no log store wired up at
// all — `markdown-docs` in particular runs in a nix check, where
// no credential exists and none should.
swarm_queue_client::install_crypto_provider();
let cfg = auth::Config::from_env()?;
let stdout = std::io::stdout();
query::run(&cfg, &logsql, limit, format, &mut stdout.lock())

80
swarm-logs/src/query.rs
View file

@ -46,41 +46,29 @@ pub fn run(
format: Format,
out: &mut impl Write,
) -> Result<()> {
// Minted per invocation, with the query URL as the audience and the
// `authelia.bearer.authz` scope. All three are load-bearing: authelia
// refuses a token carrying no audience at the authz endpoint the
// gateway's `auth_request` calls, and the audience it checks is the URL
// being requested — so the string sent here and the string requested
// below must be one binding, which is why `Config` holds exactly one.
//
// The scope has to be ASKED for, not merely registered: the agent client
// is granted `authelia.bearer.authz` by the `agentClients` entry in
// `swarm-authelia.nix`, but an OAuth2 server issues no scope the client
// never requested, and a scopeless token is refused at the authz endpoint
// exactly as an unauthenticated one is — a bare nginx 401 with nothing in
// it that names the scope. `swarm-otel.nix` records the same failure
// against the collector's client, on the same string.
let token = swarm_queue_client::mint_token_for_blocking(
&cfg.queue,
Some(&cfg.query_url),
Some("authelia.bearer.authz"),
)
.map_err(|e| anyhow::anyhow!("{}", swarm_queue_client::chain(&e)))
.context("minting an access token for the swarm log store")?;
let http = build_http_client(cfg)?;
let http = build_http_client()?;
let mut form = vec![("query", logsql.to_owned())];
if let Some(limit) = limit {
form.push(("limit", limit.to_string()));
}
let response = http
.post(&cfg.query_url)
.bearer_auth(&token)
let send = || -> Result<reqwest::blocking::Response> {
let bearer = cfg
.bearer()
.context("reading this agent's credential for the swarm log store")?;
http.post(&cfg.query_url)
.bearer_auth(bearer)
.form(&form)
.send()
.with_context(|| format!("querying the swarm log store at {}", cfg.query_url))?;
.with_context(|| format!("querying the swarm log store at {}", cfg.query_url))
};
let mut response = send()?;
// A 401 can be a secret re-minted between the store read and the check,
// so it is read again and the query sent once more.
if response.status() == reqwest::StatusCode::UNAUTHORIZED {
response = send()?;
}
let status = response.status();
if !status.is_success() {
@ -140,37 +128,13 @@ fn render(line: &str, format: Format) -> String {
}
}
/// The HTTP client for the query itself.
/// The HTTP client for the query itself, trusting the container's own roots.
///
/// Separate from the one `swarm-queue-client` builds for the token endpoint
/// because they are two hosts — `auth.<swarm>` and `logs.<swarm>` — and the
/// CA-trust knob is declared per endpoint. Same optional anchor, same reason:
/// without it a swarm using its own CA fails at TLS with
/// `invalid peer certificate: UnknownIssuer`, having never looked at the
/// anchor sitting on the host.
///
/// No timeout, deliberately, where the token client has one: a query over a
/// long retention window legitimately takes longer than any number that would
/// be right for a token request, and a client-side cut would truncate a
/// correct answer into a shorter one with no marker saying so.
fn build_http_client(cfg: &Config) -> Result<reqwest::blocking::Client> {
let mut builder = reqwest::blocking::Client::builder();
if let Some(path) = &cfg.queue.ca_file {
let pem = std::fs::read(path).with_context(|| {
format!(
"reading the log-store CA certificate from {}",
path.display()
)
})?;
let cert = reqwest::Certificate::from_pem(&pem).with_context(|| {
format!(
"parsing the log-store CA certificate from {} as PEM",
path.display()
)
})?;
builder = builder.add_root_certificate(cert);
}
builder
/// No timeout, deliberately: a query over a long retention window legitimately
/// takes long, and a client-side cut would truncate a correct answer into a
/// shorter one with no marker saying so.
fn build_http_client() -> Result<reqwest::blocking::Client> {
reqwest::blocking::Client::builder()
.build()
.context("building the log-store HTTP client")
}