Watch
0
0
Fork
You've already forked hyperhive
0

Compare commits

..
Author SHA1 Message Date
atlas
0f53e9be6c swarm-logs: authenticate as the agent with its own queue secret
swarm-logs reads `swarm/agents/<agent>/queue` from the swarm secret
store with the agent's own store certificate (the read hive-agent makes
for its queue connection) and presents it as
`Bearer swarm-agent.<agent>.<secret>`, which the gateway's
swarm-logs-auth checks. On a 401 it reads the secret again and retries
once, so a secret re-minted between the read and the check does not
fail the query.

The OIDC mint against the hive's shared `hive-<h>-agent` client is gone
from swarm-logs, along with the unused `HIVE_AGENT_OIDC_CA_FILE` anchor.
logs.nix now gates on the store address instead of the queue's token
endpoint and wraps the binary with the paths queue-identity.nix gives
the harness; the host forwards the query URL wherever it forwards a
store identity.

The shared client is still delivered into containers; removing that is

Part of #4459.
2026-10-13 08:38:05 +02:00
atlas
1f4736e79b swarm-logs-auth: check agent tokens on the log store's query route
A new `auth_request` responder on the log store's host. nginx sends it
every `/select/logsql/` request whose `Authorization` is
`Bearer swarm-agent.<agent>.<secret>`; it answers 200 when the secret is
the one stored at `swarm/agents/<agent>/queue` (`swarm-agent-verify`),
401 otherwise. Every other bearer still goes to authelia, so agents
still presenting the hive's shared OIDC client keep working.

`auth_request` takes a fixed URI, so the split is a `map` on
`$http_authorization` plus an internal location that rewrites to the
map's pick. Checked against nginx 1.30.5 with stub upstreams: an agent
token only the stub validator accepts gets 200, a token only the stub
authelia accepts gets 200, every wrong one 401, and the internal
locations 404 from outside.

The responder serves a unix socket in its own traverse-only runtime
directory and reads the store under its own leaf
(`swarm-logs-auth`), whose policy is `swarm/agents/+/queue`, read-only.
glue-bao-tls mints the leaf where the store runs; elsewhere the
operator places it, and eval refuses a log store host without one.

Part of #4459.
2026-10-13 08:38:05 +02:00
atlas
e7dbb94f28 swarm-agent-verify: lift the agent-token store check out of swarm-nats-auth
`CredentialSource`, `Store` and `verify` move from swarm-nats-auth's
`agent_token` module into a small library crate, so a second verifier
can make the same check: the secret in a `swarm-agent.<agent>.<secret>`
token equals the one stored at `swarm/agents/<agent>/queue`, read under
the verifier's own certificate.

`verify` now takes its timeout as an argument; swarm-nats-auth passes
`INTROSPECTION_TIMEOUT` as before. No behaviour change. The store-denial
tests moved with the code; swarm-nats-auth keeps the ones that go
through its policy and its bound.

Part of #4459.
2026-10-13 08:38:05 +02:00
32 changed files with 1050 additions and 531 deletions

View file

@ -193,6 +193,15 @@ hand-maintained per-file tree drifts out of sync with the code.
`claude --print` per classification (no tools, no session) and replies
with the rows and the overall verdict. Holds no forge credential. Design:
[`docs/swarm/review-classifier.md`](docs/swarm/review-classifier.md).
- **`swarm-agent-verify/`** — the check that a presented
`swarm-agent.<agent>.<secret>` token holds the secret stored at
`swarm/agents/<agent>/queue`, read under the verifier's own certificate.
Shared by `swarm-nats-auth` (the queue's auth callout) and
`swarm-logs-auth`.
- **`swarm-logs-auth/`** — the log store gateway's `auth_request` responder
for agents: nginx sends it `/select/logsql/` requests carrying an agent
token, it answers 200 or 401 via `swarm-agent-verify`. Runs on the log
store's host (`swarm-victorialogs.nix`).
- **`swarmctl/`** — swarm-level operator CLI, installed by the
swarm-controller module on the host that runs the daemon. Runs as
**root and acts directly** — it serves no socket, publishes no HTTP

31
Cargo.lock generated
View file

@ -4789,6 +4789,18 @@ version = "2.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13c2bddecc57b384dee18652358fb23172facb8a2c51ccc10d74c157bdea3292"
[[package]]
name = "swarm-agent-verify"
version = "0.1.0"
dependencies = [
"anyhow",
"subtle",
"swarm-queue-client",
"swarm-secret-client",
"tokio",
"tracing",
]
[[package]]
name = "swarm-authelia-bridge"
version = "0.1.0"
@ -4870,6 +4882,23 @@ dependencies = [
"reqwest",
"serde_json",
"swarm-queue-client",
"swarm-secret-client",
"tokio",
]
[[package]]
name = "swarm-logs-auth"
version = "0.1.0"
dependencies = [
"anyhow",
"axum",
"clap",
"swarm-agent-verify",
"swarm-queue-client",
"swarm-secret-client",
"tokio",
"tracing",
"tracing-subscriber",
]
[[package]]
@ -4910,7 +4939,7 @@ dependencies = [
"serde",
"serde_json",
"sha2 0.11.0",
"subtle",
"swarm-agent-verify",
"swarm-queue-client",
"swarm-secret-client",
"tokio",

View file

@ -25,6 +25,7 @@ members = [
"hive-subagent-mcp",
"hive-types",
"hivectl",
"swarm-agent-verify",
"swarm-authelia-bridge",
"swarm-authelia-bridge-sock",
"swarm-controller",
@ -34,6 +35,7 @@ members = [
"swarm-queue-client",
"swarm-review-classifier",
"swarm-logs",
"swarm-logs-auth",
"swarm-secret-client",
"swarmctl",
]
@ -104,6 +106,7 @@ hive-priv-sock = { path = "hive-priv-sock" }
hive-runtime = { path = "hive-runtime" }
hive-sock-client = { path = "hive-sock-client" }
hive-types = { path = "hive-types" }
swarm-agent-verify = { path = "swarm-agent-verify" }
swarm-authelia-bridge-sock = { path = "swarm-authelia-bridge-sock" }
swarm-matrix-client = { path = "swarm-matrix-client" }
swarm-queue-client = { path = "swarm-queue-client" }
@ -234,7 +237,7 @@ futures-util = "0.3"
hmac = "0.13"
sha2 = "0.11"
# Constant-time comparison of a presented secret against the stored one
# (`swarm-nats-auth::agent_token`). Already in the tree through the TLS stack.
# (`swarm-agent-verify`). Already in the tree through the TLS stack.
subtle = "2.6"
# The NATS protocol client, for the swarm queue's auth-callout responder.
# `default-features = false` because the default set is broad - jetstream, kv,

View file

@ -182,8 +182,9 @@ the credential from the store again for each attempt.
The hive's shared queue client is still delivered into the harness unit's
credentials directory, and `swarm.statusPublish.tokenEndpoint` still reaches
the container, but only for `swarm-logs`: the harness never connects with
them.
the container, but only for a `swarm-logs` built before it read the agent's
own queue secret: the harness never connects with them, and a current
`swarm-logs` presents that secret to the log store instead.
<details><summary>What an agent publishes over the queue</summary>

View file

@ -56,14 +56,14 @@ of the cell says how.
<!-- vale write-good.Passive = NO -->
| store path | minter | reader — pulls at runtime, holds in memory | automatic re-mint | automatic re-pull |
| ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `swarm/agents/<agent>/matrix/main` | `swarm-controller`, with the swarm's appservice token, at agent creation and in a five-minute pass | the agent container itself, under the certificate its hive passed in | ✅ the pass re-mints when the stored token is missing, unknown to the homeserver, or someone else's | ✅ `hive-matrix-daemon` exits when the homeserver rejects its token, and a five-minute timer restarts it, which reads the store again |
| `swarm/agents/<agent>/matrix/<account>` | `swarm-controller` | the agent container itself, under the certificate its hive passed in | must be stated | must be stated |
| `swarm/controller/swarm-controller/matrix/appservice-token` | `swarm-matrix-ctl`, inside the `hive-matrix` container, once | `swarm-controller`, under its own certificate | ❌ `swarm-matrix-ctl` mints it once; the container keeps its copy and republishes it when the store's differs | ✅ the controller reads it on every five-minute matrix pass |
| `swarm/controller/swarm-controller/oidc/client` | authelia, at its first boot; `swarm-secret-publish` copies it in | `swarm-controller`, under its own certificate, once at start | ❌ authelia mints it once. A re-mint is republished by `swarm-secret-publish`'s path unit | ❌ read once at start; the controller holds the old value until it restarts |
| `swarm/controller/swarm-controller/forge/token` | the forge, inside the `hive-forge` container, once; `swarm-bao-forge-publisher` on the forge's host copies it in under its own certificate, with a write-only grant | `swarm-controller`, under its own certificate, at start | ❌ the forge mints it once and keeps its copy in the container; the publisher writes that copy each time the forge container starts and each time the mint writes the file | ❌ the controller reads it at start, and in the background with backoff until it first finds it; a replaced token reaches it with the next restart |
| `swarm/agents/<agent>/bao-mtls` | the store's agent PKI mount (`deploy.bao.agentPkiMountPath`), which generates the key, at `swarm-controller`'s request at agent creation | `hive-c0re`, under the hive's own certificate, when it writes the agent's container config; then `hive-agent-bao-identity` in the agent container, under the leaf it holds | ✅ `swarm-controller`'s five-minute pass re-issues a live agent's leaf once it's past half its validity (45 of 90 days, read from the certificate itself) | ✅ `hive-agent-bao-identity` re-reads it every 10 minutes and swaps a new leaf into `/run/hive-agent-bao-identity/`, which every store login in the container presents. It reads under the leaf it holds, so it renews only while that leaf is valid |
| `swarm/agents/<agent>/queue` | `swarm-controller`, at agent creation | `hive-agent` in the agent container, under the agent's own certificate, held in memory — the identity it presents to the swarm queue, naming that one agent rather than its hive | ✅ `swarm-controller`'s five-minute pass re-mints a live agent's secret once it's 45 days old by `minted_at` on the stored object; a secret with no `minted_at` gets one stamped, value unchanged. The pass skips agents declared `Destroyed` — declaring an agent destroyed deletes every version of the path instead, the undo of the mint rather than another one | ✅ `hive-agent` reads the path before its first connect and again on every reconnect attempt, so a reconnect after a re-mint presents the new secret. An open connection keeps the secret it connected with; after a revocation the agent keeps retrying under the queue client's backoff |
| `swarm/agents/<agent>/queue` | `swarm-controller`, at agent creation | `hive-agent` in the agent container, under the agent's own certificate, held in memory — the identity it presents to the swarm queue, naming that one agent rather than its hive; `swarm-logs` in the agent container, under the same certificate, on every query, as the bearer it presents at the log store; `swarm-nats-auth` and `swarm-logs-auth`, each under its own certificate, to check a presented secret | ✅ `swarm-controller`'s five-minute pass re-mints a live agent's secret once it's 45 days old by `minted_at` on the stored object; a secret with no `minted_at` gets one stamped, value unchanged. The pass skips agents declared `Destroyed` — declaring an agent destroyed deletes every version of the path instead, the undo of the mint rather than another one | ✅ `hive-agent` reads the path before its first connect and again on every reconnect attempt, so a reconnect after a re-mint presents the new secret. An open connection keeps the secret it connected with; after a revocation the agent keeps retrying under the queue client's backoff. `swarm-logs` reads it on every query and once more after a 401 |
| `swarm/agents/<agent>/forge-token` | `swarm-controller`, at agent creation and in a pass every 5 minutes over every agent with a store identity | the agent container itself, under its own certificate, fetched to `/run/hive-agent-forge-token/token` | ✅ the controller re-mints when the stored token is missing or no longer matches the forge (last eight characters and scopes) | ✅ the agent re-fetches on a 10-minute timer |
| `swarm/agents/<agent>/forge/<label>` | `swarm-controller`, when an operator links an external forge account in the swarm UI | `hive-agent-forge-accounts` in the agent container, under the agent's own certificate, into `<state>/forge-<label>-token` and `forge-<label>.json` | ❌ an operator's token; replaced only by deleting it and linking the label again | ✅ the agent re-fetches on a 2-minute timer |
| `swarm/agents/<agent>/github-token` | `swarm-controller`, when an operator links a GitHub account in the swarm UI | `hive-agent-github-token` in the agent container, under the agent's own certificate, into `<state>/github-token` | ❌ an operator's token; replaced only by deleting it and linking it again | ✅ the agent re-fetches on a 2-minute timer, and `hive-github-notify` re-reads the file on every poll |

View file

@ -259,8 +259,9 @@ The store's **first reader** is the agent containers' queue credential:
`glue-queue-agent-credential.nix` lands it as two files, the client secret and
the client id it authenticates, because that's how a queue client reads them —
the secret by path, the id by value. No local fallback exists, and none is
possible, so absent files mean this hive's agents have no `swarm-logs`, which
is what a swarm looks like before the publisher on the authelia host has run.
possible, so absent files mean an agent built before `swarm-logs` read its own
queue secret has no working `swarm-logs`, which is what a swarm looks like
before the publisher on the authelia host has run.
The reader runs before `hive-c0re.service`, which wants it (doesn't require
it), so an agent container never renders ahead of the credential; an
unreachable store delays the daemon's start rather than failing it.
@ -274,8 +275,8 @@ it needs no read access to either — and hands them to
`systemd-nspawn --load-credential` through `hive-priv`, which runs as root.
Inside, `hive-agent.service` names the same two ids in `LoadCredential=`, which
lands them in its own `$CREDENTIALS_DIRECTORY`, owned by the agent user and by
nobody else, where the agent's `swarm-logs` reads them. The harness itself
doesn't: it connects to the queue with the agent's own credential. `hive-c0re`
nobody else, where a `swarm-logs` built before it read the agent's own queue
secret reads them. The harness itself doesn't: it connects to the queue with the agent's own credential. `hive-c0re`
never reads the bytes at any point: it runs as `hive-core`, which is the reason
a copy wasn't an option either. When the files aren't there the daemon forwards
nothing and says so in its journal, and the container starts without them.

View file

@ -82,6 +82,13 @@ A hive reads no **swarm service's** OIDC client secret. Every reader of
collector and the store forwarder each fetch theirs with a leaf of their own,
and the store's granter fetches the bao UI's.
Two services read **every agent's queue secret**
(`swarm/agents/<agent>/queue`), each under a leaf of its own whose policy
reaches that one leaf per agent and nothing else: the queue's auth callout
(`swarm-nats-auth`) and the log store's agent check (`swarm-logs-auth`, on the
log store's host). Either one compromised can present any agent's queue
secret, at the queue and at the log store's query route alike.
### Per-hive credentials live under the hive kind
`swarm_secret_client::policy` scopes the **hive** stanza narrowly (unlike the

View file

@ -154,6 +154,9 @@
services.hyperhive.deploy.swarm-ui.package =
lib.mkDefault
self.packages.${pkgs.stdenv.hostPlatform.system}.swarm-ui;
services.hyperhive.deploy.victorialogs.authPackage =
lib.mkDefault
self.packages.${pkgs.stdenv.hostPlatform.system}.swarm-logs-auth;
services.hyperhive.deploy.nats.authPackage =
lib.mkDefault
self.packages.${pkgs.stdenv.hostPlatform.system}.swarm-nats-auth;

View file

@ -7,13 +7,11 @@
# and a default inside the binary would be an address we hope points at
# something.
#
# ⚠️ Gated on the queue being configured as well as on the query URL, because
# the identity this CLI presents IS the queue's: one per-hive machine client,
# one credential pair, delivered once. A swarm with a log store and no agent
# credential has nothing for this to authenticate as, and installing a binary
# that can only fail would be worse than leaving it off PATH — an agent would
# read the 401 as "no logs", which is the exact confusion the store's machine
# route was added to end.
# ⚠️ Gated on the secret store's address as well as on the query URL, because
# the identity this CLI presents is the agent's own queue secret, read from the
# store with the certificate ./bao.nix keeps. Without a store there is nothing
# to authenticate as, and a binary that can only fail would read its 401 as
# "no logs".
{
lib,
pkgs,
@ -22,22 +20,22 @@
}:
let
cfg = config.services.hyperhive.agent.logs;
queueCfg = config.services.hyperhive.agent.queue;
configured = cfg.queryUrl != null && queueCfg.tokenEndpoint != null;
baoCfg = config.services.hyperhive.agent.bao;
configured = cfg.queryUrl != null && baoCfg.addr != null;
# The four coordinates `swarm-logs` reads, all-or-none on its side. Bound
# here as one attrset so the wrapper below cannot set three of them: a
# half-set environment is the failure the binary reports as a deployment
# bug, and it should not be reachable from the module that sets it.
# The same store coordinates ./queue-identity.nix gives the harness, except
# the CA: `%d` expands only inside a unit, so it is spelled as the harness
# unit's credentials directory, which the agent user's shells can read.
#
# 🩸 Two of these are credential **paths**. Neither the id nor the secret
# is read here — a value in `--set` lands in the wrapper script, which is
# in the world-readable nix store.
# 🩸 Paths and a name only. A value in `--set` lands in the wrapper script,
# which is in the world-readable nix store.
wrapperEnv = {
HIVE_AGENT_LOGS_QUERY_URL = cfg.queryUrl;
HIVE_AGENT_OIDC_TOKEN_ENDPOINT = queueCfg.tokenEndpoint;
HIVE_AGENT_OIDC_CLIENT_ID_FILE = queueCfg.clientIdFile;
HIVE_AGENT_OIDC_CLIENT_SECRET_FILE = queueCfg.clientSecretFile;
HIVE_AGENT_NAME = config.services.hyperhive.agent.user.name;
BAO_ADDR = baoCfg.addr;
BAO_CLIENT_CERT = baoCfg.clientCertFile;
BAO_CLIENT_KEY = baoCfg.clientKeyFile;
BAO_CACERT = "/run/credentials/hive-agent.service/hive-agent-bao-server-ca";
};
# Same shape `swarm-controller.nix` wraps `swarmctl` in — `symlinkJoin` +
@ -71,12 +69,6 @@ in
Set by the generated meta flake from the host's own
{option}`services.hyperhive.swarm.victorialogs.domain`.
⚠️ The **whole URL**, not a host to build one from, because this same
string is also the audience `swarm-logs` mints its token for. Two
spellings of one address present as a valid token refused at the
store — the rule `swarm-otel.nix` already states over its own push
targets.
`null` means this hive has not been given the log store's address for
its agents, and `swarm-logs` is then not installed at all. Deliberately
not defaulted: a guessed address resolves cleanly to the wrong place.

View file

@ -33,6 +33,7 @@
./glue-forge-oidc-client.nix
./glue-grafana-oidc-client.nix
./glue-hive-tls-bao-identity.nix
./glue-logs-auth-bao-identity.nix
./glue-matrix-ctl-bao-identity.nix
./glue-matrix-oidc-client.nix
./glue-nats-auth-bao-identity.nix

View file

@ -270,6 +270,13 @@ in
[ -s ${pkiDir}/nats-auth.pem ] || ${signLeaf} ${pkiDir} nats-auth \
${lib.escapeShellArg deployCfg.bao.natsAuthCommonName} "" clientAuth
# The log store's agent auth responder, which reads agent queue
# credentials. Minted here for the matrix-ctl leaf's reason: the log
# store is a swarm singleton, so elsewhere this is the file an operator
# copies.
[ -s ${pkiDir}/logs-auth.pem ] || ${signLeaf} ${pkiDir} logs-auth \
${lib.escapeShellArg deployCfg.bao.logsAuthCommonName} "" clientAuth
# The CI runner's token fetch. Minted here for the matrix-ctl leaf's
# reason: the runner is a swarm singleton, so on a runner host without
# the store this is the file an operator copies.

View file

@ -0,0 +1,31 @@
# Glue: point the log store's agent auth responder at the bao leaf minted for it.
#
# ONE PAIRING PER FILE — the swarm-logs-auth principal ← bao, and nothing else.
# Deleting this leaves a log store with no responder identity unless the
# operator names one, which ./swarm-victorialogs.nix refuses at eval.
#
# ⚠️ The minting is NOT here. ./glue-bao-tls.nix holds the CA and signs the
# leaf. What belongs here is the pairing: which paths the responder presents.
#
# ⚠️ Gated on the store running on this host, the only place the leaf is
# minted. A log-store host without the store gets no default here; the
# operator places the responder's leaf and names it.
#
# Everything is `mkDefault`. An operator naming their own paths wins.
{
lib,
config,
...
}:
let
deployCfg = config.services.hyperhive.deploy;
pkiDir = import ./lib/bao-pki-dir.nix;
in
{
config = lib.mkIf (deployCfg.victorialogs.enable && deployCfg.bao.enable) {
services.hyperhive.deploy.victorialogs = {
authBaoClientCertFile = lib.mkDefault "${pkiDir}/logs-auth.pem";
authBaoClientKeyFile = lib.mkDefault "${pkiDir}/logs-auth-key.pem";
};
};
}

View file

@ -271,27 +271,17 @@ in
HIVE_AGENT_OIDC_TOKEN_ENDPOINT = config.services.hyperhive.swarm.statusPublish.tokenEndpoint;
}
//
# Where an agent reads the swarm's logs, forwarded the same way and gated on
# the same credential: `swarm-logs` authenticates as the queue's own per-hive
# machine client, so an address with no credential behind it would put a
# binary on PATH that can only ever answer 401 — which an agent reads as "no
# logs", the exact confusion the store's machine route was added to end.
# Where an agent reads the swarm's logs. Gated on the store identity below,
# because `swarm-logs` authenticates with the agent's own queue secret, read
# from the store: an address with nothing to authenticate as would put a
# binary on PATH that can only ever answer 401, which an agent reads as "no
# logs".
#
# The **machine** route (`^~ /select/logsql/`), not the browser one at `/`:
# that one ends in `error_page 401 =302` and hands a machine caller
# authelia's login page as a 200 with an HTML body. See
# `swarm-victorialogs.nix`'s location for the whole reasoning.
#
# ⚠️ The full URL rather than the domain, because the same string is also
# the audience the token is minted for — `swarm-otel.nix` states that rule
# over its own push targets, and two spellings present as a valid token
# refused at the store.
lib.optionalAttrs
(
config.services.hyperhive.deploy.hive-controller.queue.agentNatsUrl != null
&& config.services.hyperhive.swarm.statusPublish.tokenEndpoint != null
)
{
lib.optionalAttrs haveBaoClientIdentity {
HIVE_AGENT_LOGS_QUERY_URL = "https://${config.services.hyperhive.swarm.victorialogs.domain}/select/logsql/query";
}
//

View file

@ -964,6 +964,17 @@ let
}
'';
# The log store's agent auth responder, which checks the secret an agent
# presents against the one stored for it: the same `+` leaf as the queue
# responder's, and nothing else.
logsAuthReaders = [
{
name = "swarm-logs-auth";
cn = baoDeploy.logsAuthCommonName;
policyText = readStanza "${credentialMountPath}/data/swarm/agents/+/queue";
}
];
# The CI runner's token fetch. The one reader of the path, which the
# controller writes and cannot read back.
ciRunnerReaders = [
@ -1797,6 +1808,19 @@ in
'';
};
logsAuthCommonName = lib.mkOption {
type = lib.types.str;
default = "swarm-logs-auth";
description = ''
Subject the store's `swarm-logs-auth` cert-auth role accepts: the
identity the log store's agent auth responder presents to read agent
queue credentials, `swarm/agents/<agent>/queue` and nothing else under
an agent.
⚠️ Reserved as a hive name by ./swarm.nix, like its siblings.
'';
};
ciRunnerCommonName = lib.mkOption {
type = lib.types.str;
default = "swarm-ci-runner";
@ -3171,6 +3195,8 @@ in
systemd.services.swarm-bao-nats-auth-policy = readerPolicyUnit "write the queue responder's agent-credential-reader bao policy and cert-auth role" natsAuthReaders;
systemd.services.swarm-bao-logs-auth-policy = readerPolicyUnit "write the log store responder's agent-credential-reader bao policy and cert-auth role" logsAuthReaders;
systemd.services.swarm-bao-ci-runner-policy = readerPolicyUnit "write the CI runner's registration-token-reader bao policy and cert-auth role" ciRunnerReaders;
systemd.services.swarm-bao-review-classifier-policy = readerPolicyUnit "write the review classifier's API-key-reader bao policy and cert-auth role" reviewClassifierReaders;
@ -3302,6 +3328,7 @@ in
"swarm-bao-nats-tls-policy.service"
"swarm-bao-hive-leaf-policy.service"
"swarm-bao-ci-runner-policy.service"
"swarm-bao-logs-auth-policy.service"
"swarm-bao-forge-publisher-policy.service"
]
++ granterAfter;

View file

@ -52,6 +52,22 @@ let
error_page 401 =302 https://${hyperhiveCfg.swarm.authelia.domain}/?rd=$target_url;
'';
# The agents' `auth_request` responder (`swarm-logs-auth`). Its socket's
# directory holds that socket alone and is traverse-only for others, the
# shape `swarm-controller.nix` gives its own.
logsAuthUnit = "swarm-logs-auth";
logsAuthSocket = "/run/${logsAuthUnit}/auth.sock";
logsAuthCredential = id: "/run/credentials/${logsAuthUnit}.service/${id}";
# Credential id → host path of the responder's store identity.
logsAuthStoreFiles = {
"bao-client.pem" = deployCfg.victorialogs.authBaoClientCertFile;
"bao-client-key.pem" = deployCfg.victorialogs.authBaoClientKeyFile;
}
// lib.optionalAttrs (deployCfg.bao.serverCaFile != null) {
"bao-ca.pem" = deployCfg.bao.serverCaFile;
};
baoCfg = hyperhiveCfg.swarm.bao;
# Shared host netns, like every sibling swarm container: the collector
# and Grafana reach this at 127.0.0.1:<port>.
privateNetwork = false;
@ -88,7 +104,74 @@ in
'';
};
options.services.hyperhive.deploy.victorialogs.authPackage = lib.mkOption {
type = lib.types.package;
defaultText = lib.literalExpression "hyperhive.packages.\${system}.swarm-logs-auth";
description = ''
The `swarm-logs-auth` package: the gateway's `auth_request` responder
that checks an agent's `swarm-agent.<agent>.<secret>` bearer on
`/select/logsql/` against the secret store.
'';
};
options.services.hyperhive.deploy.victorialogs.authBaoClientCertFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-bao-pki/logs-auth.pem";
description = ''
Client certificate `swarm-logs-auth` presents to the swarm's secret store
to read agents' queue secrets. Its subject must be
{option}`services.hyperhive.deploy.bao.logsAuthCommonName`.
Required when `deploy.victorialogs.enable` is set; unset fails at eval.
./glue-logs-auth-bao-identity.nix points it at the leaf
./glue-bao-tls.nix mints on a host that runs the store; on any other
log-store host it is a file the operator places.
A path, never a value.
'';
};
options.services.hyperhive.deploy.victorialogs.authBaoClientKeyFile = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/var/lib/swarm-bao-pki/logs-auth-key.pem";
description = ''
Private key for {option}`services.hyperhive.deploy.victorialogs.authBaoClientCertFile`.
A path, never a value.
'';
};
config = lib.mkIf deployCfg.victorialogs.enable {
assertions = [
{
assertion =
deployCfg.victorialogs.authBaoClientCertFile != null
&& deployCfg.victorialogs.authBaoClientKeyFile != null;
message = ''
services.hyperhive.deploy.victorialogs.authBaoClientCertFile and
authBaoClientKeyFile must both be set.
swarm-logs-auth reads agents' queue secrets from the swarm's secret
store with this certificate. They default to the leaf the store
mints where services.hyperhive.deploy.bao.enable is set on this
host; elsewhere, issue a leaf whose CN is
services.hyperhive.deploy.bao.logsAuthCommonName and name it in
both options.
'';
}
];
# Which check a `/select/logsql/` request gets. `auth_request` takes a
# fixed URI, so it names `/__swarm_logs_auth`, which rewrites to the
# target this map picks from the caller's `Authorization`.
services.nginx.appendHttpConfig = ''
map $http_authorization $swarm_logs_auth_target {
"~^Bearer swarm-agent\." /__swarm_logs_agent_auth;
default /__hive_authelia;
}
'';
# This store publishes its own health as prometheus metrics on the same
# listener it serves queries on, so the swarm's collector scrapes it with
# no exporter and no extra port — same arrangement as the metrics store.
@ -174,12 +257,35 @@ in
# logs. That is the rule in force, not an omission: read
# permissions are a later thing, and this location is where one
# attaches when it exists.
#
# Two checks: an agent's own `swarm-agent.` bearer goes to
# `swarm-logs-auth`, anything else to authelia (`appendHttpConfig`
# above picks).
"^~ /select/logsql/" = {
# No URI part, so the request path and its query string reach
# the store as the caller sent them.
proxyPass = "http://127.0.0.1:${toString cfg.port}";
extraConfig = ''
auth_request /__hive_authelia;
auth_request /__swarm_logs_auth;
'';
};
# `rewrite … last` re-matches locations, so the subrequest lands in
# one of the two internal locations named by the map.
"= /__swarm_logs_auth" = {
extraConfig = ''
internal;
rewrite ^ $swarm_logs_auth_target last;
'';
};
# The `Authorization` header rides the subrequest unchanged.
"= /__swarm_logs_agent_auth" = {
proxyPass = "http://unix:${logsAuthSocket}:/";
extraConfig = ''
internal;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
'';
};
@ -207,6 +313,53 @@ in
};
};
# The agents' check on `/select/logsql/`. On the host rather than in the
# store's container: it needs nothing of the store, only nginx's reach to
# its socket and its own store identity.
systemd.services.${logsAuthUnit} = {
description = "swarm log store auth_request responder for agent credentials";
wantedBy = [ "multi-user.target" ];
# Where the store's PKI is minted on this host, the leaf is one of its
# files. Ordering only: elsewhere the unit does not exist.
after = [
"network.target"
"swarm-bao-pki.service"
];
serviceConfig = {
ExecStart = lib.concatStringsSep " " [
"${deployCfg.victorialogs.authPackage}/bin/swarm-logs-auth"
"--socket ${logsAuthSocket}"
# The role ./swarm-bao.nix writes on the store's `cert` mount.
"--store-cert-role swarm-logs-auth"
];
# Paths only: `argv` is world-readable via /proc/<pid>/cmdline.
LoadCredential = lib.mapAttrsToList (id: path: "${id}:${path}") logsAuthStoreFiles;
DynamicUser = true;
RuntimeDirectory = logsAuthUnit;
# Traverse-only for others, so nginx reaches the socket path.
RuntimeDirectoryMode = "0751";
# The path never vanishes from under a running nginx; the responder
# unlinks a stale socket as it starts.
RuntimeDirectoryPreserve = "yes";
Restart = "on-failure";
RestartSec = "5s";
SyslogIdentifier = logsAuthUnit;
ProtectSystem = "strict";
ProtectHome = true;
PrivateTmp = true;
PrivateDevices = true;
NoNewPrivileges = true;
};
environment = {
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
BAO_CLIENT_CERT = logsAuthCredential "bao-client.pem";
BAO_CLIENT_KEY = logsAuthCredential "bao-client-key.pem";
}
// lib.optionalAttrs (logsAuthStoreFiles ? "bao-ca.pem") {
BAO_CACERT = logsAuthCredential "bao-ca.pem";
};
};
containers.${cfg.machine} = {
autoStart = true;
ephemeral = false;

View file

@ -55,6 +55,7 @@ let
deployCfg.bao.granterCommonName
deployCfg.bao.natsAuthCommonName
deployCfg.bao.ciRunnerCommonName
deployCfg.bao.logsAuthCommonName
deployCfg.bao.reviewClassifierCommonName
deployCfg.bao.forgePublisherCommonName
]

View file

@ -40,8 +40,45 @@ let
agentNoBao = shared.agentBare;
# `swarm-logs` authenticates with the agent's queue secret from the store, so
# the store address gates it beside the query URL.
logsUrl = "https://logs.t.local/select/logsql/query";
agentLogs = agentWith {
services.hyperhive.agent.bao.addr = "https://bao.t.local:8200";
services.hyperhive.agent.logs.queryUrl = logsUrl;
};
agentLogsNoBao = agentWith { services.hyperhive.agent.logs.queryUrl = logsUrl; };
swarmLogsIn =
machine:
lib.findFirst (
p: (p.name or "") == "swarm-logs-configured"
) null machine.environment.systemPackages;
agentBaoIdentity = machine: machine.systemd.services.hive-agent-bao-identity;
cases = [
{
# The wrapper names the store identity ./queue-identity.nix gives the
# harness, by path, and nothing of the hive's shared OIDC client.
name = "swarm-logs is wrapped with the agent's own store identity";
ok =
let
w = (swarmLogsIn agentLogs).buildCommand;
a = agentLogs.services.hyperhive.agent;
set = k: v: lib.hasInfix "--set ${k} ${lib.escapeShellArg v}" w;
in
set "HIVE_AGENT_LOGS_QUERY_URL" logsUrl
&& set "HIVE_AGENT_NAME" a.user.name
&& set "BAO_ADDR" "https://bao.t.local:8200"
&& set "BAO_CLIENT_CERT" a.bao.clientCertFile
&& set "BAO_CLIENT_KEY" a.bao.clientKeyFile
&& set "BAO_CACERT" "/run/credentials/hive-agent.service/hive-agent-bao-server-ca"
&& !(lib.hasInfix "OIDC" w);
}
{
# A query URL with nothing to authenticate as installs nothing.
name = "an agent with a log store but no secret store gets no swarm-logs";
ok = swarmLogsIn agentLogsNoBao == null && swarmLogsIn agentLogs != null;
}
{
# Both ids or neither: the secret authenticates nobody without the id it
# belongs to, and `swarm-logs` reads both.

View file

@ -169,7 +169,7 @@ let
_: u: (u.environment.BAO_CLIENT_CERT or null) == granterCertFile
) baoGrantWithConsumers.systemd.services;
# The sixteen units that write a `swarm-*` grant, by name, for the discovery
# The seventeen units that write a `swarm-*` grant, by name, for the discovery
# control below.
grantingUnitNames = [
"swarm-bao-controller-policy"
@ -186,6 +186,7 @@ let
"swarm-bao-operator-viewer-policy"
"swarm-bao-hive-leaf-policy"
"swarm-bao-ci-runner-policy"
"swarm-bao-logs-auth-policy"
"swarm-bao-forge-publisher-policy"
"swarm-bao-review-classifier-policy"
];
@ -704,6 +705,31 @@ let
in
lib.hasInfix "/nats-auth.pem ]" s && lib.hasInfix "swarm-nats-auth \"\" clientAuth" s;
}
{
# The same `+` leaf as the queue responder, read-only, and nothing else:
# this principal checks a presented secret and does nothing with it.
name = "the log store responder's grant is every agent's queue secret, read-only, and nothing else";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-logs-auth-policy.script;
in
lib.hasInfix "path \"secret/data/swarm/agents/+/queue\" {\n capabilities = [\"read\"]\n}" s
&& lib.length (lib.filter lib.isList (builtins.split "path \"" s)) == 1
&& !(lib.hasInfix "secret/data/swarm/agents/*" s)
&& !(lib.hasInfix "secret/data/swarm/hives" s)
&& !(lib.hasInfix "secret/data/swarm/services" s)
&& lib.hasInfix "auth/cert/certs/swarm-logs-auth" s
&& lib.hasInfix "allowed_common_names=swarm-logs-auth" s
&& lib.hasInfix "token_policies=swarm-logs-auth" s;
}
{
name = "the PKI unit signs the log store responder's leaf under its own subject";
ok =
let
s = baoGrantHere.systemd.services.swarm-bao-pki.script;
in
lib.hasInfix "/logs-auth.pem ]" s && lib.hasInfix "swarm-logs-auth \"\" clientAuth" s;
}
{
# 🩸 The half that makes the policies above bind: a policy grants only
# through a token that carries it, and a token is minted by a cert-auth
@ -915,24 +941,24 @@ let
}
{
# A store host without the granter's pair writes its grants some other
# way, so none of the sixteen units may exist. Without this arm
# way, so none of the seventeen units may exist. Without this arm
# `lib.mkIf haveGranter` could be dropped from any of them and every other
# case here would still pass.
name = "without the granter's pair none of the sixteen granting units render";
name = "without the granter's pair none of the seventeen granting units render";
ok =
let
s = baoGranterOptOut.systemd.services;
in
lib.all (unit: !(s ? ${unit})) (grantingUnitNames ++ [ "swarm-bao-granter-role" ])
# The control: the same store with the pair renders all sixteen.
# The control: the same store with the pair renders all seventeen.
&& lib.all (unit: baoGrantHere.systemd.services ? ${unit}) grantingUnitNames;
}
{
# 🩸 What replaced the silent skip. With no bootstrap token the sixteen still
# 🩸 What replaced the silent skip. With no bootstrap token the seventeen still
# render, and a refused granter fails them with the step that fixes it.
# A store host that never named a token is told to name one, since the
# unit that sets the granter up renders only where it has.
name = "a store host without a bootstrap token renders the sixteen, each failing loudly with the one-time step";
name = "a store host without a bootstrap token renders the seventeen, each failing loudly with the one-time step";
ok =
let
s = baoGranterNoToken.systemd.services;
@ -1695,8 +1721,8 @@ let
}
{
# What makes the case above mean something: discovery by the granter's
# certificate reaches all sixteen units, and each yields calls.
name = "the granter-policy check sees all sixteen granting units, and parses calls from each";
# certificate reaches all seventeen units, and each yields calls.
name = "the granter-policy check sees all seventeen granting units, and parses calls from each";
ok =
lib.sort lib.lessThan (lib.attrNames granterUnits) == lib.sort lib.lessThan grantingUnitNames
&& lib.all (u: baoCalls u.script != [ ]) (lib.attrValues granterUnits)

View file

@ -504,6 +504,44 @@ let
&& !(lib.hasInfix "$args" q.extraConfig)
&& !(lib.hasInfix "?" q.proxyPass);
}
{
# `auth_request` takes a fixed URI, so the split between the agents'
# responder and authelia is a map plus a rewrite in an internal
# location. Every link of that chain is pinned: the map keys on the
# agent-token prefix and falls back to authelia, the query location
# names the switch, the switch rewrites to the map's pick, and the
# agent target reaches the responder's socket, which the unit binds.
name = "the machine query location sends agent bearers to swarm-logs-auth and the rest to authelia";
ok =
let
l = allLocal.services.nginx.virtualHosts."logs.t.local".locations;
http = allLocal.services.nginx.appendHttpConfig;
unit = allLocal.systemd.services.swarm-logs-auth;
in
lib.hasInfix "map \$http_authorization \$swarm_logs_auth_target" http
&& lib.hasInfix "\"~^Bearer swarm-agent\\.\" /__swarm_logs_agent_auth;" http
&& lib.hasInfix "default /__hive_authelia;" http
&& lib.hasInfix "auth_request /__swarm_logs_auth;" l."^~ /select/logsql/".extraConfig
&& lib.hasInfix "rewrite ^ \$swarm_logs_auth_target last;" l."= /__swarm_logs_auth".extraConfig
&& lib.hasInfix "internal;" l."= /__swarm_logs_agent_auth".extraConfig
&& l."= /__swarm_logs_agent_auth".proxyPass == "http://unix:/run/swarm-logs-auth/auth.sock:/"
&& lib.hasInfix "--socket /run/swarm-logs-auth/auth.sock" unit.serviceConfig.ExecStart
&& l ? "= /__hive_authelia";
}
{
# The responder reads the store as itself, with the leaf the store's
# host mints for it.
name = "swarm-logs-auth presents its own store leaf";
ok =
let
v = allLocal.services.hyperhive.deploy.victorialogs;
lc = allLocal.systemd.services.swarm-logs-auth.serviceConfig.LoadCredential;
in
lib.hasSuffix "/logs-auth.pem" v.authBaoClientCertFile
&& lib.hasSuffix "/logs-auth-key.pem" v.authBaoClientKeyFile
&& builtins.elem "bao-client.pem:${v.authBaoClientCertFile}" lc
&& builtins.elem "bao-client-key.pem:${v.authBaoClientKeyFile}" lc;
}
{
# The absence arm for the case above, and the option's own rule — a
# service declares its entry under its own `enable` — made checkable.

View file

@ -163,6 +163,10 @@ in
# rather than every hive's.
swarm-nats-auth = mkBinPackage "swarm-nats-auth" "hyperhive swarm queue auth-callout responder";
# The log store gateway's `auth_request` responder for agents. Out of
# `daemonBins` like the responder above: it runs on the log store's host only.
swarm-logs-auth = mkBinPackage "swarm-logs-auth" "hyperhive swarm log store agent auth responder";
# The matrix container's control binary; `mint` publishes the sender token. Out of `daemonBins` for
# "runs *inside* a container, not on the host" reason as the responder
# above, and with a second one: putting it in the core bundle would place

Some files were not shown because too many files have changed in this diff Show more