vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

swarm-nats-auth: open sealed callout requests and seal the replies

Browse source 
When nats-server has an auth_callout xkey it seals each request to that
key and names its own public xkey in the Nats-Server-Xkey header. The
responder now loads the callout xkey seed from the secret store before
it connects, opens a sealed request with it, and seals its reply (grant
or deny) back to the header's key. A request with no header is answered
in plaintext, as before.

A new `ensure-xkey` verb creates the pair when the store holds no seed:
the seed at swarm/services/swarm-nats/callout/xkey-seed, the public half
at swarm/public/swarm-nats/callout-xkey. Each write is a create-only
(cas=0) write, and a run whose create fails reads the stored value back,
so two concurrent runs end up with one key. The responder itself only
reads the seed and refuses to start without it.

swarm-secret-client gains SecretStore::create, the cas=0 write.
This commit is contained in:
atlas 2026-10-05 21:25:40 +02:00 • committed by mara
commit 1e0b10bb7d
6 changed files with 669 additions and 19 deletions
Download patch file Download diff file Expand all files Collapse all files

173
swarm-nats-auth/src/main.rs
View file

@ -14,18 +14,23 @@
//! indistinguishable from the responder being down, and the queue is the
//! swarm's control path.
//!
//! A request that arrives sealed to the callout xkey is opened with it, and
//! its reply is sealed back to the server; see `xkey`. The `ensure-xkey` verb
//! creates that key in the secret store, and runs before the server does.
//!
//! # Secrets
//!
//! Every credential is taken as a **path**, never a value. Two reasons, both
//! previously learned the hard way here: a value in nix config is rendered into
//! the world-readable store, and a value in `argv` is readable by anyone via
//! `/proc/<pid>/cmdline`, which is `0444`. Paths are not secrets, so passing
//! them as flags is fine.
//! them as flags is fine. The xkey seed is the exception to files: it is read
//! from the secret store, at the path `xkey` names.
use std::path::PathBuf;
use anyhow::Context;
use clap::Parser;
use clap::{Parser, Subcommand};
use futures_util::StreamExt;
mod agent_token;
@ -33,15 +38,39 @@ mod introspect;
mod policy;
mod request;
mod respond;
mod xkey;
/// Subject the NATS server publishes authorization requests on.
const AUTH_SUBJECT: &str = "$SYS.REQ.USER.AUTH";
/// With no verb this is the responder, and its flags are required.
#[derive(Debug, Parser)]
#[command(
name = "swarm-nats-auth",
about = "Auth-callout responder for the swarm NATS queue"
about = "Auth-callout responder for the swarm NATS queue",
args_conflicts_with_subcommands = true,
subcommand_negates_reqs = true
)]
struct Cli {
#[command(subcommand)]
command: Option<Command>,
#[command(flatten)]
responder: Option<Args>,
}
#[derive(Debug, Subcommand)]
enum Command {
/// Create the callout xkey in the secret store when it holds none, and
/// check that its public half is stored beside it. Needs no NATS server.
EnsureXkey {
/// Role on the store's `cert` auth mount to log in as.
#[arg(long, default_value = "swarm-nats-auth")]
store_cert_role: String,
},
}
#[derive(Debug, clap::Args)]
struct Args {
/// NATS server to connect to.
#[arg(long, default_value = "nats://127.0.0.1:4222")]
@ -161,8 +190,17 @@ async fn main() -> anyhow::Result<()> {
// they land in victorialogs as raw byte-array spam otherwise.
.with_ansi(false)
.init();
let args = Args::parse();
let cli = Cli::parse();
match (cli.command, cli.responder) {
(Some(Command::EnsureXkey { store_cert_role }), _) => ensure_xkey(&store_cert_role).await,
(None, Some(args)) => serve(&args).await,
(None, None) => unreachable!("clap requires the responder's flags when no verb is given"),
}
}
/// The responder: answers every authorization request until its
/// subscription ends.
async fn serve(args: &Args) -> anyhow::Result<()> {
let client_secret = read_secret(&args.client_secret_file)?;
// The bucket is NOT a flag. `swarm_queue_client::status`'s own docs say
// why: reader and writer must name the same bucket, and an option is a
@ -185,6 +223,15 @@ async fn main() -> anyhow::Result<()> {
let http = reqwest::Client::new();
let issuer = nkeys::KeyPair::from_seed(&read_secret(&args.issuer_seed_file)?)
.context("parse the account signing seed")?;
// Before the connect: a responder that cannot open a sealed request must
// not start answering. Read, never created here; see `xkey`.
let callout_xkey = xkey::load(
&swarm_secret_client::SecretStore::from_env(&args.store_cert_role)
.await
.context("logging in to the secret store for the callout xkey")?,
)
.await?;
tracing::info!(public_key = %callout_xkey.public_key(), "loaded the callout xkey");
let user_seed = read_secret(&args.user_seed_file)?;
let client = async_nats::ConnectOptions::with_nkey(user_seed)
.name("swarm-nats-auth")
@ -202,15 +249,8 @@ async fn main() -> anyhow::Result<()> {
);
while let Some(msg) = requests.next().await {
// Decode failures are logged and dropped, never propagated: this loop
// is the swarm's login path, and exiting on one malformed payload
// would let any client take authentication down for everyone.
let req = match request::decode(&msg.payload) {
Ok(req) => req,
Err(e) => {
tracing::warn!(error = ?e, "undecodable auth request, ignoring");
continue;
}
let Some((framing, req)) = open_request(&callout_xkey, &msg) else {
continue;
};
// No token is a denial, not an error: an anonymous connect is a
// normal thing for a client to attempt and an abnormal thing to
@ -233,7 +273,7 @@ async fn main() -> anyhow::Result<()> {
(None, None)
}
Some(agent_token::Presented::Bearer(token)) => {
introspected(&policy, &http, &args, &client_secret, token).await
introspected(&policy, &http, args, &client_secret, token).await
}
None => (None, None),
};
@ -245,6 +285,7 @@ async fn main() -> anyhow::Result<()> {
user_nkey = %req.user_nkey,
server_id = %req.server_id.id,
granted = permissions.is_some(),
sealed = framing.is_sealed(),
caller = caller.as_deref().unwrap_or("-"),
"auth request"
);
@ -267,13 +308,65 @@ async fn main() -> anyhow::Result<()> {
),
None => respond::deny(&issuer, &req.server_id.id, &req.user_nkey),
};
if let Err(e) = client.publish(reply_to, token.into()).await {
let reply = match framing.seal(&callout_xkey, token.into_bytes()) {
Ok(reply) => reply,
Err(e) => {
tracing::warn!(error = %e, "could not seal the auth response; dropping");
continue;
}
};
if let Err(e) = client.publish(reply_to, reply.into()).await {
tracing::warn!(error = ?e, "failed to publish auth response");
}
}
anyhow::bail!("subscription to {AUTH_SUBJECT} ended")
}
/// The request in `msg`, opened when the server sealed it, and the framing
/// its reply takes. `None`, logged, for anything else.
///
/// Logged and dropped, never propagated: the loop calling this is the
/// swarm's login path, and exiting on one malformed payload would let any
/// client take authentication down for everyone.
fn open_request(
callout_xkey: &nkeys::XKey,
msg: &async_nats::Message,
) -> Option<(xkey::Framing, request::AuthRequest)> {
let server_xkey = msg
.headers
.as_ref()
.and_then(|h| h.get(xkey::SERVER_XKEY_HEADER))
.map(async_nats::HeaderValue::as_str);
let (framing, payload) = match xkey::Framing::open(callout_xkey, server_xkey, &msg.payload) {
Ok(opened) => opened,
Err(e) => {
tracing::warn!(error = %e, "unopenable auth request, ignoring");
return None;
}
};
match request::decode(&payload) {
Ok(req) => Some((framing, req)),
Err(e) => {
tracing::warn!(error = ?e, "undecodable auth request, ignoring");
None
}
}
}
/// The `ensure-xkey` verb: see [`xkey::ensure`].
async fn ensure_xkey(store_cert_role: &str) -> anyhow::Result<()> {
let store = swarm_secret_client::SecretStore::from_env(store_cert_role)
.await
.context("logging in to the secret store")?;
let key = xkey::ensure(&store).await?;
tracing::info!(
public_key = %key.public_key(),
path = xkey::PUBLIC_PATH,
"the callout xkey is in the secret store"
);
Ok(())
}
/// The OIDC path: who the `IdP` says presented `token`, and what the policy
/// grants that client. `None` permissions is a denial.
///
@ -322,6 +415,14 @@ async fn introspected(
mod tests {
use super::*;
/// The responder's flags, parsed the way `main` parses them.
fn responder<const N: usize>(argv: [&str; N]) -> Result<Args, clap::Error> {
Cli::try_parse_from(argv).map(|cli| {
assert!(cli.command.is_none(), "no verb was given");
cli.responder.expect("with no verb, the responder's flags")
})
}
/// The argument vector `swarm-nats.nix` actually builds, in the form it
/// builds it: every flag space-separated, and the suffix's value opening
/// with `-`. Read as a short-flag cluster it aborts before `main` runs,
@ -329,7 +430,7 @@ mod tests {
/// the operator as `authorization violation` on an unrelated process.
#[test]
fn the_units_own_argument_vector_parses() {
let args = Args::try_parse_from([
let args = responder([
"swarm-nats-auth",
"--user-seed-file",
"/run/credentials/callout-user.seed",
@ -366,7 +467,7 @@ mod tests {
/// the argument being unknown.
#[test]
fn an_agent_suffix_without_a_leading_dash_parses_too() {
let args = Args::try_parse_from([
let args = responder([
"swarm-nats-auth",
"--user-seed-file",
"/run/credentials/callout-user.seed",
@ -387,7 +488,7 @@ mod tests {
/// drift into disagreeing about what an agent client is called.
#[test]
fn the_default_suffix_is_what_the_module_passes() {
let args = Args::try_parse_from([
let args = responder([
"swarm-nats-auth",
"--user-seed-file",
"/run/credentials/callout-user.seed",
@ -401,4 +502,40 @@ mod tests {
.expect("the required flags alone must parse");
assert_eq!(args.agent_client_suffix, "-agent");
}
/// The verb as `swarm-bao-nats-callout-xkey` runs it, with none of the
/// responder's required flags.
#[test]
fn the_ensure_verb_parses_as_the_unit_runs_it() {
let cli = Cli::try_parse_from([
"swarm-nats-auth",
"ensure-xkey",
"--store-cert-role",
"swarm-nats-auth",
])
.expect("the verb needs no responder flags");
assert!(cli.responder.is_none());
assert!(matches!(
cli.command,
Some(Command::EnsureXkey { ref store_cert_role }) if store_cert_role == "swarm-nats-auth"
));
}
/// A bare invocation is neither: the responder's flags stay required
/// when no verb is given.
#[test]
fn no_verb_and_no_responder_flags_is_refused() {
Cli::try_parse_from(["swarm-nats-auth"]).expect_err("the responder's flags are required");
}
#[test]
fn the_verb_does_not_take_the_responders_flags() {
Cli::try_parse_from([
"swarm-nats-auth",
"--user-seed-file",
"/run/credentials/callout-user.seed",
"ensure-xkey",
])
.expect_err("a verb and the responder's flags conflict");
}
}