vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

swarm-nats-auth: refuse an auth request the server did not seal

Browse source 
Every nats-server this module starts renders `auth_callout.xkey` or does
not start, and the responder serves only that server, in the same
container and generation. A request without `Nats-Server-Xkey` therefore
comes from no supported server, so the plaintext framing is gone: such a
request is logged at warn and gets no reply, and the server that sent it
times out the connect as a denial. A signed denial would have had to go
out unsealed, which is the path this removes.

Also scopes two claims to what is true: the xkey's public half is read by
the swarm-nats-auth policy only, and the metadata-delete gap in the
replace-key procedure is about the two xkey paths, not all of
`secret/metadata/`.
This commit is contained in:
atlas 2026-10-05 22:46:52 +02:00 • committed by mara
commit 923554cd62
5 changed files with 78 additions and 79 deletions
Download patch file Download diff file Expand all files Collapse all files

6
docs/swarm/secrets.md
View file

@ -213,9 +213,9 @@ systemctl restart swarm-bao-nats-callout-xkey
`bao kv delete` isn't enough. It deletes the latest version but keeps the
path's metadata, so the store still counts the path as existing and refuses
the create that makes a new key. No principal hyperhive configures
may delete under `secret/metadata/`, so run the two deletes with a token
that may, such as the root token.
the create that makes a new key. No principal hyperhive configures may
delete either path's metadata, so run the two deletes with a token that may,
such as the root token.
## Hive-level — one of each per hive

7
nix/host-modules/swarm-nats.nix
View file

@ -221,9 +221,10 @@ let
userPublicFile = "callout-user.pub";
issuerPublicFile = "issuer.pub";
# The callout xkey's public half in the store, readable by any token. The
# Rust half is `swarm-nats-auth`'s `xkey::PUBLIC_PATH`; the seed beside it
# is `xkeySeedLeaf` in ./swarm-bao.nix.
# The callout xkey's public half in the store, readable by the
# `swarm-nats-auth` role alone. The Rust half is `swarm-nats-auth`'s
# `xkey::PUBLIC_PATH`; the seed beside it is `xkeySeedLeaf` in
# ./swarm-bao.nix.
xkeyPublicPath = "secret/swarm/public/swarm-nats/callout-xkey";
# Each placeholder in `calloutTemplate`, as a `sed` expression over the

6
swarm-nats-auth/README.md
View file

@ -37,9 +37,9 @@ not part of the path that runs.
The server seals each auth request to the callout xkey and names its own
public xkey in the `Nats-Server-Xkey` header; the responder opens the request
with the xkey's seed and seals its reply to that header's key. It answers a
request with no header in plaintext, which is what a server with no xkey
configured reads.
with the xkey's seed and seals its reply to that header's key. The responder
logs a warning for a request with no header, or one that doesn't open, and
sends no reply: the server it serves always seals, so no server waits on one.
The seed is in the swarm secret store, and the responder reads it before it
connects to the queue: a responder without it doesn't start. The

44
swarm-nats-auth/src/main.rs
View file

@ -256,7 +256,7 @@ async fn serve(args: &Args) -> anyhow::Result<()> {
);
while let Some(msg) = requests.next().await {
let Some((framing, req)) = open_request(&callout_xkey, &msg) else {
let Some((server_key, req)) = open_request(&callout_xkey, &msg) else {
continue;
};
// No token is a denial, not an error: an anonymous connect is a
@ -292,7 +292,6 @@ async fn serve(args: &Args) -> anyhow::Result<()> {
user_nkey = %req.user_nkey,
server_id = %req.server_id.id,
granted = permissions.is_some(),
sealed = framing.is_sealed(),
caller = caller.as_deref().unwrap_or("-"),
"auth request"
);
@ -315,7 +314,7 @@ async fn serve(args: &Args) -> anyhow::Result<()> {
),
None => respond::deny(&issuer, &req.server_id.id, &req.user_nkey),
};
let reply = match framing.seal(&callout_xkey, token.into_bytes()) {
let reply = match xkey::seal(&callout_xkey, &server_key, token.as_bytes()) {
Ok(reply) => reply,
Err(e) => {
tracing::warn!(error = %e, "could not seal the auth response; dropping");
@ -329,8 +328,9 @@ async fn serve(args: &Args) -> anyhow::Result<()> {
anyhow::bail!("subscription to {AUTH_SUBJECT} ended")
}
/// The request in `msg`, opened when the server sealed it, and the framing
/// its reply takes. `None`, logged, for anything else.
/// The request in `msg`, opened, and the server key its reply is sealed to.
/// `None`, logged, for anything else, a request with no
/// [`xkey::SERVER_XKEY_HEADER`] included.
///
/// Logged and dropped, never propagated: the loop calling this is the
/// swarm's login path, and exiting on one malformed payload would let any
@ -338,13 +338,13 @@ async fn serve(args: &Args) -> anyhow::Result<()> {
fn open_request(
callout_xkey: &nkeys::XKey,
msg: &async_nats::Message,
) -> Option<(xkey::Framing, request::AuthRequest)> {
let server_xkey = msg
) -> Option<(nkeys::XKey, request::AuthRequest)> {
let header = msg
.headers
.as_ref()
.and_then(|h| h.get(xkey::SERVER_XKEY_HEADER))
.map(async_nats::HeaderValue::as_str);
let (framing, payload) = match xkey::Framing::open(callout_xkey, server_xkey, &msg.payload) {
let (server_key, payload) = match xkey::open(callout_xkey, header, &msg.payload) {
Ok(opened) => opened,
Err(e) => {
tracing::warn!(error = %e, "unopenable auth request, ignoring");
@ -352,7 +352,7 @@ fn open_request(
}
};
match request::decode(&payload) {
Ok(req) => Some((framing, req)),
Ok(req) => Some((server_key, req)),
Err(e) => {
tracing::warn!(error = ?e, "undecodable auth request, ignoring");
None
@ -469,6 +469,32 @@ mod tests {
assert_eq!(args.agent_token_publish_subjects.len(), 5);
}
/// A request with no `Nats-Server-Xkey` is dropped before anything reads
/// it, so it gets no reply. The same request sealed, with the header,
/// opens.
#[test]
fn a_request_without_the_server_xkey_header_is_dropped() {
let own = nkeys::XKey::new();
let message = |headers, payload: Vec<u8>| async_nats::Message {
subject: AUTH_SUBJECT.into(),
reply: Some("_INBOX.reply".into()),
payload: payload.into(),
headers,
status: None,
description: None,
length: 0,
};
let request = xkey::tests::request_jwt();
assert!(open_request(&own, &message(None, request.clone())).is_none());
let server = nkeys::XKey::new();
let own_public = nkeys::XKey::from_public_key(&own.public_key()).expect("public");
let sealed = server.seal(&request, &own_public).expect("sealed");
let mut headers = async_nats::HeaderMap::new();
headers.insert(xkey::SERVER_XKEY_HEADER, server.public_key());
assert!(open_request(&own, &message(Some(headers), sealed)).is_some());
}
/// The control for the case above: an ordinary value parses through the
/// same flag, so a failure there is about the leading `-` and not about
/// the argument being unknown.

94
swarm-nats-auth/src/xkey.rs
View file

@ -137,54 +137,34 @@ fn from_seed(seed: &str) -> Result<XKey> {
XKey::from_seed(seed.trim()).map_err(|_| anyhow!("{SEED_PATH} does not hold a curve seed"))
}
/// How one request arrived, which is how its reply has to leave.
pub enum Framing {
/// No [`SERVER_XKEY_HEADER`]: the server reads the reply as it is.
Plain,
/// Sealed to this responder by the server key the header named.
Sealed(XKey),
/// `payload` opened with `own`, and the server key `server_key` names, which
/// the reply is sealed to.
///
/// # Errors
/// When there is no [`SERVER_XKEY_HEADER`], when it is not a public xkey, and
/// when the payload does not open with `own`.
pub fn open(own: &XKey, server_key: Option<&str>, payload: &[u8]) -> Result<(XKey, Vec<u8>)> {
let server_key = server_key
.with_context(|| format!("the request is not sealed: no {SERVER_XKEY_HEADER}"))?;
let server = XKey::from_public_key(server_key)
.map_err(|_| anyhow!("{SERVER_XKEY_HEADER} is not a public xkey"))?;
let opened = own
.open(payload, &server)
.map_err(|_| anyhow!("the request does not open with this responder's xkey"))?;
Ok((server, opened))
}
impl Framing {
/// `payload` as the server sent it, opened when `server_key` says it was
/// sealed.
///
/// # Errors
/// When the header is not a public xkey, or the payload does not open
/// with `own`.
pub fn open(own: &XKey, server_key: Option<&str>, payload: &[u8]) -> Result<(Self, Vec<u8>)> {
let Some(server_key) = server_key else {
return Ok((Self::Plain, payload.to_vec()));
};
let server = XKey::from_public_key(server_key)
.map_err(|_| anyhow!("{SERVER_XKEY_HEADER} is not a public xkey"))?;
let opened = own
.open(payload, &server)
.map_err(|_| anyhow!("the request does not open with this responder's xkey"))?;
Ok((Self::Sealed(server), opened))
}
/// `reply` framed the way the request was.
///
/// # Errors
/// When sealing fails, which it does only for a key with no seed.
pub fn seal(&self, own: &XKey, reply: Vec<u8>) -> Result<Vec<u8>> {
match self {
Self::Plain => Ok(reply),
Self::Sealed(server) => own
.seal(&reply, server)
.map_err(|_| anyhow!("sealing the reply")),
}
}
/// Whether the reply leaves sealed.
pub fn is_sealed(&self) -> bool {
matches!(self, Self::Sealed(_))
}
/// `reply` sealed by `own` to `server`.
///
/// # Errors
/// When sealing fails, which it does only for a key with no seed.
pub fn seal(own: &XKey, server: &XKey, reply: &[u8]) -> Result<Vec<u8>> {
own.seal(reply, server)
.map_err(|_| anyhow!("sealing the reply"))
}
#[cfg(test)]
mod tests {
pub(crate) mod tests {
use std::collections::HashMap;
use std::sync::Mutex;
use std::sync::atomic::{AtomicUsize, Ordering};
@ -335,7 +315,7 @@ mod tests {
assert_eq!(key.public_key(), existing.public_key());
}
fn request_jwt() -> Vec<u8> {
pub(crate) fn request_jwt() -> Vec<u8> {
let b = |s: &str| data_encoding::BASE64URL_NOPAD.encode(s.as_bytes());
format!(
"{}.{}.{}",
@ -356,9 +336,8 @@ mod tests {
let own_public = XKey::from_public_key(&own.public_key()).expect("public");
let sealed = server.seal(&request_jwt(), &own_public).expect("sealed");
let (framing, opened) =
Framing::open(&own, Some(&server.public_key()), &sealed).expect("opens");
assert!(framing.is_sealed());
let (server_key, opened) = open(&own, Some(&server.public_key()), &sealed).expect("opens");
assert_eq!(server_key.public_key(), server.public_key());
let req = crate::request::decode(&opened).expect("a request");
assert_eq!(req.user_nkey, "UCLIENT");
@ -374,9 +353,7 @@ mod tests {
&req.user_nkey,
&permissions,
);
let reply = framing
.seal(&own, token.clone().into_bytes())
.expect("sealed");
let reply = seal(&own, &server_key, token.as_bytes()).expect("sealed");
assert_ne!(reply, token.as_bytes(), "the reply leaves sealed");
let read_by_server = server
.open(&reply, &own_public)
@ -384,16 +361,11 @@ mod tests {
assert_eq!(read_by_server, token.as_bytes());
}
/// A server with no xkey configured sends no header and reads the reply
/// as it is.
#[test]
fn a_request_without_the_header_is_answered_in_plaintext() {
fn a_request_without_the_header_is_refused() {
let own = XKey::new();
let (framing, opened) = Framing::open(&own, None, &request_jwt()).expect("plain");
assert!(!framing.is_sealed());
assert_eq!(opened, request_jwt());
let reply = framing.seal(&own, b"a.b.c".to_vec()).expect("plain");
assert_eq!(reply, b"a.b.c");
let e = open(&own, None, &request_jwt()).expect_err("no header");
assert!(format!("{e:#}").contains(SERVER_XKEY_HEADER), "{e:#}");
}
#[test]
@ -402,9 +374,9 @@ mod tests {
let server = XKey::new();
let someone_else = XKey::from_public_key(&XKey::new().public_key()).expect("public");
let sealed = server.seal(&request_jwt(), &someone_else).expect("sealed");
assert!(Framing::open(&own, Some(&server.public_key()), &sealed).is_err());
assert!(open(&own, Some(&server.public_key()), &sealed).is_err());
// The header present over a payload that was never sealed.
assert!(Framing::open(&own, Some(&server.public_key()), &request_jwt()).is_err());
assert!(Framing::open(&own, Some("not-a-key"), &sealed).is_err());
assert!(open(&own, Some(&server.public_key()), &request_jwt()).is_err());
assert!(open(&own, Some("not-a-key"), &sealed).is_err());
}
}