`swarm/agents/<agent>/bao-mtls` did not exist, and neither did any per-agent identity at the secret store: `policy::agent_object_name`, `render_agent` and `render_agent_with_queue` had been written and never called outside their own tests. An agent's only "per-agent" secret today is read under the HIVE's certificate, through a wide grant on `swarm/agents/*` — so "per-agent" was presentational. The swarm now mints the certificate, so no hive ever needs the capability to mint one. `swarm-controller` is the service that does it: it already logs in to the store, and its existing grant already covers exactly the three objects written here (`create/update` on `secret/data/swarm/agents/*`, `sys/policies/acl/hive-*` and `auth/cert/certs/hive-*`). No new bao grant, and nothing co-located — a cert-auth role pins its authority by value, per role, so the controller issues from its own CA on its own host and pins that CA in the role it writes. No existing role changes. The mint node does not report success on a write. After publishing it connects again, with the leaf it just issued and under the role it just wrote, and reads the path back — so the policy, the role, the common name and the leaf are exercised in production on every agent creation. A certificate this code mints that the role this code writes will not accept turns the job node red at creation time instead of surfacing later as an agent container that cannot start. `TriggerDeploy` gains an `after_any` edge on the mint, not `after_ok`: a hive cannot pass down a certificate the swarm has not published, but a host with no authority configured must still create agents exactly as it does today. The private key is generated in memory and never written to disk on the controller — `SecretStore::connect_with_identity` takes the PEM the minter is already holding, so nothing is written out purely to be logged in with. Refs #4137
367 lines
14 KiB
Rust
367 lines
14 KiB
Rust
//! A logged-in handle on the store, built from this deployment's environment.
|
|
|
|
use rustify_derive::Endpoint;
|
|
use serde::{Serialize, de::DeserializeOwned};
|
|
use vaultrs::client::{Client, VaultClient, VaultClientSettingsBuilder};
|
|
|
|
use crate::{Error, path::MOUNT};
|
|
|
|
/// The store's address.
|
|
pub const ENV_ADDR: &str = "BAO_ADDR";
|
|
/// PEM client certificate presented to the store's listener.
|
|
pub const ENV_CLIENT_CERT: &str = "BAO_CLIENT_CERT";
|
|
/// PEM private key for [`ENV_CLIENT_CERT`].
|
|
pub const ENV_CLIENT_KEY: &str = "BAO_CLIENT_KEY";
|
|
/// CA bundle the store's own certificate is verified against. Optional:
|
|
/// absent means the system trust store, which is what a deployment with a
|
|
/// real CA wants and what a self-signed one must not be left with.
|
|
pub const ENV_CACERT: &str = "BAO_CACERT";
|
|
|
|
/// The auth mount a cert login goes through, unless a caller says otherwise.
|
|
pub const DEFAULT_CERT_MOUNT: &str = "cert";
|
|
|
|
/// Where the store is and which identity we present to it.
|
|
///
|
|
/// Separate from the connect so the environment can be checked without one:
|
|
/// every arm below is a misconfiguration an operator has to read an error
|
|
/// about, and none of them needs a reachable store to happen.
|
|
#[derive(Debug, PartialEq, Eq)]
|
|
pub struct Settings {
|
|
address: String,
|
|
cert_path: String,
|
|
key_path: String,
|
|
ca_path: Option<String>,
|
|
}
|
|
|
|
impl Settings {
|
|
/// Read the `BAO_*` variables from the process environment.
|
|
///
|
|
/// # Errors
|
|
/// [`Error::MissingEnv`] naming the first variable that is unset or empty.
|
|
pub fn from_env() -> Result<Self, Error> {
|
|
Self::from_lookup(|k| std::env::var(k).ok())
|
|
}
|
|
|
|
/// [`Settings::from_env`] against an arbitrary lookup.
|
|
///
|
|
/// [`vaultrs`]'s own defaults are deliberately not used: it looks for
|
|
/// `VAULT_ADDR` / `VAULT_CLIENT_CERT` / `VAULT_CLIENT_KEY`, and every unit
|
|
/// in this tree sets the `BAO_` spellings. Falling through to those
|
|
/// defaults yields a client with **no identity**, which fails at the TLS
|
|
/// handshake rather than anywhere that names the cause.
|
|
///
|
|
/// # Errors
|
|
/// [`Error::MissingEnv`] naming the first variable that is unset or empty.
|
|
pub fn from_lookup(get: impl Fn(&str) -> Option<String>) -> Result<Self, Error> {
|
|
let required = |var: &'static str| -> Result<String, Error> {
|
|
get(var)
|
|
.filter(|v| !v.is_empty())
|
|
.ok_or(Error::MissingEnv(var))
|
|
};
|
|
Ok(Self {
|
|
address: required(ENV_ADDR)?,
|
|
cert_path: required(ENV_CLIENT_CERT)?,
|
|
key_path: required(ENV_CLIENT_KEY)?,
|
|
ca_path: get(ENV_CACERT).filter(|v| !v.is_empty()),
|
|
})
|
|
}
|
|
}
|
|
|
|
/// A client that has already exchanged its certificate for a token.
|
|
pub struct SecretStore {
|
|
inner: VaultClient,
|
|
}
|
|
|
|
fn read_file(var: &'static str, path: &str) -> Result<Vec<u8>, Error> {
|
|
std::fs::read(path).map_err(|source| Error::Identity {
|
|
var,
|
|
path: path.to_owned(),
|
|
source,
|
|
})
|
|
}
|
|
|
|
impl SecretStore {
|
|
/// Connect using the `BAO_*` environment and log in with the certificate
|
|
/// auth method, returning a handle that already holds a token.
|
|
///
|
|
/// `cert_role` is the role configured on the store's `cert` mount; it
|
|
/// selects which policy the returned token carries.
|
|
///
|
|
/// # Errors
|
|
/// As [`SecretStore::connect`], plus [`Error::MissingEnv`].
|
|
pub async fn from_env(cert_role: &str) -> Result<Self, Error> {
|
|
Self::connect(&Settings::from_env()?, cert_role, DEFAULT_CERT_MOUNT).await
|
|
}
|
|
|
|
/// Present `settings`' identity to the store and log in at `cert_mount`.
|
|
///
|
|
/// # Errors
|
|
/// [`Error::Identity`] when a named file cannot be read, [`Error::Tls`]
|
|
/// when the cert/key pair does not form a usable identity,
|
|
/// [`Error::Settings`] when the address will not parse, and
|
|
/// [`Error::Vault`] when the store refuses the login — which is what an
|
|
/// unconfigured `cert` mount looks like from here.
|
|
pub async fn connect(
|
|
settings: &Settings,
|
|
cert_role: &str,
|
|
cert_mount: &str,
|
|
) -> Result<Self, Error> {
|
|
// One PEM blob holding both, which is the shape `Identity::from_pem`
|
|
// wants; the two stay separate on disk because the key is the half
|
|
// that gets `0600`.
|
|
let mut pem = read_file(ENV_CLIENT_CERT, &settings.cert_path)?;
|
|
pem.push(b'\n');
|
|
pem.extend_from_slice(&read_file(ENV_CLIENT_KEY, &settings.key_path)?);
|
|
Self::connect_with_identity(settings, &pem, cert_role, cert_mount).await
|
|
}
|
|
|
|
/// Log in at the store `settings` names, presenting an identity the caller
|
|
/// is already holding rather than one named by the environment.
|
|
///
|
|
/// `identity_pem` is the certificate and its private key concatenated into
|
|
/// one PEM blob — the shape [`reqwest::Identity::from_pem`] takes, and the
|
|
/// shape [`SecretStore::connect`] builds out of the two files
|
|
/// [`ENV_CLIENT_CERT`] and [`ENV_CLIENT_KEY`] name. Only the address and
|
|
/// the CA are read out of `settings` here; its two identity paths are not
|
|
/// touched.
|
|
///
|
|
/// This is what a **minter** needs. A process that has just issued a leaf
|
|
/// holds the bytes, and the safest place for a freshly minted private key
|
|
/// is the memory it was generated in: writing it to a file purely so that
|
|
/// a `Settings` could name it would put a key on disk for no other reason
|
|
/// than to log in with it once.
|
|
///
|
|
/// # Errors
|
|
/// [`Error::Tls`] when `identity_pem` is not a usable certificate/key
|
|
/// pair, [`Error::Settings`] when the address will not parse, and
|
|
/// [`Error::Vault`] when the store refuses the login — which is what a
|
|
/// certificate no cert-auth role accepts looks like from here.
|
|
pub async fn connect_with_identity(
|
|
settings: &Settings,
|
|
identity_pem: &[u8],
|
|
cert_role: &str,
|
|
cert_mount: &str,
|
|
) -> Result<Self, Error> {
|
|
let identity = reqwest::Identity::from_pem(identity_pem).map_err(Error::Tls)?;
|
|
|
|
let mut builder = VaultClientSettingsBuilder::default();
|
|
builder
|
|
.address(settings.address.clone())
|
|
.identity(Some(identity));
|
|
if let Some(ca) = &settings.ca_path {
|
|
builder.ca_certs(vec![ca.clone()]);
|
|
}
|
|
let built = builder
|
|
.build()
|
|
.map_err(|e| Error::Settings(e.to_string()))?;
|
|
|
|
let mut inner = VaultClient::new(built)?;
|
|
let auth = vaultrs::auth::cert::login(&inner, cert_mount, cert_role).await?;
|
|
inner.set_token(&auth.client_token);
|
|
Ok(Self { inner })
|
|
}
|
|
|
|
/// Read the object stored at `path`, decoded as `T`.
|
|
///
|
|
/// The shape belongs to whoever owns the path — see [`crate::matrix`] for
|
|
/// the one kind of secret that exists — so this moves bytes and knows no
|
|
/// fields.
|
|
///
|
|
/// # Errors
|
|
/// [`Error::Vault`] when the path does not exist, the token's policy does
|
|
/// not cover it, or the stored object does not decode as `T`.
|
|
pub async fn read<T: DeserializeOwned>(&self, path: &str) -> Result<T, Error> {
|
|
Ok(vaultrs::kv2::read(&self.inner, MOUNT, path).await?)
|
|
}
|
|
|
|
/// Write `value` at `path`, creating a new version.
|
|
///
|
|
/// # Errors
|
|
/// [`Error::Vault`] when the token's policy does not cover the path.
|
|
pub async fn write<T: Serialize + Sync>(&self, path: &str, value: &T) -> Result<(), Error> {
|
|
vaultrs::kv2::set(&self.inner, MOUNT, path, value).await?;
|
|
Ok(())
|
|
}
|
|
|
|
/// Replace the ACL policy named `name` with `policy`.
|
|
///
|
|
/// A whole-document write, not a merge: the store has no other verb, and
|
|
/// the caller renders the document from the current agent set anyway, so
|
|
/// a stanza that is gone from the render is meant to be gone from the
|
|
/// grant. Render the text with [`crate::policy`] rather than by hand.
|
|
///
|
|
/// # Errors
|
|
/// [`Error::Vault`] when the token's own policy does not cover
|
|
/// `sys/policies/acl/<name>` — which is what a controller scoped to the
|
|
/// `hive-*` namespace gets for any other name.
|
|
pub async fn write_policy(&self, name: &str, policy: &str) -> Result<(), Error> {
|
|
let request = WriteAclPolicy {
|
|
name: name.to_owned(),
|
|
policy: policy.to_owned(),
|
|
};
|
|
vaultrs::api::exec_with_empty(&self.inner, request).await?;
|
|
Ok(())
|
|
}
|
|
|
|
/// Create or replace the cert-auth role `name`, so that a certificate
|
|
/// issued by `certificate` and carrying `common_name` logs in with
|
|
/// `policy`.
|
|
///
|
|
/// `certificate` is the authority by value, not a path: the store keeps
|
|
/// its own copy inside the role and never reads a file of ours.
|
|
///
|
|
/// # Errors
|
|
/// [`Error::Vault`] when the token's policy does not cover
|
|
/// `auth/<mount>/certs/<name>`, or the store rejects the authority.
|
|
pub async fn write_cert_role(
|
|
&self,
|
|
mount: &str,
|
|
name: &str,
|
|
certificate: &str,
|
|
common_name: &str,
|
|
policy: &str,
|
|
) -> Result<(), Error> {
|
|
let mut opts =
|
|
vaultrs::api::auth::cert::requests::CreateCaCertificateRoleRequest::builder();
|
|
opts.allowed_common_names(vec![common_name.to_owned()])
|
|
.token_policies(vec![policy.to_owned()]);
|
|
vaultrs::auth::cert::ca_cert_role::set(
|
|
&self.inner,
|
|
mount,
|
|
name,
|
|
certificate,
|
|
Some(&mut opts),
|
|
)
|
|
.await?;
|
|
Ok(())
|
|
}
|
|
}
|
|
|
|
/// Write an ACL policy, spelled out here because [`vaultrs`] does not have it:
|
|
/// its `sys::policy` module targets `sys/policy/<name>`, the deprecated alias,
|
|
/// and the store ACLs that path separately from `sys/policies/acl/<name>`. A
|
|
/// token granted the latter — which is what every grant in this tree names —
|
|
/// is refused at the former with a 403.
|
|
#[derive(Endpoint, Serialize)]
|
|
#[endpoint(path = "sys/policies/acl/{self.name}", method = "PUT")]
|
|
struct WriteAclPolicy {
|
|
name: String,
|
|
/// Marked as the body so `name` stays in the path alone: an untagged field
|
|
/// would be serialised into the request too.
|
|
#[endpoint(body)]
|
|
policy: String,
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
/// A lookup standing in for a fully-configured unit's environment.
|
|
fn full(k: &str) -> Option<String> {
|
|
match k {
|
|
ENV_ADDR => Some("https://store.invalid:8200".to_owned()),
|
|
ENV_CLIENT_CERT => Some("/run/c.pem".to_owned()),
|
|
ENV_CLIENT_KEY => Some("/run/k.pem".to_owned()),
|
|
_ => None,
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn a_complete_environment_is_accepted() {
|
|
// The control: without this, every assertion below could be passing
|
|
// because `from_lookup` rejects everything.
|
|
let s = Settings::from_lookup(full).expect("every required variable is set");
|
|
assert_eq!(s.address, "https://store.invalid:8200");
|
|
assert_eq!(s.ca_path, None, "an absent CA is the system trust store");
|
|
}
|
|
|
|
#[test]
|
|
fn each_required_variable_is_named_when_it_is_the_missing_one() {
|
|
for var in [ENV_ADDR, ENV_CLIENT_CERT, ENV_CLIENT_KEY] {
|
|
let e = Settings::from_lookup(|k| if k == var { None } else { full(k) })
|
|
.expect_err("one required variable is absent");
|
|
assert!(
|
|
matches!(e, Error::MissingEnv(v) if v == var),
|
|
"dropping {var} should name {var}, got {e:?}"
|
|
);
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn an_empty_variable_is_as_absent_as_an_unset_one() {
|
|
// systemd writes `Environment=BAO_CACERT=` for an unset nix option, so
|
|
// empty is the shape this actually arrives in.
|
|
let e = Settings::from_lookup(|k| {
|
|
if k == ENV_ADDR {
|
|
Some(String::new())
|
|
} else {
|
|
full(k)
|
|
}
|
|
})
|
|
.expect_err("an empty address is not an address");
|
|
assert!(matches!(e, Error::MissingEnv(ENV_ADDR)), "got {e:?}");
|
|
|
|
let s = Settings::from_lookup(|k| {
|
|
if k == ENV_CACERT {
|
|
Some(String::new())
|
|
} else {
|
|
full(k)
|
|
}
|
|
})
|
|
.expect("an empty CA is optional, not fatal");
|
|
assert_eq!(s.ca_path, None);
|
|
}
|
|
|
|
#[test]
|
|
fn an_unreadable_cert_names_the_file_and_the_variable() {
|
|
let e = read_file(ENV_CLIENT_CERT, "/nonexistent/cert.pem")
|
|
.expect_err("the file does not exist");
|
|
match e {
|
|
Error::Identity { var, ref path, .. } => {
|
|
assert_eq!(var, ENV_CLIENT_CERT);
|
|
assert_eq!(path, "/nonexistent/cert.pem");
|
|
}
|
|
other => panic!("wanted an Identity error, got {other:?}"),
|
|
}
|
|
}
|
|
|
|
/// The defect this pins: the store gates `sys/policy/<name>` and
|
|
/// `sys/policies/acl/<name>` separately, so a client on the first is
|
|
/// refused by a grant naming the second — and nothing else in the tree
|
|
/// says which one is addressed.
|
|
#[test]
|
|
fn a_policy_write_addresses_the_modern_acl_path() {
|
|
use rustify::endpoint::Endpoint as _;
|
|
|
|
let request = WriteAclPolicy {
|
|
name: "hive-pr1ma".to_owned(),
|
|
policy: crate::policy::render("pr1ma").expect("a plain name is legal"),
|
|
};
|
|
assert_eq!(request.path(), "sys/policies/acl/hive-pr1ma");
|
|
}
|
|
|
|
#[test]
|
|
fn the_request_body_carries_the_policy_and_not_the_name() {
|
|
use rustify::endpoint::Endpoint as _;
|
|
|
|
let request = WriteAclPolicy {
|
|
name: "hive-pr1ma".to_owned(),
|
|
policy: crate::policy::render("pr1ma").expect("a plain name is legal"),
|
|
};
|
|
let body = request
|
|
.body()
|
|
.expect("the body serialises")
|
|
.expect("a policy write sends one");
|
|
let sent: serde_json::Value =
|
|
serde_json::from_slice(&body).expect("the body is the JSON the store parses");
|
|
assert_eq!(
|
|
sent["policy"],
|
|
crate::policy::render("pr1ma").expect("legal")
|
|
);
|
|
assert!(
|
|
sent.get("name").is_none(),
|
|
"`name` addresses the policy in the path; sending it too would make \
|
|
the store's copy of the document disagree with its own name"
|
|
);
|
|
}
|
|
}
|