Watch
0
0
Fork
You've already forked hyperhive
0

swarmctl: add bao init and bao grant-bootstrap

init prints the store's keys and root token once and keeps none; with
--and-grant it goes on to the granter bootstrap in the same process.
grant-bootstrap does that step alone, reading the root token from a
hidden prompt or stdin. The token reaches bao only as BAO_TOKEN in the
child's environment.
This commit is contained in:
atlas 2026-11-08 01:29:09 +01:00 • committed by mara
commit a20596fa59
11 changed files with 909 additions and 26 deletions

1
Cargo.lock generated
View file

@ -5369,6 +5369,7 @@ dependencies = [
"swarm-wire", "swarm-wire",
"tokio", "tokio",
"url", "url",
"zeroize",
] ]
[[package]] [[package]]

View file

@ -174,6 +174,8 @@ tracing-journald = "0.3.2"
rustix = { version = "1.1.4", default-features = false, features = [ rustix = { version = "1.1.4", default-features = false, features = [
"std", "std",
"fs", "fs",
# `swarmctl bao grant-bootstrap` turns terminal echo off for the root token.
"termios",
] } ] }
reqwest = { version = "0.13", default-features = false, features = [ reqwest = { version = "0.13", default-features = false, features = [
# RFC 7662 introspection posts an urlencoded body; without this, # RFC 7662 introspection posts an urlencoded body; without this,
@ -287,6 +289,9 @@ utoipa-axum = "0.2"
# re-issue it at half its life. `der` and `spki` are already in the tree # re-issue it at half its life. `der` and `spki` are already in the tree
# through `pkcs8`. # through `pkcs8`.
x509-cert = { version = "0.2", default-features = false, features = ["pem"] } x509-cert = { version = "0.2", default-features = false, features = ["pem"] }
# Wipes the root token and the bootstrap grant token from memory once
# `swarmctl bao` has handed them on. Already in the lock through the TLS stack.
zeroize = "1"
# Test doubles with expectations (canned replies, call counts, argument # Test doubles with expectations (canned replies, call counts, argument
# matchers). Dev-dependency only, and used behind `#[cfg(test)]`, so release # matchers). Dev-dependency only, and used behind `#[cfg(test)]`, so release
# builds never compile it. See "Testing seams and test doubles" in # builds never compile it. See "Testing seams and test doubles" in

View file

@ -23,23 +23,36 @@ _On the host running `swarm-bao`._ Everything else fetches its credentials
from the store, so it comes first. from the store, so it comes first.
```bash ```bash
sudo bao operator init # keep the keys it prints and the root token OFF this host swarmctl bao init
``` ```
It prints the unseal (or, under the default `pkcs11` seal, recovery) keys
and the root token **once**, and stores none of them: not in a file, and the root token is
never in a process's argument list (so not in `ps`). Record them off this host before you do
anything else. It refuses an initialised store, and works
from any host that can reach the store and has `bao` on `PATH`; the
wrapper installed on the store host already knows the store's address and CA.
Then bootstrap the **granter**, the one principal that writes every Then bootstrap the **granter**, the one principal that writes every
`swarm-*` policy and role from then on. This is the only step that needs `swarm-*` policy and role from then on. This is the only step that needs
the root token: the root token. Run it as root on the host running `swarm-bao`:
```bash ```bash
sudo -i swarmctl bao grant-bootstrap # prompts for the root token, input hidden
read -rs BAO_TOKEN && export BAO_TOKEN # paste the root token from `bao operator init`
bao policy write bao-bootstrap /etc/hyperhive/bao-bootstrap-policy.hcl
bao token create -policy=bao-bootstrap -ttl=24h -orphan -display-name=bao-bootstrap -field=token \
| install -D -m 0600 /dev/stdin /var/lib/swarm-bao-bootstrap/grant.token
unset BAO_TOKEN
systemctl restart swarm-bao-granter-role
``` ```
It writes the `bao-bootstrap` policy, places a 24h grant token for
`swarm-bao-granter-role` and restarts that unit. The token comes from the
hidden prompt, or from stdin when stdin isn't a terminal; never from an
argument or environment variable. If the granter exists already, it
says so and does nothing. To do both steps in one go,
without re-typing the root token, run `swarmctl bao init --and-grant`
instead (it needs the `pkcs11` seal, which unseals itself).
If the host running `swarm-bao` has no `swarmctl` (the controller runs
elsewhere), the granting units print the equivalent `bao` commands when the
store refuses them.
Check `systemctl status swarm-bao-granter-role` logs Check `systemctl status swarm-bao-granter-role` logs
`Uploaded policy: bao-granter`, then restart the granting units that `Uploaded policy: bao-granter`, then restart the granting units that
failed while they waited. The granter unit deletes the token itself after a failed while they waited. The granter unit deletes the token itself after a

View file

@ -37,6 +37,9 @@ This document contains the help content for the `swarmctl` command-line program.
* [`swarmctl forge make-admin`↴](#swarmctl-forge-make-admin) * [`swarmctl forge make-admin`↴](#swarmctl-forge-make-admin)
* [`swarmctl setup`↴](#swarmctl-setup) * [`swarmctl setup`↴](#swarmctl-setup)
* [`swarmctl setup status`↴](#swarmctl-setup-status) * [`swarmctl setup status`↴](#swarmctl-setup-status)
* [`swarmctl bao`↴](#swarmctl-bao)
* [`swarmctl bao init`↴](#swarmctl-bao-init)
* [`swarmctl bao grant-bootstrap`↴](#swarmctl-bao-grant-bootstrap)
* [`swarmctl login`↴](#swarmctl-login) * [`swarmctl login`↴](#swarmctl-login)
* [`swarmctl completions`↴](#swarmctl-completions) * [`swarmctl completions`↴](#swarmctl-completions)
@ -53,6 +56,7 @@ swarm-level operator CLI
* `user` — Manage subjects in the swarm's SSO provider * `user` — Manage subjects in the swarm's SSO provider
* `forge` — Manage human accounts on the swarm's forge * `forge` — Manage human accounts on the swarm's forge
* `setup` — Show where the first-run setup ceremony stands * `setup` — Show where the first-run setup ceremony stands
* `bao` — Initialise the swarm's secret store and bootstrap its granter
* `login` — Log in to the swarm named by `--swarm` as your own SSO account * `login` — Log in to the swarm named by `--swarm` as your own SSO account
* `completions` — Generate a shell completion script for `swarmctl` and print it to stdout * `completions` — Generate a shell completion script for `swarmctl` and print it to stdout
@ -671,6 +675,56 @@ Exits 0 when every step it can see has finished, and 1 otherwise, including when
## `swarmctl bao`
Initialise the swarm's secret store and bootstrap its granter.
Runs the `bao` CLI against the store over HTTPS, so the store need not run on this host. The root token never reaches a file, an argument list or an environment variable of this process.
**Usage:** `swarmctl bao <COMMAND>`
###### **Subcommands:**
* `init` — Initialise the store, printing its keys and the root token once
* `grant-bootstrap` — Bootstrap the granter: the one-time step that lets `swarm-bao-granter-role` write every `swarm-*` policy and role
## `swarmctl bao init`
Initialise the store, printing its keys and the root token once.
Prints the unseal or recovery keys and the root token to stdout, once, and stores none of them: record them off this host before doing anything else. Refuses an initialised store.
**Usage:** `swarmctl bao init [OPTIONS]`
###### **Options:**
* `--and-grant` — Right after initialising, bootstrap the granter in this same process, with the root token held only in memory.
Waits for the store to unseal itself first, so the store must unseal itself (a `pkcs11` seal); with a `shamir` seal, unseal by hand and run `swarmctl bao grant-bootstrap`.
* `--bao-bin <PATH>` — `bao` binary to run. Defaults to `bao` on `PATH`, which on a host that runs the store is the wrapper that already carries the store's address, CA and client certificate
* `--bootstrap-token-file <PATH>` — Where to write the 24h bootstrap token (mode 0600) for `swarm-bao-granter-role` to pick up. Must be a path on the host running that unit
* `--bootstrap-policy <PATH>` — The `bao-bootstrap` policy file written to the store
## `swarmctl bao grant-bootstrap`
Bootstrap the granter: the one-time step that lets `swarm-bao-granter-role` write every `swarm-*` policy and role.
Reads the root token from a hidden prompt, or from stdin when stdin isn't a terminal; never from an argument or an environment variable. Writes the `bao-bootstrap` policy, places a 24h grant token for the granter unit and restarts it, so run it as root on the host running that unit. Says so and does nothing when the granter exists already.
**Usage:** `swarmctl bao grant-bootstrap [OPTIONS]`
###### **Options:**
* `--bao-bin <PATH>` — `bao` binary to run. Defaults to `bao` on `PATH`, which on a host that runs the store is the wrapper that already carries the store's address, CA and client certificate
* `--bootstrap-token-file <PATH>` — Where to write the 24h bootstrap token (mode 0600) for `swarm-bao-granter-role` to pick up. Must be a path on the host running that unit
* `--bootstrap-policy <PATH>` — The `bao-bootstrap` policy file written to the store
## `swarmctl login` ## `swarmctl login`
Log in to the swarm named by `--swarm` as your own SSO account. Log in to the swarm named by `--swarm` as your own SSO account.

View file

@ -47,6 +47,15 @@ in
SWARMCTL_AUTHELIA_USERS_FILE = deployCfg.authelia.hostUsersFile; SWARMCTL_AUTHELIA_USERS_FILE = deployCfg.authelia.hostUsersFile;
}; };
}) })
# The same wrapper, gated on the store running HERE: `bao grant-bootstrap`
# places a token and restarts a unit that exist only on that host.
(lib.mkIf (deployCfg.bao.enable && deployCfg.bao.bootstrapTokenFile != null) {
services.hyperhive.deploy.swarm-controller.swarmctlEnvironment = {
SWARMCTL_BAO_BOOTSTRAP_TOKEN_FILE = deployCfg.bao.bootstrapTokenFile;
SWARMCTL_BAO_BOOTSTRAP_POLICY = "${config.environment.etc."hyperhive/bao-bootstrap-policy.hcl".source
}";
};
})
{ {
systemd.services.swarm-controller.environment = { systemd.services.swarm-controller.environment = {
OTEL_EXPORTER_OTLP_ENDPOINT = "https://${otelSwarmCfg.domain}/${otelSwarmCfg.producerName}"; OTEL_EXPORTER_OTLP_ENDPOINT = "https://${otelSwarmCfg.domain}/${otelSwarmCfg.producerName}";

View file

@ -280,16 +280,22 @@ let
} }
{ {
# Same file: `swarmctl` is told where the configured authelia is only # Same file: `swarmctl` is told where the configured authelia is only
# where authelia runs here. # where authelia runs here, and where the bao bootstrap token and policy
name = "swarmctl is given the authelia binary and users file only where authelia runs here"; # go only where the store runs here. `controllerAlone` runs neither, so
# it is the control that an entry is absent when its service is.
name = "swarmctl is given the authelia and bao-bootstrap coordinates only where those run here";
ok = ok =
let let
env = m: m.services.hyperhive.deploy.swarm-controller.swarmctlEnvironment; env = m: m.services.hyperhive.deploy.swarm-controller.swarmctlEnvironment;
d = allLocal.services.hyperhive.deploy.authelia; d = allLocal.services.hyperhive.deploy.authelia;
b = allLocal.services.hyperhive.deploy.bao;
in in
env allLocal == { env allLocal == {
SWARMCTL_AUTHELIA_BIN = "${d.package}/bin/authelia"; SWARMCTL_AUTHELIA_BIN = "${d.package}/bin/authelia";
SWARMCTL_AUTHELIA_USERS_FILE = d.hostUsersFile; SWARMCTL_AUTHELIA_USERS_FILE = d.hostUsersFile;
SWARMCTL_BAO_BOOTSTRAP_TOKEN_FILE = b.bootstrapTokenFile;
SWARMCTL_BAO_BOOTSTRAP_POLICY = "${allLocal.environment.etc."hyperhive/bao-bootstrap-policy.hcl".source
}";
} }
&& env shared.controllerAlone == { }; && env shared.controllerAlone == { };
} }

View file

@ -40,6 +40,7 @@ swarm-controller-api.workspace = true
swarm-wire.workspace = true swarm-wire.workspace = true
tokio.workspace = true tokio.workspace = true
url.workspace = true url.workspace = true
zeroize.workspace = true
[lints] [lints]
workspace = true workspace = true

View file

@ -45,25 +45,42 @@ See `swarm-authelia-bridge/README.md` for what both writers must uphold.
The nix deployment sets every path, each derived from an option it already The nix deployment sets every path, each derived from an option it already
has: `swarm-controller.nix` sets the socket, and `glue-controller-env.nix` sets has: `swarm-controller.nix` sets the socket, and `glue-controller-env.nix` sets
`SWARMCTL_AUTHELIA_BIN` and `SWARMCTL_AUTHELIA_USERS_FILE`, only where `SWARMCTL_AUTHELIA_BIN` and `SWARMCTL_AUTHELIA_USERS_FILE`, only where
authelia runs on the same host. They are required authelia runs on the same host. It also sets
`SWARMCTL_BAO_BOOTSTRAP_TOKEN_FILE` and `SWARMCTL_BAO_BOOTSTRAP_POLICY`, only
where the secret store runs on the same host. They are required
rather than defaulted — a default would be an address we _hope_ points at rather than defaulted — a default would be an address we _hope_ points at
something, and one that resolves cleanly to the wrong place is worse than something, and one that resolves cleanly to the wrong place is worse than
an error. an error.
| variable | what | | variable | what |
| ------------------------------ | -------------------------------------------------------------------- | | ----------------------------------- | -------------------------------------------------------------------- |
| `SWARMCTL_AUTHELIA_BIN` | the **configured** authelia; argon2 params must match the verifier's | | `SWARMCTL_AUTHELIA_BIN` | the **configured** authelia; argon2 params must match the verifier's |
| `SWARMCTL_AUTHELIA_USERS_FILE` | host-side path of the users database | | `SWARMCTL_AUTHELIA_USERS_FILE` | host-side path of the users database |
| `SWARMCTL_AUTHELIA_MACHINE` | container name, for `systemctl -M` | | `SWARMCTL_AUTHELIA_MACHINE` | container name, for `systemctl -M` |
| `SWARMCTL_AUTHELIA_UNIT` | authelia's unit inside that container | | `SWARMCTL_AUTHELIA_UNIT` | authelia's unit inside that container |
| `SWARM_CONTROLLER_SOCKET` | the controller's unix socket, for `agent` and `forge` verbs | | `SWARMCTL_BAO_BOOTSTRAP_TOKEN_FILE` | where `bao grant-bootstrap` places the 24h grant token |
| `SWARMCTL_LOGIN_DEFAULT_SWARM` | `https://<swarm domain>` of this host's swarm, for `login` only | | `SWARMCTL_BAO_BOOTSTRAP_POLICY` | the `bao-bootstrap` policy file `bao grant-bootstrap` writes |
| `SWARMCTL_SWARM` | the swarm's domain, bare or `https://`: reach the controller there | | `SWARM_CONTROLLER_SOCKET` | the controller's unix socket, for `agent` and `forge` verbs |
| `SWARMCTL_CA_FILE` | a PEM CA to trust beside the system roots, for `SWARMCTL_SWARM` | | `SWARMCTL_LOGIN_DEFAULT_SWARM` | `https://<swarm domain>` of this host's swarm, for `login` only |
| `SWARMCTL_SWARM` | the swarm's domain, bare or `https://`: reach the controller there |
| `SWARMCTL_CA_FILE` | a PEM CA to trust beside the system roots, for `SWARMCTL_SWARM` |
Every verb and flag: [`docs/2-handbook/swarm/swarmctl-cli.md`](../docs/2-handbook/swarm/swarmctl-cli.md). Every verb and flag: [`docs/2-handbook/swarm/swarmctl-cli.md`](../docs/2-handbook/swarm/swarmctl-cli.md).
The sections below cover why each verb behaves as it does. The sections below cover why each verb behaves as it does.
## `bao init` and `bao grant-bootstrap`
These run the `bao` CLI, so they reach the store over HTTPS wherever it runs;
the CLI's own `BAO_ADDR`, `BAO_CACERT` and `BAO_CLIENT_*` say where. The root
token is handed to each `bao` child in its own `BAO_TOKEN` environment, which
is readable only by root through `/proc`; it is never in an argument list,
this process's environment or a file, and error text from `bao` has it
replaced before it is shown. `init` prints the keys and the token to stdout
once and keeps none; `grant-bootstrap` reads the token from a hidden prompt,
or from stdin when that is not a terminal. Both wipe the strings from memory
when done, on a best-effort basis: the JSON parser and the pipe buffers touch
the heap on the way.
## `user add` ## `user add`
```console ```console

740
swarmctl/src/bao.rs Normal file
View file

@ -0,0 +1,740 @@
//! `swarmctl bao` — initialise the swarm's secret store and bootstrap its
//! granter, so the operator never copies the root token between commands.
//!
//! Every call goes through the `bao` CLI over HTTPS, so the store need not
//! run on this host; the CLI's own `BAO_ADDR`, `BAO_CACERT` and
//! `BAO_CLIENT_*` variables say where it is. Only [`grant_bootstrap`]'s last
//! two steps (the token file and the unit restart) are local: they act on the
//! host that runs `swarm-bao-granter-role`.
//!
//! The root token is handled so that it exists in exactly three places: the
//! operator's terminal, this process's memory, and the `BAO_TOKEN`
//! environment of one `bao` child at a time. It is never in a child's
//! argument list (visible to every user through `ps`), never in this
//! process's own environment, never in a file, and never in an error
//! message. The child's environment is readable only by root through
//! `/proc/<pid>/environ`.
use std::io::{self, IsTerminal as _, Write as _};
use std::path::PathBuf;
use std::process::{Command, Output, Stdio};
use std::time::{Duration, Instant};
use anyhow::{Context, Result, bail};
use clap::Args;
use serde::Deserialize;
use zeroize::{Zeroize as _, Zeroizing};
use crate::path_from;
/// The policy the 24h bootstrap token carries, and the policy the granter
/// unit creates once it has used that token. Its presence is how a store is
/// known to be bootstrapped already.
const BOOTSTRAP_POLICY: &str = "bao-bootstrap";
const GRANTER_POLICY: &str = "bao-granter";
const GRANTER_UNIT: &str = "swarm-bao-granter-role";
/// How long `--and-grant` waits for a freshly initialised store to unseal
/// itself. A `pkcs11` seal unseals within seconds; a `shamir` seal never does.
const UNSEAL_WAIT: Duration = Duration::from_mins(1);
#[derive(Args)]
pub struct ToolArgs {
/// `bao` binary to run. Defaults to `bao` on `PATH`, which on a host that
/// runs the store is the wrapper that already carries the store's
/// address, CA and client certificate.
#[arg(long, value_name = "PATH")]
bao_bin: Option<PathBuf>,
}
#[derive(Args)]
pub struct GrantArgs {
/// Where to write the 24h bootstrap token (mode 0600) for
/// `swarm-bao-granter-role` to pick up. Must be a path on the host running
/// that unit.
#[arg(long, value_name = "PATH")]
bootstrap_token_file: Option<PathBuf>,
/// The `bao-bootstrap` policy file written to the store.
#[arg(long, value_name = "PATH")]
bootstrap_policy: Option<PathBuf>,
}
#[derive(Args)]
pub struct InitArgs {
/// Right after initialising, bootstrap the granter in this same process,
/// with the root token held only in memory.
///
/// Waits for the store to unseal itself first, so the store must unseal
/// itself (a `pkcs11` seal); with a `shamir` seal, unseal by hand and run
/// `swarmctl bao grant-bootstrap`.
#[arg(long)]
and_grant: bool,
#[command(flatten)]
tool: ToolArgs,
#[command(flatten)]
grant: GrantArgs,
}
#[derive(Args)]
pub struct GrantBootstrapArgs {
#[command(flatten)]
tool: ToolArgs,
#[command(flatten)]
grant: GrantArgs,
}
/// The programs this verb runs, as paths so that tests can substitute stubs.
struct Tools {
bao: PathBuf,
systemctl: PathBuf,
unseal_wait: Duration,
}
impl ToolArgs {
fn tools(self) -> Tools {
Tools {
bao: self.bao_bin.unwrap_or_else(|| PathBuf::from("bao")),
systemctl: PathBuf::from("systemctl"),
unseal_wait: UNSEAL_WAIT,
}
}
}
struct GrantPaths {
token_file: PathBuf,
policy_file: PathBuf,
}
impl GrantArgs {
fn resolve(self) -> Result<GrantPaths> {
Ok(GrantPaths {
token_file: path_from(
self.bootstrap_token_file,
"SWARMCTL_BAO_BOOTSTRAP_TOKEN_FILE",
)?,
policy_file: path_from(self.bootstrap_policy, "SWARMCTL_BAO_BOOTSTRAP_POLICY")?,
})
}
}
/// `swarmctl bao init`.
pub fn init(args: InitArgs) -> Result<()> {
let grant = args.and_grant.then(|| args.grant.resolve()).transpose()?;
init_with(&args.tool.tools(), grant.as_ref(), &mut io::stdout().lock())
}
fn init_with(tools: &Tools, grant: Option<&GrantPaths>, out: &mut impl io::Write) -> Result<()> {
let root_token = initialise(tools, out)?;
let Some(paths) = grant else {
eprintln!(
"The store is initialised. Next, `swarmctl bao grant-bootstrap` bootstraps the \
granter."
);
return Ok(());
};
// The keys are already on the operator's terminal, so a failure from here
// on must say the init itself succeeded.
let granted = wait_unsealed(tools)
.and_then(|()| grant_bootstrap_with(tools, paths, &root_token))
.context(
"the store is initialised and its keys and root token are printed above, but the \
granter bootstrap failed; once fixed, run `swarmctl bao grant-bootstrap`",
)?;
writeln!(out, "{}", granted.describe(paths)).context("writing the result to stdout")
}
/// `swarmctl bao grant-bootstrap`.
pub fn grant_bootstrap(args: GrantBootstrapArgs) -> Result<()> {
let paths = args.grant.resolve()?;
let tools = args.tool.tools();
let root_token = read_root_token()?;
let granted = grant_bootstrap_with(&tools, &paths, &root_token)?;
println!("{}", granted.describe(&paths));
Ok(())
}
/// What `bao operator init -format=json` prints, minus the fields not shown.
///
/// Wiped on drop. The JSON parser and the child's output buffer pass through
/// the heap on the way here, so this narrows how long the secrets sit in
/// memory; it cannot make it zero.
#[derive(Deserialize)]
struct InitOutput {
#[serde(default)]
unseal_keys_b64: Vec<String>,
#[serde(default)]
recovery_keys_b64: Vec<String>,
root_token: String,
}
impl Drop for InitOutput {
fn drop(&mut self) {
self.unseal_keys_b64.zeroize();
self.recovery_keys_b64.zeroize();
self.root_token.zeroize();
}
}
/// Initialises the store, writes the shares and the root token to `out`, and
/// returns the root token for the caller to use once and drop.
fn initialise(tools: &Tools, out: &mut impl io::Write) -> Result<Zeroizing<String>> {
let status = run(&mut tools.bao_cmd(&["operator", "init", "-status"]))?;
match status.status.code() {
Some(0) => bail!(
"the secret store is already initialised; its keys and root token were printed \
when it was, and are not stored anywhere to print again"
),
Some(2) => {}
_ => bail!(
"could not read the store's init status: {}",
failure(&status, &[])
),
}
let created = run(&mut tools.bao_cmd(&["operator", "init", "-format=json"]))?;
let raw = Zeroizing::new(created.stdout);
if !created.status.success() {
bail!(
"`bao operator init` failed: {}",
failure_text(&created.stderr, &[])
);
}
// The parser's own error can quote the value it choked on, so none of it
// is passed on.
let parsed: InitOutput = serde_json::from_slice(&raw).map_err(|_| {
anyhow::anyhow!("`bao operator init` printed output that is not the expected JSON")
})?;
if parsed.unseal_keys_b64.is_empty() && parsed.recovery_keys_b64.is_empty() {
bail!("`bao operator init` printed no unseal or recovery keys");
}
eprintln!(
"Record the keys and the root token below off this host now: swarmctl keeps none of \
them and cannot show them again."
);
let printed = (|| -> io::Result<()> {
for (i, key) in parsed.unseal_keys_b64.iter().enumerate() {
writeln!(out, "Unseal key {}: {key}", i + 1)?;
}
for (i, key) in parsed.recovery_keys_b64.iter().enumerate() {
writeln!(out, "Recovery key {}: {key}", i + 1)?;
}
writeln!(out, "Initial root token: {}", parsed.root_token)?;
out.flush()
})();
printed.context("writing the keys and root token to stdout")?;
Ok(Zeroizing::new(parsed.root_token.clone()))
}
/// Polls until `bao status` reports an unsealed store (exit 0).
fn wait_unsealed(tools: &Tools) -> Result<()> {
let deadline = Instant::now() + tools.unseal_wait;
loop {
let status = run(&mut tools.bao_cmd(&["status"]))?;
match status.status.code() {
Some(0) => return Ok(()),
Some(2) if Instant::now() < deadline => std::thread::sleep(Duration::from_secs(1)),
Some(2) => bail!(
"the store is still sealed after {}s; with a shamir seal, unseal it with \
`bao operator unseal` and run `swarmctl bao grant-bootstrap`",
tools.unseal_wait.as_secs()
),
_ => bail!(
"could not read the store's seal status: {}",
failure(&status, &[])
),
}
}
}
#[derive(Debug)]
enum Granted {
AlreadyDone,
Done,
}
impl Granted {
fn describe(&self, paths: &GrantPaths) -> String {
match self {
Self::AlreadyDone => format!(
"the granter is already bootstrapped (policy {GRANTER_POLICY} exists); nothing done"
),
Self::Done => format!(
"granter bootstrap started: grant token written to {}, {GRANTER_UNIT} restarted",
paths.token_file.display()
),
}
}
}
/// The S3 bootstrap of `docs/1-getting-started/setup.md`: write the
/// `bao-bootstrap` policy, mint the 24h grant token, place it for the granter
/// unit and restart that unit.
fn grant_bootstrap_with(tools: &Tools, paths: &GrantPaths, root: &str) -> Result<Granted> {
let policies = run(tools.bao_cmd(&["policy", "list"]).env("BAO_TOKEN", root))?;
ensure(&policies, "bao policy list", &[root])?;
if String::from_utf8_lossy(&policies.stdout)
.lines()
.any(|line| line.trim() == GRANTER_POLICY)
{
return Ok(Granted::AlreadyDone);
}
let write = run(tools
.bao_cmd(&["policy", "write", BOOTSTRAP_POLICY])
.arg(&paths.policy_file)
.env("BAO_TOKEN", root))?;
ensure(&write, "bao policy write", &[root])?;
let minted = run(tools
.bao_cmd(&[
"token",
"create",
&format!("-policy={BOOTSTRAP_POLICY}"),
"-ttl=24h",
"-orphan",
&format!("-display-name={BOOTSTRAP_POLICY}"),
"-field=token",
])
.env("BAO_TOKEN", root))?;
ensure(&minted, "bao token create", &[root])?;
let minted = Zeroizing::new(minted.stdout);
let grant_token = Zeroizing::new(String::from_utf8_lossy(&minted).trim().to_owned());
if grant_token.is_empty() {
bail!("`bao token create` printed no token");
}
let secrets = [root, grant_token.as_str()];
let contents = Zeroizing::new(format!("{}\n", *grant_token));
crate::write_atomic(&paths.token_file, &contents).with_context(|| {
format!(
"placing the bootstrap token at {}",
paths.token_file.display()
)
})?;
let restart = run(Command::new(&tools.systemctl).args(["restart", GRANTER_UNIT]))?;
ensure(&restart, "systemctl restart", &secrets)?;
Ok(Granted::Done)
}
impl Tools {
fn bao_cmd(&self, args: &[&str]) -> Command {
let mut cmd = Command::new(&self.bao);
cmd.args(args);
cmd
}
}
/// Runs `cmd` to completion with stdin closed.
///
/// `BAO_TOKEN` is dropped from the child's inherited environment unless the
/// caller sets it, so an ambient token in the operator's shell is never used
/// by accident. Spawn errors name the program and nothing else.
fn run(cmd: &mut Command) -> Result<Output> {
let program = cmd.get_program().to_string_lossy().into_owned();
if !cmd.get_envs().any(|(key, _)| key == "BAO_TOKEN") {
cmd.env_remove("BAO_TOKEN");
}
cmd.stdin(Stdio::null())
.output()
.with_context(|| format!("running {program}"))
}
/// Fails with the child's stderr, secrets replaced, when it did not succeed.
fn ensure(out: &Output, what: &str, secrets: &[&str]) -> Result<()> {
if out.status.success() {
return Ok(());
}
bail!("`{what}` failed: {}", failure(out, secrets))
}
fn failure(out: &Output, secrets: &[&str]) -> String {
format!("{} ({})", failure_text(&out.stderr, secrets), out.status)
}
/// A child's stderr as one trimmed line of text with every secret replaced.
fn failure_text(stderr: &[u8], secrets: &[&str]) -> String {
let mut text = String::from_utf8_lossy(stderr).trim().to_owned();
for secret in secrets.iter().filter(|s| !s.is_empty()) {
text = text.replace(secret, "[redacted]");
}
text
}
/// The root token: from a no-echo prompt on a terminal, otherwise the first
/// line of stdin. Never from argv or the environment.
fn read_root_token() -> Result<Zeroizing<String>> {
let stdin = io::stdin();
let token = if stdin.is_terminal() {
prompt_no_echo(&stdin)?
} else {
read_token_line(&mut stdin.lock())?
};
if token.is_empty() {
bail!("no root token was given");
}
Ok(token)
}
fn prompt_no_echo(stdin: &io::Stdin) -> Result<Zeroizing<String>> {
use rustix::termios::{LocalModes, OptionalActions, tcgetattr, tcsetattr};
let saved = tcgetattr(stdin).context("reading terminal settings")?;
let mut quiet = saved.clone();
quiet.local_modes.remove(LocalModes::ECHO);
tcsetattr(stdin, OptionalActions::Flush, &quiet).context("turning terminal echo off")?;
eprint!("Root token (input hidden): ");
io::stderr().flush().ok();
let read = read_token_line(&mut stdin.lock());
// Restored before the error, if any, is looked at.
let restored = tcsetattr(stdin, OptionalActions::Flush, &saved);
eprintln!();
restored.context("restoring terminal echo")?;
read
}
fn read_token_line(input: &mut impl io::BufRead) -> Result<Zeroizing<String>> {
let mut line = Zeroizing::new(String::new());
input
.read_line(&mut line)
.context("reading the root token")?;
Ok(Zeroizing::new(line.trim().to_owned()))
}
#[cfg(test)]
mod tests {
use std::fs;
use std::os::unix::fs::PermissionsExt as _;
use super::*;
use crate::tests::write_executable;
const ROOT: &str = "hvs.ROOTTOKEN";
const GRANT: &str = "hvs.GRANTTOKEN";
/// Stand-ins for `bao` and `systemctl` that log every call (argument list
/// and the `BAO_TOKEN` they were given) and answer from files in `dir`.
struct Fixture {
dir: PathBuf,
tools: Tools,
paths: GrantPaths,
}
const STUB: &str = r#"#!/bin/sh
D=$(dirname "$0")
echo "ARGV $(basename "$0") $*" >> "$D/log"
echo "ENV BAO_TOKEN=${BAO_TOKEN-}" >> "$D/log"
case "$(basename "$0") $1 $2 $3" in
"bao operator init -status") exit "$(cat "$D/init_status")" ;;
"bao operator init -format=json") cat "$D/init.json"; exit 0 ;;
"bao status "*) exit "$(cat "$D/seal_status")" ;;
"bao policy list"*)
if [ -e "$D/list_fails" ]; then echo "permission denied for token $BAO_TOKEN" >&2; exit 2; fi
cat "$D/policies"; exit 0 ;;
"bao policy write"*) exit 0 ;;
"bao token create"*) echo "hvs.GRANTTOKEN"; exit 0 ;;
"systemctl restart "*)
if [ -e "$D/restart_fails" ]; then echo "unit failed, token file: $(cat "$D/grant.token")" >&2; exit 5; fi
exit 0 ;;
esac
echo "unexpected call" >&2
exit 9
"#;
impl Fixture {
fn new(tag: &str) -> Self {
let dir =
std::env::temp_dir().join(format!("swarmctl-bao-{tag}-{}", std::process::id()));
let _ = fs::remove_dir_all(&dir);
fs::create_dir_all(&dir).expect("temp dir");
write_executable(&dir.join("bao"), STUB);
write_executable(&dir.join("systemctl"), STUB);
let fixture = Self {
tools: Tools {
bao: dir.join("bao"),
systemctl: dir.join("systemctl"),
unseal_wait: Duration::ZERO,
},
paths: GrantPaths {
token_file: dir.join("grant.token"),
policy_file: dir.join("policy.hcl"),
},
dir,
};
fixture.set("init_status", "2");
fixture.set("seal_status", "0");
fixture.set("policies", "default\nroot\n");
fixture.set(
"init.json",
r#"{"unseal_keys_b64":["UK1","UK2"],"recovery_keys_b64":["RK1"],"root_token":"hvs.ROOTTOKEN"}"#,
);
fixture
}
fn set(&self, name: &str, body: &str) {
fs::write(self.dir.join(name), body).expect("write fixture file");
}
fn log(&self) -> String {
fs::read_to_string(self.dir.join("log")).unwrap_or_default()
}
/// The recorded argument lists, one per call, in order.
fn calls(&self) -> Vec<String> {
self.log()
.lines()
.filter_map(|l| l.strip_prefix("ARGV ").map(str::to_owned))
.collect()
}
}
impl Drop for Fixture {
fn drop(&mut self) {
fs::remove_dir_all(&self.dir).ok();
}
}
fn count(haystack: &str, needle: &str) -> usize {
haystack.matches(needle).count()
}
#[test]
fn init_prints_the_keys_and_the_root_token_once() {
let f = Fixture::new("init-once");
let mut out = Vec::new();
let token = initialise(&f.tools, &mut out).expect("init succeeds");
let out = String::from_utf8(out).expect("utf8");
assert_eq!(&*token, ROOT);
for line in [
"Unseal key 1: UK1",
"Unseal key 2: UK2",
"Recovery key 1: RK1",
"Initial root token: hvs.ROOTTOKEN",
] {
assert_eq!(count(&out, line), 1, "{line} in {out:?}");
}
assert_eq!(count(&out, ROOT), 1, "the token appears exactly once");
// Presence control: the log does record what init was asked.
assert_eq!(
f.calls(),
[
"bao operator init -status",
"bao operator init -format=json"
]
);
}
#[test]
fn init_hands_bao_no_token_in_argv_or_environment() {
let f = Fixture::new("init-argv");
initialise(&f.tools, &mut Vec::new()).expect("init succeeds");
let log = f.log();
assert!(!log.contains(ROOT), "{log}");
assert!(
log.contains("ENV BAO_TOKEN=\n"),
"control: ENV lines are logged: {log}"
);
}
#[test]
fn init_refuses_an_initialised_store_without_initialising() {
let f = Fixture::new("init-refuse");
f.set("init_status", "0");
let mut out = Vec::new();
let err = initialise(&f.tools, &mut out).expect_err("must refuse");
assert!(
format!("{err:#}").contains("already initialised"),
"{err:#}"
);
assert!(out.is_empty(), "nothing is printed");
assert!(
!f.log().contains("-format=json"),
"init was not attempted: {}",
f.log()
);
}
#[test]
fn init_reports_an_unreadable_status_instead_of_initialising() {
let f = Fixture::new("init-status-err");
f.set("init_status", "1");
let err = initialise(&f.tools, &mut Vec::new()).expect_err("must stop");
assert!(format!("{err:#}").contains("init status"), "{err:#}");
assert!(!f.log().contains("-format=json"), "{}", f.log());
}
#[test]
fn a_malformed_init_reply_does_not_leak_its_contents() {
let f = Fixture::new("init-garbled");
f.set(
"init.json",
r#"{"unseal_keys_b64":"hvs.LEAKED","root_token":"x"}"#,
);
// Control: the parser's own error does quote the value.
let raw = fs::read(f.dir.join("init.json")).expect("read");
let parser = serde_json::from_slice::<InitOutput>(&raw)
.err()
.expect("rejects");
assert!(parser.to_string().contains("hvs.LEAKED"), "{parser}");
let err = initialise(&f.tools, &mut Vec::new()).expect_err("must fail");
assert!(!format!("{err:#}").contains("hvs.LEAKED"), "{err:#}");
}
#[test]
fn grant_runs_the_bootstrap_steps_in_order_with_no_token_in_any_argv() {
let f = Fixture::new("grant-order");
let granted = grant_bootstrap_with(&f.tools, &f.paths, ROOT).expect("grant succeeds");
assert!(matches!(granted, Granted::Done));
let policy = f.paths.policy_file.display();
assert_eq!(
f.calls(),
[
"bao policy list".to_owned(),
format!("bao policy write bao-bootstrap {policy}"),
"bao token create -policy=bao-bootstrap -ttl=24h -orphan \
-display-name=bao-bootstrap -field=token"
.to_owned(),
"systemctl restart swarm-bao-granter-role".to_owned(),
]
);
let log = f.log();
for argv in f.calls() {
assert!(!argv.contains(ROOT) && !argv.contains(GRANT), "{argv}");
}
// Presence control: the root token did reach the three bao calls, by
// the environment of the child alone.
assert_eq!(count(&log, &format!("ENV BAO_TOKEN={ROOT}\n")), 3, "{log}");
let mode = fs::metadata(&f.paths.token_file)
.expect("stat")
.permissions()
.mode()
& 0o7777;
assert_eq!(mode, 0o600);
assert_eq!(
fs::read_to_string(&f.paths.token_file).expect("read"),
format!("{GRANT}\n")
);
}
#[test]
fn grant_on_a_bootstrapped_store_changes_nothing() {
let f = Fixture::new("grant-again");
f.set("policies", "bao-granter\ndefault\nroot\n");
let granted = grant_bootstrap_with(&f.tools, &f.paths, ROOT).expect("exits ok");
assert!(matches!(granted, Granted::AlreadyDone));
assert_eq!(f.calls(), ["bao policy list"]);
assert!(!f.paths.token_file.exists());
assert!(granted.describe(&f.paths).contains("already bootstrapped"));
}
#[test]
fn a_failing_bao_call_does_not_put_the_root_token_in_the_error() {
let f = Fixture::new("grant-fail-bao");
f.set("list_fails", "");
let err = grant_bootstrap_with(&f.tools, &f.paths, ROOT).expect_err("must fail");
let text = format!("{err:#}");
// The stub echoed the token on stderr; only the redaction keeps it out.
assert!(text.contains("permission denied"), "{text}");
assert!(text.contains("[redacted]"), "{text}");
assert!(!text.contains(ROOT), "{text}");
}
#[test]
fn a_failing_restart_does_not_put_either_token_in_the_error() {
let f = Fixture::new("grant-fail-restart");
f.set("restart_fails", "");
let err = grant_bootstrap_with(&f.tools, &f.paths, ROOT).expect_err("must fail");
let text = format!("{err:#}");
// The stub printed the grant token's file contents on stderr.
assert!(text.contains("unit failed"), "{text}");
assert!(text.contains("[redacted]"), "{text}");
assert!(!text.contains(GRANT) && !text.contains(ROOT), "{text}");
}
#[test]
fn and_grant_initialises_then_bootstraps_in_one_process() {
let f = Fixture::new("and-grant");
let mut out = Vec::new();
init_with(&f.tools, Some(&f.paths), &mut out).expect("succeeds");
let out = String::from_utf8(out).expect("utf8");
assert_eq!(count(&out, ROOT), 1, "{out}");
assert!(out.contains("granter bootstrap started"), "{out}");
let calls = f.calls();
assert_eq!(
calls[..3],
[
"bao operator init -status",
"bao operator init -format=json",
"bao status"
]
);
assert_eq!(calls[3], "bao policy list");
assert_eq!(
calls.last().map(String::as_str),
Some("systemctl restart swarm-bao-granter-role")
);
assert!(calls.iter().all(|c| !c.contains(ROOT)), "{calls:?}");
}
#[test]
fn init_without_and_grant_never_touches_the_granter() {
let f = Fixture::new("init-only");
init_with(&f.tools, None, &mut Vec::new()).expect("succeeds");
assert_eq!(f.calls().len(), 2, "{:?}", f.calls());
}
#[test]
fn a_sealed_store_after_init_keeps_the_printed_token_and_leaks_nothing() {
let f = Fixture::new("and-grant-sealed");
f.set("seal_status", "2");
let mut out = Vec::new();
let err = init_with(&f.tools, Some(&f.paths), &mut out).expect_err("sealed store");
let text = format!("{err:#}");
assert!(text.contains("printed above"), "{text}");
assert!(text.contains("still sealed"), "{text}");
assert!(!text.contains(ROOT), "{text}");
assert_eq!(count(&String::from_utf8(out).expect("utf8"), ROOT), 1);
assert!(
!f.calls().iter().any(|c| c.starts_with("bao policy")),
"no grant attempted"
);
}
#[test]
fn a_seal_status_error_is_not_read_as_sealed() {
let f = Fixture::new("seal-error");
f.set("seal_status", "1");
let err = wait_unsealed(&f.tools).expect_err("must fail");
assert!(format!("{err:#}").contains("seal status"), "{err:#}");
}
#[test]
fn a_missing_bao_binary_names_the_binary() {
let f = Fixture::new("no-bao");
let tools = Tools {
bao: f.dir.join("absent"),
systemctl: f.tools.systemctl.clone(),
unseal_wait: Duration::ZERO,
};
let err = grant_bootstrap_with(&tools, &f.paths, ROOT).expect_err("must fail");
let text = format!("{err:#}");
assert!(text.contains("absent"), "{text}");
assert!(!text.contains(ROOT), "{text}");
}
#[test]
fn the_root_token_is_the_first_line_of_a_piped_stdin() {
let token = read_token_line(&mut "hvs.ROOTTOKEN\nsecond\n".as_bytes()).expect("reads");
assert_eq!(&*token, ROOT);
let token = read_token_line(&mut " hvs.ROOTTOKEN ".as_bytes()).expect("reads");
assert_eq!(&*token, ROOT);
let token = read_token_line(&mut "".as_bytes()).expect("reads");
assert!(token.is_empty());
}
}

View file

@ -19,7 +19,8 @@
//! work they ask for is only the controller's to do (see [`agent`]). They //! work they ask for is only the controller's to do (see [`agent`]). They
//! reach it over its unix socket on its host, or, with `--swarm`, through //! reach it over its unix socket on its host, or, with `--swarm`, through
//! the swarm domain's gateway as the operator's own SSO account //! the swarm domain's gateway as the operator's own SSO account
//! ([`remote`]). The `user` verbs act on the local users file only. //! ([`remote`]). The `user` verbs act on the local users file only; the `bao`
//! verbs run the `bao` CLI against the secret store.
//! //!
//! Distinct from `hivectl`, which drives one hive's `hive-c0re` over its //! Distinct from `hivectl`, which drives one hive's `hive-c0re` over its
//! admin socket. This crate deliberately does not link `swarm-controller`, //! admin socket. This crate deliberately does not link `swarm-controller`,
@ -27,6 +28,7 @@
mod agent; mod agent;
mod agents_md; mod agents_md;
mod bao;
mod forge; mod forge;
mod inference; mod inference;
mod login; mod login;
@ -220,6 +222,15 @@ enum Verb {
#[command(subcommand)] #[command(subcommand)]
command: SetupVerb, command: SetupVerb,
}, },
/// Initialise the swarm's secret store and bootstrap its granter.
///
/// Runs the `bao` CLI against the store over HTTPS, so the store need
/// not run on this host. The root token never reaches a file, an
/// argument list or an environment variable of this process.
Bao {
#[command(subcommand)]
command: BaoVerb,
},
/// Log in to the swarm named by `--swarm` as your own SSO account. /// Log in to the swarm named by `--swarm` as your own SSO account.
/// ///
/// With no `--swarm` or `SWARMCTL_SWARM`, logs in to /// With no `--swarm` or `SWARMCTL_SWARM`, logs in to
@ -286,6 +297,25 @@ struct SetupStatusArgs {
socket: ControllerSocketArg, socket: ControllerSocketArg,
} }
#[derive(Subcommand)]
enum BaoVerb {
/// Initialise the store, printing its keys and the root token once.
///
/// Prints the unseal or recovery keys and the root token to stdout, once,
/// and stores none of them: record them off this host before doing
/// anything else. Refuses an initialised store.
Init(bao::InitArgs),
/// Bootstrap the granter: the one-time step that lets `swarm-bao-granter-role`
/// write every `swarm-*` policy and role.
///
/// Reads the root token from a hidden prompt, or from stdin when stdin isn't
/// a terminal; never from an argument or an environment variable. Writes
/// the `bao-bootstrap` policy, places a 24h grant token for the granter unit
/// and restarts it, so run it as root on the host running that unit. Says so
/// and does nothing when the granter exists already.
GrantBootstrap(bao::GrantBootstrapArgs),
}
#[derive(Subcommand)] #[derive(Subcommand)]
enum AgentVerb { enum AgentVerb {
/// Queue creation of a new agent on a hive in this swarm. /// Queue creation of a new agent on a hive in this swarm.
@ -798,6 +828,13 @@ fn main() -> Result<()> {
let target = remote.target(args.controller_socket)?; let target = remote.target(args.controller_socket)?;
forge::make_admin(&target, &args.name) forge::make_admin(&target, &args.name)
} }
// Acts through the `bao` CLI only: none of the `SWARMCTL_AUTHELIA_*` values apply.
Verb::Bao {
command: BaoVerb::Init(args),
} => bao::init(args),
Verb::Bao {
command: BaoVerb::GrantBootstrap(args),
} => bao::grant_bootstrap(args),
// Same socket-resolution reasoning as `Create` above. // Same socket-resolution reasoning as `Create` above.
Verb::Schedule { command } => schedule(&remote, command), Verb::Schedule { command } => schedule(&remote, command),
// Neither the controller socket nor the users file is required: a // Neither the controller socket nor the users file is required: a
@ -1718,7 +1755,7 @@ mod tests {
/// rather than from this one: a write fd on it held here could be inherited /// rather than from this one: a write fd on it held here could be inherited
/// by a child another test thread forks, and exec'ing a file some process /// by a child another test thread forks, and exec'ing a file some process
/// still holds open for writing fails with `ETXTBSY`. /// still holds open for writing fails with `ETXTBSY`.
fn write_executable(path: &std::path::Path, body: &str) { pub(crate) fn write_executable(path: &std::path::Path, body: &str) {
let status = std::process::Command::new("/bin/sh") let status = std::process::Command::new("/bin/sh")
.args([ .args([
"-c", "-c",

View file

@ -111,7 +111,7 @@ pub(crate) struct Observed {
const STEPS: [(&str, &str, bool); 10] = [ const STEPS: [(&str, &str, bool); 10] = [
( (
"secret store initialised", "secret store initialised",
"setup.md § 1: `sudo bao operator init`, on the host running swarm-bao", "setup.md § 1: `swarmctl bao init`",
false, false,
), ),
( (
@ -121,7 +121,7 @@ const STEPS: [(&str, &str, bool); 10] = [
), ),
( (
"granter bootstrapped", "granter bootstrapped",
"setup.md § 1: the bao-bootstrap block, then `systemctl restart swarm-bao-granter-role`", "setup.md § 1: `swarmctl bao grant-bootstrap`, on the host running swarm-bao",
false, false,
), ),
( (