hives read the forge through their own swarm-minted read account
swarm-controller now gives every hive in its directory a forge account, `hive-<hive>`, in a new `hives` team of `agent-configs` (read on `repo.code` + `repo.pulls`, every repo including later ones), and mints one `read:repository` token named `swarm-hive` into the store at `swarm/hives/<hive>/forge/token`. A five-minute pass queues a `MintHiveForgeToken` node for a hive whose token is missing or no longer matches the forge, whose account is missing, or whose team membership or team settings drifted. A hive whose store or forge read failed is never queued, so an outage cannot rotate every hive's token. The token decision reuses `agent_token`'s classifier and replace step, now parameterised by name and scopes. An agent may be called `hive-<x>`. The hive account is created with `hive-<hive>@hives.hyperhive.local` and the job refuses an account by that name with any other email before it locks it down, adds it to the team or mints for it; agent creation already refuses the hive's account by the same email check. The controller's bao policy gains the `swarm/hives/+/forge/token` leaf (create/read/update), pinned by the bao-grants module-eval assertion. The hive's own policy already reads `swarm/hives/<hive>/*`. hive-c0re reads that token through `forge::reader_token()`, falling back to the core token file when the store holds none, and uses it for: fetching a merged config rev (swarm deploy, `hivectl forge reconcile-config`), reseeding a purged `applied`, the meta-update cascade's fast-forward, seeding `proposed` on first deploy, and finding the PR that merged a refused rev. Those reads no longer require a local `hive-forge` container. The `/knowledge` clone moves from the forge host's boot sweep into the knowledge pull, so every hive clones it. Posting the deploy-failure comment on that PR stays on the core token. Refs #4851
This commit is contained in:
parent
c8472966e9
commit
21e3f626f4
17 changed files with 1222 additions and 138 deletions
|
|
@ -15,7 +15,7 @@ each agent on relevant activity.
|
||||||
|
|
||||||
## Token scopes
|
## Token scopes
|
||||||
|
|
||||||
Two scope sets:
|
Three scope sets:
|
||||||
|
|
||||||
**`AGENT_TOKEN_SCOPES`** (swarm-controller's `forge::agent_token`, pinned
|
**`AGENT_TOKEN_SCOPES`** (swarm-controller's `forge::agent_token`, pinned
|
||||||
by a test). swarm-controller mints every agent token with it:
|
by a test). swarm-controller mints every agent token with it:
|
||||||
|
|
@ -40,6 +40,53 @@ hive-c0re only mints the token when the file at
|
||||||
a token minted with admin scopes keeps them until an operator deletes that
|
a token minted with admin scopes keeps them until an operator deletes that
|
||||||
file and restarts hive-c0re.
|
file and restarts hive-c0re.
|
||||||
|
|
||||||
|
**`HIVE_TOKEN_SCOPES`** (swarm-controller's `forge::hive_token`, pinned by a
|
||||||
|
test): `read:repository` alone, for each hive's read-only account (next
|
||||||
|
section).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## How a hive reads the forge
|
||||||
|
|
||||||
|
Each hive in swarm-controller's hive directory has its own forge account,
|
||||||
|
`hive-<hive>`. swarm-controller creates it, makes it a member of the `hives`
|
||||||
|
team in `agent-configs`, and mints one token named `swarm-hive` into the
|
||||||
|
swarm secret store at `swarm/hives/<hive>/forge/token`. The team has read
|
||||||
|
permission on `repo.code` and `repo.pulls` of every repo in the org,
|
||||||
|
including repos created later. A pass at start and every five minutes
|
||||||
|
creates what's missing and re-mints a token that's missing or no longer
|
||||||
|
matches the forge. A pass whose store or forge reads fail queues nothing, so
|
||||||
|
an outage never rotates every hive's token.
|
||||||
|
|
||||||
|
The account name isn't reserved, so an agent may carry the name `hive-<x>` too.
|
||||||
|
swarm-controller creates the hive's account with the email
|
||||||
|
`hive-<hive>@hives.hyperhive.local` and refuses an account by that name with
|
||||||
|
any other email: it never adds an agent's account to the team or mints a
|
||||||
|
token for it. An agent's own creation refuses the hive's account the same
|
||||||
|
way.
|
||||||
|
|
||||||
|
hive-c0re reads the token with `forge::reader_token()`, under the hive's own
|
||||||
|
store certificate, every time it reads the forge. Where the store holds none,
|
||||||
|
it falls back to the `core` token file, which only the host running the forge
|
||||||
|
container has. These reads go through it, and none of them needs a local
|
||||||
|
forge container:
|
||||||
|
|
||||||
|
- fetching a merged config commit for a swarm deploy, and
|
||||||
|
`hivectl forge reconcile-config`
|
||||||
|
- reseeding a purged `applied` repo, and the meta-update cascade's
|
||||||
|
fast-forward of `applied/main`
|
||||||
|
- seeding `proposed` from `agent-configs/<agent>` on first deploy
|
||||||
|
- finding the PR that merged a config commit the hive refused to deploy
|
||||||
|
- cloning and pulling `/knowledge`
|
||||||
|
|
||||||
|
Posting the refusal as a comment on that PR still needs the `core` token, so
|
||||||
|
only the forge host posts it.
|
||||||
|
|
||||||
|
Roll a change to any of this out store host first (its granter writes the
|
||||||
|
controller's grant on `swarm/hives/+/forge/token`), then the controller host,
|
||||||
|
then the hives. A hive that starts before its token exists reads with the
|
||||||
|
core file, or not at all, until the controller's next pass.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Per-agent forge accounts
|
## Per-agent forge accounts
|
||||||
|
|
|
||||||
|
|
@ -87,10 +87,12 @@ message only, no forced wake, and it carries the standard "this was a
|
||||||
broadcast" hint. hive-c0re logs a diff or per-agent send failure
|
broadcast" hint. hive-c0re logs a diff or per-agent send failure
|
||||||
without blocking the pull itself.
|
without blocking the pull itself.
|
||||||
|
|
||||||
`knowledge::ensure_local_clone` creates (or refreshes) the local
|
On a hive with no local clone yet, `knowledge::pull()` takes the clone
|
||||||
clone once at startup. If the repo is brand new and
|
instead of pulling, so every hive gets `/knowledge` from its first pull,
|
||||||
empty, `ensure_local_clone` seeds it with the default README
|
whether or not it runs the forge. It authenticates with the hive's forge
|
||||||
before returning.
|
read token when it has one (see
|
||||||
|
[forge.md](forge.md#how-a-hive-reads-the-forge)) and clones the public
|
||||||
|
repo anonymously otherwise.
|
||||||
|
|
||||||
## State
|
## State
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -67,6 +67,7 @@ of the cell says how.
|
||||||
| `swarm/agents/<agent>/forge/<label>` | `swarm-controller`, when an operator links an external forge account in the swarm UI | `hive-agent-forge-accounts` in the agent container, under the agent's own certificate, into `<state>/forge-<label>-token` and `forge-<label>.json` | ❌ an operator's token; replaced only by deleting it and linking the label again | ✅ the agent re-fetches on a 2-minute timer |
|
| `swarm/agents/<agent>/forge/<label>` | `swarm-controller`, when an operator links an external forge account in the swarm UI | `hive-agent-forge-accounts` in the agent container, under the agent's own certificate, into `<state>/forge-<label>-token` and `forge-<label>.json` | ❌ an operator's token; replaced only by deleting it and linking the label again | ✅ the agent re-fetches on a 2-minute timer |
|
||||||
| `swarm/agents/<agent>/github-token` | `swarm-controller`, when an operator links a GitHub account in the swarm UI | `hive-agent-github-token` in the agent container, under the agent's own certificate, into `<state>/github-token` | ❌ an operator's token; replaced only by deleting it and linking it again | ✅ the agent re-fetches on a 2-minute timer |
|
| `swarm/agents/<agent>/github-token` | `swarm-controller`, when an operator links a GitHub account in the swarm UI | `hive-agent-github-token` in the agent container, under the agent's own certificate, into `<state>/github-token` | ❌ an operator's token; replaced only by deleting it and linking it again | ✅ the agent re-fetches on a 2-minute timer |
|
||||||
| `swarm/hives/<hive>/matrix/sender-token` | `swarm-controller`, with the swarm's appservice token, for every hive in its directory in a five-minute pass | `swarm-controller` under its own certificate, before it decides whether to mint, and hive-c0re's `stored_sender_token()`, under the hive's own certificate | ✅ the controller's pass re-mints when the stored token is missing, unknown to the homeserver, or someone else's | ✅ hive-c0re's matrix sweep reads the store every run and overwrites its token file when the store's token differs |
|
| `swarm/hives/<hive>/matrix/sender-token` | `swarm-controller`, with the swarm's appservice token, for every hive in its directory in a five-minute pass | `swarm-controller` under its own certificate, before it decides whether to mint, and hive-c0re's `stored_sender_token()`, under the hive's own certificate | ✅ the controller's pass re-mints when the stored token is missing, unknown to the homeserver, or someone else's | ✅ hive-c0re's matrix sweep reads the store every run and overwrites its token file when the store's token differs |
|
||||||
|
| `swarm/hives/<hive>/forge/token` | `swarm-controller`, with the forge's admin API, for every hive in its directory in a five-minute pass | `swarm-controller` under its own certificate, before it decides whether to mint, and hive-c0re's `forge::reader_token()`, under the hive's own certificate | ✅ the controller's pass re-mints when the stored token is missing or no longer matches the forge (last eight characters and scopes) | ✅ hive-c0re reads the store on every forge read |
|
||||||
| `swarm/hives/<hive>/queue/agent` | authelia | `swarm-bao-queue-agent` on the hive's host, under its own per-hive certificate; no agent's policy reaches it | must be stated | must be stated |
|
| `swarm/hives/<hive>/queue/agent` | authelia | `swarm-bao-queue-agent` on the hive's host, under its own per-hive certificate; no agent's policy reaches it | must be stated | must be stated |
|
||||||
| `swarm/services/<clientId>/oidc/client` | authelia | that service's own reader, under its own certificate — the queue responder's under `swarm-nats-auth`'s, the homeserver's under `swarm-matrix-ctl`'s; the bao UI's by the store's granter | must be stated | must be stated |
|
| `swarm/services/<clientId>/oidc/client` | authelia | that service's own reader, under its own certificate — the queue responder's under `swarm-nats-auth`'s, the homeserver's under `swarm-matrix-ctl`'s; the bao UI's by the store's granter | must be stated | must be stated |
|
||||||
| `swarm/services/ci-runner/registration-token` | `swarm-controller`, from the forge's admin API, at start and in a five-minute pass; its grant on the path is write-only | `swarm-bao-ci-runner` on the runner's host, under the runner's own certificate, into a file only `swarm-ci-runner-register` in the runner's container loads; no other principal's policy reads it | ❌ the forge hands out one token until an admin resets it; the controller writes a reset token within five minutes | ❌ `swarm-bao-ci-runner` reads it when it starts, at boot or deploy, and restarts the runner when the value changed |
|
| `swarm/services/ci-runner/registration-token` | `swarm-controller`, from the forge's admin API, at start and in a five-minute pass; its grant on the path is write-only | `swarm-bao-ci-runner` on the runner's host, under the runner's own certificate, into a file only `swarm-ci-runner-register` in the runner's container loads; no other principal's policy reads it | ❌ the forge hands out one token until an admin resets it; the controller writes a reset token within five minutes | ❌ `swarm-bao-ci-runner` reads it when it starts, at boot or deploy, and restarts the runner when the value changed |
|
||||||
|
|
|
||||||
|
|
@ -1,15 +1,18 @@
|
||||||
//! Optional Forgejo wiring — the `core/meta` repo, meta read-access
|
//! Optional Forgejo wiring — the `core/meta` repo, meta read-access
|
||||||
//! grants, and each agent's read-only grant on `internal/docs` (the
|
//! grants, and each agent's read-only grant on `internal/docs` (the
|
||||||
//! swarm-controller creates that repo, with the other swarm-wide orgs,
|
//! swarm-controller creates that repo, with the other swarm-wide orgs,
|
||||||
//! teams and repos).
|
//! teams and repos). Those writes, as `core`, run only beside a local
|
||||||
//! No-op when `hive-forge` isn't running. Full design: `docs/integrations/forge.md`.
|
//! `hive-forge` container; reads of the forge go through [`reader_token`] on
|
||||||
|
//! every hive. Full design: `docs/integrations/forge.md`.
|
||||||
|
|
||||||
mod pr_comment;
|
mod pr_comment;
|
||||||
|
mod reader;
|
||||||
mod reconcile;
|
mod reconcile;
|
||||||
mod repos;
|
mod repos;
|
||||||
mod users;
|
mod users;
|
||||||
|
|
||||||
pub use pr_comment::{config_repo, merged_pr_for_commit, post_pr_comment};
|
pub use pr_comment::{config_repo, merged_pr_for_commit, post_pr_comment};
|
||||||
|
pub use reader::reader_token;
|
||||||
pub use reconcile::{fetch_forge_main, reconcile_config_apply, reconcile_config_status};
|
pub use reconcile::{fetch_forge_main, reconcile_config_apply, reconcile_config_status};
|
||||||
pub use repos::{
|
pub use repos::{
|
||||||
clone_config_into_proposed, ensure_meta_remote, ensure_repo, fast_forward_applied_main,
|
clone_config_into_proposed, ensure_meta_remote, ensure_repo, fast_forward_applied_main,
|
||||||
|
|
@ -71,14 +74,20 @@ fn git_url_with_base(base: &str, repo: &str) -> String {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The `http.extraHeader` value authenticating as the forge core user.
|
/// The `http.extraHeader` value authenticating as the forge core user.
|
||||||
|
pub(crate) fn core_auth_header(token: &str) -> String {
|
||||||
|
basic_auth_header("core", token)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The `http.extraHeader` value carrying `token`. Forgejo takes the account
|
||||||
|
/// from the token, so `login` only labels the request.
|
||||||
///
|
///
|
||||||
/// Basic auth over a header rather than userinfo in the URL, so the secret
|
/// Basic auth over a header rather than userinfo in the URL, so the secret
|
||||||
/// reaches git through the environment (see [`forge_git_url`]). Pair with
|
/// reaches git through the environment (see [`forge_git_url`]). Pair with
|
||||||
/// [`crate::lifecycle::git_command_authed`], which is the only thing that
|
/// [`crate::lifecycle::git_command_authed`], which is the only thing that
|
||||||
/// should ever hold the result.
|
/// should ever hold the result.
|
||||||
pub(crate) fn core_auth_header(token: &str) -> String {
|
fn basic_auth_header(login: &str, token: &str) -> String {
|
||||||
use base64::Engine as _;
|
use base64::Engine as _;
|
||||||
let basic = base64::engine::general_purpose::STANDARD.encode(format!("core:{token}"));
|
let basic = base64::engine::general_purpose::STANDARD.encode(format!("{login}:{token}"));
|
||||||
format!("Authorization: Basic {basic}")
|
format!("Authorization: Basic {basic}")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -223,8 +232,7 @@ pub async fn sync_agent(name: &str, core_token: Option<&str>) -> bool {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The `core_token.is_some()` half of [`ensure_all`]: the meta repo, the
|
/// The `core_token.is_some()` half of [`ensure_all`]: the meta repo, the
|
||||||
/// local knowledge clone, the core avatar, and removing this hive's own
|
/// core avatar, and removing this hive's own config-PR hook —
|
||||||
/// config-PR hook —
|
|
||||||
/// every step that needs an authenticated forge client. The swarm-wide
|
/// every step that needs an authenticated forge client. The swarm-wide
|
||||||
/// objects (orgs, the `operators` team, mirrors, `internal/docs`,
|
/// objects (orgs, the `operators` team, mirrors, `internal/docs`,
|
||||||
/// `internal/knowledge`, the `agent-configs` avatar) are the
|
/// `internal/knowledge`, the `agent-configs` avatar) are the
|
||||||
|
|
@ -243,16 +251,6 @@ async fn ensure_all_orgs_and_repos(token: &str) {
|
||||||
format!("forge: core/meta repo provisioning failed: {e}"),
|
format!("forge: core/meta repo provisioning failed: {e}"),
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
// Clone knowledge repo locally so it can be bind-mounted into agents.
|
|
||||||
// The swarm-controller creates and seeds the repo itself.
|
|
||||||
if let Err(e) = crate::knowledge::ensure_local_clone(token).await {
|
|
||||||
tracing::warn!(error = ?e, "knowledge: ensure_local_clone failed");
|
|
||||||
crate::warnings::set_boot_warning(
|
|
||||||
"forge_knowledge_local_clone",
|
|
||||||
"crit",
|
|
||||||
format!("knowledge: local clone failed: {e}"),
|
|
||||||
);
|
|
||||||
}
|
|
||||||
if let Err(e) = ensure_core_avatar(token).await {
|
if let Err(e) = ensure_core_avatar(token).await {
|
||||||
tracing::warn!(error = ?e, "forge: ensure_core_avatar failed");
|
tracing::warn!(error = ?e, "forge: ensure_core_avatar failed");
|
||||||
crate::warnings::set_boot_warning(
|
crate::warnings::set_boot_warning(
|
||||||
|
|
|
||||||
|
|
@ -1,10 +1,10 @@
|
||||||
//! Commenting on an agent's config PRs as the core forge user — how a hive
|
//! Commenting on an agent's config PRs as the core forge user — how a hive
|
||||||
//! reports a merged config commit it refused to deploy back onto the PR that
|
//! reports a merged config commit it refused to deploy back onto the PR that
|
||||||
//! merged it.
|
//! merged it. Finding that PR only reads, so it uses [`reader_token`].
|
||||||
|
|
||||||
use anyhow::{Context, Result};
|
use anyhow::{Context, Result};
|
||||||
|
|
||||||
use super::{CONFIG_ORG, api, core_token};
|
use super::{CONFIG_ORG, api, core_token, reader_token};
|
||||||
|
|
||||||
/// Full `owner/name` path of an agent's config repo on the forge.
|
/// Full `owner/name` path of an agent's config repo on the forge.
|
||||||
pub fn config_repo(agent: &str) -> String {
|
pub fn config_repo(agent: &str) -> String {
|
||||||
|
|
@ -14,13 +14,13 @@ pub fn config_repo(agent: &str) -> String {
|
||||||
/// The number of the merged PR that put commit `sha` on `repo`'s base branch.
|
/// The number of the merged PR that put commit `sha` on `repo`'s base branch.
|
||||||
///
|
///
|
||||||
/// # Errors
|
/// # Errors
|
||||||
/// Absent core token, malformed repo, no such PR, or transport/API failure.
|
/// No read token, malformed repo, no such PR, or transport/API failure.
|
||||||
pub async fn merged_pr_for_commit(repo: &str, sha: &str) -> Result<u64> {
|
pub async fn merged_pr_for_commit(repo: &str, sha: &str) -> Result<u64> {
|
||||||
let token = core_token().context("forge core token absent")?;
|
let reader = reader_token().await.context("no forge read token")?;
|
||||||
let (owner, name) = repo
|
let (owner, name) = repo
|
||||||
.split_once('/')
|
.split_once('/')
|
||||||
.with_context(|| format!("forge repo `{repo}` is not owner/name"))?;
|
.with_context(|| format!("forge repo `{repo}` is not owner/name"))?;
|
||||||
let pull = api(&token)?
|
let pull = api(reader.value())?
|
||||||
.repo_get_commit_pull_request(owner, name, sha)
|
.repo_get_commit_pull_request(owner, name, sha)
|
||||||
.await
|
.await
|
||||||
.context("GET pull request of commit")?;
|
.context("GET pull request of commit")?;
|
||||||
|
|
|
||||||
159
hive-c0re/src/forge/reader.rs
Normal file
159
hive-c0re/src/forge/reader.rs
Normal file
|
|
@ -0,0 +1,159 @@
|
||||||
|
//! The credential this hive reads the forge with: its own read-only account,
|
||||||
|
//! whose token `swarm-controller` mints into the store at
|
||||||
|
//! `swarm/hives/<hive>/forge/token`, and the core token file where the store
|
||||||
|
//! holds none.
|
||||||
|
//!
|
||||||
|
//! The file is the fallback so a hive deployed before the controller mints its
|
||||||
|
//! token keeps the reads it had: the forge host holds the file, no other hive
|
||||||
|
//! does.
|
||||||
|
|
||||||
|
use super::{basic_auth_header, core_token};
|
||||||
|
|
||||||
|
/// The basic-auth user [`ReaderToken::auth_header`] names for the core token.
|
||||||
|
const CORE_LOGIN: &str = "core";
|
||||||
|
|
||||||
|
/// A forge token and the account it belongs to.
|
||||||
|
///
|
||||||
|
/// No `Debug` derive: `value` is a live forge credential.
|
||||||
|
pub struct ReaderToken {
|
||||||
|
login: String,
|
||||||
|
value: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl std::fmt::Debug for ReaderToken {
|
||||||
|
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||||
|
f.debug_struct("ReaderToken")
|
||||||
|
.field("login", &self.login)
|
||||||
|
.field("value", &"<redacted>")
|
||||||
|
.finish()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl ReaderToken {
|
||||||
|
/// The token itself, for an API client.
|
||||||
|
pub fn value(&self) -> &str {
|
||||||
|
&self.value
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The `http.extraHeader` value for git; see [`basic_auth_header`].
|
||||||
|
pub fn auth_header(&self) -> String {
|
||||||
|
basic_auth_header(&self.login, &self.value)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The token to read the forge with, or `None` when this hive has neither.
|
||||||
|
///
|
||||||
|
/// 🩸 Logs the store path and which source won, never a value.
|
||||||
|
pub async fn reader_token() -> Option<ReaderToken> {
|
||||||
|
pick(stored_reader_token().await, core_token())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The store's token wins over the core file's.
|
||||||
|
fn pick(stored: Option<ReaderToken>, core: Option<String>) -> Option<ReaderToken> {
|
||||||
|
let picked = stored.or_else(|| {
|
||||||
|
core.map(|value| ReaderToken {
|
||||||
|
login: CORE_LOGIN.to_owned(),
|
||||||
|
value,
|
||||||
|
})
|
||||||
|
});
|
||||||
|
if let Some(t) = &picked {
|
||||||
|
tracing::debug!(login = %t.login, "forge: reading as this account");
|
||||||
|
} else {
|
||||||
|
tracing::debug!("forge: no read token in the store or the core token file");
|
||||||
|
}
|
||||||
|
picked
|
||||||
|
}
|
||||||
|
|
||||||
|
/// This hive's token from the store, under the hive's own identity.
|
||||||
|
///
|
||||||
|
/// `None`, never an error, for every way it can come up empty — no hive name,
|
||||||
|
/// no `BAO_*` identity, an unreachable store, nothing at the path — the same
|
||||||
|
/// as `matrix::stored_sender_token`, and for the same reason: each sends the
|
||||||
|
/// caller to the fallback.
|
||||||
|
async fn stored_reader_token() -> Option<ReaderToken> {
|
||||||
|
let hive = std::env::var("HYPERHIVE_HIVE_NAME")
|
||||||
|
.ok()
|
||||||
|
.filter(|h| !h.is_empty())?;
|
||||||
|
let path = match swarm_secret_client::forge::hive_token_path(&hive) {
|
||||||
|
Ok(path) => path,
|
||||||
|
Err(e) => {
|
||||||
|
tracing::warn!(error = %e, "forge: this hive's name does not form a store path");
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let store = match swarm_secret_client::SecretStore::from_env(&hive).await {
|
||||||
|
Ok(store) => store,
|
||||||
|
Err(e) => {
|
||||||
|
tracing::debug!(error = %e, "forge: no swarm secret store to read the hive token from");
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
match store
|
||||||
|
.read::<swarm_secret_client::forge::Credential>(&path)
|
||||||
|
.await
|
||||||
|
{
|
||||||
|
Ok(credential) if !credential.value.trim().is_empty() => Some(ReaderToken {
|
||||||
|
login: swarm_secret_client::forge::hive_login(&hive),
|
||||||
|
value: credential.value.trim().to_owned(),
|
||||||
|
}),
|
||||||
|
Ok(_) => {
|
||||||
|
tracing::warn!(%path, "forge: the stored hive token is empty");
|
||||||
|
None
|
||||||
|
}
|
||||||
|
Err(e) => {
|
||||||
|
tracing::debug!(%path, error = %e, "forge: no hive token in the store");
|
||||||
|
None
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn stored() -> ReaderToken {
|
||||||
|
ReaderToken {
|
||||||
|
login: "hive-alpha".to_owned(),
|
||||||
|
value: "from-the-store".to_owned(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_store_wins_over_the_core_file() {
|
||||||
|
let t = pick(Some(stored()), Some("from-the-file".to_owned())).expect("a token");
|
||||||
|
assert_eq!(t.value(), "from-the-store");
|
||||||
|
assert_eq!(t.login, "hive-alpha");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_core_file_is_the_fallback() {
|
||||||
|
let t = pick(None, Some("from-the-file".to_owned())).expect("a token");
|
||||||
|
assert_eq!(t.value(), "from-the-file");
|
||||||
|
assert_eq!(t.login, CORE_LOGIN);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn neither_is_none() {
|
||||||
|
assert!(pick(None, None).is_none());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_header_names_the_tokens_own_account() {
|
||||||
|
use base64::Engine as _;
|
||||||
|
let header = stored().auth_header();
|
||||||
|
let encoded = header
|
||||||
|
.strip_prefix("Authorization: Basic ")
|
||||||
|
.expect("a basic auth header");
|
||||||
|
let decoded = base64::engine::general_purpose::STANDARD
|
||||||
|
.decode(encoded)
|
||||||
|
.expect("base64");
|
||||||
|
assert_eq!(decoded, b"hive-alpha:from-the-store");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn debug_never_prints_the_value() {
|
||||||
|
let shown = format!("{:?}", stored());
|
||||||
|
assert!(!shown.contains("from-the-store"), "{shown}");
|
||||||
|
assert!(shown.contains("hive-alpha"), "{shown}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -17,7 +17,7 @@ use anyhow::{Context, Result};
|
||||||
|
|
||||||
use hive_host_sock::{HostResponse, ReconcileDirection};
|
use hive_host_sock::{HostResponse, ReconcileDirection};
|
||||||
|
|
||||||
use super::{CONFIG_ORG, core_auth_header, core_token, forge_git_url, is_present};
|
use super::{CONFIG_ORG, forge_git_url, reader_token};
|
||||||
|
|
||||||
/// Scratch ref the forge `main` is fetched into — outside the normal
|
/// Scratch ref the forge `main` is fetched into — outside the normal
|
||||||
/// branch/tag namespace so it never collides with real refs.
|
/// branch/tag namespace so it never collides with real refs.
|
||||||
|
|
@ -27,11 +27,11 @@ const FORGE_MAIN_REF: &str = "refs/hyperhive/forge-config-main";
|
||||||
/// ref (read-only, no working-tree change) and return the applied dir.
|
/// ref (read-only, no working-tree change) and return the applied dir.
|
||||||
/// Also how a swarm deploy of a merged commit gets that commit's objects.
|
/// Also how a swarm deploy of a merged commit gets that commit's objects.
|
||||||
pub async fn fetch_forge_main(agent: &str) -> Result<PathBuf> {
|
pub async fn fetch_forge_main(agent: &str) -> Result<PathBuf> {
|
||||||
if !is_present().await {
|
let Some(reader) = reader_token().await else {
|
||||||
anyhow::bail!("forge is not running");
|
anyhow::bail!(
|
||||||
}
|
"no forge read token: the swarm store holds none for this hive and there is no core \
|
||||||
let Some(token) = core_token() else {
|
token file"
|
||||||
anyhow::bail!("forge core token not available");
|
);
|
||||||
};
|
};
|
||||||
let dir = crate::paths::applied_dir(agent);
|
let dir = crate::paths::applied_dir(agent);
|
||||||
if !dir.join(".git").exists() {
|
if !dir.join(".git").exists() {
|
||||||
|
|
@ -41,7 +41,7 @@ pub async fn fetch_forge_main(agent: &str) -> Result<PathBuf> {
|
||||||
crate::lifecycle::git_authed(
|
crate::lifecycle::git_authed(
|
||||||
&dir,
|
&dir,
|
||||||
&["fetch", "--force", &url, &format!("main:{FORGE_MAIN_REF}")],
|
&["fetch", "--force", &url, &format!("main:{FORGE_MAIN_REF}")],
|
||||||
&core_auth_header(&token),
|
&reader.auth_header(),
|
||||||
)
|
)
|
||||||
.await
|
.await
|
||||||
.context("fetch forge config main (does forge main exist yet?)")?;
|
.context("fetch forge config main (does forge main exist yet?)")?;
|
||||||
|
|
|
||||||
|
|
@ -17,7 +17,7 @@ use crate::coordinator::Coordinator;
|
||||||
|
|
||||||
use super::{
|
use super::{
|
||||||
CONFIG_ORG, SHARED_DOCS_REPO, SHARED_ORG, api, core_auth_header, core_token, forge_git_url,
|
CONFIG_ORG, SHARED_DOCS_REPO, SHARED_ORG, api, core_auth_header, core_token, forge_git_url,
|
||||||
forge_http_base, is_present,
|
forge_http_base, is_present, reader_token,
|
||||||
};
|
};
|
||||||
|
|
||||||
/// Creation options for an empty repo defaulting to `main`.
|
/// Creation options for an empty repo defaulting to `main`.
|
||||||
|
|
@ -223,17 +223,14 @@ pub async fn ensure_meta_remote(name: &str) -> Result<()> {
|
||||||
/// snapshot. A config deployed through hive-c0re without a forge merge never
|
/// snapshot. A config deployed through hive-c0re without a forge merge never
|
||||||
/// reaches it.
|
/// reaches it.
|
||||||
///
|
///
|
||||||
/// Best-effort: returns `false` (not an error) rather than bailing when the
|
/// Best-effort: returns `false` (not an error) rather than bailing when there
|
||||||
/// forge is absent, the core token isn't minted yet, `applied` already has a
|
/// is no [`reader_token`], `applied` already has a `.git` (nothing to do), or
|
||||||
/// `.git` (nothing to do), or any git step fails — callers fall back to a
|
/// any git step fails — callers fall back to a less-current source or the
|
||||||
/// less-current source or the original hard error. Only `true` once `applied` is
|
/// original hard error. Only `true` once `applied` is
|
||||||
/// actually seeded and tagged `deployed/0`, mirroring first-spawn's own
|
/// actually seeded and tagged `deployed/0`, mirroring first-spawn's own
|
||||||
/// `setup_applied` seed shape exactly.
|
/// `setup_applied` seed shape exactly.
|
||||||
pub async fn fetch_config_main_into_applied(name: &str) -> bool {
|
pub async fn fetch_config_main_into_applied(name: &str) -> bool {
|
||||||
if !is_present().await {
|
let Some(reader) = reader_token().await else {
|
||||||
return false;
|
|
||||||
}
|
|
||||||
let Some(token) = core_token() else {
|
|
||||||
return false;
|
return false;
|
||||||
};
|
};
|
||||||
let applied = crate::paths::applied_dir(name);
|
let applied = crate::paths::applied_dir(name);
|
||||||
|
|
@ -249,7 +246,7 @@ pub async fn fetch_config_main_into_applied(name: &str) -> bool {
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
let url = forge_git_url(&format!("{CONFIG_ORG}/{name}"));
|
let url = forge_git_url(&format!("{CONFIG_ORG}/{name}"));
|
||||||
let auth = core_auth_header(&token);
|
let auth = reader.auth_header();
|
||||||
let out = crate::lifecycle::git_command_authed(&auth)
|
let out = crate::lifecycle::git_command_authed(&auth)
|
||||||
.current_dir(&applied)
|
.current_dir(&applied)
|
||||||
.args([
|
.args([
|
||||||
|
|
@ -298,14 +295,11 @@ pub async fn fetch_config_main_into_applied(name: &str) -> bool {
|
||||||
/// ancestry-gated advance for a reviewed PR merge, which this is not.
|
/// ancestry-gated advance for a reviewed PR merge, which this is not.
|
||||||
///
|
///
|
||||||
/// Best-effort, same shape as [`fetch_config_main_into_applied`]: returns
|
/// Best-effort, same shape as [`fetch_config_main_into_applied`]: returns
|
||||||
/// `false` (not an error) when the forge is absent, the core token isn't
|
/// `false` (not an error) when there is no [`reader_token`], `applied` has
|
||||||
/// minted, `applied` has no `.git` yet (nothing to fast-forward — reseed
|
/// no `.git` yet (nothing to fast-forward — reseed instead), or any git step
|
||||||
/// instead), or any git step fails.
|
/// fails.
|
||||||
pub async fn fast_forward_applied_main(name: &str) -> bool {
|
pub async fn fast_forward_applied_main(name: &str) -> bool {
|
||||||
if !is_present().await {
|
let Some(reader) = reader_token().await else {
|
||||||
return false;
|
|
||||||
}
|
|
||||||
let Some(token) = core_token() else {
|
|
||||||
return false;
|
return false;
|
||||||
};
|
};
|
||||||
let applied = crate::paths::applied_dir(name);
|
let applied = crate::paths::applied_dir(name);
|
||||||
|
|
@ -313,7 +307,7 @@ pub async fn fast_forward_applied_main(name: &str) -> bool {
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
let url = forge_git_url(&format!("{CONFIG_ORG}/{name}"));
|
let url = forge_git_url(&format!("{CONFIG_ORG}/{name}"));
|
||||||
let auth = core_auth_header(&token);
|
let auth = reader.auth_header();
|
||||||
let out = crate::lifecycle::git_command_authed(&auth)
|
let out = crate::lifecycle::git_command_authed(&auth)
|
||||||
.current_dir(&applied)
|
.current_dir(&applied)
|
||||||
.args(["fetch", "--no-tags", &url, "refs/heads/main"])
|
.args(["fetch", "--no-tags", &url, "refs/heads/main"])
|
||||||
|
|
@ -356,8 +350,8 @@ pub async fn fast_forward_applied_main(name: &str) -> bool {
|
||||||
/// is the fallback, not the default.
|
/// is the fallback, not the default.
|
||||||
///
|
///
|
||||||
/// Best-effort like [`fetch_config_main_into_applied`], and `false` is an
|
/// Best-effort like [`fetch_config_main_into_applied`], and `false` is an
|
||||||
/// ordinary answer rather than an error: the forge is absent, the core token
|
/// ordinary answer rather than an error: there is no [`reader_token`], the
|
||||||
/// is not minted yet, or `agent-configs/<name>` has no `main` — the last of
|
/// forge is unreachable, or `agent-configs/<name>` has no `main` — the last of
|
||||||
/// which is the *normal* case right after `swarm-controller` creates the
|
/// which is the *normal* case right after `swarm-controller` creates the
|
||||||
/// agent, before it has pushed anything into the repo it just made.
|
/// agent, before it has pushed anything into the repo it just made.
|
||||||
///
|
///
|
||||||
|
|
@ -366,17 +360,14 @@ pub async fn fast_forward_applied_main(name: &str) -> bool {
|
||||||
/// `.git` behind either — so the caller's "does this repo already exist"
|
/// `.git` behind either — so the caller's "does this repo already exist"
|
||||||
/// check reads exactly the same before and after a failed attempt.
|
/// check reads exactly the same before and after a failed attempt.
|
||||||
pub async fn clone_config_into_proposed(dir: &Path, name: &str) -> bool {
|
pub async fn clone_config_into_proposed(dir: &Path, name: &str) -> bool {
|
||||||
if !is_present().await {
|
let Some(reader) = reader_token().await else {
|
||||||
return false;
|
|
||||||
}
|
|
||||||
let Some(token) = core_token() else {
|
|
||||||
return false;
|
return false;
|
||||||
};
|
};
|
||||||
let url = forge_git_url(&format!("{CONFIG_ORG}/{name}"));
|
let url = forge_git_url(&format!("{CONFIG_ORG}/{name}"));
|
||||||
// `--branch main` fails outright against a repo with no commits, so an
|
// `--branch main` fails outright against a repo with no commits, so an
|
||||||
// empty repo reads as "nothing to take" instead of cloning to an unborn
|
// empty repo reads as "nothing to take" instead of cloning to an unborn
|
||||||
// HEAD that would then look like a seeded checkout.
|
// HEAD that would then look like a seeded checkout.
|
||||||
let out = crate::lifecycle::git_command_authed(&core_auth_header(&token))
|
let out = crate::lifecycle::git_command_authed(&reader.auth_header())
|
||||||
.args([
|
.args([
|
||||||
"clone",
|
"clone",
|
||||||
"--branch",
|
"--branch",
|
||||||
|
|
|
||||||
|
|
@ -27,7 +27,7 @@ use super::git::{
|
||||||
/// there's genuinely nothing to clone yet — `swarm-controller` creates
|
/// there's genuinely nothing to clone yet — `swarm-controller` creates
|
||||||
/// `agent-configs/<name>` before telling a hive to deploy, but hasn't
|
/// `agent-configs/<name>` before telling a hive to deploy, but hasn't
|
||||||
/// necessarily pushed `main` to it yet, or the forge is unreachable, or
|
/// necessarily pushed `main` to it yet, or the forge is unreachable, or
|
||||||
/// the core token isn't minted.
|
/// this hive has no forge read token.
|
||||||
///
|
///
|
||||||
/// **Seeding is the whole of hive-c0re's write.** Changes to the repo
|
/// **Seeding is the whole of hive-c0re's write.** Changes to the repo
|
||||||
/// arrive as PRs from a clone, via the forge, like any other code
|
/// arrive as PRs from a clone, via the forge, like any other code
|
||||||
|
|
|
||||||
|
|
@ -22,7 +22,7 @@
|
||||||
use anyhow::{Context, Result};
|
use anyhow::{Context, Result};
|
||||||
|
|
||||||
use crate::coordinator::Coordinator;
|
use crate::coordinator::Coordinator;
|
||||||
use crate::forge::{core_auth_header, forge_git_url};
|
use crate::forge::forge_git_url;
|
||||||
|
|
||||||
pub const ORG: &str = "internal";
|
pub const ORG: &str = "internal";
|
||||||
pub const REPO: &str = "knowledge";
|
pub const REPO: &str = "knowledge";
|
||||||
|
|
@ -38,14 +38,12 @@ pub use crate::paths::KNOWLEDGE_DIR as LOCAL_DIR;
|
||||||
pub const CONTAINER_MOUNT: &str = "/knowledge";
|
pub const CONTAINER_MOUNT: &str = "/knowledge";
|
||||||
|
|
||||||
/// Clone `internal/knowledge` to [`LOCAL_DIR`] if it is not already a git
|
/// Clone `internal/knowledge` to [`LOCAL_DIR`] if it is not already a git
|
||||||
/// repository. `core_token` authenticates the HTTPS clone so private repos
|
/// repository, authenticated with `auth` (an `http.extraHeader` value) when
|
||||||
/// work. Idempotent — skips if `LOCAL_DIR/.git` exists.
|
/// given. Idempotent — skips if `LOCAL_DIR/.git` exists.
|
||||||
///
|
///
|
||||||
/// The swarm controller seeds a README into a freshly created repo. A clone
|
/// The swarm controller seeds a README into a freshly created repo. A clone
|
||||||
/// taken before that lands is empty; the next [`pull`] fills it.
|
/// taken before that lands is empty; the next [`pull`] fills it.
|
||||||
///
|
async fn ensure_local_clone(auth: Option<&str>) -> Result<()> {
|
||||||
/// Called once at hive-c0re startup after `forge::ensure_all`.
|
|
||||||
pub async fn ensure_local_clone(core_token: &str) -> Result<()> {
|
|
||||||
let git_dir = std::path::Path::new(LOCAL_DIR).join(".git");
|
let git_dir = std::path::Path::new(LOCAL_DIR).join(".git");
|
||||||
if git_dir.exists() {
|
if git_dir.exists() {
|
||||||
tracing::debug!("knowledge: local clone already present at {LOCAL_DIR}");
|
tracing::debug!("knowledge: local clone already present at {LOCAL_DIR}");
|
||||||
|
|
@ -53,7 +51,11 @@ pub async fn ensure_local_clone(core_token: &str) -> Result<()> {
|
||||||
}
|
}
|
||||||
std::fs::create_dir_all(LOCAL_DIR).context("create knowledge local dir")?;
|
std::fs::create_dir_all(LOCAL_DIR).context("create knowledge local dir")?;
|
||||||
let url = forge_git_url(&format!("{ORG}/{REPO}"));
|
let url = forge_git_url(&format!("{ORG}/{REPO}"));
|
||||||
let out = crate::lifecycle::git_command_authed(&core_auth_header(core_token))
|
let mut cmd = match auth {
|
||||||
|
Some(header) => crate::lifecycle::git_command_authed(header),
|
||||||
|
None => crate::lifecycle::git_command(),
|
||||||
|
};
|
||||||
|
let out = cmd
|
||||||
.args(["clone", &url, LOCAL_DIR])
|
.args(["clone", &url, LOCAL_DIR])
|
||||||
.output()
|
.output()
|
||||||
.await
|
.await
|
||||||
|
|
@ -86,15 +88,21 @@ async fn head_sha() -> Option<String> {
|
||||||
/// `--ff-only` so a force-push to the knowledge repo never wedges the local
|
/// `--ff-only` so a force-push to the knowledge repo never wedges the local
|
||||||
/// copy silently.
|
/// copy silently.
|
||||||
///
|
///
|
||||||
|
/// When there is no clone yet, takes one instead ([`ensure_local_clone`]),
|
||||||
|
/// so every hive gets `/knowledge` from its first pull.
|
||||||
|
///
|
||||||
/// When the pull actually moves `HEAD` (a real change, not a no-op),
|
/// When the pull actually moves `HEAD` (a real change, not a no-op),
|
||||||
/// broadcasts a short `git diff --stat` summary to every live agent's
|
/// broadcasts a short `git diff --stat` summary to every live agent's
|
||||||
/// inbox via `coord` — inbox-only, no forced wake.
|
/// inbox via `coord` — inbox-only, no forced wake.
|
||||||
pub async fn pull(coord: &Coordinator) -> Result<()> {
|
pub async fn pull(coord: &Coordinator) -> Result<()> {
|
||||||
// Sanity: if the clone is missing (e.g. storage was wiped), refuse
|
// The repo is public (the swarm controller makes it so), so an
|
||||||
// to pull and let the caller decide whether to re-clone.
|
// unauthenticated clone or pull is enough — but authenticate when a token
|
||||||
|
// is around, which keeps this working if the repo is ever made private
|
||||||
|
// again.
|
||||||
|
let auth = crate::forge::reader_token().await.map(|t| t.auth_header());
|
||||||
let git_dir = std::path::Path::new(LOCAL_DIR).join(".git");
|
let git_dir = std::path::Path::new(LOCAL_DIR).join(".git");
|
||||||
if !git_dir.exists() {
|
if !git_dir.exists() {
|
||||||
anyhow::bail!("knowledge: {LOCAL_DIR}/.git not found — clone first");
|
return ensure_local_clone(auth.as_deref()).await;
|
||||||
}
|
}
|
||||||
// Rewrite any credentialed `origin` left by an older clone, which spliced
|
// Rewrite any credentialed `origin` left by an older clone, which spliced
|
||||||
// `core:<token>@` into the URL and persisted it in `.git/config` — the one
|
// `core:<token>@` into the URL and persisted it in `.git/config` — the one
|
||||||
|
|
@ -129,10 +137,6 @@ pub async fn pull(coord: &Coordinator) -> Result<()> {
|
||||||
.await;
|
.await;
|
||||||
|
|
||||||
let before = head_sha().await;
|
let before = head_sha().await;
|
||||||
// The repo is public (the swarm controller makes it so), so an
|
|
||||||
// unauthenticated pull is enough — but authenticate when a token is around,
|
|
||||||
// which keeps this working if the repo is ever made private again.
|
|
||||||
let auth = crate::forge::core_token().map(|t| core_auth_header(&t));
|
|
||||||
let mut cmd = match &auth {
|
let mut cmd = match &auth {
|
||||||
Some(header) => crate::lifecycle::git_command_authed(header),
|
Some(header) => crate::lifecycle::git_command_authed(header),
|
||||||
None => crate::lifecycle::git_command(),
|
None => crate::lifecycle::git_command(),
|
||||||
|
|
|
||||||
|
|
@ -429,6 +429,10 @@ let
|
||||||
capabilities = ["create", "read", "update"]
|
capabilities = ["create", "read", "update"]
|
||||||
}
|
}
|
||||||
|
|
||||||
|
path "${credentialMountPath}/data/swarm/hives/+/forge/token" {
|
||||||
|
capabilities = ["create", "read", "update"]
|
||||||
|
}
|
||||||
|
|
||||||
path "${credentialMountPath}/data/${ciRunnerTokenLeaf}" {
|
path "${credentialMountPath}/data/${ciRunnerTokenLeaf}" {
|
||||||
capabilities = ["create", "update"]
|
capabilities = ["create", "update"]
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -1299,17 +1299,19 @@ let
|
||||||
lib.hasInfix "path \"secret/data/swarm/agents/*\" {\n capabilities = [\"create\", \"read\", \"update\"]" s;
|
lib.hasInfix "path \"secret/data/swarm/agents/*\" {\n capabilities = [\"create\", \"read\", \"update\"]" s;
|
||||||
}
|
}
|
||||||
{
|
{
|
||||||
# `matrix_account::hive_sender` writes every hive's sender token, and
|
# `matrix_account::hive_sender` writes every hive's sender token and
|
||||||
# nothing else under `hives/`, where each hive's queue credential also
|
# `forge::hive_token` every hive's forge token, and nothing else under
|
||||||
# lives. `+` is one segment, which keeps this to the one leaf; a `*` is a
|
# `hives/`, where each hive's queue credential also lives. `+` is one
|
||||||
# glob only at the end of a path. Pinned as the whole stanza, so an
|
# segment, which keeps each stanza to its one leaf; a `*` is a glob only
|
||||||
# added capability fails.
|
# at the end of a path. Pinned as whole stanzas, so an added capability
|
||||||
name = "the controller writes each hive's sender token and nothing else under hives";
|
# fails.
|
||||||
|
name = "the controller writes each hive's sender and forge tokens and nothing else under hives";
|
||||||
ok =
|
ok =
|
||||||
let
|
let
|
||||||
s = baoGrantHere.systemd.services.swarm-bao-controller-policy.script;
|
s = baoGrantHere.systemd.services.swarm-bao-controller-policy.script;
|
||||||
in
|
in
|
||||||
lib.hasInfix "path \"secret/data/swarm/hives/+/matrix/sender-token\" {\n capabilities = [\"create\", \"read\", \"update\"]\n}" s
|
lib.hasInfix "path \"secret/data/swarm/hives/+/matrix/sender-token\" {\n capabilities = [\"create\", \"read\", \"update\"]\n}" s
|
||||||
|
&& lib.hasInfix "path \"secret/data/swarm/hives/+/forge/token\" {\n capabilities = [\"create\", \"read\", \"update\"]\n}" s
|
||||||
&& !(lib.hasInfix "secret/data/swarm/hives/*" s)
|
&& !(lib.hasInfix "secret/data/swarm/hives/*" s)
|
||||||
&& !(lib.hasInfix "secret/metadata/swarm/hives" s);
|
&& !(lib.hasInfix "secret/metadata/swarm/hives" s);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -34,6 +34,7 @@ use utoipa::ToSchema;
|
||||||
use crate::webhook::DeliveryKind;
|
use crate::webhook::DeliveryKind;
|
||||||
|
|
||||||
pub mod agent_token;
|
pub mod agent_token;
|
||||||
|
pub mod hive_token;
|
||||||
pub mod legacy_tokens;
|
pub mod legacy_tokens;
|
||||||
pub mod objects;
|
pub mod objects;
|
||||||
pub mod runner_token;
|
pub mod runner_token;
|
||||||
|
|
@ -455,13 +456,20 @@ impl Client {
|
||||||
/// Create `agent`'s account. Idempotent: an existing user (409/422) is
|
/// Create `agent`'s account. Idempotent: an existing user (409/422) is
|
||||||
/// folded into success, same as [`Self::ensure_org_repo`].
|
/// folded into success, same as [`Self::ensure_org_repo`].
|
||||||
async fn create_agent_user(&self, agent: &str) -> Result<()> {
|
async fn create_agent_user(&self, agent: &str) -> Result<()> {
|
||||||
|
self.create_user(agent, &agent_email(agent)).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Create the account `login` with `email` and a throwaway password.
|
||||||
|
/// Idempotent like [`Self::create_agent_user`], and like it proves only
|
||||||
|
/// that *some* account named `login` exists: the caller checks the email.
|
||||||
|
async fn create_user(&self, login: &str, email: &str) -> Result<()> {
|
||||||
let password = crate::webhook::generate_hex_secret()
|
let password = crate::webhook::generate_hex_secret()
|
||||||
.context("generating a throwaway password for the agent's forge account")?;
|
.context("generating a throwaway password for a forge account")?;
|
||||||
let res = self
|
let res = self
|
||||||
.api
|
.api
|
||||||
.admin_create_user(CreateUserOption {
|
.admin_create_user(CreateUserOption {
|
||||||
created_at: None,
|
created_at: None,
|
||||||
email: agent_email(agent),
|
email: email.to_owned(),
|
||||||
full_name: None,
|
full_name: None,
|
||||||
login_name: None,
|
login_name: None,
|
||||||
must_change_password: Some(false),
|
must_change_password: Some(false),
|
||||||
|
|
@ -469,24 +477,24 @@ impl Client {
|
||||||
restricted: None,
|
restricted: None,
|
||||||
send_notify: None,
|
send_notify: None,
|
||||||
source_id: None,
|
source_id: None,
|
||||||
username: agent.to_owned(),
|
username: login.to_owned(),
|
||||||
visibility: None,
|
visibility: None,
|
||||||
})
|
})
|
||||||
.await;
|
.await;
|
||||||
match res {
|
match res {
|
||||||
Ok(_) => {
|
Ok(_) => {
|
||||||
tracing::info!(%agent, "swarm forge: created agent forge user");
|
tracing::info!(%login, "swarm forge: created forge user");
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
let existing =
|
let existing =
|
||||||
is_ambiguous_validation_failure(&e) && self.api.user_get(agent).await.is_ok();
|
is_ambiguous_validation_failure(&e) && self.api.user_get(login).await.is_ok();
|
||||||
if folds_into_success(&e, existing) {
|
if folds_into_success(&e, existing) {
|
||||||
tracing::debug!(%agent, "swarm forge: agent forge user already exists");
|
tracing::debug!(%login, "swarm forge: forge user already exists");
|
||||||
Ok(())
|
Ok(())
|
||||||
} else {
|
} else {
|
||||||
tracing::error!(%agent, error = %e, "swarm forge: forge refused to create agent user");
|
tracing::error!(%login, error = %e, "swarm forge: forge refused to create user");
|
||||||
Err(e).with_context(|| format!("create forge user {agent}"))
|
Err(e).with_context(|| format!("create forge user {login}"))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -1426,13 +1434,14 @@ mod tests {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// What [`stub_forge`] saw: method, path and JSON body of each request.
|
/// What [`stub_forge`] saw: method, path and JSON body of each request.
|
||||||
type Seen = std::sync::Arc<std::sync::Mutex<Vec<(String, String, serde_json::Value)>>>;
|
pub(super) type Seen =
|
||||||
|
std::sync::Arc<std::sync::Mutex<Vec<(String, String, serde_json::Value)>>>;
|
||||||
|
|
||||||
/// A stand-in forge on a loopback port — never the real one — answering
|
/// A stand-in forge on a loopback port — never the real one — answering
|
||||||
/// the admin user create (`POST`) with `create`, the pre-lockdown
|
/// the admin user create (`POST`) with `create`, the pre-lockdown
|
||||||
/// `user_get` (`GET`) with `get`, and the admin user edit (`PATCH`) with
|
/// `user_get` (`GET`) with `get`, and the admin user edit (`PATCH`) with
|
||||||
/// `edit` (status, JSON body each), and recording every request.
|
/// `edit` (status, JSON body each), and recording every request.
|
||||||
async fn stub_forge(
|
pub(super) async fn stub_forge(
|
||||||
create: (u16, &'static str),
|
create: (u16, &'static str),
|
||||||
get: (u16, &'static str),
|
get: (u16, &'static str),
|
||||||
edit: (u16, &'static str),
|
edit: (u16, &'static str),
|
||||||
|
|
@ -1470,7 +1479,7 @@ mod tests {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The `PATCH` [`Client::ensure_agent_user`] sent, if any.
|
/// The `PATCH` [`Client::ensure_agent_user`] sent, if any.
|
||||||
fn lockdown_patch(seen: &Seen, agent: &str) -> Option<serde_json::Value> {
|
pub(super) fn lockdown_patch(seen: &Seen, agent: &str) -> Option<serde_json::Value> {
|
||||||
let path = format!("/api/v1/admin/users/{agent}");
|
let path = format!("/api/v1/admin/users/{agent}");
|
||||||
seen.lock()
|
seen.lock()
|
||||||
.unwrap()
|
.unwrap()
|
||||||
|
|
|
||||||
|
|
@ -98,9 +98,20 @@ fn last_eight(token: &str) -> Option<&str> {
|
||||||
/// Every token not named [`AGENT_TOKEN_NAME`] is ignored, so the
|
/// Every token not named [`AGENT_TOKEN_NAME`] is ignored, so the
|
||||||
/// `hyperhive-*` tokens `hive-c0re` left behind never affect the decision.
|
/// `hyperhive-*` tokens `hive-c0re` left behind never affect the decision.
|
||||||
pub fn classify(stored: Option<&forge::Credential>, listed: &[AccessToken]) -> Decision {
|
pub fn classify(stored: Option<&forge::Credential>, listed: &[AccessToken]) -> Decision {
|
||||||
|
classify_named(AGENT_TOKEN_NAME, AGENT_TOKEN_SCOPES, stored, listed)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// [`classify`] for the token named `name` that should carry `scopes`, so a
|
||||||
|
/// second kind of token this controller mints decides the same way.
|
||||||
|
pub(super) fn classify_named(
|
||||||
|
name: &str,
|
||||||
|
scopes: &str,
|
||||||
|
stored: Option<&forge::Credential>,
|
||||||
|
listed: &[AccessToken],
|
||||||
|
) -> Decision {
|
||||||
let ours: Vec<&AccessToken> = listed
|
let ours: Vec<&AccessToken> = listed
|
||||||
.iter()
|
.iter()
|
||||||
.filter(|t| t.name.as_deref() == Some(AGENT_TOKEN_NAME))
|
.filter(|t| t.name.as_deref() == Some(name))
|
||||||
.collect();
|
.collect();
|
||||||
let token = match ours[..] {
|
let token = match ours[..] {
|
||||||
[] => return Decision::Mint(MintReason::NotOnForge),
|
[] => return Decision::Mint(MintReason::NotOnForge),
|
||||||
|
|
@ -114,7 +125,7 @@ pub fn classify(stored: Option<&forge::Credential>, listed: &[AccessToken]) -> D
|
||||||
return Decision::Mint(MintReason::LastEightMismatch);
|
return Decision::Mint(MintReason::LastEightMismatch);
|
||||||
}
|
}
|
||||||
let listed_scopes = normalized_scopes(token.scopes.iter().flatten().map(String::as_str));
|
let listed_scopes = normalized_scopes(token.scopes.iter().flatten().map(String::as_str));
|
||||||
if listed_scopes != normalized_scopes(AGENT_TOKEN_SCOPES.split(',')) {
|
if listed_scopes != normalized_scopes(scopes.split(',')) {
|
||||||
return Decision::Mint(MintReason::ScopeMismatch);
|
return Decision::Mint(MintReason::ScopeMismatch);
|
||||||
}
|
}
|
||||||
Decision::Keep
|
Decision::Keep
|
||||||
|
|
@ -216,46 +227,16 @@ impl Client {
|
||||||
Decision::Mint(reason) => reason,
|
Decision::Mint(reason) => reason,
|
||||||
};
|
};
|
||||||
|
|
||||||
if listed
|
let credential = self
|
||||||
.iter()
|
.replace_token(
|
||||||
.any(|t| t.name.as_deref() == Some(AGENT_TOKEN_NAME))
|
&store,
|
||||||
{
|
|
||||||
match self
|
|
||||||
.api
|
|
||||||
.admin_delete_user_access_token(agent, AGENT_TOKEN_NAME)
|
|
||||||
.await
|
|
||||||
{
|
|
||||||
Ok(()) => {}
|
|
||||||
Err(e) if is_not_found(&e) => {}
|
|
||||||
Err(e) => {
|
|
||||||
return Err(e)
|
|
||||||
.with_context(|| format!("deleting {agent}'s {AGENT_TOKEN_NAME} token"));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
let created = self
|
|
||||||
.api
|
|
||||||
.admin_create_user_access_token(
|
|
||||||
agent,
|
agent,
|
||||||
CreateAccessTokenOption {
|
AGENT_TOKEN_NAME,
|
||||||
name: AGENT_TOKEN_NAME.to_owned(),
|
AGENT_TOKEN_SCOPES,
|
||||||
repositories: None,
|
&listed,
|
||||||
scopes: Some(AGENT_TOKEN_SCOPES.split(',').map(str::to_owned).collect()),
|
&path,
|
||||||
},
|
|
||||||
)
|
)
|
||||||
.await
|
.await?;
|
||||||
.with_context(|| format!("creating {agent}'s {AGENT_TOKEN_NAME} token"))?;
|
|
||||||
let Some(value) = created.sha1.filter(|v| !v.is_empty()) else {
|
|
||||||
bail!("the forge created {agent}'s token but returned no value for it");
|
|
||||||
};
|
|
||||||
let credential = forge::Credential {
|
|
||||||
value,
|
|
||||||
name: AGENT_TOKEN_NAME.to_owned(),
|
|
||||||
};
|
|
||||||
store
|
|
||||||
.write(&path, &credential)
|
|
||||||
.await
|
|
||||||
.with_context(|| format!("storing {agent}'s forge token at {path}"))?;
|
|
||||||
|
|
||||||
let as_agent = Forgejo::new(Auth::Token(&credential.value), self.url.clone())
|
let as_agent = Forgejo::new(Auth::Token(&credential.value), self.url.clone())
|
||||||
.context("building a forge client with the new token")?;
|
.context("building a forge client with the new token")?;
|
||||||
|
|
@ -270,6 +251,57 @@ impl Client {
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Delete `user`'s token named `name` if `listed` has one, create a new
|
||||||
|
/// one carrying `scopes`, and store it at `path`. A crash between any two
|
||||||
|
/// steps leaves a state [`classify_named`] reads as a mint again.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
/// When the forge refuses the delete or the create, answers the create
|
||||||
|
/// with no value, or the store refuses the write.
|
||||||
|
pub(super) async fn replace_token(
|
||||||
|
&self,
|
||||||
|
store: &swarm_secret_client::SecretStore,
|
||||||
|
user: &str,
|
||||||
|
name: &str,
|
||||||
|
scopes: &str,
|
||||||
|
listed: &[AccessToken],
|
||||||
|
path: &str,
|
||||||
|
) -> Result<forge::Credential> {
|
||||||
|
if listed.iter().any(|t| t.name.as_deref() == Some(name)) {
|
||||||
|
match self.api.admin_delete_user_access_token(user, name).await {
|
||||||
|
Ok(()) => {}
|
||||||
|
Err(e) if is_not_found(&e) => {}
|
||||||
|
Err(e) => {
|
||||||
|
return Err(e).with_context(|| format!("deleting {user}'s {name} token"));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let created = self
|
||||||
|
.api
|
||||||
|
.admin_create_user_access_token(
|
||||||
|
user,
|
||||||
|
CreateAccessTokenOption {
|
||||||
|
name: name.to_owned(),
|
||||||
|
repositories: None,
|
||||||
|
scopes: Some(scopes.split(',').map(str::to_owned).collect()),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.with_context(|| format!("creating {user}'s {name} token"))?;
|
||||||
|
let Some(value) = created.sha1.filter(|v| !v.is_empty()) else {
|
||||||
|
bail!("the forge created {user}'s token but returned no value for it");
|
||||||
|
};
|
||||||
|
let credential = forge::Credential {
|
||||||
|
value,
|
||||||
|
name: name.to_owned(),
|
||||||
|
};
|
||||||
|
store
|
||||||
|
.write(path, &credential)
|
||||||
|
.await
|
||||||
|
.with_context(|| format!("storing {user}'s forge token at {path}"))?;
|
||||||
|
Ok(credential)
|
||||||
|
}
|
||||||
|
|
||||||
/// What the forge and the store say about `agent`'s token.
|
/// What the forge and the store say about `agent`'s token.
|
||||||
pub(super) async fn observe_agent(
|
pub(super) async fn observe_agent(
|
||||||
&self,
|
&self,
|
||||||
|
|
|
||||||
671
swarm-controller/src/forge/hive_token.rs
Normal file
671
swarm-controller/src/forge/hive_token.rs
Normal file
|
|
@ -0,0 +1,671 @@
|
||||||
|
//! Each hive's read-only account on the swarm's forge, `hive-<hive>`: created
|
||||||
|
//! here with the admin API, put in the [`HIVES_TEAM`] of `agent-configs` (read
|
||||||
|
//! on every config repo, including ones created later), and given one token,
|
||||||
|
//! stored at `swarm/hives/<hive>/forge/token`, where hive-c0re reads it under
|
||||||
|
//! the hive's own store identity.
|
||||||
|
//!
|
||||||
|
//! The roster is the controller's hive directory, as for
|
||||||
|
//! `matrix_account::hive_sender`. The token decision is
|
||||||
|
//! [`super::agent_token`]'s, under this module's name and scopes, so a stored
|
||||||
|
//! token the forge still lists with the same last eight characters and scopes
|
||||||
|
//! is kept, and a pass whose reads fail queues nothing.
|
||||||
|
//!
|
||||||
|
//! ⚠️ The account name is not reserved: an agent may be called `hive-<x>`.
|
||||||
|
//! Each side refuses an account carrying the other's email, so neither ever
|
||||||
|
//! locks down, joins the team with, or mints a token for the other's account.
|
||||||
|
|
||||||
|
use std::sync::Arc;
|
||||||
|
|
||||||
|
use anyhow::{Context, Result, bail};
|
||||||
|
use forgejo_api::structs::{
|
||||||
|
AccessToken, CreateTeamOption, CreateTeamOptionPermission, EditTeamOption,
|
||||||
|
EditTeamOptionPermission, RepoSearchQuery, Team, TeamPermission,
|
||||||
|
};
|
||||||
|
use forgejo_api::{Auth, Forgejo};
|
||||||
|
use swarm_secret_client::forge;
|
||||||
|
|
||||||
|
use super::agent_token::{Decision, Observed, classify_named, is_not_found};
|
||||||
|
use super::{CONFIG_ORG, Client, folds_into_success};
|
||||||
|
|
||||||
|
/// The forge's name for every hive token this module mints.
|
||||||
|
pub const HIVE_TOKEN_NAME: &str = "swarm-hive";
|
||||||
|
|
||||||
|
/// What a hive token may do: read repositories. Which repositories is the
|
||||||
|
/// account's business, and the account is only in [`HIVES_TEAM`].
|
||||||
|
pub const HIVE_TOKEN_SCOPES: &str = "read:repository";
|
||||||
|
|
||||||
|
/// The team in `agent-configs` every hive account is a member of.
|
||||||
|
const HIVES_TEAM: &str = "hives";
|
||||||
|
|
||||||
|
const HIVES_TEAM_DESCRIPTION: &str = "hyperhive hives — read on every agent config repo";
|
||||||
|
|
||||||
|
/// What a hive reads in a config repo: the git tree, and which PR merged a
|
||||||
|
/// commit.
|
||||||
|
const HIVES_TEAM_UNITS: [&str; 2] = ["repo.code", "repo.pulls"];
|
||||||
|
|
||||||
|
/// How often [`spawn`] re-checks every hive.
|
||||||
|
const RECONCILE_INTERVAL: std::time::Duration = std::time::Duration::from_mins(5);
|
||||||
|
|
||||||
|
/// The email this module creates `hive`'s account with, and the one it
|
||||||
|
/// requires before acting on an account by that name.
|
||||||
|
///
|
||||||
|
/// Under its own domain, so it can equal neither email an agent account
|
||||||
|
/// carries (`super::agent_email`, `super::legacy_agent_email`) whatever the
|
||||||
|
/// agent is called.
|
||||||
|
fn hive_email(hive: &str) -> String {
|
||||||
|
format!("{}@hives.hyperhive.local", forge::hive_login(hive))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether the account named [`forge::hive_login`]`(hive)`, carrying `email`,
|
||||||
|
/// is the one this module creates.
|
||||||
|
fn is_hive_account(hive: &str, email: Option<&str>) -> bool {
|
||||||
|
email == Some(hive_email(hive).as_str())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whether `t` has every setting [`Client::ensure_hives_team`] gives it.
|
||||||
|
fn hives_team_matches(t: &Team) -> bool {
|
||||||
|
let units_match = t.units.as_ref().is_some_and(|units| {
|
||||||
|
let mut have: Vec<&str> = units.iter().map(String::as_str).collect();
|
||||||
|
let mut want = HIVES_TEAM_UNITS.to_vec();
|
||||||
|
have.sort_unstable();
|
||||||
|
want.sort_unstable();
|
||||||
|
have == want
|
||||||
|
});
|
||||||
|
units_match
|
||||||
|
&& t.permission == Some(TeamPermission::Read)
|
||||||
|
&& t.includes_all_repositories == Some(true)
|
||||||
|
&& t.can_create_org_repo == Some(false)
|
||||||
|
&& t.description.as_deref() == Some(HIVES_TEAM_DESCRIPTION)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// [`HIVES_TEAM`] as one pass found it.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
pub enum TeamSeen {
|
||||||
|
/// Present with every setting [`hives_team_matches`] checks.
|
||||||
|
Matches(i64),
|
||||||
|
/// Present with some other setting, which may be a wider grant.
|
||||||
|
Drifted(i64),
|
||||||
|
Absent,
|
||||||
|
/// The read failed.
|
||||||
|
Unknown,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What one pass found for one hive.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
pub struct HiveSeen {
|
||||||
|
pub token: Observed,
|
||||||
|
/// Whether the account is in [`HIVES_TEAM`]; `None` when that read failed
|
||||||
|
/// or the team id is not known.
|
||||||
|
pub member: Option<bool>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The hives a pass queues a `MintHiveForgeToken` job for.
|
||||||
|
///
|
||||||
|
/// A token that has to be minted, or an account that does not exist, always
|
||||||
|
/// queues. A kept token queues only when the account or the team is not as
|
||||||
|
/// the job leaves them; that job finds the token current again and does not
|
||||||
|
/// rotate it.
|
||||||
|
///
|
||||||
|
/// [`Observed::Unknown`] never queues: a forge or store outage must not turn
|
||||||
|
/// into a rotation of every hive's token. Neither does a
|
||||||
|
/// [`Decision::Conflict`], which the job would refuse anyway.
|
||||||
|
pub fn plan(team: TeamSeen, hives: &[(String, HiveSeen)]) -> Vec<String> {
|
||||||
|
let team_needs_work = matches!(team, TeamSeen::Absent | TeamSeen::Drifted(_));
|
||||||
|
hives
|
||||||
|
.iter()
|
||||||
|
.filter(|(_, seen)| match seen.token {
|
||||||
|
Observed::Unknown | Observed::Decided(Decision::Conflict) => false,
|
||||||
|
Observed::NoForgeUser | Observed::Decided(Decision::Mint(_)) => true,
|
||||||
|
Observed::Decided(Decision::Keep) => team_needs_work || seen.member == Some(false),
|
||||||
|
})
|
||||||
|
.map(|(hive, _)| hive.clone())
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Client {
|
||||||
|
/// Make sure `hive`'s account exists, is in [`HIVES_TEAM`], and holds a
|
||||||
|
/// live [`HIVE_TOKEN_NAME`] token in the store, minting one when it does
|
||||||
|
/// not. The whole job of the `MintHiveForgeToken` node.
|
||||||
|
///
|
||||||
|
/// A rotation is [`Client::replace_token`]'s delete then create. The new
|
||||||
|
/// token is used once, and the forge's and the store's state is read back
|
||||||
|
/// and classified again, so the node does not report success on a write
|
||||||
|
/// nobody has read.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
/// When the store or the forge refuses a step, when an account by the
|
||||||
|
/// hive's name exists and is not the hive's, or when the read-back does
|
||||||
|
/// not show the new token.
|
||||||
|
pub async fn ensure_hive_forge_token(&self, hive: &str) -> Result<()> {
|
||||||
|
let login = forge::hive_login(hive);
|
||||||
|
let path = forge::hive_token_path(hive)?;
|
||||||
|
self.ensure_hive_user(hive, &login).await?;
|
||||||
|
let team = self.ensure_hives_team().await?;
|
||||||
|
self.api
|
||||||
|
.org_add_team_member(team, &login)
|
||||||
|
.await
|
||||||
|
.with_context(|| format!("adding {login} to {CONFIG_ORG}/{HIVES_TEAM}"))?;
|
||||||
|
|
||||||
|
let store = crate::store::connect()
|
||||||
|
.await
|
||||||
|
.context("logging in to the swarm secret store")?;
|
||||||
|
let (stored, listed) = self.read_hive_token(&store, &login, &path).await?;
|
||||||
|
let reason =
|
||||||
|
match classify_named(HIVE_TOKEN_NAME, HIVE_TOKEN_SCOPES, stored.as_ref(), &listed) {
|
||||||
|
Decision::Keep => {
|
||||||
|
tracing::debug!(hive, "hive forge token is current; left as it is");
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
Decision::Conflict => bail!(
|
||||||
|
"the forge lists more than one {HIVE_TOKEN_NAME:?} token for {login:?}; \
|
||||||
|
delete them by id and re-run"
|
||||||
|
),
|
||||||
|
Decision::Mint(reason) => reason,
|
||||||
|
};
|
||||||
|
let credential = self
|
||||||
|
.replace_token(
|
||||||
|
&store,
|
||||||
|
&login,
|
||||||
|
HIVE_TOKEN_NAME,
|
||||||
|
HIVE_TOKEN_SCOPES,
|
||||||
|
&listed,
|
||||||
|
&path,
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
Forgejo::new(Auth::Token(&credential.value), self.url.clone())
|
||||||
|
.context("building a forge client with the new token")?
|
||||||
|
.repo_search(RepoSearchQuery::default())
|
||||||
|
.await
|
||||||
|
.with_context(|| format!("using {login}'s new forge token"))?;
|
||||||
|
let (stored, listed) = self.read_hive_token(&store, &login, &path).await?;
|
||||||
|
let now = classify_named(HIVE_TOKEN_NAME, HIVE_TOKEN_SCOPES, stored.as_ref(), &listed);
|
||||||
|
if now != Decision::Keep {
|
||||||
|
bail!("{login}'s new forge token does not read back as current: {now:?}");
|
||||||
|
}
|
||||||
|
tracing::info!(hive, ?reason, %path, "hive forge token minted and stored");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The stored token at `path` and every token the forge lists for
|
||||||
|
/// `login`, which must exist.
|
||||||
|
async fn read_hive_token(
|
||||||
|
&self,
|
||||||
|
store: &swarm_secret_client::SecretStore,
|
||||||
|
login: &str,
|
||||||
|
path: &str,
|
||||||
|
) -> Result<(Option<forge::Credential>, Vec<AccessToken>)> {
|
||||||
|
let stored = store
|
||||||
|
.read_optional(path)
|
||||||
|
.await
|
||||||
|
.with_context(|| format!("reading {path}"))?;
|
||||||
|
let Some(listed) = self.list_agent_tokens(login).await? else {
|
||||||
|
bail!("the forge has no user {login:?} after this job created it");
|
||||||
|
};
|
||||||
|
Ok((stored, listed))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Create `hive`'s account if there is none, refuse one that is not the
|
||||||
|
/// hive's, and lock it out of creating repos.
|
||||||
|
async fn ensure_hive_user(&self, hive: &str, login: &str) -> Result<()> {
|
||||||
|
self.create_user(login, &hive_email(hive)).await?;
|
||||||
|
let user = self
|
||||||
|
.api
|
||||||
|
.user_get(login)
|
||||||
|
.await
|
||||||
|
.with_context(|| format!("read forge user {login}"))?;
|
||||||
|
if !is_hive_account(hive, user.email.as_deref()) {
|
||||||
|
bail!(
|
||||||
|
"forge account `{login}` exists but is not hive `{hive}`'s (email {:?}, expected \
|
||||||
|
`{}`); refusing to give it the hive's access",
|
||||||
|
user.email,
|
||||||
|
hive_email(hive)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
self.api
|
||||||
|
.admin_edit_user(login, super::repo_creation_lockdown(login))
|
||||||
|
.await
|
||||||
|
.with_context(|| format!("disable repo creation for forge user {login}"))?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The id of [`HIVES_TEAM`] in `agent-configs`, creating it or
|
||||||
|
/// correcting its settings first when needed. Its members are not
|
||||||
|
/// touched here.
|
||||||
|
async fn ensure_hives_team(&self) -> Result<i64> {
|
||||||
|
match self.observe_hives_team().await {
|
||||||
|
TeamSeen::Matches(id) => Ok(id),
|
||||||
|
TeamSeen::Drifted(id) => {
|
||||||
|
self.api
|
||||||
|
.org_edit_team(
|
||||||
|
id,
|
||||||
|
EditTeamOption {
|
||||||
|
can_create_org_repo: Some(false),
|
||||||
|
description: Some(HIVES_TEAM_DESCRIPTION.to_owned()),
|
||||||
|
includes_all_repositories: Some(true),
|
||||||
|
name: HIVES_TEAM.to_owned(),
|
||||||
|
permission: Some(EditTeamOptionPermission::Read),
|
||||||
|
units: Some(HIVES_TEAM_UNITS.map(str::to_owned).to_vec()),
|
||||||
|
units_map: None,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.with_context(|| {
|
||||||
|
format!("reconcile {CONFIG_ORG}/{HIVES_TEAM} team settings")
|
||||||
|
})?;
|
||||||
|
tracing::info!("swarm forge: reconciled {CONFIG_ORG}/{HIVES_TEAM} team settings");
|
||||||
|
Ok(id)
|
||||||
|
}
|
||||||
|
TeamSeen::Absent => self.create_hives_team().await,
|
||||||
|
TeamSeen::Unknown => bail!("could not read the teams of {CONFIG_ORG}"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn create_hives_team(&self) -> Result<i64> {
|
||||||
|
let team = CreateTeamOption {
|
||||||
|
can_create_org_repo: Some(false),
|
||||||
|
description: Some(HIVES_TEAM_DESCRIPTION.to_owned()),
|
||||||
|
includes_all_repositories: Some(true),
|
||||||
|
name: HIVES_TEAM.to_owned(),
|
||||||
|
permission: Some(CreateTeamOptionPermission::Read),
|
||||||
|
units: Some(HIVES_TEAM_UNITS.map(str::to_owned).to_vec()),
|
||||||
|
units_map: None,
|
||||||
|
};
|
||||||
|
match self.api.org_create_team(CONFIG_ORG, team).await {
|
||||||
|
Ok(created) => {
|
||||||
|
tracing::info!("swarm forge: created {CONFIG_ORG}/{HIVES_TEAM} team");
|
||||||
|
created
|
||||||
|
.id
|
||||||
|
.context("the forge created the hives team but returned no id")
|
||||||
|
}
|
||||||
|
// A duplicate is a 409 or a 422; either counts once a re-read
|
||||||
|
// finds the team, which also supplies its id.
|
||||||
|
Err(e) => match self.observe_hives_team().await {
|
||||||
|
TeamSeen::Matches(id) | TeamSeen::Drifted(id) if folds_into_success(&e, true) => {
|
||||||
|
Ok(id)
|
||||||
|
}
|
||||||
|
_ => Err(e).with_context(|| format!("create team {CONFIG_ORG}/{HIVES_TEAM}")),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn observe_hives_team(&self) -> TeamSeen {
|
||||||
|
match self.api.org_list_teams(CONFIG_ORG).await {
|
||||||
|
Ok((_, teams)) => teams
|
||||||
|
.into_iter()
|
||||||
|
.find(|t| t.name.as_deref() == Some(HIVES_TEAM))
|
||||||
|
.map_or(TeamSeen::Absent, |t| match t.id {
|
||||||
|
Some(id) if hives_team_matches(&t) => TeamSeen::Matches(id),
|
||||||
|
Some(id) => TeamSeen::Drifted(id),
|
||||||
|
None => TeamSeen::Unknown,
|
||||||
|
}),
|
||||||
|
Err(e) => {
|
||||||
|
tracing::warn!(error = %e, "hive forge tokens: reading the teams of {CONFIG_ORG} failed");
|
||||||
|
TeamSeen::Unknown
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What the store, the forge and the team say about `hive`.
|
||||||
|
async fn observe_hive(
|
||||||
|
&self,
|
||||||
|
store: &swarm_secret_client::SecretStore,
|
||||||
|
team: TeamSeen,
|
||||||
|
hive: &str,
|
||||||
|
) -> HiveSeen {
|
||||||
|
let login = forge::hive_login(hive);
|
||||||
|
let stored = match forge::hive_token_path(hive) {
|
||||||
|
Ok(path) => store.read_optional::<forge::Credential>(&path).await,
|
||||||
|
Err(e) => Err(e),
|
||||||
|
};
|
||||||
|
let token = match stored {
|
||||||
|
Err(e) => {
|
||||||
|
tracing::warn!(hive, error = %e, "hive forge token: store read failed");
|
||||||
|
Observed::Unknown
|
||||||
|
}
|
||||||
|
Ok(stored) => match self.list_agent_tokens(&login).await {
|
||||||
|
Ok(Some(listed)) => Observed::Decided(classify_named(
|
||||||
|
HIVE_TOKEN_NAME,
|
||||||
|
HIVE_TOKEN_SCOPES,
|
||||||
|
stored.as_ref(),
|
||||||
|
&listed,
|
||||||
|
)),
|
||||||
|
Ok(None) => Observed::NoForgeUser,
|
||||||
|
Err(e) => {
|
||||||
|
tracing::warn!(hive, error = %format!("{e:#}"), "hive forge token: forge read failed");
|
||||||
|
Observed::Unknown
|
||||||
|
}
|
||||||
|
},
|
||||||
|
};
|
||||||
|
let member = match team {
|
||||||
|
TeamSeen::Matches(id) | TeamSeen::Drifted(id) => {
|
||||||
|
match self.api.org_list_team_member(id, &login).await {
|
||||||
|
Ok(_) => Some(true),
|
||||||
|
Err(e) if is_not_found(&e) => Some(false),
|
||||||
|
Err(e) => {
|
||||||
|
tracing::warn!(hive, error = %e, "hive forge token: team membership read failed");
|
||||||
|
None
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
TeamSeen::Absent => Some(false),
|
||||||
|
TeamSeen::Unknown => None,
|
||||||
|
};
|
||||||
|
HiveSeen { token, member }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One pass: the team, then every hive in `hives`.
|
||||||
|
async fn observe_hives(&self, hives: &[String]) -> Result<(TeamSeen, Vec<(String, HiveSeen)>)> {
|
||||||
|
let store = crate::store::connect()
|
||||||
|
.await
|
||||||
|
.context("logging in to the swarm secret store")?;
|
||||||
|
let team = self.observe_hives_team().await;
|
||||||
|
let mut observed = Vec::with_capacity(hives.len());
|
||||||
|
for hive in hives {
|
||||||
|
observed.push((hive.clone(), self.observe_hive(&store, team, hive).await));
|
||||||
|
}
|
||||||
|
Ok((team, observed))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Check every hive now and every [`RECONCILE_INTERVAL`] after, and hand the
|
||||||
|
/// hives that need a job to `enqueue`, which inserts a `MintHiveForgeToken`
|
||||||
|
/// node for each.
|
||||||
|
///
|
||||||
|
/// The roster is the controller's hive directory. A pass that fails is
|
||||||
|
/// logged and retried on the next tick; it never stops the daemon.
|
||||||
|
pub fn spawn(
|
||||||
|
client: Arc<Client>,
|
||||||
|
hives: Vec<String>,
|
||||||
|
enqueue: impl Fn(Vec<String>) + Send + 'static,
|
||||||
|
) {
|
||||||
|
tokio::spawn(async move {
|
||||||
|
let mut ticker = tokio::time::interval(RECONCILE_INTERVAL);
|
||||||
|
loop {
|
||||||
|
ticker.tick().await;
|
||||||
|
match client.observe_hives(&hives).await {
|
||||||
|
Ok((team, observed)) => {
|
||||||
|
let queued = plan(team, &observed);
|
||||||
|
if queued.is_empty() {
|
||||||
|
tracing::debug!(checked = observed.len(), "hive forge tokens: all current");
|
||||||
|
} else {
|
||||||
|
tracing::info!(
|
||||||
|
checked = observed.len(),
|
||||||
|
queued = queued.len(),
|
||||||
|
"hive forge tokens: queueing jobs"
|
||||||
|
);
|
||||||
|
enqueue(queued);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Err(e) => tracing::warn!(
|
||||||
|
error = %format!("{e:#}"),
|
||||||
|
retry_in_s = RECONCILE_INTERVAL.as_secs(),
|
||||||
|
"hive forge tokens: pass failed; retrying next tick"
|
||||||
|
),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::super::agent_token::MintReason;
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
const VALUE: &str = "0123456789abcdef0123456789abcdef01234567";
|
||||||
|
|
||||||
|
fn stored() -> forge::Credential {
|
||||||
|
forge::Credential {
|
||||||
|
value: VALUE.to_owned(),
|
||||||
|
name: HIVE_TOKEN_NAME.to_owned(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn token(name: &str, last_eight: &str, scopes: &[&str]) -> AccessToken {
|
||||||
|
AccessToken {
|
||||||
|
created_at: None,
|
||||||
|
id: Some(1),
|
||||||
|
name: Some(name.to_owned()),
|
||||||
|
repositories: None,
|
||||||
|
scopes: Some(scopes.iter().map(|s| (*s).to_owned()).collect()),
|
||||||
|
sha1: None,
|
||||||
|
token_last_eight: Some(last_eight.to_owned()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn decide(stored: Option<&forge::Credential>, listed: &[AccessToken]) -> Decision {
|
||||||
|
classify_named(HIVE_TOKEN_NAME, HIVE_TOKEN_SCOPES, stored, listed)
|
||||||
|
}
|
||||||
|
|
||||||
|
fn seen(token: Observed, member: Option<bool>) -> HiveSeen {
|
||||||
|
HiveSeen { token, member }
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_token_reads_repositories_and_nothing_else() {
|
||||||
|
assert_eq!(HIVE_TOKEN_SCOPES, "read:repository");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn nothing_on_the_forge_or_in_the_store_creates() {
|
||||||
|
assert_eq!(decide(None, &[]), Decision::Mint(MintReason::NotOnForge));
|
||||||
|
let ours = token(HIVE_TOKEN_NAME, "01234567", &["read:repository"]);
|
||||||
|
assert_eq!(decide(None, &[ours]), Decision::Mint(MintReason::NotStored));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_current_token_is_kept() {
|
||||||
|
let ours = token(HIVE_TOKEN_NAME, "01234567", &["read:repository"]);
|
||||||
|
assert_eq!(decide(Some(&stored()), &[ours]), Decision::Keep);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn another_value_or_wider_scopes_rotate() {
|
||||||
|
let other = token(HIVE_TOKEN_NAME, "ffffffff", &["read:repository"]);
|
||||||
|
assert_eq!(
|
||||||
|
decide(Some(&stored()), &[other]),
|
||||||
|
Decision::Mint(MintReason::LastEightMismatch)
|
||||||
|
);
|
||||||
|
let wider = token(HIVE_TOKEN_NAME, "01234567", &["write:repository"]);
|
||||||
|
assert_eq!(
|
||||||
|
decide(Some(&stored()), &[wider]),
|
||||||
|
Decision::Mint(MintReason::ScopeMismatch)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_agent_token_on_the_account_is_not_the_hive_token() {
|
||||||
|
// The two names must differ, or a hive account would be judged by the
|
||||||
|
// agent token's scopes and the other way round.
|
||||||
|
assert_ne!(HIVE_TOKEN_NAME, super::super::agent_token::AGENT_TOKEN_NAME);
|
||||||
|
let agents = token(
|
||||||
|
super::super::agent_token::AGENT_TOKEN_NAME,
|
||||||
|
"01234567",
|
||||||
|
&["read:repository"],
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
decide(Some(&stored()), &[agents]),
|
||||||
|
Decision::Mint(MintReason::NotOnForge)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn create_and_rotate_are_queued_and_keep_is_not() {
|
||||||
|
let hives = [
|
||||||
|
(
|
||||||
|
"keep".to_owned(),
|
||||||
|
seen(Observed::Decided(Decision::Keep), Some(true)),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"rotate".to_owned(),
|
||||||
|
seen(
|
||||||
|
Observed::Decided(Decision::Mint(MintReason::LastEightMismatch)),
|
||||||
|
Some(true),
|
||||||
|
),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"create".to_owned(),
|
||||||
|
seen(Observed::NoForgeUser, Some(false)),
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"conflict".to_owned(),
|
||||||
|
seen(Observed::Decided(Decision::Conflict), Some(true)),
|
||||||
|
),
|
||||||
|
];
|
||||||
|
assert_eq!(plan(TeamSeen::Matches(7), &hives), ["rotate", "create"]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_unknown_hive_is_never_queued() {
|
||||||
|
// Not even when the team is missing, which queues every hive whose
|
||||||
|
// token is known.
|
||||||
|
let hives = [
|
||||||
|
("a".to_owned(), seen(Observed::Unknown, Some(false))),
|
||||||
|
("b".to_owned(), seen(Observed::Unknown, None)),
|
||||||
|
];
|
||||||
|
for team in [
|
||||||
|
TeamSeen::Matches(7),
|
||||||
|
TeamSeen::Drifted(7),
|
||||||
|
TeamSeen::Absent,
|
||||||
|
TeamSeen::Unknown,
|
||||||
|
] {
|
||||||
|
assert!(plan(team, &hives).is_empty(), "{team:?}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_kept_token_outside_the_team_is_queued() {
|
||||||
|
let keep = |member| seen(Observed::Decided(Decision::Keep), member);
|
||||||
|
let hives = [
|
||||||
|
("in".to_owned(), keep(Some(true))),
|
||||||
|
("out".to_owned(), keep(Some(false))),
|
||||||
|
("unread".to_owned(), keep(None)),
|
||||||
|
];
|
||||||
|
assert_eq!(plan(TeamSeen::Matches(7), &hives), ["out"]);
|
||||||
|
assert_eq!(plan(TeamSeen::Unknown, &hives), ["out"]);
|
||||||
|
// A missing or drifted team queues every known hive: the job fixes the
|
||||||
|
// team, and finds each token current.
|
||||||
|
assert_eq!(plan(TeamSeen::Absent, &hives), ["in", "out", "unread"]);
|
||||||
|
assert_eq!(plan(TeamSeen::Drifted(7), &hives), ["in", "out", "unread"]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn only_the_hives_own_email_is_its_account() {
|
||||||
|
assert!(is_hive_account(
|
||||||
|
"alpha",
|
||||||
|
Some("hive-alpha@hives.hyperhive.local")
|
||||||
|
));
|
||||||
|
assert!(!is_hive_account("alpha", None));
|
||||||
|
assert!(!is_hive_account(
|
||||||
|
"beta",
|
||||||
|
Some("hive-alpha@hives.hyperhive.local")
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn an_agent_called_hive_x_is_not_hive_xs_account() {
|
||||||
|
// Both ways round: the hive job refuses the agent's account, and the
|
||||||
|
// agent's own guard (`disable_repo_creation`) refuses the hive's,
|
||||||
|
// because neither accepts the other's email.
|
||||||
|
let login = forge::hive_login("alpha");
|
||||||
|
for agents in [
|
||||||
|
super::super::agent_email(&login),
|
||||||
|
super::super::legacy_agent_email(&login),
|
||||||
|
] {
|
||||||
|
assert!(!is_hive_account("alpha", Some(&agents)), "{agents}");
|
||||||
|
assert_ne!(hive_email("alpha"), agents);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The job-side guard, against a stand-in forge: an existing account
|
||||||
|
/// named `hive-alpha` that carries an agent's email is refused before the
|
||||||
|
/// lockdown, so nothing after it (team, token) runs either.
|
||||||
|
#[tokio::test]
|
||||||
|
async fn the_job_refuses_an_agent_account_named_like_the_hive() {
|
||||||
|
use super::super::tests::{lockdown_patch, stub_forge};
|
||||||
|
|
||||||
|
let (client, seen) = stub_forge(
|
||||||
|
(409, r#"{"message":"user already exists"}"#),
|
||||||
|
(200, r#"{"email":"hive-alpha@hyperhive.local"}"#),
|
||||||
|
(200, "{}"),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
let err = client
|
||||||
|
.ensure_hive_user("alpha", "hive-alpha")
|
||||||
|
.await
|
||||||
|
.expect_err("an agent's account is not the hive's");
|
||||||
|
let msg = format!("{err:#}");
|
||||||
|
assert!(msg.contains("not hive `alpha`'s"), "{msg}");
|
||||||
|
assert!(lockdown_patch(&seen, "hive-alpha").is_none());
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The other way round: creating an agent called `hive-alpha` refuses
|
||||||
|
/// hive `alpha`'s account rather than lock it down as the agent's.
|
||||||
|
#[tokio::test]
|
||||||
|
async fn an_agent_create_refuses_the_hives_account() {
|
||||||
|
use super::super::tests::{lockdown_patch, stub_forge};
|
||||||
|
|
||||||
|
let (client, seen) = stub_forge(
|
||||||
|
(409, r#"{"message":"user already exists"}"#),
|
||||||
|
(200, r#"{"email":"hive-alpha@hives.hyperhive.local"}"#),
|
||||||
|
(200, "{}"),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
client
|
||||||
|
.ensure_agent_user("hive-alpha")
|
||||||
|
.await
|
||||||
|
.expect_err("the hive's account is not the agent's");
|
||||||
|
assert!(lockdown_patch(&seen, "hive-alpha").is_none());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn the_job_locks_down_the_hives_own_account() {
|
||||||
|
use super::super::tests::{lockdown_patch, stub_forge};
|
||||||
|
|
||||||
|
let (client, seen) = stub_forge(
|
||||||
|
(409, r#"{"message":"user already exists"}"#),
|
||||||
|
(200, r#"{"email":"hive-alpha@hives.hyperhive.local"}"#),
|
||||||
|
(200, "{}"),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
client
|
||||||
|
.ensure_hive_user("alpha", "hive-alpha")
|
||||||
|
.await
|
||||||
|
.expect("the hive's own account");
|
||||||
|
let body = lockdown_patch(&seen, "hive-alpha").expect("no lockdown PATCH sent");
|
||||||
|
assert_eq!(body["max_repo_creation"], 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn team(permission: TeamPermission, units: &[&str]) -> Team {
|
||||||
|
Team {
|
||||||
|
can_create_org_repo: Some(false),
|
||||||
|
description: Some(HIVES_TEAM_DESCRIPTION.to_owned()),
|
||||||
|
id: Some(7),
|
||||||
|
includes_all_repositories: Some(true),
|
||||||
|
name: Some(HIVES_TEAM.to_owned()),
|
||||||
|
organization: None,
|
||||||
|
permission: Some(permission),
|
||||||
|
units: Some(units.iter().map(|u| (*u).to_owned()).collect()),
|
||||||
|
units_map: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_team_matches_only_as_a_read_team_on_every_repo() {
|
||||||
|
assert!(hives_team_matches(&team(
|
||||||
|
TeamPermission::Read,
|
||||||
|
&["repo.pulls", "repo.code"]
|
||||||
|
)));
|
||||||
|
assert!(!hives_team_matches(&team(
|
||||||
|
TeamPermission::Write,
|
||||||
|
&HIVES_TEAM_UNITS
|
||||||
|
)));
|
||||||
|
assert!(!hives_team_matches(&team(
|
||||||
|
TeamPermission::Read,
|
||||||
|
&["repo.code"]
|
||||||
|
)));
|
||||||
|
let mut some_repos = team(TeamPermission::Read, &HIVES_TEAM_UNITS);
|
||||||
|
some_repos.includes_all_repositories = Some(false);
|
||||||
|
assert!(!hives_team_matches(&some_repos));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -134,6 +134,10 @@ enum SwarmNodeKind {
|
||||||
/// in the swarm store, where that hive's matrix sweep reads it. See
|
/// in the swarm store, where that hive's matrix sweep reads it. See
|
||||||
/// `matrix_account::hive_sender`.
|
/// `matrix_account::hive_sender`.
|
||||||
MintHiveSenderToken { hive: String },
|
MintHiveSenderToken { hive: String },
|
||||||
|
/// Make sure `hive`'s read-only forge account (`hive-<hive>`) exists, is in
|
||||||
|
/// the `hives` team, and holds a live token in the swarm store, where
|
||||||
|
/// that hive's hive-c0re reads it. See `forge::hive_token`.
|
||||||
|
MintHiveForgeToken { hive: String },
|
||||||
/// Re-mint `agent`'s queue secret if it is still stored and old enough.
|
/// Re-mint `agent`'s queue secret if it is still stored and old enough.
|
||||||
/// See `agent_renewal`.
|
/// See `agent_renewal`.
|
||||||
///
|
///
|
||||||
|
|
@ -172,6 +176,7 @@ impl hive_jobq_wire::WireNode for SwarmNodeKind {
|
||||||
SwarmNodeKind::MintAgentForgeToken { .. } => "mint_agent_forge_token".to_owned(),
|
SwarmNodeKind::MintAgentForgeToken { .. } => "mint_agent_forge_token".to_owned(),
|
||||||
SwarmNodeKind::MintAgentMatrixAccount { .. } => "mint_agent_matrix_account".to_owned(),
|
SwarmNodeKind::MintAgentMatrixAccount { .. } => "mint_agent_matrix_account".to_owned(),
|
||||||
SwarmNodeKind::MintHiveSenderToken { .. } => "mint_hive_sender_token".to_owned(),
|
SwarmNodeKind::MintHiveSenderToken { .. } => "mint_hive_sender_token".to_owned(),
|
||||||
|
SwarmNodeKind::MintHiveForgeToken { .. } => "mint_hive_forge_token".to_owned(),
|
||||||
SwarmNodeKind::RenewAgentQueueCredential { .. } => {
|
SwarmNodeKind::RenewAgentQueueCredential { .. } => {
|
||||||
"renew_agent_queue_credential".to_owned()
|
"renew_agent_queue_credential".to_owned()
|
||||||
}
|
}
|
||||||
|
|
@ -199,7 +204,8 @@ impl hive_jobq_wire::WireNode for SwarmNodeKind {
|
||||||
| SwarmNodeKind::RenewAgentQueueCredential { agent } => {
|
| SwarmNodeKind::RenewAgentQueueCredential { agent } => {
|
||||||
serde_json::json!({ "agent": agent })
|
serde_json::json!({ "agent": agent })
|
||||||
}
|
}
|
||||||
SwarmNodeKind::MintHiveSenderToken { hive } => serde_json::json!({ "hive": hive }),
|
SwarmNodeKind::MintHiveSenderToken { hive }
|
||||||
|
| SwarmNodeKind::MintHiveForgeToken { hive } => serde_json::json!({ "hive": hive }),
|
||||||
SwarmNodeKind::TriggerDeploy { hive, agent, rev } => {
|
SwarmNodeKind::TriggerDeploy { hive, agent, rev } => {
|
||||||
serde_json::json!({ "agent": agent, "hive": hive, "rev": rev })
|
serde_json::json!({ "agent": agent, "hive": hive, "rev": rev })
|
||||||
}
|
}
|
||||||
|
|
@ -353,12 +359,10 @@ async fn run_swarm_node(
|
||||||
SwarmNodeKind::MintHiveSenderToken { hive } => {
|
SwarmNodeKind::MintHiveSenderToken { hive } => {
|
||||||
mint_hive_sender(deps.matrix_homeserver.as_deref(), &hive).await
|
mint_hive_sender(deps.matrix_homeserver.as_deref(), &hive).await
|
||||||
}
|
}
|
||||||
SwarmNodeKind::RenewAgentQueueCredential { agent } => {
|
SwarmNodeKind::MintHiveForgeToken { hive } => {
|
||||||
match agent_renewal::renew(&agent).await {
|
mint_hive_forge_token(deps.forge, &hive).await
|
||||||
Ok(()) => Outcome::Done,
|
|
||||||
Err(e) => Outcome::Failed(format!("{e:#}")),
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
SwarmNodeKind::RenewAgentQueueCredential { agent } => renew_queue_credential(&agent).await,
|
||||||
SwarmNodeKind::SetAgentWanted { hive, agent } => match deps.wanted {
|
SwarmNodeKind::SetAgentWanted { hive, agent } => match deps.wanted {
|
||||||
None => Outcome::Failed(
|
None => Outcome::Failed(
|
||||||
"no swarm queue is configured on this host, so no wanted-state \
|
"no swarm queue is configured on this host, so no wanted-state \
|
||||||
|
|
@ -410,6 +414,36 @@ async fn mint_forge_token(
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The `RenewAgentQueueCredential` arm, lifted out for the same reason.
|
||||||
|
async fn renew_queue_credential(agent: &str) -> hive_jobq::scheduler::Outcome {
|
||||||
|
use hive_jobq::scheduler::Outcome;
|
||||||
|
|
||||||
|
match agent_renewal::renew(agent).await {
|
||||||
|
Ok(()) => Outcome::Done,
|
||||||
|
Err(e) => Outcome::Failed(format!("{e:#}")),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The `MintHiveForgeToken` arm, lifted out for the same reason.
|
||||||
|
async fn mint_hive_forge_token(
|
||||||
|
forge: Option<Arc<forge::Client>>,
|
||||||
|
hive: &str,
|
||||||
|
) -> hive_jobq::scheduler::Outcome {
|
||||||
|
use hive_jobq::scheduler::Outcome;
|
||||||
|
|
||||||
|
let Some(client) = forge else {
|
||||||
|
return Outcome::Failed(
|
||||||
|
"no forge configured on this host (SWARM_CONTROLLER_FORGE_URL / \
|
||||||
|
SWARM_CONTROLLER_FORGE_TOKEN_FILE unset), so no hive forge token can be minted"
|
||||||
|
.to_owned(),
|
||||||
|
);
|
||||||
|
};
|
||||||
|
match client.ensure_hive_forge_token(hive).await {
|
||||||
|
Ok(()) => Outcome::Done,
|
||||||
|
Err(e) => Outcome::Failed(format!("{e:#}")),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// The `MintAgentMatrixAccount` arm, lifted out for the same reason.
|
/// The `MintAgentMatrixAccount` arm, lifted out for the same reason.
|
||||||
async fn mint_matrix_account(
|
async fn mint_matrix_account(
|
||||||
homeserver: Option<&str>,
|
homeserver: Option<&str>,
|
||||||
|
|
@ -2275,6 +2309,27 @@ fn queue_hive_sender_mints(
|
||||||
Ok(ids)
|
Ok(ids)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Insert one `MintHiveForgeToken` job per hive and return the nodes' ids.
|
||||||
|
/// `forge::hive_token::spawn`'s periodic pass is the one caller.
|
||||||
|
fn queue_hive_forge_token_mints(
|
||||||
|
sched: &Mutex<hive_jobq::scheduler::Scheduler<SwarmNodeKind, SwarmResourceKind>>,
|
||||||
|
hives: Vec<String>,
|
||||||
|
) -> Result<Vec<hive_jobq::NodeId>> {
|
||||||
|
let mut sched = sched
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||||
|
let mut ids = Vec::with_capacity(hives.len());
|
||||||
|
for hive in hives {
|
||||||
|
let queued = sched
|
||||||
|
.insert_job(None, |b| {
|
||||||
|
vec![b.node(SwarmNodeKind::MintHiveForgeToken { hive }).guid()]
|
||||||
|
})
|
||||||
|
.map_err(|e| anyhow::anyhow!("{e}"))?;
|
||||||
|
ids.extend(queued);
|
||||||
|
}
|
||||||
|
Ok(ids)
|
||||||
|
}
|
||||||
|
|
||||||
/// The homeserver agents' own matrix accounts are minted on. Unset is this
|
/// The homeserver agents' own matrix accounts are minted on. Unset is this
|
||||||
/// controller creating agents with no matrix account, which the mint node
|
/// controller creating agents with no matrix account, which the mint node
|
||||||
/// reports by name rather than `main` refusing to start.
|
/// reports by name rather than `main` refusing to start.
|
||||||
|
|
@ -2315,13 +2370,14 @@ fn spawn_matrix_account_backfill(
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Start the forge's periodic passes — the swarm-wide objects, the runner
|
/// Start the forge's periodic passes — the swarm-wide objects, the runner
|
||||||
/// registration token, and agents' tokens (which also sweeps legacy tokens;
|
/// registration token, hives' read tokens, and agents' tokens (which also
|
||||||
/// see `forge::legacy_tokens`) —
|
/// sweeps legacy tokens; see `forge::legacy_tokens`) —
|
||||||
/// when a forge is configured. Lifted out of `main` for
|
/// when a forge is configured. Lifted out of `main` for
|
||||||
/// `clippy::too_many_lines`.
|
/// `clippy::too_many_lines`.
|
||||||
fn spawn_forge_workers(
|
fn spawn_forge_workers(
|
||||||
jobq: &Arc<Mutex<hive_jobq::scheduler::Scheduler<SwarmNodeKind, SwarmResourceKind>>>,
|
jobq: &Arc<Mutex<hive_jobq::scheduler::Scheduler<SwarmNodeKind, SwarmResourceKind>>>,
|
||||||
forge_client: Option<Arc<forge::Client>>,
|
forge_client: Option<Arc<forge::Client>>,
|
||||||
|
hives: &[HiveEntry],
|
||||||
) {
|
) {
|
||||||
let Some(client) = forge_client else {
|
let Some(client) = forge_client else {
|
||||||
return;
|
return;
|
||||||
|
|
@ -2329,6 +2385,13 @@ fn spawn_forge_workers(
|
||||||
forge::objects::spawn(Arc::clone(&client));
|
forge::objects::spawn(Arc::clone(&client));
|
||||||
forge::runner_token::spawn(Arc::clone(&client));
|
forge::runner_token::spawn(Arc::clone(&client));
|
||||||
let sched = Arc::clone(jobq);
|
let sched = Arc::clone(jobq);
|
||||||
|
let hive_names = hives.iter().map(|h| h.name.clone()).collect();
|
||||||
|
forge::hive_token::spawn(Arc::clone(&client), hive_names, move |hives| {
|
||||||
|
if let Err(e) = queue_hive_forge_token_mints(&sched, hives) {
|
||||||
|
tracing::warn!(error = %format!("{e:#}"), "hive forge tokens: queueing failed");
|
||||||
|
}
|
||||||
|
});
|
||||||
|
let sched = Arc::clone(jobq);
|
||||||
forge::agent_token::spawn(client, move |agents| {
|
forge::agent_token::spawn(client, move |agents| {
|
||||||
if let Err(e) = queue_forge_token_mints(&sched, agents) {
|
if let Err(e) = queue_forge_token_mints(&sched, agents) {
|
||||||
tracing::warn!(error = %format!("{e:#}"), "agent forge tokens: queueing failed");
|
tracing::warn!(error = %format!("{e:#}"), "agent forge tokens: queueing failed");
|
||||||
|
|
@ -2953,7 +3016,7 @@ async fn main() -> Result<()> {
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
spawn_forge_workers(&jobq, forge_client.clone());
|
spawn_forge_workers(&jobq, forge_client.clone(), &hives);
|
||||||
let config_prs = forge_client.clone().map(config_pr::spawn);
|
let config_prs = forge_client.clone().map(config_pr::spawn);
|
||||||
let state_forge = keep_forge_for_state(forge_client, webhook_secret.clone());
|
let state_forge = keep_forge_for_state(forge_client, webhook_secret.clone());
|
||||||
|
|
||||||
|
|
@ -4647,6 +4710,39 @@ mod tests {
|
||||||
assert_eq!(hives, ["alpha", "beta"]);
|
assert_eq!(hives, ["alpha", "beta"]);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The hive forge-token pass's queueing: one node per hive, naming the
|
||||||
|
/// hive rather than an agent.
|
||||||
|
#[test]
|
||||||
|
fn a_queued_hive_forge_token_mint_is_one_node_per_hive() {
|
||||||
|
use hive_jobq_wire::WireNode as _;
|
||||||
|
|
||||||
|
let sched = std::sync::Mutex::new(hive_jobq::scheduler::Scheduler::new(
|
||||||
|
hive_jobq::Graph::new(),
|
||||||
|
hive_jobq::resources::ResourceTable::new(),
|
||||||
|
));
|
||||||
|
let ids = super::queue_hive_forge_token_mints(
|
||||||
|
&sched,
|
||||||
|
vec!["alpha".to_owned(), "beta".to_owned()],
|
||||||
|
)
|
||||||
|
.expect("two jobs insert");
|
||||||
|
assert_eq!(ids.len(), 2);
|
||||||
|
let guard = sched
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||||
|
let mut hives: Vec<String> = guard
|
||||||
|
.graph()
|
||||||
|
.nodes()
|
||||||
|
.map(|n| {
|
||||||
|
assert_eq!(n.payload.label(), "mint_hive_forge_token");
|
||||||
|
let data = n.payload.data(n.id.get());
|
||||||
|
assert!(data.get("agent").is_none(), "{data}");
|
||||||
|
data["hive"].as_str().expect("hive is a string").to_owned()
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
hives.sort();
|
||||||
|
assert_eq!(hives, ["alpha", "beta"]);
|
||||||
|
}
|
||||||
|
|
||||||
/// The manual route and the periodic pass both go through
|
/// The manual route and the periodic pass both go through
|
||||||
/// `queue_forge_token_mints`, so this is the assertion that a backfilled
|
/// `queue_forge_token_mints`, so this is the assertion that a backfilled
|
||||||
/// mint is the same pair of nodes agent creation inserts: the forge user,
|
/// mint is the same pair of nodes agent creation inserts: the forge user,
|
||||||
|
|
|
||||||
|
|
@ -1,14 +1,17 @@
|
||||||
//! The forge agreement: where an agent's forge tokens live in the store, and
|
//! The forge agreement: where agents' and hives' forge tokens live in the store,
|
||||||
//! what the objects at those paths hold.
|
//! and what the objects at those paths hold.
|
||||||
//!
|
//!
|
||||||
//! Two kinds. The agent's own token on the swarm's forge
|
//! Three kinds. The agent's own token on the swarm's forge
|
||||||
//! ([`agent_token_path`]): `swarm-controller` mints it with the forge's admin
|
//! ([`agent_token_path`]): `swarm-controller` mints it with the forge's admin
|
||||||
//! API, and `nix/agent-modules/forge-token.nix` reads it back under the agent's
|
//! API, and `nix/agent-modules/forge-token.nix` reads it back under the agent's
|
||||||
//! own certificate. And the agent's accounts on external forges
|
//! own certificate. And the agent's accounts on external forges
|
||||||
//! ([`account_path`]): an operator hands the controller a URL and a token, the
|
//! ([`account_path`]): an operator hands the controller a URL and a token, the
|
||||||
//! controller stores them there, and `nix/agent-modules/forge-accounts.nix`
|
//! controller stores them there, and `nix/agent-modules/forge-accounts.nix`
|
||||||
//! lists [`accounts_dir`] and reads each account. Neither end is senior, so
|
//! lists [`accounts_dir`] and reads each account. A hive's read-only token
|
||||||
//! both halves are stated once, here, beside [`crate::matrix`].
|
//! ([`hive_token_path`]) is the third: `swarm-controller` mints it for the
|
||||||
|
//! hive's own forge account, and hive-c0re reads it under the hive's
|
||||||
|
//! certificate. Neither end is senior, so both halves are stated once, here,
|
||||||
|
//! beside [`crate::matrix`].
|
||||||
|
|
||||||
use serde::{Deserialize, Serialize};
|
use serde::{Deserialize, Serialize};
|
||||||
|
|
||||||
|
|
@ -32,7 +35,34 @@ pub fn agent_token_path(agent: &str) -> Result<String, Error> {
|
||||||
Ok(format!("{prefix}/forge-token"))
|
Ok(format!("{prefix}/forge-token"))
|
||||||
}
|
}
|
||||||
|
|
||||||
/// What an agent's forge-token path holds.
|
/// The forge account `hive` reads the forge as: `swarm-controller` creates it
|
||||||
|
/// and mints [`hive_token_path`]'s token for it, and hive-c0re names it in the
|
||||||
|
/// basic-auth header that token travels in.
|
||||||
|
///
|
||||||
|
/// The `hive-` prefix keeps the account reading as a hive's rather than an
|
||||||
|
/// agent's, as [`crate::matrix::hive_localpart`] does on the homeserver. It
|
||||||
|
/// does not make the name unique: an agent may be called `hive-<x>` too, and
|
||||||
|
/// `swarm-controller` tells the two apart by the account's email.
|
||||||
|
#[must_use]
|
||||||
|
pub fn hive_login(hive: &str) -> String {
|
||||||
|
format!("hive-{hive}")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The path holding `hive`'s read-only token for the swarm's forge.
|
||||||
|
///
|
||||||
|
/// Under [`Kind::Hive`], so the grant that reaches it is the hive's own
|
||||||
|
/// `swarm/hives/<this hive>/*` stanza: hive `alpha` reads its own token and
|
||||||
|
/// gets a 403 on `beta`'s.
|
||||||
|
///
|
||||||
|
/// # Errors
|
||||||
|
/// [`Error::PathSegment`] when `hive` contains anything but `[A-Za-z0-9_-]`,
|
||||||
|
/// which is what keeps one hive's name from addressing another hive's secret.
|
||||||
|
pub fn hive_token_path(hive: &str) -> Result<String, Error> {
|
||||||
|
let prefix = principal_prefix(Kind::Hive, hive)?;
|
||||||
|
Ok(format!("{prefix}/forge/token"))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What an agent's forge-token path, and a hive's [`hive_token_path`], holds.
|
||||||
///
|
///
|
||||||
/// No `Debug` derive: `value` is a live forge credential, and a derived
|
/// No `Debug` derive: `value` is a live forge credential, and a derived
|
||||||
/// `Debug` is one `{:?}` away from a log line.
|
/// `Debug` is one `{:?}` away from a log line.
|
||||||
|
|
@ -242,6 +272,44 @@ mod tests {
|
||||||
assert!(!covered, "argus's policy must not reach atlas's token");
|
assert!(!covered, "argus's policy must not reach atlas's token");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn the_hive_token_lands_under_the_hive_prefix() {
|
||||||
|
// Spelled out: the controller's bao stanza names the same leaf.
|
||||||
|
assert_eq!(
|
||||||
|
hive_token_path("pr1ma").expect("a plain name is legal"),
|
||||||
|
"swarm/hives/pr1ma/forge/token"
|
||||||
|
);
|
||||||
|
assert_eq!(hive_login("pr1ma"), "hive-pr1ma");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_traversal_in_the_hive_name_is_refused() {
|
||||||
|
for bad in ["../beta", "a/b", "a.b", ""] {
|
||||||
|
let e = hive_token_path(bad).expect_err("a traversal is not legal");
|
||||||
|
assert!(matches!(e, Error::PathSegment { kind: "hive", .. }), "{e}");
|
||||||
|
}
|
||||||
|
// The control: the legal charset stays reachable.
|
||||||
|
assert!(hive_token_path("a-b_C9").is_ok());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_hive_reaches_its_own_forge_token_and_no_other_hives() {
|
||||||
|
let covered_by = |hive: &str, path: &str| {
|
||||||
|
crate::policy::render(hive)
|
||||||
|
.expect("legal")
|
||||||
|
.lines()
|
||||||
|
.any(|line| {
|
||||||
|
line.strip_prefix("path \"")
|
||||||
|
.and_then(|rest| rest.split_once("\" {"))
|
||||||
|
.and_then(|(p, _)| p.strip_suffix('*'))
|
||||||
|
.is_some_and(|prefix| format!("secret/data/{path}").starts_with(prefix))
|
||||||
|
})
|
||||||
|
};
|
||||||
|
let alpha = hive_token_path("alpha").expect("legal");
|
||||||
|
assert!(covered_by("alpha", &alpha), "alpha cannot read {alpha}");
|
||||||
|
assert!(!covered_by("beta", &alpha), "beta reaches {alpha}");
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn the_value_field_matches_what_the_nix_reader_asks_for() {
|
fn the_value_field_matches_what_the_nix_reader_asks_for() {
|
||||||
let json = serde_json::to_string(&Credential {
|
let json = serde_json::to_string(&Credential {
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue