Every run is recorded when it ends: a try-out when its reply settles it, a requested review once its review is posted or the post fails, and a request turned away by a full queue once its comment is posted. A duplicate request that starts nothing is not recorded. The newest 100 are one JSON array under `recent` in a new controller-owned KV bucket, `review-classifier-runs`, written with compare-and-set as `schedule-definitions` is (five attempts). A failed write is logged; the review is already posted by then. Each recorded run is also one info event, "review classifier run finished", with the record's fields as tracing fields. The controller logs natively to the journal, so they reach VictoriaLogs as F_<NAME>. GET /api/review-classifier/runs returns the list, newest first, gated by the gateway like the try-out GET. post_classifier_review now returns the posted review's html_url, which the record keeps as review_url. Closes #5153, part of #5096.
326 lines
13 KiB
Rust
326 lines
13 KiB
Rust
//! The review classifier's forge account, [`LOGIN`]: created here with the
|
|
//! admin API and given one token, stored at
|
|
//! `swarm_secret_client::review_classifier::forge_token_path`. Requesting
|
|
//! this account as a reviewer starts a classifier run, and this daemon posts
|
|
//! the result as the account's review (`crate::review_classifier::review`).
|
|
//! The classifier itself never holds the token.
|
|
//!
|
|
//! The account gets no repo access here. A repo admin adds it to a repo by
|
|
//! hand: read to request it as a reviewer, write (or a place on the branch
|
|
//! protection's approvals whitelist) for its approval to count.
|
|
//!
|
|
//! The token decision is [`super::agent_token`]'s, under this module's name
|
|
//! and scopes, so a stored token the forge still lists with the same last
|
|
//! eight characters and scopes is kept, and a pass whose reads fail queues
|
|
//! nothing.
|
|
|
|
use std::sync::Arc;
|
|
|
|
use anyhow::{Context, Result, bail};
|
|
use forgejo_api::structs::{CreatePullReviewOptions, RepoSearchQuery};
|
|
use forgejo_api::{Auth, Forgejo};
|
|
use swarm_secret_client::forge;
|
|
use swarm_secret_client::review_classifier::forge_token_path;
|
|
|
|
use super::Client;
|
|
use super::agent_token::{Decision, Observed, classify_named};
|
|
|
|
/// The classifier's forge login: the reviewer someone requests.
|
|
pub const LOGIN: &str = "review-classifier";
|
|
|
|
/// The forge's name for the account's one token.
|
|
pub const TOKEN_NAME: &str = "swarm-review-classifier";
|
|
|
|
/// What the token may do: post reviews, which the forge files under the
|
|
/// repository scope. Which repos is the account's access, granted by hand.
|
|
pub const TOKEN_SCOPES: &str = "write:repository";
|
|
|
|
/// How often [`spawn`] re-checks the account and its token.
|
|
const RECONCILE_INTERVAL: std::time::Duration = std::time::Duration::from_mins(5);
|
|
|
|
/// The email this module creates the account with, and the one it requires
|
|
/// before acting on an account named [`LOGIN`]. Under its own domain, so it
|
|
/// equals no agent's or hive's account email.
|
|
const EMAIL: &str = "review-classifier@services.hyperhive.local";
|
|
|
|
/// Whether a pass that found `seen` queues a `MintClassifierForgeToken` job.
|
|
/// [`Observed::Unknown`] never does, so a forge or store outage can't rotate
|
|
/// the token, and neither does a [`Decision::Conflict`], which the job would
|
|
/// refuse.
|
|
pub fn needs_job(seen: Observed) -> bool {
|
|
matches!(
|
|
seen,
|
|
Observed::NoForgeUser | Observed::Decided(Decision::Mint(_))
|
|
)
|
|
}
|
|
|
|
/// One review as the classifier's account posts it.
|
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
|
pub struct Review {
|
|
/// `APPROVED`, `REQUEST_CHANGES` or `COMMENT`.
|
|
pub event: &'static str,
|
|
pub body: String,
|
|
}
|
|
|
|
impl Client {
|
|
/// Make sure [`LOGIN`] exists and holds a live [`TOKEN_NAME`] token in
|
|
/// the store, minting one when it does not. The whole job of the
|
|
/// `MintClassifierForgeToken` node.
|
|
///
|
|
/// # Errors
|
|
/// When the store or the forge refuses a step, when an account named
|
|
/// [`LOGIN`] exists with another email, or when the read-back does not
|
|
/// show the new token.
|
|
pub async fn ensure_classifier_forge_token(&self) -> Result<()> {
|
|
let path = forge_token_path()?;
|
|
self.ensure_classifier_user().await?;
|
|
let store = crate::store::connect()
|
|
.await
|
|
.context("logging in to the swarm secret store")?;
|
|
let (stored, listed) = self.read_classifier_token(&store, &path).await?;
|
|
let reason = match classify_named(TOKEN_NAME, TOKEN_SCOPES, stored.as_ref(), &listed) {
|
|
Decision::Keep => {
|
|
tracing::debug!("review classifier forge token is current; left as it is");
|
|
return Ok(());
|
|
}
|
|
Decision::Conflict => bail!(
|
|
"the forge lists more than one {TOKEN_NAME:?} token for {LOGIN:?}; delete them \
|
|
by id and re-run"
|
|
),
|
|
Decision::Mint(reason) => reason,
|
|
};
|
|
let credential = self
|
|
.replace_token(&store, LOGIN, TOKEN_NAME, TOKEN_SCOPES, &listed, &path)
|
|
.await?;
|
|
|
|
Forgejo::new(Auth::Token(&credential.value), self.url.clone())
|
|
.context("building a forge client with the new token")?
|
|
.repo_search(RepoSearchQuery::default())
|
|
.await
|
|
.with_context(|| format!("using {LOGIN}'s new forge token"))?;
|
|
let (stored, listed) = self.read_classifier_token(&store, &path).await?;
|
|
let now = classify_named(TOKEN_NAME, TOKEN_SCOPES, stored.as_ref(), &listed);
|
|
if now != Decision::Keep {
|
|
bail!("{LOGIN}'s new forge token does not read back as current: {now:?}");
|
|
}
|
|
tracing::info!(?reason, %path, "review classifier forge token minted and stored");
|
|
Ok(())
|
|
}
|
|
|
|
/// Post `review` on PR `number` of `owner/repo` as [`LOGIN`], on
|
|
/// `commit` when one is given. Reads the token from the store on every
|
|
/// call. The posted review's URL, when the forge names one.
|
|
///
|
|
/// # Errors
|
|
/// When the store holds no token yet, or the forge refuses the review.
|
|
pub async fn post_classifier_review(
|
|
&self,
|
|
owner: &str,
|
|
repo: &str,
|
|
number: i64,
|
|
commit: Option<&str>,
|
|
review: Review,
|
|
) -> Result<Option<String>> {
|
|
let path = forge_token_path()?;
|
|
let store = crate::store::connect()
|
|
.await
|
|
.context("logging in to the swarm secret store")?;
|
|
let Some(credential) = store
|
|
.read_optional::<forge::Credential>(&path)
|
|
.await
|
|
.with_context(|| format!("reading {path}"))?
|
|
else {
|
|
bail!("nothing at {path}: the classifier's forge token is not minted yet");
|
|
};
|
|
Forgejo::new(Auth::Token(&credential.value), self.url.clone())
|
|
.context("building a forge client with the classifier's token")?
|
|
.repo_create_pull_review(
|
|
owner,
|
|
repo,
|
|
number,
|
|
CreatePullReviewOptions {
|
|
body: Some(review.body),
|
|
comments: None,
|
|
commit_id: commit.map(str::to_owned),
|
|
event: Some(review.event.to_owned()),
|
|
},
|
|
)
|
|
.await
|
|
.with_context(|| format!("posting {LOGIN}'s review on PR {number} of {owner}/{repo}"))
|
|
.map(|posted| posted.html_url.map(String::from))
|
|
}
|
|
|
|
/// The stored token at `path` and every token the forge lists for
|
|
/// [`LOGIN`], which must exist.
|
|
async fn read_classifier_token(
|
|
&self,
|
|
store: &swarm_secret_client::SecretStore,
|
|
path: &str,
|
|
) -> Result<(
|
|
Option<forge::Credential>,
|
|
Vec<forgejo_api::structs::AccessToken>,
|
|
)> {
|
|
let stored = store
|
|
.read_optional(path)
|
|
.await
|
|
.with_context(|| format!("reading {path}"))?;
|
|
let Some(listed) = self.list_agent_tokens(LOGIN).await? else {
|
|
bail!("the forge has no user {LOGIN:?} after this job created it");
|
|
};
|
|
Ok((stored, listed))
|
|
}
|
|
|
|
/// Create [`LOGIN`] if there is none, refuse one with another email, and
|
|
/// lock it out of creating repos.
|
|
async fn ensure_classifier_user(&self) -> Result<()> {
|
|
self.create_user(LOGIN, EMAIL).await?;
|
|
let user = self
|
|
.api
|
|
.user_get(LOGIN)
|
|
.await
|
|
.with_context(|| format!("read forge user {LOGIN}"))?;
|
|
if user.email.as_deref() != Some(EMAIL) {
|
|
bail!(
|
|
"forge account `{LOGIN}` exists but is not the review classifier's (email {:?}, \
|
|
expected `{EMAIL}`); refusing to mint a token for it",
|
|
user.email
|
|
);
|
|
}
|
|
self.api
|
|
.admin_edit_user(LOGIN, super::repo_creation_lockdown(LOGIN))
|
|
.await
|
|
.with_context(|| format!("disable repo creation for forge user {LOGIN}"))?;
|
|
Ok(())
|
|
}
|
|
|
|
/// What the store and the forge say about the account's token.
|
|
async fn observe_classifier(&self) -> Result<Observed> {
|
|
let path = forge_token_path()?;
|
|
let store = crate::store::connect()
|
|
.await
|
|
.context("logging in to the swarm secret store")?;
|
|
let stored = match store.read_optional::<forge::Credential>(&path).await {
|
|
Ok(stored) => stored,
|
|
Err(e) => {
|
|
tracing::warn!(error = %e, "review classifier forge token: store read failed");
|
|
return Ok(Observed::Unknown);
|
|
}
|
|
};
|
|
Ok(match self.list_agent_tokens(LOGIN).await {
|
|
Ok(Some(listed)) => Observed::Decided(classify_named(
|
|
TOKEN_NAME,
|
|
TOKEN_SCOPES,
|
|
stored.as_ref(),
|
|
&listed,
|
|
)),
|
|
Ok(None) => Observed::NoForgeUser,
|
|
Err(e) => {
|
|
tracing::warn!(error = %format!("{e:#}"), "review classifier forge token: forge read failed");
|
|
Observed::Unknown
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
/// Check the account now and every [`RECONCILE_INTERVAL`] after, and call
|
|
/// `enqueue`, which inserts a `MintClassifierForgeToken` node, when it needs
|
|
/// a job. A pass that fails is logged and retried on the next tick.
|
|
pub fn spawn(client: Arc<Client>, enqueue: impl Fn() + Send + 'static) {
|
|
tokio::spawn(async move {
|
|
let mut ticker = tokio::time::interval(RECONCILE_INTERVAL);
|
|
loop {
|
|
ticker.tick().await;
|
|
let Some(outcome) = super::within_forge_deadline(
|
|
"review classifier forge token",
|
|
client.observe_classifier(),
|
|
)
|
|
.await
|
|
else {
|
|
continue;
|
|
};
|
|
match outcome {
|
|
Ok(seen) if needs_job(seen) => {
|
|
tracing::info!(?seen, "review classifier forge token: queueing a job");
|
|
enqueue();
|
|
}
|
|
Ok(seen) => tracing::debug!(?seen, "review classifier forge token: no job"),
|
|
Err(e) => tracing::warn!(
|
|
error = %format!("{e:#}"),
|
|
retry_in_s = RECONCILE_INTERVAL.as_secs(),
|
|
"review classifier forge token: pass failed; retrying next tick"
|
|
),
|
|
}
|
|
}
|
|
});
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::super::agent_token::MintReason;
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn the_token_writes_repositories_and_nothing_else() {
|
|
assert_eq!(TOKEN_SCOPES, "write:repository");
|
|
assert_ne!(TOKEN_NAME, super::super::agent_token::AGENT_TOKEN_NAME);
|
|
assert_ne!(TOKEN_NAME, super::super::hive_token::HIVE_TOKEN_NAME);
|
|
}
|
|
|
|
#[test]
|
|
fn a_mint_or_a_missing_account_queues_and_nothing_else_does() {
|
|
assert!(needs_job(Observed::NoForgeUser));
|
|
assert!(needs_job(Observed::Decided(Decision::Mint(
|
|
MintReason::NotStored
|
|
))));
|
|
assert!(!needs_job(Observed::Decided(Decision::Keep)));
|
|
assert!(!needs_job(Observed::Decided(Decision::Conflict)));
|
|
assert!(!needs_job(Observed::Unknown));
|
|
}
|
|
|
|
#[test]
|
|
fn the_accounts_email_is_no_agents() {
|
|
assert_ne!(EMAIL, super::super::agent_email(LOGIN));
|
|
assert_ne!(EMAIL, super::super::legacy_agent_email(LOGIN));
|
|
}
|
|
|
|
/// An existing account named like the classifier with an agent's email
|
|
/// is refused before the lockdown, so no token is minted for it.
|
|
#[tokio::test]
|
|
async fn the_job_refuses_an_account_that_is_not_the_classifiers() {
|
|
use super::super::tests::{lockdown_patch, stub_forge};
|
|
|
|
let (client, seen) = stub_forge(
|
|
(409, r#"{"message":"user already exists"}"#),
|
|
(200, r#"{"email":"review-classifier@hyperhive.local"}"#),
|
|
(200, "{}"),
|
|
)
|
|
.await;
|
|
let err = client
|
|
.ensure_classifier_user()
|
|
.await
|
|
.expect_err("an agent's account is not the classifier's");
|
|
assert!(format!("{err:#}").contains("not the review classifier's"));
|
|
assert!(lockdown_patch(&seen, LOGIN).is_none());
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn the_job_locks_down_the_classifiers_own_account() {
|
|
use super::super::tests::{lockdown_patch, stub_forge};
|
|
|
|
let (client, seen) = stub_forge(
|
|
(409, r#"{"message":"user already exists"}"#),
|
|
(
|
|
200,
|
|
r#"{"email":"review-classifier@services.hyperhive.local"}"#,
|
|
),
|
|
(200, "{}"),
|
|
)
|
|
.await;
|
|
client
|
|
.ensure_classifier_user()
|
|
.await
|
|
.expect("the classifier's own account");
|
|
let body = lockdown_patch(&seen, LOGIN).expect("no lockdown PATCH sent");
|
|
assert_eq!(body["max_repo_creation"], 0);
|
|
}
|
|
}
|