vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

request the review classifier's review when a PR's head moves

Browse source 
The router gets a request_reviewer node and a pull_request_sync source;
the built-in head-classifier flow asks swarm-controller, over
$SWARM.router.sink.request-reviewer, to request review-classifier on
every push, unless its review of the head already stands. The controller
subscribes to pull_request_sync on its pull-request hook and answers
the router; nats-auth grants the router the publish and the controller
the subscribe.
This commit is contained in:
atlas 2026-10-24 14:28:27 +02:00
commit 0b1669a364
20 changed files with 1231 additions and 64 deletions
Download patch file Download diff file Expand all files Collapse all files

5
docs/swarm/README.md
View file

@ -32,7 +32,8 @@ grafana, the controller, authelia, …) →
its repo's own yes/no rules and posts the result as a review. →
[`review-classifier.md`](review-classifier.md)
- **notification router** — forge events turned into todos on the agents
they concern: assigned, unassigned, cleared on close. →
they concern: assigned, unassigned, cleared on close; and the classifier's
review requested on a push. →
[`notify-router.md`](notify-router.md)
- **swarm UI** — the operator's day-to-day surface, on the swarm apex,
`admins` only. → [`ui.md`](ui.md)
@ -415,7 +416,7 @@ itself, all under `https://<swarm.domain>/webhook/forge/`:
- an instance-wide `pull_request_review_request` system hook, which starts
the [review classifier](review-classifier.md#trigger-and-lifecycle)
- instance-wide `issues` + `issue_assign` and `pull_request` +
`pull_request_assign` system hooks, whose deliveries it
`pull_request_assign` + `pull_request_sync` system hooks, whose deliveries it
keeps in the `SWARM_EVENTS` stream for the
[notification router](notify-router.md#the-path)

82
docs/swarm/notify-router.md
View file

@ -41,8 +41,9 @@ router notices an unassign only on an issue it has seen an assign event for.
1. swarm-controller registers two instance-wide hooks, one for the `issues`
and `issue_assign` events at `/webhook/forge/issues` and one for
`pull_request` and `pull_request_assign` at `/webhook/forge/pull-request`.
At each start it adds any of these events a registered hook lacks.
`pull_request`, `pull_request_assign` and `pull_request_sync` (a push to an
open PR's branch) at `/webhook/forge/pull-request`. At each start it adds
any of these events a registered hook lacks.
2. On each delivery the controller checks the HMAC and publishes the body,
unchanged, to the `SWARM_EVENTS` stream on
`$SWARM.events.forge.<X-Forgejo-Event-Type>`, such as
@ -68,16 +69,19 @@ The router holds no forge token and no webhook secret, and registers no hook.
A flow is a graph of nodes joined by edges: one `source` node naming the forge
event types it starts from, `match` nodes that pass when every condition
holds, and `todo` nodes that emit one todo op per recipient. A match node can
have more than one child, and each one runs, so one event can reach several
todos. A flow must be a tree: one source, one parent for every other node,
every node reachable, no children under a `todo`. The most is 64 nodes, 16
holds, `todo` nodes that emit one todo op per recipient, and `request_reviewer`
nodes that ask for a review. A match node can have more than one child, and
each one runs, so one event can reach several leaves. A flow must be a tree:
one source, one parent for every other node, every node reachable, no children
under a `todo` or a `request_reviewer`. The most is 64 nodes, 16
levels deep, and 100 custom flows. A `todo` node sends to the agents an event
field holds (`assignees_added`, `todo_holders`, …) or to one named agent.
The router ships the three flows above, each a source → match on `action` →
todo. The code defines them (`swarm_wire::notify_flows::builtins`), so a
build that changes one reaches every swarm. A custom flow has the same type and
todo, and `head-classifier`: source `pull_request_sync` → `request_reviewer`
for `review-classifier` ([below](#requesting-a-review)). The code defines them
(`swarm_wire::notify_flows::builtins`), so a build that changes one reaches
every swarm. Each switches off on `/notifications`. A custom flow has the same type and
the same evaluator, so it can do anything a built-in does.
**The stored document.** swarm-controller keeps the rest in one document, the
@ -86,8 +90,8 @@ switched off, and the custom flows. Only the controller writes it, through
`GET` and `PUT /api/notify/flows`. A `PUT` replaces the whole document in one
compare-and-set. The controller answers 400 unless every flow passes the tree
check above and reads only what exists: a source event from `issues`,
`issue_assign`, `pull_request` or `pull_request_assign`, and condition and
recipient fields from the event's field list. The same goes for the `{field}`
`issue_assign`, `pull_request`, `pull_request_assign` or `pull_request_sync`,
and condition and recipient fields from the event's field list. The same goes for the `{field}`
placeholders in a todo's key and text. A `PUT` must carry the `revision` it read, `0` for no
document, and gets 409 when the stored revision is another. Swarm UI's
[`/notifications`](ui.md#notification-flows) page shows the document and
@ -100,8 +104,8 @@ accepts:
- `source_events`: `[{name, description}]`, the source events.
- `fields`: `[{name, description}]`, the condition and recipient fields, and
the `{field}` placeholders.
- `node_kinds`: `[{kind, description, fields}]` for `source`, `match` and
`todo`; each field is `{name, description, required, values}`, and `values`
- `node_kinds`: `[{kind, description, fields}]` for `source`, `match`, `todo`
and `request_reviewer`; each field is `{name, description, required, values}`, and `values`
lists the accepted `{name, description}` pairs (the `upsert` and `clear` of a
todo's `op`), empty for other fields.
- `limits`: `{max_nodes, max_depth, max_flows}`.
@ -109,6 +113,39 @@ accepts:
A controller that predates `schema` leaves it out of the answer; a client reads
it as optional.
**Requesting a review.** A `request_reviewer` node has one field, `reviewer`,
a forge login. For the PR the event concerns (`repo` and `number`; an event
about an issue sends nothing), the router sends a NATS request on
`$SWARM.router.sink.request-reviewer` with `{repo, number, reviewer}` and
swarm-controller answers with one line:
| Reply | Meaning |
| --------------- | ---------------------------------------------------------------------------------- |
| `requested` | the forge accepted the request |
| `skipped:<why>` | the reviewer's review of the PR's head already stands; the controller sent nothing |
| `error:<msg>` | the forge refused or didn't answer, or the request didn't decode |
The controller asks the forge under its own token, which the router never
holds. It skips a reviewer whose latest review is neither stale (the PR's head
moved past the commit it reviewed) nor dismissed, because requesting a review
from someone who already submitted one makes the forge dismiss it, approval
included. The controller asks again once the review is older than the head.
`hive-forge pr assign-reviewer` applies the same rule (`hive_types::review`).
The router sends the request once, after it queued the event's todo ops and
wrote the snapshot, and logs the answer. A controller that's down or answers
`error:` doesn't hold up the event or any agent's todos, and the request is
not retried: the next push asks again. A request repeated for the same head is
harmless: a reviewer with a pending request is a no-op for the forge, and one
who has reviewed gets `skipped`.
`head-classifier` requests the review classifier's review on a push to any open
PR. The classifier then runs as it does for any requested review
([`review-classifier.md`](review-classifier.md#trigger-and-lifecycle)), through
its own request queue, which replaces a waiting request for the same PR. A
repo the classifier's account can't read, or a PR that account opened, answers
`error:`.
**What the router runs.** It reads the key at most every 10 seconds, when an
event arrives, and runs the built-ins you left on, then the custom flows that
are on and pass the tree check. It logs a custom flow that fails the check,
@ -127,20 +164,25 @@ document never leaves it with no flows. The controller refuses to overwrite a
document of a version it doesn't know.
**Deploying in any order.** The controller, the router and the UI can be on
different hosts. A controller that predates this leaves the key absent, and the
different hosts. A router that predates `request_reviewer` reads a document
holding such a node (a custom flow with one) as one it can't read and keeps the
flows it had; it also has no `head-classifier`. A router that has it, with a
controller that predates it, gets no answer to the request and logs that. A controller that predates this leaves the key absent, and the
router runs the built-ins as before. A router that predates this ignores the
document, so the page and the API show a state the router isn't yet running.
## Credentials and grants
| Principal | Grant |
| --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `swarm-notify-router`, in its container | store: `read` on `swarm/services/<router client id>/oidc/client`. Queue: its own consumer on `SWARM_EVENTS` (create, pull, ack), publish on `$SWARM.router.todo.*`, create `ROUTER_TODOS`, its own `notify-router-state` bucket, read on the `notify-flows` bucket, and `STREAM.INFO` on the agent-status bucket, which lists the agents |
| swarm-controller | publish on `$SWARM.events.forge.*`; creates and writes the `notify-flows` bucket (every bucket, by its `$KV.*.>` grant) |
| an agent, under its own credential | create its own consumer `todo-<agent>` in the form whose filter is `$SWARM.router.todo.<agent>`, pull from it, and ack what it delivered |
| Principal | Grant |
| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `swarm-notify-router`, in its container | store: `read` on `swarm/services/<router client id>/oidc/client`. Queue: its own consumer on `SWARM_EVENTS` (create, pull, ack), publish on `$SWARM.router.todo.*` and on `$SWARM.router.sink.request-reviewer`, create `ROUTER_TODOS`, its own `notify-router-state` bucket, read on the `notify-flows` bucket, and `STREAM.INFO` on the agent-status bucket, which lists the agents |
| swarm-controller | publish on `$SWARM.events.forge.*`; subscribe to `$SWARM.router.sink.request-reviewer` and answer each request once, for 30 seconds; creates and writes the `notify-flows` bucket (every bucket, by its `$KV.*.>` grant) |
| an agent, under its own credential | create its own consumer `todo-<agent>` in the form whose filter is `$SWARM.router.todo.<agent>`, pull from it, and ack what it delivered |
No other client may publish on `$SWARM.router.todo.*` or
`$SWARM.events.forge.*`, so each has one writer. The agent's create subject
No other client may publish on `$SWARM.router.todo.*`, `$SWARM.router.sink.request-reviewer` or
`$SWARM.events.forge.*`, so each has one writer, and the controller alone
hears the review requests. The review request needs no secret of its own: the
controller asks the forge under the token it already holds. The agent's create subject
carries the consumer's filter, and the queue server checks it against the
request, so an agent can't create a consumer that delivers another agent's
todos.

4
docs/swarm/review-classifier.md
View file

@ -86,6 +86,10 @@ as a reviewer on a PR starts a run:
4. The controller posts the result on the PR as the service account's
review, on the commit it read.
A push starts no run by itself. The notification router's `head-classifier`
flow requests the account's review on every push to an open PR, unless its
review of the new head already stands ([`notify-router.md`](notify-router.md#requesting-a-review)).
When a post can't reach the secret store or the forge, because the
connection fails or a try gets no answer within two minutes, the controller
tries it up to six times over about five minutes, enough to outlast a

3
docs/swarm/services.md
View file

@ -491,7 +491,8 @@ swarm-controller keeps every `issues` and `pull_request` delivery from its
instance-wide hooks in the `SWARM_EVENTS` stream. The router reads that
stream and queues todos for the agents an event concerns: an agent assigned to
an issue or pull request gets a todo, one taken off it gets that todo rewritten
to say so, and closing it clears them. Each agent's harness takes its todos
to say so, and closing it clears them. A push to an open pull request makes
it ask swarm-controller to request the review classifier's review. Each agent's harness takes its todos
from the `ROUTER_TODOS` stream, so an agent that's down gets them when it's
back.

2
docs/swarm/ui.md
View file

@ -224,7 +224,7 @@ in two groups. Each **built-in** flow has an on/off switch; switching one off
stops the router running it from its next event, and switching it on again
restores it. Each **custom** flow shows its name and whether it's on. Every
flow draws as an indented tree: the forge events it starts from, the
conditions under it, and the todo each branch ends in.
conditions under it, and the todo or review request each branch ends in.
A built-in flow can be **duplicated** into a custom one, and **new flow** starts a
custom flow with just a source; either opens the flow in the editor. A custom

8
frontend/packages/swarm-ui/test/flow-types.test.mjs
View file

@ -73,6 +73,14 @@ test("describeNode: says what each kind does", () => {
"clear todos forge:{repo}#{number}:* of {todo_holders}",
);
assert.equal(describeNode(flow.nodes[4]), "todo to triage, key k: look");
assert.equal(
describeNode({
id: "r",
kind: "request_reviewer",
reviewer: "review-classifier",
}),
"request a review from review-classifier",
);
});
test("withBuiltin: switching off adds once, switching on removes", () => {

11
swarm-controller/src/forge.rs
View file

@ -55,6 +55,7 @@ pub mod legacy_tokens;
pub mod objects;
pub mod operator_profiles;
pub mod pull_diff;
pub mod review_request;
pub mod runner_token;
pub mod site_admin;
@ -1760,12 +1761,12 @@ fn hook_registration(kind: DeliveryKind) -> (&'static [&'static str], HookScope<
// classifier's account access can request its review.
DeliveryKind::ReviewRequest => (&["pull_request_review_request"], HookScope::Instance),
// Instance-wide, for the notification router, which reads the assign
// events. They are named: Forgejo's web form shows assignment as a
// box of its own, so a hook edited there can drop it while keeping
// `issues` or `pull_request`.
// events and a push to a PR's branch. They are named: Forgejo's web
// form shows each as a box of its own, so a hook edited there can drop
// one while keeping `issues` or `pull_request`.
DeliveryKind::Issues => (&["issues", "issue_assign"], HookScope::Instance),
DeliveryKind::PullRequest => (
&["pull_request", "pull_request_assign"],
&["pull_request", "pull_request_assign", "pull_request_sync"],
HookScope::Instance,
),
}
@ -2111,7 +2112,7 @@ mod tests {
assert_eq!(events(DeliveryKind::Issues), ["issues", "issue_assign"]);
assert_eq!(
events(DeliveryKind::PullRequest),
["pull_request", "pull_request_assign"]
["pull_request", "pull_request_assign", "pull_request_sync"]
);
// The events a stored flow's source may name are the ones the hooks
// deliver to the router.

194
swarm-controller/src/forge/review_request.rs Normal file
View file

@ -0,0 +1,194 @@
//! A PR's reviews, and asking for one more, under the controller's token.
//!
//! The router's `request_reviewer` sink reaches the forge through here
//! ([`crate::request_reviewer`]). Which reviews still stand is
//! [`hive_types::review`], the rule `hive-forge pr assign-reviewer` applies
//! too.
use anyhow::{Context, Result};
use forgejo_api::structs::PullReviewRequestOptions;
use hive_types::review::{Listed, ReviewInfo, latest_reviews};
use super::Client;
/// One review listing takes this many pages at most, 50 reviews a page.
const MAX_PAGES: u32 = 10;
const PAGE_SIZE: u32 = 50;
impl Client {
/// The latest verdict of each reviewer of PR `number` of `owner/repo`,
/// judged against the PR's head as it is now.
///
/// # Errors
/// A forge failure, with the [`forgejo_api::ForgejoError`] underneath.
pub async fn pull_reviews(
&self,
owner: &str,
repo: &str,
number: i64,
) -> Result<Vec<ReviewInfo>> {
let pull = self
.api
.call(|api| api.repo_get_pull_request(owner, repo, number).into_future())
.await
.with_context(|| format!("read PR {number} of {owner}/{repo}"))?;
let head = pull.head.and_then(|h| h.sha);
let mut reviews = Vec::new();
for page in 1..=MAX_PAGES {
let (_, batch) = self
.api
.call(|api| {
api.repo_list_pull_reviews(owner, repo, number)
.page(page)
.page_size(PAGE_SIZE)
.into_future()
})
.await
.with_context(|| format!("list the reviews of PR {number} of {owner}/{repo}"))?;
let short = batch.len() < PAGE_SIZE as usize;
reviews.extend(batch);
if short {
break;
}
}
Ok(latest_reviews(
reviews.iter().map(|r| Listed {
login: r.user.as_ref().and_then(|u| u.login.as_deref()),
state: r.state.as_deref(),
commit_id: r.commit_id.as_deref(),
stale: r.stale,
dismissed: r.dismissed,
}),
head.as_deref(),
))
}
/// Ask `reviewer` to review PR `number` of `owner/repo`. From a reviewer
/// whose review stands, the forge dismisses it: check
/// [`hive_types::review::standing`] first.
///
/// # Errors
/// The forge refused: no such reviewer, the PR's author, a reviewer with
/// no access to the repo.
pub async fn request_review(
&self,
owner: &str,
repo: &str,
number: i64,
reviewer: &str,
) -> Result<()> {
let body = PullReviewRequestOptions {
reviewers: Some(vec![reviewer.to_owned()]),
team_reviewers: None,
};
self.api
.call(|api| {
api.repo_create_pull_review_requests(owner, repo, number, body.clone())
.into_future()
})
.await
.with_context(|| {
format!("request {reviewer}'s review of PR {number} of {owner}/{repo}")
})?;
Ok(())
}
}
#[cfg(test)]
mod tests {
use std::sync::{Arc, Mutex};
use axum::http::{Method, StatusCode, Uri};
use super::*;
/// A stand-in forge holding PR 7 of `o/r` at head `new`, with `reviews`
/// listed, answering the request post with `post`. Records each request as
/// method, path and body.
async fn forge(
reviews: &'static str,
post: u16,
) -> (Client, Arc<Mutex<Vec<(String, String, String)>>>) {
let seen = Arc::new(Mutex::new(Vec::new()));
let recorded = Arc::clone(&seen);
let app = axum::Router::new().fallback(
move |method: Method, uri: Uri, body: axum::body::Bytes| {
let recorded = Arc::clone(&recorded);
async move {
recorded.lock().unwrap().push((
method.to_string(),
uri.path().to_owned(),
String::from_utf8_lossy(&body).into_owned(),
));
match (method.as_str(), uri.path()) {
("GET", "/api/v1/repos/o/r/pulls/7") => (
StatusCode::OK,
r#"{"number":7,"head":{"sha":"new"},"requested_reviewers":[]}"#
.to_owned(),
),
("GET", "/api/v1/repos/o/r/pulls/7/reviews") => {
(StatusCode::OK, reviews.to_owned())
}
("POST", "/api/v1/repos/o/r/pulls/7/requested_reviewers") => (
StatusCode::from_u16(post).unwrap(),
if post == 201 {
"[]".to_owned()
} else {
r#"{"message":"refused"}"#.to_owned()
},
),
_ => (StatusCode::NOT_FOUND, r#"{"message":"no"}"#.to_owned()),
}
}
},
);
let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap();
let url = url::Url::parse(&format!("http://{}/", listener.local_addr().unwrap())).unwrap();
tokio::spawn(async move { axum::serve(listener, app).await });
(Client::for_test_forge(url), seen)
}
const REVIEWS: &str = r#"[
{"id":1,"user":{"login":"review-classifier"},"state":"APPROVED","commit_id":"old","stale":false,"dismissed":false},
{"id":2,"user":{"login":"argus"},"state":"APPROVED","commit_id":"new","stale":false,"dismissed":false}
]"#;
/// The head is read from the PR, so a review of an older commit is
/// superseded though the forge still reports `stale: false`.
#[tokio::test]
async fn reviews_are_judged_against_the_head_read_now() {
let (client, _) = forge(REVIEWS, 201).await;
let reviews = client.pull_reviews("o", "r", 7).await.unwrap();
let standing = |login| hive_types::review::standing(&reviews, login).is_some();
assert!(
!standing("review-classifier"),
"reviewed `old`, head is `new`"
);
assert!(standing("argus"), "control: reviewed the head");
}
#[tokio::test]
async fn a_request_posts_the_reviewer_to_the_pull_requests_reviewer_list() {
let (client, seen) = forge("[]", 201).await;
client
.request_review("o", "r", 7, "review-classifier")
.await
.unwrap();
let seen = seen.lock().unwrap();
let (method, path, body) = seen.last().unwrap();
assert_eq!(method, "POST");
assert_eq!(path, "/api/v1/repos/o/r/pulls/7/requested_reviewers");
let body: serde_json::Value = serde_json::from_str(body).unwrap();
assert_eq!(body["reviewers"], serde_json::json!(["review-classifier"]));
}
#[tokio::test]
async fn a_refused_request_is_an_error_naming_the_reviewer() {
let (client, _) = forge("[]", 422).await;
let err = client
.request_review("o", "r", 7, "review-classifier")
.await
.unwrap_err();
assert!(format!("{err:#}").contains("review-classifier"), "{err:#}");
}
}

5
swarm-controller/src/main.rs
View file

@ -55,6 +55,7 @@ mod otel_http_client;
mod pulls;
mod queue_identity;
mod read_policy;
mod request_reviewer;
mod review_classifier;
mod routes;
mod schedules;
@ -239,9 +240,11 @@ async fn main() -> Result<()> {
serve(listener, state).await
}
/// Start taking the queued review requests, then serve `state` on `listener`.
/// Start taking the queued review requests and answering the notification
/// router's, then serve `state` on `listener`.
async fn serve(listener: tokio::net::UnixListener, state: AppState) -> Result<()> {
review_classifier::review::spawn_queue(&state);
request_reviewer::spawn(&state);
axum::serve(listener, build_app(state))
.await
.context("serving swarm-controller")

5
swarm-controller/src/notify_flows.rs
View file

@ -294,7 +294,8 @@ mod tests {
let Json(view) = get_flows(State(state.clone())).await.expect("get");
assert_eq!(view.revision, 0);
assert!(view.disabled_builtins.is_empty() && view.flows.is_empty());
assert_eq!(view.builtins.len(), 3);
assert_eq!(view.builtins.len(), 4);
assert!(view.builtins.iter().any(|f| f.id == "head-classifier"));
let json = serde_json::to_value(&view).expect("serializes");
assert_eq!(json["schema"]["limits"]["max_nodes"], 64);
assert_eq!(json["schema"]["node_kinds"][0]["kind"], "source");
@ -321,7 +322,7 @@ mod tests {
.expect("read")
.expect("stored");
let doc = Document::decode(&bytes).expect("the router reads it");
assert_eq!(doc.active().0.len(), 3, "2 built-ins and the custom flow");
assert_eq!(doc.active().0.len(), 4, "3 built-ins and the custom flow");
// A refused save changes nothing: an orphan node, and a built-in that does not exist.
let mut orphan = custom("orphan");

388
swarm-controller/src/request_reviewer.rs Normal file
View file

@ -0,0 +1,388 @@
//! Requesting a review for the notification router, which holds no forge
//! token.
//!
//! The router sends a [`RequestReviewer`] on
//! `swarm_wire::router_sink::REQUEST_REVIEWER_SUBJECT` and [`spawn`]'s
//! responder answers with an [`Outcome`] on the reply subject, having acted
//! under the controller's token. A reviewer whose review of the PR's head
//! already stands is skipped: requesting it again makes the forge dismiss it,
//! approval included. What stands is [`hive_types::review`], the rule
//! `hive-forge pr assign-reviewer` applies too.
//!
//! The request may be repeated (the router's event comes back after a failed
//! step): a reviewer with a request pending is a no-op for the forge, and one
//! who has reviewed since stands and is skipped.
use std::future::Future;
use std::sync::Arc;
use std::time::Duration;
use futures_util::StreamExt as _;
use hive_types::review::{ReviewInfo, standing};
use swarm_wire::router_sink::{Outcome, REQUEST_REVIEWER_SUBJECT, RequestReviewer, WORK_DEADLINE};
use crate::forge;
use crate::jobq::Late;
use crate::review_classifier::split_repo;
use crate::routes::AppState;
/// How long to wait before subscribing again.
const RETRY: Duration = Duration::from_secs(5);
/// The forge as the responder uses it: [`forge::Client`] in production, a fake
/// in the tests.
pub(crate) trait ReviewForge {
fn reviews(
&self,
owner: &str,
repo: &str,
number: i64,
) -> impl Future<Output = anyhow::Result<Vec<ReviewInfo>>> + Send;
fn request(
&self,
owner: &str,
repo: &str,
number: i64,
reviewer: &str,
) -> impl Future<Output = anyhow::Result<()>> + Send;
}
impl ReviewForge for forge::Client {
async fn reviews(
&self,
owner: &str,
repo: &str,
number: i64,
) -> anyhow::Result<Vec<ReviewInfo>> {
self.pull_reviews(owner, repo, number).await
}
async fn request(
&self,
owner: &str,
repo: &str,
number: i64,
reviewer: &str,
) -> anyhow::Result<()> {
self.request_review(owner, repo, number, reviewer).await
}
}
/// Answer the router's review requests for as long as the controller runs.
pub(crate) fn spawn(state: &AppState) {
let Some(client) = state.status.as_ref().map(|s| s.queue_client()) else {
tracing::warn!("no swarm queue is configured on this host; review requests go unanswered");
return;
};
let forge = Arc::clone(&state.forge);
tokio::spawn(async move { follow(&client, &forge).await });
}
/// Subscribe to the requests and answer each, and subscribe again if the
/// subscription ends.
async fn follow<F: ReviewForge>(client: &async_nats::Client, forge: &Late<Arc<F>>) {
loop {
match client.subscribe(REQUEST_REVIEWER_SUBJECT).await {
Ok(mut requests) => {
tracing::info!(
subject = REQUEST_REVIEWER_SUBJECT,
"request reviewer: answering requests"
);
while let Some(msg) = requests.next().await {
let Some(reply) = msg.reply.clone() else {
continue;
};
let outcome = answer(forge.get().map(Arc::as_ref), &msg.payload).await;
match &outcome {
Outcome::Error(why) => {
tracing::warn!(%why, "request reviewer: not requested");
}
other => {
tracing::info!(outcome = %other.encode(), "request reviewer: answered");
}
}
if let Err(e) = client.publish(reply, outcome.encode().into()).await {
tracing::warn!(error = %e, "request reviewer: answering failed");
}
}
tracing::warn!("request reviewer: subscription ended");
}
Err(e) => tracing::warn!(error = %e, "request reviewer: subscribe failed"),
}
tokio::time::sleep(RETRY).await;
}
}
/// What to answer to one request `payload`, `forge` being `None` while the
/// controller has no forge access.
async fn answer<F: ReviewForge>(forge: Option<&F>, payload: &[u8]) -> Outcome {
let request: RequestReviewer = match serde_json::from_slice(payload) {
Ok(request) => request,
Err(e) => return Outcome::Error(format!("the request does not decode: {e}")),
};
let Some(forge) = forge else {
return Outcome::Error("this controller has no forge access yet".to_owned());
};
match tokio::time::timeout(WORK_DEADLINE, act(forge, &request)).await {
Ok(outcome) => outcome,
Err(_) => Outcome::Error(format!(
"the forge did not answer within {}s",
WORK_DEADLINE.as_secs()
)),
}
}
async fn act<F: ReviewForge>(forge: &F, request: &RequestReviewer) -> Outcome {
let RequestReviewer {
repo,
number,
reviewer,
} = request;
let (owner, name) = match split_repo(repo) {
Ok(split) => split,
Err(why) => return Outcome::Error(why),
};
let Ok(number_i64) = i64::try_from(*number) else {
return Outcome::Error(format!("PR number {number} is out of range"));
};
if reviewer.trim().is_empty() {
return Outcome::Error("no reviewer named".to_owned());
}
let reviews = match forge.reviews(owner, name, number_i64).await {
Ok(reviews) => reviews,
Err(e) => return Outcome::Error(format!("{e:#}")),
};
if let Some(review) = standing(&reviews, reviewer) {
return Outcome::Skipped(format!(
"{reviewer} already has a standing {} review on {repo}#{number}",
review.state
));
}
match forge.request(owner, name, number_i64, reviewer).await {
Ok(()) => Outcome::Requested,
Err(e) => Outcome::Error(format!("{e:#}")),
}
}
#[cfg(test)]
mod tests {
use std::sync::Mutex;
use super::*;
#[derive(Default)]
struct Fake {
reviews: Vec<ReviewInfo>,
reviews_fail: bool,
request_fail: bool,
asked: Mutex<Vec<(String, i64, String)>>,
}
impl ReviewForge for Fake {
async fn reviews(&self, _: &str, _: &str, _: i64) -> anyhow::Result<Vec<ReviewInfo>> {
if self.reviews_fail {
anyhow::bail!("the forge is down");
}
Ok(self.reviews.clone())
}
async fn request(
&self,
owner: &str,
repo: &str,
number: i64,
reviewer: &str,
) -> anyhow::Result<()> {
if self.request_fail {
anyhow::bail!("422: the reviewer is the PR's author");
}
self.asked.lock().unwrap().push((
format!("{owner}/{repo}"),
number,
reviewer.to_owned(),
));
Ok(())
}
}
fn review(login: &str, verdict: &str, stale: bool, dismissed: bool) -> ReviewInfo {
ReviewInfo {
login: login.to_owned(),
state: verdict.to_owned(),
stale,
dismissed,
}
}
fn payload(repo: &str, number: u64, reviewer: &str) -> Vec<u8> {
serde_json::to_vec(&RequestReviewer {
repo: repo.to_owned(),
number,
reviewer: reviewer.to_owned(),
})
.unwrap()
}
fn asked(fake: &Fake) -> Vec<(String, i64, String)> {
fake.asked.lock().unwrap().clone()
}
#[tokio::test]
async fn a_reviewer_with_no_review_is_requested() {
let fake = Fake::default();
let got = answer(Some(&fake), &payload("o/r", 7, "review-classifier")).await;
assert_eq!(got, Outcome::Requested);
assert_eq!(
asked(&fake),
[("o/r".to_owned(), 7, "review-classifier".to_owned())]
);
}
/// A review that still stands is skipped, because requesting from its
/// reviewer would make the forge dismiss it.
#[tokio::test]
async fn a_standing_review_is_skipped_and_a_stale_or_dismissed_one_is_requested_again() {
let reviewer = "review-classifier";
let standing = Fake {
reviews: vec![review(reviewer, "APPROVED", false, false)],
..Fake::default()
};
let got = answer(Some(&standing), &payload("o/r", 7, reviewer)).await;
assert!(
matches!(&got, Outcome::Skipped(why) if why.contains("APPROVED") && why.contains("o/r#7")),
"{got:?}"
);
assert!(asked(&standing).is_empty(), "nothing was requested");
for (stale, dismissed) in [(true, false), (false, true)] {
let fake = Fake {
reviews: vec![review(reviewer, "APPROVED", stale, dismissed)],
..Fake::default()
};
let got = answer(Some(&fake), &payload("o/r", 7, reviewer)).await;
assert_eq!(
got,
Outcome::Requested,
"stale {stale}, dismissed {dismissed}"
);
assert_eq!(asked(&fake).len(), 1);
}
// Another reviewer's standing review is no reason to skip this one.
let other = Fake {
reviews: vec![review("argus", "APPROVED", false, false)],
..Fake::default()
};
let got = answer(Some(&other), &payload("o/r", 7, reviewer)).await;
assert_eq!(got, Outcome::Requested);
}
#[tokio::test]
async fn a_forge_failure_is_an_error_and_a_failed_read_requests_nothing() {
let down = Fake {
reviews_fail: true,
..Fake::default()
};
let got = answer(Some(&down), &payload("o/r", 7, "review-classifier")).await;
assert!(
matches!(&got, Outcome::Error(e) if e.contains("the forge is down")),
"{got:?}"
);
assert!(asked(&down).is_empty());
let refused = Fake {
request_fail: true,
..Fake::default()
};
let got = answer(Some(&refused), &payload("o/r", 7, "review-classifier")).await;
assert!(
matches!(&got, Outcome::Error(e) if e.contains("422")),
"{got:?}"
);
}
#[tokio::test]
async fn a_request_that_is_malformed_or_has_no_forge_is_an_error() {
let fake = Fake::default();
for bad in [
b"not json".to_vec(),
payload("no-slash", 7, "review-classifier"),
payload("o/r", 7, " "),
payload("o/r", u64::MAX, "review-classifier"),
] {
let got = answer(Some(&fake), &bad).await;
assert!(
matches!(got, Outcome::Error(_)),
"{:?}",
String::from_utf8_lossy(&bad)
);
}
assert!(asked(&fake).is_empty());
let none: Option<&Fake> = None;
let got = answer(none, &payload("o/r", 7, "review-classifier")).await;
assert!(
matches!(&got, Outcome::Error(e) if e.contains("no forge")),
"{got:?}"
);
}
/// The built-in flow's reviewer is the account the controller makes.
#[test]
fn the_built_in_flow_requests_the_classifiers_account() {
assert_eq!(
swarm_wire::notify_flows::CLASSIFIER_ACCOUNT,
forge::classifier_account::LOGIN
);
}
/// Against a real server: a request sent the way the router sends it comes
/// back with the outcome, and a subscriber before the controller is up
/// leaves the sender with no responders.
#[tokio::test]
async fn a_request_over_the_queue_is_answered_with_its_outcome() {
let nats = crate::test_nats::TestNats::start();
let router = nats.client().await;
let ask = |body: Vec<u8>| {
let router = router.clone();
async move {
let sent = async_nats::Request::new()
.payload(body.into())
.timeout(Some(Duration::from_secs(5)));
router.send_request(REQUEST_REVIEWER_SUBJECT, sent).await
}
};
assert!(
ask(payload("o/r", 7, "review-classifier")).await.is_err(),
"no controller listening"
);
let fake = Arc::new(Fake {
reviews: vec![review("argus", "APPROVED", false, false)],
..Fake::default()
});
let late: Late<Arc<Fake>> = Arc::default();
late.set(Arc::clone(&fake)).ok().unwrap();
let controller = nats.client().await;
tokio::spawn(async move { follow(&controller, &late).await });
// The subscription is processed once a request gets through.
let mut reply = None;
for _ in 0..50 {
if let Ok(msg) = ask(payload("o/r", 7, "review-classifier")).await {
reply = Some(msg);
break;
}
tokio::time::sleep(Duration::from_millis(100)).await;
}
let reply = reply.expect("the controller answers");
assert_eq!(
Outcome::decode(&String::from_utf8_lossy(&reply.payload)),
Outcome::Requested
);
let skipped = ask(payload("o/r", 7, "argus")).await.unwrap();
assert!(matches!(
Outcome::decode(&String::from_utf8_lossy(&skipped.payload)),
Outcome::Skipped(_)
));
}
}

9
swarm-controller/src/review_classifier/review.rs
View file

@ -2,10 +2,11 @@
//! ([`LOGIN`]) runs the repo's rules against the PR and posts the result as
//! that account's review, on the commit the run read.
//!
//! Only a review request starts a run; a push does not. A fresh request
//! retires the account's earlier review: the forge clears the `official` flag
//! on a reviewer's previous reviews when they are requested again, and only
//! official reviews count toward approvals or block a merge.
//! Only a review request starts a run. A push makes none itself: the
//! notification router's `head-classifier` flow requests the review. A fresh
//! request retires the account's earlier review: the forge clears the
//! `official` flag on a reviewer's previous reviews when they are requested
//! again, and only official reviews count toward approvals or block a merge.
//!
//! A request for a head whose newest recorded review approved or requested
//! changes runs nothing: the account posts that verdict again, saying so. A

56
swarm-nats-auth/src/policy.rs
View file

@ -293,6 +293,10 @@ impl Policy {
/// controller creates the bucket and is its one writer.
/// - **The roster.** `STREAM.INFO` on the agent-status bucket, which lists
/// its keys (`<hive>.<agent>`) and writes none.
/// - **Review requests.** Publish on the one sink subject
/// `swarm_wire::router_sink::REQUEST_REVIEWER_SUBJECT` and hear the answer
/// on its own inbox. The controller is the subject's one subscriber and
/// holds the forge token; no wildcard over `$SWARM.router.sink`.
fn notify_router_permissions(client_id: &str) -> Permissions {
use swarm_wire::{forge_events, notify_flows, notify_router, router_todo};
@ -320,6 +324,7 @@ impl Policy {
format!("$JS.API.STREAM.INFO.{flows}"),
format!("$JS.API.DIRECT.GET.{flows}.>"),
format!("$JS.API.STREAM.INFO.{}", Self::agent_status_stream()),
swarm_wire::router_sink::REQUEST_REVIEWER_SUBJECT.to_owned(),
Self::inbox(client_id),
]);
Permissions {
@ -347,7 +352,8 @@ impl Policy {
/// What a reader may subscribe to: every agent's terminal, its
/// subagents' terminals and its turn state, which the controller relays
/// to swarm-ui, every hive's permissions import, and its own reply inbox.
/// to swarm-ui, every hive's permissions import, the notification router's
/// review requests, and its own reply inbox.
///
/// The three families are the publishers' (`hive-agent`'s `swarm_term`
/// and `swarm_agent_state`, `swarm_wire::subagent_term`), spelled
@ -359,6 +365,7 @@ impl Policy {
swarm_wire::subagent_term::subject_wildcard(),
swarm_queue_client::agent_state_subject_wildcard(),
swarm_wire::agent_permissions::import_subject_wildcard(),
swarm_wire::router_sink::REQUEST_REVIEWER_SUBJECT.to_owned(),
Self::inbox(client_id),
]
}
@ -1737,7 +1744,7 @@ mod tests {
#[test]
fn the_notify_router_drains_its_own_consumer_and_writes_only_its_own_state() {
use swarm_queue_client::client_inbox_prefix;
use swarm_wire::{forge_events, notify_flows, notify_router, router_todo};
use swarm_wire::{forge_events, notify_flows, notify_router, router_sink, router_todo};
let p = with_router()
.permissions(NOTIFY_ROUTER)
@ -1758,6 +1765,7 @@ mod tests {
router_todo::subject("atlas"),
format!("$KV.{state}.42.5571"),
format!("$JS.API.DIRECT.GET.KV_{state}.$KV.{state}.42.5571"),
router_sink::REQUEST_REVIEWER_SUBJECT.to_owned(),
"$JS.API.STREAM.INFO.KV_agent-status".to_owned(),
// The flows document: read it.
format!("$JS.API.STREAM.INFO.KV_{flows}"),
@ -1783,6 +1791,8 @@ mod tests {
forge_events::subject("issues"),
"$KV.agent-status.alpha.atlas".to_owned(),
"$KV.agent-schedules.atlas".to_owned(),
"$SWARM.router.sink.other".to_owned(),
"$SWARM.router.sink.>".to_owned(),
// The flows document: not write it, nor make the bucket.
format!("$KV.{flows}.{}", notify_flows::KEY),
format!("$JS.API.STREAM.CREATE.KV_{flows}"),
@ -1798,6 +1808,48 @@ mod tests {
assert_eq!(policy().permissions(NOTIFY_ROUTER), None);
}
/// The router asks and the controller answers: nobody else may send the
/// request, and nobody else may hear it, since the answer acts under the
/// controller's forge token.
#[test]
fn only_the_router_may_send_a_review_request_and_only_the_reader_may_hear_it() {
use swarm_wire::router_sink::{REPLY_DEADLINE, REQUEST_REVIEWER_SUBJECT};
let p = with_router();
let grants = [
(
"the reader",
p.permissions("swarm-controller").expect("reader"),
),
("a hive", p.permissions("hive-alpha").expect("a hive")),
("the router", p.permissions(NOTIFY_ROUTER).expect("router")),
("the minter", p.permissions(CLAUDE_LOGIN).expect("minter")),
("the builder", p.permissions(BUILDER).expect("builder")),
(
"the classifier",
p.permissions(CLASSIFIER).expect("classifier"),
),
("atlas", p.agent_token_permissions("atlas").expect("atlas")),
];
for (who, grant) in &grants {
let sends = grant
.publish
.iter()
.any(|g| covers(g, REQUEST_REVIEWER_SUBJECT));
let hears = may_subscribe(grant, REQUEST_REVIEWER_SUBJECT);
assert_eq!(sends, *who == "the router", "{who} sends: {sends}");
assert_eq!(hears, *who == "the reader", "{who} hears: {hears}");
}
// The controller answers on the reply subject for as long as the
// router waits.
let reader = &grants[0].1;
assert!(
reader.responses >= Some(REPLY_DEADLINE),
"{:?}",
reader.responses
);
}
#[test]
fn only_the_router_may_queue_an_agents_todo_and_only_the_reader_may_keep_a_forge_event() {
use swarm_wire::{forge_events, router_todo};

105
swarm-notify-router/src/flow.rs
View file

@ -5,8 +5,9 @@
//! type and this walk runs both. A `match` node passes when every one of its
//! conditions holds, and then each of its children runs, so one event can
//! reach several branches, in edge order, so a clear listed before an upsert
//! is queued before it. A `todo` node emits one op per recipient. The router
//! runs only flows that pass [`Flow::validate`].
//! is queued before it. A `todo` node emits one op per recipient and a
//! `request_reviewer` node one request for a review. The router runs only
//! flows that pass [`Flow::validate`].
use std::collections::BTreeMap;
@ -32,17 +33,27 @@ pub struct Action {
pub op: RouterTodoOp,
}
/// Walk `flow` for one event of type `event_type` and return the ops it
/// emits. Assumes [`Flow::validate`] passed.
pub fn evaluate(flow: &Flow, event_type: &str, fields: &Fields) -> Vec<Action> {
/// One request for `reviewer`'s review of pull request `number` of `repo`.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct ReviewRequest {
pub flow: String,
pub node: String,
pub repo: String,
pub number: u64,
pub reviewer: String,
}
/// Walk `flow` for one event of type `event_type`, calling `leaf` on each
/// `todo` and `request_reviewer` node the event reaches. Assumes
/// [`Flow::validate`] passed.
fn walk<'f>(flow: &'f Flow, event_type: &str, fields: &Fields, mut leaf: impl FnMut(&'f Node)) {
let Some(root) = flow
.nodes
.iter()
.find(|n| matches!(&n.kind, NodeKind::Source { events } if events.iter().any(|e| e == event_type)))
else {
return Vec::new();
return;
};
let mut actions = Vec::new();
let mut stack = vec![root.id.as_str()];
while let Some(id) = stack.pop() {
let Some(node) = flow.node(id) else { continue };
@ -53,12 +64,55 @@ pub fn evaluate(flow: &Flow, event_type: &str, fields: &Fields) -> Vec<Action> {
stack.extend(flow.children(id).into_iter().rev());
}
}
NodeKind::Todo(sink) => actions.extend(emit(flow, node, sink, fields)),
NodeKind::Todo(_) | NodeKind::RequestReviewer { .. } => leaf(node),
}
}
}
/// Walk `flow` for one event of type `event_type` and return the todo ops it
/// emits. Assumes [`Flow::validate`] passed.
pub fn evaluate(flow: &Flow, event_type: &str, fields: &Fields) -> Vec<Action> {
let mut actions = Vec::new();
walk(flow, event_type, fields, |node| {
if let NodeKind::Todo(sink) = &node.kind {
actions.extend(emit(flow, node, sink, fields));
}
});
actions
}
/// Walk `flow` for one event of type `event_type` and return the review
/// requests it makes. An event that is not about a pull request makes none.
/// Assumes [`Flow::validate`] passed.
pub fn requests(flow: &Flow, event_type: &str, fields: &Fields) -> Vec<ReviewRequest> {
let mut requests = Vec::new();
walk(flow, event_type, fields, |node| {
if let NodeKind::RequestReviewer { reviewer } = &node.kind
&& let Some(request) = request(flow, node, reviewer, fields)
{
requests.push(request);
}
});
requests
}
fn request(flow: &Flow, node: &Node, reviewer: &str, fields: &Fields) -> Option<ReviewRequest> {
let text = |name: &str| match fields.get(name) {
Some(Value::Text(v)) => Some(v.as_str()),
_ => None,
};
if text("kind") != Some("PR") {
return None;
}
Some(ReviewRequest {
flow: flow.id.clone(),
node: node.id.clone(),
repo: text("repo")?.to_owned(),
number: text("number")?.parse().ok()?,
reviewer: reviewer.to_owned(),
})
}
fn emit(flow: &Flow, node: &Node, sink: &TodoSink, fields: &Fields) -> Vec<Action> {
let recipients = match &sink.to {
Recipients::Agent(agent) => vec![agent.clone()],
@ -333,6 +387,41 @@ mod tests {
assert_eq!(got[0].op.source, "router:assigned");
}
/// A request sits under the same match gating as a todo, and names the
/// pull request the event concerns.
#[test]
fn a_reviewer_request_runs_under_its_match_and_only_for_a_pull_request() {
let mut f = flow("assigned");
f.nodes[0].kind = NodeKind::Source {
events: vec!["pull_request_sync".to_owned()],
};
f.nodes[2].kind = NodeKind::RequestReviewer {
reviewer: "argus".to_owned(),
};
assert_eq!(f.validate(), Ok(()));
let pull = |action: &str, kind: &'static str| {
let mut fields = event(action, &[], &[], &[]);
fields.insert("kind", text(kind));
fields
};
let got = requests(&f, "pull_request_sync", &pull("assigned", "PR"));
assert_eq!(
got,
[ReviewRequest {
flow: "assigned".to_owned(),
node: "sink".to_owned(),
repo: "hyperhive/hyperhive".to_owned(),
number: 7,
reviewer: "argus".to_owned(),
}]
);
assert!(requests(&f, "pull_request_sync", &pull("opened", "PR")).is_empty());
assert!(requests(&f, "pull_request_sync", &pull("assigned", "issue")).is_empty());
assert!(requests(&f, "pull_request", &pull("assigned", "PR")).is_empty());
// A request is no todo.
assert!(evaluate(&f, "pull_request_sync", &pull("assigned", "PR")).is_empty());
}
#[test]
fn a_placeholder_expands_to_its_field_and_an_unknown_one_is_an_error() {
let fields = event("assigned", &["a", "b"], &[], &[]);

52
swarm-notify-router/src/main.rs
View file

@ -7,6 +7,12 @@
//! Each agent's harness applies them to its own todo list. The flows are the
//! built-in ones, changed by the `notify-flows` document ([`stored`]).
//!
//! A `request_reviewer` node is a request to swarm-controller, which holds the
//! forge token (`swarm_wire::router_sink`). It is sent once, after the event's
//! ops are queued and its snapshot is written, and the answer is only logged:
//! a controller that is down or slow must not hold up every agent's todos, and
//! the next push asks again.
//!
//! Per issue it keeps a snapshot in its state bucket ([`event::Snapshot`]):
//! the assignees it last saw and the agents it queued a todo for. An event is
//! acked once its ops are queued and the snapshot is written, so a crash
@ -26,6 +32,7 @@ use async_nats::jetstream::{self, consumer::pull, kv, message::PublishMessage};
use futures_util::{StreamExt as _, TryStreamExt as _};
use swarm_queue_client::{ClientSecret, QueueConfig};
use swarm_secret_client::stored_secret::StoredSecret;
use swarm_wire::router_sink::{self, Outcome, RequestReviewer};
use swarm_wire::{forge_events, notify_router, router_todo};
mod event;
@ -103,6 +110,7 @@ async fn main() -> Result<()> {
struct Router {
js: jetstream::Context,
nats: async_nats::Client,
events: jetstream::consumer::Consumer<pull::Config>,
state: kv::Store,
flows: stored::Flows,
@ -144,6 +152,7 @@ impl Router {
};
Ok(Self {
js,
nats: queue.clone(),
events,
state,
flows: stored::Flows::builtins(),
@ -200,9 +209,52 @@ impl Router {
let bytes = serde_json::to_vec(&routed.next).expect("a snapshot encodes");
self.state.put(&key, bytes.into()).await.map_err(retry)?;
}
for request in &routed.requests {
self.request_review(request).await;
}
Ok(())
}
/// Ask swarm-controller for one review and log what it answered. One
/// attempt: nothing here fails the event.
async fn request_review(&self, request: &flow::ReviewRequest) {
let wire = RequestReviewer {
repo: request.repo.clone(),
number: request.number,
reviewer: request.reviewer.clone(),
};
let body = serde_json::to_vec(&wire).expect("a review request encodes");
let sent = async_nats::Request::new()
.payload(body.into())
.timeout(Some(router_sink::REPLY_DEADLINE));
let (flow, repo, number, reviewer) = (
&request.flow,
&request.repo,
request.number,
&request.reviewer,
);
match self
.nats
.send_request(router_sink::REQUEST_REVIEWER_SUBJECT, sent)
.await
{
Ok(reply) => match Outcome::decode(&String::from_utf8_lossy(&reply.payload)) {
Outcome::Requested => {
tracing::info!(%flow, %repo, number, %reviewer, "notify router: review requested");
}
Outcome::Skipped(why) => {
tracing::info!(%flow, %repo, number, %reviewer, %why, "notify router: review not requested");
}
Outcome::Error(why) => {
tracing::warn!(%flow, %repo, number, %reviewer, %why, "notify router: the controller could not request the review");
}
},
Err(e) => {
tracing::warn!(%flow, %repo, number, %reviewer, error = %e, "notify router: no answer to a review request; not retried");
}
}
}
/// Queue one op on its agent's subject and wait for the stream's ack.
async fn queue(&self, seq: u64, action: &flow::Action) -> Result<()> {
let subject = router_todo::subject(&action.agent);

48
swarm-notify-router/src/route.rs
View file

@ -8,13 +8,16 @@ use swarm_wire::router_todo::Op;
use crate::event::{Delivery, Snapshot, fields, is_assign};
use swarm_wire::notify_flows::Flow;
use crate::flow::{Action, evaluate};
use crate::flow::{Action, ReviewRequest, evaluate, requests};
/// What one event comes to.
#[derive(Debug, PartialEq, Eq)]
pub struct Routed {
/// The ops to queue, one per agent that exists.
pub actions: Vec<Action>,
/// The reviews to ask the forge for. Not limited to agents: a reviewer is
/// a forge account.
pub requests: Vec<ReviewRequest>,
/// Recipients that are not agents of this swarm, so got nothing.
pub skipped: Vec<String>,
pub next: Snapshot,
@ -32,6 +35,10 @@ pub fn route(
let fields = fields(event_type, delivery, snapshot);
let mut actions = Vec::new();
let mut skipped = Vec::new();
let mut review_requests = Vec::new();
for flow in flows {
review_requests.extend(requests(flow, event_type, &fields));
}
for action in flows.iter().flat_map(|f| evaluate(f, event_type, &fields)) {
if is_agent(&action.agent) {
actions.push(action);
@ -62,6 +69,7 @@ pub fn route(
next.holders.retain(|h| !cleared.contains(h));
Routed {
actions,
requests: review_requests,
skipped,
next,
}
@ -186,6 +194,44 @@ mod tests {
);
}
/// A push to a PR makes one request for the classifier's review, whether
/// or not the classifier is an agent, and touches nothing else.
#[test]
fn a_push_to_a_pull_request_requests_the_classifiers_review_once() {
let flows = builtins();
let pull = Delivery {
is_pull: true,
..delivery("synchronized", &[])
};
let before = Snapshot {
assignees: Some(BTreeSet::from(["atlas".to_owned()])),
holders: BTreeSet::from(["atlas".to_owned()]),
};
let r = route("pull_request_sync", &pull, &before, &flows, |_| false);
assert_eq!(
r.requests,
[ReviewRequest {
flow: "head-classifier".to_owned(),
node: "request".to_owned(),
repo: "o/r".to_owned(),
number: 9,
reviewer: "review-classifier".to_owned(),
}]
);
assert!(r.actions.is_empty() && r.skipped.is_empty());
assert_eq!(r.next, before);
// The source gates: another PR event requests nothing.
for other in ["pull_request", "pull_request_assign", "issues"] {
let r = route(other, &pull, &before, &flows, |_| true);
assert!(r.requests.is_empty(), "{other}");
}
// Only a PR can be asked to review.
let issue = delivery("synchronized", &[]);
let r = route("pull_request_sync", &issue, &before, &flows, |_| true);
assert!(r.requests.is_empty());
}
#[test]
fn an_event_no_flow_matches_changes_nothing() {
let before = Snapshot {

39
swarm-notify-router/src/stored.rs
View file

@ -188,19 +188,28 @@ mod tests {
#[test]
fn with_no_document_the_router_runs_the_builtins() {
let mut f = Flows::builtins();
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
assert_eq!(
ids(&f),
["assigned", "unassigned", "closed", "head-classifier"]
);
f.apply(Read::Absent);
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
assert_eq!(
ids(&f),
["assigned", "unassigned", "closed", "head-classifier"]
);
// An empty document is the same.
f.apply(at(1, doc(&[], &[])));
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
assert_eq!(
ids(&f),
["assigned", "unassigned", "closed", "head-classifier"]
);
}
#[test]
fn a_document_switches_builtins_off_and_adds_custom_flows() {
let mut f = Flows::builtins();
f.apply(at(1, doc(&["unassigned"], &["triage"])));
assert_eq!(ids(&f), ["assigned", "closed", "triage"]);
assert_eq!(ids(&f), ["assigned", "closed", "head-classifier", "triage"]);
}
#[test]
@ -208,7 +217,7 @@ mod tests {
let mut f = Flows::builtins();
f.apply(at(1, doc(&["closed"], &["triage"])));
let before = ids(&f).join(",");
assert_eq!(before, "assigned,unassigned,triage");
assert_eq!(before, "assigned,unassigned,head-classifier,triage");
for bad in [
junk(2, b"not json"),
junk(3, br#"{"version":2,"flows":[]}"#),
@ -221,14 +230,20 @@ mod tests {
// The next good revision replaces them: the control that the bad
// ones were kept out by being bad, not by the loader being stuck.
f.apply(at(5, doc(&[], &[])));
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
assert_eq!(
ids(&f),
["assigned", "unassigned", "closed", "head-classifier"]
);
}
#[test]
fn a_bad_first_document_is_the_builtins_not_nothing() {
let mut f = Flows::builtins();
f.apply(junk(1, br#"{"version":9}"#));
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
assert_eq!(
ids(&f),
["assigned", "unassigned", "closed", "head-classifier"]
);
}
#[test]
@ -236,7 +251,10 @@ mod tests {
let mut f = Flows::builtins();
f.apply(at(1, doc(&["closed"], &["triage"])));
f.apply(Read::Absent);
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
assert_eq!(
ids(&f),
["assigned", "unassigned", "closed", "head-classifier"]
);
}
#[test]
@ -254,6 +272,9 @@ mod tests {
revision: 1,
bytes: serde_json::to_vec(&d).unwrap(),
});
assert_eq!(ids(&f), ["assigned", "unassigned", "closed"]);
assert_eq!(
ids(&f),
["assigned", "unassigned", "closed", "head-classifier"]
);
}
}

1
swarm-wire/src/lib.rs
View file

@ -21,6 +21,7 @@ pub mod notices;
pub mod notify_flows;
pub mod notify_router;
pub mod review_classifier;
pub mod router_sink;
pub mod router_todo;
pub mod subagent_term;
pub mod wanted;

173
swarm-wire/src/notify_flows.rs
View file

@ -4,8 +4,10 @@
//! A flow is a graph of nodes joined by edges, rooted at one `source` node.
//! A `match` node passes when every one of its conditions holds, and then
//! each of its children runs. A `todo` node is a leaf and emits one todo op
//! per recipient. Flows are trees: one source, one parent for every other
//! node, every node reachable from the source ([`Flow::validate`]).
//! per recipient. A `request_reviewer` node is a leaf that asks the forge for
//! a review of the pull request the event concerns. Flows are trees: one
//! source, one parent for every other node, every node reachable from the
//! source ([`Flow::validate`]).
//!
//! The document under [`KEY`] in [`BUCKET`] holds the ids of the built-in
//! flows that are switched off and the custom flows. The built-ins
@ -56,6 +58,10 @@ pub const SOURCE_EVENTS: &[(&str, &str)] = &[
"pull_request_assign",
"A pull request assigned or unassigned",
),
(
"pull_request_sync",
"A push to the branch of an open pull request",
),
];
/// The fields of an event a condition or a recipient can name, with what each
@ -81,6 +87,10 @@ pub const FIELDS: &[(&str, &str)] = &[
),
];
/// The forge account of the review classifier, whose review the built-in
/// `head-classifier` flow requests.
pub const CLASSIFIER_ACCOUNT: &str = "review-classifier";
/// The todo ops a `todo` node's `op` takes, one per [`Op`] variant.
const OPS: &[(&str, &str)] = &[
(
@ -171,6 +181,13 @@ pub enum NodeKind {
Match { when: Vec<Condition> },
/// Emits a todo op to each recipient.
Todo(TodoSink),
/// Asks the forge for `reviewer`'s review of the pull request the event
/// concerns, unless their review of its head already stands. Does nothing
/// for an issue.
RequestReviewer {
/// A forge login.
reviewer: String,
},
}
/// `field` equals `value`. A list field holds when one of its items does.
@ -315,6 +332,16 @@ fn is_id(id: &str) -> bool {
.all(|b| b.is_ascii_alphanumeric() || b == b'-' || b == b'_')
}
/// Whether `login` can name a forge account: Forgejo allows letters, digits,
/// `-`, `_` and `.`, and no more than 40 of them.
fn is_login(login: &str) -> bool {
!login.is_empty()
&& login.len() <= 40
&& login
.bytes()
.all(|b| b.is_ascii_alphanumeric() || matches!(b, b'-' | b'_' | b'.'))
}
fn is_field(name: &str) -> bool {
FIELDS.iter().any(|(n, _)| *n == name)
}
@ -437,6 +464,15 @@ pub fn schema() -> Schema {
),
],
},
KindSchema {
kind: "request_reviewer".to_owned(),
description: "A leaf: asks the forge for a review of the pull request the event concerns, unless the reviewer's review of its head already stands; does nothing for an issue".to_owned(),
fields: vec![kind_field(
"reviewer",
"The forge login to request a review from",
true,
)],
},
],
limits: Limits {
max_nodes: MAX_NODES,
@ -512,8 +548,12 @@ impl Flow {
return Err(format!("{} source nodes, not one", roots.len()));
};
for node in &self.nodes {
if matches!(node.kind, NodeKind::Todo(_)) && !self.children(&node.id).is_empty() {
return Err(format!("todo {} has children", node.id));
if matches!(
node.kind,
NodeKind::Todo(_) | NodeKind::RequestReviewer { .. }
) && !self.children(&node.id).is_empty()
{
return Err(format!("{} has children, and is a leaf", node.id));
}
}
// From the one root, every node reached exactly once: a node reached
@ -587,6 +627,11 @@ impl NodeKind {
}
}
Self::Todo(sink) => sink.validate()?,
Self::RequestReviewer { reviewer } => {
if !is_login(reviewer) {
return Err(format!("reviewer {reviewer:?} is not a forge login"));
}
}
}
Ok(())
}
@ -632,6 +677,7 @@ impl TodoSink {
/// then tells each agent that held one with a `closed` todo. The clear is the
/// first child of the match and runs first: its prefix also covers the
/// `closed` key.
/// - `head-classifier`: a push to a PR requests the review classifier's review.
#[must_use]
pub fn builtins() -> Vec<Flow> {
let assignment_key = "forge:{repo}#{number}:assignment";
@ -662,6 +708,7 @@ pub fn builtins() -> Vec<Flow> {
},
),
closed(),
head_classifier(),
]
}
@ -696,6 +743,33 @@ fn closed() -> Flow {
flow
}
/// source(`pull_request_sync`) → request the review classifier's review.
fn head_classifier() -> Flow {
Flow {
id: "head-classifier".to_owned(),
name: "head-classifier".to_owned(),
enabled: true,
nodes: vec![
Node {
id: "source".to_owned(),
kind: NodeKind::Source {
events: vec!["pull_request_sync".to_owned()],
},
},
Node {
id: "request".to_owned(),
kind: NodeKind::RequestReviewer {
reviewer: CLASSIFIER_ACCOUNT.to_owned(),
},
},
],
edges: vec![Edge {
from: "source".to_owned(),
to: "request".to_owned(),
}],
}
}
/// source(`events`) → match(`action` = `action`) → `sink`.
fn chain(id: &str, events: &[&str], action: &str, sink: TodoSink) -> Flow {
let node = |id: &str, kind| Node {
@ -808,7 +882,7 @@ mod tests {
assert!(with_field("no_such_field").is_err(), "control");
let kinds: Vec<&str> = s.node_kinds.iter().map(|k| k.kind.as_str()).collect();
assert_eq!(kinds, ["source", "match", "todo"]);
assert_eq!(kinds, ["source", "match", "todo", "request_reviewer"]);
let op_names = |op: &str| -> Option<Op> {
serde_json::from_value(serde_json::Value::String(op.to_owned())).ok()
};
@ -917,10 +991,15 @@ mod tests {
#[test]
fn a_node_that_reads_what_does_not_exist_is_refused() {
let mut event = builtin("assigned");
event.nodes[0].kind = NodeKind::Source {
events: vec!["pull_request_syn".to_owned()],
};
assert!(event.validate().unwrap_err().contains("pull_request_syn"));
// Control: the spelling that exists passes.
event.nodes[0].kind = NodeKind::Source {
events: vec!["pull_request_sync".to_owned()],
};
assert!(event.validate().unwrap_err().contains("pull_request_sync"));
assert_eq!(event.validate(), Ok(()));
let mut field = builtin("assigned");
field.nodes[1].kind = NodeKind::Match {
@ -944,6 +1023,81 @@ mod tests {
assert_eq!(custom("c").validate(), Ok(()));
}
/// A reviewer request is a leaf naming a forge login; the wire spells the
/// kind as the schema lists it.
#[test]
fn a_request_reviewer_node_is_a_leaf_naming_a_login() {
let with = |reviewer: &str| {
let mut f = custom("c");
f.nodes[2].kind = NodeKind::RequestReviewer {
reviewer: reviewer.to_owned(),
};
f.validate()
};
assert_eq!(with("review-classifier"), Ok(()));
assert_eq!(with("a.b_c-1"), Ok(()));
for bad in ["", "two words", "a/b", "x@y", &"a".repeat(41)] {
assert!(with(bad).is_err(), "{bad:?}");
}
let mut parent = custom("c");
parent.nodes[2].kind = NodeKind::RequestReviewer {
reviewer: "review-classifier".to_owned(),
};
parent.nodes.push(Node {
id: "m2".to_owned(),
kind: NodeKind::Match { when: Vec::new() },
});
parent.edges.push(edge("sink", "m2"));
assert!(
parent.validate().unwrap_err().contains("leaf"),
"a reviewer request with a child"
);
let json = serde_json::to_value(Node {
id: "ask".to_owned(),
kind: NodeKind::RequestReviewer {
reviewer: "review-classifier".to_owned(),
},
})
.unwrap();
assert_eq!(
json,
serde_json::json!({
"id": "ask",
"kind": "request_reviewer",
"reviewer": "review-classifier",
})
);
let kind = json["kind"].as_str().unwrap();
assert!(
schema().node_kinds.iter().any(|k| k.kind == kind),
"the schema lists the kind the wire uses"
);
let node: Node = serde_json::from_value(json).unwrap();
assert!(matches!(node.kind, NodeKind::RequestReviewer { .. }));
}
/// The built-in that requests the classifier on every push.
#[test]
fn head_classifier_requests_the_classifier_on_a_push() {
let f = builtin("head-classifier");
assert_eq!(f.validate(), Ok(()));
assert_eq!(
f.nodes[0].kind,
NodeKind::Source {
events: vec!["pull_request_sync".to_owned()]
}
);
assert_eq!(
f.nodes[1].kind,
NodeKind::RequestReviewer {
reviewer: CLASSIFIER_ACCOUNT.to_owned()
}
);
assert_eq!(f.children("source"), ["request"]);
}
#[test]
fn a_placeholder_must_name_an_event_field() {
let with = |key: &str, text: &str| {
@ -1030,7 +1184,7 @@ mod tests {
let ids = |d: &Document| d.active().0.into_iter().map(|f| f.id).collect::<Vec<_>>();
assert_eq!(
ids(&Document::default()),
["assigned", "unassigned", "closed"],
["assigned", "unassigned", "closed", "head-classifier"],
"an empty document is the built-ins"
);
let mut off = custom("off");
@ -1042,7 +1196,10 @@ mod tests {
disabled_builtins: vec!["unassigned".to_owned(), "gone".to_owned()],
flows: vec![custom("triage"), off, broken, custom("assigned")],
};
assert_eq!(ids(&doc), ["assigned", "closed", "triage"]);
assert_eq!(
ids(&doc),
["assigned", "closed", "head-classifier", "triage"]
);
let (_, skipped) = doc.active();
assert_eq!(skipped.len(), 2, "{skipped:?}");
assert!(skipped.iter().any(|s| s.starts_with("flow broken:")));

105
swarm-wire/src/router_sink.rs Normal file
View file

@ -0,0 +1,105 @@
//! The notification router's sinks that act on the forge, which it asks
//! swarm-controller to run.
//!
//! The router has no forge token. For a `request_reviewer` node
//! ([`crate::notify_flows::NodeKind::RequestReviewer`]) it sends a request on
//! [`REQUEST_REVIEWER_SUBJECT`] and swarm-controller answers on the request's
//! reply subject with an [`Outcome`], having acted under its own token.
//!
//! Three crates name these: the router sends, the controller answers, and the
//! auth-callout responder grants each its side. The controller is the
//! subject's one subscriber and the router its one publisher.
use std::time::Duration;
use serde::{Deserialize, Serialize};
/// The subject a request for a review goes to.
pub const REQUEST_REVIEWER_SUBJECT: &str = "$SWARM.router.sink.request-reviewer";
/// How long the router waits for the answer. The controller's reply grant
/// lasts this long, so its own work stops sooner ([`WORK_DEADLINE`]).
pub const REPLY_DEADLINE: Duration = Duration::from_secs(30);
/// How long the controller works on one request before it answers
/// [`Outcome::Error`].
pub const WORK_DEADLINE: Duration = Duration::from_secs(20);
/// Ask for `reviewer`'s review of pull request `number` of `repo`.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct RequestReviewer {
/// `owner/name`.
pub repo: String,
pub number: u64,
/// The reviewer's forge login.
pub reviewer: String,
}
/// What the controller did with a [`RequestReviewer`], on the wire as one
/// line: `requested`, `skipped:<why>` or `error:<message>`.
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum Outcome {
/// The forge accepted the request.
Requested,
/// Nothing was sent, and why.
Skipped(String),
/// The request failed, and why.
Error(String),
}
impl Outcome {
/// The wire line.
#[must_use]
pub fn encode(&self) -> String {
match self {
Self::Requested => "requested".to_owned(),
Self::Skipped(why) => format!("skipped:{why}"),
Self::Error(why) => format!("error:{why}"),
}
}
/// The outcome a reply line says; a line that is none of the three is an
/// [`Outcome::Error`] saying so.
#[must_use]
pub fn decode(line: &str) -> Self {
if line == "requested" {
Self::Requested
} else if let Some(why) = line.strip_prefix("skipped:") {
Self::Skipped(why.to_owned())
} else if let Some(why) = line.strip_prefix("error:") {
Self::Error(why.to_owned())
} else {
Self::Error(format!("unreadable reply {line:?}"))
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn an_outcome_round_trips_its_line() {
for outcome in [
Outcome::Requested,
Outcome::Skipped("a review stands".to_owned()),
Outcome::Error("the forge said: no: why".to_owned()),
] {
assert_eq!(Outcome::decode(&outcome.encode()), outcome);
}
assert_eq!(Outcome::Requested.encode(), "requested");
assert_eq!(Outcome::Skipped("x".to_owned()).encode(), "skipped:x");
assert_eq!(Outcome::Error("x".to_owned()).encode(), "error:x");
}
#[test]
fn a_line_that_is_no_outcome_reads_as_an_error() {
assert!(matches!(Outcome::decode("ok"), Outcome::Error(e) if e.contains("ok")));
}
/// The reply grant covers the work and the router's wait covers the grant.
#[test]
fn the_controller_stops_working_before_the_router_stops_waiting() {
assert!(WORK_DEADLINE < REPLY_DEADLINE);
}
}