Watch
0
0
Fork
You've already forked hyperhive
0

review classifier: the review table shows a pass/fail symbol, certainty against threshold, and an emoji on the overall line

The review body's table is now: result | check | certainty / threshold |
reason. The result cell is a symbol for what the row did to the review,
after polarity (✅ counted and passed, ❌ counted and failed, ❔ not
counted or no answer), then the model's verdict. The overall line carries
✅ / ❌. Polarity, threshold and counted are folded into those cells.

The prompt asks for a reason under 350 characters; the enforced cap
stays 500, so an overshoot is truncated less often.
This commit is contained in:
atlas 2026-10-14 23:31:46 +02:00 • committed by mara
commit 731cef4c2d
5 changed files with 172 additions and 43 deletions

View file

@ -12,6 +12,11 @@ use swarm_queue_client::review_classifier::Verdict;
/// truncated to this, ending in `…`.
pub const REASON_MAX_CHARS: usize = 500;
/// The length the prompt asks a reason to stay under. Below `REASON_MAX_CHARS`
/// so a model that overshoots the ask still lands inside the cap, uncut.
pub const REASON_ASKED_CHARS: usize = 350;
const _: () = assert!(REASON_ASKED_CHARS < REASON_MAX_CHARS);
/// One validated answer.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Answer {

View file

@ -9,7 +9,7 @@ use std::fmt::Write as _;
use swarm_queue_client::review_classifier::{Classification, Polarity, TryOut};
use crate::answer::REASON_MAX_CHARS;
use crate::answer::REASON_ASKED_CHARS;
/// The system prompt, the same for every run.
pub const SYSTEM: &str = "\
@ -89,7 +89,7 @@ pub fn classification_prompt<'a>(shared: &'a str, c: &Classification) -> Prompt<
};
let question = format!(
"## The question\n\nClassification `{}` ({wanted}):\n\n{}\n\n\
Answer with the JSON object only. Keep the reason under {REASON_MAX_CHARS} characters.\n",
Answer with the JSON object only. Keep the reason under {REASON_ASKED_CHARS} characters.\n",
c.name,
c.description.trim()
);
@ -296,6 +296,16 @@ diff --git a/src/main.rs b/src/main.rs
assert_eq!(shared, shared_context(&run()));
}
#[test]
fn the_question_asks_for_a_reason_under_the_asked_length_not_the_cap() {
let c = classification("datetime", "d", Polarity::Positive);
let question = classification_prompt("", &c).question;
assert!(
question.contains("Keep the reason under 350 characters."),
"{question}"
);
}
#[test]
fn the_shared_context_has_the_full_commit_message_and_a_collapsed_lockfile() {
let shared = shared_context(&run());