Watch
0
0
Fork
You've already forked hyperhive
0

review-classifier: ask a check once more when its answer isn't valid JSON

An answer that fails to validate is asked for a second time, for that
classification only, with a reminder to reply with the JSON object alone.
The second ask comes out of the classification's CLASSIFICATION_DEADLINE:
it gets what the first ask left, and is skipped when less than 20 seconds
remain, so RUN_DEADLINE's bound still holds. A failed ask (runtime error,
timeout) is not repeated. If the second answer does not validate either,
the row has no answer, as before.

Model::ask takes the deadline for the ask; the ACP runtime no longer holds
its own.

The parser also takes the first fenced block of a reply that has prose
around the fence; prose around a bare object is still rejected.
This commit is contained in:
atlas 2026-10-16 15:32:07 +02:00 • committed by mara
commit ce7b3e055f
8 changed files with 345 additions and 66 deletions

View file

@ -54,6 +54,10 @@ pub fn shared_context(run: &TryOut) -> String {
out
}
/// Appended to the question when it is asked a second time.
const JSON_REMINDER: &str = "Your previous answer was not a JSON object. Reply with the JSON \
object and nothing else: no text before or after it, and no code fence.";
/// One classification's prompt: the run's shared context unchanged, then the
/// question.
pub struct Prompt<'a> {
@ -67,6 +71,16 @@ impl Prompt<'_> {
pub fn text(&self) -> String {
format!("{}{}", self.shared, self.question)
}
/// The same prompt with a reminder to answer with the JSON object alone,
/// for asking again after an answer that was not that object.
#[must_use]
pub fn with_json_reminder(&self) -> Self {
Self {
shared: self.shared,
question: format!("{}\n{JSON_REMINDER}\n", self.question.trim_end()),
}
}
}
#[cfg(test)]