review-classifier: ask a check once more when its answer isn't valid JSON
An answer that fails to validate is asked for a second time, for that classification only, with a reminder to reply with the JSON object alone. The second ask comes out of the classification's CLASSIFICATION_DEADLINE: it gets what the first ask left, and is skipped when less than 20 seconds remain, so RUN_DEADLINE's bound still holds. A failed ask (runtime error, timeout) is not repeated. If the second answer does not validate either, the row has no answer, as before. Model::ask takes the deadline for the ask; the ACP runtime no longer holds its own. The parser also takes the first fenced block of a reply that has prose around the fence; prose around a bare object is still rejected.
This commit is contained in:
parent
b8897e7c7a
commit
ce7b3e055f
8 changed files with 345 additions and 66 deletions
|
|
@ -22,6 +22,8 @@ forge and holds no forge credential.
|
|||
- An answer is a JSON object with a verdict, a certainty and a reason; the
|
||||
prompt asks for a reason under 350 characters, and the classifier truncates one over the 500 cap to 499 plus `…`. A failed run, a
|
||||
timeout or an answer that doesn't validate leaves that row without a verdict, and it doesn't count.
|
||||
An answer that doesn't validate is asked for once more first, with a
|
||||
reminder to reply with the JSON object alone, within the same three minutes.
|
||||
- A run with no API key, or with no classification answered, fails as a
|
||||
whole: the caller sees an error, never a verdict.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue