Watch
0
0
Fork
You've already forked hyperhive
0

review classifier: classifications of one PR share a cached prefix

On the API key runtime the classifier now hands claude its prompt as one
stream-json user message (`--input-format stream-json`) with two text
blocks: the PR's shared context, carrying `cache_control: ephemeral`, and
the classification's question.

As plain stdin text the whole prompt was one block, and claude puts its
only message breakpoint on the last block, after the question. The cache
writes an entry only where a breakpoint is, so no entry ended at the
shared-context/question boundary. Every classification wrote the whole
prompt and none read it (production: cacheRead ~0, cacheCreation ~input).
The system-prompt prefix (~270 tokens) is under the minimum cacheable
length. Captures against a local listener with a dummy key (claude
2.1.223) show the client breakpoint survives on the shared block, and that
claude adds none there on its own.

The PR stays in the user role: moving it into the system prompt would give
untrusted PR text the system prompt's authority.

`Model::ask` takes a `Prompt { shared, question }`. The ACP runtime joins
it back into the same text it sent before.

Refs #5245
This commit is contained in:
atlas 2026-10-14 17:47:10 +02:00 • committed by mara
commit 6ed88ad70a
5 changed files with 223 additions and 34 deletions

View file

@ -54,19 +54,46 @@ pub fn shared_context(run: &TryOut) -> String {
out
}
/// The prompt for one classification: `shared` unchanged, then the question.
/// One classification's prompt: the run's shared context unchanged, then the
/// question.
pub struct Prompt<'a> {
pub shared: &'a str,
pub question: String,
}
impl Prompt<'_> {
/// The prompt as one text, for a runtime that takes no content blocks.
#[must_use]
pub fn text(&self) -> String {
format!("{}{}", self.shared, self.question)
}
}
#[cfg(test)]
impl Prompt<'static> {
/// A prompt whose whole text is `text`.
pub fn of(text: &str) -> Self {
Self {
shared: "",
question: text.to_owned(),
}
}
}
/// The prompt for one classification: `shared`, then the question.
#[must_use]
pub fn classification_prompt(shared: &str, c: &Classification) -> String {
pub fn classification_prompt<'a>(shared: &'a str, c: &Classification) -> Prompt<'a> {
let wanted = match c.polarity {
Polarity::Positive => "a match is the wanted outcome",
Polarity::Negative => "a match is a violation",
};
format!(
"{shared}## The question\n\nClassification `{}` ({wanted}):\n\n{}\n\n\
let question = format!(
"## The question\n\nClassification `{}` ({wanted}):\n\n{}\n\n\
Answer with the JSON object only. Keep the reason under {REASON_MAX_CHARS} characters.\n",
c.name,
c.description.trim()
)
);
Prompt { shared, question }
}
/// The lockfiles whose diffs collapse to one line. The same list as
@ -259,10 +286,11 @@ diff --git a/src/main.rs b/src/main.rs
]
.map(|c| classification_prompt(&shared, &c));
for prompt in &prompts {
assert!(prompt.starts_with(&shared), "{prompt}");
assert_eq!(prompt.shared, shared);
assert!(prompt.text().starts_with(&shared), "{}", prompt.text());
}
// The question is all that differs, and none of it is in the prefix.
assert_ne!(prompts[0], prompts[1]);
assert_ne!(prompts[0].question, prompts[1].question);
assert!(!shared.contains("datetime") && !shared.contains("no-ads"));
// And the prefix is a function of the run alone.
assert_eq!(shared, shared_context(&run()));