vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

review classifier: classifications of one PR share a cached prefix

Browse source 
On the API key runtime the classifier now hands claude its prompt as one
stream-json user message (`--input-format stream-json`) with two text
blocks: the PR's shared context, carrying `cache_control: ephemeral`, and
the classification's question.

As plain stdin text the whole prompt was one block, and claude puts its
only message breakpoint on the last block, after the question. The cache
writes an entry only where a breakpoint is, so no entry ended at the
shared-context/question boundary. Every classification wrote the whole
prompt and none read it (production: cacheRead ~0, cacheCreation ~input).
The system-prompt prefix (~270 tokens) is under the minimum cacheable
length. Captures against a local listener with a dummy key (claude
2.1.223) show the client breakpoint survives on the shared block, and that
claude adds none there on its own.

The PR stays in the user role: moving it into the system prompt would give
untrusted PR text the system prompt's authority.

`Model::ask` takes a `Prompt { shared, question }`. The ACP runtime joins
it back into the same text it sent before.

Refs #5245
This commit is contained in:
atlas 2026-10-14 17:47:10 +02:00 • committed by mara
commit 6ed88ad70a
5 changed files with 223 additions and 34 deletions
Download patch file Download diff file Expand all files Collapse all files

13
docs/swarm/review-classifier.md
View file

@ -121,7 +121,9 @@ ephemeral, and nothing carries over from one run to the next.
On the API key runtime, each classification is one `claude --print` the
service spawns through the `hive-claude` crate directly, with no session to
resume, no tools (`--tools ""`), no MCP server (`--strict-mcp-config` and no
config) and an empty directory of its own.
config) and an empty directory of its own. The prompt reaches it on stdin as
one user message in `--input-format stream-json`, so that it can carry content
blocks (see [Shared context](#shared-context)).
On the ACP runtime, each classification is one prompt on a new `opencode acp`
session. Its config and opencode's state live in a directory of its own,
@ -272,6 +274,15 @@ Only the last part differs per classification: that classification's
description and the task to classify the PR against it. The identical prefix
lets the runtime's prompt cache reuse it across questions.
On the API key runtime the commits and diff are one content block of the user
message and the question another, with a cache breakpoint at the end of the
first. The cache writes an entry only where a breakpoint is, and `claude` sets
one only after the last block, so that breakpoint is what lets the next
question read the commits and diff from the cache. The PR stays in the user
message rather than the system prompt, which tells the model to judge it as
data. A shared context shorter than the model's minimum cacheable length is
not cached at all.
No object hash is in that prefix: the header names the repo and the PR number,
not the head commit, each commit's heading gives its position instead of its
sha, and the diff loses each file's `index <hash>..<hash>` line, keeping its