review classifier: model, effort and review concurrency as swarm-wide settings
The classifier ran every classification on a hard-coded `haiku` with no
`--effort`, and the controller ran at most two requested reviews at once,
also hard-coded, so changing any of them took a code change and a
redeploy.
swarm-controller now keeps one settings document,
`{model, effort, max_concurrent_reviews}`, in its own
`review-classifier-settings` KV bucket under `all`, the way it keeps the
schedule definitions. No document means today's values: haiku, no
`--effort`, two reviews. `GET /api/review-classifier/settings` reads it;
`PUT` replaces it whole, needs the gateway's `Remote-User` like the
try-out POST, and refuses a bad model (the inference route's check), an
effort outside the five levels or a limit outside 1..=8 with
problem+json 400.
Every run request (try-out and requested review) now carries the model
and effort, and the classifier passes them to claude instead of its
constant, which is gone. A try-out reads the settings when it is posted;
a review reads them for the limit before it waits for a place and again
for the model and effort once it has one. The review semaphore is
replaced by a running count under a resizable limit, so a PUT, or the
next review's read, changes how many run without a restart, and a lower
limit lets the runs already going finish. MAX_WAITING_REVIEWS is
unchanged.
Closes #5151, part of #5096.
This commit is contained in:
parent
78412b3d3b
commit
2c5eef73cd
10 changed files with 868 additions and 69 deletions
|
|
@ -128,11 +128,49 @@ credential: `swarm-controller` reads the PR and sends it along.
|
|||
|
||||
### Swarm-level settings
|
||||
|
||||
The operator sets the classifier's settings for the whole swarm:
|
||||
The model, the effort and how many requested reviews run at once are one
|
||||
document for the whole swarm. `swarm-controller` keeps it in its
|
||||
`review-classifier-settings` KV bucket under the key `all`, and no other
|
||||
service reads it. With no document stored, the classifier runs on `haiku` at
|
||||
claude's default effort, two reviews at a time.
|
||||
|
||||
- the runtime (see [Runtimes](#runtimes)) and its credentials
|
||||
- the model and effort, and related inference settings
|
||||
- the maximum number of concurrent runs
|
||||
`GET /api/review-classifier/settings` answers the settings in force.
|
||||
`PUT /api/review-classifier/settings` replaces them whole and answers what it
|
||||
stored. Both take and give one shape:
|
||||
|
||||
```json
|
||||
{ "model": "haiku", "effort": null, "max_concurrent_reviews": 2 }
|
||||
```
|
||||
|
||||
- `model`: the model every classification runs on, as claude's `--model`
|
||||
takes it: one word with no whitespace, not starting with `-`
|
||||
- `effort`: `low`, `medium`, `high`, `xhigh` or `max`; `null`, or left out of
|
||||
a PUT, passes no `--effort`
|
||||
- `max_concurrent_reviews`: 1 to 8, how many requested reviews run at once.
|
||||
Try-outs don't count toward it. Up to eight more reviews wait for a place
|
||||
|
||||
The controller reads the settings for every run and sends the model and
|
||||
effort in the run request, so a change applies from the next run with no
|
||||
restart. A try-out reads them when it's posted, a review when it gets its
|
||||
place. Lowering the limit lets the runs already going finish.
|
||||
|
||||
The PUT needs the `Remote-User` header the gateway sets, as the try-out POST
|
||||
does. Every error is problem+json with the reason in `detail`:
|
||||
|
||||
- **400** for a body that isn't JSON or isn't this shape (a missing `model`
|
||||
or `max_concurrent_reviews`, an unknown field, an effort outside the five
|
||||
levels, a `max_concurrent_reviews` that isn't a whole number), an empty
|
||||
`model`, one with whitespace or one starting with `-`, or a
|
||||
`max_concurrent_reviews` outside 1 to 8
|
||||
- **401** for a PUT with no `Remote-User`
|
||||
- **503** when this host has no swarm queue, or it isn't connected
|
||||
- **500** when the controller can't read or write the settings
|
||||
|
||||
When the controller can't read the settings, a try-out answers the same 503
|
||||
or 500, and a review posts a no-verdict comment. The runtime (see
|
||||
[Runtimes](#runtimes)) isn't a setting yet, and the operator writes the API
|
||||
key by hand in the secret store
|
||||
([`services.md`](services.md#review-classifier)).
|
||||
|
||||
The per-agent equivalent is [`inference.md`](inference.md), where the
|
||||
controller writes model and effort to a queue KV bucket
|
||||
|
|
|
|||
Loading…
Reference in a new issue