review classifier: the most classifications a run carries is a setting
The limit was a constant of ten in swarm-wire, so a repo with eleven rules could not be reviewed and nothing short of a rebuild changed it. It is now `max_classifications` in the classifier settings, beside `max_concurrent_reviews`: 1 to 1000, ten until set, and a document stored before the field decodes with the default. The classifier settings form in the swarm UI gets a "rules per review" field, and the try-out's add button stops at the saved limit instead of a hard-coded ten. The controller checks a try-out, and a requested review's rules file, against the setting where the run starts. The rules file itself may hold up to the 1000 the setting allows, so raising the limit needs no change to a repo. A review whose rules exceed the limit posts a no-verdict comment saying so. A run's reply deadline was a constant that covered ten classifications, a minute plus three each. It is now computed from the run's own size, so a bigger run waits as long as it needs and a default-sized one still waits 31 minutes. The classifier's answer window in the queue permissions covers the biggest run the setting allows.
This commit is contained in:
parent
e436d356f3
commit
cffe8895df
2 changed files with 400 additions and 101 deletions
|
|
@ -603,7 +603,8 @@ adds to `verdicts` but runs no checks, so it adds nothing to `checks` or
|
|||
as `queue_failed` and a `no_verdict`. `no_answer` is a classification that
|
||||
failed, so a `no_verdict` run with a `no_answer` check is the usual cause of
|
||||
a comment instead of a review. The `check` values are the names in a repo's
|
||||
rules file, at most 10 per repo.
|
||||
rules file, at most the settings' `max_classifications` per repo (ten until
|
||||
set).
|
||||
|
||||
The `review-classifier` dashboard in Grafana charts these.
|
||||
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ with exactly these four keys:
|
|||
`negative` means a match is a violation
|
||||
|
||||
Any other key, at the top or in an entry, is an error: a misspelt key fails
|
||||
the load instead of vanishing. An empty list, more than ten entries, an empty
|
||||
the load instead of vanishing. An empty list, more than a thousand entries, an empty
|
||||
name or description, and two entries with one name are errors too. A file
|
||||
that exists but fails these checks never falls back to a pass: the run posts
|
||||
no verdict.
|
||||
|
|
@ -215,6 +215,7 @@ replaces it.
|
|||
"model": "haiku",
|
||||
"effort": null,
|
||||
"max_concurrent_reviews": 2,
|
||||
"max_classifications": 10,
|
||||
"runtime": "api-key",
|
||||
"acp": null
|
||||
}
|
||||
|
|
@ -228,6 +229,15 @@ replaces it.
|
|||
offers it
|
||||
- `max_concurrent_reviews`: 1 to 8, how many requested reviews run at once.
|
||||
Try-outs don't count toward it. The rest wait in the queue for a place
|
||||
- `max_classifications`: 1 to 1000, ten by default (also for a document
|
||||
stored before the field existed): the most classifications one run
|
||||
carries. The controller refuses a try-out with more, and a review of a repo
|
||||
whose rules file holds more posts a no-verdict comment saying so. A rules
|
||||
file may hold up to 1000 either way, so raising the limit needs no change to
|
||||
the repo. The classifications of a run go one after another, each for up to
|
||||
three minutes, so the controller waits a minute plus three per
|
||||
classification for the reply (31 minutes for ten), and the classifier may
|
||||
answer for as long as the biggest run the limit allows
|
||||
- `runtime`: `api-key` or `acp` (see [Runtimes](#runtimes))
|
||||
- `acp`: the provider the ACP runtime runs on, or `null`. It stays stored
|
||||
while the runtime is `api-key`, so switching back and forth needs no
|
||||
|
|
@ -255,11 +265,12 @@ does. Every error is problem+json with the reason in `detail`:
|
|||
|
||||
- **400** for a body that isn't JSON or isn't this shape: an unknown field, a
|
||||
`null` other than for `effort` or `acp`, an effort outside the five levels,
|
||||
a `max_concurrent_reviews` that isn't a whole number, a runtime other than
|
||||
a `max_concurrent_reviews` or `max_classifications` that isn't a whole
|
||||
number, a runtime other than
|
||||
the two, or an `acp` without `base_url`, `model` or `prices`. Also **400**
|
||||
when the document the PUT would store breaks a rule: an empty `model`, one
|
||||
with whitespace or one starting with `-`, a `max_concurrent_reviews` outside
|
||||
1 to 8, an `acp.base_url` that isn't `https://`, an empty `acp.model` or one
|
||||
1 to 8, a `max_classifications` outside 1 to 1000, an `acp.base_url` that isn't `https://`, an empty `acp.model` or one
|
||||
with whitespace, a negative price, or the `acp` runtime with no `acp`
|
||||
- **401** for a PUT with no `Remote-User`
|
||||
- **503** when this host has no swarm queue, or it isn't connected
|
||||
|
|
@ -588,7 +599,7 @@ The route answers **201** with `branch`, `path` (the file written) and
|
|||
file changed on the default branch during the push
|
||||
- **422** for a default branch holding both rules files or one the `rules`
|
||||
route refuses; classifications the file already holds unchanged; or a
|
||||
merge that breaks a rule, such as more than ten classifications
|
||||
merge that breaks a rule, such as more than a thousand classifications
|
||||
- **503** when this host has no forge token, the service account's token
|
||||
isn't minted yet, or the forge failed or didn't answer within two minutes
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue