Watch
0
0
Fork
You've already forked hyperhive
0
Commit graph hyperhive/swarm-controller/src/review_classifier
Author SHA1 Message Date
atlas
b1a00a3222 request the review classifier when a pr is opened or reopened 2026-10-28 10:22:13 +01:00
atlas
eaf26aaadd review classifier: export request, verdict, check and run metrics, with a dashboard
The controller counts review requests, verdicts and per-check results and times each run; the classifier service counts retried asks. A provisioned Grafana dashboard charts them.
2026-10-24 23:24:31 +02:00
atlas
26c4ff89d5 approve a requested review when the repo has no review rules
On deploy: a review requested of the review classifier on a PR whose repo
holds neither .hyperhive/review-classifier.yml nor .json on its default
branch now posts an APPROVED review at once, saying no review rules are
configured for the repo and it was approved without checks. No model call
is made. Before, that case posted a "No verdict" comment. A rules file that
exists but fails to load or parse still posts no verdict. The rule model is
unchanged and instance-wide: every repo without a rules file is approved.
2026-10-24 17:28:40 +02:00
atlas
0b1669a364 request the review classifier's review when a PR's head moves
The router gets a request_reviewer node and a pull_request_sync source;
the built-in head-classifier flow asks swarm-controller, over
$SWARM.router.sink.request-reviewer, to request review-classifier on
every push, unless its review of the head already stands. The controller
subscribes to pull_request_sync on its pull-request hook and answers
the router; nats-auth grants the router the publish and the controller
the subscribe.
2026-10-24 17:23:36 +02:00
atlas
bc4e0c23a0 treefmt 2026-10-24 03:23:17 +02:00
atlas
5ebec6d09b review classifier: a request for a moved head reviews the PR's current head
A request whose head the PR has moved on from was dropped without a word. It now runs on the current head, as before the queue. The re-post shortcut, keyed on the request's head, applies only while the PR is still at that head, so a moved PR never re-posts an old head's verdict.
2026-10-24 03:22:44 +02:00
atlas
efbcc3dce6 review classifier: group the queue consumer's handles into one struct
take and consume carried the forge, the classifier queue, the review poster and the run store as four separate parameters, which took take to eight and needed a too_many_arguments expectation. They travel together everywhere they are used, so they are now one Handles value, cloned per taken request. take has five parameters and no lint attribute. No behaviour change.
2026-10-24 03:22:44 +02:00
atlas
d6aa83146a review classifier: queued review requests survive a restart and are never refused 2026-10-24 03:22:44 +02:00
atlas
6ea13c02e3 move the review classifier, agent login and inference types into swarm-wire 2026-10-18 19:27:30 +02:00
atlas
738dd1c95b swarm-controller: split main.rs into jobq, routes and startup modules
no behaviour change: a pure move of items and tests into new modules, plus
visibility edits (pub(crate)) and path updates in sibling modules that named
crate-root items. same routes, same OpenAPI, 561 tests before and after.

why: #5470, new features stop landing in one 6.4k-line file by default.

files outside swarm-controller/src: docs/agent-lifecycle/agent-roster.md,
swarm-ui JobsPage.tsx and AgentsPage.tsx, nix/host-modules/swarm-controller.nix
and swarm-grafana.nix hold comments that named main.rs for items that moved.
2026-10-18 13:44:43 +02:00
atlas
eb92e83c95 review classifier: re-post a clean verdict on a same-head re-request
A review request for a head whose newest recorded review approved or
requested changes now runs nothing: the classifier account posts that
verdict and table again, opening "Already reviewed <sha>, verdict
unchanged". A head whose newest review had no clean verdict, a failed
post, or no record gets a fresh run, as before.

An accepted request now keeps its key until its record is written, so a
request right after a run finishes reads that run's verdict instead of
starting a second run.

Part of #5387; the restart-loss half waits on #5219.
2026-10-17 12:38:06 +02:00
atlas
a2dca32bb8 review classifier: plain table cells, no backslash escapes 2026-10-16 23:34:45 +02:00
atlas
4098fd6d35 swarm-controller: the review classifier reads a diff with 25 lines of context, not 200
The wide diff's context drops from 200 to 25 lines, so a block's
enclosing gate stays in view when it opens within 25 lines above the
change (the forge's own diff carries 3). The fallback note, the doc and
the gate fixture (now 20 lines out, not 47) follow.
2026-10-15 12:43:12 +02:00
atlas
2b6d480d9c swarm-controller: the review classifier reads a diff with 200 lines of context
The classifier read the forge's PR `.diff`, whose context is fixed at 3
lines. A block moved into a glue module lost its enclosing gate (a
`config = lib.mkIf …` 13 to 191 lines up in the misfired moves #5199,
#5200, #5201), so the glue restating that gate read as a new condition.

The controller now builds the diff itself: it lists the PR's changed files,
reads each at the merge base and at the head through the git tree and blob
API with the token it already holds, and diffs them with 200 lines of
context. It sends the forge's `.diff` instead when the PR changes more than
100 files, the forge names no merge base, or the wide diff is over 256 KiB,
and the posted review then says so with the reason.

Fixes #5227
2026-10-15 12:43:12 +02:00
atlas
9f0fb71fee review classifier: retry a review post for long enough to outlast a swarm-bao restart
The retries covered about a minute, and the swarm-bao restart that lost
#5286's review lasted 3m19s. Each try now gets its own forge deadline,
a try with no answer by then is retried like a refused connection, and
the waits are 5s, 15s, 45s, 90s and 150s: six tries over about five
minutes.

The docs also said to request the review again after a post-failed run.
That does nothing while the account's earlier request is still pending,
so they now say to remove the request and then request it again.
2026-10-15 09:58:46 +02:00
atlas
2e53e46d94 review classifier: retry a review post that failed to reach the store or forge
A review request's verdict was lost when the post failed: on 2026-10-14
the classifier approved #5286, but its post hit swarm-bao mid-restart
(cert login: connection refused), the run was recorded post-failed, and
nothing on the PR showed it.

post() now tries the post up to four times, waiting 5s, 15s and 45s,
when it fails because the connection to the secret store or the forge
failed or timed out. Any other error fails at once, as before. The
verdict is computed once and reposted as is. All tries share the one
forge deadline the single post had, and the warn log carries the number
of tries.
2026-10-15 09:58:46 +02:00
atlas
e7ed28b382 review classifier: the certainty column's header is just certainty, so the column stays narrow 2026-10-15 08:26:26 +02:00
atlas
731cef4c2d review classifier: the review table shows a pass/fail symbol, certainty against threshold, and an emoji on the overall line
The review body's table is now: result | check | certainty / threshold |
reason. The result cell is a symbol for what the row did to the review,
after polarity (✅ counted and passed, ❌ counted and failed, ❔ not
counted or no answer), then the model's verdict. The overall line carries
✅ / ❌. Polarity, threshold and counted are folded into those cells.

The prompt asks for a reason under 350 characters; the enforced cap
stays 500, so an overshoot is truncated less often.
2026-10-15 08:26:26 +02:00
atlas
49b15e7775 review try-out: open a PR with the classifications on the page
The try-out page gets an "open a PR" button. It posts the page's
classifications to the new POST /api/review-classifier/open-pr, which
merges them by name into the repo's rules file on the default branch
and has the review-classifier forge account commit the result on a new
branch, review-classifier/<user>-<8 hex>, through the contents API. The
commit's author is the forge account named like the session's
Remote-User, read with the controller's admin token. The answer links
to the forge's compare page, where the user opens the PR as themselves.

The push uses the account's existing token from the store
(swarm/services/review-classifier/forge/token, scope write:repository),
so the account needs write access to the repo; a refused push answers
403 naming that. In a YAML rules file, the entries the merge leaves
alone keep their text, as long as the result reads back as the merged
list; otherwise the file is written out whole.

DefaultBranchFiles now carries each file's blob sha, which the forge
needs to replace a file, and the repo's web URL for the link.

Fixes #5259
Refs #5096
2026-10-14 18:36:36 +02:00
atlas
1b4e764e5c review classifier: run on ACP when the swarm settings choose it
The swarm settings gain `runtime` (`api-key`, the default, or `acp`) and
an `acp` provider record: an https base URL, the provider's model, and
its prices per million tokens. A PUT now changes only the fields its
body names, so the settings form that sends model, effort and limit
leaves the runtime alone; `null` clears `effort` or `acp`. The body stays
strict, and the stored document keeps #5192's lenient read.

The controller sends the runtime in each run request. On `acp` the
classifier runs one prompt on a new `opencode acp` session per
classification through hive-runtime's AcpRuntime, in a scratch
directory, with a config rendered from the settings. opencode runs
`read` and other tools without asking, so OPENCODE_PERMISSION
{"*":"deny"} takes every tool away from the model, and a deny-all policy
refuses any request on top. The title call, which resends the prompt,
is turned off. The deadline cancels the turn, and a turn that does not
end within 15s of that is dropped, which also covers an agent stuck
before its prompt. The answer is the final assistant text, as ACP has
no result event.

The provider key is read per run from
swarm/services/review-classifier/acp-provider, which the bao UI's
admins login may create and update, and reaches opencode through
AcpRuntime::with_provider_key and the trampoline's allowlist.

Each ACP classification's cost is computed from the configured prices
and the reported tokens, and added to
hyperhive.review_classifier.cost.usage labelled repo, pr, model and
currency.

Part of #5096
2026-10-14 10:14:36 +02:00
atlas
c42321ff0e review classifier: escape $ in verdict cells so the forge doesn't render them as math
Forgejo's markdown renders $...$ and $$...$$ as KaTeX; a reason quoting
nix `${...}` put two $ in one cell and the span between them became a KaTeX
parse error. The cell escaper now backslash-escapes $ like its other markup.

Rule source: the math inline parser triggers on '$' and only starts a span
there (modules/markup/markdown/math/inline_parser.go, Trigger/Parse in
go-gitea/gitea, which Forgejo carries); goldmark's CommonMark backslash
escape consumes '\$' first, so the parser never sees it. Escaping each $
also covers '$$'. Backslashes are already escaped, so '\\$' stays literal.
2026-10-13 21:21:21 +02:00
atlas
daaf920e06 review classifier: keep stored runs this build can't decode
A run record this build couldn't decode was dropped from the list it
read, and the next recorded run wrote the list back without it, so a
rollback followed by a roll-forward lost the newer build's runs for
good. The stored list is now a list of entries, each a decoded run or
the raw JSON of one that didn't decode. The route still shows only the
decoded runs. An undecodable entry keeps its place, counts toward the
100, and is written back as it was read. A read that finds any logs one
WARN with their count and the first error.

The settings decode derived its known fields from what
`Settings::default()` serializes, so a later field skipped when unset
would have been stripped from a valid stored document and lost on the
next PUT. The known fields are now an explicit list. A test serializes
a fully populated `Settings`, checks that its keys match the list, and
checks that `decode` round-trips it.
2026-10-13 08:10:51 +02:00
atlas
6f0a72b246 review classifier: recover from a stored record it can't decode
The controller keeps the classifier's newest runs as one JSON array and
its settings as one JSON object, each under one KV key. A document this
build could not decode, as after a rollback past a build that added an
outcome or a settings field, failed every read and every write until
someone deleted the key by hand. A classifier newer than its controller
refused every run request for the missing `model`.

The run store now decodes the array one record at a time: a record that
does not decode is dropped with a WARN naming the bucket, key, index and
error, and a document that is not an array reads as no runs. Listing
keeps working, and the next recorded run writes back only the records
that decoded.

The stored settings ignore a field this build does not know; one that
still does not decode reads as the defaults with a WARN. The PUT body
keeps `deny_unknown_fields`. A PUT now always writes, so a PUT of the
defaults also replaces an undecodable document.

A run request without `model` runs on haiku, the swarm-wide setting's
default, now one constant in swarm-queue-client.

`schedules::read` gains a sibling, `read_with`, taking the decoder.

Closes #5162
2026-10-13 08:10:51 +02:00
atlas
2c5eef73cd review classifier: model, effort and review concurrency as swarm-wide settings
The classifier ran every classification on a hard-coded `haiku` with no
`--effort`, and the controller ran at most two requested reviews at once,
also hard-coded, so changing any of them took a code change and a
redeploy.

swarm-controller now keeps one settings document,
`{model, effort, max_concurrent_reviews}`, in its own
`review-classifier-settings` KV bucket under `all`, the way it keeps the
schedule definitions. No document means today's values: haiku, no
`--effort`, two reviews. `GET /api/review-classifier/settings` reads it;
`PUT` replaces it whole, needs the gateway's `Remote-User` like the
try-out POST, and refuses a bad model (the inference route's check), an
effort outside the five levels or a limit outside 1..=8 with
problem+json 400.

Every run request (try-out and requested review) now carries the model
and effort, and the classifier passes them to claude instead of its
constant, which is gone. A try-out reads the settings when it is posted;
a review reads them for the limit before it waits for a place and again
for the model and effort once it has one. The review semaphore is
replaced by a running count under a resizable limit, so a PUT, or the
next review's read, changes how many run without a restart, and a lower
limit lets the runs already going finish. MAX_WAITING_REVIEWS is
unchanged.

Closes #5151, part of #5096.
2026-10-13 01:45:19 +02:00
atlas
2fc0166427 swarm-controller: record the review classifier's runs and list them
Every run is recorded when it ends: a try-out when its reply settles it,
a requested review once its review is posted or the post fails, and a
request turned away by a full queue once its comment is posted. A
duplicate request that starts nothing is not recorded. The newest 100
are one JSON array under `recent` in a new controller-owned KV bucket,
`review-classifier-runs`, written with compare-and-set as
`schedule-definitions` is (five attempts). A failed write is logged; the
review is already posted by then.

Each recorded run is also one info event, "review classifier run
finished", with the record's fields as tracing fields. The controller
logs natively to the journal, so they reach VictoriaLogs as F_<NAME>.

GET /api/review-classifier/runs returns the list, newest first, gated
by the gateway like the try-out GET. post_classifier_review now returns
the posted review's html_url, which the record keeps as review_url.

Closes #5153, part of #5096.
2026-10-12 21:48:51 +02:00
atlas
60ea7fdc04 swarm-controller: harden the review classifier's real reviews
Follow-ups from the review of #5142.

A `Done` answer with fewer or more rows than the classifications sent, or
none, now posts the no-verdict comment instead of the classifier's overall
verdict, so a short answer can't approve.

Requests are accepted per repo, PR and head into a set bounded at the two
running plus eight waiting. A request for a key already running or waiting
starts nothing and logs at info; a request with every place taken starts no
run and posts a no-verdict comment saying so. Before, every delivery spawned
a task waiting on the semaphore, so toggling the review request queued one
paid run per toggle.

Table cells get a zero-width space after `#`, `!`, `://` and `www.`, so a
model's reason can't render an issue or PR reference or a bare URL as a link.

The secret publisher's policy denies the classifier's forge token leaf, the
way #5139 denied Grafana's admin password, with a module-eval-bao-grants arm
pinning it. The controller posts reviews with whatever token is there, so a
publisher write picked the account reviews come from.

Closes #5143, part of #5096.
2026-10-12 18:38:05 +02:00
atlas
a669cd96b6 swarm-controller: post the review classifier's verdict on a review request
Requesting the forge account `review-classifier` as a reviewer now runs the
repo's rules against the PR and posts the result as that account's review,
on the head commit the run read. A push starts no run; only a fresh review
request does.

- An instance-wide `pull_request_review_request` system hook delivers to
  `/webhook/forge/review-request`. A request for any other reviewer, or a
  removed request, does nothing.
- The controller reads the PR, loads the rules with the same reader as
  `GET /api/review-classifier/rules`, sends one request on the try-out
  subject, and posts APPROVED or REQUEST_CHANGES with the table as the body.
- With no clean verdict (PR or rules don't load, the run fails, or any
  classification has no answer) it posts a COMMENT saying why, never an
  approval. The event is one function, `no_clean_verdict_event`.
- Table cells are flattened, markup-escaped, and every `@` gets a
  zero-width space, so a model's reason can't break the table or mention
  anyone.
- At most two requested reviews run at once; the rest wait in memory.
- The account is minted the way hive accounts are: a five-minute pass
  queues a `MintClassifierForgeToken` node when the token at
  `swarm/services/review-classifier/forge/token` is missing or stale. The
  token is `write:repository`. The controller's bao policy gains that leaf
  (create/read/update), pinned in module-eval bao-grants. The classifier
  never sees the token.

The account gets no repo access automatically; a repo admin adds it by
hand. A re-request retires the account's earlier review: Forgejo clears
`official` on a reviewer's earlier reviews when it is requested again
(models/issues/review.go AddReviewRequest, v16.0.5), and approvals and
rejection blocks count official reviews only.

Refs #5137
2026-10-12 16:11:50 +02:00