vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

review classifier: run on ACP when the swarm settings choose it

Browse source 
The swarm settings gain `runtime` (`api-key`, the default, or `acp`) and
an `acp` provider record: an https base URL, the provider's model, and
its prices per million tokens. A PUT now changes only the fields its
body names, so the settings form that sends model, effort and limit
leaves the runtime alone; `null` clears `effort` or `acp`. The body stays
strict, and the stored document keeps #5192's lenient read.

The controller sends the runtime in each run request. On `acp` the
classifier runs one prompt on a new `opencode acp` session per
classification through hive-runtime's AcpRuntime, in a scratch
directory, with a config rendered from the settings. opencode runs
`read` and other tools without asking, so OPENCODE_PERMISSION
{"*":"deny"} takes every tool away from the model, and a deny-all policy
refuses any request on top. The title call, which resends the prompt,
is turned off. The deadline cancels the turn, and a turn that does not
end within 15s of that is dropped, which also covers an agent stuck
before its prompt. The answer is the final assistant text, as ACP has
no result event.

The provider key is read per run from
swarm/services/review-classifier/acp-provider, which the bao UI's
admins login may create and update, and reaches opencode through
AcpRuntime::with_provider_key and the trampoline's allowlist.

Each ACP classification's cost is computed from the configured prices
and the reported tokens, and added to
hyperhive.review_classifier.cost.usage labelled repo, pr, model and
currency.

Part of #5096
This commit is contained in:
atlas 2026-10-13 10:15:27 +02:00 • committed by mara
commit 1b4e764e5c
24 changed files with 1622 additions and 262 deletions
Download patch file Download diff file Expand all files Collapse all files

1
docs/swarm/credentials.md
View file

@ -75,6 +75,7 @@ of the cell says how.
| `swarm/services/<clientId>/oidc/client` | authelia | that service's own reader, under its own certificate — the queue responder's under `swarm-nats-auth`'s, the homeserver's under `swarm-matrix-ctl`'s, the forge's under `swarm-forge-publisher`'s, the review classifier's under `swarm-review-classifier`'s; the bao UI's by the store's granter | must be stated | must be stated |
| `swarm/services/ci-runner/registration-token` | `swarm-controller`, from the forge's admin API, at start and in a five-minute pass; its grant on the path is write-only | `swarm-bao-ci-runner` on the runner's host, under the runner's own certificate, into a file only `swarm-ci-runner-register` in the runner's container loads; no other principal's policy reads it | ❌ the forge hands out one token until an admin resets it; the controller writes a reset token within five minutes | ✅ `swarm-bao-ci-runner` reads it at boot, on deploy and every ten minutes, and restarts the runner when the value changed |
| `swarm/services/review-classifier/anthropic-api-key` | the operator, by hand in the store's UI, field `api_key` | `swarm-review-classifier` in its container, under its own certificate, at the start of every run; held for that run and handed to `claude`'s environment only; no other principal's policy reads it | ❌ an operator's key; replaced by writing a new one | ✅ read on every run, so a written or removed key takes effect on the next run |
| `swarm/services/review-classifier/acp-provider` | the operator, by hand in the store's UI, field `api_key` | `swarm-review-classifier` in its container, under its own certificate, at the start of every ACP run; held for that run and handed to `opencode`'s environment only; no other principal's policy reads it | ❌ an operator's key; replaced by writing a new one | ✅ read on every run, so a written or removed key takes effect on the next run |
| `swarm/services/review-classifier/forge/token` | `swarm-controller`, with the forge's admin API, for its `review-classifier` account, at start and in a five-minute pass; the secret publisher's policy denies it | `swarm-controller` under its own certificate, before it decides whether to mint, and on every review it posts as the account; the classifier service's policy doesn't read it | ✅ the controller's pass re-mints when the stored token is missing or no longer matches the forge (last eight characters and scopes) | ✅ the controller reads the store on every review it posts |
| `swarm/services/builder/forge/token` | `swarm-controller`, with the forge's admin API, for its `builder` account (read on every config repo through the `hives` team), at start and in a five-minute pass; the secret publisher's policy denies it | `swarm-builder` in its container, under its own certificate, on every build; held for that build and handed to `git` through the environment only; `swarm-controller` under its own certificate, before it decides whether to mint | ✅ the controller's pass re-mints when the stored token is missing or no longer matches the forge (last eight characters and scopes) | ✅ the builder reads the store on every build |
| _(not in the store)_ a hive's mTLS leaf | the store's own PKI, or an operator placing it by hand | its own client, off disk — the exception above, because it's what makes every other row's pull possible | must be stated | must be stated |

159
docs/swarm/review-classifier.md
View file

@ -118,12 +118,22 @@ It looks like an agent from the inside, with three things missing: it has no
full agent identity, no state directory and no tools. Every model session is
ephemeral, and nothing carries over from one run to the next.
On the claude runtimes, each classification is one `claude --print` the
On the API key runtime, each classification is one `claude --print` the
service spawns through the `hive-claude` crate directly, with no session to
resume, no tools (`--tools ""`), no MCP server (`--strict-mcp-config` and no
config) and an empty directory of its own. A classification needs no durable
session, so the service doesn't use [`hive-runtime`](../../hive-runtime/README.md),
whose `Runtime` trait drives the agents' durable sessions.
config) and an empty directory of its own.
On the ACP runtime, each classification is one prompt on a new `opencode acp`
session in an empty directory of its own. The service drives it through
[`hive-runtime`](../../hive-runtime/README.md)'s `AcpRuntime`, as the subagent
daemon does a one-run subagent, and stops the agent when the classification
ends. The agent has no tool. opencode runs some tools, `read` among them,
without asking the client, so the service sets opencode's own permission to
deny every tool (`OPENCODE_PERMISSION={"*":"deny"}`), which leaves the model
with no tool offered at all, and refuses any permission request on top. The
config it writes for opencode names one provider and model, from the settings,
and turns off opencode's title call, which would send the prompt a second
time.
The service is `swarm-review-classifier`, and its container is the one
[`services.md`](services.md#review-classifier) describes. It holds no forge
@ -131,51 +141,75 @@ credential: `swarm-controller` reads the PR and sends it along.
### Swarm-level settings
The model, the effort and how many requested reviews run at once are one
document for the whole swarm. `swarm-controller` keeps it in its
The runtime, the model, the effort and how many requested reviews run at once
are one document for the whole swarm. `swarm-controller` keeps it in its
`review-classifier-settings` KV bucket under the key `all`, and no other
service reads it. With no document stored, the classifier runs on `haiku` at
claude's default effort, two reviews at a time. The controller
ignores a stored field it doesn't know, and a document it still can't decode
counts as none, with a warning in its log, until the next PUT replaces it.
service reads it. With no document stored, the classifier runs on the API key
runtime on `haiku` at claude's default effort, two reviews at a time. The
controller ignores a stored field it doesn't know, and a document it still
can't decode counts as none, with a warning in its log, until the next PUT
replaces it.
`GET /api/review-classifier/settings` answers the settings in force.
`PUT /api/review-classifier/settings` replaces them whole and answers what it
stored. Both take and give one shape:
`GET /api/review-classifier/settings` answers the settings in force:
```json
{ "model": "haiku", "effort": null, "max_concurrent_reviews": 2 }
{
"model": "haiku",
"effort": null,
"max_concurrent_reviews": 2,
"runtime": "api-key",
"acp": null
}
```
- `model`: the model every classification runs on, as claude's `--model`
takes it: one word with no whitespace, not starting with `-`
- `effort`: `low`, `medium`, `high`, `xhigh` or `max`; `null`, or left out of
a PUT, passes no `--effort`
- `model`: the model every classification on the API key runtime runs on, as
claude's `--model` takes it: one word with no whitespace, not starting with
`-`
- `effort`: `low`, `medium`, `high`, `xhigh` or `max`; `null` runs at the
runtime's own default. On the ACP runtime it applies only where the agent
offers it
- `max_concurrent_reviews`: 1 to 8, how many requested reviews run at once.
Try-outs don't count toward it. Up to eight more reviews wait for a place
- `runtime`: `api-key` or `acp` (see [Runtimes](#runtimes))
- `acp`: the provider the ACP runtime runs on, or `null`. It stays stored
while the runtime is `api-key`, so switching back and forth needs no
re-entry. It holds:
- `base_url`: the provider's OpenAI-compatible API root, `https://` only
- `model`: the model as the provider names it, with no whitespace
- `prices`: what the provider charges in USD per million tokens,
`input` and `output`, and optionally `cache_read` and `cache_write`, which
otherwise count as `input`. A price is 0 or more
The controller reads the settings for every run and sends the model and
effort in the run request, so a change applies from the next run with no
`PUT /api/review-classifier/settings` takes any of these fields, changes only
those, and answers the whole document it stored. A field left out keeps its
stored value; `null` sets `effort` or `acp` to none. A form that sends the
model, effort and limit alone therefore leaves the runtime as it was. No secret
goes in the document: an `api_key` field is an unknown field.
The controller reads the settings for every run and sends the runtime, model
and effort in the run request, so a change applies from the next run with no
restart. A try-out reads them when it's posted, a review when it gets its
place. Lowering the limit lets the runs already going finish. A run request
with no model runs on `haiku`.
with no model runs on `haiku`, and one with no runtime on the API key.
The PUT needs the `Remote-User` header the gateway sets, as the try-out POST
does. Every error is problem+json with the reason in `detail`:
- **400** for a body that isn't JSON or isn't this shape (a missing `model`
or `max_concurrent_reviews`, an unknown field, an effort outside the five
levels, a `max_concurrent_reviews` that isn't a whole number), an empty
`model`, one with whitespace or one starting with `-`, or a
`max_concurrent_reviews` outside 1 to 8
- **400** for a body that isn't JSON or isn't this shape: an unknown field, a
`null` other than for `effort` or `acp`, an effort outside the five levels,
a `max_concurrent_reviews` that isn't a whole number, a runtime other than
the two, or an `acp` without `base_url`, `model` or `prices`. Also **400**
when the document the PUT would store breaks a rule: an empty `model`, one
with whitespace or one starting with `-`, a `max_concurrent_reviews` outside
1 to 8, an `acp.base_url` that isn't `https://`, an empty `acp.model` or one
with whitespace, a negative price, or the `acp` runtime with no `acp`
- **401** for a PUT with no `Remote-User`
- **503** when this host has no swarm queue, or it isn't connected
- **500** when the controller can't read or write the settings
When the controller can't read the settings, a try-out answers the same 503
or 500, and a review posts a no-verdict comment. The runtime (see
[Runtimes](#runtimes)) isn't a setting yet, and the operator writes the API
key by hand in the secret store
or 500, and a review posts a no-verdict comment. The operator writes each
runtime's key by hand in the secret store
([`services.md`](services.md#review-classifier)).
The per-agent equivalent is [`inference.md`](inference.md), where the
@ -251,7 +285,7 @@ An answer:
The verdict field holds `match`, `no-match` or `not-applicable`, and the
certainty field a whole number. A runtime error, a timeout, or an answer that
isn't that JSON object leaves the classification without a verdict: it's listed
with its error, and doesn't count. A run with no API key, or in which no
with its error, and doesn't count. A run with no key for its runtime, or in which no
classification got an answer, fails as a whole, and shows its error instead of a
verdict.
@ -304,14 +338,44 @@ link.
## Runtimes
The classifier runs on one of three runtimes, chosen at swarm level:
The classifier runs on one of two runtimes, the settings' `runtime`:
- **claude subscription**: the `claude` CLI on a Claude subscription login
- **API key**: the `claude` CLI on an API key
- **ACP**: an ACP agent and its provider, as for an ACP agent today
- **API key** (`api-key`, the default): the `claude` CLI on the Anthropic API
key at `secret/swarm/services/review-classifier/anthropic-api-key`, at
`model` and `effort`
- **ACP** (`acp`): `opencode` on the provider `acp` names, with the provider
key at `secret/swarm/services/review-classifier/acp-provider`, at
`acp.model`
Model names don't carry across runtimes. A claude alias means nothing to an
ACP provider, so the model setting belongs to the runtime it names.
ACP provider, so each runtime has a model field of its own.
To switch to ACP, an admin:
1. writes the provider key in the secret store's UI under the OIDC login, at
`secret/swarm/services/review-classifier/acp-provider`, field `api_key`
2. PUTs the runtime and the provider:
```json
{
"runtime": "acp",
"acp": {
"base_url": "https://llm.example/v1",
"model": "Qwen/Qwen3-35B",
"prices": { "input": 0.5, "output": 2 }
}
}
```
The next run goes to that provider. `{"runtime": "api-key"}` switches back and
keeps the provider for next time.
A split deploy runs on the API key: a classifier older than the ACP runtime
ignores `runtime` and runs every classification on `claude`, and a controller
older than it ignores the stored `runtime` and `acp` and sends neither.
A third runtime, **claude subscription** (the `claude` CLI on a Claude
subscription login), comes later.
> **Open point.** The subscription runtime needs a login, which is the one
> piece of durable state the service would keep. A swarm-level claude
@ -442,11 +506,14 @@ claude runtimes, every classifier run starts its `claude` process with
the metrics store by repo and PR. The process exports to the hive-tier collector on
the classifier's host, where that host runs one. On a subscription, cost is notional.
On the ACP runtime, the harness counts the agent's `usage_update` running
totals and exports them as `hyperhive.agent.cost.usage`, labelled with only
`model` and `currency`
([`observability.md`](../scheduler/observability.md#acp-reported-cost)). That
cost has no repo or PR label yet.
On the ACP runtime, the service computes each classification's cost itself,
from the settings' `acp.prices` and the token counts the agent reports for the
prompt, and adds it to the counter `hyperhive.review_classifier.cost.usage`,
labelled `repo`, `pr`, `model` (the provider's) and `currency` (`USD`). It
exports to the same collector `claude` does. A classification stopped at its
deadline, or whose agent reports no token counts, adds nothing. opencode
reports no cost of its own for a provider without prices in its config, which
is why the prices are a setting.
The classifier logs each run's result to its journal, which ships to
VictoriaLogs like every swarm service container's
@ -468,17 +535,9 @@ These get settled, and filled in here, as the work reaches them:
whether anything gets redacted.
- **Prompt injection.** A PR's diff and commit messages are model input, so
a PR can try to talk the classifier into approving it.
- **The "no tools" mechanism on ACP.** How the ACP runtime runs a session with
no tools at all. The claude runtimes pass `--tools ""`.
- **Credential paths.** Where in bao the ACP and subscription runtimes'
credentials live. The API key is at
`secret/swarm/services/review-classifier/anthropic-api-key`, field `api_key`,
and the service account's forge token at
- **Subscription credential.** Where in bao the subscription runtime's login
lives. The service account's forge token is at
`secret/swarm/services/review-classifier/forge/token`.
- **Model naming across runtimes.** How the swarm-level model setting
handles runtime-specific model names.
- **ACP cost labels.** How an ACP run's cost gets a repo and PR label, so
it reads from the metrics store like the claude runtimes' cost.
- **Reviewer handoff.** How a later reviewer, human or agent, learns the
classifier passed a PR before it starts its own review.
- **Try-out branch access.** The service account's access for pushing try-out

27
docs/swarm/secrets.md
View file

@ -407,15 +407,15 @@ a role and a policy of its own, and reaches one path, except the review
classifier's, the builder's, the forge host's and Grafana's, which reach two
each:
| unit | option pair under `deploy.bao.` | reads, or writes |
| --------------------------------------------------- | ------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------- |
| `swarm-bao-queue-agent` | `queueAgentClientCertFile` / `queueAgentClientKeyFile` | `swarm/hives/<hive>/queue/agent` |
| `swarm-bao-grafana-oidc`, `swarm-bao-grafana-admin` | `grafanaOidcClientCertFile` / `grafanaOidcClientKeyFile` | reads `swarm/services/<grafana client id>/oidc/client`; creates and reads `swarm/services/<grafana client id>/admin/password` |
| `swarm-bao-otel-oidc` | `otelOidcClientCertFile` / `otelOidcClientKeyFile` | `swarm/services/<collector client id>/oidc/client` |
| `swarm-bao-forwarder-oidc` | `forwarderOidcClientCertFile` / `forwarderOidcClientKeyFile` | `swarm/services/<store forwarder client id>/oidc/client` |
| `swarm-bao-ci-runner` | `ciRunnerClientCertFile` / `ciRunnerClientKeyFile` | `swarm/services/ci-runner/registration-token` |
| `swarm-review-classifier`, in its container | `reviewClassifierClientCertFile` / `reviewClassifierClientKeyFile` | `swarm/services/review-classifier/anthropic-api-key` and `swarm/services/<classifier client id>/oidc/client` |
| `swarm-bao-forge-publisher`, `swarm-bao-forge-oidc` | `forgePublisherClientCertFile` / `forgePublisherClientKeyFile` | writes `swarm/controller/swarm-controller/forge/token`; reads `swarm/services/<forge client id>/oidc/client` |
| unit | option pair under `deploy.bao.` | reads, or writes |
| --------------------------------------------------- | ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `swarm-bao-queue-agent` | `queueAgentClientCertFile` / `queueAgentClientKeyFile` | `swarm/hives/<hive>/queue/agent` |
| `swarm-bao-grafana-oidc`, `swarm-bao-grafana-admin` | `grafanaOidcClientCertFile` / `grafanaOidcClientKeyFile` | reads `swarm/services/<grafana client id>/oidc/client`; creates and reads `swarm/services/<grafana client id>/admin/password` |
| `swarm-bao-otel-oidc` | `otelOidcClientCertFile` / `otelOidcClientKeyFile` | `swarm/services/<collector client id>/oidc/client` |
| `swarm-bao-forwarder-oidc` | `forwarderOidcClientCertFile` / `forwarderOidcClientKeyFile` | `swarm/services/<store forwarder client id>/oidc/client` |
| `swarm-bao-ci-runner` | `ciRunnerClientCertFile` / `ciRunnerClientKeyFile` | `swarm/services/ci-runner/registration-token` |
| `swarm-review-classifier`, in its container | `reviewClassifierClientCertFile` / `reviewClassifierClientKeyFile` | `swarm/services/review-classifier/anthropic-api-key`, `swarm/services/review-classifier/acp-provider` and `swarm/services/<classifier client id>/oidc/client` |
| `swarm-bao-forge-publisher`, `swarm-bao-forge-oidc` | `forgePublisherClientCertFile` / `forgePublisherClientKeyFile` | writes `swarm/controller/swarm-controller/forge/token`; reads `swarm/services/<forge client id>/oidc/client` |
The first exists **per hive**, because the path it reads carries a hive name
and every hive runs its own reader. Its subject is
@ -534,10 +534,11 @@ The OIDC tab logs in through authelia as the client `swarm-bao-ui`
(`swarm.bao.ui.oidc.clientId`). An `admins` member gets a token under
`swarm-operator-viewer`: `list` and `read` on `secret/metadata/*`, and
`create` and `update` on
`secret/data/swarm/services/review-classifier/anthropic-api-key`. That shows
the key tree and each key's versions and timestamps, and no value: the store
refuses every `secret/data/` read. The one write is the review classifier's API
key, which the operator fills in by hand. Anything more needs a token.
`secret/data/swarm/services/review-classifier/anthropic-api-key` and
`secret/data/swarm/services/review-classifier/acp-provider`. That shows the key
tree and each key's versions and timestamps, and no value: the store refuses
every `secret/data/` read. The two writes are the review classifier's keys,
which the operator fills in by hand. Anything more needs a token.
| piece | written by |
| ----------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |

19
docs/swarm/services.md
View file

@ -271,13 +271,18 @@ identity: the host that runs the store has it, and a services host with the
store elsewhere gets the classifier only once it holds a copy of the pair (see
below). Set `deploy.review-classifier.enable` directly to override either way.
The operator writes the Anthropic key by hand, in the secret store's UI under
the OIDC login, at
`secret/swarm/services/review-classifier/anthropic-api-key`, field `api_key`.
Nothing writes it for you: until it exists the service runs and every run fails
with `no API key…`. The service reads the key on every run, so writing or
removing it takes effect on the next run, with no restart. Every run spends
money on that key.
The operator writes the key of the runtime the swarm settings choose by hand,
in the secret store's UI under the OIDC login, field `api_key`:
- the API key runtime's Anthropic key at
`secret/swarm/services/review-classifier/anthropic-api-key`
- the ACP runtime's provider key at
`secret/swarm/services/review-classifier/acp-provider`
Nothing writes them for you: until the chosen one exists the service runs and
every run fails with `no API key…` or `no ACP provider key…`. The service reads
the key on every run, so writing or removing it takes effect on the next run,
with no restart. Every run spends money on that key.
It needs neither the forge nor a hive on its host. It reaches the queue and the
secret store by name, under its own `review-classifier` leaf. On a host