review classifier: run on ACP when the swarm settings choose it
The swarm settings gain `runtime` (`api-key`, the default, or `acp`) and an `acp` provider record: an https base URL, the provider's model, and its prices per million tokens. A PUT now changes only the fields its body names, so the settings form that sends model, effort and limit leaves the runtime alone; `null` clears `effort` or `acp`. The body stays strict, and the stored document keeps #5192's lenient read. The controller sends the runtime in each run request. On `acp` the classifier runs one prompt on a new `opencode acp` session per classification through hive-runtime's AcpRuntime, in a scratch directory, with a config rendered from the settings. opencode runs `read` and other tools without asking, so OPENCODE_PERMISSION {"*":"deny"} takes every tool away from the model, and a deny-all policy refuses any request on top. The title call, which resends the prompt, is turned off. The deadline cancels the turn, and a turn that does not end within 15s of that is dropped, which also covers an agent stuck before its prompt. The answer is the final assistant text, as ACP has no result event. The provider key is read per run from swarm/services/review-classifier/acp-provider, which the bao UI's admins login may create and update, and reaches opencode through AcpRuntime::with_provider_key and the trampoline's allowlist. Each ACP classification's cost is computed from the configured prices and the reported tokens, and added to hyperhive.review_classifier.cost.usage labelled repo, pr, model and currency. Part of #5096
This commit is contained in:
atlas
mara
parent
38ee68c31d
commit
1b4e764e5c
24 changed files with 1622 additions and 262 deletions
|
|
@ -75,6 +75,7 @@ of the cell says how.
|
|||
| `swarm/services/<clientId>/oidc/client` | authelia | that service's own reader, under its own certificate — the queue responder's under `swarm-nats-auth`'s, the homeserver's under `swarm-matrix-ctl`'s, the forge's under `swarm-forge-publisher`'s, the review classifier's under `swarm-review-classifier`'s; the bao UI's by the store's granter | must be stated | must be stated |
|
||||
| `swarm/services/ci-runner/registration-token` | `swarm-controller`, from the forge's admin API, at start and in a five-minute pass; its grant on the path is write-only | `swarm-bao-ci-runner` on the runner's host, under the runner's own certificate, into a file only `swarm-ci-runner-register` in the runner's container loads; no other principal's policy reads it | ❌ the forge hands out one token until an admin resets it; the controller writes a reset token within five minutes | ✅ `swarm-bao-ci-runner` reads it at boot, on deploy and every ten minutes, and restarts the runner when the value changed |
|
||||
| `swarm/services/review-classifier/anthropic-api-key` | the operator, by hand in the store's UI, field `api_key` | `swarm-review-classifier` in its container, under its own certificate, at the start of every run; held for that run and handed to `claude`'s environment only; no other principal's policy reads it | ❌ an operator's key; replaced by writing a new one | ✅ read on every run, so a written or removed key takes effect on the next run |
|
||||
| `swarm/services/review-classifier/acp-provider` | the operator, by hand in the store's UI, field `api_key` | `swarm-review-classifier` in its container, under its own certificate, at the start of every ACP run; held for that run and handed to `opencode`'s environment only; no other principal's policy reads it | ❌ an operator's key; replaced by writing a new one | ✅ read on every run, so a written or removed key takes effect on the next run |
|
||||
| `swarm/services/review-classifier/forge/token` | `swarm-controller`, with the forge's admin API, for its `review-classifier` account, at start and in a five-minute pass; the secret publisher's policy denies it | `swarm-controller` under its own certificate, before it decides whether to mint, and on every review it posts as the account; the classifier service's policy doesn't read it | ✅ the controller's pass re-mints when the stored token is missing or no longer matches the forge (last eight characters and scopes) | ✅ the controller reads the store on every review it posts |
|
||||
| `swarm/services/builder/forge/token` | `swarm-controller`, with the forge's admin API, for its `builder` account (read on every config repo through the `hives` team), at start and in a five-minute pass; the secret publisher's policy denies it | `swarm-builder` in its container, under its own certificate, on every build; held for that build and handed to `git` through the environment only; `swarm-controller` under its own certificate, before it decides whether to mint | ✅ the controller's pass re-mints when the stored token is missing or no longer matches the forge (last eight characters and scopes) | ✅ the builder reads the store on every build |
|
||||
| _(not in the store)_ a hive's mTLS leaf | the store's own PKI, or an operator placing it by hand | its own client, off disk — the exception above, because it's what makes every other row's pull possible | must be stated | must be stated |
|
||||
|
|
|
|||
|
|
@ -118,12 +118,22 @@ It looks like an agent from the inside, with three things missing: it has no
|
|||
full agent identity, no state directory and no tools. Every model session is
|
||||
ephemeral, and nothing carries over from one run to the next.
|
||||
|
||||
On the claude runtimes, each classification is one `claude --print` the
|
||||
On the API key runtime, each classification is one `claude --print` the
|
||||
service spawns through the `hive-claude` crate directly, with no session to
|
||||
resume, no tools (`--tools ""`), no MCP server (`--strict-mcp-config` and no
|
||||
config) and an empty directory of its own. A classification needs no durable
|
||||
session, so the service doesn't use [`hive-runtime`](../../hive-runtime/README.md),
|
||||
whose `Runtime` trait drives the agents' durable sessions.
|
||||
config) and an empty directory of its own.
|
||||
|
||||
On the ACP runtime, each classification is one prompt on a new `opencode acp`
|
||||
session in an empty directory of its own. The service drives it through
|
||||
[`hive-runtime`](../../hive-runtime/README.md)'s `AcpRuntime`, as the subagent
|
||||
daemon does a one-run subagent, and stops the agent when the classification
|
||||
ends. The agent has no tool. opencode runs some tools, `read` among them,
|
||||
without asking the client, so the service sets opencode's own permission to
|
||||
deny every tool (`OPENCODE_PERMISSION={"*":"deny"}`), which leaves the model
|
||||
with no tool offered at all, and refuses any permission request on top. The
|
||||
config it writes for opencode names one provider and model, from the settings,
|
||||
and turns off opencode's title call, which would send the prompt a second
|
||||
time.
|
||||
|
||||
The service is `swarm-review-classifier`, and its container is the one
|
||||
[`services.md`](services.md#review-classifier) describes. It holds no forge
|
||||
|
|
@ -131,51 +141,75 @@ credential: `swarm-controller` reads the PR and sends it along.
|
|||
|
||||
### Swarm-level settings
|
||||
|
||||
The model, the effort and how many requested reviews run at once are one
|
||||
document for the whole swarm. `swarm-controller` keeps it in its
|
||||
The runtime, the model, the effort and how many requested reviews run at once
|
||||
are one document for the whole swarm. `swarm-controller` keeps it in its
|
||||
`review-classifier-settings` KV bucket under the key `all`, and no other
|
||||
service reads it. With no document stored, the classifier runs on `haiku` at
|
||||
claude's default effort, two reviews at a time. The controller
|
||||
ignores a stored field it doesn't know, and a document it still can't decode
|
||||
counts as none, with a warning in its log, until the next PUT replaces it.
|
||||
service reads it. With no document stored, the classifier runs on the API key
|
||||
runtime on `haiku` at claude's default effort, two reviews at a time. The
|
||||
controller ignores a stored field it doesn't know, and a document it still
|
||||
can't decode counts as none, with a warning in its log, until the next PUT
|
||||
replaces it.
|
||||
|
||||
`GET /api/review-classifier/settings` answers the settings in force.
|
||||
`PUT /api/review-classifier/settings` replaces them whole and answers what it
|
||||
stored. Both take and give one shape:
|
||||
`GET /api/review-classifier/settings` answers the settings in force:
|
||||
|
||||
```json
|
||||
{ "model": "haiku", "effort": null, "max_concurrent_reviews": 2 }
|
||||
{
|
||||
"model": "haiku",
|
||||
"effort": null,
|
||||
"max_concurrent_reviews": 2,
|
||||
"runtime": "api-key",
|
||||
"acp": null
|
||||
}
|
||||
```
|
||||
|
||||
- `model`: the model every classification runs on, as claude's `--model`
|
||||
takes it: one word with no whitespace, not starting with `-`
|
||||
- `effort`: `low`, `medium`, `high`, `xhigh` or `max`; `null`, or left out of
|
||||
a PUT, passes no `--effort`
|
||||
- `model`: the model every classification on the API key runtime runs on, as
|
||||
claude's `--model` takes it: one word with no whitespace, not starting with
|
||||
`-`
|
||||
- `effort`: `low`, `medium`, `high`, `xhigh` or `max`; `null` runs at the
|
||||
runtime's own default. On the ACP runtime it applies only where the agent
|
||||
offers it
|
||||
- `max_concurrent_reviews`: 1 to 8, how many requested reviews run at once.
|
||||
Try-outs don't count toward it. Up to eight more reviews wait for a place
|
||||
- `runtime`: `api-key` or `acp` (see [Runtimes](#runtimes))
|
||||
- `acp`: the provider the ACP runtime runs on, or `null`. It stays stored
|
||||
while the runtime is `api-key`, so switching back and forth needs no
|
||||
re-entry. It holds:
|
||||
- `base_url`: the provider's OpenAI-compatible API root, `https://` only
|
||||
- `model`: the model as the provider names it, with no whitespace
|
||||
- `prices`: what the provider charges in USD per million tokens,
|
||||
`input` and `output`, and optionally `cache_read` and `cache_write`, which
|
||||
otherwise count as `input`. A price is 0 or more
|
||||
|
||||
The controller reads the settings for every run and sends the model and
|
||||
effort in the run request, so a change applies from the next run with no
|
||||
`PUT /api/review-classifier/settings` takes any of these fields, changes only
|
||||
those, and answers the whole document it stored. A field left out keeps its
|
||||
stored value; `null` sets `effort` or `acp` to none. A form that sends the
|
||||
model, effort and limit alone therefore leaves the runtime as it was. No secret
|
||||
goes in the document: an `api_key` field is an unknown field.
|
||||
|
||||
The controller reads the settings for every run and sends the runtime, model
|
||||
and effort in the run request, so a change applies from the next run with no
|
||||
restart. A try-out reads them when it's posted, a review when it gets its
|
||||
place. Lowering the limit lets the runs already going finish. A run request
|
||||
with no model runs on `haiku`.
|
||||
with no model runs on `haiku`, and one with no runtime on the API key.
|
||||
|
||||
The PUT needs the `Remote-User` header the gateway sets, as the try-out POST
|
||||
does. Every error is problem+json with the reason in `detail`:
|
||||
|
||||
- **400** for a body that isn't JSON or isn't this shape (a missing `model`
|
||||
or `max_concurrent_reviews`, an unknown field, an effort outside the five
|
||||
levels, a `max_concurrent_reviews` that isn't a whole number), an empty
|
||||
`model`, one with whitespace or one starting with `-`, or a
|
||||
`max_concurrent_reviews` outside 1 to 8
|
||||
- **400** for a body that isn't JSON or isn't this shape: an unknown field, a
|
||||
`null` other than for `effort` or `acp`, an effort outside the five levels,
|
||||
a `max_concurrent_reviews` that isn't a whole number, a runtime other than
|
||||
the two, or an `acp` without `base_url`, `model` or `prices`. Also **400**
|
||||
when the document the PUT would store breaks a rule: an empty `model`, one
|
||||
with whitespace or one starting with `-`, a `max_concurrent_reviews` outside
|
||||
1 to 8, an `acp.base_url` that isn't `https://`, an empty `acp.model` or one
|
||||
with whitespace, a negative price, or the `acp` runtime with no `acp`
|
||||
- **401** for a PUT with no `Remote-User`
|
||||
- **503** when this host has no swarm queue, or it isn't connected
|
||||
- **500** when the controller can't read or write the settings
|
||||
|
||||
When the controller can't read the settings, a try-out answers the same 503
|
||||
or 500, and a review posts a no-verdict comment. The runtime (see
|
||||
[Runtimes](#runtimes)) isn't a setting yet, and the operator writes the API
|
||||
key by hand in the secret store
|
||||
or 500, and a review posts a no-verdict comment. The operator writes each
|
||||
runtime's key by hand in the secret store
|
||||
([`services.md`](services.md#review-classifier)).
|
||||
|
||||
The per-agent equivalent is [`inference.md`](inference.md), where the
|
||||
|
|
@ -251,7 +285,7 @@ An answer:
|
|||
The verdict field holds `match`, `no-match` or `not-applicable`, and the
|
||||
certainty field a whole number. A runtime error, a timeout, or an answer that
|
||||
isn't that JSON object leaves the classification without a verdict: it's listed
|
||||
with its error, and doesn't count. A run with no API key, or in which no
|
||||
with its error, and doesn't count. A run with no key for its runtime, or in which no
|
||||
classification got an answer, fails as a whole, and shows its error instead of a
|
||||
verdict.
|
||||
|
||||
|
|
@ -304,14 +338,44 @@ link.
|
|||
|
||||
## Runtimes
|
||||
|
||||
The classifier runs on one of three runtimes, chosen at swarm level:
|
||||
The classifier runs on one of two runtimes, the settings' `runtime`:
|
||||
|
||||
- **claude subscription**: the `claude` CLI on a Claude subscription login
|
||||
- **API key**: the `claude` CLI on an API key
|
||||
- **ACP**: an ACP agent and its provider, as for an ACP agent today
|
||||
- **API key** (`api-key`, the default): the `claude` CLI on the Anthropic API
|
||||
key at `secret/swarm/services/review-classifier/anthropic-api-key`, at
|
||||
`model` and `effort`
|
||||
- **ACP** (`acp`): `opencode` on the provider `acp` names, with the provider
|
||||
key at `secret/swarm/services/review-classifier/acp-provider`, at
|
||||
`acp.model`
|
||||
|
||||
Model names don't carry across runtimes. A claude alias means nothing to an
|
||||
ACP provider, so the model setting belongs to the runtime it names.
|
||||
ACP provider, so each runtime has a model field of its own.
|
||||
|
||||
To switch to ACP, an admin:
|
||||
|
||||
1. writes the provider key in the secret store's UI under the OIDC login, at
|
||||
`secret/swarm/services/review-classifier/acp-provider`, field `api_key`
|
||||
2. PUTs the runtime and the provider:
|
||||
|
||||
```json
|
||||
{
|
||||
"runtime": "acp",
|
||||
"acp": {
|
||||
"base_url": "https://llm.example/v1",
|
||||
"model": "Qwen/Qwen3-35B",
|
||||
"prices": { "input": 0.5, "output": 2 }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The next run goes to that provider. `{"runtime": "api-key"}` switches back and
|
||||
keeps the provider for next time.
|
||||
|
||||
A split deploy runs on the API key: a classifier older than the ACP runtime
|
||||
ignores `runtime` and runs every classification on `claude`, and a controller
|
||||
older than it ignores the stored `runtime` and `acp` and sends neither.
|
||||
|
||||
A third runtime, **claude subscription** (the `claude` CLI on a Claude
|
||||
subscription login), comes later.
|
||||
|
||||
> **Open point.** The subscription runtime needs a login, which is the one
|
||||
> piece of durable state the service would keep. A swarm-level claude
|
||||
|
|
@ -442,11 +506,14 @@ claude runtimes, every classifier run starts its `claude` process with
|
|||
the metrics store by repo and PR. The process exports to the hive-tier collector on
|
||||
the classifier's host, where that host runs one. On a subscription, cost is notional.
|
||||
|
||||
On the ACP runtime, the harness counts the agent's `usage_update` running
|
||||
totals and exports them as `hyperhive.agent.cost.usage`, labelled with only
|
||||
`model` and `currency`
|
||||
([`observability.md`](../scheduler/observability.md#acp-reported-cost)). That
|
||||
cost has no repo or PR label yet.
|
||||
On the ACP runtime, the service computes each classification's cost itself,
|
||||
from the settings' `acp.prices` and the token counts the agent reports for the
|
||||
prompt, and adds it to the counter `hyperhive.review_classifier.cost.usage`,
|
||||
labelled `repo`, `pr`, `model` (the provider's) and `currency` (`USD`). It
|
||||
exports to the same collector `claude` does. A classification stopped at its
|
||||
deadline, or whose agent reports no token counts, adds nothing. opencode
|
||||
reports no cost of its own for a provider without prices in its config, which
|
||||
is why the prices are a setting.
|
||||
|
||||
The classifier logs each run's result to its journal, which ships to
|
||||
VictoriaLogs like every swarm service container's
|
||||
|
|
@ -468,17 +535,9 @@ These get settled, and filled in here, as the work reaches them:
|
|||
whether anything gets redacted.
|
||||
- **Prompt injection.** A PR's diff and commit messages are model input, so
|
||||
a PR can try to talk the classifier into approving it.
|
||||
- **The "no tools" mechanism on ACP.** How the ACP runtime runs a session with
|
||||
no tools at all. The claude runtimes pass `--tools ""`.
|
||||
- **Credential paths.** Where in bao the ACP and subscription runtimes'
|
||||
credentials live. The API key is at
|
||||
`secret/swarm/services/review-classifier/anthropic-api-key`, field `api_key`,
|
||||
and the service account's forge token at
|
||||
- **Subscription credential.** Where in bao the subscription runtime's login
|
||||
lives. The service account's forge token is at
|
||||
`secret/swarm/services/review-classifier/forge/token`.
|
||||
- **Model naming across runtimes.** How the swarm-level model setting
|
||||
handles runtime-specific model names.
|
||||
- **ACP cost labels.** How an ACP run's cost gets a repo and PR label, so
|
||||
it reads from the metrics store like the claude runtimes' cost.
|
||||
- **Reviewer handoff.** How a later reviewer, human or agent, learns the
|
||||
classifier passed a PR before it starts its own review.
|
||||
- **Try-out branch access.** The service account's access for pushing try-out
|
||||
|
|
|
|||
|
|
@ -407,15 +407,15 @@ a role and a policy of its own, and reaches one path, except the review
|
|||
classifier's, the builder's, the forge host's and Grafana's, which reach two
|
||||
each:
|
||||
|
||||
| unit | option pair under `deploy.bao.` | reads, or writes |
|
||||
| --------------------------------------------------- | ------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `swarm-bao-queue-agent` | `queueAgentClientCertFile` / `queueAgentClientKeyFile` | `swarm/hives/<hive>/queue/agent` |
|
||||
| `swarm-bao-grafana-oidc`, `swarm-bao-grafana-admin` | `grafanaOidcClientCertFile` / `grafanaOidcClientKeyFile` | reads `swarm/services/<grafana client id>/oidc/client`; creates and reads `swarm/services/<grafana client id>/admin/password` |
|
||||
| `swarm-bao-otel-oidc` | `otelOidcClientCertFile` / `otelOidcClientKeyFile` | `swarm/services/<collector client id>/oidc/client` |
|
||||
| `swarm-bao-forwarder-oidc` | `forwarderOidcClientCertFile` / `forwarderOidcClientKeyFile` | `swarm/services/<store forwarder client id>/oidc/client` |
|
||||
| `swarm-bao-ci-runner` | `ciRunnerClientCertFile` / `ciRunnerClientKeyFile` | `swarm/services/ci-runner/registration-token` |
|
||||
| `swarm-review-classifier`, in its container | `reviewClassifierClientCertFile` / `reviewClassifierClientKeyFile` | `swarm/services/review-classifier/anthropic-api-key` and `swarm/services/<classifier client id>/oidc/client` |
|
||||
| `swarm-bao-forge-publisher`, `swarm-bao-forge-oidc` | `forgePublisherClientCertFile` / `forgePublisherClientKeyFile` | writes `swarm/controller/swarm-controller/forge/token`; reads `swarm/services/<forge client id>/oidc/client` |
|
||||
| unit | option pair under `deploy.bao.` | reads, or writes |
|
||||
| --------------------------------------------------- | ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `swarm-bao-queue-agent` | `queueAgentClientCertFile` / `queueAgentClientKeyFile` | `swarm/hives/<hive>/queue/agent` |
|
||||
| `swarm-bao-grafana-oidc`, `swarm-bao-grafana-admin` | `grafanaOidcClientCertFile` / `grafanaOidcClientKeyFile` | reads `swarm/services/<grafana client id>/oidc/client`; creates and reads `swarm/services/<grafana client id>/admin/password` |
|
||||
| `swarm-bao-otel-oidc` | `otelOidcClientCertFile` / `otelOidcClientKeyFile` | `swarm/services/<collector client id>/oidc/client` |
|
||||
| `swarm-bao-forwarder-oidc` | `forwarderOidcClientCertFile` / `forwarderOidcClientKeyFile` | `swarm/services/<store forwarder client id>/oidc/client` |
|
||||
| `swarm-bao-ci-runner` | `ciRunnerClientCertFile` / `ciRunnerClientKeyFile` | `swarm/services/ci-runner/registration-token` |
|
||||
| `swarm-review-classifier`, in its container | `reviewClassifierClientCertFile` / `reviewClassifierClientKeyFile` | `swarm/services/review-classifier/anthropic-api-key`, `swarm/services/review-classifier/acp-provider` and `swarm/services/<classifier client id>/oidc/client` |
|
||||
| `swarm-bao-forge-publisher`, `swarm-bao-forge-oidc` | `forgePublisherClientCertFile` / `forgePublisherClientKeyFile` | writes `swarm/controller/swarm-controller/forge/token`; reads `swarm/services/<forge client id>/oidc/client` |
|
||||
|
||||
The first exists **per hive**, because the path it reads carries a hive name
|
||||
and every hive runs its own reader. Its subject is
|
||||
|
|
@ -534,10 +534,11 @@ The OIDC tab logs in through authelia as the client `swarm-bao-ui`
|
|||
(`swarm.bao.ui.oidc.clientId`). An `admins` member gets a token under
|
||||
`swarm-operator-viewer`: `list` and `read` on `secret/metadata/*`, and
|
||||
`create` and `update` on
|
||||
`secret/data/swarm/services/review-classifier/anthropic-api-key`. That shows
|
||||
the key tree and each key's versions and timestamps, and no value: the store
|
||||
refuses every `secret/data/` read. The one write is the review classifier's API
|
||||
key, which the operator fills in by hand. Anything more needs a token.
|
||||
`secret/data/swarm/services/review-classifier/anthropic-api-key` and
|
||||
`secret/data/swarm/services/review-classifier/acp-provider`. That shows the key
|
||||
tree and each key's versions and timestamps, and no value: the store refuses
|
||||
every `secret/data/` read. The two writes are the review classifier's keys,
|
||||
which the operator fills in by hand. Anything more needs a token.
|
||||
|
||||
| piece | written by |
|
||||
| ----------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
|
||||
|
|
|
|||
|
|
@ -271,13 +271,18 @@ identity: the host that runs the store has it, and a services host with the
|
|||
store elsewhere gets the classifier only once it holds a copy of the pair (see
|
||||
below). Set `deploy.review-classifier.enable` directly to override either way.
|
||||
|
||||
The operator writes the Anthropic key by hand, in the secret store's UI under
|
||||
the OIDC login, at
|
||||
`secret/swarm/services/review-classifier/anthropic-api-key`, field `api_key`.
|
||||
Nothing writes it for you: until it exists the service runs and every run fails
|
||||
with `no API key…`. The service reads the key on every run, so writing or
|
||||
removing it takes effect on the next run, with no restart. Every run spends
|
||||
money on that key.
|
||||
The operator writes the key of the runtime the swarm settings choose by hand,
|
||||
in the secret store's UI under the OIDC login, field `api_key`:
|
||||
|
||||
- the API key runtime's Anthropic key at
|
||||
`secret/swarm/services/review-classifier/anthropic-api-key`
|
||||
- the ACP runtime's provider key at
|
||||
`secret/swarm/services/review-classifier/acp-provider`
|
||||
|
||||
Nothing writes them for you: until the chosen one exists the service runs and
|
||||
every run fails with `no API key…` or `no ACP provider key…`. The service reads
|
||||
the key on every run, so writing or removing it takes effect on the next run,
|
||||
with no restart. Every run spends money on that key.
|
||||
|
||||
It needs neither the forge nor a hive on its host. It reaches the queue and the
|
||||
secret store by name, under its own `review-classifier` leaf. On a host
|
||||
|
|
|
|||
Loading…
Reference in a new issue