Watch
0
0
Fork
You've already forked hyperhive
0
Commit graph hyperhive/nix/module-eval/bao-grants.nix
Author SHA1 Message Date
atlas
525cbe0355 count readers, not writers, of the pull token in the cache's roster pin 2026-10-29 00:09:12 +01:00
atlas
f22f4ee6ac let the operator's oidc login write the cache's tokens and facts 2026-10-29 00:09:12 +01:00
atlas
099af86cd2 raise the cert role token ttl to 1h 2026-10-28 20:35:11 +01:00
atlas
00df3ccffb give cert-auth role tokens a 15m ttl so logins stop leaving 32-day leases 2026-10-28 20:35:11 +01:00
atlas
fd63b402e1 fetch the self-check's password as a systemd credential 2026-10-28 13:50:35 +01:00
atlas
ec06a91deb mint per-agent proxy credentials and a per-hive squid self-check credential 2026-10-28 13:50:35 +01:00
atlas
b845630023 keep the cache's key and pull token for hive-priv in a root-only directory 2026-10-28 10:23:01 +01:00
atlas
57b04d6c49 let the operator set, replace and remove an agent's backend env from the swarm ui 2026-10-28 08:49:32 +01:00
atlas
1e5070796d copy the cache's pull facts to each hive and create them in the in-swarm cache
the in-swarm cache stores its address, name and public key beside its tokens, create-only. swarm-controller copies the four cache facts to swarm/hives/<hive>/attic/* every five minutes, with read on the sources and write on the copies. part 1 of 3: nothing reads the copies until the hive reader and probe land.
2026-10-28 01:19:56 +01:00
atlas
d5c5862654 keep the matrix appservice registration in bao, with the host file as the last copy 2026-10-27 17:33:06 +01:00
atlas
a572fb6a7f forward the attic container's journal to the swarm collector 2026-10-27 16:03:23 +01:00
atlas
99fab06348 keep authelia's keys in bao, seeded from the existing key files 2026-10-27 16:01:57 +01:00
atlas
458a046b8d run attic as a swarm service with deploy.attic.enable 2026-10-27 13:04:17 +01:00
atlas
0849afec80 push green meta builds to the attic cache and record cached 2026-10-27 13:02:43 +01:00
atlas
a4e48a7bca store the forge webhook secret and grafana's secret key in bao, seeded from the existing files 2026-10-27 13:00:13 +01:00
atlas
31670da795 granter role no-op deletes the dead bootstrap token 2026-10-26 19:19:06 +01:00
atlas
3db835fb57 granter role unit: no-op when the granter login works, delete the token after success 2026-10-26 18:49:02 +01:00
atlas
03796f1964 add one shared templated hive policy keyed on the cert alias name 2026-10-26 11:36:55 +01:00
atlas
4cdf807afc add one shared templated agent policy and name each agent's bao entity after it 2026-10-25 21:50:06 +01:00
atlas
4d9b7e0ca9 agent store leaf: 7 days instead of 90 2026-10-25 10:23:35 +01:00
atlas
868c6cf96c drop the shared hive agent login
Every agent now connects to the queue and reads logs with its own secret
from the store (#5181), so the per-hive `hive-<h>-agent` OIDC client has
no reader left. Remove it end to end:

- hive-c0re no longer forwards `hive-queue-agent-{secret,client-id}` into
  containers, and no longer forwards HIVE_AGENT_OIDC_TOKEN_ENDPOINT; the
  agent module that loaded them (nix/agent-modules/queue.nix) is gone, its
  options refused with a removal message.
- The host reader (glue-queue-agent-credential.nix, swarm-bao-queue-agent),
  its store role and policy unit (swarm-bao-queue-agent-policy), its PKI
  leaf and the deploy.bao.queueAgent* / queue.agentCredentialDir options
  are gone; the options are refused with a removal message.
- authelia mints no `hive-<h>-agent` client, and swarm-secret-publish no
  longer writes swarm/hives/<h>/queue/agent.
- swarm-secret-client drops queue::agent_client_path / queue::Credential.
- HIVE_AGENT_NATS_URL is now gated on the agents' queue address alone.

The log store's authelia check stays for operators. swarm-nats-auth still
refuses a `-agent` client id; that refusal and `agentClientSuffix` are left
for a follow-up.

Refs #4459
Refs #4765
2026-10-24 14:00:26 +02:00
atlas
0f7b403803 agents get a todo when assigned to an issue or PR, rewritten when unassigned and cleared on close
A new swarm service, swarm-notify-router, in its own container on the host
that sets deploy.notify-router.enable (defaulted on by allSwarmServices where
the host has the router's store leaf, like the claude login minter).

- swarm-controller registers instance-wide hooks for `issues` +
  `issue_assign` and `pull_request` + `pull_request_assign`, updates the
  event list of a hook already registered with fewer (leaving its active flag as it is), and keeps each
  verified body unchanged in the SWARM_EVENTS stream on
  $SWARM.events.forge.<X-Forgejo-Event-Type>, deduped by X-Forgejo-Delivery.
  A delivery the stream did not keep answers 503.
- swarm-notify-router drains its durable consumer on that stream, runs its
  flows (source -> match -> todo sink, stored as nodes plus edges and checked
  to be a tree) and queues each op on $SWARM.router.todo.<agent> in the
  work-queue stream ROUTER_TODOS. Built-in flows: assigned, unassigned (same
  key, rewritten) and closed (prefix-clears every holder's router todos). A
  per-issue snapshot in its notify-router-state bucket holds the last
  assignees and the todo holders; only agents in the agent-status bucket get
  ops, and an assignee that was not one is left out of the snapshot so the
  next assign event retries it. After the first event that must be retried,
  the rest of the batch is nakked so redelivery keeps stream order.
- hive-agent creates its own durable consumer todo-<agent> and applies each
  op to its todo store as subsystem `router`, then acks; a refused op stops
  the rest of its batch the same way. The store's (subsystem, key) dedup
  makes a redelivery harmless. New Todos::clear_prefix.
- swarm-nats-auth: a --notify-router-client arm; the controller may publish
  $SWARM.events.forge.*; each agent gets three {agent} templates (extended
  consumer create whose filter the server checks, MSG.NEXT, ACK).
- Credentials: a notify-router store leaf minted by glue-bao-tls, a
  swarm-notify-router bao policy (read on its own queue client secret), an
  authelia machine client, the secret publisher entry, a journal forwarder
  route. No manual step.

hive-forge-notify is unchanged, so agents also still get forge's own
notifications.

Refs #5571
2026-10-22 07:28:53 +02:00
atlas
9a65643769 swarm-bao granter stops retrying a rejected bootstrap token
swarm-bao-granter-role exits 4 when the store is unsealed but refuses the
bootstrap token, and Restart=on-failure retried that every 30s for as long
as the token file existed. RestartPreventExitStatus=4 makes that permanent
failure leave the unit failed once. Exit 1 (store sealed or unreachable)
keeps retrying.

On deploy: the unit goes red once instead of looping every 30s. The stale
token on muede-lpt2 still needs the operator step (remove it or mint a
fresh one); this does not clear it.
2026-10-21 09:22:19 +02:00
atlas
de4871cab1 swarm-bao: the agent-token checkers' cert roles issue batch tokens 2026-10-17 16:28:27 +02:00
atlas
e92f118bda add swarm-metrics, an agent CLI for the metrics store
swarm-metrics reads the swarm's VictoriaMetrics from inside an agent
container: query, query-range, series, labels and label-values, each one
GET against the store's Prometheus read API through the gateway, printing
the store's JSON. It authenticates the way swarm-logs does, with the
agent's own queue secret read from bao under the agent's leaf, sent as
a swarm-agent bearer.

On the metrics store's host the read API's auth map gains an agent arm
ahead of the Grafana one. It routes to swarm-metrics-agent-auth, a
swarm-logs-auth instance logged in under the existing swarm-metrics-auth
leaf, whose bao grant now includes swarm/agents/+/queue read. The check is
rate-limited per address like the log store's
(deploy.victoriametrics.agentAuth{Rate,Burst,VerdictTtl}).

nix/agent-modules/metrics.nix installs the wrapped binary when the hive
forwards HIVE_AGENT_METRICS_URL (from swarm.victoriametrics.domain) and a
store address, the same gate as swarm-logs. Adds the base:swarm-metrics
skill and the CLI reference with its freshness check.

Fixes #5395
2026-10-17 10:10:17 +02:00
atlas
b68a8ef290 swarm-authelia: run the secret publisher and a journal forwarder inside the container
swarm-secret-publish and its path unit now run inside authelia's container,
beside the mint, and read each client secret at the path authelia writes it.
The host unit and host path unit, which read the secrets through the
container's root on the host, are gone, and so is
deploy.authelia.hostClientSecretDir, which only they read. The publisher keeps
its store identity, the secret-publisher leaf. The host hands the leaf and the
store's CA into the container as nspawn system credentials, and the unit loads
them; where this host mints the leaf, the container waits for swarm-bao-pki.
Authelia does not wait for the publisher, which retries every 30 seconds while
the store is down.

The container also runs the journal forwarder the other service containers
run, pushing to a receiver of its own on the swarm collector (serviceProducers
swarm-authelia) as a new machine OIDC client (swarm.authelia.otel.clientId)
that authelia registers for itself and the publisher carries to the store. The
fetch unit reads it back under the publisher's leaf, also handed to it as a
system credential, so the swarm-secret-publisher policy gains read on that one
leaf; its stanza keeps create and update, which the exact path would otherwise
take from the services/* prefix. Without the leaf the forwarder is off.

The container drops --link-journal=host, and the host collector stops reading
it. The services' "nothing at <path>" errors name the publisher in authelia's
container.
2026-10-17 00:28:16 +02:00
atlas
b072fbf4d6 push ci run metrics to victoriametrics, with a grafana board
A swarm-ci-metrics host unit on a 5-minute timer lists every forge
repo's Actions runs as the new read-only ci-metrics account and pushes
each finished run's duration and queue wait over OTLP, stamped at the
run's stop time. The first pass backfills from run 1. swarm-controller
mints the account's token into bao and keeps a ci-metrics team with
read on the Actions unit in every organisation.

Closes #5376
2026-10-16 20:43:11 +02:00
atlas
265f8a47cc make bao readers want swarm-bao-pki and drop its start limit
A pki start refused by its start limit failed every reader that
Requires= it with `dependency`. That leaves a unit inactive, not
failed, so Restart=on-failure never fired and readers without a
refresh timer never ran again (#5381: grafana 502 for ~43 min).

Readers and granting units now Wants= + After= pki; one whose leaf is
still missing fails its login with exit 1 into its existing retry.
pki gets StartLimitIntervalSec=0: it is deliberately not
RemainAfterExit, so each reader's start re-runs it.
2026-10-16 18:29:38 +02:00
atlas
141be16c6d hive-forge: the controller-token mint writes straight to the store, and the on-disk token goes
`forgejo-swarm-controller-account`, in the forge's container, now puts
the swarm-controller's forge token into the swarm secret store itself,
under the forge publisher's leaf. The container already receives that
leaf as nspawn system credentials for its journal forwarder; the mint
loads the same three. The token reaches bao on stdin (`value=-`) and
the forge's API through a header file, never as an argument, and never
touches disk.

The mint's guard is a marker holding the published token's name, at
/var/lib/forgejo/swarm-controller-token-name, written last: after the
put, and after every other `swarm-controller-boot-*` token of the
account is revoked through forgejo's admin API, authenticated as the
new token. So a put that failed, or a marker lost while the store kept
an older token, leaves no live token behind once a mint completes. The
mint logs in to the store and checks that the account is a site admin
before it mints, so a store that is down or an account the admin API
would refuse mints nothing on each retry. Failures retry under
store-retry; forgejo does not wait for the mint. No bao read and no
grant change: the forge publisher keeps create/update on the token.

Removed: the host's `swarm-bao-forge-publisher` service and path unit,
which read the token file through the container's root, and their
entries in the store and policy ordering glue. The forge publisher's
policy unit stays; `swarm-bao-forge-oidc` still uses the leaf.

On deploy, an existing forge host has no marker, so the mint runs once:
it mints a new token, publishes it, revokes the old one, and deletes
the old /var/lib/forgejo/swarm-controller-token. The controller's next
forge call gets a 401, re-reads the store and carries on.
2026-10-16 17:48:56 +02:00
atlas
7294707d7d swarm-victoriametrics: forward the container's journal to the swarm collector
The metrics store's container now runs the journal forwarder the other
service containers run, pushing to a receiver of its own on the swarm
collector (serviceProducers swarm-victoriametrics) as a new machine OIDC
client (swarm.victoriametrics.otel.clientId), registered by a glue file
wherever authelia runs and published to the store by the secret publisher.

The container keeps its loopback-only netns. The collector cannot dial a
unix socket, so swarm-victoriametrics-egress-relay, a socket-activated
systemd-socket-proxyd inside the container, takes its connections on
127.0.0.1:4380 and passes them to /run/swarm-victoriametrics-egress/egress.sock.
The host's nginx listens there and sends /swarm-victoriametrics/ to the swarm
collector and /api/oidc/token to authelia, both by name and verified, and
refuses every other path. nginx gets ReadWritePaths on that directory and
removes a stale socket before it starts.

The container cannot reach the store, so the client secret is fetched on the
host by swarm-victoriametrics-forwarder-oidc under swarm-metrics-auth's leaf,
the shape swarm-bao-forwarder-oidc has, and bound into the container
read-only. The forwarder module gains hostSecretFile for that: set, it renders
no fetch unit. swarm-metrics-auth's policy gains one read stanza on the
forwarder's client path.

The container drops --link-journal=host, and the host collector stops reading
it.

Also lists the forge publisher's and matrix-ctl's leaves in the forwarder
secrets row of docs/swarm/secrets.md, which the hive-forge/hive-matrix change
missed.
2026-10-16 14:47:31 +02:00
atlas
ce44b6e77e hive-forge, hive-matrix: forward each container's journal to the swarm collector
Each container now runs the journal forwarder the CI runner's, Grafana's and
the log store's containers run, pushing to a receiver of its own on the swarm
collector (serviceProducers hive-forge and hive-matrix). Each authenticates as
a new machine OIDC client of its own (swarm.forge.otel.clientId,
swarm.matrix.otel.clientId), registered by a glue file wherever authelia runs,
minted by authelia and published to the store by the secret publisher. The
forge's and the homeserver's browser login clients are unchanged. The client
ids are swarm-forge-collector and swarm-matrix-collector, not
hive-<x>-collector, which is the client id a hive named <x>-collector gets.

The fetch unit inside each container reads the client secret with the store
identity the host already holds for that service. The forge publisher's leaf
is handed into the forge container as an nspawn system credential, as the
grafana and log store containers do. matrix-ctl's leaf is already bound into
the homeserver's container, so its forwarder reads it at those paths, as
swarm-matrix-appservice-publish does; with no matrix-ctl identity the
homeserver's forwarder is off. Each of the two policies gains one read stanza
on its forwarder's client path. The host units that use those identities are
unchanged.

The two containers drop --link-journal=host, and the host collector stops
reading them.
2026-10-16 13:50:23 +02:00
atlas
c6ca9ab845 swarm-ci-runner, swarm-grafana, swarm-victorialogs: forward each container's journal to the swarm collector
Each container now runs the journal forwarder the queue's and the builder's
containers run, pushing to a receiver of its own on the swarm collector. Each
authenticates as a new machine OIDC client of its own
(swarm.<service>.otel.clientId), registered by a glue file wherever authelia
runs, minted by authelia and published to the store by the secret publisher.
Grafana's browser login client is unchanged.

The fetch unit inside each container reads the client secret with the store
identity the host already holds for that service (the runner's, Grafana's,
and the log store responder's), now also handed into the container as an
nspawn system credential. Each of those three policies gains one read stanza
on its forwarder's client path. The host units that use those identities are
unchanged.

The three containers drop --link-journal=host, and the host collector stops
reading them.
2026-10-16 11:22:51 +02:00
atlas
eff5c43bdc grafana: query the log store with the metrics bearer, by its swarm name
Grafana's VictoriaLogs datasource now targets https://logs.<swarm> and sends
the same static bearer as the metrics datasource. The log store's vhost routes
a 64-hex Authorization bearer to a second swarm-metrics-auth instance
(swarm-logs-grafana-auth, cert role swarm-logs-auth, which gains a read on the
bearer's store path). swarm-agent. tokens and everything else keep their routes.

Part of #3742 (DoD 3, logs).
2026-10-16 02:38:47 +02:00
atlas
92559af8e7 gateway: serve the hive dashboard on <hive>.hives.<swarm> behind authelia, drop htpasswd
The dashboard, its API, the swagger UI and the agent routes move to their
own vhost, `swarm.dashboardDomain` = `<hive>.hives.<swarm.domain>`, under
the swarm domain so authelia's session cookie reaches it. `/`, `/api/` and
`/api/docs/` ask authelia through `auth_request`, and authelia's
`operatorsOnly` rule for each hive's dashboard name lets only the admins
group in. The hive domain keeps matrix discovery and redirects every other
path to the new name.

The name rides as a SAN on the hive's own gateway leaf: the per-hive bao
role allows it, and hive-tls asks for it and checks coverage.

HTTP Basic auth is gone: the htpasswd writer and its test, the
`hivectl gateway` verbs and host-socket requests, the `gateway.auth`
options (now a removed-option error), the unauthorized page and the bcrypt
dependency. tmpfiles deletes a leftover gateway.htpasswd, which held
bcrypt hashes at 0644.

c0re hands out dashboard and agent URLs on the new name
(HYPERHIVE_DASHBOARD_DOMAIN), and the swarm controller's hive roster
carries each hive's dashboard name for the swarm UI's hive list.

Closes #3469
2026-10-15 21:44:27 +02:00
atlas
7d09776742 swarm UI: an alerts page, read by the controller with a Viewer Grafana token from bao
swarm-bao-grafana-controller-token, on the host that runs Grafana, logs in
to Grafana's API over its socket as the built-in admin (the password
swarm-bao-grafana-admin minted, #5138) and keeps a `swarm-controller`
service account that is an enabled Viewer. It mints a token for that
account and writes it to bao at
swarm/controller/swarm-controller/grafana/token under Grafana's leaf. A
ten-minute timer runs it again. The unit keeps a stored token Grafana
accepts. If Grafana refuses the stored token with 401, or bao holds
none, the unit deletes every token of the account and mints a new one.
Grafana's grant gains create/read/update on that leaf, and the
controller's gains read.

swarm-controller serves GET /api/alerts from Grafana's
/api/prometheus/grafana/api/v1/alerts. It reads the token from bao,
keeps it in memory, and re-reads bao once on a 401. The HTTP client has
5s connect and 15s request timeouts. Any failure is a 503 that names its
cause. swarm-ui gets an /alerts page listing them, built the same way as
the hives page.

This is option (c) on #5129, the operator's choice.
2026-10-15 15:16:21 +02:00
atlas
d5de1e4943 swarm UI: log an agent into claude from its card; the credentials go to bao and the agent picks them up
A new swarm service, `swarm-claude-login`, in its own container on the host
that sets `deploy.claude-login.enable` (defaulted on by `allSwarmServices`
where the host has the minter's store leaf, like the review classifier).

- swarm-ui: an agent card showing "login needed" gets a "log in" button.
  Its dialog starts the login, shows the URL claude printed, and takes the
  pasted code.
- swarm-controller: `POST/DELETE /api/agents/{name}/claude-login` and
  `POST /api/agents/{name}/claude-login/code`, behind the swarm UI's
  authelia-gated `/api/`, relay the steps to the minter as requests on
  `$SWARM.claude-login.request`.
- swarm-claude-login: runs `claude auth login` over piped stdio with HOME
  and CLAUDE_CONFIG_DIR in a per-login scratch dir, scrapes the first
  https URL, writes the code to stdin, and stores the `.credentials.json`
  at `swarm/agents/<agent>/claude-credentials` with a KV v2 write, a new
  version over any earlier one. The scratch dir goes on every exit path; a
  login waits 15 minutes for its code.
- hive-agent: while parked in `wait_for_login`, reads that leaf every 30 s
  under the agent's own cert and installs it only when its KV version is
  newer than the one recorded in `<harness>/claude-credentials-version`.
  An equal or older version is never put back, because claude rotates the
  refresh token and the stored copy is stale after the first refresh.
- swarm-secret-client: `claude_login` paths and `SecretStore::read_versioned`.
- Credentials: a `claude-login` store leaf minted by glue-bao-tls, a
  `swarm-claude-login` policy (create/update on
  `swarm/agents/+/claude-credentials`, read on its own queue client
  secret), an authelia machine client, and a `--claude-login-client`
  grant in the queue's auth-callout responder. The unit runs as a static
  system user.

The hive's own web-UI login is unchanged.

Refs #5268
2026-10-15 13:26:29 +02:00
atlas
2ab8be61a2 swarm-bao, swarm-nats: bound each attempt of the hand-copied store-retry units
The granting units (granterPolicyUnit's 17, swarm-bao-granter-role,
swarm-bao-controller-policy, swarm-bao-agent-pki) and swarm-bao-nats-tls
copy store-retry's restart budget by hand, are Type=oneshot, and set no
TimeoutStartSec, so systemd gives each attempt no start timeout. A store
that accepts a connection and never answers leaves the attempt running
forever: Restart=on-failure never fires, units ordered after it wait, and
swarm-bao-nats-tls's daily timer starts a unit that is still activating.

Each now sets TimeoutStartSec: 60 for the granting units and the
granter's role, 2min for controller-policy (it may generate the services
root), 5min for agent-pki and nats-tls (each may wait on a blocking
restart of a server in a container). module-eval-bao-grants and
module-eval-nats-tls assert the bound is finite.

The restart budget itself is unchanged; that is #5305.

Closes #5306
2026-10-15 12:43:01 +02:00
atlas
6aab2ebb65 swarm metrics store: the read API needs Grafana's bearer or an operator session
The metrics vhost forwarded the store's read API (query, series, labels,
export, vmui, ...) with no credential, so anything that could reach
metrics.<swarm> could read every hive's and agent's metrics. mara ruled on
checked by an auth_request responder, and operators use their authelia
session.

- swarm-metrics-auth (new crate): auth_request responder on the metrics
  store's host. Compares the presented bearer, in constant time, with the
  one stored at swarm/services/<grafana client id>/metrics/bearer, read
  under its own bao leaf and cached for 60s. Static user, the post-#5264
  swarm-logs-auth unit shape.
- swarm-bao-grafana-metrics-bearer: on Grafana's host, creates that bearer
  in bao if absent (create without update, Grafana's existing leaf) and
  writes "Bearer <value>" into the container; the datasource sends it as
  its Authorization header via $__file{}.
- The vhost: read locations take auth_request through a map (a bearer goes
  to swarm-metrics-auth, anything else to authelia, no login redirect);
  vmui takes an authelia session with the login redirect; forceSSL like
  the logs vhost. The authelia subrequest location is renamed from
  /__metrics_push_authz to /__metrics_authelia since it now serves reads.
- Authelia: the collector keeps every path; everyone else is refused the
  collector's push and /metrics paths; group admins is admitted to the
  rest; then deny.
- bao: a swarm-metrics-auth cert role and policy (read on the one leaf),
  its leaf minted by glue-bao-tls, the CN reserved as a hive name, and
  Grafana's grant gains create on the bearer leaf.

Fixes #3596
2026-10-15 10:48:20 +02:00
atlas
1b4e764e5c review classifier: run on ACP when the swarm settings choose it
The swarm settings gain `runtime` (`api-key`, the default, or `acp`) and
an `acp` provider record: an https base URL, the provider's model, and
its prices per million tokens. A PUT now changes only the fields its
body names, so the settings form that sends model, effort and limit
leaves the runtime alone; `null` clears `effort` or `acp`. The body stays
strict, and the stored document keeps #5192's lenient read.

The controller sends the runtime in each run request. On `acp` the
classifier runs one prompt on a new `opencode acp` session per
classification through hive-runtime's AcpRuntime, in a scratch
directory, with a config rendered from the settings. opencode runs
`read` and other tools without asking, so OPENCODE_PERMISSION
{"*":"deny"} takes every tool away from the model, and a deny-all policy
refuses any request on top. The title call, which resends the prompt,
is turned off. The deadline cancels the turn, and a turn that does not
end within 15s of that is dropped, which also covers an agent stuck
before its prompt. The answer is the final assistant text, as ACP has
no result event.

The provider key is read per run from
swarm/services/review-classifier/acp-provider, which the bao UI's
admins login may create and update, and reaches opencode through
AcpRuntime::with_provider_key and the trampoline's allowlist.

Each ACP classification's cost is computed from the configured prices
and the reported tokens, and added to
hyperhive.review_classifier.cost.usage labelled repo, pr, model and
currency.

Part of #5096
2026-10-14 10:14:36 +02:00
atlas
47d3ccfede module-eval: bao-grants counts the eighteen granting units 2026-10-14 03:30:46 +02:00
atlas
b68b74d687 swarm-builder: build each agent config PR head and post a commit status
A new `swarm-builder` service, in its own container on the host that sets
`deploy.builder.enable` (defaulted on by `allSwarmServices` where the host
has the builder's store leaf, like the review classifier).

When an `agent-configs/<agent>` PR is opened, reopened or synchronized,
swarm-controller queues a `build_agent` jobq node. The node sets a
`swarm-builder` commit status to pending on the PR head, sends
`{agent, rev, pr}` to the builder over NATS request-reply
(`$SWARM.builder.build`), and writes success (store path), failure (last
line nix printed) or error (no builder / no reply) on the head. The node
fails when the build does, so the Jobs page matches the PR.

The builder builds nix/builder/check-flake's `check` with every input
overridden: hyperhive and nixpkgs to the host's own
`c0re.hyperhiveFlake`/`c0re.nixpkgsFlake` (what a hive there renders into
its meta flake), and the agent to the PR head on the forge. Evaluation is
pure and runs in the container; builds go to the host nix daemon as an
untrusted client. The private config repo is fetched with the token of a
new read-only `builder` forge account (in the `hives` team), which the
controller mints into `swarm/services/builder/forge/token`; git gets it
from a credential helper reading the child's environment, so it is in no
argument or URL.

Credentials: a `builder` store leaf minted by glue-bao-tls on the store
host, a `swarm-builder` reader policy (forge token + its own OIDC client
secret), an authelia machine client, and a `--builder-client` grant in the
queue's auth-callout responder. The controller gains a grant on the token
path and the secret publisher a deny on it.

This is the agent-config build check atlas scoped on #5081 (slice 3):
nothing is pushed to a cache, nothing is written to KV, and a red status
gates no deploy. The check composes agent-base + the config without the
values a hive adds (CA, docs, claude-code path, ports, input follows), so
it can disagree with a hive's own build.
2026-10-14 03:30:46 +02:00
atlas
1f4736e79b swarm-logs-auth: check agent tokens on the log store's query route
A new `auth_request` responder on the log store's host. nginx sends it
every `/select/logsql/` request whose `Authorization` is
`Bearer swarm-agent.<agent>.<secret>`; it answers 200 when the secret is
the one stored at `swarm/agents/<agent>/queue` (`swarm-agent-verify`),
401 otherwise. Every other bearer still goes to authelia, so agents
still presenting the hive's shared OIDC client keep working.

`auth_request` takes a fixed URI, so the split is a `map` on
`$http_authorization` plus an internal location that rewrites to the
map's pick. Checked against nginx 1.30.5 with stub upstreams: an agent
token only the stub validator accepts gets 200, a token only the stub
authelia accepts gets 200, every wrong one 401, and the internal
locations 404 from outside.

The responder serves a unix socket in its own traverse-only runtime
directory and reads the store under its own leaf
(`swarm-logs-auth`), whose policy is `swarm/agents/+/queue`, read-only.
glue-bao-tls mints the leaf where the store runs; elsewhere the
operator places it, and eval refuses a log store host without one.

Part of #4459.
2026-10-13 08:38:05 +02:00
atlas
18ed3f5040 module-eval: evaluate shared fixture configs once
Thirty-three module-eval fixtures were a second (or later) full
`nixosSystem` evaluation of a config another fixture already evaluated:
16 shapes defined identically in two to seven cluster files, plus
duplicates inside agent-queue-bao, core-toggle and
swarm-services-switch. Each cluster imports ./lib.nix itself, so nothing
was shared between them.

nix/module-eval/shared.nix now defines each of those 16 shapes once.
nix/checks.nix instantiates it a single time and passes it as `shared`
to the 23 clusters that read one, and each cluster rebinds the fixture
under its existing local name. Assertions are unchanged. Case text
changes only in core-toggle, where two cases built the forge-host
fixture inline and now read the file's `forgeHere`, and in two inline
`let` bindings (bao-otel-collector, grafana) that now take the shared
fixture.

Six folds are value-equal rather than textually equal: they drop an
explicit `deploy.hive-controller.enable = false` or
`deploy.swarm-otel.enable = false` that equals what the option already
resolves to for that config. Nothing under nix/host-modules sets either
option apart from the `mkDefault allSwarmServices` on swarm-otel, and
the only option-priority read there is `domain.highestPrio`.

Part of #5171.
2026-10-13 08:14:50 +02:00
atlas
48a728df17 swarm-bao-pki: sign every other hive's three store leaves
The store host's `swarm-bao-pki` now also signs, for every hive in
`swarm.hives` other than its own, the reader (`<hive>`), queue-agent
(`<queueAgentCommonNamePrefix>-<hive>`) and gateway-leaf
(`<hiveLeafCommonNamePrefix>-<hive>`) leaves, into
/var/lib/swarm-bao-pki/hives/<hive>/ with the basenames the own hive's
leaves use. Same CA, same `signLeaf`, idempotent on absence like every
other leaf in the unit. `signLeaf` now takes the CA from the top of the
PKI directory rather than from the leaf's directory; every existing
caller passed that directory, so their output is unchanged.

The operator still copies the files to the other hive's host and names
them in config; only the openssl step against the CA goes away
(mara on #5016, c95685: "ok mint").

Closes #5016
2026-10-12 21:41:28 +02:00
atlas
60ea7fdc04 swarm-controller: harden the review classifier's real reviews
Follow-ups from the review of #5142.

A `Done` answer with fewer or more rows than the classifications sent, or
none, now posts the no-verdict comment instead of the classifier's overall
verdict, so a short answer can't approve.

Requests are accepted per repo, PR and head into a set bounded at the two
running plus eight waiting. A request for a key already running or waiting
starts nothing and logs at info; a request with every place taken starts no
run and posts a no-verdict comment saying so. Before, every delivery spawned
a task waiting on the semaphore, so toggling the review request queued one
paid run per toggle.

Table cells get a zero-width space after `#`, `!`, `://` and `www.`, so a
model's reason can't render an issue or PR reference or a bare URL as a link.

The secret publisher's policy denies the classifier's forge token leaf, the
way #5139 denied Grafana's admin password, with a module-eval-bao-grants arm
pinning it. The controller posts reviews with whatever token is there, so a
publisher write picked the account reviews come from.

Closes #5143, part of #5096.
2026-10-12 18:38:05 +02:00
atlas
133d5578d2 bao: let the UI's admins login write the review classifier's API key
The docs tell the operator to write
`secret/swarm/services/review-classifier/anthropic-api-key` by hand in
the bao UI, but the UI's OIDC login for authelia's `admins` group lands
on `swarm-operator-viewer`, which is list+read on `secret/metadata/*`
and nothing under `secret/data/`. bao refuses the write, so an admin
can't set the key and every classifier run fails with "no API key".

The viewer policy gains `create` and `update` on that one leaf, no
`read`: a UI session still never sees a secret value. The leaf is a
named binding, shared with the classifier's reader policy.

module-eval pins the viewer's two stanzas and that no `data/` grant
carries `read`. docs/swarm/secrets.md and services.md describe the
write.

Closes #5148
2026-10-12 18:17:28 +02:00
atlas
a669cd96b6 swarm-controller: post the review classifier's verdict on a review request
Requesting the forge account `review-classifier` as a reviewer now runs the
repo's rules against the PR and posts the result as that account's review,
on the head commit the run read. A push starts no run; only a fresh review
request does.

- An instance-wide `pull_request_review_request` system hook delivers to
  `/webhook/forge/review-request`. A request for any other reviewer, or a
  removed request, does nothing.
- The controller reads the PR, loads the rules with the same reader as
  `GET /api/review-classifier/rules`, sends one request on the try-out
  subject, and posts APPROVED or REQUEST_CHANGES with the table as the body.
- With no clean verdict (PR or rules don't load, the run fails, or any
  classification has no answer) it posts a COMMENT saying why, never an
  approval. The event is one function, `no_clean_verdict_event`.
- Table cells are flattened, markup-escaped, and every `@` gets a
  zero-width space, so a model's reason can't break the table or mention
  anyone.
- At most two requested reviews run at once; the rest wait in memory.
- The account is minted the way hive accounts are: a five-minute pass
  queues a `MintClassifierForgeToken` node when the token at
  `swarm/services/review-classifier/forge/token` is missing or stale. The
  token is `write:repository`. The controller's bao policy gains that leaf
  (create/read/update), pinned in module-eval bao-grants. The classifier
  never sees the token.

The account gets no repo access automatically; a repo admin adds it by
hand. A re-request retires the account's earlier review: Forgejo clears
`official` on a reviewer's earlier reviews when it is requested again
(models/issues/review.go AddReviewRequest, v16.0.5), and approvals and
rejection blocks count official reviews only.

Refs #5137
2026-10-12 16:11:50 +02:00
atlas
eecd2b62e8 grafana: fix the sso.md admin claim, deny the publisher the admin password
docs/swarm/sso.md still said Grafana ships admin/admin and that a dead
provider locks out the built-in admin. The password is minted into the
store and the API still takes basic auth for it, so the paragraph now says
where an operator reads it in an emergency. secrets.md says how to rotate
the password and to deploy the store's host first, and no longer calls SSO
the only way in.

The secret publisher's `services/*` write covered the admin-password
leaf, so a file-copier could set the password Grafana applies at its next
start. The leaf gets the same exact-path deny as the callout seeds, with
an arm in module-eval-bao-grants.
2026-10-12 15:16:53 +02:00
atlas
993f5fd99d grafana: mint the admin password into the store and apply it on every start
Grafana's built-in admin kept the nixpkgs default password `admin`, and
Grafana's API takes basic auth whatever `auth.disable_login_form` says, so
the public vhost gave server-admin to `admin:admin`.

`swarm-bao-grafana-admin` creates a random password in the store under
Grafana's existing leaf when none is stored (`create` without `update`, so
the first one is kept), then copies it into the container.
`security.admin_password` reads it for a new database, and a `grafana cli
admin reset-admin-password` in `grafana.service`'s `preStart` applies it to
an existing one on every start.

Closes #5138
2026-10-12 15:16:53 +02:00
atlas
5d8ab675cd nix: run the review classifier in a container of its own when a host enables it
deploy.review-classifier.enable (off by default, not derived from
allSwarmServices) puts swarm-review-classifier in its own network
namespace on the bridge, with claude-code and the daemon. A host unit
copies the classifier's store leaf into the container; the daemon reads
its queue client secret and the API key from the store with it.

- glue-bao-tls mints the review-classifier leaf; swarm-bao writes the
  swarm-review-classifier role, reading exactly the API key path and the
  classifier's queue client secret; swarm.nix reserves its CN.
- glue-review-classifier-oidc-client registers the machine client where
  authelia runs, and the publisher carries its secret to the store.
- swarm-nats passes the client id to the responder as
  --review-classifier-client.
- Where the host runs a hive-tier collector, claude exports its metrics
  there; the daemon adds the repo and the PR to the resource attributes.
- module-eval: the new grant and leaf (bao-grants, now sixteen granting
  units), the container's presence and its unit (core-toggle), and the
  client registration and publishing (secret-publisher).
2026-10-12 13:11:21 +02:00