Watch
0
0
Fork
You've already forked hyperhive
0
hyperhive/docs/integrations/forge.md
atlas abc942cff3 docs: the swarm mints agent forge tokens; hive-c0re and tea-login no longer do
credentials.md gains the forge-token row and drops the claim that the
forge token never passes through the store. setup.md says plainly that
an agent spawned on the hive alone, ruth's bootstrap included, now gets
no forge user from anything. CLI references regenerated.

Refs #3782
2026-09-24 17:48:53 +02:00

349 lines
19 KiB
Markdown

# hive-forge
Private Forgejo instance running in a nixos-container, used as the
swarm's persistent code-collaboration surface (issues, PRs, reviews,
attachments). Configured via `services.hyperhive.swarm.forge.*` (what the
forge IS to every hive — names, ports, URLs) plus
`services.hyperhive.deploy.forgejo.*` (what the host running it decides —
which build it runs, how it's served, what it mirrors, where its
host-local secrets sit).
Container
shape, ROOT_URL / sub-domain routing, and operator-vs-in-cluster URL
handling live in [`docs/networking/gateway.md`](../networking/gateway.md); this file owns the
per-agent integration story and the notification pump that wakes
each agent on relevant activity.
## Token scopes
Two scope sets live in `hive-c0re::forge`:
**`TOKEN_SCOPES`** (per-agent tokens, and `hivectl forge create-user`
accounts). swarm-controller mints agent tokens with a byte-identical copy,
`forge::agent_token::AGENT_TOKEN_SCOPES`, pinned by a test:
| Scope | Why |
| -------------------- | ------------------------------------------------------------------------------------------------------- |
| `write:repository` | Create, clone, push, delete repos; merge PRs. |
| `write:issue` | Open / comment / review issues **and** pull requests (Forgejo namespaces PR conversation under issues). |
| `write:user` | Edit own profile, create repos under own user. |
| `write:organization` | Create + manage orgs (lets agents share a forge namespace). |
| `read:user` | Token-owner endpoint used for self-identification at harness startup. |
| `write:misc` | Hooks, attachments, the rest of the long tail. |
| `read:notification` | Poll `GET /notifications` for unread events. |
| `write:notification` | Mark notifications read via `PATCH /notifications/threads/{id}`. |
**`CORE_TOKEN_SCOPES`** (hive-c0re's own `core` user): everything in
`TOKEN_SCOPES` plus `read:admin` and `write:admin`. Site-admin
membership alone isn't sufficient — Forgejo's token scope gate runs
before the user-permission check, so `/api/v1/admin/*` returns
`403 Forbidden` for any token without the admin scope bits, even when
the bearer is a site admin.
**Migration note**: if `PATCH /api/v1/admin/users/{name}` returns 403
on an existing deploy, the core token predates the admin-scope
addition. Delete `/var/lib/hyperhive/forge-core-token` and restart
hive-c0re to re-mint with the new scopes.
---
## Per-agent forge accounts
Each agent gets its own Forgejo user and access token. swarm-controller
creates the user when it creates the agent, and mints one token named
`swarm-agent` with the admin API into the swarm secret store at
`swarm/agents/<agent>/forge-token`. A pass at start and every five minutes
re-mints any agent's token that's missing or no longer matches the forge;
a rotation deletes the old `swarm-agent` token first, so each agent holds at
most one. The agent fetches the token under its own store certificate into
`/run/hive-agent-forge-token/token` (`nix/agent-modules/forge-token.nix`),
and `hive-forge`, the git credential helper, the `forge_notify` poller and
the avatar sync read it from there, falling back to `<state>/forge-token`,
the file hive-c0re wrote before. hive-c0re no longer creates agent users or
mints agent tokens; the `hyperhive-<unix-seconds>` tokens it minted stay on
the forge until removed.
Two things live in the `agent-configs` Forgejo organization:
- A config repo per agent (`agent-configs/<name>`). The
agent is a **write collaborator on its own** repo — it can push
config-change branches and open config PRs (Forgejo `pull_request`
webhook at `/webhook/config-pr` queues a `MergeConfigPr` approval;
`hive-c0re/src/forge/config_pr_poll.rs` re-scans every 5 minutes as a
fault-tolerance backstop) — but
`main` is branch-protected core-only: only hive-c0re's verify-and-ff-push
merge handler lands on `main`, the operator team must approve first, and
the agent can neither push `main` directly nor self-merge. `main` is
fast-forward-only — hive-c0re never force-pushes (the merge handler's ff
push lands fine; the `push_config` mirror pushes `main` + the add-only
status tags without force, and treats a non-fast-forward rejection of
`main` after a rolled-back deploy as expected — the forge keeps the
approved history, the `failed/<id>` tag records the divergence).
Repos stay private, so an agent can't read another
agent's config. (Agents remain read-only collaborators on `core/meta`.)
hive-c0re also references this repo as the agent's **persistent meta
flake input** (`agent-<n>.url = git+http://<forge>/agent-configs/<n>.git`;
see [approvals.md § Meta flake](../agent-lifecycle/approvals.md)), fetching it as the `core`
user via a git credential helper that reads the live forge-core token —
so the config lives on the forge, not a hand-synced local checkout.
- The dashboard links each container's "config" anchor to this
config repo, so operators can go straight from the SW4RM tab into
the rendered repo without an extra `git` step.
The `hive-forge` CLI (separate workspace crate, see
[`README.md`](../README.md) file map) wraps the Forgejo REST API
with the per-agent token; agents call it for issue / PR / comment
ops as if it were a peer. All REST calls across the workspace
(`hive-forge` verbs, hive-c0re provisioning, this poller) go through
the typed `forgejo-api` crate; only non-`/api/v1` web-router routes
(attachment / artifact downloads, log streaming) and the poller's
enrichment fetches of server-provided subject URLs stay on raw
reqwest.
## Notification poller (`hive-forge-notify/src/notify.rs`)
Its own long-running per-agent daemon (`hive-forge-notify`, a sibling of
`hive-bash-daemon` / `hive-matrix-daemon`) — it used
to be a background task inside the `hive-agent` serve loop. Polls
`GET /api/v1/notifications?all=false` every 30 seconds (Forgejo's
unread-only filter), formats each notification as a broker
`Wake { from: "forge" }` message, and delivers it to the agent's own
inbox so claude's normal turn loop picks it up.
The crate builds a second, independent binary for a different host — see
[github.md](github.md#notifications).
The host-specific calls — list unread, mark read, resolve own login —
live behind `Source` in `hive-forge-notify/src/source.rs`;
classification, formatting, dedupe and todo delivery live in `notify.rs`,
common to both binaries. Todo
keys here are the bare thread ids, and must stay that way: renaming them
would orphan every in-flight forge todo on the first restart after a
deploy.
### Mark-read on delivery
On a **successful** broker delivery, `forge_notify` marks the thread
read on forge straight away (`PATCH /notifications/threads/{id}`). The
broker inbox is the durable work queue now — each delivered wake is a
sqlite row with its own ack lifecycle — so the forge unread flag no
longer needs to track whether the agent has _processed_ a
notification. Clearing it on delivery keeps forge's unread set **small
by construction**: at rest it holds only threads that failed to
deliver plus whatever arrived since the last 30s poll.
That size property is the whole point. A container rebuild starts the
poller with no memory of what it delivered, re-scans `?all=false`, and
finds nothing stale — the delivered threads are already read on forge.
Forge's own read-state is thus the durable, cross-rebuild record of
what `forge_notify` has delivered; there is **no persisted cursor**.
**Gotcha:** don't reintroduce a persisted dedup cursor here. A design
that leaves threads unread and tracks delivery via a separately persisted
cursor is fragile — losing that cursor across a rebuild re-delivers the
agent's entire still-unread backlog as a flood of fresh wakes. Forge's
own read-state is the only durable record this design needs.
**Read-before-comment guard doesn't block a fresh wake.** hive-forge's
read-before-comment guard (which keys off forge unread-state) doesn't
force the agent to view a thread before commenting on it: the broker
wake already carries the notification body, so _delivery is the read_.
An agent that wants the full thread still runs `hive-forge comments` /
`view`.
**In-process dedupe (small, ephemeral).** A single-process map (thread
id → last-delivered `updated_at`) guards the narrow window where a
mark-read call _transiently fails_ and the thread reappears unread in
the next poll before its `updated_at` bumps — so a flaky PATCH doesn't
re-fire the wake. it's **not persisted** and resets on restart (forge
read-state covers the durable case). Each poll prunes it to the ids in
the single `limit=UNREAD_FETCH_LIMIT` (50) fetch page, so it can never
exceed that many entries (a debug assertion pins the invariant; the
fetch limit and the bound are the same constant). `forge_notify` leaves
a failed _delivery_ unread and out of the map, so it resurfaces next tick.
`forge_notify` marks self-echo notifications (the agent's own writes, see below)
read directly without a delivery — same `mark_read` call, no wake.
### Activation gates (graceful no-ops)
The poller starts disabled and stays that way for any of:
- `HIVE_FORGE_URL` not set (no forge configured for this hive), or
not parseable as a URL.
- no token in `$HIVE_FORGE_TOKEN_FILE` or `<state>/forge-token` (agent has no forge
account — pre-provisioning or destroy-without-purge race).
- Initial client construction fails (the typed `forgejo-api` client
for the API calls, or the plain reqwest client kept for the
best-effort enrichment fetches of server-provided subject URLs;
both unlikely; treated as fatal-to-the-task only).
Disabled = the spawned task returns immediately. All other failure
modes (HTTP errors, parse errors, mark-read failures) are
best-effort: logged at debug/warn and retried next tick.
### Self-notification filtering
Forgejo fires notifications for the agent's own actions (it opened a
PR, posted a comment, submitted a review). Surfacing those would
loop claude on its own writes. `forge_notify` drops the comment/review case
silently (mark-read without delivery):
- **Self-authored comments / reviews** — comment payload's
`user.login` matches `own_login`.
- **Self-authored creations** (an agent opening its own PR/issue) — the
already-fetched subject payload's poster `user.login` matches
`own_login`. `forge_notify` drops only _creations_ this way; a later state
change on the agent's own subject comes from someone else, so it still surfaces.
`forge_notify` fetches `own_login` at startup via `GET /api/v1/user`. On fetch
failure the filter degrades open (no filtering) rather than crashing
the task — a noisy inbox beats a silently stuck poller — but the fetch
is **re-attempted on each poll tick** until it succeeds, so a boot-time
failure (the forge not yet reachable) self-heals instead of leaving
self-echo filtering off for the whole process lifetime.
### Body excerpt + truncation + heading escape
The wake message embeds the comment / review body so the agent sees
actual content without a follow-up fetch — **not** a new issue/PR's own
description, which the wrapper table below deliberately omits. Three
pipeline steps in order:
1. **Truncate** to `BODY_TRUNCATE = 500` chars at a char-boundary;
appends `…` when cut. Truncation happens BEFORE escape so the
mention-overflow diff (next step) compares like-for-like against
the raw body.
2. **Mention overflow extraction** — when truncation actually
trimmed content, walk the full body line-by-line and surface any
`@username` lines that fell outside the embed window. Rendered as
a trailing `mentions (truncated from body):\n > <line>` block.
Mention detection requires the `@` to be at line start or
following a non-username byte, so email-style `foo@bar.com` does
NOT count.
3. **ATX heading escape** — for each line that's a strict
`CommonMark` ATX heading (1-6 leading `#`s followed by a space,
tab, or end-of-line), prepend `\` so the embedded body doesn't
blow into a top-level h1/h2 inside the wrapper message when the
dashboard renders it. Lines like `#tag`, `#123`, `#!/bin/bash` <!-- lint:allow: syntax examples, not real tags -->
**aren't** headings — no escape, no cosmetic noise. Indented
"headings" inside lists / nested quotes keep their leading
whitespace.
The strict ATX rule is deliberate: `\#tag` and `#tag` render
identically, so an over-eager escape just adds visual clutter
without changing behavior. Setext-style headings (`title\n====`)
aren't handled — rarer in practice, would need multi-line
lookahead.
### Wrapper format
Five shapes, distinguished by the notification's classification:
| Trigger | Wrapper |
| ----------------------------------- | ------------------------------------------------------------------------------ |
| Comment on issue / PR | `[comment on PR #N owner/repo] title\nurl: ...\n\nauthor: body\nassignee: ...` |
| Review submission | `[PR approved #N owner/repo] title\nurl: ...\n\nauthor: body\nassignee: ...` |
| New issue / PR | `[new PR #N owner/repo] title\nurl: ...\nassignee: ...` |
| Later activity (open, not creation) | `[activity on PR #N owner/repo] title\nurl: ...\nassignee: ...` |
| State change | `[PR merged #N owner/repo] title\nurl: ...\nassignee: ...` |
The issue/PR's own description is deliberately never embedded here (only a
comment/review body is — that's the actual new content a notification
exists to surface): repeating a subject's own description on every "new"
or "activity on" wake for it's noise the agent already has via the URL,
not context (mara, triage: "just the issue title should be enough").
Review labels come from the Forgejo `state` field: `APPROVED` →
`approved`, `REQUEST_CHANGES` → `changes requested`, `COMMENT` →
`review comment`. `forge_notify` drops `PENDING` (review saved but not
submitted yet — no peer-visible event). Unknown states fall back to
the generic comment wrapper.
A review submitted with **no body** renders `reviewed by: <author>` in
place of the `<author>: <body>` line — deliberately worded to not collide
with the meta-suffix `reviewer:` line (requested reviewers, below).
### Merge/close vs a later comment
A notification carrying a `latest_comment_url` normally takes the comment
path. But a merged/closed subject **keeps** its `latest_comment_url` set,
so a just-merged PR that had any prior discussion would route to the
comment path and render `[comment on PR]` (with a stale pre-merge comment
body) instead of `[PR merged]` — the agent never learns its PR merged.
When the notification IS the merge/close transition — its
event time (`updated_at`) is within `NEW_ITEM_TOLERANCE_SECS` of the
subject's `closed_at` (set for both `merged` and `closed`) — the
state-change path wins even with a comment url present
(`state_change_is_current`). A genuine **later** comment on an
already-closed subject bumps `updated_at` well past `closed_at`, so it
stays on the comment path and keeps its comment body. Missing/unparseable
timestamps default to the state-change path, so a merge is never silently
hidden behind a stale comment.
#### Merge racing a comment
The one gap the timestamp cut leaves: a genuine comment posted **within
`NEW_ITEM_TOLERANCE_SECS` of the merge** bumps `updated_at` close enough
to `closed_at` that `state_change_is_current` returns `true` — so it takes
the state-change path, dropping its body. Best of both worlds:
on the merge/close path `forge_notify` fetches the `latest_comment_url` comment and, when
its `created_at` is strictly **after** the subject's `closed_at`
(`comment_is_after_close`) — that is, it raced the merge rather than being the
pre-merge last comment the subject keeps — append it as a
`comment by <author>: <excerpt>` block before the meta suffix
(`fresh_post_close_comment_tail`). The wake carries **both** `[PR merged]`
and the racing comment. `forge_notify` leaves the kept pre-merge comment (created before
`closed_at`) off, drops a self-authored racing comment (don't
echo the agent's own write), and appends nothing for a missing/unparseable `created_at`/
`closed_at` (conservative — only surface a comment it can
positively place after the close). Cost: one extra comment fetch on
merge/close notifications, acceptable given how rare they are.
### "new" vs "activity on"
A review submitted with **no body** carries no `latest_comment_url`,
so it misses the comment path and lands on the state-change path with
`state == "open"` — exactly like a freshly opened PR. Labeling that
`new PR` is misleading: agents dismiss it as a duplicate of the
original open notification and miss the review. The `open`
state only earns the `new <kind>` label when the notification's event
time (`updated_at`) is within `NEW_ITEM_TOLERANCE_SECS` (120s) of the
subject's `created_at`. `forge_notify` labels anything later `activity on <kind>`
— neutral and non-misleading, since it can't cheaply say _what_ the
activity was without an extra reviews fetch. Missing/unparseable
timestamps default to `new` (preserve prior behavior rather than mask a
genuine new item). A small dependency-free
RFC 3339 helper (`parse_rfc3339`) parses timestamps.
`forge_notify` extracts the number from `subject.html_url`'s last path segment
(strips `#anchor` first), and the repo slug from `repository.full_name`.
Both degrade gracefully when absent (number → blank, repo → blank)
so unexpected Forgejo shapes don't crash the formatter.
### Meta suffix
Every wrapper ends with one or more of:
- `assignee: <list>` — always present; `unassigned` when empty so
the line shape is stable.
- `reviewer: <list>` — PR notifications only, present only when
`requested_reviewers` is non-empty.
### Review-request override
For new PRs, the kind label flips to `[review requested #N
owner/repo]` when `own_login` appears in `requested_reviewers`,
regardless of the Forgejo `reason` field. Forgejo doesn't reliably
set `reason == "review_requested"` (often null instead), so the
fallback checks the subject payload directly. `is_new` gates detection
so the label only fires once on PR creation, not on every
subsequent comment.
### Subscription management
The poller does **not** autounsubscribe from repo watches — it
delivers every unread notification it's handed. A hive-forge CLI
subscription verb bounds the firehose explicitly (dropping broad repo
watches an agent doesn't need) — the poller never guesses which
watches to drop. See the `subscription` verb in
[`docs/tools/forge.md`](../tools/forge.md).