docs: move implementation sections out of the handbook pages
Three handbook pages carried sections an operator never reads, named after source paths and build internals. They move to pages under 4-implementation, and each handbook page keeps a short pointer. - forge.md: the notification poller (what it polls, how it formats, de-duplicates and marks notifications read) is now agent-tools/forge-notify.md. The forge page keeps a short Notifications section. - matrix.md: the retired hive-side matrix code, the evaluation assertions and the fluffychat-web build fixes are now networking/matrix-homeserver.md. - github.md: the Implementation section (how `gh` and `git push` authenticate, and the GitHub poller) is now agent-tools/github.md. docs/README.md and the agent-tools README list the new pages, and AGENTS.md's note on mixed operator and implementation docs now says to move the detail and leave a pointer.
This commit is contained in:
parent
b6af390581
commit
97671e28d5
9 changed files with 504 additions and 466 deletions
18
AGENTS.md
18
AGENTS.md
|
|
@ -222,10 +222,14 @@ in this file and drifted.
|
||||||
mechanical fix gets a one-line comment, a genuinely non-obvious
|
mechanical fix gets a one-line comment, a genuinely non-obvious
|
||||||
invariant or decision earns real prose. Don't restate a module's `//!`
|
invariant or decision earns real prose. Don't restate a module's `//!`
|
||||||
doc in a comment further down the same file.
|
doc in a comment further down the same file.
|
||||||
- **Operator-facing doc with real implementation detail mixed in**: wrap
|
- **Operator-facing doc with implementation detail mixed in**: a section
|
||||||
the implementation part in a `<details><summary>…</summary>` block
|
of source paths, wire formats or build internals belongs on a page under
|
||||||
(blank line after `<summary>` so the markdown inside renders) rather
|
`docs/4-implementation/`, with a short pointer left in the handbook page
|
||||||
than a sibling page — see `docs/2-handbook/external-accounts/github.md`'s
|
(see how `docs/2-handbook/swarm/forge.md` points to
|
||||||
"Implementation" section. It only collapses in a rendered browser, and
|
`docs/4-implementation/agent-tools/forge-notify.md`). A short aside that
|
||||||
the `<summary>` line isn't markdown-processed, so keep it plain prose
|
an operator may want but most do not goes in a
|
||||||
with no backticks.
|
`<details><summary>…</summary>` block (blank line after `<summary>` so the
|
||||||
|
markdown inside renders), as in `docs/2-handbook/swarm/schedules.md`'s
|
||||||
|
"Implementation" section. It only collapses in a rendered browser, and the
|
||||||
|
`<summary>` line isn't markdown-processed, so keep it plain prose with no
|
||||||
|
backticks.
|
||||||
|
|
|
||||||
|
|
@ -61,93 +61,8 @@ reads it: [credentials-reference.md](../swarm/credentials-reference.md).
|
||||||
scope is the real blast-radius limiter, and the container boundary is
|
scope is the real blast-radius limiter, and the container boundary is
|
||||||
the enforcement. See [security.md](../agents/security.md).
|
the enforcement. See [security.md](../agents/security.md).
|
||||||
|
|
||||||
## Implementation
|
## How it works
|
||||||
|
|
||||||
<details>
|
How the agent's `gh` and `git push` authenticate, and how its GitHub
|
||||||
<summary>How the agent's `gh`/`git push` actually authenticate, and how
|
notification poller works, is in
|
||||||
the notification poller works</summary>
|
[the implementation page](../../4-implementation/agent-tools/github.md).
|
||||||
|
|
||||||
### How the agent uses it
|
|
||||||
|
|
||||||
When enabled, the container gets:
|
|
||||||
|
|
||||||
- **A `gh` wrapper** on `PATH` (shadowing the raw `gh`) that exports
|
|
||||||
`GH_TOKEN` from the token file at invocation, then execs real `gh`. So
|
|
||||||
`gh pr create`, `gh api …`, etc. just work — `gh` derives the identity
|
|
||||||
from the token.
|
|
||||||
- **A git credential helper** (`git-credential-hive-github`), wired via a
|
|
||||||
host-scoped `/etc/gitconfig` entry for `https://github.com`, so
|
|
||||||
`git push https://github.com/<owner>/<repo>` authenticates as
|
|
||||||
`x-access-token` + the PAT (GitHub ignores the username for PAT auth).
|
|
||||||
Host-scoped, so it never touches the forge (`localhost:3000`) or any
|
|
||||||
other remote.
|
|
||||||
|
|
||||||
Both scripts read the token from `<state>/github-token` **at invocation
|
|
||||||
time**, so a PAT written (or rotated) mid-session takes effect immediately
|
|
||||||
— no container rebuild or restart. Until the file exists, `gh` / `git push`
|
|
||||||
fail unauthenticated. The build bakes the token path into the scripts
|
|
||||||
(rather than reading it from an env var), because claude's Bash tool runs in a
|
|
||||||
minimal environment that wouldn't carry one.
|
|
||||||
|
|
||||||
### Notifications
|
|
||||||
|
|
||||||
`hive-github-notify` polls github.com for the agent, turning each unread
|
|
||||||
notification thread into a todo. it's a **separate binary and a
|
|
||||||
separate systemd unit** from the internal forge's poller
|
|
||||||
(`hive-forge-notify`, see [forge.md](../swarm/forge.md#notification-poller-hive-forge-notifysrcnotifyrs)),
|
|
||||||
installed by `nix/agent-modules/github.nix` under
|
|
||||||
`services.hyperhive.agent.github.enable`. Both binaries ship from the one
|
|
||||||
`hive-forge-notify` derivation, so the unit is a second `ExecStart`
|
|
||||||
path, not a new package.
|
|
||||||
|
|
||||||
Two units rather than one daemon with two loops, because it puts the
|
|
||||||
decision in nix: a hive built without this module has **no github poller
|
|
||||||
in its closure at all**, which is what makes GitHub access separable
|
|
||||||
rather than merely switched off. it's also why this isn't a cargo
|
|
||||||
feature — a feature would unify across the workspace and cost every
|
|
||||||
crate its build cache.
|
|
||||||
|
|
||||||
At runtime the poller needs the PAT above. No PAT, no polling: the unit
|
|
||||||
logs why and exits 0, which is why it's `Restart = on-failure` and
|
|
||||||
never `always` — a clean exit on a PAT-less agent must not become a
|
|
||||||
restart loop. The poller waits about ten minutes for the PAT to appear
|
|
||||||
before it gives up. A path unit (`hive-github-notify.path`) starts the poller
|
|
||||||
again when `<state>/github-token` changes, so a PAT that lands later still
|
|
||||||
enables polling. Once polling, it re-reads the file on every
|
|
||||||
tick, so it picks up a rotated PAT on the next tick without a restart.
|
|
||||||
|
|
||||||
Forgejo models its notifications API on GitHub's, so one tolerant
|
|
||||||
parse serves both: `id`, `repository.full_name`,
|
|
||||||
`subject {title,url,latest_comment_url}` and `updated_at` line up field
|
|
||||||
for field. Lenient deserializers absorb the two real differences —
|
|
||||||
GitHub sends the thread id as a _string_ where Forgejo
|
|
||||||
sends a number, and says `PullRequest` where Forgejo says `Pull`.
|
|
||||||
`hive-github-notify` prefixes todo keys `gh:<id>` so a github thread id can't collide with a
|
|
||||||
forge one.
|
|
||||||
|
|
||||||
Two host differences worth knowing before touching this code:
|
|
||||||
|
|
||||||
- **Auth scheme, not just value.** Forgejo takes
|
|
||||||
`Authorization: token <t>`; GitHub wants `Bearer <t>` plus `Accept:
|
|
||||||
application/vnd.github+json`, `X-GitHub-Api-Version` and a
|
|
||||||
`User-Agent`. Sending Forgejo's form to GitHub doesn't error — it
|
|
||||||
authenticates as _nobody_ and silently drops to the unauthenticated
|
|
||||||
rate limit. The cheap way to tell the two apart is the rate-limit
|
|
||||||
header: `x-ratelimit-remaining` near 5000 is an authenticated user,
|
|
||||||
near 60 is anonymous.
|
|
||||||
- **GitHub sets the cadence.** It returns `X-Poll-Interval` (60s in
|
|
||||||
practice, slower than the poller's own tick) and rate-limits callers who
|
|
||||||
ignore it, so the loop re-arms to the server's interval whenever that's
|
|
||||||
_slower_ than the poller's. A hint faster than the poller's own tick
|
|
||||||
isn't a reason to poll harder.
|
|
||||||
|
|
||||||
⚠️ **This needs the `notifications` scope on the PAT.** A token minted
|
|
||||||
for `gh` + `git push` typically carries `repo` only, which is enough to
|
|
||||||
push and open PRs but **not** to read the notification stream (nor to
|
|
||||||
mark a thread read, which is the same scope). A PAT without it doesn't
|
|
||||||
break anything: the poller logs the refusal and stays quiet, and the
|
|
||||||
agent never gets GitHub wakes. If an agent's GitHub
|
|
||||||
notifications never arrive, check the token's scopes first — the
|
|
||||||
symptom is silence, not an error.
|
|
||||||
|
|
||||||
</details>
|
|
||||||
|
|
|
||||||
|
|
@ -123,279 +123,11 @@ the typed `forgejo-api` crate; only non-`/api/v1` web-router routes
|
||||||
enrichment fetches of server-provided subject URLs stay on raw
|
enrichment fetches of server-provided subject URLs stay on raw
|
||||||
reqwest.
|
reqwest.
|
||||||
|
|
||||||
## Notification poller (`hive-forge-notify/src/notify.rs`)
|
## Notifications
|
||||||
|
|
||||||
Its own long-running per-agent daemon (`hive-forge-notify`, a sibling of
|
Each agent runs a small daemon that polls its forge account for unread
|
||||||
`hive-bash-daemon` / `hive-matrix-daemon`) — it used
|
notifications every 30 seconds and hands each one to the agent as a todo, so
|
||||||
to be a background task inside the `hive-agent` serve loop. Polls
|
an assignment, a review or a comment wakes the agent. It starts once the agent
|
||||||
`GET /api/v1/notifications?all=false` every 30 seconds (Forgejo's
|
holds its forge token, and a token minted later starts it too. How it formats,
|
||||||
unread-only filter), formats each notification as a broker
|
de-duplicates and marks notifications read is in
|
||||||
`Wake { from: "forge" }` message, and delivers it to the agent's own
|
[the forge notification poller](../../4-implementation/agent-tools/forge-notify.md).
|
||||||
inbox so claude's normal turn loop picks it up.
|
|
||||||
|
|
||||||
The daemon waits about ten minutes for the agent's forge token, then exits 0,
|
|
||||||
which its `Restart=on-failure` doesn't restart. A path unit
|
|
||||||
(`hive-forge-notify.path`) starts it again when the fetched token file or the
|
|
||||||
state-dir `forge-token` changes, so a token minted later still starts the
|
|
||||||
polling.
|
|
||||||
|
|
||||||
The crate builds a second, independent binary for a different host — see
|
|
||||||
[github.md](../external-accounts/github.md#notifications).
|
|
||||||
|
|
||||||
The host-specific calls — list unread, mark read, resolve own login —
|
|
||||||
live behind `Source` in `hive-forge-notify/src/source.rs`;
|
|
||||||
classification, formatting, dedupe and todo delivery live in `notify.rs`,
|
|
||||||
common to both binaries. Todo
|
|
||||||
keys here are the bare thread ids, and must stay that way: renaming them
|
|
||||||
would orphan every in-flight forge todo on the first restart after a
|
|
||||||
deploy.
|
|
||||||
|
|
||||||
### Mark-read on delivery
|
|
||||||
|
|
||||||
On a **successful** broker delivery, `forge_notify` marks the thread
|
|
||||||
read on forge straight away (`PATCH /notifications/threads/{id}`). The
|
|
||||||
broker inbox is the durable work queue now — each delivered wake is a
|
|
||||||
sqlite row with its own ack lifecycle — so the forge unread flag no
|
|
||||||
longer needs to track whether the agent has _processed_ a
|
|
||||||
notification. Clearing it on delivery keeps forge's unread set **small
|
|
||||||
by construction**: at rest it holds only threads that failed to
|
|
||||||
deliver plus whatever arrived since the last 30s poll.
|
|
||||||
|
|
||||||
That size property is the whole point. A container rebuild starts the
|
|
||||||
poller with no memory of what it delivered, re-scans `?all=false`, and
|
|
||||||
finds nothing stale — the delivered threads are already read on forge.
|
|
||||||
Forge's own read-state is thus the durable, cross-rebuild record of
|
|
||||||
what `forge_notify` has delivered; there is **no persisted cursor**.
|
|
||||||
|
|
||||||
**Gotcha:** don't reintroduce a persisted dedup cursor here. A design
|
|
||||||
that leaves threads unread and tracks delivery via a separately persisted
|
|
||||||
cursor is fragile — losing that cursor across a rebuild re-delivers the
|
|
||||||
agent's entire still-unread backlog as a flood of fresh wakes. Forge's
|
|
||||||
own read-state is the only durable record this design needs.
|
|
||||||
|
|
||||||
**Read-before-comment guard doesn't block a fresh wake.** hive-forge's
|
|
||||||
read-before-comment guard (which keys off forge unread-state) doesn't
|
|
||||||
force the agent to view a thread before commenting on it: the broker
|
|
||||||
wake already carries the notification body, so _delivery is the read_.
|
|
||||||
An agent that wants the full thread still runs `hive-forge comments` /
|
|
||||||
`view`.
|
|
||||||
|
|
||||||
**In-process dedupe (small, ephemeral).** A single-process map (thread
|
|
||||||
id → last-delivered `updated_at`) guards the narrow window where a
|
|
||||||
mark-read call _transiently fails_ and the thread reappears unread in
|
|
||||||
the next poll before its `updated_at` bumps — so a flaky PATCH doesn't
|
|
||||||
re-fire the wake. it's **not persisted** and resets on restart (forge
|
|
||||||
read-state covers the durable case). Each poll prunes it to the ids in
|
|
||||||
the single `limit=UNREAD_FETCH_LIMIT` (50) fetch page, so it can never
|
|
||||||
exceed that many entries (a debug assertion pins the invariant; the
|
|
||||||
fetch limit and the bound are the same constant). `forge_notify` leaves
|
|
||||||
a failed _delivery_ unread and out of the map, so it resurfaces next tick.
|
|
||||||
|
|
||||||
`forge_notify` marks self-echo notifications (the agent's own writes, see below)
|
|
||||||
read directly without a delivery — same `mark_read` call, no wake.
|
|
||||||
|
|
||||||
### Activation gates (graceful no-ops)
|
|
||||||
|
|
||||||
The poller starts switched off and stays that way for any of:
|
|
||||||
|
|
||||||
- `HIVE_FORGE_URL` not set or not parseable as a URL (a hand-run
|
|
||||||
harness; hive-c0re sets it from `swarm.forge.domain` on every hive).
|
|
||||||
- no token in `$HIVE_FORGE_TOKEN_FILE` or `<state>/forge-token` (agent has no forge
|
|
||||||
account — pre-provisioning or destroy-without-purge race).
|
|
||||||
- Initial client construction fails (the typed `forgejo-api` client
|
|
||||||
for the API calls, or the plain reqwest client kept for the
|
|
||||||
best-effort enrichment fetches of server-provided subject URLs;
|
|
||||||
both unlikely; treated as fatal-to-the-task only).
|
|
||||||
|
|
||||||
Switched off = the spawned task returns immediately. All other failure
|
|
||||||
modes (HTTP errors, parse errors, mark-read failures) are
|
|
||||||
best-effort: logged at debug/warn and retried next tick.
|
|
||||||
|
|
||||||
### Self-notification filtering
|
|
||||||
|
|
||||||
Forgejo fires notifications for the agent's own actions (it opened a
|
|
||||||
PR, posted a comment, submitted a review). Surfacing those would
|
|
||||||
loop claude on its own writes. `forge_notify` drops the comment/review case
|
|
||||||
silently (mark-read without delivery):
|
|
||||||
|
|
||||||
- **Self-authored comments / reviews** — comment payload's
|
|
||||||
`user.login` matches `own_login`.
|
|
||||||
- **Self-authored creations** (an agent opening its own PR/issue) — the
|
|
||||||
already-fetched subject payload's poster `user.login` matches
|
|
||||||
`own_login`. `forge_notify` drops only _creations_ this way; a later state
|
|
||||||
change on the agent's own subject comes from someone else, so it still surfaces.
|
|
||||||
|
|
||||||
`forge_notify` fetches `own_login` at startup via `GET /api/v1/user`. On fetch
|
|
||||||
failure the filter degrades open (no filtering) rather than crashing
|
|
||||||
the task — a noisy inbox beats a silently stuck poller — but the fetch
|
|
||||||
is **re-attempted on each poll tick** until it succeeds, so a boot-time
|
|
||||||
failure (the forge not yet reachable) self-heals instead of leaving
|
|
||||||
self-echo filtering off for the whole process lifetime.
|
|
||||||
|
|
||||||
### Body excerpt + truncation + heading escape
|
|
||||||
|
|
||||||
The wake message embeds the comment / review body so the agent sees
|
|
||||||
actual content without a follow-up fetch — **not** a new issue/PR's own
|
|
||||||
description, which the wrapper table below deliberately omits. Three
|
|
||||||
pipeline steps in order:
|
|
||||||
|
|
||||||
1. **Truncate** to `BODY_TRUNCATE = 500` chars at a char-boundary;
|
|
||||||
appends `…` when cut. Truncation happens BEFORE escape so the
|
|
||||||
mention-overflow diff (next step) compares like-for-like against
|
|
||||||
the raw body.
|
|
||||||
2. **Mention overflow extraction** — when truncation actually
|
|
||||||
trimmed content, walk the full body line-by-line and surface any
|
|
||||||
`@username` lines that fell outside the embed window. Rendered as
|
|
||||||
a trailing `mentions (truncated from body):\n > <line>` block.
|
|
||||||
Mention detection requires the `@` to be at line start or
|
|
||||||
following a non-username byte, so email-style `foo@bar.com` does
|
|
||||||
NOT count.
|
|
||||||
3. **ATX heading escape** — for each line that's a strict
|
|
||||||
`CommonMark` ATX heading (1-6 leading `#`s followed by a space,
|
|
||||||
tab, or end-of-line), prepend `\` so the embedded body doesn't
|
|
||||||
blow into a top-level h1/h2 inside the wrapper message when the
|
|
||||||
dashboard renders it. Lines like `#tag`, `#123`, `#!/bin/bash` <!-- lint:allow: syntax examples, not real tags -->
|
|
||||||
**aren't** headings — no escape, no cosmetic noise. Indented
|
|
||||||
"headings" inside lists / nested quotes keep their leading
|
|
||||||
whitespace.
|
|
||||||
|
|
||||||
The strict ATX rule is deliberate: `\#tag` and `#tag` render
|
|
||||||
identically, so an over-eager escape just adds visual clutter
|
|
||||||
without changing behavior. Setext-style headings (`title\n====`)
|
|
||||||
aren't handled — rarer in practice, would need multi-line
|
|
||||||
lookahead.
|
|
||||||
|
|
||||||
### Wrapper format
|
|
||||||
|
|
||||||
Five shapes, distinguished by the notification's classification:
|
|
||||||
|
|
||||||
| Trigger | Wrapper |
|
|
||||||
| ----------------------------------- | ------------------------------------------------------------------------------ |
|
|
||||||
| Comment on issue / PR | `[comment on PR #N owner/repo] title\nurl: ...\n\nauthor: body\nassignee: ...` |
|
|
||||||
| Review submission | `[PR approved #N owner/repo] title\nurl: ...\n\nauthor: body\nassignee: ...` |
|
|
||||||
| New issue / PR | `[new PR #N owner/repo] title\nurl: ...\nassignee: ...` |
|
|
||||||
| Later activity (open, not creation) | `[activity on PR #N owner/repo] title\nurl: ...\nassignee: ...` |
|
|
||||||
| State change | `[PR merged #N owner/repo] title\nurl: ...\nassignee: ...` |
|
|
||||||
|
|
||||||
The issue/PR's own description is deliberately never embedded here (only a
|
|
||||||
comment/review body is — that's the actual new content a notification
|
|
||||||
exists to surface): repeating a subject's own description on every "new"
|
|
||||||
or "activity on" wake for it's noise the agent already has via the URL,
|
|
||||||
not context (mara, triage: "just the issue title should be enough").
|
|
||||||
|
|
||||||
Review labels come from the Forgejo `state` field: `APPROVED` →
|
|
||||||
`approved`, `REQUEST_CHANGES` → `changes requested`, `COMMENT` →
|
|
||||||
`review comment`. `forge_notify` drops `PENDING` (review saved but not
|
|
||||||
submitted yet — no peer-visible event). Unknown states fall back to
|
|
||||||
the generic comment wrapper.
|
|
||||||
|
|
||||||
A review submitted with **no body** renders `reviewed by: <author>` in
|
|
||||||
place of the `<author>: <body>` line — deliberately worded to not collide
|
|
||||||
with the meta-suffix `reviewer:` line (requested reviewers, below).
|
|
||||||
|
|
||||||
### Inline review comments
|
|
||||||
|
|
||||||
A review's line comments belong to neither the review body nor the
|
|
||||||
`latest_comment_url`. A review with only inline comments has no such url
|
|
||||||
and would otherwise land as a bare `activity on PR`. Every PR wake therefore
|
|
||||||
fetches the PR's reviews (`/pulls/N/reviews`, up to four pages of 50) and,
|
|
||||||
for the reviews new to the agent, appends an
|
|
||||||
`inline review comments (read them with `hive-forge pr reviews N`):` block
|
|
||||||
saying where they sit: one `author: path:line, path, ...` line per review
|
|
||||||
author (bare `path` when Forgejo dropped the position because the line left
|
|
||||||
the diff; control characters in a path become spaces). The comment text stays
|
|
||||||
out of the wake, and the agent reads the full review with the CLI.
|
|
||||||
|
|
||||||
A review counts as new when it landed strictly after the thread's previously
|
|
||||||
delivered `updated_at`. Without a baseline every review counts, and so does
|
|
||||||
one with no submit time, so nothing disappears. It doesn't count when the
|
|
||||||
agent wrote it, when it's a `PENDING` draft, or when `comments_count` is 0.
|
|
||||||
The notifier fetches the newest 5 such reviews and lists at most 10 locations
|
|
||||||
(`(+N more)` for the rest). Best effort: a failed fetch yields less text and
|
|
||||||
never drops the wake. Cost: one reviews fetch per PR wake, plus one per new
|
|
||||||
commented review.
|
|
||||||
|
|
||||||
### Merge/close vs a later comment
|
|
||||||
|
|
||||||
A notification carrying a `latest_comment_url` normally takes the comment
|
|
||||||
path. But a merged/closed subject **keeps** its `latest_comment_url` set,
|
|
||||||
so a just-merged PR that had any prior discussion would route to the
|
|
||||||
comment path and render `[comment on PR]` (with a stale pre-merge comment
|
|
||||||
body) instead of `[PR merged]` — the agent never learns its PR merged.
|
|
||||||
When the notification IS the merge/close transition — its
|
|
||||||
event time (`updated_at`) is within `NEW_ITEM_TOLERANCE_SECS` of the
|
|
||||||
subject's `closed_at` (set for both `merged` and `closed`) — the
|
|
||||||
state-change path wins even with a comment url present
|
|
||||||
(`state_change_is_current`). A genuine **later** comment on an
|
|
||||||
already-closed subject bumps `updated_at` well past `closed_at`, so it
|
|
||||||
stays on the comment path and keeps its comment body. Missing/unparseable
|
|
||||||
timestamps default to the state-change path, so a merge is never silently
|
|
||||||
hidden behind a stale comment.
|
|
||||||
|
|
||||||
#### Merge racing a comment
|
|
||||||
|
|
||||||
The one gap the timestamp cut leaves: a genuine comment posted **within
|
|
||||||
`NEW_ITEM_TOLERANCE_SECS` of the merge** bumps `updated_at` close enough
|
|
||||||
to `closed_at` that `state_change_is_current` returns `true` — so it takes
|
|
||||||
the state-change path, dropping its body. Best of both worlds:
|
|
||||||
on the merge/close path `forge_notify` fetches the `latest_comment_url` comment and, when
|
|
||||||
its `created_at` is strictly **after** the subject's `closed_at`
|
|
||||||
(`comment_is_after_close`) — in other words, it raced the merge rather than being the
|
|
||||||
pre-merge last comment the subject keeps — append it as a
|
|
||||||
`comment by <author>: <excerpt>` block before the meta suffix
|
|
||||||
(`fresh_post_close_comment_tail`). The wake carries **both** `[PR merged]`
|
|
||||||
and the racing comment. `forge_notify` leaves the kept pre-merge comment (created before
|
|
||||||
`closed_at`) off, drops a self-authored racing comment (don't
|
|
||||||
echo the agent's own write), and appends nothing for a missing/unparseable `created_at`/
|
|
||||||
`closed_at` (conservative — only surface a comment it can
|
|
||||||
positively place after the close). Cost: one extra comment fetch on
|
|
||||||
merge/close notifications, acceptable given how rare they are.
|
|
||||||
|
|
||||||
### "new" vs "activity on"
|
|
||||||
|
|
||||||
A review submitted with **no body** carries no `latest_comment_url`,
|
|
||||||
so it misses the comment path and lands on the state-change path with
|
|
||||||
`state == "open"` — exactly like a freshly opened PR. Labeling that
|
|
||||||
`new PR` is misleading: agents dismiss it as a duplicate of the
|
|
||||||
original open notification and miss the review. The `open`
|
|
||||||
state only earns the `new <kind>` label when the notification's event
|
|
||||||
time (`updated_at`) is within `NEW_ITEM_TOLERANCE_SECS` (120s) of the
|
|
||||||
subject's `created_at`. `forge_notify` labels anything later `activity on <kind>`
|
|
||||||
— neutral and non-misleading, since it can't cheaply say _what_ the
|
|
||||||
activity was without an extra reviews fetch. Missing/unparseable
|
|
||||||
timestamps default to `new` (preserve prior behavior rather than mask a
|
|
||||||
genuine new item). A small dependency-free
|
|
||||||
RFC 3339 helper (`parse_rfc3339`) parses timestamps.
|
|
||||||
|
|
||||||
`forge_notify` extracts the number from `subject.html_url`'s last path segment
|
|
||||||
(strips `#anchor` first), and the repo slug from `repository.full_name`.
|
|
||||||
Both degrade gracefully when absent (number → blank, repo → blank)
|
|
||||||
so unexpected Forgejo shapes don't crash the formatter.
|
|
||||||
|
|
||||||
### Meta suffix
|
|
||||||
|
|
||||||
Every wrapper ends with one or more of:
|
|
||||||
|
|
||||||
- `assignee: <list>` — always present; `unassigned` when empty so
|
|
||||||
the line shape is stable.
|
|
||||||
- `reviewer: <list>` — PR notifications only, present only when
|
|
||||||
`requested_reviewers` is non-empty.
|
|
||||||
|
|
||||||
### Review-request override
|
|
||||||
|
|
||||||
For new PRs, the kind label flips to `[review requested #N
|
|
||||||
owner/repo]` when `own_login` appears in `requested_reviewers`,
|
|
||||||
regardless of the Forgejo `reason` field. Forgejo doesn't reliably
|
|
||||||
set `reason == "review_requested"` (often null instead), so the
|
|
||||||
fallback checks the subject payload directly. `is_new` gates detection
|
|
||||||
so the label only fires once on PR creation, not on every
|
|
||||||
subsequent comment.
|
|
||||||
|
|
||||||
### Subscription management
|
|
||||||
|
|
||||||
The poller does **not** autounsubscribe from repo watches — it
|
|
||||||
delivers every unread notification it's handed. A hive-forge CLI
|
|
||||||
subscription verb bounds the firehose explicitly (dropping broad repo
|
|
||||||
watches an agent doesn't need) — the poller never guesses which
|
|
||||||
watches to drop. See the `subscription` verb in
|
|
||||||
[`docs/4-implementation/agent-tools/forge.md`](../../4-implementation/agent-tools/forge.md).
|
|
||||||
|
|
|
||||||
|
|
@ -327,14 +327,9 @@ Initial rollout settings:
|
||||||
stored yet the account comes up without backup and picks it up on the
|
stored yet the account comes up without backup and picks it up on the
|
||||||
next daemon start. A failure is a `WARN` and never stops the daemon.
|
next daemon start. A failure is a `WARN` and never stops the daemon.
|
||||||
|
|
||||||
## Retired hive-side matrix code
|
The module's retired hive-side code, its evaluation assertions and the
|
||||||
|
fluffychat-web build fixes are in the
|
||||||
Earlier versions of hive-c0re created a private "hive" Matrix Space and a
|
[implementation notes](../../4-implementation/networking/matrix-homeserver.md).
|
||||||
`hive-chat` room, invited every agent to both, kept the hive's sender token
|
|
||||||
on disk and offered `hivectl matrix invite`. Matrix is swarm-level, so
|
|
||||||
hive-c0re dropped all of it and no longer talks to the homeserver. The Space, the
|
|
||||||
room and the files `/var/lib/hyperhive/matrix/{access-token,space-room-id,chat-room-id}`
|
|
||||||
are harmless leftovers, so delete them by hand if you like.
|
|
||||||
|
|
||||||
## Configuration tuning
|
## Configuration tuning
|
||||||
|
|
||||||
|
|
@ -358,85 +353,3 @@ size of a single matrix client request body. Matches the matrix-spec
|
||||||
recommendation for media uploads and the upstream tuwunel default.
|
recommendation for media uploads and the upstream tuwunel default.
|
||||||
Raise for deployments that need large file transfers; lower for
|
Raise for deployments that need large file transfers; lower for
|
||||||
resource-constrained hosts where a 20 MB request is unexpectedly large.
|
resource-constrained hosts where a 20 MB request is unexpectedly large.
|
||||||
|
|
||||||
## Assertion rationale
|
|
||||||
|
|
||||||
`config.assertions` in this module fail eval early rather than ship
|
|
||||||
surprising behaviour:
|
|
||||||
|
|
||||||
- **`services.hyperhive.swarm.matrix.gatewayHost != ""`** — same footgun as `forge.domain`:
|
|
||||||
empty string renders `.<hive>`-shaped garbage in both nginx
|
|
||||||
`server_name` (treated as wildcard catch-all, surprising) and
|
|
||||||
`/etc/hosts` (malformed entry). `null` is the right opt-out shape;
|
|
||||||
a config assertion rejects empty string explicitly.
|
|
||||||
SSO is unconditional, so the three below are requirements of running a
|
|
||||||
homeserver at all rather than of a setting:
|
|
||||||
|
|
||||||
- **Keep `deploy.matrix.sso.clientSecretFile` set** — it defaults to
|
|
||||||
`/run/hive-matrix-oidc/<client id>.secret` wherever the homeserver runs,
|
|
||||||
and `swarm-bao-matrix-oidc` writes the secret there from the swarm secret
|
|
||||||
store under matrix-ctl's leaf. The path is on the host's tmpfs, so every
|
|
||||||
reboot fetches the secret again and it never reaches persistent disk. After
|
|
||||||
a reboot the homeserver doesn't start until the store answers; the
|
|
||||||
store-retry timer then starts it with no manual step. A path
|
|
||||||
outside `/run/hive-matrix-oidc/`, or a null, fails at eval, not at boot:
|
|
||||||
tuwunel reads its identity providers from the config file, so a half-configured
|
|
||||||
one can stop the homeserver from starting outright rather than merely
|
|
||||||
hiding a login button.
|
|
||||||
- **Set `swarm.authelia.url`** — without a provider URL there
|
|
||||||
is nothing to discover against.
|
|
||||||
- **Set `swarm.matrix.gatewayHost != null`** — the SSO callback URL is
|
|
||||||
format-locked to `<homeserver>/_matrix/client/unstable/login/sso/callback/<client_id>`,
|
|
||||||
and the identity provider needs a public name to redirect the
|
|
||||||
browser to.
|
|
||||||
|
|
||||||
`server_name`'s own bogus-value guard lives in `hive-network.nix`
|
|
||||||
(`services.hyperhive.domain != null`), not here — see
|
|
||||||
[`docs/4-implementation/networking/network.md`](../../4-implementation/networking/network.md).
|
|
||||||
|
|
||||||
## fluffychat-web build fixes
|
|
||||||
|
|
||||||
`pkgs.fluffychat-web` ships from `flutter341.buildFlutterApplication`,
|
|
||||||
which has two upstream gaps for fluffychat's web target:
|
|
||||||
|
|
||||||
- The dart web-worker entry point (`web/native_executor.dart`) isn't
|
|
||||||
compiled — `buildFlutterApplication` only runs `flutter build web`
|
|
||||||
on the main entry.
|
|
||||||
- `native_imaging`'s C source isn't built — emscripten isn't a
|
|
||||||
flutter-builder native build input.
|
|
||||||
|
|
||||||
Both fixed in `nix/host-modules/hive-matrix.nix` via two derivations:
|
|
||||||
|
|
||||||
- **`fluffychat-web-imaging`** builds `Imaging.{js,wasm}` from the
|
|
||||||
`native_imaging` C source via `pkgs.emscripten`. Source comes
|
|
||||||
from `pkgs.fluffychat-web.passthru.pubspecLock.dependencySources.native_imaging`
|
|
||||||
— already in the build closure of the flutter app, so no parallel
|
|
||||||
hash pin and version autosyncs with nixpkgs bumps. Build closure
|
|
||||||
is ~3.6 GiB (emscripten LLVM); runtime closure is just the two
|
|
||||||
output files. `dontConfigure = true` because cmake runs inside
|
|
||||||
`js/Makefile` via `emcmake cmake`, not at the package root. The
|
|
||||||
build script needs `HOME` + `EM_CACHE` writable for emscripten's
|
|
||||||
on-demand sysroot build (libc, libc++ → wasm).
|
|
||||||
- **`fluffychat-web-fixed`** is `pkgs.fluffychat-web` plus a
|
|
||||||
`postInstall` patch that (a) compiles `web/native_executor.dart`
|
|
||||||
via `dart compile js` (dart from the flutter341 closure, no
|
|
||||||
incremental cost) and (b) installs `fluffychat-web-imaging`'s
|
|
||||||
outputs into `$out`.
|
|
||||||
|
|
||||||
Two subtle details worth knowing before touching either derivation:
|
|
||||||
|
|
||||||
- **`make -C js`** instead of `cd js; make` — keeps the build-phase
|
|
||||||
pwd at the source root so `installPhase` doesn't have to know
|
|
||||||
about the cd. Robust against future reorders / `dontBuild`.
|
|
||||||
- **`web/native_executor.dart`** as a build-CWD-relative path,
|
|
||||||
*not* `$src/web/...` — `dart`'s `package_config.json` walk-up
|
|
||||||
needs to hit `buildFlutterApplication`'s pub-get output
|
|
||||||
(`.dart_tool/` in the build CWD). Walking up from a read-only
|
|
||||||
`$src/` store path finds no `.dart_tool/` and errors with
|
|
||||||
"Couldn't resolve the package 'matrix'."
|
|
||||||
|
|
||||||
Drop both derivations when nixpkgs's flutter builder grows worker
|
|
||||||
+ emcc support upstream.
|
|
||||||
|
|
||||||
Mount point is `chat.<swarm-domain>/` (the `gatewayHost` vhost);
|
|
||||||
upstream `--base-href "/"` is correct at sub-domain root, no override.
|
|
||||||
|
|
|
||||||
|
|
@ -18,6 +18,11 @@ behavior. The operator's own tools are in
|
||||||
- **[forge](forge.md)** — the `hive-forge` Forgejo CLI every agent has
|
- **[forge](forge.md)** — the `hive-forge` Forgejo CLI every agent has
|
||||||
for issues, PRs, and comments. Not an MCP tool — a binary agents
|
for issues, PRs, and comments. Not an MCP tool — a binary agents
|
||||||
shell out to instead of ad-hoc curl.
|
shell out to instead of ad-hoc curl.
|
||||||
|
- **[forge-notify](forge-notify.md)** — the daemon that wakes an agent on
|
||||||
|
forge activity: what it polls, how it formats and de-duplicates a
|
||||||
|
notification, and when it marks one read.
|
||||||
|
- **[github](github.md)** — how an agent's `gh` and `git push` authenticate
|
||||||
|
to GitHub, and its GitHub notification poller.
|
||||||
- **[forge-cli](forge-cli.md)** — the exhaustive, autogenerated
|
- **[forge-cli](forge-cli.md)** — the exhaustive, autogenerated
|
||||||
flag-by-flag reference for `hive-forge`, kept in lockstep with the
|
flag-by-flag reference for `hive-forge`, kept in lockstep with the
|
||||||
binary by CI the same way `hivectl-cli.md` is.
|
binary by CI the same way `hivectl-cli.md` is.
|
||||||
|
|
|
||||||
280
docs/4-implementation/agent-tools/forge-notify.md
Normal file
280
docs/4-implementation/agent-tools/forge-notify.md
Normal file
|
|
@ -0,0 +1,280 @@
|
||||||
|
# The forge notification poller
|
||||||
|
|
||||||
|
How the daemon that wakes an agent on forge activity works: what it polls, how it formats and de-duplicates a notification, and when it marks one read. The operator-side picture, with the forge's per-agent accounts, is in [the swarm forge](../../2-handbook/swarm/forge.md); the verbs an agent calls are in [`forge.md`](forge.md).
|
||||||
|
|
||||||
|
## Notification poller (`hive-forge-notify/src/notify.rs`)
|
||||||
|
|
||||||
|
Its own long-running per-agent daemon (`hive-forge-notify`, a sibling of
|
||||||
|
`hive-bash-daemon` / `hive-matrix-daemon`) — it used
|
||||||
|
to be a background task inside the `hive-agent` serve loop. Polls
|
||||||
|
`GET /api/v1/notifications?all=false` every 30 seconds (Forgejo's
|
||||||
|
unread-only filter), formats each notification as a broker
|
||||||
|
`Wake { from: "forge" }` message, and delivers it to the agent's own
|
||||||
|
inbox so claude's normal turn loop picks it up.
|
||||||
|
|
||||||
|
The daemon waits about ten minutes for the agent's forge token, then exits 0,
|
||||||
|
which its `Restart=on-failure` doesn't restart. A path unit
|
||||||
|
(`hive-forge-notify.path`) starts it again when the fetched token file or the
|
||||||
|
state-dir `forge-token` changes, so a token minted later still starts the
|
||||||
|
polling.
|
||||||
|
|
||||||
|
The crate builds a second, independent binary for a different host — see
|
||||||
|
[github.md](github.md#notifications).
|
||||||
|
|
||||||
|
The host-specific calls — list unread, mark read, resolve own login —
|
||||||
|
live behind `Source` in `hive-forge-notify/src/source.rs`;
|
||||||
|
classification, formatting, dedupe and todo delivery live in `notify.rs`,
|
||||||
|
common to both binaries. Todo
|
||||||
|
keys here are the bare thread ids, and must stay that way: renaming them
|
||||||
|
would orphan every in-flight forge todo on the first restart after a
|
||||||
|
deploy.
|
||||||
|
|
||||||
|
### Mark-read on delivery
|
||||||
|
|
||||||
|
On a **successful** broker delivery, `forge_notify` marks the thread
|
||||||
|
read on forge straight away (`PATCH /notifications/threads/{id}`). The
|
||||||
|
broker inbox is the durable work queue now — each delivered wake is a
|
||||||
|
sqlite row with its own ack lifecycle — so the forge unread flag no
|
||||||
|
longer needs to track whether the agent has _processed_ a
|
||||||
|
notification. Clearing it on delivery keeps forge's unread set **small
|
||||||
|
by construction**: at rest it holds only threads that failed to
|
||||||
|
deliver plus whatever arrived since the last 30s poll.
|
||||||
|
|
||||||
|
That size property is the whole point. A container rebuild starts the
|
||||||
|
poller with no memory of what it delivered, re-scans `?all=false`, and
|
||||||
|
finds nothing stale — the delivered threads are already read on forge.
|
||||||
|
Forge's own read-state is thus the durable, cross-rebuild record of
|
||||||
|
what `forge_notify` has delivered; there is **no persisted cursor**.
|
||||||
|
|
||||||
|
**Gotcha:** don't reintroduce a persisted dedup cursor here. A design
|
||||||
|
that leaves threads unread and tracks delivery via a separately persisted
|
||||||
|
cursor is fragile — losing that cursor across a rebuild re-delivers the
|
||||||
|
agent's entire still-unread backlog as a flood of fresh wakes. Forge's
|
||||||
|
own read-state is the only durable record this design needs.
|
||||||
|
|
||||||
|
**Read-before-comment guard doesn't block a fresh wake.** hive-forge's
|
||||||
|
read-before-comment guard (which keys off forge unread-state) doesn't
|
||||||
|
force the agent to view a thread before commenting on it: the broker
|
||||||
|
wake already carries the notification body, so _delivery is the read_.
|
||||||
|
An agent that wants the full thread still runs `hive-forge comments` /
|
||||||
|
`view`.
|
||||||
|
|
||||||
|
**In-process dedupe (small, ephemeral).** A single-process map (thread
|
||||||
|
id → last-delivered `updated_at`) guards the narrow window where a
|
||||||
|
mark-read call _transiently fails_ and the thread reappears unread in
|
||||||
|
the next poll before its `updated_at` bumps — so a flaky PATCH doesn't
|
||||||
|
re-fire the wake. it's **not persisted** and resets on restart (forge
|
||||||
|
read-state covers the durable case). Each poll prunes it to the ids in
|
||||||
|
the single `limit=UNREAD_FETCH_LIMIT` (50) fetch page, so it can never
|
||||||
|
exceed that many entries (a debug assertion pins the invariant; the
|
||||||
|
fetch limit and the bound are the same constant). `forge_notify` leaves
|
||||||
|
a failed _delivery_ unread and out of the map, so it resurfaces next tick.
|
||||||
|
|
||||||
|
`forge_notify` marks self-echo notifications (the agent's own writes, see below)
|
||||||
|
read directly without a delivery — same `mark_read` call, no wake.
|
||||||
|
|
||||||
|
### Activation gates (graceful no-ops)
|
||||||
|
|
||||||
|
The poller starts switched off and stays that way for any of:
|
||||||
|
|
||||||
|
- `HIVE_FORGE_URL` not set or not parseable as a URL (a hand-run
|
||||||
|
harness; hive-c0re sets it from `swarm.forge.domain` on every hive).
|
||||||
|
- no token in `$HIVE_FORGE_TOKEN_FILE` or `<state>/forge-token` (agent has no forge
|
||||||
|
account — pre-provisioning or destroy-without-purge race).
|
||||||
|
- Initial client construction fails (the typed `forgejo-api` client
|
||||||
|
for the API calls, or the plain reqwest client kept for the
|
||||||
|
best-effort enrichment fetches of server-provided subject URLs;
|
||||||
|
both unlikely; treated as fatal-to-the-task only).
|
||||||
|
|
||||||
|
Switched off = the spawned task returns immediately. All other failure
|
||||||
|
modes (HTTP errors, parse errors, mark-read failures) are
|
||||||
|
best-effort: logged at debug/warn and retried next tick.
|
||||||
|
|
||||||
|
### Self-notification filtering
|
||||||
|
|
||||||
|
Forgejo fires notifications for the agent's own actions (it opened a
|
||||||
|
PR, posted a comment, submitted a review). Surfacing those would
|
||||||
|
loop claude on its own writes. `forge_notify` drops the comment/review case
|
||||||
|
silently (mark-read without delivery):
|
||||||
|
|
||||||
|
- **Self-authored comments / reviews** — comment payload's
|
||||||
|
`user.login` matches `own_login`.
|
||||||
|
- **Self-authored creations** (an agent opening its own PR/issue) — the
|
||||||
|
already-fetched subject payload's poster `user.login` matches
|
||||||
|
`own_login`. `forge_notify` drops only _creations_ this way; a later state
|
||||||
|
change on the agent's own subject comes from someone else, so it still surfaces.
|
||||||
|
|
||||||
|
`forge_notify` fetches `own_login` at startup via `GET /api/v1/user`. On fetch
|
||||||
|
failure the filter degrades open (no filtering) rather than crashing
|
||||||
|
the task — a noisy inbox beats a silently stuck poller — but the fetch
|
||||||
|
is **re-attempted on each poll tick** until it succeeds, so a boot-time
|
||||||
|
failure (the forge not yet reachable) self-heals instead of leaving
|
||||||
|
self-echo filtering off for the whole process lifetime.
|
||||||
|
|
||||||
|
### Body excerpt + truncation + heading escape
|
||||||
|
|
||||||
|
The wake message embeds the comment / review body so the agent sees
|
||||||
|
actual content without a follow-up fetch — **not** a new issue/PR's own
|
||||||
|
description, which the wrapper table below deliberately omits. Three
|
||||||
|
pipeline steps in order:
|
||||||
|
|
||||||
|
1. **Truncate** to `BODY_TRUNCATE = 500` chars at a char-boundary;
|
||||||
|
appends `…` when cut. Truncation happens BEFORE escape so the
|
||||||
|
mention-overflow diff (next step) compares like-for-like against
|
||||||
|
the raw body.
|
||||||
|
2. **Mention overflow extraction** — when truncation actually
|
||||||
|
trimmed content, walk the full body line-by-line and surface any
|
||||||
|
`@username` lines that fell outside the embed window. Rendered as
|
||||||
|
a trailing `mentions (truncated from body):\n > <line>` block.
|
||||||
|
Mention detection requires the `@` to be at line start or
|
||||||
|
following a non-username byte, so email-style `foo@bar.com` does
|
||||||
|
NOT count.
|
||||||
|
3. **ATX heading escape** — for each line that's a strict
|
||||||
|
`CommonMark` ATX heading (1-6 leading `#`s followed by a space,
|
||||||
|
tab, or end-of-line), prepend `\` so the embedded body doesn't
|
||||||
|
blow into a top-level h1/h2 inside the wrapper message when the
|
||||||
|
dashboard renders it. Lines like `#tag`, `#123`, `#!/bin/bash` <!-- lint:allow: syntax examples, not real tags -->
|
||||||
|
**aren't** headings — no escape, no cosmetic noise. Indented
|
||||||
|
"headings" inside lists / nested quotes keep their leading
|
||||||
|
whitespace.
|
||||||
|
|
||||||
|
The strict ATX rule is deliberate: `\#tag` and `#tag` render
|
||||||
|
identically, so an over-eager escape just adds visual clutter
|
||||||
|
without changing behavior. Setext-style headings (`title\n====`)
|
||||||
|
aren't handled — rarer in practice, would need multi-line
|
||||||
|
lookahead.
|
||||||
|
|
||||||
|
### Wrapper format
|
||||||
|
|
||||||
|
Five shapes, distinguished by the notification's classification:
|
||||||
|
|
||||||
|
| Trigger | Wrapper |
|
||||||
|
| ----------------------------------- | ------------------------------------------------------------------------------ |
|
||||||
|
| Comment on issue / PR | `[comment on PR #N owner/repo] title\nurl: ...\n\nauthor: body\nassignee: ...` |
|
||||||
|
| Review submission | `[PR approved #N owner/repo] title\nurl: ...\n\nauthor: body\nassignee: ...` |
|
||||||
|
| New issue / PR | `[new PR #N owner/repo] title\nurl: ...\nassignee: ...` |
|
||||||
|
| Later activity (open, not creation) | `[activity on PR #N owner/repo] title\nurl: ...\nassignee: ...` |
|
||||||
|
| State change | `[PR merged #N owner/repo] title\nurl: ...\nassignee: ...` |
|
||||||
|
|
||||||
|
The issue/PR's own description is deliberately never embedded here (only a
|
||||||
|
comment/review body is — that's the actual new content a notification
|
||||||
|
exists to surface): repeating a subject's own description on every "new"
|
||||||
|
or "activity on" wake for it's noise the agent already has via the URL,
|
||||||
|
not context (mara, triage: "just the issue title should be enough").
|
||||||
|
|
||||||
|
Review labels come from the Forgejo `state` field: `APPROVED` →
|
||||||
|
`approved`, `REQUEST_CHANGES` → `changes requested`, `COMMENT` →
|
||||||
|
`review comment`. `forge_notify` drops `PENDING` (review saved but not
|
||||||
|
submitted yet — no peer-visible event). Unknown states fall back to
|
||||||
|
the generic comment wrapper.
|
||||||
|
|
||||||
|
A review submitted with **no body** renders `reviewed by: <author>` in
|
||||||
|
place of the `<author>: <body>` line — deliberately worded to not collide
|
||||||
|
with the meta-suffix `reviewer:` line (requested reviewers, below).
|
||||||
|
|
||||||
|
### Inline review comments
|
||||||
|
|
||||||
|
A review's line comments belong to neither the review body nor the
|
||||||
|
`latest_comment_url`. A review with only inline comments has no such url
|
||||||
|
and would otherwise land as a bare `activity on PR`. Every PR wake therefore
|
||||||
|
fetches the PR's reviews (`/pulls/N/reviews`, up to four pages of 50) and,
|
||||||
|
for the reviews new to the agent, appends an
|
||||||
|
`inline review comments (read them with `hive-forge pr reviews N`):` block
|
||||||
|
saying where they sit: one `author: path:line, path, ...` line per review
|
||||||
|
author (bare `path` when Forgejo dropped the position because the line left
|
||||||
|
the diff; control characters in a path become spaces). The comment text stays
|
||||||
|
out of the wake, and the agent reads the full review with the CLI.
|
||||||
|
|
||||||
|
A review counts as new when it landed strictly after the thread's previously
|
||||||
|
delivered `updated_at`. Without a baseline every review counts, and so does
|
||||||
|
one with no submit time, so nothing disappears. It doesn't count when the
|
||||||
|
agent wrote it, when it's a `PENDING` draft, or when `comments_count` is 0.
|
||||||
|
The notifier fetches the newest 5 such reviews and lists at most 10 locations
|
||||||
|
(`(+N more)` for the rest). Best effort: a failed fetch yields less text and
|
||||||
|
never drops the wake. Cost: one reviews fetch per PR wake, plus one per new
|
||||||
|
commented review.
|
||||||
|
|
||||||
|
### Merge/close vs a later comment
|
||||||
|
|
||||||
|
A notification carrying a `latest_comment_url` normally takes the comment
|
||||||
|
path. But a merged/closed subject **keeps** its `latest_comment_url` set,
|
||||||
|
so a just-merged PR that had any prior discussion would route to the
|
||||||
|
comment path and render `[comment on PR]` (with a stale pre-merge comment
|
||||||
|
body) instead of `[PR merged]` — the agent never learns its PR merged.
|
||||||
|
When the notification IS the merge/close transition — its
|
||||||
|
event time (`updated_at`) is within `NEW_ITEM_TOLERANCE_SECS` of the
|
||||||
|
subject's `closed_at` (set for both `merged` and `closed`) — the
|
||||||
|
state-change path wins even with a comment url present
|
||||||
|
(`state_change_is_current`). A genuine **later** comment on an
|
||||||
|
already-closed subject bumps `updated_at` well past `closed_at`, so it
|
||||||
|
stays on the comment path and keeps its comment body. Missing/unparseable
|
||||||
|
timestamps default to the state-change path, so a merge is never silently
|
||||||
|
hidden behind a stale comment.
|
||||||
|
|
||||||
|
#### Merge racing a comment
|
||||||
|
|
||||||
|
The one gap the timestamp cut leaves: a genuine comment posted **within
|
||||||
|
`NEW_ITEM_TOLERANCE_SECS` of the merge** bumps `updated_at` close enough
|
||||||
|
to `closed_at` that `state_change_is_current` returns `true` — so it takes
|
||||||
|
the state-change path, dropping its body. Best of both worlds:
|
||||||
|
on the merge/close path `forge_notify` fetches the `latest_comment_url` comment and, when
|
||||||
|
its `created_at` is strictly **after** the subject's `closed_at`
|
||||||
|
(`comment_is_after_close`) — in other words, it raced the merge rather than being the
|
||||||
|
pre-merge last comment the subject keeps — append it as a
|
||||||
|
`comment by <author>: <excerpt>` block before the meta suffix
|
||||||
|
(`fresh_post_close_comment_tail`). The wake carries **both** `[PR merged]`
|
||||||
|
and the racing comment. `forge_notify` leaves the kept pre-merge comment (created before
|
||||||
|
`closed_at`) off, drops a self-authored racing comment (don't
|
||||||
|
echo the agent's own write), and appends nothing for a missing/unparseable `created_at`/
|
||||||
|
`closed_at` (conservative — only surface a comment it can
|
||||||
|
positively place after the close). Cost: one extra comment fetch on
|
||||||
|
merge/close notifications, acceptable given how rare they are.
|
||||||
|
|
||||||
|
### "new" vs "activity on"
|
||||||
|
|
||||||
|
A review submitted with **no body** carries no `latest_comment_url`,
|
||||||
|
so it misses the comment path and lands on the state-change path with
|
||||||
|
`state == "open"` — exactly like a freshly opened PR. Labeling that
|
||||||
|
`new PR` is misleading: agents dismiss it as a duplicate of the
|
||||||
|
original open notification and miss the review. The `open`
|
||||||
|
state only earns the `new <kind>` label when the notification's event
|
||||||
|
time (`updated_at`) is within `NEW_ITEM_TOLERANCE_SECS` (120s) of the
|
||||||
|
subject's `created_at`. `forge_notify` labels anything later `activity on <kind>`
|
||||||
|
— neutral and non-misleading, since it can't cheaply say _what_ the
|
||||||
|
activity was without an extra reviews fetch. Missing/unparseable
|
||||||
|
timestamps default to `new` (preserve prior behavior rather than mask a
|
||||||
|
genuine new item). A small dependency-free
|
||||||
|
RFC 3339 helper (`parse_rfc3339`) parses timestamps.
|
||||||
|
|
||||||
|
`forge_notify` extracts the number from `subject.html_url`'s last path segment
|
||||||
|
(strips `#anchor` first), and the repo slug from `repository.full_name`.
|
||||||
|
Both degrade gracefully when absent (number → blank, repo → blank)
|
||||||
|
so unexpected Forgejo shapes don't crash the formatter.
|
||||||
|
|
||||||
|
### Meta suffix
|
||||||
|
|
||||||
|
Every wrapper ends with one or more of:
|
||||||
|
|
||||||
|
- `assignee: <list>` — always present; `unassigned` when empty so
|
||||||
|
the line shape is stable.
|
||||||
|
- `reviewer: <list>` — PR notifications only, present only when
|
||||||
|
`requested_reviewers` is non-empty.
|
||||||
|
|
||||||
|
### Review-request override
|
||||||
|
|
||||||
|
For new PRs, the kind label flips to `[review requested #N
|
||||||
|
owner/repo]` when `own_login` appears in `requested_reviewers`,
|
||||||
|
regardless of the Forgejo `reason` field. Forgejo doesn't reliably
|
||||||
|
set `reason == "review_requested"` (often null instead), so the
|
||||||
|
fallback checks the subject payload directly. `is_new` gates detection
|
||||||
|
so the label only fires once on PR creation, not on every
|
||||||
|
subsequent comment.
|
||||||
|
|
||||||
|
### Subscription management
|
||||||
|
|
||||||
|
The poller does **not** autounsubscribe from repo watches — it
|
||||||
|
delivers every unread notification it's handed. A hive-forge CLI
|
||||||
|
subscription verb bounds the firehose explicitly (dropping broad repo
|
||||||
|
watches an agent doesn't need) — the poller never guesses which
|
||||||
|
watches to drop. See the `subscription` verb in
|
||||||
|
[`docs/4-implementation/agent-tools/forge.md`](forge.md).
|
||||||
86
docs/4-implementation/agent-tools/github.md
Normal file
86
docs/4-implementation/agent-tools/github.md
Normal file
|
|
@ -0,0 +1,86 @@
|
||||||
|
# GitHub accounts: how the agent authenticates
|
||||||
|
|
||||||
|
How the agent's `gh` and `git push` authenticate, and how the GitHub notification poller works. What an operator does to give an agent an account is in [GitHub accounts](../../2-handbook/external-accounts/github.md).
|
||||||
|
|
||||||
|
## How the agent uses it
|
||||||
|
|
||||||
|
When enabled, the container gets:
|
||||||
|
|
||||||
|
- **A `gh` wrapper** on `PATH` (shadowing the raw `gh`) that exports
|
||||||
|
`GH_TOKEN` from the token file at invocation, then execs real `gh`. So
|
||||||
|
`gh pr create`, `gh api …`, etc. just work — `gh` derives the identity
|
||||||
|
from the token.
|
||||||
|
- **A git credential helper** (`git-credential-hive-github`), wired via a
|
||||||
|
host-scoped `/etc/gitconfig` entry for `https://github.com`, so
|
||||||
|
`git push https://github.com/<owner>/<repo>` authenticates as
|
||||||
|
`x-access-token` + the PAT (GitHub ignores the username for PAT auth).
|
||||||
|
Host-scoped, so it never touches the forge (`localhost:3000`) or any
|
||||||
|
other remote.
|
||||||
|
|
||||||
|
Both scripts read the token from `<state>/github-token` **at invocation
|
||||||
|
time**, so a PAT written (or rotated) mid-session takes effect immediately
|
||||||
|
— no container rebuild or restart. Until the file exists, `gh` / `git push`
|
||||||
|
fail unauthenticated. The build bakes the token path into the scripts
|
||||||
|
(rather than reading it from an env var), because claude's Bash tool runs in a
|
||||||
|
minimal environment that wouldn't carry one.
|
||||||
|
|
||||||
|
## Notifications
|
||||||
|
|
||||||
|
`hive-github-notify` polls github.com for the agent, turning each unread
|
||||||
|
notification thread into a todo. it's a **separate binary and a
|
||||||
|
separate systemd unit** from the internal forge's poller
|
||||||
|
(`hive-forge-notify`, see [the forge notification poller](forge-notify.md)),
|
||||||
|
installed by `nix/agent-modules/github.nix` under
|
||||||
|
`services.hyperhive.agent.github.enable`. Both binaries ship from the one
|
||||||
|
`hive-forge-notify` derivation, so the unit is a second `ExecStart`
|
||||||
|
path, not a new package.
|
||||||
|
|
||||||
|
Two units rather than one daemon with two loops, because it puts the
|
||||||
|
decision in nix: a hive built without this module has **no github poller
|
||||||
|
in its closure at all**, which is what makes GitHub access separable
|
||||||
|
rather than merely switched off. it's also why this isn't a cargo
|
||||||
|
feature — a feature would unify across the workspace and cost every
|
||||||
|
crate its build cache.
|
||||||
|
|
||||||
|
At runtime the poller needs the PAT above. No PAT, no polling: the unit
|
||||||
|
logs why and exits 0, which is why it's `Restart = on-failure` and
|
||||||
|
never `always` — a clean exit on a PAT-less agent must not become a
|
||||||
|
restart loop. The poller waits about ten minutes for the PAT to appear
|
||||||
|
before it gives up. A path unit (`hive-github-notify.path`) starts the poller
|
||||||
|
again when `<state>/github-token` changes, so a PAT that lands later still
|
||||||
|
enables polling. Once polling, it re-reads the file on every
|
||||||
|
tick, so it picks up a rotated PAT on the next tick without a restart.
|
||||||
|
|
||||||
|
Forgejo models its notifications API on GitHub's, so one tolerant
|
||||||
|
parse serves both: `id`, `repository.full_name`,
|
||||||
|
`subject {title,url,latest_comment_url}` and `updated_at` line up field
|
||||||
|
for field. Lenient deserializers absorb the two real differences —
|
||||||
|
GitHub sends the thread id as a _string_ where Forgejo
|
||||||
|
sends a number, and says `PullRequest` where Forgejo says `Pull`.
|
||||||
|
`hive-github-notify` prefixes todo keys `gh:<id>` so a github thread id can't collide with a
|
||||||
|
forge one.
|
||||||
|
|
||||||
|
Two host differences worth knowing before touching this code:
|
||||||
|
|
||||||
|
- **Auth scheme, not just value.** Forgejo takes
|
||||||
|
`Authorization: token <t>`; GitHub wants `Bearer <t>` plus `Accept:
|
||||||
|
application/vnd.github+json`, `X-GitHub-Api-Version` and a
|
||||||
|
`User-Agent`. Sending Forgejo's form to GitHub doesn't error — it
|
||||||
|
authenticates as _nobody_ and silently drops to the unauthenticated
|
||||||
|
rate limit. The cheap way to tell the two apart is the rate-limit
|
||||||
|
header: `x-ratelimit-remaining` near 5000 is an authenticated user,
|
||||||
|
near 60 is anonymous.
|
||||||
|
- **GitHub sets the cadence.** It returns `X-Poll-Interval` (60s in
|
||||||
|
practice, slower than the poller's own tick) and rate-limits callers who
|
||||||
|
ignore it, so the loop re-arms to the server's interval whenever that's
|
||||||
|
_slower_ than the poller's. A hint faster than the poller's own tick
|
||||||
|
isn't a reason to poll harder.
|
||||||
|
|
||||||
|
⚠️ **This needs the `notifications` scope on the PAT.** A token minted
|
||||||
|
for `gh` + `git push` typically carries `repo` only, which is enough to
|
||||||
|
push and open PRs but **not** to read the notification stream (nor to
|
||||||
|
mark a thread read, which is the same scope). A PAT without it doesn't
|
||||||
|
break anything: the poller logs the refusal and stays quiet, and the
|
||||||
|
agent never gets GitHub wakes. If an agent's GitHub
|
||||||
|
notifications never arrive, check the token's scopes first — the
|
||||||
|
symptom is silence, not an error.
|
||||||
95
docs/4-implementation/networking/matrix-homeserver.md
Normal file
95
docs/4-implementation/networking/matrix-homeserver.md
Normal file
|
|
@ -0,0 +1,95 @@
|
||||||
|
# The matrix homeserver: implementation notes
|
||||||
|
|
||||||
|
What the homeserver's NixOS module does that an operator never sees: the hive-side code it replaced, the evaluation assertions it makes, and the fluffychat-web build fixes. The operator's view of the homeserver is in [the swarm matrix page](../../2-handbook/swarm/matrix.md).
|
||||||
|
|
||||||
|
## Retired hive-side matrix code
|
||||||
|
|
||||||
|
Earlier versions of hive-c0re created a private "hive" Matrix Space and a
|
||||||
|
`hive-chat` room, invited every agent to both, kept the hive's sender token
|
||||||
|
on disk and offered `hivectl matrix invite`. Matrix is swarm-level, so
|
||||||
|
hive-c0re dropped all of it and no longer talks to the homeserver. The Space, the
|
||||||
|
room and the files `/var/lib/hyperhive/matrix/{access-token,space-room-id,chat-room-id}`
|
||||||
|
are harmless leftovers, so delete them by hand if you like.
|
||||||
|
|
||||||
|
## Assertion rationale
|
||||||
|
|
||||||
|
`config.assertions` in this module fail eval early rather than ship
|
||||||
|
surprising behaviour:
|
||||||
|
|
||||||
|
- **`services.hyperhive.swarm.matrix.gatewayHost != ""`** — same footgun as `forge.domain`:
|
||||||
|
empty string renders `.<hive>`-shaped garbage in both nginx
|
||||||
|
`server_name` (treated as wildcard catch-all, surprising) and
|
||||||
|
`/etc/hosts` (malformed entry). `null` is the right opt-out shape;
|
||||||
|
a config assertion rejects empty string explicitly.
|
||||||
|
SSO is unconditional, so the three below are requirements of running a
|
||||||
|
homeserver at all rather than of a setting:
|
||||||
|
|
||||||
|
- **Keep `deploy.matrix.sso.clientSecretFile` set** — it defaults to
|
||||||
|
`/run/hive-matrix-oidc/<client id>.secret` wherever the homeserver runs,
|
||||||
|
and `swarm-bao-matrix-oidc` writes the secret there from the swarm secret
|
||||||
|
store under matrix-ctl's leaf. The path is on the host's tmpfs, so every
|
||||||
|
reboot fetches the secret again and it never reaches persistent disk. After
|
||||||
|
a reboot the homeserver doesn't start until the store answers; the
|
||||||
|
store-retry timer then starts it with no manual step. A path
|
||||||
|
outside `/run/hive-matrix-oidc/`, or a null, fails at eval, not at boot:
|
||||||
|
tuwunel reads its identity providers from the config file, so a half-configured
|
||||||
|
one can stop the homeserver from starting outright rather than merely
|
||||||
|
hiding a login button.
|
||||||
|
- **Set `swarm.authelia.url`** — without a provider URL there
|
||||||
|
is nothing to discover against.
|
||||||
|
- **Set `swarm.matrix.gatewayHost != null`** — the SSO callback URL is
|
||||||
|
format-locked to `<homeserver>/_matrix/client/unstable/login/sso/callback/<client_id>`,
|
||||||
|
and the identity provider needs a public name to redirect the
|
||||||
|
browser to.
|
||||||
|
|
||||||
|
`server_name`'s own bogus-value guard lives in `hive-network.nix`
|
||||||
|
(`services.hyperhive.domain != null`), not here — see
|
||||||
|
[`docs/4-implementation/networking/network.md`](network.md).
|
||||||
|
|
||||||
|
## fluffychat-web build fixes
|
||||||
|
|
||||||
|
`pkgs.fluffychat-web` ships from `flutter341.buildFlutterApplication`,
|
||||||
|
which has two upstream gaps for fluffychat's web target:
|
||||||
|
|
||||||
|
- The dart web-worker entry point (`web/native_executor.dart`) isn't
|
||||||
|
compiled — `buildFlutterApplication` only runs `flutter build web`
|
||||||
|
on the main entry.
|
||||||
|
- `native_imaging`'s C source isn't built — emscripten isn't a
|
||||||
|
flutter-builder native build input.
|
||||||
|
|
||||||
|
Both fixed in `nix/host-modules/hive-matrix.nix` via two derivations:
|
||||||
|
|
||||||
|
- **`fluffychat-web-imaging`** builds `Imaging.{js,wasm}` from the
|
||||||
|
`native_imaging` C source via `pkgs.emscripten`. Source comes
|
||||||
|
from `pkgs.fluffychat-web.passthru.pubspecLock.dependencySources.native_imaging`
|
||||||
|
— already in the build closure of the flutter app, so no parallel
|
||||||
|
hash pin and version autosyncs with nixpkgs bumps. Build closure
|
||||||
|
is ~3.6 GiB (emscripten LLVM); runtime closure is just the two
|
||||||
|
output files. `dontConfigure = true` because cmake runs inside
|
||||||
|
`js/Makefile` via `emcmake cmake`, not at the package root. The
|
||||||
|
build script needs `HOME` + `EM_CACHE` writable for emscripten's
|
||||||
|
on-demand sysroot build (libc, libc++ → wasm).
|
||||||
|
- **`fluffychat-web-fixed`** is `pkgs.fluffychat-web` plus a
|
||||||
|
`postInstall` patch that (a) compiles `web/native_executor.dart`
|
||||||
|
via `dart compile js` (dart from the flutter341 closure, no
|
||||||
|
incremental cost) and (b) installs `fluffychat-web-imaging`'s
|
||||||
|
outputs into `$out`.
|
||||||
|
|
||||||
|
Two subtle details worth knowing before touching either derivation:
|
||||||
|
|
||||||
|
- **`make -C js`** instead of `cd js; make` — keeps the build-phase
|
||||||
|
pwd at the source root so `installPhase` doesn't have to know
|
||||||
|
about the cd. Robust against future reorders / `dontBuild`.
|
||||||
|
- **`web/native_executor.dart`** as a build-CWD-relative path,
|
||||||
|
_not_ `$src/web/...` — `dart`'s `package_config.json` walk-up
|
||||||
|
needs to hit `buildFlutterApplication`'s pub-get output
|
||||||
|
(`.dart_tool/` in the build CWD). Walking up from a read-only
|
||||||
|
`$src/` store path finds no `.dart_tool/` and errors with
|
||||||
|
"Couldn't resolve the package 'matrix'."
|
||||||
|
|
||||||
|
Drop both derivations when nixpkgs's flutter builder grows worker
|
||||||
|
|
||||||
|
- emcc support upstream.
|
||||||
|
|
||||||
|
Mount point is `chat.<swarm-domain>/` (the `gatewayHost` vhost);
|
||||||
|
upstream `--base-href "/"` is correct at sub-domain root, no override.
|
||||||
|
|
@ -99,9 +99,9 @@ Accounts outside the swarm that an agent can hold. Start at
|
||||||
[`2-handbook/external-accounts/`](2-handbook/external-accounts/README.md).
|
[`2-handbook/external-accounts/`](2-handbook/external-accounts/README.md).
|
||||||
|
|
||||||
- **How to give an agent a GitHub account (`gh` + `git push`)? how's
|
- **How to give an agent a GitHub account (`gh` + `git push`)? how's
|
||||||
the PAT injected?** → [`2-handbook/external-accounts/github.md`](2-handbook/external-accounts/github.md)
|
the PAT injected?** → [`2-handbook/external-accounts/github.md`](2-handbook/external-accounts/github.md);
|
||||||
(operator content up top; the `gh`/git-push + notification-poller
|
the `gh`/git-push and notification-poller mechanics are in
|
||||||
mechanics are in a collapsed "Implementation" section at the bottom).
|
[`4-implementation/agent-tools/github.md`](4-implementation/agent-tools/github.md).
|
||||||
- **How to give an agent GitLab accounts (`glab` + `git push`, more than one
|
- **How to give an agent GitLab accounts (`glab` + `git push`, more than one
|
||||||
instance)?** → [`2-handbook/external-accounts/gitlab.md`](2-handbook/external-accounts/gitlab.md).
|
instance)?** → [`2-handbook/external-accounts/gitlab.md`](2-handbook/external-accounts/gitlab.md).
|
||||||
|
|
||||||
|
|
@ -119,6 +119,11 @@ Nothing of it sits in this tree.
|
||||||
- **What can an agent do inside its container? bash, subagent, forge, matrix, and the
|
- **What can an agent do inside its container? bash, subagent, forge, matrix, and the
|
||||||
`hive-forge`, `swarm-logs` and `swarm-metrics` CLI references?** →
|
`hive-forge`, `swarm-logs` and `swarm-metrics` CLI references?** →
|
||||||
[`4-implementation/agent-tools/`](4-implementation/agent-tools/README.md).
|
[`4-implementation/agent-tools/`](4-implementation/agent-tools/README.md).
|
||||||
|
- **How does the forge notification poller wake an agent, and how does it
|
||||||
|
format and de-duplicate what it delivers?** →
|
||||||
|
[`4-implementation/agent-tools/forge-notify.md`](4-implementation/agent-tools/forge-notify.md).
|
||||||
|
- **How do an agent's `gh` and `git push` authenticate to GitHub?** →
|
||||||
|
[`4-implementation/agent-tools/github.md`](4-implementation/agent-tools/github.md).
|
||||||
|
|
||||||
### Turn loop, scheduler, CI, observability
|
### Turn loop, scheduler, CI, observability
|
||||||
|
|
||||||
|
|
@ -141,6 +146,9 @@ Nothing of it sits in this tree.
|
||||||
discovery work?** → [`4-implementation/networking/gateway.md`](4-implementation/networking/gateway.md).
|
discovery work?** → [`4-implementation/networking/gateway.md`](4-implementation/networking/gateway.md).
|
||||||
- **How does DNS resolution work in agent containers? What's the
|
- **How does DNS resolution work in agent containers? What's the
|
||||||
bridge network for?** → [`4-implementation/networking/network.md`](4-implementation/networking/network.md).
|
bridge network for?** → [`4-implementation/networking/network.md`](4-implementation/networking/network.md).
|
||||||
|
- **What does the matrix homeserver's module assert at evaluation, how
|
||||||
|
fluffychat-web gets built, and what replaced the hive-side matrix code?** →
|
||||||
|
[`4-implementation/networking/matrix-homeserver.md`](4-implementation/networking/matrix-homeserver.md).
|
||||||
- **Where do agent snapshots go? How does the swarm's `btrfs receive`
|
- **Where do agent snapshots go? How does the swarm's `btrfs receive`
|
||||||
endpoint authenticate a pushing hive?** →
|
endpoint authenticate a pushing hive?** →
|
||||||
[`4-implementation/networking/snapshot-store.md`](4-implementation/networking/snapshot-store.md).
|
[`4-implementation/networking/snapshot-store.md`](4-implementation/networking/snapshot-store.md).
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue