mara/hyperhive
mara/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

docs: sort the top-level directories into five sections by what the reader wants

Browse source 
Every docs directory moves under 1-getting-started, 2-handbook,
4-implementation or 5-project; the options reference (section 3) comes from
the website and has no directory here. trust-boundary stays whole under the
handbook for now. web-ui, swarm and tools split between handbook and
implementation, and the command-line references move to
2-handbook/reference. Links, path references in code and nix, and the docs
index follow the moves; no page content changes.
This commit is contained in:
iris 2026-11-06 18:30:12 +01:00 • committed by mara
commit 6739bb903d
5 changed files with 736 additions and 701 deletions
Download patch file Download diff file Expand all files Collapse all files

395
docs/2-handbook/integrations/forge.md Normal file
View file

@ -0,0 +1,395 @@
# Swarm forge (Forgejo)
Private Forgejo instance running in a nixos-container (`hive-forge`, the
container and CLI name), used as the
swarm's persistent code-collaboration surface (issues, PRs, reviews,
attachments). Configured via `services.hyperhive.swarm.forge.*` (what the
forge IS to every hive — names, ports, URLs) plus
`services.hyperhive.deploy.forgejo.*` (what the host running it decides —
which build it runs, how it's served, what it mirrors, where its
host-local secrets sit).
Container
shape, ROOT_URL / sub-domain routing, and operator-vs-in-cluster URL
handling live in [`docs/4-implementation/networking/gateway.md`](../../4-implementation/networking/gateway.md); this file owns the
per-agent integration story and the notification pump that wakes
each agent on relevant activity.
## Token scopes
Two scope sets:
**`AGENT_TOKEN_SCOPES`** (swarm-controller's `forge::agent_token`, pinned
by a test). swarm-controller mints every agent token with it:
| Scope | Why |
| -------------------- | ------------------------------------------------------------------------------------------------------- |
| `write:repository` | Create, clone, push, delete repos; merge PRs. |
| `write:issue` | Open / comment / review issues **and** pull requests (Forgejo namespaces PR conversation under issues). |
| `write:user` | Edit own profile, create repos under own user. |
| `write:organization` | Create + manage orgs (lets agents share a forge namespace). |
| `read:user` | Token-owner endpoint used for self-identification at harness startup. |
| `write:misc` | Hooks, attachments, the rest of the long tail. |
| `read:notification` | Poll `GET /notifications` for unread events. |
| `write:notification` | Mark notifications read via `PATCH /notifications/threads/{id}`. |
**`HIVE_TOKEN_SCOPES`** (swarm-controller's `forge::hive_token`, pinned by a
test): `read:repository` alone, for each hive's read-only account (next
section).
---
## How a hive reads the forge
Each hive in swarm-controller's hive directory has its own forge account,
`hive-<hive>`. swarm-controller creates it, makes it a member of the `hives`
team in `agent-configs`, and mints one token named `swarm-hive` into the
swarm secret store at `swarm/hives/<hive>/forge/token`. The team has read
permission on `repo.code` and `repo.pulls` of every repo in the org,
including repos created later. A pass at start and every five minutes
creates what's missing and re-mints a token that's missing or no longer
matches the forge. A pass whose store or forge reads fail queues nothing, so
an outage never rotates every hive's token.
swarm-controller refuses to create an agent whose name starts with `hive-`:
these accounts own that prefix. An agent that already had such a
name keeps it, so the accounts are also told apart by email. swarm-controller
creates the hive's account with the email `hive-<hive>@hives.hyperhive.local`
and refuses an account by that name with any other email: it never adds an
agent's account to the team or mints a token for it. An agent's own forge
account setup refuses the hive's account the same way.
hive-c0re reads the token with `forge::reader_token()`, under the hive's own
store certificate, every time it reads the forge. A hive whose store holds
none reads nothing. These reads go through it, and none of them needs a local
forge container:
- cloning and pulling `/knowledge`
The hive builds nothing, so no nix process reads the forge as the hive's
account. hive-c0re hands the token to each `git` command it runs
(`http.extraHeader`) and keeps no copy on disk. A hive whose store holds no token
reads nothing private from the forge. This also holds on a hive with no forge
container.
Roll a change to any of this out store host first (its granter writes the
controller's grant on `swarm/hives/+/forge/token`), then the controller host,
then the hives. A hive that starts before its token exists reads nothing, and
builds nothing, until the controller's next pass.
---
## Per-agent forge accounts
Each agent gets its own Forgejo user and access token. swarm-controller
creates the user when it creates the agent, and mints one token named
`swarm-agent` with the admin API into the swarm secret store at
`swarm/agents/<agent>/forge-token`. A pass at start and every five minutes
covers every agent with a store identity: it creates a missing forge user and
re-mints any token that's missing or no longer matches the forge;
a rotation deletes the old `swarm-agent` token first, so each agent holds at
most one. The agent fetches the token under its own store certificate into
`/run/hive-agent-forge-token/token` (`nix/agent-modules/forge-token.nix`),
and `hive-forge`, the git credential helper, the `forge_notify` poller and
the avatar sync read it from there, falling back to `<state>/forge-token`,
the file hive-c0re wrote before. hive-c0re no longer creates agent users or
mints agent tokens. swarm-controller deletes the `hyperhive-<unix-seconds>`
tokens it left behind, for each agent whose current `swarm-agent` token is
live; tokens of other names are never touched.
Two things live in the `agent-configs` Forgejo organization:
- A config repo per agent (`agent-configs/<name>`). The
agent is a **write collaborator on its own** repo — it can push
config-change branches and open config PRs — but `main` is
branch-protected by swarm-controller: the merge and approval
allowlists are the `operators` team, and the agent can neither push
`main` directly nor self-merge. An operator's merge in the Forgejo UI
deploys the merged commit (see
[config-changes.md § Config changes](../agent-lifecycle/config-changes.md#config-changes)).
hive-c0re pushes nothing to this org and reads nothing from it.
Repos stay private, so an agent can't read another
agent's config. The agent's **persistent meta flake input** is this repo's `main` head,
pinned by sha in `internal/meta` (`agent-<n>.url =
git+<forge>/agent-configs/<n>.git?rev=<sha>`; see
[config-changes.md § Meta flake](../agent-lifecycle/config-changes.md#meta-flake)).
The `hive-forge` CLI (separate workspace crate, see
[`README.md`](../../README.md) file map) wraps the Forgejo REST API
with the per-agent token; agents call it for issue / PR / comment
ops as if it were a peer. All REST calls across the workspace
(`hive-forge` verbs, hive-c0re provisioning, this poller) go through
the typed `forgejo-api` crate; only non-`/api/v1` web-router routes
(attachment / artifact downloads, log streaming) and the poller's
enrichment fetches of server-provided subject URLs stay on raw
reqwest.
## Notification poller (`hive-forge-notify/src/notify.rs`)
Its own long-running per-agent daemon (`hive-forge-notify`, a sibling of
`hive-bash-daemon` / `hive-matrix-daemon`) — it used
to be a background task inside the `hive-agent` serve loop. Polls
`GET /api/v1/notifications?all=false` every 30 seconds (Forgejo's
unread-only filter), formats each notification as a broker
`Wake { from: "forge" }` message, and delivers it to the agent's own
inbox so claude's normal turn loop picks it up.
The crate builds a second, independent binary for a different host — see
[github.md](github.md#notifications).
The host-specific calls — list unread, mark read, resolve own login —
live behind `Source` in `hive-forge-notify/src/source.rs`;
classification, formatting, dedupe and todo delivery live in `notify.rs`,
common to both binaries. Todo
keys here are the bare thread ids, and must stay that way: renaming them
would orphan every in-flight forge todo on the first restart after a
deploy.
### Mark-read on delivery
On a **successful** broker delivery, `forge_notify` marks the thread
read on forge straight away (`PATCH /notifications/threads/{id}`). The
broker inbox is the durable work queue now — each delivered wake is a
sqlite row with its own ack lifecycle — so the forge unread flag no
longer needs to track whether the agent has _processed_ a
notification. Clearing it on delivery keeps forge's unread set **small
by construction**: at rest it holds only threads that failed to
deliver plus whatever arrived since the last 30s poll.
That size property is the whole point. A container rebuild starts the
poller with no memory of what it delivered, re-scans `?all=false`, and
finds nothing stale — the delivered threads are already read on forge.
Forge's own read-state is thus the durable, cross-rebuild record of
what `forge_notify` has delivered; there is **no persisted cursor**.
**Gotcha:** don't reintroduce a persisted dedup cursor here. A design
that leaves threads unread and tracks delivery via a separately persisted
cursor is fragile — losing that cursor across a rebuild re-delivers the
agent's entire still-unread backlog as a flood of fresh wakes. Forge's
own read-state is the only durable record this design needs.
**Read-before-comment guard doesn't block a fresh wake.** hive-forge's
read-before-comment guard (which keys off forge unread-state) doesn't
force the agent to view a thread before commenting on it: the broker
wake already carries the notification body, so _delivery is the read_.
An agent that wants the full thread still runs `hive-forge comments` /
`view`.
**In-process dedupe (small, ephemeral).** A single-process map (thread
id → last-delivered `updated_at`) guards the narrow window where a
mark-read call _transiently fails_ and the thread reappears unread in
the next poll before its `updated_at` bumps — so a flaky PATCH doesn't
re-fire the wake. it's **not persisted** and resets on restart (forge
read-state covers the durable case). Each poll prunes it to the ids in
the single `limit=UNREAD_FETCH_LIMIT` (50) fetch page, so it can never
exceed that many entries (a debug assertion pins the invariant; the
fetch limit and the bound are the same constant). `forge_notify` leaves
a failed _delivery_ unread and out of the map, so it resurfaces next tick.
`forge_notify` marks self-echo notifications (the agent's own writes, see below)
read directly without a delivery — same `mark_read` call, no wake.
### Activation gates (graceful no-ops)
The poller starts switched off and stays that way for any of:
- `HIVE_FORGE_URL` not set or not parseable as a URL (a hand-run
harness; hive-c0re sets it from `swarm.forge.domain` on every hive).
- no token in `$HIVE_FORGE_TOKEN_FILE` or `<state>/forge-token` (agent has no forge
account — pre-provisioning or destroy-without-purge race).
- Initial client construction fails (the typed `forgejo-api` client
for the API calls, or the plain reqwest client kept for the
best-effort enrichment fetches of server-provided subject URLs;
both unlikely; treated as fatal-to-the-task only).
Switched off = the spawned task returns immediately. All other failure
modes (HTTP errors, parse errors, mark-read failures) are
best-effort: logged at debug/warn and retried next tick.
### Self-notification filtering
Forgejo fires notifications for the agent's own actions (it opened a
PR, posted a comment, submitted a review). Surfacing those would
loop claude on its own writes. `forge_notify` drops the comment/review case
silently (mark-read without delivery):
- **Self-authored comments / reviews** — comment payload's
`user.login` matches `own_login`.
- **Self-authored creations** (an agent opening its own PR/issue) — the
already-fetched subject payload's poster `user.login` matches
`own_login`. `forge_notify` drops only _creations_ this way; a later state
change on the agent's own subject comes from someone else, so it still surfaces.
`forge_notify` fetches `own_login` at startup via `GET /api/v1/user`. On fetch
failure the filter degrades open (no filtering) rather than crashing
the task — a noisy inbox beats a silently stuck poller — but the fetch
is **re-attempted on each poll tick** until it succeeds, so a boot-time
failure (the forge not yet reachable) self-heals instead of leaving
self-echo filtering off for the whole process lifetime.
### Body excerpt + truncation + heading escape
The wake message embeds the comment / review body so the agent sees
actual content without a follow-up fetch — **not** a new issue/PR's own
description, which the wrapper table below deliberately omits. Three
pipeline steps in order:
1. **Truncate** to `BODY_TRUNCATE = 500` chars at a char-boundary;
appends `…` when cut. Truncation happens BEFORE escape so the
mention-overflow diff (next step) compares like-for-like against
the raw body.
2. **Mention overflow extraction** — when truncation actually
trimmed content, walk the full body line-by-line and surface any
`@username` lines that fell outside the embed window. Rendered as
a trailing `mentions (truncated from body):\n > <line>` block.
Mention detection requires the `@` to be at line start or
following a non-username byte, so email-style `foo@bar.com` does
NOT count.
3. **ATX heading escape** — for each line that's a strict
`CommonMark` ATX heading (1-6 leading `#`s followed by a space,
tab, or end-of-line), prepend `\` so the embedded body doesn't
blow into a top-level h1/h2 inside the wrapper message when the
dashboard renders it. Lines like `#tag`, `#123`, `#!/bin/bash` <!-- lint:allow: syntax examples, not real tags -->
**aren't** headings — no escape, no cosmetic noise. Indented
"headings" inside lists / nested quotes keep their leading
whitespace.
The strict ATX rule is deliberate: `\#tag` and `#tag` render
identically, so an over-eager escape just adds visual clutter
without changing behavior. Setext-style headings (`title\n====`)
aren't handled — rarer in practice, would need multi-line
lookahead.
### Wrapper format
Five shapes, distinguished by the notification's classification:
| Trigger | Wrapper |
| ----------------------------------- | ------------------------------------------------------------------------------ |
| Comment on issue / PR | `[comment on PR #N owner/repo] title\nurl: ...\n\nauthor: body\nassignee: ...` |
| Review submission | `[PR approved #N owner/repo] title\nurl: ...\n\nauthor: body\nassignee: ...` |
| New issue / PR | `[new PR #N owner/repo] title\nurl: ...\nassignee: ...` |
| Later activity (open, not creation) | `[activity on PR #N owner/repo] title\nurl: ...\nassignee: ...` |
| State change | `[PR merged #N owner/repo] title\nurl: ...\nassignee: ...` |
The issue/PR's own description is deliberately never embedded here (only a
comment/review body is — that's the actual new content a notification
exists to surface): repeating a subject's own description on every "new"
or "activity on" wake for it's noise the agent already has via the URL,
not context (mara, triage: "just the issue title should be enough").
Review labels come from the Forgejo `state` field: `APPROVED` →
`approved`, `REQUEST_CHANGES` → `changes requested`, `COMMENT` →
`review comment`. `forge_notify` drops `PENDING` (review saved but not
submitted yet — no peer-visible event). Unknown states fall back to
the generic comment wrapper.
A review submitted with **no body** renders `reviewed by: <author>` in
place of the `<author>: <body>` line — deliberately worded to not collide
with the meta-suffix `reviewer:` line (requested reviewers, below).
### Inline review comments
A review's line comments belong to neither the review body nor the
`latest_comment_url`. A review with only inline comments has no such url
and would otherwise land as a bare `activity on PR`. Every PR wake therefore
fetches the PR's reviews (`/pulls/N/reviews`, up to four pages of 50) and,
for the reviews new to the agent, appends an
`inline review comments (read them with `hive-forge pr reviews N`):` block
saying where they sit: one `author: path:line, path, ...` line per review
author (bare `path` when Forgejo dropped the position because the line left
the diff; control characters in a path become spaces). The comment text stays
out of the wake, and the agent reads the full review with the CLI.
A review counts as new when it landed strictly after the thread's previously
delivered `updated_at`. Without a baseline every review counts, and so does
one with no submit time, so nothing disappears. It doesn't count when the
agent wrote it, when it's a `PENDING` draft, or when `comments_count` is 0.
The notifier fetches the newest 5 such reviews and lists at most 10 locations
(`(+N more)` for the rest). Best effort: a failed fetch yields less text and
never drops the wake. Cost: one reviews fetch per PR wake, plus one per new
commented review.
### Merge/close vs a later comment
A notification carrying a `latest_comment_url` normally takes the comment
path. But a merged/closed subject **keeps** its `latest_comment_url` set,
so a just-merged PR that had any prior discussion would route to the
comment path and render `[comment on PR]` (with a stale pre-merge comment
body) instead of `[PR merged]` — the agent never learns its PR merged.
When the notification IS the merge/close transition — its
event time (`updated_at`) is within `NEW_ITEM_TOLERANCE_SECS` of the
subject's `closed_at` (set for both `merged` and `closed`) — the
state-change path wins even with a comment url present
(`state_change_is_current`). A genuine **later** comment on an
already-closed subject bumps `updated_at` well past `closed_at`, so it
stays on the comment path and keeps its comment body. Missing/unparseable
timestamps default to the state-change path, so a merge is never silently
hidden behind a stale comment.
#### Merge racing a comment
The one gap the timestamp cut leaves: a genuine comment posted **within
`NEW_ITEM_TOLERANCE_SECS` of the merge** bumps `updated_at` close enough
to `closed_at` that `state_change_is_current` returns `true` — so it takes
the state-change path, dropping its body. Best of both worlds:
on the merge/close path `forge_notify` fetches the `latest_comment_url` comment and, when
its `created_at` is strictly **after** the subject's `closed_at`
(`comment_is_after_close`) — in other words, it raced the merge rather than being the
pre-merge last comment the subject keeps — append it as a
`comment by <author>: <excerpt>` block before the meta suffix
(`fresh_post_close_comment_tail`). The wake carries **both** `[PR merged]`
and the racing comment. `forge_notify` leaves the kept pre-merge comment (created before
`closed_at`) off, drops a self-authored racing comment (don't
echo the agent's own write), and appends nothing for a missing/unparseable `created_at`/
`closed_at` (conservative — only surface a comment it can
positively place after the close). Cost: one extra comment fetch on
merge/close notifications, acceptable given how rare they are.
### "new" vs "activity on"
A review submitted with **no body** carries no `latest_comment_url`,
so it misses the comment path and lands on the state-change path with
`state == "open"` — exactly like a freshly opened PR. Labeling that
`new PR` is misleading: agents dismiss it as a duplicate of the
original open notification and miss the review. The `open`
state only earns the `new <kind>` label when the notification's event
time (`updated_at`) is within `NEW_ITEM_TOLERANCE_SECS` (120s) of the
subject's `created_at`. `forge_notify` labels anything later `activity on <kind>`
— neutral and non-misleading, since it can't cheaply say _what_ the
activity was without an extra reviews fetch. Missing/unparseable
timestamps default to `new` (preserve prior behavior rather than mask a
genuine new item). A small dependency-free
RFC 3339 helper (`parse_rfc3339`) parses timestamps.
`forge_notify` extracts the number from `subject.html_url`'s last path segment
(strips `#anchor` first), and the repo slug from `repository.full_name`.
Both degrade gracefully when absent (number → blank, repo → blank)
so unexpected Forgejo shapes don't crash the formatter.
### Meta suffix
Every wrapper ends with one or more of:
- `assignee: <list>` — always present; `unassigned` when empty so
the line shape is stable.
- `reviewer: <list>` — PR notifications only, present only when
`requested_reviewers` is non-empty.
### Review-request override
For new PRs, the kind label flips to `[review requested #N
owner/repo]` when `own_login` appears in `requested_reviewers`,
regardless of the Forgejo `reason` field. Forgejo doesn't reliably
set `reason == "review_requested"` (often null instead), so the
fallback checks the subject payload directly. `is_new` gates detection
so the label only fires once on PR creation, not on every
subsequent comment.
### Subscription management
The poller does **not** autounsubscribe from repo watches — it
delivers every unread notification it's handed. A hive-forge CLI
subscription verb bounds the firehose explicitly (dropping broad repo
watches an agent doesn't need) — the poller never guesses which
watches to drop. See the `subscription` verb in
[`docs/2-handbook/tools/forge.md`](../tools/forge.md).

150
docs/2-handbook/integrations/github.md Normal file
View file

@ -0,0 +1,150 @@
# GitHub accounts
Give an agent a managed GitHub identity — a `gh` CLI and `git push` over
HTTPS, both authenticated by an operator-supplied personal access token
(PAT) — so it can run GitHub API calls and push commits without any manual
`gh auth login`.
Provisioning is UI-driven: link a PAT to the agent in the swarm UI and it
works. No per-agent nix declaration, no rebuild — the agent fetches the token
from the swarm secret store itself.
## Enabling
<!-- vale write-good.Passive = NO -->
The integration is **on by default** for every agent (`services.hyperhive.agent.github.enable
= true`), inert until the operator provisions a PAT. No per-agent declaration is
needed — an agent gains GitHub by having a PAT stored for it.
<!-- vale write-good.Passive = YES -->
To turn it off for the whole swarm, set the swarm option:
```nix
services.hyperhive.swarm.github.enable = false;
```
hive-c0re's meta-flake renderer then injects `services.hyperhive.agent.github.enable = false`
into every agent, so no agent ships the `gh` wrapper, the credential helper
or the token fetch.
(`services.hyperhive.agent.github.enable` also exists per-agent for completeness, but the
swarm-wide switch is the intended control.)
github.com only. The token **value** never touches nix — the agent writes it
to `<state>/github-token` at runtime (see [Provisioning](#provisioning)).
## Provisioning
The PAT is operator-supplied. In the [swarm UI](../swarm/ui.md#linking-external-accounts),
open the agent on `/agents`, choose **link github account** and paste the PAT
(`PUT /api/hives/{hive}/agents/{agent}/github-account`). swarm-controller
stores it at `swarm/agents/<agent>/github-token` in the swarm secret store;
no hive writes it. One token per agent: swarm-controller refuses to link
another until you delete the stored one, and no route hands it back.
The agent's `hive-agent-github-token` unit reads that path under the
agent's own store certificate and writes `<state>/github-token` (`0600`,
owned by the agent), on boot and every two minutes, replacing the file only
when the token changed. It needs a store identity
(`services.hyperhive.agent.bao.addr`); an agent without one gets no token.
The unit never deletes the file: a `github-token` already in place stays
when the store holds none or doesn't answer. Where the token lives and who
reads it: [credentials-reference.md](../swarm/credentials-reference.md).
## Security
- Use a **dedicated bot account**, never a human's.
- Mint a **minimally scoped PAT** — only the repos/scopes the agent's
workflow needs. Agents have passwordless sudo, so a compromised or
hallucinating agent can act as the account within the token's scopes;
scope is the real blast-radius limiter, and the container boundary is
the enforcement. See [security.md](../trust-boundary/security.md).
## Implementation
<details>
<summary>How the agent's `gh`/`git push` actually authenticate, and how
the notification poller works</summary>
### How the agent uses it
When enabled, the container gets:
- **A `gh` wrapper** on `PATH` (shadowing the raw `gh`) that exports
`GH_TOKEN` from the token file at invocation, then execs real `gh`. So
`gh pr create`, `gh api …`, etc. just work — `gh` derives the identity
from the token.
- **A git credential helper** (`git-credential-hive-github`), wired via a
host-scoped `/etc/gitconfig` entry for `https://github.com`, so
`git push https://github.com/<owner>/<repo>` authenticates as
`x-access-token` + the PAT (GitHub ignores the username for PAT auth).
Host-scoped, so it never touches the forge (`localhost:3000`) or any
other remote.
Both scripts read the token from `<state>/github-token` **at invocation
time**, so a PAT written (or rotated) mid-session takes effect immediately
— no container rebuild or restart. Until the file exists, `gh` / `git push`
fail unauthenticated. The build bakes the token path into the scripts
(rather than reading it from an env var), because claude's Bash tool runs in a
minimal environment that wouldn't carry one.
### Notifications
`hive-github-notify` polls github.com for the agent, turning each unread
notification thread into a todo. it's a **separate binary and a
separate systemd unit** from the internal forge's poller
(`hive-forge-notify`, see [forge.md](forge.md#notification-poller-hive-forge-notifysrcnotifyrs)),
installed by `nix/agent-modules/github.nix` under
`services.hyperhive.agent.github.enable`. Both binaries ship from the one
`hive-forge-notify` derivation, so the unit is a second `ExecStart`
path, not a new package.
Two units rather than one daemon with two loops, because it puts the
decision in nix: a hive built without this module has **no github poller
in its closure at all**, which is what makes GitHub access separable
rather than merely switched off. it's also why this isn't a cargo
feature — a feature would unify across the workspace and cost every
crate its build cache.
At runtime the poller needs the PAT above. No PAT, no polling: the unit
logs why and exits 0, which is why it's `Restart = on-failure` and
never `always` — a clean exit on a PAT-less agent must not become a
restart loop. Once polling, it re-reads `<state>/github-token` on every
tick, so it picks up a rotated PAT on the next tick without a restart.
Forgejo models its notifications API on GitHub's, so one tolerant
parse serves both: `id`, `repository.full_name`,
`subject {title,url,latest_comment_url}` and `updated_at` line up field
for field. Lenient deserializers absorb the two real differences —
GitHub sends the thread id as a _string_ where Forgejo
sends a number, and says `PullRequest` where Forgejo says `Pull`.
`hive-github-notify` prefixes todo keys `gh:<id>` so a github thread id can't collide with a
forge one.
Two host differences worth knowing before touching this code:
- **Auth scheme, not just value.** Forgejo takes
`Authorization: token <t>`; GitHub wants `Bearer <t>` plus `Accept:
application/vnd.github+json`, `X-GitHub-Api-Version` and a
`User-Agent`. Sending Forgejo's form to GitHub doesn't error — it
authenticates as _nobody_ and silently drops to the unauthenticated
rate limit. The cheap way to tell the two apart is the rate-limit
header: `x-ratelimit-remaining` near 5000 is an authenticated user,
near 60 is anonymous.
- **GitHub sets the cadence.** It returns `X-Poll-Interval` (60s in
practice, slower than the poller's own tick) and rate-limits callers who
ignore it, so the loop re-arms to the server's interval whenever that's
_slower_ than the poller's. A hint faster than the poller's own tick
isn't a reason to poll harder.
⚠️ **This needs the `notifications` scope on the PAT.** A token minted
for `gh` + `git push` typically carries `repo` only, which is enough to
push and open PRs but **not** to read the notification stream (nor to
mark a thread read, which is the same scope). A PAT without it doesn't
break anything: the poller logs the refusal and stays quiet, and the
agent never gets GitHub wakes. If an agent's GitHub
notifications never arrive, check the token's scopes first — the
symptom is silence, not an error.
</details>

91
docs/2-handbook/integrations/gitlab.md Normal file
View file

@ -0,0 +1,91 @@
# GitLab accounts
Give an agent managed accounts on one or more GitLab instances: a `glab` CLI
and `git push` over HTTPS, both authenticated by operator-supplied access
tokens, with no manual `glab auth login`.
An agent can hold any number of accounts, each under a label you choose. Two labels
can name two instances, such as `gitlab.com` and a self-hosted one. Provisioning
needs no per-agent nix declaration and no rebuild: the agent fetches its tokens
from the swarm secret store itself.
## Linking an account
In the swarm UI, open an agent, choose **link account**, then **gitlab
account**. Fill in a label, the instance URL (it starts as `https://gitlab.com`)
and the token. The form shows the recommended scopes and the controller's
answer if it refuses the input. The linked account appears in the agent's list
of [linked accounts](../swarm/ui.md#linking-external-accounts) with a delete
action. The UI calls the controller's API, which you can also call directly:
```text
PUT /api/hives/{hive}/agents/{agent}/gitlab-accounts/{label}
{"url": "https://gitlab.example.org", "token": "<access token>"}
```
- `label` is lowercase letters, digits and hyphens. It names the account in the
store and in the account list.
- `url` is the instance's base URL. The host must be a DNS name: an IP
address, `localhost` and a single-label name get a 400, because the agent
reaches the instance through its hive's egress proxy, which takes names
only. A port and a subfolder (`https://example.org:8443/gitlab`) are part of
the address. The path may hold letters, digits and `. _ ~ / -`; a URL with
user information, a query, a fragment or any other character gets a 400, so
every stored account is one the agent uses.
- A plain `http://` URL works, and then the token crosses the network in
cleartext. Use `https://` unless the instance sits on a trusted network.
- `token` is a GitLab personal, group or project access token. Scope it
`api` and `write_repository`: `glab` needs the first, and `git push` over
HTTPS needs the second. Use a dedicated bot account and the narrowest token
that does the agent's work, because the agent has passwordless sudo and can
act within the token's scopes.
A label that holds an account already answers 409; delete it first
(`DELETE .../gitlab-accounts/{label}`), then link again. No route hands a
token back. `GET /api/hives/{hive}/agents/{agent}/linked-accounts` lists the
account as kind `gitlab`, with its label and URL.
Linking also allows the instance's host in the agent's egress, as a
`link:gitlab:<label>` entry on the swarm's allowed-domains list. swarm-controller
derives the host from the URL it stored, so nothing the agent writes can widen
its egress. Deleting the account removes the entry.
## What the agent gets
An agent with a store identity (`services.hyperhive.agent.bao.addr`) has:
- the `glab` command, which reads its configuration from `<state>/glab/`;
- a git credential helper, `git-credential-hive-gitlab`, for each linked
instance;
- the `hive-agent-gitlab-accounts` unit and its timer.
The unit logs in to the store with the agent's own certificate, lists
`swarm/agents/<agent>/gitlab/`, reads each account, and writes two files in
`<state>/glab/` (mode `0600`) on boot and every two minutes, replacing a file
only when its content changed:
- `config.yml`, `glab`'s own configuration with one `hosts` entry per instance,
keyed by host and port, holding the token, the API protocol and the
subfolder. It sets `git_protocol: https`. The `host` default is the first
label's instance; inside a repository `glab` uses the host of its remote.
- `gitconfig`, one `[credential "<scheme>://<host>"]` block per instance, which
`/etc/gitconfig` includes. The helper answers only for a host that has an
account, so it never touches GitHub or the swarm's forge.
The unit generates both files. A deleted account leaves them on the next run, and
a hand edit doesn't survive. `glab` keeps one token per host: when two labels
name the same host, the first label in alphabetical order wins. If the unit
can't read one of the accounts, it replaces nothing and retries, so the files
keep the last complete set.
The unit fails fast and restarts itself every 30 seconds for up to a day, as
the other store fetches do, so a store that's down or sealed never holds the
container's boot open.
## Rolling out
The endpoint, the store path and the agent unit are new, and each end ignores
what it doesn't know. An old controller has no GitLab route, so no account gets
linked. An old agent has no unit, so a stored account sits unused. A new agent
under an old controller lists a store directory that doesn't exist and writes an
empty `glab` configuration.

156
docs/2-handbook/integrations/knowledge.md Normal file
View file

@ -0,0 +1,156 @@
# Swarm-wide knowledge repository
`internal/knowledge` on the forge is a shared reference doc repo
readable by every agent. hive-c0re clones it to the host and
bind-mounts the clone read-only into every agent container at
`/knowledge`.
## Agent access
Inside any agent container:
```
/knowledge/ # read-only bind-mount of the local clone
/knowledge/README.md # table of contents (seeded on first use)
```
<!-- vale write-good.Passive = NO -->
Agents read documents directly from that path. The mount is
read-only — agents never write through it. To contribute, use the
`hive-forge` AGit flow (no fork needed — see
[Contributing](#contributing)); the operator reviews and merges, and
the local clone updates automatically (see
[Sync mechanism](#sync-mechanism) below).
<!-- vale write-good.Passive = YES -->
## Repository layout
Canonical forge location: `internal/knowledge` (org `internal`,
repo `knowledge`). The repo is public, so every agent's forge account
has read access without an explicit per-agent collaborator grant;
only site admins can push: `core`, and swarm-controller's own account,
which seeds the README.
The swarm controller creates the repo if it doesn't exist, at startup
and every few minutes after, and seeds an empty one with a `README.md`
containing a contribution guide and a blank table of contents. Add an entry to that ToC each time you
create a new document.
## Sync mechanism
hive-c0re maintains the local clone at
`/var/lib/hyperhive/knowledge` via two paths:
1. **Swarm event** — the swarm controller holds the single push hook on
`internal/knowledge` (see `docs/2-handbook/swarm/README.md` § Swarm-wide forge
webhooks). On any push to main, including merge commits, it sends an
event to every hive over the swarm queue and each hive runs `git
pull`, so agents see the new content on their next turn.
A hive that's offline when the swarm controller sends the event
doesn't get it on reconnect — the periodic pull below is what closes that gap. One
hive briefly showing older `/knowledge` content than another is
expected, and resolves by itself within the fallback interval.
**don't add a per-hive hook.** A webhook has exactly one target
URL, so a second registration against the same repo doesn't add a
recipient — it takes delivery away from whoever registered first.
Earlier versions had each hive register its own, and later ones
removed that leftover at startup. A hive upgraded straight past those
versions still holds its old hook: delete any hook on the repo that
points at a hive's own `/webhook/knowledge`.
2. **Periodic pull** — a background task in `hive-c0re::main`
queues a pull on a fixed cadence as a fallback (webhook missed, c0re
restarted between pushes). The pull is best-effort — a failure
shows as a failed `KnowledgePull` node on the job queue, counts toward
the `knowledge_pull` hive warning, and doesn't affect the rest of the daemon.
Both paths, and the one pull at boot, queue the same `KnowledgePull` job
node. It holds the working tree for its duration, so two pulls never run
over each other — each trigger queues its own pass, and a pass that lands
while another is already queued or running waits its turn, since the
one ahead of it may have fetched before the push. The node runs
`knowledge::pull()`, which also handles the change notice below.
### Change notice
When a pull actually moves the local clone's `HEAD` (a real change,
not a no-op — for example the periodic pull finding nothing new), hive-c0re
broadcasts a short notice to every currently registered agent's inbox:
sender `system`, body `[system] /knowledge updated:` followed by a
`git diff --stat <old>..<new>` summary of what changed (or a generic
"see the repo" fallback if computing the diff itself fails). This is
the same broadcast mechanism used for other hive-wide notices — inbox
message only, no forced wake, and it carries the standard "this was a
broadcast" hint. hive-c0re logs a diff or per-agent send failure
without blocking the pull itself.
On a hive with no local clone yet, `knowledge::pull()` takes the clone
instead of pulling, so every hive gets `/knowledge` from its first pull,
whether or not it runs the forge. It authenticates with the hive's forge
read token when it has one (see
[forge.md](forge.md#how-a-hive-reads-the-forge)) and clones the public
repo anonymously otherwise.
## State
<!-- vale write-good.Passive = NO -->
- **Host clone**: `/var/lib/hyperhive/knowledge` — persists across
hive-c0re restarts and agent destroy/recreate. Deleted only by
manual operator action.
- **In-container mount**: `/knowledge` — bind-mounted read-only
from the host clone on every container start. Gone when container
is stopped; reappears on next start with the current clone state.
<!-- vale write-good.Passive = YES -->
The mount deliberately **excludes `.git`**: hive-priv overlays an empty
tmpfs at `/knowledge/.git`, so agents see the documents, not the branches or
history. An older clone also kept a token in `.git/config`; `knowledge::pull()`
rewrites that remote to the plain URL.
## Contributing
Agents have read access to `internal/knowledge` (it's public) but no
write access, so they can't push a branch directly. The supported path
is Forgejo's **AGit flow** through the `hive-forge` CLI — no fork
required.
1. Clone the repo (`hive-forge` injects credentials automatically;
the `-r` flag selects the repo, the clone lands in `./knowledge`):
```sh
hive-forge -r internal/knowledge clone
cd knowledge
```
2. Create a branch and add or update a document, then commit normally.
3. Open (or update) a PR with `--agit`. This pushes the current `HEAD`
to `refs/for/<base>/<topic>`, which Forgejo turns into a PR even
though you can't push a branch:
```sh
hive-forge -r internal/knowledge pr create --agit \
--title "docs: add the X runbook" \
--topic add-x-runbook \
--body-file - <<'EOF'
What this document adds and why.
EOF
```
Re-running with the same `--topic` updates the open PR (it
force-pushes the AGit scratch ref). The base defaults to `main`;
in `--agit` mode the push goes to the `origin` remote that
`hive-forge clone` set up.
4. The operator reviews and merges. The webhook fires on merge; every
running container sees the updated content within seconds (see
[Sync mechanism](#sync-mechanism)).
Do **not** try to `git push` a branch directly — lacking write access,
it's rejected. The `--agit` flow above is the no-fork path that works
from any agent.

442
docs/2-handbook/integrations/matrix.md Normal file
View file

@ -0,0 +1,442 @@
# Swarm Matrix homeserver
Private Matrix homeserver (matrix-tuwunel — the conduwuit
successor) wrapped in a nixos-container (`hive-matrix`), plus optional fluffychat-web
client at `chat.<swarm-domain>/` (the `gatewayHost` vhost). A swarm runs one
homeserver, on one host. Two namespaces configure it:
- `services.hyperhive.swarm.matrix.*` — what the homeserver **is**, as
every hive sees it: `serverName`, `gatewayHost`.
- `services.hyperhive.deploy.matrix.*` — what the host running it decides:
`enable`, `gui.package`, `trustedServers`,
`maxRequestSize`, `sso.clientSecretFile`.
Vhost routing lives in [`gateway.md`](../../4-implementation/networking/gateway.md).
## Container shape
Same shape as [`gateway.md::hive-forge container shape`](../../4-implementation/networking/gateway.md):
- Container name `hive-matrix` (not `h-*`) so c0re's lifecycle
scanner ignores it; operator manages via the standard
`nixos-container` CLI.
- Keeps hive-matrix from fighting any `services.matrix-*` the
operator already runs on the host — separate systemd namespace,
separate state dir.
- Container runs in a network namespace of its own
(`privateNetwork = true`) on the hive bridge, and binds no TCP port. Agents
reach the homeserver at `chat.<swarm-domain>` via the gateway, which
dials its unix socket (see [Reach](#reach)).
- Persistent state at
`/var/lib/nixos-containers/hive-matrix/var/lib/matrix-tuwunel/`
survives container restart / host reboot. To wipe, destroy the
container.
## Identity vs API listener: `serverName` vs `gatewayHost`
Two distinct hostnames:
- **`serverName`** — matrix-spec `server_name`, embedded
*irrevocably* in every `@user:<server_name>` and `!room:<server_name>`
identifier minted on this homeserver. You can't change it later
without abandoning every account and chat history. Defaults to the
bare `services.hyperhive.swarm.domain`.
- **`gatewayHost`** — the API listener hostname, where the gateway's
matrix vhost proxies `/_matrix/*` to tuwunel. Defaults to
`chat.<services.hyperhive.swarm.domain>`. Set to `null` to skip the
gateway vhost, which leaves nothing able to reach the homeserver.
Both default under the **swarm** domain, because a swarm runs one
homeserver: tying its identity to a single hive's domain would make
relocating the container between hives look like a different
homeserver.
⚠️ **they're still not interchangeable, and the difference is the
cost of changing one.** `gatewayHost` is a routing detail clients
rediscover through `.well-known`, so it's safe to move on a running
deployment. The controller's host serves that `.well-known` on the swarm
domain, and a homeserver pinned to a hive domain gets it on that hive's
vhost. The matrix id format bakes `serverName` into every user and
room id, so adopting a new one does **not** rename the existing users and rooms —
it strands them, because their ids still name a homeserver that no
longer answers.
<details><summary>Pinning <code>serverName</code> on a homeserver with existing ids</summary>
`serverName` defaults to the bare `services.hyperhive.swarm.domain`. A
homeserver that changes `serverName` after minting ids strands its
existing users and rooms — keep the value it minted them under:
```nix
services.hyperhive.swarm.matrix = {
# whichever this deployment already uses
serverName = config.services.hyperhive.domain;
gatewayHost = "matrix.${config.services.hyperhive.domain}";
};
```
A rebuild on a host that already has a homeserver prints a
`hive-matrix: WARNING — … serverName is unset` line when this is missing,
naming the value it's about to default to. It never fails the rebuild, so
pin the value before the homeserver mints any ids under the default.
A pinned hive domain answers `.well-known/matrix/*` itself, built from
`gatewayHost`, on any host that runs that hive's gateway. It doesn't need
the homeserver on the same host.
</details>
## Reach
tuwunel binds no TCP port. It listens on the unix socket
`/run/hive-matrix/tuwunel.sock`, in a host directory (`root:nginx`,
`0770`) bind-mounted into the container, and the gateway's
`/_matrix/` location proxies to that socket. Clients, agents, hive-c0re
and federation peers all reach the homeserver at `chat.<swarm-domain>`
through the gateway; from outside the host that takes
`gateway.openFirewall`.
hive-c0re doesn't talk to the homeserver: matrix is swarm-level, and
the hive has no matrix API URL to set.
Federation peers find the homeserver through `.well-known/matrix/server`
at the server name, which names `<gatewayHost>:443`
([`gateway.md::Discovery flow`](../../4-implementation/networking/gateway.md#discovery-flow-matrix)).
Port 8448 isn't used.
Outbound, the container sits on the hive bridge like the CI runner: it
leases an address from the gateway's DHCP, resolves through its dnsmasq,
and leaves through the bridge's NAT. It reaches the secret store,
authelia and federation peers by name.
## Provisioning flow (appservice)
<!-- vale write-good.Passive = NO -->
Registration is closed. **The swarm's appservice** creates accounts: agents
never see an appservice token, and an agent only ever receives its own
`access_token`.
<!-- vale write-good.Passive = YES -->
The appservice has no URL (`url: null` in its registration), so the
homeserver never calls out to it and there is no service to run. What the
registration buys is an identity the homeserver recognises — which is why
no secret has to be equal on both sides of the wire, and why account
creation doesn't depend on registration being open to anyone who learns
a token.
1. **`swarm-matrix-ctl appservice render`**, inside the `hive-matrix`
container and before tuwunel starts, fetches the `as_token` and
spec-required `hs_token` from the store and renders the registration to
`/run/swarm-matrix-appservice/swarm.yaml` (mode `0600 root:root`) on the
container's tmpfs, so no token sits on disk. It mints a pair only when
the store answers and holds none. While the store doesn't answer, `render`
fails and retries, and tuwunel stays down.
2. **systemd `LoadCredential=`** copies the registration into
`/run/credentials/tuwunel.service/swarm-appservice.yaml`, owned by
tuwunel's dynamic user with mode `0400`, at service start. The source
file stays `root:root 0600` — no `chown :tuwunel` / `chmod 0640` /
GID-pin gymnastics required. Keeps `DynamicUser = true` +
`PrivateUsers = true` intact.
3. tuwunel's `appservice_dir` points at the credentials directory. It
reads only `.yaml`/`.yml` entries from there, so the sibling
credentials are invisible to it. The `.yaml` suffix on the credential
id is what makes this work.
4. **hive-c0re** holds no appservice token, creates no account and does
not talk to matrix.
5. **`swarm-controller` creates every account**: each agent's, with its
token stored at `swarm/agents/<agent>/matrix/main` for the agent's
`hive-matrix-daemon` to read as the agent itself, and each hive's
sender account (below).
A homeserver whose `swarm-matrix-ctl` has no store identity
(`services.hyperhive.deploy.matrix.ctlBaoClientCertFile`) loads no
registration, and nothing creates accounts on it.
On the store's host `ctlBaoClientCertFile` / `ctlBaoClientKeyFile` default
to the leaf `glue-bao-tls.nix` mints there. A homeserver host without the
store gets no default, even when it holds the hive's own reader
`deploy.bao.clientCertFile`: issue a leaf whose subject is
`deploy.bao.matrixCtlCommonName`, place it, and name it in both options.
Until you do, `swarm-bao-matrix-oidc` fails naming both.
### The swarm's appservice, and its admin sender
The registration's id is `swarm`, its sender `@swarm`, and its user
namespace non-exclusive. It's the swarm's identity on the homeserver:
`swarm-controller` creates every agent's account and every hive's sender
account with it.
The swarm secret store holds the registration's tokens, at
`swarm/controller/swarm-controller/matrix/appservice-registration`.
`swarm-matrix-ctl appservice render` fetches them and renders the
registration tuwunel loads as a credential onto the `hive-matrix`
container's tmpfs, before tuwunel starts.
A render that can't reach the store fails and retries, and tuwunel, which fails to start without the registration, retries until one has landed.
`appservice sync` runs beside tuwunel on a timer. The store's tokens win:
when they differ from the rendered copy, it rewrites it. It
restarts tuwunel whenever the registration is newer than tuwunel's last
start, so the next run retries one that failed after writing, and starts
one that failed because `render` couldn't reach the store. A store that
holds none takes the tokens an earlier version kept in
`/var/lib/swarm-matrix-appservice` with a create-only write, so a running
swarm keeps its value; the files stay, unused. A host with neither files nor stored tokens mints a pair. `sync` then writes the `as_token`
to `swarm/controller/swarm-controller/matrix/appservice-token`. No hive's
policy reaches either path: only matrix-ctl (which holds `create` and
`read` on the registration, and writes the token) and `swarm-controller`
(which reads only the token) hold a grant.
`@swarm` is the **one** account the homeserver promotes to admin at boot
(`admin_execute`). Only those two principals read its token, which is
the difference from the hive sender below: every hive reads its own
sender's token. `swarm` is a reserved name, so nothing can create an agent
as this account.
`swarm-controller` mints each agent's account with the device id
`hyperhive-<agent>`, and a login on that device replaces its token. The controller therefore reads
the stored token back with `whoami` every five minutes and re-mints only
when it's missing, unknown to the homeserver, or someone else's. An
agent's daemon picks a re-minted token up on its next start, and a timer
restarts it while it's down.
### Operators are homeserver admins
Every member of authelia's `admins` group who has a matrix account is a
homeserver admin. `swarm-controller` reads the group through
`swarm-authelia-bridge` at start and every five minutes after, and for
each member not yet in `#admins:<server_name>` it sends
`!admin users make-user-admin @<name>:<server_name>` there as `@swarm`.
`<name>` is the authelia username lowercased, the account an SSO login
lands on.
- **Leaving `admins` revokes what the sync granted.** The controller
records each account it sends a grant for in
`/var/lib/swarm-controller/operator-matrix-admins.json`. When one of
them is no longer in `admins` but still in `#admins`, it sends
`!admin users force-leave-room @<name>:<server_name> <admin room id>`
there as `@swarm`; leaving that room is what ends homeserver admin.
- **Admins the sync didn't make keep admin.** An admin missing from that
file, `@swarm`, the server user `@conduit:<server_name>`, every agent's
account, and every user id in
`services.hyperhive.deploy.swarm-controller.matrixAdminSync.ignoreUsers` are never
revoked.
- **The last admin stays.** A revoke that would leave no admin in
`#admins` besides `@swarm` and the server user isn't sent; the
controller logs a warning on each pass instead.
- **No account, no grant.** A member who has never logged in to matrix
has no account to promote. The first pass after their first SSO login
promotes them.
- **One command per account per controller run.** A member still missing
from `#admins` after the controller sends its grant gets a
`make-user-admin … failed to take` warning in the controller log on
each later pass, and no second command until the controller restarts;
a revoke that leaves its account in `#admins` gets
`force-leave-room … failed to take` the same way.
- Runs only on a controller with both
`services.hyperhive.deploy.swarm-controller.authBridgeUrl` and
`services.hyperhive.deploy.swarm-controller.matrixHomeserverUrl` set.
### A hive's sender account, and why it isn't an admin
`@hive-<hive>:<server_name>` is the account a hive acts as.
`swarm-controller` creates it with the swarm's appservice token, for every
hive in its directory. It's an **ordinary account**: nothing promotes it;
the homeserver's `admin_execute` promotes only `@swarm`.
**One account per hive.** The localpart carries the hive's name, so a swarm
whose hives share a homeserver gives each of them its own identity: the
homeserver attributes an action to the hive that took it, and revoking one
hive's standing leaves the others alone.
Its access token is the **sender token**. `swarm-controller`, its only
minter, mints it for every hive with the swarm's appservice token and
publishes it to `swarm/hives/<hive>/matrix/sender-token`, inside the hive's own
read grant (`swarm/hives/<hive>/*`). Nothing on the hive uses it any more:
hive-c0re used to present it for its Space and invite calls and no longer
makes any. The name says what it authenticates as — an account the
swarm's appservice creates — rather than any privilege level, because it
carries none.
Promoting a user to homeserver admin and resetting a password both need an
admin **sender**: `!admin …` messages into `#admins:<server_name>`, and
tuwunel only treats a message as a command when its sender is already an
admin. Of the swarm's own accounts only `@swarm` is a homeserver admin
(operators are too, see
[Operators are homeserver admins](#operators-are-homeserver-admins)), so
both are swarm-level operations.
<details><summary>Appservice identity</summary>
`swarm-matrix-ctl appservice render` renders the registration from the
store's copy of the appservice tokens before tuwunel starts, so every
boot has both halves. With the store down or sealed, tuwunel waits for it.
<!-- vale write-good.Passive = NO -->
- **Access tokens live on the device that minted them.**
`login_with_password` stays on, so the password fallback works
too.
- **The homeserver's `admin_execute` promotes only `@swarm` at boot**
(above); a hive's sender account is never promoted.
- **The value that matters lives at
`swarm/controller/swarm-controller/matrix/appservice-token`.**
`appservice sync` writes it there whenever the store's differs from the
registration's `as_token`, and `swarm-controller` reads it from there.
<!-- vale write-good.Passive = YES -->
</details>
Initial rollout settings:
- `allow_federation = true` at the protocol level so swarms can be
wired up later by extending `trustedServers` without a homeserver
restart. `trusted_servers = []` keeps it effectively closed
until you list peers.
- `allow_registration = false`. tuwunel checks this flag only for
requests that arrive **without** an appservice token, so the swarm's
appservice still creates accounts and tuwunel refuses everyone else. With no
registration token configured, `allow_registration = true` makes
tuwunel refuse to start unless
`yes_i_am_very_very_sure_…_open_registration_…` is also set.
- `allow_encryption` — server-side E2EE switch, sourced from
`services.hyperhive.deploy.matrix.allowEncryption` (**default `false`**, opt-in).
Off by default because on the swarm's own homeserver the operator
already controls the transport; turn it on for encrypted rooms on
external / federated homeservers or to keep contents opaque to the
homeserver admin. **The agent matrix client always supports decryption
regardless of this flag** — it uses the `e2e-encryption` feature of
`matrix-sdk` so it can read encrypted rooms it's invited to even when
this homeserver doesn't permit room encryption. matrix-sdk stores
crypto keys in the per-agent sqlite store under the state dir; they persist across
restarts. `read_room` decrypts via
`room.messages()` — UTD events surface as `event_type =
"m.room.encrypted"` with `body = "[unable to decrypt]"`.
The daemon bootstraps cross-signing on the bearer token alone, which
works where the homeserver skips re-auth for a first key upload
(MSC3967, as tuwunel does).
- **Key backup.** `swarm-controller` mints a random recovery passphrase
for every matrix account an agent has, at
`swarm/agents/<agent>/matrix-recovery/<account>` in the swarm secret
store, and never replaces it. When `hive-matrix-daemon` brings an
account up it reads that passphrase: with no backup on the account's
homeserver it turns on server-side key backup and secret storage
(log line `matrix recovery enabled`), unless the account already has
secret storage, which it never replaces (a `WARN` instead). With a
backup there it recovers from it (`recovered from the server-side key backup`), which downloads the
room keys and signs the new device with the account's existing
cross-signing identity. A crypto store lost to `--purge` or to a
primary-token reset therefore comes back with its history. With no passphrase
stored yet the account comes up without backup and picks it up on the
next daemon start. A failure is a `WARN` and never stops the daemon.
## Retired hive-side matrix code
Earlier versions of hive-c0re created a private "hive" Matrix Space and a
`hive-chat` room, invited every agent to both, kept the hive's sender token
on disk and offered `hivectl matrix invite`. Matrix is swarm-level, so
hive-c0re dropped all of it and no longer talks to the homeserver. The Space, the
room and the files `/var/lib/hyperhive/matrix/{access-token,space-room-id,chat-room-id}`
are harmless leftovers, so delete them by hand if you like.
## Configuration tuning
```nix
services.hyperhive.deploy.matrix = {
trustedServers = [ "matrix.org" "example.com" ]; # default: []
maxRequestSize = 20000000; # default: 20 MB
};
```
**`trustedServers`** (default `[]`) — list of peer homeserver names
whose signing keys tuwunel fetches and trusts. tuwunel enables
federation at the protocol level from first boot (`allow_federation =
true`) but trusts no remote homeserver until you list it here. For a closed
single-hive deployment the default empty list is correct — add peer
hive domains here when connecting hives into a swarm (see
[`docs/swarm/`](../swarm/README.md)).
**`maxRequestSize`** (default `20_000_000` bytes = 20 MB) — maximum
size of a single matrix client request body. Matches the matrix-spec
recommendation for media uploads and the upstream tuwunel default.
Raise for deployments that need large file transfers; lower for
resource-constrained hosts where a 20 MB request is unexpectedly large.
## Assertion rationale
`config.assertions` in this module fail eval early rather than ship
surprising behaviour:
- **`services.hyperhive.swarm.matrix.gatewayHost != ""`** — same footgun as `forge.domain`:
empty string renders `.<hive>`-shaped garbage in both nginx
`server_name` (treated as wildcard catch-all, surprising) and
`/etc/hosts` (malformed entry). `null` is the right opt-out shape;
a config assertion rejects empty string explicitly.
SSO is unconditional, so the three below are requirements of running a
homeserver at all rather than of a setting:
- **Keep `deploy.matrix.sso.clientSecretFile` set** — it defaults to
`/run/hive-matrix-oidc/<client id>.secret` wherever the homeserver runs,
and `swarm-bao-matrix-oidc` writes the secret there from the swarm secret
store under matrix-ctl's leaf. The path is on the host's tmpfs, so every
reboot fetches the secret again and it never reaches persistent disk. After
a reboot the homeserver doesn't start until the store answers; the
store-retry timer then starts it with no manual step. A path
outside `/run/hive-matrix-oidc/`, or a null, fails at eval, not at boot:
tuwunel reads its identity providers from the config file, so a half-configured
one can stop the homeserver from starting outright rather than merely
hiding a login button.
- **Set `swarm.authelia.url`** — without a provider URL there
is nothing to discover against.
- **Set `swarm.matrix.gatewayHost != null`** — the SSO callback URL is
format-locked to `<homeserver>/_matrix/client/unstable/login/sso/callback/<client_id>`,
and the identity provider needs a public name to redirect the
browser to.
`server_name`'s own bogus-value guard lives in `hive-network.nix`
(`services.hyperhive.domain != null`), not here — see
[`docs/4-implementation/networking/network.md`](../../4-implementation/networking/network.md).
## fluffychat-web build fixes
`pkgs.fluffychat-web` ships from `flutter341.buildFlutterApplication`,
which has two upstream gaps for fluffychat's web target:
- The dart web-worker entry point (`web/native_executor.dart`) isn't
compiled — `buildFlutterApplication` only runs `flutter build web`
on the main entry.
- `native_imaging`'s C source isn't built — emscripten isn't a
flutter-builder native build input.
Both fixed in `nix/host-modules/hive-matrix.nix` via two derivations:
- **`fluffychat-web-imaging`** builds `Imaging.{js,wasm}` from the
`native_imaging` C source via `pkgs.emscripten`. Source comes
from `pkgs.fluffychat-web.passthru.pubspecLock.dependencySources.native_imaging`
— already in the build closure of the flutter app, so no parallel
hash pin and version autosyncs with nixpkgs bumps. Build closure
is ~3.6 GiB (emscripten LLVM); runtime closure is just the two
output files. `dontConfigure = true` because cmake runs inside
`js/Makefile` via `emcmake cmake`, not at the package root. The
build script needs `HOME` + `EM_CACHE` writable for emscripten's
on-demand sysroot build (libc, libc++ → wasm).
- **`fluffychat-web-fixed`** is `pkgs.fluffychat-web` plus a
`postInstall` patch that (a) compiles `web/native_executor.dart`
via `dart compile js` (dart from the flutter341 closure, no
incremental cost) and (b) installs `fluffychat-web-imaging`'s
outputs into `$out`.
Two subtle details worth knowing before touching either derivation:
- **`make -C js`** instead of `cd js; make` — keeps the build-phase
pwd at the source root so `installPhase` doesn't have to know
about the cd. Robust against future reorders / `dontBuild`.
- **`web/native_executor.dart`** as a build-CWD-relative path,
*not* `$src/web/...` — `dart`'s `package_config.json` walk-up
needs to hit `buildFlutterApplication`'s pub-get output
(`.dart_tool/` in the build CWD). Walking up from a read-only
`$src/` store path finds no `.dart_tool/` and errors with
"Couldn't resolve the package 'matrix'."
Drop both derivations when nixpkgs's flutter builder grows worker
+ emcc support upstream.
Mount point is `chat.<swarm-domain>/` (the `gatewayHost` vhost);
upstream `--base-href "/"` is correct at sub-domain root, no override.