vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

Compare commits

base: vinzenz:1e67f5624993551b22a54efd5a43a92041ced6d7
Branches Tags
vinzenz:main
..
compare: vinzenz:cc2503d9c744168c166b63f4bf6713c155c67928
Branches Tags
vinzenz:main
Author SHA1 Message Date
atlas
cc2503d9c7 treefmt: a generated CLI doc is the generator's, not prettier's 
docs/tools/swarmctl-cli.md is rendered by `swarmctl markdown-docs`, and
nix/checks.nix's swarmctl-docs-fresh check diffs the committed copy
against fresh binary output. Enabling prettier on markdown rewrote its
list bullets and footer indentation, which no regeneration can settle:
formatting it fails the freshness check, not formatting it fails
treefmt. .prettierignore already carried hivectl-cli.md for exactly
this reason; swarmctl's doc was added later and the entry was not.

Restore the file to its generated bytes, list it alongside hivectl's,
and state the invariant where the next CLI doc gets added.
 2026-09-02 15:25:07 +02:00
atlas
36b67fcda3 docs/ci: the local-checks block named a command that does not exist 
`docs/scheduler/ci.md` told the reader to run `nix develop -c treefmt`.
treefmt is not on the devshell PATH — `nix develop -c which treefmt` exits
1 while `which cargo` resolves, so the check discriminates and the
documented command has never worked from that shell.

The same line enumerated the formatter set as "rustfmt + nixfmt + taplo",
which was already wrong before this branch: keep-sorted has been enabled
all along, and the commit before this adds prettier. Replaced the list
with a pointer to nix/treefmt.nix so it can't drift out of sync again.

Also drops "+ rustfmt" from the nix-flake-check row: rustfmt is one of
treefmt's formatters, not a separate thing that runs alongside it.
 2026-09-02 15:25:07 +02:00
atlas
39b95c2ede treefmt: apply prettier 
Pure `nix fmt` output from the commit before this one — no hand edits.
203 files: 52 md, 42 tsx, 32 js, 32 css, 21 ts, 13 html, 8 json, 3 mjs.

Reproduce with `nix develop -c nix fmt` on the parent commit; the result
should be byte-identical to this tree.

None of the 13 `.prettierignore` entries appears here — verified by
intersecting the changed-file list against the ignore file, with a
control proving the intersection finds a match when one exists.
 2026-09-02 15:25:07 +02:00
atlas
5d24bedd60 treefmt: format the web files with prettier 
`nix/treefmt.nix` gated nix, rust and toml and nothing else, so markdown,
css, js/ts/tsx, html and json had no formatting gate at all. Operator
asked for "the web stuff to be formatted as well" (#1997).

This commit is the one-line enable; the reformat it produces is the
commit after it, kept separate so this one stays reviewable.

Scope is treefmt-nix's `programs.prettier` default rather than a
hand-written `includes` list. The default is wider than the requested
md/css/js/ts/tsx/html/json — it also covers yaml, scss, vue, mdx, json5,
cjs/mjs/jsx — but measured against this tree that width costs exactly 3
extra files, all `frontend/packages/*/build.mjs`, which are web build
scripts. `.forgejo/workflows/{ci,coverage}.yml` are in the default set
and already conform, so they reformat to nothing today; from here on they
are gated too, which is what a repo formatter is for.

`.prettierignore` is honoured: prettier applies it even to the explicit
paths treefmt passes, so the `hivectl markdown-docs` generated file and
the 12 files awaiting manual list-item re-wrap stay untouched.
 2026-09-02 15:25:07 +02:00
205 changed files with 10075 additions and 6049 deletions
Download patch file Download diff file Expand all files Collapse all files

10
.prettierignore
View file

@ -1,7 +1,11 @@
# Auto-generated by `hivectl markdown-docs` — do not format. # Generated by `<tool> markdown-docs` — do not format. The matching
# The hivectl-docs CI check diffs this against fresh binary output, # *-docs-fresh check in nix/checks.nix diffs each against fresh binary
# so reformatting it would break that check. # output, so any reformatting fails that check permanently: the
# generator, not the formatter, owns these files' layout. Every CLI
# reference doc belongs here — adding one and forgetting this line is
# what made swarmctl's fail.
docs/tools/hivectl-cli.md docs/tools/hivectl-cli.md
docs/tools/swarmctl-cli.md
# Files with multi-line list-item continuations that prettier strips to col 0. # Files with multi-line list-item continuations that prettier strips to col 0.
# prettier's `proseWrap: "preserve"` prevents prose reflow but not list-item # prettier's `proseWrap: "preserve"` prevents prose reflow but not list-item

6
claude-plugins/plugins/base/skills/claude-subagents/SKILL.md
View file

@ -33,7 +33,7 @@ claude --name <memorable> --model <cheaper-than-you> --dangerously-skip-permissi
by name for follow-ups. by name for follow-ups.
- **`--model <cheaper-than-you>`** - think about which model the task - **`--model <cheaper-than-you>`** - think about which model the task
actually needs; don't spend a bigger model's tokens than your own on actually needs; don't spend a bigger model's tokens than your own on
mechanical work a cheaper one handles fine. Never use a *bigger* model mechanical work a cheaper one handles fine. Never use a _bigger_ model
than yourself for a sub-agent - if the task needs that much capability, than yourself for a sub-agent - if the task needs that much capability,
it's not the "mechanical batch" case this skill is for. it's not the "mechanical batch" case this skill is for.
- **`--dangerously-skip-permissions`** - required in headless (`-p`) - **`--dangerously-skip-permissions`** - required in headless (`-p`)
@ -68,7 +68,7 @@ work while the sub-agent grinds.
cases: "leave it as-is where genuinely unclear; do not guess." cases: "leave it as-is where genuinely unclear; do not guess."
- **Ask for a report file** - per-item results + anything skipped and - **Ask for a report file** - per-item results + anything skipped and
why, so you can verify without re-deriving. why, so you can verify without re-deriving.
- **Tune on ONE item first**, eyeball the result, fix the prompt, *then* - **Tune on ONE item first**, eyeball the result, fix the prompt, _then_
turn it loose on the full batch. A prompt bug replicated across 100 turn it loose on the full batch. A prompt bug replicated across 100
items is 100 cleanups. items is 100 cleanups.
@ -98,7 +98,7 @@ running at once instead of handing the whole thing to one.
loop. loop.
- **Mind the container's memory cap before picking N.** Your whole - **Mind the container's memory cap before picking N.** Your whole
container shares one `MemoryMax` (a few GB by default) with every container shares one `MemoryMax` (a few GB by default) with every
subagent you spawn *and* your own process. A `claude` process plus its subagent you spawn _and_ your own process. A `claude` process plus its
MCP servers can hold several hundred MB to ~1GB depending on the task; MCP servers can hold several hundred MB to ~1GB depending on the task;
spawning a dozen at once on a small container doesn't just slow spawning a dozen at once on a small container doesn't just slow
things down, it can OOM the whole container — taking your own things down, it can OOM the whole container — taking your own

6
claude-plugins/plugins/base/skills/forge-issue-filing/SKILL.md
View file

@ -15,7 +15,7 @@ steps in the right order.
## 1. Check for a duplicate before filing ## 1. Check for a duplicate before filing
`hive-forge list` has a server-side full-text `--search` flag that covers `hive-forge list` has a server-side full-text `--search` flag that covers
title *and* body - use it before writing anything: title _and_ body - use it before writing anything:
```sh ```sh
hive-forge list --search "keyword or two" --state all hive-forge list --search "keyword or two" --state all
@ -43,8 +43,8 @@ Prefer naming the broken behavior or missing capability over a specific
error string or one-off symptom - "hive-forge list has no way to filter by error string or one-off symptom - "hive-forge list has no way to filter by
milestone" beats "list command confusing" or "can't find issues for milestone" beats "list command confusing" or "can't find issues for
milestone 11 easily". If you're not sure which framing is more durable, milestone 11 easily". If you're not sure which framing is more durable,
lean toward whatever a colleague hitting the *same underlying gap* through lean toward whatever a colleague hitting the _same underlying gap_ through
a *different* symptom would still recognize. a _different_ symptom would still recognize.
## 3. Body: what's wrong, not how to fix it ## 3. Body: what's wrong, not how to fix it

11
claude-plugins/plugins/base/skills/forge-triage/SKILL.md
View file

@ -20,7 +20,7 @@ dimension directly.
hardcoded, so check `repo-labels` first if you don't already know the hardcoded, so check `repo-labels` first if you don't already know the
scopes in use. scopes in use.
- **PRs with no formally requested reviewer** - `hive-forge lint - **PRs with no formally requested reviewer** - `hive-forge lint
no-reviewer` flags PRs with zero requested reviewers at all; add no-reviewer` flags PRs with zero requested reviewers at all; add
`--reviewer <name>` to instead flag PRs where that specific person `--reviewer <name>` to instead flag PRs where that specific person
isn't among the requested reviewers. This checks the forge's actual isn't among the requested reviewers. This checks the forge's actual
reviewer-request state, not a text `@name` mention anywhere in the reviewer-request state, not a text `@name` mention anywhere in the
@ -42,12 +42,13 @@ dimension directly.
Run the checks relevant to what you're verifying rather than assuming Run the checks relevant to what you're verifying rather than assuming
one covers everything - "is triage caught up" usually means unassigned one covers everything - "is triage caught up" usually means unassigned
+ unlabeled at minimum, with no-reviewer added if reviews matter for
your workflow. Each verb supports `--json` if you want to fold the - unlabeled at minimum, with no-reviewer added if reviews matter for
results into something else instead of reading the human-readable output. your workflow. Each verb supports `--json` if you want to fold the
results into something else instead of reading the human-readable output.
This is reactive/on-demand coverage checking, not a scheduled rollup or This is reactive/on-demand coverage checking, not a scheduled rollup or
historical trend - if you need to track compliance *over time* (was historical trend - if you need to track compliance _over time_ (was
triage worse last week than this week), that's a different, bigger triage worse last week than this week), that's a different, bigger
ask than running these checks - don't build that unprompted just ask than running these checks - don't build that unprompted just
because the checks exist. because the checks exist.

6
claude-plugins/plugins/base/skills/headless-screenshot/SKILL.md
View file

@ -6,7 +6,7 @@ description: Take a real screenshot of an HTML/CSS page from inside a container
# Headless Screenshot # Headless Screenshot
"No browser in this container" usually means no browser is "No browser in this container" usually means no browser is
*pre-installed*, not that one is unreachable. `nixpkgs#chromium` is a _pre-installed_, not that one is unreachable. `nixpkgs#chromium` is a
`nix shell` away, and a single headless invocation renders a page and `nix shell` away, and a single headless invocation renders a page and
writes a PNG without a running server, a display, or any scripted writes a PNG without a running server, a display, or any scripted
interaction library. interaction library.
@ -65,7 +65,7 @@ peer agent's GUI-testing capability, try rendering it yourself first:
This is a static render, not a browser automation tool. Verifying an This is a static render, not a browser automation tool. Verifying an
interaction (a click, a hover state, a form submission) needs interaction (a click, a hover state, a form submission) needs
puppeteer/playwright or similar - reach for those only when the puppeteer/playwright or similar - reach for those only when the
question is genuinely about *behavior*, not appearance. A missing question is genuinely about _behavior_, not appearance. A missing
emoji/glyph rendering as a box in the screenshot is usually a font emoji/glyph rendering as a box in the screenshot is usually a font
availability artifact of the headless environment, not a real bug - availability artifact of the headless environment, not a real bug -
don't mistake one for the other when reporting results. don't mistake one for the other when reporting results.
@ -80,7 +80,7 @@ judge icon/glyph layout, not just to note "there'd be an icon here").
- **Tofu boxes instead of emoji.** Plain `nixpkgs#chromium` has no - **Tofu boxes instead of emoji.** Plain `nixpkgs#chromium` has no
color-emoji font, so any 🖥/📡/🔨-style glyph renders as an empty color-emoji font, so any 🖥/📡/🔨-style glyph renders as an empty
box. Add `nixpkgs#noto-fonts-color-emoji` to the `nix shell` - but a box. Add `nixpkgs#noto-fonts-color-emoji` to the `nix shell` - but a
`FONTCONFIG_FILE` that lists *only* the emoji dir throws away the `FONTCONFIG_FILE` that lists _only_ the emoji dir throws away the
base fonts and blanks all normal text instead, which is its own base fonts and blanks all normal text instead, which is its own
false alarm. Merge, don't override: false alarm. Merge, don't override:
``` ```

2
claude-plugins/plugins/base/skills/pr-review/SKILL.md
View file

@ -10,7 +10,7 @@ work in your own style. That framing should hold throughout.
## Before you start ## Before you start
1. **Read the linked issue/task** - understand *why* the change exists, 1. **Read the linked issue/task** - understand _why_ the change exists,
not just what it does. not just what it does.
2. **Check CI status first.** A failing build or test suite is an 2. **Check CI status first.** A failing build or test suite is an
automatic block - don't write a detailed review of code that doesn't automatic block - don't write a detailed review of code that doesn't

24
docs/agent-lifecycle/agent-hierarchy.md
View file

@ -28,7 +28,7 @@ requester-as-parent edge written at its `init_config` approval (so
agent, including the bootstrap container (`ruth`) — it's just another agent, including the bootstrap container (`ruth`) — it's just another
root. The manager is reparentable like any other agent; there's no root. The manager is reparentable like any other agent; there's no
"structurally root" carve-out. Its privileges live on its MCP socket, "structurally root" carve-out. Its privileges live on its MCP socket,
not its tree position (see *Manager special-casing today* below). not its tree position (see _Manager special-casing today_ below).
### Reparenting ### Reparenting
@ -88,15 +88,15 @@ umount-old / mount-new / restart-cascade step.
## Planned topology semantics (once ancestor-based enforcement lands) ## Planned topology semantics (once ancestor-based enforcement lands)
| operation | who can do it | | operation | who can do it |
| ----------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | | ----------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| `kill` / `start` / `restart` / `update` (any descendant) | any ancestor | | `kill` / `start` / `restart` / `update` (any descendant) | any ancestor |
| `request_init_config` (spawn a new child) | any agent, child added under self | | `request_init_config` (spawn a new child) | any agent, child added under self |
| config change via forge PR (any descendant's config) | any ancestor | | config change via forge PR (any descendant's config) | any ancestor |
| `get_logs` (any descendant) | any ancestor | | `get_logs` (any descendant) | any ancestor |
| moderate reminders (cancel any open thread of a descendant) | any ancestor | | moderate reminders (cancel any open thread of a descendant) | any ancestor |
| `send` / `recv` routing | parent ↔ same-parent siblings ↔ self ↔ descendants; explicit allow-list for anyone else | | `send` / `recv` routing | parent ↔ same-parent siblings ↔ self ↔ descendants; explicit allow-list for anyone else |
| `request_update_meta_inputs` (bump meta lock) | root agents only (today: just `manager`) | | `request_update_meta_inputs` (bump meta lock) | root agents only (today: just `manager`) |
"Ancestor" walks `ContainerView.parent` chains; cycles are guarded by a "Ancestor" walks `ContainerView.parent` chains; cycles are guarded by a
visited-set at dispatch time (a malformed `topology.json` can't lock visited-set at dispatch time (a malformed `topology.json` can't lock
@ -138,11 +138,11 @@ other agents don't:
capability. capability.
- **Prompt/tools** — the system prompt uses `<!-- role:agent -->` / - **Prompt/tools** — the system prompt uses `<!-- role:agent -->` /
`<!-- role:manager -->` marker blocks, and a `Flavor::{Agent, `<!-- role:manager -->` marker blocks, and a `Flavor::{Agent,
Manager}` switch picks the MCP tool allow-list claude sees. Both are Manager}` switch picks the MCP tool allow-list claude sees. Both are
already parametrised on a single flavour value, so the planned already parametrised on a single flavour value, so the planned
per-capability-group version (`cap:<group>` prompt blocks + a per-capability-group version (`cap:<group>` prompt blocks + a
matching tool allow-list) is additive rather than a rewrite. matching tool allow-list) is additive rather than a rewrite.
- **State dirs** — *not* special-cased: `HYPERHIVE_STATE_DIR` is - **State dirs** — _not_ special-cased: `HYPERHIVE_STATE_DIR` is
injected uniformly via `systemd.globalEnvironment` for every injected uniformly via `systemd.globalEnvironment` for every
container including the manager, so all token/state paths resolve container including the manager, so all token/state paths resolve
through it the same way everywhere. through it the same way everywhere.

2
docs/getting-started/setup.md
View file

@ -114,7 +114,7 @@ Nothing to run — it is served on the swarm apex
decide whether you can actually open it: decide whether you can actually open it:
- **You are in `admins`** (step 3). The gateway asks authelia whether - **You are in `admins`** (step 3). The gateway asks authelia whether
you have a session; the rule that makes it mean *operator* wants the you have a session; the rule that makes it mean _operator_ wants the
group. Without it you log in and still get bounced. group. Without it you log in and still get bounced.
- **The name resolves to this host.** It is published to the hive's own - **The name resolves to this host.** It is published to the hive's own
resolver and to `/etc/hosts` when `gateway.localHostsEntry` is on; from resolver and to `/etc/hosts` when `gateway.localHostsEntry` is on; from

18
docs/integrations/forge.md
View file

@ -112,7 +112,7 @@ On a **successful** broker delivery, `forge_notify` marks the thread
read on forge straight away (`PATCH /notifications/threads/{id}`). The read on forge straight away (`PATCH /notifications/threads/{id}`). The
broker inbox is the durable work queue now — each delivered wake is a broker inbox is the durable work queue now — each delivered wake is a
sqlite row with its own ack lifecycle — so the forge unread flag no sqlite row with its own ack lifecycle — so the forge unread flag no
longer needs to track whether the agent has *processed* a longer needs to track whether the agent has _processed_ a
notification. Clearing it on delivery keeps forge's unread set **tiny notification. Clearing it on delivery keeps forge's unread set **tiny
by construction**: at rest it holds only threads that failed to by construction**: at rest it holds only threads that failed to
deliver plus whatever arrived since the last 30s poll. deliver plus whatever arrived since the last 30s poll.
@ -132,19 +132,19 @@ own read-state is the only durable record this design needs.
**Read-before-comment guard doesn't block a fresh wake.** hive-forge's **Read-before-comment guard doesn't block a fresh wake.** hive-forge's
read-before-comment guard (which keys off forge unread-state) does not read-before-comment guard (which keys off forge unread-state) does not
force the agent to view a thread before commenting on it: the broker force the agent to view a thread before commenting on it: the broker
wake already carries the notification body, so *delivery is the read*. wake already carries the notification body, so _delivery is the read_.
An agent that wants the full thread still runs `hive-forge comments` / An agent that wants the full thread still runs `hive-forge comments` /
`view`. `view`.
**In-process dedupe (tiny, ephemeral).** A single-process map (thread **In-process dedupe (tiny, ephemeral).** A single-process map (thread
id → last-delivered `updated_at`) guards the narrow window where a id → last-delivered `updated_at`) guards the narrow window where a
mark-read call *transiently fails* and the thread reappears unread in mark-read call _transiently fails_ and the thread reappears unread in
the next poll before its `updated_at` bumps — so a flaky PATCH doesn't the next poll before its `updated_at` bumps — so a flaky PATCH doesn't
re-fire the wake. It is **not persisted** and resets on restart (forge re-fire the wake. It is **not persisted** and resets on restart (forge
read-state covers the durable case). Each poll prunes it to the ids in read-state covers the durable case). Each poll prunes it to the ids in
the single `limit=UNREAD_FETCH_LIMIT` (50) fetch page, so it can never the single `limit=UNREAD_FETCH_LIMIT` (50) fetch page, so it can never
exceed that many entries (a debug assertion pins the invariant; the exceed that many entries (a debug assertion pins the invariant; the
fetch limit and the bound are the same constant). A failed *delivery* fetch limit and the bound are the same constant). A failed _delivery_
is left unread and out of the map, so it resurfaces next tick. is left unread and out of the map, so it resurfaces next tick.
Self-echo notifications (the agent's own writes, see below) are marked Self-echo notifications (the agent's own writes, see below) are marked
@ -225,13 +225,13 @@ lookahead.
Five shapes, distinguished by the notification's classification: Five shapes, distinguished by the notification's classification:
| Trigger | Wrapper | | Trigger | Wrapper |
| ----------------------------------- | --------------------------------------------------------------------------------- | | ----------------------------------- | ------------------------------------------------------------------------------ |
| Comment on issue / PR | `[comment on PR #N owner/repo] title\nurl: ...\n\nauthor: body\nassignee: ...` | | Comment on issue / PR | `[comment on PR #N owner/repo] title\nurl: ...\n\nauthor: body\nassignee: ...` |
| Review submission | `[PR approved #N owner/repo] title\nurl: ...\n\nauthor: body\nassignee: ...` | | Review submission | `[PR approved #N owner/repo] title\nurl: ...\n\nauthor: body\nassignee: ...` |
| New issue / PR | `[new PR #N owner/repo] title\nurl: ...\nassignee: ...` | | New issue / PR | `[new PR #N owner/repo] title\nurl: ...\nassignee: ...` |
| Later activity (open, not creation) | `[activity on PR #N owner/repo] title\nurl: ...\nassignee: ...` | | Later activity (open, not creation) | `[activity on PR #N owner/repo] title\nurl: ...\nassignee: ...` |
| State change | `[PR merged #N owner/repo] title\nurl: ...\nassignee: ...` | | State change | `[PR merged #N owner/repo] title\nurl: ...\nassignee: ...` |
The issue/PR's own description is deliberately never embedded here (only a The issue/PR's own description is deliberately never embedded here (only a
comment/review body is — that's the actual new content a notification comment/review body is — that's the actual new content a notification

8
docs/integrations/github.md
View file

@ -109,7 +109,7 @@ Forgejo's notifications API is modelled on GitHub's, so one tolerant
parse serves both: `id`, `repository.full_name`, parse serves both: `id`, `repository.full_name`,
`subject {title,url,latest_comment_url}` and `updated_at` line up field `subject {title,url,latest_comment_url}` and `updated_at` line up field
for field. The two real differences are absorbed by lenient for field. The two real differences are absorbed by lenient
deserializers — GitHub sends the thread id as a *string* where Forgejo deserializers — GitHub sends the thread id as a _string_ where Forgejo
sends a number, and says `PullRequest` where Forgejo says `Pull`. Todo sends a number, and says `PullRequest` where Forgejo says `Pull`. Todo
keys are prefixed `gh:<id>` so a github thread id cannot collide with a keys are prefixed `gh:<id>` so a github thread id cannot collide with a
forge one. forge one.
@ -118,16 +118,16 @@ Two host differences worth knowing before touching this code:
- **Auth scheme, not just value.** Forgejo takes - **Auth scheme, not just value.** Forgejo takes
`Authorization: token <t>`; GitHub wants `Bearer <t>` plus `Accept: `Authorization: token <t>`; GitHub wants `Bearer <t>` plus `Accept:
application/vnd.github+json`, `X-GitHub-Api-Version` and a application/vnd.github+json`, `X-GitHub-Api-Version` and a
`User-Agent`. Sending Forgejo's form to GitHub does not error — it `User-Agent`. Sending Forgejo's form to GitHub does not error — it
authenticates as *nobody* and silently drops to the unauthenticated authenticates as _nobody_ and silently drops to the unauthenticated
rate limit. The cheap way to tell the two apart is the rate-limit rate limit. The cheap way to tell the two apart is the rate-limit
header: `x-ratelimit-remaining` near 5000 is an authenticated user, header: `x-ratelimit-remaining` near 5000 is an authenticated user,
near 60 is anonymous. near 60 is anonymous.
- **GitHub sets the cadence.** It returns `X-Poll-Interval` (60s in - **GitHub sets the cadence.** It returns `X-Poll-Interval` (60s in
practice, slower than our own tick) and rate-limits callers who ignore practice, slower than our own tick) and rate-limits callers who ignore
it, so the loop re-arms to the server's interval whenever that is it, so the loop re-arms to the server's interval whenever that is
*slower* than ours. A hint faster than our own tick is not a reason to _slower_ than ours. A hint faster than our own tick is not a reason to
poll harder. poll harder.
⚠️ **This needs the `notifications` scope on the PAT.** A token minted ⚠️ **This needs the `notifications` scope on the PAT.** A token minted

2
docs/integrations/knowledge.md
View file

@ -42,7 +42,7 @@ hive-c0re maintains the local clone at
`internal/knowledge` (see `docs/swarm/README.md` § Swarm-wide forge `internal/knowledge` (see `docs/swarm/README.md` § Swarm-wide forge
webhooks). On any push to main, including merge commits, it sends an webhooks). On any push to main, including merge commits, it sends an
event to every hive over the swarm queue and each hive runs `git event to every hive over the swarm queue and each hive runs `git
pull`, so agents see the new content on their next turn. pull`, so agents see the new content on their next turn.
A hive that is offline when the event is sent does not get it on A hive that is offline when the event is sent does not get it on
reconnect — the periodic pull below is what closes that gap. So one reconnect — the periodic pull below is what closes that gap. So one

34
docs/networking/network.md
View file

@ -43,13 +43,13 @@ untouched by any of it.
└─────────┘ └─────────┘ └─────────┘ └─────────┘ └─────────┘ └─────────┘
``` ```
| container | netns | IPv4 | listens / reached via | | container | netns | IPv4 | listens / reached via |
| -------------- | ----------------------- | -------------------- | -------------------------------------------------------------------------------------------- | | -------------- | ----------------------- | -------------- | ----------------------------------------------------------------------------------------------- |
| `hive-gateway` | host (shared) | host addresses | nginx `:80`/`:443` (every vhost); dnsmasq `bridgeIp:53` + DHCP `:67` on the bridge | | `hive-gateway` | host (shared) | host addresses | nginx `:80`/`:443` (every vhost); dnsmasq `bridgeIp:53` + DHCP `:67` on the bridge |
| `hive-forge` | host (shared) | host addresses | forgejo `:3000` http, `:2222` git-ssh; fronted by the `forge.<swarm-domain>` vhost | | `hive-forge` | host (shared) | host addresses | forgejo `:3000` http, `:2222` git-ssh; fronted by the `forge.<swarm-domain>` vhost |
| `hive-matrix` | host (shared) | host addresses | tuwunel `:8008` (+ optional federation port); fronted by the matrix vhost | | `hive-matrix` | host (shared) | host addresses | tuwunel `:8008` (+ optional federation port); fronted by the matrix vhost |
| `hive-ci` | private, veth on bridge | DHCP pool | outbound only (runner → forge); no inbound surface | | `hive-ci` | private, veth on bridge | DHCP pool | outbound only (runner → forge); no inbound surface |
| `h-<agent>` | private, veth on bridge | DHCP pool | web UI via UDS `/run/hive-agent/<name>` → nginx sub-path; in-container UI port hashed 8100–8999 | | `h-<agent>` | private, veth on bridge | DHCP pool | web UI via UDS `/run/hive-agent/<name>` → nginx sub-path; in-container UI port hashed 8100–8999 |
The flows, end to end: The flows, end to end:
@ -57,7 +57,7 @@ The flows, end to end:
host firewall (udp 67 hole) → dnsmasq pool → lease + router option. host firewall (udp 67 hole) → dnsmasq pool → lease + router option.
- **DNS** — agents and the service containers query `bridgeIp:53`; hive - **DNS** — agents and the service containers query `bridgeIp:53`; hive
zones are answered authoritatively with the bridge IP, everything else zones are answered authoritatively with the bridge IP, everything else
forwards to the host's resolvers (see *Resolver behaviour* below). Each forwards to the host's resolvers (see _Resolver behaviour_ below). Each
container points its own `resolv.conf` there, and one that instead container points its own `resolv.conf` there, and one that instead
inherits the host's resolves no swarm name at all — those records exist inherits the host's resolves no swarm name at all — those records exist
only on the bridge. only on the bridge.
@ -169,7 +169,7 @@ agent containers.
The **host** firewall is the only firewall. The shared-netns infra The **host** firewall is the only firewall. The shared-netns infra
containers (gateway, forge, matrix) set containers (gateway, forge, matrix) set
`networking.firewall.enable = false`: a NixOS firewall inside a `networking.firewall.enable = false`: a NixOS firewall inside a
shared-netns container runs against the *host* ruleset — at container shared-netns container runs against the _host_ ruleset — at container
boot its `firewall-start` flushes the `nixos-fw` chains, rebuilds them boot its `firewall-start` flushes the `nixos-fw` chains, rebuilds them
from the container's (empty) port list, and deletes the host's from the container's (empty) port list, and deletes the host's
`nixos-nat-*` chains without recreating them, silently wiping the `nixos-nat-*` chains without recreating them, silently wiping the
@ -183,7 +183,7 @@ By default agents can only reach the host on 80/443 (+53 DNS), so a
host-side service on another port — e.g. a dev OTLP collector you want host-side service on another port — e.g. a dev OTLP collector you want
agents to reach directly — is unreachable. (hyperhive's own telemetry agents to reach directly — is unreachable. (hyperhive's own telemetry
needs none of this: `otel.enable` opens its collector's port itself, and needs none of this: `otel.enable` opens its collector's port itself, and
`otel.endpoint` is the *upstream*, which no agent ever dials. See `otel.endpoint` is the _upstream_, which no agent ever dials. See
`docs/scheduler/observability.md`.) `docs/scheduler/observability.md`.)
`services.hyperhive.network.exposeHostPorts = [ 4318 ];` opens each `services.hyperhive.network.exposeHostPorts = [ 4318 ];` opens each
@ -206,12 +206,12 @@ Each agent container runs in a private network namespace with a dedicated
veth pair attached to the bridge. The following table summarises what veth pair attached to the bridge. The following table summarises what
the nix side sets up unconditionally: the nix side sets up unconditionally:
| effect | mechanism | | effect | mechanism |
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| IP forwarding | `boot.kernel.sysctl."net.ipv4.ip_forward" = 1` | | IP forwarding | `boot.kernel.sysctl."net.ipv4.ip_forward" = 1` |
| Internet NAT | `networking.nat { enable = true; internalInterfaces = [ bridgeName ]; }` — MASQUERADE on packets leaving via any external NIC | | Internet NAT | `networking.nat { enable = true; internalInterfaces = [ bridgeName ]; }` — MASQUERADE on packets leaving via any external NIC |
| Loopback DROP | `networking.firewall.extraInputRules` — drops bridge-subnet → `127.0.0.0/8` traffic; defence-in-depth against routing table leaks | | Loopback DROP | `networking.firewall.extraInputRules` — drops bridge-subnet → `127.0.0.0/8` traffic; defence-in-depth against routing table leaks |
| Gateway access | `networking.firewall.interfaces.<bridge>.allowedTCPPorts = [ 80 443 ]` — lets isolated agents (private netns, veth on bridge) reach nginx on the host | | Gateway access | `networking.firewall.interfaces.<bridge>.allowedTCPPorts = [ 80 443 ]` — lets isolated agents (private netns, veth on bridge) reach nginx on the host |
| c0re signal | `HIVE_NETWORK_BRIDGE`, `HIVE_NETWORK_SUBNET` in `systemd.services.hive-c0re.environment` — both **required**; `hive-c0re` refuses to start without them | | c0re signal | `HIVE_NETWORK_BRIDGE`, `HIVE_NETWORK_SUBNET` in `systemd.services.hive-c0re.environment` — both **required**; `hive-c0re` refuses to start without them |
`HIVE_NETWORK_SUBNET` is the host-side bridge IP + prefix (e.g. `HIVE_NETWORK_SUBNET` is the host-side bridge IP + prefix (e.g.
@ -262,7 +262,7 @@ wiring is runtime:
**Why isolation is safe**: hive-c0re's control-plane sockets are unix **Why isolation is safe**: hive-c0re's control-plane sockets are unix
domain sockets bind-mounted into containers, not network listeners — see domain sockets bind-mounted into containers, not network listeners — see
the *Control plane (no network)* bullet under [Network the _Control plane (no network)_ bullet under [Network
map](#network-map) above. `PRIVATE_NETWORK=1` has no effect on a path map](#network-map) above. `PRIVATE_NETWORK=1` has no effect on a path
that never touches the network stack. that never touches the network stack.

24
docs/networking/snapshot-store.md
View file

@ -43,8 +43,8 @@ Note that the mesh is gated on `swarm.wireguard.enable`, **not** on
## Pointing a hive at it ## Pointing a hive at it
The block above configures the host that *receives*. Every hive that The block above configures the host that _receives_. Every hive that
*pushes* separately needs to be told where the store is: _pushes_ separately needs to be told where the store is:
```nix ```nix
services.hyperhive.swarm.snapshotStore = { services.hyperhive.swarm.snapshotStore = {
@ -65,9 +65,9 @@ inconsistency:
Note the option lives under `swarm.*` while the receiving host's lives Note the option lives under `swarm.*` while the receiving host's lives
under `services.hyperhive.snapshotStore`. That is the distinction the under `services.hyperhive.snapshotStore`. That is the distinction the
two namespaces carry throughout: `swarm.*` describes *the swarm* as seen two namespaces carry throughout: `swarm.*` describes _the swarm_ as seen
from this host, and a bare `services.hyperhive.<service>` describes *a from this host, and a bare `services.hyperhive.<service>` describes _a
role this host performs*. A store host sets both --- one to run the role this host performs_. A store host sets both --- one to run the
receiver, one only if it also runs a hive that pushes. receiver, one only if it also runs a hive that pushes.
With it set, `hivectl agent <name> subvol snapshot push <label> With it set, `hivectl agent <name> subvol snapshot push <label>
@ -88,7 +88,7 @@ that address provably came from the holder of that private key. A
packet that reaches the receiver has therefore already been packet that reaches the receiver has therefore already been
authenticated by the kernel. authenticated by the kernel.
Layering TLS client certs on top would authenticate *the same fact* a Layering TLS client certs on top would authenticate _the same fact_ a
second time, and add a credential with an expiry --- a migration that second time, and add a credential with an expiry --- a migration that
fails because a renewal quietly didn't happen, discovered on the day fails because a renewal quietly didn't happen, discovered on the day
you need to move an agent. you need to move an agent.
@ -98,14 +98,14 @@ you need to move an agent.
The destination is keyed by **agent**. The destination is keyed by **agent**.
This is not cosmetic. After a migration, an agent's next incremental This is not cosmetic. After a migration, an agent's next incremental
send arrives from a *different* hive than the previous one. Keying by send arrives from a _different_ hive than the previous one. Keying by
hive would split that agent's snapshot chain across two directories, hive would split that agent's snapshot chain across two directories,
and `btrfs send -p` would fail to find its parent --- breaking exactly and `btrfs send -p` would fail to find its parent --- breaking exactly
the case the store exists to serve. the case the store exists to serve.
## What the sender can and cannot choose ## What the sender can and cannot choose
A `btrfs send` stream carries no notion of *which agent* it belongs to, A `btrfs send` stream carries no notion of _which agent_ it belongs to,
and the subvolume name inside it is chosen by the sender. So the and the subvolume name inside it is chosen by the sender. So the
protocol is one `agent <name>` header line, then the raw stream. protocol is one `agent <name>` header line, then the raw stream.
@ -129,7 +129,7 @@ address being set are assertions, not documentation --- bound to
state. state.
Binding is not sufficient on its own. NixOS's firewall is default-deny Binding is not sufficient on its own. NixOS's firewall is default-deny
and filters in netfilter, *before* a packet reaches a bound socket, so and filters in netfilter, _before_ a packet reaches a bound socket, so
the port is opened explicitly --- and scoped to the mesh interface: the port is opened explicitly --- and scoped to the mesh interface:
```nix ```nix
@ -176,8 +176,8 @@ and means it should get the treatment a backup host gets --- restricted
access, and a decision (rather than an omission) on encryption at rest. access, and a decision (rather than an omission) on encryption at rest.
The trap is the label: this box holds backup-grade data while not being The trap is the label: this box holds backup-grade data while not being
called a backup, so it can end up with backup-grade *exposure* and called a backup, so it can end up with backup-grade _exposure_ and
non-backup-grade *controls*. Nobody puts a migration staging area on non-backup-grade _controls_. Nobody puts a migration staging area on
the access-review list. the access-review list.
### What a snapshot contains ### What a snapshot contains
@ -202,7 +202,7 @@ gap is tracked separately.
### Retention ### Retention
Retention lives on the *sending* side (last-N by count, swept Retention lives on the _sending_ side (last-N by count, swept
periodically), not here. Count rather than age is deliberate: a count periodically), not here. Count rather than age is deliberate: a count
is bounded by construction, whereas an age policy silently scales disk is bounded by construction, whereas an age policy silently scales disk
usage with how hot a hive runs. usage with how hot a hive runs.

10
docs/process/gotchas.md
View file

@ -78,8 +78,8 @@ existing containers can't be started.
### `RestrictAddressFamilies` fails as "Address family not supported by protocol" ### `RestrictAddressFamilies` fails as "Address family not supported by protocol"
A unit whose `RestrictAddressFamilies` omits a family gets `EAFNOSUPPORT` A unit whose `RestrictAddressFamilies` omits a family gets `EAFNOSUPPORT`
(errno 97) back from `socket()`. Clients surface that as *"tcp open error: (errno 97) back from `socket()`. Clients surface that as _"tcp open error:
Address family not supported by protocol"* — the message names the Address family not supported by protocol"_ — the message names the
**protocol** and never the **sandbox**, so it reads like a dead network, a **protocol** and never the **sandbox**, so it reads like a dead network, a
missing route, or an IPv6 problem. missing route, or an IPv6 problem.
@ -98,7 +98,7 @@ re-checks it when the program changes.** A unit that only served a unix
socket when it was written is correct at `[ "AF_UNIX" ]` and silently wrong socket when it was written is correct at `[ "AF_UNIX" ]` and silently wrong
the day someone adds an HTTP client. Check the unit in the same commit as the day someone adds an HTTP client. Check the unit in the same commit as
the client — and when narrowing it, prefer a test that derives the required the client — and when narrowing it, prefer a test that derives the required
families from the code (which fails on the *next* client too) over one that families from the code (which fails on the _next_ client too) over one that
asserts today's list. asserts today's list.
### `register_agent` is idempotent ### `register_agent` is idempotent
@ -120,7 +120,7 @@ operator's host-level `allowUnfree` does **not** propagate in.
Operators don't need to set anything on their side. Operators don't need to set anything on their side.
That same isolation is why an agent can't pick a claude out of a That same isolation is why an agent can't pick a claude out of a
*different* nixpkgs by itself: a container only ever sees the one _different_ nixpkgs by itself: a container only ever sees the one
nixpkgs the meta flake injects, so an `agent.nix` naming the host's nixpkgs the meta flake injects, so an `agent.nix` naming the host's
`nixpkgs-unstable` has nothing to name. A release channel can trail `nixpkgs-unstable` has nothing to name. A release channel can trail
unstable by weeks on this package, which is what unstable by weeks on this package, which is what
@ -140,7 +140,7 @@ an input only because a docs tree has no runtime dependencies.
The `storePath` trap is worth spelling out, because it is not confined The `storePath` trap is worth spelling out, because it is not confined
to options the operator writes: **any** option of type `package` fed a to options the operator writes: **any** option of type `package` fed a
store-path *string* coerces through `lib.toDerivation`, i.e. store-path _string_ coerces through `lib.toDerivation`, i.e.
`builtins.storePath`. `environment.systemPackages` and `builtins.storePath`. `environment.systemPackages` and
`systemd.services.<name>.path` both do it (the latter takes plain `systemd.services.<name>.path` both do it (the latter takes plain
strings like `/run/wrappers` happily, but anything under strings like `/run/wrappers` happily, but anything under

6
docs/process/pr-review-gate.md
View file

@ -8,7 +8,7 @@ human review that already happened.
Whether a PR can merge, and what counts toward "can", is configured Whether a PR can merge, and what counts toward "can", is configured
per repo in its branch-protection settings — not a fact true of every per repo in its branch-protection settings — not a fact true of every
hive or every repo. The pieces a repo *can* require: hive or every repo. The pieces a repo _can_ require:
- **CI is green** — the repo's required status checks pass on the - **CI is green** — the repo's required status checks pass on the
PR's current head commit, if the repo requires any. PR's current head commit, if the repo requires any.
@ -40,8 +40,8 @@ independently.
Auto-merge isn't "no human ever looked at this." Whoever arms it has Auto-merge isn't "no human ever looked at this." Whoever arms it has
already judged the PR sound at a coarse level — the signal it sends is already judged the PR sound at a coarse level — the signal it sends is
roughly *"apart from maybe minor tweaks a reviewer can still catch, roughly _"apart from maybe minor tweaks a reviewer can still catch,
I think this is fine."* That's the human-in-the-loop step, and it I think this is fine."_ That's the human-in-the-loop step, and it
already happened. No large changes are expected to surface after already happened. No large changes are expected to surface after
that point — a reviewer's job past that point is to flag it if one that point — a reviewer's job past that point is to flag it if one
does, not to assume none ever will. does, not to assume none ever will.

18
docs/scheduler/ci.md
View file

@ -18,11 +18,11 @@ Two things worth knowing before you rely on it:
already has forge access; if you take outside/fork contributions, already has forge access; if you take outside/fork contributions,
gate CI behind Forgejo's fork-PR-approval setting or restrict the gate CI behind Forgejo's fork-PR-approval setting or restrict the
workflow to push-only triggers (forks can't push to your branches). workflow to push-only triggers (forks can't push to your branches).
See *Security* below for the full threat model. See _Security_ below for the full threat model.
- **Watch your disk.** CI builds through the host's nix store with no - **Watch your disk.** CI builds through the host's nix store with no
automatic garbage collection of its own — a busy CI day can fill automatic garbage collection of its own — a busy CI day can fill
`/nix/store` until every job fails with `ENOSPC`. Add the daily + `/nix/store` until every job fails with `ENOSPC`. Add the daily +
disk-pressure GC config from *Host store maintenance* below to your disk-pressure GC config from _Host store maintenance_ below to your
host's NixOS config (not optional if you plan to actually use this). host's NixOS config (not optional if you plan to actually use this).
Everything below this point is implementation detail: exact Everything below this point is implementation detail: exact
@ -36,10 +36,10 @@ defined in [`.forgejo/workflows/ci.yml`](../../.forgejo/workflows/ci.yml). All
three are required checks (forge branch protection) — a hit on any of them three are required checks (forge branch protection) — a hit on any of them
blocks merge. blocks merge.
| Job | What it runs | | Job | What it runs |
| --- | --- | | ---------------------- | --------------------------------------------------------------------------------- |
| **nix flake check** | treefmt + rustfmt formatting, `cargo clippy -D warnings`, `cargo test`, module evaluation | | **nix flake check** | treefmt formatting, `cargo clippy -D warnings`, `cargo test`, module evaluation |
| **tracker-tag lint** | flags `#NNN` issue tags in source and comments (`scripts/check-issue-refs.sh`) | | **tracker-tag lint** | flags `#NNN` issue tags in source and comments (`scripts/check-issue-refs.sh`) |
| **comment-block lint** | flags contiguous comment blocks over 30 lines (`scripts/check-comment-blocks.sh`) | | **comment-block lint** | flags contiguous comment blocks over 30 lines (`scripts/check-comment-blocks.sh`) |
`hive-forge ci-rerun --pr N` dispatches a `workflow_dispatch` retrigger `hive-forge ci-rerun --pr N` dispatches a `workflow_dispatch` retrigger
@ -69,7 +69,7 @@ wastes a remote-builder slot. Use the devshell equivalents instead:
```sh ```sh
nix develop -c cargo clippy --all-targets -- -D warnings nix develop -c cargo clippy --all-targets -- -D warnings
nix develop -c cargo test nix develop -c cargo test
nix develop -c treefmt # same as nix fmt; treefmt covers rustfmt + nixfmt + taplo nix fmt # authoritative; the formatter set is nix/treefmt.nix
sh scripts/check-issue-refs.sh # tracker-tag lint sh scripts/check-issue-refs.sh # tracker-tag lint
sh scripts/check-comment-blocks.sh # comment-block lint sh scripts/check-comment-blocks.sh # comment-block lint
``` ```
@ -90,7 +90,7 @@ slow); run those manually before pushing Rust changes.
The internal forge is always present (mandatory), so the runner always has a The internal forge is always present (mandatory), so the runner always has a
hive-forge instance to register against — nothing extra to enable beyond hive-forge instance to register against — nothing extra to enable beyond
`services.hyperhive.deploy.forgejo.ci.enable = true` (see *For operators* above). `services.hyperhive.deploy.forgejo.ci.enable = true` (see _For operators_ above).
Optional tuning: `services.hyperhive.deploy.forgejo.ci.name` (runner name in forge Optional tuning: `services.hyperhive.deploy.forgejo.ci.name` (runner name in forge
admin panel), `concurrency` (parallel job capacity), `labels` (workflow admin panel), `concurrency` (parallel job capacity), `labels` (workflow
@ -131,7 +131,7 @@ Gated on `HYPERHIVE_FORGE_CI_ENABLED` (the nix module sets it on `hive-c0re.serv
- The container boots immediately — nothing gates its start on registration. - The container boots immediately — nothing gates its start on registration.
- tmpfiles seeds `/run/hive-ci/runner-token` with `TOKEN=placeholder` so the runner's `EnvironmentFile` always exists. - tmpfiles seeds `/run/hive-ci/runner-token` with `TOKEN=placeholder` so the runner's `EnvironmentFile` always exists.
- `gitea-runner-hive.service` has an `ExecStartPre` precond (ahead of the nix-daemon wait) that **fails fast** unless it is already registered (`.runner` present) or a real, non-placeholder token is in place. `Restart=on-failure` (no start-limit cap) self-heals it: a runner that precond-fails at boot keeps retrying until hive-c0re writes the token (c0re's explicit restart is the primary path; the retry is the safety net). - `gitea-runner-hive.service` has an `ExecStartPre` precond (ahead of the nix-daemon wait) that **fails fast** unless it is already registered (`.runner` present) or a real, non-placeholder token is in place. `Restart=on-failure` (no start-limit cap) self-heals it: a runner that precond-fails at boot keeps retrying until hive-c0re writes the token (c0re's explicit restart is the primary path; the retry is the safety net).
- **Convergence**: because the token write targets the *host* file, even if c0re's restart races the container being down, the container later starts, reads the now-real token, passes the precond, and registers on its own. - **Convergence**: because the token write targets the _host_ file, even if c0re's restart races the container being down, the container later starts, reads the now-real token, passes the precond, and registers on its own.
## Actions checkout mirror ## Actions checkout mirror

124
docs/scheduler/coordinator.md
View file

@ -19,7 +19,7 @@ changes) is submitted to the global job-DAG queue (`hive-c0re/src/job_queue/`)
as a **DAG of primitive nodes**. One scheduler task drives all DAGs; as a **DAG of primitive nodes**. One scheduler task drives all DAGs;
concurrency comes from the resource classes below, not from multiple workers. concurrency comes from the resource classes below, not from multiple workers.
Special cases like graceful stop, deferred starts, and the meta-update Special cases like graceful stop, deferred starts, and the meta-update
cascade need no bespoke code paths — each is expressed as a DAG *shape* cascade need no bespoke code paths — each is expressed as a DAG _shape_
built from the same primitive nodes. built from the same primitive nodes.
### Two levels: DAG and node ### Two levels: DAG and node
@ -42,54 +42,54 @@ there is no malformed spec to reject.
Nix-heavy — hold one of the `buildSlots` permits for the node's duration: Nix-heavy — hold one of the `buildSlots` permits for the node's duration:
| Node | Wraps | | Node | Wraps |
| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Prebuild` | `lifecycle::prebuild_toplevel` — build the toplevel out-of-band while the container keeps serving (its meta preamble is the upstream `MetaSync` node). Skipped when the container is already down; `Swap` builds inline instead | | `Prebuild` | `lifecycle::prebuild_toplevel` — build the toplevel out-of-band while the container keeps serving (its meta preamble is the upstream `MetaSync` node). Skipped when the container is already down; `Swap` builds inline instead |
| `Swap` | drop-in rewrite + `nixos-container update` profile-swap (requires the container stopped); the post-swap bookkeeping tail lives in the sibling `RebuildBookkeeping` node | | `Swap` | drop-in rewrite + `nixos-container update` profile-swap (requires the container stopped); the post-swap bookkeeping tail lives in the sibling `RebuildBookkeeping` node |
| `Create` | first-spawn `nixos-container create` proper; assumes the upstream `Provision` node already registered the agent in meta | | `Create` | first-spawn `nixos-container create` proper; assumes the upstream `Provision` node already registered the agent in meta |
| `MetaLock` | meta flake lock bump (`lock_update` / boot-sweep `lock_update_hyperhive`, commit fused — see below); fans out child `Rebuild` DAGs on completion | | `MetaLock` | meta flake lock bump (`lock_update` / boot-sweep `lock_update_hyperhive`, commit fused — see below); fans out child `Rebuild` DAGs on completion |
| `DeployWindow` | resource-holding root of the merge-config-PR deploy subtree — declares the build slot, the lease and the meta window, then completes immediately so its children run under them (see _Approvals_ below) | | `DeployWindow` | resource-holding root of the merge-config-PR deploy subtree — declares the build slot, the lease and the meta window, then completes immediately so its children run under them (see _Approvals_ below) |
| `DeployApply` | the deploy's irreversible half: ff-merge the reviewed PR head, two-phase meta deploy, container rebuild | | `DeployApply` | the deploy's irreversible half: ff-merge the reviewed PR head, two-phase meta deploy, container rebuild |
Cheap — no build slot: Cheap — no build slot:
| Node | Behavior | | Node | Behavior |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------ | | -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `MergeVerify` | the deploy's pre-merge gate — PR-head drift check, fetch, `verify_commit` eval. Mutates nothing, so a rejection here needs no compensation | | `MergeVerify` | the deploy's pre-merge gate — PR-head drift check, fetch, `verify_commit` eval. Mutates nothing, so a rejection here needs no compensation |
| `DeployTail` | the deploy's `AfterAny` compensation + bookkeeping tail: (1) rolls `applied/main` back from the parked `refs/hyperhive/rollback/<id>` and aborts the staged meta lock when the deploy never confirmed good; (2) mirrors whichever deploy tag landed to the forge config repo, always, best-effort; (3) posts the failing build log back onto the config PR when the deploy failed. Named for (2)/(3), which run on the success path too — not `AbortDeploy`. Infallible by construction | | `DeployTail` | the deploy's `AfterAny` compensation + bookkeeping tail: (1) rolls `applied/main` back from the parked `refs/hyperhive/rollback/<id>` and aborts the staged meta lock when the deploy never confirmed good; (2) mirrors whichever deploy tag landed to the forge config repo, always, best-effort; (3) posts the failing build log back onto the config PR when the deploy failed. Named for (2)/(3), which run on the success path too — not `AbortDeploy`. Infallible by construction |
| `MetaSync` | the rebuild's meta preamble — rebuild-dir prep, idempotent meta `sync_agents`, optional per-agent relock. Holds the `MetaWindow` resource (below); deliberately its own node so the window never covers `Prebuild`'s multi-minute build | | `MetaSync` | the rebuild's meta preamble — rebuild-dir prep, idempotent meta `sync_agents`, optional per-agent relock. Holds the `MetaWindow` resource (below); deliberately its own node so the window never covers `Prebuild`'s multi-minute build |
| `Provision` | first-spawn pre-create provisioning — proposed/applied repos, state subvolume, meta registration (`sync_agents`); runs ahead of `Create` so the `nixos-container create --flake meta#<name>` ref resolves. Store/meta-only, no container yet | | `Provision` | first-spawn pre-create provisioning — proposed/applied repos, state subvolume, meta registration (`sync_agents`); runs ahead of `Create` so the `nixos-container create --flake meta#<name>` ref resolves. Store/meta-only, no container yet |
| `Reconcile` | idempotent power converge: read `wanted` (below) + observed state; start if `Up` & down (cold-start fallback included), stop if `Offline` & up, else noop | | `Reconcile` | idempotent power converge: read `wanted` (below) + observed state; start if `Up` & down (cold-start fallback included), stop if `Offline` & up, else noop |
| `Start` | mechanical container start — runtime dir + drop-ins, `start_with_fallback`, MCP listener registration, the manager kick. Fanned out by a `Reconcile` that observed `wanted = Up` and the container down | | `Start` | mechanical container start — runtime dir + drop-ins, `start_with_fallback`, MCP listener registration, the manager kick. Fanned out by a `Reconcile` that observed `wanted = Up` and the container down |
| `Stop` | mechanical container stop — `nixos-container` kill, MCP listener unregister, the `Killed` manager notify. Fanned out by a `Reconcile` that observed `wanted = Offline` and up | | `Stop` | mechanical container stop — `nixos-container` kill, MCP listener unregister, the `Killed` manager notify. Fanned out by a `Reconcile` that observed `wanted = Offline` and up |
| `StopForUpdate` | mechanical `nixos-container stop` for the profile swap; never touches `wanted`; noop if already stopped | | `StopForUpdate` | mechanical `nixos-container stop` for the profile swap; never touches `wanted`; noop if already stopped |
| `RebuildBookkeeping` | the swap's Ok-only bookkeeping tail — rev marker, forge/matrix sync, manager kick, rescan, meta-inputs snapshot; `AfterOk(Swap)` so it runs only on a successful swap (the `Rebuilt` manager event is emitted by the DAG's `EmitRebuilt` tail node, not here). Split out of `Swap` for dashboard visibility + retry granularity, declares no resources of its own — a coordinated child of the `AgentWindow` brace | | `RebuildBookkeeping` | the swap's Ok-only bookkeeping tail — rev marker, forge/matrix sync, manager kick, rescan, meta-inputs snapshot; `AfterOk(Swap)` so it runs only on a successful swap (the `Rebuilt` manager event is emitted by the DAG's `EmitRebuilt` tail node, not here). Split out of `Swap` for dashboard visibility + retry granularity, declares no resources of its own — a coordinated child of the `AgentWindow` brace |
| `AgentWindow` | pure resource holder — the brace for one agent's rebuild. Declares the build slot + agent lease atomically and holds both for its whole subtree, so `Prebuild` and the `Signal`→`Drain` quiesce window run concurrently instead of one nested under the other. Performs no work; see _Braces_ | | `AgentWindow` | pure resource holder — the brace for one agent's rebuild. Declares the build slot + agent lease atomically and holds both for its whole subtree, so `Prebuild` and the `Signal`→`Drain` quiesce window run concurrently instead of one nested under the other. Performs no work; see _Braces_ |
| `Signal` | set the graceful fence + kick, so the harness runs one stop-checkpoint turn | | `Signal` | set the graceful fence + kick, so the harness runs one stop-checkpoint turn |
| `Drain` | await the harness clearing the fence, bounded by the 3-min graceful-stop timeout; resolves ok either way | | `Drain` | await the harness clearing the fence, bounded by the 3-min graceful-stop timeout; resolves ok either way |
| `PauseSignal` | write the pause marker + mark `pause_pending`. No kick, unlike `Signal` — the harness's between-turns poll is already responsive enough, and `Signal`'s kick-message body ("you were just (re)started") would be actively misleading here | | `PauseSignal` | write the pause marker + mark `pause_pending`. No kick, unlike `Signal` — the harness's between-turns poll is already responsive enough, and `Signal`'s kick-message body ("you were just (re)started") would be actively misleading here |
| `PauseDrain` | await the harness reporting `PauseAcknowledged`, bounded timeout; best-effort like `Drain` | | `PauseDrain` | await the harness reporting `PauseAcknowledged`, bounded timeout; best-effort like `Drain` |
| `DestroyContainer` | `nixos-container destroy` + un-registration (drop from the roster, clear the ephemeral runtime dir). Runs downstream of a `Stop`, so deliberately excluded from `takes_container_down` — the container is already down by the time it claims | | `DestroyContainer` | `nixos-container destroy` + un-registration (drop from the roster, clear the ephemeral runtime dir). Runs downstream of a `Stop`, so deliberately excluded from `takes_container_down` — the container is already down by the time it claims |
| `PurgeState` | the `purge = true` half of a destroy: delete the agent's state subvolume (via hive-priv) plus its state/applied dirs. Own node because it's conditional and the irreversible step | | `PurgeState` | the `purge = true` half of a destroy: delete the agent's state subvolume (via hive-priv) plus its state/applied dirs. Own node because it's conditional and the irreversible step |
| `DestroyBookkeeping` | the post-destroy tail — meta sync, fail pending approvals, drop the power intent, notify the manager, rescan, re-emit the tombstone, resync tmpfiles. Same split rationale as `RebuildBookkeeping`/`Swap`. Its `purge` flag only selects the wording of the approval-failure reason and the manager notification — the destructive work is `PurgeState`'s | | `DestroyBookkeeping` | the post-destroy tail — meta sync, fail pending approvals, drop the power intent, notify the manager, rescan, re-emit the tombstone, resync tmpfiles. Same split rationale as `RebuildBookkeeping`/`Swap`. Its `purge` flag only selects the wording of the approval-failure reason and the manager notification — the destructive work is `PurgeState`'s |
| `SetWanted` | write the durable power intent (`wanted = Up`/`Offline`) as the head node of a power-op DAG, replacing the old pre-submit side effect. Takes the agent lease even though it's a store write, so the intent write and the tail `Reconcile` are atomic per-agent — two racing power ops can't clobber each other's intent before either reconciles | | `SetWanted` | write the durable power intent (`wanted = Up`/`Offline`) as the head node of a power-op DAG, replacing the old pre-submit side effect. Takes the agent lease even though it's a store write, so the intent write and the tail `Reconcile` are atomic per-agent — two racing power ops can't clobber each other's intent before either reconciles |
| `FinalizeDeploy` | deploy phase 3 — drop the rollback ref, plant `deployed/<id>`, commit the staged `flake.lock`. The first two git steps are fatal on purpose, so a confirmed-good deploy's outcome and the repo's state can't disagree | | `FinalizeDeploy` | deploy phase 3 — drop the rollback ref, plant `deployed/<id>`, commit the staged `flake.lock`. The first two git steps are fatal on purpose, so a confirmed-good deploy's outcome and the repo's state can't disagree |
| `ResolveApproval` | tail of an approval-carrying DAG — resolve the approval row from how the work ended (`AfterAny`, one node emitted per outcome). Agentless: the approval row already names its agent | | `ResolveApproval` | tail of an approval-carrying DAG — resolve the approval row from how the work ended (`AfterAny`, one node emitted per outcome). Agentless: the approval row already names its agent |
| `EmitRebuilt` | tail of a rebuild/perm-change — emit the agent's `Rebuilt` manager event (ok/fail per outcome, nothing on cancel). One node per agent *and* per outcome | | `EmitRebuilt` | tail of a rebuild/perm-change — emit the agent's `Rebuilt` manager event (ok/fail per outcome, nothing on cancel). One node per agent _and_ per outcome |
| `WriteDropin` | `set_nspawn_flags` + `set_resource_limits` + daemon-reload | | `WriteDropin` | `set_nspawn_flags` + `set_resource_limits` + daemon-reload |
| `WritePermFile` | commit `tool-groups.json` / `capabilities.json` (single git commit under `META_LOCK`) + emit the P3RM1SS10NS snapshots | | `WritePermFile` | commit `tool-groups.json` / `capabilities.json` (single git commit under `META_LOCK`) + emit the P3RM1SS10NS snapshots |
| `Reparent` | `set-parent` / `set-parent-bulk`: apply every `(child, new_parent)` move under one `META_LOCK` commit (`meta::bulk_commit_topology`), send the per-agent move notifications, rescan + diff-emit. Agentless like `MetaLock` — a bulk move can span multiple agents, and a reparent touches the meta repo, not any one container. `moves` is typed `(Ident, Option<Ident>)` pairs, not raw strings. Rides the existing `Template::MetaUpdate` variant rather than a dedicated one — it's internal-only (never reaches the graph wire), so the stand-in only affects `terminal_hook` dispatch (resolves to no hook either way) and history-retention bucketing | | `Reparent` | `set-parent` / `set-parent-bulk`: apply every `(child, new_parent)` move under one `META_LOCK` commit (`meta::bulk_commit_topology`), send the per-agent move notifications, rescan + diff-emit. Agentless like `MetaLock` — a bulk move can span multiple agents, and a reparent touches the meta repo, not any one container. `moves` is typed `(Ident, Option<Ident>)` pairs, not raw strings. Rides the existing `Template::MetaUpdate` variant rather than a dedicated one — it's internal-only (never reaches the graph wire), so the stand-in only affects `terminal_hook` dispatch (resolves to no hook either way) and history-retention bucketing |
| `ForgeSweep` | one-shot boot-time forge user/token sweep for every container (`forge::ensure_all`) as a first-class node, so it shows as real work on the dashboard instead of running invisibly in a bare `tokio::spawn`. Agentless | | `ForgeSweep` | one-shot boot-time forge user/token sweep for every container (`forge::ensure_all`) as a first-class node, so it shows as real work on the dashboard instead of running invisibly in a bare `tokio::spawn`. Agentless |
| `MatrixSweep` | same as `ForgeSweep`, for matrix (`matrix::ensure_all`). The periodic 30-min re-sweep stays a background loop in `main.rs`; only the boot-time instance is a node | | `MatrixSweep` | same as `ForgeSweep`, for matrix (`matrix::ensure_all`). The periodic 30-min re-sweep stays a background loop in `main.rs`; only the boot-time instance is a node |
| `WebhookRegister` | one-shot boot-time Forgejo webhook registration (`internal/knowledge` push→pull, `agent-configs` PR→approval). No-op until the core token, hive domain, and HMAC secret are all available. Agentless | | `WebhookRegister` | one-shot boot-time Forgejo webhook registration (`internal/knowledge` push→pull, `agent-configs` PR→approval). No-op until the core token, hive domain, and HMAC secret are all available. Agentless |
| `KnowledgePull` | one-shot boot-time `/knowledge` pull (`knowledge::pull`), reconciling commits that landed while `hive-c0re` was down. Same rationale as `MatrixSweep`: the periodic hourly re-pull stays a background loop | | `KnowledgePull` | one-shot boot-time `/knowledge` pull (`knowledge::pull`), reconciling commits that landed while `hive-c0re` was down. Same rationale as `MatrixSweep`: the periodic hourly re-pull stays a background loop |
| `WantedPull` | one-shot boot-time pull of the agent set the swarm controller declares for this hive (`wanted::pull`), converging the agents it names. No background loop behind this one — boot is the whole cadence; the deploy event (`swarm_status`) is the fast path, this repairs a missed one. Agentless | | `WantedPull` | one-shot boot-time pull of the agent set the swarm controller declares for this hive (`wanted::pull`), converging the agents it names. No background loop behind this one — boot is the whole cadence; the deploy event (`swarm_status`) is the fast path, this repairs a missed one. Agentless |
There is deliberately **no `GitCommit` node**: `meta.rs` fuses each mutation There is deliberately **no `GitCommit` node**: `meta.rs` fuses each mutation
with its commit under its internal `META_LOCK` mutex, so a standalone commit with its commit under its internal `META_LOCK` mutex, so a standalone commit
node would open a dirty-working-tree window between nodes. node would open a dirty-working-tree window between nodes.
Two further layers protect the meta repo across *windows* that span multiple Two further layers protect the meta repo across _windows_ that span multiple
`META_LOCK` acquisitions — above all the approval deploy's prepare→finalize `META_LOCK` acquisitions — above all the approval deploy's prepare→finalize
span, which keeps a bumped `flake.lock` **staged uncommitted** for the whole span, which keeps a bumped `flake.lock` **staged uncommitted** for the whole
container build: container build:
@ -104,7 +104,7 @@ container build:
because a resource is held by a subtree root across its whole subtree, which because a resource is held by a subtree root across its whole subtree, which
a `MutexGuard` (bounded by one executor fn) cannot — that is what lets a a `MutexGuard` (bounded by one executor fn) cannot — that is what lets a
multi-node deploy own one window. For the same reason the window must stay multi-node deploy own one window. For the same reason the window must stay
*off* long store-only work: the rebuild's meta preamble is its own _off_ long store-only work: the rebuild's meta preamble is its own
`MetaSync` node, a sibling of (never a parent of) `Prebuild`, so the `MetaSync` node, a sibling of (never a parent of) `Prebuild`, so the
toplevel build runs outside the window and `buildSlots > 1` still gives toplevel build runs outside the window and `buildSlots > 1` still gives
concurrent rebuilds across agents. concurrent rebuilds across agents.
@ -119,10 +119,10 @@ container build:
The `stop` / `start` power ops write the durable `wanted` intent via a head The `stop` / `start` power ops write the durable `wanted` intent via a head
`SetWanted` node (not a pre-submit side effect) — it holds the agent lease, `SetWanted` node (not a pre-submit side effect) — it holds the agent lease,
so intent-write + reconcile is atomic per-agent. `restart` is the exception: so intent-write + reconcile is atomic per-agent. `restart` is the exception:
it writes *no* intent (no `SetWanted` head) — it bounces the container and it writes _no_ intent (no `SetWanted` head) — it bounces the container and
lets the tail `Reconcile` converge to the agent's existing `wanted`, so a lets the tail `Reconcile` converge to the agent's existing `wanted`, so a
deliberately-stopped agent is not forced back up by a hive-wide restart. The deliberately-stopped agent is not forced back up by a hive-wide restart. The
hive-wide power ops — `restart`, `stop`, and `start` — take an agent *list*: hive-wide power ops — `restart`, `stop`, and `start` — take an agent _list_:
a hive-wide `hivectl restart` / `stop` / `start` is ONE DAG with a per-agent a hive-wide `hivectl restart` / `stop` / `start` is ONE DAG with a per-agent
subgraph each (independent roots, run concurrently on their own leases), not subgraph each (independent roots, run concurrently on their own leases), not
N separate DAGs. N separate DAGs.
@ -132,12 +132,12 @@ async `lifecycle::is_running` read), so they live in `job_queue/power.rs`,
not the pure/sync `templates.rs`. Per-agent shape rule: `stop`/`start` carry not the pure/sync `templates.rs`. Per-agent shape rule: `stop`/`start` carry
a head `SetWanted` (intent) — `restart` does not; the tail `Reconcile` a head `SetWanted` (intent) — `restart` does not; the tail `Reconcile`
(convergence guarantee — cheap, noops when already converged) is ALWAYS (convergence guarantee — cheap, noops when already converged) is ALWAYS
present; only the *mechanical* nodes (`Signal`/`Drain`/`StopForUpdate`) are present; only the _mechanical_ nodes (`Signal`/`Drain`/`StopForUpdate`) are
state-conditional — skipped for a *down* agent (nothing to quiesce/stop). Keeping `Reconcile` in every shape state-conditional — skipped for a _down_ agent (nothing to quiesce/stop). Keeping `Reconcile` in every shape
closes the TOCTOU window: if an agent flips state between the `is_running` closes the TOCTOU window: if an agent flips state between the `is_running`
read and node exec, the tail `Reconcile` still converges it in-DAG (with read and node exec, the tail `Reconcile` still converges it in-DAG (with
`StopForUpdate`-noop as the backstop) — no reliance on an external reconcile `StopForUpdate`-noop as the backstop) — no reliance on an external reconcile
sweep. `start` folds the per-agent stale-rev upgrade in (a *down + stale* sweep. `start` folds the per-agent stale-rev upgrade in (a _down + stale_
agent's subgraph is a rebuild-then-start). agent's subgraph is a rebuild-then-start).
```text ```text
@ -163,7 +163,7 @@ Notable collapses:
leaves it stopped). leaves it stopped).
- **The swap-failure recovery-start is structural**: `Reconcile` deps on - **The swap-failure recovery-start is structural**: `Reconcile` deps on
`Swap` with the one `AfterAny` edge in the system — it runs after `Swap` `Swap` with the one `AfterAny` edge in the system — it runs after `Swap`
terminal ok *or* fail, bringing a wanted-up agent back on its old config. terminal ok _or_ fail, bringing a wanted-up agent back on its old config.
- **Deferred start is automatic**: `Reconcile` holds no build slot, so the - **Deferred start is automatic**: `Reconcile` holds no build slot, so the
next DAG's `Prebuild` starts as soon as `Swap` frees the slot. next DAG's `Prebuild` starts as soon as `Swap` frees the slot.
- **Graceful stop needs no watcher thread**: `Signal`/`Drain` are cheap, so a - **Graceful stop needs no watcher thread**: `Signal`/`Drain` are cheap, so a
@ -171,7 +171,7 @@ Notable collapses:
drains overlap; each DAG's tail `Reconcile` does the actual stop. drains overlap; each DAG's tail `Reconcile` does the actual stop.
- **The meta-update cascade grows in the same DAG on completion**: - **The meta-update cascade grows in the same DAG on completion**:
`MetaLock`'s executor computes the affected agent set after the bump lands `MetaLock`'s executor computes the affected agent set after the bump lands
and grows one `rebuild` subgraph per agent into its *own* DAG via and grows one `rebuild` subgraph per agent into its _own_ DAG via
`append_subgraph` (rooted on the `MetaLock`, `relock = false` so the cascade `append_subgraph` (rooted on the `MetaLock`, `relock = false` so the cascade
doesn't revert the bump). Not child DAGs — one DAG, no `parent_id`. A failed doesn't revert the bump). Not child DAGs — one DAG, no `parent_id`. A failed
bump appends nothing (no cancel-children dance). Same shape as the startup bump appends nothing (no cancel-children dance). Same shape as the startup
@ -180,9 +180,9 @@ Notable collapses:
### Desired-state (spec vs status) ### Desired-state (spec vs status)
Per-agent power *intent* — `wanted: Up | Offline` — is durable as the Per-agent power _intent_ — `wanted: Up | Offline` — is durable as the
`agent_power` table in the coordinator DB (`hive-c0re/src/stores/power.rs`). `agent_power` table in the coordinator DB (`hive-c0re/src/stores/power.rs`).
`container_view` remains the observed *status*; `Reconcile` nodes converge the `container_view` remains the observed _status_; `Reconcile` nodes converge the
two. Setting `wanted` is never a queued node: the power layer two. Setting `wanted` is never a queued node: the power layer
(`job_queue/power.rs`) writes the row synchronously, then inserts the DAG (`job_queue/power.rs`) writes the row synchronously, then inserts the DAG
whose `Reconcile` reads the fresh value — rapid toggles are last-writer-wins. whose `Reconcile` reads the fresh value — rapid toggles are last-writer-wins.
@ -237,15 +237,15 @@ than to one DAG shape it happens to appear in. A **brace** is the one sanctioned
exception: a pure-resource-holder root that declares on behalf of a subtree exception: a pure-resource-holder root that declares on behalf of a subtree
coordinated with itself, whose members then declare nothing. coordinated with itself, whose members then declare nothing.
It is forced rather than stylistic. Declaring a resource means *"I need this It is forced rather than stylistic. Declaring a resource means _"I need this
exclusively"*, and the agent lease is single-unit — so **two siblings that both exclusively"_, and the agent lease is single-unit — so **two siblings that both
declared it could never run concurrently.** For a subtree whose whole point is declared it could never run concurrently.** For a subtree whose whole point is
concurrency (`Prebuild` beside the `Signal` → `Drain` quiesce window), declaring concurrency (`Prebuild` beside the `Signal` → `Drain` quiesce window), declaring
the requirement truthfully on every node and running those nodes in parallel are the requirement truthfully on every node and running those nodes in parallel are
mutually exclusive. One holder above them speaks for the subtree. mutually exclusive. One holder above them speaks for the subtree.
This is the opposite of the failure the declare-your-own rule exists to prevent, This is the opposite of the failure the declare-your-own rule exists to prevent,
not a relapse into it: there the requirement was *implicit*, inferred from a not a relapse into it: there the requirement was _implicit_, inferred from a
node's kind and true only by accident of placement. Here it is explicit, on one node's kind and true only by accident of placement. Here it is explicit, on one
node, with the omission below it documented on the brace itself. node, with the omission below it documented on the brace itself.
@ -257,7 +257,7 @@ Two consequences worth knowing:
leave a gap another DAG could claim the agent in, mid-bounce. A brace supplies leave a gap another DAG could claim the agent in, mid-bounce. A brace supplies
that continuity directly, so the nesting is no longer load-bearing. that continuity directly, so the nesting is no longer load-bearing.
- **Observability is unaffected.** `running_transients` keys off a node's - **Observability is unaffected.** `running_transients` keys off a node's
*payload* agent, not off a declared lease edge, so every child still lights its _payload_ agent, not off a declared lease edge, so every child still lights its
own dashboard pill and still reports its own `takes_container_down` to the own dashboard pill and still reports its own `takes_container_down` to the
crash watcher. A brace itself reports `false`: it parents the stopping nodes crash watcher. A brace itself reports `false`: it parents the stopping nodes
but does not stop anything, and claiming otherwise would widen crash but does not stop anything, and claiming otherwise would widen crash
@ -323,12 +323,12 @@ cancelled-while-queued, which fails the approval instead of dangling it).
### Wire shape ### Wire shape
`RebuildQueueChanged { seq }` (event name kept) — **a bare trigger, no `RebuildQueueChanged { seq }` (event name kept) — **a bare trigger, no
payload.** It says *the queue changed*; a client that wants to know how payload.** It says _the queue changed_; a client that wants to know how
re-fetches `GET /api/jobq/graph`. re-fetches `GET /api/jobq/graph`.
That endpoint serves the graph generically (`hive-jobq-wire`): every node That endpoint serves the graph generically (`hive-jobq-wire`): every node
carries `id`, `parent`, `deps`, `state`, `label` (the node kind's own wire carries `id`, `parent`, `deps`, `state`, `label` (the node kind's own wire
string — the kind *is* the phase label, there is no separate sub-step string — the kind _is_ the phase label, there is no separate sub-step
string) and free-form `data` for what only some kinds have (`agent`, string) and free-form `data` for what only some kinds have (`agent`,
`approval_id`, `inputs`, `build_log_id`). Group roots ride as ordinary `approval_id`, `inputs`, `build_log_id`). Group roots ride as ordinary
nodes, so a group's state is just the root's own `state`. nodes, so a group's state is just the root's own `state`.
@ -341,7 +341,7 @@ live-log panel off the running node.
The event carries no payload by design: shipping a typed projection of The event carries no payload by design: shipping a typed projection of
the whole queue in the event itself would be a second rendering of the the whole queue in the event itself would be a second rendering of the
same graph that has to be kept in agreement by hand with the endpoint same graph that has to be kept in agreement by hand with the endpoint
every consumer actually reads. Telling a client *when* to refetch is every consumer actually reads. Telling a client _when_ to refetch is
the event's whole job. the event's whole job.
--- ---
@ -362,7 +362,7 @@ On startup, `auto_update::run` classifies every agent by rev freshness (the
per-agent `.{name}.hyperhive-rev` marker under `/var/lib/hyperhive/applied/` per-agent `.{name}.hyperhive-rev` marker under `/var/lib/hyperhive/applied/`
vs the current flake path) and persisted `wanted` intent, then: vs the current flake path) and persisted `wanted` intent, then:
1. **Config path** — when *any* marker is stale, submit one `Boot` 1. **Config path** — when _any_ marker is stale, submit one `Boot`
DAG: a `MetaLock` (hyperhive input bump, non-fatal) that grows an in-DAG DAG: a `MetaLock` (hyperhive input bump, non-fatal) that grows an in-DAG
`Rebuild` subgraph for each stale agent whose `wanted = Up` (topology-sorted, `Rebuild` subgraph for each stale agent whose `wanted = Up` (topology-sorted,
parents first). Stale but wanted-offline agents get no boot-time nix work — their parents first). Stale but wanted-offline agents get no boot-time nix work — their
@ -453,7 +453,7 @@ graft's completion then plants the deploy tag — so "did the agent come back
up?" is answered by `Reconcile` succeeding, the same way it is for every up?" is answered by `Reconcile` succeeding, the same way it is for every
other rebuild, instead of by a fused inline start. other rebuild, instead of by a fused inline start.
The grafted nodes land *inside* `DeployWindow`'s subtree, so they re-enter The grafted nodes land _inside_ `DeployWindow`'s subtree, so they re-enter
the meta window and build slot it already holds rather than deadlocking the meta window and build slot it already holds rather than deadlocking
against it. against it.
@ -514,7 +514,7 @@ The same drop-in carries `CPUWeight=` / `IOWeight=` from
`agentCpuWeight` / `agentIoWeight`. Those are a different kind of `agentCpuWeight` / `agentIoWeight`. Those are a different kind of
setting: the quota and the memory max are **hard caps** that throttle setting: the quota and the memory max are **hard caps** that throttle
an agent even on a completely idle host, while the weights are cgroup an agent even on a completely idle host, while the weights are cgroup
v2 **relative shares** that only decide who yields *under contention*. v2 **relative shares** that only decide who yields _under contention_.
A low-weight container still gets the whole machine when nothing else A low-weight container still gets the whole machine when nothing else
wants it. wants it.
@ -529,7 +529,7 @@ Two things to know about the weights:
- They are **hive-wide** — unlike the caps there is no per-agent - They are **hive-wide** — unlike the caps there is no per-agent
override in `meta/resource-limits.json`, so every agent carries the override in `meta/resource-limits.json`, so every agent carries the
same value and the weight does *not* rank agents against each other. same value and the weight does _not_ rank agents against each other.
What `80` buys is that agents yield to everything **not** on this What `80` buys is that agents yield to everything **not** on this
drop-in path: host services (nginx and dnsmasq among them) and the drop-in path: host services (nginx and dnsmasq among them) and the
infra containers (`hive-ci`, `hive-forge`, `hive-gateway`, infra containers (`hive-ci`, `hive-forge`, `hive-gateway`,

20
docs/scheduler/jobq.md
View file

@ -2,7 +2,7 @@
Every container operation — rebuild, first-spawn, a config-PR deploy, Every container operation — rebuild, first-spawn, a config-PR deploy,
power changes — runs through one shared job queue. This page explains power changes — runs through one shared job queue. This page explains
what the job queue *is*, as a general idea, independent of what any one what the job queue _is_, as a general idea, independent of what any one
subsystem uses it for. For the hive-c0re-specific step catalogue and the subsystem uses it for. For the hive-c0re-specific step catalogue and the
engineering internals (scheduler, leases, resource windows) see engineering internals (scheduler, leases, resource windows) see
[`coordinator.md`](coordinator.md) instead. [`coordinator.md`](coordinator.md) instead.
@ -23,7 +23,7 @@ Two ideas are all there is to it:
that's the whole reason things queue instead of all firing at once. that's the whole reason things queue instead of all firing at once.
The engine's whole job is: whenever a step's ordering and resource needs The engine's whole job is: whenever a step's ordering and resource needs
are both satisfied, run it. It has no opinion on what the steps *do* — are both satisfied, run it. It has no opinion on what the steps _do_ —
that's supplied by whoever builds the graph. hive-c0re is the one thing that's supplied by whoever builds the graph. hive-c0re is the one thing
building graphs on it today, but nothing about the engine is specific to building graphs on it today, but nothing about the engine is specific to
containers or rebuilds; there's nothing stopping another subsystem from containers or rebuilds; there's nothing stopping another subsystem from
@ -38,14 +38,14 @@ nested under it are that job's steps, in order (occasionally a couple run
side by side). A step shows one of: side by side). A step shows one of:
| Glyph | Meaning | | Glyph | Meaning |
| ----- | -------------------------------------------------------- | | ----- | ------------------------------------------------------- |
| `⏸` | queued, waiting its turn | | `⏸` | queued, waiting its turn |
| `▶` | running | | `▶` | running |
| `◐` | its own work is done, waiting on a step nested under it | | `◐` | its own work is done, waiting on a step nested under it |
| `✔` | finished successfully | | `✔` | finished successfully |
| `✖` | failed | | `✖` | failed |
| `⊘` | cancelled | | `⊘` | cancelled |
| `·` | skipped (not needed for this run) | | `·` | skipped (not needed for this run) |
A step that isn't needed for a given run shows as `·` rather than being A step that isn't needed for a given run shows as `·` rather than being
left out of the tree entirely, so the same kind of operation keeps a left out of the tree entirely, so the same kind of operation keeps a

94
docs/scheduler/observability.md
View file

@ -46,7 +46,7 @@ no longer survives independently of anything host-side.
**It has no application-level auth.** The receiver takes any OTLP that reaches **It has no application-level auth.** The receiver takes any OTLP that reaches
it; what bounds who can reach it is the firewall — `exposeHostPorts` opens the it; what bounds who can reach it is the firewall — `exposeHostPorts` opens the
port on the bridge interface only. So "unauthenticated to a bridge address" port on the bridge interface only. So "unauthenticated to a bridge address"
means *reachable from an agent container*, not *presents a credential*. means _reachable from an agent container_, not _presents a credential_.
The consequence, stated because it is a choice rather than an oversight: **any The consequence, stated because it is a choice rather than an oversight: **any
agent can push arbitrary OTLP, and it is forwarded on under the operator's agent can push arbitrary OTLP, and it is forwarded on under the operator's
@ -56,10 +56,10 @@ smuggled out in resource attributes on an otherwise-legitimate export.
That is a **different risk from the one the collector fixes**, and strictly That is a **different risk from the one the collector fixes**, and strictly
smaller than what preceded it: before, every agent held the upstream credential smaller than what preceded it: before, every agent held the upstream credential
itself, so it could do all of the above *and* use the token anywhere else. The itself, so it could do all of the above _and_ use the token anywhere else. The
collector removes the token and keeps the pipe. Agents are inside the trust collector removes the token and keeps the pipe. Agents are inside the trust
boundary (`docs/trust-boundary/security.md`: capability = accepted risk), so an agent being boundary (`docs/trust-boundary/security.md`: capability = accepted risk), so an agent being
able to *send* is an accepted extension of that boundary — but it is not able to _send_ is an accepted extension of that boundary — but it is not
closed by this design, and nothing here should be read as closing it. closed by this design, and nothing here should be read as closing it.
**The `agent` label is self-reported, and no planned authentication changes **The `agent` label is self-reported, and no planned authentication changes
@ -73,13 +73,13 @@ is getting a credential:
- **agent→collector** (this section's hop) stays open on the bridge. Nothing - **agent→collector** (this section's hop) stays open on the bridge. Nothing
downstream can tell one agent's export from another's. downstream can tell one agent's export from another's.
- **hive→swarm** is where the planned ingest auth goes. The swarm tier stamps - **hive→swarm** is where the planned ingest auth goes. The swarm tier stamps
`hive=` from the connection it authenticated, so *that* label becomes `hive=` from the connection it authenticated, so _that_ label becomes
unforgeable. unforgeable.
So a verified `hive` is reachable and a verified `agent` is not — and that falls So a verified `hive` is reachable and a verified `agent` is not — and that falls
out of the topology rather than being a gap someone forgot to close. The swarm out of the topology rather than being a gap someone forgot to close. The swarm
runs one collector, and the mechanism gives it no finer grain: a bearer-token runs one collector, and the mechanism gives it no finer grain: a bearer-token
check never reveals *which* token matched, and a receiver reads request metadata check never reveals _which_ token matched, and a receiver reads request metadata
rather than the claims it authenticated with. rather than the claims it authenticated with.
If you need per-agent numbers you can act on, take them from the agent's own If you need per-agent numbers you can act on, take them from the agent's own
@ -100,10 +100,10 @@ two-tier architecture, the security model, and how the options interact.
Telemetry crosses two collectors, and which one you configure depends on what Telemetry crosses two collectors, and which one you configure depends on what
the host is: the host is:
| | runs where | receives from | does | | | runs where | receives from | does |
|---|---|---|---| | ------------------------------------ | ---------------------- | --------------------------------- | --------------------------------------------------------------------- |
| **hive tier** — `otel.enable` | every hive with agents | that hive's agents, on the bridge | forwards to the swarm tier. Holds no credential, picks no destination | | **hive tier** — `otel.enable` | every hive with agents | that hive's agents, on the bridge | forwards to the swarm tier. Holds no credential, picks no destination |
| **swarm tier** — `deploy.swarm-otel` | once per swarm | every hive's collector | writes the swarm's store and exports upstream | | **swarm tier** — `deploy.swarm-otel` | once per swarm | every hive's collector | writes the swarm's store and exports upstream |
An all-local host runs both, and needs nothing said about the hop between them. An all-local host runs both, and needs nothing said about the hop between them.
@ -119,16 +119,16 @@ services.hyperhive.otel = {
every agent needs the credential to authenticate — and the harness delivers every agent needs the credential to authenticate — and the harness delivers
that token into the agent's own `~/.claude/settings.json`, a file the agent can that token into the agent's own `~/.claude/settings.json`, a file the agent can
read. `0600` protects it from other containers, not from the agent itself. As read. `0600` protects it from other containers, not from the agent itself. As
long as the direct path stays *selectable*, that hole stays selectable; an long as the direct path stays _selectable_, that hole stays selectable; an
option that can reintroduce it is a hole with extra steps. option that can reintroduce it is a hole with extra steps.
**Why the tiers stay separate on one box.** They are not collapsed when **Why the tiers stay separate on one box.** They are not collapsed when
co-located: an all-local hive is a statement about *where* processes run, not co-located: an all-local hive is a statement about _where_ processes run, not
about the shape of the deployment. A boundary that disappears locally is one about the shape of the deployment. A boundary that disappears locally is one
the local deployment stops testing. the local deployment stops testing.
**`endpoint` keeps meaning "where telemetry goes upstream."** Neither tier **`endpoint` keeps meaning "where telemetry goes upstream."** Neither tier
redefines it — the agent-facing value is *derived* redefines it — the agent-facing value is _derived_
(`http://<bridgeIp>:<collector.port>`), so an existing deployment's `endpoint` (`http://<bridgeIp>:<collector.port>`), so an existing deployment's `endpoint`
keeps working unchanged. The bridge port is contributed to `exposeHostPorts` keeps working unchanged. The bridge port is contributed to `exposeHostPorts`
automatically; there is nothing to open by hand. automatically; there is nothing to open by hand.
@ -160,9 +160,9 @@ swarm tier and no metrics appearing for that hive.
⚠️ **`endpoint` must be valid for `protocol`.** The upstream exporter follows ⚠️ **`endpoint` must be valid for `protocol`.** The upstream exporter follows
`otel.protocol` (`grpc` → the gRPC exporter, otherwise OTLP/HTTP), and the gRPC `otel.protocol` (`grpc` → the gRPC exporter, otherwise OTLP/HTTP), and the gRPC
exporter takes an *address*: `https://host/path` is a legal exporter takes an _address_: `https://host/path` is a legal
`OTEL_EXPORTER_OTLP_ENDPOINT` for HTTP but fails as gRPC with *"missing port in `OTEL_EXPORTER_OTLP_ENDPOINT` for HTTP but fails as gRPC with _"missing port in
address"*. The collector's config is validated at build time, so a mismatch is address"_. The collector's config is validated at build time, so a mismatch is
a build error naming the reason rather than telemetry silently going nowhere. a build error naming the reason rather than telemetry silently going nowhere.
## Network access ## Network access
@ -176,7 +176,7 @@ services.hyperhive.network.exposeHostPorts = [ 5432 ];
``` ```
and point whatever consumes it at `10.42.0.1:5432` rather than loopback: inside and point whatever consumes it at `10.42.0.1:5432` rather than loopback: inside
a container, loopback is the *container*. The bridge IP is the host's address on a container, loopback is the _container_. The bridge IP is the host's address on
the `hive-br0` bridge. The service must also bind an address the bridge can the `hive-br0` bridge. The service must also bind an address the bridge can
reach — a `127.0.0.1`-only listener stays unreachable no matter what the reach — a `127.0.0.1`-only listener stays unreachable no matter what the
firewall allows. See `docs/networking/network.md::Reaching host services` for details. firewall allows. See `docs/networking/network.md::Reaching host services` for details.
@ -196,12 +196,12 @@ yourself from a tool call or `hivectl shell`.
Every agent's export therefore includes these resource attributes Every agent's export therefore includes these resource attributes
automatically: automatically:
| Attribute | Value | | Attribute | Value |
|-----------|-------| | -------------- | ------------------------------------------------------------ |
| `service.name` | `hyperhive-agent` (constant) | | `service.name` | `hyperhive-agent` (constant) |
| `agent` | agent logical name (e.g. `iris`) | | `agent` | agent logical name (e.g. `iris`) |
| `hive` | hive display name (`services.hyperhive.hiveName`) | | `hive` | hive display name (`services.hyperhive.hiveName`) |
| `swarm` | swarm display name (`services.hyperhive.swarm.name`, if set) | | `swarm` | swarm display name (`services.hyperhive.swarm.name`, if set) |
Additional labels can be appended via `extraResourceAttributes` (see option Additional labels can be appended via `extraResourceAttributes` (see option
reference above); custom per-data-point labels can be passed with reference above); custom per-data-point labels can be passed with
@ -222,14 +222,14 @@ exists, so off-the-shelf OTEL/Grafana container dashboards work. Resource
`service.name = hyperhive-c0re`; each data point is tagged `container.name` `service.name = hyperhive-c0re`; each data point is tagged `container.name`
(= the `h-<agent>` machine) and the hive `agent` label: (= the `h-<agent>` machine) and the hive `agent` label:
| Metric | Unit | Kind | Source | | Metric | Unit | Kind | Source |
|--------|------|------|--------| | ----------------------------------- | ---- | ------- | ----------------------------------------------------------------------------------------------------------- |
| `container.cpu.time` | `s` | counter | cumulative `cpu.stat` `usage_usec` → seconds | | `container.cpu.time` | `s` | counter | cumulative `cpu.stat` `usage_usec` → seconds |
| `container.memory.usage` | `By` | gauge | `memory.current` | | `container.memory.usage` | `By` | gauge | `memory.current` |
| `hyperhive.container.memory.limit` | `By` | gauge | `memory.max` (custom — semconv has no `.limit` metric; omitted when unlimited) | | `hyperhive.container.memory.limit` | `By` | gauge | `memory.max` (custom — semconv has no `.limit` metric; omitted when unlimited) |
| `hyperhive.container.memory.peak` | `By` | gauge | `memory.peak` (custom — no semconv metric; omitted if unavailable) | | `hyperhive.container.memory.peak` | `By` | gauge | `memory.peak` (custom — no semconv metric; omitted if unavailable) |
| `hyperhive.container.storage.usage` | `By` | gauge | state dir + writable rootfs (custom — semconv only has `disk.io`; omitted until the slow disk sampler runs) | | `hyperhive.container.storage.usage` | `By` | gauge | state dir + writable rootfs (custom — semconv only has `disk.io`; omitted until the slow disk sampler runs) |
| `hyperhive.container.cpu.percent` | `%` | gauge | host-normalised percent (custom — the value the dashboard LOAD tab shows, no `rate()` needed) | | `hyperhive.container.cpu.percent` | `%` | gauge | host-normalised percent (custom — the value the dashboard LOAD tab shows, no `rate()` needed) |
The `hyperhive.`-prefixed metrics have no semconv equivalent (memory The `hyperhive.`-prefixed metrics have no semconv equivalent (memory
limit + peak, on-disk footprint, and an instantaneous cpu percent kept limit + peak, on-disk footprint, and an instantaneous cpu percent kept
@ -252,14 +252,14 @@ turn, its own outcome classification, the loose-ends backlog, and session
boundaries. Token usage, cost, and tool-call counts are **not** duplicated boundaries. Token usage, cost, and tool-call counts are **not** duplicated
here; that's already covered by Claude's own export. here; that's already covered by Claude's own export.
| Metric | Unit | Kind | Attributes | | Metric | Unit | Kind | Attributes |
|--------|------|------|------------| | -------------------------------------- | ---- | --------- | ----------------------------------------------------------------------------------------------------------- |
| `hyperhive.agent.turn.duration` | `ms` | histogram | `wake_from`, `result_kind`, `model` | | `hyperhive.agent.turn.duration` | `ms` | histogram | `wake_from`, `result_kind`, `model` |
| `hyperhive.agent.turn.count` | — | counter | `wake_from`, `result_kind`, `model` | | `hyperhive.agent.turn.count` | — | counter | `wake_from`, `result_kind`, `model` |
| `hyperhive.agent.session.count` | — | counter | `model` (incremented once per fresh, non-`--continue`'d session) | | `hyperhive.agent.session.count` | — | counter | `model` (incremented once per fresh, non-`--continue`'d session) |
| `hyperhive.agent.loose_ends.threads` | — | gauge | none | | `hyperhive.agent.loose_ends.threads` | — | gauge | none |
| `hyperhive.agent.loose_ends.reminders` | — | gauge | none | | `hyperhive.agent.loose_ends.reminders` | — | gauge | none |
| `hyperhive.agent.claude_md.lines` | — | gauge | none — recorded from the `CLAUDE.md`-size watch's own ~15-minute tick, **not** per turn like the rows above | | `hyperhive.agent.claude_md.lines` | — | gauge | none — recorded from the `CLAUDE.md`-size watch's own ~15-minute tick, **not** per turn like the rows above |
Resource attributes (`service.name`, `agent`, `hive`, `swarm`) come from the Resource attributes (`service.name`, `agent`, `hive`, `swarm`) come from the
same container-wide `OTEL_RESOURCE_ATTRIBUTES` as everything else in this same container-wide `OTEL_RESOURCE_ATTRIBUTES` as everything else in this
@ -277,11 +277,11 @@ that hosts no agents still reports, and "this hive is quiet" is
distinguishable from "this hive is gone". Select them with distinguishable from "this hive is gone". Select them with
`{hive!="",agent=""}`. `{hive!="",agent=""}`.
| Metric | Unit | Kind | Meaning | | Metric | Unit | Kind | Meaning |
|--------|------|------|---------| | ------------------------- | ---- | ----- | -------------------------------------------------------------------------------------------------------------------- |
| `process.uptime` | `s` | gauge | seconds since this hive's `hive-c0re` started exporting; a restart reads as a drop to ~0 | | `process.uptime` | `s` | gauge | seconds since this hive's `hive-c0re` started exporting; a restart reads as a drop to ~0 |
| `hyperhive.hive.degraded` | `1` | gauge | `1` while the hive reports itself unhealthy — the same verdict `/health/ready` gives and the swarm status view shows | | `hyperhive.hive.degraded` | `1` | gauge | `1` while the hive reports itself unhealthy — the same verdict `/health/ready` gives and the swarm status view shows |
| `hyperhive.hive.warnings` | `1` | gauge | how many warnings are currently raised, split by a `level` attribute (`warn`, `crit`) | | `hyperhive.hive.warnings` | `1` | gauge | how many warnings are currently raised, split by a `level` attribute (`warn`, `crit`) |
Both levels are reported every cycle, `0` included, so a healthy hive is Both levels are reported every cycle, `0` included, so a healthy hive is
visible as zeros rather than as missing series. visible as zeros rather than as missing series.
@ -303,10 +303,10 @@ Forgejo's own native `/metrics` endpoint has no equivalent: it exposes
counts of durable rows (issues, comments, repos), and neither a commit nor a counts of durable rows (issues, comments, repos), and neither a commit nor a
push is stored anywhere as a row to count. push is stored anywhere as a row to count.
| Metric | Unit | Kind | Attributes | | Metric | Unit | Kind | Attributes |
|--------|------|------|------------| | ---------------------------- | ---- | ------- | ------------------- |
| `hyperhive.vcs.commit.count` | — | counter | `repo` (`org/repo`) | | `hyperhive.vcs.commit.count` | — | counter | `repo` (`org/repo`) |
| `hyperhive.vcs.push.count` | — | counter | `repo` (`org/repo`) | | `hyperhive.vcs.push.count` | — | counter | `repo` (`org/repo`) |
A push with zero commits (a branch delete, or a force-push that doesn't add A push with zero commits (a branch delete, or a force-push that doesn't add
new commits) still increments `push.count`; `commit.count` only advances new commits) still increments `push.count`; `commit.count` only advances

48
docs/swarm/README.md
View file

@ -7,7 +7,7 @@ the additional config needed when the swarm spans multiple hosts.
For the full option reference rather than prose: `services.hyperhive.swarm.*` For the full option reference rather than prose: `services.hyperhive.swarm.*`
(swarm-wide facts, identical on every host) and `services.hyperhive.deploy.*` (swarm-wide facts, identical on every host) and `services.hyperhive.deploy.*`
(this host's own deployment decisions — does *this* machine run grafana, (this host's own deployment decisions — does _this_ machine run grafana,
the swarm controller, authelia, …) are separate generated pages, `nix the swarm controller, authelia, …) are separate generated pages, `nix
build .#docs-swarm` / `.#docs-deploy` or the website's `/options/swarm.html` build .#docs-swarm` / `.#docs-deploy` or the website's `/options/swarm.html`
/ `/options/deploy.html`. / `/options/deploy.html`.
@ -20,7 +20,7 @@ build .#docs-swarm` / `.#docs-deploy` or the website's `/options/swarm.html`
- **swarm** — one or more hives whose operators have declared them - **swarm** — one or more hives whose operators have declared them
as peers. Agents can be qualified as `agent@hive-domain`. as peers. Agents can be qualified as `agent@hive-domain`.
- **peer hive** — any hive in `services.hyperhive.swarm.hives` other - **peer hive** — any hive in `services.hyperhive.swarm.hives` other
than this one. Peers are *derived*, not declared: the directory lists than this one. Peers are _derived_, not declared: the directory lists
every hive including yourself, and `hiveName` says which one you are. every hive including yourself, and `hiveName` says which one you are.
## Hive identity config ## Hive identity config
@ -47,7 +47,7 @@ and deploys — an eval failure asking the operator to write the address
down is the cheaper outcome. **Upgrading past this release means setting down is the cheaper outcome. **Upgrading past this release means setting
both once.** both once.**
`domain` is required too, but you no longer *write* it: it is read from `domain` is required too, but you no longer _write_ it: it is read from
this hive's own entry in the directory, whose `domain` defaults to this hive's own entry in the directory, whose `domain` defaults to
`<name>.<swarm.domain>`. So a conventional swarm states no addresses at `<name>.<swarm.domain>`. So a conventional swarm states no addresses at
all, and a hive addressed by something else states it in the one place all, and a hive addressed by something else states it in the one place
@ -70,7 +70,7 @@ form qualified labels (`iris@pr1ma.example.com`).
`swarm.name` is purely display — it surfaces in the dashboard chrome `swarm.name` is purely display — it surfaces in the dashboard chrome
header and per-agent system prompts, and federated hives at different header and per-agent system prompts, and federated hives at different
domains can share one. `hiveName` surfaces in the same places but is domains can share one. `hiveName` surfaces in the same places but is
*not* only display: it is the leftmost label of the hive's domain. That _not_ only display: it is the leftmost label of the hive's domain. That
`swarm.name` sits under `swarm` and `hiveName` does not is the whole `swarm.name` sits under `swarm` and `hiveName` does not is the whole
distinction — one names this hive, the other names the group it belongs distinction — one names this hive, the other names the group it belongs
to. to.
@ -120,14 +120,14 @@ services.hyperhive.swarm.hives = {
``` ```
One attrset describing **every** hive in the swarm, **including this One attrset describing **every** hive in the swarm, **including this
one**, keyed by that hive's `hiveName`. It is meant to be *identical on one**, keyed by that hive's `hiveName`. It is meant to be _identical on
every host* — write it once, share it, and each host reads it correctly every host_ — write it once, share it, and each host reads it correctly
because `services.hyperhive.hiveName` says which entry is itself. because `services.hyperhive.hiveName` says which entry is itself.
Empty (the default) means this host isn't in a swarm. Once non-empty it Empty (the default) means this host isn't in a swarm. Once non-empty it
**must** contain an entry for `hiveName`; eval fails naming the missing **must** contain an entry for `hiveName`; eval fails naming the missing
hive. That assertion is load-bearing rather than pedantic — "my peers" hive. That assertion is load-bearing rather than pedantic — "my peers"
is derived as *everything that isn't me*, so a directory that doesn't is derived as _everything that isn't me_, so a directory that doesn't
contain you derives every hive as a peer and you peer with yourself. contain you derives every hive as a peer and you peer with yourself.
`domain` is required per entry and deliberately undefaulted: it is `domain` is required per entry and deliberately undefaulted: it is
@ -138,7 +138,7 @@ evaluates cleanly points at a real machine that isn't the one you meant.
> inside a swarm comes from the swarm root ([`ca.md`](ca.md)): every > inside a swarm comes from the swarm root ([`ca.md`](ca.md)): every
> hive chains to it, so one anchor replaces per-hive pinning entirely. > hive chains to it, so one anchor replaces per-hive pinning entirely.
> What that genuinely drops is trusting a hive whose root this swarm > What that genuinely drops is trusting a hive whose root this swarm
> does *not* own — another swarm's, or one keeping its own CA. That is > does _not_ own — another swarm's, or one keeping its own CA. That is
> a cross-swarm problem and wants a mechanism designed for it. (An > a cross-swarm problem and wants a mechanism designed for it. (An
> earlier `certFingerprint` field existed for exactly that gap, pinning > earlier `certFingerprint` field existed for exactly that gap, pinning
> a peer's TLS leaf for hive-c0re's own peer HTTPS checks — removed > a peer's TLS leaf for hive-c0re's own peer HTTPS checks — removed
@ -162,7 +162,7 @@ evaluates cleanly points at a real machine that isn't the one you meant.
⚠️ **That container currently trusts no swarm-internal CA**, so a ⚠️ **That container currently trusts no swarm-internal CA**, so a
self-signed gateway certificate does not federate. The swarm root self-signed gateway certificate does not federate. The swarm root
can't simply be listed there: `security.pki.certificateFiles` is can't simply be listed there: `security.pki.certificateFiles` is
read when the system is *built*, and the root is a runtime file (its read when the system is _built_, and the root is a runtime file (its
key must never enter the store), so there is no build-time name for key must never enter the store), so there is no build-time name for
it. Bridging that needs a runtime mechanism and is tracked as its own it. Bridging that needs a runtime mechanism and is tracked as its own
issue. Until then, federation needs CA-issued certs (ACME). See issue. Until then, federation needs CA-issued certs (ACME). See
@ -184,7 +184,7 @@ swarm.hives = { … }; swarm.hives = { … }; # byte-identical
``` ```
That is the point of the shape, and it removes a class of bug rather That is the point of the shape, and it removes a class of bug rather
than saving typing: a per-host peer list let two hosts hold *different* than saving typing: a per-host peer list let two hosts hold _different_
facts about the same third hive — a stale endpoint, a rotated facts about the same third hive — a stale endpoint, a rotated
fingerprint — with nothing to detect the disagreement. One entry per fingerprint — with nothing to detect the disagreement. One entry per
hive makes it unrepresentable. hive makes it unrepresentable.
@ -277,7 +277,7 @@ port}` tells this hive where the swarm's `btrfs receive` endpoint is, so
`hivectl agent <name> subvol snapshot push` has somewhere to stream to. `hivectl agent <name> subvol snapshot push` has somewhere to stream to.
It is genuinely swarm-scoped rather than per-peer — a swarm has exactly It is genuinely swarm-scoped rather than per-peer — a swarm has exactly
one store, because the receiver keys destinations by *agent* so a one store, because the receiver keys destinations by _agent_ so a
migrating agent keeps one unbroken incremental chain. See migrating agent keeps one unbroken incremental chain. See
[snapshot-store.md](../networking/snapshot-store.md). [snapshot-store.md](../networking/snapshot-store.md).
@ -305,12 +305,12 @@ A hive publishes only once it has been given the three
`swarm.statusPublish` coordinates below. A hive that has not reads `swarm.statusPublish` coordinates below. A hive that has not reads
`never_reported` — it is not broken, it just has nothing to say upward. `never_reported` — it is not broken, it just has nothing to say upward.
| freshness | what to do about it | | freshness | what to do about it |
|---|---| | ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `fresh` | nothing — reported within `staleAfterSeconds` | | `fresh` | nothing — reported within `staleAfterSeconds` |
| `stale` | the hive stopped reporting. Its last payload is still shown, so check `age_seconds` and the payload for what it managed to say | | `stale` | the hive stopped reporting. Its last payload is still shown, so check `age_seconds` and the payload for what it managed to say |
| `never_reported` | this hive has never reported at all — normally a deployment that hasn't happened, not an outage | | `never_reported` | this hive has never reported at all — normally a deployment that hasn't happened, not an outage |
| `unknown` | something is publishing under a name that is not in `swarm.hives` — a typo in the roster, or a hive that was removed and is still running | | `unknown` | something is publishing under a name that is not in `swarm.hives` — a typo in the roster, or a hive that was removed and is still running |
Every row also carries `last_seen_unix` and `age_seconds` if you want to Every row also carries `last_seen_unix` and `age_seconds` if you want to
apply your own threshold. The timestamp is the one the queue recorded on apply your own threshold. The timestamp is the one the queue recorded on
@ -328,11 +328,11 @@ Three options, on the **hive**, set together or not at all — a
half-configured hive is an eval error rather than one that quietly never half-configured hive is an eval error rather than one that quietly never
reports: reports:
| option | what to set it to | | option | what to set it to |
|---|---| | ------------------ | ------------------------------------------------------ |
| `natsUrl` | where the swarm queue listens, as this hive reaches it | | `natsUrl` | where the swarm queue listens, as this hive reaches it |
| `tokenEndpoint` | the swarm IdP's `/api/oidc/token` | | `tokenEndpoint` | the swarm IdP's `/api/oidc/token` |
| `clientSecretFile` | path to this hive's client secret, plaintext | | `clientSecretFile` | path to this hive's client secret, plaintext |
On a host that runs the queue and the IdP itself, all three default to On a host that runs the queue and the IdP itself, all three default to
the local ones and there is nothing to set. Any other hive needs them the local ones and there is nothing to set. Any other hive needs them
@ -364,8 +364,8 @@ on the `agent-configs` org, both under
`https://<swarm.domain>/webhook/forge/`. `https://<swarm.domain>/webhook/forge/`.
The controller **interprets** a delivery and sends hives a specific The controller **interprets** a delivery and sends hives a specific
message — *the knowledge repo changed*, *deploy agent `foo` at rev message — _the knowledge repo changed_, _deploy agent `foo` at rev
`abc123`* — rather than forwarding forge payloads for each hive to `abc123`_ — rather than forwarding forge payloads for each hive to
re-derive. Approval happens once, at the swarm level: a hive receives a re-derive. Approval happens once, at the swarm level: a hive receives a
decision, not an event to adjudicate. decision, not an event to adjudicate.

Some files were not shown because too many files have changed in this diff Show more