docs: fix 329 Microsoft.Contractions hits with proper safety checks

Apply contraction fixes across ~40 doc files (setup, integrations, lifecycle,
networking, scheduler, swarm, tools, trust-boundary, UI, etc.). Skipped 14 hits:
- 10 where words appear in ALL CAPS for deliberate emphasis (is NOT, do NOT, etc.)
- 4 where text could not be safely located due to markdown formatting or column position

Applied via systematic scan with checks for fenced code blocks, inline code spans,
and intentional caps. Preserves sentence-initial capitalization throughout.
This commit is contained in:
iris 2026-09-07 14:28:23 +02:00 committed by mara
commit 78021ce982
38 changed files with 320 additions and 320 deletions

View file

@ -54,7 +54,7 @@ declarations.
agent?** → [`integrations/matrix.md`](integrations/matrix.md) (the homeserver);
[`tools/matrix.md`](tools/matrix.md) for the MCP tool surface and
`hyperhive.matrixAccounts`.
- **How do I give an agent a GitHub account (`gh` + `git push`)? How is
- **How do I give an agent a GitHub account (`gh` + `git push`)? how's
the PAT injected?** → [`integrations/github.md`](integrations/github.md)
(operator content up top; the `gh`/git-push + notification-poller
mechanics are in a collapsed "Implementation" section at the bottom).
@ -79,7 +79,7 @@ declarations.
## Scheduler, CI, observability
- **What is the job queue, as a general idea (not hive-c0re specifics)?**
- **what's the job queue, as a general idea (not hive-c0re specifics)?**
[`scheduler/jobq.md`](scheduler/jobq.md) — operator-facing, no implementation detail.
- **How does the rebuild queue work? What are the concrete step kinds,
queue sources, scheduler internals?** → [`scheduler/coordinator.md`](scheduler/coordinator.md).

View file

@ -127,7 +127,7 @@ other agents don't:
equivalent on their own socket.
- **Storage/mounts** — only the manager container gets
`/var/lib/hyperhive/agents` bind-mounted RW at `/agents` (so it can
manage any agent's state dir — config is not authored there, since a
manage any agent's state dir — config isn't authored there, since a
real config change is a PR from a clone), plus RO mounts for
`/applied` (diff against what's deployed) and `/meta` (system-wide
deploy log). Planned: each agent gets RW to `/agents/<descendant>/`

View file

@ -27,7 +27,7 @@ CLI) before it takes effect. What you'll see, and what to do with it:
- **New agent** (`InitConfig` then `Spawn`) — creating a brand-new
agent is two approvals. `InitConfig` creates the config repo and
seeds it from a template; `Spawn` creates the container from that
config. Tailoring the template first is not a separate mechanism —
config. Tailoring the template first isn't a separate mechanism —
it's the config-change flow above, a PR you review like any other. Every later change goes through the config-change flow
above — there's no repeat "spawn" for an existing agent.
- **Meta/flake update** (`UpdateMetaInputs`) — an agent asked to bump
@ -66,11 +66,11 @@ request.
`hive-forge` — the same way it would change any other repo.
The bind-mounted `/agents/<name>/config/` is a **copy for reading** a
config, not the tree to edit: authoring in place there produces no PR
and no approval. (It is currently mounted read-write, which is a
and no approval. (it's currently mounted read-write, which is a
defect tracked separately, not an authoring path.)
Branch protection (push/merge allowlist = `core`, approvals allowlist
= operator team; see "Forge mirror" and #1787) makes the agent a
write collaborator that **cannot merge its own config PR**.
write collaborator that **can't merge its own config PR**.
2. hive-c0re's `/webhook/config-pr` endpoint receives the Forgejo
`pull_request` event (opened / synchronized / reopened) and queues a
`MergeConfigPr` approval; a poll fallback catches any missed webhook.
@ -138,7 +138,7 @@ in-container store. The operator then **spawns** the agent (the
`Spawn` approval / `◆ R3QU3ST SP4WN` button), which creates the
container from that config.
Changing what the template seeded is not a special case: like every
Changing what the template seeded isn't a special case: like every
later change, it's a PR on that config repo (`MergeConfigPr`), made
from a clone, reviewed and approved by the operator. The PR flow is
the one path — an operator can equally drive both steps herself
@ -240,7 +240,7 @@ Transient broker errors (sqlite lock contention, etc.) get the same
- **Recurring rows** re-arm to the next interval slot — the retry
self-heals on the next worker pass.
- **One-shot rows** are deleted unconditionally after their single
fan-out pass; a broker error on a one-shot is not retried (the
fan-out pass; a broker error on a one-shot isn't retried (the
operator advisory and `last_result` are the only audit trail).
### Reminder delivery: file-path semantics
@ -259,7 +259,7 @@ container, e.g. `/agents/<name>/state/foo.md`). On delivery hive-c0re:
dir is canonicalized and re-verified to live under the agent's host
state root. The final file is opened with
`O_NOFOLLOW | O_CREAT | O_TRUNC` so an existing symlink at the
basename cannot redirect the write to an arbitrary host path.
basename can't redirect the write to an arbitrary host path.
4. **Writes the body to disk** and delivers a short pointer message in its
place, keeping the agent's inbox / wake-prompt small while the bulky
payload is read out of band.
@ -279,7 +279,7 @@ reuses prior config + login. With `purge = true` the agent's
`/var/lib/hyperhive/{agents,applied}/<name>/` trees are also
wiped (config history + creds + notes gone forever). The
root/bootstrap container is destroyable like any other — hive-c0re
recreates it on the next startup if it's absent, so destroying it is
recreates it on the next startup if it's absent, so destroying it's
transient.
## Meta flake
@ -390,7 +390,7 @@ Why two physical repos: the submitting agent's `/agents/<n>/config/` is
RW — a buggy or hostile agent can `git clean -fdx` its own
proposed tree. The applied repo is never bind-mounted (except
the read-only `.git` exposure described below) so a destructive
move inside the container cannot reach it.
move inside the container can't reach it.
The container's `--flake` ref is `/var/lib/hyperhive/meta#<name>`
(see "Meta flake" above). The agent's own `applied/<n>/flake.nix`
@ -493,7 +493,7 @@ Every parent agent's container has its **direct children's** config
repos bind-mounted **read-only** (topology-driven:
`hive-c0re/src/lifecycle/host_config.rs` calls `bind_child_agent_dirs`
for each entry in
`topology::children_of(agent_name)`). It is a copy to *read* a child's
`topology::children_of(agent_name)`). it's a copy to *read* a child's
current config — not an editing surface.
An agent with the `approvals` tool group submits a change the same way
@ -514,7 +514,7 @@ bind mounts via `set_nspawn_flags`:
deploy flake.
The root agent holds this role; a sub-manager that only manages a
subtree does not, and only has its direct children's config dirs.
subtree doesn't, and only has its direct children's config dirs.
Each proposed repo (`/agents/<n>/config/`) is pre-configured
with `applied` as a git remote pointing at
@ -534,7 +534,7 @@ cat /meta/flake.lock | jq '.nodes | with_entries(select(.key | startswith("agent
```
The RO binds block push at the kernel level — git plumbing inside the
container cannot corrupt either authoritative repo.
container can't corrupt either authoritative repo.
## Startup migrations (older hosts)

View file

@ -340,7 +340,7 @@ A parent agent gets each direct child's `state` dir bind-mounted
which includes writing into a child's state for recovery (e.g. seeding
notes, clearing a stuck sentinel) as well as reading it.
**`harness` is not mounted at all.** It holds the child's own runtime
**`harness` isn't mounted at all.** It holds the child's own runtime
material — `bash-tasks/`, the turn-stats and event sqlite dbs — and
nothing argues for a parent reading it, let alone writing it. hive-c0re
reads a child's harness dir **directly on the host** when it wants
@ -404,7 +404,7 @@ Contents:
hive-wide `services.hyperhive.c0re.agentCpuQuota` / `agentMemoryMax`,
so an agent can override only its memory and still track the hive
default for CPU. The `CPUWeight=` / `IOWeight=` shares in the same
drop-in have **no** per-agent override — they are hive-wide only and
drop-in have **no** per-agent override — they're hive-wide only and
come straight off `HiveEnv`, so this file has no field for them.
The root agent has the meta dir RO-mounted at `/meta/`.
@ -476,14 +476,14 @@ Only present on the one host running
so it survives restarts and redeploys.
- `webhook-secret` — the HMAC key the swarm's forge webhooks are signed
with. **Keep it.** It is handed to Forgejo when a hook is registered,
with. **Keep it.** it's handed to Forgejo when a hook is registered,
so replacing the file means every subsequent delivery fails
verification until the hook is re-registered with the new value. It is
verification until the hook is re-registered with the new value. it's
generated automatically on first start; there is nothing to configure.
If the file is unreadable at startup the daemon still starts and logs
`webhook secret unavailable`; the webhook endpoint then answers 503
rather than accepting deliveries it cannot verify. Everything else the
rather than accepting deliveries it can't verify. Everything else the
controller serves is unaffected.
## Run-time dirs

View file

@ -10,9 +10,9 @@ sequence. All `hivectl` commands below run as **root on the host** (not
inside an agent container); the `request_*` steps run from ruth's own
turn via the MCP tools.
**Bringing up a hive that does not host its own swarm services?** Read
**Bringing up a hive that doesn't host its own swarm services?** Read
[`swarm/secrets.md`](../swarm/secrets.md) first. Everything below assumes
each credential is generated where it is read, which is true on an
each credential is generated where it's read, which is true on an
all-local deploy and not otherwise — that page says which files an
operator has to place, and where.
@ -73,8 +73,8 @@ half, `.clientCertFile`, `.clientKeyFile` and `.serverCaFile` at the reader's.
Those are `mkDefault`s, so naming your own paths wins. Do that when your
certificates come from a real internal CA; the store has no opinion about
which. A hive that does **not** deploy the store names the reader's three
itself: that leaf is issued out of band, and it is the one credential the store
cannot hand you, being what opens it. ⚠️ Not the gateway's HTTPS certificates and not the hive CA — this is
itself: that leaf is issued out of band, and it's the one credential the store
can't hand you, being what opens it. ⚠️ Not the gateway's HTTPS certificates and not the hive CA — this is
**mTLS between services and the store**, a separate trust domain, because a
store that took its identity from an authority it will itself distribute could
never come up before that authority.
@ -95,7 +95,7 @@ start. Skipping this step looks like a broken proxy.
swarmctl user add mara --display-name Mara --email mara@example.com --group admins
```
⚠️ **Keep `--group admins`.** It is not decoration: operator-only
⚠️ **Keep `--group admins`.** it's not decoration: operator-only
surfaces (the swarm UI below) are gated on that group, and an account
without it authenticates successfully and is then refused — which reads
like a broken login rather than a missing group.
@ -109,14 +109,14 @@ Detail, including what the password is and why this stays manual:
### 5 · Swarm UI (only when `deploy.swarm-ui`, on by default with the controller)
Nothing to run — it is served on the swarm apex
Nothing to run — it's served on the swarm apex
(`https://<swarm.domain>/`) as soon as the host rebuilds. Two things
decide whether you can actually open it:
- **You are in `admins`** (step 3). The gateway asks authelia whether
you have a session; the rule that makes it mean _operator_ wants the
group. Without it you log in and still get bounced.
- **The name resolves to this host.** It is published to the hive's own
- **The name resolves to this host.** it's published to the hive's own
resolver and to `/etc/hosts` when `gateway.localHostsEntry` is on; from
anywhere else it needs a real DNS record like any other public name.

View file

@ -134,7 +134,7 @@ agent's entire still-unread backlog as a flood of fresh wakes. Forge's
own read-state is the only durable record this design needs.
**Read-before-comment guard doesn't block a fresh wake.** hive-forge's
read-before-comment guard (which keys off forge unread-state) does not
read-before-comment guard (which keys off forge unread-state) doesn't
force the agent to view a thread before commenting on it: the broker
wake already carries the notification body, so _delivery is the read_.
An agent that wants the full thread still runs `hive-forge comments` /
@ -144,7 +144,7 @@ An agent that wants the full thread still runs `hive-forge comments` /
id → last-delivered `updated_at`) guards the narrow window where a
mark-read call _transiently fails_ and the thread reappears unread in
the next poll before its `updated_at` bumps — so a flaky PATCH doesn't
re-fire the wake. It is **not persisted** and resets on restart (forge
re-fire the wake. it's **not persisted** and resets on restart (forge
read-state covers the durable case). Each poll prunes it to the ids in
the single `limit=UNREAD_FETCH_LIMIT` (50) fetch page, so it can never
exceed that many entries (a debug assertion pins the invariant; the
@ -222,7 +222,7 @@ pipeline steps in order:
The strict ATX rule is deliberate: `\#tag` and `#tag` render
identically, so an over-eager escape just adds visual clutter
without changing behavior. Setext-style headings (`title\n====`)
are not handled — rarer in practice, would need multi-line
aren't handled — rarer in practice, would need multi-line
lookahead.
### Wrapper format
@ -240,7 +240,7 @@ Five shapes, distinguished by the notification's classification:
The issue/PR's own description is deliberately never embedded here (only a
comment/review body is — that's the actual new content a notification
exists to surface): repeating a subject's own description on every "new"
or "activity on" wake for it is noise the agent already has via the URL,
or "activity on" wake for it's noise the agent already has via the URL,
not context (mara, triage: "just the issue title should be enough").
Review labels come from the Forgejo `state` field: `APPROVED`

View file

@ -28,7 +28,7 @@ into every agent, so no agent ships the `gh` wrapper or credential helper.
(`hyperhive.github.enable` also exists per-agent for completeness, but the
hive-wide host switch is the intended control.)
github.com only. The token **value** never touches nix — it is written to
github.com only. The token **value** never touches nix — it's written to
`<state>/github-token` separately (see [Provisioning](#provisioning)).
## Provisioning
@ -88,7 +88,7 @@ minimal environment that wouldn't carry one.
### Notifications
`hive-github-notify` polls github.com for the agent, turning each unread
notification thread into a todo. It is a **separate binary and a
notification thread into a todo. it's a **separate binary and a
separate systemd unit** from the internal forge's poller
(`hive-forge-notify`, see [forge.md](forge.md#notification-poller-hive-forge-notifysrcnotifyrs)),
installed by `nix/agent-modules/github.nix` under
@ -99,12 +99,12 @@ path, not a new package.
Two units rather than one daemon with two loops, because it puts the
decision in nix: a hive built without this module has **no github poller
in its closure at all**, which is what makes GitHub access separable
rather than merely switched off. It is also why this is not a cargo
rather than merely switched off. it's also why this isn't a cargo
feature — a feature would unify across the workspace and cost every
crate its build cache.
At runtime the poller needs the PAT above. No PAT, no polling: the unit
logs why and exits 0, which is why it is `Restart = on-failure` and
logs why and exits 0, which is why it's `Restart = on-failure` and
never `always` — a clean exit on a PAT-less agent must not become a
restart loop.
@ -114,7 +114,7 @@ parse serves both: `id`, `repository.full_name`,
for field. The two real differences are absorbed by lenient
deserializers — GitHub sends the thread id as a _string_ where Forgejo
sends a number, and says `PullRequest` where Forgejo says `Pull`. Todo
keys are prefixed `gh:<id>` so a github thread id cannot collide with a
keys are prefixed `gh:<id>` so a github thread id can't collide with a
forge one.
Two host differences worth knowing before touching this code:
@ -122,15 +122,15 @@ Two host differences worth knowing before touching this code:
- **Auth scheme, not just value.** Forgejo takes
`Authorization: token <t>`; GitHub wants `Bearer <t>` plus `Accept:
application/vnd.github+json`, `X-GitHub-Api-Version` and a
`User-Agent`. Sending Forgejo's form to GitHub does not error — it
`User-Agent`. Sending Forgejo's form to GitHub doesn't error — it
authenticates as _nobody_ and silently drops to the unauthenticated
rate limit. The cheap way to tell the two apart is the rate-limit
header: `x-ratelimit-remaining` near 5000 is an authenticated user,
near 60 is anonymous.
- **GitHub sets the cadence.** It returns `X-Poll-Interval` (60s in
practice, slower than our own tick) and rate-limits callers who ignore
it, so the loop re-arms to the server's interval whenever that is
_slower_ than ours. A hint faster than our own tick is not a reason to
it, so the loop re-arms to the server's interval whenever that's
_slower_ than ours. A hint faster than our own tick isn't a reason to
poll harder.
⚠️ **This needs the `notifications` scope on the PAT.** A token minted

View file

@ -44,13 +44,13 @@ hive-c0re maintains the local clone at
event to every hive over the swarm queue and each hive runs `git
pull`, so agents see the new content on their next turn.
A hive that is offline when the event is sent does not get it on
A hive that's offline when the event is sent doesn't get it on
reconnect — the periodic pull below is what closes that gap. So one
hive briefly showing older `/knowledge` content than another is
expected, and resolves by itself within the fallback interval.
**Do not add a per-hive hook.** A webhook has exactly one target
URL, so a second registration against the same repo does not add a
**don't add a per-hive hook.** A webhook has exactly one target
URL, so a second registration against the same repo doesn't add a
recipient — it takes delivery away from whoever registered first.
Earlier versions had each hive register its own; hive-c0re now
removes its own leftover at startup, so no operator step is needed
@ -59,7 +59,7 @@ pull`, so agents see the new content on their next turn.
2. **Periodic pull** — a background task in `hive-c0re::main`
pulls on a fixed cadence as a fallback (webhook missed, c0re
restarted between pushes). The pull is best-effort — a failure
logs a warning and does not affect the rest of the daemon.
logs a warning and doesn't affect the rest of the daemon.
Both paths share the same `knowledge::pull()` function, which also
handles the change notice below — neither path can forget to wire it

View file

@ -31,7 +31,7 @@ Two distinct hostnames:
- **`serverName`** — matrix-spec `server_name`, embedded
*irrevocably* in every `@user:<server_name>` and `!room:<server_name>`
identifier minted on this homeserver. Cannot be changed later
identifier minted on this homeserver. can't be changed later
without abandoning every account and chat history. Defaults to the
bare `services.hyperhive.swarm.domain`; clients auto-discover the
actual API endpoint via the `.well-known/matrix/{client,server}`
@ -46,9 +46,9 @@ homeserver: tying its identity to a single hive's domain would make
relocating the container between hives look like a different
homeserver.
⚠️ **They are still not interchangeable, and the difference is the
⚠️ **they're still not interchangeable, and the difference is the
cost of changing one.** `gatewayHost` is a routing detail clients
rediscover through `.well-known`, so it is safe to move on a running
rediscover through `.well-known`, so it's safe to move on a running
deployment. `serverName` is baked into every user and room id, so
adopting a new one does **not** rename the existing users and rooms —
it strands them, because their ids still name a homeserver that no
@ -71,8 +71,8 @@ services.hyperhive.swarm.matrix = {
A rebuild on a host that already has a homeserver prints a
`hive-matrix: WARNING — … serverName is unset` line when this is missing,
naming the value it is about to default to. That warning is why this
section exists; it never fails the rebuild, so it is on you to act on it
naming the value it's about to default to. That warning is why this
section exists; it never fails the rebuild, so it's on you to act on it
before the ids are minted.
## Default-closed firewall
@ -147,13 +147,13 @@ Initial rollout settings:
external / federated homeservers or to keep contents opaque to the
homeserver admin. **The agent matrix client always supports decryption
regardless of this flag** — it uses the `e2e-encryption` feature of
`matrix-sdk` so it can read encrypted rooms it is invited to even when
`matrix-sdk` so it can read encrypted rooms it's invited to even when
this homeserver doesn't permit room encryption. Crypto keys are stored
in the per-agent sqlite store under the state dir and persist across
restarts (lost on `--purge`). `read_room` decrypts via
`room.messages()` — UTD events surface as `event_type =
"m.room.encrypted"` with `body = "[unable to decrypt]"`.
Cross-signing and automatic key backup are not enabled for the first
Cross-signing and automatic key backup aren't enabled for the first
pass: static bearer-token bot accounts can't bootstrap cross-signing
without MSC3967.
@ -227,7 +227,7 @@ homeserver at all rather than of a setting:
tuwunel reads its identity providers from the config file, so a
half-configured one can stop the homeserver from starting outright
rather than merely hiding a login button. On a host that also runs
the swarm's authelia it is wired up for you.
the swarm's authelia it's wired up for you.
- **`swarm.authelia.url` is required** — without a provider URL there
is nothing to discover against.
- **`gatewayHost != null` is required** — the SSO callback URL is

View file

@ -107,7 +107,7 @@ now set unconditionally for every agent. The mechanism:
The dir is `0751`, owned by the agent's container uid/gid, so
nginx reaches `web.sock` through `o=--x` (traverse) and the socket's
own `0666`. The gateway is one of three principals sharing that dir
and does not own its ownership rules — see
and doesn't own its ownership rules — see
[`docs/trust-boundary/boundary.md`](../trust-boundary/boundary.md#the-per-agent-socket-dir).
3. **Marker gate**. After successful `bind_unix`, the harness drops
`<dir>/hyperhive-socket-bound` next to the socket. c0re's
@ -118,13 +118,13 @@ now set unconditionally for every agent. The mechanism:
`/var/lib/hive-gateway/conf/agents.conf` — a plain nginx include
file with one `location /agent/<name>/` block per agent. Always
a UDS upstream (`http://unix:/run/hive-agent/<name>/web.sock:/`);
if the socket is not yet bound, nginx returns 502 caught by the
if the socket isn't yet bound, nginx returns 502 caught by the
`error_page 502 503 504 = /__hive_agent_unreachable` directive.
nginx includes `/var/lib/hive-gateway/conf/agents.conf` — the same
path c0re writes, since both run on the host.
After each write, c0re triggers the appropriate nginx action via
`hive-priv` (which is root; hive-c0re runs as the unprivileged
`hive-core` user and cannot act on a system unit).
`hive-core` user and can't act on a system unit).
`hive-priv` queries `ActiveState` and dispatches:
- active → `systemctl reload nginx` (SIGHUP, zero-downtime)
- failed → `systemctl reset-failed nginx` + `systemctl start nginx`
@ -206,7 +206,7 @@ On by default, and listens on `httpsPort` (default 443) on every vhost beside th
The issuer is a **host-held hive CA**, not a bare self-signed leaf. A host service (`hive-tls-ca.service`, from the `hive-tls` module) generates a long-lived CA (`services.hyperhive.deploy.hive-controller.tls.caValidityDays`, default ~20y) under `services.hyperhive.deploy.hive-controller.tls.stateDir` (default `/var/lib/hive-tls`), then signs a gateway **leaf** (`leafValidityDays`, default 30d) with it. `hive-gateway-self-signed-cert` then imports the leaf into nginx's state dir (`/var/lib/hive-gateway/tls/{cert,key}.pem`).
⚠️ **Do not collapse that import unit into pointing nginx at the CA dir.**
⚠️ **don't collapse that import unit into pointing nginx at the CA dir.**
It does two jobs, and skipping it has taken the gateway down in production
before. It re-modes the leaf (`hive-tls-ca` writes the key `0600
root:root`; nginx's pre-start `nginx -t` runs as the *nginx user*, so a
@ -216,13 +216,13 @@ swarm-services fallback below is for.
**Why a CA, not a bare leaf**: a bare self-signed leaf is its own trust anchor, so every regeneration is a new anchor every consumer must re-trust — and a runtime-generated leaf can't be wired into an agent's build-time trust store at all. With a stable CA, agents and federation peers trust it *once*; leaf rotation never re-breaks them.
**What consumers trust**: `trust-bundle.pem` in the same state dir, not `ca.pem`. The hive CA is itself issued under the swarm root ([`swarm/ca.md`](../swarm/ca.md) has the hierarchy), and an intermediate is not a chain a verifier can terminate at — so the bundle carries the hive CA plus whatever it is rooted at. nginx is handed the leaf with the hive CA appended for the same reason. Everything that trusts the hive's TLS reads the bundle: agents (via `security.pki.certificateFiles`), the CI and forge containers, and a federating peer.
**What consumers trust**: `trust-bundle.pem` in the same state dir, not `ca.pem`. The hive CA is itself issued under the swarm root ([`swarm/ca.md`](../swarm/ca.md) has the hierarchy), and an intermediate isn't a chain a verifier can terminate at — so the bundle carries the hive CA plus whatever it's rooted at. nginx is handed the leaf with the hive CA appended for the same reason. Everything that trusts the hive's TLS reads the bundle: agents (via `security.pki.certificateFiles`), the CI and forge containers, and a federating peer.
**Why on by default**: matrix-dart-sdk (FluffyChat's SDK) hardcodes `https://<host>/.well-known/matrix/client` for homeserver discovery and refuses to fall back to plain http. Without TLS the browser client cannot bootstrap.
**Why on by default**: matrix-dart-sdk (FluffyChat's SDK) hardcodes `https://<host>/.well-known/matrix/client` for homeserver discovery and refuses to fall back to plain http. Without TLS the browser client can't bootstrap.
**Cert shape**: leaf subject CN = bare hive domain; subjectAltName is `<hive>` plus wildcard `*.<hive>`, so all current and future sub-domain vhosts validate under the same leaf + the hive CA. A swarm service whose name is *not* under this hive's domain cannot be added here — the hive CA is name-constrained to `<hive>`, and a violating SAN invalidates the whole leaf, not just that name. Those names get the swarm-services leaf instead ([`swarm/ca.md`](../swarm/ca.md)).
**Cert shape**: leaf subject CN = bare hive domain; subjectAltName is `<hive>` plus wildcard `*.<hive>`, so all current and future sub-domain vhosts validate under the same leaf + the hive CA. A swarm service whose name is *not* under this hive's domain can't be added here — the hive CA is name-constrained to `<hive>`, and a violating SAN invalidates the whole leaf, not just that name. Those names get the swarm-services leaf instead ([`swarm/ca.md`](../swarm/ca.md)).
**Rotation**: `hive-tls-ca.service` is idempotent — it re-signs the leaf when it is missing or within 30 days of expiry, always under the same CA (so consumer trust is undisturbed). The CA itself is regenerated only if missing or already expired. To force a leaf rotation, delete `gateway.pem` under the state dir and restart the unit, then reload `nginx`.
**Rotation**: `hive-tls-ca.service` is idempotent — it re-signs the leaf when it's missing or within 30 days of expiry, always under the same CA (so consumer trust is undisturbed). The CA itself is regenerated only if missing or already expired. To force a leaf rotation, delete `gateway.pem` under the state dir and restart the unit, then reload `nginx`.
**Cert prompts**: browsers still warn once per host until the hive's `trust-bundle.pem` is added to the browser/OS trust store (an anchor, not the leaf, is the thing to trust). Agent trust is wired separately (see the agent-trust work for `/run/hive-ca`).
@ -242,7 +242,7 @@ nginx reads the directory directly and uses `cert.pem` + `key.pem` (override `tl
`tls.certDir` and `tls.acme.enable` set together is an assertion error.
**Key file permissions**: nixpkgs's `security.acme` outputs private keys as `0640 root:acme` by default. nginx runs as the `nginx` user and cannot read a key with that ownership. Fix with:
**Key file permissions**: nixpkgs's `security.acme` outputs private keys as `0640 root:acme` by default. nginx runs as the `nginx` user and can't read a key with that ownership. Fix with:
```nix
security.acme.certs."example.com".group = "nginx";
@ -258,7 +258,7 @@ for an operator who wants their own TLS terminator:
- give the gateway the real cert via `tls.certDir` (or `tls.acme`) so it
serves proper TLS directly — no separate proxy needed; or
- front it over a **unix socket** rather than a plain-http TCP port (the
intended direction for "bring your own proxy" — the gateway is not meant
intended direction for "bring your own proxy" — the gateway isn't meant
to expose an unencrypted TCP upstream).
Because of this, `.well-known/matrix/{client,server}` discovery responses
@ -369,7 +369,7 @@ that path.
services.hyperhive.swarm.forge.rootUrl = "https://forge.example.com/";
```
`rootUrl` (default **null**) overrides the Forgejo `ROOT_URL` that is
`rootUrl` (default **null**) overrides the Forgejo `ROOT_URL` that's
auto-derived from `forge.domain` + gateway state. The auto-derivation
covers most cases:
@ -455,7 +455,7 @@ legacy single-proxy block and all traffic is forwarded to the agent
daemon as before.
**`extraFiles`**: per-agent `hyperhive.frontend.extraFiles` are in
`mergedDist`, not in the base `cfg.frontend` dist. They are not under
`mergedDist`, not in the base `cfg.frontend` dist. they're not under
the nix-store `alias` path, so requests for them fall through
`try_files` to `@<name>_dynamic` and are served by the agent daemon
as before.
@ -493,7 +493,7 @@ default status code would point at the wrong component. The per-agent
routes qualify (a 502 there means the harness is restarting, not that
the gateway is broken), and so does `auth.<swarm>` — a dead authelia
upstream almost always means the user store was never bootstrapped, and
a bare 502 blames the proxy, which is the one part that is working.
a bare 502 blames the proxy, which is the one part that's working.
Forge / matrix / fluffychat still get nginx defaults: their upstreams
being down means what the status code says, so a themed page would add
@ -542,7 +542,7 @@ The daemon hashes passwords with BCrypt (cost 12) and writes
`$2y$`-prefixed hashes that nginx accepts natively. No external
`htpasswd` binary is required.
**What is not gated:** per-agent UI routes emitted into `agents.conf`
**what's not gated:** per-agent UI routes emitted into `agents.conf`
(served under `/agent/<name>/`) inherit no auth from `/` — nginx
applies `auth_basic` per-location. Full per-agent coverage is a
follow-up.
@ -618,18 +618,18 @@ the opposite arm run as a control:
- `verify` + `depth` — the chain is leaf -> intermediate -> root.
- `trusted_cert` — the bundle; nginx reads ALL certs in the file, which
the bundle's own doc warns is not true of every consumer.
the bundle's own doc warns isn't true of every consumer.
- `ssl_name` — checks the HOSTNAME too. Without it a chain-only check
accepts any certificate this CA ever signed, and for an internal CA
that is every service on the hive.
- `server_name on` — sends SNI, or the far end cannot pick a cert.
that's every service on the hive.
- `server_name on` — sends SNI, or the far end can't pick a cert.
**⚠️ Session-cache footgun**: `proxy_ssl_session_reuse` is left at its
default (on), deliberately — this is used on per-request auth
subrequests, so the handshake it avoids is paid on every request. Worth
knowing when testing though: the session cache is keyed by upstream
address and NOT by trust config, so two locations pointing at one
upstream with different trust do not verify independently.
upstream with different trust don't verify independently.
**⚠️ Host-header clobber footgun**: `verifiedProxyTo` also pins `Host`
(and reinstates the rest of nginx's `recommendedProxySettings` header

View file

@ -75,7 +75,7 @@ The flows, end to end:
`/run/hive/mcp.sock`, privileged helper `/run/hive/priv.sock`,
operator admin `/run/hyperhive/host.sock`, and the per-agent UI
sockets under `/run/hive-agent/` are unix domain sockets
bind-mounted through the VFS; private netns does not affect them.
bind-mounted through the VFS; private netns doesn't affect them.
## Container shape (where dnsmasq lives)
@ -100,8 +100,8 @@ listener on `bridgeIp` is on the host's bridge interface.
```
Requires `services.hyperhive.domain` to be set — the dnsmasq resolver
is authoritative for `<hive-domain>` and its sub-domains. You do not
write it: it is read from this hive's entry in the swarm directory
is authoritative for `<hive-domain>` and its sub-domains. You don't
write it: it's read from this hive's entry in the swarm directory
(`docs/swarm/README.md` § Hive identity config).
## Bridge addressing
@ -159,7 +159,7 @@ agent containers.
interfaces stay closed. The hive resolver isn't an external-facing
service.
- Port 67 (UDP) admits DHCP requests to the dnsmasq pool. dnsmasq
receives DHCP via a regular UDP socket (it does not use a
receives DHCP via a regular UDP socket (it doesn't use a
netfilter-bypassing raw socket), so the hole is mandatory — without
it containers never get a lease and fall back to 169.254.x.x.
- Ports 80 and 443 let isolated agents reach nginx (gateway
@ -174,7 +174,7 @@ boot its `firewall-start` flushes the `nixos-fw` chains, rebuilds them
from the container's (empty) port list, and deletes the host's
`nixos-nat-*` chains without recreating them, silently wiping the
bridge holes above plus the agents' NAT. Private-netns containers
(agents, hive-ci) may keep their own firewall — it is scoped to their
(agents, hive-ci) may keep their own firewall — it's scoped to their
namespace.
### Reaching host services (`exposeHostPorts`)
@ -225,7 +225,7 @@ address arithmetic.
`PRIVATE_NETWORK=1`, `LOCAL_ADDRESS=` (empty), `HOST_ADDRESS=<bridge-ip>`,
and `HOST_BRIDGE=<bridgeName>` via `lifecycle::set_nspawn_flags` when
creating or updating containers. Both variables are validated **once at
daemon startup**, not per container: they are process-global, so a
daemon startup**, not per container: they're process-global, so a
missing or malformed value is a misconfigured daemon rather than one bad
container, and failing at boot gives a single diagnostic instead of one
per agent. There is no non-isolated mode to fall back to. `LOCAL_ADDRESS` is left empty so the
@ -236,7 +236,7 @@ to all containers — agents and service containers alike.
`HOST_ADDRESS` is the bridge gateway IP (the address part of
`HIVE_NETWORK_SUBNET`, via `lifecycle::bridge_gateway_ip` — taken verbatim
so a non-`.1` operator override still resolves to wherever the bridge
actually lives). It is **load-bearing**: nixos-container's container-side
actually lives). it's **load-bearing**: nixos-container's container-side
network setup only installs a default route (`ip route add default via
$HOST_ADDRESS`) when `HOST_ADDRESS` is non-empty. In bridge mode the
host-side address/route setup is skipped, so writing it only affects the
@ -248,7 +248,7 @@ but no path off the bridge subnet (no internet, no `api.anthropic.com`).
nixos-container copies the **host's** `/etc/resolv.conf` into the container
at every start. The host resolver (e.g. `127.0.0.53` from systemd-resolved,
or a LAN router) is unreachable from a private netns and isn't
authoritative for the hive's own zones, so it is replaced with the
authoritative for the hive's own zones, so it's replaced with the
bridge dnsmasq at boot. Because the copy happens on every start, a
declarative `environment.etc."resolv.conf"` would be clobbered — so the
wiring is runtime:
@ -257,7 +257,7 @@ wiring is runtime:
gateway IP) into each container's `/etc`.
- the `hyperhive-isolated-dns` oneshot (`nix/agent-modules/network.nix`), gated on that
marker, rewrites `/etc/resolv.conf` to `nameserver <gateway-ip>` at boot.
It is ordered `before` the harness (`hive-ag3nt`), the matrix daemon, and
it's ordered `before` the harness (`hive-ag3nt`), the matrix daemon, and
`tea-login` so the resolver is correct before the first DNS lookup.
**Why isolation is safe**: hive-c0re's control-plane sockets are unix

View file

@ -4,14 +4,14 @@ The swarm's `btrfs receive` endpoint. Hives push agent snapshots to it
over the WireGuard mesh; a destination hive later pulls one back to
complete a migration.
Two things it is not, both worth stating because both are common
Two things it's not, both worth stating because both are common
assumptions:
- **It is not the swarm controller**, and does not depend on one. It is
- **it's not the swarm controller**, and doesn't depend on one. it's
a NixOS host role: a btrfs subvolume tree, a socket-activated
receiver, and the `wg-hive` interface the swarm module already brings
up. That is why it can be deployed before any controller exists.
- **It is not a backup product.** It happens to hold the data a backup
up. that's why it can be deployed before any controller exists.
- **it's not a backup product.** It happens to hold the data a backup
would hold, and it should be operated accordingly (see
[Operating it](#operating-it)) --- but nothing in it does scheduling,
verification, or restore orchestration.
@ -56,15 +56,15 @@ services.hyperhive.swarm.snapshotStore = {
Two deliberate asymmetries in that pair, both liable to be misread as
inconsistency:
- **`address` has no default.** It is a deployment fact a pushing hive
cannot derive, and a wrong guess means streaming an agent's state at
- **`address` has no default.** it's a deployment fact a pushing hive
can't derive, and a wrong guess means streaming an agent's state at
whatever happens to answer. Unset, a push fails naming this option.
- **`port` does default** (`51821`), because it is a convention both
- **`port` does default** (`51821`), because it's a convention both
ends read from the same option docs --- a default there is
coordination, not a guess.
Note the option lives under `swarm.*` while the receiving host's lives
under `services.hyperhive.snapshotStore`. That is the distinction the
under `services.hyperhive.snapshotStore`. that's the distinction the
two namespaces carry throughout: `swarm.*` describes _the swarm_ as seen
from this host, and a bare `services.hyperhive.<service>` describes _a
role this host performs_. A store host sets both --- one to run the
@ -78,7 +78,7 @@ and no credential argument, because the mesh is the authentication.
## The mesh is the authentication
There are no certificates here, and no key material of its own. That is
There are no certificates here, and no key material of its own. that's
deliberate rather than an omission.
WireGuard's cryptokey routing already binds a peer's source address to
@ -97,16 +97,16 @@ you need to move an agent.
The destination is keyed by **agent**.
This is not cosmetic. After a migration, an agent's next incremental
This isn't cosmetic. After a migration, an agent's next incremental
send arrives from a _different_ hive than the previous one. Keying by
hive would split that agent's snapshot chain across two directories,
and `btrfs send -p` would fail to find its parent --- breaking exactly
the case the store exists to serve.
## What the sender can and cannot choose
## What the sender can and can't choose
A `btrfs send` stream carries no notion of _which agent_ it belongs to,
and the subvolume name inside it is chosen by the sender. So the
and the subvolume name inside it's chosen by the sender. So the
protocol is one `agent <name>` header line, then the raw stream.
The rule that matters:
@ -116,7 +116,7 @@ The rule that matters:
Validation is an allowlist --- `[A-Za-z0-9_-]+` and nothing else. No
slash and no dot means neither directory traversal nor an absolute path
can survive it. It is deliberately an allowlist and not a list of
can survive it. it's deliberately an allowlist and not a list of
forbidden characters: a blocklist only ever excludes the attacks
somebody already thought of.
@ -128,7 +128,7 @@ address being set are assertions, not documentation --- bound to
`0.0.0.0` this socket is an unauthenticated remote write into agent
state.
Binding is not sufficient on its own. NixOS's firewall is default-deny
Binding isn't sufficient on its own. NixOS's firewall is default-deny
and filters in netfilter, _before_ a packet reaches a bound socket, so
the port is opened explicitly --- and scoped to the mesh interface:
@ -165,7 +165,7 @@ nobody runs it day to day, so it rots and is discovered broken in
production.
⚠️ **The assumption to keep true over time:** the store host runs
nothing else. That is true on day one and quietly false the day someone
nothing else. that's true on day one and quietly false the day someone
notices the box has spare disk. Nothing in the config objects when it
stops being true.
@ -207,7 +207,7 @@ periodically), not here. Count rather than age is deliberate: a count
is bounded by construction, whereas an age policy silently scales disk
usage with how hot a hive runs.
Per-agent or per-hive `btrfs qgroup` quotas are not configured yet.
Per-agent or per-hive `btrfs qgroup` quotas aren't configured yet.
Without them one runaway hive can fill the store and take out every
other hive's snapshots.

View file

@ -126,7 +126,7 @@ request still carries an optional `wait_seconds` (long-poll the first
message, once one arrives — or one is already pending — the call
drains up to `max` in total): the harness's own turn-driving loop
uses it internally (`hive-agent`'s `recv_next`, 180s). The
agent-facing MCP `recv` tool does not expose this parameter — it
agent-facing MCP `recv` tool doesn't expose this parameter — it
always passes `wait_seconds: None`, an immediate peek.
Per-row bookkeeping inside the broker:
@ -484,7 +484,7 @@ must be *fixed*, not silenced. Common patterns:
If a lint seems wrong for a specific call site, file an issue and ask
mara — don't add `#[allow]` speculatively. The gate is intentional.
**The devshell checks are not the full `nix flake check`.** Clippy /
**The devshell checks aren't the full `nix flake check`.** Clippy /
fmt / `cargo test` cover most gates, but `nix flake check` runs extra
check derivations they don't:
@ -535,7 +535,7 @@ Shape contract — every one of these:
1. **Always `exit 0`**, even on internal failure. A non-zero exit
would mark the unit `failed`, which in turn aborts
`nixos-container update` and blocks rebuilds. The agent's
capability surface is not allowed to gate the container
capability surface isn't allowed to gate the container
build.
2. **No `set -e`** in the script body. Subshell failures must not
propagate. Use `... || true` on every external call that can

View file

@ -138,7 +138,7 @@ reference-less `-source` (so the runtime closure never arrives), and
pure evaluation rejects. `hyperhive.docs.source` gets away with being
an input only because a docs tree has no runtime dependencies.
The `storePath` trap is worth spelling out, because it is not confined
The `storePath` trap is worth spelling out, because it's not confined
to options the operator writes: **any** option of type `package` fed a
store-path _string_ coerces through `lib.toDerivation`, i.e.
`builtins.storePath`. `environment.systemPackages` and
@ -150,7 +150,7 @@ farm built from the interpolated string — before it can go anywhere a
package is expected.
The catch is that a path written into a generated flake is text, not a
reference — the container's closure does not keep the binary alive.
reference — the container's closure doesn't keep the binary alive.
The **host** does: the package is interpolated into
`/etc/hyperhive/serve.json`, so it lands in the host's system closure
and is gc-rooted by the running generation. `builtins.toJSON` preserves
@ -257,7 +257,7 @@ store files. Any subsequent write into the copied tree (adding new
files in subdirectories) fails with `EPERM`. Fix: pass
`--no-preserve=mode,ownership` so the output tree is writable.
### `nix build flake#name` does not walk into `nixosConfigurations`
### `nix build flake#name` doesn't walk into `nixosConfigurations`
`nix build` resolves the fragment (`#name`) against the flake's
**top-level output attrs** — not against `nixosConfigurations`
@ -339,7 +339,7 @@ error: getting Git object '<hash>': object not found (libgit2 error code = 9)
when a commit that was reachable at some earlier evaluation is now gone
(GC'd, rebased away, or pruned). The failure is persistent: clearing
`~/.cache/nix/{eval-cache-v6,gitv3,fetcher-cache-v4.sqlite}` does not
`~/.cache/nix/{eval-cache-v6,gitv3,fetcher-cache-v4.sqlite}` doesn't
help because the missing object is a structural gap in the git object
graph itself, not in nix's caches.
@ -410,7 +410,7 @@ connects to the compositor at `127.0.0.1:<vnc_port>`.
and `XDG_RUNTIME_DIR=/run/gui` as global system environment
variables (gated on `hyperhive.gui.enable`) so every systemd service
in the container inherits them. Without this, services starting
Wayland clients could not find the compositor — libwayland falls
Wayland clients couldn't find the compositor — libwayland falls
back to a headless display or errors out, the app "works" on a
second invisible display, and the VNC session shows a blank weston
desktop (#540 double-screen).
@ -432,7 +432,7 @@ connects to the compositor at `127.0.0.1:<vnc_port>`.
- **`Type = "simple"` (not `notify`)**: `switch-to-configuration`
must never block on weston signalling readiness. A misconfigured
weston degrades to a `Restart=on-failure` loop visible in
`journalctl`, it does not abort the `nixos-container update`.
`journalctl`, it doesn't abort the `nixos-container update`.
Same reasoning as the `tea-login` unit in `nix/agent-modules/forge.nix`.
- **`[core] idle-time=0`**: disables weston's 300-second idle
timeout. Without it the VNC desktop fades to black and

View file

@ -5,7 +5,7 @@ executing CI jobs from `.forgejo/workflows/ci.yml` on every PR.
## For operators
**Enabling it is one line**: `services.hyperhive.deploy.forgejo.ci.enable = true`
**Enabling it's one line**: `services.hyperhive.deploy.forgejo.ci.enable = true`
in the host NixOS config. No manual token provisioning — hive-c0re
registers the runner with the forge automatically.
@ -45,7 +45,7 @@ blocks merge.
`hive-forge ci-rerun --pr N` dispatches a `workflow_dispatch` retrigger
without an empty commit.
**`ci-rerun --pr` verifies the code but does not reliably move the PR's own
**`ci-rerun --pr` verifies the code but doesn't reliably move the PR's own
status checks.** Measured directly (raw `GET /repos/.../actions/tasks`
JSON): a genuine PR-triggered run carries `event: pull_request` with the
`#<n>` pseudo-ref as its `head_branch`; a `ci-rerun --pr`-dispatched run
@ -95,12 +95,12 @@ hive-forge instance to register against — nothing extra to enable beyond
Optional tuning: `services.hyperhive.deploy.forgejo.ci.name` (runner name in forge
admin panel), `concurrency` (parallel job capacity), `labels` (workflow
targeting), `jobTimeout` (per-job wall-clock cap, default `"1h"`, Go duration
string e.g. `"3h"` — a job that exceeds it is killed so a hung or runaway
string e.g. `"3h"` — a job that exceeds it's killed so a hung or runaway
build can't hold the runner's single slot indefinitely).
## Container design
- **Private netns, bridge-attached**: the container runs in its own network namespace (`privateNetwork = true`, `hostBridge`) and reaches hive-forge through the gateway at `http://<forge.domain>` (resolved to the bridge IP via `networking.extraHosts`). It cannot reach host-loopback services — the core dashboard at `127.0.0.1:7000` and the raw forge port are unreachable from CI. Requires `deploy.forgejo.behindGateway = true`.
- **Private netns, bridge-attached**: the container runs in its own network namespace (`privateNetwork = true`, `hostBridge`) and reaches hive-forge through the gateway at `http://<forge.domain>` (resolved to the bridge IP via `networking.extraHosts`). It can't reach host-loopback services — the core dashboard at `127.0.0.1:7000` and the raw forge port are unreachable from CI. Requires `deploy.forgejo.behindGateway = true`.
- **Non-ephemeral**: runner credentials persist across restarts (written to container's stateDir on first registration, reused thereafter).
- **Sandbox fallback**: nspawn containers can't create user-namespaces, so nix's sandboxing would always fail. Module sets `nix.settings.sandbox-fallback = true` in the container — nix builds run unsandboxed (safe because the container is already isolated). See `docs/process/gotchas.md`.
- **Credential isolation**: the forge admin token (`forge-core-token`) never enters the container. hive-c0re holds it and performs all forge API calls (runner validation + registration-token mint, in `forge/ci_runner.rs`); via hive-priv it writes only the runner registration token to the host env-file `/run/hive-ci/runner-token`, which the container bind-mounts read-only.
@ -130,7 +130,7 @@ Gated on `HYPERHIVE_FORGE_CI_ENABLED` (the nix module sets it on `hive-c0re.serv
- The container boots immediately — nothing gates its start on registration.
- tmpfiles seeds `/run/hive-ci/runner-token` with `TOKEN=placeholder` so the runner's `EnvironmentFile` always exists.
- `gitea-runner-hive.service` has an `ExecStartPre` precond (ahead of the nix-daemon wait) that **fails fast** unless it is already registered (`.runner` present) or a real, non-placeholder token is in place. `Restart=on-failure` (no start-limit cap) self-heals it: a runner that precond-fails at boot keeps retrying until hive-c0re writes the token (c0re's explicit restart is the primary path; the retry is the safety net).
- `gitea-runner-hive.service` has an `ExecStartPre` precond (ahead of the nix-daemon wait) that **fails fast** unless it's already registered (`.runner` present) or a real, non-placeholder token is in place. `Restart=on-failure` (no start-limit cap) self-heals it: a runner that precond-fails at boot keeps retrying until hive-c0re writes the token (c0re's explicit restart is the primary path; the retry is the safety net).
- **Convergence**: because the token write targets the _host_ file, even if c0re's restart races the container being down, the container later starts, reads the now-real token, passes the precond, and registers on its own.
## Actions checkout mirror
@ -173,16 +173,16 @@ to avoid provisioning collisions.
### What unsandboxed builds mean
nspawn containers cannot create user-namespaces, so `nix.settings.sandbox-fallback = true` is set in the container. This means every `nix build` (and `nix flake check`) runs **without a build sandbox** — the build process has full access to the container filesystem, network, and any bind-mounts during the build phase.
nspawn containers can't create user-namespaces, so `nix.settings.sandbox-fallback = true` is set in the container. This means every `nix build` (and `nix flake check`) runs **without a build sandbox** — the build process has full access to the container filesystem, network, and any bind-mounts during the build phase.
A malicious `default.nix` or build script in a PR can therefore:
- **Make arbitrary network requests** to any address reachable from the container. The container runs in its own netns behind the hive bridge, so it reaches the forge only through the gateway (`http://<forge.domain>`, public/read endpoints — no admin credentials) and **cannot** reach host-loopback services: the unauthenticated core dashboard at `127.0.0.1:7000` and the raw forge port are off-limits (bridge→127.0.0.0/8 is dropped).
- **Make arbitrary network requests** to any address reachable from the container. The container runs in its own netns behind the hive bridge, so it reaches the forge only through the gateway (`http://<forge.domain>`, public/read endpoints — no admin credentials) and **can't** reach host-loopback services: the unauthenticated core dashboard at `127.0.0.1:7000` and the raw forge port are off-limits (bridge→127.0.0.0/8 is dropped).
- **Write to the container filesystem**, including corrupting the runner's state dir or `.runner` credentials.
The core admin token (`forge-core-token`) is **not** bind-mounted into the container. It is held and used only by hive-c0re on the host (`forge/ci_runner.rs`), which mints per-runner registration tokens; only that registration token reaches the container's env-file. A build process can still reach forge over the network, but cannot use the admin token to issue privileged API calls.
The core admin token (`forge-core-token`) is **not** bind-mounted into the container. it's held and used only by hive-c0re on the host (`forge/ci_runner.rs`), which mints per-runner registration tokens; only that registration token reaches the container's env-file. A build process can still reach forge over the network, but can't use the admin token to issue privileged API calls.
Note: `nix flake check --no-build` (eval-only) reduces the attack surface but does not eliminate it — `builtins.fetchGit`, `builtins.fetchurl`, and import-from-derivation can reach the network and filesystem during evaluation. The default CI workflow runs full `nix flake check` (builds derivations), which is the higher-risk path.
Note: `nix flake check --no-build` (eval-only) reduces the attack surface but doesn't eliminate it — `builtins.fetchGit`, `builtins.fetchurl`, and import-from-derivation can reach the network and filesystem during evaluation. The default CI workflow runs full `nix flake check` (builds derivations), which is the higher-risk path.
### Mitigation

View file

@ -35,7 +35,7 @@ A DAG is **declared, not described**: a template builds it through
`hive_jobq::JobBuilder`, naming each node it depends on via the handle
`b.node(kind)` handed back, and the builder inserts the nodes itself. A handle
only exists for a node already declared, so every edge points backwards and a
cycle cannot be written down — there is no submit-time validation pass, because
cycle can't be written down — there is no submit-time validation pass, because
there is no malformed spec to reject.
### Node inventory (primitives)
@ -100,9 +100,9 @@ container build:
`DeployWindow` — the deploy subtree's root, which holds it across every
phase below it (it declares `Resource::MetaWindow`). Two meta
mutations can therefore never interleave, so no commit lands inside another
node's staged window. It is a queue resource rather than a runtime mutex
node's staged window. it's a queue resource rather than a runtime mutex
because a resource is held by a subtree root across its whole subtree, which
a `MutexGuard` (bounded by one executor fn) cannot — that is what lets a
a `MutexGuard` (bounded by one executor fn) can't — that's what lets a
multi-node deploy own one window. For the same reason the window must stay
_off_ long store-only work: the rebuild's meta preamble is its own
`MetaSync` node, a sibling of (never a parent of) `Prebuild`, so the
@ -121,7 +121,7 @@ The `stop` / `start` power ops write the durable `wanted` intent via a head
so intent-write + reconcile is atomic per-agent. `restart` is the exception:
it writes _no_ intent (no `SetWanted` head) — it bounces the container and
lets the tail `Reconcile` converge to the agent's existing `wanted`, so a
deliberately-stopped agent is not forced back up by a hive-wide restart. The
deliberately-stopped agent isn't forced back up by a hive-wide restart. The
hive-wide power ops — `restart`, `stop`, and `start` — take an agent _list_:
a hive-wide `hivectl restart` / `stop` / `start` is ONE DAG with a per-agent
subgraph each (independent roots, run concurrently on their own leases), not
@ -130,7 +130,7 @@ N separate DAGs.
**These are built dynamically from each agent's live running state** (an
async `lifecycle::is_running` read), so they live in `job_queue/power.rs`,
not the pure/sync `templates.rs`. Per-agent shape rule: `stop`/`start` carry
a head `SetWanted` (intent) — `restart` does not; the tail `Reconcile`
a head `SetWanted` (intent) — `restart` doesn't; the tail `Reconcile`
(convergence guarantee — cheap, noops when already converged) is ALWAYS
present; only the _mechanical_ nodes (`Signal`/`Drain`/`StopForUpdate`) are
state-conditional — skipped for a _down_ agent (nothing to quiesce/stop). Keeping `Reconcile` in every shape
@ -237,7 +237,7 @@ than to one DAG shape it happens to appear in. A **brace** is the one sanctioned
exception: a pure-resource-holder root that declares on behalf of a subtree
coordinated with itself, whose members then declare nothing.
It is forced rather than stylistic. Declaring a resource means _"I need this
it's forced rather than stylistic. Declaring a resource means _"I need this
exclusively"_, and the agent lease is single-unit — so **two siblings that both
declared it could never run concurrently.** For a subtree whose whole point is
concurrency (`Prebuild` beside the `Signal``Drain` quiesce window), declaring
@ -246,7 +246,7 @@ mutually exclusive. One holder above them speaks for the subtree.
This is the opposite of the failure the declare-your-own rule exists to prevent,
not a relapse into it: there the requirement was _implicit_, inferred from a
node's kind and true only by accident of placement. Here it is explicit, on one
node's kind and true only by accident of placement. Here it's explicit, on one
node, with the omission below it documented on the brace itself.
Two consequences worth knowing:
@ -260,7 +260,7 @@ Two consequences worth knowing:
_payload_ agent, not off a declared lease edge, so every child still lights its
own dashboard pill and still reports its own `takes_container_down` to the
crash watcher. A brace itself reports `false`: it parents the stopping nodes
but does not stop anything, and claiming otherwise would widen crash
but doesn't stop anything, and claiming otherwise would widen crash
suppression across the build and tail, where a vanished container is still a
real crash.
@ -349,7 +349,7 @@ the event's whole job.
## Container view
`container_view.rs` maintains an in-memory snapshot of every nixos-container's
systemd service state. It is polled on coordinator startup and re-scanned after
systemd service state. it's polled on coordinator startup and re-scanned after
every lifecycle operation (spawn, rebuild, kill) so the dashboard always reflects
the actual container status without a live `nixos-container list` call on each
render.
@ -429,7 +429,7 @@ Sequence for a rebuild DAG (each step is its own queue node):
1. `MetaSync` — rebuild-dir prep, meta `sync_agents`, and (unless this is a
meta-update cascade child) the per-agent relock. Short, and the only step
that mutates the meta repo, so it is the only one holding the global deploy
that mutates the meta repo, so it's the only one holding the global deploy
window.
2. `Prebuild` — build the new `system.build.toplevel` **before** stopping.
The container keeps serving the previous generation while eval + fetch +
@ -446,11 +446,11 @@ Sequence for a rebuild DAG (each step is its own queue node):
slot, so the next DAG's `Prebuild` overlaps the container boot.
The approval deploy uses this same chain rather than a rebuild path of its
own. Its `DeployApply` node does not build: it merges, opens the two-phase
own. Its `DeployApply` node doesn't build: it merges, opens the two-phase
meta deploy, and returns the chain above as a subgraph the scheduler grafts
into the live DAG under that node. A `FinalizeDeploy` node gated on the
graft's completion then plants the deploy tag — so "did the agent come back
up?" is answered by `Reconcile` succeeding, the same way it is for every
up?" is answered by `Reconcile` succeeding, the same way it's for every
other rebuild, instead of by a fused inline start.
The grafted nodes land _inside_ `DeployWindow`'s subtree, so they re-enter
@ -479,7 +479,7 @@ flags → `systemctl daemon-reload` → `start`.
### Prebuild attr path
`nix build` does not auto-resolve `meta#<name>` against `nixosConfigurations` the
`nix build` doesn't auto-resolve `meta#<name>` against `nixosConfigurations` the
way `nixos-container` does internally. The explicit attr path
`<flake-root>#nixosConfigurations.<name>.config.system.build.toplevel` is required;
using the bare `meta#<name>` ref would make nix look in `packages`, `legacyPackages`,
@ -523,11 +523,11 @@ wants it.
| `services.hyperhive.c0re.agentCpuQuota` | `"200%"` | CPU cap per agent, as a percentage of one core (`"200%"` = 2 cores). Raise if agents hit CPU limits during builds or heavy tool use. |
| `services.hyperhive.c0re.agentMemoryMax` | `"4G"` | Memory cap per agent. Raise for agents that run large nix builds or hold big in-memory data. |
| `services.hyperhive.c0re.agentCpuWeight` | `80` | `cpu.weight` share per agent, `1``10000` or `null` to omit the setting. Kernel default is `100`, so `80` makes agents yield. |
| `services.hyperhive.c0re.agentIoWeight` | `80` | `io.weight` share per agent, same range and `null` handling. See the caveat below — it is a no-op on many hosts. |
| `services.hyperhive.c0re.agentIoWeight` | `80` | `io.weight` share per agent, same range and `null` handling. See the caveat below — it's a no-op on many hosts. |
Two things to know about the weights:
- They are **hive-wide** — unlike the caps there is no per-agent
- they're **hive-wide** — unlike the caps there is no per-agent
override in `meta/resource-limits.json`, so every agent carries the
same value and the weight does _not_ rank agents against each other.
What `80` buys is that agents yield to everything **not** on this

View file

@ -33,33 +33,33 @@ swarm's store. No agent holds a copy, and neither does this hive.
The hive collector reaches the swarm collector by its gateway name
(`swarm.otel.domain`, default `otel.<swarm domain>`) — the same DNS-and-CA-trust
shape every hive-to-swarm-service hop uses, not a URL an operator has to point
anywhere. A hive that does not run the swarm's services still resolves that
anywhere. A hive that doesn't run the swarm's services still resolves that
name through the gateway; nothing here needs setting for the split-host case.
⚠️ **The collector is therefore in the path of all telemetry.** It runs on the
same host as the agents and restarts on failure, and telemetry is not the
control plane — degraded telemetry is not degraded operation — but the export
same host as the agents and restarts on failure, and telemetry isn't the
control plane — degraded telemetry isn't degraded operation — but the export
no longer survives independently of anything host-side.
### what the agent→collector hop is and isn't
**It has no application-level auth.** The receiver takes any OTLP that reaches
it; what bounds who can reach it is the firewall — `exposeHostPorts` opens the
it; what bounds who can reach it's the firewall — `exposeHostPorts` opens the
port on the bridge interface only. So "unauthenticated to a bridge address"
means _reachable from an agent container_, not _presents a credential_.
The consequence, stated because it is a choice rather than an oversight: **any
agent can push arbitrary OTLP, and it is forwarded on under the operator's
The consequence, stated because it's a choice rather than an oversight: **any
agent can push arbitrary OTLP, and it's forwarded on under the operator's
credential.** Neither tier can tell a container's genuine Claude Code stats
from anything else shaped like OTLP arriving on that port — including data
smuggled out in resource attributes on an otherwise-legitimate export.
That is a **different risk from the one the collector fixes**, and strictly
that's a **different risk from the one the collector fixes**, and strictly
smaller than what preceded it: before, every agent held the upstream credential
itself, so it could do all of the above _and_ use the token anywhere else. The
collector removes the token and keeps the pipe. Agents are inside the trust
boundary (`docs/trust-boundary/security.md`: capability = accepted risk), so an agent being
able to _send_ is an accepted extension of that boundary — but it is not
able to _send_ is an accepted extension of that boundary — but it's not
closed by this design, and nothing here should be read as closing it.
**The `agent` label is self-reported, and no planned authentication changes
@ -76,7 +76,7 @@ is getting a credential:
`hive=` from the connection it authenticated, so _that_ label becomes
unforgeable.
So a verified `hive` is reachable and a verified `agent` is not — and that falls
So a verified `hive` is reachable and a verified `agent` isn't — and that falls
out of the topology rather than being a gap someone forgot to close. The swarm
runs one collector, and the mechanism gives it no finer grain: a bearer-token
check never reveals _which_ token matched, and a receiver reads request metadata
@ -91,7 +91,7 @@ Every `services.hyperhive.otel.*` option's full type/default/description/
example is generated straight from the nix module (`nix/host-modules/
otel.nix`) into [`/options/`](/options/) (host options — `nix build
.#docs-host` for a local render). That page is kept honest by the build in a
way a hand-copied version here cannot be, so it is the reference, not this
way a hand-copied version here can't be, so it's the reference, not this
doc. What follows is what a flat per-option listing can't express: the
two-tier architecture, the security model, and how the options interact.
@ -120,9 +120,9 @@ every agent needs the credential to authenticate — and the harness delivers
that token into the agent's own `~/.claude/settings.json`, a file the agent can
read. `0600` protects it from other containers, not from the agent itself. As
long as the direct path stays _selectable_, that hole stays selectable; an
option that can reintroduce it is a hole with extra steps.
option that can reintroduce it's a hole with extra steps.
**Why the tiers stay separate on one box.** They are not collapsed when
**Why the tiers stay separate on one box.** they're not collapsed when
co-located: an all-local hive is a statement about _where_ processes run, not
about the shape of the deployment. A boundary that disappears locally is one
the local deployment stops testing.
@ -136,14 +136,14 @@ automatically; there is nothing to open by hand.
### Authenticated ingest
The swarm tier gives **each hive its own receiver**, and stamps the `hive` label
from whichever receiver accepted a sample. A hive therefore cannot report
metrics as another hive, and cannot relabel its own by editing what it sends —
the label is not taken from the payload at all.
from whichever receiver accepted a sample. A hive therefore can't report
metrics as another hive, and can't relabel its own by editing what it sends —
the label isn't taken from the payload at all.
**On an all-local swarm there is nothing to set.** Each hive already has an
identity, and its collector reads the secret that host's own authelia minted.
**On a hive that does not host the swarm's services**, the secret has to arrive
**On a hive that doesn't host the swarm's services**, the secret has to arrive
somehow — copy it across and name it:
```nix
@ -286,7 +286,7 @@ distinguishable from "this hive is gone". Select them with
Both levels are reported every cycle, `0` included, so a healthy hive is
visible as zeros rather than as missing series.
`hyperhive.hive.degraded` is what a dashboard should alert on: it is
`hyperhive.hive.degraded` is what a dashboard should alert on: it's
`hive-c0re`'s own readiness verdict, so it stays in step with `/health/ready`
and with what the swarm controller sees. `hyperhive.hive.warnings` is the
detail behind it — `warn`-level entries mean "an operator should look" and do
@ -341,7 +341,7 @@ hive-metric <name> <value> [--type counter|gauge] [--temporality delta|cumulativ
itself).
- `--labels key=value` — extra per-data-point labels. May be repeated.
The resource labels (agent, hive, swarm, service.name) are inherited
automatically from `OTEL_RESOURCE_ATTRIBUTES` — do not re-specify them.
automatically from `OTEL_RESOURCE_ATTRIBUTES` — don't re-specify them.
### Examples
@ -356,10 +356,10 @@ hive-metric queue_depth 17 --type gauge
hive-metric api_latency_ms 142.5 --type gauge --labels model=sonnet --labels tier=api
```
### Error when OTEL is not configured
### Error when OTEL isn't configured
When `services.hyperhive.otel.enable = false` (the default), the
`OTEL_EXPORTER_OTLP_ENDPOINT` env var is not set and `hive-metric` exits
`OTEL_EXPORTER_OTLP_ENDPOINT` env var isn't set and `hive-metric` exits
with an informative error message. No silently-dropped metrics.
### Wire format

View file

@ -47,14 +47,14 @@ and deploys — an eval failure asking the operator to write the address
down is the cheaper outcome. **Upgrading past this release means setting
both once.**
`domain` is required too, but you no longer _write_ it: it is read from
`domain` is required too, but you no longer _write_ it: it's read from
this hive's own entry in the directory, whose `domain` defaults to
`<name>.<swarm.domain>`. So a conventional swarm states no addresses at
all, and a hive addressed by something else states it in the one place
the other hives read — `swarm.hives.edge.domain = "edge.elsewhere.example";`.
Setting `services.hyperhive.domain` directly still works and still wins,
with a **deprecation warning**. The reason it's deprecated is not tidiness:
with a **deprecation warning**. The reason it's deprecated isn't tidiness:
that option is local to one host, the directory is copied to every host,
so a value written only there leaves every peer pointing somewhere else
with nothing detecting the disagreement.
@ -70,8 +70,8 @@ form qualified labels (`iris@pr1ma.example.com`).
`swarm.name` is purely display — it surfaces in the dashboard chrome
header and per-agent system prompts, and federated hives at different
domains can share one. `hiveName` surfaces in the same places but is
_not_ only display: it is the leftmost label of the hive's domain. That
`swarm.name` sits under `swarm` and `hiveName` does not is the whole
_not_ only display: it's the leftmost label of the hive's domain. That
`swarm.name` sits under `swarm` and `hiveName` doesn't is the whole
distinction — one names this hive, the other names the group it belongs
to.
@ -120,7 +120,7 @@ services.hyperhive.swarm.hives = {
```
One attrset describing **every** hive in the swarm, **including this
one**, keyed by that hive's `hiveName`. It is meant to be _identical on
one**, keyed by that hive's `hiveName`. it's meant to be _identical on
every host_ — write it once, share it, and each host reads it correctly
because `services.hyperhive.hiveName` says which entry is itself.
@ -130,7 +130,7 @@ hive. That assertion is load-bearing rather than pedantic — "my peers"
is derived as _everything that isn't me_, so a directory that doesn't
contain you derives every hive as a peer and you peer with yourself.
`domain` is required per entry and deliberately undefaulted: it is
`domain` is required per entry and deliberately undefaulted: it's
conventionally `<name>.<swarm.domain>`, but a wrong domain that
evaluates cleanly points at a real machine that isn't the one you meant.
@ -138,7 +138,7 @@ evaluates cleanly points at a real machine that isn't the one you meant.
> inside a swarm comes from the swarm root ([`ca.md`](ca.md)): every
> hive chains to it, so one anchor replaces per-hive pinning entirely.
> What that genuinely drops is trusting a hive whose root this swarm
> does _not_ own — another swarm's, or one keeping its own CA. That is
> does _not_ own — another swarm's, or one keeping its own CA. that's
> a cross-swarm problem and wants a mechanism designed for it. (An
> earlier `certFingerprint` field existed for exactly that gap, pinning
> a peer's TLS leaf for hive-c0re's own peer HTTPS checks — removed
@ -160,7 +160,7 @@ evaluates cleanly points at a real machine that isn't the one you meant.
**container's** trust bundle, independent of this directory.
⚠️ **That container currently trusts no swarm-internal CA**, so a
self-signed gateway certificate does not federate. The swarm root
self-signed gateway certificate doesn't federate. The swarm root
can't be listed there: `security.pki.certificateFiles` is
read when the system is _built_, and the root is a runtime file (its
key must never enter the store), so there is no build-time name for
@ -183,7 +183,7 @@ hiveName = "pr1ma"; hiveName = "edge";
swarm.hives = { … }; swarm.hives = { … }; # byte-identical
```
That is the point of the shape, and it removes a class of bug rather
that's the point of the shape, and it removes a class of bug rather
than saving typing: a per-host peer list let two hosts hold _different_
facts about the same third hive — a stale endpoint, a rotated
fingerprint — with nothing to detect the disagreement. One entry per
@ -276,7 +276,7 @@ service it points at: `services.hyperhive.swarm.snapshotStore.{address,
port}` tells this hive where the swarm's `btrfs receive` endpoint is, so
`hivectl agent <name> subvol snapshot push` has somewhere to stream to.
It is genuinely swarm-scoped rather than per-peer — a swarm has exactly
it's genuinely swarm-scoped rather than per-peer — a swarm has exactly
one store, because the receiver keys destinations by _agent_ so a
migrating agent keeps one unbroken incremental chain. See
[snapshot-store.md](../networking/snapshot-store.md).
@ -288,9 +288,9 @@ daemon on this host. **Off by default and deliberately not derived from
`services.hyperhive.enable`**: a swarm has one controller, so enabling it
is a statement about swarm topology, not about whether hyperhive is
installed. Every hive runs `hive-c0re` (the agents on that host); one
hive additionally runs this (what is true across hives).
hive additionally runs this (what's true across hives).
What it serves, why it is a unix socket rather than a port, and the
What it serves, why it's a unix socket rather than a port, and the
socket-directory constraint that governs where `socketPath` may point:
[`swarm-controller/README.md`](../../swarm-controller/README.md).
@ -298,19 +298,19 @@ socket-directory constraint that governs where `socketPath` may point:
One row per hive in `swarm.hives`, saying when it last reported and what
it said. Hives publish upward through the swarm queue; the controller
never reaches down to collect, so a hive that cannot reach the swarm
still knows its own state — you just cannot see it from here.
never reaches down to collect, so a hive that can't reach the swarm
still knows its own state — you just can't see it from here.
A hive publishes only once it has been given the three
`swarm.statusPublish` coordinates below. A hive that has not reads
`never_reported` — it is not broken, it just has nothing to say upward.
`never_reported` — it's not broken, it just hasn'thing to say upward.
| freshness | what to do about it |
| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| `fresh` | nothing — reported within `staleAfterSeconds` |
| `stale` | the hive stopped reporting. Its last payload is still shown, so check `age_seconds` and the payload for what it managed to say |
| `never_reported` | this hive has never reported at all — normally a deployment that hasn't happened, not an outage |
| `unknown` | something is publishing under a name that is not in `swarm.hives` — a typo in the roster, or a hive that was removed and is still running |
| freshness | what to do about it |
| ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| `fresh` | nothing — reported within `staleAfterSeconds` |
| `stale` | the hive stopped reporting. Its last payload is still shown, so check `age_seconds` and the payload for what it managed to say |
| `never_reported` | this hive has never reported at all — normally a deployment that hasn't happened, not an outage |
| `unknown` | something is publishing under a name that's not in `swarm.hives` — a typo in the roster, or a hive that was removed and is still running |
Every row also carries `last_seen_unix` and `age_seconds` if you want to
apply your own threshold. The timestamp is the one the queue recorded on
@ -341,20 +341,20 @@ not distribute it. Copy `hive-<hiveName>.secret` out of the swarm host's
`deploy.authelia.hostClientSecretDir` with whatever secret management the
deployment already uses.
The identity is not a choice — a hive authenticates as `hive-<hiveName>`
The identity isn't a choice — a hive authenticates as `hive-<hiveName>`
and publishes under `hiveName`, the same name that keys `swarm.hives`.
If a hive stops reporting, its own dashboard is the place to look: a
failure to publish raises a warning banner there after three consecutive
misses. It stays `warn` rather than `crit` on purpose — a hive that
cannot reach the queue is not itself unhealthy, so it does not start
calling itself degraded for being unable to say it is fine.
can't reach the queue isn't itself unhealthy, so it doesn't start
calling itself degraded for being unable to say it's fine.
The endpoint answers **503** when this host has no swarm queue
configured, or has one and cannot read it — deliberately not an empty
configured, or has one and can't read it — deliberately not an empty
list, which would look like a silent swarm rather than a controller that
cannot see. The body says which. Status survives a controller restart:
it is stored in the queue, not in the daemon.
can't see. The body says which. Status survives a controller restart:
it's stored in the queue, not in the daemon.
### Swarm-wide forge webhooks
@ -375,19 +375,19 @@ leftover at startup. A webhook has exactly one target URL, so per-hive
registration never added a recipient — it took delivery away from
whichever hive registered before it.
**The `agent-configs` org is not yet.** Each hive still registers its own
**The `agent-configs` org isn't yet.** Each hive still registers its own
`pull_request` hook there, so that repo has two — the hive's and the
controller's — and **both are expected; do not delete either.** Removing
controller's — and **both are expected; don't delete either.** Removing
a hive's stops it acting on config PRs; removing the controller's just
gets recreated on its next start.
Nothing to configure. The hooks are registered only when this host also
serves the swarm UI vhost — that is what publishes the endpoint, and a
hook the forge cannot reach would collect failed deliveries while
serves the swarm UI vhost — that's what publishes the endpoint, and a
hook the forge can't reach would collect failed deliveries while
looking healthy. The HMAC secret is generated on first start and kept
(see [`docs/agent-lifecycle/persistence.md`](../agent-lifecycle/persistence.md)).
To check it is working, push to `internal/knowledge` and look for
To check it's working, push to `internal/knowledge` and look for
`webhook: verified delivery` in `journalctl -u swarm-controller`. A
refused delivery logs `webhook: refused delivery` with the reason.

View file

@ -5,7 +5,7 @@ hive's own CA, and that hive CA signs the gateway leaf. A peer that
trusts the root once validates every hive in the swarm, present and
future, instead of being pinned to each one by hand.
That is the whole point of the hierarchy — it turns per-peer trust from
that's the whole point of the hierarchy — it turns per-peer trust from
O(n²) hand-pinning into one anchor per swarm.
## Two provisioning modes, one structure
@ -19,7 +19,7 @@ What differs is who puts the artifacts on disk:
`services.hyperhive.swarm.ca.autoConfigure` selects between them, and is
**off by default**: a swarm's services and its hives can live on
different hosts, and a host cannot tell whether it is the one holding
different hosts, and a host can't tell whether it's the one holding
the root, so setting the swarm CA up is an operator action rather than
something a host assumes. Turn it on for an all-on-one-host deployment
and the hierarchy costs no configuration.
@ -43,13 +43,13 @@ The swarm's service names (`swarm.forge.domain`,
`swarm.matrix.gatewayHost`, `swarm.authelia.domain`) default to
siblings of the hive domain — `forge.<swarm.domain>`, not
`forge.<hive domain>`. The hive CA's leaf is a **single-label** wildcard
over the hive's own domain, so it cannot cover them; only the
over the hive's own domain, so it can't cover them; only the
swarm-services leaf can, and this host signs that one only under
`autoConfigure`. A hive with neither serves the hive leaf on those names
and every client sees a name mismatch.
So `hive-tls` emits an eval-time **warning** naming the uncovered names.
It is deliberately not an assertion: this module can see what _it_ is
it's deliberately not an assertion: this module can see what _it_ is
able to issue, but not an operator-installed sub-CA in
`swarm.ca.stateDir`, an external ACME setup, or a certificate delivered
by any other means. If you have arranged one, the warning is expected
@ -89,16 +89,16 @@ host:
scp /var/lib/swarm-ca/root.pem <peer-host>:/var/lib/swarm-ca/root.pem
```
That is the point of the hierarchy: the root is installed **once per
that's the point of the hierarchy: the root is installed **once per
swarm host**, not once per peer, so a hive joining later needs no edit on
the hives already running. A hive whose cert already chains to a public
CA has nothing to install — `peer-config` says so instead.
Handing a peer this hive's own `ca.pem` would not work even as a
one-off: once a hive CA is an intermediate under the swarm root, it is
no longer something a verifier can build a chain _to_ — OpenSSL will not
one-off: once a hive CA is an intermediate under the swarm root, it's
no longer something a verifier can build a chain _to_ — OpenSSL won't
terminate a chain at a trusted non-self-signed certificate without
`-partial_chain`. That is why the tool distributes the root, not a
`-partial_chain`. that's why the tool distributes the root, not a
per-hive file.
## Adopting the hierarchy on an existing hive
@ -107,23 +107,23 @@ A hive that predates the swarm root carries a self-signed `ca.pem`, and
adopting the hierarchy means replacing it. That invalidates an anchor
consumers already trust, and they refresh on their own schedule — agents
only pick up new trust when their container restarts, peers only on
their own rebuild. **Who is allowed to decide that is what splits the
their own rebuild. **Who is allowed to decide that's what splits the
two cases.**
### Where this host owns the root (`autoConfigure`)
Adoption happens by itself, once. `hive-tls-ca.service` notices that
`ca.pem` does not chain to the root, keeps the old certificate as
`ca.pem` doesn't chain to the root, keeps the old certificate as
`ca-previous.pem`, and re-issues under the root; the next leaf is signed
by the new CA.
It is safe to automate here precisely because this is the
it's safe to automate here precisely because this is the
all-on-one-host shape: every consumer is on this box, so "when will they
have refreshed" is knowable rather than guessed.
The old CA stays in `trust-bundle.pem` afterwards, so adoption is
**additive to the anchor set before it is subtractive** — a container
that has not restarted yet still validates. Removing `ca-previous.pem`
**additive to the anchor set before it's subtractive** — a container
that hasn't restarted yet still validates. Removing `ca-previous.pem`
is a deliberate later step: how long is long enough is a property of the
deployment, not something the unit can know.
@ -141,15 +141,15 @@ rm <deploy.hive-controller.tls.stateDir>/ca.pem <deploy.hive-controller.tls.stat
systemctl restart hive-tls-ca.service
```
Failing rather than warning is deliberate: a hive whose CA does not
Failing rather than warning is deliberate: a hive whose CA doesn't
chain to the root it has been given is misconfigured, and a warning in a
build log is not something anyone reads twice.
build log isn't something anyone reads twice.
To keep the current CA on purpose — a hive that deliberately stays
outside the hierarchy, or one mid-migration — `touch` the marker file
named in the message. That is a decision, and it is recorded as one.
named in the message. that's a decision, and it's recorded as one.
A hive with **no** root configured at all is not affected by any of
A hive with **no** root configured at all isn't affected by any of
this: it self-signs exactly as it always has.
## Distributing the root
@ -159,11 +159,11 @@ The root **key** is a runtime file for the reason in
The root **certificate** is a runtime file as a _consequence_ — it lives
beside the key under `swarm.ca.stateDir` — and that has a cost worth
naming, because it is subtle and it bites at a distance:
naming, because it's subtle and it bites at a distance:
> **Nothing whose trust store is assembled at build time can reference
> the swarm root.** `security.pki.certificateFiles` is read inside the
> derivation; the root does not exist there.
> derivation; the root doesn't exist there.
Two consumers, and only one of them is fine:

View file

@ -19,9 +19,9 @@ so the store is a perfectly good place for it.
**The generator and the reader are usually in different containers.** They share
the host's network namespace, which makes them feel co-located, but their
filesystem roots are separate. That is why delivery is a **host-side copy rather
filesystem roots are separate. that's why delivery is a **host-side copy rather
than a bind mount**: `nixos-container` refuses to start when a bind source is
missing, and a secret minted on another container's first boot does not exist
missing, and a secret minted on another container's first boot doesn't exist
yet. Binding it would make one container wait on a file that waits on a
container that starts after it.
@ -30,20 +30,20 @@ container that starts after it.
Every row below is read against one of these. This is a different cut
from the [deployment shapes](services.md#deployment-shapes) --- those
say _where services run_, these say _who is responsible for a secret
file being there_ --- so the two lists do not line up one-to-one, and
file being there_ --- so the two lists don't line up one-to-one, and
neither is a renaming of the other.
| topology | what it means | who places secrets |
| ------------------ | ---------------------------------------------------------- | ---------------------------------------------------------------------------- |
| **all-local** | one host runs the swarm's shared services and its own hive | nobody — each secret is generated where it is read, or copied by a host unit |
| **swarm-managed** | the swarm's services run on a host with `swarmctl` | `swarmctl` writes what it owns; the rest is still generated in place |
| **hive elsewhere** | a hive that federates with a swarm it does not host | the operator provides the file and names it in config |
| topology | what it means | who places secrets |
| ------------------ | ---------------------------------------------------------- | --------------------------------------------------------------------------- |
| **all-local** | one host runs the swarm's shared services and its own hive | nobody — each secret is generated where it's read, or copied by a host unit |
| **swarm-managed** | the swarm's services run on a host with `swarmctl` | `swarmctl` writes what it owns; the rest is still generated in place |
| **hive elsewhere** | a hive that federates with a swarm it doesn't host | the operator provides the file and names it in config |
## Swarm-level — one of each per swarm
| secret | generated by | lives at | hive elsewhere |
| --------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| swarm root CA cert | `swarm-ca.nix` first-boot unit, when `autoConfigure` is set | `/var/lib/swarm-ca/root.pem` | operator copies the **cert** in; it is public |
| swarm root CA cert | `swarm-ca.nix` first-boot unit, when `autoConfigure` is set | `/var/lib/swarm-ca/root.pem` | operator copies the **cert** in; it's public |
| swarm root CA key | same unit | `/var/lib/swarm-ca/root-key.pem`, `0600` | stays on whichever host holds it — see the constraint below |
| swarm-services sub-CA (cert + key) | `swarm-ca.nix`, signed by the root | `/var/lib/swarm-ca/services-ca{,-key}.pem` | issued where the root lives |
| authelia session, JWT and storage-encryption keys | authelia's first-boot unit, in-container | `/var/lib/authelia-swarm/{session,jwt,storage-encryption}.key` | generated in place; nothing outside that container reads them |
@ -56,31 +56,31 @@ neither is a renaming of the other.
| wireguard private key | **the operator**`wg genkey` | whatever `deploy.wireguard.privateKeyFile` names | always operator-provided; nothing generates this for you |
| queue auth-callout nkeys (user seed + account seed) | `swarm-nats-callout-keys` first-boot unit, when `deploy.nats.autoGenerateCallout` is set | `/var/lib/swarm-nats-callout/{callout-user,issuer}.seed`, `0600` | operator mints both with `nk` and names them in `deploy.nats.calloutUserSeedFile` / `deploy.nats.calloutIssuerSeedFile` |
| the secret store's own contents | openbao, on first `bao operator init`**an operator action, not a unit** | inside the `swarm-bao` container, at its own `/var/lib/openbao`, kept across rebuilds by `ephemeral = false`. ⚠️ **Not a host path**: `nixos-container destroy swarm-bao` takes the raft data with it, so back up the container's tree, not `/var/lib/`. Only the store's TLS material (`/var/lib/swarm-bao-tls`) and its PKCS11 token (`/var/lib/swarm-bao-token`) are host-level | n/a — there is one store; a hive elsewhere is a _client_ of it and holds none of this |
| the secret store's unseal material | the HSM/TPM under `deploy.bao.seal = "pkcs11"`; openbao itself under `"shamir"` | in the token; or held by whoever ran `bao operator init`, which is what `"shamir"` means and why it is stated rather than inferred | n/a — only the host running the store seals anything |
| the secret store's unseal material | the HSM/TPM under `deploy.bao.seal = "pkcs11"`; openbao itself under `"shamir"` | in the token; or held by whoever ran `bao operator init`, which is what `"shamir"` means and why it's stated rather than inferred | n/a — only the host running the store seals anything |
The three keys authelia mints for itself are generated in-container precisely
because nothing outside that container ever reads them. **That is the test worth
because nothing outside that container ever reads them. **that's the test worth
applying to any secret added here** — and the client secret's plaintext half is
the one row that fails it, which is the entire reason a delivery step exists.
There are two telemetry collectors and they land on opposite sides of that test.
The **hive's** collector needs no delivery step. It authenticates to the swarm's
collector as its own hive, and it is a host unit rather than a container, so on
collector as its own hive, and it's a host unit rather than a container, so on
an all-local swarm it reads authelia's file where it lies and no second copy is
made. On any other topology it is an ordinary "operator provides the file"
made. On any other topology it's an ordinary "operator provides the file"
case — see `services.hyperhive.otel.clientSecretFile`.
The **swarm's** collector does need one. It runs in a container, so its copy is
placed by `swarm-otel-oidc-secret.service` and lands at
`/var/lib/swarm-otel-oidc/<client-id>.secret` — the same shape as the forge and
homeserver rows below, and for the same reason: the container that mints the
secret is not the container that reads it.
secret isn't the container that reads it.
The copy is only made when authelia is enabled **on this host** and something
published is being scraped; otherwise no secret is needed and none is placed.
⚠️ **Do not read that delivery unit as the only way this collector is fed.**
⚠️ **don't read that delivery unit as the only way this collector is fed.**
Whether it authenticates follows the _credential_, never another service's
placement: a swarm collector may run on a host that holds neither store and no
authelia, and then the secret is an ordinary operator-provided file named in
@ -91,7 +91,7 @@ not the definition of the case.
### Minting the queue's callout nkeys
`deploy.nats.autoGenerateCallout` mints both keypairs on the host before the queue
starts. It is on by default only under `singleHostSwarm` — the one
starts. It's on by default only under `singleHostSwarm` — the one
topology where the queue, its responder and the operator are the same person. On
every other topology, mint them yourself:
@ -108,12 +108,12 @@ admit anyone to the queue, so it belongs wherever the responder runs and nowhere
else.
A hive that sets neither the public keys nor `autoGenerateCallout` fails at
eval, naming the option it wants. That is deliberate: a queue that started
eval, naming the option it wants. that's deliberate: a queue that started
without them would accept `CONNECT {"user":"auth"}` from anyone sharing the
host's network namespace, and nothing would look wrong until somebody connected.
**All four or none** — the seed paths are required too, not just the public
keys. They are two halves of the same pair: the server verifies with the public
keys. they're two halves of the same pair: the server verifies with the public
half, the responder signs with the private one. Supplying only the public keys
used to pass eval and leave the queue with an auth-callout nobody answers, which
refuses every client rather than degrading — and a refusal reaches the client as
@ -139,8 +139,8 @@ bounded wait, 120s — and then **fail loudly** rather than skipping. A silent s
produces a service whose login button always fails, which is a symptom several
layers from its cause.
The store's **first reader** is the matrix registration token, and it is worth
saying why that one: it is an opaque 32-byte value with no second file and no
The store's **first reader** is the matrix registration token, and it's worth
saying why that one: it's an opaque 32-byte value with no second file and no
format. Authelia's OIDC secret needs a `.secret` _and_ a matching `.digest`, so
starting there would have meant debugging "can a reader authenticate and get
bytes back" and "did we write authelia's file format right" at once, with an
@ -153,8 +153,8 @@ locally minted token in place, so a hive with no store behaves exactly as it
did before.
⚠️ **Service↔store mTLS is its own trust domain.** A credential you must
already hold to authenticate cannot be fetched from the thing it authenticates
you to, so the store's identity cannot come from an authority the store
already hold to authenticate can't be fetched from the thing it authenticates
you to, so the store's identity can't come from an authority the store
distributes — which excludes the hive CA and the swarm CA both, and has nothing
to do with the gateway's HTTPS certificates either way. `glue-bao-tls.nix`
mints a CA that signs exactly two things, the store's server certificate and a
@ -163,14 +163,14 @@ internal CA deletes that file and names its own paths in
`deploy.bao.serverCertFile` / `clientCaFile`; the store itself has no opinion.
A hive that reads from a store on **another** machine names the reader's half —
`clientCertFile`, `clientKeyFile`, `serverCaFile` — and places that leaf by hand.
It is the one credential that cannot come out of the store, being what opens it;
it's the one credential that can't come out of the store, being what opens it;
everything else a hive needs does.
## The constraint that decides where the root lives
A hive CA carries `nameConstraints=permitted;DNS:<hive domain>`, and **a swarm
service name is a sibling of the hive domain rather than a child** — `forge.<swarm>`
next to `<hive>.<swarm>`. So a hive CA cannot issue a certificate for a swarm
next to `<hive>.<swarm>`. So a hive CA can't issue a certificate for a swarm
service. Not by policy: by construction, and openssl enforces it.
Whatever holds the swarm root is therefore what makes swarm-service certificates
@ -188,4 +188,4 @@ possible at all. Two things follow:
State three things, in the row you add above: **who mints it**, **which
container reads it**, and **what happens when they differ**. If they differ, it
needs a delivery unit, and the unit copies — it does not bind.
needs a delivery unit, and the unit copies — it doesn't bind.

View file

@ -10,9 +10,9 @@ services.hyperhive.deploy.allSwarmServices = true;
```
**`deploy.allSwarmServices` is what "the swarm's shared services run
here" means: every once-per-swarm service that is _optional_ takes its
`enable` from it.** That is the whole rule, stated once — the per-service
sections below do not repeat it, so a service that stops deriving is a
here" means: every once-per-swarm service that's _optional_ takes its
`enable` from it.** that's the whole rule, stated once — the per-service
sections below don't repeat it, so a service that stops deriving is a
visible difference rather than one more paragraph saying the same thing.
`singleHostSwarm` is the all-on-one-box switch above it: it defaults
@ -20,13 +20,13 @@ both `deploy.allSwarmServices` and `swarm.ca.autoConfigure` (the swarm CA
is generated here). Each derived toggle can still be set on its own,
which wins, so "all local except X" needs no further option.
**Both default to off**, and that is deliberate: a host cannot tell
whether it is meant to be the swarm's service host, so this is an
**Both default to off**, and that's deliberate: a host can't tell
whether it's meant to be the swarm's service host, so this is an
operator saying so rather than something inferred. With them off, a hive
is a _client_ of those services — it configures how to reach them and
runs none of them.
The forge is the exception, and not because it is per-hive: it is
The forge is the exception, and not because it's per-hive: it's
swarm-wide but **not optional**, being the canonical store for the meta
flake and every agent's config repo, so it deploys with hyperhive itself
and has no `enable` to derive from anything.
@ -82,7 +82,7 @@ starts with nobody in it rather than failing to start — a provider with
no subjects yet is the correct state before anything has provisioned
them. Session and storage keys are generated in the container on first
boot and never rotated automatically; replacing one invalidates data
already written (sessions, the encrypted store), so that is an operator
already written (sessions, the encrypted store), so that's an operator
action.
Storage is local sqlite and the notifier writes to a file. Both are
@ -112,11 +112,11 @@ services.hyperhive.deploy.grafana.enable = false;
⚠️ **This starts a database that grows for as long as the swarm runs.**
See `retentionPeriod` below before leaving it at its default.
| Option | When you'd touch it |
| ---------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `deploy.victoriametrics.retentionPeriod` | Default `5y`. Lower it once you have measured how fast this swarm actually fills a disk — the default is deliberately generous because too-short silently discards history you cannot get back. |
| `swarm.grafana.oidc.role` | Default `Admin` for everyone who logs in. Lower to `Viewer`/`Editor` if the swarm grows operators who should not be able to reconfigure Grafana. |
| `deploy.grafana.datasourceUrl` | Only if you front VictoriaMetrics with something else. It defaults to the store on this host, which is the only thing it can reach. |
| Option | When you'd touch it |
| ----------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `deploy.victoriametrics.retentionPeriod` | Default `5y`. Lower it once you have measured how fast this swarm actually fills a disk — the default is deliberately generous because too-short silently discards history you can't get back. |
| `swarm.grafana.oidc.role` | Default `Admin` for everyone who logs in. Lower to `Viewer`/`Editor` if the swarm grows operators who shouldn't be able to reconfigure Grafana. |
| `deploy.grafana.datasourceUrl` | Only if you front VictoriaMetrics with something else. It defaults to the store on this host, which is the only thing it can reach. |
**Logging in.** Grafana is behind swarm SSO, so the accounts are the
authelia ones — there is no separate Grafana password, and the local
@ -138,7 +138,7 @@ the hive's daemons and infra containers, and the harness units inside every
agent container. The collector below is what writes to it.
**Reading them.** Open Grafana, pick **Explore**, and choose the
`VictoriaLogs` datasource — it is provisioned for you. Grafana's _Logs
`VictoriaLogs` datasource — it's provisioned for you. Grafana's _Logs
Drilldown_ app is deliberately not installed: it only supports Loki, and
no setting here changes that, so Explore is the log browser for this
swarm.
@ -170,7 +170,7 @@ Every hive's own collector reaches this one by its gateway name,
`swarm.otel.domain` (default `otel.<swarm domain>`) — the same
by-domain-through-the-gateway shape every other swarm service uses, not a
loopback URL an operator has to redirect. There is nothing to set on a hive
that does not run the swarm's services; the name resolves through the
that doesn't run the swarm's services; the name resolves through the
gateway either way.
| Option | When you'd touch it |

View file

@ -1,6 +1,6 @@
# Swarm SSO
The swarm runs one authelia, and it is two things at once: the **session
The swarm runs one authelia, and it's two things at once: the **session
provider** every protected vhost checks (`auth_request`), and — once any
client is declared — an **OIDC provider** issuing tokens to relying
parties: the forge and the matrix homeserver.
@ -18,7 +18,7 @@ name all follow `deploy.authelia`, so there is nothing to turn on
separately. (Details, including why a client hive must not declare that
vhost: [`../networking/gateway.md`](../networking/gateway.md).)
**Authelia does not start until at least one user exists.** The user
**Authelia doesn't start until at least one user exists.** The user
store is generated empty — deliberately, since seeding a default account
would put a credential in a config file — but authelia validates it at
startup and treats "no users" as fatal:
@ -31,7 +31,7 @@ error reading the authentication database: could not validate the schema:
It then exits 1 and systemd restarts it, so a swarm that has been
enabled but not bootstrapped shows a **crash-looping unit** and `502 Bad
Gateway` from the vhost — not a login page with nobody able to use it.
The gateway is working in that state; the upstream is not up.
The gateway is working in that state; the upstream isn't up.
⚠️ So the step below is **required to finish the install**, not an
optional first-login convenience. Run it before concluding anything is
@ -48,14 +48,14 @@ this password is stored nowhere — record it now
```
The password is generated, hashed, and printed once; only the hash is
kept. `swarmctl` reads and writes authelia's `users.yml` directly — it is
kept. `swarmctl` reads and writes authelia's `users.yml` directly — it's
the one user store, shared with `swarm-authelia-bridge`, which creates
agent identities in the same file. No restart: authelia watches it. Full
reference: [`../tools/swarmctl-cli.md`](../tools/swarmctl-cli.md).
You can edit `users.yml` by hand, and `swarmctl` will read what you
wrote. ⚠️ It rewrites the whole file on every change, so **comments and
formatting do not survive**; values and unrecognised keys do.
formatting don't survive**; values and unrecognised keys do.
This step stays manual on purpose. Bootstrapping an identity provider
non-interactively means a secret arriving from somewhere — a file, an
@ -78,12 +78,12 @@ mara is now in groups: admins
Two behaviours worth knowing before you rely on them:
- **`--remove-group` fails if the user is not in that group.** Every
other flag is idempotent — setting what is already set is fine, so a
"make these four things true" call does not break when one of them
- **`--remove-group` fails if the user isn't in that group.** Every
other flag is idempotent — setting what's already set is fine, so a
"make these four things true" call doesn't break when one of them
already was. Revocation is the exception on purpose: a typo'd group
name that reported success would leave an account holding access you
believe you took away, and that is the one outcome nobody re-checks.
believe you took away, and that's the one outcome nobody re-checks.
- **The resulting group list is printed** because group names have no
registry anywhere. A misspelled `--add-group` creates a real group that
no access-control rule mentions, so the user gains nothing and no error
@ -105,7 +105,7 @@ the delivery step below and the rest of authelia's keys don't.
## Getting the plaintext to the relying party
Three cases, and they are genuinely different mechanisms rather than one
Three cases, and they're genuinely different mechanisms rather than one
mechanism with flags.
### 1. All-local — one host runs both
@ -116,12 +116,12 @@ service's container, and the service's own module contributes its client
entry — callback URL included — to authelia's client list.
The callback is built once and read twice, so the redirect URI authelia is
told to allow and the one the service actually sends cannot drift apart. A
told to allow and the one the service actually sends can't drift apart. A
mismatch there is a rejected login with no error text worth reading.
⚠️ The delivery is a copy, not a `bindMounts` entry, and deliberately so:
nixos-container refuses to start a container whose bind source is
missing, and this secret does not exist until authelia's first boot has
missing, and this secret doesn't exist until authelia's first boot has
run. Binding it would make the service wait on a file that waits on a
container that starts after it — on a fresh hive, a permanent stall
presenting as "the forge is broken", several layers from its cause.
@ -155,7 +155,7 @@ evaluation error.
## Where each relying party differs
The registration half is identical; what each service does with the
result is not.
result isn't.
| | forge | matrix |
| ------------------------- | ---------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
@ -175,7 +175,7 @@ Two consequences worth stating plainly:
spoken by non-browser clients holding matrix access tokens — every
agent's own daemon — as well as by federation, so `/_matrix/` is served
directly and authenticates itself. The forward-auth vhosts protect
browser surfaces; this is not one of them.
browser surfaces; this isn't one of them.
## Machine clients
@ -198,7 +198,7 @@ valid token refused at the target, which reads like a broken credential
rather than a broken name.
**Tokens are signed (`RS256`), not opaque**, because a resource server
that cannot call the provider back is a real case here: the telemetry
that can't call the provider back is a real case here: the telemetry
collector verifies offline against `/jwks.json`, and an opaque token gives
it nothing to verify. The queue's responder introspects instead — a
different question asked of the same token, and the reason both
@ -240,13 +240,13 @@ All three matter. A change that silently deleted the browser page would
pass a deny-only check, and one that quietly stopped denying would pass a
page-only check. This was verified that way when the split was introduced.
## What this does not do
## What this doesn't do
- **It does not disable local login.** Each service keeps its password
- **It doesn't disable local login.** Each service keeps its password
database and gains a second door. An identity provider that can take a
service offline when it hiccups is worse than one with two ways in.
Making authelia the only path is a separate, reversible switch per
service (tuwunel's `login_with_password`, forgejo's own setting).
- **It does not provision users.** Agents are created and destroyed
- **It doesn't provision users.** Agents are created and destroyed
continuously, so the subject set belongs to a program rather than to a
config file; today that program is `swarmctl`.

View file

@ -23,7 +23,7 @@ reader and to a certificate — so there is no separate option to pin.
> The apex must differ from `services.hyperhive.domain`. The gateway's
> default server already answers for the hive domain, and two vhosts
> claiming one `server_name` do not error — nginx picks one — so this is
> claiming one `server_name` don't error — nginx picks one — so this is
> an assertion rather than a runtime surprise.
## 🔑 You must be in the `admins` group
@ -57,8 +57,8 @@ between "an operator's page" and "anyone with a session".
## What it costs to be reachable
The apex is published to the hive's resolver like every other swarm
service, so agent containers can resolve it. That is deliberate and it is
not a hole: **reachability is not the access control here.** An agent
service, so agent containers can resolve it. that's deliberate and it's
not a hole: **reachability isn't the access control here.** An agent
that resolves the name and connects still has no operator session, and
the subrequest denies it.

View file

@ -59,7 +59,7 @@ response, so no redundant loose-end follows. Narrow best-effort race: a
actually finishes can still occasionally get both.
Tasks marked `interrupted` had their process killed by a harness
restart; a best-effort todo is still surfaced so the agent is not
restart; a best-effort todo is still surfaced so the agent isn't
silently blocked.
Exposed as `mcp__bash__status`.

View file

@ -569,7 +569,7 @@ Merge the PR
###### **Options:**
* `--method <METHOD>` — Merge strategy (default: a merge commit). Squash is not offered
* `--method <METHOD>` — Merge strategy (default: a merge commit). Squash isn't offered
Default value: `merge`
@ -580,7 +580,7 @@ Merge the PR
Rebase the head branch onto the base, then fast-forward
* `--keep-branch` — Keep the head branch after merging (deleted by default)
* `--force` — Merge even if the PR is not mergeable, CI is not green, or a review requests changes
* `--force` — Merge even if the PR isn't mergeable, CI isn't green, or a review requests changes

View file

@ -32,7 +32,7 @@ list (show/create/edit/status/merge/reviews/commits/diff/view/comment/
comments/close/reopen/labels/assign/dependency/reaction/timeline as applicable).
The flat forms are two different groups, and `--help` hides both, so the
difference cannot be seen from the CLI and is stated here instead:
difference can't be seen from the CLI and is stated here instead:
- **Renamed — these no longer run.** `issue-create` `issue-edit` `pr-create`
`pr-status` `pr-merge` `pr-reviews` `pr-commits` `pr-assign-reviewer` `diff`
@ -40,7 +40,7 @@ difference cannot be seen from the CLI and is stated here instead:
are removed outright in a later release.
- **Kind-agnostic — these stay.** `view` `comment` `comments` `close` `assign`
`labels` `timeline` are the only way to act on a number without already
knowing whether it is an issue or a PR, which no namespaced form offers.
knowing whether it's an issue or a PR, which no namespaced form offers.
Hidden, but supported; prefer `issue <verb>` / `pr <verb>` when the kind is
known, since those validate it.
@ -217,7 +217,7 @@ plain comment show under `last comment`, not `reviews`.
### Repo management
Agents **cannot create repos directly via forge token** — Forgejo
Agents **can't create repos directly via forge token** — Forgejo
disables push-to-create and the agent token doesn't have the Create
scope. Two paths exist depending on where the repo should live:
@ -225,7 +225,7 @@ scope. Two paths exist depending on where the repo should live:
MCP tool (requires the `forge` tool group). hive-c0re creates the repo in
the c0re-owned `agents/` org, adds you as a write collaborator (not
owner), and enables branch protection (operator-team merge approval
required — you cannot self-merge). Clone URL is returned immediately.
required — you can't self-merge). Clone URL is returned immediately.
This is the standard path for agents that need a working repo.
**Other repos** — Use the CLI verbs below (`repo-create` / `repo-add-collaborator`).
@ -346,7 +346,7 @@ to discover valid label names before triaging or to audit the label set.
normal query).
- ⚠️ `--assignee` / `--author` are deliberately **not** validated: someone
who has left still legitimately appears on old issues, so a login that
isn't a current member is not necessarily a typo.
isn't a current member isn't necessarily a typo.
- `list --limit N` is a *request*: the forge clamps page size to its own
`api.MAX_RESPONSE_ITEMS` (50 by default), so `--limit 400` returns at
most 50 rows. The stderr trailer reports the real total from the

View file

@ -132,7 +132,7 @@ For an existing agent, persists the token to its state dir; for a human/other ac
Show + reconcile the divergence between an agent's local applied config checkout and its forge `agent-configs/<agent>` main.
Always prints the diff first. `--from forge` resets the local checkout to forge main (effective on the next deploy); `--from local` is not supported yet. With no `--from`, prompts for the direction.
Always prints the diff first. `--from forge` resets the local checkout to forge main (effective on the next deploy); `--from local` isn't supported yet. With no `--from`, prompts for the direction.
**Usage:** `hivectl forge reconcile-config [OPTIONS] <AGENT>`
@ -406,7 +406,7 @@ Start this EXISTING agent container. Fails immediately if `name` has no config/t
###### **Options:**
* `--paused` — Start (or leave) the agent paused: if it's currently down, the pause marker is written before the container boots, so it comes up paused instead of driving turns immediately. If it's already running, this pauses it in place and does not attempt a (re)start
* `--paused` — Start (or leave) the agent paused: if it's currently down, the pause marker is written before the container boots, so it comes up paused instead of driving turns immediately. If it's already running, this pauses it in place and doesn't attempt a (re)start

View file

@ -50,7 +50,7 @@ hivectl forge reconcile-config iris --verbose # include the full diff, not
- `reconcile-config <agent>` shows the divergence between the agent's local
applied config checkout and its forge `agent-configs/<agent>` `main`, then
reconciles. `--from forge` resets the local checkout to forge `main` (takes
effect on the next deploy — it does not auto-rebuild). `--from local` is not
effect on the next deploy — it doesn't auto-rebuild). `--from local` isn't
supported yet (forge `main` is core-only branch-protected; resolve via a
config PR). With no `--from` it prompts for the direction after the diff.
@ -126,7 +126,7 @@ hivectl gateway list-users # list all usernames, one pe
```
Passwords are hashed with BCrypt (cost 12) by the daemon. The file is
created if it does not exist. Re-running `create-user` with the same
created if it doesn't exist. Re-running `create-user` with the same
username updates the password hash in place.
## Agents

View file

@ -2,7 +2,7 @@
Two tool groups govern agent lifecycle management and config changes.
Both are scoped to **direct children only** (topology-enforced: the
server rejects any name that is not a direct child of the calling
server rejects any name that's not a direct child of the calling
agent per `topology.json`). Privileged agents (e.g. ruth) may operate
on any sub-agent — the topology scope applies to all others.

View file

@ -66,7 +66,7 @@ the machine name (`h-<name>`). `lines` defaults to 50, host-capped at 500.
## `read_host_journal` capability
Capability-gated (not a tool group) — the operator enables it in the
P3RM1SS10NS C4P4B1L1T13S section. Unlike tool groups this is not
P3RM1SS10NS C4P4B1L1T13S section. Unlike tool groups this isn't
configurable from `agent.nix`.
### `get_host_journal(unit?, container?, lines?, priority?, grep?, since?, until?)`

View file

@ -27,7 +27,7 @@ swarm-level operator CLI
* `--authelia-bin <PATH>` — authelia binary used to hash passwords. The argon2 parameters must match the verifier's, so this has to be the *configured* package rather than whatever is on `PATH`
* `--users-file <PATH>` — Host-side path of authelia's users database — i.e. the path inside the container, prefixed with the container's root.
This is the only user store: it is read before every change and written in place, and `swarm-authelia-bridge` writes the same file.
This is the only user store: it's read before every change and written in place, and `swarm-authelia-bridge` writes the same file.
@ -53,7 +53,7 @@ Add a user, generating a password for them
###### **Arguments:**
* `<USERNAME>` — Login name. Conservative ASCII only — it is a YAML map key and reaches access-control rules and logs
* `<USERNAME>` — Login name. Conservative ASCII only — it's a YAML map key and reaches access-control rules and logs
###### **Options:**
@ -79,8 +79,8 @@ Every flag is optional and they compose, so one call can set several things at o
* `--display-name <TEXT>` — Name shown in the SSO UI
* `--email <ADDRESS>`
* `--add-group <GROUP>` — Repeatable. Adding a group the user is already in is not an error
* `--remove-group <GROUP>` — Repeatable. Fails if the user is not in the group — a revocation that reports success without revoking is the failure nobody re-checks
* `--add-group <GROUP>` — Repeatable. Adding a group the user is already in isn't an error
* `--remove-group <GROUP>` — Repeatable. Fails if the user isn't in the group — a revocation that reports success without revoking is the failure nobody re-checks

View file

@ -55,7 +55,7 @@ gateway, which fronts all operator traffic and is where operator auth lives,
and (b) network isolation, which keeps agents — and `hive-ci`'s untrusted PR
builds — off host-loopback so nothing can reach `127.0.0.1:<dashboard_port>`
directly. This is deliberate given the load-bearing role of network isolation
above, but it is a standing invariant: the `/api` must never be bound to a
above, but it's a standing invariant: the `/api` must never be bound to a
non-loopback address or exposed outside the gateway, and every new
operator-authority route inherits that assumption. `hive-ci` is treated like an
agent for this purpose — it runs untrusted PR code and is netns-isolated for
@ -97,7 +97,7 @@ exact same path, so the group grant always holds.
### the per-agent socket dir
`/run/hive-agent/<name>/` is shared by **three principals that share no
group**, which is why its mode is what it is:
group**, which is why its mode is what it's:
| principal | reaches | needs |
| ------------------- | ---------------------------------------- | ------------ |
@ -110,7 +110,7 @@ agent's container uid/gid** — `o=--x` is traverse without listing, and
both sockets are `0666`, which is all a dialer needs.
**Ownership is declared, not repaired.** The tmpfiles.d entry written by
`SyncAgentTmpfiles` names the uid/gid directly. Do not add a chown
`SyncAgentTmpfiles` names the uid/gid directly. don't add a chown
alongside it: `d` re-applies on every boot _and_ every agent
spawn/destroy, so ownership set afterwards is reverted the next time any
agent changes — which is exactly how this dir spent a long time at
@ -119,7 +119,7 @@ agent changes — which is exactly how this dir spent a long time at
The mode is load-bearing, not cosmetic. Write permission on a
_directory_ is what confers the right to unlink its entries, whoever owns
them, and the sticky bit is the only thing that would restrain that (it
is not set here). A world-writable socket dir therefore lets anything
isn't set here). A world-writable socket dir therefore lets anything
able to reach the path delete an agent's socket and bind its own — and
nginx reaches all of `/run/hive-agent` as a plain host path. Dropping
`o=w` removes that permission rather than qualifying it.
@ -142,11 +142,11 @@ This dir has exactly one writer, so it needs no world write at all.
`hivectl` drives the whole hive — spawn / kill / destroy / rebuild /
deploy — over the **host admin socket** `/run/hyperhive/host.sock`,
socket-activated by the `hive-c0re.socket` unit. That socket _is_ the
full-control surface, so who can connect to it is a real trust
full-control surface, so who can connect to it's a real trust
boundary.
By default the socket is `0660` group-owned by **`hive-admin`**, an
empty group — so it is effectively **root-only** until an operator is
empty group — so it's effectively **root-only** until an operator is
explicitly granted access. Grant sudoless `hivectl` by listing login
users in `services.hyperhive.c0re.adminUsers`; each is added to
`hive-admin`, and members connect without `sudo`. The runtime dir

View file

@ -4,14 +4,14 @@
The sections below document specific mechanisms (the state-file endpoint,
nixbld isolation, privilege separation). This section frames the model they
serve: **what hyperhive defends, what it deliberately does not, and where the
operator is accepting risk.** It is the reference for "is it safe to give an
serve: **what hyperhive defends, what it deliberately doesn't, and where the
operator is accepting risk.** it's the reference for "is it safe to give an
agent capability X?".
### The trust boundary is the container, not credential storage
An agent is **trusted code running inside its own nspawn container**. The
boundary that matters is the container: a sub-agent cannot see the host
boundary that matters is the container: a sub-agent can't see the host
netns, another agent's container, or another agent's state dir. Within its
own container the agent is privileged — it has **passwordless `sudo` by
default**. Isolating credentials _from the agent itself_ is therefore **not a
@ -20,10 +20,10 @@ run arbitrary commands as root inside its container. (The narrow exception is
_cross-tenant_ leakage — e.g. the unsandboxed-nix-build `0600` token policy
below stops a build's nixbld user reading the agent's own forge token, and the
state-file endpoint stops one agent proxying another's files. Those harden the
boundary; they do not sandbox the agent from itself.)
boundary; they don't sandbox the agent from itself.)
The corollary: **don't reason about security as "can the agent be stopped from
touching its credentials". Reason about it as "what is the blast radius if this
touching its credentials". Reason about it as "what's the blast radius if this
agent does the worst possible thing with everything it can reach".**
### Scoped tokens bound the blast radius
@ -31,7 +31,7 @@ agent does the worst possible thing with everything it can reach".**
Each agent gets its own scoped credentials, never shared:
- **forge token** → that agent's Forgejo account only (its own repos +
collaborator grants; cannot act as another agent or as `core`).
collaborator grants; can't act as another agent or as `core`).
- **matrix token** → that agent's matrix account only.
So a compromised/confused agent's reach on the forge or matrix is bounded by
@ -66,7 +66,7 @@ checkpoints**, not about sandboxing the agent from its own tools:
merges restricted to the operators team + a required operators-team approval
(`apply_operator_branch_protection` / the config-repo equivalent) — and an
agent (a write collaborator, not a repo admin) can neither change those
settings nor merge its own PR. It is **not** set up for external VCS (GitHub
settings nor merge its own PR. it's **not** set up for external VCS (GitHub
etc.), though — there, operator-merge is process + accepted risk, not a
technical control.
- **Approvals** — config changes, schedule additions, and other
@ -107,7 +107,7 @@ before touching the filesystem:
(canonical host form or the in-container view `/agents/<n>/state/`)
- `/var/lib/hyperhive/shared/` — shared docs (`/shared/` in-container)
`/state/...` without an agent prefix is explicitly _not_ accepted — it is
`/state/...` without an agent prefix is explicitly _not_ accepted — it's
ambiguous from the host's perspective.
Defense-in-depth layers (in order):
@ -145,7 +145,7 @@ before building `agent_notes_dir(name)`, so a `..` component can't traverse.
Agent containers bind-mount the host's `nix-daemon` socket. The host daemon may
have `sandbox-fallback = false` (strict NixOS defaults), which causes `nix build`
inside nspawn containers to fail — containers lack kernel user namespaces, so nix
cannot set up its build sandbox. the agent modules set `sandbox-fallback = true`
can't set up its build sandbox. the agent modules set `sandbox-fallback = true`
so that builds fall back to unsandboxed execution rather than failing outright.
### Threat model
@ -156,22 +156,22 @@ any file in the container that the nixbld user can read.
The blast radius also has a **network** dimension. hive-ci runs its unsandboxed
builds of untrusted PR code in its own private netns behind the hive bridge: a
build reaches the forge only through the gateway and cannot reach host-loopback
build reaches the forge only through the gateway and can't reach host-loopback
services — including the core dashboard at `127.0.0.1:<dashboard_port>`, which
has no application-layer auth of its own (see [`docs/scheduler/ci.md`](../scheduler/ci.md)). The `0600`
token policy bounds file reads; network isolation bounds network reach.
**What is NOT exposed**:
**what's NOT exposed**:
- `/home/<name>/.claude/` — mode `0700`, owned by the per-agent
user `<name>`. nixbld users cannot read it.
user `<name>`. nixbld users can't read it.
- `$HYPERHIVE_STATE_DIR/forge-token` (= `/agents/<name>/state/forge-token`)
— written at mode `0600` and chowned to the per-agent uid:gid (see
`hive-c0re/src/forge/mod.rs`'s module doc for exactly where). nixbld users
cannot read it.
can't read it.
**Policy**: all credential files written to agent state directories MUST be mode
`0600` or stricter. Do not create world-readable secret files in agent state dirs.
`0600` or stricter. don't create world-readable secret files in agent state dirs.
### Long-term fix
@ -185,14 +185,14 @@ bind-mount compatibility with user namespace UID mapping and is tracked as a TOD
### Background
`hive-c0re` runs as the unprivileged system user `hive-core`
(`/var/lib/hyperhive` owned by `hive-core:hive-core`). It cannot
(`/var/lib/hyperhive` owned by `hive-core:hive-core`). It can't
directly invoke `nixos-container`, `journalctl -M`, or act on a system
unit (`systemctl reload nginx`) — those require root. `hive-priv` fills
this gap.
⚠️ **`ReloadGatewayNginx` acts on a host unit, so nothing implicitly
scopes it.** Its containment is the unit name hard-coded in `hive-priv`:
a caller cannot name the unit, so the verb cannot be steered at another
a caller can't name the unit, so the verb can't be steered at another
service. **A privileged verb needs something bounding what it can act
on; when that isn't a namespace, it has to be a constant the caller
can't supply.**
@ -228,12 +228,12 @@ known operations; there is no arbitrary command pass-through:
allowlist before any operation: only names matching the agent-name
convention (char-validated) or the known sibling service containers
(`hive-forge`, `hive-matrix`, `hive-ci`) are accepted. `hive-gateway` is
a host unit, not a container, so it is not in this list — see
a host unit, not a container, so it's not in this list — see
`ReloadGatewayNginx` above for how its access is scoped instead.
Arbitrary container names are rejected.
**Socket-activated** — systemd starts `hive-priv` on the first
incoming connection (`LISTEN_FDS=1`); it is not running between calls.
incoming connection (`LISTEN_FDS=1`); it's not running between calls.
The `ProtectSystem=strict` + `ReadWritePaths` sandbox limits filesystem
writes to only the paths `hive-priv` legitimately needs.

View file

@ -109,7 +109,7 @@ user-configured MCP server can push its own todos the same way. See
what each built-in todo producer watches and how the store + `get_loose_ends`
merge work.
Plugin install failures are not fatal: each entry comes back as a
Plugin install failures aren't fatal: each entry comes back as a
human-readable failure string that gets routed via
`Surface::send_to_parent` to the agent's topology parent (the
broker resolves `<parent>` per `topology::resolve_recipient`; root

View file

@ -119,7 +119,7 @@ Constraints: `target` must start with an alphanumeric or `_` and
contain only alphanumerics, `_`, `.`, `/`, `-`. `..` segments are
rejected by a config assertion. The merge step refuses to overwrite
files already present in the default dist — pick a target name that
does not collide with existing paths (`static/`, `index.html`, etc.).
doesn't collide with existing paths (`static/`, `index.html`, etc.).
The default dist ships at `hyperhive.frontend.dist` (the
`hyperhive-frontend` package output, read-only). To replace the
@ -150,8 +150,8 @@ loopback default would only ever be correct when the forge shares the
agent's network namespace, and inside a container `localhost` is the
agent itself, so the default was a value that built fine and then talked
to the wrong machine. With `null` the `tea-login` and `forge-avatar-sync`
units are not generated at all: an absent integration rather than a
misdirected one. You do not normally set this — hive-c0re renders the
units aren't generated at all: an absent integration rather than a
misdirected one. You don't normally set this — hive-c0re renders the
host's real forge URL into every agent, and refuses to write a meta
flake without one, so `null` only survives where the agent modules are
evaluated outside a hive.
@ -201,14 +201,14 @@ hyperhive.claudePluginsAutoUpdate = false; # default
- **`claudePlugins`** — list of plugin specs passed to
`claude plugin install <spec>`. Each spec is installed on every boot
(`install` is expected to be idempotent); failures log a warning but
do not abort boot. Defaults to Anthropic's `skill-creator` (so every
don't abort boot. Defaults to Anthropic's `skill-creator` (so every
agent can author, refine, and evaluate its own skills) plus
hyperhive's own `base` plugin — skills that apply to every agent
regardless of role (currently just `state-hygiene`) — all without
any per-agent wiring.
> Both plugin lists follow ordinary NixOS list-option semantics: a
> per-agent definition **replaces** the default, it does not extend it.
> per-agent definition **replaces** the default, it doesn't extend it.
> An agent that sets `claudePlugins` and still wants the defaults has
> to list `skill-creator@claude-plugins-official` and `base@hyperhive`
> explicitly alongside its own entries — likewise for the two default
@ -238,7 +238,7 @@ Non-compile subcommands (`new`, `add`, third-party `cargo-*`) are
left untouched.
Set to `false` for agents that parse cargo's JSON output
programmatically and do not pass `--message-format json` themselves.
programmatically and don't pass `--message-format json` themselves.
## API-key backend (`useApiKey` / `backendEnvironmentFile`)
@ -288,7 +288,7 @@ The file lives in the agent's bind-mounted **harness** dir, not `state/`
(see `docs/agent-lifecycle/persistence.md`'s "Parent access to child state"), so an API
key placed here is reachable by nothing but this agent and the host.
⚠️ Verified end-to-end against OpenRouter has not happened as of this
⚠️ Verified end-to-end against OpenRouter hasn't happened as of this
writing — `ANTHROPIC_BASE_URL` support in the shipped Claude CLI is
documented behavior, not something this hive has run a live turn
against yet. Tool use, streaming, and MCP all need to keep working

View file

@ -83,7 +83,7 @@ at_unix_timestamp?)`.
`cancel_loose_end`. Omit `agent` to list your own threads. Pass
`agent: "<name>"` to inspect a direct child agent (always accessible
per topology enforcement); non-children require the
`query_agent_state` capability. The `"*"` hive-wide query is not
`query_agent_state` capability. The `"*"` hive-wide query isn't
available on the agent socket.
- `cancel_loose_end` — hard-delete a `reminder`, cancel a pending
`approval` row, or clear a `todo` row (loose-ends-v2). Agents may
@ -139,7 +139,7 @@ hive_name?, swarm_name?, matrix_accounts? }`. `matrix_accounts` is a
disabled for agents). The repo is created in the c0re-owned `agents`
org; the calling agent gets write collaborator access; the default
branch is branch-protected (operator-team must approve merges, so the
agent cannot self-merge). Opt-in; not in any default preset.
agent can't self-merge). Opt-in; not in any default preset.
See [`docs/tools/forge.md — Repo management`](../tools/forge.md).
- **Web egress** (`web_tools`) — enables Claude's built-in `WebFetch`
and `WebSearch` tools (not MCP tools; added directly to the

View file

@ -203,7 +203,7 @@ operations: rebuilds, meta-update cascades, and first-spawns. One
operation runs at a time; the worker drains FIFO. **Is a mounted
`JobqGraph`** (the shared generic graph-viewer Preact component,
`@hive/shared/jobq-graph.js` — the same one swarm-ui uses) —
`builds.js` does not render the queue itself; it just mounts the component
`builds.js` doesn't render the queue itself; it just mounts the component
(`mountJobqGraph(container, props)`, since this page has no JSX
pipeline) with an `onUpdate` callback to drive the two things below it
that the generic view doesn't show. The component owns fetching, cold
@ -234,7 +234,7 @@ deep-link** — the generic graph wire doesn't carry those fields, and
rows are meant to present exactly what the endpoint provides rather
than reconstruct chrome the backend no longer sends. Settled entries
render their **full step tree**, not just a bare summary — the wire
does not filter `Done` nodes out.
doesn't filter `Done` nodes out.
**State filter (hyperhive#2606).** A row of per-state checkboxes above
the tree — one per lifecycle state, matching the row glyphs — lets the
@ -355,7 +355,7 @@ internal failure); the page reads `detail` for display. The host coordinator per
the agent's `matrixAccounts.<account>.tokenFile` via the same
privileged write path as the hive-internal `matrix-token`; the token is
**never** echoed back, and the page clears the secret inputs on submit
regardless of outcome. The account list reflects what is *provisioned*
regardless of outcome. The account list reflects what's *provisioned*
(an account with a stored token), so a config-declared-but-unprovisioned
account appears only once it has been provisioned through the form.
@ -525,7 +525,7 @@ Per-schedule action column:
(one-shots may also be paused to delay a one-time fire, so the
button always appears). Paused rows show a `⏸ paused` badge in the
`next` column and are sorted to the middle bucket (after active,
before cancelled). A paused schedule does not fire regardless of
before cancelled). A paused schedule doesn't fire regardless of
`next_fire_at`; resuming (`▶`) clears the pause without adjusting
the next-fire time. POSTs to `/api/schedules/{id}/pause` or
`/api/schedules/{id}/resume`.
@ -846,13 +846,13 @@ progresses (a rebuild reads `stop_for_update`, then `swap`, then
`reconcile` rather than one constant `rebuilding` for its whole
life). Two consequences for anything rendering it:
- The label vocabulary is **open** — it is the node's own wire tag
- The label vocabulary is **open** — it's the node's own wire tag
(`NodeKind::as_str`, the same strings the graph wire's node
labels carry), not a fixed set. Treat it as an opaque display
string; do not switch on specific values. `restarting` in
string; don't switch on specific values. `restarting` in
particular no longer exists, because no node kind is unique to a
restart.
- It is **not** exclusively operator-initiated, and **not** limited
- it's **not** exclusively operator-initiated, and **not** limited
to rebuild-shaped work — `running_transients()` on the backend is
a status-only test (any `Running` node whose payload names a
non-empty agent lights a pill), so work the operator never
@ -1135,7 +1135,7 @@ below — some endpoints aren't in it yet.
Refuses `Running` / terminal-state entries (in-flight
rebuilds can't be safely interrupted). Always 200; body is
`{"cancelled": true}` on a successful flip or
`{"cancelled": false}` when the entry was not in `Queued` state.
`{"cancelled": false}` when the entry wasn't in `Queued` state.
- `POST /api/agent/{name}/mark-all-read` — ack all pending broker
messages for `{name}`. Backfills `delivered_at` for rows
not yet delivered and sets `acked_at = now`. Returns
@ -1287,7 +1287,7 @@ below — some endpoints aren't in it yet.
the audit trail distinguishes scheduled fires from operator-
triggered ones.
- `POST /api/schedules/{id}/pause` — pause a recurring or
one-shot schedule. A paused schedule does not fire regardless
one-shot schedule. A paused schedule doesn't fire regardless
of `next_fire_at`; the row gains a `⏸ paused` badge.
- `POST /api/schedules/{id}/resume` — resume a paused schedule.
Clears the pause flag without adjusting `next_fire_at`.

View file

@ -225,7 +225,7 @@ returns inside the builder still commit whatever was appended
before they returned.
**Keyed DOM caching:** for sections whose rows hold interactive state
(textarea drafts, checkboxes, focused inputs) `paintAtomic` is not
(textarea drafts, checkboxes, focused inputs) `paintAtomic` isn't
enough — wiping and rebuilding still destroys the state even if the
flash is hidden. The keyed pattern keeps a `Map<id, {el, fingerprint}>`
where the fingerprint is `JSON.stringify({...visible fields...})`.
@ -234,7 +234,7 @@ textarea draft, checkbox state, and event listeners); only cache-miss
rows are rebuilt and inserted. Used for: containers (`containerRowCache`),
rebuild-queue entries (`rebuildQueueRowCache`). The spawn-form input+focus and meta-input
checkboxes use a lighter snapshot-then-restore pattern (snapshot
before `replaceChildren`, restore after) since they are single
before `replaceChildren`, restore after) since they're single
values rather than per-row caches.
**`<details>` open-state preservation:** any collapsible element