diff --git a/.prettierignore b/.prettierignore index 8aab0453..eed7adfa 100644 --- a/.prettierignore +++ b/.prettierignore @@ -4,25 +4,25 @@ # generator, not the formatter, owns these files' layout. Every CLI # reference doc belongs here — adding one and forgetting this line is # what made swarmctl's fail. -docs/tools/hivectl-cli.md -docs/tools/swarmctl-cli.md -docs/tools/forge-cli.md -docs/tools/swarm-logs-cli.md -docs/tools/swarm-metrics-cli.md +docs/2-handbook/reference/hivectl-cli.md +docs/2-handbook/reference/swarmctl-cli.md +docs/2-handbook/reference/forge-cli.md +docs/2-handbook/reference/swarm-logs-cli.md +docs/2-handbook/reference/swarm-metrics-cli.md # Files with multi-line list-item continuations that prettier strips to col 0. # prettier's `proseWrap: "preserve"` prevents prose reflow but not list-item # indentation normalisation, so these files need manual re-wrap before # adding them back to the formatter scope. AGENTS.md -docs/agent-lifecycle/config-changes.md -docs/process/conventions.md -docs/networking/gateway.md -docs/integrations/matrix.md -docs/agent-lifecycle/persistence.md -docs/web-ui/terminal-rendering.md -docs/tools/forge.md -docs/tools/matrix.md -docs/turn-loop/README.md -docs/web-ui/agent.md -docs/web-ui/dashboard.md +docs/2-handbook/agent-lifecycle/config-changes.md +docs/5-project/process/conventions.md +docs/4-implementation/networking/gateway.md +docs/2-handbook/integrations/matrix.md +docs/2-handbook/agent-lifecycle/persistence.md +docs/4-implementation/web-ui/terminal-rendering.md +docs/2-handbook/tools/forge.md +docs/2-handbook/tools/matrix.md +docs/4-implementation/turn-loop/README.md +docs/2-handbook/web-ui/agent.md +docs/2-handbook/web-ui/dashboard.md diff --git a/AGENTS.md b/AGENTS.md index e4a27928..a14a9879 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -10,20 +10,20 @@ when you need depth on a subsystem. This file is the index. the env var — inside an agent container `localhost` is the *agent*, not the forge, so a hardcoded loopback address fails to connect. - Operator/agent trust-boundary design: - **[docs/trust-boundary/boundary.md](docs/trust-boundary/boundary.md)** (`area/ops` issues + **[docs/2-handbook/trust-boundary/boundary.md](docs/2-handbook/trust-boundary/boundary.md)** (`area/ops` issues for the deployment/gateway/privsep work — the separator is a **slash**, and `--label area:ops` now fails with `did you mean "area/ops"?`). - Agent trust model (trust boundary, prompt-injection threat model, capability = accepted risk), credential isolation + sandbox threat model: - **[docs/trust-boundary/security.md](docs/trust-boundary/security.md)**. + **[docs/2-handbook/trust-boundary/security.md](docs/2-handbook/trust-boundary/security.md)**. - Odd-looking nspawn / container plumbing usually has a story: - **[docs/process/gotchas.md](docs/process/gotchas.md)**. + **[docs/5-project/process/gotchas.md](docs/5-project/process/gotchas.md)**. ## Build & check Always through the devshell — a bare `cargo` fails (`failed to find tool "cc"`, `cannot find -lsqlite3`). Details: -[`docs/process/conventions.md`](docs/process/conventions.md#building--local-checks). +[`docs/5-project/process/conventions.md`](docs/5-project/process/conventions.md#building--local-checks). ```sh nix develop -c cargo clippy --all-targets -- -D warnings @@ -33,8 +33,8 @@ nix fmt # treefmt is the CI gate, NOT bare cargo fmt Changed a `hivectl` / `swarmctl` verb or flag? Regenerate its doc — only CI's `nix flake check` catches the drift: -`nix develop -c cargo run --bin hivectl -- markdown-docs > docs/tools/hivectl-cli.md` -(same for `swarmctl` → `docs/tools/swarmctl-cli.md`). CI red but local +`nix develop -c cargo run --bin hivectl -- markdown-docs > docs/2-handbook/reference/hivectl-cli.md` +(same for `swarmctl` → `docs/2-handbook/reference/swarmctl-cli.md`). CI red but local green → build the suspect check (e.g. `nix build .#checks.x86_64-linux.hivectl-docs`) before blaming the builder. @@ -58,10 +58,10 @@ Per-hive (host + in-container): crate. - **`hivectl/`** — operator CLI; talks to `hive-c0re` over the host admin socket, does NOT link it. Verb reference (CI-enforced): - [`docs/tools/hivectl-cli.md`](docs/tools/hivectl-cli.md). + [`docs/2-handbook/reference/hivectl-cli.md`](docs/2-handbook/reference/hivectl-cli.md). - **`hive-priv/`** — minimal root helper, socket-activated at `/run/hive/priv.sock`; the few root ops (bind-mount edits, nsenter) - `hive-c0re` delegates. See `docs/trust-boundary/boundary.md`. + `hive-c0re` delegates. See `docs/2-handbook/trust-boundary/boundary.md`. - **`hive-agent/`** — the in-container serve loop: turn-loop *policy* (`turn.rs`) over `hive-runtime`, per-agent web UI, event + turn-stats sinks, system-prompt renderer, claude launch config (`mcp_config.rs`). @@ -130,7 +130,7 @@ Swarm-level: **`users.yml` in place**, shared with `swarm-authelia-bridge`; that crate's README says what both writers must uphold. Verb reference (CI-enforced): - [`docs/tools/swarmctl-cli.md`](docs/tools/swarmctl-cli.md). + [`docs/2-handbook/reference/swarmctl-cli.md`](docs/2-handbook/reference/swarmctl-cli.md). - **`swarm-authelia-bridge/`** (+ **`swarm-authelia-bridge-sock/`** wire types) — the other writer of authelia's users DB; runs inside `swarm-authelia`. @@ -164,20 +164,20 @@ Swarm-level: - **`swarm-builder/`** — builds agent config PR heads for commit status, builds each agent from `internal/meta` before its deploy, and evaluates the agent archetypes the create form lists. - Design: [`docs/swarm/builder.md`](docs/swarm/builder.md). + Design: [`docs/4-implementation/swarm/builder.md`](docs/4-implementation/swarm/builder.md). - **`swarm-ci-metrics/`** — polls the forge's Actions runs and pushes each finished run's duration and queue wait to the metrics store. Design: - [`docs/swarm/ci-metrics.md`](docs/swarm/ci-metrics.md). + [`docs/4-implementation/swarm/ci-metrics.md`](docs/4-implementation/swarm/ci-metrics.md). - **`swarm-claude-login/`** — runs `claude auth login` for one agent and stores the credentials at `swarm/agents//claude-credentials`. - Design: [`docs/swarm/claude-login.md`](docs/swarm/claude-login.md). + Design: [`docs/2-handbook/swarm/claude-login.md`](docs/2-handbook/swarm/claude-login.md). - **`swarm-notify-router/`** — forge events from swarm-controller's webhooks to todos on the agents they concern, through each agent's own consumer on the `ROUTER_TODOS` stream. Design: - [`docs/swarm/notify-router.md`](docs/swarm/notify-router.md). + [`docs/4-implementation/swarm/notify-router.md`](docs/4-implementation/swarm/notify-router.md). - **`swarm-review-classifier/`** — one `claude --print` per PR classification, no tools, no forge credential. Design: - [`docs/swarm/review-classifier.md`](docs/swarm/review-classifier.md). + [`docs/4-implementation/swarm/review-classifier.md`](docs/4-implementation/swarm/review-classifier.md). ### External dependencies with no directory here @@ -213,7 +213,7 @@ in this file and drifted. - **Never add `#[allow(clippy::…)]`** — fix the lint instead (extract a helper, add backticks, etc.). Worked examples: - [`docs/process/conventions.md`](docs/process/conventions.md#building--local-checks). + [`docs/5-project/process/conventions.md`](docs/5-project/process/conventions.md#building--local-checks). - **Pre-push lint hook** (catches tracker-tag and comment-block failures before CI does — install once per clone): `ln -sf ../../scripts/pre-push .git/hooks/pre-push` @@ -225,7 +225,7 @@ in this file and drifted. - **Operator-facing doc with real implementation detail mixed in**: wrap the implementation part in a `
…` block (blank line after `` so the markdown inside renders) rather - than a sibling page — see `docs/integrations/github.md`'s + than a sibling page — see `docs/2-handbook/integrations/github.md`'s "Implementation" section. It only collapses in a rendered browser, and the `` line isn't markdown-processed, so keep it plain prose with no backticks. diff --git a/Cargo.toml b/Cargo.toml index 7c137435..68d0c5df 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -290,5 +290,5 @@ x509-cert = { version = "0.2", default-features = false, features = ["pem"] } # Test doubles with expectations (canned replies, call counts, argument # matchers). Dev-dependency only, and used behind `#[cfg(test)]`, so release # builds never compile it. See "Testing seams and test doubles" in -# `docs/process/conventions.md`. +# `docs/5-project/process/conventions.md`. mockall = "0.15" diff --git a/README.md b/README.md index d5e83869..327c2243 100644 --- a/README.md +++ b/README.md @@ -81,24 +81,24 @@ box it gets: - **messaging** — peers and the operator talk over matrix; `send` / `recv` is the hive-local broker for system traffic and agents without matrix, `remind` to wake itself, scheduled prompts the operator sets for it at swarm level, and a todo list that forge notifications and other - producers feed. → [schedules](docs/swarm/schedules.md) + producers feed. → [schedules](docs/2-handbook/swarm/schedules.md) - **background bash** — long-running commands as tracked tasks (`run` / `status` / `kill`), so a build doesn't block the turn. → - [bash](docs/tools/bash.md) + [bash](docs/2-handbook/tools/bash.md) - **subagents** — nested sessions on the agent's own runtime, optionally re-prompted toward a goal over several turns. → - [subagent](docs/tools/subagent.md) + [subagent](docs/2-handbook/tools/subagent.md) - **forge** — its own Forgejo account, the `hive-forge` CLI, git credentials, and notifications turned into todos. Optional linked accounts on other forges, GitLab (`glab` + push), and GitHub (`gh` + push) via `services.hyperhive.agent.github.enable`. → - [forge](docs/integrations/forge.md) · [gitlab](docs/integrations/gitlab.md) · - [github](docs/integrations/github.md) + [forge](docs/2-handbook/integrations/forge.md) · [gitlab](docs/2-handbook/integrations/gitlab.md) · + [github](docs/2-handbook/integrations/github.md) - **matrix** — its own account(s) on the swarm homeserver, as MCP tools. → - [matrix](docs/tools/matrix.md) + [matrix](docs/2-handbook/tools/matrix.md) - **knowledge** — the swarm's shared `internal/knowledge` repo, read-only at `/knowledge`, plus hyperhive's own docs in the container. → - [knowledge](docs/integrations/knowledge.md) + [knowledge](docs/2-handbook/integrations/knowledge.md) - **web UI** — a per-agent page behind the gateway: live terminal of the turn stream, model / effort switching, cancel. - **GUI** (opt-in, `services.hyperhive.agent.gui.enable`) — a Wayland @@ -106,7 +106,7 @@ box it gets: the web UI. - **reverse engineering** (opt-in, `services.hyperhive.agent.rea.enable`, or the `reverse-engineer` archetype) — REA and Ghidra 12.1 in the container, with - their MCP tools allowed. → [archetypes](docs/swarm/archetypes.md) + their MCP tools allowed. → [archetypes](docs/2-handbook/swarm/archetypes.md) - **your own tools** — any extra MCP server (stdio or http) via `services.hyperhive.agent.extraMcpServers`, and any NixOS config you'd put on a machine. @@ -114,7 +114,7 @@ box it gets: ## Quick start: an all-local swarm The whole swarm on one box — a hive plus every shared service. Good for a -dev box or a single-host deployment; [`docs/swarm/`](docs/swarm/README.md) +dev box or a single-host deployment; [`docs/swarm/`](docs/2-handbook/swarm/README.md) covers spreading a swarm across hosts. ```nix @@ -158,7 +158,7 @@ After `nixos-rebuild switch` the swarm UI answers on the swarm domain, and the hive brings up the manager agent (`ruth`). A few steps stay deliberately human — initialising the secret store, giving ruth her store identity, creating your SSO account. Walk through -**[`docs/getting-started/setup.md`](docs/getting-started/setup.md)** +**[`docs/1-getting-started/setup.md`](docs/1-getting-started/setup.md)** once, top to bottom. ## Operating it @@ -167,14 +167,14 @@ once, top to bottom. (`admins` group): hive roster and status, every agent with its live terminal and wanted state, creating agents, linking external forge and matrix accounts, the job graph, a cross-repo issue report. → - [`docs/swarm/ui.md`](docs/swarm/ui.md) + [`docs/2-handbook/swarm/ui.md`](docs/2-handbook/swarm/ui.md) - **`swarmctl`** — the swarm's CLI, on the swarm-controller host: create agents and mint their identities, manage SSO users, make forge admins. - → [reference](docs/tools/swarmctl-cli.md) + → [reference](docs/2-handbook/reference/swarmctl-cli.md) - **per-hive** — each hive's dashboard (container state, rebuild queue) and `hivectl` for host-level administration and - container shells. → [`docs/web-ui/`](docs/web-ui/README.md) · - [hivectl](docs/tools/hivectl.md) + container shells. → [`docs/web-ui/`](docs/2-handbook/web-ui/README.md) · + [hivectl](docs/2-handbook/tools/hivectl.md) You are an SSO user too: `swarmctl user add --group admins`, and a first login through authelia creates your forge and matrix accounts; @@ -187,10 +187,10 @@ one becomes a homeserver admin within five minutes of that first login. pick the page for your task rather than reading front to back. Some common entry points: -- connecting hives into a swarm → [`swarm/`](docs/swarm/README.md) -- how config changes flow → [`agent-lifecycle/config-changes.md`](docs/agent-lifecycle/config-changes.md) -- trust boundary + threat model → [`trust-boundary/`](docs/trust-boundary/boundary.md) -- what an agent's turn looks like → [`turn-loop/`](docs/turn-loop/README.md) +- connecting hives into a swarm → [`swarm/`](docs/2-handbook/swarm/README.md) +- how config changes flow → [`agent-lifecycle/config-changes.md`](docs/2-handbook/agent-lifecycle/config-changes.md) +- trust boundary + threat model → [`trust-boundary/`](docs/2-handbook/trust-boundary/boundary.md) +- what an agent's turn looks like → [`turn-loop/`](docs/4-implementation/turn-loop/README.md) - every `services.hyperhive.*` option → [options reference](https://hyperhive.darkest.space/options/) ## Hacking on it @@ -205,6 +205,6 @@ sudo nixos-rebuild switch --flake .# ``` Conventions (commit style, no `#[allow(clippy::…)]`, the pre-push hook) -live in [`docs/process/conventions.md`](docs/process/conventions.md). +live in [`docs/5-project/process/conventions.md`](docs/5-project/process/conventions.md). Issues + backlog are on the [forge](https://forge.darkest.space/hyperhive/hyperhive/issues). diff --git a/docs/getting-started/nixos.md b/docs/1-getting-started/nixos.md similarity index 95% rename from docs/getting-started/nixos.md rename to docs/1-getting-started/nixos.md index a116ff7b..eadf59ef 100644 --- a/docs/getting-started/nixos.md +++ b/docs/1-getting-started/nixos.md @@ -84,11 +84,11 @@ Run `nixos-rebuild switch --flake .#my-host`. The swarm UI then answers at Leave the other options at their defaults until you need them: - Spreading a swarm over more than one host, and the credentials a split swarm - needs: [`swarm/README.md`](../swarm/README.md) and - [`swarm/credentials.md`](../swarm/credentials.md). + needs: [`swarm/README.md`](../2-handbook/swarm/README.md) and + [`swarm/credentials.md`](../2-handbook/swarm/credentials.md). - Every option, with its type and default: [the options reference](https://hyperhive.darkest.space/options/). -- The secret store's seal and the granter: [`swarm/bao.md`](../swarm/bao.md). +- The secret store's seal and the granter: [`swarm/bao.md`](../4-implementation/swarm/bao.md). ## Next diff --git a/docs/getting-started/setup.md b/docs/1-getting-started/setup.md similarity index 91% rename from docs/getting-started/setup.md rename to docs/1-getting-started/setup.md index a30ffbb7..b3fdf7e2 100644 --- a/docs/getting-started/setup.md +++ b/docs/1-getting-started/setup.md @@ -10,7 +10,7 @@ Every command runs as **root on the host**. > **Not all-local?** Each step says which host it runs on. A split swarm > also has credentials you can't generate where they're read — each -> host's mTLS identity at the secret store. Read [`swarm/credentials.md`](../swarm/credentials.md) first; it +> host's mTLS identity at the secret store. Read [`swarm/credentials.md`](../2-handbook/swarm/credentials.md) first; it > says which files you place, and where. ## 1 · Secret store @@ -53,7 +53,7 @@ active` while the store comes up; the units retry every 30s for a day. With the default `pkcs11` seal the store unseals itself from here on. With `deploy.bao.seal = "shamir"`, run `bao operator unseal` after every restart. What the granter is, why it's root-equivalent, and the store's TLS: -[`swarm/bao.md`](../swarm/bao.md). +[`swarm/bao.md`](../4-implementation/swarm/bao.md). ## 2 · Your SSO account @@ -71,13 +71,13 @@ It prints a generated password once — record it. Keep both flags: - **`--email`** — the forge won't create an account without one. Fix either afterwards with `swarmctl user update mara --add-group admins ---email …`. Details: [`swarm/sso.md`](../swarm/sso.md). +--email …`. Details: [`swarm/sso.md`](../2-handbook/swarm/sso.md). ## 3 · Swarm UI login Open `https:///` and sign in with the account from step 2. The name resolves on the box itself through `/etc/hosts`; from anywhere else it -needs a real DNS record. → [`swarm/ui.md`](../swarm/ui.md) +needs a real DNS record. → [`swarm/ui.md`](../2-handbook/swarm/ui.md) Signing in to the forge once through SSO creates your forge account under the same username. Then, _on the swarm-controller's host_: @@ -114,7 +114,7 @@ container keeps starting with the hive meanwhile. From here on, ask ruth. Open her card on `/agents` in the swarm UI and use the message box under her terminal. She is the manager agent: her MCP surface is the privileged one that configures agents, and she can recover the others. Changes to -an agent's config arrive as PRs you merge. → [`agent-lifecycle/agent-roster.md`](../agent-lifecycle/agent-roster.md) +an agent's config arrive as PRs you merge. → [`agent-lifecycle/agent-roster.md`](../2-handbook/agent-lifecycle/agent-roster.md) To create an agent yourself instead, use the swarm UI, or: @@ -129,10 +129,10 @@ swarm UI's job view for progress. Later config changes are PRs on `agent-configs/iris`; you merge them on the forge, and the merge deploys them. → -[`agent-lifecycle/config-changes.md`](../agent-lifecycle/config-changes.md#config-changes) +[`agent-lifecycle/config-changes.md`](../2-handbook/agent-lifecycle/config-changes.md#config-changes) ## After setup: use the swarm UI Day-to-day operation happens in the swarm UI at `https:///`: agents, hives, jobs and messages. Swarm-level operations take over from the -per-hive dashboard and `hivectl`. → [`swarm/ui.md`](../swarm/ui.md) +per-hive dashboard and `hivectl`. → [`swarm/ui.md`](../2-handbook/swarm/ui.md) diff --git a/docs/agent-lifecycle/agent-roster.md b/docs/2-handbook/agent-lifecycle/agent-roster.md similarity index 95% rename from docs/agent-lifecycle/agent-roster.md rename to docs/2-handbook/agent-lifecycle/agent-roster.md index 26746ae3..000a2f10 100644 --- a/docs/agent-lifecycle/agent-roster.md +++ b/docs/2-handbook/agent-lifecycle/agent-roster.md @@ -13,7 +13,7 @@ This doc covers what the roster file holds, the map-shaped alternative ## Where the roster lives "The roster" means two different, non-overlapping things depending on -scope. The [README](../../README.md)'s control-plane row ("`swarm-controller` +scope. The [README](../../../README.md)'s control-plane row ("`swarm-controller` holds the hive directory, the agent roster and the job graph") is the **swarm-wide** one: `GET /api/agents` on swarm-controller reads straight from the swarm's authelia identity store over @@ -96,7 +96,7 @@ state dirs work as for every other agent. into the caller's own inbox) is on this socket too, unprivileged on both flavours; matrix, bash and forge don't call it — their notifications push a todo on the harness's in-agent socket (see - [`docs/turn-loop/mcp.md`](../turn-loop/mcp.md#waking-the-agent-from-inside-the-container)). + [`docs/4-implementation/turn-loop/mcp.md`](../../4-implementation/turn-loop/mcp.md#waking-the-agent-from-inside-the-container)). - **Storage/mounts** — the `manage_root_agent` permission, which the swarm UI's setup assistant creates ruth with, gets a container `/var/lib/hyperhive/agents` bind-mounted RW at `/agents` (so it can @@ -115,7 +115,7 @@ state dirs work as for every other agent. from an [archetype](../swarm/archetypes.md) is told it has no config repo. - **Tool allow-list** — the MCP tools claude sees come from the agent's permissions alone, the same mechanism for every agent - ([`docs/turn-loop/mcp.md`](../turn-loop/mcp.md)). Ruth's permissions + ([`docs/4-implementation/turn-loop/mcp.md`](../../4-implementation/turn-loop/mcp.md)). Ruth's permissions are the default plus `manage_root_agent`, stored for her like any grant. - **State dirs** — _not_ treated as an exception: `HYPERHIVE_STATE_DIR` is injected uniformly via `systemd.globalEnvironment` for every diff --git a/docs/agent-lifecycle/config-changes.md b/docs/2-handbook/agent-lifecycle/config-changes.md similarity index 95% rename from docs/agent-lifecycle/config-changes.md rename to docs/2-handbook/agent-lifecycle/config-changes.md index 3bcf8403..e0e2f1ab 100644 --- a/docs/agent-lifecycle/config-changes.md +++ b/docs/2-handbook/agent-lifecycle/config-changes.md @@ -116,7 +116,7 @@ agent's forge config repo, the one config PRs merge into) and one (the dashboard port, the pinned claude-code, extra OTEL labels, the queue address) aren't in it: `hive-priv` writes them to the container's `/etc/hyperhive-hive-facts` at -start (see `docs/process/conventions.md`). +start (see `docs/5-project/process/conventions.md`). A config that sits only in a local checkout, not on forge `main`, never deploys. @@ -134,14 +134,14 @@ is unset, since every agent would come up with no forge. swarm-controller commits `internal/meta`: one `flake.nix` and `flake.lock` for every agent any hive places (the pins: -[`docs/swarm/builder.md`](../swarm/builder.md#meta-flake-lock)). The swarm +[`docs/4-implementation/swarm/builder.md`](../../4-implementation/swarm/builder.md#meta-flake-lock)). The swarm builder builds every agent from it, and a hive deploys that build. While the swarm has recorded no build of an agent, that agent's deploy fails and names the agent. The controller queues its update of the repo before the deploy request for a merged config commit and for a new agent, and sends the deploy only when that update succeeded and the swarm builder built the agent from the updated repo (the -[deploy gate](../swarm/builder.md#deploy-gate)). A failed build skips the deploy, and the timer builds again. A merge whose webhook delivery never arrives gets the same update, build and deploy within five minutes, from the controller's drift pass (see the paragraph on config drift in the deploy gate section). +[deploy gate](../../4-implementation/swarm/builder.md#deploy-gate)). A failed build skips the deploy, and the timer builds again. A merge whose webhook delivery never arrives gets the same update, build and deploy within five minutes, from the controller's drift pass (see the paragraph on config drift in the deploy gate section). - **Choosing the closure.** The hive doesn't choose. It deploys the closure the agent's record in `agent-closures` declares, whichever `internal/meta` commit @@ -163,7 +163,7 @@ the swarm builder built the agent from the updated repo (the `{"op":"update_container","name":"atlas","stream":true,"toplevel":"/nix/store/-nixos-system-atlas"}`. A rebuild with no record, or none the builder pushed, fails with an error that names the agent; see - [`docs/swarm/builder.md`](../swarm/builder.md#rebuilds-download-the-closure). + [`docs/4-implementation/swarm/builder.md`](../../4-implementation/swarm/builder.md#rebuilds-download-the-closure). - **Hyperhive.** The commit pins one hyperhive rev for the swarm, so an agent runs the hyperhive of the controller's deploy, not its hive's. - **Marker.** After a successful swap or create `hive-c0re` writes the closure's @@ -218,7 +218,7 @@ each `git` call inline, never writing it into a repo's config, so no stored token sits in a repo an agent might one day read. The dashboard deep-links into this org — a `config repo` link -per container row. See `docs/web-ui/dashboard.md`. +per container row. See `docs/2-handbook/web-ui/dashboard.md`. ### Submitting agent's view of config repos @@ -234,7 +234,7 @@ path reaches the same file without the review. Agents holding the `manage_root_agent` permission (granted per agent in the swarm's `agent-permissions` record; see -`docs/process/conventions.md#permissions`) get additional host-side bind +`docs/5-project/process/conventions.md#permissions`) get additional host-side bind mounts via `set_nspawn_flags`: - `/var/lib/hyperhive/agents//state` → `/agents//state` (RW) for @@ -329,7 +329,7 @@ Today hive-c0re sends two: the `/knowledge updated` broadcast detected"`; a swarm-level grafana alert matches that log line directly and pages the operator, with no agent (and no per-hive dashboard banner) in the loop. See -[`docs/scheduler/observability.md`](../scheduler/observability.md#container-crash-alerting-hive-c0re). +[`docs/4-implementation/scheduler/observability.md`](../../4-implementation/scheduler/observability.md#container-crash-alerting-hive-c0re). The remaining lower-urgency lifecycle notices — `Rebuilt`, `Killed`, `Destroyed`, `NeedsLogin`, `LoggedIn` — are "FYI, check diff --git a/docs/agent-lifecycle/persistence.md b/docs/2-handbook/agent-lifecycle/persistence.md similarity index 98% rename from docs/agent-lifecycle/persistence.md rename to docs/2-handbook/agent-lifecycle/persistence.md index 57dfbc14..8e5d00bc 100644 --- a/docs/agent-lifecycle/persistence.md +++ b/docs/2-handbook/agent-lifecycle/persistence.md @@ -62,8 +62,8 @@ table, and the per-agent power-intent registry: This is the durable power *intent* the job queue reconciles the observed container state against; intent survives hive-c0re restarts even though in-flight queue work doesn't. See - [`docs/scheduler/coordinator.md`'s Desired-state - section](../scheduler/coordinator.md#desired-state-spec-vs-status) for who + [`docs/4-implementation/scheduler/coordinator.md`'s Desired-state + section](../../4-implementation/scheduler/coordinator.md#desired-state-spec-vs-status) for who writes and reads it and how reconciliation works. On a host whose hive-c0re once kept scheduled prompts, the file also @@ -114,7 +114,7 @@ back through when debugging a regression. Age-only within the history sooner than a quiet one. The trade-off (accepted): a misbehaving harness can skip its own cleanup — but a compromised harness is already inside the container trust boundary -([`docs/trust-boundary/security.md`](../trust-boundary/security.md)), and these are ephemeral local +([`docs/2-handbook/trust-boundary/security.md`](../trust-boundary/security.md)), and these are ephemeral local artifacts, so cleaning them up where they live is the honest fix. @@ -236,7 +236,7 @@ destroy/recreate, gone on `--purge`. Empty marker file. Its presence parks the agent's turn loop: the harness keeps serving its web UI and MCP daemons but drives no turns, and inbox messages queue unacked until it's removed (see -[turn loop](../turn-loop/README.md#the-loop)). +[turn loop](../../4-implementation/turn-loop/README.md#the-loop)). Unusually, it's read and written from **both** sides of the harness @@ -278,7 +278,7 @@ Under `/var/lib/hyperhive/agents//`: has a value. - `hyperhive-harness.json` — rate-limited / needs-login flags read by the dashboard's async container-state fetch. See - `docs/web-ui/dashboard.md::Container row`. + `docs/2-handbook/web-ui/dashboard.md::Container row`. The agent writes its own instructions to `AGENTS.md` here; claude and opencode load it automatically from their cwd, `state/`. At boot, @@ -289,7 +289,7 @@ Under `/var/lib/hyperhive/agents//`: to the swarm, which reads that copy, and applies the git diffs the swarm sends to edit it. A new agent's first start writes the text given at create, only when no file exists. This file stays the only copy that counts: - [`docs/swarm/README.md`](../swarm/README.md#an-agents-agentsmd). + [`docs/2-handbook/swarm/README.md`](../swarm/README.md#an-agents-agentsmd). - `harness/` — harness-internal ephemeral state; not intended for agent consumption. Bind-mounted to `/agents//harness` inside the container (`$HYPERHIVE_HARNESS_DIR`). Contents: @@ -667,7 +667,7 @@ subvolume creation/deletion above. This is the same subvolume `hivectl agent subvol snapshot push` sends to the swarm's snapshot store — see -[`docs/networking/snapshot-store.md`](../networking/snapshot-store.md) for what a pushed +[`docs/4-implementation/networking/snapshot-store.md`](../../4-implementation/networking/snapshot-store.md) for what a pushed snapshot contains and how the store authenticates a sender. The subvolume is the whole agent state root, so `work/` is part of every @@ -704,7 +704,7 @@ the store answers that it holds none, never because a read failed. place; a missing file is an empty set. A file the controller can't read or parse stops the sync, grants included, and the controller logs `operator matrix admin: pass failed` on every pass. See - [`docs/integrations/matrix.md`](../integrations/matrix.md#operators-are-homeserver-admins). + [`docs/2-handbook/integrations/matrix.md`](../integrations/matrix.md#operators-are-homeserver-admins). ## Run-time dirs diff --git a/docs/integrations/forge.md b/docs/2-handbook/integrations/forge.md similarity index 98% rename from docs/integrations/forge.md rename to docs/2-handbook/integrations/forge.md index 7b7b85eb..0279b8ba 100644 --- a/docs/integrations/forge.md +++ b/docs/2-handbook/integrations/forge.md @@ -10,7 +10,7 @@ which build it runs, how it's served, what it mirrors, where its host-local secrets sit). Container shape, ROOT_URL / sub-domain routing, and operator-vs-in-cluster URL -handling live in [`docs/networking/gateway.md`](../networking/gateway.md); this file owns the +handling live in [`docs/4-implementation/networking/gateway.md`](../../4-implementation/networking/gateway.md); this file owns the per-agent integration story and the notification pump that wakes each agent on relevant activity. @@ -114,7 +114,7 @@ git+/agent-configs/.git?rev=`; see [config-changes.md § Meta flake](../agent-lifecycle/config-changes.md#meta-flake)). The `hive-forge` CLI (separate workspace crate, see -[`README.md`](../README.md) file map) wraps the Forgejo REST API +[`README.md`](../../README.md) file map) wraps the Forgejo REST API with the per-agent token; agents call it for issue / PR / comment ops as if it were a peer. All REST calls across the workspace (`hive-forge` verbs, hive-c0re provisioning, this poller) go through @@ -392,4 +392,4 @@ delivers every unread notification it's handed. A hive-forge CLI subscription verb bounds the firehose explicitly (dropping broad repo watches an agent doesn't need) — the poller never guesses which watches to drop. See the `subscription` verb in -[`docs/tools/forge.md`](../tools/forge.md). +[`docs/2-handbook/tools/forge.md`](../tools/forge.md). diff --git a/docs/integrations/github.md b/docs/2-handbook/integrations/github.md similarity index 100% rename from docs/integrations/github.md rename to docs/2-handbook/integrations/github.md diff --git a/docs/integrations/gitlab.md b/docs/2-handbook/integrations/gitlab.md similarity index 100% rename from docs/integrations/gitlab.md rename to docs/2-handbook/integrations/gitlab.md diff --git a/docs/integrations/knowledge.md b/docs/2-handbook/integrations/knowledge.md similarity index 98% rename from docs/integrations/knowledge.md rename to docs/2-handbook/integrations/knowledge.md index 37b898f1..263a645a 100644 --- a/docs/integrations/knowledge.md +++ b/docs/2-handbook/integrations/knowledge.md @@ -44,7 +44,7 @@ hive-c0re maintains the local clone at `/var/lib/hyperhive/knowledge` via two paths: 1. **Swarm event** — the swarm controller holds the single push hook on - `internal/knowledge` (see `docs/swarm/README.md` § Swarm-wide forge + `internal/knowledge` (see `docs/2-handbook/swarm/README.md` § Swarm-wide forge webhooks). On any push to main, including merge commits, it sends an event to every hive over the swarm queue and each hive runs `git pull`, so agents see the new content on their next turn. diff --git a/docs/integrations/matrix.md b/docs/2-handbook/integrations/matrix.md similarity index 98% rename from docs/integrations/matrix.md rename to docs/2-handbook/integrations/matrix.md index 9bf0808c..29b0bc68 100644 --- a/docs/integrations/matrix.md +++ b/docs/2-handbook/integrations/matrix.md @@ -11,11 +11,11 @@ homeserver, on one host. Two namespaces configure it: `enable`, `gui.package`, `trustedServers`, `maxRequestSize`, `sso.clientSecretFile`. -Vhost routing lives in [`gateway.md`](../networking/gateway.md). +Vhost routing lives in [`gateway.md`](../../4-implementation/networking/gateway.md). ## Container shape -Same shape as [`gateway.md::hive-forge container shape`](../networking/gateway.md): +Same shape as [`gateway.md::hive-forge container shape`](../../4-implementation/networking/gateway.md): - Container name `hive-matrix` (not `h-*`) so c0re's lifecycle scanner ignores it; operator manages via the standard @@ -101,7 +101,7 @@ the hive has no matrix API URL to set. Federation peers find the homeserver through `.well-known/matrix/server` at the server name, which names `:443` -([`gateway.md::Discovery flow`](../networking/gateway.md#discovery-flow-matrix)). +([`gateway.md::Discovery flow`](../../4-implementation/networking/gateway.md#discovery-flow-matrix)). Port 8448 isn't used. Outbound, the container sits on the hive bridge like the CI runner: it @@ -392,7 +392,7 @@ homeserver at all rather than of a setting: `server_name`'s own bogus-value guard lives in `hive-network.nix` (`services.hyperhive.domain != null`), not here — see -[`docs/networking/network.md`](../networking/network.md). +[`docs/4-implementation/networking/network.md`](../../4-implementation/networking/network.md). ## fluffychat-web build fixes diff --git a/docs/tools/forge-cli.md b/docs/2-handbook/reference/forge-cli.md similarity index 100% rename from docs/tools/forge-cli.md rename to docs/2-handbook/reference/forge-cli.md diff --git a/docs/tools/hivectl-cli.md b/docs/2-handbook/reference/hivectl-cli.md similarity index 100% rename from docs/tools/hivectl-cli.md rename to docs/2-handbook/reference/hivectl-cli.md diff --git a/docs/tools/swarm-logs-cli.md b/docs/2-handbook/reference/swarm-logs-cli.md similarity index 100% rename from docs/tools/swarm-logs-cli.md rename to docs/2-handbook/reference/swarm-logs-cli.md diff --git a/docs/tools/swarm-metrics-cli.md b/docs/2-handbook/reference/swarm-metrics-cli.md similarity index 100% rename from docs/tools/swarm-metrics-cli.md rename to docs/2-handbook/reference/swarm-metrics-cli.md diff --git a/docs/tools/swarmctl-cli.md b/docs/2-handbook/reference/swarmctl-cli.md similarity index 100% rename from docs/tools/swarmctl-cli.md rename to docs/2-handbook/reference/swarmctl-cli.md diff --git a/docs/swarm/README.md b/docs/2-handbook/swarm/README.md similarity index 95% rename from docs/swarm/README.md rename to docs/2-handbook/swarm/README.md index 32b6cbed..be942f7f 100644 --- a/docs/swarm/README.md +++ b/docs/2-handbook/swarm/README.md @@ -8,8 +8,8 @@ host is a swarm of one. This page is for the operator. It covers the control plane, how a hive joins the directory, and how hives and agents report upward. The steps for a -fresh swarm are in [`setup.md`](../getting-started/setup.md); the all-local -config is the [README quick start](../../README.md#quick-start-an-all-local-swarm). +fresh swarm are in [`setup.md`](../../1-getting-started/setup.md); the all-local +config is the [README quick start](../../../README.md#quick-start-an-all-local-swarm). Option reference: `services.hyperhive.swarm.*` (swarm-wide facts, identical on every host) and `services.hyperhive.deploy.*` (whether _this_ host runs @@ -24,17 +24,17 @@ grafana, the controller, authelia, …) → - **scheduled prompts** — named prompts for one or more agents at set times, kept by the swarm and fired in each agent's container. → [`schedules.md`](schedules.md) - **inference settings** — the model and effort the swarm sets for an agent, - over its own. → [`inference.md`](inference.md) + over its own. → [`inference.md`](../../4-implementation/swarm/inference.md) - **resource caps** — the CPU and memory cap the swarm sets for an agent, over its hive's. → - [`coordinator.md`](../scheduler/coordinator.md#container-resource-limits) + [`coordinator.md`](../../4-implementation/scheduler/coordinator.md#container-resource-limits) - **review classifier** — checks a PR against its repo's own yes/no rules and posts the result as a review. → - [`review-classifier.md`](review-classifier.md) + [`review-classifier.md`](../../4-implementation/swarm/review-classifier.md) - **notification router** — forge events turned into todos on the agents they concern: assigned, unassigned, cleared on close; and the classifier's review requested on a push. → - [`notify-router.md`](notify-router.md) + [`notify-router.md`](../../4-implementation/swarm/notify-router.md) - **swarm UI** — the operator's day-to-day surface, on the swarm apex, `admins` only. → [`ui.md`](ui.md) - **shared services** — one forge, homeserver, SSO, queue and metrics/logs @@ -45,12 +45,12 @@ grafana, the controller, authelia, …) → every credential the swarm holds, who mints it, who reads it, how it renews and where it lives → [`credentials-reference.md`](credentials-reference.md) · how the store comes up, who writes its grants and how it unseals → - [`bao.md`](bao.md) + [`bao.md`](../../4-implementation/swarm/bao.md) - **swarm CA** — the store root every hive's gateway leaf chains to, and what to - hand a peer. → [`ca.md`](ca.md) + hand a peer. → [`ca.md`](../../4-implementation/swarm/ca.md) - **snapshot store** — the swarm's one `btrfs receive` endpoint, `swarm.snapshotStore.{address,port}`. → - [`snapshot-store.md`](../networking/snapshot-store.md) + [`snapshot-store.md`](../../4-implementation/networking/snapshot-store.md) ## Swarm controller @@ -74,7 +74,7 @@ fact about swarm topology, not about whether this host runs a hive. What it serves, why it's a unix socket rather than a port, and the socket-directory constraint that governs where `socketPath` may point: -[`swarm-controller/README.md`](../../swarm-controller/README.md). +[`swarm-controller/README.md`](../../../swarm-controller/README.md). ### Per-hive status (`GET /api/hives/status`) @@ -102,7 +102,7 @@ needs. A hive on an older release lists none. The snapshot also carries `egress_deny_applied`, the revision of the egress deny list the hive last rendered into its proxy (`0` when the swarm has none). A hive on an older release, or one that hasn't read the list yet, leaves it out; see -[the egress proxy](../networking/network.md#egress-proxy-squid). +[the egress proxy](../../4-implementation/networking/network.md#egress-proxy-squid). Every row also carries `last_seen_unix` and `age_seconds` if you want to apply your own threshold. The timestamp is the one the queue recorded on @@ -292,7 +292,7 @@ Swarm-side, `GET /api/agents//state/stream` relays that subject as SSE, resolving the agent's hive at request time exactly as the terminal stream does. The payload passes through opaquely. Separately, the controller keeps each agent's latest `model`, `effort` and their sources for -`GET /api/agents//inference` ([`inference.md`](inference.md)), and each +`GET /api/agents//inference` ([`inference.md`](../../4-implementation/swarm/inference.md)), and each agent's latest `needs_login` for the `needs_login` field of `GET /api/agents/status`. That field is `null` until a header carrying the flag arrives, so for up to a minute after the controller starts, and for an agent @@ -434,12 +434,12 @@ itself, all under `https:///webhook/forge/`: - a `pull_request` hook on the `agent-configs` org - an instance-wide `push` system hook, for commit metrics - an instance-wide `pull_request_review_request` system hook, which starts - the [review classifier](review-classifier.md#trigger-and-lifecycle) + the [review classifier](../../4-implementation/swarm/review-classifier.md#trigger-and-lifecycle) - instance-wide `issues` + `issue_assign` and `pull_request` + `pull_request_assign` + `pull_request_sync` + `pull_request_review_approved` + `pull_request_review_rejected` + `pull_request_review_comment` system hooks, whose deliveries it keeps in the `SWARM_EVENTS` stream for the - [notification router](notify-router.md#the-path) + [notification router](../../4-implementation/swarm/notify-router.md#the-path) The controller **interprets** a delivery and sends hives a specific message — _the knowledge repo changed_, _deploy agent `foo` at rev @@ -478,7 +478,7 @@ host also serves the swarm UI vhost — that's what publishes the endpoint, and a hook the forge can't reach would collect failed deliveries while looking healthy. The controller generates the HMAC secret on first start and keeps it -(see [`docs/agent-lifecycle/persistence.md`](../agent-lifecycle/persistence.md)). +(see [`docs/2-handbook/agent-lifecycle/persistence.md`](../agent-lifecycle/persistence.md)). To check it's working, push to `internal/knowledge` and look for `webhook: verified delivery` in `journalctl -u swarm-controller`. A @@ -559,10 +559,10 @@ system prompts — and federated hives at different domains can share one. hive's domain. `swarm.name` names the group; `hiveName` names this hive. The env-var chain and `qualify()` semantics: -[`conventions.md`](../process/conventions.md) § Hive identity. +[`conventions.md`](../../5-project/process/conventions.md) § Hive identity. Trust inside a swarm comes from the secret store's `pki` root -([`ca.md`](ca.md)): every hive's gateway leaf chains to it. Trusting a +([`ca.md`](../../4-implementation/swarm/ca.md)): every hive's gateway leaf chains to it. Trusting a hive whose root this swarm doesn't own — another swarm's — has no mechanism. @@ -672,11 +672,11 @@ If both hosts are behind NAT, you need a STUN relay or a third host ## Cross-references - [`ui.md`](ui.md) — the swarm UI, the operator surface for "what hives exist" -- [`../networking/snapshot-store.md`](../networking/snapshot-store.md) — the +- [`../networking/snapshot-store.md`](../../4-implementation/networking/snapshot-store.md) — the swarm's `btrfs receive` endpoint and the `swarm.snapshotStore` option -- [`../process/conventions.md`](../process/conventions.md) § Hive identity — +- [`../process/conventions.md`](../../5-project/process/conventions.md) § Hive identity — env vars, qualified labels - [`../integrations/matrix.md`](../integrations/matrix.md) — matrix federation, TLS cert autogeneration, firewall posture -- [`../networking/gateway.md`](../networking/gateway.md) — nginx vhosts and +- [`../networking/gateway.md`](../../4-implementation/networking/gateway.md) — nginx vhosts and the `.well-known/matrix/` autodiscovery scheme diff --git a/docs/swarm/archetypes.md b/docs/2-handbook/swarm/archetypes.md similarity index 98% rename from docs/swarm/archetypes.md rename to docs/2-handbook/swarm/archetypes.md index 8b5018bf..1b9da25a 100644 --- a/docs/swarm/archetypes.md +++ b/docs/2-handbook/swarm/archetypes.md @@ -48,7 +48,7 @@ hyperhiveArchetypes.reviewer = { imports it into each agent created from the archetype. Today the swarm reads one flake: the hyperhive source the -[builder](builder.md)'s host runs, so only the built-ins appear. The meta +[builder](../../4-implementation/swarm/builder.md)'s host runs, so only the built-ins appear. The meta flake has one archetype source too, its own `hyperhive` input, and refuses an archetype from any other source name. diff --git a/docs/swarm/claude-login.md b/docs/2-handbook/swarm/claude-login.md similarity index 100% rename from docs/swarm/claude-login.md rename to docs/2-handbook/swarm/claude-login.md diff --git a/docs/swarm/credentials-reference.md b/docs/2-handbook/swarm/credentials-reference.md similarity index 99% rename from docs/swarm/credentials-reference.md rename to docs/2-handbook/swarm/credentials-reference.md index 53852111..0bba22dd 100644 --- a/docs/swarm/credentials-reference.md +++ b/docs/2-handbook/swarm/credentials-reference.md @@ -154,7 +154,7 @@ pieces fit, and the reasoning behind them, is in [`credentials.md`](credentials. | secret | generated by | lives at | | ---------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | hive leaf certs | `hive-tls.nix`, issued by the secret store's `pki` mount | `/.pem` | -| the hive's own store identity | `glue-bao-tls.nix` on the store's host; placed by hand on every other host ([`ca.md`](ca.md#the-gateway-leaf-comes-from-the-secret-store)) | `deploy.hive-controller.tls.baoHiveIdentityCertFile` / `baoHiveIdentityKeyFile` | +| the hive's own store identity | `glue-bao-tls.nix` on the store's host; placed by hand on every other host ([`ca.md`](../../4-implementation/swarm/ca.md#the-gateway-leaf-comes-from-the-secret-store)) | `deploy.hive-controller.tls.baoHiveIdentityCertFile` / `baoHiveIdentityKeyFile` | | the cache's public key and pull token, for hive-priv | `swarm-controller` copies them to `swarm/hives//attic/{public-key,pull-token}` (see the cache-facts row above) | `hive-priv-cache-facts` (root) reads those two under the hive's own store identity every ten minutes and writes `/run/hyperhive-priv/attic/public-key` and `netrc` (tmpfs, root, each `0600`, directory `0700`); the cache's address comes from `services.hyperhive.swarm.attic.domain` and `cache`, not from the store. A store that holds no facts, or that doesn't answer, leaves the files as they stand, and a hive that never got them has `hive-priv` answer `not available`. | | the forge's copy of its OIDC secret | `swarm-bao-forge-oidc.service` reads it out of the swarm secret store under the forge publisher's leaf, wherever authelia runs | `/run/hive-forge-oidc/.secret` on the host (tmpfs), bound into the forge container, handed to `forgejo-sso-source` through `LoadCredential` | | the homeserver's copy of its OIDC secret | `swarm-bao-matrix-oidc.service` reads it out of the swarm secret store under matrix-ctl's leaf, wherever authelia runs | `/run/hive-matrix-oidc/.secret` on the host (tmpfs), bound into the homeserver container, handed to tuwunel through `LoadCredential` | diff --git a/docs/swarm/credentials.md b/docs/2-handbook/swarm/credentials.md similarity index 99% rename from docs/swarm/credentials.md rename to docs/2-handbook/swarm/credentials.md index 79b6afa1..abdeb246 100644 --- a/docs/swarm/credentials.md +++ b/docs/2-handbook/swarm/credentials.md @@ -452,7 +452,7 @@ On a host that mints its own PKI, `glue-bao-tls.nix` signs all ten and defaults all twenty options, and there is nothing to do. Elsewhere you copy each leaf from the store's host and name it here: a service principal's from the top of `/var/lib/swarm-bao-pki/`, another hive's leaves from -`/var/lib/swarm-bao-pki/hives//` ([bao.md § TLS](bao.md#tls) lists that +`/var/lib/swarm-bao-pki/hives//` ([bao.md § TLS](../../4-implementation/swarm/bao.md#tls) lists that directory). That's one file per principal rather than one file shared by ten, which is the whole of what this buys. `swarm-ci-runner.nix` refuses the build wherever the runner runs without its pair, as @@ -610,7 +610,7 @@ works too. The granter holds no `sys/auth`, so enabling the mount stays a bootstrap-token step. A store set up before the mount existed needs the -[one-time granter step](../getting-started/setup.md) again: it re-writes +[one-time granter step](../../1-getting-started/setup.md) again: it re-writes `bao-bootstrap`, which covers `sys/auth/oidc`, and the granter's own policy, which covers `auth/oidc/`. Until then `swarm-bao-operator-viewer-policy` writes the viewer policy, logs the step, and diff --git a/docs/swarm/schedules.md b/docs/2-handbook/swarm/schedules.md similarity index 99% rename from docs/swarm/schedules.md rename to docs/2-handbook/swarm/schedules.md index a9dc7a0e..526a3628 100644 --- a/docs/swarm/schedules.md +++ b/docs/2-handbook/swarm/schedules.md @@ -41,7 +41,7 @@ kqzpmwab "standup" every 1d from 2026-10-05T07:00:00Z next 2026-10-05T07:00: - `schedule list` shows one row per prompt: id, name, cadence, start, next fire time, agents, and the first line of the body. -The full flag reference is [`swarmctl-cli.md`](../tools/swarmctl-cli.md). +The full flag reference is [`swarmctl-cli.md`](../reference/swarmctl-cli.md). ## When a prompt fires diff --git a/docs/swarm/services.md b/docs/2-handbook/swarm/services.md similarity index 97% rename from docs/swarm/services.md rename to docs/2-handbook/swarm/services.md index 115e0dd1..4a0b3a36 100644 --- a/docs/swarm/services.md +++ b/docs/2-handbook/swarm/services.md @@ -9,7 +9,7 @@ Some things exist once per **swarm** rather than once per hive: the forge, the matrix homeserver, SSO, the secret store, the queue, and the metrics and log stack. This page says which host runs them and what a hive that runs none of them configures instead. The all-local quick start sets everything with one -line → [README](../../README.md#quick-start-an-all-local-swarm). +line → [README](../../../README.md#quick-start-an-all-local-swarm). Two options say where they live, and everything else derives: @@ -73,7 +73,7 @@ the shapes worth naming are the ones they produce: - **All-local.** Everything on one machine: `singleHostSwarm = true`, plus `deploy.hive-controller.enable = true` for a hive to run agents on. After the first switch, the steps in - [`setup.md`](../getting-started/setup.md) remain. + [`setup.md`](../../1-getting-started/setup.md) remain. - **Services on the swarm controller host.** Set `deploy.allSwarmServices` and `deploy.swarm-controller.enable` there, with hives elsewhere. The controller doesn't derive from @@ -119,7 +119,7 @@ gateway's nginx and the container's processes can open. Agent subjects come from swarm-controller's agent-creation job, written into the users database by `swarm-authelia-bridge`; human ones come from -`swarmctl user add` → [setup.md § 2](../getting-started/setup.md#2--your-sso-account). +`swarmctl user add` → [setup.md § 2](../../1-getting-started/setup.md#2--your-sso-account). The bridge listens on a unix socket on the authelia host, and the gateway publishes it as @@ -335,7 +335,7 @@ the only way in, and the collector is the only intended writer. **Who may query.** `/select/logsql/` takes one of three credentials. An agent presents its `swarm-agent..` bearer, checked by `swarm-logs-auth`. The agent must also hold the `logs` permission, -which is off by default; without it the answer is a 403 (`docs/process/conventions.md`, Permissions). Grafana's datasource presents the same bearer as its +which is off by default; without it the answer is a 403 (`docs/5-project/process/conventions.md`, Permissions). Grafana's datasource presents the same bearer as its metrics datasource, which `swarm-bao-grafana-metrics-bearer` creates, and a second `swarm-metrics-auth` instance (`swarm-logs-grafana-auth`) on the log store's host checks it against the same store path; the gateway tells it from @@ -439,7 +439,7 @@ doesn't gate them: that option says this host _runs_ the store, while the swarm has one either way, reached by its swarm name through the gateway. Agent-side configuration, and what a hive's own collector does, are in -[`../scheduler/observability.md`](../scheduler/observability.md). +[`../scheduler/observability.md`](../../4-implementation/scheduler/observability.md). ### CI runner @@ -452,7 +452,7 @@ It needs neither the forge nor a hive on its host. It reaches the forge at publishes to the secret store, which it reads under its own `ci-runner` leaf. On a host without the store, that leaf is the one file to copy. Registration, the container and the threat model are in -[`../scheduler/ci.md`](../scheduler/ci.md). +[`../scheduler/ci.md`](../../4-implementation/scheduler/ci.md). The runner's built-in Prometheus endpoint is on, bound to `127.0.0.1:9101` in the container's own network namespace. The container's journal forwarder @@ -486,14 +486,14 @@ status blocks nothing: the merge still deploys. Separately, the builder builds each agent from `internal/meta` before the swarm sends its deploy, and a failed build holds that deploy back (`build_meta` job; -[deploy gate](builder.md#deploy-gate)). Those jobs share the one-build slot with +[deploy gate](../../4-implementation/swarm/builder.md#deploy-gate)). Those jobs share the one-build slot with the PR checks. It needs neither the forge nor a hive on its host. It reaches the queue, the secret store and the forge by name, under its own `builder` leaf, and fetches the private config repo with the read-only token of its own `builder` forge account. On a host without the store, that leaf is the one file to copy, as -for the CI runner. What it builds and why are in [`builder.md`](builder.md). +for the CI runner. What it builds and why are in [`builder.md`](../../4-implementation/swarm/builder.md). ### Binary cache @@ -514,7 +514,7 @@ It needs no manual secret. Two host units, under the cache's own `attic` leaf, create its signing key, the cache and the push and pull tokens in the secret store, each only while absent. On a host without the store, that leaf is the one file to copy. The builder pushes green builds to it. How it works, and the -external alternative, are in [`builder.md`](builder.md#binary-cache). +external alternative, are in [`builder.md`](../../4-implementation/swarm/builder.md#binary-cache). The container forwards atticd's journal to the swarm collector like the other service containers (see [Telemetry collector](#telemetry-collector-otel)). The two host @@ -536,7 +536,7 @@ Actions run to the metrics store, through its own route on the swarm collector; the first pass pushes every run since the first. It reads the forge with the read-only token of its own `ci-metrics` forge account, under its own `ci-metrics` leaf. On a host without the store, that leaf is the one file to -copy. What it reads and pushes are in [`ci-metrics.md`](ci-metrics.md). +copy. What it reads and pushes are in [`ci-metrics.md`](../../4-implementation/swarm/ci-metrics.md). ### Review classifier @@ -564,7 +564,7 @@ It needs neither the forge nor a hive on its host. It reaches the queue and the secret store by name, under its own `review-classifier` leaf. On a host without the store, that leaf is the one file to copy, as for the CI runner. The swarm UI's try-out drives it through swarm-controller; what a run is -and what it costs are in [`review-classifier.md`](review-classifier.md). +and what it costs are in [`review-classifier.md`](../../4-implementation/swarm/review-classifier.md). ### Claude login minter @@ -606,4 +606,4 @@ It needs neither the forge nor a hive on its host, and holds no forge token. It reaches the queue and the secret store by name, under its own `notify-router` leaf. On a host without the store, that leaf is the one file to copy, as for the CI runner. The flows and the delivery path are in -[`notify-router.md`](notify-router.md). +[`notify-router.md`](../../4-implementation/swarm/notify-router.md). diff --git a/docs/swarm/sso.md b/docs/2-handbook/swarm/sso.md similarity index 98% rename from docs/swarm/sso.md rename to docs/2-handbook/swarm/sso.md index 4caeedb3..57179793 100644 --- a/docs/swarm/sso.md +++ b/docs/2-handbook/swarm/sso.md @@ -16,7 +16,7 @@ authelia binds no TCP port: its listeners are unix sockets only the gateway can publishes it as `auth.` — vhost, dnsmasq record and TLS name all follow `deploy.authelia`, so there is nothing to turn on separately. (Details, including why a client hive must not declare that -vhost: [`../networking/gateway.md`](../networking/gateway.md).) +vhost: [`../networking/gateway.md`](../../4-implementation/networking/gateway.md).) **Nobody can log in until the first user exists.** authelia exits at startup on an empty user set: @@ -50,7 +50,7 @@ this password is stored nowhere — record it now the hash. It reads and writes authelia's `users.yml` directly — it's the one user store, shared with `swarm-authelia-bridge`, which creates agent identities in the same file. No restart: authelia watches it. Full -reference: [`../tools/swarmctl-cli.md`](../tools/swarmctl-cli.md). +reference: [`../tools/swarmctl-cli.md`](../reference/swarmctl-cli.md). You can edit `users.yml` by hand, and `swarmctl` will read what you wrote. ⚠️ It rewrites the whole file on every change, so **comments and @@ -294,7 +294,7 @@ token. Its access token carries `authelia.bearer.authz` with the audience `https:///api/`, so the gateway admits it at the controller's API and nowhere else, under the same access rule as a browser session there. Revocation is per account: once authelia stops accepting the account, the -gateway refuses its token too. See [`swarmctl/README.md`](../../swarmctl/README.md#remote-use-swarmctl-login). +gateway refuses its token too. See [`swarmctl/README.md`](../../../swarmctl/README.md#remote-use-swarmctl-login). ## What this doesn't do diff --git a/docs/swarm/ui.md b/docs/2-handbook/swarm/ui.md similarity index 97% rename from docs/swarm/ui.md rename to docs/2-handbook/swarm/ui.md index 1306888f..ad4a0922 100644 --- a/docs/swarm/ui.md +++ b/docs/2-handbook/swarm/ui.md @@ -34,7 +34,7 @@ smaller; short confirm dialogs stay short. | `/jobs` | the controller's job graph, where agent creation and credential mints show progress, and under it the swarm builder's recent builds (agent config PRs, and agents built from `internal/meta` before a deploy), with what nix printed when one failed, and the review classifier's recent runs, a try-out on a real PR, and its settings | | `/messages` | the swarm-wide message feed: every message between agents, and to them from the operator, reminders and compaction, with a history window and a filter on sender or recipient; a folded **broadcast** card above the feed sends one message to more than one agent at once, and a folded **schedules** card holds the scheduled prompts: add, edit, pause, resume and remove | -Everything it shows comes from [`swarm-controller`](../../swarm-controller/README.md). +Everything it shows comes from [`swarm-controller`](../../../swarm-controller/README.md). Pages that refresh on a timer share one polling rule: the page sends no new request while the last one is still running, so an older answer never overtakes a slow one, and a failed refresh keeps the rows already on screen under an @@ -205,7 +205,7 @@ with another browser. ### Review The **review** card on the **jobs** tab is the one place for the -[review classifier](review-classifier.md), with three sections in a column. The +[review classifier](../../4-implementation/swarm/review-classifier.md), with three sections in a column. The card starts shut and reads nothing until you open it; the old `/review` address redirects to the jobs tab. **try a classification** and **classifier settings** come first, each folded to a single bar: the bar of the settings @@ -215,7 +215,7 @@ ask for their own data. #### Recent runs -The **recent runs** section lists what the [review classifier](review-classifier.md) has +The **recent runs** section lists what the [review classifier](../../4-implementation/swarm/review-classifier.md) has run, newest first, one card per run. A card shows the repo and PR, the head commit, whether the run was a try-out or a requested review, how long ago it started, how long it took, how many classifications ran, and the outcome as a @@ -243,7 +243,7 @@ stores and posts nothing. #### Try a classification -The **try a classification** section shows how the [review classifier](review-classifier.md) +The **try a classification** section shows how the [review classifier](../../4-implementation/swarm/review-classifier.md) answers a classification on a real PR, before you put it in a repo's config. Give it a repo as `owner/name` and a PR number, then write a classification: a name, a description, a polarity (positive or negative) and a threshold from 0 @@ -266,7 +266,7 @@ leaves your classifications as they were. A run takes a minute or more, and up to half an hour for ten classifications. The page says it's running and shows how long it has been, then lists each classification with its verdict, the model's certainty, the threshold, whether the verdict counted and the reason; the posted review folds -those fields into four columns ([`review-classifier.md`](review-classifier.md)). A classification that couldn't run shows its error where the reason +those fields into four columns ([`review-classifier.md`](../../4-implementation/swarm/review-classifier.md)). A classification that couldn't run shows its error where the reason would be. Under the table are the overall verdict, approve or request changes, and the head commit the run read. A failed run shows its error, for example when the swarm has no API key yet. @@ -285,12 +285,12 @@ already holds replaces it, and the file's other rules stay. It needs no PR number. If the forge has no account of yours yet, log in to the forge once first. If the classifier's account may not push to the repo, the page says so: a repo admin adds it as a collaborator with write access -([the service account](review-classifier.md#the-service-account)). +([the service account](../../4-implementation/swarm/review-classifier.md#the-service-account)). #### Classifier settings The **classifier settings** section shows the runtime, model and -effort the [review classifier](review-classifier.md) runs with and how many +effort the [review classifier](../../4-implementation/swarm/review-classifier.md) runs with and how many requested reviews run at once, and lets you change them. The runtime is **API key** (claude on the Anthropic key) or **ACP** (opencode on a provider). On the API key runtime the model is free text, one word as claude's `--model` takes @@ -343,7 +343,7 @@ routes. ### Notification flows -The **notification router** card on `/swarm` (beta, folded; `/notifications` redirects to it) lists the [notification router's flows](notify-router.md#flows) +The **notification router** card on `/swarm` (beta, folded; `/notifications` redirects to it) lists the [notification router's flows](../../4-implementation/swarm/notify-router.md#flows) in two groups. Each **built-in** flow has an on/off switch; switching one off stops the router running it from its next event, and switching it on again restores it. Each **custom** flow shows its name and whether it's on. Every @@ -371,7 +371,7 @@ first: when, which flow and node, the event and issue, who got it, and whether i went through (`GET /api/notify/deliveries`). Both come from the router; while the running router is older than the controller, each shows the controller's "not available" message in place of its numbers or table, and the rest of the page -works. See [deliveries and dry runs](notify-router.md#deliveries-and-dry-runs). +works. See [deliveries and dry runs](../../4-implementation/swarm/notify-router.md#deliveries-and-dry-runs). #### Editing a flow @@ -428,7 +428,7 @@ are those with an open issue, so one with none is missing), and a todo's agent i dropdown of the swarm's agents. A value set in a flow that the list no longer holds stays selected, marked "not listed." If a list fails to load, or the controller predates the schema, the box is plain text. The values come from the -controller's schema ([notify-router](notify-router.md)), so a new one shows up +controller's schema ([notify-router](../../4-implementation/swarm/notify-router.md)), so a new one shows up without a UI change. **preview** runs the draft, saved or not, over the newest forge events and @@ -632,7 +632,7 @@ the whole value. A field you leave empty stays out of it, so the agent keeps its own value for that field; an empty max subagents is no cap. **clear the swarm's setting** sends the `DELETE`. The card reads `GET /api/agents/{agent}/inference`; before the agent's first report reaches the controller, it shows the swarm's stored values only. -[inference](inference.md) describes the settings themselves. +[inference](../../4-implementation/swarm/inference.md) describes the settings themselves. ### Editing an agent's instructions @@ -681,7 +681,7 @@ with `internet_log: true`; the other two leave `internet_log` out, which ends logging. The **would-deny** tab of the **egress** dashboard in Grafana shows the hosts a `log` agent reached. The hive's egress proxy applies the flag to each agent's proxy credential (see -[the egress proxy](../networking/network.md#egress-proxy-squid)). An agent sends its +[the egress proxy](../../4-implementation/networking/network.md#egress-proxy-squid)). An agent sends its credential from its next start after it has fetched it, so a flag change applies to an agent's traffic from then on. @@ -766,7 +766,7 @@ seconds and reloads its proxy, so the name is reachable within a minute. The proxy's private-range rules still apply: an entry whose name resolves to a loopback, private or bridge address stays blocked. An entry adds to the names a hive's own `allowedDomains` lists, which stay allowed. See [the egress -proxy](../networking/network.md#egress-proxy-squid) for what each hive does +proxy](../../4-implementation/networking/network.md#egress-proxy-squid) for what each hive does when the controller is old or unreachable. ### Creating an agent @@ -894,7 +894,7 @@ a link dialog closes. write-only and starts empty; **remove** sends the `DELETE` below. A swarm-controller without the route answers 404, and the dialog says so. What the agent does with it: - [API-key provider](../turn-loop/config.md#api-key-provider-useapikey--backendenvironmentfile). + [API-key provider](../../4-implementation/turn-loop/config.md#api-key-provider-useapikey--backendenvironmentfile). #### Deleting a linked account @@ -970,7 +970,7 @@ swarmctl user update --add-group admins # an account that already exists ``` `--email` isn't needed for the UI, but the forge won't create your account -without one → [setup.md § 2](../getting-started/setup.md#2--your-sso-account). +without one → [setup.md § 2](../../1-getting-started/setup.md#2--your-sso-account). Why a group and not "any session": agents are authelia subjects too, so _authenticated_ includes every agent in the swarm. The group is the only @@ -1089,4 +1089,4 @@ swarm's front page opens with a name mismatch. ## Cross-references - [`sso.md`](sso.md) — the authelia instance itself, and the user store. -- [`../networking/gateway.md`](../networking/gateway.md) — the full vhost map and TLS modes. +- [`../networking/gateway.md`](../../4-implementation/networking/gateway.md) — the full vhost map and TLS modes. diff --git a/docs/tools/README.md b/docs/2-handbook/tools/README.md similarity index 88% rename from docs/tools/README.md rename to docs/2-handbook/tools/README.md index d70bbcef..9de2186d 100644 --- a/docs/tools/README.md +++ b/docs/2-handbook/tools/README.md @@ -12,12 +12,12 @@ debug agent behavior. - **[hivectl](hivectl.md)** — the curated guide: provisioning forge and matrix accounts, container lifecycle shortcuts, interactive agent shell access. -- **[hivectl-cli](hivectl-cli.md)** — the exhaustive, autogenerated +- **[hivectl-cli](../reference/hivectl-cli.md)** — the exhaustive, autogenerated flag-by-flag reference, kept in lockstep with the binary by CI. ## For the swarm operator -- **[swarmctl-cli](swarmctl-cli.md)** — the exhaustive, autogenerated +- **[swarmctl-cli](../reference/swarmctl-cli.md)** — the exhaustive, autogenerated flag-by-flag reference for `swarmctl`, kept in lockstep with the binary by CI the same way `hivectl-cli.md` is. `swarmctl` itself runs as root on the swarm-controller host, not through `hivectl` — @@ -37,7 +37,7 @@ debug agent behavior. - **[forge](forge.md)** — the `hive-forge` Forgejo CLI every agent has for issues, PRs, and comments. Not an MCP tool — a binary agents shell out to instead of ad-hoc curl. -- **[forge-cli](forge-cli.md)** — the exhaustive, autogenerated +- **[forge-cli](../reference/forge-cli.md)** — the exhaustive, autogenerated flag-by-flag reference for `hive-forge`, kept in lockstep with the binary by CI the same way `hivectl-cli.md` is. - **[matrix](matrix.md)** — the matrix MCP tool surface diff --git a/docs/tools/bash.md b/docs/2-handbook/tools/bash.md similarity index 100% rename from docs/tools/bash.md rename to docs/2-handbook/tools/bash.md diff --git a/docs/tools/forge.md b/docs/2-handbook/tools/forge.md similarity index 99% rename from docs/tools/forge.md rename to docs/2-handbook/tools/forge.md index 3030945e..b2d997ef 100644 --- a/docs/tools/forge.md +++ b/docs/2-handbook/tools/forge.md @@ -6,11 +6,11 @@ proper Rust binary). Use it instead of ad-hoc curl pipelines. This page is the curated guide. For the exhaustive flag-by-flag reference autogenerated from the binary's own command tree, see -[`forge-cli.md`](forge-cli.md) — emitted by the hidden `hive-forge +[`forge-cli.md`](../reference/forge-cli.md) — emitted by the hidden `hive-forge markdown-docs` subcommand and kept in lockstep with the code by the `hive-forge-docs` flake check (CI fails if the committed copy drifts). Regenerate with `nix build .#hive-forge && ./result/bin/hive-forge -markdown-docs > docs/tools/forge-cli.md`. +markdown-docs > docs/2-handbook/reference/forge-cli.md`. ## Credentials and repo defaults diff --git a/docs/tools/hivectl.md b/docs/2-handbook/tools/hivectl.md similarity index 95% rename from docs/tools/hivectl.md rename to docs/2-handbook/tools/hivectl.md index 21bad35e..2b7cf2e3 100644 --- a/docs/tools/hivectl.md +++ b/docs/2-handbook/tools/hivectl.md @@ -6,7 +6,7 @@ admin socket `/run/hyperhive/host.sock`, which is root-only by default — so it needs `sudo` unless you grant sudoless access by listing your login user in `services.hyperhive.deploy.hive-controller.adminUsers` (adds you to the `hive-admin` group that owns the socket; see -[`docs/trust-boundary/boundary.md`](../trust-boundary/boundary.md#host-admin-socket-access-hivectl)). +[`docs/2-handbook/trust-boundary/boundary.md`](../trust-boundary/boundary.md#host-admin-socket-access-hivectl)). Available via the `hive-c0re` package in the host NixOS config. Unlike the `hive-c0re` daemon subcommands (which go through the broker), @@ -15,11 +15,11 @@ shortcuts and interactive agent shell access. This page is the curated guide. For the exhaustive flag-by-flag reference autogenerated from the binary's own command tree, see -[`hivectl-cli.md`](hivectl-cli.md) — emitted by the hidden `hivectl +[`hivectl-cli.md`](../reference/hivectl-cli.md) — emitted by the hidden `hivectl markdown-docs` subcommand and kept in lockstep with the code by the `hivectl-docs` flake check (CI fails if the committed copy drifts). Regenerate with `nix build .#default && ./result/bin/hivectl -markdown-docs > docs/tools/hivectl-cli.md`. +markdown-docs > docs/2-handbook/reference/hivectl-cli.md`. ## Forge @@ -149,7 +149,7 @@ hivectl agent iris watch # tail iris's live events; Ctrl-C to stop Dials the same unix socket the gateway's nginx `proxy_pass`es through (`/run/hive-agent//web.sock` — see [Per-agent unix-socket -upstream](../networking/gateway.md#per-agent-unix-socket-upstream)) directly and +upstream](../../4-implementation/networking/gateway.md#per-agent-unix-socket-upstream)) directly and speaks a bare HTTP/1.1 request for the agent's existing `/events/stream` SSE endpoint over it. No gateway hop, no daemon round-trip for the stream itself — the daemon socket is only used for the "does this agent @@ -162,7 +162,7 @@ Prints one compact line per event — reuses the `_icon`/`_summary` fields the harness already stamps onto stream-json events for the web UI (`stream_enrich.rs`), so tool calls and turn markers read as short glyph-prefixed lines instead of raw JSON. Not an attempt at the web -UI's full collapsible-details rendering (`docs/web-ui/terminal-rendering.md`) +UI's full collapsible-details rendering (`docs/4-implementation/web-ui/terminal-rendering.md`) — that's presentation for a browser, this is a `tail -f`. ## Open diff --git a/docs/tools/matrix.md b/docs/2-handbook/tools/matrix.md similarity index 98% rename from docs/tools/matrix.md rename to docs/2-handbook/tools/matrix.md index 1a970670..b1a1dc31 100644 --- a/docs/tools/matrix.md +++ b/docs/2-handbook/tools/matrix.md @@ -146,7 +146,7 @@ pending invites, resolve_invite to accept or reject`. The daemon clears the keyed todo when a `resolve_invite` (or `join_room`) call resolves the invite. -See [`docs/integrations/matrix.md`](../integrations/matrix.md) for the homeserver setup, +See [`docs/2-handbook/integrations/matrix.md`](../integrations/matrix.md) for the homeserver setup, provisioning flow, and federation config. ## Extra MCP servers (per-agent) diff --git a/docs/tools/subagent.md b/docs/2-handbook/tools/subagent.md similarity index 98% rename from docs/tools/subagent.md rename to docs/2-handbook/tools/subagent.md index 35ccbe6f..02f876d4 100644 --- a/docs/tools/subagent.md +++ b/docs/2-handbook/tools/subagent.md @@ -325,7 +325,7 @@ it came from. One without a `model` runs on claude's own default, which the daemon doesn't check. The list is: - the swarm's `subagent_models` for this agent, when it sets one - ([`inference.md`](../swarm/inference.md)). It replaces the agent's own + ([`inference.md`](../../4-implementation/swarm/inference.md)). It replaces the agent's own list, so it can name models that list leaves out. - otherwise `services.hyperhive.agent.availableModels`, which reaches the daemon as `HIVE_AVAILABLE_MODELS`. An empty list restricts nothing. @@ -340,7 +340,7 @@ as `/` on `opencode`. ## How many subagents may run at once When the swarm sets `max_subagents` for the agent -([`inference.md`](../swarm/inference.md)), the daemon refuses a `start`, or a +([`inference.md`](../../4-implementation/swarm/inference.md)), the daemon refuses a `start`, or a `continue` of a session that isn't running, once that many subagents are running, before anything spawns: @@ -405,7 +405,7 @@ daemon reads from the swarm on each `start`/`continue`: `Edit`, `Glob`, `Grep`, `Read`, `Skill`, `Write`, plus `WebFetch`/ `WebSearch` for an agent granted `web_tools` and not otherwise. A running subagent keeps the tools it started with. See -[the harness's own allowlist](../turn-loop/mcp.md#tool-allowlist-hive_sh4repermissionsallowed_builtin_tools) +[the harness's own allowlist](../../4-implementation/turn-loop/mcp.md#tool-allowlist-hive_sh4repermissionsallowed_builtin_tools) for what that list contains and why. The exception: **a subagent gets Claude's built-in `Bash` iff its parent diff --git a/docs/trust-boundary/boundary.md b/docs/2-handbook/trust-boundary/boundary.md similarity index 93% rename from docs/trust-boundary/boundary.md rename to docs/2-handbook/trust-boundary/boundary.md index 810aa484..d6fc058d 100644 --- a/docs/trust-boundary/boundary.md +++ b/docs/2-handbook/trust-boundary/boundary.md @@ -23,7 +23,7 @@ always on), the gateway proxies all operator-facing traffic, and gateway. - **Agent** — speaks only for itself, only over its per-agent unix socket. The socket's identity _is_ the agent (see - `docs/process/conventions.md`, "identity = socket"). An agent must not + `docs/5-project/process/conventions.md`, "identity = socket"). An agent must not be able to reach the core daemon's HTTP surface, another agent's socket, or another agent's web UI. @@ -50,7 +50,7 @@ defence-in-depth layered on top. Network isolation is complete and always on: every agent container runs in a private netns behind the hive bridge, and there is no -shared-netns mode. See `docs/networking/network.md`. +shared-netns mode. See `docs/4-implementation/networking/network.md`. Concretely, the core daemon's dashboard `/api` carries **no application-layer authentication** — the core daemon serves operator-authority @@ -81,7 +81,7 @@ The boundary rests on three layers: 3. **Privsep** — defence in depth on the core process; `hive-c0re` runs as the unprivileged `hive-core` user and delegates root operations to `hive-priv`, a narrow socket-activated helper. See - [`docs/trust-boundary/security.md`](security.md) for the privilege boundary table. + [`docs/2-handbook/trust-boundary/security.md`](security.md) for the privilege boundary table. ### hive-priv socket activation @@ -121,7 +121,7 @@ root:root`, no symlink followed) and leaves an existing one alone. The container's own activation (`hive-agent-user-migrate`) then chowns it to the agent user and sets `0751`. Without a user namespace that uid is the host inode's owner. (A namespaced agent, see "User namespace" in -`docs/agent-lifecycle/persistence.md`, gets the dir from `EnsureAgentSocketDir` +`docs/2-handbook/agent-lifecycle/persistence.md`, gets the dir from `EnsureAgentSocketDir` owned by its data owner instead, because the container can't chown through the id map.) Don't add another host-side chown or chmod: two owners of one path revert each other. Until the container activates, @@ -140,12 +140,12 @@ of `/run` read-only. Dropping `o=w` removes that permission rather than qualifying it. ⚠️ **The gateway's nginx and dnsmasq are host services, next to -`hive-c0re`** (see `docs/networking/gateway.md`) — there is no namespace between +`hive-c0re`** (see `docs/4-implementation/networking/gateway.md`) — there is no namespace between them and the rest of the host. That costs no network isolation: nginx binds the host's `:80`/`:443` and reaches `localhost` upstreams, which would require opening a netns anyway. 🔑 It does mean nothing _implicitly_ scopes the privileged reload verb — -see [`docs/trust-boundary/security.md`](security.md#hive-c0re-privilege-separation) for +see [`docs/2-handbook/trust-boundary/security.md`](security.md#hive-c0re-privilege-separation) for how `PrivRequest::ReloadGatewayNginx`'s containment works. ⚠️ This dir has exactly one writer, so it needs no world write at all. diff --git a/docs/trust-boundary/security.md b/docs/2-handbook/trust-boundary/security.md similarity index 99% rename from docs/trust-boundary/security.md rename to docs/2-handbook/trust-boundary/security.md index 12fa1bfc..a6a25fcb 100644 --- a/docs/trust-boundary/security.md +++ b/docs/2-handbook/trust-boundary/security.md @@ -138,7 +138,7 @@ checkpoints**, not about sandboxing the agent from its own tools: off-hive, on any host that sets `deploy.builder.enable` (the all-swarm-services switch defaults it on): it builds the PR's head with a read-only forge token, in its own netns, as an untrusted - client of its host's nix daemon (see [`builder.md`](../swarm/builder.md)). + client of its host's nix daemon (see [`builder.md`](../../4-implementation/swarm/builder.md)). ### Capability = accepted risk @@ -235,7 +235,7 @@ The blast radius also has a **network** dimension. `swarm-ci-runner` runs its unsandboxed builds of untrusted PR code in its own private netns behind the bridge: a build reaches the forge by name and can't reach host-loopback services — including the core dashboard at `127.0.0.1:`, which -has no application-layer auth of its own (see [`docs/scheduler/ci.md`](../scheduler/ci.md)). The `0600` +has no application-layer auth of its own (see [`docs/4-implementation/scheduler/ci.md`](../../4-implementation/scheduler/ci.md)). The `0600` token policy bounds file reads; network isolation bounds network reach. **What's NOT exposed**: diff --git a/docs/web-ui/README.md b/docs/2-handbook/web-ui/README.md similarity index 85% rename from docs/web-ui/README.md rename to docs/2-handbook/web-ui/README.md index d9cba937..dd2967ae 100644 --- a/docs/web-ui/README.md +++ b/docs/2-handbook/web-ui/README.md @@ -4,8 +4,8 @@ The per-hive dashboard: container state and the rebuild queue for one hive. For swarm-wide day to day, see [the swarm UI](../swarm/ui.md) instead. If you want implementation detail — wire formats, DOM structure, event plumbing — see [Dashboard -layout](dashboard.md), [Per-agent page](agent.md), [Shape](shape.md), -and [CSS theme variables](css-vars.md) below; this page only covers +layout](dashboard.md), [Per-agent page](agent.md), [Shape](../../4-implementation/web-ui/shape.md), +and [CSS theme variables](../../4-implementation/web-ui/css-vars.md) below; this page only covers what you actually do here. ## Where things are @@ -55,7 +55,7 @@ merging a config change or bumping a flake input. **Grant or revoke a permission.** Not on the hive dashboard: an agent's permissions live in the swarm. Set them with `swarmctl agent permissions`, or on the swarm UI's create-agent screen when creating the agent; see -[`process/conventions.md`](../process/conventions.md#permissions). +[`process/conventions.md`](../../5-project/process/conventions.md#permissions). **Read an agent's logs.** Logs aren't in the hive dashboard. Every hive ships its agents', daemons' and infra containers' journals to the @@ -77,13 +77,13 @@ JS renders — no full-page reloads. badge-derivation rules. - **[Per-agent page](agent.md)** — the per-agent terminal, side panel, actions, and per-agent endpoints. -- **[Shape (shared by both)](shape.md)** — the SPA skeleton, SSE +- **[Shape (shared by both)](../../4-implementation/web-ui/shape.md)** — the SPA skeleton, SSE multiplexing, terminal pane, listener bind, per-agent relative paths, `data-async` form pattern, atomic repaint. -- **[CSS theme variables](css-vars.md)** — the Catppuccin Mocha custom +- **[CSS theme variables](../../4-implementation/web-ui/css-vars.md)** — the Catppuccin Mocha custom properties declared once in `base.css` and the rule that per-page stylesheets reference (never redeclare) them. -- **[swarm-ui design guide](design-guide.md)** — visual language, +- **[swarm-ui design guide](../../5-project/web-ui/design-guide.md)** — visual language, motion, theming, error-UX, and component-first principles for the swarm-level Preact app specifically (not this page's dashboard/agent UIs). `/components` on a running swarm-ui is the companion living @@ -95,7 +95,7 @@ Task-oriented jumps straight to the relevant section, for when you're touching the code rather than using the UI: - **"How does the dashboard SPA stay live without polling?"** → - [`shape.md`](shape.md) (SSE multiplexing, Worker-death self-heal, + [`shape.md`](../../4-implementation/web-ui/shape.md) (SSE multiplexing, Worker-death self-heal, atomic repaint). - **"What does a container row contain?"** → [`dashboard.md`](dashboard.md) (Container row, Selection bar). @@ -103,7 +103,7 @@ touching the code rather than using the UI: [`dashboard.md`](dashboard.md) (Dashboard endpoints, Dashboard event channel). - **"How does the per-agent terminal render tool calls?"** → - [`terminal-rendering.md`](terminal-rendering.md) (full row taxonomy + [`terminal-rendering.md`](../../4-implementation/web-ui/terminal-rendering.md) (full row taxonomy and dispatch walkthrough); for a high-level summary see [`agent.md`](agent.md) (Per-stream rendering). - **"What actions does the agent page offer?"** → @@ -111,5 +111,5 @@ touching the code rather than using the UI: - **"What are the per-agent HTTP endpoints?"** → [`agent.md`](agent.md) (Per-agent endpoints). - **"Which CSS variable to use / where are colours defined?"** → - [`css-vars.md`](css-vars.md) (Palette, single-source `base.css` + [`css-vars.md`](../../4-implementation/web-ui/css-vars.md) (Palette, single-source `base.css` rule). diff --git a/docs/web-ui/agent.md b/docs/2-handbook/web-ui/agent.md similarity index 98% rename from docs/web-ui/agent.md rename to docs/2-handbook/web-ui/agent.md index af241216..3a73a45d 100644 --- a/docs/web-ui/agent.md +++ b/docs/2-handbook/web-ui/agent.md @@ -1,7 +1,7 @@ # Per-agent page > Part of [Web UI](README.md). See also: -> [Shape (shared)](shape.md) · [Dashboard layout](dashboard.md) +> [Shape (shared)](../../4-implementation/web-ui/shape.md) · [Dashboard layout](dashboard.md) Two fixed-position layers frame a full-viewport terminal: a header and @@ -55,7 +55,7 @@ at a given viewport width. Two columns: the built-in default); the model's also says what the last turn resolved it to. Both are set on the swarm: the swarm UI's agents page, or `swarmctl agent inference set` - ([`inference.md`](../swarm/inference.md)). + ([`inference.md`](../../4-implementation/swarm/inference.md)). - On an ACP agent each badge shows the session's own value when it doesn't offer the requested one. The effort badge is hidden (`state.effort` is `null`) until the first turn has attached a @@ -183,10 +183,10 @@ SSE frame and the replayed history rows. The web UI: The backfill/live-tail dedupe, sticky-bottom autoscroll, and "↓ N new" pill are the shared terminal-pane mechanics described in -[Shape](shape.md#shared-terminal-pane) — this page's log is one +[Shape](../../4-implementation/web-ui/shape.md#shared-terminal-pane) — this page's log is one instance of that same factory. -Per-stream rendering (see [`docs/web-ui/terminal-rendering.md`](terminal-rendering.md) for +Per-stream rendering (see [`docs/4-implementation/web-ui/terminal-rendering.md`](../../4-implementation/web-ui/terminal-rendering.md) for the full row taxonomy and dispatch logic): - `Stream` `tool_use` → diff --git a/docs/web-ui/dashboard.md b/docs/2-handbook/web-ui/dashboard.md similarity index 98% rename from docs/web-ui/dashboard.md rename to docs/2-handbook/web-ui/dashboard.md index 028a8c0b..adf1e53f 100644 --- a/docs/web-ui/dashboard.md +++ b/docs/2-handbook/web-ui/dashboard.md @@ -1,7 +1,7 @@ # Dashboard layout > Part of [Web UI](README.md). See also: -> [Shape (shared)](shape.md) · [Per-agent page](agent.md) +> [Shape (shared)](../../4-implementation/web-ui/shape.md) · [Per-agent page](agent.md) This page covers the per-hive UI that hive-c0re serves. Swarm-wide surfaces — the agent roster, agent creation, linking external forge and @@ -43,7 +43,7 @@ active; hash changes don't reload the page. entry through an RAII guard, and each raise logs one `hive warning raised` line. The operator sees the registry as metrics and Grafana alerts: see - [Hive-scoped metrics](../scheduler/observability.md#hive-scoped-metrics-hive-c0re). + [Hive-scoped metrics](../../4-implementation/scheduler/observability.md#hive-scoped-metrics-hive-c0re). A container crash is a swarm-level Grafana alert on the crash watcher's own log line, so a hive with no otel export has no operator-visible warning signal beyond `/health/ready`. @@ -71,7 +71,7 @@ see below.) Per-container CPU, memory and disk load isn't shown in the hive UI. The same figures go out as OTEL metrics and appear in Grafana (see -[observability](../scheduler/observability.md)). An agent's CPU and memory +[observability](../../4-implementation/scheduler/observability.md)). An agent's CPU and memory caps live per agent in the swarm UI. **K3PT ST4T3** — agents with kept state and **no container** (size + age + @@ -213,7 +213,7 @@ dashboard doesn't have to learn that. Gateway-on: base URL is `/agent/` (same origin, gateway proxies to the per-agent harness — TCP or unix-domain depending on the agent's `HIVE_WEB_SOCKET` opt-in, see -`docs/networking/gateway.md::Per-agent unix-socket upstream`). +`docs/4-implementation/networking/gateway.md::Per-agent unix-socket upstream`). Gateway-off is the flag-absent fallback (a dashboard served outside a hyperhive deploy): base URL is `http://:` (direct TCP). Forge diff --git a/docs/crates/README.md b/docs/4-implementation/crates/README.md similarity index 89% rename from docs/crates/README.md rename to docs/4-implementation/crates/README.md index e2caa230..a048e16d 100644 --- a/docs/crates/README.md +++ b/docs/4-implementation/crates/README.md @@ -9,5 +9,5 @@ drift out of sync the way a hand-copied mirror would — edit the crate's own README to change what shows up here. For the one-line-per-crate index (which crate does what, at a glance) -see the main [`AGENTS.md`](../../AGENTS.md#repo-map) repo map instead; +see the main [`AGENTS.md`](../../../AGENTS.md#repo-map) repo map instead; this section is each crate's own, longer word on itself. diff --git a/docs/networking/gateway.md b/docs/4-implementation/networking/gateway.md similarity index 98% rename from docs/networking/gateway.md rename to docs/4-implementation/networking/gateway.md index 0a2dfd04..16c6d445 100644 --- a/docs/networking/gateway.md +++ b/docs/4-implementation/networking/gateway.md @@ -20,7 +20,7 @@ You rarely switch it on yourself. `gateway.enable` defaults to off, and every mo | `chat./` | fluffychat-web static | `hive-matrix.nix`, `swarm.matrix.gatewayHost != null` | | `chat./config.json` | inline JSON (FluffyChat boot config) | `hive-matrix.nix`, `swarm.matrix.gatewayHost != null` | | `/metrics` | the queue exporter's `/metrics` on loopback, behind an authelia `auth_request`; every other path, `/` included, returns `403`, so `/varz` and `/connz` are never reachable | `swarm-nats.nix`, the queue's host | -| `grafana.`, `metrics.`, `logs.`, `otel.`, `bao.` | the matching swarm service | that service's module → [`swarm/services.md`](../swarm/services.md) | +| `grafana.`, `metrics.`, `logs.`, `otel.`, `bao.` | the matching swarm service | that service's module → [`swarm/services.md`](../../2-handbook/swarm/services.md) | A service module declares its vhost as `services.hyperhive.gateway.vhosts.`: the locations, `forceSSL` and any `extraConfig` it needs. The gateway renders it into `services.nginx.virtualHosts.` and adds the `listen` set, the certificate for that name and the [security headers](#security-headers) (ahead of the vhost's own `extraConfig`), and drops `addSSL` from a vhost that sets `forceSSL`. The eval fails for a vhost that sets `listen`, `addSSL`, `onlySSL`, `enableACME`, `useACMEHost`, `sslCertificate` or `sslCertificateKey` itself, and for any vhost declared while `gateway.enable` is false. A name with a certificate from outside the gateway's leaves (the queue's) sets `certificate = { cert; key; }` instead. `hive-gateway/vhosts.nix` declares the gateway's own catch-all (`_`) the same way, `hive-c0re/vhosts.nix` the dashboard and the hive domain, and `glue-matrix-hive-discovery.nix` adds the matrix redirect to the hive domain where the homeserver runs, and discovery there when the hive domain is the pinned server name. The catch-all sets `securityHeaders = false` because it sends no response, and the gateway renders a vhost's `default` at `mkDefault` priority. @@ -42,7 +42,7 @@ The catch-all `_` vhost answers any other `Host` with `444` (connection closed, Only the host that **runs** authelia declares `auth.`; a hive that merely uses SSO knows `swarm.authelia.url` but doesn't answer for that name. The server name must be exactly `swarm.authelia.domain` — authelia checks that `authelia_url` sits inside its session cookie domain at startup, and refuses to boot otherwise. The vhost carries no `auth_request` (that would put the login page behind the login it serves) and passes `X-Forwarded-{Proto,Host,Uri,For}`, because authelia decides by the original request. -⚠️ `auth.` answering with the **sso unavailable** page means authelia isn't answering at all — check `journalctl -M swarm-authelia -u authelia-swarm`. While the user set is empty, a placeholder user marked `disabled` keeps authelia's user store non-empty, so it boots fine and serves a login page that refuses everyone; add the first account with `swarmctl user add …` ([`setup.md`](../getting-started/setup.md)). +⚠️ `auth.` answering with the **sso unavailable** page means authelia isn't answering at all — check `journalctl -M swarm-authelia -u authelia-swarm`. While the user set is empty, a placeholder user marked `disabled` keeps authelia's user store non-empty, so it boots fine and serves a login page that refuses everyone; add the first account with `swarmctl user add …` ([`setup.md`](../../1-getting-started/setup.md)). Per-agent UIs stay sub-path; forge and matrix get sub-domains → [Sub-domain shape](#sub-domain-shape-rationale). @@ -116,7 +116,7 @@ Give the gateway the real cert (`tls.certDir` or `tls.acme`) so it serves proper ## Dashboard access -The dashboard answers on `.hives.` (`deploy.hive-controller.dashboardDomain`, derived from the hive name and `swarm.domain`), behind an authelia `auth_request`. Log in with your authelia account; only accounts in the `admins` group get in, the same rule as the swarm UI. Add one with `swarmctl user add --group admins` ([`swarm/sso.md`](../swarm/sso.md)). +The dashboard answers on `.hives.` (`deploy.hive-controller.dashboardDomain`, derived from the hive name and `swarm.domain`), behind an authelia `auth_request`. Log in with your authelia account; only accounts in the `admins` group get in, the same rule as the swarm UI. Add one with `swarmctl user add --group admins` ([`swarm/sso.md`](../../2-handbook/swarm/sso.md)). The name sits under the swarm domain so authelia's session cookie reaches it, whatever the hive's own domain is. Any path on the hive domain other than the matrix routes redirects there, so old bookmarks still land. @@ -146,7 +146,7 @@ It shares the host network namespace (`privateNetwork = false`): the container i State lives at `/var/lib/nixos-containers/hive-forge/var/lib/forgejo/` and survives restarts and reboots; destroy the container to wipe it. -Only the swarm's forge host runs it (`services.hyperhive.deploy.forgejo.enable`, see [`swarm/services.md`](../swarm/services.md)). Every other hive reaches that host's gateway by `forge.`. +Only the swarm's forge host runs it (`services.hyperhive.deploy.forgejo.enable`, see [`swarm/services.md`](../../2-handbook/swarm/services.md)). Every other hive reaches that host's gateway by `forge.`. ### Network and port configuration @@ -281,7 +281,7 @@ Every agent binds its web UI on a unix-domain socket at nginx reaches `web.sock` through `o=--x` (traverse) and the socket's own `0666`. The gateway is one of three principals sharing that dir and doesn't own its ownership rules — see - [`docs/trust-boundary/boundary.md`](../trust-boundary/boundary.md#the-per-agent-socket-dir). + [`docs/2-handbook/trust-boundary/boundary.md`](../../2-handbook/trust-boundary/boundary.md#the-per-agent-socket-dir). 3. **Marker gate**. After a successful `bind_unix`, the harness drops `/hyperhive-socket-bound` next to the socket. c0re's `agent_sockets::write` filters its JSON map by marker presence — @@ -429,7 +429,7 @@ Three render sites follow it: the agent-name link, the favicon fetch (`/icon`), and the nav-strip `container`-kind links from `DashboardState.links` (`GET /api/dashboard-state`). Forge links come from `swarm.forge.publicUrl`; with it unset the dashboard hides them rather than -guess. See `docs/web-ui/dashboard.md::Container row` for the frontend side. +guess. See `docs/2-handbook/web-ui/dashboard.md::Container row` for the frontend side. ### Dialing another vhost by name (`verifiedProxyTo`) diff --git a/docs/networking/network.md b/docs/4-implementation/networking/network.md similarity index 98% rename from docs/networking/network.md rename to docs/4-implementation/networking/network.md index 6235eade..f80b8612 100644 --- a/docs/networking/network.md +++ b/docs/4-implementation/networking/network.md @@ -74,7 +74,7 @@ The flows, end to end: the host forwards + masquerades out its uplink. - **Swarm** — peer hives connect over the `wg-hive` WireGuard mesh and reach each other's gateway/forge across it - ([`docs/swarm/`](../swarm/README.md)). + ([`docs/swarm/`](../../2-handbook/swarm/README.md)). - **Control plane (no network)** — per-agent broker socket `/run/hive/mcp.sock`, privileged helper `/run/hive/priv.sock`, operator admin `/run/hyperhive/host.sock`, and the per-agent UI @@ -105,7 +105,7 @@ The hive domain (`services.hyperhive.domain`, which the resolver answers for) comes from this hive's entry in the swarm directory: `swarm.hives..domain`, default `.`. Eval fails until `swarm.domain` and that entry exist -([`swarm/README.md`](../swarm/README.md) § Hive identity config). +([`swarm/README.md`](../../2-handbook/swarm/README.md) § Hive identity config). ## Bridge addressing @@ -197,7 +197,7 @@ agents to reach directly — is unreachable. (hyperhive's own services need none of this: a module that means to be reachable from agents opens its own port here, which the hive collector and `nats.enable` both do. Note that `deploy.swarm-otel.upstream.endpoint` is the _upstream_, which no agent ever dials. See -`docs/scheduler/observability.md`.) +`docs/4-implementation/scheduler/observability.md`.) `services.hyperhive.network.exposeHostPorts = [ 4318 ];` opens each listed TCP port `P` on the bridge-interface `allowedTCPPorts`, so an @@ -242,7 +242,7 @@ host. It never decrypts TLS, so it holds no signing key. It caches nothing. minute. Nothing rotates a credential: to replace one, delete its leaf in the store and the next pass mints a new one. HTTP Basic crosses the bridge in clear between an agent and the proxy. See - [the credentials reference](../swarm/credentials-reference.md). + [the credentials reference](../../2-handbook/swarm/credentials-reference.md). A request without a valid credential gets a 407, once the destination checks below have passed it. @@ -365,7 +365,7 @@ host. It never decrypts TLS, so it holds no signing key. It caches nothing. nothing on its wire changes. - **Applied.** Each hive reports the revision of the list it last rendered as `egress_deny_applied` in its status snapshot (see [the hive - snapshot](../swarm/README.md)). `GET /api/egress/denied-domains` answers the + snapshot](../../2-handbook/swarm/README.md)). `GET /api/egress/denied-domains` answers the list's `revision` and `behind`, the hives that haven't reported it: an older `hive-c0re` never does, so it shows there rather than silently allowing the names. A hive that reads the list reports within about a minute. Reporting @@ -392,7 +392,7 @@ host. It never decrypts TLS, so it holds no signing key. It caches nothing. `allowedDomains` is the hive's own list, always allowed whether or not the swarm has a list of its own (next bullet). - **Swarm allow-list.** The operator adds domains in the swarm UI - ([allowed domains](../swarm/ui.md#allowed-domains)); the swarm controller + ([allowed domains](../../2-handbook/swarm/ui.md#allowed-domains)); the swarm controller keeps them in one value of the `egress-allow` bucket, each for every agent or for one agent. Every 30 seconds `hive-c0re` reads it and writes two files beside the users file, `services.hyperhive.deploy.hive-controller.squid.allowedFile` @@ -577,4 +577,4 @@ loopback — the hive-c0re admin socket is a UDS, not TCP. ## Cross-references -- `docs/networking/gateway.md` — vhost map + the gateway's other duties +- `docs/4-implementation/networking/gateway.md` — vhost map + the gateway's other duties diff --git a/docs/networking/snapshot-store.md b/docs/4-implementation/networking/snapshot-store.md similarity index 97% rename from docs/networking/snapshot-store.md rename to docs/4-implementation/networking/snapshot-store.md index d52fcd3e..216df7a4 100644 --- a/docs/networking/snapshot-store.md +++ b/docs/4-implementation/networking/snapshot-store.md @@ -35,7 +35,7 @@ services.hyperhive.deploy.wireguard = { The store host is a swarm member like any other: it gets an entry in `services.hyperhive.swarm.hives`, the same directory every host holds. See -[swarm/](../swarm/README.md) for the mesh itself. +[swarm/](../../2-handbook/swarm/README.md) for the mesh itself. Note that `deploy.wireguard.enable` gates the mesh, **not** `deploy.hive-controller.enable` --- a store host runs no hive and would otherwise get no @@ -192,8 +192,8 @@ the access-review list. The snapshot covers an agent's **state subvolume**, which is the parent of `state/`, `claude/` and `harness/` (see -[`docs/agent-lifecycle/persistence.md`'s btrfs subvolume -section](../agent-lifecycle/persistence.md#btrfs-subvolumes-for-varlibhyperhiveagentsname) +[`docs/2-handbook/agent-lifecycle/persistence.md`'s btrfs subvolume +section](../../2-handbook/agent-lifecycle/persistence.md#btrfs-subvolumes-for-varlibhyperhiveagentsname) for how and when hive-c0re creates that subvolume). Consequences: diff --git a/docs/scheduler/ci.md b/docs/4-implementation/scheduler/ci.md similarity index 98% rename from docs/scheduler/ci.md rename to docs/4-implementation/scheduler/ci.md index 9bf4e3a5..b8d85984 100644 --- a/docs/scheduler/ci.md +++ b/docs/4-implementation/scheduler/ci.md @@ -42,7 +42,7 @@ internals, container design, and the full security writeup. The jobs run on every PR (and on `workflow_dispatch` for manual re-triggers), defined in -[`.forgejo/workflows/ci.yml`](../../.forgejo/workflows/ci.yml). The first four +[`.forgejo/workflows/ci.yml`](../../../.forgejo/workflows/ci.yml). The first four below are required checks (forge branch protection) — a hit on any of them blocks merge. The two vale jobs split what used to be a single check: the error-only one exists so it _can_ become a required check without also @@ -114,7 +114,7 @@ slow); run those manually before pushing Rust changes. Tests that boot NixOS VMs are the `vmTests..` flake output (`nix/vm-tests.nix`), not `checks`, so the per-PR **nix flake check** job never -builds them. [`.forgejo/workflows/vm-nightly.yml`](../../.forgejo/workflows/vm-nightly.yml) +builds them. [`.forgejo/workflows/vm-nightly.yml`](../../../.forgejo/workflows/vm-nightly.yml) builds all of them nightly at 02:30; a failing test fails that run. To run one by hand, start the workflow from the forge UI (Actions, vm-nightly, Run workflow, optional `test` input), or run @@ -159,7 +159,7 @@ needs `deploy.forgejo.enable` or a hive on its host. - **Private netns, bridge-attached**: the container runs in its own network namespace (`privateNetwork = true`, `hostBridge`), takes its address from the gateway's DHCP, and resolves through the gateway's dnsmasq on the bridge IP. It can't reach host-loopback services — the core dashboard at `127.0.0.1:7000` and the raw forge port are unreachable from CI. - **Non-ephemeral**: runner credentials (`.runner`) persist across restarts, written to the container's state dir on registration and reused after. -- **Sandbox fallback**: nspawn containers can't create user-namespaces, so nix's sandboxing would always fail. Module sets `nix.settings.sandbox-fallback = true` in the container — nix builds run unsandboxed (safe because the container is already isolated). See `docs/process/gotchas.md`. +- **Sandbox fallback**: nspawn containers can't create user-namespaces, so nix's sandboxing would always fail. Module sets `nix.settings.sandbox-fallback = true` in the container — nix builds run unsandboxed (safe because the container is already isolated). See `docs/5-project/process/gotchas.md`. - **Credential isolation**: no forge admin credential enters the container. swarm-controller holds the forge admin token; the runner holds only the registration token and its own `.runner`, and the host unit that fetches the token holds a store identity that reads that one path. Only `swarm-ci-runner-register.service` loads the registration token; the daemon unit and the jobs it runs have it neither in their environment nor among their credentials. ## Registration @@ -360,4 +360,4 @@ builder — disk-pressure collection at least runs while nothing is mid-copy. - `nix/host-modules/swarm-ci-runner.nix`: runner configuration, token fetch, re-registration step, container setup. - `swarm-controller/src/forge/runner_token.rs`: publishes the registration token. - `.forgejo/workflows/ci.yml`: workflow definition. -- `docs/process/gotchas.md`: nix sandboxing limitations in containers. +- `docs/5-project/process/gotchas.md`: nix sandboxing limitations in containers. diff --git a/docs/scheduler/coordinator.md b/docs/4-implementation/scheduler/coordinator.md similarity index 98% rename from docs/scheduler/coordinator.md rename to docs/4-implementation/scheduler/coordinator.md index 68f707cb..0591c60b 100644 --- a/docs/scheduler/coordinator.md +++ b/docs/4-implementation/scheduler/coordinator.md @@ -1,8 +1,8 @@ # hive-c0re coordinator internals Architecture notes for the `hive-c0re` coordinator daemon's internal subsystems. -For the public API surface (dashboard, socket protocol) see `docs/process/conventions.md` -and `docs/agent-lifecycle/persistence.md`. +For the public API surface (dashboard, socket protocol) see `docs/5-project/process/conventions.md` +and `docs/2-handbook/agent-lifecycle/persistence.md`. --- @@ -321,7 +321,7 @@ classification (the one-time migration; thereafter the DB is authoritative). Agents build from the swarm's `internal/meta` flake, which swarm-controller commits and the swarm builder builds from (see -[Config changes](../agent-lifecycle/config-changes.md#the-swarms-meta-flake)). +[Config changes](../../2-handbook/agent-lifecycle/config-changes.md#the-swarms-meta-flake)). The hive keeps no flake of its own and deploys the closure the swarm builds from it, as the agent's `agent-closures` record declares (`attic/closure.rs`). `agent_set.rs` keeps the @@ -529,7 +529,7 @@ config — all nspawn containers live in `machine.slice`. ## See also -- `docs/agent-lifecycle/config-changes.md` — config PRs, meta flake, system messages -- `docs/agent-lifecycle/persistence.md` — SQLite schema, state-dir layout -- `docs/process/conventions.md` — wire protocol, recipient sentinels -- `docs/agent-lifecycle/agent-roster.md` — the agent roster and privilege +- `docs/2-handbook/agent-lifecycle/config-changes.md` — config PRs, meta flake, system messages +- `docs/2-handbook/agent-lifecycle/persistence.md` — SQLite schema, state-dir layout +- `docs/5-project/process/conventions.md` — wire protocol, recipient sentinels +- `docs/2-handbook/agent-lifecycle/agent-roster.md` — the agent roster and privilege diff --git a/docs/scheduler/jobq.md b/docs/4-implementation/scheduler/jobq.md similarity index 97% rename from docs/scheduler/jobq.md rename to docs/4-implementation/scheduler/jobq.md index 15bca667..3b9104a9 100644 --- a/docs/scheduler/jobq.md +++ b/docs/4-implementation/scheduler/jobq.md @@ -34,7 +34,7 @@ swarm controller and hive-c0re each build their own graph on it. Two views, one per graph, drawn by the same component: the swarm UI's `/jobs` page shows the swarm controller's graph, and the hive dashboard's **R3BU1LD QU3U3** tab (see -[`web-ui/dashboard.md`](../web-ui/dashboard.md)) shows that hive's. Each +[`web-ui/dashboard.md`](../../2-handbook/web-ui/dashboard.md)) shows that hive's. Each **row** is one job; the rows nested under it are that job's steps, in order (occasionally a couple run side by side). A step shows one of: diff --git a/docs/scheduler/observability.md b/docs/4-implementation/scheduler/observability.md similarity index 99% rename from docs/scheduler/observability.md rename to docs/4-implementation/scheduler/observability.md index 1be92cf8..e20ebf7e 100644 --- a/docs/scheduler/observability.md +++ b/docs/4-implementation/scheduler/observability.md @@ -40,7 +40,7 @@ switch on: there is no per-hive or per-agent opt-in. `upstream.endpoint` is where telemetry ends up after it leaves the swarm — optional, because the swarm's own metrics store (`deploy.victoriametrics`) is a destination in its own right. With both, telemetry goes to both. See -[`swarm/services.md`](../swarm/services.md#metrics-victoriametrics--grafana). +[`swarm/services.md`](../../2-handbook/swarm/services.md#metrics-victoriametrics--grafana). **Telemetry leaves a hive exactly one way: through the collector that runs on the host.** Agents never talk to `upstream.endpoint` themselves — @@ -50,7 +50,7 @@ can reach. They dial it as every hive and every swarm, which each hive's own dnsmasq answers with its bridge IP, so an agent's build carries no hive-specific address and no swarm domain. The otel module opens that port on the bridge itself. That collector forwards to the swarm's -([`swarm/services.md`](../swarm/services.md#telemetry-collector-otel)), the +([`swarm/services.md`](../../2-handbook/swarm/services.md#telemetry-collector-otel)), the single process holding the upstream credential and the only writer to the swarm's stores. No agent holds a copy, and neither does the hive. @@ -104,7 +104,7 @@ smuggled out in resource attributes on an otherwise-legitimate export. That's a **different risk from the one the collector fixes**: an agent that held the upstream credential could do everything above _and_ use the token anywhere else. The collector removes the token and keeps the pipe. Agents are inside the trust -boundary (`docs/trust-boundary/security.md`: capability = accepted risk), so an agent being +boundary (`docs/2-handbook/trust-boundary/security.md`: capability = accepted risk), so an agent being able to _send_ is an accepted extension of that boundary — but it's not closed by this design, and don't read anything here as closing it. @@ -160,7 +160,7 @@ run lands a changed value it restarts the collector. A timer runs it again every ten minutes, so a secret rotated later reaches the collector within about ten minutes. On a hive that runs the store the leaf is already there; elsewhere it's the credential you -place — see [secret placement](../swarm/credentials.md). +place — see [secret placement](../../2-handbook/swarm/credentials.md). To deliver the secret some other way, name the file. The fetch unit isn't rendered, and the collector waits up to 120s for the file to appear: @@ -387,7 +387,7 @@ shares a filesystem with the units it reads, because it's a host systemd service rather than a container. It forwards the crash log line alongside the rest of the hive's telemetry. A swarm-level grafana alert, `hh-container-crashed` in -[`nix/host-modules/swarm-grafana.nix`](../../nix/host-modules/swarm-grafana.nix), +[`nix/host-modules/swarm-grafana.nix`](../../../nix/host-modules/swarm-grafana.nix), matches it directly and pages the operator — no agent, and no per-hive dashboard, in the loop. diff --git a/docs/swarm/bao.md b/docs/4-implementation/swarm/bao.md similarity index 96% rename from docs/swarm/bao.md rename to docs/4-implementation/swarm/bao.md index 707c377b..e72fb868 100644 --- a/docs/swarm/bao.md +++ b/docs/4-implementation/swarm/bao.md @@ -3,11 +3,11 @@ The swarm runs one OpenBao store, `swarm-bao`. Every swarm-level credential an agent or service needs comes from it under that principal's own certificate identity. What lives in it and who reads what: -[`credentials-reference.md`](credentials-reference.md). This page covers the store itself — how it +[`credentials-reference.md`](../../2-handbook/swarm/credentials-reference.md). This page covers the store itself — how it comes up, who may write its grants, and how it's sealed and reached. The operator steps (init, the one-time granter bootstrap) are in -[`../getting-started/setup.md`](../getting-started/setup.md#1--secret-store); +[`../getting-started/setup.md`](../../1-getting-started/setup.md#1--secret-store); this page is the why behind them. @@ -119,7 +119,7 @@ Those are `mkDefault`s, so naming your own paths wins. Do that when your certificates come from a real internal CA; the store has no opinion about which. A host that does **not** deploy the store names the reader's three itself, plus a pair for every principal it runs — see -[per-principal identities](credentials-reference.md#store-identities-one-per-principal) for the list. +[per-principal identities](../../2-handbook/swarm/credentials-reference.md#store-identities-one-per-principal) for the list. They're the credentials the store can't hand you, being what opens it, so the store's host signs them and the operator copies them to the host that needs them and names them in config. The service principals' leaves are at the top of diff --git a/docs/swarm/builder.md b/docs/4-implementation/swarm/builder.md similarity index 99% rename from docs/swarm/builder.md rename to docs/4-implementation/swarm/builder.md index 18ca2094..31b823db 100644 --- a/docs/swarm/builder.md +++ b/docs/4-implementation/swarm/builder.md @@ -10,7 +10,7 @@ The builder also builds each agent from `internal/meta` before the swarm sends its deploy, and a failed build holds the deploy back: see [Deploy gate](#deploy-gate). -Where it runs and how it's enabled: [`services.md`](services.md#builder). +Where it runs and how it's enabled: [`services.md`](../../2-handbook/swarm/services.md#builder). ## What happens on a config PR @@ -135,7 +135,7 @@ The builder also answers `$SWARM.builder.archetypes`: it evaluates the forge token, and answers with the JSON. An evaluation doesn't wait for a build in progress and has five minutes; the controller waits six. The builder logs each evaluation's time as `elapsed_ms`, and a timeout's error carries the last -lines nix printed. What the controller does with the answer: [`archetypes.md`](archetypes.md). +lines nix printed. What the controller does with the answer: [`archetypes.md`](../../2-handbook/swarm/archetypes.md). ## Meta-flake lock @@ -179,7 +179,7 @@ the deploy node, and the graph shows the update as failed and the deploy as skipped. Each ten-minute retry of a failed update declares a fresh build and deploy behind it for every agent whose latest deploy the failed update skipped, so the run that succeeds sends them (see -[Config changes](../agent-lifecycle/config-changes.md#the-swarms-meta-flake)). A render +[Config changes](../../2-handbook/agent-lifecycle/config-changes.md#the-swarms-meta-flake)). A render equal to the committed `flake.nix` sends no lock. The controller sends a deploy to every agent a hive places whenever the pin in @@ -592,7 +592,7 @@ status. The same `git` fetches a config's own inputs, so the builder registers the helper for the forge's URL only, and the helper answers only an `https` request for the forge's host: any other host gets nothing. The builder holds no forge write credential; swarm-controller writes the status with its own forge token. Details per path: -[`credentials-reference.md`](credentials-reference.md). +[`credentials-reference.md`](../../2-handbook/swarm/credentials-reference.md). The builder exports build counts, durations and queue wait; see [Swarm builder metrics](../scheduler/observability.md#swarm-builder-metrics). diff --git a/docs/swarm/ca.md b/docs/4-implementation/swarm/ca.md similarity index 99% rename from docs/swarm/ca.md rename to docs/4-implementation/swarm/ca.md index 06883b6b..8dea0bc6 100644 --- a/docs/swarm/ca.md +++ b/docs/4-implementation/swarm/ca.md @@ -24,7 +24,7 @@ IP SANs and no `localhost`. A request for any other name is a refusal from the store naming that name. The hive's policy grants `update` on that one issue path and `read` on its own `attic/public-key`, `attic/pull-token` and `squid-selfcheck` -(see [the cache facts for hive-priv](credentials-reference.md) and the +(see [the cache facts for hive-priv](../../2-handbook/swarm/credentials-reference.md) and the squid self-check password), and nothing else. `swarm-bao-hive-leaf-policy.service`, on the store's host, writes the role, the policy and the cert-auth role for every diff --git a/docs/swarm/ci-metrics.md b/docs/4-implementation/swarm/ci-metrics.md similarity index 100% rename from docs/swarm/ci-metrics.md rename to docs/4-implementation/swarm/ci-metrics.md diff --git a/docs/swarm/inference.md b/docs/4-implementation/swarm/inference.md similarity index 92% rename from docs/swarm/inference.md rename to docs/4-implementation/swarm/inference.md index 8ed30eb1..33454297 100644 --- a/docs/swarm/inference.md +++ b/docs/4-implementation/swarm/inference.md @@ -20,18 +20,18 @@ swarmctl agent inference clear iris - `--subagent-models` takes a comma-separated list. It replaces the agent's `availableModels` as the models its subagent `start` and `continue` calls may name, read on each call - ([`subagent.md`](../tools/subagent.md#which-models-a-subagent-may-run-on)). + ([`subagent.md`](../../2-handbook/tools/subagent.md#which-models-a-subagent-may-run-on)). - You set the subagent cap, `max_subagents`, in the swarm UI's inference dialog; `set` has no flag for it and keeps the stored one. It's a whole number from 0 to 32, where 0 allows no subagents; unset means no cap - ([`subagent.md`](../tools/subagent.md#how-many-subagents-may-run-at-once)). + ([`subagent.md`](../../2-handbook/tools/subagent.md#how-many-subagents-may-run-at-once)). - `get` shows each field, naming the unset ones (or says the swarm sets nothing for the agent), and then the model and effort the agent reports running on, each with where it comes from. - `clear` removes every field; the agent goes back to its own values. -The full flag reference is [`swarmctl-cli.md`](../tools/swarmctl-cli.md). The -swarm UI's agents page edits the same settings; see [UI](ui.md). The +The full flag reference is [`swarmctl-cli.md`](../../2-handbook/reference/swarmctl-cli.md). The +swarm UI's agents page edits the same settings; see [UI](../../2-handbook/swarm/ui.md). The swarm is the only place to change an agent's model or effort at runtime: its web UI shows both but has no picker and no `/model` or `/effort` command. @@ -112,7 +112,7 @@ warning in its log, until the next `PUT` replaces it. An agent's own queue credential grants it `$JS.API.DIRECT.GET.KV_agent-inference.$KV.agent-inference.`, the direct read of its own key, and nothing else in this bucket — the same shape -as its schedule key ([`schedules.md`](schedules.md)). +as its schedule key ([`schedules.md`](../../2-handbook/swarm/schedules.md)). The harness (`hive-agent`'s `swarm_inference`) applies a set field to the in-memory model or effort without writing `hyperhive-model` or @@ -123,7 +123,7 @@ The subagent daemon (`hive-subagent-mcp`'s `swarm_models`) reads same credential, on each `start` or `continue`. `effective` comes from the agent's turn-state header on -`$SWARM.agent-state.` ([`README.md`](README.md)), which carries +`$SWARM.agent-state.` ([`README.md`](../../2-handbook/swarm/README.md)), which carries `model`, `effort`, `model_source` and `effort_source`. swarm-controller subscribes to every agent's header with the reader grant it already holds for the state relay and keeps the latest per agent in memory. A header without diff --git a/docs/swarm/notify-router.md b/docs/4-implementation/swarm/notify-router.md similarity index 98% rename from docs/swarm/notify-router.md rename to docs/4-implementation/swarm/notify-router.md index 27fb64f5..779c0dd2 100644 --- a/docs/swarm/notify-router.md +++ b/docs/4-implementation/swarm/notify-router.md @@ -5,7 +5,7 @@ agents they concern. No hive is in the path, and the agent can be on any hive: swarm-controller keeps the events, the router decides, and each agent's harness takes its own todos off the queue. -Where it runs and how it's enabled: [`services.md`](services.md#notification-router). +Where it runs and how it's enabled: [`services.md`](../../2-handbook/swarm/services.md#notification-router). It runs beside `hive-forge-notify`, which still delivers forge notifications the way it did. An agent can get the same news from both. @@ -96,7 +96,7 @@ check above and reads only what exists: a source event from `issues`, three `pull_request_review_*` events, and condition and recipient fields from the event's field list. The same goes for the `{field}` placeholders in a todo's key and text. A `PUT` must carry the `revision` it read, `0` for no document, and gets 409 when the stored revision is another. Swarm UI's -[notification router card on `/swarm`](ui.md#notification-flows) shows the document and +[notification router card on `/swarm`](../../2-handbook/swarm/ui.md#notification-flows) shows the document and switches built-ins. **The schema.** `GET /api/notify/flows` also returns a `schema` object, built @@ -263,7 +263,7 @@ serves them next to `GET /api/notify/flows`: The dry-run request carries a whole document. The `GET` sends the saved one; the `POST` takes a `PUT` body (it ignores a `revision`), answers 400 with the `PUT` error for a document that fails the check, and forwards a valid one. -[notification router card on `/swarm`](ui.md#notification-flows) shows both: the recent deliveries, +[notification router card on `/swarm`](../../2-handbook/swarm/ui.md#notification-flows) shows both: the recent deliveries, and the counts beside each node. **Deploying in any order.** A router that predates these subjects doesn't diff --git a/docs/swarm/review-classifier.md b/docs/4-implementation/swarm/review-classifier.md similarity index 97% rename from docs/swarm/review-classifier.md rename to docs/4-implementation/swarm/review-classifier.md index e60810ac..e9e1cf7f 100644 --- a/docs/swarm/review-classifier.md +++ b/docs/4-implementation/swarm/review-classifier.md @@ -75,7 +75,7 @@ The classifier has a dedicated forge service account, `review-classifier` as a reviewer on a PR starts a run: 1. The forge sends the `pull_request_review_request` delivery to - [`swarm-controller`](../../swarm-controller/README.md), the swarm's one + [`swarm-controller`](../../../swarm-controller/README.md), the swarm's one webhook owner, on its instance-wide hook `/webhook/forge/review-request`. A delivery for any other reviewer, or a removed request, does nothing. 2. The controller reads the PR and the repo's rules from the default branch, @@ -125,7 +125,7 @@ the `official` flag on a reviewer's earlier reviews when it's requested again, and only official reviews count toward the required approvals or block a merge. Removing the request before the new review lands restores the earlier one. A repo whose branch protection also dismisses stale approvals (see -[PR review gate](../process/pr-review-gate.md)) stops counting an approval +[PR review gate](../../5-project/process/pr-review-gate.md)) stops counting an approval on an older commit even without a re-request. How many requested reviews run at once is a swarm-wide setting, two by @@ -155,7 +155,7 @@ earlier verdict waits for a run place too, and frees it before it posts. A swarm-level service in its own container, on whichever host the operator puts it. It talks to `swarm-controller` through async messages only, and gets its secrets from the swarm's secret store, bao -([`bao.md`](bao.md), [`credentials.md`](credentials.md)). +([`bao.md`](bao.md), [`credentials.md`](../../2-handbook/swarm/credentials.md)). It looks like an agent from the inside, with three things missing: it has no full agent identity, no state directory and no tools. Every model session is @@ -178,7 +178,7 @@ removed after it. Its working directory is one empty directory that every classification shares: opencode writes that path into the system message, so one path keeps the system message the same from one classification to the next, and the provider's prompt cache can hold. The service drives it through -[`hive-runtime`](../../hive-runtime/README.md)'s `AcpRuntime`, as the subagent +[`hive-runtime`](../../../hive-runtime/README.md)'s `AcpRuntime`, as the subagent daemon does a one-run subagent, and stops the agent when the classification ends. The agent has no tool. opencode runs some tools, `read` among them, without asking the client, so the service sets opencode's own permission to @@ -194,7 +194,7 @@ own: opencode merges an agent's permission over `OPENCODE_PERMISSION`, so one there would offer the model tools again. The service is `swarm-review-classifier`, and its container is the one -[`services.md`](services.md#review-classifier) describes. It holds no forge +[`services.md`](../../2-handbook/swarm/services.md#review-classifier) describes. It holds no forge credential: `swarm-controller` reads the PR and sends it along. ### Swarm-level settings @@ -268,7 +268,7 @@ does. Every error is problem+json with the reason in `detail`: When the controller can't read the settings, a try-out answers the same 503 or 500, and a review posts a no-verdict comment. The operator writes each runtime's key by hand in the secret store -([`services.md`](services.md#review-classifier)). +([`services.md`](../../2-handbook/swarm/services.md#review-classifier)). The per-agent equivalent is [`inference.md`](inference.md), where the controller writes model and effort to a queue KV bucket @@ -423,7 +423,7 @@ row that failed it, so _match_ is ✅ on a positive rule and ❌ on a negative o _unsure_), or without an answer. The certainty cell reads _certainty / threshold_. The approval is a real one: it counts toward a repo's required approvals -like any other ([`pr-review-gate.md`](../process/pr-review-gate.md)). The +like any other ([`pr-review-gate.md`](../../5-project/process/pr-review-gate.md)). The Repos that rely on it typically raise their required approval count to account for it. @@ -492,7 +492,7 @@ subscription login), comes later. ## Swarm UI -Two pages in the [swarm UI](ui.md), `admins` only like the rest of it. +Two pages in the [swarm UI](../../2-handbook/swarm/ui.md), `admins` only like the rest of it. ### Recent runs @@ -685,7 +685,7 @@ lists their labels, and the `review-classifier` Grafana dashboard charts them. The classifier logs each run's result to its journal, which ships to VictoriaLogs like every swarm service container's -([`services.md`](services.md#logs-victorialogs)). +([`services.md`](../../2-handbook/swarm/services.md#logs-victorialogs)). ## Still open diff --git a/docs/turn-loop/README.md b/docs/4-implementation/turn-loop/README.md similarity index 96% rename from docs/turn-loop/README.md rename to docs/4-implementation/turn-loop/README.md index 47716fd7..e7ade378 100644 --- a/docs/turn-loop/README.md +++ b/docs/4-implementation/turn-loop/README.md @@ -11,7 +11,7 @@ Every turn runs through `hive-runtime`, on the runtime - **`claude`** (default) — one `claude --print` process per turn. → [claude-invocation.md](claude-invocation.md) - **`acp`** — any Agent Client Protocol agent, run as one long-lived child - of the harness. → [`hive-runtime/README.md`](../../hive-runtime/README.md) + of the harness. → [`hive-runtime/README.md`](../../../hive-runtime/README.md) Both report the turn in claude's `stream-json` shape, so the loop below is the same for either. @@ -27,7 +27,7 @@ agents) runs: queued and unacked, so a resume drains the backlog instead of losing it; reminders and todo wakes buffer in their channels. Set it with `hivectl agent pause` or the dashboard toggle; see - [persistence](../agent-lifecycle/persistence.md#-harnesspaused-per-agent). + [persistence](../../2-handbook/agent-lifecycle/persistence.md#-harnesspaused-per-agent). 1. Long-poll `Recv` on its socket. The host-side broker (`broker.rs::recv_blocking_batch`) returns immediately if there's a pending message, otherwise waits up to 30 s for a broker `Sent` @@ -111,21 +111,21 @@ doc for the trait shape. `serve_main` reads `HIVE_PORT` (default `DEFAULT_WEB_PORT`) + `HIVE_LABEL` (default `"hive"` for standalone runs; the meta flake sets it unconditionally for any container-deployed agent; -see `docs/process/conventions.md::Hive identity` for the env stack), +see `docs/5-project/process/conventions.md::Hive identity` for the env stack), opens turn-stats sqlite, prepares the on-boot files (see [claude-invocation](claude-invocation.md#on-boot-files)), installs claude plugins, spawns `web_ui::serve` + `vacuum::run`, and either drops into `serve_loop` directly (`Online`) or parks on the login flow first (`NeedsLogin`). `hive-forge-notify` polls forge notifications on its own, not this loop — see -[`forge.md`](../integrations/forge.md). +[`forge.md`](../../2-handbook/integrations/forge.md). Boot also opens the todos store and the socket in-container producers dial. Matrix / bash / forge-notify daemons and the in-process `disk_watch` todo producer (low state-disk space) are the built-in producers, but the socket accepts any `subsystem` marker — a user-configured MCP server can push its own todos the same way. See -[`docs/agent-lifecycle/persistence.md`](../agent-lifecycle/persistence.md#state-dirs-per-agent) for +[`docs/2-handbook/agent-lifecycle/persistence.md`](../../2-handbook/agent-lifecycle/persistence.md#state-dirs-per-agent) for what each built-in todo producer watches and how the store + `get_loose_ends` merge work. @@ -214,7 +214,7 @@ path = [ "/run/wrappers" "/run/current-system/sw" ]; `/run/wrappers` (not `/run/wrappers/bin`) comes first so setuid wrappers — notably `sudo` — resolve before bare nix-store binaries; see -[`docs/process/gotchas.md`](../process/gotchas.md) ("`systemd.services.*.path` appends +[`docs/5-project/process/gotchas.md`](../../5-project/process/gotchas.md) ("`systemd.services.*.path` appends `/bin` to every entry") for why the trailing `/bin` matters in general. It's load-bearing here because the harness runs as the per-agent user: without the wrapper dir on `PATH`, `sudo` resolves to diff --git a/docs/turn-loop/claude-invocation.md b/docs/4-implementation/turn-loop/claude-invocation.md similarity index 97% rename from docs/turn-loop/claude-invocation.md rename to docs/4-implementation/turn-loop/claude-invocation.md index 2e2bb643..72a8a635 100644 --- a/docs/turn-loop/claude-invocation.md +++ b/docs/4-implementation/turn-loop/claude-invocation.md @@ -2,7 +2,7 @@ How a turn runs on the `claude` runtime, the default (`services.hyperhive.agent.runtime`). An `acp` agent's turns: -→ [`hive-runtime/README.md`](../../hive-runtime/README.md). +→ [`hive-runtime/README.md`](../../../hive-runtime/README.md). ``` claude --print --verbose --output-format stream-json --model \ @@ -39,7 +39,7 @@ writes its store path into the container's `/etc/hyperhive-hive-facts` as that runs that build when the path exists, and the container's own `claude-code` when the variable is unset or the path no longer exists, saying so on stderr in the second case. A bump reaches an agent at its next container -start, not at a rebuild. See docs/process/gotchas.md::`claude-code` is unfree. +start, not at a rebuild. See docs/5-project/process/gotchas.md::`claude-code` is unfree. Hive-enforced settings ship at `/etc/claude-code/managed-settings.json` (claude-code's canonical managed-settings path — precedence #1, @@ -170,7 +170,7 @@ for the window on turns the model didn't report one. `0` disables proactive compaction (the reactive path always applies). The proactive path is best-effort — a failed checkpoint or `/compact` never fails the turn that already succeeded. An ACP agent gets the same policy; how it compacts is in -[`hive-runtime/README.md`](../../hive-runtime/README.md#acp-backend-compaction). +[`hive-runtime/README.md`](../../../hive-runtime/README.md#acp-backend-compaction). The operator can force a compaction any time via `POST /api/compact`. It's **deferred**: the handler sets `Bus::request_compact()` and returns @@ -216,7 +216,7 @@ agent's durable bind-mounted dir. AGENTS.md autoload walks upward from `/state` — drop a per-agent AGENTS.md there if you want long-term hints that survive destroy/recreate. A `CLAUDE.md` in `/state` or above it shadows `AGENTS.md`: claude then loads only -the `CLAUDE.md` (see `docs/agent-lifecycle/persistence.md` for the +the `CLAUDE.md` (see `docs/2-handbook/agent-lifecycle/persistence.md` for the boot-time rename). The wake prompt is intentionally minimal: the popped message's @@ -263,7 +263,7 @@ needs to `chown` a bind mount), once at startup: share everything else. `` and `` blocks gate the config-repo paragraphs: an agent with `HIVE_ARCHETYPE` set (the meta flake sets it on an agent built - from an [archetype](../swarm/archetypes.md)) gets the archetype block and + from an [archetype](../../2-handbook/swarm/archetypes.md)) gets the archetype block and every other agent the repo block, with `{archetype}` filled in with the archetype's name. Five placeholders are then substituted: `{label}` (short agent name), `{qualified_label}` diff --git a/docs/turn-loop/config.md b/docs/4-implementation/turn-loop/config.md similarity index 99% rename from docs/turn-loop/config.md rename to docs/4-implementation/turn-loop/config.md index b4f22e0c..7d688dbd 100644 --- a/docs/turn-loop/config.md +++ b/docs/4-implementation/turn-loop/config.md @@ -306,7 +306,7 @@ controller derives only from an operator's save, never from text it finds in the store, so an environment saved before this existed needs one more save. An agent that has just the file gets no entry, and an operator adds its provider's host in the swarm UI (see -[allowed domains](../swarm/ui.md#allowed-domains)). +[allowed domains](../../2-handbook/swarm/ui.md#allowed-domains)). Provision the file once, out of band (never through nix — an API key in the store is world-readable and travels with the flake closure): @@ -323,7 +323,7 @@ KEYS The file lives in the agent's bind-mounted **harness** dir, not `state/` — `harness/` survives container rebuilds exactly like `state/` does, but (unlike `state/`) is never bind-mounted into a parent agent's container -(see `docs/agent-lifecycle/persistence.md`'s "Parent access to child state"), so an API +(see `docs/2-handbook/agent-lifecycle/persistence.md`'s "Parent access to child state"), so an API key placed here is reachable by nothing but this agent and the host. The file doesn't stay root-owned, though: the container's activation chowns the whole harness dir to the agent, so after the next boot the file is agent-owned diff --git a/docs/turn-loop/mcp.md b/docs/4-implementation/turn-loop/mcp.md similarity index 91% rename from docs/turn-loop/mcp.md rename to docs/4-implementation/turn-loop/mcp.md index 4e0e21bd..9fbc6e49 100644 --- a/docs/turn-loop/mcp.md +++ b/docs/4-implementation/turn-loop/mcp.md @@ -20,7 +20,7 @@ entry on the schema's stdio default — it spawns the tools land in claude as `mcp__hyperhive__`. Each entry also has an `availableToSubagents` toggle (default `false`) — see -[`docs/tools/subagent.md`](../tools/subagent.md#mcp-servers-available-to-a-subagent) +[`docs/2-handbook/tools/subagent.md`](../../2-handbook/tools/subagent.md#mcp-servers-available-to-a-subagent) for what reaches a subagent's own `--mcp-config` and what doesn't. An `"http"` entry that needs request headers (a bearer-authenticated remote server) sets `headersFile` to an absolute path: a file holding a JSON object @@ -29,7 +29,7 @@ file), so the secret never enters the nix store, and claude re-reads a rotated t on its next reconnect. ACP agents don't read it. The agent's permissions gate tool access -([`docs/process/conventions.md#permissions`](../process/conventions.md)). +([`docs/5-project/process/conventions.md#permissions`](../../5-project/process/conventions.md)). The default (`Permission::DEFAULT`) is `messaging`, `meta`, `inbox` and `execution`; `swarmctl agent permissions set` changes it, and the next turn runs on the change. @@ -48,7 +48,7 @@ runs on the change. (default: empty = unrestricted) limits which names `send` accepts — useful for sandboxing. The broker refuses `to: "operator"`. The error points an agent that has the matrix MCP tools at matrix - ([matrix](../tools/matrix.md)), and tells one without them that the + ([matrix](../../2-handbook/tools/matrix.md)), and tells one without them that the operator reads its `set_status` line, its turn output on the dashboard, and its logs. - `recv` — drain inbox. Always an immediate peek, never blocks. `max` @@ -74,7 +74,7 @@ runs on the change. **System messages** (from sender `system`): the broker delivers them as regular inbox messages (same `recv` path). A container crash never reaches an agent — it only produces a log line + swarm-level grafana -alert, no agent turn at all (`docs/scheduler/observability.md`). Lifecycle +alert, no agent turn at all (`docs/4-implementation/scheduler/observability.md`). Lifecycle transitions the job-queue scheduler or crash watcher drive directly — stop/kill, destroy, a flake-rev login or logout state change — reach no individual agent: they publish onto a swarm-wide NATS @@ -89,8 +89,8 @@ at_unix_timestamp?)`. reminders, open todos pushed by an in-container subsystem (matrix, bash, forge, or any user-configured MCP server), an undelivered-inbox count, and unread matrix notifications. Always the caller's own rows. Full per-variant wire shape in - [`docs/process/conventions.md` § Loose-ends wire - shape](../process/conventions.md#loose-ends-wire-shape). + [`docs/5-project/process/conventions.md` § Loose-ends wire + shape](../../5-project/process/conventions.md#loose-ends-wire-shape). - `cancel_loose_end` — hard-delete a `reminder` or clear a `todo` row (loose-ends-v2). Agents may only cancel rows they own. - `remind` — schedule a reminder in this agent's own inbox. Large @@ -139,30 +139,30 @@ hive_name?, swarm_name?, matrix_accounts? }`. `matrix_accounts` is a ## Privileged tools (by permission) - **Bash execution** (`execution`) — background shell tasks. See - [`docs/tools/bash.md`](../tools/bash.md). + [`docs/2-handbook/tools/bash.md`](../../2-handbook/tools/bash.md). - **Subagent spawning** — nested sessions on the agent's own runtime (claude or ACP, via `hive-runtime`) as background tasks, shipped default-on like bash execution; no permission gates it. See - [`docs/tools/subagent.md`](../tools/subagent.md). + [`docs/2-handbook/tools/subagent.md`](../../2-handbook/tools/subagent.md). - **Config changes** — no permission: they go through a forge PR on `agent-configs/` — see - [`docs/agent-lifecycle/config-changes.md`](../agent-lifecycle/config-changes.md). + [`docs/2-handbook/agent-lifecycle/config-changes.md`](../../2-handbook/agent-lifecycle/config-changes.md). - **Web egress** (`web_tools`) — enables Claude's built-in `WebFetch` and `WebSearch` tools (not MCP tools; added directly to the `--allowedTools` list). Off by default. - **Recovery mounts** (`manage_root_agent`) — no tool: every agent's state and config mounted into this agent's container. See - [`docs/process/conventions.md#permissions`](../process/conventions.md). + [`docs/5-project/process/conventions.md#permissions`](../../5-project/process/conventions.md). - **Matrix MCP + extra servers** — `mcp__matrix__*` tools and per-agent extra MCP config. See - [`docs/tools/matrix.md`](../tools/matrix.md). + [`docs/2-handbook/tools/matrix.md`](../../2-handbook/tools/matrix.md). ## Waking the agent from inside the container The built-in producers (matrix, bash, forge) upsert a todo on the harness's in-agent socket (`HIVE_AGENT_SOCKET`, `UpsertTodo` — see -[`docs/tools/bash.md` § Completion as a todo -(loose-ends v2)](../tools/bash.md#completion-as-a-todo-loose-ends-v2) +[`docs/2-handbook/tools/bash.md` § Completion as a todo +(loose-ends v2)](../../2-handbook/tools/bash.md#completion-as-a-todo-loose-ends-v2) for the full upsert/signal/clear mechanism): a new or changed summary makes the harness signal its own turn loop, with no hive-c0re round-trip. An external MCP server (or any other in-container process) @@ -205,7 +205,7 @@ The built-in list and the `--tools` value it resolves to live in `hive-sh4re` (the `Permission` enum itself is in `hive-types`), not in the harness, because the subagent daemon spawns its own `claude` and must resolve the same set — -see [`docs/tools/subagent.md`](../tools/subagent.md). +see [`docs/2-handbook/tools/subagent.md`](../../2-handbook/tools/subagent.md). - Allowed built-ins: `Edit`, `Glob`, `Grep`, `Read`, `Skill`, `Write`. `Skill` is what makes an installed plugin's `SKILL.md` invokable — diff --git a/docs/web-ui/css-vars.md b/docs/4-implementation/web-ui/css-vars.md similarity index 100% rename from docs/web-ui/css-vars.md rename to docs/4-implementation/web-ui/css-vars.md diff --git a/docs/web-ui/shape.md b/docs/4-implementation/web-ui/shape.md similarity index 98% rename from docs/web-ui/shape.md rename to docs/4-implementation/web-ui/shape.md index d1c9c5c1..300bfeda 100644 --- a/docs/web-ui/shape.md +++ b/docs/4-implementation/web-ui/shape.md @@ -1,7 +1,7 @@ # Shape (shared by both) -> Part of [Web UI](README.md). See also: -> [Dashboard layout](dashboard.md) · [Per-agent page](agent.md) +> Part of [Web UI](../../2-handbook/web-ui/README.md). See also: +> [Dashboard layout](../../2-handbook/web-ui/dashboard.md) · [Per-agent page](../../2-handbook/web-ui/agent.md) ## Shared routes @@ -272,7 +272,7 @@ The per-agent UI optionally binds a `UnixListener` instead of TCP when the nix module sets `HIVE_WEB_SOCKET` — the unix-socket transition mechanics (per-agent `/run/hive-agent//` bind-mount, `.bound` marker filtering, `agent-sockets.json` consumer on the -gateway side) live in [`docs/networking/gateway.md::Per-agent unix-socket +gateway side) live in [`docs/4-implementation/networking/gateway.md::Per-agent unix-socket upstream`](../networking/gateway.md). The env var is opt-in per agent so the two modes coexist while sub-agents transition. diff --git a/docs/web-ui/terminal-rendering.md b/docs/4-implementation/web-ui/terminal-rendering.md similarity index 100% rename from docs/web-ui/terminal-rendering.md rename to docs/4-implementation/web-ui/terminal-rendering.md diff --git a/docs/process/conventions.md b/docs/5-project/process/conventions.md similarity index 96% rename from docs/process/conventions.md rename to docs/5-project/process/conventions.md index 4b1898dd..76d454e9 100644 --- a/docs/process/conventions.md +++ b/docs/5-project/process/conventions.md @@ -43,7 +43,7 @@ agent's build, and neither do the hive's other per-host values: the dashboard port (`HIVE_DASHBOARD_PORT`), the pinned claude-code (`HYPERHIVE_CLAUDE_CODE_PATH`), extra OTEL labels (`HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`) and the queue's address -(`HIVE_AGENT_NATS_URL`, from `deploy.hive-controller.queue.agentNatsUrl`) and the egress proxy's address and direct hosts (`HIVE_SQUID_PROXY`, `HIVE_EGRESS_NO_PROXY`, see [the egress proxy](../networking/network.md#egress-proxy-squid)). `hive-priv` writes them to +(`HIVE_AGENT_NATS_URL`, from `deploy.hive-controller.queue.agentNatsUrl`) and the egress proxy's address and direct hosts (`HIVE_SQUID_PROXY`, `HIVE_EGRESS_NO_PROXY`, see [the egress proxy](../../4-implementation/networking/network.md#egress-proxy-squid)). `hive-priv` writes them to `/etc/hyperhive-hive-facts` before every container start, leaving out any the hive doesn't set; the agent then uses its own default. The agent's environment generator (`nix/agent-modules/hive-facts.nix`) passes all but the two egress facts, which only the proxy fetch reads, to every unit, @@ -77,8 +77,8 @@ is caller-chosen so the wake prompt can label the source verbatim notifications upsert a todo on the harness's in-agent socket (`HIVE_AGENT_SOCKET`), which signals the same turn loop without a hive-c0re round-trip. See -[`docs/turn-loop/mcp.md` § Waking the agent from inside the -container](../turn-loop/mcp.md#waking-the-agent-from-inside-the-container). +[`docs/4-implementation/turn-loop/mcp.md` § Waking the agent from inside the +container](../../4-implementation/turn-loop/mcp.md#waking-the-agent-from-inside-the-container). Identity = socket means anything that can connect to `/run/hive/mcp.sock` is implicitly trusted to inject wakes. That's @@ -108,7 +108,7 @@ JSON line-delimited over unix sockets in both directions (host admin / manager / agent). SSE streams (`/dashboard/stream` on hive-c0re, `/events/stream` on the per-agent web UIs) are `text/event-stream`; each frame carries a `seq` field for the snapshot-dedupe dance -(see `docs/web-ui/shape.md`). Request/response types live in `hive-sh4re` +(see `docs/4-implementation/web-ui/shape.md`). Request/response types live in `hive-sh4re` — change them in one place. The dashboard event vocabulary lives in `hive-c0re::dashboard_events::DashboardEvent`. @@ -286,7 +286,7 @@ host access it has. | `messaging` | `send`, `recv`, `ack_until` | | `meta` | `get_agent_meta` (`set_status` is always-on, see below) | | `inbox` | `get_loose_ends`, `cancel_loose_end`, `remind` | -| `execution` | the `bash` MCP server (`mcp__bash__run`/`status`/`kill`), rendered into the agent's MCP config only with it; a subagent gets Claude's own `Bash` instead. See `docs/tools/bash.md`. | +| `execution` | the `bash` MCP server (`mcp__bash__run`/`status`/`kill`), rendered into the agent's MCP config only with it; a subagent gets Claude's own `Bash` instead. See `docs/2-handbook/tools/bash.md`. | | `web_tools` | the Claude built-ins `WebFetch`/`WebSearch` | | `manage_root_agent` | every agent's state (rw) bind-mounted into the holder's container, plus `/meta` (ro) | | `logs` | the log store: `swarm-logs` queries. Checked by `swarm-logs-auth` at the gateway, not by a tool list. Off by default. | @@ -351,7 +351,7 @@ the field out and a `DELETE` (`swarmctl agent permissions clear`) also keep the stored value; a `DELETE` on an agent that has one rewrites the record to just that value. Read it with `swarm_queue_client::agent_permissions::unrestricted`. The hive reads it for -the agent's squid policy: [the egress proxy](../networking/network.md#egress-proxy-squid) +the agent's squid policy: [the egress proxy](../../4-implementation/networking/network.md#egress-proxy-squid) lets an unrestricted agent's credential reach any destination, and holds the others to the base list and the allow-list. An agent sends its credential once it has fetched it, from its next start. @@ -361,7 +361,7 @@ explicit `unrestricted_internet: false`; read the pair with `unrestricted_internet` without `internet_log` turns it off, the route refuses `internet_log: true` unless the stored or requested flag is `false`, and a `DELETE` keeps it. A hive that doesn't read the field sees `false` and enforces -the list. The proxy's handling is under _Log mode_ in [the egress proxy](../networking/network.md#egress-proxy-squid). +the list. The proxy's handling is under _Log mode_ in [the egress proxy](../../4-implementation/networking/network.md#egress-proxy-squid). **How a change applies** — - The agent's harness reads its own key every 10 seconds and builds every @@ -433,7 +433,7 @@ into the `Swap` node, then runs `nixos-container update` + stop + start across the `StopForUpdate → Swap → RebuildBookkeeping` brace and the tail `Reconcile` node. `flake.nix` itself lives in the agent's `agent-configs/` repo and rides along with its pinned commit (see -`docs/agent-lifecycle/config-changes.md::Deploy markers per agent`). +`docs/2-handbook/agent-lifecycle/config-changes.md::Deploy markers per agent`). Anything that changes per-container state on the host should be re-applied here so a manual `↻ R3BU1LD` from the dashboard is @@ -497,20 +497,20 @@ mara — don't add `#[allow]` speculatively. The gate is intentional. fmt / `cargo test` cover most gates, but `nix flake check` runs extra check derivations they don't: -- **`hivectl-docs`** regenerates `docs/tools/hivectl-cli.md` from +- **`hivectl-docs`** regenerates `docs/2-handbook/reference/hivectl-cli.md` from hivectl's clap tree and **fails if the committed copy is stale**. **After any change to a hivectl verb or flag, regenerate it**: ```sh - nix develop -c cargo run --bin hivectl -- markdown-docs > docs/tools/hivectl-cli.md + nix develop -c cargo run --bin hivectl -- markdown-docs > docs/2-handbook/reference/hivectl-cli.md ``` clippy / fmt / `cargo test` all pass *without* this — only the flake check catches the drift, and `ci-log` often can't show you why (it 500s on a fast failure), so you're left guessing "builder flake" when it's a stale doc. - **`swarmctl-docs`** is the same check for `swarmctl` / - `docs/tools/swarmctl-cli.md`: + `docs/2-handbook/reference/swarmctl-cli.md`: ```sh - nix develop -c cargo run --bin swarmctl -- markdown-docs > docs/tools/swarmctl-cli.md + nix develop -c cargo run --bin swarmctl -- markdown-docs > docs/2-handbook/reference/swarmctl-cli.md ``` - there's also a flake `cargo-test` check and NixOS module evaluation in the set. @@ -554,7 +554,7 @@ provisioning time: (The matrix profile avatar is **not** a oneshot — `hive-matrix-daemon` sets it over its live authenticated Client; see -`docs/agent-lifecycle/persistence.md::matrix avatar`.) +`docs/2-handbook/agent-lifecycle/persistence.md::matrix avatar`.) Shape contract — every one of these: @@ -575,7 +575,7 @@ Shape contract — every one of these: 5. **Re-runnable**: a second invocation produces the same final state (idempotent uploads, idempotent config rewrites). Used by the `.path` watchers that re-fire on token appearance (see - `docs/agent-lifecycle/persistence.md::Matrix per-agent daemon`). + `docs/2-handbook/agent-lifecycle/persistence.md::Matrix per-agent daemon`). The service stays root-owned so the bootstrap ordering doesn't need a user-existence check before each fire. diff --git a/docs/process/gotchas.md b/docs/5-project/process/gotchas.md similarity index 97% rename from docs/process/gotchas.md rename to docs/5-project/process/gotchas.md index 5fdcb766..43c73a5c 100644 --- a/docs/process/gotchas.md +++ b/docs/5-project/process/gotchas.md @@ -205,7 +205,7 @@ durable knowledge here (`notes.md`, anything else) — the harness's own internal files (`hyperhive-events.sqlite`, `hyperhive-turn-stats.sqlite`, `hyperhive-model`) live in the separate `harness` dir instead, so they don't clutter what claude sees as "my notes dir" (see -[`docs/agent-lifecycle/persistence.md`](../agent-lifecycle/persistence.md)). Survives `destroy`/recreate +[`docs/2-handbook/agent-lifecycle/persistence.md`](../../2-handbook/agent-lifecycle/persistence.md)). Survives `destroy`/recreate alongside the claude dir. ### Persistent work dir per agent @@ -309,14 +309,14 @@ nix build "git+/internal/meta.git?rev=#nixosConfigurations.argus.con nixos-containers' start script bind-mounts the host's nix-daemon socket directory into every container (hyperhive adds no bind for it; an agent with `services.hyperhive.agent.nixBuilds = "none"` gets it shadowed, see -`docs/trust-boundary/security.md`). nspawn +`docs/2-handbook/trust-boundary/security.md`). nspawn containers don't get user-namespaces by default, so `nix build` invocations _inside_ the container can't set up the build sandbox and fail outright if the host daemon's `nix.settings.sandbox-fallback` is `false` (nixpkgs default). `nix/agent-modules/default.nix` does `lib.mkForce true` so builds fall back to unsandboxed local builds rather than failing. Security -implications: `docs/trust-boundary/security.md`. +implications: `docs/2-handbook/trust-boundary/security.md`. ### Linking workspace binaries locally needs `nix develop` @@ -336,7 +336,7 @@ nix develop -c cargo run -p hive-c0re --bin hivectl -- ``` This is also how you regenerate committed generated docs locally — -for example `docs/tools/hivectl-cli.md` via the `hivectl markdown-docs` +for example `docs/2-handbook/reference/hivectl-cli.md` via the `hivectl markdown-docs` subcommand (its `hivectl-docs` flake check otherwise only fails in CI on drift). @@ -392,7 +392,7 @@ own self-consistent object store and is immune to the issue. ### `hive-forge`: prefer over raw curl pipelines -Full CLI reference: [`docs/tools/forge.md`](../tools/forge.md). +Full CLI reference: [`docs/2-handbook/tools/forge.md`](../../2-handbook/tools/forge.md). Never use raw `curl` for forge access. ## GUI (weston/VNC) @@ -402,13 +402,13 @@ Never use raw `curl` for forge access. `nix/agent-modules/weston-vnc.nix` adds an optional Weston Wayland compositor with the `vnc-backend` plugin, surfaced as `services.hyperhive.agent.gui.enable = true` per-agent. The harness's -`/screen/ws` WebSocket relay (`docs/web-ui/agent.md::Per-agent endpoints`) +`/screen/ws` WebSocket relay (`docs/2-handbook/web-ui/agent.md::Per-agent endpoints`) connects to the compositor at `127.0.0.1:`. - **Port allocation**: a **fixed** port (`services.hyperhive.agent.gui.vncPort`, default 5900). No per-agent hashing: network isolation is unconditional (each agent has its own netns — see - `docs/networking/network.md#container-isolation`), so the VNC port is + `docs/4-implementation/networking/network.md#container-isolation`), so the VNC port is container-local and can't collide across agents. The harness learns the port from the `HIVE_GUI_VNC_PORT` env var (set on the harness service when `gui.enable`) — no marker file, no runtime hash. (Unlike @@ -463,7 +463,7 @@ connects to the compositor at `127.0.0.1:`. must never block on weston signalling readiness. A misconfigured weston degrades to a `Restart=on-failure` loop visible in `journalctl`, it doesn't abort the `nixos-container update`. - Same reasoning as the best-effort oneshots in `docs/process/conventions.md`. + Same reasoning as the best-effort oneshots in `docs/5-project/process/conventions.md`. - **`[core] idle-time=0`**: disables weston's 300-second idle timeout. Without it the VNC desktop fades to black and desktop-shell shows its unlock screen — useless for an diff --git a/docs/process/pr-review-gate.md b/docs/5-project/process/pr-review-gate.md similarity index 100% rename from docs/process/pr-review-gate.md rename to docs/5-project/process/pr-review-gate.md diff --git a/docs/web-ui/design-guide.md b/docs/5-project/web-ui/design-guide.md similarity index 97% rename from docs/web-ui/design-guide.md rename to docs/5-project/web-ui/design-guide.md index b704fcdb..82ef74ca 100644 --- a/docs/web-ui/design-guide.md +++ b/docs/5-project/web-ui/design-guide.md @@ -8,7 +8,7 @@ things look like — that's `ComponentsPage` (`/components`), the living, always-current demo of every building block in `src/ui/`. Code can't go stale the way a doc's screenshots can, so treat `ComponentsPage` as the source of truth for appearance and this doc as the source of truth for intent. -Colour variables specifically are `docs/web-ui/css-vars.md`'s job, not +Colour variables specifically are `docs/4-implementation/web-ui/css-vars.md`'s job, not repeated here. ## Visual language @@ -61,7 +61,7 @@ smooth them instead. ## Theming The mechanical contract (base16 slots, semantic vars, what a page's CSS -is and isn't allowed to reference) lives in `docs/web-ui/css-vars.md` — +is and isn't allowed to reference) lives in `docs/4-implementation/web-ui/css-vars.md` — read that for the how. This section is the _policy_ layered on top: - **User-theming compatible by construction.** The whole point of the diff --git a/docs/README.md b/docs/README.md index 8caf44da..0d5ff323 100644 --- a/docs/README.md +++ b/docs/README.md @@ -3,132 +3,167 @@ Depth reference for hyperhive — the substrate, not the pitch (that's the [top-level README](../README.md) / [website](https://hyperhive.darkest.space)). Every page here stands alone; pick the one matching your task rather than -reading top to bottom. For the autogenerated NixOS options reference -(every `services.hyperhive.*` / `hyperhive.*` option, host and agent), see -[the options site](https://hyperhive.darkest.space/options/) instead — -this tree is prose, that one's generated straight from the module -declarations. +reading top to bottom. The docs sit in five sections, by what the reader wants: -## Getting started +1. [Getting started](#1-getting-started): what hyperhive is, and how to bring one up. +2. [Handbook](#2-handbook): running and using a hive and a swarm. +3. [Options reference](#3-options-reference): every NixOS option. +4. [Implementation](#4-implementation): how it works inside. +5. [Project](#5-project): how the project itself works. + +## 1. Getting started - **Adding hyperhive to a NixOS flake? Which options does it need?** → - [`getting-started/nixos.md`](getting-started/nixos.md). -- **Bringing a fresh swarm online?** → [`getting-started/setup.md`](getting-started/setup.md) + [`1-getting-started/nixos.md`](1-getting-started/nixos.md). +- **Bringing a fresh swarm online?** → [`1-getting-started/setup.md`](1-getting-started/setup.md) (the hand steps after the first switch). -- **What does the dashboard look like, and how to use it?** → - [`web-ui/`](web-ui/README.md) — the operator-facing starting point; - its own sub-pages ([`shape`](web-ui/shape.md), - [`dashboard`](web-ui/dashboard.md), [`agent`](web-ui/agent.md), - [`css-vars`](web-ui/css-vars.md), [`terminal-rendering`](web-ui/terminal-rendering.md)) - go deeper into implementation. -- **What tools does an agent (or the operator) have available?** → - [`tools/`](tools/README.md) — `hivectl` (yours) plus every agent's MCP - tool surface (bash, forge, lifecycle, matrix). -## Agent lifecycle +## 2. Handbook + +Using and operating the hive and the swarm. Some pages here still carry +implementation detail next to the operator content; sorting those out is +later work. + +### Web UI and tools + +- **What does the dashboard look like, and how to use it?** → + [`2-handbook/web-ui/`](2-handbook/web-ui/README.md) — the operator-facing starting point, + with [`dashboard`](2-handbook/web-ui/dashboard.md) and [`agent`](2-handbook/web-ui/agent.md). +- **What tools does an agent (or the operator) have available?** → + [`2-handbook/tools/`](2-handbook/tools/README.md) — `hivectl` (yours) plus every agent's MCP + tool surface (bash, forge, lifecycle, matrix). +- **What does `hivectl` do? Provisioning, gateway users, container + shells?** → [`2-handbook/tools/hivectl.md`](2-handbook/tools/hivectl.md) (the curated guide); + [`2-handbook/reference/hivectl-cli.md`](2-handbook/reference/hivectl-cli.md) for the exhaustive, + autogenerated flag reference. The other command-line references sit beside it in + [`2-handbook/reference/`](2-handbook/reference/). + +### Agent lifecycle - **How do config changes flow from manager to operator to container?** → - [`agent-lifecycle/config-changes.md`](agent-lifecycle/config-changes.md) + [`2-handbook/agent-lifecycle/config-changes.md`](2-handbook/agent-lifecycle/config-changes.md) (config PRs, the meta flake, system messages). - **What state survives destroy / purge / restart?** → - [`agent-lifecycle/persistence.md`](agent-lifecycle/persistence.md). + [`2-handbook/agent-lifecycle/persistence.md`](2-handbook/agent-lifecycle/persistence.md). - **Who can do what to whom — the agent roster and privilege?** → - [`agent-lifecycle/agent-roster.md`](agent-lifecycle/agent-roster.md). -- **How does claude get its prompt, and what tools does it have?** → - [`turn-loop/`](turn-loop/README.md) — the loop, binary shape, turn - outcomes; sub-pages: [`claude-invocation`](turn-loop/claude-invocation.md), - [`config`](turn-loop/config.md), [`mcp`](turn-loop/mcp.md). + [`2-handbook/agent-lifecycle/agent-roster.md`](2-handbook/agent-lifecycle/agent-roster.md). -## Trust boundary & security +### Trust boundary and security - **What's the operator/agent trust boundary? What's a permission?** → - [`trust-boundary/boundary.md`](trust-boundary/boundary.md). + [`2-handbook/trust-boundary/boundary.md`](2-handbook/trust-boundary/boundary.md). - **Agent trust model, prompt-injection threat model, credential - isolation?** → [`trust-boundary/security.md`](trust-boundary/security.md). + isolation?** → [`2-handbook/trust-boundary/security.md`](2-handbook/trust-boundary/security.md). -## Accounts & integrations +### Accounts and integrations - **How do per-agent forge accounts work? What does `forge_notify` poll, - and how does it format wake messages?** → [`integrations/forge.md`](integrations/forge.md) - (the swarm's Forgejo); [`tools/forge.md`](tools/forge.md) for the + and how does it format wake messages?** → [`2-handbook/integrations/forge.md`](2-handbook/integrations/forge.md) + (the swarm's Forgejo); [`2-handbook/tools/forge.md`](2-handbook/tools/forge.md) for the `hive-forge` CLI verbs agents actually call. - **How does the matrix-tuwunel container work? Multiple accounts per - agent?** → [`integrations/matrix.md`](integrations/matrix.md) (the homeserver); - [`tools/matrix.md`](tools/matrix.md) for the MCP tool surface and + agent?** → [`2-handbook/integrations/matrix.md`](2-handbook/integrations/matrix.md) (the homeserver); + [`2-handbook/tools/matrix.md`](2-handbook/tools/matrix.md) for the MCP tool surface and `services.hyperhive.agent.matrixAccounts`. - **How to give an agent a GitHub account (`gh` + `git push`)? how's - the PAT injected?** → [`integrations/github.md`](integrations/github.md) + the PAT injected?** → [`2-handbook/integrations/github.md`](2-handbook/integrations/github.md) (operator content up top; the `gh`/git-push + notification-poller mechanics are in a collapsed "Implementation" section at the bottom). - **How to give an agent GitLab accounts (`glab` + `git push`, more than one - instance)?** → [`integrations/gitlab.md`](integrations/gitlab.md). + instance)?** → [`2-handbook/integrations/gitlab.md`](2-handbook/integrations/gitlab.md). - **What's `/knowledge`? How does the swarm-wide knowledge repo sync, - and how to contribute a document?** → [`integrations/knowledge.md`](integrations/knowledge.md). -- **What does `hivectl` do? Provisioning, gateway users, container - shells?** → [`tools/hivectl.md`](tools/hivectl.md) (the curated guide); - [`tools/hivectl-cli.md`](tools/hivectl-cli.md) for the exhaustive, - autogenerated flag reference. + and how to contribute a document?** → [`2-handbook/integrations/knowledge.md`](2-handbook/integrations/knowledge.md). -## Networking & swarms +### Swarm -- **What nginx vhosts does the gateway serve? How does matrix - discovery work?** → [`networking/gateway.md`](networking/gateway.md). -- **How does DNS resolution work in agent containers? What's the - bridge network for?** → [`networking/network.md`](networking/network.md). -- **How are two hives connected into a swarm?** → [`swarm/`](swarm/README.md) +- **How are two hives connected into a swarm?** → [`2-handbook/swarm/`](2-handbook/swarm/README.md) (peer hives, TLS trust). -- **Where do agent snapshots go? How does the swarm's `btrfs receive` - endpoint authenticate a pushing hive?** → - [`networking/snapshot-store.md`](networking/snapshot-store.md). - **Who mints each credential, who reads it, and how does it rotate — and - where's that shape headed?** → [`swarm/credentials.md`](swarm/credentials.md) + where's that shape headed?** → [`2-handbook/swarm/credentials.md`](2-handbook/swarm/credentials.md) (the rules, where each secret goes and why) and - [`swarm/credentials-reference.md`](swarm/credentials-reference.md) (the + [`2-handbook/swarm/credentials-reference.md`](2-handbook/swarm/credentials-reference.md) (the tables: each credential, current state and target state, and where each file lives today). -- **How does the secret store come up, who writes its grants, how's it - sealed?** → [`swarm/bao.md`](swarm/bao.md). - **How does an agent get a prompt at set times, and why doesn't it fire - twice after downtime?** → [`swarm/schedules.md`](swarm/schedules.md). -- **How to set an agent's model or effort from the swarm, and what happens - to its own pick?** → [`swarm/inference.md`](swarm/inference.md). -- **How does a PR get checked against its repo's own yes/no rules before - a full review?** → [`swarm/review-classifier.md`](swarm/review-classifier.md). -- **Does an agent config PR build before it's merged?** → - [`swarm/builder.md`](swarm/builder.md). + twice after downtime?** → [`2-handbook/swarm/schedules.md`](2-handbook/swarm/schedules.md). - **How to create an agent from a ready-made set of permissions and - settings?** → [`swarm/archetypes.md`](swarm/archetypes.md). -- **How long do CI runs take, per repo and PR, and where does the board get - it?** → [`swarm/ci-metrics.md`](swarm/ci-metrics.md). + settings?** → [`2-handbook/swarm/archetypes.md`](2-handbook/swarm/archetypes.md). - **How to log an agent in to claude from the swarm UI, and why doesn't a stale stored copy log it out again?** → - [`swarm/claude-login.md`](swarm/claude-login.md). -- **How does an agent get a todo when someone assigns it an issue, and lose it - when the issue closes?** → [`swarm/notify-router.md`](swarm/notify-router.md). + [`2-handbook/swarm/claude-login.md`](2-handbook/swarm/claude-login.md). -## Scheduler, CI, observability +## 3. Options reference +The autogenerated NixOS options reference (every `services.hyperhive.*` / +`hyperhive.*` option, host and agent) lives on the website, built from the +module declarations: [the options site](https://hyperhive.darkest.space/options/). +Nothing of it sits in this tree. + +## 4. Implementation + +### Turn loop, scheduler, CI, observability + +- **How does claude get its prompt, and what tools does it have?** → + [`4-implementation/turn-loop/`](4-implementation/turn-loop/README.md) — the loop, binary shape, turn + outcomes; sub-pages: [`claude-invocation`](4-implementation/turn-loop/claude-invocation.md), + [`config`](4-implementation/turn-loop/config.md), [`mcp`](4-implementation/turn-loop/mcp.md). - **what's the job queue, as a general idea (not hive-c0re specifics)?** → - [`scheduler/jobq.md`](scheduler/jobq.md) — operator-facing, no implementation detail. + [`4-implementation/scheduler/jobq.md`](4-implementation/scheduler/jobq.md) — operator-facing, no implementation detail. - **How does the rebuild queue work? What are the concrete step kinds, - queue sources, scheduler internals?** → [`scheduler/coordinator.md`](scheduler/coordinator.md). + queue sources, scheduler internals?** → [`4-implementation/scheduler/coordinator.md`](4-implementation/scheduler/coordinator.md). - **How does the CI runner work? How does it register?** → - [`scheduler/ci.md`](scheduler/ci.md). + [`4-implementation/scheduler/ci.md`](4-implementation/scheduler/ci.md). - **How are Claude Code metrics (tokens, cost, tool calls) exported to - Prometheus/Grafana?** → [`scheduler/observability.md`](scheduler/observability.md). + Prometheus/Grafana?** → [`4-implementation/scheduler/observability.md`](4-implementation/scheduler/observability.md). -## Crate reference +### Networking + +- **What nginx vhosts does the gateway serve? How does matrix + discovery work?** → [`4-implementation/networking/gateway.md`](4-implementation/networking/gateway.md). +- **How does DNS resolution work in agent containers? What's the + bridge network for?** → [`4-implementation/networking/network.md`](4-implementation/networking/network.md). +- **Where do agent snapshots go? How does the swarm's `btrfs receive` + endpoint authenticate a pushing hive?** → + [`4-implementation/networking/snapshot-store.md`](4-implementation/networking/snapshot-store.md). + +### Swarm services + +- **How does the secret store come up, who writes its grants, how's it + sealed?** → [`4-implementation/swarm/bao.md`](4-implementation/swarm/bao.md). +- **How to set an agent's model or effort from the swarm, and what happens + to its own pick?** → [`4-implementation/swarm/inference.md`](4-implementation/swarm/inference.md). +- **How does a PR get checked against its repo's own yes/no rules before + a full review?** → [`4-implementation/swarm/review-classifier.md`](4-implementation/swarm/review-classifier.md). +- **Does an agent config PR build before it's merged?** → + [`4-implementation/swarm/builder.md`](4-implementation/swarm/builder.md). +- **How long do CI runs take, per repo and PR, and where does the board get + it?** → [`4-implementation/swarm/ci-metrics.md`](4-implementation/swarm/ci-metrics.md). +- **How does an agent get a todo when someone assigns it an issue, and lose it + when the issue closes?** → [`4-implementation/swarm/notify-router.md`](4-implementation/swarm/notify-router.md). + +### Web UI internals + +- **How's the dashboard built, and how does the terminal render?** → + [`shape`](4-implementation/web-ui/shape.md), + [`css-vars`](4-implementation/web-ui/css-vars.md), + [`terminal-rendering`](4-implementation/web-ui/terminal-rendering.md). + +### Crate reference - **What does a specific Rust crate do, on its own terms?** → - [`crates/`](crates/README.md) — every workspace crate's own + [`4-implementation/crates/`](4-implementation/crates/README.md) — every workspace crate's own `README.md`, one level up from source; the crate itself is still the source of truth, this is just a walkable mirror. -## Process & conventions +## 5. Project + +How the project itself works: processes, conventions, design guides. - **Naming, commit style, wire protocol, the `data-async` pattern?** → - [`process/conventions.md`](process/conventions.md). -- **Why does the nspawn flag look like that?** → [`process/gotchas.md`](process/gotchas.md) + [`5-project/process/conventions.md`](5-project/process/conventions.md). +- **Why does the nspawn flag look like that?** → [`5-project/process/gotchas.md`](5-project/process/gotchas.md) (bind mounts, conf flags, other NixOS/nspawn quirks). - **What does a PR review verdict actually gate?** → - [`process/pr-review-gate.md`](process/pr-review-gate.md). + [`5-project/process/pr-review-gate.md`](5-project/process/pr-review-gate.md). +- **How are the web UIs designed to look and behave?** → + [`5-project/web-ui/design-guide.md`](5-project/web-ui/design-guide.md). diff --git a/frontend/packages/agent/src/agent.css b/frontend/packages/agent/src/agent.css index 03f9e623..edc17b48 100644 --- a/frontend/packages/agent/src/agent.css +++ b/frontend/packages/agent/src/agent.css @@ -16,7 +16,7 @@ /* ─── full-screen layout overrides ───────────────────────────────── The agent page mounts a full-viewport terminal under a fixed - frosted-glass header. See docs/web-ui/agent.md::Per-agent + frosted-glass header. See docs/2-handbook/web-ui/agent.md::Per-agent page for the layer / pill / side-panel structure; rules below override the in-page defaults from @hive/shared. */ @@ -184,7 +184,7 @@ h3 { } .agent-icon { /* Square identity anchor — explicit 5em sizing + align-self. - See docs/web-ui/agent.md::Per-agent page (Agent icon) for the + See docs/2-handbook/web-ui/agent.md::Per-agent page (Agent icon) for the intrinsic-dim-pushes-parent-flex-open + sticks-to-top rationale. */ width: 5em; @@ -684,7 +684,7 @@ pre.diff { /* Tail pill (↓ N new): z-index bumped above the floating chrome so the pill sits on the top layer instead of being clipped by it. (Pill is anchored in `.agent-main` rather than `.terminal-wrap` — see - docs/web-ui/agent.md::Per-agent page Terminal-wrap paragraph for the + docs/2-handbook/web-ui/agent.md::Per-agent page Terminal-wrap paragraph for the backdrop-filter stacking-context rationale.) */ .agent-main .tail-pill { bottom: 0.6em; diff --git a/frontend/packages/agent/src/colors.css b/frontend/packages/agent/src/colors.css index 72edd746..ef910c14 100644 --- a/frontend/packages/agent/src/colors.css +++ b/frontend/packages/agent/src/colors.css @@ -2,5 +2,5 @@ esbuild emits its own `dist/static/colors.css`, linked first by every agent page (before theme.css). This is the theme swap contract: a stylix-generated variant replaces only this file. See - @hive/shared/colors.css + docs/web-ui/css-vars.md. */ + @hive/shared/colors.css + docs/4-implementation/web-ui/css-vars.md. */ @import "@hive/shared/colors.css"; diff --git a/frontend/packages/agent/src/components/Header.tsx b/frontend/packages/agent/src/components/Header.tsx index 54ad3558..00b7d276 100644 --- a/frontend/packages/agent/src/components/Header.tsx +++ b/frontend/packages/agent/src/components/Header.tsx @@ -53,7 +53,7 @@ export interface HeaderProps { * a variable pill count) grows to fit badges+pills together, keeping * the header a stable height even on widescreen. Meta-nav * (``) lives here too, as a fixed-size trigger, not an - * inline link list in the title row (docs/web-ui/agent.md). */ + * inline link list in the title row (docs/2-handbook/web-ui/agent.md). */ pills?: ComponentChildren; } diff --git a/frontend/packages/agent/src/index.html b/frontend/packages/agent/src/index.html index 4c9ce097..0b1f9494 100644 --- a/frontend/packages/agent/src/index.html +++ b/frontend/packages/agent/src/index.html @@ -19,7 +19,7 @@ + tree; docs/2-handbook/web-ui/agent.md for the design. -->
diff --git a/frontend/packages/agent/src/screen.html b/frontend/packages/agent/src/screen.html index b4be39a7..4cea97a3 100644 --- a/frontend/packages/agent/src/screen.html +++ b/frontend/packages/agent/src/screen.html @@ -36,7 +36,7 @@