docs: restructure into topic subdirectories, collapse duplicated index

Per mara's go-ahead on hyperhive#3902 ("getting started is good, but
terminal rendering does not go in there i think"):

Moved 21 top-level docs/*.md files into 7 new topic subdirectories
(existing web-ui/, turn-loop/, swarm/, tools/, crates/ untouched):
  getting-started/  setup.md
  agent-lifecycle/  agent-hierarchy.md, approvals.md, persistence.md
  trust-boundary/   boundary.md, security.md
  integrations/     forge.md, matrix.md, github.md, knowledge.md
  networking/       gateway.md, network.md, snapshot-store.md
  scheduler/        jobq.md, coordinator.md, ci.md, observability.md
  process/          conventions.md, gotchas.md, pr-review-gate.md
  web-ui/           terminal-rendering.md (moved into the EXISTING dir,
                    per mara's correction to the original getting-started
                    guess -- it's UI implementation detail, not onboarding)

The physical layout now matches docs/README.md's own topical headers,
which already amounted to this taxonomy -- see the scoping comment on
the issue for the two findings that motivated this (a genuine
duplication between CLAUDE.md's old "Reading paths" list and
docs/README.md's grouped one, since drifted out of sync with each
other; and the flat layout not matching the grouping we already had).

Fixed every cross-reference this moved across the whole repo (~120
files: docs/ internal links at every depth, Rust doc comments, nix
module option docs, crate READMEs) -- verified two ways: a grep sweep
confirming zero remaining references to any old path, and a script
that resolves every markdown link in docs/**/*.md + CLAUDE.md +
README.md against the filesystem and reports anything that doesn't
exist (zero broken links).

Collapsed CLAUDE.md's "Reading paths" section (the duplicate) down to
a pointer at docs/README.md, now the single index. Rewrote
docs/README.md itself to use the new subdirectory paths and added the
one doc it was missing that CLAUDE.md's old copy had (pr-review-gate.md).

Classified all 22 docs/*.md files first via a haiku subagent (mara's
suggestion) on two axes -- proposed grouping and operator-vs-
implementation focus -- before finalizing the taxonomy; spot-checked
the report and found internal inconsistencies (its classification
table disagreed with its own summary section for a few files), so this
taxonomy is my original proposal + the one correction mara gave
directly, not a blind application of the subagent's table. The
operator-focus data it gathered is still useful for a follow-up
content pass (docs skewing 'mixed' rather than pure operator-facing),
not addressed in this PR -- structure only.

nix fmt clean, both pre-push lints clean.
This commit is contained in:
iris 2026-09-02 01:47:05 +02:00 committed by mara
commit 07b62612b0
124 changed files with 301 additions and 377 deletions

View file

@ -0,0 +1,142 @@
# Hive-wide knowledge repository
`internal/knowledge` on the forge is a shared reference doc repo
readable by every agent. hive-c0re clones it to the host and
bind-mounts the clone read-only into every agent container at
`/knowledge`.
## Agent access
Inside any agent container:
```
/knowledge/ # read-only bind-mount of the local clone
/knowledge/README.md # table of contents (seeded on first use)
```
Agents read documents directly from that path. The mount is
read-only — agents never write through it. To contribute, use the
`hive-forge` AGit flow (no fork needed — see
[Contributing](#contributing)); the operator reviews and merges, and
the local clone updates automatically (see
[Sync mechanism](#sync-mechanism) below).
## Repository layout
Canonical forge location: `internal/knowledge` (org `internal`,
repo `knowledge`). The repo is public, so every agent's forge account
has read access without an explicit per-agent collaborator grant;
only the `core` account has push access, for auto-seeding.
The repo is auto-created at hive-c0re startup if it doesn't exist,
seeded with a `README.md` containing a contribution guide and a
blank table of contents. Add an entry to that ToC each time you
create a new document.
## Sync mechanism
hive-c0re maintains the local clone at
`/var/lib/hyperhive/knowledge` via two paths:
1. **Swarm event** — the swarm controller holds the single push hook on
`internal/knowledge` (see `docs/swarm/README.md` § Swarm-wide forge
webhooks). On any push to main, including merge commits, it sends an
event to every hive over the swarm queue and each hive runs `git
pull`, so agents see the new content on their next turn.
A hive that is offline when the event is sent does not get it on
reconnect — the periodic pull below is what closes that gap. So one
hive briefly showing older `/knowledge` content than another is
expected, and resolves by itself within the fallback interval.
**Do not add a per-hive hook.** A webhook has exactly one target
URL, so a second registration against the same repo does not add a
recipient — it takes delivery away from whoever registered first.
Earlier versions had each hive register its own; hive-c0re now
removes its own leftover at startup, so no operator step is needed
to migrate.
2. **Periodic pull** — a background task in `hive-c0re::main`
pulls on a fixed cadence as a fallback (webhook missed, c0re
restarted between pushes). The pull is best-effort — a failure
logs a warning and does not affect the rest of the daemon.
Both paths share the same `knowledge::pull()` function, which also
handles the change notice below — neither path can forget to wire it
in since the broadcast logic lives once, in `pull()` itself, not at
each call site.
### Change notice
When a pull actually moves the local clone's `HEAD` (a real change,
not a no-op — e.g. the periodic pull finding nothing new), hive-c0re
broadcasts a short notice to every currently-registered agent's inbox:
sender `system`, body `[system] /knowledge updated:` followed by a
`git diff --stat <old>..<new>` summary of what changed (or a generic
"see the repo" fallback if computing the diff itself fails). This is
the same broadcast mechanism used for other hive-wide notices — inbox
message only, no forced wake, and it carries the standard "this was a
broadcast" hint. A diff or per-agent send failure is logged but never
blocks the pull itself.
The local clone is created (or refreshed) once at startup via
`knowledge::ensure_local_clone`. If the repo is brand new and
empty, `ensure_local_clone` seeds it with the default README
before returning.
## State
- **Host clone**: `/var/lib/hyperhive/knowledge` — persists across
hive-c0re restarts and agent destroy/recreate. Deleted only by
manual operator action.
- **In-container mount**: `/knowledge` — bind-mounted read-only
from the host clone on every container start. Gone when container
is stopped; reappears on next start with the current clone state.
The mount deliberately **excludes `.git`**: the host clone embeds the `core`
token in `.git/config` (it rides the clone URL), so hive-priv overlays an empty
tmpfs at `/knowledge/.git` — agents see the documents, not the repo metadata or
token.
## Contributing
Agents have read access to `internal/knowledge` (it's public) but no
write access, so they can't push a branch directly. The supported path
is Forgejo's **AGit flow** through the `hive-forge` CLI — no fork
required.
1. Clone the repo (credentials are injected automatically; the `-r`
flag selects the repo, the clone lands in `./knowledge`):
```sh
hive-forge -r internal/knowledge clone
cd knowledge
```
2. Create a branch and add or update a document, then commit normally.
3. Open (or update) a PR with `--agit`. This pushes the current `HEAD`
to `refs/for/<base>/<topic>`, which Forgejo turns into a PR even
though you can't push a branch:
```sh
hive-forge -r internal/knowledge pr-create --agit \
--title "docs: add the X runbook" \
--topic add-x-runbook \
--body-file - <<'EOF'
What this document adds and why.
EOF
```
Re-running with the same `--topic` updates the open PR (it
force-pushes the AGit scratch ref). The base defaults to `main`;
in `--agit` mode the push goes to the `origin` remote that
`hive-forge clone` set up.
4. The operator reviews and merges. The webhook fires on merge; every
running container sees the updated content within seconds (see
[Sync mechanism](#sync-mechanism)).
Do **not** try to `git push` a branch directly — lacking write access,
it's rejected. The `--agit` flow above is the no-fork path that works
from any agent.