ci: gate documentation pointers so a dead one fails the build

Pointing at a doc instead of duplicating its prose next to the code is what
the comment-block lint pushes people toward, and nothing checked that the
pointers still resolve. The reorg that prompted the preceding commit broke
41 references and every one of them merged green.

Two arms: repo-root-relative `docs/` paths, and relative markdown links
resolved against the linking file. Scope is every tracked file rather than a
type list — the dead references were in css, html and an ignore file as well
as in markdown and nix.

Three controls, because a link checker whose extractor quietly matches
nothing passes forever while checking nothing: each arm must have found
candidates, and at least one pointer must have resolved. Any of those failing
exits non-zero and says the script is broken rather than the tree.

Escape hatch is the `lint:allow` marker the tracker-tag lint already uses.
Its only current users are in reference-docs.nix, where a path naming a file
the derivation synthesizes into $out is correct precisely because the repo
does not have it.

Also runs in the pre-push hook alongside the other two.

.forgejo/workflows/ci.yml

@@ -53,3 +53,17 @@ jobs:
         # protection) — a hit blocks merge. See
         # scripts/check-comment-blocks.sh.
         run: sh scripts/check-comment-blocks.sh
+
+  doc-refs:
+    name: doc-pointer lint
+    runs-on: [hive-ci]
+    # Pure git+grep — seconds.
+    timeout-minutes: 5
+    steps:
+      - uses: actions/checkout@v3
+      - name: lint
+        # Flags `docs/…md` paths and relative markdown links that no
+        # longer resolve. Pointing at a doc is what the comment-block
+        # lint above pushes people toward, so the pointers need a gate of
+        # their own. See scripts/check-doc-refs.sh.
+        run: sh scripts/check-doc-refs.sh

.prettierignore

@@ -8,12 +8,12 @@ docs/tools/hivectl-cli.md
 # indentation normalisation, so these files need manual re-wrap before
 # adding them back to the formatter scope.
 CLAUDE.md
-docs/approvals.md
-docs/conventions.md
-docs/gateway.md
-docs/matrix.md
-docs/persistence.md
-docs/terminal-rendering.md
+docs/agent-lifecycle/approvals.md
+docs/process/conventions.md
+docs/networking/gateway.md
+docs/integrations/matrix.md
+docs/agent-lifecycle/persistence.md
+docs/web-ui/terminal-rendering.md
 docs/tools/forge.md
 docs/tools/matrix.md
 docs/turn-loop/README.md

docs/agent-lifecycle/approvals.md

@@ -209,7 +209,7 @@ One-shot rows fire once (if past due, on the next worker pass) and are deleted b
 ### Scheduled prompt delivery: todo, not a broker message
 
 An agent target's delivery is `push_todo` (`Coordinator::push_todo`,
-`docs/coordinator.md` covers the mechanism generally), not a broker
+`docs/scheduler/coordinator.md` covers the mechanism generally), not a broker
 `Message` — a scheduled prompt wakes its target with a todo instead of
 driving an immediate turn, by design. `key = "schedule:<id>"` per
 target drives `push_todo`'s own upsert-by-key dedup: a re-fire of the
@@ -419,7 +419,7 @@ submitter pushes again (or closes it) to retry.
 
 Long-running approval work — `MergeConfigPr`, `UpdateMetaInputs`,
 `Spawn` — runs as a DAG on the global job queue
-(`docs/coordinator.md::Job queue`), submitted by the approval handler
+(`docs/scheduler/coordinator.md::Job queue`), submitted by the approval handler
 rather than run inline:
 
 | `ApprovalKind` | DAG submitted | source |

docs/agent-lifecycle/persistence.md

@@ -466,7 +466,7 @@ subvolume creation/deletion above.
 
 This is the same subvolume `hivectl agent <name> subvol snapshot push`
 sends to the swarm's snapshot store — see
-[`docs/snapshot-store.md`](../networking/snapshot-store.md) for what a pushed
+[`docs/networking/snapshot-store.md`](../networking/snapshot-store.md) for what a pushed
 snapshot contains and how the store authenticates a sender.
 
 ## `/var/lib/swarm-controller/` (swarm-controller host only)

docs/networking/network.md

@@ -184,7 +184,7 @@ host-side service on another port — e.g. a dev OTLP collector you want
 agents to reach directly — is unreachable. (hyperhive's own telemetry
 needs none of this: `otel.enable` opens its collector's port itself, and
 `otel.endpoint` is the *upstream*, which no agent ever dials. See
-`docs/observability.md`.)
+`docs/scheduler/observability.md`.)
 
 `services.hyperhive.network.exposeHostPorts = [ 4318 ];` opens each
 listed TCP port `P` on the bridge-interface `allowedTCPPorts`, so an
@@ -274,4 +274,4 @@ loopback — the hive-c0re admin socket is a UDS, not TCP.
 
 ## Cross-references
 
-- `docs/gateway.md` — vhost map + the gateway's other duties
+- `docs/networking/gateway.md` — vhost map + the gateway's other duties

docs/networking/snapshot-store.md

@@ -184,7 +184,7 @@ the access-review list.
 
 The snapshot covers an agent's **state subvolume**, which is the parent
 of `state/`, `claude/` and `harness/` (see
-[`docs/persistence.md`'s btrfs subvolume
+[`docs/agent-lifecycle/persistence.md`'s btrfs subvolume
 section](../agent-lifecycle/persistence.md#btrfs-subvolumes-for-varlibhyperhiveagentsname)
 for how and when that subvolume is created). Consequences:
 

docs/process/conventions.md

@@ -195,7 +195,7 @@ Per-variant fields:
   — `agent` is the affected agent (target of the spawn / config
   commit), not the asker. `description` is the manager's free-text
   blurb shown on the dashboard card. `commit_ref` is the
-  kind-specific payload (see `docs/approvals.md::Approval kinds
+  kind-specific payload (see `docs/agent-lifecycle/approvals.md::Approval kinds
   (wire shapes)`).
 - `Reminder { id, owner, message, due_at, age_seconds }` —
   `due_at` is the absolute time the scheduler is targeting (RFC
@@ -424,7 +424,7 @@ start runs across the `StopForUpdate → Swap → RebuildBookkeeping`
 brace and the tail `Reconcile` node. `flake.nix` itself is no longer
 regenerated host-side on rebuild — it's tracked in the agent's
 proposed/applied repos and rides along on every fetch (see
-`docs/approvals.md::Two repos per agent`).
+`docs/agent-lifecycle/approvals.md::Two repos per agent`).
 
 Anything that changes per-container state on the host should be
 re-applied here so a manual `↻ R3BU1LD` from the dashboard is
@@ -528,7 +528,7 @@ the state dir at provisioning time:
 
 (The matrix profile avatar is **not** a oneshot — `hive-matrix-daemon`
 sets it over its live authenticated Client; see
-`docs/persistence.md::matrix avatar`.)
+`docs/agent-lifecycle/persistence.md::matrix avatar`.)
 
 Shape contract — every one of these:
 
@@ -549,7 +549,7 @@ Shape contract — every one of these:
 5. **Re-runnable**: a second invocation produces the same final
    state (idempotent uploads, idempotent config rewrites). Used
    by the `.path` watchers that re-fire on token appearance (see
-   `docs/persistence.md::Matrix per-agent daemon`).
+   `docs/agent-lifecycle/persistence.md::Matrix per-agent daemon`).
 
 The artefact lives under the agent user's home where applicable
 (`~/.config/tea/config.yml`) and is chown'd to that user, but the

docs/process/gotchas.md

@@ -289,7 +289,7 @@ and fail outright if the host daemon's
 `nix.settings.sandbox-fallback` is `false` (nixpkgs default).
 `nix/agent-modules/default.nix` does `lib.mkForce true` so builds
 fall back to unsandboxed local builds rather than failing. Security
-implications: `docs/security.md`.
+implications: `docs/trust-boundary/security.md`.
 
 ### Linking workspace binaries locally needs `nix develop`
 
@@ -377,7 +377,7 @@ connects to the compositor at `127.0.0.1:<vnc_port>`.
 - **Port allocation**: a **fixed** port (`hyperhive.gui.vncPort`,
   default 5900). No per-agent hashing: network isolation is
   unconditional (each agent has its own netns — see
-  `docs/network.md#container-isolation`), so the VNC port is
+  `docs/networking/network.md#container-isolation`), so the VNC port is
   container-local and can't collide across agents. The harness learns
   the port from the `HIVE_GUI_VNC_PORT` env var (set on the harness
   service when `gui.enable`) — no marker file, no runtime hash. (Unlike

docs/scheduler/ci.md

@@ -102,7 +102,7 @@ build can't hold the runner's single slot indefinitely).
 
 - **Private netns, bridge-attached**: the container runs in its own network namespace (`privateNetwork = true`, `hostBridge`) and reaches hive-forge through the gateway at `http://<forge.domain>` (resolved to the bridge IP via `networking.extraHosts`). It cannot reach host-loopback services — the core dashboard at `127.0.0.1:7000` and the raw forge port are unreachable from CI. Requires `forge.behindGateway = true`.
 - **Non-ephemeral**: runner credentials persist across restarts (written to container's stateDir on first registration, reused thereafter).
-- **Sandbox fallback**: nspawn containers can't create user-namespaces, so nix's sandboxing would always fail. Module sets `nix.settings.sandbox-fallback = true` in the container — nix builds run unsandboxed (safe because the container is already isolated). See `docs/gotchas.md`.
+- **Sandbox fallback**: nspawn containers can't create user-namespaces, so nix's sandboxing would always fail. Module sets `nix.settings.sandbox-fallback = true` in the container — nix builds run unsandboxed (safe because the container is already isolated). See `docs/process/gotchas.md`.
 - **Credential isolation**: the forge admin token (`forge-core-token`) never enters the container. hive-c0re holds it and performs all forge API calls (runner validation + registration-token mint, in `forge/ci_runner.rs`); via hive-priv it writes only the runner registration token to the host env-file `/run/hive-ci/runner-token`, which the container bind-mounts read-only.
 
 ## Auto-registration flow
@@ -235,4 +235,4 @@ GC on the coordinator host won't reclaim space on the builder.
 
 - `nix/host-modules/hive-ci.nix`: runner configuration, auto-registration script, container setup.
 - `.forgejo/workflows/ci.yml`: workflow definition.
-- `docs/gotchas.md`: nix sandboxing limitations in containers.
+- `docs/process/gotchas.md`: nix sandboxing limitations in containers.

docs/scheduler/coordinator.md

@@ -1,8 +1,8 @@
 # hive-c0re coordinator internals
 
 Architecture notes for the `hive-c0re` coordinator daemon's internal subsystems.
-For the public API surface (dashboard, socket protocol) see `docs/conventions.md`
-and `docs/persistence.md`.
+For the public API surface (dashboard, socket protocol) see `docs/process/conventions.md`
+and `docs/agent-lifecycle/persistence.md`.
 
 ---
 
@@ -571,7 +571,7 @@ which is almost never desired. Leave off on non-x86 hosts.
 
 ## See also
 
-- `docs/approvals.md` — approval flow + scheduled prompts
-- `docs/persistence.md` — SQLite schema, state-dir layout
-- `docs/conventions.md` — wire protocol, recipient sentinels
-- `docs/agent-hierarchy.md` — topology and parent/child relations
+- `docs/agent-lifecycle/approvals.md` — approval flow + scheduled prompts
+- `docs/agent-lifecycle/persistence.md` — SQLite schema, state-dir layout
+- `docs/process/conventions.md` — wire protocol, recipient sentinels
+- `docs/agent-lifecycle/agent-hierarchy.md` — topology and parent/child relations

docs/scheduler/observability.md

@@ -58,7 +58,7 @@ That is a **different risk from the one the collector fixes**, and strictly
 smaller than what preceded it: before, every agent held the upstream credential
 itself, so it could do all of the above *and* use the token anywhere else. The
 collector removes the token and keeps the pipe. Agents are inside the trust
-boundary (`docs/security.md`: capability = accepted risk), so an agent being
+boundary (`docs/trust-boundary/security.md`: capability = accepted risk), so an agent being
 able to *send* is an accepted extension of that boundary — but it is not
 closed by this design, and nothing here should be read as closing it.
 
@@ -179,7 +179,7 @@ and point whatever consumes it at `10.42.0.1:5432` rather than loopback: inside
 a container, loopback is the *container*. The bridge IP is the host's address on
 the `hive-br0` bridge. The service must also bind an address the bridge can
 reach — a `127.0.0.1`-only listener stays unreachable no matter what the
-firewall allows. See `docs/network.md::Reaching host services` for details.
+firewall allows. See `docs/networking/network.md::Reaching host services` for details.
 
 ⚠️ **None of this is needed for hyperhive's own telemetry** — `otel.enable`
 contributes the collector's port and derives the agent-facing endpoint itself.

docs/trust-boundary/boundary.md

@@ -19,7 +19,7 @@ always on), the gateway proxies all operator-facing traffic, and
   reachable via the gateway.
 - **Agent** — speaks only for itself, only over its per-agent
   unix socket. The socket's identity _is_ the agent (see
-  `docs/conventions.md`, "identity = socket"). An agent must not
+  `docs/process/conventions.md`, "identity = socket"). An agent must not
   be able to reach the core daemon's HTTP surface, another
   agent's socket, or another agent's web UI.
 
@@ -46,7 +46,7 @@ defence-in-depth layered on top.
 
 Network isolation is complete and always on: every agent container
 runs in a private netns behind the hive bridge, and there is no
-shared-netns mode. See `docs/network.md`.
+shared-netns mode. See `docs/networking/network.md`.
 
 Concretely, the core daemon's dashboard `/api` carries **no
 application-layer authentication** — operator-authority routes are served
@@ -125,7 +125,7 @@ nginx reaches all of `/run/hive-agent` as a plain host path. Dropping
 `o=w` removes that permission rather than qualifying it.
 
 ⚠️ **The gateway's nginx and dnsmasq are host services, next to
-`hive-c0re`** (see `docs/gateway.md`) — there is no namespace between
+`hive-c0re`** (see `docs/networking/gateway.md`) — there is no namespace between
 them and the rest of the host. That costs no network isolation: nginx
 binds the host's `:80`/`:443` and reaches `localhost` upstreams, which a
 netns would have to be opened up for anyway.

frontend/packages/dashboard/src/core.html

@@ -51,7 +51,7 @@
          cgroup v2 on the host. Polled only while this sub-tab is active. -->
     <section class="core-pane" id="core-pane-load" data-tab-pane="load"
              role="tabpanel" aria-labelledby="core-tab-load">
-      <p class="meta">live cpu + memory per agent container, from cgroup v2 on the host. cpu is % of total host capacity (all cores), sampled over ~200ms each refresh; polled every 5s while this tab is open. network is omitted on purpose — agents share the host netns, so there is no per-container counter (see <code>docs/network.md</code>).</p>
+      <p class="meta">live cpu + memory per agent container, from cgroup v2 on the host. cpu is % of total host capacity (all cores), sampled over ~200ms each refresh; polled every 5s while this tab is open. network is omitted on purpose — agents share the host netns, so there is no per-container counter (see <code>docs/networking/network.md</code>).</p>
       <div id="container-load-section">
         <p class="meta">loading…</p>
       </div>

frontend/packages/dashboard/src/swarm.js

@@ -619,7 +619,7 @@ export function renderContainers(s) {
   // TCP fallback — the gateway proxies the prefix to the per-agent
   // harness (unix-domain via `agent-sockets.json`, or a computed TCP
   // loopback port while the socket marker is absent). See
-  // `docs/web-ui/dashboard.md::Container row` + `docs/gateway.md::Vhost map`.
+  // `docs/web-ui/dashboard.md::Container row` + `docs/networking/gateway.md::Vhost map`.
   const gatewayLinks = !!(s && s.gateway_enabled);
   // Forge public URL: state.forge_public_url (set by the NixOS module
   // from services.hyperhive.forge.publicUrl) or null — never guessed

frontend/packages/shared/src/terminal/terminal.css

@@ -14,7 +14,7 @@
    No `.term-input` here — composers are a separate concern, owned by
    each page's own CSS. Row taxonomy + layout contract documented in
    `docs/web-ui/shape.md::Shared terminal pane` and
-   `docs/terminal-rendering.md`. */
+   `docs/web-ui/terminal-rendering.md`. */
 
 .terminal-wrap {
   position: relative;
@@ -106,7 +106,7 @@
    is now four: the four severities every row already carries. Structural
    identity (this is a turn boundary, this is a tool call) is carried by
    the row's icon (`◆`, `🔧`, `✅`, …) and summary text instead of colour —
-   see docs/terminal-rendering.md. */
+   see docs/web-ui/terminal-rendering.md. */
 .live .level-debug { color: var(--muted); }
 .live .level-info  { color: var(--fg); }
 .live .level-warn  { color: var(--amber); border-left-color: var(--amber); }

nix/host-modules/swarm-otel.nix

@@ -1,7 +1,7 @@
 # The swarm's telemetry collector: one per swarm, in a `swarm-otel`
 # nixos-container beside the swarm's other shared services. This is the
 # **swarm** tier; `otel.nix` is the hive tier that forwards into it.
-# `docs/observability.md` owns the tier model and why the two stay separate
+# `docs/scheduler/observability.md` owns the tier model and why the two stay separate
 # when co-located.
 #
 # A container rather than a second host unit, for the same reason every

nix/packages/reference-docs.nix

@@ -17,6 +17,12 @@
 # `docs.nix` (website repo) needs no changes to pick either up — it
 # already walks every subdirectory generically.
 #
+# For crates that is `$out/crates/<crate>.md`, one per workspace crate
+# with a README.md (mara's ask: "behaves as if hive-core/README.md lives
+# at docs/crates/hive-core.md" — lint:allow, a path this derivation
+# creates in $out, so the repo has no such file). The crate's own README
+# stays the single source of truth.
+#
 # Output layout:
 #   $out/                   — the repo docs/ tree verbatim
 #   $out/crates/<crate>.md  — one per workspace crate README
@@ -74,7 +80,7 @@ stdenv.mkDerivation {
         # Rewrite relative links that were correct from the crate's own
         # position in the repo tree but aren't once the file is
         # virtually one level under docs/ instead:
-        #   ../docs/x.md      (crate root -> repo docs/)   -> ../x.md
+        #   ../docs/x.md      (crate root -> repo docs/)   -> ../x.md  lint:allow (stand-in name)
         #     (crates/ -> docs/ is one directory shallower than
         #     <crate>/ -> docs/ was, so the leading docs/ segment drops)
         #   ../<sibling-crate> (that crate's own directory, no file)

scripts/check-doc-refs.sh

@@ -0,0 +1,98 @@
+#!/bin/sh
+# CI lint: flags documentation pointers that no longer resolve.
+#
+# A pointer at a doc is the hive's preferred alternative to duplicating that
+# doc's prose next to the code. That trade only pays while the pointer
+# resolves, and nothing was checking. A `docs/` reorganisation left 15 distinct
+# dead paths across nix, css, html, js, markdown and .prettierignore.
+#
+# Emits a CI error annotation per hit and exits 1 if any pointer is dead, 0
+# otherwise. Runs as its own job; seconds, pure git+grep.
+#
+# Two arms, because a pointer is written two ways:
+#   root — a repo-root-relative path in prose or a comment, `docs/<dir>/<f>.md`
+#   rel  — a markdown link resolved against the linking file, [x](../<dir>/<f>.md)
+# (spelled with placeholders on purpose: a literal example path here would be
+# a dead pointer this script then reports against itself)
+#
+# Scope is every tracked file, deliberately unrestricted by type: prose
+# describes a path in words, and a stale path in a config file (an ignore
+# rule, a build input) is a live defect rather than a stale comment.
+#
+# Escape hatch: a line containing `lint:allow` is exempt. Reserve it for a
+# path that is correct while absent — a file another derivation synthesizes,
+# or a literal example — and keep the reason next to the marker.
+set -eu
+
+root_pattern='docs/[A-Za-z0-9._/-]*\.md'
+rel_pattern='\]\(\.\.*/[A-Za-z0-9._/-]*\.md'
+
+tmp="$(mktemp -d)"
+trap 'rm -rf "$tmp"' EXIT
+
+# Lines carrying the marker, as `file:line`, to skip below.
+git ls-files -z | xargs -0 -r grep -n 'lint:allow' /dev/null 2>/dev/null \
+  | cut -d: -f1,2 > "$tmp/allow" || true
+
+exempt() { grep -qxF "$1:$2" "$tmp/allow" 2>/dev/null; }
+
+git ls-files -z | xargs -0 -r grep -nEo "$root_pattern" /dev/null 2>/dev/null \
+  > "$tmp/root" || true
+git ls-files -z | xargs -0 -r grep -nEo "$rel_pattern" /dev/null 2>/dev/null \
+  > "$tmp/rel" || true
+
+dead=0
+alive=0
+
+while IFS=: read -r file lineno ref; do
+  [ -n "${ref:-}" ] || continue
+  exempt "$file" "$lineno" && continue
+  if [ -f "$ref" ]; then
+    alive=$((alive + 1))
+  else
+    dead=$((dead + 1))
+    printf '::error file=%s,line=%s::doc pointer does not resolve: %s\n' \
+      "$file" "$lineno" "$ref"
+  fi
+done < "$tmp/root"
+
+while IFS=: read -r file lineno match; do
+  [ -n "${match:-}" ] || continue
+  exempt "$file" "$lineno" && continue
+  link="${match#](}"
+  # Resolved from the linking file's directory, so `../` counts the same way
+  # the renderer counts it. `-m` keeps a link that escapes the repo root
+  # resolvable enough to report rather than erroring out here.
+  target="$(cd "$(dirname "$file")" && realpath -m "$link")"
+  if [ -f "$target" ]; then
+    alive=$((alive + 1))
+  else
+    dead=$((dead + 1))
+    printf '::error file=%s,line=%s::relative doc link does not resolve: %s\n' \
+      "$file" "$lineno" "$link"
+  fi
+done < "$tmp/rel"
+
+# A lint whose extractor matches nothing passes forever and reports success
+# while checking not one thing. Both arms therefore have to have found
+# something that resolves before a clean run means anything: if a rename, a
+# pattern edit or a tree move silences the search, that is a failure of this
+# script, not a property of the tree.
+if [ "$(wc -l < "$tmp/root")" -eq 0 ]; then
+  echo 'check-doc-refs: root-pointer search matched nothing — pattern or tree changed' >&2
+  exit 1
+fi
+if [ "$(wc -l < "$tmp/rel")" -eq 0 ]; then
+  echo 'check-doc-refs: relative-link search matched nothing — pattern or tree changed' >&2
+  exit 1
+fi
+if [ "$alive" -eq 0 ]; then
+  echo 'check-doc-refs: not one pointer resolved — the resolver is broken, not the tree' >&2
+  exit 1
+fi
+
+if [ "$dead" -gt 0 ]; then
+  printf 'check-doc-refs: %s dead doc pointer(s); %s resolved\n' "$dead" "$alive" >&2
+  exit 1
+fi
+exit 0

scripts/pre-push

@@ -1,6 +1,6 @@
 #!/bin/sh
-# Git pre-push hook: runs the tracker-tag lint and the comment-block
-# lint against the working tree before any push lands on the remote.
+# Git pre-push hook: runs the tracker-tag, comment-block and doc-pointer
+# lints against the working tree before any push lands on the remote.
 # Catches issues that would fail CI and require a follow-up commit
 # (common failure mode: a nix comment containing a hash-issue-number
 # tag added mid-session).
@@ -28,5 +28,11 @@ if ! sh "$repo_root/scripts/check-comment-blocks.sh"; then
   exit 1
 fi
 
+echo "pre-push: running doc-pointer lint..." >&2
+if ! sh "$repo_root/scripts/check-doc-refs.sh"; then
+  echo "pre-push: doc-pointer lint FAILED — fix before pushing" >&2
+  exit 1
+fi
+
 echo "pre-push: lints passed ✓" >&2
 exit 0