ci: gate documentation pointers so a dead one fails the build

Pointing at a doc instead of duplicating its prose next to the code is what
the comment-block lint pushes people toward, and nothing checked that the
pointers still resolve. The reorg that prompted the preceding commit broke
41 references and every one of them merged green.

Two arms: repo-root-relative `docs/` paths, and relative markdown links
resolved against the linking file. Scope is every tracked file rather than a
type list — the dead references were in css, html and an ignore file as well
as in markdown and nix.

Three controls, because a link checker whose extractor quietly matches
nothing passes forever while checking nothing: each arm must have found
candidates, and at least one pointer must have resolved. Any of those failing
exits non-zero and says the script is broken rather than the tree.

Escape hatch is the `lint:allow` marker the tracker-tag lint already uses.
Its only current users are in reference-docs.nix, where a path naming a file
the derivation synthesizes into $out is correct precisely because the repo
does not have it.

Also runs in the pre-push hook alongside the other two.

.forgejo/workflows/ci.yml

@@ -53,3 +53,17 @@ jobs:
         # protection) — a hit blocks merge. See
         # scripts/check-comment-blocks.sh.
         run: sh scripts/check-comment-blocks.sh
+
+  doc-refs:
+    name: doc-pointer lint
+    runs-on: [hive-ci]
+    # Pure git+grep — seconds.
+    timeout-minutes: 5
+    steps:
+      - uses: actions/checkout@v3
+      - name: lint
+        # Flags `docs/…md` paths and relative markdown links that no
+        # longer resolve. Pointing at a doc is what the comment-block
+        # lint above pushes people toward, so the pointers need a gate of
+        # their own. See scripts/check-doc-refs.sh.
+        run: sh scripts/check-doc-refs.sh

nix/packages/reference-docs.nix

@@ -17,6 +17,12 @@
 # `docs.nix` (website repo) needs no changes to pick either up — it
 # already walks every subdirectory generically.
 #
+# For crates that is `$out/crates/<crate>.md`, one per workspace crate
+# with a README.md (mara's ask: "behaves as if hive-core/README.md lives
+# at docs/crates/hive-core.md" — lint:allow, a path this derivation
+# creates in $out, so the repo has no such file). The crate's own README
+# stays the single source of truth.
+#
 # Output layout:
 #   $out/                   — the repo docs/ tree verbatim
 #   $out/crates/<crate>.md  — one per workspace crate README
@@ -74,7 +80,7 @@ stdenv.mkDerivation {
         # Rewrite relative links that were correct from the crate's own
         # position in the repo tree but aren't once the file is
         # virtually one level under docs/ instead:
-        #   ../docs/x.md      (crate root -> repo docs/)   -> ../x.md
+        #   ../docs/x.md      (crate root -> repo docs/)   -> ../x.md  lint:allow (stand-in name)
         #     (crates/ -> docs/ is one directory shallower than
         #     <crate>/ -> docs/ was, so the leading docs/ segment drops)
         #   ../<sibling-crate> (that crate's own directory, no file)

scripts/check-doc-refs.sh

@@ -0,0 +1,98 @@
+#!/bin/sh
+# CI lint: flags documentation pointers that no longer resolve.
+#
+# A pointer at a doc is the hive's preferred alternative to duplicating that
+# doc's prose next to the code. That trade only pays while the pointer
+# resolves, and nothing was checking. A `docs/` reorganisation left 15 distinct
+# dead paths across nix, css, html, js, markdown and .prettierignore.
+#
+# Emits a CI error annotation per hit and exits 1 if any pointer is dead, 0
+# otherwise. Runs as its own job; seconds, pure git+grep.
+#
+# Two arms, because a pointer is written two ways:
+#   root — a repo-root-relative path in prose or a comment, `docs/<dir>/<f>.md`
+#   rel  — a markdown link resolved against the linking file, [x](../<dir>/<f>.md)
+# (spelled with placeholders on purpose: a literal example path here would be
+# a dead pointer this script then reports against itself)
+#
+# Scope is every tracked file, deliberately unrestricted by type: prose
+# describes a path in words, and a stale path in a config file (an ignore
+# rule, a build input) is a live defect rather than a stale comment.
+#
+# Escape hatch: a line containing `lint:allow` is exempt. Reserve it for a
+# path that is correct while absent — a file another derivation synthesizes,
+# or a literal example — and keep the reason next to the marker.
+set -eu
+
+root_pattern='docs/[A-Za-z0-9._/-]*\.md'
+rel_pattern='\]\(\.\.*/[A-Za-z0-9._/-]*\.md'
+
+tmp="$(mktemp -d)"
+trap 'rm -rf "$tmp"' EXIT
+
+# Lines carrying the marker, as `file:line`, to skip below.
+git ls-files -z | xargs -0 -r grep -n 'lint:allow' /dev/null 2>/dev/null \
+  | cut -d: -f1,2 > "$tmp/allow" || true
+
+exempt() { grep -qxF "$1:$2" "$tmp/allow" 2>/dev/null; }
+
+git ls-files -z | xargs -0 -r grep -nEo "$root_pattern" /dev/null 2>/dev/null \
+  > "$tmp/root" || true
+git ls-files -z | xargs -0 -r grep -nEo "$rel_pattern" /dev/null 2>/dev/null \
+  > "$tmp/rel" || true
+
+dead=0
+alive=0
+
+while IFS=: read -r file lineno ref; do
+  [ -n "${ref:-}" ] || continue
+  exempt "$file" "$lineno" && continue
+  if [ -f "$ref" ]; then
+    alive=$((alive + 1))
+  else
+    dead=$((dead + 1))
+    printf '::error file=%s,line=%s::doc pointer does not resolve: %s\n' \
+      "$file" "$lineno" "$ref"
+  fi
+done < "$tmp/root"
+
+while IFS=: read -r file lineno match; do
+  [ -n "${match:-}" ] || continue
+  exempt "$file" "$lineno" && continue
+  link="${match#](}"
+  # Resolved from the linking file's directory, so `../` counts the same way
+  # the renderer counts it. `-m` keeps a link that escapes the repo root
+  # resolvable enough to report rather than erroring out here.
+  target="$(cd "$(dirname "$file")" && realpath -m "$link")"
+  if [ -f "$target" ]; then
+    alive=$((alive + 1))
+  else
+    dead=$((dead + 1))
+    printf '::error file=%s,line=%s::relative doc link does not resolve: %s\n' \
+      "$file" "$lineno" "$link"
+  fi
+done < "$tmp/rel"
+
+# A lint whose extractor matches nothing passes forever and reports success
+# while checking not one thing. Both arms therefore have to have found
+# something that resolves before a clean run means anything: if a rename, a
+# pattern edit or a tree move silences the search, that is a failure of this
+# script, not a property of the tree.
+if [ "$(wc -l < "$tmp/root")" -eq 0 ]; then
+  echo 'check-doc-refs: root-pointer search matched nothing — pattern or tree changed' >&2
+  exit 1
+fi
+if [ "$(wc -l < "$tmp/rel")" -eq 0 ]; then
+  echo 'check-doc-refs: relative-link search matched nothing — pattern or tree changed' >&2
+  exit 1
+fi
+if [ "$alive" -eq 0 ]; then
+  echo 'check-doc-refs: not one pointer resolved — the resolver is broken, not the tree' >&2
+  exit 1
+fi
+
+if [ "$dead" -gt 0 ]; then
+  printf 'check-doc-refs: %s dead doc pointer(s); %s resolved\n' "$dead" "$alive" >&2
+  exit 1
+fi
+exit 0

scripts/pre-push

@@ -1,6 +1,6 @@
 #!/bin/sh
-# Git pre-push hook: runs the tracker-tag lint and the comment-block
-# lint against the working tree before any push lands on the remote.
+# Git pre-push hook: runs the tracker-tag, comment-block and doc-pointer
+# lints against the working tree before any push lands on the remote.
 # Catches issues that would fail CI and require a follow-up commit
 # (common failure mode: a nix comment containing a hash-issue-number
 # tag added mid-session).
@@ -28,5 +28,11 @@ if ! sh "$repo_root/scripts/check-comment-blocks.sh"; then
   exit 1
 fi
 
+echo "pre-push: running doc-pointer lint..." >&2
+if ! sh "$repo_root/scripts/check-doc-refs.sh"; then
+  echo "pre-push: doc-pointer lint FAILED — fix before pushing" >&2
+  exit 1
+fi
+
 echo "pre-push: lints passed ✓" >&2
 exit 0