Watch
0
0
Fork
You've already forked hyperhive
0

Compare commits

..
19 changed files with 39 additions and 163 deletions

View file

@ -53,17 +53,3 @@ jobs:
# protection) — a hit blocks merge. See
# scripts/check-comment-blocks.sh.
run: sh scripts/check-comment-blocks.sh
doc-refs:
name: doc-pointer lint
runs-on: [hive-ci]
# Pure git+grep — seconds.
timeout-minutes: 5
steps:
- uses: actions/checkout@v3
- name: lint
# Flags `docs/…md` paths and relative markdown links that no
# longer resolve. Pointing at a doc is what the comment-block
# lint above pushes people toward, so the pointers need a gate of
# their own. See scripts/check-doc-refs.sh.
run: sh scripts/check-doc-refs.sh

View file

@ -8,12 +8,12 @@ docs/tools/hivectl-cli.md
# indentation normalisation, so these files need manual re-wrap before
# adding them back to the formatter scope.
CLAUDE.md
docs/agent-lifecycle/approvals.md
docs/process/conventions.md
docs/networking/gateway.md
docs/integrations/matrix.md
docs/agent-lifecycle/persistence.md
docs/web-ui/terminal-rendering.md
docs/approvals.md
docs/conventions.md
docs/gateway.md
docs/matrix.md
docs/persistence.md
docs/terminal-rendering.md
docs/tools/forge.md
docs/tools/matrix.md
docs/turn-loop/README.md

View file

@ -209,7 +209,7 @@ One-shot rows fire once (if past due, on the next worker pass) and are deleted b
### Scheduled prompt delivery: todo, not a broker message
An agent target's delivery is `push_todo` (`Coordinator::push_todo`,
`docs/scheduler/coordinator.md` covers the mechanism generally), not a broker
`docs/coordinator.md` covers the mechanism generally), not a broker
`Message` — a scheduled prompt wakes its target with a todo instead of
driving an immediate turn, by design. `key = "schedule:<id>"` per
target drives `push_todo`'s own upsert-by-key dedup: a re-fire of the
@ -419,7 +419,7 @@ submitter pushes again (or closes it) to retry.
Long-running approval work — `MergeConfigPr`, `UpdateMetaInputs`,
`Spawn` — runs as a DAG on the global job queue
(`docs/scheduler/coordinator.md::Job queue`), submitted by the approval handler
(`docs/coordinator.md::Job queue`), submitted by the approval handler
rather than run inline:
| `ApprovalKind` | DAG submitted | source |

View file

@ -466,7 +466,7 @@ subvolume creation/deletion above.
This is the same subvolume `hivectl agent <name> subvol snapshot push`
sends to the swarm's snapshot store — see
[`docs/networking/snapshot-store.md`](../networking/snapshot-store.md) for what a pushed
[`docs/snapshot-store.md`](../networking/snapshot-store.md) for what a pushed
snapshot contains and how the store authenticates a sender.
## `/var/lib/swarm-controller/` (swarm-controller host only)

View file

@ -184,7 +184,7 @@ host-side service on another port — e.g. a dev OTLP collector you want
agents to reach directly — is unreachable. (hyperhive's own telemetry
needs none of this: `otel.enable` opens its collector's port itself, and
`otel.endpoint` is the *upstream*, which no agent ever dials. See
`docs/scheduler/observability.md`.)
`docs/observability.md`.)
`services.hyperhive.network.exposeHostPorts = [ 4318 ];` opens each
listed TCP port `P` on the bridge-interface `allowedTCPPorts`, so an
@ -274,4 +274,4 @@ loopback — the hive-c0re admin socket is a UDS, not TCP.
## Cross-references
- `docs/networking/gateway.md` — vhost map + the gateway's other duties
- `docs/gateway.md` — vhost map + the gateway's other duties

View file

@ -184,7 +184,7 @@ the access-review list.
The snapshot covers an agent's **state subvolume**, which is the parent
of `state/`, `claude/` and `harness/` (see
[`docs/agent-lifecycle/persistence.md`'s btrfs subvolume
[`docs/persistence.md`'s btrfs subvolume
section](../agent-lifecycle/persistence.md#btrfs-subvolumes-for-varlibhyperhiveagentsname)
for how and when that subvolume is created). Consequences:

View file

@ -195,7 +195,7 @@ Per-variant fields:
— `agent` is the affected agent (target of the spawn / config
commit), not the asker. `description` is the manager's free-text
blurb shown on the dashboard card. `commit_ref` is the
kind-specific payload (see `docs/agent-lifecycle/approvals.md::Approval kinds
kind-specific payload (see `docs/approvals.md::Approval kinds
(wire shapes)`).
- `Reminder { id, owner, message, due_at, age_seconds }` —
`due_at` is the absolute time the scheduler is targeting (RFC
@ -424,7 +424,7 @@ start runs across the `StopForUpdate → Swap → RebuildBookkeeping`
brace and the tail `Reconcile` node. `flake.nix` itself is no longer
regenerated host-side on rebuild — it's tracked in the agent's
proposed/applied repos and rides along on every fetch (see
`docs/agent-lifecycle/approvals.md::Two repos per agent`).
`docs/approvals.md::Two repos per agent`).
Anything that changes per-container state on the host should be
re-applied here so a manual `↻ R3BU1LD` from the dashboard is
@ -528,7 +528,7 @@ the state dir at provisioning time:
(The matrix profile avatar is **not** a oneshot — `hive-matrix-daemon`
sets it over its live authenticated Client; see
`docs/agent-lifecycle/persistence.md::matrix avatar`.)
`docs/persistence.md::matrix avatar`.)
Shape contract — every one of these:
@ -549,7 +549,7 @@ Shape contract — every one of these:
5. **Re-runnable**: a second invocation produces the same final
state (idempotent uploads, idempotent config rewrites). Used
by the `.path` watchers that re-fire on token appearance (see
`docs/agent-lifecycle/persistence.md::Matrix per-agent daemon`).
`docs/persistence.md::Matrix per-agent daemon`).
The artefact lives under the agent user's home where applicable
(`~/.config/tea/config.yml`) and is chown'd to that user, but the

View file

@ -289,7 +289,7 @@ and fail outright if the host daemon's
`nix.settings.sandbox-fallback` is `false` (nixpkgs default).
`nix/agent-modules/default.nix` does `lib.mkForce true` so builds
fall back to unsandboxed local builds rather than failing. Security
implications: `docs/trust-boundary/security.md`.
implications: `docs/security.md`.
### Linking workspace binaries locally needs `nix develop`
@ -377,7 +377,7 @@ connects to the compositor at `127.0.0.1:<vnc_port>`.
- **Port allocation**: a **fixed** port (`hyperhive.gui.vncPort`,
default 5900). No per-agent hashing: network isolation is
unconditional (each agent has its own netns — see
`docs/networking/network.md#container-isolation`), so the VNC port is
`docs/network.md#container-isolation`), so the VNC port is
container-local and can't collide across agents. The harness learns
the port from the `HIVE_GUI_VNC_PORT` env var (set on the harness
service when `gui.enable`) — no marker file, no runtime hash. (Unlike

View file

@ -102,7 +102,7 @@ build can't hold the runner's single slot indefinitely).
- **Private netns, bridge-attached**: the container runs in its own network namespace (`privateNetwork = true`, `hostBridge`) and reaches hive-forge through the gateway at `http://<forge.domain>` (resolved to the bridge IP via `networking.extraHosts`). It cannot reach host-loopback services — the core dashboard at `127.0.0.1:7000` and the raw forge port are unreachable from CI. Requires `forge.behindGateway = true`.
- **Non-ephemeral**: runner credentials persist across restarts (written to container's stateDir on first registration, reused thereafter).
- **Sandbox fallback**: nspawn containers can't create user-namespaces, so nix's sandboxing would always fail. Module sets `nix.settings.sandbox-fallback = true` in the container — nix builds run unsandboxed (safe because the container is already isolated). See `docs/process/gotchas.md`.
- **Sandbox fallback**: nspawn containers can't create user-namespaces, so nix's sandboxing would always fail. Module sets `nix.settings.sandbox-fallback = true` in the container — nix builds run unsandboxed (safe because the container is already isolated). See `docs/gotchas.md`.
- **Credential isolation**: the forge admin token (`forge-core-token`) never enters the container. hive-c0re holds it and performs all forge API calls (runner validation + registration-token mint, in `forge/ci_runner.rs`); via hive-priv it writes only the runner registration token to the host env-file `/run/hive-ci/runner-token`, which the container bind-mounts read-only.
## Auto-registration flow
@ -235,4 +235,4 @@ GC on the coordinator host won't reclaim space on the builder.
- `nix/host-modules/hive-ci.nix`: runner configuration, auto-registration script, container setup.
- `.forgejo/workflows/ci.yml`: workflow definition.
- `docs/process/gotchas.md`: nix sandboxing limitations in containers.
- `docs/gotchas.md`: nix sandboxing limitations in containers.

View file

@ -1,8 +1,8 @@
# hive-c0re coordinator internals
Architecture notes for the `hive-c0re` coordinator daemon's internal subsystems.
For the public API surface (dashboard, socket protocol) see `docs/process/conventions.md`
and `docs/agent-lifecycle/persistence.md`.
For the public API surface (dashboard, socket protocol) see `docs/conventions.md`
and `docs/persistence.md`.
---
@ -571,7 +571,7 @@ which is almost never desired. Leave off on non-x86 hosts.
## See also
- `docs/agent-lifecycle/approvals.md` — approval flow + scheduled prompts
- `docs/agent-lifecycle/persistence.md` — SQLite schema, state-dir layout
- `docs/process/conventions.md` — wire protocol, recipient sentinels
- `docs/agent-lifecycle/agent-hierarchy.md` — topology and parent/child relations
- `docs/approvals.md` — approval flow + scheduled prompts
- `docs/persistence.md` — SQLite schema, state-dir layout
- `docs/conventions.md` — wire protocol, recipient sentinels
- `docs/agent-hierarchy.md` — topology and parent/child relations

View file

@ -58,7 +58,7 @@ That is a **different risk from the one the collector fixes**, and strictly
smaller than what preceded it: before, every agent held the upstream credential
itself, so it could do all of the above *and* use the token anywhere else. The
collector removes the token and keeps the pipe. Agents are inside the trust
boundary (`docs/trust-boundary/security.md`: capability = accepted risk), so an agent being
boundary (`docs/security.md`: capability = accepted risk), so an agent being
able to *send* is an accepted extension of that boundary — but it is not
closed by this design, and nothing here should be read as closing it.
@ -179,7 +179,7 @@ and point whatever consumes it at `10.42.0.1:5432` rather than loopback: inside
a container, loopback is the *container*. The bridge IP is the host's address on
the `hive-br0` bridge. The service must also bind an address the bridge can
reach — a `127.0.0.1`-only listener stays unreachable no matter what the
firewall allows. See `docs/networking/network.md::Reaching host services` for details.
firewall allows. See `docs/network.md::Reaching host services` for details.
⚠️ **None of this is needed for hyperhive's own telemetry** — `otel.enable`
contributes the collector's port and derives the agent-facing endpoint itself.

View file

@ -19,7 +19,7 @@ always on), the gateway proxies all operator-facing traffic, and
reachable via the gateway.
- **Agent** — speaks only for itself, only over its per-agent
unix socket. The socket's identity _is_ the agent (see
`docs/process/conventions.md`, "identity = socket"). An agent must not
`docs/conventions.md`, "identity = socket"). An agent must not
be able to reach the core daemon's HTTP surface, another
agent's socket, or another agent's web UI.
@ -46,7 +46,7 @@ defence-in-depth layered on top.
Network isolation is complete and always on: every agent container
runs in a private netns behind the hive bridge, and there is no
shared-netns mode. See `docs/networking/network.md`.
shared-netns mode. See `docs/network.md`.
Concretely, the core daemon's dashboard `/api` carries **no
application-layer authentication** — operator-authority routes are served
@ -125,7 +125,7 @@ nginx reaches all of `/run/hive-agent` as a plain host path. Dropping
`o=w` removes that permission rather than qualifying it.
⚠️ **The gateway's nginx and dnsmasq are host services, next to
`hive-c0re`** (see `docs/networking/gateway.md`) — there is no namespace between
`hive-c0re`** (see `docs/gateway.md`) — there is no namespace between
them and the rest of the host. That costs no network isolation: nginx
binds the host's `:80`/`:443` and reaches `localhost` upstreams, which a
netns would have to be opened up for anyway.

View file

@ -51,7 +51,7 @@
cgroup v2 on the host. Polled only while this sub-tab is active. -->
<section class="core-pane" id="core-pane-load" data-tab-pane="load"
role="tabpanel" aria-labelledby="core-tab-load">
<p class="meta">live cpu + memory per agent container, from cgroup v2 on the host. cpu is % of total host capacity (all cores), sampled over ~200ms each refresh; polled every 5s while this tab is open. network is omitted on purpose — agents share the host netns, so there is no per-container counter (see <code>docs/networking/network.md</code>).</p>
<p class="meta">live cpu + memory per agent container, from cgroup v2 on the host. cpu is % of total host capacity (all cores), sampled over ~200ms each refresh; polled every 5s while this tab is open. network is omitted on purpose — agents share the host netns, so there is no per-container counter (see <code>docs/network.md</code>).</p>
<div id="container-load-section">
<p class="meta">loading…</p>
</div>

View file

@ -619,7 +619,7 @@ export function renderContainers(s) {
// TCP fallback — the gateway proxies the prefix to the per-agent
// harness (unix-domain via `agent-sockets.json`, or a computed TCP
// loopback port while the socket marker is absent). See
// `docs/web-ui/dashboard.md::Container row` + `docs/networking/gateway.md::Vhost map`.
// `docs/web-ui/dashboard.md::Container row` + `docs/gateway.md::Vhost map`.
const gatewayLinks = !!(s && s.gateway_enabled);
// Forge public URL: state.forge_public_url (set by the NixOS module
// from services.hyperhive.forge.publicUrl) or null — never guessed

View file

@ -14,7 +14,7 @@
No `.term-input` here — composers are a separate concern, owned by
each page's own CSS. Row taxonomy + layout contract documented in
`docs/web-ui/shape.md::Shared terminal pane` and
`docs/web-ui/terminal-rendering.md`. */
`docs/terminal-rendering.md`. */
.terminal-wrap {
position: relative;
@ -106,7 +106,7 @@
is now four: the four severities every row already carries. Structural
identity (this is a turn boundary, this is a tool call) is carried by
the row's icon (`◆`, `🔧`, `✅`, …) and summary text instead of colour —
see docs/web-ui/terminal-rendering.md. */
see docs/terminal-rendering.md. */
.live .level-debug { color: var(--muted); }
.live .level-info { color: var(--fg); }
.live .level-warn { color: var(--amber); border-left-color: var(--amber); }

View file

@ -1,7 +1,7 @@
# The swarm's telemetry collector: one per swarm, in a `swarm-otel`
# nixos-container beside the swarm's other shared services. This is the
# **swarm** tier; `otel.nix` is the hive tier that forwards into it.
# `docs/scheduler/observability.md` owns the tier model and why the two stay separate
# `docs/observability.md` owns the tier model and why the two stay separate
# when co-located.
#
# A container rather than a second host unit, for the same reason every

View file

@ -17,12 +17,6 @@
# `docs.nix` (website repo) needs no changes to pick either up — it
# already walks every subdirectory generically.
#
# For crates that is `$out/crates/<crate>.md`, one per workspace crate
# with a README.md (mara's ask: "behaves as if hive-core/README.md lives
# at docs/crates/hive-core.md" — lint:allow, a path this derivation
# creates in $out, so the repo has no such file). The crate's own README
# stays the single source of truth.
#
# Output layout:
# $out/ — the repo docs/ tree verbatim
# $out/crates/<crate>.md — one per workspace crate README
@ -80,7 +74,7 @@ stdenv.mkDerivation {
# Rewrite relative links that were correct from the crate's own
# position in the repo tree but aren't once the file is
# virtually one level under docs/ instead:
# ../docs/x.md (crate root -> repo docs/) -> ../x.md lint:allow (stand-in name)
# ../docs/x.md (crate root -> repo docs/) -> ../x.md
# (crates/ -> docs/ is one directory shallower than
# <crate>/ -> docs/ was, so the leading docs/ segment drops)
# ../<sibling-crate> (that crate's own directory, no file)

View file

@ -1,98 +0,0 @@
#!/bin/sh
# CI lint: flags documentation pointers that no longer resolve.
#
# A pointer at a doc is the hive's preferred alternative to duplicating that
# doc's prose next to the code. That trade only pays while the pointer
# resolves, and nothing was checking. A `docs/` reorganisation left 15 distinct
# dead paths across nix, css, html, js, markdown and .prettierignore.
#
# Emits a CI error annotation per hit and exits 1 if any pointer is dead, 0
# otherwise. Runs as its own job; seconds, pure git+grep.
#
# Two arms, because a pointer is written two ways:
# root — a repo-root-relative path in prose or a comment, `docs/<dir>/<f>.md`
# rel — a markdown link resolved against the linking file, [x](../<dir>/<f>.md)
# (spelled with placeholders on purpose: a literal example path here would be
# a dead pointer this script then reports against itself)
#
# Scope is every tracked file, deliberately unrestricted by type: prose
# describes a path in words, and a stale path in a config file (an ignore
# rule, a build input) is a live defect rather than a stale comment.
#
# Escape hatch: a line containing `lint:allow` is exempt. Reserve it for a
# path that is correct while absent — a file another derivation synthesizes,
# or a literal example — and keep the reason next to the marker.
set -eu
root_pattern='docs/[A-Za-z0-9._/-]*\.md'
rel_pattern='\]\(\.\.*/[A-Za-z0-9._/-]*\.md'
tmp="$(mktemp -d)"
trap 'rm -rf "$tmp"' EXIT
# Lines carrying the marker, as `file:line`, to skip below.
git ls-files -z | xargs -0 -r grep -n 'lint:allow' /dev/null 2>/dev/null \
| cut -d: -f1,2 > "$tmp/allow" || true
exempt() { grep -qxF "$1:$2" "$tmp/allow" 2>/dev/null; }
git ls-files -z | xargs -0 -r grep -nEo "$root_pattern" /dev/null 2>/dev/null \
> "$tmp/root" || true
git ls-files -z | xargs -0 -r grep -nEo "$rel_pattern" /dev/null 2>/dev/null \
> "$tmp/rel" || true
dead=0
alive=0
while IFS=: read -r file lineno ref; do
[ -n "${ref:-}" ] || continue
exempt "$file" "$lineno" && continue
if [ -f "$ref" ]; then
alive=$((alive + 1))
else
dead=$((dead + 1))
printf '::error file=%s,line=%s::doc pointer does not resolve: %s\n' \
"$file" "$lineno" "$ref"
fi
done < "$tmp/root"
while IFS=: read -r file lineno match; do
[ -n "${match:-}" ] || continue
exempt "$file" "$lineno" && continue
link="${match#](}"
# Resolved from the linking file's directory, so `../` counts the same way
# the renderer counts it. `-m` keeps a link that escapes the repo root
# resolvable enough to report rather than erroring out here.
target="$(cd "$(dirname "$file")" && realpath -m "$link")"
if [ -f "$target" ]; then
alive=$((alive + 1))
else
dead=$((dead + 1))
printf '::error file=%s,line=%s::relative doc link does not resolve: %s\n' \
"$file" "$lineno" "$link"
fi
done < "$tmp/rel"
# A lint whose extractor matches nothing passes forever and reports success
# while checking not one thing. Both arms therefore have to have found
# something that resolves before a clean run means anything: if a rename, a
# pattern edit or a tree move silences the search, that is a failure of this
# script, not a property of the tree.
if [ "$(wc -l < "$tmp/root")" -eq 0 ]; then
echo 'check-doc-refs: root-pointer search matched nothing — pattern or tree changed' >&2
exit 1
fi
if [ "$(wc -l < "$tmp/rel")" -eq 0 ]; then
echo 'check-doc-refs: relative-link search matched nothing — pattern or tree changed' >&2
exit 1
fi
if [ "$alive" -eq 0 ]; then
echo 'check-doc-refs: not one pointer resolved — the resolver is broken, not the tree' >&2
exit 1
fi
if [ "$dead" -gt 0 ]; then
printf 'check-doc-refs: %s dead doc pointer(s); %s resolved\n' "$dead" "$alive" >&2
exit 1
fi
exit 0

View file

@ -1,6 +1,6 @@
#!/bin/sh
# Git pre-push hook: runs the tracker-tag, comment-block and doc-pointer
# lints against the working tree before any push lands on the remote.
# Git pre-push hook: runs the tracker-tag lint and the comment-block
# lint against the working tree before any push lands on the remote.
# Catches issues that would fail CI and require a follow-up commit
# (common failure mode: a nix comment containing a hash-issue-number
# tag added mid-session).
@ -28,11 +28,5 @@ if ! sh "$repo_root/scripts/check-comment-blocks.sh"; then
exit 1
fi
echo "pre-push: running doc-pointer lint..." >&2
if ! sh "$repo_root/scripts/check-doc-refs.sh"; then
echo "pre-push: doc-pointer lint FAILED — fix before pushing" >&2
exit 1
fi
echo "pre-push: lints passed ✓" >&2
exit 0