Per mara's go-ahead on hyperhive#3902 ("getting started is good, but
terminal rendering does not go in there i think"):
Moved 21 top-level docs/*.md files into 7 new topic subdirectories
(existing web-ui/, turn-loop/, swarm/, tools/, crates/ untouched):
getting-started/ setup.md
agent-lifecycle/ agent-hierarchy.md, approvals.md, persistence.md
trust-boundary/ boundary.md, security.md
integrations/ forge.md, matrix.md, github.md, knowledge.md
networking/ gateway.md, network.md, snapshot-store.md
scheduler/ jobq.md, coordinator.md, ci.md, observability.md
process/ conventions.md, gotchas.md, pr-review-gate.md
web-ui/ terminal-rendering.md (moved into the EXISTING dir,
per mara's correction to the original getting-started
guess -- it's UI implementation detail, not onboarding)
The physical layout now matches docs/README.md's own topical headers,
which already amounted to this taxonomy -- see the scoping comment on
the issue for the two findings that motivated this (a genuine
duplication between CLAUDE.md's old "Reading paths" list and
docs/README.md's grouped one, since drifted out of sync with each
other; and the flat layout not matching the grouping we already had).
Fixed every cross-reference this moved across the whole repo (~120
files: docs/ internal links at every depth, Rust doc comments, nix
module option docs, crate READMEs) -- verified two ways: a grep sweep
confirming zero remaining references to any old path, and a script
that resolves every markdown link in docs/**/*.md + CLAUDE.md +
README.md against the filesystem and reports anything that doesn't
exist (zero broken links).
Collapsed CLAUDE.md's "Reading paths" section (the duplicate) down to
a pointer at docs/README.md, now the single index. Rewrote
docs/README.md itself to use the new subdirectory paths and added the
one doc it was missing that CLAUDE.md's old copy had (pr-review-gate.md).
Classified all 22 docs/*.md files first via a haiku subagent (mara's
suggestion) on two axes -- proposed grouping and operator-vs-
implementation focus -- before finalizing the taxonomy; spot-checked
the report and found internal inconsistencies (its classification
table disagreed with its own summary section for a few files), so this
taxonomy is my original proposal + the one correction mara gave
directly, not a blind application of the subagent's table. The
operator-focus data it gathered is still useful for a follow-up
content pass (docs skewing 'mixed' rather than pure operator-facing),
not addressed in this PR -- structure only.
nix fmt clean, both pre-push lints clean.
Nine of the 38 .nix tokens mentioned anywhere in *.rs did not name
anything that exists. Twelve mentions, five distinct targets:
hive-c0re.nix, hive-gateway.nix and hive-forge.nix are all directories
now; nix/modules/ is not a directory we have; hive-forge-tools.nix was
a bash script the binary replaced and is gone.
Where the reference is load-bearing it is corrected rather than
deleted, because the reference is the point: a comment saying a
constant must match a nix literal is only useful if you can open the
file it names. Where the module member was unambiguous the path now
names it exactly.
paths.rs's STATE_ROOT marker was the worst of them: it claimed the
value came from services.hyperhive.c0re.statePath, in hive-c0re.nix.
Neither exists. The option is not declared anywhere and the file is a
directory, so a "must match" contract pointed at two things that
cannot be opened. /var/lib/hyperhive is hardcoded on both sides, which
is what the comment now says.
hive-forge-tools.nix keeps no replacement: naming a file that was
deliberately deleted helps nobody, and "replaces a prior bash script"
is complete without it.
Measured before and after with the same command: 9 unresolved of 38
before, 4 of 35 after. The remaining four are an example path in a doc
comment, an upstream nixpkgs path, and two from one synthetic test
fixture.
A webhook has exactly one target URL, so every hive registering one
against the shared internal/knowledge repository was last-writer-wins
rather than idempotent: all but the most recent silently stopped
receiving deliveries. The swarm controller holds the single registration
and now addresses an event to each hive over the queue instead.
This is a migration, not a deletion. Not registering any more fixes
nothing on a hive that has already run — the hook it created persists on
the forge, so the contention would survive on exactly the deployments
that have it while fresh installs looked fixed. The hive that created a
hook removes it.
It removes only its OWN, matched on the full URL rather than the
/webhook/knowledge suffix. A hook with that suffix and a different base
belongs to another hive, possibly one not yet upgraded, and deleting it
would break that hive's knowledge sync until it caught up. Reaping a
neighbour's registration is the behaviour being removed here; doing it
while fixing it would only invert the direction.
The predecessor did reap by suffix, to clear loopback hooks left by an
older single-hive layout. That was safe when a hive was alone on its
forge and is not safe now. The hive-side registrars also acted as reapers
of hooks under their own path, which is why the swarm hook lives under
/webhook/forge/; removing this registrar removes that reaper too.
Intended, and stated because no reviewer would infer it from the diff.
The receive endpoint goes with it. A live HMAC-verified
/webhook/knowledge that nothing can legitimately reach would tell the
next reader that this is how a hive learns about knowledge changes.
Docs move in the same commit: docs/swarm/README.md said two hooks exist
per swarm-wide repo and neither should be deleted, which is now true for
agent-configs and wrong for internal/knowledge — a half-correct
description being worse than an uncorrected one.
A consumer that wants "how much is in flight" — a summary line, a badge,
a health check — had to fetch the whole graph and tally it client-side,
on every poll, in every consumer.
`hive_jobq_wire::state_rollup` counts `roots` and their subtrees by
state, straight off a `Graph<N, R>` with **no bound on either
parameter**. A node's state is a scheduler concept, so counting by state
needs to know nothing about what the payload or the resource are;
bounding it like the projection does would make a host implement two
display traits to be allowed to count, which is a requirement about
rendering imposed on arithmetic.
It takes the roots for the same reason `wire_snapshot` does — which
groups are in view is the host's policy, and nothing is ever removed
from a graph — so passing the same set makes the roll-up describe
exactly the graph beside it.
Each entry carries BOTH counts: `nodes` (the whole subtree) and `roots`
(just the group tops). One rebuild is ~7 nodes and 1 root, so a summary
meaning *operations* and one meaning *steps* are different numbers over
the same queue, and picking one here would make this crate decide what
counts as a job — the domain question it exists not to answer. It
reports both structural facts; the viewer chooses.
A pair, not a map: JSON object keys are strings, so a map would spell
the state twice and give the wire no ordering. Every state rides with
its zeros in a fixed order, so a consumer can index positionally and
never handles a missing bucket. Tallying positionally against
`ALL_STATES` means a new upstream `State` fails the exhaustive match in
`state_index` rather than silently landing in an existing bucket.
hive-c0re serves it at `GET /api/jobq/rollup`. The queue-side method is
a call site, not an implementation: it supplies the lock and the same
`visible_roots` as `graph_snapshot`, so the summary cannot describe a
different visible set than the graph it summarises.
`graph_node` projects a `hive_jobq::Node` onto the wire type from the
parent commit: everything the crate records, with hive-c0re's own fields
(`agent`, `approval_id`, `inputs`, `build_log_id`) collected into the
opaque payload slot instead of standing as named columns. A node with
nothing domain-specific to say serialises no `data` key at all, so the
slot costs nothing when it is unused.
`graph_dep` turns a `DepWhen` into the set of outcomes that satisfy the
edge by asking it about each of the four terminal states, rather than
leaking the bitflags representation onto the wire.
`Resource::wire_name` gives the resource vocabulary a string form —
hive-jobq is generic over the resource type, so a viewer that renders any
graph cannot be handed this enum. The `agent:` prefix keeps per-agent
leases from colliding with a global resource sharing an agent's name.
`graph_snapshot` deliberately reuses `visible_dags` for retention: every
live group plus the newest terminal ones. Serving the raw graph would
grow without bound — evicted groups' nodes linger until bounded pruning
lands. Within a retained group nothing is filtered: group roots ride as
ordinary nodes, and `Done` nodes stay, which is the projection defect
behind the "rebuild shows a single node" report.
The endpoint lands in the same commit rather than after it. Without a
consumer the whole projection is dead code, and a wire type nobody
produces cannot be reviewed for whether it says the right things.
Its OpenAPI body is `serde_json::Value`, matching `api_state`: no type in
`hive-host-sock` derives `ToSchema`, and that crate stays dependency-lean
on purpose.
Follow-up from #2872: the comment said 'the rest of the (much larger)
route table below is undocumented for now' -- that was true when the
annotation sweep started, not anymore. Trimmed to state the current
fact plainly instead of narrating the sweep's history (mara: is that
level of detail relevant here) -- this is a plain rustdoc comment on
ApiDoc, not part of the actual OpenAPI JSON (that comes from the
separate #[openapi(info(description = ...))] attribute below it), so
keeping it terse and mechanical is the right call.
Swagger UI itself is nginx-hosted now (iris's 86a39c4c), so c0re
carrying its own vendored copy via utoipa-swagger-ui was a straight
duplicate — dropped the dependency (root Cargo.toml + hive-c0re's),
swapped the SwaggerUi::new(...).url(...) mount for a plain
/api/openapi.json GET route serving the same OpenApi doc as JSON.
Verified: cargo build/clippy/test -p hive-c0re clean, Cargo.lock
dropped utoipa-swagger-ui + utoipa-swagger-ui-vendored with no other
changes, nix fmt clean.
Adds CPU/memory cap columns and an inline edit form to the container-load
table in /core.html, backed by a new POST /api/resource-limits/{name}
dashboard endpoint.
## Backend (hive-c0re)
lifecycle_ops.rs — new post_resource_limits handler:
- Parses ResourceLimitsForm { cpu_quota, memory_max } (both optional; empty
string = clear override, fall back to hive-wide default).
- Validates each non-empty value via resource_limits::validate_cpu_quota /
validate_memory_max — returns 422 UNPROCESSABLE_ENTITY with a human-
readable message on invalid input so the dashboard can surface it inline.
- Calls meta::commit_resource_limits (staged git write under META_LOCK, same
as hivectl set-limits).
- Re-applies the drop-in immediately via lifecycle::write_dropins so the new
ceilings take effect on the next container start without waiting for a
rebuild.
- Triggers rescan_containers_and_emit so ContainerView.cpu_quota/memory_max
update via SSE without waiting for the next periodic sweep.
dashboard/mod.rs — registers the route:
POST /api/resource-limits/{name}
## Frontend (core.js + system-sections.css)
core.js:
- containersState derived from /api/state snapshot alongside tombstonesState
— supplies configured cpu_quota/memory_max to the LOAD table.
- lastLoadRows stash lets SSE-triggered re-renders call renderContainerLoad
without waiting for the next 5s poll.
- renderContainerLoad: adds cpu cap / mem cap columns (muted; tooltip
'configured ceiling — takes effect on next start') sourced from
ContainerView, plus a per-row S3T toggle button that expands an inline
edit form with cpu_quota / memory_max text inputs and a S4V3 button.
The edit form shows a restart hint, surfaces validation errors inline, and
collapses on success.
- container_state_changed SSE handler: updates containersState in place and
re-renders the LOAD table so the cap columns flip immediately after a save.
system-sections.css:
- CSS for the new cap columns (.cload-cap-th, .cload-cap) and inline edit
form (.cload-edit-row, .cload-edit-form, .cload-edit-label, etc.).
- Remove dead .rqe-step rule (step sub-step label retired from the wire in
'job_queue: retire the now-off-wire step sub-step label').
When ContainerView.paused is true, show a clickable yellow `⏸ paused`
badge on the agent card that POSTs to the new /api/resume/{name} endpoint
to un-park the turn loop. The badge doubles as the resume button so the
state is self-documenting and one click to fix.
The agent action menu gains ⏸ P4US3 (when not paused) and ▶ R3SUM3
(when paused), orthogonal to the running/stopped start/stop actions.
On the backend, /api/pause/{name} and /api/resume/{name} POST routes
wire to Coordinator::set_paused and trigger an immediate rescan so the
badge flips via the existing SSE ContainerUpdate without polling.
Depends on the ContainerView.paused field and Coordinator::set_paused
added in the parent PR.
The raw-graph wire drops the inline build_log_id; the client fetches a
node's captured output on demand. Two handlers resolve node id -> log-row
id (JobQueue::build_log_id_of, now keyed by the wire u64) then delegate to
the existing get_full / raw handlers: /api/build-log/<node_id> serves the
BuildLogFull JSON ({stdout, stderr} + header), /raw serves text/plain.
404 when the node has no linked log.
Per mara's feedback on PR #2407 ("better: you can also provide url in
dashboard, same as with matrix, no host config"), drops
services.hyperhive.extraForges and the admin-API mint/revoke flow
entirely. The operator now creates a token on the external forge
themselves and pastes a label + base URL + access token into the
dashboard's FORGES tab, the same shape as the GitHub PAT flow plus the
base-URL field from the matrix extra-account flow. hive-c0re only ever
writes/deletes two local files per account (forge-<label>-token,
forge-<label>.json sidecar for the URL) via hive-priv — no remote
account creation, no admin token, no revoke-on-the-remote-side, no nix
config to enumerate.
- nix/host-modules/hive-forge/default.nix: removed the extraForges
option, its label-format assertion, and the HYPERHIVE_EXTRA_FORGES
env forwarding.
- hive-c0re/src/forge/extra.rs: deleted (REST admin-API provisioning,
no longer needed).
- hive-c0re/src/dashboard/extra_forges.rs: GET /api/extra-forges?
agent= lists an agent's stored forges by scanning its state dir
(mirrors matrix_accounts.rs's filename-scan listing), POST
/api/extra-forge-account (agent/label/base_url/token/
action=add|remove) stores or removes an account.
- hive-sh4re/priv_proto.rs + hive-priv/main.rs: new
WriteAgentExtraForgeAccount/DeleteAgentExtraForgeAccount priv
requests (adds base_url, writes/deletes a JSON sidecar alongside the
token).
- hive-c0re/src/priv_client.rs: matching wrapper functions.
- frontend/packages/dashboard/src/credentials.{html,js}: FORGES tab is
a per-agent list + add-account paste form (label/base_url/token), no
grant/revoke-from-catalog UI.
- docs/web-ui/dashboard.md: FORGES tab section rewritten.
Supersedes the design in PR #2407 (already approved+green on the old
admin-API model) — opening as a fresh PR against the same issues
rather than force-pushing over the approved one.
New POST /api/infra-container/{name}/{action} dashboard route (start/
stop/restart on hive-ci/hive-forge/hive-gateway/hive-matrix), reusing
the existing priv_client::control_infra_container helper the
infra_admin agent path already uses, plus an audit_log entry per
attempt. Adds infra_containers to the /api/state StateSnapshot (name +
live running status via systemctl is-active). New 1NFR4 sub-tab on the
C0R3 dashboard page: one row per infra container with a running/
stopped badge and start/stop/restart buttons, polled every 5s while
the sub-tab is open.
Two issues flagged by argus in PR #2388 review:
1. Empty-key fallback: when load_or_generate() failed, webhook_secret was
String::new(). An attacker knowing this could forge deliveries with a
valid HMAC of the empty key. Fix: change to Option<String>; on None,
skip hook registration entirely and return 503 from /webhook/* handlers
(rather than 401 with a misleadingly-verifiable empty-key HMAC).
2. Stale hook cleanup: on upgrade from old code, old loopback hooks
(http://127.0.0.1:.../webhook/knowledge, .../webhook/config-pr) were
left alongside the new domain-URL hook. Fix: during ensure_webhook /
ensure_config_pr_webhook, after listing hooks, delete any that end with
our path suffix but point at a different base URL.
clippy + nix fmt clean.
Both webhook registrations (knowledge push + config-PR pull_request) now
use the public hive domain instead of loopback:
https://<HYPERHIVE_HIVE_DOMAIN>/webhook/{knowledge,config-pr}
This routes deliveries through the gateway, bypassing the Forgejo SSRF
guard that blocked loopback delivery and silently broke the config-PR
merge flow since launch.
Changes:
- webhook_secret: new module — auto-generate + persist a 32-byte HMAC
secret to STATE_ROOT/webhook-secret on first startup; verify
X-Hub-Signature-256 on every incoming webhook POST (HMAC-SHA256).
- forge/mod.rs: ensure_config_pr_webhook now takes hive_domain +
webhook_secret; sets secret in Forgejo hook config.
- workers/knowledge.rs: ensure_webhook same update.
- dashboard/webhook.rs: both handlers read raw Bytes first, verify HMAC,
then parse JSON. Returns 401 on signature mismatch.
- dashboard/mod.rs: AppState carries webhook_secret; serve() takes it.
- main.rs: load/generate secret at startup; pass to registration tasks
+ dashboard; add 5-minute config-PR polling fallback task.
- forge/config_pr_poll.rs: new — scan agent-configs/* for open PRs with
no pending MergeConfigPr approval; queue them. Idempotent.
- stores/approvals.rs: has_pending_merge_config_pr() for poll dedup.
- nix/modules/hive-gateway.nix: remove dashboardAuth from /webhook/
location (HMAC replaces basic auth for webhook endpoints; Forgejo
cannot send HTTP Basic credentials with webhook deliveries).
