07b62612b0
Per mara's go-ahead on hyperhive#3902 ("getting started is good, but
terminal rendering does not go in there i think"):
Moved 21 top-level docs/*.md files into 7 new topic subdirectories
(existing web-ui/, turn-loop/, swarm/, tools/, crates/ untouched):
getting-started/ setup.md
agent-lifecycle/ agent-hierarchy.md, approvals.md, persistence.md
trust-boundary/ boundary.md, security.md
integrations/ forge.md, matrix.md, github.md, knowledge.md
networking/ gateway.md, network.md, snapshot-store.md
scheduler/ jobq.md, coordinator.md, ci.md, observability.md
process/ conventions.md, gotchas.md, pr-review-gate.md
web-ui/ terminal-rendering.md (moved into the EXISTING dir,
per mara's correction to the original getting-started
guess -- it's UI implementation detail, not onboarding)
The physical layout now matches docs/README.md's own topical headers,
which already amounted to this taxonomy -- see the scoping comment on
the issue for the two findings that motivated this (a genuine
duplication between CLAUDE.md's old "Reading paths" list and
docs/README.md's grouped one, since drifted out of sync with each
other; and the flat layout not matching the grouping we already had).
Fixed every cross-reference this moved across the whole repo (~120
files: docs/ internal links at every depth, Rust doc comments, nix
module option docs, crate READMEs) -- verified two ways: a grep sweep
confirming zero remaining references to any old path, and a script
that resolves every markdown link in docs/**/*.md + CLAUDE.md +
README.md against the filesystem and reports anything that doesn't
exist (zero broken links).
Collapsed CLAUDE.md's "Reading paths" section (the duplicate) down to
a pointer at docs/README.md, now the single index. Rewrote
docs/README.md itself to use the new subdirectory paths and added the
one doc it was missing that CLAUDE.md's old copy had (pr-review-gate.md).
Classified all 22 docs/*.md files first via a haiku subagent (mara's
suggestion) on two axes -- proposed grouping and operator-vs-
implementation focus -- before finalizing the taxonomy; spot-checked
the report and found internal inconsistencies (its classification
table disagreed with its own summary section for a few files), so this
taxonomy is my original proposal + the one correction mara gave
directly, not a blind application of the subagent's table. The
operator-focus data it gathered is still useful for a follow-up
content pass (docs skewing 'mixed' rather than pure operator-facing),
not addressed in this PR -- structure only.
nix fmt clean, both pre-push lints clean.
2.8 KiB
The job queue, for operators
Every container operation — rebuild, first-spawn, a config-PR deploy,
power changes — runs through one shared job queue. This page explains
what the job queue is, as a general idea, independent of what any one
subsystem uses it for. For the hive-c0re-specific step catalogue and the
engineering internals (scheduler, leases, resource windows) see
coordinator.md instead.
What the job queue is, in the abstract
"jobq" is a generic engine for running many interdependent jobs under limited concurrency — it has no idea what a "container" or a "rebuild" is. Two ideas are all there is to it:
- A job is a small graph of steps, not one opaque blob. Steps can depend on each other (this step only starts once that one finishes), so a big operation is really a short, ordered sequence — not a single black box that's either "done" or "not done."
- A step can need a shared resource, which only so many steps can hold at once (a "slot"). If every currently-running step already holds the slots it needs, a new step that wants the same one waits its turn — that's the whole reason things queue instead of all firing at once.
The engine's whole job is: whenever a step's ordering and resource needs are both satisfied, run it. It has no opinion on what the steps do — that's supplied by whoever builds the graph. hive-c0re is the one thing building graphs on it today, but nothing about the engine is specific to containers or rebuilds; there's nothing stopping another subsystem from using the same engine for its own unrelated queue.
Watching it happen
Each row you see in a queue view (the BU1LDS page's R3BU1LD QU3U3
— see web-ui/dashboard.md — and swarm-ui's
/jobs page both render the same underlying graph) is one job; the rows
nested under it are that job's steps, in order (occasionally a couple run
side by side). A step shows one of:
| Glyph | Meaning |
|---|---|
⏸ |
queued, waiting its turn |
▶ |
running |
◐ |
its own work is done, waiting on a step nested under it |
✔ |
finished successfully |
✖ |
failed |
⊘ |
cancelled |
· |
skipped (not needed for this run) |
A step that isn't needed for a given run shows as · rather than being
left out of the tree entirely, so the same kind of operation keeps a
recognizable shape run to run, whichever steps it actually needed.