docs/jobq.md: add the missing abstract jobq section
mara: "the issue wanted jobq the abstract thing docs, but you documented the hive nodes" — the original PR only had the core-specific node table (the issue's second ask); it was missing the first ask, an operator-facing explanation of the jobq engine itself (graph of steps + shared resource slots) independent of what hive-c0re specifically builds on top of it. Adds that as its own section up front, and reframes the existing node table under an explicit "core-specific nodes" heading so the two asks read as clearly separate pieces.
This commit is contained in:
parent
582aebc5eb
commit
dcc50902db
1 changed files with 34 additions and 5 deletions
39
docs/jobq.md
39
docs/jobq.md
|
|
@ -1,14 +1,40 @@
|
||||||
# The job queue, for operators
|
# The job queue, for operators
|
||||||
|
|
||||||
Every container operation — rebuild, first-spawn, a config-PR deploy,
|
Every container operation — rebuild, first-spawn, a config-PR deploy,
|
||||||
power changes — runs through one shared job queue. This page explains what
|
power changes — runs through one shared job queue. This page has two
|
||||||
you see when you watch it: the **BU1LDS** page's R3BU1LD QU3U3 (see
|
parts: what the job queue *is*, as a general idea, and what hive-c0re
|
||||||
[`web-ui/dashboard.md`](web-ui/dashboard.md)) and swarm-ui's `/jobs` page
|
specifically uses it for — the step labels you actually see on the
|
||||||
both render the same underlying graph. For the engineering internals
|
**BU1LDS** page's R3BU1LD QU3U3 (see
|
||||||
|
[`web-ui/dashboard.md`](web-ui/dashboard.md)) and swarm-ui's `/jobs` page,
|
||||||
|
which both render the same underlying graph. For the engineering internals
|
||||||
(scheduler, leases, resource windows) see [`coordinator.md`](coordinator.md)
|
(scheduler, leases, resource windows) see [`coordinator.md`](coordinator.md)
|
||||||
instead — this page stays at the level of "what does this step mean," not
|
instead — this page stays at the level of "what does this step mean," not
|
||||||
"how is it implemented."
|
"how is it implemented."
|
||||||
|
|
||||||
|
## What the job queue is, in the abstract
|
||||||
|
|
||||||
|
"jobq" is a generic engine for running many interdependent jobs under
|
||||||
|
limited concurrency — it has no idea what a "container" or a "rebuild" is.
|
||||||
|
Two ideas are all there is to it:
|
||||||
|
|
||||||
|
- **A job is a small graph of steps**, not one opaque blob. Steps can
|
||||||
|
depend on each other (this step only starts once that one finishes), so
|
||||||
|
a big operation is really a short, ordered sequence — not a single
|
||||||
|
black box that's either "done" or "not done."
|
||||||
|
- **A step can need a shared resource**, which only so many steps can hold
|
||||||
|
at once (a "slot"). If every currently-running step already holds the
|
||||||
|
slots it needs, a new step that wants the same one waits its turn —
|
||||||
|
that's the whole reason things queue instead of all firing at once.
|
||||||
|
|
||||||
|
The engine's whole job is: whenever a step's ordering and resource needs
|
||||||
|
are both satisfied, run it. It has no opinion on what the steps *do* —
|
||||||
|
that's supplied by whoever builds the graph. hive-c0re is the one thing
|
||||||
|
building graphs on it today (there's nothing stopping another subsystem
|
||||||
|
from using the same engine for its own unrelated queue), and it always
|
||||||
|
builds them from one fixed catalogue of step kinds — that catalogue is
|
||||||
|
"the core-specific nodes" in the next section, and is as close to the
|
||||||
|
implementation as this page gets.
|
||||||
|
|
||||||
## Operations and steps
|
## Operations and steps
|
||||||
|
|
||||||
Each **row** in the queue view is one operation you asked for (or that the
|
Each **row** in the queue view is one operation you asked for (or that the
|
||||||
|
|
@ -30,7 +56,10 @@ shows one of:
|
||||||
A step's label is a short, fixed word — `Prebuild`, `Swap`, `Reconcile`,
|
A step's label is a short, fixed word — `Prebuild`, `Swap`, `Reconcile`,
|
||||||
and so on. The table below is what each one means, in plain terms.
|
and so on. The table below is what each one means, in plain terms.
|
||||||
|
|
||||||
## What each step does
|
## The core-specific nodes
|
||||||
|
|
||||||
|
Every step you'll actually see comes from hive-c0re's fixed catalogue —
|
||||||
|
what each one means, in plain terms, no internals:
|
||||||
|
|
||||||
| Label | What it means |
|
| Label | What it means |
|
||||||
| --------------------- | ---------------------------------------------------------------------------------------------------------- |
|
| --------------------- | ---------------------------------------------------------------------------------------------------------- |
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue