docs/jobq.md: drop the core-specific node table, keep only the abstract engine explanation
mara: "only the abstract jobq part was asked for in the first place" — the previous revision still carried the full hive-c0re step-kind table under a "core-specific nodes" heading; that catalogue belongs in coordinator.md (where it already lived) alongside the rest of hive-c0re's job-queue internals, not duplicated here. jobq.md is now just the domain-agnostic model (graph of steps + shared resource slots) plus the generic row/step/glyph framing for watching it in the dashboard — no hive-c0re-specific step names anywhere. Coordinator.md's job-queue section, docs/README.md, and the root CLAUDE.md reading-paths index are updated to match.
This commit is contained in:
iris
mara
parent
dcc50902db
commit
8363a459bf
4 changed files with 43 additions and 92 deletions
10
CLAUDE.md
10
CLAUDE.md
|
|
@ -242,11 +242,11 @@ read them à la carte.
|
||||||
endpoint authenticate a pushing hive, and what does a snapshot
|
endpoint authenticate a pushing hive, and what does a snapshot
|
||||||
actually contain?"** →
|
actually contain?"** →
|
||||||
[`docs/snapshot-store.md`](docs/snapshot-store.md).
|
[`docs/snapshot-store.md`](docs/snapshot-store.md).
|
||||||
- **"What am I looking at in the build queue? What does each step
|
- **"What is the job queue, as a general idea?"** →
|
||||||
mean?"** → [`docs/jobq.md`](docs/jobq.md) (operator-facing, no
|
[`docs/jobq.md`](docs/jobq.md) (operator-facing, no hive-c0re specifics).
|
||||||
implementation detail).
|
- **"How does the rebuild queue work? What are the concrete step kinds,
|
||||||
- **"How does the rebuild queue work internally? What are queue kinds and
|
queue sources, scheduler internals?"** →
|
||||||
sources?"** → [`docs/coordinator.md`](docs/coordinator.md).
|
[`docs/coordinator.md`](docs/coordinator.md).
|
||||||
- **"How does the CI runner work? What's the auto-registration flow?"** →
|
- **"How does the CI runner work? What's the auto-registration flow?"** →
|
||||||
[`docs/ci.md`](docs/ci.md).
|
[`docs/ci.md`](docs/ci.md).
|
||||||
- **"What is `/knowledge`? How does the hive-wide knowledge repo sync,
|
- **"What is `/knowledge`? How does the hive-wide knowledge repo sync,
|
||||||
|
|
|
||||||
|
|
@ -79,11 +79,10 @@ declarations.
|
||||||
|
|
||||||
## Scheduler, CI, observability
|
## Scheduler, CI, observability
|
||||||
|
|
||||||
- **What am I looking at in the build queue / R3BU1LD QU3U3? What does
|
- **What is the job queue, as a general idea (not hive-c0re specifics)?** →
|
||||||
each step mean?** → [`jobq.md`](jobq.md) — operator-facing, no
|
[`jobq.md`](jobq.md) — operator-facing, no implementation detail.
|
||||||
implementation detail.
|
- **How does the rebuild queue work? What are the concrete step kinds,
|
||||||
- **How does the rebuild queue work internally? What are queue kinds and
|
queue sources, scheduler internals?** → [`coordinator.md`](coordinator.md).
|
||||||
sources?** → [`coordinator.md`](coordinator.md).
|
|
||||||
- **How does the CI runner work? What's the auto-registration flow?** →
|
- **How does the CI runner work? What's the auto-registration flow?** →
|
||||||
[`ci.md`](ci.md).
|
[`ci.md`](ci.md).
|
||||||
- **How do I export Claude Code metrics (tokens, cost, tool calls) to
|
- **How do I export Claude Code metrics (tokens, cost, tool calls) to
|
||||||
|
|
|
||||||
|
|
@ -8,9 +8,10 @@ and `docs/persistence.md`.
|
||||||
|
|
||||||
## Job queue
|
## Job queue
|
||||||
|
|
||||||
**For what each step means to someone watching the dashboard, without the
|
**For the job queue as a general idea — graph of steps, shared resource
|
||||||
implementation detail below, see [`jobq.md`](jobq.md) instead** — this
|
slots, no hive-c0re specifics — see [`jobq.md`](jobq.md) instead.** This
|
||||||
section stays at the internals level (module layout, scheduler mechanics,
|
section covers the concrete node inventory hive-c0re builds on top of that
|
||||||
|
engine, plus the internals (module layout, scheduler mechanics,
|
||||||
resource/lease semantics).
|
resource/lease semantics).
|
||||||
|
|
||||||
Every container/meta operation (rebuild, meta-update, first-spawn, power
|
Every container/meta operation (rebuild, meta-update, first-spawn, power
|
||||||
|
|
|
||||||
101
docs/jobq.md
101
docs/jobq.md
|
|
@ -1,15 +1,11 @@
|
||||||
# The job queue, for operators
|
# The job queue, for operators
|
||||||
|
|
||||||
Every container operation — rebuild, first-spawn, a config-PR deploy,
|
Every container operation — rebuild, first-spawn, a config-PR deploy,
|
||||||
power changes — runs through one shared job queue. This page has two
|
power changes — runs through one shared job queue. This page explains
|
||||||
parts: what the job queue *is*, as a general idea, and what hive-c0re
|
what the job queue *is*, as a general idea, independent of what any one
|
||||||
specifically uses it for — the step labels you actually see on the
|
subsystem uses it for. For the hive-c0re-specific step catalogue and the
|
||||||
**BU1LDS** page's R3BU1LD QU3U3 (see
|
engineering internals (scheduler, leases, resource windows) see
|
||||||
[`web-ui/dashboard.md`](web-ui/dashboard.md)) and swarm-ui's `/jobs` page,
|
[`coordinator.md`](coordinator.md) instead.
|
||||||
which both render the same underlying graph. For the engineering internals
|
|
||||||
(scheduler, leases, resource windows) see [`coordinator.md`](coordinator.md)
|
|
||||||
instead — this page stays at the level of "what does this step mean," not
|
|
||||||
"how is it implemented."
|
|
||||||
|
|
||||||
## What the job queue is, in the abstract
|
## What the job queue is, in the abstract
|
||||||
|
|
||||||
|
|
@ -29,73 +25,28 @@ Two ideas are all there is to it:
|
||||||
The engine's whole job is: whenever a step's ordering and resource needs
|
The engine's whole job is: whenever a step's ordering and resource needs
|
||||||
are both satisfied, run it. It has no opinion on what the steps *do* —
|
are both satisfied, run it. It has no opinion on what the steps *do* —
|
||||||
that's supplied by whoever builds the graph. hive-c0re is the one thing
|
that's supplied by whoever builds the graph. hive-c0re is the one thing
|
||||||
building graphs on it today (there's nothing stopping another subsystem
|
building graphs on it today, but nothing about the engine is specific to
|
||||||
from using the same engine for its own unrelated queue), and it always
|
containers or rebuilds; there's nothing stopping another subsystem from
|
||||||
builds them from one fixed catalogue of step kinds — that catalogue is
|
using the same engine for its own unrelated queue.
|
||||||
"the core-specific nodes" in the next section, and is as close to the
|
|
||||||
implementation as this page gets.
|
|
||||||
|
|
||||||
## Operations and steps
|
## Watching it happen
|
||||||
|
|
||||||
Each **row** in the queue view is one operation you asked for (or that the
|
Each **row** you see in a queue view (the **BU1LDS** page's R3BU1LD QU3U3
|
||||||
system triggered on your behalf, e.g. a config-lock bump cascading into a
|
— see [`web-ui/dashboard.md`](web-ui/dashboard.md) — and swarm-ui's
|
||||||
rebuild). The rows nested under it are that operation's **steps**, run in
|
`/jobs` page both render the same underlying graph) is one job; the rows
|
||||||
order (occasionally with a couple of steps running side by side). A step
|
nested under it are that job's steps, in order (occasionally a couple run
|
||||||
shows one of:
|
side by side). A step shows one of:
|
||||||
|
|
||||||
| Glyph | Meaning |
|
| Glyph | Meaning |
|
||||||
| ----- | ----------------------------------------------------- |
|
| ----- | -------------------------------------------------------- |
|
||||||
| `⏸` | queued, waiting its turn |
|
| `⏸` | queued, waiting its turn |
|
||||||
| `▶` | running |
|
| `▶` | running |
|
||||||
| `◐` | its own work is done, waiting on a step nested under it |
|
| `◐` | its own work is done, waiting on a step nested under it |
|
||||||
| `✔` | finished successfully |
|
| `✔` | finished successfully |
|
||||||
| `✖` | failed |
|
| `✖` | failed |
|
||||||
| `⊘` | cancelled |
|
| `⊘` | cancelled |
|
||||||
| `·` | skipped (not needed for this run) |
|
| `·` | skipped (not needed for this run) |
|
||||||
|
|
||||||
A step's label is a short, fixed word — `Prebuild`, `Swap`, `Reconcile`,
|
A step that isn't needed for a given run shows as `·` rather than being
|
||||||
and so on. The table below is what each one means, in plain terms.
|
left out of the tree entirely, so the same kind of operation keeps a
|
||||||
|
recognizable shape run to run, whichever steps it actually needed.
|
||||||
## The core-specific nodes
|
|
||||||
|
|
||||||
Every step you'll actually see comes from hive-c0re's fixed catalogue —
|
|
||||||
what each one means, in plain terms, no internals:
|
|
||||||
|
|
||||||
| Label | What it means |
|
|
||||||
| --------------------- | ---------------------------------------------------------------------------------------------------------- |
|
|
||||||
| `MetaSync` | Refreshes the agent's config from the shared repo before building — housekeeping, not user-visible work. |
|
|
||||||
| `Prebuild` | Builds the new version of the container **in the background** while the current one keeps running. Nothing is disrupted yet — this is where a build failure shows up, before anything is touched. |
|
|
||||||
| `StopForUpdate` | Stops the container so the new version can be put in place. Skipped if it was already stopped. |
|
|
||||||
| `Swap` | Switches the container over to the version built by `Prebuild`. Fast, since the build already happened. |
|
|
||||||
| `RebuildBookkeeping` | Records that the rebuild succeeded — updates the version marker, syncs forge/matrix accounts, notifies other agents. |
|
|
||||||
| `Reconcile` | Brings the container to whatever state it's supposed to be in (running or stopped) and makes sure it actually got there. Runs at the end of nearly every operation as a final "did it work?" check. |
|
|
||||||
| `Create` | First-time creation of a brand-new container. |
|
|
||||||
| `WriteDropin` | Applies the container's resource limits (CPU/memory) and any nspawn flag changes. |
|
|
||||||
| `WritePermFile` | Applies a permission change — which tool groups or capabilities an agent has. |
|
|
||||||
| `Reparent` | Moves one or more agents to a new parent in the topology tree. |
|
|
||||||
| `MetaLock` | Bumps the shared configuration lock file every agent's build is derived from. Can cascade into a rebuild for every agent affected by the bump — you'll see those appear as extra rows under the same operation. |
|
|
||||||
| `AgentWindow` | Groups one agent's rebuild steps together — no visible work of its own, just a wrapper row. |
|
|
||||||
| `Signal` | Asks a running agent to finish its current turn before being stopped (a graceful stop). |
|
|
||||||
| `Drain` | Waits for that agent to confirm it's done, up to a few minutes, before continuing. |
|
|
||||||
| `DeployWindow` | The starting point of an approved config-PR deploy — no visible work, just where the deploy begins. |
|
|
||||||
| `MergeVerify` | Double-checks the approved PR hasn't changed since it was reviewed, right before merging it. |
|
|
||||||
| `DeployApply` | Merges the approved PR and carries out the deploy — this is where the real changes start happening. |
|
|
||||||
| `DeployTail` | Cleans up after the deploy: rolls the merge back if it didn't finish cleanly, otherwise mirrors the result to the forge. |
|
|
||||||
|
|
||||||
Not every operation uses every step — a rebuild looks roughly like
|
|
||||||
`MetaSync → Prebuild → StopForUpdate → Swap → RebuildBookkeeping →
|
|
||||||
Reconcile`; a plain stop or start is much shorter (`Reconcile` alone, or
|
|
||||||
`Signal → Drain → Reconcile` for a graceful one). A step that isn't
|
|
||||||
needed for a given run shows as `·` (skipped) rather than being left out
|
|
||||||
of the tree, so the shape stays recognizable across different operations.
|
|
||||||
|
|
||||||
## Where this shows up
|
|
||||||
|
|
||||||
- **BU1LDS → R3BU1LD QU3U3** (single hive): the primary place to watch
|
|
||||||
this. Includes a live build-log panel for whatever `Prebuild`/`Swap`
|
|
||||||
step is currently running.
|
|
||||||
- **swarm-ui → `/jobs`**: the same graph, viewed across every hive in the
|
|
||||||
swarm.
|
|
||||||
- The dashboard's per-agent status pill lights up while any step that
|
|
||||||
touches that agent's container is in flight — you don't need the queue
|
|
||||||
page open to notice a rebuild is happening.
|
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue