From 8363a459bfd62a71d5c52f20b5a0893b3151fef4 Mon Sep 17 00:00:00 2001 From: iris Date: Wed, 26 Aug 2026 22:48:17 +0200 Subject: [PATCH] docs/jobq.md: drop the core-specific node table, keep only the abstract engine explanation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit mara: "only the abstract jobq part was asked for in the first place" — the previous revision still carried the full hive-c0re step-kind table under a "core-specific nodes" heading; that catalogue belongs in coordinator.md (where it already lived) alongside the rest of hive-c0re's job-queue internals, not duplicated here. jobq.md is now just the domain-agnostic model (graph of steps + shared resource slots) plus the generic row/step/glyph framing for watching it in the dashboard — no hive-c0re-specific step names anywhere. Coordinator.md's job-queue section, docs/README.md, and the root CLAUDE.md reading-paths index are updated to match. --- CLAUDE.md | 10 ++--- docs/README.md | 9 ++-- docs/coordinator.md | 7 +-- docs/jobq.md | 101 ++++++++++++-------------------------------- 4 files changed, 39 insertions(+), 88 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 15efcb93..d1a69727 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -242,11 +242,11 @@ read them à la carte. endpoint authenticate a pushing hive, and what does a snapshot actually contain?"** → [`docs/snapshot-store.md`](docs/snapshot-store.md). -- **"What am I looking at in the build queue? What does each step - mean?"** → [`docs/jobq.md`](docs/jobq.md) (operator-facing, no - implementation detail). -- **"How does the rebuild queue work internally? What are queue kinds and - sources?"** → [`docs/coordinator.md`](docs/coordinator.md). +- **"What is the job queue, as a general idea?"** → + [`docs/jobq.md`](docs/jobq.md) (operator-facing, no hive-c0re specifics). +- **"How does the rebuild queue work? What are the concrete step kinds, + queue sources, scheduler internals?"** → + [`docs/coordinator.md`](docs/coordinator.md). - **"How does the CI runner work? What's the auto-registration flow?"** → [`docs/ci.md`](docs/ci.md). - **"What is `/knowledge`? How does the hive-wide knowledge repo sync, diff --git a/docs/README.md b/docs/README.md index 1bda4de1..5a8c37eb 100644 --- a/docs/README.md +++ b/docs/README.md @@ -79,11 +79,10 @@ declarations. ## Scheduler, CI, observability -- **What am I looking at in the build queue / R3BU1LD QU3U3? What does - each step mean?** → [`jobq.md`](jobq.md) — operator-facing, no - implementation detail. -- **How does the rebuild queue work internally? What are queue kinds and - sources?** → [`coordinator.md`](coordinator.md). +- **What is the job queue, as a general idea (not hive-c0re specifics)?** → + [`jobq.md`](jobq.md) — operator-facing, no implementation detail. +- **How does the rebuild queue work? What are the concrete step kinds, + queue sources, scheduler internals?** → [`coordinator.md`](coordinator.md). - **How does the CI runner work? What's the auto-registration flow?** → [`ci.md`](ci.md). - **How do I export Claude Code metrics (tokens, cost, tool calls) to diff --git a/docs/coordinator.md b/docs/coordinator.md index 4e2f6c6b..e6bd2ee2 100644 --- a/docs/coordinator.md +++ b/docs/coordinator.md @@ -8,9 +8,10 @@ and `docs/persistence.md`. ## Job queue -**For what each step means to someone watching the dashboard, without the -implementation detail below, see [`jobq.md`](jobq.md) instead** — this -section stays at the internals level (module layout, scheduler mechanics, +**For the job queue as a general idea — graph of steps, shared resource +slots, no hive-c0re specifics — see [`jobq.md`](jobq.md) instead.** This +section covers the concrete node inventory hive-c0re builds on top of that +engine, plus the internals (module layout, scheduler mechanics, resource/lease semantics). Every container/meta operation (rebuild, meta-update, first-spawn, power diff --git a/docs/jobq.md b/docs/jobq.md index 43d0e2be..79a83e81 100644 --- a/docs/jobq.md +++ b/docs/jobq.md @@ -1,15 +1,11 @@ # The job queue, for operators Every container operation — rebuild, first-spawn, a config-PR deploy, -power changes — runs through one shared job queue. This page has two -parts: what the job queue *is*, as a general idea, and what hive-c0re -specifically uses it for — the step labels you actually see on the -**BU1LDS** page's R3BU1LD QU3U3 (see -[`web-ui/dashboard.md`](web-ui/dashboard.md)) and swarm-ui's `/jobs` page, -which both render the same underlying graph. For the engineering internals -(scheduler, leases, resource windows) see [`coordinator.md`](coordinator.md) -instead — this page stays at the level of "what does this step mean," not -"how is it implemented." +power changes — runs through one shared job queue. This page explains +what the job queue *is*, as a general idea, independent of what any one +subsystem uses it for. For the hive-c0re-specific step catalogue and the +engineering internals (scheduler, leases, resource windows) see +[`coordinator.md`](coordinator.md) instead. ## What the job queue is, in the abstract @@ -29,73 +25,28 @@ Two ideas are all there is to it: The engine's whole job is: whenever a step's ordering and resource needs are both satisfied, run it. It has no opinion on what the steps *do* — that's supplied by whoever builds the graph. hive-c0re is the one thing -building graphs on it today (there's nothing stopping another subsystem -from using the same engine for its own unrelated queue), and it always -builds them from one fixed catalogue of step kinds — that catalogue is -"the core-specific nodes" in the next section, and is as close to the -implementation as this page gets. +building graphs on it today, but nothing about the engine is specific to +containers or rebuilds; there's nothing stopping another subsystem from +using the same engine for its own unrelated queue. -## Operations and steps +## Watching it happen -Each **row** in the queue view is one operation you asked for (or that the -system triggered on your behalf, e.g. a config-lock bump cascading into a -rebuild). The rows nested under it are that operation's **steps**, run in -order (occasionally with a couple of steps running side by side). A step -shows one of: +Each **row** you see in a queue view (the **BU1LDS** page's R3BU1LD QU3U3 +— see [`web-ui/dashboard.md`](web-ui/dashboard.md) — and swarm-ui's +`/jobs` page both render the same underlying graph) is one job; the rows +nested under it are that job's steps, in order (occasionally a couple run +side by side). A step shows one of: -| Glyph | Meaning | -| ----- | ----------------------------------------------------- | -| `⏸` | queued, waiting its turn | -| `▶` | running | -| `◐` | its own work is done, waiting on a step nested under it | -| `✔` | finished successfully | -| `✖` | failed | -| `⊘` | cancelled | -| `·` | skipped (not needed for this run) | +| Glyph | Meaning | +| ----- | -------------------------------------------------------- | +| `⏸` | queued, waiting its turn | +| `▶` | running | +| `◐` | its own work is done, waiting on a step nested under it | +| `✔` | finished successfully | +| `✖` | failed | +| `⊘` | cancelled | +| `·` | skipped (not needed for this run) | -A step's label is a short, fixed word — `Prebuild`, `Swap`, `Reconcile`, -and so on. The table below is what each one means, in plain terms. - -## The core-specific nodes - -Every step you'll actually see comes from hive-c0re's fixed catalogue — -what each one means, in plain terms, no internals: - -| Label | What it means | -| --------------------- | ---------------------------------------------------------------------------------------------------------- | -| `MetaSync` | Refreshes the agent's config from the shared repo before building — housekeeping, not user-visible work. | -| `Prebuild` | Builds the new version of the container **in the background** while the current one keeps running. Nothing is disrupted yet — this is where a build failure shows up, before anything is touched. | -| `StopForUpdate` | Stops the container so the new version can be put in place. Skipped if it was already stopped. | -| `Swap` | Switches the container over to the version built by `Prebuild`. Fast, since the build already happened. | -| `RebuildBookkeeping` | Records that the rebuild succeeded — updates the version marker, syncs forge/matrix accounts, notifies other agents. | -| `Reconcile` | Brings the container to whatever state it's supposed to be in (running or stopped) and makes sure it actually got there. Runs at the end of nearly every operation as a final "did it work?" check. | -| `Create` | First-time creation of a brand-new container. | -| `WriteDropin` | Applies the container's resource limits (CPU/memory) and any nspawn flag changes. | -| `WritePermFile` | Applies a permission change — which tool groups or capabilities an agent has. | -| `Reparent` | Moves one or more agents to a new parent in the topology tree. | -| `MetaLock` | Bumps the shared configuration lock file every agent's build is derived from. Can cascade into a rebuild for every agent affected by the bump — you'll see those appear as extra rows under the same operation. | -| `AgentWindow` | Groups one agent's rebuild steps together — no visible work of its own, just a wrapper row. | -| `Signal` | Asks a running agent to finish its current turn before being stopped (a graceful stop). | -| `Drain` | Waits for that agent to confirm it's done, up to a few minutes, before continuing. | -| `DeployWindow` | The starting point of an approved config-PR deploy — no visible work, just where the deploy begins. | -| `MergeVerify` | Double-checks the approved PR hasn't changed since it was reviewed, right before merging it. | -| `DeployApply` | Merges the approved PR and carries out the deploy — this is where the real changes start happening. | -| `DeployTail` | Cleans up after the deploy: rolls the merge back if it didn't finish cleanly, otherwise mirrors the result to the forge. | - -Not every operation uses every step — a rebuild looks roughly like -`MetaSync → Prebuild → StopForUpdate → Swap → RebuildBookkeeping → -Reconcile`; a plain stop or start is much shorter (`Reconcile` alone, or -`Signal → Drain → Reconcile` for a graceful one). A step that isn't -needed for a given run shows as `·` (skipped) rather than being left out -of the tree, so the shape stays recognizable across different operations. - -## Where this shows up - -- **BU1LDS → R3BU1LD QU3U3** (single hive): the primary place to watch - this. Includes a live build-log panel for whatever `Prebuild`/`Swap` - step is currently running. -- **swarm-ui → `/jobs`**: the same graph, viewed across every hive in the - swarm. -- The dashboard's per-agent status pill lights up while any step that - touches that agent's container is in flight — you don't need the queue - page open to notice a rebuild is happening. +A step that isn't needed for a given run shows as `·` rather than being +left out of the tree entirely, so the same kind of operation keeps a +recognizable shape run to run, whichever steps it actually needed.