vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs/jobq.md: drop the core-specific node table, keep only the abstract engine explanation

Browse source 
mara: "only the abstract jobq part was asked for in the first place" —
the previous revision still carried the full hive-c0re step-kind table
under a "core-specific nodes" heading; that catalogue belongs in
coordinator.md (where it already lived) alongside the rest of
hive-c0re's job-queue internals, not duplicated here.

jobq.md is now just the domain-agnostic model (graph of steps + shared
resource slots) plus the generic row/step/glyph framing for watching it
in the dashboard — no hive-c0re-specific step names anywhere.
Coordinator.md's job-queue section, docs/README.md, and the root
CLAUDE.md reading-paths index are updated to match.
This commit is contained in:
iris 2026-08-26 22:48:17 +02:00 • committed by mara
commit 8363a459bf
4 changed files with 43 additions and 92 deletions
Download patch file Download diff file Expand all files Collapse all files

10
CLAUDE.md
View file

@ -242,11 +242,11 @@ read them à la carte.
endpoint authenticate a pushing hive, and what does a snapshot
actually contain?"** →
[`docs/snapshot-store.md`](docs/snapshot-store.md).
- **"What am I looking at in the build queue? What does each step
mean?"** → [`docs/jobq.md`](docs/jobq.md) (operator-facing, no
implementation detail).
- **"How does the rebuild queue work internally? What are queue kinds and
sources?"** → [`docs/coordinator.md`](docs/coordinator.md).
- **"What is the job queue, as a general idea?"** →
[`docs/jobq.md`](docs/jobq.md) (operator-facing, no hive-c0re specifics).
- **"How does the rebuild queue work? What are the concrete step kinds,
queue sources, scheduler internals?"** →
[`docs/coordinator.md`](docs/coordinator.md).
- **"How does the CI runner work? What's the auto-registration flow?"** →
[`docs/ci.md`](docs/ci.md).
- **"What is `/knowledge`? How does the hive-wide knowledge repo sync,

9
docs/README.md
View file

@ -79,11 +79,10 @@ declarations.
## Scheduler, CI, observability
- **What am I looking at in the build queue / R3BU1LD QU3U3? What does
each step mean?** → [`jobq.md`](jobq.md) — operator-facing, no
implementation detail.
- **How does the rebuild queue work internally? What are queue kinds and
sources?** → [`coordinator.md`](coordinator.md).
- **What is the job queue, as a general idea (not hive-c0re specifics)?** →
[`jobq.md`](jobq.md) — operator-facing, no implementation detail.
- **How does the rebuild queue work? What are the concrete step kinds,
queue sources, scheduler internals?** → [`coordinator.md`](coordinator.md).
- **How does the CI runner work? What's the auto-registration flow?** →
[`ci.md`](ci.md).
- **How do I export Claude Code metrics (tokens, cost, tool calls) to

7
docs/coordinator.md
View file

@ -8,9 +8,10 @@ and `docs/persistence.md`.
## Job queue
**For what each step means to someone watching the dashboard, without the
implementation detail below, see [`jobq.md`](jobq.md) instead** — this
section stays at the internals level (module layout, scheduler mechanics,
**For the job queue as a general idea — graph of steps, shared resource
slots, no hive-c0re specifics — see [`jobq.md`](jobq.md) instead.** This
section covers the concrete node inventory hive-c0re builds on top of that
engine, plus the internals (module layout, scheduler mechanics,
resource/lease semantics).
Every container/meta operation (rebuild, meta-update, first-spawn, power

101
docs/jobq.md
View file

@ -1,15 +1,11 @@
# The job queue, for operators
Every container operation — rebuild, first-spawn, a config-PR deploy,
power changes — runs through one shared job queue. This page has two
parts: what the job queue *is*, as a general idea, and what hive-c0re
specifically uses it for — the step labels you actually see on the
**BU1LDS** page's R3BU1LD QU3U3 (see
[`web-ui/dashboard.md`](web-ui/dashboard.md)) and swarm-ui's `/jobs` page,
which both render the same underlying graph. For the engineering internals
(scheduler, leases, resource windows) see [`coordinator.md`](coordinator.md)
instead — this page stays at the level of "what does this step mean," not
"how is it implemented."
power changes — runs through one shared job queue. This page explains
what the job queue *is*, as a general idea, independent of what any one
subsystem uses it for. For the hive-c0re-specific step catalogue and the
engineering internals (scheduler, leases, resource windows) see
[`coordinator.md`](coordinator.md) instead.
## What the job queue is, in the abstract
@ -29,73 +25,28 @@ Two ideas are all there is to it:
The engine's whole job is: whenever a step's ordering and resource needs
are both satisfied, run it. It has no opinion on what the steps *do* —
that's supplied by whoever builds the graph. hive-c0re is the one thing
building graphs on it today (there's nothing stopping another subsystem
from using the same engine for its own unrelated queue), and it always
builds them from one fixed catalogue of step kinds — that catalogue is
"the core-specific nodes" in the next section, and is as close to the
implementation as this page gets.
building graphs on it today, but nothing about the engine is specific to
containers or rebuilds; there's nothing stopping another subsystem from
using the same engine for its own unrelated queue.
## Operations and steps
## Watching it happen
Each **row** in the queue view is one operation you asked for (or that the
system triggered on your behalf, e.g. a config-lock bump cascading into a
rebuild). The rows nested under it are that operation's **steps**, run in
order (occasionally with a couple of steps running side by side). A step
shows one of:
Each **row** you see in a queue view (the **BU1LDS** page's R3BU1LD QU3U3
— see [`web-ui/dashboard.md`](web-ui/dashboard.md) — and swarm-ui's
`/jobs` page both render the same underlying graph) is one job; the rows
nested under it are that job's steps, in order (occasionally a couple run
side by side). A step shows one of:
| Glyph | Meaning |
| ----- | ----------------------------------------------------- |
| `⏸` | queued, waiting its turn |
| `▶` | running |
| `◐` | its own work is done, waiting on a step nested under it |
| `✔` | finished successfully |
| `✖` | failed |
| `⊘` | cancelled |
| `·` | skipped (not needed for this run) |
| Glyph | Meaning |
| ----- | -------------------------------------------------------- |
| `⏸` | queued, waiting its turn |
| `▶` | running |
| `◐` | its own work is done, waiting on a step nested under it |
| `✔` | finished successfully |
| `✖` | failed |
| `⊘` | cancelled |
| `·` | skipped (not needed for this run) |
A step's label is a short, fixed word — `Prebuild`, `Swap`, `Reconcile`,
and so on. The table below is what each one means, in plain terms.
## The core-specific nodes
Every step you'll actually see comes from hive-c0re's fixed catalogue —
what each one means, in plain terms, no internals:
| Label | What it means |
| --------------------- | ---------------------------------------------------------------------------------------------------------- |
| `MetaSync` | Refreshes the agent's config from the shared repo before building — housekeeping, not user-visible work. |
| `Prebuild` | Builds the new version of the container **in the background** while the current one keeps running. Nothing is disrupted yet — this is where a build failure shows up, before anything is touched. |
| `StopForUpdate` | Stops the container so the new version can be put in place. Skipped if it was already stopped. |
| `Swap` | Switches the container over to the version built by `Prebuild`. Fast, since the build already happened. |
| `RebuildBookkeeping` | Records that the rebuild succeeded — updates the version marker, syncs forge/matrix accounts, notifies other agents. |
| `Reconcile` | Brings the container to whatever state it's supposed to be in (running or stopped) and makes sure it actually got there. Runs at the end of nearly every operation as a final "did it work?" check. |
| `Create` | First-time creation of a brand-new container. |
| `WriteDropin` | Applies the container's resource limits (CPU/memory) and any nspawn flag changes. |
| `WritePermFile` | Applies a permission change — which tool groups or capabilities an agent has. |
| `Reparent` | Moves one or more agents to a new parent in the topology tree. |
| `MetaLock` | Bumps the shared configuration lock file every agent's build is derived from. Can cascade into a rebuild for every agent affected by the bump — you'll see those appear as extra rows under the same operation. |
| `AgentWindow` | Groups one agent's rebuild steps together — no visible work of its own, just a wrapper row. |
| `Signal` | Asks a running agent to finish its current turn before being stopped (a graceful stop). |
| `Drain` | Waits for that agent to confirm it's done, up to a few minutes, before continuing. |
| `DeployWindow` | The starting point of an approved config-PR deploy — no visible work, just where the deploy begins. |
| `MergeVerify` | Double-checks the approved PR hasn't changed since it was reviewed, right before merging it. |
| `DeployApply` | Merges the approved PR and carries out the deploy — this is where the real changes start happening. |
| `DeployTail` | Cleans up after the deploy: rolls the merge back if it didn't finish cleanly, otherwise mirrors the result to the forge. |
Not every operation uses every step — a rebuild looks roughly like
`MetaSync → Prebuild → StopForUpdate → Swap → RebuildBookkeeping →
Reconcile`; a plain stop or start is much shorter (`Reconcile` alone, or
`Signal → Drain → Reconcile` for a graceful one). A step that isn't
needed for a given run shows as `·` (skipped) rather than being left out
of the tree, so the shape stays recognizable across different operations.
## Where this shows up
- **BU1LDS → R3BU1LD QU3U3** (single hive): the primary place to watch
this. Includes a live build-log panel for whatever `Prebuild`/`Swap`
step is currently running.
- **swarm-ui → `/jobs`**: the same graph, viewed across every hive in the
swarm.
- The dashboard's per-agent status pill lights up while any step that
touches that agent's container is in flight — you don't need the queue
page open to notice a rebuild is happening.
A step that isn't needed for a given run shows as `·` rather than being
left out of the tree entirely, so the same kind of operation keeps a
recognizable shape run to run, whichever steps it actually needed.