hyperhive/docs/jobq.md

The job queue, for operators

Every container operation — rebuild, first-spawn, a config-PR deploy, power changes — runs through one shared job queue. This page explains what you see when you watch it: the BU1LDS page's R3BU1LD QU3U3 (see web-ui/dashboard.md) and swarm-ui's /jobs page both render the same underlying graph. For the engineering internals (scheduler, leases, resource windows) see coordinator.md instead — this page stays at the level of "what does this step mean," not "how is it implemented."

Operations and steps

Each row in the queue view is one operation you asked for (or that the system triggered on your behalf, e.g. a config-lock bump cascading into a rebuild). The rows nested under it are that operation's steps, run in order (occasionally with a couple of steps running side by side). A step shows one of:

Glyph	Meaning
⏸	queued, waiting its turn
▶	running
◐	its own work is done, waiting on a step nested under it
✔	finished successfully
✖	failed
⊘	cancelled
·	skipped (not needed for this run)

A step's label is a short, fixed word — Prebuild, Swap, Reconcile, and so on. The table below is what each one means, in plain terms.

What each step does

Label	What it means
MetaSync	Refreshes the agent's config from the shared repo before building — housekeeping, not user-visible work.
Prebuild	Builds the new version of the container in the background while the current one keeps running. Nothing is disrupted yet — this is where a build failure shows up, before anything is touched.
StopForUpdate	Stops the container so the new version can be put in place. Skipped if it was already stopped.
Swap	Switches the container over to the version built by Prebuild. Fast, since the build already happened.
RebuildBookkeeping	Records that the rebuild succeeded — updates the version marker, syncs forge/matrix accounts, notifies other agents.
Reconcile	Brings the container to whatever state it's supposed to be in (running or stopped) and makes sure it actually got there. Runs at the end of nearly every operation as a final "did it work?" check.
Create	First-time creation of a brand-new container.
WriteDropin	Applies the container's resource limits (CPU/memory) and any nspawn flag changes.
WritePermFile	Applies a permission change — which tool groups or capabilities an agent has.
Reparent	Moves one or more agents to a new parent in the topology tree.
MetaLock	Bumps the shared configuration lock file every agent's build is derived from. Can cascade into a rebuild for every agent affected by the bump — you'll see those appear as extra rows under the same operation.
AgentWindow	Groups one agent's rebuild steps together — no visible work of its own, just a wrapper row.
Signal	Asks a running agent to finish its current turn before being stopped (a graceful stop).
Drain	Waits for that agent to confirm it's done, up to a few minutes, before continuing.
DeployWindow	The starting point of an approved config-PR deploy — no visible work, just where the deploy begins.
MergeVerify	Double-checks the approved PR hasn't changed since it was reviewed, right before merging it.
DeployApply	Merges the approved PR and carries out the deploy — this is where the real changes start happening.
DeployTail	Cleans up after the deploy: rolls the merge back if it didn't finish cleanly, otherwise mirrors the result to the forge.

Not every operation uses every step — a rebuild looks roughly like MetaSync → Prebuild → StopForUpdate → Swap → RebuildBookkeeping → Reconcile; a plain stop or start is much shorter (Reconcile alone, or Signal → Drain → Reconcile for a graceful one). A step that isn't needed for a given run shows as · (skipped) rather than being left out of the tree, so the shape stays recognizable across different operations.

Where this shows up

• BU1LDS → R3BU1LD QU3U3 (single hive): the primary place to watch this. Includes a live build-log panel for whatever Prebuild/Swap step is currently running.
• swarm-ui → /jobs: the same graph, viewed across every hive in the swarm.
• The dashboard's per-agent status pill lights up while any step that touches that agent's container is in flight — you don't need the queue page open to notice a rebuild is happening.