# The job queue, for operators

Every container operation — rebuild, first-spawn, a config-PR deploy,
power changes — runs through one shared job queue. This page explains what
you see when you watch it: the **BU1LDS** page's R3BU1LD QU3U3 (see
[`web-ui/dashboard.md`](web-ui/dashboard.md)) and swarm-ui's `/jobs` page
both render the same underlying graph. For the engineering internals
(scheduler, leases, resource windows) see [`coordinator.md`](coordinator.md)
instead — this page stays at the level of "what does this step mean," not
"how is it implemented."

## Operations and steps

Each **row** in the queue view is one operation you asked for (or that the
system triggered on your behalf, e.g. a config-lock bump cascading into a
rebuild). The rows nested under it are that operation's **steps**, run in
order (occasionally with a couple of steps running side by side). A step
shows one of:

| Glyph | Meaning                                              |
| ----- | ----------------------------------------------------- |
| `⏸`   | queued, waiting its turn                               |
| `▶`   | running                                                |
| `◐`   | its own work is done, waiting on a step nested under it |
| `✔`   | finished successfully                                  |
| `✖`   | failed                                                 |
| `⊘`   | cancelled                                               |
| `·`   | skipped (not needed for this run)                      |

A step's label is a short, fixed word — `Prebuild`, `Swap`, `Reconcile`,
and so on. The table below is what each one means, in plain terms.

## What each step does

| Label                | What it means                                                                                             |
| --------------------- | ---------------------------------------------------------------------------------------------------------- |
| `MetaSync`            | Refreshes the agent's config from the shared repo before building — housekeeping, not user-visible work.  |
| `Prebuild`            | Builds the new version of the container **in the background** while the current one keeps running. Nothing is disrupted yet — this is where a build failure shows up, before anything is touched. |
| `StopForUpdate`       | Stops the container so the new version can be put in place. Skipped if it was already stopped.            |
| `Swap`                | Switches the container over to the version built by `Prebuild`. Fast, since the build already happened.   |
| `RebuildBookkeeping`  | Records that the rebuild succeeded — updates the version marker, syncs forge/matrix accounts, notifies other agents. |
| `Reconcile`           | Brings the container to whatever state it's supposed to be in (running or stopped) and makes sure it actually got there. Runs at the end of nearly every operation as a final "did it work?" check. |
| `Create`              | First-time creation of a brand-new container.                                                              |
| `WriteDropin`         | Applies the container's resource limits (CPU/memory) and any nspawn flag changes.                          |
| `WritePermFile`       | Applies a permission change — which tool groups or capabilities an agent has.                              |
| `Reparent`            | Moves one or more agents to a new parent in the topology tree.                                             |
| `MetaLock`            | Bumps the shared configuration lock file every agent's build is derived from. Can cascade into a rebuild for every agent affected by the bump — you'll see those appear as extra rows under the same operation. |
| `AgentWindow`         | Groups one agent's rebuild steps together — no visible work of its own, just a wrapper row.                |
| `Signal`              | Asks a running agent to finish its current turn before being stopped (a graceful stop).                    |
| `Drain`               | Waits for that agent to confirm it's done, up to a few minutes, before continuing.                         |
| `DeployWindow`        | The starting point of an approved config-PR deploy — no visible work, just where the deploy begins.        |
| `MergeVerify`         | Double-checks the approved PR hasn't changed since it was reviewed, right before merging it.                |
| `DeployApply`         | Merges the approved PR and carries out the deploy — this is where the real changes start happening. |
| `DeployTail`          | Cleans up after the deploy: rolls the merge back if it didn't finish cleanly, otherwise mirrors the result to the forge. |

Not every operation uses every step — a rebuild looks roughly like
`MetaSync → Prebuild → StopForUpdate → Swap → RebuildBookkeeping →
Reconcile`; a plain stop or start is much shorter (`Reconcile` alone, or
`Signal → Drain → Reconcile` for a graceful one). A step that isn't
needed for a given run shows as `·` (skipped) rather than being left out
of the tree, so the shape stays recognizable across different operations.

## Where this shows up

- **BU1LDS → R3BU1LD QU3U3** (single hive): the primary place to watch
  this. Includes a live build-log panel for whatever `Prebuild`/`Swap`
  step is currently running.
- **swarm-ui → `/jobs`**: the same graph, viewed across every hive in the
  swarm.
- The dashboard's per-agent status pill lights up while any step that
  touches that agent's container is in flight — you don't need the queue
  page open to notice a rebuild is happening.