job_queue: trim NodeKind/Resource doc comments now that coordinator.md covers them

docs/scheduler/coordinator.md

@@ -44,7 +44,7 @@ Nix-heavy — hold one of the `buildSlots` permits for the node's duration:
 
 | Node             | Wraps                                                                                                                                          |
 | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
-| `Prebuild`       | `lifecycle::prebuild_toplevel` — build the toplevel out-of-band while the container keeps serving (its meta preamble is the upstream `MetaSync` node) |
+| `Prebuild`       | `lifecycle::prebuild_toplevel` — build the toplevel out-of-band while the container keeps serving (its meta preamble is the upstream `MetaSync` node). Skipped when the container is already down; `Swap` builds inline instead |
 | `Swap`           | drop-in rewrite + `nixos-container update` profile-swap (requires the container stopped); the post-swap bookkeeping tail lives in the sibling `RebuildBookkeeping` node |
 | `Create`         | first-spawn `nixos-container create` proper; assumes the upstream `Provision` node already registered the agent in meta                        |
 | `MetaLock`       | meta flake lock bump (`lock_update` / boot-sweep `lock_update_hyperhive`, commit fused — see below); fans out child `Rebuild` DAGs on completion |
@@ -56,22 +56,23 @@ Cheap — no build slot:
 | Node            | Behavior                                                                                                                             |
 | --------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
 | `MergeVerify`   | the deploy's pre-merge gate — PR-head drift check, fetch, `verify_commit` eval. Mutates nothing, so a rejection here needs no compensation |
-| `DeployTail`    | the deploy's `AfterAny` compensation + bookkeeping tail — rolls `applied/main` back from the parked `refs/hyperhive/rollback/<id>` and aborts the staged meta lock when the deploy never confirmed good, then mirrors the config repo to the forge. Infallible by construction |
+| `DeployTail`    | the deploy's `AfterAny` compensation + bookkeeping tail: (1) rolls `applied/main` back from the parked `refs/hyperhive/rollback/<id>` and aborts the staged meta lock when the deploy never confirmed good; (2) mirrors whichever deploy tag landed to the forge config repo, always, best-effort; (3) posts the failing build log back onto the config PR when the deploy failed. Named for (2)/(3), which run on the success path too — not `AbortDeploy`. Infallible by construction |
 | `MetaSync`      | the rebuild's meta preamble — rebuild-dir prep, idempotent meta `sync_agents`, optional per-agent relock. Holds the `MetaWindow` resource (below); deliberately its own node so the window never covers `Prebuild`'s multi-minute build |
 | `Provision`     | first-spawn pre-create provisioning — proposed/applied repos, state subvolume, meta registration (`sync_agents`); runs ahead of `Create` so the `nixos-container create --flake meta#<name>` ref resolves. Store/meta-only, no container yet |
 | `Reconcile`     | idempotent power converge: read `wanted` (below) + observed state; start if `Up` & down (cold-start fallback included), stop if `Offline` & up, else noop |
 | `Start`         | mechanical container start — runtime dir + drop-ins, `start_with_fallback`, MCP listener registration, the manager kick. Fanned out by a `Reconcile` that observed `wanted = Up` and the container down |
 | `Stop`          | mechanical container stop — `nixos-container` kill, MCP listener unregister, the `Killed` manager notify. Fanned out by a `Reconcile` that observed `wanted = Offline` and up |
 | `StopForUpdate` | mechanical `nixos-container stop` for the profile swap; never touches `wanted`; noop if already stopped                               |
-| `RebuildBookkeeping` | the swap's Ok-only bookkeeping tail — rev marker, forge/matrix sync, manager kick, rescan, meta-inputs snapshot; `AfterOk(Swap)` so it runs only on a successful swap (the `Rebuilt` manager event is emitted by the DAG's `EmitRebuilt` tail node, not here) |
+| `RebuildBookkeeping` | the swap's Ok-only bookkeeping tail — rev marker, forge/matrix sync, manager kick, rescan, meta-inputs snapshot; `AfterOk(Swap)` so it runs only on a successful swap (the `Rebuilt` manager event is emitted by the DAG's `EmitRebuilt` tail node, not here). Split out of `Swap` for dashboard visibility + retry granularity, declares no resources of its own — a coordinated child of the `AgentWindow` brace |
 | `AgentWindow`   | pure resource holder — the brace for one agent's rebuild. Declares the build slot + agent lease atomically and holds both for its whole subtree, so `Prebuild` and the `Signal`→`Drain` quiesce window run concurrently instead of one nested under the other. Performs no work; see _Braces_ |
 | `Signal`        | set the graceful fence + kick, so the harness runs one stop-checkpoint turn                                                           |
 | `Drain`         | await the harness clearing the fence, bounded by the 3-min graceful-stop timeout; resolves ok either way                              |
-| `PauseSignal`   | write the pause marker + mark `pause_pending`. No kick, unlike `Signal` — the harness's own between-turns poll is already responsive enough |
+| `PauseSignal`   | write the pause marker + mark `pause_pending`. No kick, unlike `Signal` — the harness's between-turns poll is already responsive enough, and `Signal`'s kick-message body ("you were just (re)started") would be actively misleading here |
 | `PauseDrain`    | await the harness reporting `PauseAcknowledged`, bounded timeout; best-effort like `Drain`                                            |
 | `DestroyContainer` | `nixos-container destroy` + un-registration (drop from the roster, clear the ephemeral runtime dir). Runs downstream of a `Stop`, so deliberately excluded from `takes_container_down` — the container is already down by the time it claims |
 | `PurgeState`    | the `purge = true` half of a destroy: delete the agent's state subvolume (via hive-priv) plus its state/applied dirs. Own node because it's conditional and the irreversible step |
-| `DestroyBookkeeping` | the post-destroy tail — meta sync, fail pending approvals, drop the power intent, notify the manager, rescan, re-emit the tombstone, resync tmpfiles. Same split rationale as `RebuildBookkeeping`/`Swap` |
+| `DestroyBookkeeping` | the post-destroy tail — meta sync, fail pending approvals, drop the power intent, notify the manager, rescan, re-emit the tombstone, resync tmpfiles. Same split rationale as `RebuildBookkeeping`/`Swap`. Its `purge` flag only selects the wording of the approval-failure reason and the manager notification — the destructive work is `PurgeState`'s |
+| `SetWanted`     | write the durable power intent (`wanted = Up`/`Offline`) as the head node of a power-op DAG, replacing the old pre-submit side effect. Takes the agent lease even though it's a store write, so the intent write and the tail `Reconcile` are atomic per-agent — two racing power ops can't clobber each other's intent before either reconciles |
 | `FinalizeDeploy` | deploy phase 3 — drop the rollback ref, plant `deployed/<id>`, commit the staged `flake.lock`. The first two git steps are fatal on purpose, so a confirmed-good deploy's outcome and the repo's state can't disagree |
 | `ResolveApproval` | tail of an approval-carrying DAG — resolve the approval row from how the work ended (`AfterAny`, one node emitted per outcome). Agentless: the approval row already names its agent |
 | `EmitRebuilt`   | tail of a rebuild/perm-change — emit the agent's `Rebuilt` manager event (ok/fail per outcome, nothing on cancel). One node per agent *and* per outcome |
@@ -220,9 +221,9 @@ resources are free. Resources:
    (`SetWanted` is a store write, not a container op, but takes the lease anyway
    so a power-op DAG's intent write + reconcile is atomic — two racing ops can't
    clobber intent before either reconciles.) **Lease-exempt**: `MetaSync`,
-   `Prebuild`, `MetaLock`, `WritePermFile`, `Reparent` — they touch the store /
+   `Prebuild`, `Provision`, `MetaLock`, `WritePermFile`, `Reparent` — they
-   meta, not the running container, which is exactly why a stop can land while
+   touch the store / meta, not the running container, which is exactly why a
-   another DAG's prebuild is still building. Also exempt, for a different
+   stop can land while another DAG's prebuild is still building. Also exempt, for a different
    reason, are the rebuild subtree's own members (`StopForUpdate`, `Swap`,
    `Signal`, `Drain`, `RebuildBookkeeping`): they genuinely do touch the
    container, but their `AgentWindow` brace holds the lease above them — see
@@ -306,6 +307,13 @@ compensation step that survives a hive-c0re restart — the pre-merge
 local variable, so `DeployTail` can still undo a half-finished deploy after a
 crash.
 
+`DeployWindow` declares all three resources (build slot, lease, meta window)
+on itself rather than letting each phase declare its own, because the queue
+acquires a node's resources atomically (all-or-nothing): a child that took
+the build slot while its parent held the meta window could block waiting for
+a resource its own parent already committed to, a lock-ordering hazard that
+one multi-resource root avoids by construction.
+
 `Spawn` and `UpdateMetaInputs` approvals map onto the ordinary `spawn` /
 `meta-update` shapes. The scheduler fires `actions::resolve_approval_dag`
 exactly once when **any** approval-carrying DAG settles terminal — deploys
@@ -427,8 +435,9 @@ Sequence for a rebuild DAG (each step is its own queue node):
    The container keeps serving the previous generation while eval + fetch +
    build happen out-of-band. `nixos-container update` then finds the result
    cached and skips straight to the profile-swap. Build failures surface
-   here, before the running container is touched. (Runs even for a stopped
+   here, before the running container is touched. Skipped when the
-   container — same total nix work, one uniform DAG shape.)
+   container is already down — a stopped agent has no uptime to preserve,
+   so `Swap` builds inline instead of paying the double eval.
 3. `StopForUpdate` — bring the container down (noop when already stopped).
 4. `Swap` — `nixos-container update --flake meta#<name>` profile-swap
    (near-instant after the prebuild).

hive-c0re/src/job_queue/model.rs

@@ -26,118 +26,59 @@ use hive_jobq::TerminalState;
 /// functions themselves, which is why there is no `GitCommit` node —
 /// a standalone commit node would open a dirty-working-tree window
 /// between nodes that the fused `meta.rs` ops deliberately close.
+///
+/// What each kind does, the resource/lease rules, and the DAG shapes they
+/// compose into live in the node-inventory table and surrounding sections
+/// of `docs/scheduler/coordinator.md` — this enum is deliberately not a
+/// second copy of that; each variant below gets a one-line pointer.
 #[derive(Debug, Clone, PartialEq, Eq, Serialize)]
 #[serde(tag = "kind", rename_all = "snake_case")]
 pub enum NodeKind {
-    /// The rebuild's meta-repo preamble: `lifecycle::prepare_rebuild_dirs`,
+    /// The rebuild's meta-repo preamble. `relock = false` only for
-    /// an idempotent meta `sync_agents`, and an optional per-agent relock.
+    /// meta-update cascade rebuilds (re-locking would revert the bump the
-    /// `relock = false` only for meta-update cascade rebuilds (re-locking
+    /// cascade just committed).
-    /// would revert the bump the cascade just committed).
-    ///
-    /// Its own node — ahead of, and *not* an ancestor of, [`NodeKind::Prebuild`]
-    /// — precisely because it is the only part of the rebuild that mutates the
-    /// meta repo and so holds the global
-    /// [`Resource::MetaWindow`](super::resource::Resource::MetaWindow). Fusing
-    /// it into `Prebuild` (or making it `Prebuild`'s parent, which holds a
-    /// resource across the whole subtree) would extend that global window over
-    /// the multi-minute toplevel build and serialize rebuilds hive-wide.
-    /// Store/meta work only — build-slot- and lease-exempt.
     MetaSync { agent: String, relock: bool },
-    /// Out-of-band toplevel build while the container keeps serving:
+    /// Out-of-band toplevel build while the container keeps serving.
-    /// `lifecycle::prebuild_toplevel`, reading a meta repo the upstream
-    /// [`NodeKind::MetaSync`] has already synced. The warm build is skipped
-    /// when the container is already down — it only exists to shrink the
-    /// swap's downtime, which a stopped agent doesn't need (`Swap` builds
-    /// inline instead).
     Prebuild { agent: String },
-    /// `nixos-container update` profile-swap (requires the container
+    /// `nixos-container update` profile-swap.
-    /// stopped). Re-applies nspawn flags + resource limits first —
-    /// rebuild is the reconcile verb. The post-rebuild bookkeeping tail
-    /// lives in the sibling `RebuildBookkeeping` node.
     Swap { agent: String },
-    /// The post-`Swap` bookkeeping tail as a first-class node: rev marker,
+    /// The post-`Swap` bookkeeping tail as a first-class node.
-    /// forge + matrix sync, manager kick, container rescan, meta-inputs
-    /// snapshot. Split out of `Swap` for dashboard visibility + retry
-    /// granularity. Deps `AfterOk(Swap)`, so it runs only when the profile
-    /// swap succeeded; the tail `Reconcile` deps `AfterAny(RebuildBookkeeping)`, so on
-    /// swap failure this node is cancel-cascaded (a terminal state) and
-    /// recovery still runs. Store/forge/matrix work only — no nix build.
-    ///
-    /// Declares **no resources**: it is a coordinated child of
-    /// [`NodeKind::AgentWindow`], which holds the agent lease (and the build
-    /// slot) for the whole rebuild subtree. See `templates.rs`'s module doc for
-    /// why the brace is the one place a resource is declared on behalf of
-    /// others.
     RebuildBookkeeping { agent: String },
-    /// First-spawn pre-create provisioning: proposed/applied repos,
+    /// First-spawn pre-create provisioning.
-    /// state subvolume, and meta registration (`sync_agents`). Runs
-    /// ahead of `Create` so the `nixos-container create --flake
-    /// meta#<name>` ref resolves. Store/meta-only — no container yet —
-    /// so it's lease- and build-slot-exempt like `Prebuild`.
     Provision { agent: String },
     /// First-spawn `nixos-container create` proper. Assumes the
     /// upstream `Provision` node already registered the agent in meta.
     Create { agent: String },
-    /// `nixos-container destroy` plus the un-registration that follows it:
+    /// `nixos-container destroy` plus the un-registration that follows it.
-    /// drop the agent from the coordinator's roster and clear its ephemeral
-    /// runtime dir.
     ///
-    /// **Deliberately not in [`NodeKind::takes_container_down`]**, and that
+    /// Deliberately excluded from [`NodeKind::takes_container_down`] — see
-    /// is the design rather than an oversight. This node runs *downstream of
+    /// that method for why a node downstream of a `Stop` must not claim the
-    /// a `Stop`*, which already carries the flag honestly, so by the time it
+    /// container going down as its own.
-    /// claims there is nothing left to take down. A container still live here
-    /// is a real bug and must page someone — a `true` would absorb exactly
-    /// that signal, and the flag's whole asymmetry (see that method) is that
-    /// a wrong `true` silently swallows a crash.
     DestroyContainer { agent: String },
     /// The `purge = true` half of a destroy: delete the agent's state
     /// subvolume (via hive-priv, since a subvolume root defeats
-    /// `remove_dir_all`) plus its state and applied dirs. Its own node
+    /// `remove_dir_all`) plus its state and applied dirs.
-    /// because it is conditional — a plain destroy never inserts it — and
-    /// because it is the irreversible step, so it earns a distinct row in
-    /// the graph rather than hiding inside a bookkeeping tail.
     PurgeState { agent: String },
-    /// The post-destroy bookkeeping tail: meta sync, fail the agent's pending
+    /// The post-destroy bookkeeping tail.
-    /// approvals, drop the durable power intent, notify the manager, rescan
-    /// containers, re-emit the tombstone + schedule snapshots, resync
-    /// tmpfiles. Split from [`NodeKind::DestroyContainer`] for the same
-    /// reason [`NodeKind::RebuildBookkeeping`] is split from `Swap`:
-    /// dashboard visibility and retry granularity for work that is pure
-    /// store/meta bookkeeping and touches no container.
-    ///
-    /// `purge` only selects the wording of the approval-failure reason and
-    /// the manager notification; the destructive work is `PurgeState`'s.
     DestroyBookkeeping { agent: String, purge: bool },
-    /// Meta flake lock bump. `sweep = false`: `meta::lock_update`
+    /// Meta flake lock bump. `sweep = false`: `meta::lock_update` with this
-    /// (commit fused, under `META_LOCK`) with this node's own `inputs`;
+    /// node's own `inputs`; `sweep = true`: `meta::lock_update_hyperhive`
-    /// `sweep = true`: `meta::lock_update_hyperhive`, *non-fatal* (a
+    /// (boot sweep, `inputs` empty). See _Notable collapses_ for how the
-    /// failed boot-time bump must not cancel the fan-out rebuilds).
+    /// scheduler appends the fan-out `Rebuild` DAGs on completion.
-    /// On success the scheduler appends child `Rebuild` DAGs: the
-    /// precomputed `fanout` list when present (boot sweep), else the
-    /// post-bump affected set (`meta_update_cascade_agents`).
-    ///
-    /// `inputs` are the flake inputs to bump — empty means "all", and the
-    /// boot sweep leaves them empty since it bumps `hyperhive` alone. They
-    /// ride this node because it is the only thing that reads them.
     MetaLock {
         sweep: bool,
         fanout: Option<Vec<String>>,
         inputs: Vec<String>,
     },
-    /// Idempotent power converge *planner*: read `wanted` + observed
+    /// Idempotent power converge *planner*: fans a child
-    /// state and decide the action (start if `Up` & down, stop if
+    /// [`NodeKind::Start`] / [`NodeKind::Stop`] out at runtime rather than
-    /// `Offline` & up, else noop). The mechanical work is not done in
+    /// doing the mechanical work itself.
-    /// this node — it fans a child [`NodeKind::Start`] / [`NodeKind::Stop`]
-    /// DAG out at runtime so the sub-step is a first-class DAG node.
     Reconcile { agent: String },
-    /// Mechanical container start: the start preamble (runtime dir +
+    /// Mechanical container start. Fanned out by a [`NodeKind::Reconcile`]
-    /// drop-ins), `start_with_fallback`, MCP listener registration, and
+    /// that observed `wanted = Up` and the container down.
-    /// the manager kick. Fanned out by a [`NodeKind::Reconcile`] that
-    /// observed `wanted = Up` and the container down.
     Start { agent: String },
-    /// Mechanical container stop: `nixos-container` kill, MCP listener
+    /// Mechanical container stop. Fanned out by a [`NodeKind::Reconcile`]
-    /// unregister, and the `Killed` manager notify. Fanned out by a
+    /// that observed `wanted = Offline` and up.
-    /// [`NodeKind::Reconcile`] that observed `wanted = Offline` and up.
     Stop { agent: String },
     /// Mechanical `nixos-container stop` for the profile swap. Never
     /// touches `wanted`. Noop if already stopped.
@@ -150,10 +91,7 @@ pub enum NodeKind {
     /// downstream `Reconcile` performs the actual stop.
     Drain { agent: String },
     /// Write the pause marker (`Coordinator::set_paused`) + mark
-    /// `pause_pending`. No kick, unlike `Signal` — the harness's own
+    /// `pause_pending`.
-    /// between-turns poll (`PAUSE_POLL`, 1s default) is already
-    /// responsive enough, and `Signal`'s kick-message body ("you were
-    /// just (re)started") would be actively misleading here.
     PauseSignal { agent: String },
     /// Await the harness reporting `PauseAcknowledged`, bounded by a
     /// timeout. Resolves ok either way — pausing is best-effort from
@@ -161,197 +99,65 @@ pub enum NodeKind {
     PauseDrain { agent: String },
     /// `set_nspawn_flags` + `set_resource_limits` + daemon-reload.
     WriteDropin { agent: String },
-    /// Commit `tool-groups.json` / `capabilities.json` per its `payload`
+    /// Commit `tool-groups.json` / `capabilities.json` per its `payload`.
-    /// (commit fused under `META_LOCK`). The payload rides this node — the only
-    /// consumer — rather than the generic DAG container.
     WritePermFile { agent: String, payload: PermPayload },
     /// Topology move(s) — `set-parent` (len 1) or `set-parent-bulk` (len N) —
-    /// as a single queue node. Agentless like [`NodeKind::MetaLock`]: a
+    /// as a single queue node. `moves` is typed `(Ident, Option<Ident>)`
-    /// reparent touches the meta repo, not any one container, and a bulk
+    /// pairs, applied in order under one `META_LOCK` acquisition.
-    /// move spans multiple agents anyway. Declares the meta window, same
-    /// precedent as [`NodeKind::WritePermFile`] (also a small
-    /// git-commit-under-`META_LOCK` op) — a reparent's commit must not land
-    /// inside another node's staged deploy `prepare_deploy`→`finalize_deploy`
-    /// window. `(child, new_parent)` pairs, applied in order under one
-    /// `META_LOCK` acquisition / one git commit (`meta::bulk_commit_topology`
-    /// handles both the single- and multi-move case uniformly).
     Reparent {
         moves: Vec<(hive_types::Ident, Option<hive_types::Ident>)>,
     },
-    /// Group root of the approval-deploy (`MergeConfigPr`) subtree, and the
+    /// Group root of the approval-deploy (`MergeConfigPr`) subtree — the
-    /// node that **owns the deploy window**. It performs no work of its own —
+    /// **brace** that owns the deploy window. See _Braces_ and _Approvals_.
-    /// it exists so the resources it declares (the global
-    /// [`Resource::MetaWindow`](super::resource::Resource::MetaWindow), the
-    /// agent lease, a build slot) are held continuously across every child
-    /// phase, which a per-node acquisition could not guarantee.
-    ///
-    /// All three resources are declared *here*, on one node, on purpose. The
-    /// queue acquires a node's resources atomically (all-or-nothing), so a
-    /// single multi-resource root can never hold one and block on another —
-    /// whereas letting a child take the build slot while its parent held the
-    /// meta window would introduce exactly that pattern, and with it a
-    /// lock-ordering argument that has to be re-verified on every future edit.
-    /// Cheap, too: the window has to span the container build regardless (see
-    /// [`NodeKind::DeployApply`]), so nothing is over-serialised by hoisting
-    /// the slot and the lease up alongside it.
-    ///
-    /// Carries the approval row every phase below it re-reads, like each of
-    /// those phases does — the id is the node's own payload, not something a
-    /// DAG-level catch-all hands down.
     DeployWindow { agent: String, approval_id: i64 },
-    /// Group root of a rebuild subtree — the **brace**: declares the agent
+    /// Group root of a rebuild subtree — the **brace**, scoped to one
-    /// lease and the build slot, holds both for the whole subtree, and performs
+    /// agent. Why braces exist and what they cost: _Braces_.
-    /// no work of its own. Same pure-resource-holder shape as
-    /// [`NodeKind::DeployWindow`], scoped to one agent.
-    ///
-    /// Its children declare no resources and borrow these grants, which is what
-    /// lets `Prebuild` run beside the `Signal` → `Drain` window. Why braces
-    /// exist and what they cost: `docs/scheduler/coordinator.md`, _Braces_.
     AgentWindow { agent: String },
-    /// Deploy phase 1 — **verify only, mutates nothing.** Drift-gate the
+    /// Deploy phase 1 — **verify only, mutates nothing.**
-    /// approval's PR head, fetch it into the applied repo, and eval-verify the
-    /// merge head. Any failure here aborts the deploy with the forge state
-    /// untouched, so it is safely retryable and cancel-safe: nothing downstream
-    /// has happened yet.
     MergeVerify { agent: String, approval_id: i64 },
-    /// Deploy phase 2 — the irreversible fast-forward plus the *opening* half of
+    /// Deploy phase 2 — the irreversible fast-forward plus the *opening*
-    /// the two-phase meta deploy: park the rollback ref, ff-merge the reviewed
+    /// half of the two-phase meta deploy. Does **not** build the container
-    /// head to `main` via the forge API, ff `applied/main`, and
+    /// itself; grows the ordinary rebuild subgraph in as its own children
-    /// `meta::prepare_deploy` (which stages `flake.lock` uncommitted).
+    /// (see the rebuild-path section's approval-deploy paragraph).
-    ///
-    /// It does **not** run the container rebuild itself. It grows the ordinary
-    /// rebuild subgraph into this DAG as its own children
-    /// ([`super::templates::deploy_rebuild_nodes`], `relock = false` — the lock
-    /// is already staged), so the multi-minute build renders as the same real
-    /// nodes every other rebuild does instead of one opaque box. Closing the
-    /// staged-lock window is likewise its own node
-    /// ([`NodeKind::FinalizeDeploy`]), and the compensation path is
-    /// [`NodeKind::DeployTail`].
     DeployApply { agent: String, approval_id: i64 },
-    /// Deploy phase 3 — close the two-phase meta deploy once the rebuild
+    /// Deploy phase 3 — closes the two-phase meta deploy once
-    /// subgraph under [`NodeKind::DeployApply`] has come up clean: drop the
+    /// [`NodeKind::DeployApply`]'s rebuild subgraph comes up clean.
-    /// rollback ref, plant the `deployed/<id>` tag, commit the staged
-    /// `flake.lock` (`meta::finalize_deploy`).
-    ///
-    /// Its two git steps are **fatal**, deliberately. They are the writes that
-    /// tell [`NodeKind::DeployTail`] a deploy confirmed good, so a node that
-    /// merely warned on them could report success while leaving the tail
-    /// looking at the git state of a failure — and the tail would then roll a
-    /// *good* deploy back. Failing loudly keeps the node's outcome and the
-    /// repo's state saying the same thing.
-    ///
-    /// The trailing `meta::finalize_deploy` stays warn-only: by then the
-    /// container already runs the new config, and an uncommitted staged lock is
-    /// something the operator can commit by hand.
     FinalizeDeploy { agent: String, approval_id: i64 },
-    /// Deploy compensation **and bookkeeping** tail — `AfterAny`
+    /// Deploy compensation **and bookkeeping** tail. See the `DeployTail`
-    /// [`NodeKind::DeployApply`], so it runs on success, failure, and cancel
+    /// node-inventory row for its three responsibilities and why it isn't
-    /// alike, in the same spirit as the rebuild template's tail `Reconcile`
+    /// named `AbortDeploy`.
-    /// ("always runs, decides internally"). It:
-    /// 1. compensates a merge that landed but was never finalized — roll `main`
-    ///    back, reset the tree, `meta::abort_deploy`, plant `failed/<id>`;
-    /// 2. mirrors whichever deploy tag got planted to the forge config repo
-    ///    (`forge::push_config`), always, best-effort;
-    /// 3. posts the failing build log back onto the config PR when the deploy
-    ///    failed, so the manager sees the rejection without leaving the forge.
-    ///
-    /// Steps 2 and 3 are why this is `DeployTail` and not `AbortDeploy`: it has
-    /// work to do on the success path too, and a node name that claims
-    /// otherwise would be a lie on the dashboard.
-    ///
-    /// For (1) it needs no knowledge of how far the deploy got, because that state is
-    /// parked in the applied repo rather than passed between nodes:
-    /// `DeployApply` writes the pre-merge `main` sha to
-    /// `refs/hyperhive/rollback/<approval-id>` before the fast-forward and
-    /// [`NodeKind::FinalizeDeploy`] deletes it. So the ref existing *is*
-    /// the "a merge landed and was not finalized" signal, and its absence makes
-    /// this node a no-op. Parking it in git rather than in a node payload also
-    /// means it survives a `hive-c0re` restart mid-deploy, which an in-memory
-    /// queue does not.
     DeployTail { agent: String, approval_id: i64 },
-    /// Tail node of an approval-carrying DAG (spawn / opaque deploy / config-PR
+    /// Tail node of an approval-carrying DAG (spawn / opaque deploy /
-    /// merge): resolve the approval row from how the work actually ended.
+    /// config-PR merge): resolve the approval row from how the work ended.
-    ///
-    /// Weak-edged (`DepWhen::AFTER_ANY`) like [`NodeKind::DeployTail`], so it runs on
-    /// success, failure **and cancel** alike and decides internally. It reads its
-    /// dependencies' terminal states from the node's dependency list rather than
-    /// re-deriving them from the world the way `DeployTail` reads git: a node is
-    /// *told* how the work it follows ended, it does not go back out and ask.
-    ///
-    /// Agentless on purpose: the approval row already names its agent, so
-    /// carrying one here would be a second copy free to drift. Like
-    /// [`NodeKind::MetaLock`] it reports `""` from [`NodeKind::agent`] and takes
-    /// no lease — which is also what lets one close a multi-agent DAG.
     ResolveApproval {
         approval_id: i64,
-        /// Which outcome this node reports. A template emits **one per outcome**,
+        /// Which outcome this node reports. A template emits **one per
-        /// each edged to accept only that one, so exactly one is ever runnable
+        /// outcome**, each edged to accept only that one, so exactly one is
-        /// and the executor has nothing to decide — it resolves the row the way
+        /// ever runnable. The `Cancelled` one is also the node
-        /// its own variant says. The `Cancelled` one is also the node that
+        /// [`super::JobQueue::cancel`] spares, since its edge is the only
-        /// [`super::JobQueue::cancel`] spares, since its edge is the only one
+        /// one that accepts a dropped dependency.
-        /// that accepts a dropped dependency.
         outcome: TerminalState,
     },
-    /// Tail node of a rebuild / perm-change: emit this agent's `Rebuilt` manager
+    /// Tail node of a rebuild / perm-change: emit this agent's `Rebuilt`
-    /// event — `ok` when its deps are `Done`, `!ok` carrying the failure note when
+    /// manager event. One node per agent *and* per outcome; no cancel
-    /// they `Failed`, and **nothing at all** when they `Cancelled` (a cancelled DAG
+    /// variant (a cancelled DAG never ran, so there is no rebuild to
-    /// never ran, so there is no rebuild to report).
+    /// report).
-    ///
-    /// One node per **agent** — a multi-agent DAG reports each agent's own
-    /// outcome rather than painting all of them with the whole DAG's roll-up —
-    /// and one per **outcome**: `ok` isn't computed here, it's which of the pair
-    /// the graph let run.
-    ///
-    /// No cancel variant, deliberately: a DAG dropped before it started has no
-    /// rebuild to report, and neither tail's edge accepts `Cancelled`, so both
-    /// are cancelled with the rest and nothing is emitted.
     EmitRebuilt { agent: String, ok: bool },
-    /// Write the agent's durable power intent (`wanted = Up` when `up`, else
+    /// Write the agent's durable power intent (`wanted = Up` when `up`,
-    /// `Offline`) as a first-class DAG node, at the head of a power-op
+    /// else `Offline`) as the head node of a power-op DAG.
-    /// template so the downstream `Reconcile` reads it. Replaces the old
-    /// pre-submit `set_wanted` side effect: the intent write is now part of
-    /// the atomic DAG (crash-safe, per-agent — a multi-agent DAG carries one
-    /// `SetWanted` per agent). Build-slot-exempt (a store write), but
-    /// **lease-needing**: it takes the agent's lifecycle lease so the whole
-    /// power-op DAG (intent write → reconcile) is atomic per-agent — two
-    /// racing ops (e.g. restart vs stop) can't clobber each other's intent
-    /// before either reconciles, which is the point of moving the write into
-    /// the DAG. (In `stale_start` the lease is thus held across the head
-    /// `Prebuild`, but that's a no-op there — the agent is down, so prebuild
-    /// is skipped.)
     SetWanted { agent: String, up: bool },
     /// One-shot boot-time forge user/token sweep for every existing
-    /// container (`forge::ensure_all`) — moved out of a bare `tokio::spawn`
+    /// container (`forge::ensure_all`). Agentless.
-    /// so it shows as real work on the dashboard instead of running
-    /// invisibly until it fails. Agentless: it sweeps every container, not
-    /// one. Store/network-I/O only — build-slot- and lease-exempt, same
-    /// class as [`NodeKind::MetaSync`]/[`NodeKind::Provision`].
     ForgeSweep,
-    /// One-shot boot-time matrix user/space sweep (`matrix::ensure_all`),
+    /// One-shot boot-time matrix user/space sweep (`matrix::ensure_all`).
-    /// for the same dashboard-visibility reason as [`NodeKind::ForgeSweep`].
+    /// Agentless.
-    /// The *periodic* re-sweep (every 30 min, recovering token files
-    /// `hive-matrix-daemon` deleted) stays a background loop in
-    /// `main.rs` — only the boot-time instance is a DAG node.
     MatrixSweep,
-    /// One-shot boot-time Forgejo webhook registration, for
+    /// One-shot boot-time Forgejo webhook registration. Agentless.
-    /// `internal/knowledge` (push → git pull) and the `agent-configs` org
-    /// (`pull_request` → config-PR approval). No-op when the core token, hive
-    /// domain, or HMAC secret aren't available yet — mirrors the guard the
-    /// `tokio::spawn` block it replaced already used. Agentless, build-slot-
-    /// and lease-exempt.
     WebhookRegister,
-    /// One-shot boot-time `/knowledge` pull (`knowledge::pull`), reconciling
+    /// One-shot boot-time `/knowledge` pull (`knowledge::pull`). Agentless.
-    /// any commits that landed while `hive-c0re` was down. Same rationale as
-    /// [`NodeKind::MatrixSweep`]: the periodic hourly re-pull stays a
-    /// background loop, only the boot-time instance is a DAG node.
     KnowledgePull,
-    /// One-shot boot-time pull of the agent set the swarm controller declares
+    /// One-shot boot-time pull of the agent set the swarm controller
-    /// for this hive (`wanted::pull`), converging the agents it names.
+    /// declares for this hive (`wanted::pull`). Agentless.
-    ///
-    /// Unlike the three above there is **no** background loop behind this one:
-    /// boot is the whole cadence. The per-agent fast path is the deploy event
-    /// (`swarm_status`), and this is what repairs a missed one. Agentless — it
-    /// reads the whole declaration, not one agent.
     WantedPull,
 }
 

hive-c0re/src/job_queue/resource.rs

@@ -15,46 +15,21 @@
 //! the meta window) cannot deadlock against each other.
 
 /// The two resource classes the queue gates concurrency on, as the crate's
-/// generic resource type `R`.
+/// generic resource type `R`. What each holds, who declares it, and the
+/// exemptions: see `docs/scheduler/coordinator.md`'s _Scheduler semantics_
+/// (build slots, the per-agent lease) and _Two further layers protect the
+/// meta repo_ (the deploy window) sections — this enum stays the one-line
+/// summary, not a second copy.
 #[derive(Debug, Clone, PartialEq, Eq, Hash)]
 pub enum Resource {
     /// One of the `buildSlots` permits, held by a nix-heavy node for its
-    /// duration. Capacity is `services.hyperhive.c0re.buildSlots` (default 1),
+    /// duration.
-    /// set on the [`hive_jobq::resources::ResourceTable`] at construction.
     BuildSlot,
-    /// The per-agent lifecycle lease — globally exclusive per agent across all
+    /// The per-agent lifecycle lease — globally exclusive per agent across
-    /// DAGs (unconfigured, so the crate's default capacity 1 applies). Held by
+    /// all DAGs.
-    /// a DAG's first container-affecting node for that agent and re-entered by
-    /// the rest of that agent's subtree via the crate's recursive lock, so two
-    /// DAGs never interleave container ops on one agent.
-    ///
-    /// Nodes that touch the *store or meta repo* rather than the running
-    /// container do not declare it — `MetaSync`, `Prebuild`, `Provision`,
-    /// `MetaLock`, `WritePermFile`, `Reparent`. That exemption is what lets a
-    /// `Prebuild` overlap another DAG's work on the same agent. `Provision`
-    /// precedes the container's existence entirely, so the lease is first taken
-    /// at the `Create` it feeds.
     Agent(String),
-    /// The meta-repo mutation window — a global singleton (default capacity 1)
+    /// The meta-repo mutation window — a global singleton held by any node
-    /// held by any node that mutates the meta repo, so two meta mutations never
+    /// that mutates the meta repo, so two meta mutations never interleave.
-    /// interleave. That exclusion is load-bearing: a commit landing inside
-    /// another node's staged `prepare_deploy`→`finalize_deploy` window would
-    /// sweep the staged `flake.lock` into its own commit and neuter
-    /// `abort_deploy`. Replaces the former runtime `meta::exclusive()` mutex: a
-    /// `MutexGuard` cannot span scheduler nodes, but a resource held by a
-    /// subtree root *can* — which is what lets the two-phase deploy
-    /// (`prepare_deploy` stages `flake.lock` uncommitted across the whole
-    /// container build, `finalize_deploy`/`abort_deploy` resolve it) be
-    /// decomposed into sub-nodes instead of one opaque node. Descendants of a
-    /// holder re-enter it through the crate's recursive lock, exactly like
-    /// [`Resource::Agent`].
-    ///
-    /// Deliberately **not** declared by `Prebuild`: the window must stay off the
-    /// multi-minute toplevel build, which only *reads* the store. Holding a
-    /// hive-global cap-1 across it would serialize every agent's rebuild behind
-    /// every other's — which is why the meta preamble is its own `MetaSync`
-    /// node, and a sibling of `Prebuild` rather than its parent (a resource is
-    /// held across the holder's whole subtree).
     MetaWindow,
 }