vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

job_queue: declare a node's resources where the node is constructed

Browse source 
Resources were derived from the node's kind: `templates::node` called
`NodeKind::resource_deps()`, which fanned out to `needs_build_slot` /
`needs_lease` / `needs_meta_window`. That made the requirement a property
of the *kind*, so a kind that happened to run under an ancestor already
holding the resource could get away with declaring nothing.

Three did. `Start`, `Stop` and `PostSwap` appear in none of the three
predicates, and that was only safe because one construction site fans
them out from inside a lease-holding `Reconcile` — a fact about today's
DAG shape, not about the nodes.

Each of the 41 construction sites now says what it holds. `Start` /
`Stop` / `PostSwap` declare the agent lease; per the contract that is a
re-entrant borrow, which a new test pins rather than argues.

`running_transients` reads the node's declared deps instead of
re-deriving from the kind. That closes the blank-pill gap: the pill went
blank during container start, stop and the post-swap tail because the
declaration was missing, not because the filter was wrong.

The deleted predicates carried the only written record of three design
decisions; each moved to the `Resource` variant it constrains rather than
dying with its function.
This commit is contained in:
atlas 2026-08-02 15:57:51 +02:00 • committed by mara
commit 10dbdb444d
9 changed files with 256 additions and 177 deletions
Download patch file Download diff file Expand all files Collapse all files

86
hive-c0re/src/job_queue/model.rs
View file

@ -21,7 +21,8 @@ use hive_jobq::TerminalState;
/// The primitive operations — each kind maps to one executor fn in
/// `exec.rs`, a thin wrapper over existing `lifecycle.rs` / `meta.rs`
/// code. Concurrency is gated by two resource classes (see
/// [`NodeKind::needs_build_slot`] / [`NodeKind::needs_lease`]); the
/// [`Resource`](super::resource::Resource), declared per node where the node
/// is constructed rather than derived from its kind); the
/// meta *repo* is serialized by `meta::META_LOCK` inside the wrapped
/// functions themselves, which is why there is no `GitCommit` node —
/// a standalone commit node would open a dirty-working-tree window
@ -62,8 +63,10 @@ pub enum NodeKind {
/// swap succeeded; the tail `Reconcile` deps `AfterAny(PostSwap)`, so on
/// swap failure this node is cancel-cascaded (a terminal state) and
/// recovery still runs. Store/forge/matrix work only — no nix build, so
/// build-slot-exempt; the agent lease taken at `Swap` is held across the
/// whole chain until `Reconcile` settles, so it's not re-declared here.
/// build-slot-exempt. It *does* declare the agent lease: an ancestor in the
/// stop chain already holds it, so this is a re-entrant borrow rather than a
/// second unit — declaring it keeps the requirement true of this node rather
/// than of the one DAG shape it happens to be used in.
PostSwap { agent: String },
/// First-spawn pre-create provisioning: proposed/applied repos,
/// state subvolume, and meta registration (`sync_agents`). Runs
@ -124,7 +127,7 @@ pub enum NodeKind {
/// Topology move(s) — `set-parent` (len 1) or `set-parent-bulk` (len N) —
/// as a single queue node. Agentless like [`NodeKind::MetaLock`]: a
/// reparent touches the meta repo, not any one container, and a bulk
/// move spans multiple agents anyway. `needs_meta_window() = true`, same
/// move spans multiple agents anyway. Declares the meta window, same
/// precedent as [`NodeKind::WritePermFile`] (also a small
/// git-commit-under-`META_LOCK` op) — a reparent's commit must not land
/// inside another node's staged deploy `prepare_deploy`→`finalize_deploy`
@ -347,42 +350,6 @@ impl NodeKind {
}
}
/// Nix-heavy kinds hold one of the `buildSlots` semaphore permits
/// for the node's duration.
pub fn needs_build_slot(&self) -> bool {
matches!(
self,
NodeKind::Prebuild { .. }
| NodeKind::Swap { .. }
| NodeKind::Create { .. }
| NodeKind::MetaLock { .. }
| NodeKind::DeployWindow { .. }
)
}
/// Container-affecting kinds require the DAG to hold the agent's
/// lifecycle lease (acquired at the first such node, held until the
/// DAG is terminal). Lease-exempt kinds (`MetaSync`, `Prebuild`,
/// `Provision`, `MetaLock`, `WritePermFile`, `Reparent`) touch the store / meta repo, not the
/// running container — which is exactly why a `Prebuild` can overlap
/// another DAG's work on the same agent. `Provision` precedes the
/// container's existence entirely, so the lease is first taken at the
/// `Create` node it feeds.
pub fn needs_lease(&self) -> bool {
matches!(
self,
NodeKind::Swap { .. }
| NodeKind::Create { .. }
| NodeKind::Reconcile { .. }
| NodeKind::StopForUpdate { .. }
| NodeKind::Signal { .. }
| NodeKind::Drain { .. }
| NodeKind::WriteDropin { .. }
| NodeKind::DeployWindow { .. }
| NodeKind::SetWanted { .. }
)
}
/// Whether running this node is *expected* to take the agent's container
/// down. Feeds `TransientState::deliberate_stop`, which the crash watcher
/// reads to tell an intentional stop from a crash.
@ -420,45 +387,6 @@ impl NodeKind {
// their own answer.
// - `DeployWindow` brackets a deploy without itself stopping anything.
}
/// Kinds that **mutate the meta repo** and so must hold the global
/// [`Resource::MetaWindow`](super::resource::Resource::MetaWindow) for
/// their duration: no two meta mutations may interleave, because a commit
/// landing inside another node's staged `prepare_deploy`→`finalize_deploy`
/// window would sweep the staged `flake.lock` into its own commit and
/// neuter `abort_deploy`.
///
/// This is the queue-primitive replacement for the former runtime
/// `meta::exclusive()` mutex — same global serialisation, but held by the
/// scheduler and therefore able to span a whole subtree, which a
/// `MutexGuard` cannot.
///
/// Note what is **not** here: [`NodeKind::Prebuild`]. The window must stay
/// off the multi-minute toplevel build, which only *reads* the store — the
/// old mutex was scoped to drop before it, and holding a cap-1 global
/// across it would serialize every agent's rebuild behind every other's.
/// That is why the meta preamble is its own [`NodeKind::MetaSync`] node,
/// and why that node is a sibling rather than `Prebuild`'s parent (a
/// resource held by a parent covers its whole subtree).
///
/// Two of these kinds run *inside* a [`NodeKind::DeployWindow`]'s subtree
/// (the appended rebuild's `MetaSync`, and [`NodeKind::FinalizeDeploy`]).
/// They still declare the window: a descendant re-enters an ancestor's hold
/// through the crate's recursive lock, exactly as `Start` / `Stop` re-enter
/// a `Reconcile`'s agent lease. Declaring it is what keeps the requirement
/// true of the *node* rather than of one particular DAG shape.
pub fn needs_meta_window(&self) -> bool {
matches!(
self,
NodeKind::MetaSync { .. }
| NodeKind::Provision { .. }
| NodeKind::MetaLock { .. }
| NodeKind::WritePermFile { .. }
| NodeKind::Reparent { .. }
| NodeKind::DeployWindow { .. }
| NodeKind::FinalizeDeploy { .. }
)
}
}
/// Submit-time spec for a whole DAG: the group's metadata plus the declared —