refactor(#2949): kill Declare — a running node declares onto its own builder
A node no longer hands back a recipe for the scheduler to replay later. It declares straight onto a builder it was given, and that builder is inserted as part of completing the node. Deleted: `pub type Declare`, `struct NodeOutput` (+ its hand-written `Debug`), `JobQueue::append_subgraph`. Nothing added to `Dag` / `DagView`. jobq gains `Scheduler::new_job()` (the only way to obtain a `JobBuilder`) and `complete_growing(id, outcome, grown)`, which inserts under `id` and *then* completes it, so a DAG cannot roll terminal while grown work is still pending. `complete()` and `complete_growing()` share a private `finish()` rather than one redirecting through the other. The DAG-gone guard lives beside the graph now, where it cannot be skipped, instead of being a caller-side lookup. The growth executors return data (`run_meta_lock -> (Vec<String>, RebuildOpts)`, `run_reconcile -> Option<NodeKind>`) rather than taking the builder: a `&Job` parameter is live for the whole function body, and `&RefCell<T>` is never `Send`, so an async fn taking one cannot be spawned. `run_node` threads the builder by value and hands it back. A node can now declare work and then fail, which was previously inexpressible. `grown` is dropped in that case — failure cancel-cascades downstream, so inserting it would only add nodes to immediately cancel — and the log line carries `grown_nodes` so the drop is visible.
This commit is contained in:
atlas
mara
parent
2454a1ea6a
commit
82ef06f445
8 changed files with 388 additions and 344 deletions
|
|
@ -55,16 +55,6 @@ use resource::Resource;
|
|||
/// borrowed one; only `hive_jobq` can make or insert it.
|
||||
pub type Job = hive_jobq::JobBuilder<NodeKind, Resource>;
|
||||
|
||||
/// A job's shape as a **recipe**: given a builder, declare the nodes.
|
||||
///
|
||||
/// What a template returns and what an executor hands back, because neither
|
||||
/// can build a job itself — `hive_jobq` creates the builder inside its own
|
||||
/// insertion call and never lets one out. So the transferable thing is the
|
||||
/// declaring closure, and the queue runs it at the moment it inserts.
|
||||
///
|
||||
/// `Send` because an executor's output crosses the scheduler's task boundary.
|
||||
pub type Declare = Box<dyn FnOnce(&Job) + Send>;
|
||||
|
||||
/// A handle to one node a template declared — where its edges, grouping and
|
||||
/// resources are declared. `Copy`; naming a node as a dependency does not
|
||||
/// consume the ability to name it again.
|
||||
|
|
@ -163,6 +153,18 @@ impl Default for JobQueue {
|
|||
}
|
||||
}
|
||||
|
||||
/// A node runner's `Result` as the scheduler's [`Outcome`].
|
||||
///
|
||||
/// The failure reason + `finished_at` are stamped onto the graph `Node` by the
|
||||
/// scheduler (the reason rides `Outcome::Failed`); there is no host-side copy,
|
||||
/// so nothing needs clearing on success.
|
||||
fn outcome_of(result: Result<(), String>) -> Outcome {
|
||||
match result {
|
||||
Ok(()) => Outcome::Done,
|
||||
Err(e) => Outcome::Failed(truncate_error(&e)),
|
||||
}
|
||||
}
|
||||
|
||||
/// Insert a declared `job` into the shared graph and record its per-node
|
||||
/// `node_rt`, returning the inserted ids.
|
||||
///
|
||||
|
|
@ -221,7 +223,7 @@ impl JobQueue {
|
|||
/// roots re-parented to the container). Returns the container's id as the
|
||||
/// DAG id — its rolled-up state is the DAG state.
|
||||
///
|
||||
/// Takes the spec's recipe by generic, not as a boxed [`Declare`]: a spec
|
||||
/// Takes the spec's recipe by generic, not as a boxed closure: a spec
|
||||
/// travels from the template that built it directly into this call, so
|
||||
/// there is nothing to allocate for.
|
||||
///
|
||||
|
|
@ -254,38 +256,6 @@ impl JobQueue {
|
|||
Ok(container.get())
|
||||
}
|
||||
|
||||
/// Append a whole *subgraph* into a live DAG at runtime — the single
|
||||
/// in-DAG-growth primitive. The subgraph is inserted as a [`insert_group`]
|
||||
/// rooted under `dep_on` (the emitting node): the subgraph's own root becomes
|
||||
/// a *child* of `dep_on`, its steps children of that root, and the group's
|
||||
/// agent lease is hoisted onto that root. Ordering root→`dep_on` is the parent
|
||||
/// gate — the children run once `dep_on` reaches `Finishing`. Because the
|
||||
/// emitting node stays `Finishing` until this appended subtree is terminal and
|
||||
/// the DAG's terminal node deps on the top root, roll-up keeps the DAG from
|
||||
/// settling early with no explicit wiring. A no-op if the DAG is gone.
|
||||
pub fn append_subgraph(&self, dag_id: u64, declare: Declare, dep_on: NodeId) {
|
||||
let mut inner = self.lock();
|
||||
if inner.container(dag_id).is_none() {
|
||||
return;
|
||||
}
|
||||
// Insert the subgraph as a group rooted under the emitting node: the
|
||||
// subgraph's own root becomes a child of `dep_on`, its steps children of
|
||||
// that root. No terminal-node wiring — roll-up carries terminality: the
|
||||
// emitter stays `Finishing` until this appended subtree settles, and the
|
||||
// container node rolls up terminal only once its whole subtree (incl. this
|
||||
// appended work) has settled, so the DAG hook waits for free.
|
||||
if let Err(e) = insert_group(&mut inner, declare, Some(dep_on)) {
|
||||
tracing::error!(
|
||||
dag = dag_id,
|
||||
error = %e,
|
||||
"job_queue: append_subgraph insert failed"
|
||||
);
|
||||
return;
|
||||
}
|
||||
drop(inner);
|
||||
self.notify.notify_one();
|
||||
}
|
||||
|
||||
/// Claim every currently-runnable node, acquiring its resources, and mark it
|
||||
/// `Running`. Delegates readiness + resource acquisition to the crate's
|
||||
/// settle loop; builds a [`Claim`] per started node from its payload + its
|
||||
|
|
@ -325,15 +295,48 @@ impl JobQueue {
|
|||
/// ([`NodeKind::ResolveApproval`] / [`NodeKind::EmitRebuilt`]), which the
|
||||
/// scheduler claims and runs like any other node.
|
||||
pub fn complete_node(&self, node_id: NodeId, result: Result<(), String>) {
|
||||
// Deliberately not `complete_node_growing(.., self.new_job())`: that
|
||||
// would take the lock twice (once to mint an empty builder, once to
|
||||
// complete) to express "grew nothing". The shared part is the outcome
|
||||
// mapping, and that's a free fn.
|
||||
let mut inner = self.lock();
|
||||
// The failure reason + `finished_at` are stamped onto the graph `Node`
|
||||
// by the scheduler (the reason rides `Outcome::Failed`); no host-side
|
||||
// copy, so there is nothing to clear here.
|
||||
let outcome = match result {
|
||||
Ok(()) => Outcome::Done,
|
||||
Err(e) => Outcome::Failed(truncate_error(&e)),
|
||||
};
|
||||
inner.sched.complete(node_id, outcome);
|
||||
inner.sched.complete(node_id, outcome_of(result));
|
||||
drop(inner);
|
||||
self.notify.notify_one();
|
||||
}
|
||||
|
||||
/// A builder for a node to declare more work into while it runs.
|
||||
///
|
||||
/// Handed to [`exec::run_node`] and returned to
|
||||
/// [`JobQueue::complete_node_growing`]. Only `hive_jobq` can construct one,
|
||||
/// which is why this goes through the scheduler rather than
|
||||
/// `Job::default()`.
|
||||
#[must_use]
|
||||
pub fn new_job(&self) -> Job {
|
||||
self.lock().sched.new_job()
|
||||
}
|
||||
|
||||
/// [`JobQueue::complete_node`] plus the work the node declared while it ran.
|
||||
///
|
||||
/// `grown` is inserted **under `node_id`** before the completion, so the DAG
|
||||
/// cannot roll terminal with the appended work still pending — the property
|
||||
/// the old two-call `append_subgraph` + `complete_node` sequence had to
|
||||
/// arrange by hand at every call site.
|
||||
pub fn complete_node_growing(&self, node_id: NodeId, result: Result<(), String>, grown: Job) {
|
||||
let mut inner = self.lock();
|
||||
// A rejected grown job is logged, not propagated: the node's own work
|
||||
// already ran, and refusing to complete it here would both misreport
|
||||
// that and wedge the DAG on a node stuck `Running`.
|
||||
if let Err(e) = inner
|
||||
.sched
|
||||
.complete_growing(node_id, outcome_of(result), grown)
|
||||
{
|
||||
tracing::error!(
|
||||
node = node_id.get(),
|
||||
error = %e,
|
||||
"job_queue: work grown by a completing node was rejected"
|
||||
);
|
||||
}
|
||||
drop(inner);
|
||||
self.notify.notify_one();
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in a new issue