refactor(job-queue): a job is a recipe, not a value you carry
Follows the jobq change: a builder can no longer be constructed or inserted outside `hive_jobq`, so `DagSpec` cannot hold one. It carries a `Declare` — `Box<dyn FnOnce(&Job) + Send>` — and the queue runs it against a builder jobq owns, at the moment it inserts. `NodeOutput.append_subgraph` becomes `Vec<Declare>` for the same reason, and this is where the shape was always heading: that field's doc already said an executor "cannot reach the queue, so it hands the declaration back", while its type was a `Vec<Job>` the executor had built itself. The rejected `build_nodes -> Vec<NodeSpec>` was the first version of that escape hatch; a recipe is the last one, because there is no job-shaped value to hand over at all. Templates and the power-op assemblers move their owned data into the closure and are otherwise unchanged — `rebuild_nodes`, `node` and the tail helpers already took `&Job` and returned handles, so only each template's outermost frame moved. Two `Debug` impls are hand-written: a closure has nothing to show, and its nodes do not exist until the queue runs it. `NodeOutput` reports how many subgraphs were emitted, `DagSpec` its source and reason. `append_subgraph`'s `is_empty()` early-return is gone — you cannot ask a recipe whether it will declare anything without running it. It now inserts and returns an empty id list if nothing was declared, which takes the queue lock in a case that previously skipped it. The two in-DAG-growth tests build `Declare`s now, so they exercise the shape an executor actually produces rather than one only a test could construct. 45 job-queue tests unchanged and passing.
This commit is contained in:
parent
f7548e4535
commit
9c97365f8f
8 changed files with 336 additions and 284 deletions
|
|
@ -470,15 +470,27 @@ impl NodeKind {
|
|||
/// Type-specific payloads (`PermChange`'s file payload) ride the node that
|
||||
/// consumes them ([`NodeKind::WritePermFile`]), not this generic spec.
|
||||
///
|
||||
/// There is no separate per-node spec type: the nodes live in the builder,
|
||||
/// which inserts them itself. A shape that has been declared is therefore
|
||||
/// always insertable — a dangling edge or a cycle cannot be expressed, so
|
||||
/// there is nothing left for a submit-time validation pass to reject.
|
||||
#[derive(Debug)]
|
||||
/// There is no separate per-node spec type, and no built job either: `declare`
|
||||
/// is a *recipe* the queue runs against a builder `hive_jobq` owns, at the
|
||||
/// moment it inserts. A shape that has been declared is therefore always
|
||||
/// insertable — a dangling edge or a cycle cannot be expressed, so there is
|
||||
/// nothing left for a submit-time validation pass to reject.
|
||||
pub struct DagSpec {
|
||||
pub source: Source,
|
||||
/// Free-form "why".
|
||||
pub reason: String,
|
||||
/// The DAG's declared nodes, with their edges, grouping and resources.
|
||||
pub job: super::Job,
|
||||
/// Declares the DAG's nodes — their edges, grouping and resources — onto
|
||||
/// the builder the queue hands it.
|
||||
pub declare: super::Declare,
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for DagSpec {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
// The recipe is a closure; there is nothing to show of it, and its
|
||||
// nodes do not exist until the queue runs it.
|
||||
f.debug_struct("DagSpec")
|
||||
.field("source", &self.source)
|
||||
.field("reason", &self.reason)
|
||||
.finish_non_exhaustive()
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in a new issue