jobq: one insertion entry point, and make it atomic
Three findings from the operator's review, all correct. 1. Two insert_job's. Graph::insert_job had no caller outside hive-jobq's own tests -- production only ever went through Scheduler::insert_job. It existed because the graph-level one got written first. Deleted; the tests moved onto a Scheduler, which is where insertion belongs anyway. 2. insert_job was not atomic, and the previous commit made that worse: a forward edge or forward parent surfaced mid-loop, leaving the nodes before it in the graph, and resolve_wanted ran after every insert, so an unknown handle failed once the whole job was already committed. The module documented this under "Partial insertion" instead of fixing it -- prose describing a hole is not a design. All three are decidable from what the builder holds, so check_declaration_order now runs before the first insert and the loop indexes ids directly. A malformed job leaves the graph untouched. What remains mid-insert is the graph's own rejection (out-of-group dep, empty DepWhen); closing that needs a dry-run validate on Graph, which is a separate change. 3. DagSpec no longer boxes its recipe: it is generic over the closure, which travels from the template that built it straight into submit. The box bought type inference, and paying for it costs annotations -- `|b: &Job|` at each declaration site (the field needs an HRTB, and an unannotated closure binds one lifetime) and `+ use<>` on each returning signature (or the opaque type captures the caller's borrows). Erasure is still needed where several recipe shapes share one type: the boxed Declare stays for the executor's append_subgraph, and a test table uses an erase() helper.
This commit is contained in:
parent
bf138ae79a
commit
f035b63b9a
8 changed files with 211 additions and 176 deletions
|
|
@ -110,15 +110,19 @@ impl<N, R: Clone + Eq + Hash> Scheduler<N, R> {
|
|||
/// insert one itself, so there is no way to end up with a job-shaped value
|
||||
/// being passed around as a spec.
|
||||
///
|
||||
/// The scheduler-side counterpart of [`Graph::insert_job`]: same
|
||||
/// resolution, but each node goes through [`Scheduler::append`], so a
|
||||
/// caller never has to reach past the scheduler at the graph underneath.
|
||||
/// Call [`Scheduler::settle`] afterwards to start whatever became
|
||||
/// runnable.
|
||||
/// The one insertion entry point: every node goes through
|
||||
/// [`Scheduler::append`], so a caller never has to reach past the scheduler
|
||||
/// at the graph underneath. Call [`Scheduler::settle`] afterwards to start
|
||||
/// whatever became runnable.
|
||||
///
|
||||
/// **Atomic in the job's own shape.** A forward edge, a forward parent, or
|
||||
/// a request for a handle this job never declared is rejected *before* the
|
||||
/// first node is inserted, so a malformed job leaves the graph untouched
|
||||
/// rather than half-built.
|
||||
///
|
||||
/// # Errors
|
||||
/// Propagates [`BuildError`] — a forward reference in the job's own
|
||||
/// declarations, or a graph rejection.
|
||||
/// declarations, a handle from a different job, or a graph rejection.
|
||||
pub fn insert_job(
|
||||
&mut self,
|
||||
root_parent: Option<NodeId>,
|
||||
|
|
@ -126,10 +130,9 @@ impl<N, R: Clone + Eq + Hash> Scheduler<N, R> {
|
|||
) -> Result<Vec<NodeId>, BuildError> {
|
||||
let job = JobBuilder::new();
|
||||
let wanted = declare(&job);
|
||||
let ids = job.insert_with(root_parent, |payload, deps, parent| {
|
||||
job.insert_with(root_parent, &wanted, |payload, deps, parent| {
|
||||
self.append(payload, deps, parent)
|
||||
})?;
|
||||
crate::builder::resolve_wanted(&wanted, &ids)
|
||||
})
|
||||
}
|
||||
|
||||
/// Claim every currently-runnable pending node and start it: node-deps
|
||||
|
|
|
|||
Loading…
Reference in a new issue