jobq: one insertion entry point, and make it atomic
Three findings from the operator's review, all correct. 1. Two insert_job's. Graph::insert_job had no caller outside hive-jobq's own tests -- production only ever went through Scheduler::insert_job. It existed because the graph-level one got written first. Deleted; the tests moved onto a Scheduler, which is where insertion belongs anyway. 2. insert_job was not atomic, and the previous commit made that worse: a forward edge or forward parent surfaced mid-loop, leaving the nodes before it in the graph, and resolve_wanted ran after every insert, so an unknown handle failed once the whole job was already committed. The module documented this under "Partial insertion" instead of fixing it -- prose describing a hole is not a design. All three are decidable from what the builder holds, so check_declaration_order now runs before the first insert and the loop indexes ids directly. A malformed job leaves the graph untouched. What remains mid-insert is the graph's own rejection (out-of-group dep, empty DepWhen); closing that needs a dry-run validate on Graph, which is a separate change. 3. DagSpec no longer boxes its recipe: it is generic over the closure, which travels from the template that built it straight into submit. The box bought type inference, and paying for it costs annotations -- `|b: &Job|` at each declaration site (the field needs an HRTB, and an unannotated closure binds one lifetime) and `+ use<>` on each returning signature (or the opaque type captures the caller's borrows). Erasure is still needed where several recipe shapes share one type: the boxed Declare stays for the executor's append_subgraph, and a test table uses an erase() helper.
This commit is contained in:
parent
bf138ae79a
commit
f035b63b9a
8 changed files with 211 additions and 176 deletions
|
|
@ -475,16 +475,23 @@ impl NodeKind {
|
|||
/// moment it inserts. A shape that has been declared is therefore always
|
||||
/// insertable — a dangling edge or a cycle cannot be expressed, so there is
|
||||
/// nothing left for a submit-time validation pass to reject.
|
||||
pub struct DagSpec {
|
||||
///
|
||||
/// Generic over the recipe rather than boxing it: a spec goes from the template
|
||||
/// that returns it straight to the `submit` that consumes it, so the closure's
|
||||
/// concrete type is known the whole way and needs neither an allocation nor a
|
||||
/// `Send` bound. (The executor's `append_subgraph` is the case that *does* need
|
||||
/// a boxed [`super::Declare`] — its recipes are collected into a `Vec` and
|
||||
/// applied later, across a task boundary.)
|
||||
pub struct DagSpec<F> {
|
||||
pub source: Source,
|
||||
/// Free-form "why".
|
||||
pub reason: String,
|
||||
/// Declares the DAG's nodes — their edges, grouping and resources — onto
|
||||
/// the builder the queue hands it.
|
||||
pub declare: super::Declare,
|
||||
pub declare: F,
|
||||
}
|
||||
|
||||
impl std::fmt::Debug for DagSpec {
|
||||
impl<F> std::fmt::Debug for DagSpec<F> {
|
||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
// The recipe is a closure; there is nothing to show of it, and its
|
||||
// nodes do not exist until the queue runs it.
|
||||
|
|
|
|||
Loading…
Reference in a new issue