vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2

jobq: one insertion entry point, and make it atomic

Browse source 
Three findings from the operator's review, all correct.

1. Two insert_job's. Graph::insert_job had no caller outside hive-jobq's
   own tests -- production only ever went through Scheduler::insert_job.
   It existed because the graph-level one got written first. Deleted; the
   tests moved onto a Scheduler, which is where insertion belongs anyway.

2. insert_job was not atomic, and the previous commit made that worse: a
   forward edge or forward parent surfaced mid-loop, leaving the nodes
   before it in the graph, and resolve_wanted ran after every insert, so
   an unknown handle failed once the whole job was already committed.
   The module documented this under "Partial insertion" instead of fixing
   it -- prose describing a hole is not a design.

   All three are decidable from what the builder holds, so
   check_declaration_order now runs before the first insert and the loop
   indexes ids directly. A malformed job leaves the graph untouched.
   What remains mid-insert is the graph's own rejection (out-of-group
   dep, empty DepWhen); closing that needs a dry-run validate on Graph,
   which is a separate change.

3. DagSpec no longer boxes its recipe: it is generic over the closure,
   which travels from the template that built it straight into submit.
   The box bought type inference, and paying for it costs annotations --
   `|b: &Job|` at each declaration site (the field needs an HRTB, and an
   unannotated closure binds one lifetime) and `+ use<>` on each
   returning signature (or the opaque type captures the caller's borrows).
   Erasure is still needed where several recipe shapes share one type:
   the boxed Declare stays for the executor's append_subgraph, and a test
   table uses an erase() helper.
This commit is contained in:
atlas 2026-08-02 14:39:37 +02:00 • committed by mara
commit f035b63b9a
8 changed files with 211 additions and 176 deletions
Download patch file Download diff file Expand all files Collapse all files

29
hive-jobq/src/lib.rs
View file

@ -458,35 +458,6 @@ impl<N, R> Graph<N, R> {
Ok(id)
}
/// Insert a whole job under `root_parent`, returning the ids of the nodes
/// `declare` **asked for**, in the order it named them.
///
/// `declare` receives a fresh [`JobBuilder`], names the job's nodes on it,
/// and returns the handles whose ids it wants back. The builder never
/// leaves this call, so a job cannot be built in one place and inserted in
/// another. The graph-only counterpart of
/// [`crate::scheduler::Scheduler::insert_job`] — prefer that one when a
/// scheduler owns the graph, since it also records what it started.
///
/// Asking is how a caller addresses a node it created: the alternative — a
/// map of everything inserted, or a positional vector — either hands back a
/// lookup nobody performs or reintroduces the counting this API exists to
/// remove.
///
/// # Errors
/// Propagates [`BuildError`] — a forward reference in the job's own
/// declarations, a handle from a different job, or a graph rejection.
pub fn insert_job(
&mut self,
root_parent: Option<NodeId>,
declare: impl FnOnce(&JobBuilder<N, R>) -> Vec<NodeGuid>,
) -> Result<Vec<NodeId>, BuildError> {
let job = JobBuilder::new();
let wanted = declare(&job);
let ids = job.insert_into(self, root_parent)?;
crate::builder::resolve_wanted(&wanted, &ids)
}
/// Borrow a node by id.
#[must_use]
pub fn node(&self, id: NodeId) -> Option<&Node<N, R>> {