refactor(job-queue): a job is a recipe, not a value you carry

Follows the jobq change: a builder can no longer be constructed or
inserted outside `hive_jobq`, so `DagSpec` cannot hold one. It carries a
`Declare` — `Box<dyn FnOnce(&Job) + Send>` — and the queue runs it
against a builder jobq owns, at the moment it inserts.

`NodeOutput.append_subgraph` becomes `Vec<Declare>` for the same reason,
and this is where the shape was always heading: that field's doc already
said an executor "cannot reach the queue, so it hands the declaration
back", while its type was a `Vec<Job>` the executor had built itself.
The rejected `build_nodes -> Vec<NodeSpec>` was the first version of that
escape hatch; a recipe is the last one, because there is no job-shaped
value to hand over at all.

Templates and the power-op assemblers move their owned data into the
closure and are otherwise unchanged — `rebuild_nodes`, `node` and the
tail helpers already took `&Job` and returned handles, so only each
template's outermost frame moved.

Two `Debug` impls are hand-written: a closure has nothing to show, and
its nodes do not exist until the queue runs it. `NodeOutput` reports how
many subgraphs were emitted, `DagSpec` its source and reason.

`append_subgraph`'s `is_empty()` early-return is gone — you cannot ask a
recipe whether it will declare anything without running it. It now
inserts and returns an empty id list if nothing was declared, which
takes the queue lock in a case that previously skipped it.

The two in-DAG-growth tests build `Declare`s now, so they exercise the
shape an executor actually produces rather than one only a test could
construct. 45 job-queue tests unchanged and passing.
This commit is contained in:
atlas 2026-08-02 13:40:03 +02:00 committed by mara
commit 9c97365f8f
8 changed files with 336 additions and 284 deletions

View file

@ -51,10 +51,20 @@ pub use model::{DagSpec, DagView, NodeKind, PermPayload, Source, State};
use resource::Resource;
/// A job under construction: `hive_jobq`'s builder over this queue's payload
/// ([`NodeKind`]) and resource ([`Resource`]) types. Templates declare into
/// one of these; [`JobQueue::submit`] inserts it.
/// ([`NodeKind`]) and resource ([`Resource`]) types. Templates declare into a
/// borrowed one; only `hive_jobq` can make or insert it.
pub type Job = hive_jobq::JobBuilder<NodeKind, Resource>;
/// A job's shape as a **recipe**: given a builder, declare the nodes.
///
/// What a template returns and what an executor hands back, because neither
/// can build a job itself — `hive_jobq` creates the builder inside its own
/// insertion call and never lets one out. So the transferable thing is the
/// declaring closure, and the queue runs it at the moment it inserts.
///
/// `Send` because an executor's output crosses the scheduler's task boundary.
pub type Declare = Box<dyn FnOnce(&Job) + Send>;
/// A handle to one node a template declared — where its edges, grouping and
/// resources are declared. `Copy`; naming a node as a dependency does not
/// consume the ability to name it again.
@ -169,12 +179,12 @@ impl Default for JobQueue {
/// Propagates a crate graph-insert error (malformed dep/parent / dep-scope).
fn insert_group(
inner: &mut QueueInner,
job: Job,
declare: Declare,
group_parent: Option<NodeId>,
) -> anyhow::Result<Vec<NodeId>> {
let ids = inner
.sched
.insert_job(job, group_parent)
.insert_job(group_parent, declare)
.map_err(|e| anyhow::anyhow!("job_queue: graph insert failed: {e}"))?;
for &id in ids.values() {
inner.node_rt.insert(id, NodeRuntime::default());
@ -226,7 +236,7 @@ impl JobQueue {
)
.map_err(|e| anyhow::anyhow!("job_queue: container insert failed: {e}"))?;
inner.node_rt.insert(container, NodeRuntime::default());
insert_group(&mut inner, spec.job, Some(container))?;
insert_group(&mut inner, spec.declare, Some(container))?;
// Settle the container's own (no-op) logic immediately so it parks in
// `Finishing` and its children become runnable — it never needs claiming
// or executing, and stays out of `claim_ready`. It rolls up terminal when
@ -247,10 +257,7 @@ impl JobQueue {
/// the DAG's terminal node deps on the top root, roll-up keeps the DAG from
/// settling early with no explicit wiring. Returns the new node ids; empty if
/// the DAG is gone or `nodes` is empty.
pub fn append_subgraph(&self, dag_id: u64, job: Job, dep_on: NodeId) -> Vec<NodeId> {
if job.is_empty() {
return Vec::new();
}
pub fn append_subgraph(&self, dag_id: u64, declare: Declare, dep_on: NodeId) -> Vec<NodeId> {
let mut inner = self.lock();
if inner.container(dag_id).is_none() {
return Vec::new();
@ -261,7 +268,7 @@ impl JobQueue {
// emitter stays `Finishing` until this appended subtree settles, and the
// container node rolls up terminal only once its whole subtree (incl. this
// appended work) has settled, so the DAG hook waits for free.
let ids = match insert_group(&mut inner, job, Some(dep_on)) {
let ids = match insert_group(&mut inner, declare, Some(dep_on)) {
Ok(ids) => ids,
Err(e) => {
tracing::error!(