refactor(job-queue): a job is a recipe, not a value you carry

Follows the jobq change: a builder can no longer be constructed or
inserted outside `hive_jobq`, so `DagSpec` cannot hold one. It carries a
`Declare` — `Box<dyn FnOnce(&Job) + Send>` — and the queue runs it
against a builder jobq owns, at the moment it inserts.

`NodeOutput.append_subgraph` becomes `Vec<Declare>` for the same reason,
and this is where the shape was always heading: that field's doc already
said an executor "cannot reach the queue, so it hands the declaration
back", while its type was a `Vec<Job>` the executor had built itself.
The rejected `build_nodes -> Vec<NodeSpec>` was the first version of that
escape hatch; a recipe is the last one, because there is no job-shaped
value to hand over at all.

Templates and the power-op assemblers move their owned data into the
closure and are otherwise unchanged — `rebuild_nodes`, `node` and the
tail helpers already took `&Job` and returned handles, so only each
template's outermost frame moved.

Two `Debug` impls are hand-written: a closure has nothing to show, and
its nodes do not exist until the queue runs it. `NodeOutput` reports how
many subgraphs were emitted, `DagSpec` its source and reason.

`append_subgraph`'s `is_empty()` early-return is gone — you cannot ask a
recipe whether it will declare anything without running it. It now
inserts and returns an empty id list if nothing was declared, which
takes the queue lock in a case that previously skipped it.

The two in-DAG-growth tests build `Declare`s now, so they exercise the
shape an executor actually produces rather than one only a test could
construct. 45 job-queue tests unchanged and passing.
This commit is contained in:
atlas 2026-08-02 13:40:03 +02:00 committed by mara
commit 9c97365f8f
8 changed files with 336 additions and 284 deletions

View file

@ -10,7 +10,7 @@ use std::sync::Arc;
use anyhow::{Context as _, Result};
use super::{Claim, Job};
use super::{Claim, Declare};
use hive_jobq::TerminalState;
use super::model::NodeKind;
@ -27,7 +27,7 @@ pub const GRACEFUL_STOP_TIMEOUT: std::time::Duration = std::time::Duration::from
/// Extra signal an executor hands back to the scheduler alongside
/// success.
#[derive(Debug, Default)]
#[derive(Default)]
pub struct NodeOutput {
/// Whole per-agent *subgraphs* to append into *this same* DAG at
/// runtime — the single in-DAG-growth channel. Each [`Job`] is one
@ -42,7 +42,17 @@ pub struct NodeOutput {
/// *before* the emitting node's completion so the DAG never rolls terminal
/// with the appended work still pending — keeping the lease-window
/// transient held across the sub-step.
pub append_subgraph: Vec<Job>,
pub append_subgraph: Vec<Declare>,
}
impl std::fmt::Debug for NodeOutput {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
// The subgraphs are closures — how many were emitted is the only thing
// there is to say about them before the queue runs them.
f.debug_struct("NodeOutput")
.field("append_subgraph", &self.append_subgraph.len())
.finish()
}
}
/// Build-log sink for one claimed node.
@ -342,17 +352,18 @@ async fn run_meta_lock(
.unwrap_or_default()
.iter()
.map(|agent| {
let job = Job::new();
super::templates::rebuild_nodes(
&job,
agent,
super::templates::RebuildOpts {
relock: true,
graceful: true,
},
None,
);
job
let agent = agent.clone();
Box::new(move |b: &super::Job| {
super::templates::rebuild_nodes(
b,
&agent,
super::templates::RebuildOpts {
relock: true,
graceful: true,
},
None,
);
}) as Declare
})
.collect();
return Ok(NodeOutput { append_subgraph });
@ -374,17 +385,18 @@ async fn run_meta_lock(
let append_subgraph = cascade
.iter()
.map(|agent| {
let job = Job::new();
super::templates::rebuild_nodes(
&job,
agent,
super::templates::RebuildOpts {
relock: false,
graceful: false,
},
None,
);
job
let agent = agent.clone();
Box::new(move |b: &super::Job| {
super::templates::rebuild_nodes(
b,
&agent,
super::templates::RebuildOpts {
relock: false,
graceful: false,
},
None,
);
}) as Declare
})
.collect();
Ok(NodeOutput { append_subgraph })
@ -404,10 +416,10 @@ async fn run_reconcile(coord: &Arc<Coordinator>, claim: &Claim) -> Result<NodeOu
// One node targeting this agent, rooted on this reconcile node. `NodeKind`
// carries the agent it targets, so stamp `claim.agent` into the fanned-out
// Start/Stop kind (one in-DAG-growth channel).
let sub = |kind| {
let job = Job::new();
let _ = super::templates::node(&job, kind);
vec![job]
let sub = |kind: NodeKind| {
vec![Box::new(move |b: &super::Job| {
let _ = super::templates::node(b, kind);
}) as Declare]
};
let append_subgraph = match reconcile_action(wanted, running) {
ReconcileAction::Start => sub(NodeKind::Start {