refactor(job-queue): a job is a recipe, not a value you carry
Follows the jobq change: a builder can no longer be constructed or inserted outside `hive_jobq`, so `DagSpec` cannot hold one. It carries a `Declare` — `Box<dyn FnOnce(&Job) + Send>` — and the queue runs it against a builder jobq owns, at the moment it inserts. `NodeOutput.append_subgraph` becomes `Vec<Declare>` for the same reason, and this is where the shape was always heading: that field's doc already said an executor "cannot reach the queue, so it hands the declaration back", while its type was a `Vec<Job>` the executor had built itself. The rejected `build_nodes -> Vec<NodeSpec>` was the first version of that escape hatch; a recipe is the last one, because there is no job-shaped value to hand over at all. Templates and the power-op assemblers move their owned data into the closure and are otherwise unchanged — `rebuild_nodes`, `node` and the tail helpers already took `&Job` and returned handles, so only each template's outermost frame moved. Two `Debug` impls are hand-written: a closure has nothing to show, and its nodes do not exist until the queue runs it. `NodeOutput` reports how many subgraphs were emitted, `DagSpec` its source and reason. `append_subgraph`'s `is_empty()` early-return is gone — you cannot ask a recipe whether it will declare anything without running it. It now inserts and returns an empty id list if nothing was declared, which takes the queue lock in a case that previously skipped it. The two in-DAG-growth tests build `Declare`s now, so they exercise the shape an executor actually produces rather than one only a test could construct. 45 job-queue tests unchanged and passing.
This commit is contained in:
parent
f7548e4535
commit
9c97365f8f
8 changed files with 336 additions and 284 deletions
|
|
@ -26,7 +26,7 @@ use std::sync::Arc;
|
|||
|
||||
use super::model::{DagSpec, NodeKind};
|
||||
use super::templates::{RebuildOpts, node, rebuild_nodes};
|
||||
use super::{Job, Source, templates};
|
||||
use super::{Declare, Job, Source, templates};
|
||||
use crate::coordinator::Coordinator;
|
||||
use crate::lifecycle;
|
||||
|
||||
|
|
@ -169,11 +169,11 @@ fn restart_chain(b: &Job, agent: &str, graceful: bool, running: bool) {
|
|||
/// and each keeps its own root, so the per-agent subgraphs are independent and
|
||||
/// run concurrently, each on its own lease. Rebasing one subgraph's indices
|
||||
/// onto another's used to be a function.
|
||||
fn power_dag(source: Source, reason: String, job: Job) -> DagSpec {
|
||||
fn power_dag(source: Source, reason: String, declare: Declare) -> DagSpec {
|
||||
DagSpec {
|
||||
source,
|
||||
reason,
|
||||
job,
|
||||
declare,
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -189,11 +189,16 @@ pub(crate) fn stop_spec(
|
|||
source: Source,
|
||||
reason: String,
|
||||
) -> DagSpec {
|
||||
let job = Job::new();
|
||||
for (agent, running) in targets {
|
||||
stop_chain(&job, agent, graceful, *running);
|
||||
}
|
||||
power_dag(source, reason, job)
|
||||
let targets = targets.to_vec();
|
||||
power_dag(
|
||||
source,
|
||||
reason,
|
||||
Box::new(move |b| {
|
||||
for (agent, running) in targets {
|
||||
stop_chain(b, &agent, graceful, running);
|
||||
}
|
||||
}),
|
||||
)
|
||||
}
|
||||
|
||||
/// Assemble the start DAG from explicit `(agent, running, stale)` targets.
|
||||
|
|
@ -207,11 +212,16 @@ pub(crate) fn start_spec(
|
|||
source: Source,
|
||||
reason: String,
|
||||
) -> DagSpec {
|
||||
let job = Job::new();
|
||||
for (agent, running, stale) in targets {
|
||||
start_chain(&job, agent, *running, *stale);
|
||||
}
|
||||
power_dag(source, reason, job)
|
||||
let targets = targets.to_vec();
|
||||
power_dag(
|
||||
source,
|
||||
reason,
|
||||
Box::new(move |b| {
|
||||
for (agent, running, stale) in targets {
|
||||
start_chain(b, &agent, running, stale);
|
||||
}
|
||||
}),
|
||||
)
|
||||
}
|
||||
|
||||
/// Assemble the restart DAG from explicit `(agent, running)` targets.
|
||||
|
|
@ -221,11 +231,16 @@ pub(crate) fn restart_spec(
|
|||
source: Source,
|
||||
reason: String,
|
||||
) -> DagSpec {
|
||||
let job = Job::new();
|
||||
for (agent, running) in targets {
|
||||
restart_chain(&job, agent, graceful, *running);
|
||||
}
|
||||
power_dag(source, reason, job)
|
||||
let targets = targets.to_vec();
|
||||
power_dag(
|
||||
source,
|
||||
reason,
|
||||
Box::new(move |b| {
|
||||
for (agent, running) in targets {
|
||||
restart_chain(b, &agent, graceful, running);
|
||||
}
|
||||
}),
|
||||
)
|
||||
}
|
||||
|
||||
/// Restart a single agent. Thin wrapper over [`restart_many`].
|
||||
|
|
|
|||
Loading…
Reference in a new issue