jobq: one insertion entry point, and make it atomic
Three findings from the operator's review, all correct. 1. Two insert_job's. Graph::insert_job had no caller outside hive-jobq's own tests -- production only ever went through Scheduler::insert_job. It existed because the graph-level one got written first. Deleted; the tests moved onto a Scheduler, which is where insertion belongs anyway. 2. insert_job was not atomic, and the previous commit made that worse: a forward edge or forward parent surfaced mid-loop, leaving the nodes before it in the graph, and resolve_wanted ran after every insert, so an unknown handle failed once the whole job was already committed. The module documented this under "Partial insertion" instead of fixing it -- prose describing a hole is not a design. All three are decidable from what the builder holds, so check_declaration_order now runs before the first insert and the loop indexes ids directly. A malformed job leaves the graph untouched. What remains mid-insert is the graph's own rejection (out-of-group dep, empty DepWhen); closing that needs a dry-run validate on Graph, which is a separate change. 3. DagSpec no longer boxes its recipe: it is generic over the closure, which travels from the template that built it straight into submit. The box bought type inference, and paying for it costs annotations -- `|b: &Job|` at each declaration site (the field needs an HRTB, and an unannotated closure binds one lifetime) and `+ use<>` on each returning signature (or the opaque type captures the caller's borrows). Erasure is still needed where several recipe shapes share one type: the boxed Declare stays for the executor's append_subgraph, and a test table uses an erase() helper.
This commit is contained in:
parent
bf138ae79a
commit
f035b63b9a
8 changed files with 211 additions and 176 deletions
|
|
@ -179,7 +179,7 @@ impl Default for JobQueue {
|
||||||
/// Propagates a crate graph-insert error (malformed dep/parent / dep-scope).
|
/// Propagates a crate graph-insert error (malformed dep/parent / dep-scope).
|
||||||
fn insert_group(
|
fn insert_group(
|
||||||
inner: &mut QueueInner,
|
inner: &mut QueueInner,
|
||||||
declare: Declare,
|
declare: impl FnOnce(&Job),
|
||||||
group_parent: Option<NodeId>,
|
group_parent: Option<NodeId>,
|
||||||
) -> anyhow::Result<()> {
|
) -> anyhow::Result<()> {
|
||||||
inner
|
inner
|
||||||
|
|
@ -216,15 +216,19 @@ impl JobQueue {
|
||||||
self.inner.lock().expect("job_queue mutex poisoned")
|
self.inner.lock().expect("job_queue mutex poisoned")
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Submit a DAG. Validates the spec, inserts a [`NodeKind::Dag`] **container
|
/// Submit a DAG: insert a [`NodeKind::Dag`] **container node** carrying the
|
||||||
/// node** carrying the group's metadata, then inserts the template's nodes as
|
/// group's metadata, then insert the template's nodes as its subtree (their
|
||||||
/// its subtree (their roots re-parented to the container). Returns the
|
/// roots re-parented to the container). Returns the container's id as the
|
||||||
/// container's id as the DAG id — its rolled-up state is the DAG state.
|
/// DAG id — its rolled-up state is the DAG state.
|
||||||
|
///
|
||||||
|
/// Takes the spec's recipe by generic, not as a boxed [`Declare`]: a spec
|
||||||
|
/// travels from the template that built it directly into this call, so
|
||||||
|
/// there is nothing to allocate for.
|
||||||
///
|
///
|
||||||
/// # Errors
|
/// # Errors
|
||||||
/// Propagates the spec-validation error (empty / cyclic / bad parent) or a
|
/// Propagates a graph-insert error (dependencies that aren't
|
||||||
/// graph-insert error (dependencies that aren't dependency-topological).
|
/// dependency-topological).
|
||||||
pub fn submit(&self, spec: DagSpec) -> anyhow::Result<u64> {
|
pub fn submit<F: FnOnce(&Job)>(&self, spec: DagSpec<F>) -> anyhow::Result<u64> {
|
||||||
let mut inner = self.lock();
|
let mut inner = self.lock();
|
||||||
let container = inner
|
let container = inner
|
||||||
.sched
|
.sched
|
||||||
|
|
|
||||||
|
|
@ -475,16 +475,23 @@ impl NodeKind {
|
||||||
/// moment it inserts. A shape that has been declared is therefore always
|
/// moment it inserts. A shape that has been declared is therefore always
|
||||||
/// insertable — a dangling edge or a cycle cannot be expressed, so there is
|
/// insertable — a dangling edge or a cycle cannot be expressed, so there is
|
||||||
/// nothing left for a submit-time validation pass to reject.
|
/// nothing left for a submit-time validation pass to reject.
|
||||||
pub struct DagSpec {
|
///
|
||||||
|
/// Generic over the recipe rather than boxing it: a spec goes from the template
|
||||||
|
/// that returns it straight to the `submit` that consumes it, so the closure's
|
||||||
|
/// concrete type is known the whole way and needs neither an allocation nor a
|
||||||
|
/// `Send` bound. (The executor's `append_subgraph` is the case that *does* need
|
||||||
|
/// a boxed [`super::Declare`] — its recipes are collected into a `Vec` and
|
||||||
|
/// applied later, across a task boundary.)
|
||||||
|
pub struct DagSpec<F> {
|
||||||
pub source: Source,
|
pub source: Source,
|
||||||
/// Free-form "why".
|
/// Free-form "why".
|
||||||
pub reason: String,
|
pub reason: String,
|
||||||
/// Declares the DAG's nodes — their edges, grouping and resources — onto
|
/// Declares the DAG's nodes — their edges, grouping and resources — onto
|
||||||
/// the builder the queue hands it.
|
/// the builder the queue hands it.
|
||||||
pub declare: super::Declare,
|
pub declare: F,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl std::fmt::Debug for DagSpec {
|
impl<F> std::fmt::Debug for DagSpec<F> {
|
||||||
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||||
// The recipe is a closure; there is nothing to show of it, and its
|
// The recipe is a closure; there is nothing to show of it, and its
|
||||||
// nodes do not exist until the queue runs it.
|
// nodes do not exist until the queue runs it.
|
||||||
|
|
|
||||||
|
|
@ -26,11 +26,11 @@ use std::sync::Arc;
|
||||||
|
|
||||||
use super::model::{DagSpec, NodeKind};
|
use super::model::{DagSpec, NodeKind};
|
||||||
use super::templates::{RebuildOpts, node, rebuild_nodes};
|
use super::templates::{RebuildOpts, node, rebuild_nodes};
|
||||||
use super::{Declare, Job, Source, templates};
|
use super::{Job, Source, templates};
|
||||||
use crate::coordinator::Coordinator;
|
use crate::coordinator::Coordinator;
|
||||||
use crate::lifecycle;
|
use crate::lifecycle;
|
||||||
|
|
||||||
fn submit_and_emit(coord: &Arc<Coordinator>, spec: super::DagSpec) -> u64 {
|
fn submit_and_emit<F: FnOnce(&Job)>(coord: &Arc<Coordinator>, spec: super::DagSpec<F>) -> u64 {
|
||||||
let id = coord
|
let id = coord
|
||||||
.job_queue
|
.job_queue
|
||||||
.submit(spec)
|
.submit(spec)
|
||||||
|
|
@ -169,7 +169,7 @@ fn restart_chain(b: &Job, agent: &str, graceful: bool, running: bool) {
|
||||||
/// and each keeps its own root, so the per-agent subgraphs are independent and
|
/// and each keeps its own root, so the per-agent subgraphs are independent and
|
||||||
/// run concurrently, each on its own lease. Rebasing one subgraph's indices
|
/// run concurrently, each on its own lease. Rebasing one subgraph's indices
|
||||||
/// onto another's used to be a function.
|
/// onto another's used to be a function.
|
||||||
fn power_dag(source: Source, reason: String, declare: Declare) -> DagSpec {
|
fn power_dag<F: FnOnce(&Job)>(source: Source, reason: String, declare: F) -> DagSpec<F> {
|
||||||
DagSpec {
|
DagSpec {
|
||||||
source,
|
source,
|
||||||
reason,
|
reason,
|
||||||
|
|
@ -188,12 +188,12 @@ pub(crate) fn stop_spec(
|
||||||
graceful: bool,
|
graceful: bool,
|
||||||
source: Source,
|
source: Source,
|
||||||
reason: String,
|
reason: String,
|
||||||
) -> DagSpec {
|
) -> DagSpec<impl FnOnce(&Job) + use<>> {
|
||||||
let targets = targets.to_vec();
|
let targets = targets.to_vec();
|
||||||
power_dag(
|
power_dag(
|
||||||
source,
|
source,
|
||||||
reason,
|
reason,
|
||||||
Box::new(move |b| {
|
Box::new(move |b: &Job| {
|
||||||
for (agent, running) in targets {
|
for (agent, running) in targets {
|
||||||
stop_chain(b, &agent, graceful, running);
|
stop_chain(b, &agent, graceful, running);
|
||||||
}
|
}
|
||||||
|
|
@ -211,12 +211,12 @@ pub(crate) fn start_spec(
|
||||||
targets: &[(String, bool, bool)],
|
targets: &[(String, bool, bool)],
|
||||||
source: Source,
|
source: Source,
|
||||||
reason: String,
|
reason: String,
|
||||||
) -> DagSpec {
|
) -> DagSpec<impl FnOnce(&Job) + use<>> {
|
||||||
let targets = targets.to_vec();
|
let targets = targets.to_vec();
|
||||||
power_dag(
|
power_dag(
|
||||||
source,
|
source,
|
||||||
reason,
|
reason,
|
||||||
Box::new(move |b| {
|
Box::new(move |b: &Job| {
|
||||||
for (agent, running, stale) in targets {
|
for (agent, running, stale) in targets {
|
||||||
start_chain(b, &agent, running, stale);
|
start_chain(b, &agent, running, stale);
|
||||||
}
|
}
|
||||||
|
|
@ -230,12 +230,12 @@ pub(crate) fn restart_spec(
|
||||||
graceful: bool,
|
graceful: bool,
|
||||||
source: Source,
|
source: Source,
|
||||||
reason: String,
|
reason: String,
|
||||||
) -> DagSpec {
|
) -> DagSpec<impl FnOnce(&Job) + use<>> {
|
||||||
let targets = targets.to_vec();
|
let targets = targets.to_vec();
|
||||||
power_dag(
|
power_dag(
|
||||||
source,
|
source,
|
||||||
reason,
|
reason,
|
||||||
Box::new(move |b| {
|
Box::new(move |b: &Job| {
|
||||||
for (agent, running) in targets {
|
for (agent, running) in targets {
|
||||||
restart_chain(b, &agent, graceful, running);
|
restart_chain(b, &agent, graceful, running);
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -241,7 +241,7 @@ pub(crate) fn rebuild_nodes<'a>(
|
||||||
/// already holding it rather than deadlocking against it.
|
/// already holding it rather than deadlocking against it.
|
||||||
pub(crate) fn deploy_rebuild_nodes(agent: &str, approval_id: i64) -> Declare {
|
pub(crate) fn deploy_rebuild_nodes(agent: &str, approval_id: i64) -> Declare {
|
||||||
let agent = agent.to_owned();
|
let agent = agent.to_owned();
|
||||||
Box::new(move |b| {
|
Box::new(move |b: &Job| {
|
||||||
let roots = rebuild_nodes(
|
let roots = rebuild_nodes(
|
||||||
b,
|
b,
|
||||||
&agent,
|
&agent,
|
||||||
|
|
@ -274,12 +274,17 @@ pub(crate) fn deploy_rebuild_nodes(agent: &str, approval_id: i64) -> Declare {
|
||||||
/// whole `StopForUpdate`→`Swap`→`PostSwap` subtree, so those three cover every
|
/// whole `StopForUpdate`→`Swap`→`PostSwap` subtree, so those three cover every
|
||||||
/// node. Edging `Reconcile` alone would not do: it is `AfterAny` `Prebuild`, so
|
/// node. Edging `Reconcile` alone would not do: it is `AfterAny` `Prebuild`, so
|
||||||
/// it reaches `Done` even after a failed swap and the tail would report success.
|
/// it reaches `Done` even after a failed swap and the tail would report success.
|
||||||
pub fn rebuild(agent: &str, source: Source, reason: String, relock: bool) -> DagSpec {
|
pub fn rebuild(
|
||||||
|
agent: &str,
|
||||||
|
source: Source,
|
||||||
|
reason: String,
|
||||||
|
relock: bool,
|
||||||
|
) -> DagSpec<impl FnOnce(&Job) + use<>> {
|
||||||
let agent = agent.to_owned();
|
let agent = agent.to_owned();
|
||||||
DagSpec {
|
DagSpec {
|
||||||
source,
|
source,
|
||||||
reason,
|
reason,
|
||||||
declare: Box::new(move |b| {
|
declare: Box::new(move |b: &Job| {
|
||||||
let roots = rebuild_nodes(
|
let roots = rebuild_nodes(
|
||||||
b,
|
b,
|
||||||
&agent,
|
&agent,
|
||||||
|
|
@ -319,12 +324,16 @@ pub fn rebuild(agent: &str, source: Source, reason: String, relock: bool) -> Dag
|
||||||
///
|
///
|
||||||
/// The window still spans the container build, as it must: `prepare_deploy`
|
/// The window still spans the container build, as it must: `prepare_deploy`
|
||||||
/// leaves `flake.lock` staged-uncommitted for the build's whole duration.
|
/// leaves `flake.lock` staged-uncommitted for the build's whole duration.
|
||||||
pub fn approval_deploy(agent: &str, approval_id: i64, reason: String) -> DagSpec {
|
pub fn approval_deploy(
|
||||||
|
agent: &str,
|
||||||
|
approval_id: i64,
|
||||||
|
reason: String,
|
||||||
|
) -> DagSpec<impl FnOnce(&Job) + use<>> {
|
||||||
let agent = agent.to_owned();
|
let agent = agent.to_owned();
|
||||||
DagSpec {
|
DagSpec {
|
||||||
source: Source::Approval,
|
source: Source::Approval,
|
||||||
reason,
|
reason,
|
||||||
declare: Box::new(move |b| {
|
declare: Box::new(move |b: &Job| {
|
||||||
let a = || agent.clone();
|
let a = || agent.clone();
|
||||||
let window = node(
|
let window = node(
|
||||||
b,
|
b,
|
||||||
|
|
@ -371,12 +380,16 @@ pub fn approval_deploy(agent: &str, approval_id: i64, reason: String) -> DagSpec
|
||||||
/// single-node lifecycle DAGs that exercise per-agent lease serialization
|
/// single-node lifecycle DAGs that exercise per-agent lease serialization
|
||||||
/// in the queue tests); production paths no longer emit a bare reconcile.
|
/// in the queue tests); production paths no longer emit a bare reconcile.
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
pub fn reconcile_only(agent: &str, source: Source, reason: String) -> DagSpec {
|
pub fn reconcile_only(
|
||||||
|
agent: &str,
|
||||||
|
source: Source,
|
||||||
|
reason: String,
|
||||||
|
) -> DagSpec<impl FnOnce(&Job) + use<>> {
|
||||||
let agent = agent.to_owned();
|
let agent = agent.to_owned();
|
||||||
DagSpec {
|
DagSpec {
|
||||||
source,
|
source,
|
||||||
reason,
|
reason,
|
||||||
declare: Box::new(move |b| {
|
declare: Box::new(move |b: &Job| {
|
||||||
let _reconcile = node(b, NodeKind::Reconcile { agent });
|
let _reconcile = node(b, NodeKind::Reconcile { agent });
|
||||||
}),
|
}),
|
||||||
}
|
}
|
||||||
|
|
@ -393,12 +406,12 @@ pub fn reconcile_only(agent: &str, source: Source, reason: String) -> DagSpec {
|
||||||
/// container was never created). Closed by a `ResolveApproval` tail root edged
|
/// container was never created). Closed by a `ResolveApproval` tail root edged
|
||||||
/// `AfterAny` onto `Provision` — the DAG's only other group-root, so its roll-up
|
/// `AfterAny` onto `Provision` — the DAG's only other group-root, so its roll-up
|
||||||
/// already carries the whole cascade.
|
/// already carries the whole cascade.
|
||||||
pub fn spawn(agent: &str, approval_id: i64, reason: String) -> DagSpec {
|
pub fn spawn(agent: &str, approval_id: i64, reason: String) -> DagSpec<impl FnOnce(&Job) + use<>> {
|
||||||
let agent = agent.to_owned();
|
let agent = agent.to_owned();
|
||||||
DagSpec {
|
DagSpec {
|
||||||
source: Source::Approval,
|
source: Source::Approval,
|
||||||
reason,
|
reason,
|
||||||
declare: Box::new(move |b| {
|
declare: Box::new(move |b: &Job| {
|
||||||
let a = || agent.clone();
|
let a = || agent.clone();
|
||||||
let provision = node(b, NodeKind::Provision { agent: a() });
|
let provision = node(b, NodeKind::Provision { agent: a() });
|
||||||
let create = node(b, NodeKind::Create { agent: a() }).part_of(provision);
|
let create = node(b, NodeKind::Create { agent: a() }).part_of(provision);
|
||||||
|
|
@ -417,12 +430,17 @@ pub fn spawn(agent: &str, approval_id: i64, reason: String) -> DagSpec {
|
||||||
/// effect in the container. Group-roots are `WritePermFile` plus the rebuild
|
/// effect in the container. Group-roots are `WritePermFile` plus the rebuild
|
||||||
/// subgraph's `MetaSync` / `Prebuild` / `Reconcile`, so the `EmitRebuilt` tail
|
/// subgraph's `MetaSync` / `Prebuild` / `Reconcile`, so the `EmitRebuilt` tail
|
||||||
/// edges all four.
|
/// edges all four.
|
||||||
pub fn perm_change(agent: &str, source: Source, reason: String, payload: PermPayload) -> DagSpec {
|
pub fn perm_change(
|
||||||
|
agent: &str,
|
||||||
|
source: Source,
|
||||||
|
reason: String,
|
||||||
|
payload: PermPayload,
|
||||||
|
) -> DagSpec<impl FnOnce(&Job) + use<>> {
|
||||||
let agent = agent.to_owned();
|
let agent = agent.to_owned();
|
||||||
DagSpec {
|
DagSpec {
|
||||||
source,
|
source,
|
||||||
reason,
|
reason,
|
||||||
declare: Box::new(move |b| {
|
declare: Box::new(move |b: &Job| {
|
||||||
let write = node(
|
let write = node(
|
||||||
b,
|
b,
|
||||||
NodeKind::WritePermFile {
|
NodeKind::WritePermFile {
|
||||||
|
|
@ -463,11 +481,11 @@ pub fn meta_update(
|
||||||
source: Source,
|
source: Source,
|
||||||
reason: String,
|
reason: String,
|
||||||
approval_id: Option<i64>,
|
approval_id: Option<i64>,
|
||||||
) -> DagSpec {
|
) -> DagSpec<impl FnOnce(&Job) + use<>> {
|
||||||
DagSpec {
|
DagSpec {
|
||||||
source,
|
source,
|
||||||
reason,
|
reason,
|
||||||
declare: Box::new(move |b| {
|
declare: Box::new(move |b: &Job| {
|
||||||
let lock = node(
|
let lock = node(
|
||||||
b,
|
b,
|
||||||
NodeKind::MetaLock {
|
NodeKind::MetaLock {
|
||||||
|
|
@ -501,11 +519,11 @@ pub fn reparent(
|
||||||
moves: Vec<(hive_types::Ident, Option<hive_types::Ident>)>,
|
moves: Vec<(hive_types::Ident, Option<hive_types::Ident>)>,
|
||||||
source: Source,
|
source: Source,
|
||||||
reason: String,
|
reason: String,
|
||||||
) -> DagSpec {
|
) -> DagSpec<impl FnOnce(&Job) + use<>> {
|
||||||
DagSpec {
|
DagSpec {
|
||||||
source,
|
source,
|
||||||
reason,
|
reason,
|
||||||
declare: Box::new(move |b| {
|
declare: Box::new(move |b: &Job| {
|
||||||
let _reparent = node(b, NodeKind::Reparent { moves });
|
let _reparent = node(b, NodeKind::Reparent { moves });
|
||||||
}),
|
}),
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -9,15 +9,29 @@
|
||||||
use super::model::NodeKind;
|
use super::model::NodeKind;
|
||||||
use super::*;
|
use super::*;
|
||||||
|
|
||||||
fn submit(q: &JobQueue, spec: DagSpec) -> u64 {
|
fn submit<F: FnOnce(&Job)>(q: &JobQueue, spec: DagSpec<F>) -> u64 {
|
||||||
q.submit(spec).expect("valid spec")
|
q.submit(spec).expect("valid spec")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Erase a spec's recipe to the boxed [`Declare`] so specs of *different*
|
||||||
|
/// shapes can share one type — e.g. a table of `(name, spec)` cases.
|
||||||
|
///
|
||||||
|
/// Production never needs this: each submit path builds one spec and hands it
|
||||||
|
/// straight to `submit`, so the concrete closure type is known end to end. A
|
||||||
|
/// test table is the case where several shapes must be one type.
|
||||||
|
fn erase<F: FnOnce(&Job) + Send + 'static>(spec: DagSpec<F>) -> DagSpec<Declare> {
|
||||||
|
DagSpec {
|
||||||
|
source: spec.source,
|
||||||
|
reason: spec.reason,
|
||||||
|
declare: Box::new(spec.declare),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
fn ident(s: &str) -> hive_types::Ident {
|
fn ident(s: &str) -> hive_types::Ident {
|
||||||
hive_types::Ident::parse(s).expect("valid test ident")
|
hive_types::Ident::parse(s).expect("valid test ident")
|
||||||
}
|
}
|
||||||
|
|
||||||
fn rebuild(agent: &str, reason: &str) -> DagSpec {
|
fn rebuild(agent: &str, reason: &str) -> DagSpec<impl FnOnce(&Job) + use<>> {
|
||||||
templates::rebuild(agent, Source::Manual, reason.to_owned(), true)
|
templates::rebuild(agent, Source::Manual, reason.to_owned(), true)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -25,14 +39,22 @@ fn rebuild(agent: &str, reason: &str) -> DagSpec {
|
||||||
/// shape (`[Signal→Drain→] StopForUpdate → Reconcile`, no `SetWanted` head)
|
/// shape (`[Signal→Drain→] StopForUpdate → Reconcile`, no `SetWanted` head)
|
||||||
/// most queue-mechanics tests assume. Mirrors the pre-dynamic
|
/// most queue-mechanics tests assume. Mirrors the pre-dynamic
|
||||||
/// `templates::restart` (which is now the state-aware `submit::restart_spec`).
|
/// `templates::restart` (which is now the state-aware `submit::restart_spec`).
|
||||||
fn restart_online(agents: &[&str], graceful: bool, reason: &str) -> DagSpec {
|
fn restart_online(
|
||||||
|
agents: &[&str],
|
||||||
|
graceful: bool,
|
||||||
|
reason: &str,
|
||||||
|
) -> DagSpec<impl FnOnce(&Job) + use<>> {
|
||||||
let targets: Vec<(String, bool)> = agents.iter().map(|a| ((*a).to_owned(), true)).collect();
|
let targets: Vec<(String, bool)> = agents.iter().map(|a| ((*a).to_owned(), true)).collect();
|
||||||
submit::restart_spec(&targets, graceful, Source::Manual, reason.to_owned())
|
submit::restart_spec(&targets, graceful, Source::Manual, reason.to_owned())
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Stop DAG spec with every agent treated as **running** — the online shape
|
/// Stop DAG spec with every agent treated as **running** — the online shape
|
||||||
/// (`SetWanted → [Signal→Drain→](graceful) Reconcile`).
|
/// (`SetWanted → [Signal→Drain→](graceful) Reconcile`).
|
||||||
fn stop_online(agents: &[&str], graceful: bool, reason: &str) -> DagSpec {
|
fn stop_online(
|
||||||
|
agents: &[&str],
|
||||||
|
graceful: bool,
|
||||||
|
reason: &str,
|
||||||
|
) -> DagSpec<impl FnOnce(&Job) + use<>> {
|
||||||
let targets: Vec<(String, bool)> = agents.iter().map(|a| ((*a).to_owned(), true)).collect();
|
let targets: Vec<(String, bool)> = agents.iter().map(|a| ((*a).to_owned(), true)).collect();
|
||||||
submit::stop_spec(&targets, graceful, Source::Manual, reason.to_owned())
|
submit::stop_spec(&targets, graceful, Source::Manual, reason.to_owned())
|
||||||
}
|
}
|
||||||
|
|
@ -197,7 +219,7 @@ fn graceful_rebuild_chain_drains_before_stopping() {
|
||||||
DagSpec {
|
DagSpec {
|
||||||
source: Source::AutoUpdate,
|
source: Source::AutoUpdate,
|
||||||
reason: "sweep".to_owned(),
|
reason: "sweep".to_owned(),
|
||||||
declare: Box::new(|b| {
|
declare: Box::new(|b: &Job| {
|
||||||
templates::rebuild_nodes(
|
templates::rebuild_nodes(
|
||||||
b,
|
b,
|
||||||
"agent-a",
|
"agent-a",
|
||||||
|
|
@ -246,7 +268,7 @@ fn non_graceful_rebuild_has_no_signal_or_drain() {
|
||||||
DagSpec {
|
DagSpec {
|
||||||
source: Source::Manual,
|
source: Source::Manual,
|
||||||
reason: "manual".to_owned(),
|
reason: "manual".to_owned(),
|
||||||
declare: Box::new(|b| {
|
declare: Box::new(|b: &Job| {
|
||||||
templates::rebuild_nodes(
|
templates::rebuild_nodes(
|
||||||
b,
|
b,
|
||||||
"agent-a",
|
"agent-a",
|
||||||
|
|
@ -695,7 +717,7 @@ fn append_subgraph_roots_on_emitter_and_rebases_local_deps() {
|
||||||
let spec = DagSpec {
|
let spec = DagSpec {
|
||||||
source: Source::AutoUpdate,
|
source: Source::AutoUpdate,
|
||||||
reason: "sweep".to_owned(),
|
reason: "sweep".to_owned(),
|
||||||
declare: Box::new(|b| {
|
declare: Box::new(|b: &Job| {
|
||||||
let _lock = templates::node(
|
let _lock = templates::node(
|
||||||
b,
|
b,
|
||||||
NodeKind::MetaLock {
|
NodeKind::MetaLock {
|
||||||
|
|
@ -715,7 +737,7 @@ fn append_subgraph_roots_on_emitter_and_rebases_local_deps() {
|
||||||
// match the sweep arm of `run_meta_lock` or this stops tracking production.
|
// match the sweep arm of `run_meta_lock` or this stops tracking production.
|
||||||
let subgraph = |agent: &str| -> Declare {
|
let subgraph = |agent: &str| -> Declare {
|
||||||
let agent = agent.to_owned();
|
let agent = agent.to_owned();
|
||||||
Box::new(move |b| {
|
Box::new(move |b: &Job| {
|
||||||
templates::rebuild_nodes(
|
templates::rebuild_nodes(
|
||||||
b,
|
b,
|
||||||
&agent,
|
&agent,
|
||||||
|
|
@ -821,7 +843,7 @@ fn meta_update_grows_cascade_in_dag() {
|
||||||
// Simulate the executor growing the cascade in-DAG (`relock = false` — a
|
// Simulate the executor growing the cascade in-DAG (`relock = false` — a
|
||||||
// cascade child must not re-lock and revert the parent's bump).
|
// cascade child must not re-lock and revert the parent's bump).
|
||||||
for agent in ["alice", "bob"] {
|
for agent in ["alice", "bob"] {
|
||||||
let declare: Declare = Box::new(move |b| {
|
let declare: Declare = Box::new(move |b: &Job| {
|
||||||
templates::rebuild_nodes(
|
templates::rebuild_nodes(
|
||||||
b,
|
b,
|
||||||
agent,
|
agent,
|
||||||
|
|
@ -1073,25 +1095,37 @@ fn cancelled_power_op_runs_no_compensating_node() {
|
||||||
for graceful in [false, true] {
|
for graceful in [false, true] {
|
||||||
for running in [false, true] {
|
for running in [false, true] {
|
||||||
let targets = vec![("agent-a".to_owned(), running)];
|
let targets = vec![("agent-a".to_owned(), running)];
|
||||||
|
// Erased to `DagSpec<Declare>`: three different recipe types have to
|
||||||
|
// sit in one array.
|
||||||
let cases = [
|
let cases = [
|
||||||
(
|
(
|
||||||
"restart",
|
"restart",
|
||||||
false,
|
false,
|
||||||
submit::restart_spec(&targets, graceful, Source::Manual, "bounce".to_owned()),
|
erase(submit::restart_spec(
|
||||||
|
&targets,
|
||||||
|
graceful,
|
||||||
|
Source::Manual,
|
||||||
|
"bounce".to_owned(),
|
||||||
|
)),
|
||||||
),
|
),
|
||||||
(
|
(
|
||||||
"stop",
|
"stop",
|
||||||
true,
|
true,
|
||||||
submit::stop_spec(&targets, graceful, Source::Manual, "stop".to_owned()),
|
erase(submit::stop_spec(
|
||||||
|
&targets,
|
||||||
|
graceful,
|
||||||
|
Source::Manual,
|
||||||
|
"stop".to_owned(),
|
||||||
|
)),
|
||||||
),
|
),
|
||||||
(
|
(
|
||||||
"start",
|
"start",
|
||||||
true,
|
true,
|
||||||
submit::start_spec(
|
erase(submit::start_spec(
|
||||||
&[("agent-a".to_owned(), running, false)],
|
&[("agent-a".to_owned(), running, false)],
|
||||||
Source::Manual,
|
Source::Manual,
|
||||||
"start".to_owned(),
|
"start".to_owned(),
|
||||||
),
|
)),
|
||||||
),
|
),
|
||||||
];
|
];
|
||||||
for (name, writes_intent, spec) in cases {
|
for (name, writes_intent, spec) in cases {
|
||||||
|
|
|
||||||
|
|
@ -6,15 +6,15 @@
|
||||||
//! get wrong.
|
//! get wrong.
|
||||||
//!
|
//!
|
||||||
//! **An insertion API, not a spec factory.** A builder is only ever handed to a
|
//! **An insertion API, not a spec factory.** A builder is only ever handed to a
|
||||||
//! closure by an insertion entry point ([`Graph::insert_job`],
|
//! closure by the single insertion entry point
|
||||||
//! [`crate::scheduler::Scheduler::insert_job`]), which inserts the declared
|
//! ([`crate::scheduler::Scheduler::insert_job`]), which inserts the declared
|
||||||
//! nodes and returns the ids the graph minted. It cannot be constructed, held
|
//! nodes and returns the ids the job asked for. It cannot be constructed, held
|
||||||
//! or inserted from outside this crate, and there is no intermediate
|
//! or inserted from outside this crate, and there is no intermediate
|
||||||
//! node-description type to keep in sync with [`Graph::insert`]'s signature —
|
//! node-description type to keep in sync with [`crate::Graph::insert`]'s signature —
|
||||||
//! so a job has no representation that can be passed around instead of being
|
//! so a job has no representation that can be passed around instead of being
|
||||||
//! inserted.
|
//! inserted.
|
||||||
//!
|
//!
|
||||||
//! **Payload-agnostic.** Generic over the same `N` and `R` as [`Graph`]: the
|
//! **Payload-agnostic.** Generic over the same `N` and `R` as [`crate::Graph`]: the
|
||||||
//! builder knows nothing about what a node *does*, only how nodes relate.
|
//! builder knows nothing about what a node *does*, only how nodes relate.
|
||||||
//!
|
//!
|
||||||
//! # Declaration order
|
//! # Declaration order
|
||||||
|
|
@ -29,7 +29,7 @@
|
||||||
use std::cell::RefCell;
|
use std::cell::RefCell;
|
||||||
use std::collections::HashMap;
|
use std::collections::HashMap;
|
||||||
|
|
||||||
use crate::{Dep, DepWhen, Graph, GraphError, NodeId, TerminalState};
|
use crate::{Dep, DepWhen, GraphError, NodeId, TerminalState};
|
||||||
|
|
||||||
/// An opaque identity for a node **within the job being built**.
|
/// An opaque identity for a node **within the job being built**.
|
||||||
///
|
///
|
||||||
|
|
@ -88,24 +88,48 @@ pub enum BuildError {
|
||||||
Graph(#[from] GraphError),
|
Graph(#[from] GraphError),
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Look each handle a job asked for up in what the insert actually minted,
|
/// Reject a job whose own declarations don't hold up — **before anything is
|
||||||
/// preserving the order it asked in — the last step of both insertion entry
|
/// inserted**, so these three failures cannot leave a partial job behind.
|
||||||
/// points.
|
///
|
||||||
|
/// Each is decidable from what the builder already holds: a node may only name
|
||||||
|
/// handles declared before it, and a job may only ask for handles it declared.
|
||||||
|
/// Running this first is what lets the insert loop index `ids` directly instead
|
||||||
|
/// of discovering a bad reference halfway through mutating the graph.
|
||||||
///
|
///
|
||||||
/// # Errors
|
/// # Errors
|
||||||
/// [`BuildError::UnknownNode`] for a handle this job never issued.
|
/// [`BuildError::ForwardEdge`] / [`BuildError::ForwardParent`] for a reference
|
||||||
pub(crate) fn resolve_wanted(
|
/// to a later node, [`BuildError::UnknownNode`] for a requested handle this job
|
||||||
|
/// never declared.
|
||||||
|
fn check_declaration_order<N, R>(
|
||||||
|
pending: &[Pending<N, R>],
|
||||||
wanted: &[NodeGuid],
|
wanted: &[NodeGuid],
|
||||||
ids: &HashMap<NodeGuid, NodeId>,
|
) -> Result<(), BuildError> {
|
||||||
) -> Result<Vec<NodeId>, BuildError> {
|
let mut declared: std::collections::HashSet<NodeGuid> = std::collections::HashSet::new();
|
||||||
wanted
|
for node in pending {
|
||||||
.iter()
|
for (dep, _) in &node.deps {
|
||||||
.map(|g| {
|
if !declared.contains(dep) {
|
||||||
ids.get(g)
|
return Err(BuildError::ForwardEdge {
|
||||||
.copied()
|
node: node.guid,
|
||||||
.ok_or(BuildError::UnknownNode { node: *g })
|
dep: *dep,
|
||||||
})
|
});
|
||||||
.collect()
|
}
|
||||||
|
}
|
||||||
|
if let Some(parent) = node.parent
|
||||||
|
&& !declared.contains(&parent)
|
||||||
|
{
|
||||||
|
return Err(BuildError::ForwardParent {
|
||||||
|
node: node.guid,
|
||||||
|
parent,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
declared.insert(node.guid);
|
||||||
|
}
|
||||||
|
for guid in wanted {
|
||||||
|
if !declared.contains(guid) {
|
||||||
|
return Err(BuildError::UnknownNode { node: *guid });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
/// One node as the builder holds it: edges and parent still name *handles*, so
|
/// One node as the builder holds it: edges and parent still name *handles*, so
|
||||||
|
|
@ -143,8 +167,8 @@ impl<N, R> JobBuilder<N, R> {
|
||||||
/// A fresh, empty builder.
|
/// A fresh, empty builder.
|
||||||
///
|
///
|
||||||
/// **Crate-private, and that is the API.** A builder is only ever handed to
|
/// **Crate-private, and that is the API.** A builder is only ever handed to
|
||||||
/// a closure by an insertion entry point ([`Graph::insert_job`],
|
/// a closure by the single insertion entry point
|
||||||
/// [`crate::scheduler::Scheduler::insert_job`]), which inserts the declared
|
/// ([`crate::scheduler::Scheduler::insert_job`]), which inserts the declared
|
||||||
/// nodes and returns the ids. Nothing job-shaped is constructible or
|
/// nodes and returns the ids. Nothing job-shaped is constructible or
|
||||||
/// carryable outside this crate — otherwise it is a spec factory again,
|
/// carryable outside this crate — otherwise it is a spec factory again,
|
||||||
/// just with a builder's name on it.
|
/// just with a builder's name on it.
|
||||||
|
|
@ -195,64 +219,35 @@ impl<N, R> JobBuilder<N, R> {
|
||||||
///
|
///
|
||||||
/// [`BuildError::ForwardEdge`] / [`BuildError::ForwardParent`] if a node
|
/// [`BuildError::ForwardEdge`] / [`BuildError::ForwardParent`] if a node
|
||||||
/// references one declared after it, or [`BuildError::Graph`] if the graph
|
/// references one declared after it, or [`BuildError::Graph`] if the graph
|
||||||
/// rejects a node (see [`Graph::insert`]).
|
/// rejects a node (see [`crate::Graph::insert`]).
|
||||||
pub(crate) fn insert_into(
|
|
||||||
self,
|
|
||||||
graph: &mut Graph<N, R>,
|
|
||||||
root_parent: Option<NodeId>,
|
|
||||||
) -> Result<HashMap<NodeGuid, NodeId>, BuildError> {
|
|
||||||
self.insert_with(root_parent, |payload, deps, parent| {
|
|
||||||
graph.insert(payload, deps, parent)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
/// [`JobBuilder::insert_into`] against an arbitrary sink — the same
|
|
||||||
/// resolution, for a caller that inserts through something wrapping the
|
|
||||||
/// graph (e.g. [`crate::scheduler::Scheduler::insert_job`], which has
|
|
||||||
/// bookkeeping of its own to do per node).
|
|
||||||
///
|
|
||||||
/// # Errors
|
|
||||||
///
|
|
||||||
/// As [`JobBuilder::insert_into`].
|
|
||||||
///
|
|
||||||
/// # Partial insertion
|
|
||||||
///
|
|
||||||
/// An error leaves the nodes inserted *before* it in the sink. Callers that
|
|
||||||
/// need all-or-nothing should insert into a scratch graph, or treat a
|
|
||||||
/// failure as fatal — every variant is a programming error in the job's own
|
|
||||||
/// shape, not a runtime condition to recover from.
|
|
||||||
pub(crate) fn insert_with(
|
pub(crate) fn insert_with(
|
||||||
self,
|
self,
|
||||||
root_parent: Option<NodeId>,
|
root_parent: Option<NodeId>,
|
||||||
|
wanted: &[NodeGuid],
|
||||||
mut insert: impl FnMut(N, Vec<Dep<R>>, Option<NodeId>) -> Result<NodeId, GraphError>,
|
mut insert: impl FnMut(N, Vec<Dep<R>>, Option<NodeId>) -> Result<NodeId, GraphError>,
|
||||||
) -> Result<HashMap<NodeGuid, NodeId>, BuildError> {
|
) -> Result<Vec<NodeId>, BuildError> {
|
||||||
|
let pending = self.nodes.into_inner();
|
||||||
|
check_declaration_order(&pending, wanted)?;
|
||||||
|
|
||||||
let mut ids: HashMap<NodeGuid, NodeId> = HashMap::new();
|
let mut ids: HashMap<NodeGuid, NodeId> = HashMap::new();
|
||||||
for pending in self.nodes.into_inner() {
|
for node in pending {
|
||||||
let parent = match pending.parent {
|
let parent = match node.parent {
|
||||||
None => root_parent,
|
None => root_parent,
|
||||||
Some(p) => Some(*ids.get(&p).ok_or(BuildError::ForwardParent {
|
Some(p) => Some(ids[&p]),
|
||||||
node: pending.guid,
|
|
||||||
parent: p,
|
|
||||||
})?),
|
|
||||||
};
|
};
|
||||||
let mut deps: Vec<Dep<R>> = Vec::with_capacity(pending.deps.len());
|
let mut deps: Vec<Dep<R>> = Vec::with_capacity(node.deps.len());
|
||||||
for (on, when) in pending.deps {
|
for (on, when) in node.deps {
|
||||||
let id = *ids.get(&on).ok_or(BuildError::ForwardEdge {
|
deps.push(Dep::Node { id: ids[&on], when });
|
||||||
node: pending.guid,
|
|
||||||
dep: on,
|
|
||||||
})?;
|
|
||||||
deps.push(Dep::Node { id, when });
|
|
||||||
}
|
}
|
||||||
deps.extend(
|
deps.extend(
|
||||||
pending
|
node.resources
|
||||||
.resources
|
|
||||||
.into_iter()
|
.into_iter()
|
||||||
.map(|(name, count)| Dep::Resource { name, count }),
|
.map(|(name, count)| Dep::Resource { name, count }),
|
||||||
);
|
);
|
||||||
let id = insert(pending.payload, deps, parent)?;
|
let id = insert(node.payload, deps, parent)?;
|
||||||
ids.insert(pending.guid, id);
|
ids.insert(node.guid, id);
|
||||||
}
|
}
|
||||||
Ok(ids)
|
Ok(wanted.iter().map(|g| ids[g]).collect())
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Apply `f` to the node named by `guid`.
|
/// Apply `f` to the node named by `guid`.
|
||||||
|
|
@ -393,22 +388,25 @@ impl<N, R> NodeRef<'_, N, R> {
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::BuildError;
|
use super::BuildError;
|
||||||
|
use crate::resources::ResourceTable;
|
||||||
|
use crate::scheduler::Scheduler;
|
||||||
use crate::{Dep, DepWhen, Graph, NodeId};
|
use crate::{Dep, DepWhen, Graph, NodeId};
|
||||||
|
|
||||||
/// A graph whose payload is a name and whose resources are strings.
|
/// A scheduler over a graph whose payload is a name and whose resources are
|
||||||
fn graph() -> Graph<&'static str, &'static str> {
|
/// strings — the only way in, since insertion is a scheduler operation.
|
||||||
Graph::new()
|
fn sched() -> Scheduler<&'static str, &'static str> {
|
||||||
|
Scheduler::new(Graph::new(), ResourceTable::new())
|
||||||
}
|
}
|
||||||
|
|
||||||
fn deps_of(g: &Graph<&'static str, &'static str>, id: NodeId) -> Vec<Dep<&'static str>> {
|
fn deps_of(g: &Scheduler<&'static str, &'static str>, id: NodeId) -> Vec<Dep<&'static str>> {
|
||||||
g.node(id).expect("node present").deps.clone()
|
g.graph().node(id).expect("node present").deps.clone()
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The point of the handle layer: an edge declared against a *handle* comes
|
/// The point of the handle layer: an edge declared against a *handle* comes
|
||||||
/// out addressing the id that node was actually minted as.
|
/// out addressing the id that node was actually minted as.
|
||||||
#[test]
|
#[test]
|
||||||
fn edges_resolve_to_minted_ids() {
|
fn edges_resolve_to_minted_ids() {
|
||||||
let mut g = graph();
|
let mut g = sched();
|
||||||
let ids = g
|
let ids = g
|
||||||
.insert_job(None, |b| {
|
.insert_job(None, |b| {
|
||||||
let first = b.node("a");
|
let first = b.node("a");
|
||||||
|
|
@ -431,7 +429,7 @@ mod tests {
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn parent_resolves_to_a_minted_id() {
|
fn parent_resolves_to_a_minted_id() {
|
||||||
let mut g = graph();
|
let mut g = sched();
|
||||||
let ids = g
|
let ids = g
|
||||||
.insert_job(None, |b| {
|
.insert_job(None, |b| {
|
||||||
let root = b.node("a");
|
let root = b.node("a");
|
||||||
|
|
@ -442,15 +440,15 @@ mod tests {
|
||||||
let [root, child] = ids[..] else {
|
let [root, child] = ids[..] else {
|
||||||
panic!("two ids back")
|
panic!("two ids back")
|
||||||
};
|
};
|
||||||
assert_eq!(g.node(root).expect("root").parent, None);
|
assert_eq!(g.graph().node(root).expect("root").parent, None);
|
||||||
assert_eq!(g.node(child).expect("child").parent, Some(root));
|
assert_eq!(g.graph().node(child).expect("child").parent, Some(root));
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A handle is `Copy`, so naming the same node as a dependency twice must
|
/// A handle is `Copy`, so naming the same node as a dependency twice must
|
||||||
/// not consume it — the fan-out every composite job needs.
|
/// not consume it — the fan-out every composite job needs.
|
||||||
#[test]
|
#[test]
|
||||||
fn one_handle_can_be_depended_on_twice() {
|
fn one_handle_can_be_depended_on_twice() {
|
||||||
let mut g = graph();
|
let mut g = sched();
|
||||||
let ids = g
|
let ids = g
|
||||||
.insert_job(None, |b| {
|
.insert_job(None, |b| {
|
||||||
let shared = b.node("a");
|
let shared = b.node("a");
|
||||||
|
|
@ -481,7 +479,7 @@ mod tests {
|
||||||
/// Resource deps ride along with the node deps, in one insert.
|
/// Resource deps ride along with the node deps, in one insert.
|
||||||
#[test]
|
#[test]
|
||||||
fn resources_become_resource_deps() {
|
fn resources_become_resource_deps() {
|
||||||
let mut g = graph();
|
let mut g = sched();
|
||||||
let ids = g
|
let ids = g
|
||||||
.insert_job(None, |b| {
|
.insert_job(None, |b| {
|
||||||
vec![
|
vec![
|
||||||
|
|
@ -516,7 +514,7 @@ mod tests {
|
||||||
/// of quietly reordering.
|
/// of quietly reordering.
|
||||||
#[test]
|
#[test]
|
||||||
fn a_forward_edge_is_rejected_by_name() {
|
fn a_forward_edge_is_rejected_by_name() {
|
||||||
let mut g = graph();
|
let mut g = sched();
|
||||||
let mut named = None;
|
let mut named = None;
|
||||||
let err = g
|
let err = g
|
||||||
.insert_job(None, |b| {
|
.insert_job(None, |b| {
|
||||||
|
|
@ -541,7 +539,7 @@ mod tests {
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn a_forward_parent_is_rejected_by_name() {
|
fn a_forward_parent_is_rejected_by_name() {
|
||||||
let mut g = graph();
|
let mut g = sched();
|
||||||
let mut named = None;
|
let mut named = None;
|
||||||
let err = g
|
let err = g
|
||||||
.insert_job(None, |b| {
|
.insert_job(None, |b| {
|
||||||
|
|
@ -571,7 +569,7 @@ mod tests {
|
||||||
/// would resolve, wrongly, to the second job's own node.
|
/// would resolve, wrongly, to the second job's own node.
|
||||||
#[test]
|
#[test]
|
||||||
fn a_handle_from_another_job_is_not_silently_resolved() {
|
fn a_handle_from_another_job_is_not_silently_resolved() {
|
||||||
let mut g = graph();
|
let mut g = sched();
|
||||||
let mut foreign = None;
|
let mut foreign = None;
|
||||||
g.insert_job(None, |b| {
|
g.insert_job(None, |b| {
|
||||||
foreign = Some(b.node("first job").guid());
|
foreign = Some(b.node("first job").guid());
|
||||||
|
|
@ -596,7 +594,7 @@ mod tests {
|
||||||
/// pre-empt it.
|
/// pre-empt it.
|
||||||
#[test]
|
#[test]
|
||||||
fn graph_rejection_surfaces_as_is() {
|
fn graph_rejection_surfaces_as_is() {
|
||||||
let mut g = graph();
|
let mut g = sched();
|
||||||
let err = g
|
let err = g
|
||||||
.insert_job(None, |b| {
|
.insert_job(None, |b| {
|
||||||
let root = b.node("root");
|
let root = b.node("root");
|
||||||
|
|
@ -614,8 +612,8 @@ mod tests {
|
||||||
/// template be written without knowing the container it will live under.
|
/// template be written without knowing the container it will live under.
|
||||||
#[test]
|
#[test]
|
||||||
fn root_parent_adopts_only_the_jobs_own_roots() {
|
fn root_parent_adopts_only_the_jobs_own_roots() {
|
||||||
let mut g = graph();
|
let mut g = sched();
|
||||||
let container = g.insert("container", Vec::new(), None).expect("container");
|
let container = g.append("container", Vec::new(), None).expect("container");
|
||||||
|
|
||||||
let ids = g
|
let ids = g
|
||||||
.insert_job(Some(container), |b| {
|
.insert_job(Some(container), |b| {
|
||||||
|
|
@ -627,8 +625,8 @@ mod tests {
|
||||||
let [root, child] = ids[..] else {
|
let [root, child] = ids[..] else {
|
||||||
panic!("two ids back")
|
panic!("two ids back")
|
||||||
};
|
};
|
||||||
assert_eq!(g.node(root).expect("root").parent, Some(container));
|
assert_eq!(g.graph().node(root).expect("root").parent, Some(container));
|
||||||
assert_eq!(g.node(child).expect("child").parent, Some(root));
|
assert_eq!(g.graph().node(child).expect("child").parent, Some(root));
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A handle that names no node in *this* job is refused rather than
|
/// A handle that names no node in *this* job is refused rather than
|
||||||
|
|
@ -636,7 +634,7 @@ mod tests {
|
||||||
/// short vector would misalign every id after it.
|
/// short vector would misalign every id after it.
|
||||||
#[test]
|
#[test]
|
||||||
fn asking_for_a_foreign_handle_is_an_error() {
|
fn asking_for_a_foreign_handle_is_an_error() {
|
||||||
let mut g = graph();
|
let mut g = sched();
|
||||||
let mut foreign = None;
|
let mut foreign = None;
|
||||||
g.insert_job(None, |b| {
|
g.insert_job(None, |b| {
|
||||||
foreign = Some(b.node("first job").guid());
|
foreign = Some(b.node("first job").guid());
|
||||||
|
|
@ -656,9 +654,9 @@ mod tests {
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn an_empty_builder_inserts_nothing() {
|
fn an_empty_builder_inserts_nothing() {
|
||||||
let mut g = graph();
|
let mut g = sched();
|
||||||
let ids = g.insert_job(None, |_| Vec::new()).expect("insert");
|
let ids = g.insert_job(None, |_| Vec::new()).expect("insert");
|
||||||
assert!(ids.is_empty());
|
assert!(ids.is_empty());
|
||||||
assert_eq!(g.nodes().count(), 0);
|
assert_eq!(g.graph().nodes().count(), 0);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -458,35 +458,6 @@ impl<N, R> Graph<N, R> {
|
||||||
Ok(id)
|
Ok(id)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Insert a whole job under `root_parent`, returning the ids of the nodes
|
|
||||||
/// `declare` **asked for**, in the order it named them.
|
|
||||||
///
|
|
||||||
/// `declare` receives a fresh [`JobBuilder`], names the job's nodes on it,
|
|
||||||
/// and returns the handles whose ids it wants back. The builder never
|
|
||||||
/// leaves this call, so a job cannot be built in one place and inserted in
|
|
||||||
/// another. The graph-only counterpart of
|
|
||||||
/// [`crate::scheduler::Scheduler::insert_job`] — prefer that one when a
|
|
||||||
/// scheduler owns the graph, since it also records what it started.
|
|
||||||
///
|
|
||||||
/// Asking is how a caller addresses a node it created: the alternative — a
|
|
||||||
/// map of everything inserted, or a positional vector — either hands back a
|
|
||||||
/// lookup nobody performs or reintroduces the counting this API exists to
|
|
||||||
/// remove.
|
|
||||||
///
|
|
||||||
/// # Errors
|
|
||||||
/// Propagates [`BuildError`] — a forward reference in the job's own
|
|
||||||
/// declarations, a handle from a different job, or a graph rejection.
|
|
||||||
pub fn insert_job(
|
|
||||||
&mut self,
|
|
||||||
root_parent: Option<NodeId>,
|
|
||||||
declare: impl FnOnce(&JobBuilder<N, R>) -> Vec<NodeGuid>,
|
|
||||||
) -> Result<Vec<NodeId>, BuildError> {
|
|
||||||
let job = JobBuilder::new();
|
|
||||||
let wanted = declare(&job);
|
|
||||||
let ids = job.insert_into(self, root_parent)?;
|
|
||||||
crate::builder::resolve_wanted(&wanted, &ids)
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Borrow a node by id.
|
/// Borrow a node by id.
|
||||||
#[must_use]
|
#[must_use]
|
||||||
pub fn node(&self, id: NodeId) -> Option<&Node<N, R>> {
|
pub fn node(&self, id: NodeId) -> Option<&Node<N, R>> {
|
||||||
|
|
|
||||||
|
|
@ -110,15 +110,19 @@ impl<N, R: Clone + Eq + Hash> Scheduler<N, R> {
|
||||||
/// insert one itself, so there is no way to end up with a job-shaped value
|
/// insert one itself, so there is no way to end up with a job-shaped value
|
||||||
/// being passed around as a spec.
|
/// being passed around as a spec.
|
||||||
///
|
///
|
||||||
/// The scheduler-side counterpart of [`Graph::insert_job`]: same
|
/// The one insertion entry point: every node goes through
|
||||||
/// resolution, but each node goes through [`Scheduler::append`], so a
|
/// [`Scheduler::append`], so a caller never has to reach past the scheduler
|
||||||
/// caller never has to reach past the scheduler at the graph underneath.
|
/// at the graph underneath. Call [`Scheduler::settle`] afterwards to start
|
||||||
/// Call [`Scheduler::settle`] afterwards to start whatever became
|
/// whatever became runnable.
|
||||||
/// runnable.
|
///
|
||||||
|
/// **Atomic in the job's own shape.** A forward edge, a forward parent, or
|
||||||
|
/// a request for a handle this job never declared is rejected *before* the
|
||||||
|
/// first node is inserted, so a malformed job leaves the graph untouched
|
||||||
|
/// rather than half-built.
|
||||||
///
|
///
|
||||||
/// # Errors
|
/// # Errors
|
||||||
/// Propagates [`BuildError`] — a forward reference in the job's own
|
/// Propagates [`BuildError`] — a forward reference in the job's own
|
||||||
/// declarations, or a graph rejection.
|
/// declarations, a handle from a different job, or a graph rejection.
|
||||||
pub fn insert_job(
|
pub fn insert_job(
|
||||||
&mut self,
|
&mut self,
|
||||||
root_parent: Option<NodeId>,
|
root_parent: Option<NodeId>,
|
||||||
|
|
@ -126,10 +130,9 @@ impl<N, R: Clone + Eq + Hash> Scheduler<N, R> {
|
||||||
) -> Result<Vec<NodeId>, BuildError> {
|
) -> Result<Vec<NodeId>, BuildError> {
|
||||||
let job = JobBuilder::new();
|
let job = JobBuilder::new();
|
||||||
let wanted = declare(&job);
|
let wanted = declare(&job);
|
||||||
let ids = job.insert_with(root_parent, |payload, deps, parent| {
|
job.insert_with(root_parent, &wanted, |payload, deps, parent| {
|
||||||
self.append(payload, deps, parent)
|
self.append(payload, deps, parent)
|
||||||
})?;
|
})
|
||||||
crate::builder::resolve_wanted(&wanted, &ids)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Claim every currently-runnable pending node and start it: node-deps
|
/// Claim every currently-runnable pending node and start it: node-deps
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue