refactor(job-queue): build DAGs by naming nodes, not counting them
Every template built a `Vec<NodeSpec>` whose edges and parents were positional indices into that vector, so a shape was expressed as arithmetic: `base + 1`, `stop_root + 2`, `sfu + 1`, and a `reconcile_index()` helper that read the emitted vector's length to find out where its own last node had landed. `concat_subgraphs` existed solely to rebase one per-agent subgraph's indices onto another's. Templates now declare into a `hive_jobq::JobBuilder` and hold the handles they get back, so an edge names the node it waits on. The arithmetic is gone, and with it: - `NodeSpec` and the job-queue's own index-based `Dep`. - `insert_group`'s index resolution — it wraps `Scheduler::insert_job`. - `concat_subgraphs` — per-agent chains share one builder and each keeps its own root, so independence is structural rather than computed. - `reconcile_index` and `dep_index`. - `templates::validate` and its petgraph toposort. It rejected dangling deps and cycles; both are now unrepresentable, since a handle only exists for an already-declared node and every edge therefore points backwards. (petgraph stays in the tree for `agent_config::topology`.) `NodeOutput.append_subgraph` becomes `Vec<Job>`: an executor cannot reach the queue, so it hands back declarations and the scheduler inserts them under its own lock. That is what the in-DAG growth path always wanted — a transferable declaration, not a vector of specs. Resource declaration is unchanged in behaviour: the `templates::node` helper applies `NodeKind::resource_deps()` at the construction site, so every node still declares what its kind needs. Moving that declaration to the call sites is #2818's job; this leaves it one place to delete. Three tests went with the guard they covered — they hand-built malformed specs out of indices, which is the representation that made those shapes possible. Two more now read a DAG's shape off the queue rather than out of a spec vector, which is where it is observable. The remaining 45 job-queue tests are unchanged and still pass: lease serialization, roll-up, cancel-cascade, in-DAG growth and per-agent concurrency all behave as before.
This commit is contained in:
atlas
mara
parent
9be7731c5e
commit
e7c3cf5a3d
9 changed files with 528 additions and 696 deletions
|
|
@ -47,9 +47,19 @@ use hive_jobq::{Dep, Graph, NodeId};
|
|||
use tokio::sync::Notify;
|
||||
|
||||
pub use hive_jobq::TerminalState;
|
||||
pub use model::{DagSpec, DagView, NodeKind, NodeSpec, PermPayload, Source, State};
|
||||
pub use model::{DagSpec, DagView, NodeKind, PermPayload, Source, State};
|
||||
use resource::Resource;
|
||||
|
||||
/// A job under construction: `hive_jobq`'s builder over this queue's payload
|
||||
/// ([`NodeKind`]) and resource ([`Resource`]) types. Templates declare into
|
||||
/// one of these; [`JobQueue::submit`] inserts it.
|
||||
pub type Job = hive_jobq::JobBuilder<NodeKind, Resource>;
|
||||
|
||||
/// A handle to one node a template declared — where its edges, grouping and
|
||||
/// resources are declared. `Copy`; naming a node as a dependency does not
|
||||
/// consume the ability to name it again.
|
||||
pub type Handle<'a> = hive_jobq::NodeRef<'a, NodeKind, Resource>;
|
||||
|
||||
/// How many terminal DAGs (`Done` / `Failed` / `Cancelled`) the snapshot
|
||||
/// retains, newest first. A flat cap over the whole sorted list: the
|
||||
/// dashboard renders one recent-builds list, so one number bounds it.
|
||||
|
|
@ -143,56 +153,33 @@ impl Default for JobQueue {
|
|||
}
|
||||
}
|
||||
|
||||
/// Insert `nodes` into the shared graph, honouring the spec's explicit **parent
|
||||
/// axis**: a node with `parent = None` is a top-level group root (re-parented to
|
||||
/// `group_parent`, which is `None` for `submit` and the emitting node for
|
||||
/// `append_subgraph`); a node with `parent = Some(idx)` becomes a child of the
|
||||
/// already-inserted node at spec index `idx`. `deps` are translated to crate
|
||||
/// `Dep::Node` edges verbatim — templates declare the parent axis + sibling
|
||||
/// ordering directly, so there is no dep-on-root to drop and no lease to hoist:
|
||||
/// each node declares its own `Dep::Resource`, and the crate's borrow model
|
||||
/// keeps a resource continuous across a subtree (a root owns it, descendants
|
||||
/// borrow it). Independent group roots (multiple `parent = None` nodes) carry no
|
||||
/// cross-links, so a multi-agent DAG's per-agent subgraphs run concurrently, each
|
||||
/// on its own lease. Records per-node `node_rt`. Returns the inserted ids
|
||||
/// (index-aligned with `nodes`). A node with `parent = None` is re-parented to
|
||||
/// `group_parent` (the DAG container for a template, or the emitting node for a
|
||||
/// runtime-appended subgraph); a node's `parent` / dep targets must precede it
|
||||
/// in `nodes` (submit-time `validate` enforces density + acyclicity).
|
||||
/// Insert a declared `job` into the shared graph and record its per-node
|
||||
/// `node_rt`, returning the inserted ids.
|
||||
///
|
||||
/// A node that declared no parent hangs under `group_parent` — the DAG
|
||||
/// container for a template, the emitting node for a runtime-appended
|
||||
/// subgraph. Templates declare the parent axis + sibling ordering directly, so
|
||||
/// there is no dep-on-root to drop and no lease to hoist: each node declares
|
||||
/// its own resources, and the crate's borrow model keeps a resource continuous
|
||||
/// across a subtree (a root owns it, descendants borrow it). Independent group
|
||||
/// roots carry no cross-links, so a multi-agent DAG's per-agent subgraphs run
|
||||
/// concurrently, each on its own lease.
|
||||
///
|
||||
/// # Errors
|
||||
/// Propagates a crate graph-insert error (malformed dep/parent / dep-scope).
|
||||
fn insert_group(
|
||||
inner: &mut QueueInner,
|
||||
nodes: &[NodeSpec],
|
||||
job: Job,
|
||||
group_parent: Option<NodeId>,
|
||||
) -> anyhow::Result<Vec<NodeId>> {
|
||||
let mut ids: Vec<NodeId> = Vec::with_capacity(nodes.len());
|
||||
for ns in nodes {
|
||||
let payload = ns.kind.clone();
|
||||
let mut deps: Vec<Dep<Resource>> = payload
|
||||
.resource_deps()
|
||||
.into_iter()
|
||||
.map(|(name, count)| Dep::Resource { name, count })
|
||||
.collect();
|
||||
for d in &ns.deps {
|
||||
deps.push(Dep::Node {
|
||||
id: ids[dep_index(d.on)],
|
||||
when: d.when,
|
||||
});
|
||||
}
|
||||
let parent = match ns.parent {
|
||||
Some(idx) => Some(ids[dep_index(idx)]),
|
||||
None => group_parent,
|
||||
};
|
||||
let id = inner
|
||||
.sched
|
||||
.append(payload, deps, parent)
|
||||
.map_err(|e| anyhow::anyhow!("job_queue: graph insert failed: {e}"))?;
|
||||
ids.push(id);
|
||||
let ids = inner
|
||||
.sched
|
||||
.insert_job(job, group_parent)
|
||||
.map_err(|e| anyhow::anyhow!("job_queue: graph insert failed: {e}"))?;
|
||||
for &id in ids.values() {
|
||||
inner.node_rt.insert(id, NodeRuntime::default());
|
||||
}
|
||||
Ok(ids)
|
||||
Ok(ids.into_values().collect())
|
||||
}
|
||||
|
||||
impl JobQueue {
|
||||
|
|
@ -225,7 +212,6 @@ impl JobQueue {
|
|||
/// Propagates the spec-validation error (empty / cyclic / bad parent) or a
|
||||
/// graph-insert error (dependencies that aren't dependency-topological).
|
||||
pub fn submit(&self, spec: DagSpec) -> anyhow::Result<u64> {
|
||||
templates::validate(&spec)?;
|
||||
let mut inner = self.lock();
|
||||
let container = inner
|
||||
.sched
|
||||
|
|
@ -240,7 +226,7 @@ impl JobQueue {
|
|||
)
|
||||
.map_err(|e| anyhow::anyhow!("job_queue: container insert failed: {e}"))?;
|
||||
inner.node_rt.insert(container, NodeRuntime::default());
|
||||
insert_group(&mut inner, &spec.nodes, Some(container))?;
|
||||
insert_group(&mut inner, spec.job, Some(container))?;
|
||||
// Settle the container's own (no-op) logic immediately so it parks in
|
||||
// `Finishing` and its children become runnable — it never needs claiming
|
||||
// or executing, and stays out of `claim_ready`. It rolls up terminal when
|
||||
|
|
@ -261,8 +247,8 @@ impl JobQueue {
|
|||
/// the DAG's terminal node deps on the top root, roll-up keeps the DAG from
|
||||
/// settling early with no explicit wiring. Returns the new node ids; empty if
|
||||
/// the DAG is gone or `nodes` is empty.
|
||||
pub fn append_subgraph(&self, dag_id: u64, nodes: &[NodeSpec], dep_on: NodeId) -> Vec<NodeId> {
|
||||
if nodes.is_empty() {
|
||||
pub fn append_subgraph(&self, dag_id: u64, job: Job, dep_on: NodeId) -> Vec<NodeId> {
|
||||
if job.is_empty() {
|
||||
return Vec::new();
|
||||
}
|
||||
let mut inner = self.lock();
|
||||
|
|
@ -275,7 +261,7 @@ impl JobQueue {
|
|||
// emitter stays `Finishing` until this appended subtree settles, and the
|
||||
// container node rolls up terminal only once its whole subtree (incl. this
|
||||
// appended work) has settled, so the DAG hook waits for free.
|
||||
let ids = match insert_group(&mut inner, nodes, Some(dep_on)) {
|
||||
let ids = match insert_group(&mut inner, job, Some(dep_on)) {
|
||||
Ok(ids) => ids,
|
||||
Err(e) => {
|
||||
tracing::error!(
|
||||
|
|
@ -679,12 +665,6 @@ impl QueueInner {
|
|||
}
|
||||
}
|
||||
|
||||
/// A spec dependency index (`Dep.on`, a wire `u64`) as a `usize` for indexing
|
||||
/// into the node/id vectors. `templates::validate` guarantees it's in range.
|
||||
fn dep_index(on: u64) -> usize {
|
||||
usize::try_from(on).unwrap_or(usize::MAX)
|
||||
}
|
||||
|
||||
/// Truncate a node error to [`MAX_ERROR_LEN`] on a char boundary, appending `…`.
|
||||
fn truncate_error(e: &str) -> String {
|
||||
if e.len() <= MAX_ERROR_LEN {
|
||||
|
|
|
|||
Loading…
Reference in a new issue