refactor(job-queue): build DAGs by naming nodes, not counting them

Every template built a `Vec<NodeSpec>` whose edges and parents were
positional indices into that vector, so a shape was expressed as
arithmetic: `base + 1`, `stop_root + 2`, `sfu + 1`, and a
`reconcile_index()` helper that read the emitted vector's length to find
out where its own last node had landed. `concat_subgraphs` existed
solely to rebase one per-agent subgraph's indices onto another's.

Templates now declare into a `hive_jobq::JobBuilder` and hold the
handles they get back, so an edge names the node it waits on. The
arithmetic is gone, and with it:

- `NodeSpec` and the job-queue's own index-based `Dep`.
- `insert_group`'s index resolution — it wraps `Scheduler::insert_job`.
- `concat_subgraphs` — per-agent chains share one builder and each keeps
  its own root, so independence is structural rather than computed.
- `reconcile_index` and `dep_index`.
- `templates::validate` and its petgraph toposort. It rejected dangling
  deps and cycles; both are now unrepresentable, since a handle only
  exists for an already-declared node and every edge therefore points
  backwards. (petgraph stays in the tree for `agent_config::topology`.)

`NodeOutput.append_subgraph` becomes `Vec<Job>`: an executor cannot
reach the queue, so it hands back declarations and the scheduler inserts
them under its own lock. That is what the in-DAG growth path always
wanted — a transferable declaration, not a vector of specs.

Resource declaration is unchanged in behaviour: the `templates::node`
helper applies `NodeKind::resource_deps()` at the construction site, so
every node still declares what its kind needs. Moving that declaration
to the call sites is #2818's job; this leaves it one place to delete.

Three tests went with the guard they covered — they hand-built malformed
specs out of indices, which is the representation that made those shapes
possible. Two more now read a DAG's shape off the queue rather than out
of a spec vector, which is where it is observable. The remaining 45
job-queue tests are unchanged and still pass: lease serialization,
roll-up, cancel-cascade, in-DAG growth and per-agent concurrency all
behave as before.
This commit is contained in:
atlas 2026-08-02 13:05:17 +02:00 committed by mara
commit e7c3cf5a3d
9 changed files with 528 additions and 696 deletions

View file

@ -10,10 +10,10 @@ use std::sync::Arc;
use anyhow::{Context as _, Result};
use super::Claim;
use super::{Claim, Job};
use hive_jobq::TerminalState;
use super::model::{NodeKind, NodeSpec};
use super::model::NodeKind;
use crate::coordinator::Coordinator;
use crate::power::{ReconcileAction, reconcile_action};
@ -30,19 +30,19 @@ pub const GRACEFUL_STOP_TIMEOUT: std::time::Duration = std::time::Duration::from
#[derive(Debug, Default)]
pub struct NodeOutput {
/// Whole per-agent *subgraphs* to append into *this same* DAG at
/// runtime — the single in-DAG-growth channel. Each inner
/// `Vec<NodeSpec>` is one independent subgraph whose `deps` are local
/// (0-based within that subgraph); the scheduler appends each via
/// [`super::JobQueue::append_subgraph`], which rebases the deps onto the DAG's
/// node-id space and roots the subgraph on the emitting node. Used both
/// for the multi-node case (`MetaLock` growing one rebuild subgraph per
/// agent — the startup sweep's stale agents, the meta-update cascade's
/// affected agents) and the single-node case (a `Reconcile` planner
/// emitting its mechanical `Start` / `Stop` as a one-node subgraph). The
/// scheduler applies these *before* the emitting node's completion so the
/// DAG never rolls terminal with the appended work still pending — keeping
/// the lease-window transient held across the sub-step.
pub append_subgraph: Vec<Vec<NodeSpec>>,
/// runtime — the single in-DAG-growth channel. Each [`Job`] is one
/// independent subgraph, declared but not yet inserted: an executor cannot
/// reach the queue, so it hands the declaration back and the scheduler
/// inserts it via [`super::JobQueue::append_subgraph`] under its own lock,
/// rooted on the emitting node. Used both for the multi-node case
/// (`MetaLock` growing one rebuild subgraph per agent — the startup
/// sweep's stale agents, the meta-update cascade's affected agents) and
/// the single-node case (a `Reconcile` planner emitting its mechanical
/// `Start` / `Stop` as a one-node subgraph). The scheduler applies these
/// *before* the emitting node's completion so the DAG never rolls terminal
/// with the appended work still pending — keeping the lease-window
/// transient held across the sub-step.
pub append_subgraph: Vec<Job>,
}
/// Build-log sink for one claimed node.
@ -342,14 +342,17 @@ async fn run_meta_lock(
.unwrap_or_default()
.iter()
.map(|agent| {
let job = Job::new();
super::templates::rebuild_nodes(
&job,
agent,
super::templates::RebuildOpts {
relock: true,
graceful: true,
},
0,
)
None,
);
job
})
.collect();
return Ok(NodeOutput { append_subgraph });
@ -371,14 +374,17 @@ async fn run_meta_lock(
let append_subgraph = cascade
.iter()
.map(|agent| {
let job = Job::new();
super::templates::rebuild_nodes(
&job,
agent,
super::templates::RebuildOpts {
relock: false,
graceful: false,
},
0,
)
None,
);
job
})
.collect();
Ok(NodeOutput { append_subgraph })
@ -398,7 +404,11 @@ async fn run_reconcile(coord: &Arc<Coordinator>, claim: &Claim) -> Result<NodeOu
// One node targeting this agent, rooted on this reconcile node. `NodeKind`
// carries the agent it targets, so stamp `claim.agent` into the fanned-out
// Start/Stop kind (one in-DAG-growth channel).
let sub = |kind| vec![vec![super::templates::node(kind, Vec::new())]];
let sub = |kind| {
let job = Job::new();
let _ = super::templates::node(&job, kind);
vec![job]
};
let append_subgraph = match reconcile_action(wanted, running) {
ReconcileAction::Start => sub(NodeKind::Start {
agent: name.clone(),