refactor(job-queue): build DAGs by naming nodes, not counting them

Every template built a `Vec<NodeSpec>` whose edges and parents were
positional indices into that vector, so a shape was expressed as
arithmetic: `base + 1`, `stop_root + 2`, `sfu + 1`, and a
`reconcile_index()` helper that read the emitted vector's length to find
out where its own last node had landed. `concat_subgraphs` existed
solely to rebase one per-agent subgraph's indices onto another's.

Templates now declare into a `hive_jobq::JobBuilder` and hold the
handles they get back, so an edge names the node it waits on. The
arithmetic is gone, and with it:

- `NodeSpec` and the job-queue's own index-based `Dep`.
- `insert_group`'s index resolution — it wraps `Scheduler::insert_job`.
- `concat_subgraphs` — per-agent chains share one builder and each keeps
  its own root, so independence is structural rather than computed.
- `reconcile_index` and `dep_index`.
- `templates::validate` and its petgraph toposort. It rejected dangling
  deps and cycles; both are now unrepresentable, since a handle only
  exists for an already-declared node and every edge therefore points
  backwards. (petgraph stays in the tree for `agent_config::topology`.)

`NodeOutput.append_subgraph` becomes `Vec<Job>`: an executor cannot
reach the queue, so it hands back declarations and the scheduler inserts
them under its own lock. That is what the in-DAG growth path always
wanted — a transferable declaration, not a vector of specs.

Resource declaration is unchanged in behaviour: the `templates::node`
helper applies `NodeKind::resource_deps()` at the construction site, so
every node still declares what its kind needs. Moving that declaration
to the call sites is #2818's job; this leaves it one place to delete.

Three tests went with the guard they covered — they hand-built malformed
specs out of indices, which is the representation that made those shapes
possible. Two more now read a DAG's shape off the queue rather than out
of a spec vector, which is where it is observable. The remaining 45
job-queue tests are unchanged and still pass: lease serialization,
roll-up, cancel-cascade, in-DAG growth and per-agent concurrency all
behave as before.
This commit is contained in:
atlas 2026-08-02 13:05:17 +02:00 committed by mara
commit e7c3cf5a3d
9 changed files with 528 additions and 696 deletions

View file

@ -334,7 +334,7 @@ fn submit_boot_tree(
n_deferred: usize,
n_skipped: usize,
) {
use crate::job_queue::{DagSpec, NodeKind, NodeSpec, Source};
use crate::job_queue::{DagSpec, Job, NodeKind, Source, templates};
// Fully-quiet boot (nothing stale, nothing drifted) submits nothing.
if !any_stale && drifted.is_empty() {
@ -348,32 +348,27 @@ fn submit_boot_tree(
n_skipped,
);
let mut nodes: Vec<NodeSpec> = Vec::new();
let job = Job::new();
// Sweep whenever ANY marker is stale — even when every stale agent is
// wanted-offline: the hyperhive lock bump must land now so their later
// start-upgrade rebuilds build against it. No stale agents ⇒ no MetaLock
// ⇒ no meta commit on a no-change boot. The `fanout` list rides the
// MetaLock into `run_meta_lock`, which appends the rebuild subgraphs.
if any_stale {
nodes.push(NodeSpec {
kind: NodeKind::MetaLock {
let _ = templates::node(
&job,
NodeKind::MetaLock {
sweep: true,
fanout: Some(fanout),
// A sweep bumps `hyperhive` alone (`lock_update_hyperhive`),
// so it names no inputs.
inputs: Vec::new(),
},
deps: Vec::new(),
parent: None,
});
);
}
// One boot Reconcile per drifted agent — independent roots.
for name in drifted {
nodes.push(NodeSpec {
kind: NodeKind::Reconcile { agent: name },
deps: Vec::new(),
parent: None,
});
let _ = templates::node(&job, NodeKind::Reconcile { agent: name });
}
let spec = DagSpec {
@ -384,7 +379,7 @@ fn submit_boot_tree(
// Rebuilding when the sweep will grow rebuild subgraphs (per-agent
// crash-watch suppression during their Swap, applied at claim time);
// a reconcile-only boot needs no transient.
nodes,
job,
};
if let Err(e) = coord.job_queue.submit(spec) {
tracing::warn!(error = ?e, "boot: sweep DAG submit failed");