vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

feat(#2476): grow the meta-update cascade in-DAG instead of child DAGs

Browse source 
MetaLock's non-sweep completion now grows one rebuild subgraph per
affected agent into the same DAG (append_subgraph), replacing the
fan-out-child-DAGs + cancel_children dance. Drops NodeOutput.fanout and
scheduler's fanout_specs. meta_update DAG carries Rebuilding transient so
each cascade agent gets crash-watch suppression at Swap (the property the
old child Rebuild DAGs held via their own transient); MetaLock head needs
no lease so the pseudo-agent gets no pill.

append_children/parent_id and child-DAG tests are intentionally left for
the #2453 capstone.
This commit is contained in:
atlas 2026-07-15 18:56:40 +02:00 • committed by mara
commit 2b3130f63c
5 changed files with 102 additions and 66 deletions
Download patch file Download diff file Expand all files Collapse all files

50
hive-c0re/src/job_queue/scheduler.rs
View file

@ -3,21 +3,20 @@
//! executor task per claim, and on any completion re-evaluate.
//! Concurrency comes from the build-slot count, not multiple workers.
//!
//! Also owns the two DAG-lifetime side channels the sync queue core
//! can't hold itself:
//! - the per-DAG transient guard (dashboard pill + crash-watch
//! suppression), created when a DAG acquires its agent lease and
//! dropped when the DAG settles terminal;
//! - the `MetaLock` fan-out: appending child `Rebuild` DAGs once the
//! lock bump lands, so children build against the post-bump lock
//! (and a failed bump fans out nothing — replacing the old
//! pre-enqueue + cancel-children dance).
//! Also owns the per-DAG transient guard (dashboard pill + crash-watch
//! suppression) that the sync queue core can't hold itself — created when a
//! DAG acquires its agent lease, dropped when the DAG settles terminal.
//!
//! In-DAG growth (a `MetaLock` growing rebuild subgraphs after the lock
//! bump, a `Reconcile` fanning its `Start`/`Stop`) flows through
//! `NodeOutput.append_subgraph` / `append_nodes`, applied before the
//! emitting node completes — see `handle_completion`.
use std::collections::HashMap;
use std::sync::Arc;
use super::Claim;
use super::exec::{self, NodeOutput};
use super::{Claim, Source, templates};
use crate::coordinator::Coordinator;
struct NodeDone {
@ -128,10 +127,6 @@ async fn handle_completion(
coord
.job_queue
.complete_node(claim.dag_id, claim.node_id, Ok(()));
if !output.fanout.is_empty() {
let specs = fanout_specs(&claim, output.fanout);
coord.job_queue.append_children(specs);
}
}
Err(e) => {
let msg = format!("{e:#}");
@ -165,30 +160,3 @@ async fn process_terminals(
exec::on_dag_terminal(coord, &terminal).await;
}
}
/// Child `Rebuild` specs for a completed meta-update `MetaLock` fan-out,
/// grouped under the parent via `parent_id`. Meta-update children skip the
/// per-agent relock (`relock = false`) — it would revert the bump the parent
/// just committed. This is now the meta-update cascade path only: the startup
/// sweep no longer fans out child DAGs — it grows one rebuild subgraph per
/// stale agent into its own DAG via `append_subgraph` (see
/// `exec::run_meta_lock`).
fn fanout_specs(claim: &Claim, agents: Vec<String>) -> Vec<super::DagSpec> {
let reason = if let Some(approval_id) = claim.approval_id {
format!("approval #{approval_id} meta input cascade")
} else {
"meta-update cascade".to_owned()
};
agents
.into_iter()
.map(|agent| {
templates::rebuild(
&agent,
Source::MetaUpdate,
reason.clone(),
Some(claim.dag_id),
false,
)
})
.collect()
}