vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

refactor(#2949): kill Declare — a running node declares onto its own builder

Browse source 
A node no longer hands back a recipe for the scheduler to replay later. It
declares straight onto a builder it was given, and that builder is inserted
as part of completing the node.

Deleted: `pub type Declare`, `struct NodeOutput` (+ its hand-written `Debug`),
`JobQueue::append_subgraph`. Nothing added to `Dag` / `DagView`.

jobq gains `Scheduler::new_job()` (the only way to obtain a `JobBuilder`) and
`complete_growing(id, outcome, grown)`, which inserts under `id` and *then*
completes it, so a DAG cannot roll terminal while grown work is still pending.
`complete()` and `complete_growing()` share a private `finish()` rather than
one redirecting through the other. The DAG-gone guard lives beside the graph
now, where it cannot be skipped, instead of being a caller-side lookup.

The growth executors return data (`run_meta_lock -> (Vec<String>, RebuildOpts)`,
`run_reconcile -> Option<NodeKind>`) rather than taking the builder: a `&Job`
parameter is live for the whole function body, and `&RefCell<T>` is never
`Send`, so an async fn taking one cannot be spawned. `run_node` threads the
builder by value and hands it back.

A node can now declare work and then fail, which was previously inexpressible.
`grown` is dropped in that case — failure cancel-cascades downstream, so
inserting it would only add nodes to immediately cancel — and the log line
carries `grown_nodes` so the drop is visible.
This commit is contained in:
atlas 2026-08-02 17:20:43 +02:00 • committed by mara
commit 82ef06f445
8 changed files with 388 additions and 344 deletions
Download patch file Download diff file Expand all files Collapse all files

62
hive-c0re/src/job_queue/scheduler.rs
View file

@ -16,19 +16,22 @@
//! the DAG settles.
//!
//! In-DAG growth (a `MetaLock` growing rebuild subgraphs, a `Reconcile` fanning
//! its `Start`/`Stop`) flows through `NodeOutput.append_subgraph`, applied
//! before the emitting node completes — see `handle_completion`.
//! its `Start`/`Stop`) is declared onto the builder each node is handed, and
//! inserted as part of completing that node — see `handle_completion`.
use std::collections::HashMap;
use std::sync::Arc;
use super::Claim;
use super::exec::{self, NodeOutput};
use super::exec;
use super::{Claim, Job};
use crate::coordinator::Coordinator;
struct NodeDone {
claim: Claim,
result: anyhow::Result<NodeOutput>,
/// Whatever the node declared into its builder while running — usually
/// nothing. Inserted under the node as part of completing it.
grown: Job,
result: anyhow::Result<()>,
}
/// Scheduler loop. Spawned once at hive-c0re startup from `main.rs`.
@ -83,9 +86,18 @@ pub async fn run_worker(coord: Arc<Coordinator>) {
let coord = Arc::clone(&coord);
let tx = tx.clone();
tokio::spawn(async move {
let result = exec::run_node(&coord, &claim).await;
// The node's growth channel. Local state, so it costs
// nothing to carry and holds no lock while the node runs.
// The builder is passed by value and handed back: owned it
// is `Send`, a `&Job` held across an await is not.
let job = coord.job_queue.new_job();
let (grown, result) = exec::run_node(&coord, job, &claim).await;
// Send failure = scheduler gone (shutdown); drop.
let _ = tx.send(NodeDone { claim, result });
let _ = tx.send(NodeDone {
claim,
grown,
result,
});
});
}
// Newly-started owner nodes now hold their leases — surface the pills.
@ -110,27 +122,26 @@ pub async fn run_worker(coord: Arc<Coordinator>) {
}
fn handle_completion(coord: &Arc<Coordinator>, done: NodeDone) {
let NodeDone { claim, result } = done;
let NodeDone {
claim,
grown,
result,
} = done;
match result {
Ok(output) => {
Ok(()) => {
tracing::info!(
dag = claim.dag_id,
node = claim.node_id.get(),
"job_queue: node done"
);
// Append any in-DAG subgraphs BEFORE completing this node, so
// completing it doesn't roll the DAG terminal while the appended
// work is still pending. Each subgraph roots on this node
// (`AfterOk`), so it becomes ready the instant this one settles
// `Done` just below — covers both the multi-node case (a `MetaLock`
// Whatever the node declared goes in under it as part of this
// completion, so the DAG cannot roll terminal while the appended
// work is still pending. Covers the multi-node case (a `MetaLock`
// growing per-agent rebuild subgraphs) and the single-node case (a
// `Reconcile` planner's `Start` / `Stop`).
for subgraph in output.append_subgraph {
coord
.job_queue
.append_subgraph(claim.dag_id, subgraph, claim.node_id);
}
coord.job_queue.complete_node(claim.node_id, Ok(()));
// `Reconcile` planner's `Start` / `Stop`) identically.
coord
.job_queue
.complete_node_growing(claim.node_id, Ok(()), grown);
}
Err(e) => {
let msg = format!("{e:#}");
@ -140,8 +151,17 @@ fn handle_completion(coord: &Arc<Coordinator>, done: NodeDone) {
kind = claim.kind.as_str(),
agent = %claim.agent,
error = %msg,
grown_nodes = !grown.is_empty(),
"job_queue: node failed"
);
// `grown` is deliberately dropped on failure. A node that declared
// follow-up work and *then* failed does not want that work run —
// failure cancel-cascades downstream, so inserting it would only
// add nodes to immediately cancel. This preserves the old shape,
// where growth could only be expressed on the success path at all;
// the difference is that it is now possible to declare and then
// fail, so the drop has to be a decision rather than an accident.
drop(grown);
coord.job_queue.complete_node(claim.node_id, Err(msg));
}
}