vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

refactor(job-queue): a job is a recipe, not a value you carry

Browse source 
Follows the jobq change: a builder can no longer be constructed or
inserted outside `hive_jobq`, so `DagSpec` cannot hold one. It carries a
`Declare` — `Box<dyn FnOnce(&Job) + Send>` — and the queue runs it
against a builder jobq owns, at the moment it inserts.

`NodeOutput.append_subgraph` becomes `Vec<Declare>` for the same reason,
and this is where the shape was always heading: that field's doc already
said an executor "cannot reach the queue, so it hands the declaration
back", while its type was a `Vec<Job>` the executor had built itself.
The rejected `build_nodes -> Vec<NodeSpec>` was the first version of that
escape hatch; a recipe is the last one, because there is no job-shaped
value to hand over at all.

Templates and the power-op assemblers move their owned data into the
closure and are otherwise unchanged — `rebuild_nodes`, `node` and the
tail helpers already took `&Job` and returned handles, so only each
template's outermost frame moved.

Two `Debug` impls are hand-written: a closure has nothing to show, and
its nodes do not exist until the queue runs it. `NodeOutput` reports how
many subgraphs were emitted, `DagSpec` its source and reason.

`append_subgraph`'s `is_empty()` early-return is gone — you cannot ask a
recipe whether it will declare anything without running it. It now
inserts and returns an empty id list if nothing was declared, which
takes the queue lock in a case that previously skipped it.

The two in-DAG-growth tests build `Declare`s now, so they exercise the
shape an executor actually produces rather than one only a test could
construct. 45 job-queue tests unchanged and passing.
This commit is contained in:
atlas 2026-08-02 13:40:03 +02:00 • committed by mara
commit 9c97365f8f
8 changed files with 336 additions and 284 deletions
Download patch file Download diff file Expand all files Collapse all files

49
hive-c0re/src/workers/auto_update.rs
View file

@ -334,7 +334,7 @@ fn submit_boot_tree(
n_deferred: usize,
n_skipped: usize,
) {
use crate::job_queue::{DagSpec, Job, NodeKind, Source, templates};
use crate::job_queue::{DagSpec, NodeKind, Source, templates};
// Fully-quiet boot (nothing stale, nothing drifted) submits nothing.
if !any_stale && drifted.is_empty() {
@ -348,28 +348,29 @@ fn submit_boot_tree(
n_skipped,
);
let job = Job::new();
// Sweep whenever ANY marker is stale — even when every stale agent is
// wanted-offline: the hyperhive lock bump must land now so their later
// start-upgrade rebuilds build against it. No stale agents ⇒ no MetaLock
// ⇒ no meta commit on a no-change boot. The `fanout` list rides the
// MetaLock into `run_meta_lock`, which appends the rebuild subgraphs.
if any_stale {
let _ = templates::node(
&job,
NodeKind::MetaLock {
sweep: true,
fanout: Some(fanout),
// A sweep bumps `hyperhive` alone (`lock_update_hyperhive`),
// so it names no inputs.
inputs: Vec::new(),
},
);
}
// One boot Reconcile per drifted agent — independent roots.
for name in drifted {
let _ = templates::node(&job, NodeKind::Reconcile { agent: name });
}
let declare: crate::job_queue::Declare = Box::new(move |b| {
// Sweep whenever ANY marker is stale — even when every stale agent is
// wanted-offline: the hyperhive lock bump must land now so their later
// start-upgrade rebuilds build against it. No stale agents ⇒ no MetaLock
// ⇒ no meta commit on a no-change boot. The `fanout` list rides the
// MetaLock into `run_meta_lock`, which appends the rebuild subgraphs.
if any_stale {
let _ = templates::node(
b,
NodeKind::MetaLock {
sweep: true,
fanout: Some(fanout),
// A sweep bumps `hyperhive` alone (`lock_update_hyperhive`),
// so it names no inputs.
inputs: Vec::new(),
},
);
}
// One boot Reconcile per drifted agent — independent roots.
for name in drifted {
let _ = templates::node(b, NodeKind::Reconcile { agent: name });
}
});
let spec = DagSpec {
// The sweep's own rebuild subgraphs emit their `Rebuilt` events as they
@ -379,7 +380,7 @@ fn submit_boot_tree(
// Rebuilding when the sweep will grow rebuild subgraphs (per-agent
// crash-watch suppression during their Swap, applied at claim time);
// a reconcile-only boot needs no transient.
job,
declare,
};
if let Err(e) = coord.job_queue.submit(spec) {
tracing::warn!(error = ?e, "boot: sweep DAG submit failed");