vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

job_queue: fix the boot sweep's lost declarations, drop the node wrapper

Browse source 
Two review findings on the resources-at-construction change.

argus: `workers::auto_update`'s boot sweep constructs nodes through
`templates::node` too, and it was not converted. With the kind-derived
declaration gone, its sweep `MetaLock` and its per-agent `Reconcile`
silently declared no resources at all — so a boot reconcile no longer
held the agent lease and could race another DAG's container ops, and the
sweep's meta commit could land inside another node's staged deploy
window. Nothing failed to compile: removing an implicit behaviour from a
helper is invisible at every call site that relied on it.

The declarations now live in a pure `boot_nodes`, split out of
`submit_boot_tree` so they can be exercised without a `Coordinator`.
That path is the only place job nodes are built outside `job_queue/`,
which is exactly why it had no coverage; `boot_sweep_nodes_declare_
their_own_resources` closes that, asserting against declared graph edges
rather than against the kind.

mara: `templates::node` is a redundant redirect now that it no longer
derives resources — deleted, and its 43 call sites use `Job::node`
directly. The reasoning it documented moved to the module docs of
`templates.rs` and `resource.rs`, which is where it stays true.
This commit is contained in:
atlas 2026-08-02 16:22:56 +02:00 • committed by mara
commit 58a9f218f2
6 changed files with 231 additions and 180 deletions
Download patch file Download diff file Expand all files Collapse all files

2
hive-c0re/src/job_queue/exec.rs
View file

@ -424,7 +424,7 @@ async fn run_reconcile(coord: &Arc<Coordinator>, claim: &Claim) -> Result<NodeOu
// node rather than to the fact that a `Reconcile` happens to fan it out.
let lease = Resource::Agent(kind.agent().to_owned());
vec![Box::new(move |b: &super::Job| {
let _ = super::templates::node(b, kind).needs(lease);
let _ = b.node(kind).needs(lease);
}) as Declare]
};
let append_subgraph = match reconcile_action(wanted, running) {

8
hive-c0re/src/job_queue/resource.rs
View file

@ -4,10 +4,10 @@
//! payload `N`; here `R` is [`Resource`] and `N` is [`NodeKind`] directly (each
//! variant carries the agent it targets).
//!
//! **A node's resources are declared where the node is constructed**, not
//! derived from its kind — see `templates::node`. Deriving them made the
//! requirement a property of the *kind*, which let a kind that happened to run
//! under a holding ancestor declare nothing at all.
//! **A node's resources are declared where the node is constructed** with
//! `.needs(…)`, not derived from its kind. Deriving them made the requirement a
//! property of the *kind*, which let a kind that happened to run under a
//! holding ancestor declare nothing at all.
//!
//! All of a node's resource edges are acquired **atomically**
//! (`try_acquire_all`) — a node never holds one resource while waiting on

72
hive-c0re/src/job_queue/submit.rs
View file

@ -6,7 +6,7 @@
//! state, which needs an async `lifecycle::is_running` read that a pure/sync
//! template can't do. So these fns are async — they read each agent's state,
//! assemble a per-agent subgraph out of the shared pure primitives
//! (`templates::{node, rebuild_nodes}`), all declaring into ONE job
//! (`Job::node` + `templates::rebuild_nodes`), all declaring into ONE job
//! (independent per-agent roots, concurrent on their own leases).
//!
//! Dynamic shape rule: `stop`/`start` carry a head `SetWanted(w)` (durable
@ -26,7 +26,7 @@ use std::sync::Arc;
use super::model::{DagSpec, NodeKind};
use super::resource::Resource;
use super::templates::{RebuildOpts, node, rebuild_nodes};
use super::templates::{RebuildOpts, rebuild_nodes};
use super::{Job, Source, templates};
use crate::coordinator::Coordinator;
use crate::lifecycle;
@ -65,30 +65,32 @@ fn stop_chain(b: &Job, agent: &str, graceful: bool, running: bool) {
// steps are its children (borrow the lease, run once it reaches `Finishing`,
// dep-ordered among themselves).
let a = || agent.to_owned();
let wanted = node(
b,
NodeKind::SetWanted {
let wanted = b
.node(NodeKind::SetWanted {
agent: a(),
up: false,
},
)
.needs(Resource::Agent(a()));
})
.needs(Resource::Agent(a()));
// Declaration order is dependency order: the quiesce steps come first so
// the `Reconcile` that waits on them can name them.
if graceful && running {
let signal = node(b, NodeKind::Signal { agent: a() })
let signal = b
.node(NodeKind::Signal { agent: a() })
.needs(Resource::Agent(a()))
.part_of(wanted);
let drain = node(b, NodeKind::Drain { agent: a() })
let drain = b
.node(NodeKind::Drain { agent: a() })
.needs(Resource::Agent(a()))
.part_of(wanted)
.after_ok(signal);
let _ = node(b, NodeKind::Reconcile { agent: a() })
let _ = b
.node(NodeKind::Reconcile { agent: a() })
.needs(Resource::Agent(a()))
.part_of(wanted)
.after_ok(drain);
} else {
let _ = node(b, NodeKind::Reconcile { agent: a() })
let _ = b
.node(NodeKind::Reconcile { agent: a() })
.needs(Resource::Agent(a()))
.part_of(wanted);
}
@ -99,14 +101,12 @@ fn stop_chain(b: &Job, agent: &str, graceful: bool, running: bool) {
/// current derivations), otherwise a plain `Reconcile` (which starts a down
/// agent and noops an already-running one).
fn start_chain(b: &Job, agent: &str, running: bool, stale: bool) {
let wanted = node(
b,
NodeKind::SetWanted {
let wanted = b
.node(NodeKind::SetWanted {
agent: agent.to_owned(),
up: true,
},
)
.needs(Resource::Agent(agent.to_owned()));
})
.needs(Resource::Agent(agent.to_owned()));
if !running && stale {
// Rebuild subtree chained behind the `SetWanted` head. `MetaSync`,
// `Prebuild` + `Reconcile` are their own group roots (top-level, per
@ -121,14 +121,12 @@ fn start_chain(b: &Job, agent: &str, running: bool, stale: bool) {
Some(wanted),
);
} else {
let _ = node(
b,
NodeKind::Reconcile {
let _ = b
.node(NodeKind::Reconcile {
agent: agent.to_owned(),
},
)
.needs(Resource::Agent(agent.to_owned()))
.part_of(wanted);
})
.needs(Resource::Agent(agent.to_owned()))
.part_of(wanted);
}
}
@ -145,7 +143,9 @@ fn restart_chain(b: &Job, agent: &str, graceful: bool, running: bool) {
let a = || agent.to_owned();
if !running {
// Nothing to bounce — a lone Reconcile converges to intent.
let _ = node(b, NodeKind::Reconcile { agent: a() }).needs(Resource::Agent(a()));
let _ = b
.node(NodeKind::Reconcile { agent: a() })
.needs(Resource::Agent(a()));
return;
}
// Running: mechanical stop then Reconcile. The first stop node is the group
@ -157,21 +157,29 @@ fn restart_chain(b: &Job, agent: &str, graceful: bool, running: bool) {
// that step *is* the root, and the parent gate already orders it — a child
// must NOT dep on its own parent (dep-scope), so it takes no sibling edge.
if graceful {
let signal = node(b, NodeKind::Signal { agent: a() }).needs(Resource::Agent(a()));
let drain = node(b, NodeKind::Drain { agent: a() })
let signal = b
.node(NodeKind::Signal { agent: a() })
.needs(Resource::Agent(a()));
let drain = b
.node(NodeKind::Drain { agent: a() })
.needs(Resource::Agent(a()))
.part_of(signal);
let stop = node(b, NodeKind::StopForUpdate { agent: a() })
let stop = b
.node(NodeKind::StopForUpdate { agent: a() })
.needs(Resource::Agent(a()))
.part_of(signal)
.after_ok(drain);
let _ = node(b, NodeKind::Reconcile { agent: a() })
let _ = b
.node(NodeKind::Reconcile { agent: a() })
.needs(Resource::Agent(a()))
.part_of(signal)
.after_ok(stop);
} else {
let stop = node(b, NodeKind::StopForUpdate { agent: a() }).needs(Resource::Agent(a()));
let _ = node(b, NodeKind::Reconcile { agent: a() })
let stop = b
.node(NodeKind::StopForUpdate { agent: a() })
.needs(Resource::Agent(a()));
let _ = b
.node(NodeKind::Reconcile { agent: a() })
.needs(Resource::Agent(a()))
.part_of(stop);
}

193
hive-c0re/src/job_queue/templates.rs
View file

@ -1,7 +1,10 @@
//! DAG shape builders — every operation as a template over the shared
//! node primitives. Pure (no I/O); each node carries its own `agent` (there is
//! no DAG-level agent), stamped by the [`node`] helper along with the
//! resources that node's kind needs.
//! no DAG-level agent) and **declares its own resources** with `.needs(…)`
//! right where it is constructed, rather than having them derived from its
//! kind. Deriving made the requirement a property of the *kind*, so a kind that
//! happened to run under an ancestor already holding the resource could get
//! away with declaring nothing.
//!
//! ```text
//! rebuild(a): MetaSync(a) → Prebuild(a) → StopForUpdate(a) → Swap(a) →(ok) PostSwap(a) →(any) Reconcile(a)
@ -19,7 +22,7 @@
//! The hive-wide **power ops** (`stop` / `start` / `restart`) are NOT here:
//! their per-agent shape depends on live running state (an async
//! `lifecycle::is_running` read), so `submit.rs` assembles them out of the
//! primitives this module exports ([`node`], [`rebuild_nodes`]).
//! primitives this module exports ([`rebuild_nodes`]) over `Job::node`.
use hive_jobq::TerminalState;
@ -27,28 +30,6 @@ use super::model::{DagSpec, NodeKind, PermPayload, Source};
use super::resource::Resource;
use super::{Declare, Handle, Job};
/// Declare one node carrying `kind`. **Resources are not derived here** — the
/// construction site says what the node holds, with `.needs(…)`.
///
/// That is the point rather than an omission. Deriving `(name, units)` from the
/// kind made the declaration a property of the *kind*, so a kind that happened
/// to run under an ancestor holding the resource could get away with declaring
/// nothing — which is precisely how `Start` / `Stop` / `PostSwap` ended up
/// lease-exempt: one construction site fans them out from inside a
/// lease-holding `Reconcile`. The requirement is a property of the node, not of
/// the one DAG shape it is used in today.
///
/// Declaring a resource an ancestor already holds is free: a descendant
/// re-enters that grant through the crate's recursive lock rather than taking a
/// fresh unit.
///
/// The returned handle is where edges and grouping are declared, and is `Copy`
/// — naming a node as a dependency does not consume the ability to name it
/// again.
pub(crate) fn node(b: &Job, kind: NodeKind) -> Handle<'_> {
b.node(kind)
}
/// The `Rebuilt`-reporting tail pair for a rebuild-shaped DAG: the success node
/// gated on every group-root in `roots`, and the failure node gated on *its*
/// elimination.
@ -57,13 +38,10 @@ pub(crate) fn node(b: &Job, kind: NodeKind) -> Handle<'_> {
/// dropped — see [`hive_jobq::NodeRef::on_elimination_of`].
fn emit_rebuilt_tails(b: &Job, agent: &str, roots: &[Handle<'_>]) {
let ok = roots.iter().fold(
node(
b,
NodeKind::EmitRebuilt {
agent: agent.to_owned(),
ok: true,
},
),
b.node(NodeKind::EmitRebuilt {
agent: agent.to_owned(),
ok: true,
}),
hive_jobq::NodeRef::after_ok,
);
// The failure branch needs *both*: the ok branch being ruled out (that is the
@ -73,13 +51,10 @@ fn emit_rebuilt_tails(b: &Job, agent: &str, roots: &[Handle<'_>]) {
// `Reconcile` is still bringing the container back up, so reporting straight
// off the elimination would announce the failure mid-recovery.
let _failed = roots.iter().fold(
node(
b,
NodeKind::EmitRebuilt {
agent: agent.to_owned(),
ok: false,
},
)
b.node(NodeKind::EmitRebuilt {
agent: agent.to_owned(),
ok: false,
})
.on_elimination_of(ok),
hive_jobq::NodeRef::after_any,
);
@ -96,14 +71,12 @@ fn resolve_approval_tails(b: &Job, approval_id: i64, root: Handle<'_>) {
TerminalState::Failed,
TerminalState::Cancelled,
] {
let _ = node(
b,
NodeKind::ResolveApproval {
let _ = b
.node(NodeKind::ResolveApproval {
approval_id,
outcome,
},
)
.on_outcome(root, &[outcome]);
})
.on_outcome(root, &[outcome]);
}
}
@ -182,37 +155,42 @@ pub(crate) fn rebuild_nodes<'a>(
let a = || agent.to_owned();
let RebuildOpts { relock, graceful } = opts;
let mut meta_sync =
node(b, NodeKind::MetaSync { agent: a(), relock }).needs(Resource::MetaWindow);
let mut meta_sync = b
.node(NodeKind::MetaSync { agent: a(), relock })
.needs(Resource::MetaWindow);
if let Some(after) = after {
meta_sync = meta_sync.after_ok(after);
}
let prebuild = node(b, NodeKind::Prebuild { agent: a() })
let prebuild = b
.node(NodeKind::Prebuild { agent: a() })
.needs(Resource::BuildSlot)
.after_ok(meta_sync);
// The stop root hangs off `Prebuild` and owns the agent lease for
// everything below it. `StopForUpdate` parents the swap pair either way.
let stop_for_update = if graceful {
let signal = node(b, NodeKind::Signal { agent: a() })
let signal = b
.node(NodeKind::Signal { agent: a() })
.needs(Resource::Agent(a()))
.part_of(prebuild);
// `Drain` is a *child* of `Signal`, so the parent gate already orders
// it — a child must not dep on its own parent (dep-scope).
let drain = node(b, NodeKind::Drain { agent: a() })
let drain = b
.node(NodeKind::Drain { agent: a() })
.needs(Resource::Agent(a()))
.part_of(signal);
node(b, NodeKind::StopForUpdate { agent: a() })
b.node(NodeKind::StopForUpdate { agent: a() })
.needs(Resource::Agent(a()))
.part_of(signal)
.after_ok(drain)
} else {
node(b, NodeKind::StopForUpdate { agent: a() })
b.node(NodeKind::StopForUpdate { agent: a() })
.needs(Resource::Agent(a()))
.part_of(prebuild)
};
let swap = node(b, NodeKind::Swap { agent: a() })
let swap = b
.node(NodeKind::Swap { agent: a() })
.needs(Resource::BuildSlot)
.needs(Resource::Agent(a()))
.part_of(stop_for_update);
@ -220,12 +198,14 @@ pub(crate) fn rebuild_nodes<'a>(
// `StopForUpdate`, which holds it, so this is a re-entrant borrow — no
// second unit, no deadlock. Declaring it is what stops the requirement
// being true only of this one DAG shape.
let _post_swap = node(b, NodeKind::PostSwap { agent: a() })
let _post_swap = b
.node(NodeKind::PostSwap { agent: a() })
.needs(Resource::Agent(a()))
.part_of(stop_for_update)
.after_ok(swap);
let reconcile = node(b, NodeKind::Reconcile { agent: a() })
let reconcile = b
.node(NodeKind::Reconcile { agent: a() })
.needs(Resource::Agent(a()))
.after_any(prebuild);
@ -272,16 +252,14 @@ pub(crate) fn deploy_rebuild_nodes(agent: &str, approval_id: i64) -> Declare {
},
None,
);
let _finalize = node(
b,
NodeKind::FinalizeDeploy {
let _finalize = b
.node(NodeKind::FinalizeDeploy {
agent: agent.clone(),
approval_id,
},
)
.needs(Resource::MetaWindow)
.after_ok(roots.prebuild)
.after_ok(roots.reconcile);
})
.needs(Resource::MetaWindow)
.after_ok(roots.prebuild)
.after_ok(roots.reconcile);
})
}
@ -363,42 +341,34 @@ pub fn approval_deploy(
// (`MetaWindow`). All three are held for its whole subtree, which
// is what lets the appended rebuild's `MetaSync` and the
// `FinalizeDeploy` re-enter rather than contend.
let window = node(
b,
NodeKind::DeployWindow {
let window = b
.node(NodeKind::DeployWindow {
agent: a(),
approval_id,
},
)
.needs(Resource::BuildSlot)
.needs(Resource::Agent(a()))
.needs(Resource::MetaWindow);
let verify = node(
b,
NodeKind::MergeVerify {
})
.needs(Resource::BuildSlot)
.needs(Resource::Agent(a()))
.needs(Resource::MetaWindow);
let verify = b
.node(NodeKind::MergeVerify {
agent: a(),
approval_id,
},
)
.part_of(window);
let apply = node(
b,
NodeKind::DeployApply {
})
.part_of(window);
let apply = b
.node(NodeKind::DeployApply {
agent: a(),
approval_id,
},
)
.part_of(window)
.after_ok(verify);
let _tail = node(
b,
NodeKind::DeployTail {
})
.part_of(window)
.after_ok(verify);
let _tail = b
.node(NodeKind::DeployTail {
agent: a(),
approval_id,
},
)
.part_of(window)
.after_any(apply);
})
.part_of(window)
.after_any(apply);
resolve_approval_tails(b, approval_id, window);
}),
@ -423,7 +393,7 @@ pub fn reconcile_only(
declare: Box::new(move |b: &Job| {
// Name the lease before the agent string is moved into the kind.
let lease = Resource::Agent(agent.clone());
let _reconcile = node(b, NodeKind::Reconcile { agent }).needs(lease);
let _reconcile = b.node(NodeKind::Reconcile { agent }).needs(lease);
}),
}
}
@ -446,15 +416,20 @@ pub fn spawn(agent: &str, approval_id: i64, reason: String) -> DagSpec<impl FnOn
reason,
declare: Box::new(move |b: &Job| {
let a = || agent.clone();
let provision = node(b, NodeKind::Provision { agent: a() }).needs(Resource::MetaWindow);
let create = node(b, NodeKind::Create { agent: a() })
let provision = b
.node(NodeKind::Provision { agent: a() })
.needs(Resource::MetaWindow);
let create = b
.node(NodeKind::Create { agent: a() })
.needs(Resource::BuildSlot)
.needs(Resource::Agent(a()))
.part_of(provision);
let dropin = node(b, NodeKind::WriteDropin { agent: a() })
let dropin = b
.node(NodeKind::WriteDropin { agent: a() })
.needs(Resource::Agent(a()))
.part_of(create);
let _reconcile = node(b, NodeKind::Reconcile { agent: a() })
let _reconcile = b
.node(NodeKind::Reconcile { agent: a() })
.needs(Resource::Agent(a()))
.part_of(create)
.after_ok(dropin);
@ -480,14 +455,12 @@ pub fn perm_change(
source,
reason,
declare: Box::new(move |b: &Job| {
let write = node(
b,
NodeKind::WritePermFile {
let write = b
.node(NodeKind::WritePermFile {
agent: agent.clone(),
payload,
},
)
.needs(Resource::MetaWindow);
})
.needs(Resource::MetaWindow);
let roots = rebuild_nodes(
b,
&agent,
@ -526,16 +499,14 @@ pub fn meta_update(
source,
reason,
declare: Box::new(move |b: &Job| {
let lock = node(
b,
NodeKind::MetaLock {
let lock = b
.node(NodeKind::MetaLock {
sweep: false,
fanout: None,
inputs,
},
)
.needs(Resource::BuildSlot)
.needs(Resource::MetaWindow);
})
.needs(Resource::BuildSlot)
.needs(Resource::MetaWindow);
// The bump itself has no side effect, so an operator-driven one ends
// at the `MetaLock`; an approval-driven one still has its row to
// resolve and gets the per-outcome tails edged onto that single
@ -566,7 +537,9 @@ pub fn reparent(
source,
reason,
declare: Box::new(move |b: &Job| {
let _reparent = node(b, NodeKind::Reparent { moves }).needs(Resource::MetaWindow);
let _reparent = b
.node(NodeKind::Reparent { moves })
.needs(Resource::MetaWindow);
}),
}
}

66
hive-c0re/src/job_queue/tests.rs
View file

@ -766,7 +766,7 @@ fn a_fanned_out_start_declares_the_lease_and_re_enters_its_reconciles_grant() {
agent: "agent-a".to_owned(),
};
let lease = Resource::Agent(kind.agent().to_owned());
let _ = templates::node(b, kind).needs(lease);
let _ = b.node(kind).needs(lease);
}),
reconcile.node_id,
);
@ -798,6 +798,57 @@ fn a_fanned_out_start_declares_the_lease_and_re_enters_its_reconciles_grant() {
assert_eq!(state_of(&q, rival), State::Done);
}
#[test]
fn boot_sweep_nodes_declare_their_own_resources() {
// Regression, and the reason it needs its own test: `workers::auto_update`
// is the only place job nodes are constructed *outside* `job_queue/`, so
// nothing in this module covered it. When resource derivation moved to the
// construction sites, this path was missed and both kinds silently declared
// nothing — dropping the agent lease a boot `Reconcile` needs to not race
// another DAG's container ops, and letting the sweep `MetaLock` land its
// meta commit inside another node's staged deploy window. Nothing failed to
// compile; only an exhaustive caller list would have caught it.
let q = JobQueue::new(4);
let _id = submit(
&q,
DagSpec {
source: Source::AutoUpdate,
reason: "boot".to_owned(),
declare: Box::new(|b: &Job| {
crate::workers::auto_update::boot_nodes(
b,
true,
vec!["stale-agent".to_owned()],
vec!["drifted-agent".to_owned()],
);
}),
},
);
// Both are independent roots on disjoint resources, so both start at once.
let claims = q.claim_ready();
let by_kind = |kind: &str| {
claims
.iter()
.find(|c| c.kind.as_str() == kind)
.unwrap_or_else(|| panic!("no {kind} claim in {claims:?}"))
};
let mut lock = declared_resources(&q, by_kind("meta_lock").node_id);
lock.sort_by_key(|r| format!("{r:?}"));
assert_eq!(
lock,
vec![Resource::BuildSlot, Resource::MetaWindow],
"the sweep MetaLock runs a nix lock bump and commits to meta"
);
assert_eq!(
declared_resources(&q, by_kind("reconcile").node_id),
vec![Resource::Agent("drifted-agent".to_owned())],
"a boot Reconcile touches the container, so it holds that agent's lease"
);
}
#[test]
fn append_subgraph_roots_on_emitter_and_rebases_local_deps() {
// The startup-sweep mechanism: a `MetaLock` emitter grows one rebuild
@ -808,14 +859,11 @@ fn append_subgraph_roots_on_emitter_and_rebases_local_deps() {
source: Source::AutoUpdate,
reason: "sweep".to_owned(),
declare: Box::new(|b: &Job| {
let _lock = templates::node(
b,
NodeKind::MetaLock {
sweep: true,
fanout: None,
inputs: Vec::new(),
},
);
let _lock = b.node(NodeKind::MetaLock {
sweep: true,
fanout: None,
inputs: Vec::new(),
});
}),
};
let id = submit(&q, spec);

70
hive-c0re/src/workers/auto_update.rs
View file

@ -318,6 +318,49 @@ pub async fn run(coord: Arc<Coordinator>) -> Result<()> {
Ok(())
}
/// The boot DAG's node declarations, split out of [`submit_boot_tree`] so they
/// can be exercised without a live [`Coordinator`].
///
/// That split is not cosmetic: this path constructs nodes outside
/// `job_queue/`, so it is the one place a resource declaration can be forgotten
/// without any in-module test noticing. It has happened once already — the
/// sweep `MetaLock` and the boot `Reconcile`s silently declared nothing when
/// kind-derived resources were removed, which drops the agent lease a boot
/// reconcile needs to not race another DAG's container ops.
pub(crate) fn boot_nodes(
b: &crate::job_queue::Job,
any_stale: bool,
fanout: Vec<String>,
drifted: Vec<String>,
) {
use crate::job_queue::NodeKind;
use crate::job_queue::resource::Resource;
// Sweep whenever ANY marker is stale — even when every stale agent is
// wanted-offline: the hyperhive lock bump must land now so their later
// start-upgrade rebuilds build against it. No stale agents ⇒ no MetaLock
// ⇒ no meta commit on a no-change boot. The `fanout` list rides the
// MetaLock into `run_meta_lock`, which appends the rebuild subgraphs.
if any_stale {
let _ = b
.node(NodeKind::MetaLock {
sweep: true,
fanout: Some(fanout),
// A sweep bumps `hyperhive` alone (`lock_update_hyperhive`),
// so it names no inputs.
inputs: Vec::new(),
})
.needs(Resource::BuildSlot)
.needs(Resource::MetaWindow);
}
// One boot Reconcile per drifted agent — independent roots.
for name in drifted {
// Name the lease before the agent string moves into the kind.
let lease = Resource::Agent(name.clone());
let _ = b.node(NodeKind::Reconcile { agent: name }).needs(lease);
}
}
/// Submit this boot's work as **one DAG** (no anchor node, no per-agent
/// child DAGs). Node 0 is the sweep `MetaLock` (only when
/// something is stale) — its executor bumps the hyperhive lock, then grows
@ -334,7 +377,7 @@ fn submit_boot_tree(
n_deferred: usize,
n_skipped: usize,
) {
use crate::job_queue::{DagSpec, NodeKind, Source, templates};
use crate::job_queue::{DagSpec, Source};
// Fully-quiet boot (nothing stale, nothing drifted) submits nothing.
if !any_stale && drifted.is_empty() {
@ -348,29 +391,8 @@ fn submit_boot_tree(
n_skipped,
);
let declare: crate::job_queue::Declare = Box::new(move |b| {
// Sweep whenever ANY marker is stale — even when every stale agent is
// wanted-offline: the hyperhive lock bump must land now so their later
// start-upgrade rebuilds build against it. No stale agents ⇒ no MetaLock
// ⇒ no meta commit on a no-change boot. The `fanout` list rides the
// MetaLock into `run_meta_lock`, which appends the rebuild subgraphs.
if any_stale {
let _ = templates::node(
b,
NodeKind::MetaLock {
sweep: true,
fanout: Some(fanout),
// A sweep bumps `hyperhive` alone (`lock_update_hyperhive`),
// so it names no inputs.
inputs: Vec::new(),
},
);
}
// One boot Reconcile per drifted agent — independent roots.
for name in drifted {
let _ = templates::node(b, NodeKind::Reconcile { agent: name });
}
});
let declare: crate::job_queue::Declare =
Box::new(move |b| boot_nodes(b, any_stale, fanout, drifted));
let spec = DagSpec {
// The sweep's own rebuild subgraphs emit their `Rebuilt` events as they