vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

job_queue: lease release is the crate's, not the host's

Browse source 
Two more tests drove DAGs to completion to watch an agent lease free up
-- one when a single agent's subgraph settled inside a still-running
multi-agent DAG, the other when a whole power op finished. Releasing a
grant once its owner's subtree is terminal is hive-jobq's, covered by
owner_holds_grant_for_its_whole_subtree,
child_borrows_ancestor_grant_released_when_subtree_done and
leaf_owner_goes_done_directly_and_releases.

Their host-side halves are declarations asserted elsewhere: that each
agent's subgraph is an independent root holding only its own lease is in
multi_agent_restart_is_one_dag_with_concurrent_per_agent_subgraphs, and
that a power op emits no tail node is in
cancelled_power_op_runs_no_compensating_node, which checks the DAG has
no pending nodes left at all.
This commit is contained in:
atlas 2026-08-02 20:39:21 +02:00 • committed by mara
commit ca2c479b17
1 changed files with 17 additions and 71 deletions
Download patch file Download diff file Expand all files Collapse all files

88
hive-c0re/src/job_queue/tests.rs
View file

@ -673,52 +673,6 @@ fn multi_agent_restart_is_one_dag_with_concurrent_per_agent_subgraphs() {
);
}
/// A multi-agent DAG frees an agent's lease the moment THAT agent's
/// subgraph is terminal — not when the whole DAG finishes. So a
/// concurrent DAG wanting the finished agent can proceed while the rest
/// of the first DAG runs on.
#[test]
fn multi_agent_lease_frees_per_subgraph_not_whole_dag() {
let q = JobQueue::new(4);
let id = submit(&q, restart_online(&["agent-a", "agent-b"], false, "r"));
// Drive agent-a's ENTIRE subgraph to Done while leaving agent-b's
// head running (so agent-b keeps holding its lease).
let mut b_in_flight = false;
loop {
let mut progressed = false;
for c in q.claim_ready() {
if c.agent == "agent-a" {
q.complete_node(c.node_id, Ok(()));
progressed = true;
} else {
b_in_flight = true; // leave agent-b's node running
}
}
if !progressed {
break;
}
}
assert!(b_in_flight, "agent-b subgraph should still be in flight");
// The DAG as a whole is NOT terminal — agent-b runs on.
assert_eq!(state_of(&q, id), State::Running);
// agent-a's lease is freed early → a concurrent agent-a DAG runs;
// an agent-b DAG still blocks on the lease agent-b's subgraph holds.
submit(&q, restart_online(&["agent-a"], false, "concurrent-a"));
submit(&q, restart_online(&["agent-b"], false, "concurrent-b"));
let claims = q.claim_ready();
let agents: Vec<&str> = claims.iter().map(|c| c.agent.as_str()).collect();
assert!(
agents.contains(&"agent-a"),
"agent-a lease freed the moment its subgraph settled"
);
assert!(
!agents.contains(&"agent-b"),
"agent-b lease still held — its subgraph is still in flight"
);
}
#[test]
fn multi_agent_stop_is_one_dag_with_concurrent_per_agent_subgraphs() {
let q = JobQueue::new(4);
@ -1228,6 +1182,23 @@ fn rebuild_reconcile_waits_for_the_whole_build_subtree() {
// node fails, asserting the DAG reads `Failed`. That is `failed_child_rolls_
// parent_up_to_failed` in hive_jobq, restated through a c0re template.
// `multi_agent_lease_frees_per_subgraph_not_whole_dag` and
// `dag_settles_terminal_and_releases_lease_after_work` lived here.
//
// Both drove a DAG to completion to watch an agent lease free up — one when a
// single agent's subgraph settled inside a still-running multi-agent DAG, the
// other when a whole power op finished. Releasing a grant once its owner's
// subtree is terminal is hive_jobq's (`owner_holds_grant_for_its_whole_subtree`,
// `child_borrows_ancestor_grant_released_when_subtree_done`,
// `leaf_owner_goes_done_directly_and_releases`).
//
// The c0re halves are declared and asserted elsewhere: that each agent's
// subgraph is an independent root holding only its own lease is in
// `multi_agent_restart_is_one_dag_with_concurrent_per_agent_subgraphs`, and
// that a power op emits no tail node is in
// `cancelled_power_op_runs_no_compensating_node`, which checks the DAG has no
// pending nodes left at all.
// ---- cancel ----
#[test]
@ -1362,31 +1333,6 @@ fn cancelled_power_op_runs_no_compensating_node() {
// ---- terminal reporting + lease release ----
#[test]
fn dag_settles_terminal_and_releases_lease_after_work() {
let q = JobQueue::new(1);
let id = submit(&q, restart_online(&["agent-a"], false, "r"));
// restart = StopForUpdate → Reconcile.
let stop = claim_one(&q);
assert_eq!(stop.kind.as_str(), "stop_for_update");
q.complete_node(stop.node_id, Ok(()));
let rec = claim_one(&q);
assert_eq!(rec.kind.as_str(), "reconcile");
// Completing the last work node rolls the container up terminal. A power op
// has no tail node, so nothing is left to claim.
q.complete_node(rec.node_id, Ok(()));
assert!(q.claim_ready().is_empty(), "no tail node to claim");
assert_eq!(state_of(&q, id), State::Done);
// Lease released when the work chain settled: a new DAG for the agent claims
// immediately.
let next = submit(
&q,
templates::reconcile_only("agent-a", Source::Manual, "stop".to_owned()),
);
let c = claim_one(&q);
assert_eq!(c.dag_id, next);
}
/// A DAG cancelled while fully queued must still **run its tail**, or a queued
/// approval DAG cancelled by the operator would dangle its approval forever.
///