vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

job_queue: one deploy-shape table replaces two DAG walks

Browse source 
deploy_dag_runs_phases_in_order_and_tails_a_failed_apply and
deploy_dag_skips_apply_but_still_runs_tail_when_verify_fails differed
only in where they injected the failure -- apply in one, verify in the
other -- and each drove the whole DAG to watch the compensation tail run
anyway.

Both follow from a single declared edge. The tail accepts
done|failed|skipped on apply, and skipped is exactly the state apply
lands in when verify failed and it never ran. Asserting that edge covers
both cases without running anything.

The runtime halves are hive-jobq's and tested there: a failed dep
cancels its AfterOk dependents while the AfterAny one still runs, and a
parent rolls up Failed from a failed child -- which is what stops an Ok
tail laundering a failed deploy into a success.

Mutation-checked: turning the tail's after_any(apply) into
after_ok(apply) fails the surviving test on that edge alone.
This commit is contained in:
atlas 2026-08-02 20:04:38 +02:00 • committed by mara
commit 7d1709cfc5
1 changed files with 44 additions and 52 deletions
Download patch file Download diff file Expand all files Collapse all files

96
hive-c0re/src/job_queue/tests.rs
View file

@ -1599,33 +1599,35 @@ fn deploy_dag_runs_phases_in_order_and_tails_a_failed_apply() {
templates::approval_deploy("agent-a", 7, "approval #7".to_owned()),
);
let root = claim_one(&q);
assert!(
matches!(root.kind, NodeKind::DeployWindow { .. }),
"root claims first: it holds the meta window for the whole subtree"
);
q.complete_node(root.node_id, Ok(()));
let verify = claim_one(&q);
assert!(matches!(verify.kind, NodeKind::MergeVerify { .. }));
q.complete_node(verify.node_id, Ok(()));
let apply = claim_one(&q);
assert!(matches!(apply.kind, NodeKind::DeployApply { .. }));
q.complete_node(apply.node_id, Err("nixos-container update blew up".into()));
let tail = claim_one(&q);
assert!(
matches!(tail.kind, NodeKind::DeployTail { .. }),
"AfterAny tail runs on a failed apply — that's the whole point of it"
);
q.complete_node(tail.node_id, Ok(()));
settle_approval_tail(&q, 7, TerminalState::Failed);
assert_eq!(
state_of(&q, id),
State::Failed,
"an Ok tail must not launder a failed deploy into a success"
declared_shape(&q, id),
vec![
// The window is the group root and holds the meta window for the
// whole subtree; the three phases are its sub-nodes.
row("deploy_window", None, &[]),
row("merge_verify", Some("deploy_window"), &[]),
// Apply only on a clean verify — a failed verify cancel-cascades
// it, which is what leaves the forge and the applied repo untouched.
row(
"deploy_apply",
Some("deploy_window"),
&[("merge_verify", "done")]
),
// The compensation tail accepts every terminal outcome of apply,
// *including `skipped`* — which is the state apply lands in when
// verify failed and it never ran. That one edge is the entire
// "still tails a failed apply / a failed verify" behaviour, and it
// is why two separate DAG-driving tests collapsed into this table.
row(
"deploy_tail",
Some("deploy_window"),
&[("deploy_apply", "done|failed|skipped")]
),
// One approval tail per outcome, gated on the window's roll-up.
row("resolve_approval", None, &[("deploy_window", "done")]),
row("resolve_approval", None, &[("deploy_window", "failed")]),
row("resolve_approval", None, &[("deploy_window", "cancelled")]),
]
);
}
@ -1750,32 +1752,22 @@ fn deploy_dag_skips_finalize_but_still_tails_a_failed_graft() {
);
}
/// A pre-merge rejection (drift gate, eval failure) cancel-cascades the
/// irreversible half via its `AfterOk` edge, but the tail is still reached —
/// it owns the forge mirror, not just compensation.
#[test]
fn deploy_dag_skips_apply_but_still_runs_tail_when_verify_fails() {
let q = JobQueue::new(1);
let id = submit(
&q,
templates::approval_deploy("agent-a", 9, "approval #9".to_owned()),
);
let root = claim_one(&q);
q.complete_node(root.node_id, Ok(()));
let verify = claim_one(&q);
q.complete_node(verify.node_id, Err("PR head drifted since review".into()));
let tail = claim_one(&q);
assert!(
matches!(tail.kind, NodeKind::DeployTail { .. }),
"apply is cancel-cascaded, so the tail is the next claimable node"
);
q.complete_node(tail.node_id, Ok(()));
settle_approval_tail(&q, 9, TerminalState::Failed);
assert_eq!(state_of(&q, id), State::Failed);
}
// `deploy_dag_skips_apply_but_still_runs_tail_when_verify_fails` lived here.
//
// A pre-merge rejection (drift gate, eval failure) cancel-cascades the
// irreversible half via its `AfterOk` edge, while the tail is still reached —
// it owns the forge mirror, not just compensation. That test and
// `deploy_dag_runs_phases_in_order_and_tails_a_failed_apply` differed only in
// *where* they injected the failure, and each drove the whole DAG to watch the
// tail run anyway.
//
// Both outcomes follow from one declared edge, which the surviving test now
// asserts directly: the tail accepts `done|failed|skipped` on apply, and
// `skipped` is exactly the state apply lands in when verify failed and it never
// ran. The runtime halves are hive_jobq's and tested there —
// `failed_after_ok_dep_cancels_dependents_but_after_any_still_runs` and
// `failed_child_rolls_parent_up_to_failed` (an Ok tail cannot launder a failed
// deploy into a success).
// ---- history ----
//