actions: split the config-PR deploy into verify / apply / tail
`run_approval_merge_config_pr` and `run_merge_config_pr` are gone; the three phases are `run_deploy_merge_verify` (drift gate, fetch, verify — mutates nothing), `run_deploy_apply` (merge + build) and `run_deploy_tail` (compensation + push). The rollback state is a git ref in the applied repo (`refs/hyperhive/rollback/<approval-id>`) rather than a value handed between nodes, because hive-c0re can restart between the apply and the tail and the tail still has to know what to undo. Rolling `main` back on a *successful* deploy is the worst thing the tail can do, so it is guarded twice: the apply drops the rollback ref before it plants `deployed/<id>`, and the tail refuses to compensate at all if `deployed/<id>` resolves. It takes two independent git failures to get there. `run_deploy_tail` returns nothing and warns on every error — a failing compensation must not mask the deploy's own verdict, which the terminal hook takes from the DAG's roll-up.
This commit is contained in:
parent
8899c9f355
commit
27ecda7b13
2 changed files with 393 additions and 280 deletions
|
|
@ -99,7 +99,10 @@ pub(super) async fn run_node(coord: &Arc<Coordinator>, claim: &Claim) -> Result<
|
|||
NodeKind::Drain { .. } => run_drain(coord, claim, &ctx).await,
|
||||
NodeKind::WriteDropin { .. } => run_write_dropin(coord, claim).await,
|
||||
NodeKind::WritePermFile { .. } => run_write_perm_file(coord, claim, &ctx).await,
|
||||
NodeKind::ApprovalDeploy { .. } => run_approval_deploy(coord, claim).await,
|
||||
NodeKind::DeployWindow { .. } => run_deploy_window(claim),
|
||||
NodeKind::MergeVerify { .. } => run_merge_verify(coord, claim).await,
|
||||
NodeKind::DeployApply { .. } => run_deploy_apply(coord, claim).await,
|
||||
NodeKind::DeployTail { .. } => run_deploy_tail(coord, claim).await,
|
||||
NodeKind::SetWanted { up, .. } => run_set_wanted(coord, claim, *up),
|
||||
// Pure grouping container — no work; completing it lets it reach
|
||||
// `Finishing` so its child template nodes start. The DAG's terminal
|
||||
|
|
@ -578,27 +581,66 @@ async fn run_write_perm_file(
|
|||
Ok(NodeOutput::default())
|
||||
}
|
||||
|
||||
/// Opaque approval deploy pipeline for `MergeConfigPr`: verify + ff-merge the
|
||||
/// reviewed PR head, then the container rebuild. The two-phase
|
||||
/// prepare/finalize/abort meta deploy — and the approval resolution — stay
|
||||
/// inside `actions.rs` in v1 (design doc §9).
|
||||
async fn run_approval_deploy(coord: &Arc<Coordinator>, claim: &Claim) -> Result<NodeOutput> {
|
||||
let approval_id = claim
|
||||
/// The approval id every deploy phase re-reads its approval row by. Fails the
|
||||
/// node when the DAG carries none, which would mean a `MergeConfigPr` DAG was
|
||||
/// built without going through `templates::approval_deploy`.
|
||||
fn deploy_approval_id(claim: &Claim) -> Result<i64> {
|
||||
claim
|
||||
.approval_id
|
||||
.with_context(|| format!("approval_deploy dag {} has no approval_id", claim.dag_id))?;
|
||||
// The deploy window covers the whole prepare→finalize span
|
||||
// (`NodeKind::needs_meta_window`, held by the scheduler for this
|
||||
// node): `prepare_deploy` stages `flake.lock` uncommitted for the
|
||||
// entire container build, and no other meta mutation may land inside
|
||||
// that window (it would sweep the staged lock and neuter
|
||||
// `abort_deploy`). Holding it as a queue resource — rather than a
|
||||
// `MutexGuard` that cannot outlive this fn — is what lets increment 2
|
||||
// decompose this node into sub-nodes under a window-holding parent.
|
||||
crate::actions::run_approval_merge_config_pr(coord, Some(claim.dag_id), approval_id)
|
||||
.with_context(|| format!("approval deploy dag {} has no approval_id", claim.dag_id))
|
||||
}
|
||||
|
||||
/// The deploy subtree's root: pure resource holder, no work of its own.
|
||||
///
|
||||
/// It exists so the global meta window (plus the agent lease and a build slot)
|
||||
/// is held continuously across every phase below it. `prepare_deploy` leaves
|
||||
/// `flake.lock` staged-uncommitted for the whole container build, and any other
|
||||
/// meta mutation landing inside that span would sweep the staged lock into its
|
||||
/// own commit and neuter `abort_deploy` — so the window has to outlive any one
|
||||
/// node, which the `MutexGuard` this replaced could not do.
|
||||
///
|
||||
/// Completing immediately moves it to `Finishing`, which is what starts the
|
||||
/// children; the resources stay held until the whole subtree settles.
|
||||
fn run_deploy_window(claim: &Claim) -> Result<NodeOutput> {
|
||||
deploy_approval_id(claim)?;
|
||||
Ok(NodeOutput::default())
|
||||
}
|
||||
|
||||
/// Deploy phase 1 — drift gate, fetch, eval-verify. Mutates nothing, so a
|
||||
/// failure here cancel-cascades the rest of the subtree with the forge and the
|
||||
/// applied repo exactly as they were.
|
||||
async fn run_merge_verify(coord: &Arc<Coordinator>, claim: &Claim) -> Result<NodeOutput> {
|
||||
crate::actions::run_deploy_merge_verify(coord, Some(claim.dag_id), deploy_approval_id(claim)?)
|
||||
.await
|
||||
.map(|()| NodeOutput::default())
|
||||
}
|
||||
|
||||
/// Deploy phase 2 — the irreversible half: ff-merge, two-phase meta deploy,
|
||||
/// container rebuild, finalize.
|
||||
async fn run_deploy_apply(coord: &Arc<Coordinator>, claim: &Claim) -> Result<NodeOutput> {
|
||||
crate::actions::run_deploy_apply(coord, Some(claim.dag_id), deploy_approval_id(claim)?)
|
||||
.await
|
||||
.map(|()| NodeOutput::default())
|
||||
}
|
||||
|
||||
/// Deploy compensation + bookkeeping tail. `AfterAny` the apply node, so it
|
||||
/// runs on every outcome; it is deliberately infallible (see
|
||||
/// [`crate::actions::run_deploy_tail`]) — a failing tail must not flip an
|
||||
/// otherwise-successful deploy's DAG state.
|
||||
///
|
||||
/// Takes the agent from the node payload so the tail can still compensate when
|
||||
/// the approval row is gone (deny race, purge).
|
||||
async fn run_deploy_tail(coord: &Arc<Coordinator>, claim: &Claim) -> Result<NodeOutput> {
|
||||
crate::actions::run_deploy_tail(
|
||||
coord,
|
||||
Some(claim.dag_id),
|
||||
claim.kind.agent(),
|
||||
deploy_approval_id(claim)?,
|
||||
)
|
||||
.await;
|
||||
Ok(NodeOutput::default())
|
||||
}
|
||||
|
||||
/// Compute which agents a `nix flake update <inputs>` on the meta
|
||||
/// flake affects — the fan-out set for `MetaUpdate` DAGs. Empty
|
||||
/// `inputs` or any input under `hyperhive` → every container;
|
||||
|
|
|
|||
Loading…
Reference in a new issue