job_queue: grow the rebuild subgraph from DeployApply (#2664)
The config-PR deploy's apply node still did the whole container rebuild inline, through the last surviving `lifecycle::rebuild_no_meta` call. It now merges, opens the two-phase meta deploy, and returns the ordinary rebuild chain as a subgraph the scheduler grafts into the live DAG under it. A new `FinalizeDeploy` node, gated on that graft, plants the deploy tag and commits the staged lock. Net effect: "did the agent come back up?" is answered by `Reconcile` succeeding, the same way it is for every other rebuild, instead of by a fused inline start — and each deploy phase is its own queue node, so the dashboard shows which one is running. The grafted nodes root on the apply node, so they land inside `DeployWindow`'s subtree and re-enter the meta window and build slot it already holds rather than deadlocking against them. The new happy-path test runs on a one-slot queue specifically to pin that down. `FinalizeDeploy`'s two git writes are fatal, deliberately: they are what tells `DeployTail` a deploy confirmed good, so a node that merely warned on them could report success while leaving the tail looking at the git state of a failure — and the tail would then roll a good deploy back. The trailing `meta::finalize_deploy` stays warn-only, since by then the container already runs the new config. The `failed/<id>` annotated tag moves into the tail, which is now the only place holding a failed deploy. It reads the reason off the DAG via a new `JobQueue::first_error`, and is gated on `main` having actually moved — the rollback ref is parked *before* the merge, so its existence alone does not mean a merge happened, and a pre-merge rejection must not tag the previous, innocent head. Removing the last inline rebuild orphaned a chain of now-dead code: `rebuild_no_meta`, `container_exists`, `Coordinator::set_queue_build_log` and `JobQueue::set_build_log_id_running`, all deleted here.
This commit is contained in:
parent
7b2645078a
commit
3429a8c5a6
10 changed files with 388 additions and 257 deletions
|
|
@ -123,6 +123,50 @@ pub(crate) fn rebuild_nodes(agent: &str, relock: bool, base: u64) -> Vec<NodeSpe
|
|||
]
|
||||
}
|
||||
|
||||
/// The rebuild subgraph a [`NodeKind::DeployApply`] grows into its own DAG once
|
||||
/// the merge has landed and `prepare_deploy` has staged the lock, plus the
|
||||
/// [`NodeKind::FinalizeDeploy`] that closes the window behind it.
|
||||
///
|
||||
/// `relock = false` is the whole reason this composes: `prepare_deploy` already
|
||||
/// relocked and staged `flake.lock`, so the appended `MetaSync` must do the dir
|
||||
/// prep + `sync_agents` *without* re-locking over it.
|
||||
///
|
||||
/// `FinalizeDeploy` waits on **two** siblings, which together reproduce the gate
|
||||
/// the old fused node had around its inline `rebuild_no_meta` call:
|
||||
/// - `AfterOk` `Prebuild` — a parent's state is its roll-up, so this is `Done`
|
||||
/// only once `StopForUpdate` → `Swap` → `PostSwap` all are (a failed *or*
|
||||
/// cancelled child rolls the parent up `Failed`). That's the old
|
||||
/// `build_result`.
|
||||
/// - `AfterOk` `Reconcile` — the old call passed `deferred_start = false` on
|
||||
/// purpose: the container had to come back up *before* the deploy was
|
||||
/// finalized. `Reconcile` alone would not do, being `AfterAny` — it reaches
|
||||
/// `Done` even after a failed `Swap`.
|
||||
///
|
||||
/// Appended, not submitted: the roots below become children of the emitting
|
||||
/// `DeployApply` (see [`super::JobQueue::append_subgraph`]), which puts them
|
||||
/// inside the `DeployWindow`'s subtree — so the `MetaWindow` this subgraph's
|
||||
/// `MetaSync` and `FinalizeDeploy` declare is re-entered from the ancestor
|
||||
/// already holding it rather than deadlocking against it.
|
||||
pub(crate) fn deploy_rebuild_nodes(agent: &str) -> Vec<NodeSpec> {
|
||||
let mut nodes = rebuild_nodes(agent, false, 0);
|
||||
nodes.push(node(
|
||||
NodeKind::FinalizeDeploy {
|
||||
agent: agent.to_owned(),
|
||||
},
|
||||
vec![
|
||||
Dep {
|
||||
on: 1,
|
||||
when: DepWhen::AfterOk,
|
||||
},
|
||||
Dep {
|
||||
on: 5,
|
||||
when: DepWhen::AfterOk,
|
||||
},
|
||||
],
|
||||
));
|
||||
nodes
|
||||
}
|
||||
|
||||
/// One uniform rebuild shape — no `was_running` branch. `StopForUpdate`
|
||||
/// noops when already down; the tail `Reconcile` auto-noops the start
|
||||
/// when `wanted = Offline` (a rebuild of a deliberately-stopped agent
|
||||
|
|
@ -149,7 +193,9 @@ pub fn rebuild(agent: &str, source: Source, reason: String, relock: bool) -> Dag
|
|||
/// nothing, so a failure here cancel-cascades its siblings with the forge and
|
||||
/// the applied repo exactly as they were.
|
||||
/// - `DeployApply` (2, child, `AfterOk` `MergeVerify`): the irreversible half —
|
||||
/// ff-merge, `prepare_deploy`, rebuild, `finalize_deploy`.
|
||||
/// ff-merge + `prepare_deploy`. It doesn't rebuild inline; it grows
|
||||
/// [`deploy_rebuild_nodes`] into this DAG as its own children, so the build
|
||||
/// and the closing `FinalizeDeploy` are real nodes under the same window.
|
||||
/// - `DeployTail` (3, child, `AfterAny` `DeployApply`): the compensation +
|
||||
/// bookkeeping tail — rollback when a merge landed unfinalized, forge tag
|
||||
/// mirror, PR failure comment (see [`NodeKind::DeployTail`]).
|
||||
|
|
|
|||
Loading…
Reference in a new issue