mara/hyperhive
mara/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

swarm: a deploy goes out only once its closure is cached and the hive has its cache facts

Browse source
This commit is contained in:
atlas 2026-11-07 11:53:25 +01:00
commit 79c1b6d81d
1 changed files with 432 additions and 92 deletions
Download patch file Download diff file Expand all files Collapse all files

32
swarm-controller/src/meta_build.rs
View file

@ -8,14 +8,16 @@
//! key itself: the builder holds no grant on the bucket. A `Built` reply and
//! nothing else writes, so a red build leaves the previous record.
//!
//! The node ends one of two ways. `Done` is a green build. Every `Failed` skips
//! the deploy behind it, and the Jobs page shows why:
//! The node ends one of two ways. `Done` is a green build whose closure the
//! builder pushed to the cache. Every `Failed` skips the deploy behind it, and
//! the Jobs page shows why:
//! - `nix` failed, its timeout included: `Failed` with `nix`'s tail, or `nix
//! timed out`. The deploy stays held and counts as a failed build.
//! - The build got no verdict (no builder, a forge error, no `internal/meta`
//! head, a missing read grant, a failed closure write): `Failed` starting with
//! [`NO_VERDICT_PREFIX`], which the held-deploys gauge reads to label the
//! reason.
//! head, a missing read grant, a failed closure write, a closure the builder
//! built but did not push): `Failed` starting with [`NO_VERDICT_PREFIX`],
//! which the held-deploys gauge reads to label the reason. A built but
//! unpushed closure is still recorded, with `cached` false.
//!
//! Both are built again on every round of the meta timer
//! (`jobq::queue_meta_update`) until a build succeeds.
@ -384,6 +386,9 @@ pub(crate) async fn run(
nix_builds,
};
match queue.record(&agent, &closure).await {
Ok(()) if !cached => no_verdict(format!(
"built {out_path}, but the builder did not push it to the cache"
)),
Ok(()) => {
if let Some(pr) = pr {
clear_failure(head, pr, &agent, &built_rev).await;
@ -488,7 +493,7 @@ mod tests {
}
fn built() -> Self {
Self::built_cached(false)
Self::built_cached(true)
}
fn built_cached(cached: bool) -> Self {
@ -615,16 +620,25 @@ mod tests {
assert_eq!(closure.store_path, "/nix/store/x-toplevel");
assert_eq!(closure.system, "x86_64-linux");
assert!(closure.built_unix > 0);
assert!(!closure.cached);
assert!(closure.cached);
}
/// The record says what the builder said about the cache, both ways.
/// The record says what the builder said about the cache, both ways. Only a
/// pushed closure ends the node `Done`: the deploy waits on that.
#[tokio::test]
async fn the_record_carries_whether_the_builder_cached_the_closure() {
for cached in [true, false] {
let queue = FakeQueue::built_cached(cached);
let outcome = run_with(&FakeHead::reading(Ok(Some(REV))), &queue).await;
assert_eq!(outcome, Outcome::Done);
if cached {
assert_eq!(outcome, Outcome::Done);
} else {
let Outcome::Failed(error) = outcome else {
panic!("an uncached build is not Done: {outcome:?}")
};
assert!(is_no_verdict(&error), "{error}");
assert!(error.contains("did not push"), "{error}");
}
let recorded = queue.recorded.lock().unwrap();
let [(_, closure)] = recorded.as_slice() else {
panic!("one record: {recorded:?}")