Watch
0
0
Fork
You've already forked hyperhive
0

swarm-builder: refuse a build that waited too long for the slot

A request that waits behind another build, e.g. one a controller
restart orphaned, could start a full build after most of the
controller's 61-minute reply deadline was gone. The builder now waits
at most REPLY_DEADLINE - BUILD_DEADLINE - 30 s for the slot and
answers Failed, without building, past that. The 30 s covers the
forge token read, now bounded to 20 s, and the reply's trip back.

Fixes #5437
This commit is contained in:
atlas 2026-10-17 15:50:54 +02:00 • committed by mara
commit bf848ba65f
4 changed files with 180 additions and 22 deletions

View file

@ -20,7 +20,11 @@ Where it runs and how it's enabled: [`services.md`](services.md#builder).
finishes and writes its status on its own, older commit.
2. The node sets the status to `pending`, then sends the build as a request on
the queue subject `$SWARM.builder.build` and waits up to 61 minutes for the
reply.
reply. The builder runs one build at a time and refuses a request that has
waited 30 seconds for its turn, such as one behind a build a controller
restart orphaned: a build started later could still run when the 61
minutes end. Reading the forge token before a build has 20 seconds, and
the rest of the minute covers the reply's trip back.
3. The builder runs `nix build` of `nix/builder/check-flake`'s `check` output
with every input overridden:
- `hyperhive` and `nixpkgs`: the sources the builder's own host was
@ -39,6 +43,7 @@ Where it runs and how it's enabled: [`services.md`](services.md#builder).
| ----------------------------------------- | --------- | --------------------------- | ------ |
| built | `success` | the store path | done |
| `nix` failed or timed out | `failure` | the last line `nix` printed | failed |
| the builder busy for 30 seconds | `failure` | why | failed |
| no builder listening, or no reply in time | `error` | why | failed |
The full tail is in the failed node's error on the Jobs page and in the