Watch
0
0
Fork
You've already forked hyperhive
0

queue the build of a config head that moved without a webhook build

This commit is contained in:
atlas 2026-11-02 03:23:47 +01:00 • committed by mara
commit b7d8c0dfc0
2 changed files with 689 additions and 28 deletions

View file

@ -141,7 +141,7 @@ agent. The controller queues
its update of the repo before the deploy request for a merged config commit
and for a new agent, and sends the deploy only when that update succeeded and
the swarm builder built the agent from the updated repo (the
[deploy gate](../swarm/builder.md#deploy-gate)). A failed build skips the deploy, and the timer builds again.
[deploy gate](../swarm/builder.md#deploy-gate)). A failed build skips the deploy, and the timer builds again. A merge whose webhook delivery never arrives gets the same update, build and deploy within five minutes, from the controller's drift pass (see the paragraph on config drift in the deploy gate section).
- **Choosing the closure.** The hive doesn't choose. It deploys the closure the
agent's record in `agent-closures` declares, whichever `internal/meta` commit

View file

@ -121,7 +121,7 @@ status has no link.
can push to [a binary cache](#binary-cache).
- The PR check gates nothing. A merge deploys whatever the status says. The
[deploy gate](#deploy-gate) is a separate build, from `internal/meta`.
- A lost webhook delivery isn't retried: the head isn't built until the next
- A lost webhook delivery of a PR push isn't retried: the head isn't built until the next
push. The jobs graph is in memory, so a controller restart drops a build in
flight and leaves its status `pending`.
@ -236,6 +236,19 @@ is the same case: the old builder has no subscriber for the new subject, so the
controller gets no responders at once, and the deploys go out after the builder
updates. Each round adds a build and deploy node per failing agent.
A merged config PR whose webhook delivery never arrives would leave the agent
on its old config. Every five minutes, after the config-PR scan, the controller
reads `main` of each `agent-configs/<agent>` repo that has a commit and that
exactly one hive places. It compares that head with the config commit pinned for
the agent in the `flake.nix` of the `internal/meta` commit its `agent-closures`
key names, since the pin in `internal/meta`'s own head also moves for other
agents' merges. When they differ, it declares the same update, build and deploy
a merge declares, and logs at info `config drift: …`. It leaves alone an agent
with no key, a key whose flake pins no config commit for the agent, and a head
the graph already holds a deploy for, in any state: the ten-minute rounds above
build a head whose build failed again, not this pass. The graph is
in memory, so a controller restart tries each such head once more.
When the build belongs to a merged config PR and `nix` exited with an error or hit
its deadline, the controller also comments once on that PR, with its forge
account: the agent, the `internal/meta` commit of the first red build, `nix`'s last lines (at most 4000