cache-miss pass pushes a record the builder did not push

A meta build whose attic push failed (attic or its token not up yet) is
recorded with cached false and the deploy goes ahead. Nothing sent that
build again until internal/meta moved, so the create for that agent
failed on every hive and a rebuild was silently never pending.

The controller's cache-miss pass now also takes a record from the head
with cached false: it sends the same build without asking the cache, with
the pass's existing backoff (30 min doubling to 12 h), and rewrites the
record with cached true once the push lands, under the same
compare-and-set guard as a path change. The push.rs module doc no longer
claims the next build pushes again.

Mixed versions: controller-only, same wire type and builder request. An
old controller leaves an uncached record as before; a builder needs no
change.

Closes #6173

docs/4-implementation/swarm/builder.md

@@ -316,11 +316,11 @@ grant the forge refuses fails the build with no verdict.
 cache (`attic.<swarm domain>` and `swarm`) on every host, so the builder pushes
 each green meta build's closure to it, or to the cache you override them with,
 before it replies. It runs `attic push swarm:<cache> <out path>` once per build,
-not `attic watch`, and the builder pushes again when the controller runs a build again. The
-push has ten minutes, or less when the reply is due sooner, because it must end
-inside the 61 minutes the controller waits. `attic` checks the server's certificate
-against the system roots and pins nothing, so a Let's Encrypt certificate
-works.
+not `attic watch`. When a push fails, the controller's
+[cache-miss pass](#rebuild-on-a-cache-miss) sends the build again. The push has
+ten minutes, or less when the reply is due sooner, because it must end inside the
+61 minutes the controller waits. `attic` checks the server's certificate against
+the system roots and pins nothing, so a Let's Encrypt certificate works.
 
 Set both options to a name or both to null: evaluation refuses one without the
 other. Null in both is how a swarm opts out of pushing.
@@ -336,9 +336,10 @@ The token is in no argument or variable, so no process listing shows it.
 
 The reply's `cached` says whether the push finished. A push that fails, times out
 or can't read the token leaves the build green with `cached` false and logs a
-warning with the reason; the deploy goes ahead and the next build of the agent
-tries again. With no attic behind the defaults, every push fails this way. With
-both options null, the builder pushes nothing and `cached` is always false.
+warning with the reason; the deploy goes ahead, and the controller sends the
+same build again until the push lands ([below](#rebuild-on-a-cache-miss)). With
+no attic behind the defaults, every push fails this way. With both options null,
+the builder pushes nothing and `cached` is always false.
 
 Mixed versions:
 
@@ -354,8 +355,11 @@ A record written before the field existed also reads as `false`.
 
 The cache drops a closure nobody fetched for its retention period, but the
 record in `agent-closures` stays. Every ten minutes `swarm-controller` checks
-each agent a hive places: when the record is from `internal/meta`'s head and
-says the builder pushed it, the controller fetches the narinfo from
+each agent a hive places. A record from `internal/meta`'s head that says the
+builder didn't push (`cached` false) gets the build sent again without a narinfo
+check, and the controller rewrites the record with `cached` true once the push
+lands. When the record from the head says the builder pushed it, the controller
+fetches the narinfo from
 `<url>/<cache>/<hash>.narinfo` with the pull token it reads from
 `secret/swarm/services/attic/` (the same four leaves it
 [copies to the hives](#what-hives-do-with-the-cache); with any missing, or a URL
@@ -363,7 +367,7 @@ that isn't `https`, the pass does nothing). A 404 is a miss, and the controller
 sends the builder `build-meta` for that agent and the record's own commit. The
 builder builds it and pushes as for any meta build. Any other answer or a failed
 request isn't a miss, and rebuilds nothing. The next ordinary build handles a record
-from an older commit, or one with `cached` false.
+from an older commit.
 
 A pass sends at most three builds, one after the other. After an agent's rebuild
 failed, got no answer or built without pushing, the pass skips it for 30 minutes, then
@@ -371,19 +375,23 @@ for twice as long after each further failure, up to 12 hours. The wait is in mem
 so a controller restart tries every agent again. The first pass runs ten
 minutes after the controller starts.
 
-The record keeps naming what hives deploy. A rebuild that lands on the record's
-path writes nothing. A rebuild that lands on another path means the build isn't
+The record keeps naming what hives deploy. A rebuild of a cached record that lands
+on the record's path writes nothing. A rebuild of an uncached record that lands on
+its path rewrites the record with `cached` true, under the same guard as below.
+A rebuild that lands on another path means the build isn't
 reproducible: the controller logs a warning, and replaces the record only if it
 still holds the same commit and no write came since the controller read it. The
 record otherwise stays as it was.
 
 The controller counts each rebuild in `hyperhive.builder.cache_miss_rebuilds` with
-the label `outcome`: `rebuilt`, `path_changed` (the record now names the new path),
+the label `outcome`: `rebuilt`, `cached` (an uncached record now says cached),
+`cached_kept` (the record changed meanwhile and stayed), `path_changed` (the record now names the new path),
 `path_changed_kept` (the record stayed), `not_pushed`, `failed` (`nix` failed) or
 `unavailable` (the builder was busy, absent or gave no answer).
 
 Mixed versions: a builder needs no change, since a meta build already names
-its commit. An old controller does nothing about a miss, as before. A hive
+its commit. An old controller does nothing about a miss or an uncached record,
+as before. A hive
 sees nothing new.
 
 ### The swarm's own cache