Destroy was a straight-line async fn with no queue node behind it, so
nothing in the graph could answer "is this container going down on
purpose?". That gap is why an imperative crash-watch suppression guard
existed: an RAII handle held for the operation's duration, a second way
to say what every other lifecycle op already says through its node.
Reuse the existing Stop node rather than teaching a new node to stop
things:
Stop -> DestroyContainer -> (PurgeState) -> DestroyBookkeeping
Stop already declares takes_container_down honestly, so the suppression
is now derived from the graph like every other op's. It also turns the
precondition into an edge: DestroyContainer runs only under a completed
Stop, so it operates on an already-stopped container and carries
takes_container_down = false permanently. A container still alive at
that point is a real bug and stays loud instead of being absorbed by a
flag -- which matters because a wrong true silently swallows a crash
while a wrong false only costs a spurious event.
Removes suppress_crash_watch, CrashWatchSuppression, crash_suppressed,
crash_watch_suppressed and NO_NODE_LABEL. The migration call sites went
with the obsolete startup migrations, so destroy was the last caller and
intent now has exactly one home.
destroy() becomes a submit-and-return, matching every sibling endpoint
(rebuild, kill, restart, start, pause, resume) -- it was the only
lifecycle op that awaited its work. The container rescan moves into the
bookkeeping tail, so ContainerRemoved now arrives after the 200 rather
than before it.
Also drops an orphaned doc-comment in coordinator.rs: two stacked blocks
where only the second described crash_suppressed, the first documenting
a field that no longer exists. Removing the field would have re-pointed
it at recent_transient.
Deletes `TransientState`, adds `transient_kind` to `TransientCleared`, and
fixes a crash misreport — three changes over the same functions.
`TransientState` was `RunningTransient` with `agent` dropped and
`takes_container_down` renamed; that rename was the only thing it did, and
its three consumers each read a disjoint subset. `transient_snapshot` now
returns `RunningTransient` directly.
`recent_transient` was keyed by agent alone and overwritten on each clear.
An agent can clear several pills in one grace window, so a `Prebuild`
(`takes_container_down = false`) landing after a `StopForUpdate` (`true`)
left the tombstone reading `false` and the crash watcher reported a
deliberate stop as a container crash. Keyed by `(agent, label)` now, with
`recent_transient_within` folding back per agent by OR — the same question
`crash_watch` asks of the active set.
`TransientCleared` gains the label for the same reason: a client holding
two open pills for one agent could not tell which one a clear referred to.
The out-of-band suppression guard has no node and so no label; it uses
`NO_NODE_LABEL`, angle-bracketed to stay out of the `NodeKind::as_str`
namespace.
Per mara on #2822: status is the only test. The agent comes off the
node's own payload rather than a declared Resource::Agent edge, so the
lease-exempt kinds (Prebuild, MetaSync) that name an agent without
holding its lease now light a pill — they are work on that agent.
Dropping that test breaks the one-pill-per-agent invariant, since
lease-exemption is exactly what lets one DAG build for an agent while
another holds its lease. Everything keyed by agent alone had to follow:
- reconcile_transients keys (agent, label) via TransientSeen, so a
second pill cannot evict the first — and cannot lose its
takes_container_down, which the crash watcher reads at clear time.
- transient_snapshot returns a Vec per agent for the same reason. The
collapse was silent: a Prebuild could evict a StopForUpdate and its
deliberate_stop, making an intentional stop report as a crash.
- crash_watch asks whether ANY running node expects the container down.
- the dashboard renders one row per node instead of one per agent.
takes_container_down never reached the frontend; no wire change needed.
315 tests pass unchanged.
mara on !2910: "also remove imperative path for the things that are not
nodes yet, file follow up issue to fix that".
`TransientGuard`, the stored map and both manual set/clear are gone.
`transient_snapshot()` is derived and nothing else — destroy and
migration show no pill, because there is no node to derive one from. The
pill returns for free when they become nodes.
What those three guards were actually doing, though, was suppressing the
crash watcher, not drawing a pill. `migrate.rs` said so in its own
comment: without it, `crash_watch` fires `ContainerCrash` for every
migrated agent and the manager tries to recover containers that were
stopped on purpose. Destroy is the same — the container disappears
deliberately and nothing in the graph says so.
Deleting them outright would therefore have traded a dashboard pill for
false crash alerts on every destroy and every migration. So the
suppression survives as its own thing, `suppress_crash_watch`, with a
name that says what it is. It is still RAII, and still held for the
operation rather than stamped once, because the crash watcher's grace
window is finite and a destroy is not — a single tombstone would expire
mid-operation. The drop stamps the tombstone, covering the poll that
lands just after.
That leaves RAII in the codebase for exactly one purpose instead of two.
Untangling the pill from the suppression is what made the transient layer
deletable at all.
Follow-up issue for making destroy + migration real queue nodes to
follow; at that point this guard goes too.
Checked with clippy (`--all-targets -D warnings`), `cargo test -p
hive-c0re -p hive-jobq` (322 + 41 passed) and `nix fmt`.
The dashboard pill was declared once per DAG at submit time, so a rebuild
reported `rebuilding` for its entire life — through the prebuild, the
stop, the swap, the tail and the reconcile. It named the intent of the
request, not what was happening.
It is now read off the nodes actually running. A node lights a pill when
it is `Running` and declares the agent's own resource. Declaring is the
test, not targeting: `Prebuild` and `MetaSync` name an agent but are
lease-exempt on purpose (the container keeps serving), so they must not
light one. It is also not the lease *owner* — `resource_state()` answers
"who holds the slot", which is a different question from "what is
running", and a descendant that borrows an ancestor's grant never
appears in that map.
`TransientKind` is gone entirely rather than being re-derived. The label
is the node's own wire tag (`NodeKind::as_str`) — the same vocabulary
`NodeView.kind` already ships, so a pill and a DAG node name an operation
identically and there is no second taxonomy to keep in step. Work with no
node behind it (destroy, migration) supplies its own literal.
`DagSpec::transient`, `Claim::transient`, `DagMeta::transient` and
`NodeKind::Dag`'s `transient` field all go with it.
## the safety half, which is deliberately not the display half
`crash_watch::is_deliberate_stop` used to match a `TransientKind` to
decide whether a vanished container was intentional or a crash. That made
a pill's display vocabulary decide an alerting question, so renaming or
adding a label would silently move the alerting boundary.
`TransientState` now carries two independent fields: `label` (rendered,
nothing branches on it) and `deliberate_stop` (read only by the crash
watcher). The producer sets the second, because the producer is the only
thing that knows — it is not recoverable from the first.
For queue work that value is `NodeKind::takes_container_down()`, and it
is emphatically not "holds a lease": `Create` and `Start` hold the
agent's lease exactly like `Stop` does, and a container dying *while
starting* is a real crash that must keep reporting as one. The default is
`false` on purpose — a wrong `false` costs a spurious crash event, a
wrong `true` swallows a real crash silently.
## known cost, accepted on the issue
A restart no longer reads `restarting`. No `NodeKind` is unique to a
restart — `restart_chain` reuses `Signal` / `StopForUpdate` / `Drain` /
`Reconcile` — because "restart" is a property of the DAG's shape, not of
any node. A restart now reads `signal` / `stop_for_update`, then the
agent returns.
`Start` / `Stop` / `PostSwap` run inside a lease-holding ancestor and
re-declare nothing, so they light no pill and the agent reads idle for
those windows. Closing that is the resources-where-constructed work
(#2818), not this change.
Checked with clippy (`--all-targets -D warnings`), `cargo test -p
hive-c0re -p hive-jobq` (321 + 40 passed) and `nix fmt`.
