frontend: swarm.js off rebuild-queue-derived in-flight status onto transients

Fixes #2822.

`swarm.js` had two independent per-agent "is this in flight" sources:
`transientsState` (operator/worker-initiated ops the backend chose to
flag) and `inFlightOpsByAgent()`, a separate derivation straight from
`rebuildQueueState` covering everything else. Since #3010/#3016,
`running_transients()` is a status-only test — any `Running` job-queue
node naming a non-empty agent lights a transient pill, not just a
curated subset — so the second source's Running-state handling is now
provably redundant: a Running node with an agent always already has a
transient by the time `queuedOpsByAgent()` (renamed from
`inFlightOpsByAgent`) would be consulted.

## What changed

- `transientsState`: `Map<name, {kind, since_unix}>` (one pill per
  agent) -> `Map<name, Map<kind, since_unix>>` (several pills per
  agent). `applyTransientSet`/`applyTransientCleared` now add/remove
  by `(name, kind)` rather than overwrite/delete by name alone, using
  `TransientCleared`'s `transient_kind` field (landed in #3016) to
  know which pill cleared. `syncTransientsFromSnapshot` groups the
  now-flat `TransientView` list by name instead of assuming one row
  per agent.
- `inFlightOpsByAgent()` -> `queuedOpsByAgent()`: trimmed to the
  `Pending` (queued, not yet started) case only. The `Running` branch
  and its "running beats queued" priority logic are gone entirely —
  dead weight now that transients cover every running case
  unconditionally.
- Render loop: an agent's transients win outright whenever any exist
  (rendered as **one badge per pill**, not collapsed into one label —
  mara: "show all running nodes that name the agent"); the queued
  fallback only applies when a agent has zero transients. `opRunning`
  simplifies to "does this agent have at least one transient".
- `docs/web-ui/dashboard.md`'s Container-row section rewritten to
  match — it described a "transient, then in-flight-queue, in
  priority order" model that's no longer accurate now that the second
  source only ever fires for the one case the first can't represent.

## Verification

`npm run build` clean for both packages (dashboard + agent). Standalone
re-derivation of the transient-map + queued-fallback logic
(`/tmp/verify-swarm-transients.mjs`, not part of this diff) run against
constructed event sequences: single-pill lifecycle, two simultaneous
pills on one agent with independent clear-by-kind, clearing an unknown
kind is a safe no-op, a flat snapshot with duplicate agent names groups
correctly, the queued fallback only fires when no transient exists and
steps aside the instant one arrives, and a Running-state rebuild-queue
entry produces no queued badge (confirming the Pending-only trim is
correct, not just assumed). All 17 checks passed.

Verified directly against the merged backend rather than trusting
summaries: `job_queue/mod.rs::running_transients()` filters
`State::Running` only (not Pending — an earlier note of mine claiming
otherwise was imprecise paraphrasing), and `NodeView.agent` /
`running_transients()`'s agent both resolve through the same
`payload.agent()`, so a Running node's presence in `rebuild_queue`
and its presence as a transient are guaranteed consistent, not just
usually so.

#2985 (DagView/NodeView deletion) unblocks once this merges — atlas is
waiting on a ping.
This commit is contained in:
iris 2026-08-03 18:41:34 +02:00 committed by mara
commit d1f82e725e
2 changed files with 138 additions and 113 deletions

View file

@ -870,54 +870,59 @@ fetch entirely.
`config` link — remain visible regardless of run state. `config` link — remain visible regardless of run state.
When the container is running, status badges follow — `⊘ rate When the container is running, status badges follow — `⊘ rate
limited` (red, while the harness is parked after a 429), `needs limited` (red, while the harness is parked after a 429), `needs
login`, `needs update` — in-flight `◐ pending-state…` pill login`, `needs update` — plus **one `◐ pending-state…` pill per
(replaces buttons during operator-initiated start / stop / active transient** (replaces buttons during operator-initiated
restart / rebuild / destroy). Additionally, when a rebuild-queue start / stop / restart / rebuild / destroy). An agent can carry
entry for this agent is `queued` or `running` but no **several transients at once** (mara: "show all running nodes that
operator-initiated transient is set, the card surfaces a name the agent") — e.g. a lease-exempt `prebuild` running alongside
`building…` / `meta-updating…` / `starting…` / `stopping…` badge a `stop_for_update` on the same agent — and each renders as its own
(per queue `kind` — e.g. a `start` entry shows `starting…` / independent badge rather than being collapsed into one label,
`start queued`, `stop` and `graceful_stop` both show `stopping…` / matching the existing multi-badge convention this line already uses
`stop queued`) sourced from `rebuildQueueState` — so the SW4RM tab for `paused`/`needs_update`/model/ctx.
shows the same progress visible on the BU1LDS page's R3BU1LD QU3U3. The row visual splits queued vs running: a **queued** entry (no
The row visual splits queued vs running: a **queued** entry shows transient yet, see below) shows only the pending-state pill (no row
only the pending-state pill (no row tint, so a long queue doesn't tint, so a long queue doesn't paint half the tab amber); a
paint half the tab amber); a **running** entry keeps the amber **running** entry (at least one transient) keeps the amber row tint
row tint AND draws a **rotating amber ring** around the agent AND draws a **rotating amber ring** around the agent icon, so it's
icon, so it's obvious at a glance which container is actually obvious at a glance which container is actually moving.
moving.
**Pending-state derivation:** the pill is sourced from two
separate stores in priority order. (1) The **transient**
(`transientsState`) — covers the create-and-start window where the
container literally isn't up yet, before any backend state event
has fired.
A transient is **derived from the job-queue node currently **Pending-badge derivation:** two separate stores, but no longer a
running** against that agent, not declared per request, so its priority *order* between them — the second only ever applies when
label follows the operation as it progresses (a rebuild reads the first has nothing to say. (1) **Transients**
`stop_for_update`, then `swap`, then `reconcile` rather than one (`transientsState`, keyed `agent -> Map<kind, since_unix>`) — a
constant `rebuilding` for its whole life). Two consequences for transient is **derived from a job-queue node currently `Running`**
anything rendering it: against that agent, not declared per request, so its label follows
the operation as it progresses (a rebuild reads `stop_for_update`,
then `swap`, then `reconcile` rather than one constant `rebuilding`
for its whole life). Two consequences for anything rendering it:
- The label vocabulary is **open** — it is the node's own wire tag - The label vocabulary is **open** — it is the node's own wire tag
(`NodeKind::as_str`, the same strings `NodeView.kind` carries), (`NodeKind::as_str`, the same strings `NodeView.kind` carries),
not a fixed set. Treat it as an opaque display string; do not not a fixed set. Treat it as an opaque display string; do not
switch on specific values. `restarting` in particular no longer switch on specific values. `restarting` in particular no longer
exists, because no node kind is unique to a restart. exists, because no node kind is unique to a restart.
- It is **not** exclusively operator-initiated. Work the operator - It is **not** exclusively operator-initiated, and **not** limited
never clicked (a meta-update cascade, a crash-recover rebuild) to rebuild-shaped work — `running_transients()` on the backend is
lights the same pill, since it is the running node that sets it. a status-only test (any `Running` node whose payload names a
non-empty agent lights a pill), so work the operator never
clicked (a meta-update cascade, a crash-recover rebuild, a
lease-exempt `prebuild`) lights the same mechanism.
Ops with no queue node behind them (destroy, migration) supply Ops with no queue node behind them (destroy, migration) supply
their own label directly. (2) If no transient is set, the **rebuild-queue their own label directly via `TransientSet`/`TransientCleared`
entry** for this agent is consulted (`rebuildQueueState`); this events carrying no backing node at all.
covers worker-driven ops — meta-update cascades, crash-recover
rebuilds, approval-driven rebuilds — that the operator didn't (2) The **rebuild-queue fallback** (`rebuildQueueState`) only
click. `ContainerStateChanged` carries neither signal, so the fires when an agent has **zero** transients — since (1) now covers
dashboard reads from the two snapshots directly. The every `Running` node unconditionally, a `Running` rebuild-queue
`opRunning` flag (driving the `pending-running` row class + entry can never usefully reach this fallback by the time it's
spinner) is true when (1) is set OR (2) is in `running` state; consulted; the fallback exists purely for the **`Pending`
queued entries leave `opRunning` false. (queued, not yet started)** case, which `running_transients()`'s
Running-only test cannot represent. `queuedOpsByAgent()` (swarm.js)
reflects this: it only ever looks at `Pending`-state queue nodes.
`opRunning` (driving the `pending-running` row class + spinner) is
simply "does this agent have at least one transient" — a queued-only
entry (no transient yet) leaves it false.
An **active model badge** (`model · <name>`, blue) appears when the An **active model badge** (`model · <name>`, blue) appears when the
container is running and the harness has persisted a model name container is running and the harness has persisted a model name
(`harness/hyperhive-model`). Read by hive-c0re's `ContainerView` (`harness/hyperhive-model`). Read by hive-c0re's `ContainerView`

View file

@ -37,8 +37,13 @@ const containerRowCache = new Map();
// Derived transient state — cold-loaded from /api/state.transients, // Derived transient state — cold-loaded from /api/state.transients,
// then mutated live by `transient_set` / `transient_cleared`. Keyed // then mutated live by `transient_set` / `transient_cleared`. Keyed
// by agent name so add/remove are O(1). `since_unix` is wall-clock so // by agent name -> Map<transient_kind, since_unix>: an agent can hold
// the elapsed-seconds badge ticks without polling. // several pills at once (every *running* node naming the agent lights
// one now, not just a curated "worth surfacing" subset — see
// docs/web-ui.md::Container row), keyed the same way the backend's
// own tombstone map is, `(agent, label)`, so a clear only removes the
// specific pill it names rather than guessing. `since_unix` is
// wall-clock so the elapsed-seconds badge ticks without polling.
const transientsState = new Map(); const transientsState = new Map();
// In-memory set of selected agent logical names backing the sticky // In-memory set of selected agent logical names backing the sticky
@ -54,77 +59,99 @@ export function syncRebuildQueueFromSnapshot(s) {
} }
export function applyRebuildQueueChanged(ev) { export function applyRebuildQueueChanged(ev) {
rebuildQueueState = (ev.queue || []).slice(); rebuildQueueState = (ev.queue || []).slice();
// Re-render the SW4RM tab so newly-queued / newly-running // Re-render the SW4RM tab so newly-queued ops light up the right
// rebuild-queue ops light up the right card with a building... // card with a "<kind> queued…" badge, and entries that drop out of
// badge, and finished ops fall back to the regular state badges. // Pending fall back to whatever transientsState (or nothing) says
// instead. Running work is *not* driven by this event — that's
// transient_set/transient_cleared's job, since every running node
// naming an agent already lights a pill by the time it gets here.
// See docs/web-ui.md::Container row for the badge taxonomy. // See docs/web-ui.md::Container row for the badge taxonomy.
renderContainersFromState(); renderContainersFromState();
} }
// Map from agent name -> highest-priority in-flight op ({ kind, state }) // Map from agent name -> the queued op ({ kind }) backing the SW4RM
// (`running` beats `queued`). Used by the container row renderer to // card's "<kind> queued" badge. Pending only — every *running* node
// surface "building..." / "meta-updating..." badges on the SW4RM tab // naming an agent already lights a transient pill (any running node,
// when an op is still in the rebuild queue but no operator-initiated // not just a curated "worth it" subset — see docs/web-ui.md::Container
// transient is set. // row), so a Running entry here would always be redundant with
// `transientsState` by the time this is consulted. Queued (not yet
// started) work is the one state transients can't represent, since
// `running_transients()` on the backend is a Running-only test.
// //
// Agent is per-node, not per-DAG (a DAG can span agents — e.g. the // Agent is per-node, not per-DAG (a DAG can span agents — e.g. the
// startup sweep's MetaLock cascade, or a hive-wide restart), so this // startup sweep's MetaLock cascade, or a hive-wide restart), so this
// derives each agent's in-flight state from its own node(s) within the // derives each agent's queued state from its own node(s) within the
// entry rather than the DAG's overall `state`/`kind` — a DAG can be // entry rather than the DAG's overall `state`/`kind`.
// `running` overall while a given agent's subgraph hasn't started yet function queuedOpsByAgent() {
// (still queued behind an earlier node in its chain), and vice versa.
function inFlightOpsByAgent() {
const out = new Map(); const out = new Map();
for (const e of rebuildQueueState) { for (const e of rebuildQueueState) {
if (e.state !== 'Pending' && e.state !== 'Running') continue; if (e.state !== 'Pending') continue;
// spawn ops target an agent that doesn't exist yet as a // spawn ops target an agent that doesn't exist yet as a
// container — the transient store already drives the // container — the transient store already drives the
// pending row for that case. Skip here to avoid double- // pending row for that case. Skip here to avoid double-
// surfacing if the spawn op happens to land in the queue // surfacing if the spawn op happens to land in the queue
// while the row exists transiently. // while the row exists transiently.
if (e.kind === 'spawn') continue; if (e.kind === 'spawn') continue;
const perAgentState = new Map();
for (const n of e.nodes || []) { for (const n of e.nodes || []) {
if (!n.agent) continue; if (!n.agent || n.state !== 'Pending') continue;
if (n.state !== 'Pending' && n.state !== 'Running') continue; // First entry found wins — with only one state to consider
const cur = perAgentState.get(n.agent); // (Pending), there's no priority to resolve between DAGs.
if (!cur || (cur === 'Pending' && n.state === 'Running')) { if (!out.has(n.agent)) out.set(n.agent, { kind: e.kind });
perAgentState.set(n.agent, n.state);
}
}
for (const [agent, state] of perAgentState) {
const cur = out.get(agent);
if (!cur || (cur.state === 'Pending' && state === 'Running')) {
out.set(agent, { kind: e.kind, state });
}
} }
} }
return out; return out;
} }
// "<kind> queued" text for the queuedOpsByAgent fallback badge. Only
// ever applies to this fallback — a transient's own kind is an opaque
// wire tag (docs/web-ui.md::Container row) displayed as-is, never
// run through a lookup like this one.
function queuedLabelFor(kind) {
return kind === 'meta_update' ? 'meta-update queued'
: kind === 'destroy' ? 'destroy queued'
: kind === 'restart' ? 'restart queued'
: kind === 'start' ? 'start queued'
: kind === 'stop' ? 'stop queued'
: kind === 'graceful_stop' ? 'stop queued'
: kind === 'reconcile' ? 'reconcile queued'
: 'rebuild queued';
}
// ─── transients ───────────────────────────────────────────────────────────── // ─── transients ─────────────────────────────────────────────────────────────
export function syncTransientsFromSnapshot(s) { export function syncTransientsFromSnapshot(s) {
transientsState.clear(); transientsState.clear();
const nowUnix = Math.floor(Date.now() / 1000);
for (const t of s.transients || []) { for (const t of s.transients || []) {
// Snapshot is a flat list — multiple rows CAN share the same
// `name` (one row per running node) — so group into this agent's
// kind map rather than overwrite.
let byKind = transientsState.get(t.name);
if (!byKind) {
byKind = new Map();
transientsState.set(t.name, byKind);
}
// Snapshot ships `secs` (server-computed); reconstruct an // Snapshot ships `secs` (server-computed); reconstruct an
// approximate since_unix so the live ticker keeps progressing // approximate since_unix so the live ticker keeps progressing
// without surprising jumps when the next snapshot lands. // without surprising jumps when the next snapshot lands.
const nowUnix = Math.floor(Date.now() / 1000); byKind.set(t.kind, t.since_unix ?? (nowUnix - (t.secs || 0)));
transientsState.set(t.name, {
kind: t.kind,
since_unix: t.since_unix ?? (nowUnix - (t.secs || 0)),
});
} }
} }
export function applyTransientSet(ev) { export function applyTransientSet(ev) {
transientsState.set(ev.name, { let byKind = transientsState.get(ev.name);
kind: ev.transient_kind, if (!byKind) {
since_unix: ev.since_unix, byKind = new Map();
}); transientsState.set(ev.name, byKind);
}
byKind.set(ev.transient_kind, ev.since_unix);
renderContainersFromState(); renderContainersFromState();
} }
export function applyTransientCleared(ev) { export function applyTransientCleared(ev) {
if (transientsState.delete(ev.name)) renderContainersFromState(); const byKind = transientsState.get(ev.name);
if (!byKind || !byKind.delete(ev.transient_kind)) return;
// Drop the now-empty outer entry too — keeps `transientsState.size`
// meaning "agents with at least one pill" (checked at the
// no-containers-and-no-transients empty-state gate below).
if (byKind.size === 0) transientsState.delete(ev.name);
renderContainersFromState();
} }
// Re-render using the last cached snapshot (containers come from // Re-render using the last cached snapshot (containers come from
// /api/state, transients overlay from the derived map). The snapshot // /api/state, transients overlay from the derived map). The snapshot
@ -327,7 +354,7 @@ function buildContainerLi(c, node, opts) {
} = opts; } = opts;
const li = el('li', { const li = el('li', {
class: 'container-row' class: 'container-row'
+ (pending ? ' pending' : '') + (pending.length ? ' pending' : '')
+ (opRunning ? ' pending-running' : '') + (opRunning ? ' pending-running' : '')
+ (selected ? ' selected' : ''), + (selected ? ' selected' : ''),
}); });
@ -489,9 +516,14 @@ function buildContainerLi(c, node, opts) {
// badge. `needs_login` is still c0re-owned (reads auth sentinel // badge. `needs_login` is still c0re-owned (reads auth sentinel
// files on the host). rate_limited / ctx / status_text are // files on the host). rate_limited / ctx / status_text are
// agent-owned and rendered by the async dashboard-state fetch above. // agent-owned and rendered by the async dashboard-state fetch above.
if (pending) { if (pending.length) {
head.append(el('span', { class: 'pending-state' }, // One badge per pill — an agent can carry several transients at
el('span', { class: 'spinner' }, '◐'), ' ', pending + '…')); // once now (see docs/web-ui.md::Container row), each rendered
// independently rather than collapsed into one label.
for (const label of pending) {
head.append(el('span', { class: 'pending-state' },
el('span', { class: 'spinner' }, '◐'), ' ', label + '…'));
}
} else if (!c.running) { } else if (!c.running) {
head.append(el('span', head.append(el('span',
{ class: 'badge badge-muted', title: 'container is shut down — start it to bring the harness back up' }, { class: 'badge badge-muted', title: 'container is shut down — start it to bring the harness back up' },
@ -648,11 +680,10 @@ export function renderContainers(s) {
const forgeBase = (s && s.forge_public_url) || null; const forgeBase = (s && s.forge_public_url) || null;
const ul = existingUl ?? el('ul', { class: 'containers' }); const ul = existingUl ?? el('ul', { class: 'containers' });
const tree = buildAgentTree(containers); const tree = buildAgentTree(containers);
// In-flight rebuild / meta-update / destroy ops per agent name — // Queued (not-yet-running) ops per agent name — see
// see docs/web-ui.md::Container row for the building... badge // docs/web-ui.md::Container row for why this only covers the
// rationale (covers the SYST3M-shows-rebuild-but-SW4RM-shows-stopped // Pending case now (Running is fully covered by transientsState).
// gap when no operator transient is set). const queuedOps = queuedOpsByAgent();
const inFlight = inFlightOpsByAgent();
// Build the ordered list of <li> elements, reusing cached rows // Build the ordered list of <li> elements, reusing cached rows
// whose displayed state hasn't changed. // whose displayed state hasn't changed.
@ -666,31 +697,20 @@ export function renderContainers(s) {
const containerBase = gatewayLinks const containerBase = gatewayLinks
? `/agent/${encodeURIComponent(c.name)}` ? `/agent/${encodeURIComponent(c.name)}`
: `http://${hostname}:${c.port}`; : `http://${hostname}:${c.port}`;
// Pending-state derivation + queued-vs-running split — see // Pending-badge derivation: an agent's transients win outright when
// docs/web-ui.md::Container row for the transient -> in-flight // any exist (rendered one badge per pill — mara: "show all running
// queue priority order and the opRunning rationale. // nodes that name the agent"), the rebuild-queue's queued-only
const transientKind = transientsState.get(c.name)?.kind || null; // fallback otherwise. See docs/web-ui.md::Container row.
const op = !transientKind ? inFlight.get(c.name) : null; const transientKindsMap = transientsState.get(c.name);
const pending = transientKind // Sorted for stable badge order across renders — Map iteration
|| (op && (op.state === 'Running' // order is insertion order, which shifts as pills clear/re-add.
? (op.kind === 'meta_update' ? 'meta-updating' const transientKinds = transientKindsMap
: op.kind === 'destroy' ? 'destroying' ? Array.from(transientKindsMap.keys()).sort() : [];
: op.kind === 'restart' ? 'restarting' const queuedOp = transientKinds.length === 0 ? queuedOps.get(c.name) : null;
: op.kind === 'start' ? 'starting' const pending = transientKinds.length > 0
: op.kind === 'stop' ? 'stopping' ? transientKinds
: op.kind === 'graceful_stop' ? 'stopping' : (queuedOp ? [queuedLabelFor(queuedOp.kind)] : []);
: op.kind === 'reconcile' ? 'reconciling' const opRunning = transientKinds.length > 0;
: 'rebuilding')
: (op.kind === 'meta_update' ? 'meta-update queued'
: op.kind === 'destroy' ? 'destroy queued'
: op.kind === 'restart' ? 'restart queued'
: op.kind === 'start' ? 'start queued'
: op.kind === 'stop' ? 'stop queued'
: op.kind === 'graceful_stop' ? 'stop queued'
: op.kind === 'reconcile' ? 'reconcile queued'
: 'rebuild queued')));
const opRunning = transientKind != null
|| (op != null && op.state === 'Running');
const selected = selectionState.has(c.name); const selected = selectionState.has(c.name);
// Pending questions where this agent is the asker (awaiting an // Pending questions where this agent is the asker (awaiting an
// answer) or the target (owes a reply). Derived live from // answer) or the target (owes a reply). Derived live from