mara (#4170): swarm-ui's wanted-state dropdown could only ever declare
up/offline/destroy, with no way to swarm-declare the existing hive-local
turn-loop pause (`hivectl agent pause|resume`).
`AgentState::Paused` is not a fifth peer of Up/Offline/Destroyed on the
power axis this enum otherwise answers — it's Up plus an orthogonal
turn-loop pause. `hive-c0re`'s `workers::wanted` reconcile loop now
decides the two axes independently (`decide` for power, the new
`decide_pause` for the marker), so a stopped agent declared Paused
converges with both a Start and a Pause in the same pass.
Known, deliberate limitation: a Paused declaration on an agent this
hive has never deployed only reaches Deploy this pass — writing the
pause marker into a harness dir that may not exist yet was judged not
worth the risk, so it converges on the next pass once the agent is
present instead.
swarm-ui's WantedMenu gains a fourth "paused" option (warning-tone
badge). No separate "resume" entry — selecting "up" from a paused row
already clears the marker via the same decide_pause path.
Pause/resume marker writes go through one shared
Coordinator::set_paused_by_name helper, used by both the interactive
dashboard pause/resume handlers and this reconcile loop, instead of
each duplicating the parse-name/write-marker/track-rescan shape.
swarm-ui's "offline" and "paused" confirm dialogs share one
confirmTarget state and one ConfirmDialog instead of two near-identical
copies.
Closes#4170
A hive reads its agents' credentials with its own certificate, and nothing
said which paths that certificate may read, so the read half of a delivery
answered 403.
The grant is wide on purpose. An agent's path does not name the hive
hosting it -- agents move -- so a per-hive grant has to be an enumeration
the controller re-emits whenever the roster changes, and an enumeration
that can drift or land out of order advertises a boundary it does not
hold. A wide grant that says what it is beats a narrow one that only looks
narrow. mara's call, on the PR: rather a too-lax scope than one that
pretends to be strict.
What that buys, beyond honesty: the document is identical for every hive
and depends on nothing, so it is written once at startup beside the rest of
a hive's provisioning instead of on every declaration. No derived state, no
re-emission, and the ordering hazard that came with one stops existing.
What still holds is read-only. A hive cannot write an agent's credential,
so it cannot hand itself an agent's identity, and the grant reaches nothing
in the store outside the agent-credential prefix.
The fact is documented where someone meets the boundary rather than only in
this message, and the two ways to narrow it later -- scope per hive, or
give agents their own store identity -- are tracked.
The controller writes an agent's credential; the hive fetches it back with
its own certificate. Nothing said which paths that certificate may read, so
the read half of a delivery answers 403 with no way to tell why.
The grant is derived from the declaration, so it is re-rendered at the one
place the declaration changes -- WantedWriter::set -- rather than at its
caller, which would work today and break on the second caller.
Emitted before the KV write: a grant that lands late is a 403 on an agent's
first fetch, while one that shrinks early only affects an agent already
being torn down. A failed write then leaves a superset the next declaration
re-renders.
Destroyed agents are filtered out. The declared set is a hive's whole
history -- a destroyed entry stays so that redeclaring it Up is refused as
the terminal transition it is -- so granting every declared agent would
leave a torn-down agent's credentials readable forever.
The sink is a trait because a missed emission is that same untraceable 403:
the double pins which agents were published, and the no-sink and refusing
arms pin the two deployments that are not a happy path. Not covered: the
call site inside set(), which needs a live queue.
The cert role moves to its own module on the way past. It is the
controller's identity at the store, not something the matrix route owns,
and the policy writer needs the same login.
Both were prose about the model this branch replaces, caught in review.
`swarm-controller/src/wanted.rs` was the worse of the two: its header
called the lifecycle "deliberately identical" to `status` and the handle
"resolved on first use and cached", while `store`'s own doc seventy lines
below says "resolved per call rather than cached". One file, two
contradictory claims, and the `OnceCell` that would have settled it is
gone. Rewritten to say where the mirror stops rather than to patch the
stale clause, since the divergence is the point of the change.
`hive-c0re/src/workers/wanted.rs` named a `hive-wanted` bucket that no
longer exists.
Swept by content rather than fixing only the two that were named: the
sweep surfaced a third candidate, `swarm-nats-auth/src/policy.rs`'s
"one key per hive", and reading it cleared it — that sentence is about
the hive-status bucket, whose shape is unchanged. Left alone
deliberately.
A hive reads its own declaration today and that scopes cleanly: DIRECT.GET
carries the key in the subject, so the grant can name it. A *watch* cannot be
scoped that way — a consumer's filter travels in the request payload, so
$JS.API.CONSUMER.CREATE.<stream> grants the whole stream. With every hive in
one bucket, letting a hive watch its own declaration would let it read every
other hive's.
One bucket per hive (hive-wanted-<hive>) makes the stream a hive may hold
exactly as wide as what it is allowed to see, which is what #4006's live-watch
needs. That watch is a separate change; this only moves the boundary.
mara's calls, both on #4006: one stream per hive rather than teaching the auth
responder a hive roster, and a wildcard for the controller — "its okay if
swarm controller can theoretically override hive". A bucket name is a single
subject token with no prefix matching, so no wildcard narrower than * covers N
per-hive buckets; the controller's grant is account-wide by consequence, and
documented as chosen rather than left to look accidental.
The reader arm of #4005's key-layout guard asserted the opposite of that
ruling, so it is replaced rather than deleted: the hive arm survives as
no_hive_may_write_another_role_s_agent_status (with a positive control), and
the_readers_grant_is_deliberately_account_wide pins the decision and names the
ruling, so the width reads as chosen to whoever finds it next.
Two pre-existing negative assertions were silently defanged by the rename --
they matched hive-wanted.beta and $KV.hive-wanted.alpha, strings nothing
produces any more, and kept passing. Both now match current names.
swarm-controller resolves the store per hive per call instead of caching one in
a OnceCell: there is no single handle that serves N buckets, and declarations
change on operator action rather than per tick.
The hive-side loop landed without anything to converge to: nothing wrote
`$KV.hive-wanted.<hive>`, so in production only the "no key" branch ran.
This is the writer.
`WantedWriter` mirrors `StatusReader` — that module reads what hives report,
this one writes what they are told, so it holds a client rather than a bucket
handle and resolves the store on first use. It shares the status reader's
connection: the controller has exactly one by design, and a second connect
would double the auth-callout traffic and give the two paths independent
reconnect state.
The value under a hive's key is the map of every agent on that hive, so a
plain `put` of a single-agent change would drop a concurrent change to a
different agent, with only one revision of history to not recover from.
Writes are read-modify-write against the entry revision, and only
`WrongLastRevision` / `AlreadyExists` count as a lost race — every other
error returns immediately rather than spinning the retry loop and then
blaming a concurrent writer that never existed.
`apply` is split out and tested because it holds the invariant: declaring
one agent preserves the rest, and a current value that will not decode is an
error rather than a fresh start. Overwriting a document nobody can read
discards every other agent's declaration.
Two routes, no swarmctl verb and no jobq node: `create_agent` needs a graph
because it is multi-step, and one CAS'd write is not.
`build_app` is extracted from `main` in the same change because `main` sat at
exactly the `too_many_lines` limit, so adding an endpoint tripped a lint
about the startup sequence. The route list is the part that grows.
