The hive-side loop landed without anything to converge to: nothing wrote
`$KV.hive-wanted.<hive>`, so in production only the "no key" branch ran.
This is the writer.
`WantedWriter` mirrors `StatusReader` — that module reads what hives report,
this one writes what they are told, so it holds a client rather than a bucket
handle and resolves the store on first use. It shares the status reader's
connection: the controller has exactly one by design, and a second connect
would double the auth-callout traffic and give the two paths independent
reconnect state.
The value under a hive's key is the map of every agent on that hive, so a
plain `put` of a single-agent change would drop a concurrent change to a
different agent, with only one revision of history to not recover from.
Writes are read-modify-write against the entry revision, and only
`WrongLastRevision` / `AlreadyExists` count as a lost race — every other
error returns immediately rather than spinning the retry loop and then
blaming a concurrent writer that never existed.
`apply` is split out and tested because it holds the invariant: declaring
one agent preserves the rest, and a current value that will not decode is an
error rather than a fresh start. Overwriting a document nobody can read
discards every other agent's declaration.
Two routes, no swarmctl verb and no jobq node: `create_agent` needs a graph
because it is multi-step, and one CAS'd write is not.
`build_app` is extracted from `main` in the same change because `main` sat at
exactly the `too_many_lines` limit, so adding an endpoint tripped a lint
about the startup sequence. The route list is the part that grows.
mara's call on the PR: "dont make the enum open, we will just add
entries later". The catch-all variant is gone, and with it the per-agent
inert path.
What changes is where version skew lands, not whether it is handled. An
unrecognised value used to be one agent this hive left alone; it is now a
decode failure for the whole declaration, so a hive running older code
converges *nothing* rather than obeying the agents it happened to
understand. That fails closed instead of dangerous, and it is the right
trade when both ends ship together — which is what "add entries later"
assumes.
The test moved with the property rather than being rewritten in place:
`an_unknown_state_fails_the_whole_declaration` lives in
swarm-queue-client, where the decode is, with a valid entry beside it as
the control. `hive-c0re` keeps a coverage check that every state this
build knows produces an action somewhere — asserting inertness there
would be asserting something the type system no longer lets me build.
The deploy event is a nudge with no second path: core NATS is
at-most-once, so a hive that was down when the controller published
simply never learns that an agent is meant to exist here. This adds the
repair path — one boot-time DAG node that reads this hive's own key in
the `hive-wanted` bucket and converges the agents it names.
Two semantics settled on the issue thread, and both are places where a
plausible implementation is the wrong one:
- **Absence is not a deletion order.** No bucket, no key, or an agent
the value does not name all mean the controller has said nothing.
Swarm-side lifecycle does not yet cover agents that predate it, so
"converge to exactly this set" would tear down every agent the swarm
has not adopted. `plan` only ever inspects the agents a declaration
names.
- **An unrecognised state is inert.** `AgentState` is an open enum: a
value this build cannot read deserialises into `Unrecognised` and is
left alone. A closed enum would force "not `Up`" onto a state like
`paused`, so a controller that learned a new value would take agents
down on every hive not yet updated.
Divergence is measured against the hive's **stored power intent**, not
the container's observed running state — an agent that is down while its
intent says `Up` is already the boot reconcile's work, and a loop reading
`is_running` would insert a start DAG behind that reconcile's back on
every boot. A hive that already agrees with its declaration queues
nothing at all.
`queue_first_deploy` is extracted from the deploy-event path rather than
open-coded here, for the power-intent seed: without it `first_deploy`'s
tail `Reconcile` seeds `Wanted` from a container that exists but has not
started yet, which locks the agent to `Offline` on its first reconcile.
The read is authorised as-is: `store.get` takes async-nats' direct-get
arm (the KV bucket is created with `allow_direct`), which is exactly the
`$JS.API.DIRECT.GET.KV_hive-wanted.$KV.hive-wanted.<hive>` subject
`swarm-nats-auth` grants a hive. The fallback subject is not granted, and
a refused NATS request surfaces as a timeout rather than an error.
Nothing writes the bucket yet — the controller-side writer is the other
half of #3124, so this does not close it.
The controller declares each hive's wanted agent set; the hive reads its own
key. Deliberately not a mirror of `status`, which the module documents as a
table: authored by the controller rather than the hive, DECLARED rather than
observed, and — the row that decides how it must be built — unrecoverable if
the store is lost, where status regenerates because every hive republishes
what it is.
Two open functions rather than one shared `open_or_create`. `status` shares its
constructor because either end may legitimately arrive first on a fresh swarm;
here the writer is single and known, so the hive gets a read-only open
returning `Option` and holds no grant to create the bucket. Its absence is the
ordinary pre-publication state, not an error a hive could fix.
Absence is also not a deletion order, which the module says at the place an
implementer will meet it: swarm-side lifecycle does not yet cover agents that
predate it, so a hive finding no key has learned nothing about what it runs —
converging to an empty set would tear those down.
No consumer yet. This is the half that is invariant under the scope semantics
and cadence still being decided.
