feat(#3124): converge the hive onto the agent set the swarm declares

The deploy event is a nudge with no second path: core NATS is
at-most-once, so a hive that was down when the controller published
simply never learns that an agent is meant to exist here. This adds the
repair path — one boot-time DAG node that reads this hive's own key in
the `hive-wanted` bucket and converges the agents it names.

Two semantics settled on the issue thread, and both are places where a
plausible implementation is the wrong one:

- **Absence is not a deletion order.** No bucket, no key, or an agent
  the value does not name all mean the controller has said nothing.
  Swarm-side lifecycle does not yet cover agents that predate it, so
  "converge to exactly this set" would tear down every agent the swarm
  has not adopted. `plan` only ever inspects the agents a declaration
  names.
- **An unrecognised state is inert.** `AgentState` is an open enum: a
  value this build cannot read deserialises into `Unrecognised` and is
  left alone. A closed enum would force "not `Up`" onto a state like
  `paused`, so a controller that learned a new value would take agents
  down on every hive not yet updated.

Divergence is measured against the hive's **stored power intent**, not
the container's observed running state — an agent that is down while its
intent says `Up` is already the boot reconcile's work, and a loop reading
`is_running` would insert a start DAG behind that reconcile's back on
every boot. A hive that already agrees with its declaration queues
nothing at all.

`queue_first_deploy` is extracted from the deploy-event path rather than
open-coded here, for the power-intent seed: without it `first_deploy`'s
tail `Reconcile` seeds `Wanted` from a container that exists but has not
started yet, which locks the agent to `Offline` on its first reconcile.

The read is authorised as-is: `store.get` takes async-nats' direct-get
arm (the KV bucket is created with `allow_direct`), which is exactly the
`$JS.API.DIRECT.GET.KV_hive-wanted.$KV.hive-wanted.<hive>` subject
`swarm-nats-auth` grants a hive. The fallback subject is not granted, and
a refused NATS request surfaces as a timeout rather than an error.

Nothing writes the bucket yet — the controller-side writer is the other
half of #3124, so this does not close it.
This commit is contained in:
atlas 2026-09-01 13:05:53 +02:00
commit 37f3c63eeb
7 changed files with 526 additions and 34 deletions

View file

@ -317,13 +317,14 @@ pub async fn run(coord: Arc<Coordinator>) -> Result<()> {
Ok(())
}
/// Submit the boot-time forge/matrix/webhook/knowledge sweeps as DAG nodes —
/// `ForgeSweep`, `MatrixSweep`, `WebhookRegister`, `KnowledgePull`. Unlike
/// Submit the boot-time forge/matrix/webhook/knowledge/wanted-state sweeps as
/// DAG nodes — `ForgeSweep`, `MatrixSweep`, `WebhookRegister`,
/// `KnowledgePull`, `WantedPull`. Unlike
/// [`submit_boot_tree`] this runs on **every** boot, quiet or not: these
/// aren't config-drift work, they're startup housekeeping that always needs
/// to happen, and the point of moving them here is exactly so they show up
/// as real work on the dashboard instead of an invisible `tokio::spawn` that
/// only surfaces on failure. Four independent, build-slot- and lease-exempt
/// only surfaces on failure. Five independent, build-slot- and lease-exempt
/// roots — no dependency edges between them, matching the existing
/// `Reconcile`-root pattern in [`boot_nodes`].
fn submit_startup_sweep_nodes(coord: &Arc<Coordinator>) {
@ -334,6 +335,7 @@ fn submit_startup_sweep_nodes(coord: &Arc<Coordinator>) {
let _ = b.node(NodeKind::MatrixSweep);
let _ = b.node(NodeKind::WebhookRegister);
let _ = b.node(NodeKind::KnowledgePull);
let _ = b.node(NodeKind::WantedPull);
Vec::new()
}) {
tracing::warn!(error = ?e, "boot: startup sweep DAG insert failed");