feat(#3124): converge the hive onto the agent set the swarm declares
The deploy event is a nudge with no second path: core NATS is at-most-once, so a hive that was down when the controller published simply never learns that an agent is meant to exist here. This adds the repair path — one boot-time DAG node that reads this hive's own key in the `hive-wanted` bucket and converges the agents it names. Two semantics settled on the issue thread, and both are places where a plausible implementation is the wrong one: - **Absence is not a deletion order.** No bucket, no key, or an agent the value does not name all mean the controller has said nothing. Swarm-side lifecycle does not yet cover agents that predate it, so "converge to exactly this set" would tear down every agent the swarm has not adopted. `plan` only ever inspects the agents a declaration names. - **An unrecognised state is inert.** `AgentState` is an open enum: a value this build cannot read deserialises into `Unrecognised` and is left alone. A closed enum would force "not `Up`" onto a state like `paused`, so a controller that learned a new value would take agents down on every hive not yet updated. Divergence is measured against the hive's **stored power intent**, not the container's observed running state — an agent that is down while its intent says `Up` is already the boot reconcile's work, and a loop reading `is_running` would insert a start DAG behind that reconcile's back on every boot. A hive that already agrees with its declaration queues nothing at all. `queue_first_deploy` is extracted from the deploy-event path rather than open-coded here, for the power-intent seed: without it `first_deploy`'s tail `Reconcile` seeds `Wanted` from a container that exists but has not started yet, which locks the agent to `Offline` on its first reconcile. The read is authorised as-is: `store.get` takes async-nats' direct-get arm (the KV bucket is created with `allow_direct`), which is exactly the `$JS.API.DIRECT.GET.KV_hive-wanted.$KV.hive-wanted.<hive>` subject `swarm-nats-auth` grants a hive. The fallback subject is not granted, and a refused NATS request surfaces as a timeout rather than an error. Nothing writes the bucket yet — the controller-side writer is the other half of #3124, so this does not close it.
This commit is contained in:
parent
8cba57e01c
commit
37f3c63eeb
7 changed files with 526 additions and 34 deletions
|
|
@ -317,13 +317,14 @@ pub async fn run(coord: Arc<Coordinator>) -> Result<()> {
|
|||
Ok(())
|
||||
}
|
||||
|
||||
/// Submit the boot-time forge/matrix/webhook/knowledge sweeps as DAG nodes —
|
||||
/// `ForgeSweep`, `MatrixSweep`, `WebhookRegister`, `KnowledgePull`. Unlike
|
||||
/// Submit the boot-time forge/matrix/webhook/knowledge/wanted-state sweeps as
|
||||
/// DAG nodes — `ForgeSweep`, `MatrixSweep`, `WebhookRegister`,
|
||||
/// `KnowledgePull`, `WantedPull`. Unlike
|
||||
/// [`submit_boot_tree`] this runs on **every** boot, quiet or not: these
|
||||
/// aren't config-drift work, they're startup housekeeping that always needs
|
||||
/// to happen, and the point of moving them here is exactly so they show up
|
||||
/// as real work on the dashboard instead of an invisible `tokio::spawn` that
|
||||
/// only surfaces on failure. Four independent, build-slot- and lease-exempt
|
||||
/// only surfaces on failure. Five independent, build-slot- and lease-exempt
|
||||
/// roots — no dependency edges between them, matching the existing
|
||||
/// `Reconcile`-root pattern in [`boot_nodes`].
|
||||
fn submit_startup_sweep_nodes(coord: &Arc<Coordinator>) {
|
||||
|
|
@ -334,6 +335,7 @@ fn submit_startup_sweep_nodes(coord: &Arc<Coordinator>) {
|
|||
let _ = b.node(NodeKind::MatrixSweep);
|
||||
let _ = b.node(NodeKind::WebhookRegister);
|
||||
let _ = b.node(NodeKind::KnowledgePull);
|
||||
let _ = b.node(NodeKind::WantedPull);
|
||||
Vec::new()
|
||||
}) {
|
||||
tracing::warn!(error = ?e, "boot: startup sweep DAG insert failed");
|
||||
|
|
|
|||
Loading…
Reference in a new issue