fix(hive-c0re): close review findings on the job-DAG queue

- deploy-window gate (meta::exclusive) + path-limited meta commits:
  a perm/lock/topology commit can no longer sweep an ApprovalDeploy's
  staged flake.lock and neuter abort_deploy (regression test included)
- cancel surfaces now buffer terminal roll-ups the scheduler drains,
  so a queued approval DAG cancelled by the operator resolves its
  approval instead of dangling, and cancelled power ops revert their
  wanted flip to the observed state
- hivectl restart / restart-all ride the queue (lease serialization,
  transient guard) and restart sets wanted=Up like the old kill+start
- exactly one Rebuilt event per rebuild DAG, emitted at terminal
- StopForUpdate pre-seeds a missing agent_power row from the pre-stop
  observation so a rebuild can't strand an unknown agent offline
- history trim keeps terminal fan-out parents with live children
- audit_log back on db::open; swarm.js badge for reconcile DAGs
This commit is contained in:
müde 2026-07-06 21:44:43 +02:00
commit 084e12503c
12 changed files with 448 additions and 160 deletions

View file

@ -52,16 +52,32 @@ Cheap — no build slot:
There is deliberately **no `GitCommit` node**: `meta.rs` fuses each mutation
with its commit under its internal `META_LOCK` mutex, so a standalone commit
node would open a dirty-working-tree window between nodes. That same
`META_LOCK` is also why the scheduler needs no meta-repo resource class — any
executor touching the meta repo serializes inside `meta.rs`.
node would open a dirty-working-tree window between nodes.
Two further layers protect the meta repo across *windows* that span multiple
`META_LOCK` acquisitions — above all the approval deploy's prepare→finalize
span, which keeps a bumped `flake.lock` **staged uncommitted** for the whole
container build:
- **The deploy-window gate** (`meta::exclusive()`): every executor that
mutates the meta repo (`Prebuild`'s sync+relock, `MetaLock`,
`WritePermFile`, `Create`'s agent registration, and `ApprovalDeploy` for
its whole span) holds this async mutex for its mutation span, so no commit
can land inside another node's staged window. `Prebuild` drops it before
the long toplevel build (store reads only), preserving `buildSlots > 1`
concurrency.
- **Path-limited commits**: the targeted meta committers (perm files,
topology, lock bumps, finalize) commit `-- <their paths>` with path-scoped
dirty checks, so even a non-queue caller (boot migration, destroy's
`sync_agents`) can never sweep someone else's staged content into its
commit.
### Every operation as a DAG
```text
rebuild(a): Prebuild(a) → StopForUpdate(a) → Swap(a) →(after-any) Reconcile(a)
graceful-stop(a): [wanted=Offline] Signal(a) → Drain(a) → Reconcile(a)
restart(a): StopForUpdate(a) → Reconcile(a) (wanted unchanged)
restart(a): [wanted=Up] StopForUpdate(a) → Reconcile(a)
start(a): [wanted=Up] Reconcile(a) (stale rev ⇒ upgraded to rebuild)
stop(a): [wanted=Offline] Reconcile(a)
spawn(a): [wanted=Up] Create(a) → WriteDropin(a) → Reconcile(a)