feat(hivectl): queue-routed lifecycle verbs with wait + DAG progress
every agent lifecycle verb on the admin socket (rebuild / restart / restart-all / kill / stop / start) now submits job-queue DAGs and returns their ids; hivectl polls the new HostRequest::QueueDag and prints a live node-chain progress line per DAG (fan-out children included), exiting non-zero on failure — --no-wait opts out. DagView and the queue wire enums move to hive_sh4re::jobs (wire types live in the shared crate); the last fused rebuild path (lifecycle::rebuild) is gone. tracker: #2166
This commit is contained in:
müde
parent
dc6a37b29a
commit
b489454dc2
9 changed files with 641 additions and 443 deletions
|
|
@ -108,17 +108,28 @@ Notable collapses:
|
|||
|
||||
### Desired-state (spec vs status)
|
||||
|
||||
Per-agent power *intent* — `wanted: Up | Offline` — is durable in
|
||||
`/var/lib/hyperhive/db/agent_power.sqlite` (`hive-c0re/src/power.rs`).
|
||||
Per-agent power *intent* — `wanted: Up | Offline` — is durable as the
|
||||
`agent_power` table in the coordinator DB (`hive-c0re/src/power.rs`).
|
||||
`container_view` remains the observed *status*; `Reconcile` nodes converge the
|
||||
two. Setting `wanted` is never a queued node: the submit layer
|
||||
(`job_queue/submit.rs`) writes the row synchronously, then submits the DAG
|
||||
whose `Reconcile` reads the fresh value — rapid toggles are last-writer-wins.
|
||||
Power toggles never commit to the meta repo. Direct (non-queued) power paths —
|
||||
`hivectl stop/start`, the admin-socket kill, the MCP kill tool — write
|
||||
`wanted` too, so reconciles never undo an operator's stop. Agents without a
|
||||
row are seeded from observed state on first touch (running ⇒ `Up`); destroy
|
||||
removes the row.
|
||||
Power toggles never commit to the meta repo. Every operator power surface —
|
||||
dashboard buttons, the MCP tools, and `hivectl stop/start/restart/kill` —
|
||||
rides the queue through that submit layer, so intent, lease serialization,
|
||||
and crash-watch suppression can't drift per surface; the only direct starts
|
||||
left are the root-agent bootstrap and infra containers (no lease, no
|
||||
harness). Cancelling a still-queued power DAG reverts `wanted` to the
|
||||
observed state — a cancel means "don't do it", not "do it later". Agents
|
||||
without a row are seeded from observed state on first touch (running ⇒
|
||||
`Up`); destroy removes the row.
|
||||
|
||||
The admin-socket responses carry the submitted DAG ids; `hivectl` polls
|
||||
`HostRequest::QueueDag` (~1s) and prints a progress line per DAG — roll-up
|
||||
glyph, template, agent, node chain with the running node's step label — so
|
||||
CLI verbs block until their jobs finish (`--no-wait` opts out; failures exit
|
||||
non-zero). Fan-out children joining a polled parent show up in the same
|
||||
loop.
|
||||
|
||||
### Scheduler semantics
|
||||
|
||||
|
|
|
|||
|
|
@ -271,8 +271,8 @@ Agent container management. Requires the hive-c0re daemon to be running (connect
|
|||
###### **Subcommands:**
|
||||
|
||||
* `list` — Show all managed agents with their status (running / needs-login / needs-update) and technical state (deployed sha, parent, pending reminders). The host roster overview; reuses the dashboard's per-agent aggregation. Requires the daemon running
|
||||
* `restart` — Stop and start a single agent container without rebuilding config. Useful for "kick the container" when the process is stuck or the container needs a clean restart without changing the NixOS config
|
||||
* `restart-all` — Stop and restart ALL managed agent containers in sequence. Iterates the live container list and restarts each one. Any per-agent failure is reported at the end rather than stopping mid-run, so all containers get a restart attempt
|
||||
* `restart` — Stop and start a single agent container without rebuilding config. Useful for "kick the container" when the process is stuck or the container needs a clean restart without changing the NixOS config. Rides the job queue (serialized against in-flight rebuilds for the same agent); waits with live progress unless `--no-wait`
|
||||
* `restart-all` — Restart ALL managed agent containers via one restart DAG each — unrelated agents overlap, each serializes on its own lease. Waits for the whole set with live progress unless `--no-wait`
|
||||
|
||||
|
||||
|
||||
|
|
@ -290,21 +290,29 @@ Show all managed agents with their status (running / needs-login / needs-update)
|
|||
|
||||
## `hivectl agents restart`
|
||||
|
||||
Stop and start a single agent container without rebuilding config. Useful for "kick the container" when the process is stuck or the container needs a clean restart without changing the NixOS config
|
||||
Stop and start a single agent container without rebuilding config. Useful for "kick the container" when the process is stuck or the container needs a clean restart without changing the NixOS config. Rides the job queue (serialized against in-flight rebuilds for the same agent); waits with live progress unless `--no-wait`
|
||||
|
||||
**Usage:** `hivectl agents restart <NAME>`
|
||||
**Usage:** `hivectl agents restart [OPTIONS] <NAME>`
|
||||
|
||||
###### **Arguments:**
|
||||
|
||||
* `<NAME>` — Agent name (e.g. `damocles`, `ruth`)
|
||||
|
||||
###### **Options:**
|
||||
|
||||
* `--no-wait` — Return immediately after the restart DAG is queued
|
||||
|
||||
|
||||
|
||||
## `hivectl agents restart-all`
|
||||
|
||||
Stop and restart ALL managed agent containers in sequence. Iterates the live container list and restarts each one. Any per-agent failure is reported at the end rather than stopping mid-run, so all containers get a restart attempt
|
||||
Restart ALL managed agent containers via one restart DAG each — unrelated agents overlap, each serializes on its own lease. Waits for the whole set with live progress unless `--no-wait`
|
||||
|
||||
**Usage:** `hivectl agents restart-all`
|
||||
**Usage:** `hivectl agents restart-all [OPTIONS]`
|
||||
|
||||
###### **Options:**
|
||||
|
||||
* `--no-wait` — Return immediately after the restart DAGs are queued
|
||||
|
||||
|
||||
|
||||
|
|
@ -408,6 +416,7 @@ Stop containers hive-wide in one operator action. Bare `hivectl stop` stops **ev
|
|||
* `--gateway` — The gateway container (`hive-gateway`)
|
||||
* `--matrix` — The matrix container (`hive-matrix`)
|
||||
* `--graceful` — Gracefully quiesce each agent before stopping, instead of a hard stop. Each agent gets a graceful-stop DAG on the job queue: the harness is signalled, runs one stop-checkpoint turn to flush durable `/state`, drains, then the container is stopped (bounded by a 3-min timeout that falls back to a hard stop). All drains overlap. Applies to agents only
|
||||
* `--no-wait` — Return immediately after the stop DAGs are queued instead of waiting for them with live per-node progress
|
||||
|
||||
|
||||
|
||||
|
|
@ -425,6 +434,7 @@ Start containers hive-wide — the inverse of `hivectl stop`. Bare `hivectl star
|
|||
* `--forge` — The forge container (`hive-forge`)
|
||||
* `--gateway` — The gateway container (`hive-gateway`)
|
||||
* `--matrix` — The matrix container (`hive-matrix`)
|
||||
* `--no-wait` — Return immediately after the start DAGs are queued instead of waiting for them with live per-node progress
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue