vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

feat(#2448): emit one multi-agent DAG for hivectl restart / restart-all

Browse source 
A hive-wide restart was N separate single-agent DAGs (one submit::restart
per agent). Now that agent is per-node (#2445), make it ONE DAG with a
per-agent restart subgraph each.

- templates::restart takes an agent list: each agent gets an independent
  subgraph (a head SetWanted(Up) root, then its restart chain), so the N
  subgraphs run concurrently on their own leases. One agent = the ordinary
  single-agent restart; unifies the old restart + graceful_restart fns.
- submit::restart / graceful_restart stay as single-agent wrappers over
  the new submit::restart_many(agents, graceful).
- server.rs handle_restart_all + handle_restart_scoped submit one
  restart_many call instead of looping per agent. Infra containers
  unchanged (no lease/DAG, synchronous).

Scope: restart + restart-all only. Broad stop+start is the same pattern
(stop/start templates take agent lists) — a follow-up increment.
This commit is contained in:
atlas 2026-07-14 23:00:10 +02:00 • committed by mara
commit 1739716fa2
3 changed files with 105 additions and 100 deletions
Download patch file Download diff file Expand all files Collapse all files

96
hive-c0re/src/server.rs
View file

@ -552,28 +552,29 @@ fn submit_single(coord: &Arc<Coordinator>, name: &str, verb: Verb) -> HostRespon
HostResponse::queued(vec![id])
}
/// Restart every container by submitting one restart DAG per agent —
/// each serializes on its own lease, so unrelated agents' restarts
/// overlap while nothing races an in-flight rebuild. Returns once all
/// are queued; per-agent results surface on the queue.
/// Restart every container in **one** DAG — a per-agent restart subgraph
/// each, running concurrently on their own leases (so unrelated agents'
/// restarts overlap while nothing races an in-flight rebuild). Returns
/// once queued; per-node progress surfaces on the single DAG.
async fn handle_restart_all(coord: &Arc<Coordinator>) -> Result<HostResponse> {
tracing::info!("restart-all");
let agents = lifecycle::list().await?;
let mut ok_agents: Vec<String> = Vec::new();
let mut queued: Vec<u64> = Vec::new();
for agent in &agents {
let Some(logical) = agent.strip_prefix(lifecycle::AGENT_PREFIX) else {
continue;
};
queued.push(crate::job_queue::submit::restart(
let containers = lifecycle::list().await?;
let agents: Vec<String> = containers
.iter()
.filter_map(|a| a.strip_prefix(lifecycle::AGENT_PREFIX).map(str::to_owned))
.collect();
let queued = if agents.is_empty() {
Vec::new()
} else {
vec![crate::job_queue::submit::restart_many(
coord,
logical,
&agents,
false,
crate::job_queue::Source::Manual,
"manual restart via hivectl restart-all".to_owned(),
));
ok_agents.push(logical.to_owned());
}
let mut resp = HostResponse::list(ok_agents);
)]
};
let mut resp = HostResponse::list(agents);
resp.queued_dags = Some(queued);
Ok(resp)
}
@ -718,16 +719,14 @@ async fn handle_start(
}
/// Restart containers hive-wide (`hivectl restart`) — the DAG-based
/// sibling of [`handle_stop`]/[`handle_start`], replacing the old
/// client-side stop-then-start composition (issue tracker "dagify hivectl
/// commands"). Each targeted agent gets exactly one atomic DAG submitted
/// up front: the `Restart` template (mechanical stop + reconcile) in the
/// common case, or `GracefulRestart` (signal → drain → mechanical stop →
/// reconcile) with `graceful` set — no "submit a DAG, wait for it, submit
/// another" composition on either path, so a dropped `hivectl` connection
/// never strands an agent, the same way `handle_restart_all` already
/// avoids it. Infra containers have no lease/DAG and restart
/// synchronously (stop then start).
/// sibling of [`handle_stop`]/[`handle_start`]. All targeted agents ride
/// **one** DAG (a per-agent restart subgraph each: `SetWanted → [Signal →
/// Drain →] StopForUpdate → Reconcile`, independent roots that run
/// concurrently on their own leases), not N separate DAGs — a hive-wide
/// restart is one job. `graceful` prepends signal→drain per agent. No
/// client-side stop-then-start composition, so a dropped `hivectl`
/// connection never strands an agent. Infra containers have no lease/DAG
/// and restart synchronously (stop then start).
async fn handle_restart_scoped(
coord: &Arc<Coordinator>,
scope: &LifecycleScope,
@ -740,30 +739,25 @@ async fn handle_restart_scoped(
let mut errors: Vec<String> = Vec::new();
let mut queued: Vec<u64> = Vec::new();
// One atomic DAG per agent, submitted up front — `Restart`
// (mechanical stop + reconcile) or, with `--graceful`,
// `GracefulRestart` (signal → drain → mechanical stop → reconcile).
// No client- or server-side "submit a stop DAG, await it, then
// submit a start DAG" composition: that window is exactly the
// dropped-connection gap this DAG-based path exists to close.
for agent in &agents {
let id = if graceful {
crate::job_queue::submit::graceful_restart(
coord,
agent,
crate::job_queue::Source::Manual,
"manual via hivectl restart --graceful".to_owned(),
)
} else {
crate::job_queue::submit::restart(
coord,
agent,
crate::job_queue::Source::Manual,
"manual restart via hivectl restart".to_owned(),
)
};
queued.push(id);
ok_items.push(agent.clone());
// One DAG for all targeted agents — a per-agent restart subgraph each
// (`SetWanted → [Signal → Drain →] StopForUpdate → Reconcile`),
// independent roots that run concurrently on their own leases. A
// hive-wide `hivectl restart` is now a single DAG, not N. No
// client-side stop-then-start composition — the whole restart survives
// a dropped connection because the DAG owns it.
if !agents.is_empty() {
queued.push(crate::job_queue::submit::restart_many(
coord,
&agents,
graceful,
crate::job_queue::Source::Manual,
if graceful {
"manual via hivectl restart --graceful".to_owned()
} else {
"manual restart via hivectl restart".to_owned()
},
));
ok_items.extend(agents.iter().cloned());
}
for &container in &infra {