vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(#2453): remove DAG parent_id now that every op is one DAG

Browse source 
With the meta-update cascade (#2476) and startup sweep (#2450) folded
into single DAGs that grow per-agent subgraphs via append_subgraph,
nothing links parent/child DAGs anymore — parent_id is dead.

hive-c0re: drop parent_id from Dag/DagSpec (+ the DagView copy); delete
append_children and cancel_children (no callers); simplify trim_history
(no more terminal-parent-with-live-children guard — a one-big-DAG is
terminal only when its whole graph settles); drop the rebuild() parent_id
param; QueueDag returns just the polled DAG (no fan-out children to
gather). hive-sh4re: drop the DagView.parent_id wire field.

frontend: a multi-step op is one DAG now, so renderRebuildQueue drops the
childrenOf/orphans cross-DAG grouping and renders each entry flat; its
per-agent subgraphs render as nodes within the one row (split by deps).
Removed the dead rqe-child style + isChild plumbing.

Docs + the child-DAG queue tests updated/removed to match.
This commit is contained in:
atlas 2026-07-15 19:50:04 +02:00
commit edf9fd036e
15 changed files with 40 additions and 337 deletions
Download patch file Download diff file Expand all files Collapse all files

2
hive-c0re/src/bin/hivectl/dag_progress.rs
View file

@ -349,7 +349,6 @@ mod tests {
kind: Template::Rebuild,
state: State::Running,
source: Source::Manual,
parent_id: None,
reason: "manual".to_owned(),
enqueued_at: 0,
started_at: Some(1),
@ -389,7 +388,6 @@ mod tests {
kind: Template::Rebuild,
state: State::Failed,
source: Source::Manual,
parent_id: None,
reason: "manual".to_owned(),
enqueued_at: 0,
started_at: Some(1),

4
hive-c0re/src/dashboard_events.rs
View file

@ -208,8 +208,8 @@ pub enum DashboardEvent {
/// snapshot-shape rationale as `TombstonesChanged` /
/// `MetaInputsChanged`: the list is small, snapshot semantics avoid
/// the add/remove races a per-row event would have, and the
/// dashboard's grouping (`parent_id`) is most naturally re-derived
/// from the full list.
/// dashboard renders each DAG's multi-agent shape from its `nodes`
/// (grouped by `NodeView::agent`) — no cross-DAG grouping needed.
RebuildQueueChanged { seq: u64, queue: Vec<DagView> },
/// Full snapshot of all scheduled prompts. Emitted after every
/// operator mutation (new / edit / cancel / fire-now) and after the

51
hive-c0re/src/job_queue/mod.rs
View file

@ -154,20 +154,6 @@ impl JobQueue {
Ok(id)
}
/// Append fan-out children under a parent DAG (meta-update / sweep
/// cascade); returns the child ids created. No dedup (see
/// [`Self::submit`]).
pub fn append_children(&self, specs: Vec<DagSpec>) -> Vec<u64> {
let mut ids = Vec::with_capacity(specs.len());
for spec in specs {
match self.submit(spec) {
Ok(id) => ids.push(id),
Err(e) => tracing::error!(error = ?e, "job_queue: invalid fan-out child spec"),
}
}
ids
}
/// Append a node into a *live* (non-terminal) DAG at runtime,
/// depending `AfterOk` on `dep_on` (the node that emitted it). Lets a
/// planner node — e.g. [`NodeKind::Reconcile`] — fan a mechanical
@ -297,7 +283,6 @@ impl JobQueue {
template: spec.template,
source: spec.source,
reason: spec.reason,
parent_id: spec.parent_id,
approval_id: spec.approval_id,
inputs: spec.inputs,
perm_payload: spec.perm_payload,
@ -541,29 +526,6 @@ impl JobQueue {
true
}
/// Cancel every still-fully-queued child DAG of `parent`. Running
/// children are left alone. Returns the count of cancelled DAGs.
pub fn cancel_children(&self, parent: u64) -> usize {
let mut inner = self.inner.lock().expect("job_queue mutex poisoned");
let now = now_unix();
let mut count = 0;
for dag in &mut inner.dags {
if dag.parent_id == Some(parent) && dag.rollup() == State::Queued {
for n in &mut dag.nodes {
n.state = State::Cancelled;
n.finished_at = Some(now);
}
count += 1;
}
}
if count > 0 {
Self::settle(&mut inner);
drop(inner);
self.notify.notify_one();
}
count
}
/// Set the step label on a `Running` node. Returns `true` when the
/// label actually changed (callers emit a snapshot only then).
pub fn set_step(&self, dag_id: u64, node_id: NodeId, step: &str) -> bool {
@ -653,25 +615,16 @@ impl JobQueue {
}
/// Keep only the newest `MAX_HISTORY_PER_TEMPLATE` terminal DAGs
/// per template. Never evicted: live DAGs; terminal parents with
/// live children (a fan-out parent is terminal the moment its
/// `MetaLock` completes — evicting it while cascade rebuilds run
/// would orphan their dashboard group); and terminal DAGs that
/// per template. Never evicted: live DAGs; and terminal DAGs that
/// finished after `grace_cutoff` (see [`HISTORY_GRACE_SECS`]).
fn trim_history(inner: &mut Inner, grace_cutoff: i64) {
let live_parents: std::collections::HashSet<u64> = inner
.dags
.iter()
.filter(|d| !d.is_terminal())
.filter_map(|d| d.parent_id)
.collect();
let mut counts: HashMap<Template, usize> = HashMap::new();
let kept: Vec<Dag> = inner
.dags
.iter()
.rev()
.filter(|d| {
if !d.is_terminal() || live_parents.contains(&d.id) {
if !d.is_terminal() {
return true;
}
let finished = d.nodes.iter().filter_map(|n| n.finished_at).max();

13
hive-c0re/src/job_queue/model.rs
View file

@ -115,11 +115,10 @@ pub enum NodeKind {
/// deploy stays inside `actions.rs` in v1 — deliberately not
/// modeled as scheduler nodes (see the design doc §9).
ApprovalDeploy,
/// No-op anchor that completes immediately with no work. It exists so a
/// boot's `StartupSweep` + per-agent `Reconcile` child DAGs can hang off
/// one root (`parent_id`) and render as a single boot tree on the
/// dashboard. Holds no lease and no build slot; the child DAGs it anchors
/// still run concurrently — the grouping is a display link, not a dep edge.
/// No-op anchor that completes immediately with no work. Vestigial since
/// the boot sweep became one in-DAG graph (no `boot_root` anchor / child
/// DAGs) — slated for removal with the other `StartupSweep` residuals.
/// Holds no lease and no build slot.
Noop,
/// Write the agent's durable power intent (`wanted = Up` when `up`, else
/// `Offline`) as a first-class DAG node, at the head of a power-op
@ -242,8 +241,6 @@ pub struct DagSpec {
pub source: Source,
/// Free-form "why".
pub reason: String,
/// Cascade grouping (meta-update / sweep children).
pub parent_id: Option<u64>,
/// Fires the approval-resolution hook on DAG terminal.
pub approval_id: Option<i64>,
/// `MetaUpdate`-only: the inputs to bump (also part of the dedup
@ -266,7 +263,6 @@ pub struct Dag {
pub template: Template,
pub source: Source,
pub reason: String,
pub parent_id: Option<u64>,
pub approval_id: Option<i64>,
pub inputs: Vec<String>,
pub perm_payload: Option<PermPayload>,
@ -355,7 +351,6 @@ impl Dag {
kind: self.template,
state: self.rollup(),
source: self.source,
parent_id: self.parent_id,
reason: self.reason.clone(),
enqueued_at: self.created_at,
started_at,

7
hive-c0re/src/job_queue/submit.rs
View file

@ -38,10 +38,10 @@ fn submit_and_emit(coord: &Arc<Coordinator>, spec: super::DagSpec) -> u64 {
}
/// Manual/approval-independent rebuild (always relocks the agent's
/// meta input — cascade children are built by the scheduler's fan-out
/// instead of this surface).
/// meta input — the meta-update cascade grows its own rebuild subgraphs
/// in-DAG instead of going through this surface).
pub fn rebuild(coord: &Arc<Coordinator>, agent: &str, source: Source, reason: String) -> u64 {
submit_and_emit(coord, templates::rebuild(agent, source, reason, None, true))
submit_and_emit(coord, templates::rebuild(agent, source, reason, true))
}
// ---- dynamic power-op DAG assembly ----------------------------------------
@ -148,7 +148,6 @@ fn power_dag(
template,
source,
reason,
parent_id: None,
approval_id: None,
inputs: Vec::new(),
perm_payload: None,

18
hive-c0re/src/job_queue/templates.rs
View file

@ -19,7 +19,7 @@
//! rebuild(a): Prebuild(a) → StopForUpdate(a) → Swap(a) →(any) Reconcile(a)
//! spawn(a): Provision(a) → Create(a) → WriteDropin(a) → Reconcile(a) [wanted=Up at approve]
//! perm-change(a): WritePermFile(a) → «rebuild subgraph»
//! meta-update(inp): MetaLock(inp) → «fan-out rebuild(a) per affected a»
//! meta-update(inp): MetaLock(inp) →«in-DAG rebuild subgraph per affected a»
//! ```
//!
//! For the dynamic power-op shapes (`stop` / `start` / `restart`, built from
@ -83,18 +83,11 @@ pub(crate) fn rebuild_nodes(agent: &str, relock: bool, base: u32) -> Vec<NodeSpe
/// when `wanted = Offline` (a rebuild of a deliberately-stopped agent
/// leaves it stopped). `relock = false` only for meta-update cascade
/// children.
pub fn rebuild(
agent: &str,
source: Source,
reason: String,
parent_id: Option<u64>,
relock: bool,
) -> DagSpec {
pub fn rebuild(agent: &str, source: Source, reason: String, relock: bool) -> DagSpec {
DagSpec {
template: Template::Rebuild,
source,
reason,
parent_id,
approval_id: None,
inputs: Vec::new(),
perm_payload: None,
@ -111,7 +104,6 @@ pub fn approval_deploy(agent: &str, approval_id: i64, reason: String) -> DagSpec
template: Template::Rebuild,
source: Source::Approval,
reason,
parent_id: None,
approval_id: Some(approval_id),
inputs: Vec::new(),
perm_payload: None,
@ -135,7 +127,6 @@ pub fn reconcile_only(
template,
source,
reason,
parent_id: None,
approval_id: None,
inputs: Vec::new(),
perm_payload: None,
@ -153,7 +144,6 @@ pub fn spawn(agent: &str, approval_id: i64, reason: String) -> DagSpec {
template: Template::Spawn,
source: Source::Approval,
reason,
parent_id: None,
approval_id: Some(approval_id),
inputs: Vec::new(),
perm_payload: None,
@ -177,7 +167,6 @@ pub fn perm_change(agent: &str, source: Source, reason: String, payload: PermPay
template: Template::PermChange,
source,
reason,
parent_id: None,
approval_id: None,
inputs: Vec::new(),
perm_payload: Some(payload),
@ -190,7 +179,7 @@ pub fn perm_change(agent: &str, source: Source, reason: String, payload: PermPay
/// per affected agent into *this same* DAG on completion (via
/// `append_subgraph`) — appended *after* the bump lands so their prebuilds
/// run against the post-bump lock, and a failed bump appends nothing
/// (replacing the old fan-out-child-DAGs + `cancel_children` dance).
/// (replacing the old fan-out-child-DAGs dance).
/// `transient = Rebuilding` because those appended subgraphs are rebuilds:
/// it's applied per-agent at claim time (the `MetaLock` head needs no lease,
/// so the "hyperhive" pseudo-agent gets no pill), giving each cascade agent
@ -206,7 +195,6 @@ pub fn meta_update(
template: Template::MetaUpdate,
source,
reason,
parent_id: None,
approval_id,
inputs,
perm_payload: None,

207
hive-c0re/src/job_queue/tests.rs
View file

@ -1,7 +1,8 @@
//! Queue-core unit tests: submit / no-dedup, cycle rejection, resource
//! serialization (build slots / per-agent leases), lease-exempt
//! overlap, FIFO fairness, cancel semantics, `AfterAny` failure
//! routing, fan-out, and history retention. All synchronous — the
//! routing, in-DAG subgraph growth, and history retention. All
//! synchronous — the
//! scheduler's async loop is a thin claim/complete pump over the same
//! methods exercised here.
@ -13,7 +14,7 @@ fn submit(q: &JobQueue, spec: DagSpec) -> u64 {
}
fn rebuild(agent: &str, reason: &str) -> DagSpec {
templates::rebuild(agent, Source::Manual, reason.to_owned(), None, true)
templates::rebuild(agent, Source::Manual, reason.to_owned(), true)
}
/// Restart DAG spec with every agent treated as **running** — the online
@ -460,7 +461,6 @@ fn append_subgraph_roots_on_emitter_and_rebases_local_deps() {
template: Template::Boot,
source: Source::AutoUpdate,
reason: "sweep".to_owned(),
parent_id: None,
approval_id: None,
inputs: Vec::new(),
perm_payload: None,
@ -646,125 +646,6 @@ fn cancel_refuses_running_dag() {
assert_eq!(state_of(&q, id), State::Running);
}
#[test]
fn cancel_children_marks_queued_children_only() {
let q = JobQueue::new(1);
let meta = submit(
&q,
templates::meta_update(vec![], Source::Manual, "bump".to_owned(), None),
);
// Parent's MetaLock is running while children exist.
let lock = claim_one(&q);
assert_eq!(lock.dag_id, meta);
let child_a = submit(
&q,
templates::rebuild(
"agent-a",
Source::MetaUpdate,
"cascade".to_owned(),
Some(meta),
false,
),
);
let child_b = submit(
&q,
templates::rebuild(
"agent-b",
Source::MetaUpdate,
"cascade".to_owned(),
Some(meta),
false,
),
);
let unrelated = submit(&q, rebuild("agent-c", "operator queued"));
// MetaLock holds the single build slot, so both children (and the
// unrelated rebuild) are still fully queued here.
let cancelled = q.cancel_children(meta);
assert_eq!(cancelled, 2);
assert_eq!(state_of(&q, child_a), State::Cancelled);
assert_eq!(state_of(&q, child_b), State::Cancelled);
assert_eq!(state_of(&q, unrelated), State::Queued);
}
#[test]
fn cancel_children_skips_running_child() {
let q = JobQueue::new(2);
let meta = submit(
&q,
templates::meta_update(vec![], Source::Manual, "bump".to_owned(), None),
);
let lock = claim_one(&q);
let running_child = submit(
&q,
templates::rebuild(
"agent-a",
Source::MetaUpdate,
"cascade".to_owned(),
Some(meta),
false,
),
);
let queued_child = submit(
&q,
templates::rebuild(
"agent-b",
Source::MetaUpdate,
"cascade".to_owned(),
Some(meta),
false,
),
);
// Second slot lets running_child's prebuild start.
let child_claim = claim_one(&q);
assert_eq!(child_claim.dag_id, running_child);
let n = q.cancel_children(meta);
assert_eq!(n, 1);
assert_eq!(state_of(&q, running_child), State::Running);
assert_eq!(state_of(&q, queued_child), State::Cancelled);
q.complete_node(meta, lock.node_id, Ok(()));
}
// ---- fan-out ----
#[test]
fn append_children_sets_parent() {
let q = JobQueue::new(1);
let meta = submit(
&q,
templates::meta_update(vec![], Source::Manual, "bump".to_owned(), None),
);
let specs = vec![
templates::rebuild(
"alice",
Source::MetaUpdate,
"cascade".to_owned(),
Some(meta),
false,
),
templates::rebuild(
"bob",
Source::MetaUpdate,
"cascade".to_owned(),
Some(meta),
false,
),
// No dedup: a second alice child is its own DAG now.
templates::rebuild(
"alice",
Source::MetaUpdate,
"cascade again".to_owned(),
Some(meta),
false,
),
];
let ids = q.append_children(specs);
assert_eq!(ids.len(), 3);
assert_ne!(ids[0], ids[2], "no dedup: duplicate child is distinct");
let snap = q.snapshot();
let children: Vec<_> = snap.iter().filter(|d| d.parent_id == Some(meta)).collect();
assert_eq!(children.len(), 3);
}
// ---- terminal reporting + lease release ----
#[test]
@ -826,88 +707,6 @@ fn cancelled_dag_reports_terminal_once() {
assert!(q.drain_terminal().iter().all(|t| t.dag_id != id));
}
#[test]
fn cancel_children_reports_terminals() {
let q = JobQueue::new(1);
let meta = submit(
&q,
templates::meta_update(vec![], Source::Manual, "bump".to_owned(), None),
);
let _lock = claim_one(&q);
let child = submit(
&q,
templates::rebuild(
"agent-a",
Source::MetaUpdate,
"cascade".to_owned(),
Some(meta),
false,
),
);
assert_eq!(q.cancel_children(meta), 1);
let reports = q.drain_terminal();
assert_eq!(reports.len(), 1);
assert_eq!(reports[0].dag_id, child);
assert_eq!(reports[0].state, State::Cancelled);
}
/// History trim must not evict a terminal fan-out parent while its
/// children are still live — the dashboard groups children under it.
#[test]
fn trim_keeps_terminal_parent_with_live_children() {
let q = JobQueue::new(1);
// Pin agent-x's lease with a running stop DAG so the child below
// stays fully queued while we churn history.
let pin = submit(
&q,
templates::reconcile_only(
Template::Stop,
"agent-x",
Source::Manual,
"lease pin".to_owned(),
None,
),
);
let pin_claim = claim_one(&q);
assert_eq!(pin_claim.dag_id, pin);
// Terminal fan-out parent + a lease-blocked child under it.
let meta = submit(
&q,
templates::meta_update(vec![], Source::Manual, "bump".to_owned(), None),
);
let lock = claim_one(&q);
q.complete_node(meta, lock.node_id, Ok(()));
let mut child_spec = submit::restart_spec(
&[("agent-x".to_owned(), true)],
false,
Source::MetaUpdate,
"cascade".to_owned(),
);
child_spec.parent_id = Some(meta);
let child = submit(&q, child_spec);
// Churn > MAX_HISTORY_PER_TEMPLATE terminal meta_update DAGs.
for i in 0..7 {
let id = submit(
&q,
templates::meta_update(
vec![format!("input-{i}")],
Source::Manual,
"churn".to_owned(),
None,
),
);
let c = claim_one(&q);
assert_eq!(c.dag_id, id, "child is lease-blocked; churn claims freely");
q.complete_node(id, c.node_id, Ok(()));
}
let snap = q.snapshot();
assert!(
snap.iter().any(|d| d.id == meta),
"terminal parent with live child must survive trim"
);
assert!(snap.iter().any(|d| d.id == child));
}
// ---- steps, build logs, history ----
#[test]

4
hive-c0re/src/server.rs
View file

@ -145,12 +145,12 @@ async fn dispatch(req: &HostRequest, coord: Arc<Coordinator>) -> HostResponse {
}
HostRequest::Rebuild { name } => submit_single(&coord, name, Verb::Rebuild).await,
HostRequest::QueueDag { id } => {
// The polled DAG first, then its live fan-out children.
// A multi-step op is one DAG now (no fan-out children to gather).
let dags = coord
.job_queue
.snapshot()
.into_iter()
.filter(|d| d.id == *id || d.parent_id == Some(*id))
.filter(|d| d.id == *id)
.collect();
HostResponse::dags(dags)
}

2
hive-c0re/src/workers/auto_update.rs
View file

@ -112,7 +112,6 @@ pub async fn ensure_root_agent(coord: &Arc<Coordinator>) -> Result<()> {
MANAGER_NAME,
crate::job_queue::Source::AutoUpdate,
"manager migration: no applied flake".to_owned(),
None,
true,
)) {
tracing::warn!(error = ?e, "manager migration rebuild submit failed");
@ -347,7 +346,6 @@ fn submit_boot_tree(
template: Template::Boot,
source: Source::AutoUpdate,
reason,
parent_id: None,
approval_id: None,
inputs: Vec::new(),
perm_payload: None,