vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

jobq: delete DagView/NodeView, the second projection of one graph

Browse source 
Two views of the same graph existed: the typed `DagView`/`NodeView`
(`/api/state.rebuild_queue`, the `QueueDag` socket request, and the
`RebuildQueueChanged` payload) and `hive-jobq-wire`'s generic
`GraphNode` (`/api/jobq/graph`, `QueueNodes`). Every consumer has moved
to the generic one, so the typed pair is deleted rather than kept in
agreement with it.

What that removes, beyond the types: the `QueueDag` request and
`HostResponse::dags`; `Queue::snapshot`; `dag_view`, `visible_dags`,
`shown_on_wire`, `dag_finished_at` and `containers`; and the
`rebuild_queue` field on `/api/state`. `RebuildQueueChanged` keeps its
seq and loses its payload — nothing read it, and shipping the graph
both on an event and on an endpoint is the duplication this issue is
about. It stays an event rather than becoming a poll because
push-on-change is what every other live surface here does.

Two behaviours came out simpler for a structural reason. `await_dags`
needed two rules — settled means "gone from the snapshot" *or* "present
with every node terminal" — because the typed view evicted finished
groups; the generic view doesn't, so pending is just "some node isn't
terminal". And `state_of` in the tests no longer derives a roll-up at
all: a group root's own state is the scheduler's answer.

That second one found a bug. `cancelled_dag_still_runs_its_approval
tail` asserted the group reads `Cancelled` while the tail it exists to
protect was still pending — `rollup_state` flattened the surviving
child away and called the group settled. The root reads `Finishing`,
which is what the scheduler documents: own logic done, children still
running. The test now asserts that, with the reasoning inline so it
doesn't get "fixed" back.

Kept: `Source`, `State`, `PermPayload` and the `NodeId` alias in
`hive-host-sock::jobs` — shared vocabulary, still used by hivectl.
This commit is contained in:
atlas 2026-08-03 21:07:58 +02:00 • committed by mara
commit f707c60f90
10 changed files with 131 additions and 615 deletions
Download patch file Download diff file Expand all files Collapse all files

221
hive-c0re/src/job_queue/mod.rs
View file

@ -39,15 +39,14 @@ mod tests;
use std::sync::{Arc, Mutex};
use chrono::{DateTime, Utc};
use hive_host_sock::jobs::NodeView;
use hive_jobq::resources::ResourceTable;
use hive_jobq::scheduler::{Outcome, Scheduler};
use hive_jobq::{Dep, Graph, NodeId};
use hive_jobq::{Graph, NodeId};
use hive_jobq_wire::{GraphNode, GraphWire};
use tokio::sync::Notify;
pub use hive_jobq::TerminalState;
pub use model::{DagView, NodeKind, PermPayload, Source, State};
pub use model::{NodeKind, PermPayload, Source, State};
use resource::Resource;
/// A job under construction: `hive_jobq`'s builder over this queue's payload
@ -380,17 +379,6 @@ impl JobQueue {
hive_jobq_wire::state_rollup(inner.graph(), visible_roots(&inner))
}
/// Snapshot every live + retained DAG for `/api/state` + `RebuildQueueChanged`.
#[must_use]
pub fn snapshot(&self) -> Vec<DagView> {
let inner = self.lock();
let mut ids = visible_dags(&inner);
ids.sort_unstable_by_key(|c| c.get());
ids.into_iter()
.filter_map(|c| dag_view(&inner, c))
.collect()
}
/// One or more nodes plus their live subtrees, as generic wire nodes —
/// the `QueueNodes` polling surface behind `hivectl`'s wait/progress
/// loop. Sibling of [`Self::snapshot`] (which serves the same graph
@ -410,8 +398,8 @@ impl JobQueue {
/// the result rather than erroring the whole batch — some ids in a
/// batch may already be evicted while others are still live. Today
/// that only happens for a genuinely unknown id: nothing prunes the
/// graph yet (bounded-prune is a Stage-C follow-up, see
/// [`visible_dags`]), so a *completed* DAG's nodes keep riding here
/// graph yet (bounded-prune is a Stage-C follow-up; [`visible_roots`]
/// bounds the *view*, not the graph), so a completed group's nodes keep riding here
/// with a terminal `state` rather than disappearing — callers
/// watching for "done" should read the root's `state`, not absence.
#[must_use]
@ -431,191 +419,13 @@ fn find_node(sched: &Sched, id: u64) -> Option<NodeId> {
.find_map(|n| (n.id.get() == id).then_some(n.id))
}
/// Project a DAG into its wire [`DagView`]: a near-raw view of the
/// container's work nodes, with `Done` nodes excluded. Lifecycle
/// (`state` / `started_at` / `finished_at` / `error`) is read straight
/// off each `hive_jobq::Node`; the client derives the DAG label, roll-up
/// state, and DAG timestamps from the node set. Non-derivable per-node
/// payload (`approval_id`, meta `inputs`) rides the owning node. Returns
/// `None` when every work node is `Done` or `Skipped` — a fully-settled
/// DAG drops out of the snapshot entirely (a `Failed` one lingers until
/// aged out).
fn dag_view(sched: &Sched, container: NodeId) -> Option<DagView> {
// Read straight off the container's payload: the domain fields have a
// single home there, so an intermediate owned copy of them was a second
// type describing the same data rather than a grouping side-table.
//
// `created_at` is NOT among them — it comes off the container node itself
// below, where the graph stamps it for every node. Keeping a payload copy
// would record one instant in two places.
let node = sched.graph().node(container)?;
let NodeKind::Dag { source, reason } = &node.payload else {
return None;
};
let all: Vec<_> = sched.graph().descendants(container).collect();
// DAG-level timestamps are taken over *all* subtree nodes (including the
// `Done` ones excluded from the wire) — the client can't derive them
// from a `Done`-filtered node set, so the host computes them here.
let mut started: Vec<DateTime<Utc>> = Vec::new();
let mut finished: Vec<DateTime<Utc>> = Vec::new();
for node in &all {
if let Some(s) = node.started_at {
started.push(s);
}
if let Some(f) = node.finished_at {
finished.push(f);
}
}
// Decide which nodes ride the wire *before* projecting any of them: a
// `NodeView` costs a `build_logs` lookup, so building one for a node
// that's about to be dropped would be a query per finished step.
let shown = shown_on_wire(&all.iter().map(|n| n.state).collect::<Vec<_>>())?;
let mut nodes = Vec::new();
for node in shown.into_iter().map(|i| all[i]) {
let id = node.id;
let deps: Vec<u64> = node
.deps
.iter()
.filter_map(|d| match d {
Dep::Node { id, .. } => Some(id.get()),
Dep::Resource { .. } => None,
})
.collect();
// Non-derivable per-node payload rides the node that owns it. Every
// deploy phase carries the approval id, but only the subtree root
// projects it onto the wire — hanging the approval link off all of
// them would render the same card once per phase.
let approval_id = match &node.payload {
NodeKind::DeployWindow { approval_id, .. } => Some(*approval_id),
_ => None,
};
let inputs = match &node.payload {
NodeKind::MetaLock { inputs, .. } => inputs.clone(),
_ => Vec::new(),
};
// Looked up from the log row itself (`build_logs.node_id`), not a
// host-side map. One indexed query per node in the snapshot; the
// node set is bounded by `MAX_HISTORY_DAGS` and the store is a
// local sqlite file, so this is cheaper than the lock contention
// a second shared map would reintroduce.
let build_log_id = crate::build_logs::global().and_then(|h| h.id_for_node(id.get()));
// `node.parent` is the structural jobq parent. Top-level nodes
// have `parent == Some(container)` (direct children of the Dag
// container); those become `parent: None` on the wire since the
// container itself is not part of the work-node payload. Sub-nodes
// carry the id of their containing parent work-node.
let parent = node
.parent
.filter(|&p| p != container)
.map(hive_jobq::NodeId::get);
nodes.push(NodeView {
id: id.get(),
agent: node.payload.agent().to_owned(),
kind: node.payload.as_str().to_owned(),
deps,
state: node.state,
started_at: node.started_at,
finished_at: node.finished_at,
error: node.error.clone(),
approval_id,
inputs,
build_log_id,
parent,
});
}
let is_terminal = sched.graph().is_settled(container) == Some(true);
Some(DagView {
id: container.get(),
source: *source,
reason: reason.clone(),
created_at: node.created_at,
started_at: started.into_iter().min(),
finished_at: is_terminal.then(|| finished.into_iter().max()).flatten(),
nodes,
})
}
/// Which of a DAG's work nodes ride the wire, by index into `states` — or
/// `None` when the DAG has nothing left worth showing and drops out of the
/// snapshot entirely.
///
/// Two separate decisions, and conflating them pins every completed deploy in
/// the queue view forever:
/// - **`Done` drops off the wire.** A finished step isn't interesting.
/// `Skipped` stays: which branch a run *didn't* take is the readable half of
/// an outcome-branched DAG.
/// - **`Skipped` alone doesn't hold a DAG in the snapshot.** So "the node list
/// is non-empty" and "there's still something here worth showing" are
/// different questions, and only the second one may drop the DAG.
///
/// Takes states rather than projected nodes so the caller can skip the work of
/// projecting what it's about to discard, and so this is testable without a
/// graph — the states it keys on are ones only a run can produce.
fn shown_on_wire(states: &[State]) -> Option<Vec<usize>> {
let worth_showing = states
.iter()
.any(|s| !matches!(s, State::Done | State::Skipped));
if !worth_showing {
return None;
}
Some(
states
.iter()
.enumerate()
.filter(|(_, s)| !matches!(s, State::Done))
.map(|(i, _)| i)
.collect(),
)
}
/// When a DAG's work node finishes on `finished_at` — the max over its
/// subtree (read off the graph `Node`, as unix seconds), for the history
/// cap ordering.
fn dag_finished_at(sched: &Sched, container: NodeId) -> i64 {
sched
.graph()
.descendants(container)
.filter_map(|n| n.finished_at)
.map(|t| t.timestamp())
.max()
.unwrap_or(0)
}
/// Every DAG container node id in the graph.
fn containers(sched: &Sched) -> Vec<NodeId> {
sched
.graph()
.nodes()
.filter(|n| n.parent.is_none() && matches!(n.payload, NodeKind::Dag { .. }))
.map(|n| n.id)
.collect()
}
/// The **visible** DAG set for the snapshot: every live (non-terminal) DAG,
/// plus the newest [`MAX_HISTORY_DAGS`] terminal ones. Crate nodes for
/// evicted DAGs linger in the graph (bounded-prune is a Stage-C follow-up);
/// this filter is what bounds what the dashboard sees.
fn visible_dags(sched: &Sched) -> Vec<NodeId> {
let mut live: Vec<NodeId> = Vec::new();
let mut terminal: Vec<(NodeId, i64, u64)> = Vec::new();
for c in containers(sched) {
if sched.graph().is_settled(c) == Some(true) {
terminal.push((c, dag_finished_at(sched, c), c.get()));
} else {
live.push(c);
}
}
retain_history(live, terminal, MAX_HISTORY_DAGS)
}
/// The visible **group** set for [`Queue::graph_snapshot`]: every live group
/// root, plus the newest [`MAX_HISTORY_DAGS`] settled ones.
///
/// Same policy as [`visible_dags`], selected *structurally* — a root is a node
/// with no parent. The `DagView` path next door keys on `NodeKind::Dag`
/// instead, which is fine for a projection that already only means anything to
/// hive-c0re, but would make the generic endpoint depend on one node kind that
/// is itself slated for removal.
/// Selected *structurally* — a root is a node with no parent. The typed
/// projection this replaced keyed on `NodeKind::Dag` instead, which made the
/// visible set depend on one host node kind; nothing here knows what a node
/// means.
///
/// **This bound is load-bearing, not tidiness.** Nothing ever removes a node
/// from the graph (bounded pruning is a Stage-C follow-up), so serving
@ -636,10 +446,13 @@ fn visible_roots(sched: &Sched) -> Vec<NodeId> {
}
/// When a whole group last finished: the newest `finished_at` across the root
/// **and** its descendants. Unlike [`dag_finished_at`] the root itself counts,
/// because a generic group root can be an ordinary node with no children at
/// all — reading only descendants would date every such group to the epoch and
/// evict it first.
/// **and** its descendants.
///
/// The root itself counts, because a group root can be an ordinary node with
/// no children at all — reading only descendants would date every such group
/// to the epoch and evict it first. (The typed path this replaced read
/// descendants only, and could get away with it: its roots were always DAG
/// containers, which always have children.)
fn group_finished_at(sched: &Sched, root: NodeId) -> i64 {
sched
.graph()
@ -657,8 +470,8 @@ fn group_finished_at(sched: &Sched, root: NodeId) -> i64 {
.unwrap_or(0)
}
/// [`visible_dags`]'s policy, split from the graph it reads: keep every live
/// DAG, plus the newest `cap` terminal ones.
/// [`visible_roots`]'s policy, split from the graph it reads: keep every live
/// group, plus the newest `cap` terminal ones.
///
/// `terminal` rows are `(handle, finished_at, tiebreak)`. The tiebreak orders
/// DAGs that settled inside the same wall-clock second — which is *most* of