job_queue: filter jobq graph snapshot by per-node state
graph_snapshot previously filtered which whole roots got projected based on the root node's own state, so a group root that was still Running but had already-Done internal steps couldn't be filtered down to just its live nodes, and a filtered-out root hid its entire subtree even when a descendant still matched. Apply the states filter after GraphWire::wire_snapshot instead, over every node in the flattened tree, not just roots. The jobq-graph client already handles an orphaned node (parent filtered out) by promoting it to a rendered root, so this is safe on the client side with no changes needed there. Fixes hyperhive#3210
This commit is contained in:
parent
0b6b3b755d
commit
861a1f8f26
2 changed files with 98 additions and 37 deletions
|
|
@ -333,25 +333,32 @@ impl JobQueue {
|
||||||
/// visible rather than vanishing from the payload, which is what makes a
|
/// visible rather than vanishing from the payload, which is what makes a
|
||||||
/// fast rebuild render as a single node).
|
/// fast rebuild render as a single node).
|
||||||
///
|
///
|
||||||
/// `states`, when given, keeps only the **root** groups whose own state
|
/// `states`, when given, keeps every **individual node** (root or
|
||||||
/// is named — a root's state is already its subtree's rolled-up answer
|
/// descendant) whose own state is named — not just whole root groups.
|
||||||
/// (see `hive_jobq_wire`'s doc), so filtering the root filters the whole
|
/// A still-live group (root not yet terminal) can otherwise hold any
|
||||||
/// group without needing to also filter descendants. `None` (or the
|
/// number of already-finished steps inside it; filtering only at the
|
||||||
/// full state set) is the unfiltered call, matching prior behaviour.
|
/// root leaves every one of those visible regardless of the ask, which
|
||||||
/// A slice rather than a set: `State` (`hive_jobq`'s) derives `Eq` but
|
/// is exactly the clutter a state filter exists to remove. `None` (or
|
||||||
/// not `Hash`, and the whole vocabulary is 7 variants — a linear
|
/// the full state set) is the unfiltered call, matching prior
|
||||||
/// membership check per root costs nothing at that size, so there is no
|
/// behaviour. A slice rather than a set: `State` derives `Eq` but not
|
||||||
/// reason to widen `hive_jobq::State`'s derive list just for this.
|
/// `Hash`, and the vocabulary is 7 variants — a linear check per node
|
||||||
|
/// costs nothing at that size.
|
||||||
|
///
|
||||||
|
/// A node whose *parent* got filtered out still rides with its original
|
||||||
|
/// `parent` id — `<hive-jobq-graph>` (the one consumer) already treats
|
||||||
|
/// an unresolvable parent as a new root (`buildTree`'s fallback), so a
|
||||||
|
/// filtered-out ancestor surfaces a still-matching descendant one level
|
||||||
|
/// higher rather than hiding or orphaning it.
|
||||||
///
|
///
|
||||||
/// The projection itself is [`hive_jobq_wire`]'s; all this layer supplies
|
/// The projection itself is [`hive_jobq_wire`]'s; all this layer supplies
|
||||||
/// is *which* groups to show — see [`visible_roots`] for why the graph
|
/// is *which* nodes to show — see [`visible_roots`] for why the graph
|
||||||
/// can't decide that for itself.
|
/// can't decide the root-visibility half of that for itself.
|
||||||
#[must_use]
|
#[must_use]
|
||||||
pub fn graph_snapshot(&self, states: Option<&[State]>) -> Vec<GraphNode> {
|
pub fn graph_snapshot(&self, states: Option<&[State]>) -> Vec<GraphNode> {
|
||||||
let inner = self.lock();
|
let inner = self.lock();
|
||||||
let roots = visible_roots(&inner);
|
let roots = visible_roots(&inner);
|
||||||
let roots = filter_roots_by_state(&inner, roots, states);
|
let nodes = inner.graph().wire_snapshot(roots);
|
||||||
inner.graph().wire_snapshot(roots)
|
filter_nodes_by_state(nodes, states)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Per-state counts over the **same** groups [`Queue::graph_snapshot`]
|
/// Per-state counts over the **same** groups [`Queue::graph_snapshot`]
|
||||||
|
|
@ -406,30 +413,23 @@ fn find_node(sched: &Sched, id: u64) -> Option<NodeId> {
|
||||||
.find_map(|n| (n.id.get() == id).then_some(n.id))
|
.find_map(|n| (n.id.get() == id).then_some(n.id))
|
||||||
}
|
}
|
||||||
|
|
||||||
/// [`Queue::graph_snapshot`]'s `states` ask, applied to an already-bounded
|
/// [`Queue::graph_snapshot`]'s `states` ask, applied to the already-
|
||||||
/// root set: keeps only the roots whose own `state` is named.
|
/// projected node list: keeps every node — root or descendant — whose own
|
||||||
|
/// `state` is named.
|
||||||
///
|
///
|
||||||
/// Applied *after* [`visible_roots`] rather than folded into it — the
|
/// Applied *after* [`GraphWire::wire_snapshot`] rather than as a root
|
||||||
/// history cap bounds how much settled work is retained at all, which is a
|
/// pre-filter — narrowing which roots are visible at all is
|
||||||
/// different question from which of the retained (and live) groups the
|
/// [`visible_roots`]'s job (a different question: how much settled work is
|
||||||
/// caller wants shown right now. `None` (or an unrecognised/absent query)
|
/// retained, full stop); this is "of what's retained and live, which
|
||||||
/// is the identity filter.
|
/// individual nodes does the caller want shown right now." `None` (or an
|
||||||
fn filter_roots_by_state(
|
/// unrecognised/absent query) is the identity filter.
|
||||||
sched: &Sched,
|
fn filter_nodes_by_state(nodes: Vec<GraphNode>, states: Option<&[State]>) -> Vec<GraphNode> {
|
||||||
roots: Vec<NodeId>,
|
|
||||||
states: Option<&[State]>,
|
|
||||||
) -> Vec<NodeId> {
|
|
||||||
let Some(states) = states else {
|
let Some(states) = states else {
|
||||||
return roots;
|
return nodes;
|
||||||
};
|
};
|
||||||
roots
|
nodes
|
||||||
.into_iter()
|
.into_iter()
|
||||||
.filter(|&id| {
|
.filter(|n| states.contains(&n.state))
|
||||||
sched
|
|
||||||
.graph()
|
|
||||||
.node(id)
|
|
||||||
.is_some_and(|n| states.contains(&n.state))
|
|
||||||
})
|
|
||||||
.collect()
|
.collect()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1471,11 +1471,18 @@ fn history_retains_live_dags_and_the_newest_terminals() {
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// `graph_snapshot`'s `states` ask filters by **root** state — a whole group
|
/// `graph_snapshot`'s `states` ask, exercised through the real scheduler:
|
||||||
/// is kept or dropped together, never split mid-subtree. `None` is the
|
/// a queued (never-run) DAG cancels as one unit, so every node in it shares
|
||||||
/// identity filter (every visible root, current default behaviour).
|
/// one state and this only proves root-level inclusion/exclusion — the
|
||||||
|
/// per-node case (a live group holding a mix of finished and unfinished
|
||||||
|
/// steps) isn't expressible without driving the scheduler, which this test
|
||||||
|
/// module deliberately can't do (see the module doc comment). See
|
||||||
|
/// `filter_nodes_by_state_keeps_matching_nodes_from_a_mixed_state_tree`
|
||||||
|
/// below for that half, exercised directly against hand-built `GraphNode`s.
|
||||||
|
/// `None` is the identity filter (every visible node, current default
|
||||||
|
/// behaviour).
|
||||||
#[test]
|
#[test]
|
||||||
fn graph_snapshot_states_filters_whole_groups_by_root_state() {
|
fn graph_snapshot_states_filters_by_node_state() {
|
||||||
let q = JobQueue::new(2);
|
let q = JobQueue::new(2);
|
||||||
let roots_a = insert_named(&q, |builder| restart_online(builder, &["agent-a"], false));
|
let roots_a = insert_named(&q, |builder| restart_online(builder, &["agent-a"], false));
|
||||||
let roots_b = insert_named(&q, |builder| restart_online(builder, &["agent-b"], false));
|
let roots_b = insert_named(&q, |builder| restart_online(builder, &["agent-b"], false));
|
||||||
|
|
@ -1520,6 +1527,60 @@ fn graph_snapshot_states_filters_whole_groups_by_root_state() {
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// One hand-built node, bypassing the scheduler entirely — `filter_nodes_by_state`
|
||||||
|
/// is a pure `Vec<GraphNode> -> Vec<GraphNode>` transform, so this exercises it
|
||||||
|
/// directly rather than trying (and failing, per this module's own rule) to
|
||||||
|
/// drive a live group into a genuinely mixed state through the real queue.
|
||||||
|
fn node(id: u64, parent: Option<u64>, state: State) -> GraphNode {
|
||||||
|
GraphNode {
|
||||||
|
id,
|
||||||
|
parent,
|
||||||
|
state,
|
||||||
|
deps: Vec::new(),
|
||||||
|
created_at: Utc::now(),
|
||||||
|
started_at: None,
|
||||||
|
finished_at: None,
|
||||||
|
error: None,
|
||||||
|
payload: hive_jobq_wire::NodePayload {
|
||||||
|
label: id.to_string(),
|
||||||
|
data: serde_json::Value::Null,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The case `graph_snapshot_states_filters_by_node_state` above can't reach:
|
||||||
|
/// a still-live group (root `Running`) holding a mix of already-`Done` and
|
||||||
|
/// still-`Pending` steps. Filtering out `Done` must drop exactly the `Done`
|
||||||
|
/// node and nothing else — the root and the `Pending` sibling both stay,
|
||||||
|
/// even though the root itself isn't in the requested state set.
|
||||||
|
#[test]
|
||||||
|
fn filter_nodes_by_state_keeps_matching_nodes_from_a_mixed_state_tree() {
|
||||||
|
let nodes = vec![
|
||||||
|
node(1, None, State::Running), // root: whole group still live
|
||||||
|
node(2, Some(1), State::Done), // finished step, should be hidden
|
||||||
|
node(3, Some(1), State::Pending), // not-yet-run step, should stay
|
||||||
|
];
|
||||||
|
|
||||||
|
let mut kept: Vec<u64> =
|
||||||
|
filter_nodes_by_state(nodes.clone(), Some(&[State::Running, State::Pending]))
|
||||||
|
.into_iter()
|
||||||
|
.map(|n| n.id)
|
||||||
|
.collect();
|
||||||
|
kept.sort_unstable();
|
||||||
|
assert_eq!(
|
||||||
|
kept,
|
||||||
|
vec![1, 3],
|
||||||
|
"Done is filtered out even though its still-live parent (root) isn't"
|
||||||
|
);
|
||||||
|
|
||||||
|
let mut unfiltered: Vec<u64> = filter_nodes_by_state(nodes, None)
|
||||||
|
.into_iter()
|
||||||
|
.map(|n| n.id)
|
||||||
|
.collect();
|
||||||
|
unfiltered.sort_unstable();
|
||||||
|
assert_eq!(unfiltered, vec![1, 2, 3], "None is the identity filter");
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn error_truncation_cuts_on_a_char_boundary() {
|
fn error_truncation_cuts_on_a_char_boundary() {
|
||||||
// `truncate_error` is a pure `&str -> String`. This used to submit a DAG,
|
// `truncate_error` is a pure `&str -> String`. This used to submit a DAG,
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue