refactor(#2949): the build-log row carries its node id
`QueueInner` was `{ sched, node_rt }`, where `node_rt` held exactly one
datum per node: the `build_logs` row id. It existed because a `hive_jobq`
node payload is immutable after insert while the log row is created when
the build starts — so the link could not ride the node.
Invert it: the log row names its node (`build_logs.node_id`, one migration
in the existing `schema_versions` framework). Same single-home property,
in the direction the type system allows.
`QueueInner` is now just the scheduler. That is the point: the queue holds
no per-node side map, so nothing has to be locked alongside the graph.
Deleted as a consequence, each surfaced by dead-code analysis after the
edit above rather than predicted:
- `NodeRuntime`, `node_rt`, `set_build_log_id`, and `build_log_id_of`
(which linear-scanned the map to match a wire `u64` against opaque
`NodeId`s). The lookup is an indexed query now.
- `struct Ctx`, entirely. It carried `coord` + `dag_id` + `node_id` into
the executors so the build-log callback could reach the queue; without
the callback, `coord`/`dag_id` were never read and `node_id` was already
on the `Claim` both executors receive.
- `QueueInner::node_running`, which existed only for `set_build_log_id`'s
"only while running" guard.
- The `Fn(i64)` callbacks on `prebuild_toplevel` / `swap_update` /
`priv_run_inner`, replaced by a `node_id: Option<u64>` passed down. The
id travels one way now instead of being registered back.
`meta.rs`'s `nix_logged` passes `None` deliberately: its callers reach it
from outside the queue as well as inside, and nothing reads the link for
them yet.
`id_for_node` takes `MAX(id)` rather than assuming uniqueness — a retried
node opens a second row and the panel wants the current attempt. The test
moved to where the behaviour lives and covers that, plus survival across
completion and non-collision with node-less rows.
This commit is contained in:
parent
82ef06f445
commit
77cc7bea6b
7 changed files with 150 additions and 162 deletions
|
|
@ -36,7 +36,6 @@ pub mod templates;
|
|||
#[cfg(test)]
|
||||
mod tests;
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::sync::Mutex;
|
||||
|
||||
use chrono::{DateTime, Utc};
|
||||
|
|
@ -101,15 +100,6 @@ pub struct Claim {
|
|||
pub agent: String,
|
||||
}
|
||||
|
||||
/// Per-node runtime metadata the crate graph doesn't carry. Lifecycle
|
||||
/// (`started_at` / `finished_at` / `error`) lives on the `hive_jobq::Node`
|
||||
/// itself now, so only the build-log row link remains host-side (the
|
||||
/// client fetches the log by node id).
|
||||
#[derive(Debug, Default, Clone)]
|
||||
struct NodeRuntime {
|
||||
build_log_id: Option<i64>,
|
||||
}
|
||||
|
||||
/// An owned read-view of a DAG container's carried metadata ([`NodeKind::Dag`]).
|
||||
/// Derived on read from the container node — the data has a single home (the
|
||||
/// node payload); this is not a stored side-table.
|
||||
|
|
@ -119,18 +109,21 @@ struct DagMeta {
|
|||
created_at: DateTime<Utc>,
|
||||
}
|
||||
|
||||
/// The mutable queue state behind the mutex: the crate scheduler plus the
|
||||
/// per-node runtime metadata the graph can't carry. A **DAG is a single
|
||||
/// container node** ([`NodeKind::Dag`], `parent = None`) whose subtree is the
|
||||
/// DAG's work — so the container's `NodeId` is the DAG id, its rolled-up state
|
||||
/// is the DAG state, and there are no grouping side-tables: membership + meta
|
||||
/// are graph queries ([`QueueInner::container`] / [`QueueInner::dag_meta`] +
|
||||
/// the `hive_jobq::Graph` accessors). One shared crate [`Graph`] holds every DAG.
|
||||
/// The mutable queue state behind the mutex: **just the crate scheduler**.
|
||||
/// A **DAG is a single container node** ([`NodeKind::Dag`], `parent = None`)
|
||||
/// whose subtree is the DAG's work — so the container's `NodeId` is the DAG id,
|
||||
/// its rolled-up state is the DAG state, and there are no grouping side-tables:
|
||||
/// membership + meta are graph queries ([`QueueInner::container`] /
|
||||
/// [`QueueInner::dag_meta`] + the `hive_jobq::Graph` accessors). One shared
|
||||
/// crate [`Graph`] holds every DAG.
|
||||
///
|
||||
/// There is deliberately **no per-node side map** any more. The last one held
|
||||
/// the `build_logs` row id; that link now lives on the log row itself
|
||||
/// (`build_logs.node_id`), so it survives a restart and needs no lock held
|
||||
/// alongside the scheduler's — which is what lets the scheduler's own lock be
|
||||
/// the only one the run loop takes.
|
||||
struct QueueInner {
|
||||
sched: Scheduler<NodeKind, Resource>,
|
||||
/// Per-node runtime metadata (the build-log id) — mutable after
|
||||
/// insert, so it can't ride the immutable node payload.
|
||||
node_rt: HashMap<NodeId, NodeRuntime>,
|
||||
}
|
||||
|
||||
/// The queue. Lives on `Coordinator` (one per hive-c0re process); a single
|
||||
|
|
@ -208,7 +201,6 @@ impl JobQueue {
|
|||
Self {
|
||||
inner: Mutex::new(QueueInner {
|
||||
sched: Scheduler::new(Graph::new(), table),
|
||||
node_rt: HashMap::new(),
|
||||
}),
|
||||
notify: Notify::new(),
|
||||
}
|
||||
|
|
@ -244,7 +236,6 @@ impl JobQueue {
|
|||
None,
|
||||
)
|
||||
.map_err(|e| anyhow::anyhow!("job_queue: container insert failed: {e}"))?;
|
||||
inner.node_rt.insert(container, NodeRuntime::default());
|
||||
insert_group(&mut inner, spec.declare, Some(container))?;
|
||||
// Settle the container's own (no-op) logic immediately so it parks in
|
||||
// `Finishing` and its children become runnable — it never needs claiming
|
||||
|
|
@ -374,33 +365,6 @@ impl JobQueue {
|
|||
true
|
||||
}
|
||||
|
||||
/// Link a `build_logs` row to a specific `Running` node.
|
||||
pub fn set_build_log_id(&self, dag_id: u64, node_id: NodeId, log_id: i64) -> bool {
|
||||
let mut inner = self.lock();
|
||||
if inner.sched.graph().root_of(node_id).map(NodeId::get) != Some(dag_id)
|
||||
|| !inner.node_running(node_id)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
inner.node_rt.entry(node_id).or_default().build_log_id = Some(log_id);
|
||||
true
|
||||
}
|
||||
|
||||
/// The `build_logs` row id linked to the wire node id `node_id`, if any —
|
||||
/// the lookup behind the `GET /api/build-log/<node_id>` query endpoint (the
|
||||
/// client fetches a node's captured build output on demand rather than
|
||||
/// receiving it inline). Takes the raw wire `u64` (the endpoint's path
|
||||
/// param); `node_rt` is keyed by the opaque `NodeId`, so this scans for the
|
||||
/// matching id — the map is small (live + recently-terminal nodes).
|
||||
#[must_use]
|
||||
pub fn build_log_id_of(&self, node_id: u64) -> Option<i64> {
|
||||
self.lock()
|
||||
.node_rt
|
||||
.iter()
|
||||
.find(|(nid, _)| nid.get() == node_id)
|
||||
.and_then(|(_, rt)| rt.build_log_id)
|
||||
}
|
||||
|
||||
/// The first failed node's error in `dag_id`, if any has failed yet.
|
||||
///
|
||||
/// Unlike the roll-up summary this is readable *mid-flight*, which is the
|
||||
|
|
@ -498,14 +462,6 @@ impl JobQueue {
|
|||
}
|
||||
|
||||
impl QueueInner {
|
||||
/// Whether `id` is a `Running` node.
|
||||
fn node_running(&self, id: NodeId) -> bool {
|
||||
self.sched
|
||||
.graph()
|
||||
.node(id)
|
||||
.is_some_and(|n| n.state == State::Running)
|
||||
}
|
||||
|
||||
/// The container node of `dag_id` — the `NodeKind::Dag` root whose id equals
|
||||
/// `dag_id`. `NodeId` is un-fabricable from a raw `u64`, so this is a search.
|
||||
fn container(&self, dag_id: u64) -> Option<NodeId> {
|
||||
|
|
@ -593,7 +549,12 @@ impl QueueInner {
|
|||
NodeKind::MetaLock { inputs, .. } => inputs.clone(),
|
||||
_ => Vec::new(),
|
||||
};
|
||||
let build_log_id = self.node_rt.get(&id).and_then(|r| r.build_log_id);
|
||||
// Looked up from the log row itself (`build_logs.node_id`), not a
|
||||
// host-side map. One indexed query per node in the snapshot; the
|
||||
// node set is bounded by `MAX_HISTORY_DAGS` and the store is a
|
||||
// local sqlite file, so this is cheaper than the lock contention
|
||||
// a second shared map would reintroduce.
|
||||
let build_log_id = crate::build_logs::global().and_then(|h| h.id_for_node(id.get()));
|
||||
// `node.parent` is the structural jobq parent. Top-level nodes
|
||||
// have `parent == Some(container)` (direct children of the Dag
|
||||
// container); those become `parent: None` on the wire since the
|
||||
|
|
|
|||
Loading…
Reference in a new issue