refactor(#2949): the build-log row carries its node id

`QueueInner` was `{ sched, node_rt }`, where `node_rt` held exactly one
datum per node: the `build_logs` row id. It existed because a `hive_jobq`
node payload is immutable after insert while the log row is created when
the build starts — so the link could not ride the node.

Invert it: the log row names its node (`build_logs.node_id`, one migration
in the existing `schema_versions` framework). Same single-home property,
in the direction the type system allows.

`QueueInner` is now just the scheduler. That is the point: the queue holds
no per-node side map, so nothing has to be locked alongside the graph.

Deleted as a consequence, each surfaced by dead-code analysis after the
edit above rather than predicted:

- `NodeRuntime`, `node_rt`, `set_build_log_id`, and `build_log_id_of`
  (which linear-scanned the map to match a wire `u64` against opaque
  `NodeId`s). The lookup is an indexed query now.
- `struct Ctx`, entirely. It carried `coord` + `dag_id` + `node_id` into
  the executors so the build-log callback could reach the queue; without
  the callback, `coord`/`dag_id` were never read and `node_id` was already
  on the `Claim` both executors receive.
- `QueueInner::node_running`, which existed only for `set_build_log_id`'s
  "only while running" guard.
- The `Fn(i64)` callbacks on `prebuild_toplevel` / `swap_update` /
  `priv_run_inner`, replaced by a `node_id: Option<u64>` passed down. The
  id travels one way now instead of being registered back.

`meta.rs`'s `nix_logged` passes `None` deliberately: its callers reach it
from outside the queue as well as inside, and nothing reads the link for
them yet.

`id_for_node` takes `MAX(id)` rather than assuming uniqueness — a retried
node opens a second row and the panel wants the current attempt. The test
moved to where the behaviour lives and covers that, plus survival across
completion and non-collision with node-less rows.
This commit is contained in:
atlas 2026-08-02 18:26:32 +02:00 committed by mara
commit 77cc7bea6b
7 changed files with 150 additions and 162 deletions

View file

@ -36,7 +36,6 @@ pub mod templates;
#[cfg(test)]
mod tests;
use std::collections::HashMap;
use std::sync::Mutex;
use chrono::{DateTime, Utc};
@ -101,15 +100,6 @@ pub struct Claim {
pub agent: String,
}
/// Per-node runtime metadata the crate graph doesn't carry. Lifecycle
/// (`started_at` / `finished_at` / `error`) lives on the `hive_jobq::Node`
/// itself now, so only the build-log row link remains host-side (the
/// client fetches the log by node id).
#[derive(Debug, Default, Clone)]
struct NodeRuntime {
build_log_id: Option<i64>,
}
/// An owned read-view of a DAG container's carried metadata ([`NodeKind::Dag`]).
/// Derived on read from the container node — the data has a single home (the
/// node payload); this is not a stored side-table.
@ -119,18 +109,21 @@ struct DagMeta {
created_at: DateTime<Utc>,
}
/// The mutable queue state behind the mutex: the crate scheduler plus the
/// per-node runtime metadata the graph can't carry. A **DAG is a single
/// container node** ([`NodeKind::Dag`], `parent = None`) whose subtree is the
/// DAG's work — so the container's `NodeId` is the DAG id, its rolled-up state
/// is the DAG state, and there are no grouping side-tables: membership + meta
/// are graph queries ([`QueueInner::container`] / [`QueueInner::dag_meta`] +
/// the `hive_jobq::Graph` accessors). One shared crate [`Graph`] holds every DAG.
/// The mutable queue state behind the mutex: **just the crate scheduler**.
/// A **DAG is a single container node** ([`NodeKind::Dag`], `parent = None`)
/// whose subtree is the DAG's work — so the container's `NodeId` is the DAG id,
/// its rolled-up state is the DAG state, and there are no grouping side-tables:
/// membership + meta are graph queries ([`QueueInner::container`] /
/// [`QueueInner::dag_meta`] + the `hive_jobq::Graph` accessors). One shared
/// crate [`Graph`] holds every DAG.
///
/// There is deliberately **no per-node side map** any more. The last one held
/// the `build_logs` row id; that link now lives on the log row itself
/// (`build_logs.node_id`), so it survives a restart and needs no lock held
/// alongside the scheduler's — which is what lets the scheduler's own lock be
/// the only one the run loop takes.
struct QueueInner {
sched: Scheduler<NodeKind, Resource>,
/// Per-node runtime metadata (the build-log id) — mutable after
/// insert, so it can't ride the immutable node payload.
node_rt: HashMap<NodeId, NodeRuntime>,
}
/// The queue. Lives on `Coordinator` (one per hive-c0re process); a single
@ -208,7 +201,6 @@ impl JobQueue {
Self {
inner: Mutex::new(QueueInner {
sched: Scheduler::new(Graph::new(), table),
node_rt: HashMap::new(),
}),
notify: Notify::new(),
}
@ -244,7 +236,6 @@ impl JobQueue {
None,
)
.map_err(|e| anyhow::anyhow!("job_queue: container insert failed: {e}"))?;
inner.node_rt.insert(container, NodeRuntime::default());
insert_group(&mut inner, spec.declare, Some(container))?;
// Settle the container's own (no-op) logic immediately so it parks in
// `Finishing` and its children become runnable — it never needs claiming
@ -374,33 +365,6 @@ impl JobQueue {
true
}
/// Link a `build_logs` row to a specific `Running` node.
pub fn set_build_log_id(&self, dag_id: u64, node_id: NodeId, log_id: i64) -> bool {
let mut inner = self.lock();
if inner.sched.graph().root_of(node_id).map(NodeId::get) != Some(dag_id)
|| !inner.node_running(node_id)
{
return false;
}
inner.node_rt.entry(node_id).or_default().build_log_id = Some(log_id);
true
}
/// The `build_logs` row id linked to the wire node id `node_id`, if any —
/// the lookup behind the `GET /api/build-log/<node_id>` query endpoint (the
/// client fetches a node's captured build output on demand rather than
/// receiving it inline). Takes the raw wire `u64` (the endpoint's path
/// param); `node_rt` is keyed by the opaque `NodeId`, so this scans for the
/// matching id — the map is small (live + recently-terminal nodes).
#[must_use]
pub fn build_log_id_of(&self, node_id: u64) -> Option<i64> {
self.lock()
.node_rt
.iter()
.find(|(nid, _)| nid.get() == node_id)
.and_then(|(_, rt)| rt.build_log_id)
}
/// The first failed node's error in `dag_id`, if any has failed yet.
///
/// Unlike the roll-up summary this is readable *mid-flight*, which is the
@ -498,14 +462,6 @@ impl JobQueue {
}
impl QueueInner {
/// Whether `id` is a `Running` node.
fn node_running(&self, id: NodeId) -> bool {
self.sched
.graph()
.node(id)
.is_some_and(|n| n.state == State::Running)
}
/// The container node of `dag_id` — the `NodeKind::Dag` root whose id equals
/// `dag_id`. `NodeId` is un-fabricable from a raw `u64`, so this is a search.
fn container(&self, dag_id: u64) -> Option<NodeId> {
@ -593,7 +549,12 @@ impl QueueInner {
NodeKind::MetaLock { inputs, .. } => inputs.clone(),
_ => Vec::new(),
};
let build_log_id = self.node_rt.get(&id).and_then(|r| r.build_log_id);
// Looked up from the log row itself (`build_logs.node_id`), not a
// host-side map. One indexed query per node in the snapshot; the
// node set is bounded by `MAX_HISTORY_DAGS` and the store is a
// local sqlite file, so this is cheaper than the lock contention
// a second shared map would reintroduce.
let build_log_id = crate::build_logs::global().and_then(|h| h.id_for_node(id.get()));
// `node.parent` is the structural jobq parent. Top-level nodes
// have `parent == Some(container)` (direct children of the Dag
// container); those become `parent: None` on the wire since the