Compare commits

..
3 changed files with 20 additions and 44 deletions

View file

@ -53,12 +53,9 @@ hand-maintained per-file tree drifts out of sync with the code.
streamable-http listener, `hive-mcp-http` systemd unit) + its claude streamable-http listener, `hive-mcp-http` systemd unit) + its claude
launch-config layer (tool-group/capability → `--allowedTools`, launch-config layer (tool-group/capability → `--allowedTools`,
`--mcp-config` render). `--mcp-config` render).
- **`hive-jobq/`** — job-DAG scheduler, extracted from hive-c0re's - **`hive-jobq/`** — persistent job-DAG scheduler, extracted from
in-tree `job_queue` as a domain-agnostic library. **Runtime-only — hive-c0re's in-tree `job_queue` as a domain-agnostic library. One
nothing writes the graph to disk**; hive-c0re starts empty each boot persistent graph for the whole system (not a DAG per job); enqueuing
and re-derives desired state via the reconcile sweep, so node ids and
timestamps are stable within a run, not across restarts. One
shared graph for the whole system (not a DAG per job); enqueuing
inserts a self-contained sub-DAG and returns the ids of the nodes the job inserts a self-contained sub-DAG and returns the ids of the nodes the job
asked for, in the order it named them. Generic over asked for, in the order it named them. Generic over
the node payload `N` and the resource name `R`; resource deps are named the node payload `N` and the resource name `R`; resource deps are named

View file

@ -1,7 +1,7 @@
# hive-jobq # hive-jobq
A job-DAG scheduler, extracted from hive-c0re's in-tree `job_queue` A persistent job-DAG scheduler, extracted from hive-c0re's in-tree `job_queue`
as a **domain-agnostic** library. It schedules a single in-memory graph of as a **domain-agnostic** library. It schedules a single persistent graph of
nodes over named resources; it knows nothing about containers, rebuilds, or any nodes over named resources; it knows nothing about containers, rebuilds, or any
hyperhive type — the node payload `N` and resource name `R` are both generic, so hyperhive type — the node payload `N` and resource name `R` are both generic, so
the caller supplies its own domain. the caller supplies its own domain.
@ -15,13 +15,7 @@ kinds, wires deps, and supplies a runner; the scheduler decides what can start.
## Model ## Model
**Runtime-only: nothing writes this graph to disk.** The serde impls exist for One **persistent graph** for the whole system, not a DAG per job. Enqueuing
the wire projection (`hive-jobq-wire`) and a possible future store; no caller
loads one, so ids and timestamps are stable within a run, not across restarts.
hive-c0re constructs an empty graph every boot and re-derives desired state with
its reconcile sweep.
One **shared graph** for the whole system, not a DAG per job. Enqueuing
inserts a self-contained sub-DAG and returns the ids of the nodes the job inserts a self-contained sub-DAG and returns the ids of the nodes the job
*asked* for, in the order it named them; the scheduler runs a continuous loop, *asked* for, in the order it named them; the scheduler runs a continuous loop,
starting every node whose deps are satisfied: starting every node whose deps are satisfied:
@ -41,20 +35,18 @@ for the acquiring node *plus its whole parent subtree*, and a descendant needing
a resource an ancestor already holds re-uses that grant (a re-entrant borrow, a resource an ancestor already holds re-uses that grant (a re-entrant borrow,
one branch at a time) rather than taking a fresh unit. one branch at a time) rather than taking a fresh unit.
A `NodeId` is opaque, stable and monotonic **within a run** — a fresh process A `NodeId` is opaque, stable, and monotonic (safe to persist). The scheduler is
mints ids from zero, so an id stored outside it is a historical record, not a
handle that will resolve later. The scheduler is
single-threaded — it owns the resource table and mutates it directly. single-threaded — it owns the resource table and mutates it directly.
## Shape ## Shape
- **`Graph<N, R>`** — the in-memory node store. `insert` mints ids and - **`Graph<N, R>`** — the persistent node store. `insert` mints ids and
validates dep/parent references; `set_state` is the single state-transition validates dep/parent references; `set_state` is the single state-transition
choke point (and where each node's lifecycle timestamps — choke point (and where each node's lifecycle timestamps —
`started_at` / `finished_at`, `DateTime<Utc>` — are stamped). `started_at` / `finished_at`, `DateTime<Utc>` — are stamped).
- **`Node<N, R>`** — `{ id, parent, payload, deps, state, started_at, - **`Node<N, R>`** — `{ id, parent, payload, deps, state, started_at,
finished_at, error }`. All fields public; derives serde for the wire finished_at, error }`. All fields public; derives serde for persistence + the
projection (and so a store could be added — nothing calls one today). wire.
- **`Scheduler<N, R>`** — drives the graph: `settle()` starts every ready node - **`Scheduler<N, R>`** — drives the graph: `settle()` starts every ready node
(acquiring resources atomically), `complete(id, outcome)` reports a finished (acquiring resources atomically), `complete(id, outcome)` reports a finished
node's result and rolls terminality up the parent chain, releasing grants once node's result and rolls terminality up the parent chain, releasing grants once

View file

@ -1,12 +1,9 @@
//! `hive-jobq` — a job-DAG scheduler, extracted from hive-c0re's in-tree //! `hive-jobq` — a persistent job-DAG scheduler, extracted from hive-c0re's
//! `job_queue` as a domain-agnostic library. //! in-tree `job_queue` as a domain-agnostic library.
//!
//! **Runtime-only: nothing writes this graph to disk** — ids and timestamps are
//! stable within a run, not across restarts. See [`Graph`] and [`NodeId`].
//! //!
//! # Model (v2) //! # Model (v2)
//! //!
//! One **shared graph** for the whole system, not a DAG per job. Enqueuing //! One **persistent graph** for the whole system, not a DAG per job. Enqueuing
//! inserts a self-contained sub-DAG of nodes and returns their ids; the //! inserts a self-contained sub-DAG of nodes and returns their ids; the
//! scheduler runs a continuous loop, starting every node whose [`Dep`]s are //! scheduler runs a continuous loop, starting every node whose [`Dep`]s are
//! satisfied: //! satisfied:
@ -21,7 +18,7 @@
//! A node carries two independent axes: its [`Dep`]s (ordering + resource //! A node carries two independent axes: its [`Dep`]s (ordering + resource
//! needs) and its [`Node::parent`] (structural grouping) — the parent chain, //! needs) and its [`Node::parent`] (structural grouping) — the parent chain,
//! not the [`Dep::Node`] edges, is what the [`scheduler`] consults for resource //! not the [`Dep::Node`] edges, is what the [`scheduler`] consults for resource
//! re-entrancy. A [`NodeId`] is opaque, stable and monotonic within a run. The //! re-entrancy. A [`NodeId`] is opaque, stable, and monotonic (persisted). The
//! payload `N` is generic so the library stays container-agnostic. //! payload `N` is generic so the library stays container-agnostic.
//! //!
//! A resource unit is held for the acquiring node + its whole [`Node::parent`] //! A resource unit is held for the acquiring node + its whole [`Node::parent`]
@ -39,15 +36,12 @@ use chrono::{DateTime, Utc};
/// Opaque, stable, monotonic node identifier. /// Opaque, stable, monotonic node identifier.
/// ///
/// Assigned by the [`Graph`] on insert, so it is stable for the lifetime of /// Assigned by the [`Graph`] on insert and persisted, so it is stable across
/// that graph. **Not stable across restarts** — nothing persists the graph, so /// restarts.
/// a fresh process mints ids from zero again (see the module doc). Anything
/// storing an id outside the process is keeping a historical record, not a
/// handle it can resolve later.
/// ///
/// The inner field is crate-private: an id can only originate from the graph's /// The inner field is crate-private: an id can only originate from the graph's
/// monotonic counter (or deserializing a graph that was serialized from one), /// monotonic counter (or deserialization of a persisted graph), never be
/// never be fabricated by a caller — that is what makes it opaque. /// fabricated by a caller — that is what makes it opaque.
#[derive( #[derive(
Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, serde::Serialize, serde::Deserialize, Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, serde::Serialize, serde::Deserialize,
)] )]
@ -357,21 +351,14 @@ pub enum GraphError {
/// so the next minted id would collide with one already in the graph. /// so the next minted id would collide with one already in the graph.
#[error("next_id {next_id} must exceed the largest existing node id {max_id}")] #[error("next_id {next_id} must exceed the largest existing node id {max_id}")]
NextIdTooSmall { NextIdTooSmall {
/// The counter value carried by the deserialized graph. /// The persisted counter value.
next_id: u64, next_id: u64,
/// The largest id already present. /// The largest id already present.
max_id: u64, max_id: u64,
}, },
} }
/// The single in-memory graph of all nodes. /// The single persistent graph of all nodes.
///
/// **Not persisted.** The `Serialize`/`Deserialize` impls exist so a graph can
/// be projected onto a wire (`hive-jobq-wire`) and so a store *could* be added,
/// but no caller writes or loads one: hive-c0re constructs an empty graph on
/// every boot and re-derives desired state with its reconcile sweep (see
/// `hive-c0re/src/job_queue/mod.rs`). A derive is a capability, not a promise
/// that something uses it.
/// ///
/// New jobs are inserted as sub-DAGs of nodes; the scheduler walks this graph /// New jobs are inserted as sub-DAGs of nodes; the scheduler walks this graph
/// filling open slots. Completed nodes are retained (no pruning in v1). /// filling open slots. Completed nodes are retained (no pruning in v1).