Compare commits
3 changed files with 20 additions and 44 deletions
|
|
@ -53,12 +53,9 @@ hand-maintained per-file tree drifts out of sync with the code.
|
||||||
streamable-http listener, `hive-mcp-http` systemd unit) + its claude
|
streamable-http listener, `hive-mcp-http` systemd unit) + its claude
|
||||||
launch-config layer (tool-group/capability → `--allowedTools`,
|
launch-config layer (tool-group/capability → `--allowedTools`,
|
||||||
`--mcp-config` render).
|
`--mcp-config` render).
|
||||||
- **`hive-jobq/`** — job-DAG scheduler, extracted from hive-c0re's
|
- **`hive-jobq/`** — persistent job-DAG scheduler, extracted from
|
||||||
in-tree `job_queue` as a domain-agnostic library. **Runtime-only —
|
hive-c0re's in-tree `job_queue` as a domain-agnostic library. One
|
||||||
nothing writes the graph to disk**; hive-c0re starts empty each boot
|
persistent graph for the whole system (not a DAG per job); enqueuing
|
||||||
and re-derives desired state via the reconcile sweep, so node ids and
|
|
||||||
timestamps are stable within a run, not across restarts. One
|
|
||||||
shared graph for the whole system (not a DAG per job); enqueuing
|
|
||||||
inserts a self-contained sub-DAG and returns the ids of the nodes the job
|
inserts a self-contained sub-DAG and returns the ids of the nodes the job
|
||||||
asked for, in the order it named them. Generic over
|
asked for, in the order it named them. Generic over
|
||||||
the node payload `N` and the resource name `R`; resource deps are named
|
the node payload `N` and the resource name `R`; resource deps are named
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
# hive-jobq
|
# hive-jobq
|
||||||
|
|
||||||
A job-DAG scheduler, extracted from hive-c0re's in-tree `job_queue`
|
A persistent job-DAG scheduler, extracted from hive-c0re's in-tree `job_queue`
|
||||||
as a **domain-agnostic** library. It schedules a single in-memory graph of
|
as a **domain-agnostic** library. It schedules a single persistent graph of
|
||||||
nodes over named resources; it knows nothing about containers, rebuilds, or any
|
nodes over named resources; it knows nothing about containers, rebuilds, or any
|
||||||
hyperhive type — the node payload `N` and resource name `R` are both generic, so
|
hyperhive type — the node payload `N` and resource name `R` are both generic, so
|
||||||
the caller supplies its own domain.
|
the caller supplies its own domain.
|
||||||
|
|
@ -15,13 +15,7 @@ kinds, wires deps, and supplies a runner; the scheduler decides what can start.
|
||||||
|
|
||||||
## Model
|
## Model
|
||||||
|
|
||||||
**Runtime-only: nothing writes this graph to disk.** The serde impls exist for
|
One **persistent graph** for the whole system, not a DAG per job. Enqueuing
|
||||||
the wire projection (`hive-jobq-wire`) and a possible future store; no caller
|
|
||||||
loads one, so ids and timestamps are stable within a run, not across restarts.
|
|
||||||
hive-c0re constructs an empty graph every boot and re-derives desired state with
|
|
||||||
its reconcile sweep.
|
|
||||||
|
|
||||||
One **shared graph** for the whole system, not a DAG per job. Enqueuing
|
|
||||||
inserts a self-contained sub-DAG and returns the ids of the nodes the job
|
inserts a self-contained sub-DAG and returns the ids of the nodes the job
|
||||||
*asked* for, in the order it named them; the scheduler runs a continuous loop,
|
*asked* for, in the order it named them; the scheduler runs a continuous loop,
|
||||||
starting every node whose deps are satisfied:
|
starting every node whose deps are satisfied:
|
||||||
|
|
@ -41,20 +35,18 @@ for the acquiring node *plus its whole parent subtree*, and a descendant needing
|
||||||
a resource an ancestor already holds re-uses that grant (a re-entrant borrow,
|
a resource an ancestor already holds re-uses that grant (a re-entrant borrow,
|
||||||
one branch at a time) rather than taking a fresh unit.
|
one branch at a time) rather than taking a fresh unit.
|
||||||
|
|
||||||
A `NodeId` is opaque, stable and monotonic **within a run** — a fresh process
|
A `NodeId` is opaque, stable, and monotonic (safe to persist). The scheduler is
|
||||||
mints ids from zero, so an id stored outside it is a historical record, not a
|
|
||||||
handle that will resolve later. The scheduler is
|
|
||||||
single-threaded — it owns the resource table and mutates it directly.
|
single-threaded — it owns the resource table and mutates it directly.
|
||||||
|
|
||||||
## Shape
|
## Shape
|
||||||
|
|
||||||
- **`Graph<N, R>`** — the in-memory node store. `insert` mints ids and
|
- **`Graph<N, R>`** — the persistent node store. `insert` mints ids and
|
||||||
validates dep/parent references; `set_state` is the single state-transition
|
validates dep/parent references; `set_state` is the single state-transition
|
||||||
choke point (and where each node's lifecycle timestamps —
|
choke point (and where each node's lifecycle timestamps —
|
||||||
`started_at` / `finished_at`, `DateTime<Utc>` — are stamped).
|
`started_at` / `finished_at`, `DateTime<Utc>` — are stamped).
|
||||||
- **`Node<N, R>`** — `{ id, parent, payload, deps, state, started_at,
|
- **`Node<N, R>`** — `{ id, parent, payload, deps, state, started_at,
|
||||||
finished_at, error }`. All fields public; derives serde for the wire
|
finished_at, error }`. All fields public; derives serde for persistence + the
|
||||||
projection (and so a store could be added — nothing calls one today).
|
wire.
|
||||||
- **`Scheduler<N, R>`** — drives the graph: `settle()` starts every ready node
|
- **`Scheduler<N, R>`** — drives the graph: `settle()` starts every ready node
|
||||||
(acquiring resources atomically), `complete(id, outcome)` reports a finished
|
(acquiring resources atomically), `complete(id, outcome)` reports a finished
|
||||||
node's result and rolls terminality up the parent chain, releasing grants once
|
node's result and rolls terminality up the parent chain, releasing grants once
|
||||||
|
|
|
||||||
|
|
@ -1,12 +1,9 @@
|
||||||
//! `hive-jobq` — a job-DAG scheduler, extracted from hive-c0re's in-tree
|
//! `hive-jobq` — a persistent job-DAG scheduler, extracted from hive-c0re's
|
||||||
//! `job_queue` as a domain-agnostic library.
|
//! in-tree `job_queue` as a domain-agnostic library.
|
||||||
//!
|
|
||||||
//! **Runtime-only: nothing writes this graph to disk** — ids and timestamps are
|
|
||||||
//! stable within a run, not across restarts. See [`Graph`] and [`NodeId`].
|
|
||||||
//!
|
//!
|
||||||
//! # Model (v2)
|
//! # Model (v2)
|
||||||
//!
|
//!
|
||||||
//! One **shared graph** for the whole system, not a DAG per job. Enqueuing
|
//! One **persistent graph** for the whole system, not a DAG per job. Enqueuing
|
||||||
//! inserts a self-contained sub-DAG of nodes and returns their ids; the
|
//! inserts a self-contained sub-DAG of nodes and returns their ids; the
|
||||||
//! scheduler runs a continuous loop, starting every node whose [`Dep`]s are
|
//! scheduler runs a continuous loop, starting every node whose [`Dep`]s are
|
||||||
//! satisfied:
|
//! satisfied:
|
||||||
|
|
@ -21,7 +18,7 @@
|
||||||
//! A node carries two independent axes: its [`Dep`]s (ordering + resource
|
//! A node carries two independent axes: its [`Dep`]s (ordering + resource
|
||||||
//! needs) and its [`Node::parent`] (structural grouping) — the parent chain,
|
//! needs) and its [`Node::parent`] (structural grouping) — the parent chain,
|
||||||
//! not the [`Dep::Node`] edges, is what the [`scheduler`] consults for resource
|
//! not the [`Dep::Node`] edges, is what the [`scheduler`] consults for resource
|
||||||
//! re-entrancy. A [`NodeId`] is opaque, stable and monotonic within a run. The
|
//! re-entrancy. A [`NodeId`] is opaque, stable, and monotonic (persisted). The
|
||||||
//! payload `N` is generic so the library stays container-agnostic.
|
//! payload `N` is generic so the library stays container-agnostic.
|
||||||
//!
|
//!
|
||||||
//! A resource unit is held for the acquiring node + its whole [`Node::parent`]
|
//! A resource unit is held for the acquiring node + its whole [`Node::parent`]
|
||||||
|
|
@ -39,15 +36,12 @@ use chrono::{DateTime, Utc};
|
||||||
|
|
||||||
/// Opaque, stable, monotonic node identifier.
|
/// Opaque, stable, monotonic node identifier.
|
||||||
///
|
///
|
||||||
/// Assigned by the [`Graph`] on insert, so it is stable for the lifetime of
|
/// Assigned by the [`Graph`] on insert and persisted, so it is stable across
|
||||||
/// that graph. **Not stable across restarts** — nothing persists the graph, so
|
/// restarts.
|
||||||
/// a fresh process mints ids from zero again (see the module doc). Anything
|
|
||||||
/// storing an id outside the process is keeping a historical record, not a
|
|
||||||
/// handle it can resolve later.
|
|
||||||
///
|
///
|
||||||
/// The inner field is crate-private: an id can only originate from the graph's
|
/// The inner field is crate-private: an id can only originate from the graph's
|
||||||
/// monotonic counter (or deserializing a graph that was serialized from one),
|
/// monotonic counter (or deserialization of a persisted graph), never be
|
||||||
/// never be fabricated by a caller — that is what makes it opaque.
|
/// fabricated by a caller — that is what makes it opaque.
|
||||||
#[derive(
|
#[derive(
|
||||||
Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, serde::Serialize, serde::Deserialize,
|
Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, serde::Serialize, serde::Deserialize,
|
||||||
)]
|
)]
|
||||||
|
|
@ -357,21 +351,14 @@ pub enum GraphError {
|
||||||
/// so the next minted id would collide with one already in the graph.
|
/// so the next minted id would collide with one already in the graph.
|
||||||
#[error("next_id {next_id} must exceed the largest existing node id {max_id}")]
|
#[error("next_id {next_id} must exceed the largest existing node id {max_id}")]
|
||||||
NextIdTooSmall {
|
NextIdTooSmall {
|
||||||
/// The counter value carried by the deserialized graph.
|
/// The persisted counter value.
|
||||||
next_id: u64,
|
next_id: u64,
|
||||||
/// The largest id already present.
|
/// The largest id already present.
|
||||||
max_id: u64,
|
max_id: u64,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The single in-memory graph of all nodes.
|
/// The single persistent graph of all nodes.
|
||||||
///
|
|
||||||
/// **Not persisted.** The `Serialize`/`Deserialize` impls exist so a graph can
|
|
||||||
/// be projected onto a wire (`hive-jobq-wire`) and so a store *could* be added,
|
|
||||||
/// but no caller writes or loads one: hive-c0re constructs an empty graph on
|
|
||||||
/// every boot and re-derives desired state with its reconcile sweep (see
|
|
||||||
/// `hive-c0re/src/job_queue/mod.rs`). A derive is a capability, not a promise
|
|
||||||
/// that something uses it.
|
|
||||||
///
|
///
|
||||||
/// New jobs are inserted as sub-DAGs of nodes; the scheduler walks this graph
|
/// New jobs are inserted as sub-DAGs of nodes; the scheduler walks this graph
|
||||||
/// filling open slots. Completed nodes are retained (no pruning in v1).
|
/// filling open slots. Completed nodes are retained (no pruning in v1).
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue