Compare commits

...
Author SHA1 Message Date
atlas
d9da92de4f jobq: the README repeats the same false persistence claims
Six more, in the crate's standalone README: the summary, the model section,
NodeId, the Graph entry and the Node serde note. Same wrong claim as the
module doc, in the sibling file the first sweep didn't look at.

A crate's README and its //! module doc are the same document in two files,
so a fix that touches one and not the other is the default outcome rather
than an unlucky miss.
2026-08-03 17:28:09 +02:00
atlas
6e18bbc839 jobq: say the graph is runtime-only, because it is
The crate described itself as a persistent scheduler and NodeId promised
stability across restarts. Neither is true: hive-c0re constructs an empty
Graph on every boot and re-derives desired state with its reconcile sweep,
and nothing in the workspace writes or loads a graph. hive-c0re's own
job_queue module doc has said "runtime-only (no persistence)" all along —
only the extracted library's prose drifted.

Seven claims corrected across the module doc, NodeId, the id-counter error
and the Graph type, plus the repo map. The module doc now states the fact
positively rather than just dropping the word: serde exists so the graph can
be projected onto a wire and so a store could be added later, ids and
timestamps are stable within a run.

NodeId spells out the consequence, since that is the part that could mislead
someone: an id stored outside the process is a historical record, not a
handle that will resolve after a restart.
2026-08-03 17:28:09 +02:00
3 changed files with 44 additions and 20 deletions

View file

@ -53,9 +53,12 @@ hand-maintained per-file tree drifts out of sync with the code.
streamable-http listener, `hive-mcp-http` systemd unit) + its claude
launch-config layer (tool-group/capability → `--allowedTools`,
`--mcp-config` render).
- **`hive-jobq/`** — persistent job-DAG scheduler, extracted from
hive-c0re's in-tree `job_queue` as a domain-agnostic library. One
persistent graph for the whole system (not a DAG per job); enqueuing
- **`hive-jobq/`** — job-DAG scheduler, extracted from hive-c0re's
in-tree `job_queue` as a domain-agnostic library. **Runtime-only —
nothing writes the graph to disk**; hive-c0re starts empty each boot
and re-derives desired state via the reconcile sweep, so node ids and
timestamps are stable within a run, not across restarts. One
shared graph for the whole system (not a DAG per job); enqueuing
inserts a self-contained sub-DAG and returns the ids of the nodes the job
asked for, in the order it named them. Generic over
the node payload `N` and the resource name `R`; resource deps are named

View file

@ -1,7 +1,7 @@
# hive-jobq
A persistent job-DAG scheduler, extracted from hive-c0re's in-tree `job_queue`
as a **domain-agnostic** library. It schedules a single persistent graph of
A job-DAG scheduler, extracted from hive-c0re's in-tree `job_queue`
as a **domain-agnostic** library. It schedules a single in-memory graph of
nodes over named resources; it knows nothing about containers, rebuilds, or any
hyperhive type — the node payload `N` and resource name `R` are both generic, so
the caller supplies its own domain.
@ -15,7 +15,13 @@ kinds, wires deps, and supplies a runner; the scheduler decides what can start.
## Model
One **persistent graph** for the whole system, not a DAG per job. Enqueuing
**Runtime-only: nothing writes this graph to disk.** The serde impls exist for
the wire projection (`hive-jobq-wire`) and a possible future store; no caller
loads one, so ids and timestamps are stable within a run, not across restarts.
hive-c0re constructs an empty graph every boot and re-derives desired state with
its reconcile sweep.
One **shared graph** for the whole system, not a DAG per job. Enqueuing
inserts a self-contained sub-DAG and returns the ids of the nodes the job
*asked* for, in the order it named them; the scheduler runs a continuous loop,
starting every node whose deps are satisfied:
@ -35,18 +41,20 @@ for the acquiring node *plus its whole parent subtree*, and a descendant needing
a resource an ancestor already holds re-uses that grant (a re-entrant borrow,
one branch at a time) rather than taking a fresh unit.
A `NodeId` is opaque, stable, and monotonic (safe to persist). The scheduler is
A `NodeId` is opaque, stable and monotonic **within a run** — a fresh process
mints ids from zero, so an id stored outside it is a historical record, not a
handle that will resolve later. The scheduler is
single-threaded — it owns the resource table and mutates it directly.
## Shape
- **`Graph<N, R>`** — the persistent node store. `insert` mints ids and
- **`Graph<N, R>`** — the in-memory node store. `insert` mints ids and
validates dep/parent references; `set_state` is the single state-transition
choke point (and where each node's lifecycle timestamps —
`started_at` / `finished_at`, `DateTime<Utc>` — are stamped).
- **`Node<N, R>`** — `{ id, parent, payload, deps, state, started_at,
finished_at, error }`. All fields public; derives serde for persistence + the
wire.
finished_at, error }`. All fields public; derives serde for the wire
projection (and so a store could be added — nothing calls one today).
- **`Scheduler<N, R>`** — drives the graph: `settle()` starts every ready node
(acquiring resources atomically), `complete(id, outcome)` reports a finished
node's result and rolls terminality up the parent chain, releasing grants once

View file

@ -1,9 +1,12 @@
//! `hive-jobq` — a persistent job-DAG scheduler, extracted from hive-c0re's
//! in-tree `job_queue` as a domain-agnostic library.
//! `hive-jobq` — a job-DAG scheduler, extracted from hive-c0re's in-tree
//! `job_queue` as a domain-agnostic library.
//!
//! **Runtime-only: nothing writes this graph to disk** — ids and timestamps are
//! stable within a run, not across restarts. See [`Graph`] and [`NodeId`].
//!
//! # Model (v2)
//!
//! One **persistent graph** for the whole system, not a DAG per job. Enqueuing
//! One **shared graph** for the whole system, not a DAG per job. Enqueuing
//! inserts a self-contained sub-DAG of nodes and returns their ids; the
//! scheduler runs a continuous loop, starting every node whose [`Dep`]s are
//! satisfied:
@ -18,7 +21,7 @@
//! A node carries two independent axes: its [`Dep`]s (ordering + resource
//! needs) and its [`Node::parent`] (structural grouping) — the parent chain,
//! not the [`Dep::Node`] edges, is what the [`scheduler`] consults for resource
//! re-entrancy. A [`NodeId`] is opaque, stable, and monotonic (persisted). The
//! re-entrancy. A [`NodeId`] is opaque, stable and monotonic within a run. The
//! payload `N` is generic so the library stays container-agnostic.
//!
//! A resource unit is held for the acquiring node + its whole [`Node::parent`]
@ -36,12 +39,15 @@ use chrono::{DateTime, Utc};
/// Opaque, stable, monotonic node identifier.
///
/// Assigned by the [`Graph`] on insert and persisted, so it is stable across
/// restarts.
/// Assigned by the [`Graph`] on insert, so it is stable for the lifetime of
/// that graph. **Not stable across restarts** — nothing persists the graph, so
/// a fresh process mints ids from zero again (see the module doc). Anything
/// storing an id outside the process is keeping a historical record, not a
/// handle it can resolve later.
///
/// The inner field is crate-private: an id can only originate from the graph's
/// monotonic counter (or deserialization of a persisted graph), never be
/// fabricated by a caller — that is what makes it opaque.
/// monotonic counter (or deserializing a graph that was serialized from one),
/// never be fabricated by a caller — that is what makes it opaque.
#[derive(
Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, serde::Serialize, serde::Deserialize,
)]
@ -351,14 +357,21 @@ pub enum GraphError {
/// so the next minted id would collide with one already in the graph.
#[error("next_id {next_id} must exceed the largest existing node id {max_id}")]
NextIdTooSmall {
/// The persisted counter value.
/// The counter value carried by the deserialized graph.
next_id: u64,
/// The largest id already present.
max_id: u64,
},
}
/// The single persistent graph of all nodes.
/// The single in-memory graph of all nodes.
///
/// **Not persisted.** The `Serialize`/`Deserialize` impls exist so a graph can
/// be projected onto a wire (`hive-jobq-wire`) and so a store *could* be added,
/// but no caller writes or loads one: hive-c0re constructs an empty graph on
/// every boot and re-derives desired state with its reconcile sweep (see
/// `hive-c0re/src/job_queue/mod.rs`). A derive is a capability, not a promise
/// that something uses it.
///
/// New jobs are inserted as sub-DAGs of nodes; the scheduler walks this graph
/// filling open slots. Completed nodes are retained (no pruning in v1).