| Filename | Latest commit message | Latest commit date |
|---|---|---|
d1f1a361f0
The caller supplies how to run a node and spawns what it gets back; it never touches claiming or completion. The returned future runs the node *and completes it*, so "forgot to finish the node" stops being something a caller can do — completion is inside the thing they spawn. The `Option` is answered synchronously, before anything is awaited, so the run loop learns whether there was work without waiting on the node it just started. That is what lets it choose between claiming again immediately and backing off; an id alone cannot express that choice. Locking: taken twice, briefly, and never held across the await — once to claim, once inside the future to complete. A guard alive across an await point would make the future non-`Send` and unspawnable, which is also why the node itself runs unlocked for however long it takes. `Arc` + `std::sync::Mutex` keep this runtime-agnostic: no tokio in this crate. `run` receives an owned payload rather than a borrow for the same reason a `&Job` could not be threaded through the executors: a reference parameter is live for the whole future, borrowing the graph across the await and poisoning `Send`. The output carries the insert result instead of swallowing it. This crate has no logger by design, so a malformed grown job is reported to the caller, who can log it. The node completes either way — its own work already happened. |
||
| .. | ||
| src | ||
| Cargo.toml | ||
| README.md | ||
hive-jobq
A persistent job-DAG scheduler, extracted from hive-c0re's in-tree job_queue
as a domain-agnostic library. It schedules a single persistent graph of
nodes over named resources; it knows nothing about containers, rebuilds, or any
hyperhive type — the node payload N and resource name R are both generic, so
the caller supplies its own domain.
When to use it
Reach for this crate whenever you need to run a DAG of interdependent work items under bounded, named concurrency — the hive-c0re rebuild/lifecycle queue is the first consumer, but nothing here is specific to it. The caller defines the node kinds, wires deps, and supplies a runner; the scheduler decides what can start.
Model
One persistent graph for the whole system, not a DAG per job. Enqueuing inserts a self-contained sub-DAG and returns the ids of the nodes the job asked for, in the order it named them; the scheduler runs a continuous loop, starting every node whose deps are satisfied:
- Resource deps are named counting semaphores over a caller-chosen type
R— e.g.build-slot(capacity N),agent/<name>(capacity 1), or any unconfigured name (capacity 1, created on use). A node acquires all its resource deps atomically at start (all-or-nothing) — no hold-and-wait, so no deadlock. - Node deps wait on another node per
DepWhen:AfterOkneeds success (a failed dep cancels the dependent),AfterAnyonly needs terminal.
A node carries two independent axes: its Deps (ordering + resource needs) and
its parent (structural grouping). The parent chain, not the node edges, is
what the scheduler consults for resource re-entrancy: a resource unit is held
for the acquiring node plus its whole parent subtree, and a descendant needing
a resource an ancestor already holds re-uses that grant (a re-entrant borrow,
one branch at a time) rather than taking a fresh unit.
A NodeId is opaque, stable, and monotonic (safe to persist). The scheduler is
single-threaded — it owns the resource table and mutates it directly.
Shape
Graph<N, R>— the persistent node store.insertmints ids and validates dep/parent references;set_stateis the single state-transition choke point (and where each node's lifecycle timestamps —started_at/finished_at,DateTime<Utc>— are stamped).Node<N, R>—{ id, parent, payload, deps, state, started_at, finished_at, error }. All fields public; derives serde for persistence + the wire.Scheduler<N, R>— drives the graph:settle()starts every ready node (acquiring resources atomically),complete(id, outcome)reports a finished node's result and rolls terminality up the parent chain, releasing grants once a subtree is done.Outcome::{Done, Failed(String)}— the failure reason ridesFailedonto the node'serror.ResourceTable<R>— per-name capacities; unconfigured names default to capacity 1.
See the crate-root and scheduler module //! docs for the full borrow/release
model.