| Filename | Latest commit message | Latest commit date |
|---|---|---|
affedecaa5
`DepWhen` was two named cases, so every new combination wanted a new
variant. It is now a set over the terminal outcomes: a `u8` bitset
newtype, no dependency, with `AFTER_OK` / `AFTER_ANY` kept as the two
constants the templates actually use. "Run regardless" is all outcomes,
"anything that isn't a failure" is `{Done, Cancelled}`, a compensation
branch is `{Failed}` — closed under combination, so it never needs
another variant.
`TerminalState` is its own type rather than a subset of `State`, so an
edge cannot name `Pending` / `Running` / `Finishing`. Those are
meaningless in a dependency and are better unrepresentable than
validated against. The empty set is the one thing that can't be typed
away — nothing satisfies it, so `validate` rejects it next to the cycle
check.
Two consequences worth calling out:
- `cascade_cancel` collapses to one rule: a pending node is doomed once
any edge it names can no longer be satisfied. The hardcoded `AfterOk`
special case is gone, and a weak-edged node survives its dependency's
cancellation because of its own edge rather than by exemption.
- The cascade now runs on **any** terminal outcome, `Done` included.
With sets, success rules dependents out just as failure does — a
`{Failed}` branch is unsatisfiable the moment its dependency succeeds,
and leaving it `Pending` would wedge the subtree non-terminal forever.
That is a hang, not a wrong answer, so it is the load-bearing half of
this commit.
Edges are conjunctive, so "any of these N failed" is not directly
sayable. The composition that works is in the tests: the success branch
depends `AFTER_OK` on every root, so it is itself cancelled the moment
one of them doesn't succeed, and the failure branch hangs off *that*
with `{Cancelled}`. Exactly one of the two runs.
Also deletes hive-c0re's duplicate `DepWhen` enum and the
`to_crate_when` translation beside it. The copy bought nothing and had
to be widened in lockstep with the crate's edge model — it is the
in-between layer #2772 exists to remove, and it is what broke the build
when the crate's spelling changed.
All 34 jobq tests pass, including the four new ones covering both
directions of a failure-only branch, weak-edge survival of a cancelled
dependency, and the aggregator composition.
|
||
| .. | ||
| src | ||
| Cargo.toml | ||
| README.md | ||
hive-jobq
A persistent job-DAG scheduler, extracted from hive-c0re's in-tree job_queue
as a domain-agnostic library. It schedules a single persistent graph of
nodes over named resources; it knows nothing about containers, rebuilds, or any
hyperhive type — the node payload N and resource name R are both generic, so
the caller supplies its own domain.
When to use it
Reach for this crate whenever you need to run a DAG of interdependent work items under bounded, named concurrency — the hive-c0re rebuild/lifecycle queue is the first consumer, but nothing here is specific to it. The caller defines the node kinds, wires deps, and supplies a runner; the scheduler decides what can start.
Model
One persistent graph for the whole system, not a DAG per job. Enqueuing inserts a self-contained sub-DAG and returns the new node ids; the scheduler runs a continuous loop, starting every node whose deps are satisfied:
- Resource deps are named counting semaphores over a caller-chosen type
R— e.g.build-slot(capacity N),agent/<name>(capacity 1), or any unconfigured name (capacity 1, created on use). A node acquires all its resource deps atomically at start (all-or-nothing) — no hold-and-wait, so no deadlock. - Node deps wait on another node per
DepWhen:AfterOkneeds success (a failed dep cancels the dependent),AfterAnyonly needs terminal.
A node carries two independent axes: its Deps (ordering + resource needs) and
its parent (structural grouping). The parent chain, not the node edges, is
what the scheduler consults for resource re-entrancy: a resource unit is held
for the acquiring node plus its whole parent subtree, and a descendant needing
a resource an ancestor already holds re-uses that grant (a re-entrant borrow,
one branch at a time) rather than taking a fresh unit.
A NodeId is opaque, stable, and monotonic (safe to persist). The scheduler is
single-threaded — it owns the resource table and mutates it directly.
Shape
Graph<N, R>— the persistent node store.insertmints ids and validates dep/parent references;set_stateis the single state-transition choke point (and where each node's lifecycle timestamps —started_at/finished_at,DateTime<Utc>— are stamped).Node<N, R>—{ id, parent, payload, deps, state, started_at, finished_at, error }. All fields public; derives serde for persistence + the wire.Scheduler<N, R>— drives the graph:settle()starts every ready node (acquiring resources atomically),complete(id, outcome)reports a finished node's result and rolls terminality up the parent chain, releasing grants once a subtree is done.Outcome::{Done, Failed(String)}— the failure reason ridesFailedonto the node'serror.ResourceTable<R>— per-name capacities; unconfigured names default to capacity 1.
See the crate-root and scheduler module //! docs for the full borrow/release
model.