4017a57350
Adds crate READMEs (matching the hive-claude precedent) and wires readme = "README.md" into each Cargo.toml [package] for hive-jobq, hive-host-sock, and hive-priv-sock — the crates squarely in the infra lane. Each README leads with purpose + when-to-use and points at the crate-root //! docs for depth rather than duplicating them. First increment of the per-crate-README effort; the shape here is the proposed template for the remaining crates (see issue discussion).
58 lines
2.9 KiB
Markdown
58 lines
2.9 KiB
Markdown
# hive-jobq
|
|
|
|
A persistent job-DAG scheduler, extracted from hive-c0re's in-tree `job_queue`
|
|
as a **domain-agnostic** library. It schedules a single persistent graph of
|
|
nodes over named resources; it knows nothing about containers, rebuilds, or any
|
|
hyperhive type — the node payload `N` and resource name `R` are both generic, so
|
|
the caller supplies its own domain.
|
|
|
|
## When to use it
|
|
|
|
Reach for this crate whenever you need to run a DAG of interdependent work items
|
|
under bounded, named concurrency — the hive-c0re rebuild/lifecycle queue is the
|
|
first consumer, but nothing here is specific to it. The caller defines the node
|
|
kinds, wires deps, and supplies a runner; the scheduler decides what can start.
|
|
|
|
## Model
|
|
|
|
One **persistent graph** for the whole system, not a DAG per job. Enqueuing
|
|
inserts a self-contained sub-DAG and returns the new node ids; the scheduler
|
|
runs a continuous loop, starting every node whose deps are satisfied:
|
|
|
|
- **Resource deps** are named counting semaphores over a caller-chosen type `R`
|
|
— e.g. `build-slot` (capacity N), `agent/<name>` (capacity 1), or any
|
|
unconfigured name (capacity 1, created on use). A node acquires *all* its
|
|
resource deps atomically at start (all-or-nothing) — no hold-and-wait, so no
|
|
deadlock.
|
|
- **Node deps** wait on another node per `DepWhen`: `AfterOk` needs success (a
|
|
failed dep cancels the dependent), `AfterAny` only needs terminal.
|
|
|
|
A node carries two independent axes: its `Dep`s (ordering + resource needs) and
|
|
its `parent` (structural grouping). The **parent chain**, not the node edges, is
|
|
what the scheduler consults for resource re-entrancy: a resource unit is held
|
|
for the acquiring node *plus its whole parent subtree*, and a descendant needing
|
|
a resource an ancestor already holds re-uses that grant (a re-entrant borrow,
|
|
one branch at a time) rather than taking a fresh unit.
|
|
|
|
A `NodeId` is opaque, stable, and monotonic (safe to persist). The scheduler is
|
|
single-threaded — it owns the resource table and mutates it directly.
|
|
|
|
## Shape
|
|
|
|
- **`Graph<N, R>`** — the persistent node store. `insert` mints ids and
|
|
validates dep/parent references; `set_state` is the single state-transition
|
|
choke point (and where each node's lifecycle timestamps —
|
|
`started_at` / `finished_at`, `DateTime<Utc>` — are stamped).
|
|
- **`Node<N, R>`** — `{ id, parent, payload, deps, state, started_at,
|
|
finished_at, error }`. All fields public; derives serde for persistence + the
|
|
wire.
|
|
- **`Scheduler<N, R>`** — drives the graph: `settle()` starts every ready node
|
|
(acquiring resources atomically), `complete(id, outcome)` reports a finished
|
|
node's result and rolls terminality up the parent chain, releasing grants once
|
|
a subtree is done. `Outcome::{Done, Failed(String)}` — the failure reason
|
|
rides `Failed` onto the node's `error`.
|
|
- **`ResourceTable<R>`** — per-name capacities; unconfigured names default to
|
|
capacity 1.
|
|
|
|
See the crate-root and `scheduler` module `//!` docs for the full borrow/release
|
|
model.
|