hive-c0re: serialise the matrix and knowledge sweeps through the job queue
The matrix sweep and the /knowledge pull each had concurrent callers (#4723 item 4). Two overlapping knowledge pulls fail on .git/index.lock and the remote-tracking ref lock: 30 of 30 concurrent replays of the reset/clean/pull sequence in a scratch repo errored, 0 of 10 sequential ones did. Two overlapping matrix sweeps on a hive with no persisted Space / chat-room id both miss the by-name lookup and both createRoom (from reading the code, not reproduced against a homeserver). On every boot the MatrixSweep DAG node and the main.rs loop's immediate first call ran at once. Every sweep now runs as a job node, and each sweep's node holds its own capacity-1 queue resource (Resource::MatrixSweep, Resource::KnowledgeTree), the MetaWindow pattern: the scheduler never starts a second pass of one sweep while the first holds the resource, and different sweeps still run side by side. - templates::matrix_sweep / templates::knowledge_pull build the node with its resource; boot, the periodic loops and the swarm event all use them. - JobQueue::insert_unless_live folds a submission into a live node of the same kind instead of queueing another. Periodic ticks fold into a queued or running pass. The swarm knowledge event folds into a queued pull only, and queues one behind a running pull, which may have fetched before the push. - The main.rs matrix loop no longer sweeps immediately at startup; the boot MatrixSweep node is the startup pass, as KnowledgePull already was for knowledge. - The executors bound each pass (10 min matrix, 5 min knowledge), since a hung pass would otherwise hold its resource against every later one, and own the sweep-health banners, so every pass reports to them. Replaces the SweepLock version of this branch, per review. Refs #4723
This commit is contained in:
parent
2252c55df8
commit
1d4c77d2c8
14 changed files with 352 additions and 105 deletions
|
|
@ -80,16 +80,15 @@ async fn head_sha() -> Option<String> {
|
|||
.then(|| String::from_utf8_lossy(&out.stdout).trim().to_owned())
|
||||
}
|
||||
|
||||
/// Pull the latest changes in the local clone. Called from the webhook
|
||||
/// handler on every push to `internal/knowledge` main, and periodically
|
||||
/// from `main.rs` as a fallback. Uses `--ff-only` so a force-push to
|
||||
/// the knowledge repo never wedges the local copy silently.
|
||||
/// Pull the latest changes in the local clone. Runs only as a
|
||||
/// `KnowledgePull` job node (see `job_queue::templates::knowledge_pull`),
|
||||
/// whose resource keeps two pulls off this working tree at once. Uses
|
||||
/// `--ff-only` so a force-push to the knowledge repo never wedges the local
|
||||
/// copy silently.
|
||||
///
|
||||
/// When the pull actually moves `HEAD` (a real change, not a no-op),
|
||||
/// broadcasts a short `git diff --stat` summary to every live agent's
|
||||
/// inbox via `coord` — inbox-only, no forced wake, and shared by both
|
||||
/// call sites since the broadcast lives in here rather than in each
|
||||
/// caller.
|
||||
/// inbox via `coord` — inbox-only, no forced wake.
|
||||
pub async fn pull(coord: &Coordinator) -> Result<()> {
|
||||
// Sanity: if the clone is missing (e.g. storage was wiped), refuse
|
||||
// to pull and let the caller decide whether to re-clone.
|
||||
|
|
|
|||
Loading…
Reference in a new issue