hyperhive/hive-c0re
Repository files (latest commit first)
Filename Latest commit message Latest commit date
atlas 1d4c77d2c8 hive-c0re: serialise the matrix and knowledge sweeps through the job queue
The matrix sweep and the /knowledge pull each had concurrent callers
(#4723 item 4). Two overlapping knowledge pulls fail on .git/index.lock
and the remote-tracking ref lock: 30 of 30 concurrent replays of the
reset/clean/pull sequence in a scratch repo errored, 0 of 10 sequential
ones did. Two overlapping matrix sweeps on a hive with no persisted
Space / chat-room id both miss the by-name lookup and both createRoom
(from reading the code, not reproduced against a homeserver). On every
boot the MatrixSweep DAG node and the main.rs loop's immediate first
call ran at once.

Every sweep now runs as a job node, and each sweep's node holds its own
capacity-1 queue resource (Resource::MatrixSweep,
Resource::KnowledgeTree), the MetaWindow pattern: the scheduler never
starts a second pass of one sweep while the first holds the resource,
and different sweeps still run side by side.

- templates::matrix_sweep / templates::knowledge_pull build the node
  with its resource; boot, the periodic loops and the swarm event all
  use them.
- JobQueue::insert_unless_live folds a submission into a live node of
  the same kind instead of queueing another. Periodic ticks fold into a
  queued or running pass. The swarm knowledge event folds into a queued
  pull only, and queues one behind a running pull, which may have
  fetched before the push.
- The main.rs matrix loop no longer sweeps immediately at startup; the
  boot MatrixSweep node is the startup pass, as KnowledgePull already
  was for knowledge.
- The executors bound each pass (10 min matrix, 5 min knowledge), since
  a hung pass would otherwise hold its resource against every later one,
  and own the sweep-health banners, so every pass reports to them.

Replaces the SweepLock version of this branch, per review.

Refs #4723
2026-09-27 20:15:55 +02:00
..
src hive-c0re: serialise the matrix and knowledge sweeps through the job queue 2026-09-27 20:15:55 +02:00
Cargo.toml log: send records natively to journald, keep stdout off-unit 2026-09-21 15:52:57 +02:00
README.md docs: restructure into topic subdirectories, collapse duplicated index 2026-09-02 01:55:37 +02:00

hive-c0re

The unprivileged host daemon (runs as hive-core). Owns the sqlite broker, the approval/reminder/schedule queues, the generic job-DAG queue, container lifecycle, gateway/forge/matrix provisioning, per-container stats, and the axum operator dashboard. Largest crate in the workspace — bin-only, no separate lib.

When to use it

Host-level, cross-container orchestration: spawning/rebuilding/ destroying agent containers, the approval flow, dashboard-visible state, provisioning per-agent forge/matrix/gateway accounts. Agent-side behavior (turn loop, MCP tools) lives in hive-agent/hive-agent-mcp instead — this daemon only talks to agents over the socket wire types in hive-sh4re.

Shape

Cohesive clusters live in directory submodules, each re-exported at the crate root (crate::broker::… keeps resolving regardless of which subdirectory a module actually lives in). One line each — read the module's own //! doc-comment for real detail, don't expect this file to track it:

  • dashboard/ — the operator dashboard (containers, approvals, schedules, logs, topology).
  • job_queue/ — the job-DAG queue + desired-state reconciliation (docs/scheduler/coordinator.md).
  • lifecycle/ — nixos-container lifecycle + per-agent config flake generation.
  • stores/ — sqlite-backed stores (broker, queues, audit, power).
  • workers/ — background sweeps (crash watch, scheduled prompts, auto-update, knowledge sync).
  • agent_config/ — per-agent registries (tool groups, capabilities, resource limits, topology).
  • stats/ — dashboard metrics aggregation + OTEL export.
  • socket_server/ — the unix-socket request server shared by per-agent + manager sockets.
  • forge/ — optional Forgejo wiring (docs/integrations/forge.md).
  • coordinator.rs — top-level wiring for serve.
  • meta.rs, migrate.rs — the meta flake + schema/state migrations.
  • matrix.rs, gateway_nginx.rs, webhook_secret.rs, priv_client.rs — matrix provisioning, gateway vhosts, webhook secrets, and the hive-priv client respectively.

See the top-level CLAUDE.md/docs/ index for the full reading-path map.