Named counting-semaphore resources — the Dep::Resource side of the v2
model. A ResourceTable tracks per-name capacity + held counts; unconfigured
names default to capacity 1 (created lazily). try_acquire_all grants every
requested unit or none, leaving the table untouched on failure — so a node
never holds one resource while waiting for another, which is what makes the
scheduler deadlock-free without cycle detection. Duplicate names in a
request are summed; over-capacity requests can never acquire. release_all
saturates rather than underflowing. Runtime scheduler state, not persisted:
held counts are rederived from running nodes on restart. Guard objects
(RAII release, recursive re-entrancy) wrap this in a follow-up.
Adds set_boot_warning() to stats/warnings.rs: a one-shot variant of
set_warning() for steps that run once at hive-c0re boot (no periodic
retry to observe a later success and clear the banner via WarningGuard
drop). It forgets the guard, leaking it for the life of the process —
the banner clears on the next hive-c0re restart, which is also when a
config/environment fix would take effect.
Wires every remaining tracing::warn!()-and-forget site in
forge::ensure_all() to also raise a boot warning: core user/token
provisioning, per-org seeding, the operators-team grant per org, the
core/meta repo, shared docs repo, knowledge repo (+ its local clone),
both avatar uploads, and the final nixos-container list failure.
Split the core_token.is_some() body of ensure_all() into a new
ensure_all_orgs_and_repos() helper to stay under clippy's
function-length limit.
Deliberately out of scope: sync_agent() (shared with rebuild_agent(),
different blast radius) and ci_runner::ensure_ci_runner_registered()
(its own retry logic) — matrix::ensure_all() got the same slice-by-
slice treatment.
Per mara's direction — validate ids as they enter the graph so internal
iteration can trust every id the graph holds; the generational route for
removal comes later. Adds GraphError; insert() now rejects a dangling
Dep::Node / parent id (it is fallible); validate() checks all internal id
references resolve and that next_id is past the largest existing id;
deserialization runs validate() via #[serde(try_from = "GraphData<N>")], so
a loaded graph can never carry a dangling reference. 5 new tests; serde_json
added as a dev-dependency for the round-trip cases.
argus flagged that `NodeId(pub u64)` contradicted the "opaque" doc — a pub
inner field lets callers fabricate `NodeId(42)`. Make the field `pub(crate)`
so an id can only originate from the graph's monotonic counter or serde
deserialization, never a caller. Tests construct ids in-crate (unaffected);
the derived Serialize/Deserialize round-trips fine. Doc keeps "opaque" — now
accurate — with a line explaining the enforcement.
group_terminal checked only that every child was terminal, never the group
node's own state — so an empty group whose node is still Running returned
true (empty .all()), making a running node that has yet to append its
subgraph look already-finished. Now it requires the group node itself
terminal AND every child recursively terminal. Deciding when to settle a
group node to terminal once its children are done stays a scheduler concern;
this answers the dependents' question — is the whole group, node included,
finished. Adds group-node-pending-with-child-done + empty-running-group tests.
First step of extracting the job-DAG queue into a domain-agnostic
`hive-jobq` library, per the operator's v2 design: one persistent
graph, named-counter resources, recursive node groups, opaque stable
node ids, guard-object locks, a slot-filling scheduler.
This commit lands only the data model, so the shape can be reviewed
before the machinery is built on it:
- NodeId: opaque, stable, monotonic; group membership is a parent
edge, not encoded in the id (the 1/1/2 hierarchy is a derived UI
label).
- ResourceName, Dep (Node | Resource{name,count}), State.
- Node<N>: caller-defined payload N so the library stays
container-agnostic.
- Graph<N>: insert (mints stable ids), node lookup, children,
recursive group-terminal check. Retains completed groups (no
pruning in v1).
The resource-acquisition machinery (atomic all-or-nothing acquire),
the recursive-lock guards, and the scheduler loop are follow-ups.
Tests cover id minting, group terminality, and state terminality;
clippy + rustdoc clean.
Per mara's standing calibration (#2463): extraction > silencing for
too_many_lines. Splits the space + chat-room provisioning tail into its
own fn, bringing ensure_all back under the 100-line threshold without
an #[allow].
Fast-follow for #2289 — matrix::ensure_all() (admin user, per-agent
sync, hive Space + chat-room provisioning/invites) ran periodically
every 30 minutes but only ever warn!'d to the journal on failure, so a
persistent problem (missing invites, broken admin token, etc.) was
invisible to the operator.
ensure_all() now returns bool (aggregate ok/fail across every
sub-step) instead of (), and both call sites in main.rs feed that into
a debounced SweepHealth("matrix_ensure_all", warn, threshold=2) —
matches the existing knowledge_pull pattern. A lone bad sweep
self-heals silently; two consecutive failures raise a banner that
clears on the next clean sweep.
forge::ensure_all()'s remaining independent steps are still open —
that sweep only runs once at startup (no periodic loop), so the
debounced pattern doesn't map as directly; left for a follow-up.
The clippy check's comment described `-D warnings -A clippy::pedantic` — the
`-A` half dropping pedantic from the CI gate — but the args were only
`-D warnings`, so pedantic was hard-denied contrary to the doc. Operator
call: pedantic should be gated. Encode that as the single source of truth:
set the workspace lint `pedantic = deny` (errors locally and in CI), and
rewrite the checks.nix comment to match. Args unchanged; `-D warnings` still
gates rustc + non-pedantic clippy warnings. No new failures — the tree was
already pedantic-clean under CI's `-D warnings`, which denied pedantic.
Per operator review (#2547): remove the never-reached branch. render_flake
read HIVE_FORGE_URL inline with an unwrap_or_else(localhost:3000) fallback
that can't be hit — the daemon always sets HIVE_FORGE_URL (environment.nix)
— and the localhost URL wouldn't match the domain-scoped credential helper
anyway. Thread forge_base as an explicit param instead: the caller
sync_agents passes forge::forge_http_base(), tests pass it explicitly. This
removes the dead branch AND the hidden env-read, and drops the racy env
set_var from the forge-url test. render_flake is pure/param-driven again.
The persistent agent-<n> input url in the meta flake is now the forge
config repo, but the deploy + manual-rebuild re-lock must not follow it:
prepare_deploy and lock_update_for_rebuild override the input back to the
local applied mirror (git+file://<applied_dir>, main head) via
--override-input, which writes the applied rev into flake.lock.
This keeps the forge as the declared, reviewable source while:
- surviving forge unreachability (rebuilds fire on crash-restart and meta
bumps too, not just config PRs -- coupling every rebuild to forge would
be a regression),
- deploying exactly the reviewed head applied/<n>/main was fast-forwarded
to (no TOCTOU on a newer forge head merging mid-deploy),
- reusing verify_commit's local-override pattern so verify and deploy eval
the same source.
New applied_override_url helper + unit test.
hive-core now fetches each agent's config as a forge-hosted flake input
(git+http://<forge>/agent-configs/<n>.git). Add a git credential helper
(git-credential-hive-forge) that reads the live forge-core admin token on
every invocation and authenticates as the forge core user, wired via the
[credential] stanza in hive-core's $HOME/.gitconfig + on the service PATH.
Reading the token file live means zero stale copies and no resync on
rotation; no token ever lands in a flake URL or lock. Mirrors the existing
github.nix credential-helper pattern.
Trim the historical parenthetical about the dropped systemd-socket-proxyd
forwarder — argus flagged it (🟡 on the merged PR) and the constraint
(bind 0.0.0.0 or the bridge IP, not loopback) stands on its own without
the "why we dropped the proxy" narrative. No-history-in-docs preference.
package-lock.json was updated in 9edd3750 (dompurify/marked/esbuild
bumps) but npmDepsHash in nix/packages/frontend.nix wasn't recomputed,
breaking the frontend FOD build on main (hash mismatch, fixes#2539).
Recomputed via prefetch-npm-deps frontend/package-lock.json.
Now that matrix-sdk 0.18 is on main, reqwest 0.13.1 is already in the
tree transitively. Point the workspace crates at it directly.
reqwest 0.13 renamed the rustls feature set:
- rustls-tls -> rustls
- rustls-tls-native-roots -> rustls-native-certs
- (webpki-roots is now a separate feature)
hive-forge keeps its dual-trust story (system/native store for the
hive CA + bundled Mozilla roots for public CAs) by enabling
rustls-native-certs + webpki-roots explicitly.
forgejo-api 0.11 resolves cleanly against reqwest 0.13 (no conflict).
rusqlite 0.40 is intentionally NOT bumped here: matrix-sdk-sqlite 0.18
still pins rusqlite 0.37, so 0.40's libsqlite3-sys 0.38 would hit the
links="sqlite3" single-owner conflict. Deferred until upstream moves.
The #2439 agent-per-node chain removed submit-time dedup, DAG parent_id, and
child-DAG fan-out (everything grows in-DAG via append_subgraph). Update three
stale spots in docs/coordinator.md: the DAG is no longer a dedup unit;
in-DAG-appended nodes (not fan-out child DAGs) surface under the same polled
DAG id; and there is no cancel_children / parent-child cascade anymore.
#2415 moved hive-ci runner registration off the boot-critical path into
hive-c0re (forge/ci_runner.rs) + hive-priv, retiring the host-side
hive-ci-prefetch.service. Update the doc's Container-design, Auto-registration
flow, and Security sections to describe the new flow: c0re validates .runner /
mints the token out of band, hive-priv writes the host env-file + restarts the
runner, the container boots immediately with a fail-fast precond +
Restart=on-failure self-heal.
Ran nix fmt on the whole change (not just the .nix): rustfmt wraps the long
ci_enabled() is_ok_and line that tripped the treefmt CI gate. Adds the # Errors
doc section on the pub priv_client::register_ci_runner per the HOW-TO rule.
Completes #2415. Registration no longer gates container@hive-ci start:
- Retire the host-side hive-ci-prefetch.service (+ its 100-line script and
the now-dead coreTokenPath/containerRoot let-bindings) — the forge round-trip
it did on the boot path now lives in hive-c0re (forge/ci_runner.rs), run out
of band during the startup sweep.
- Drop the container@hive-ci TimeoutStartSec = mkForce 180s band-aid that
papered over that boot-path wait tripping the nspawn start timeout (the ~60s
restart loop of #2410).
- gitea-runner-hive gains an ExecStartPre precond (ahead of the nix-daemon
wait) that fails fast unless it is already registered (.runner present) or a
real, non-placeholder token is in place — so missing creds just hold the
runner down instead of blocking the container. Restart=on-failure (no start
limit) self-heals it once hive-c0re writes the token; c0re's explicit restart
is the primary path.
- Set HYPERHIVE_FORGE_CI_ENABLED=1 on hive-c0re.service so the sweep registers.
The tmpfiles TOKEN=placeholder seed + read-only bind-mount are unchanged; the
core admin token still never enters the container.
Part B (c0re half). New forge/ci_runner.rs: validate the hive-ci runner's
persisted .runner against the forge (GET /admin/runners/{id}); if absent or
stale, mint a fresh registration token (GET /admin/runners/registration-token,
raw request against the local http forge — forgejo-api 0.11 doesn't wrap it)
and hand it to hive-priv's RegisterCiRunner to write the host env-file +
restart the runner. Gated on HYPERHIVE_FORGE_CI_ENABLED; best-effort (never
aborts the startup sweep). Called from ensure_all after the org/repo seeding.
The nix boot-path change (drop prefetch gating, add runner precond, set the
env var) lands next on this branch.
Part A of moving hive-ci runner registration off the boot-critical path.
Adds a root-side hive-priv op that writes the runner registration token to
the host env-file /run/hive-ci/runner-token (in-place, preserving the inode
the container bind-mounts) and restarts the in-container gitea-runner-hive
unit. The forge admin token stays in hive-c0re; only the registration token
reaches the host env-file the container mounts read-only. The c0re-side
caller (ensure_ci_runner_registered) + the nix boot-path change land next on
this branch.
Bumps the feature (major) versions that update cleanly without breaking the
build: indicatif 0.17->0.18, tower-http 0.6->0.7, hmac 0.12->0.13,
sha2 0.10->0.11. Only adaptation needed: import hmac's KeyInit trait in
webhook_secret (new_from_slice moved from Mac to KeyInit in hmac 0.13).
Held back (require dedicated code-change PRs, out of scope for a
non-breaking bump):
- reqwest 0.13: renames the rustls-tls feature and conflicts with
forgejo-api 0.11 + matrix-sdk 0.14 which pin reqwest 0.12.
- rusqlite 0.40: libsqlite3-sys 0.38 clashes with matrix-sdk-sqlite 0.14's
0.35 (single links=sqlite3) — coupled to the matrix-sdk bump.
- rmcp 2.2, matrix-sdk 0.18: major API rewrites across the MCP/matrix crates.
Refreshes ~90 semver-compatible dependency versions (patch/minor within
the existing Cargo.toml constraints); Cargo.lock only, no source changes.
Major bumps held back by our constraints (rmcp 2.x, matrix-sdk 0.18,
reqwest 0.13, rusqlite 0.40, tower-http 0.7, sha2 0.11) are left for
separate, code-touching follow-ups.
README: add a "Overriding nixpkgs" section explaining hyperhive.inputs.
nixpkgs.follows and showing it in the quick-start flake example, since
hyperhive pins its own nixpkgs and consumers embedding it as a flake
input generally want to follow their host's nixpkgs instead.
CLAUDE.md: the repo map still described a hive-ag3nt/ directory
grouping hive-agent, hive-agent-mcp, and hive-agent-wake — that
directory doesn't exist; they're three separate top-level crates.
Also added the three wire-type crates split out of hive-sh4re
(hive-host-sock, hive-priv-sock) and hive-metric, none of which were
listed.
