Two hardening items from argus's review of #2374:
1. PR state check at submission:
- Add `pr_is_open(repo, pr)` to forge/pr_merge.rs using
`repo_get_pull_request` + `StateType` — early error if the PR is
already closed or merged instead of queuing a card that fails later
- Call it in `submit_merge_config_pr` before fetching the head sha
2. Atomic fetched_sha INSERT:
- Add `fetched_sha: Option<&str>` to `Approvals::submit_kind` so
the sha can be included in the INSERT rather than a follow-up UPDATE
- MergeConfigPr already knows the sha before inserting the row
(pr_head_sha runs first) → pass `Some(&sha)`, drop the separate
`set_fetched_sha` call → truly atomic
- ApplyCommit still needs two writes (sha resolved by git_fetch_to_tag
after the row exists) → pass `None`, `set_fetched_sha` unchanged
- All other callers (InitConfig, Spawn, UpdateMetaInputs,
SchedulePrompt) pass `None` — no behavioural change
- Add `fetched_sha_in_insert_is_readable_via_get` test covering the
MergeConfigPr path
MergeConfigPr approvals had a fully-implemented approve handler
(run_merge_config_pr, ff_push_to_main, mark_pr_merged) and dashboard
display, but no way to submit one. An agent with the `approvals` tool
group calling request_merge_config_pr(agent, pr_number) is the missing
piece.
What this adds:
- RequestMergeConfigPr variant in hive-sh4re AgentRequest + ToolGroup::Approvals
- submit_merge_config_pr: fetches PR head sha (the drift-gate reviewed sha),
queues a MergeConfigPr row, sets fetched_sha, emits approval_added with
pr_number so the dashboard card links to the forge PR
- handle_request_merge_config_pr: topology (require_descendant) +
tool-group (require_group(approvals)) guards before submit
- socket_server/mod.rs: dispatch arm for RequestMergeConfigPr
- hive-ag3nt MCP tool: request_merge_config_pr with full description
- docs/tools/lifecycle.md: documents the new tool + boundary table row
Unlike submit_apply_commit, no flake pre-flight at submission time (eval-
verify happens at approval time inside run_merge_config_pr, same as the
rest of the merge pipeline). Applied repo must already exist (guard added
with a clear error message pointing at request_apply_commit for first-spawn).
- Add `priv_proto::AGENT_RUNTIME_ROOT` to hive-sh4re as the shared
single source for the per-agent runtime root path. hive-priv now
imports it instead of carrying a local const with a stale comment
that still pointed at `coordinator::AGENT_RUNTIME_ROOT` (removed in
#2285/#2367 — moved to `paths::agent_runtime_root()`).
- Add 'must stay in sync' cross-ref comments on both sides of the
privsep boundary:
· priv_proto::META_DIR ↔ paths::meta_root()
· priv_proto::AGENT_STATE_ROOT ↔ paths::AGENTS_ROOT
· priv_proto::AGENT_RUNTIME_ROOT ↔ paths::RUNTIME_ROOT + agent_runtime_root()
· paths::AGENTS_ROOT ↔ priv_proto::AGENT_STATE_ROOT
· paths::RUNTIME_ROOT ↔ priv_proto::AGENT_RUNTIME_ROOT
The dep graph prevents a shared import (hive-sh4re is a leaf; both
hive-c0re and hive-priv depend on it but not each other), so the
lockstep comments are the enforced contract.
Replace the try-and-ignore-duplicate-column approach in apply_migrations
with proper schema versioning using a shared schema_versions table.
## mechanism
New function: db::apply_versioned_migrations(conn, subsystem,
legacy_column, migrations). Tracks the applied-migration count in a
schema_versions table (one row per subsystem key). Only migrations past
the stored version run.
Legacy detection: pre-versioning databases have no schema_versions row.
The legacy_column tuple (table, column) identifies a column that exists
only in a fully-migrated legacy database. If present, all known
migrations are skipped. If absent, migrations start from 0.
## stores migrated
- broker: removes bespoke ensure_message_columns / ensure_reminder_columns.
Unified into BROKER_MIGRATIONS (v1-v5). Legacy detector: messages.priority
(added in the last pre-versioning migration).
- approvals: 4 historical migrations (v1-v4). Legacy detector:
approvals.submitter.
- operator_questions: 3 historical migrations (v1-v3). Legacy detector:
operator_questions.target.
- scheduled_prompts: 1 historical migration (v1). Legacy detector:
scheduled_prompts.paused_at_unix.
apply_migrations removed (no callers).
## tests (db.rs)
- fresh_install_runs_all_migrations
- legacy_install_skips_all_migrations
- partial_migration_resumes_from_version
- already_at_latest_is_noop
- multiple_stores_in_same_db
The is_descendant_of and apply_set_parent cycle detection both used
hand-rolled 32-hop bounded ancestor walks. Correct in practice (no
real hive exceeds 32 levels) but carried an arbitrary ceiling and were
harder to reason about than proven graph primitives.
Changes:
- Add build_graph(): converts BTreeMap<name, parent|null> → DiGraph
with parent→child edges + BTreeMap<name, NodeIndex> index
- Add is_descendant_of_in(): pure (no disk I/O), uses
petgraph::algo::has_path_connecting from ancestor to candidate
- Rewrite is_descendant_of(): delegates to is_descendant_of_in(&read())
- Rewrite apply_set_parent() cycle detection: build_graph() + speculative
edge + is_cyclic_directed(); no depth limit
- Add tests for is_descendant_of_in (self, direct child, grandchild,
parent-is-not-child, sibling, unknown)
petgraph was already a workspace dep (used elsewhere). On-disk format
unchanged (flat JSON map). Public API surface unchanged.
On 409 (team already exists), list the org teams to find the operators
team id, then unconditionally PATCH to the desired settings via
org_edit_team. This self-heals a team that was created with the wrong
shape by an older code path (missing units, wrong permission) without
touching membership (separate endpoint, operator-managed).
Addresses mara's review: 'shouldnt we get, then change, then update'.
Unconditional PATCH is simpler than GET→diff→conditional PATCH and safe
here since we own units/permission/description fully.
mara: the background worker is redundant if c0re knows when its own
sockets go missing. damocles: 10s poll latency and redundancy are two
faces of the same issue — poll adds a reconnect window and does
redundant work when c0re could react directly.
design: c0re owns the MCP listener lifecycle, so the only time a
listener disappears without c0re knowing is when c0re itself restarts.
- replace spawn_poll (recurring 10s loop) with sync_on_start (one-shot
sweep at daemon boot): re-registers all running agents on startup
after /run/hyperhive/agents/ is cleared by the tmpfs reset.
- run_reconcile (reconcile-start path): add coord.register_agent(name)
immediately after start_with_fallback — event-driven, no poll delay.
- run_create already calls register_agent eagerly; kill/destroy paths
already call unregister_agent — no changes needed there.
tracker: #2290
Collapse the scattered ensure_agent_runtime_dir calls into the lifecycle
functions themselves so callers have a single responsibility:
- lifecycle::spawn: calls ensure_agent_runtime_dir before write_dropins.
Callers (handle_spawn, ensure_root_agent) no longer need a separate
preamble step.
- lifecycle::rebuild_no_meta spawn path: calls ensure_agent_runtime_dir
before write_dropins. apply_commit / merge_config_pr flows no longer
need a manual ensure_agent_runtime_dir.
- run_create (job-queue): drops ensure_agent_runtime_dir + register_agent.
The tail Reconcile's converge_start_preamble handles the runtime dir
and mcp_sockets::spawn_poll handles the listener. Create stays purely
'provision + create', not 'create + start'.
- handle_spawn (server.rs): drops manual preamble; lifecycle::spawn owns it.
Drops unneeded unregister_agent on failure (supervisor handles listener).
- ensure_root_agent (auto_update.rs): drops manual ensure_agent_runtime_dir.
- actions.rs apply_commit / merge_config_pr: drop manual
ensure_agent_runtime_dir; rebuild_no_meta's spawn path handles it.
Result: ensure_agent_runtime_dir lives in exactly two places —
lifecycle::spawn (direct spawn) and converge_start_preamble (start/reconcile
path). All other callers are clean call sites.
- lifecycle::StartableAgent: opaque token produced only by
converge_start_preamble. #[must_use] with a hint to call
start_with_fallback(token).
- lifecycle::converge_start_preamble(name, hive, paths): runs
ensure_agent_runtime_dir + write_dropins, returns StartableAgent.
The only way to obtain a token.
- lifecycle::start_with_fallback(token: StartableAgent): public API
now requires the token. Callers that skip the preamble get a compile
error, not a runtime outage.
- lifecycle::start_with_fallback_inner(name): private; used internally
by rebuild_no_meta where the preamble is already enforced structurally
(write_dropins was called on the line above).
- exec.rs ReconcileAction::Start: migrated to converge_start_preamble
+ start_with_fallback(token). The write_dropins + start_with_fallback
two-step is now a single typed pipeline.
- lifecycle::ensure_agent_runtime_dir(name): pure filesystem op, no
Coordinator dep. Creates /run/hyperhive/agents/<name> without touching
the MCP listener map.
- workers/mcp_sockets::spawn_poll(coord): 10 s reconcile loop (same shape
as agent_sockets::spawn_poll). Converges 'agent running => MCP listener
bound'. First tick is immediate so hive-c0re restarts re-register all
running agents without waiting a full interval. Fixes the dead-listener-
after-daemon-restart gap.
- All ensure_runtime() call sites updated:
- Prebuild/Swap/WriteDropin: Coordinator::agent_dir() (pure, no IO)
- Reconcile-Start: ensure_agent_runtime_dir + agent_dir (dir may be
missing after reboot; listener deferred to supervisor)
- run_create / handle_spawn: ensure_agent_runtime_dir + register_agent
(eager on first spawn so socket ready before harness first turn)
- apply_commit / merge_config_pr: ensure_agent_runtime_dir + agent_dir
- Manager (auto_update): ensure_agent_runtime_dir + agent_dir
(manager has no MCP listener; socket_server::start_manager owns it)
- ensure_runtime() retained in Coordinator with updated doc pointing at
the preferred split form. No callers remain outside tests.
systemd-tmpfiles d entries adjust mode+owner on existing dirs. Using
0755 root root would stomp live agents' socket dirs (owned by agent
uid:gid) on every sync_tmpfiles call, breaking the harness's ability
to bind new sockets until host_config rechowns them.
Fix: 0777 root root — matches the chmod_socket_dir(0o777) fallback
already used by host_config when the agent uid is unavailable. World-
writable dirs let the non-root harness bind sockets regardless of who
owns the dir. host_config's chown_socket_dir tightens ownership when
the agent uid is resolved.
Also add missing # Errors doc to priv_client::sync_agent_tmpfiles.
Root cause of the boot outage: container@h-* units try to start before
hive-c0re reaches ensure_runtime, so bind-mount source dirs are missing.
Fix: hive-c0re (via hive-priv, which runs as root) writes
/etc/tmpfiles.d/hyperhive-agents.conf whenever the agent set changes.
systemd-tmpfiles-setup.service (sysinit.target) reads it at every boot
BEFORE any container units start, pre-creating:
/run/hyperhive/agents/<name> — MCP socket dir (bind -> /run/hive)
/run/hive-agent/<name> — web socket dir (bind -> /run/hive-agent)
This alone removes the outage class: even if hive-c0re is slow to start,
the bind-mount sources exist and container units can activate.
Added:
- PrivRequest::SyncAgentTmpfiles { agents } in hive-sh4re
- sync_agent_tmpfiles() in hive-priv: generates content, writes atomically,
calls systemd-tmpfiles --create to apply immediately
- priv_client::sync_agent_tmpfiles() wrapper
- lifecycle::sync_tmpfiles() best-effort helper (list + priv call)
- Call sites: hive-c0re startup, handle_spawn success, destroy success
Replace the hardcoded FORGE_HTTP const with forge_http_base() which
reads HIVE_FORGE_URL from the environment (already set unconditionally
by hive-c0re.nix to http://<forge.domain>). Add forge_git_url() helper
that inserts core:<token> credentials between scheme and authority for
git push/clone URLs.
All call sites updated:
- forge/mod.rs: api() OnceLock + new forge_git_url/forge_http_base fns
- forge/repos.rs: push_meta, push_config, ensure_meta_remote
- forge/pr_merge.rs: tokenised_repo_url delegate + test loosened
- workers/knowledge.rs: clone + push URLs
- socket_server/mod.rs: clone_url in RepoCreated response
No new env var: HIVE_FORGE_URL was already the right knob (mara).
Closes#1868. Closes#2174 (this supersedes the operators-team fix from
the closed#2218, which is re-applied in the ensure_operators_team call
that was already merged separately).
