argus caught it: the function-level /// comment still described the
old submit-await-submit graceful approach after the code moved to
one atomic GracefulRestart DAG.
mara's review on #2436: no submit-await-submit composition, even
server-side. Adds Template::GracefulRestart (Signal -> Drain ->
StopForUpdate -> Reconcile, wanted=Up) mirroring how Restart already
does StopForUpdate -> Reconcile, plus submit::graceful_restart and
templates::graceful_restart. handle_restart_scoped now submits exactly
one DAG per agent up front for both the graceful and non-graceful
case -- no await_dags in the loop anymore.
hivectl restart --agent NAME previously composed stop() then start() as
two separate client-side daemon calls glued by CLI-process control flow
— not one DAG, and a dropped hivectl connection mid-restart (ssh drop,
Ctrl-C) left the agent stopped with no automatic follow-up. mara flagged
this as the first target for the 'dagify hivectl commands' issue.
New HostRequest::RestartScoped{scope, graceful} handles it server-side:
each targeted agent now rides exactly one atomic Restart-template DAG
(same one hivectl agents restart / restart-all already use) in the
common non-graceful case. --graceful has no single-DAG template yet, so
it submits the graceful-stop DAGs, awaits them server-side, then submits
the start DAGs — still one daemon call end to end, just not yet a single
DAG (noted as a follow-up). Infra containers restart synchronously as
before (no lease/DAG concept for them).
CLI-side restart() now just makes the one call + waits, same output
shape as before via render_lifecycle.
Split the priv-socket wire types (PrivRequest/PrivResponse/PrivEvent and
friends) out of hive-sh4re into their own hive-priv-sock crate, mirroring
the existing hive-host-sock split. hive-priv — the root-privileged
helper — now depends on just this narrow protocol crate instead of the
much larger daemon-shared crate, shrinking its dependency surface and
making the privsep boundary easier to audit. No server/client
implementation lives here, only the wire contract; hive-c0re still
depends on hive-sh4re directly for everything else.
dispatch was 101 lines (1 over limit) due to the AgentStatus arm.
Extract it to a dedicated handle_agent_status helper to bring dispatch
under the 100-line lint limit without the allow attribute.
Per mara review comment on PR #2379.
- Backtick-quote `pull_request` in doc comments (4x doc_markdown)
- Add #[allow(clippy::too_many_lines)] to server::dispatch (101/100;
+1 line from submit_kind fetched_sha param in 5dd0a36f)
Two hardening items from argus's review of #2374:
1. PR state check at submission:
- Add `pr_is_open(repo, pr)` to forge/pr_merge.rs using
`repo_get_pull_request` + `StateType` — early error if the PR is
already closed or merged instead of queuing a card that fails later
- Call it in `submit_merge_config_pr` before fetching the head sha
2. Atomic fetched_sha INSERT:
- Add `fetched_sha: Option<&str>` to `Approvals::submit_kind` so
the sha can be included in the INSERT rather than a follow-up UPDATE
- MergeConfigPr already knows the sha before inserting the row
(pr_head_sha runs first) → pass `Some(&sha)`, drop the separate
`set_fetched_sha` call → truly atomic
- ApplyCommit still needs two writes (sha resolved by git_fetch_to_tag
after the row exists) → pass `None`, `set_fetched_sha` unchanged
- All other callers (InitConfig, Spawn, UpdateMetaInputs,
SchedulePrompt) pass `None` — no behavioural change
- Add `fetched_sha_in_insert_is_readable_via_get` test covering the
MergeConfigPr path
Collapse the scattered ensure_agent_runtime_dir calls into the lifecycle
functions themselves so callers have a single responsibility:
- lifecycle::spawn: calls ensure_agent_runtime_dir before write_dropins.
Callers (handle_spawn, ensure_root_agent) no longer need a separate
preamble step.
- lifecycle::rebuild_no_meta spawn path: calls ensure_agent_runtime_dir
before write_dropins. apply_commit / merge_config_pr flows no longer
need a manual ensure_agent_runtime_dir.
- run_create (job-queue): drops ensure_agent_runtime_dir + register_agent.
The tail Reconcile's converge_start_preamble handles the runtime dir
and mcp_sockets::spawn_poll handles the listener. Create stays purely
'provision + create', not 'create + start'.
- handle_spawn (server.rs): drops manual preamble; lifecycle::spawn owns it.
Drops unneeded unregister_agent on failure (supervisor handles listener).
- ensure_root_agent (auto_update.rs): drops manual ensure_agent_runtime_dir.
- actions.rs apply_commit / merge_config_pr: drop manual
ensure_agent_runtime_dir; rebuild_no_meta's spawn path handles it.
Result: ensure_agent_runtime_dir lives in exactly two places —
lifecycle::spawn (direct spawn) and converge_start_preamble (start/reconcile
path). All other callers are clean call sites.
- lifecycle::ensure_agent_runtime_dir(name): pure filesystem op, no
Coordinator dep. Creates /run/hyperhive/agents/<name> without touching
the MCP listener map.
- workers/mcp_sockets::spawn_poll(coord): 10 s reconcile loop (same shape
as agent_sockets::spawn_poll). Converges 'agent running => MCP listener
bound'. First tick is immediate so hive-c0re restarts re-register all
running agents without waiting a full interval. Fixes the dead-listener-
after-daemon-restart gap.
- All ensure_runtime() call sites updated:
- Prebuild/Swap/WriteDropin: Coordinator::agent_dir() (pure, no IO)
- Reconcile-Start: ensure_agent_runtime_dir + agent_dir (dir may be
missing after reboot; listener deferred to supervisor)
- run_create / handle_spawn: ensure_agent_runtime_dir + register_agent
(eager on first spawn so socket ready before harness first turn)
- apply_commit / merge_config_pr: ensure_agent_runtime_dir + agent_dir
- Manager (auto_update): ensure_agent_runtime_dir + agent_dir
(manager has no MCP listener; socket_server::start_manager owns it)
- ensure_runtime() retained in Coordinator with updated doc pointing at
the preferred split form. No callers remain outside tests.
Root cause of the boot outage: container@h-* units try to start before
hive-c0re reaches ensure_runtime, so bind-mount source dirs are missing.
Fix: hive-c0re (via hive-priv, which runs as root) writes
/etc/tmpfiles.d/hyperhive-agents.conf whenever the agent set changes.
systemd-tmpfiles-setup.service (sysinit.target) reads it at every boot
BEFORE any container units start, pre-creating:
/run/hyperhive/agents/<name> — MCP socket dir (bind -> /run/hive)
/run/hive-agent/<name> — web socket dir (bind -> /run/hive-agent)
This alone removes the outage class: even if hive-c0re is slow to start,
the bind-mount sources exist and container units can activate.
Added:
- PrivRequest::SyncAgentTmpfiles { agents } in hive-sh4re
- sync_agent_tmpfiles() in hive-priv: generates content, writes atomically,
calls systemd-tmpfiles --create to apply immediately
- priv_client::sync_agent_tmpfiles() wrapper
- lifecycle::sync_tmpfiles() best-effort helper (list + priv call)
- Call sites: hive-c0re startup, handle_spawn success, destroy success
- subvol upgrade waits for the queued stop DAG before migrating (was
snapshotting + swapping state under a live bind mount) and for the
restart job after
- history trim gets a 5-min grace for fresh terminals so broad
stop/start waits can't miss a failed DAG evicted by the per-template
cap (cap still applies past the grace)
- restart-all returns its DAG ids so hivectl actually waits
- hard stops await their agent DAGs (bounded) before infra goes down,
restoring the agents-before-infra invariant
- hivectl wait uses node-level terminality so the after-any recovery
reconcile is watched to completion; infra render errors no longer
skip watching already-queued agent DAGs
- fold hive-bash-mcp's last local now_unix into wire_time
every agent lifecycle verb on the admin socket (rebuild / restart /
restart-all / kill / stop / start) now submits job-queue DAGs and
returns their ids; hivectl polls the new HostRequest::QueueDag and
prints a live node-chain progress line per DAG (fan-out children
included), exiting non-zero on failure — --no-wait opts out. DagView
and the queue wire enums move to hive_sh4re::jobs (wire types live in
the shared crate); the last fused rebuild path (lifecycle::rebuild)
is gone. tracker: #2166
- deploy-window gate (meta::exclusive) + path-limited meta commits:
a perm/lock/topology commit can no longer sweep an ApprovalDeploy's
staged flake.lock and neuter abort_deploy (regression test included)
- cancel surfaces now buffer terminal roll-ups the scheduler drains,
so a queued approval DAG cancelled by the operator resolves its
approval instead of dangling, and cancelled power ops revert their
wanted flip to the observed state
- hivectl restart / restart-all ride the queue (lease serialization,
transient guard) and restart sets wanted=Up like the old kill+start
- exactly one Rebuilt event per rebuild DAG, emitted at terminal
- StopForUpdate pre-seeds a missing agent_power row from the pre-stop
observation so a rebuild can't strand an unknown agent offline
- history trim keeps terminal fan-out parents with live children
- audit_log back on db::open; swarm.js badge for reconcile DAGs
jobs are now DAGs of primitive nodes (prebuild, stop-for-update, swap,
reconcile, signal, drain, ...) driven by one scheduler with N build
slots + per-agent lifecycle leases. per-agent power intent (wanted
up/offline) is durable in agent_power.sqlite; Reconcile nodes converge
observed state to it. kills the graceful-stop watcher thread, the
deferred-start follow-up, and the cascade pre-enqueue (fan-out on
MetaLock completion instead). tracker: #2166
Per argus review: std::env::var(..).ok() yields Some("") for an empty
env value, so require_hive_domain would 'succeed' with an empty domain
and emit invalid nix (swarm.peers."" = …). Filter empty so it resolves
to None → the CLI's clear 'domain unset' error fires instead.
Per operator review: hivectl shouldn't make the operator retype this
hive's own domain. Add a HostRequest::HiveDomain admin-socket query
(c0re answers from HYPERHIVE_HIVE_DOMAIN, already in its service env) and
a domain field on HostResponse.
hivectl peer-config now resolves the domain as: --domain override (kept
for offline/scripted use), else the daemon query; errors with a clear
hint when neither resolves. wg init resolves it best-effort too, so it
prints the hand-over peer-config block without --domain (skipped, not
failed, when the daemon is unreachable).
Regenerated docs/tools/hivectl-cli.md.
Replace the stringly-typed infra-control path with an InfraContainer enum
(Ci/Forge/Gateway/Matrix). The variants are the allowlist: serde rejects any
unknown or unsafe name (hive-c0re has no variant) at the wire boundary, so
hive-priv no longer needs a root-side SIBLING_CONTAINERS.contains() check on
ControlInfraContainer — the type enforces it, and 'the daemon can't stop
itself' is a compile-time guarantee.
- priv_proto: InfraContainer enum; manual Serialize/Deserialize + FromStr +
unit_name() all key off one mapping, so the wire form ('hive-ci', …) is
unchanged and there's no drift. ControlInfraContainer.container: String ->
InfraContainer.
- hive-priv / priv_client / server.rs: thread the enum; scoped_infra returns
Vec<InfraContainer>; the control handler uses unit_name().
- agent_server: the infra_admin restart gate parses the name via FromStr
instead of a slice .contains().
- SIBLING_CONTAINERS stays (validate_container_name/_system_name still use it
for journals / general container validation); a test keeps the enum and the
slice in lockstep.
Per review: c0re expands the LifecycleScope to explicit container-name
lists (scoped_agents / scoped_infra) in the dispatch arm, then hands those
lists to handle_stop / handle_start. The 'all agents' flag no longer flows
past the resolution boundary, so downstream consumers (incl. the future
graceful-stop queue) always operate on concrete names. CLI --agents flag
unchanged.
`hivectl stop` brings the whole hive down in one operator action — all
sub-agents plus the ci/forge/gateway/matrix infra containers — and
`hivectl start` brings it back up. Scope flags (--agents, --agent <name>,
--ci, --forge, --gateway, --matrix) narrow the set; a bare invocation
targets everything. hive-c0re never stops itself.
- hive-sh4re: HostRequest::{Stop,Start} + LifecycleScope wire type;
priv_proto InfraAction + ControlInfraContainer + the
CONTROLLABLE_INFRA_CONTAINERS allowlist (adds hive-matrix, excludes
hive-c0re).
- hive-priv: control_infra_container handler (systemctl <verb>
container@<name>, allowlist-validated root-side).
- hive-c0re: handle_stop / handle_start fan out agents via lifecycle and
infra via hive-priv; per-target failures are aggregated. Infra
systemctl routes through hive-priv (the privsep boundary).
- The --graceful flag is threaded through Stop now; the per-agent quiesce
itself lands with the graceful-agent-stop work.
Per mara's direction: both nixpkgs and nixpkgs-unstable are now
top-level meta flake inputs with explicit store-path URLs. Hyperhive
follows them rather than the other way around:
inputs.nixpkgs.url = "path:${pkgs.path}";
inputs.nixpkgs-unstable.url = "path:${nixpkgs-unstable}";
inputs.hyperhive.url = "...";
inputs.hyperhive.inputs.nixpkgs.follows = "nixpkgs";
inputs.hyperhive.inputs.nixpkgs-unstable.follows = "nixpkgs-unstable";
New NixOS host options (auto-set at build time, overridable):
services.hyperhive.c0re.nixpkgsFlake
default: "path:${pkgs.path}" — host's evaluated nixpkgs.
services.hyperhive.c0re.nixpkgsUnstableFlake
default: "path:${nixpkgs-unstable}" from hyperhive's flake.nix —
the channel that carries claude-code. Operators can override to
track a different unstable snapshot.
Legacy fallback (both args empty) preserved for backward compat.
Two new Rust tests cover the full-URL and fallback paths.
meta flake was using `nixpkgs.follows = "hyperhive/nixpkgs"` but
`hyperhive` is a store-path input, so nix resolves hyperhive's own
pinned lock rather than the host's follows-substituted version.
When an operator sets `inputs.hyperhive.inputs.nixpkgs.follows =
"nixpkgs"` in their host flake, the meta flake was silently ignoring
it and using hyperhive's pinned nixpkgs instead.
Fix: hive-c0re.nix injects `--nixpkgs-flake path:${pkgs.path}` into
the daemon's ExecStart. `pkgs` IS the host's nixpkgs when follows is
set; otherwise it's hyperhive's own pin — so the meta flake gets the
right nixpkgs in both cases. render_flake emits `nixpkgs.url = "..."`
(explicit) when nixpkgs_flake is non-empty, falling back to the old
`follows` form when empty for backward compat.
Add a systemd.sockets.hive-c0re unit that holds /run/hyperhive/host.sock
before hive-c0re starts. hive-c0re serve() detects LISTEN_FDS via the
listenfd crate and accepts the systemd-handed fd instead of calling bind().
Falls back to the existing bind path when LISTEN_FDS is absent so direct
invocation and CI are unaffected.
Benefits: hivectl can connect the moment the socket unit activates (no
racy window), and a hive-c0re restart never drops the socket inode.
Part of the docs-migration chore (issue #708). Remove GitHub issue
numbers from inline comments, option descriptions, and rustdoc —
these are contextless noise for anyone reading the code without
access to the original discussions. Replace with prose that captures
the same rationale directly.
No functional change. Build still clean (cargo check passes).
