diff --git a/CLAUDE.md b/CLAUDE.md index 631ef259..aea318d1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -31,18 +31,10 @@ hive-c0re/ host daemon + CLI (one binary, subcommand-dispatched) src/coordinator.rs shared state (broker/approvals/questions/transient/ sockets) + tombstone enumeration + kick_agent src/actions.rs approve/deny/destroy (transient-aware) - src/auto_update.rs startup rebuild scan + ensure_manager + - meta::lock_update_hyperhive bump - src/lifecycle.rs `nixos-container` shellouts; per-agent applied - + proposed git repo seeding; tag plumbing - src/meta.rs single hive-c0re-owned flake at /var/lib/ - hyperhive/meta/ — sync_agents, two-phase - prepare/finalize/abort, lock_update_* - src/migrate.rs startup auto-migration from pre-meta layout - (idempotent, marker-guarded phase 4) + src/auto_update.rs startup rebuild scan + ensure_manager + src/lifecycle.rs `nixos-container` shellouts, per-agent flake generator src/dashboard.rs axum HTTP: static shell + /api/state JSON + actions + journald viewer + bind-with-retry (SO_REUSEADDR) - + deployed_sha chip per container assets/ index.html, dashboard.css, app.js (include_str!) hive-ag3nt/ in-container harness crate; produces TWO binaries @@ -122,40 +114,29 @@ read them à la carte. In-flight or recent context that hasn't earned a section yet. Prune freely. -- **Just landed:** meta-flake overhaul. Each agent's applied - repo is a tiny module-only flake (`nixosModules.default = - import ./agent.nix`); `agent.nix` is a plain NixOS module - function — no extendModules, no hyperhive input visible to - the manager. A single hive-c0re-owned repo at - `/var/lib/hyperhive/meta/` declares one input per agent - (pointing at that agent's applied repo via `git+file://`) - and one `nixosConfigurations.` output per agent, - wrapping `inputs.agent-.nixosModules.default` with the - identity + `HIVE_PORT` / `HIVE_LABEL` / - `HIVE_DASHBOARD_PORT` injection. Containers run against - `meta#`. Every approve runs `nix flake lock - --update-input agent-` (two-phase: prepare on the - build path, finalize/abort on the result) — meta's git - log is the system-wide deploy audit trail; failures and - denials live as annotated tags in applied. The manager - has `/applied` and `/meta` RO-bound and the `applied` - remote pre-wired in every proposed repo so `git fetch - applied`, `git show applied/refs/tags/deployed/`, - `git -C /meta log --oneline`, `cat /meta/flake.lock` - all just work. Migration runs idempotently on - hive-c0re startup (`HIVE_SKIP_META_MIGRATION=1` skips it): - rewrites pre-meta applied flakes to module-only, wires - the proposed remote, seeds meta, and repoints every - container at `meta#` (guarded by a marker so the - expensive phase only runs once). -- **Just landed (prior overhaul still underneath):** tag- - driven config-apply. Two-repo split (proposed = manager - RW, applied = core-only); `request_apply_commit` fetches - the manager's commit into applied and pins it as - `proposal/`; approve / deny / build walk through - tags on the same commit; `applied/main` only fast- - forwards on `deployed/`. `failed/` + `denied/` are - annotated. See `docs/approvals.md`. +- **Just landed:** tag-driven config-apply overhaul. Two-repo + split kept (proposed = manager RW, applied = core-only) for + safety. New flow: at `request_apply_commit` time hive-c0re + fetches the manager's commit into applied and pins it as + `proposal/`; the manager-side repo is then irrelevant + for that approval. Approve / deny / build walk through more + tags (`approved/`, `building/`, `deployed/`, `failed/`, + `denied/`) on the same commit; `applied/main` only + fast-forwards on `deployed/`. `failed/` and `denied/` are + annotated — body is the build error or the operator's deny + note respectively. Manager has `/applied` bind-mounted RO + (whole tree) so `git fetch /applied//.git + 'refs/tags/*:refs/tags/applied/*'` mirrors every relevant + tag into its proposed clone. `agent.nix` stays the entry + point; the whole tracked tree is now preserved + through apply (arbitrary files supported). The wrapper + `flake.nix` is regenerated by hive-c0re every + spawn/rebuild but never tracked, so the applied log is + exactly the manager's commits in deploy order. Migration: + no in-place — pre-overhaul applied dirs are detected via + the missing `deployed/0` tag and `setup_applied` bails + with `destroy --purge` instructions. See + `docs/approvals.md`. - **Recent (since last compaction):** inline +/- diffs on Write/Edit, send full body via collapsed details, operator cancel + ttl on questions, deny-with-reason, dashboard diff --git a/README.md b/README.md index 5fbcf390..614c3f06 100644 --- a/README.md +++ b/README.md @@ -26,9 +26,8 @@ host (NixOS, runs hive-c0re.service) └── nixos-containers (each bind-mounts its socket dir → /run/hive, │ credentials dir → /root/.claude, │ durable notes dir → /state; - │ manager additionally gets /agents RW, - │ /applied RO (deployed-tag mirror), - │ /meta RO (swarm-wide deploy flake)) + │ manager additionally gets /agents RW + │ + /applied RO for the deployed-tag mirror) │ ├── hm1nd hive-m1nd serve : claude turn loop + │ MCP (send / recv / request_spawn / kill / start / @@ -55,30 +54,21 @@ load; collapsible inbox + collapsible journald viewer + collapsible `agent.nix` viewer per agent on the dashboard. Config changes flow the other way: manager edits files under -`/agents//config/` — `agent.nix` is a plain NixOS module function -`{ config, pkgs, lib, ... }: { ... }`, and arbitrary sibling files in -the commit are preserved → commits → submits the sha via -`request_apply_commit`. Hive-c0re immediately fetches that commit from -the proposed repo into the applied repo and pins it as `proposal/` -— immutable from the manager's side from then on. Operator clicks -◆ APPR0VE → hive-c0re fast-forwards `applied//main` to the proposal, -runs `nix flake lock --update-input agent-` against the host-wide -meta flake at `/var/lib/hyperhive/meta/`, builds via -`nixos-container update --flake meta#`, and either commits -the lock + tags `deployed/` on success or `git restore`s the lock + -annotates `failed/` with the build error + rolls back -`applied//main` on failure. Denials leave a `denied/` annotated -tag carrying the operator's note. - -Meta's git log is the swarm-wide deploy audit trail (one commit per -successful deploy). Per-agent applied repos carry the tag-rich state -machine for inside-baseball decisions. The manager sees both — proposed -repos ship with an `applied` remote pre-wired, and `/meta/` is RO-bound -inside the container — so `git fetch applied`, -`git show applied/refs/tags/deployed/`, `git log /meta`, -`cat /meta/flake.lock` all just work without constructing paths by -hand. See [`docs/approvals.md`](docs/approvals.md) for the full state -machine + lock-flow walkthrough. +`/agents//config/` (`agent.nix` is the entry point, but arbitrary +sibling files in the commit are preserved) → commits → submits the sha +via `request_apply_commit`. Hive-c0re immediately fetches that commit +from the proposed repo into the applied repo and pins it as +`proposal/` — from this moment the proposal is immutable from the +manager's side. Operator clicks ◆ APPR0VE on the dashboard → hive-c0re +moves the working tree to the proposal, runs `nixos-container update`, +and either fast-forwards `applied/main` (tagging `deployed/`) or +annotates `failed/` with the build error and rolls back to the +previous deployed tree. Denials leave a `denied/` annotated tag +carrying the operator's note. The manager sees everything that +shipped (or didn't) via a read-only `/applied//.git` mirror inside +its container; `git show applied/deployed/` etc. is the audit +trail. See [`docs/approvals.md`](docs/approvals.md) for the full tag +state machine. For decisions the manager needs human signal on, `ask_operator(question, options?, multi?)` queues a free-text/checkbox/radio form on the dashboard; the answer arrives later as a `HelperEvent::OperatorAnswered` diff --git a/docs/approvals.md b/docs/approvals.md index de35a2fa..47fa5942 100644 --- a/docs/approvals.md +++ b/docs/approvals.md @@ -37,59 +37,6 @@ step — the operator just sees the name. On approve, hive-c0re creates the container in a background task while the dashboard shows a spinner. -## Meta flake - -The hive-c0re-owned repo at `/var/lib/hyperhive/meta/` -declares one flake input per agent (`agent-.url = -"git+file:///var/lib/hyperhive/applied/"`) and one -`nixosConfigurations.` output per agent. Each output wraps -`inputs.agent-.nixosModules.default` with the identity + -`HIVE_PORT` / `HIVE_LABEL` / `HIVE_DASHBOARD_PORT` injection -module that `setup_applied` used to generate inline. -Containers run against `--flake /var/lib/hyperhive/meta#`. - -Per-deploy lock flow (two-phase, owned by -`actions::run_apply_commit` → `meta::{prepare,finalize,abort} -_deploy`): - -1. `meta::prepare_deploy(name)` runs - `nix flake lock --update-input agent-` without - committing. Working tree of meta now points the input at - `applied//main` (which `run_apply_commit` already - fast-forwarded to `proposal/`). -2. `lifecycle::rebuild_no_meta` runs - `nixos-container update --flake meta#`. Nix - evaluates against the staged lock. -3. On success — `meta::finalize_deploy(name, sha, "deployed/ - ")` stages `flake.lock` and commits with - `deploy deployed/ `. Meta's git log gains - one entry per successful deploy. -4. On failure — `meta::abort_deploy()` runs - `git restore flake.lock` so the meta history shows only - successes; the failure stays as an annotated `failed/` - tag in `applied/`. - -Single-phase variants exist for paths without -rollback semantics: `meta::lock_update_for_rebuild(name)` for -the manual `↻ R3BU1LD` button (commits if the lock changed) -and `meta::lock_update_hyperhive()` for the -auto-update flake-rev bump (one shot before per-agent -rebuilds, commits if the lock changed). - -`meta::sync_agents(hyperhive_flake, dashboard_port, &agents)` -is the idempotent reconciler called by `spawn`, `destroy`, -`rebuild`, and the startup migration. Renders `flake.nix` -from the agent list; if it differs from disk, runs -`nix flake lock` + commits as `regenerate meta flake` (or -`seed meta from N agent(s)` on the very first call). - -The manager has `/meta` RO-bound inside its container: -`git -C /meta log --oneline` is the swarm-wide deploy log, -`cat /meta/flake.lock | jq '.nodes["agent-"].locked'` -resolves which sha each agent is pinned at right now. -Dashboard surfaces the same info as a `deployed:` chip -per container row. - ## Two repos per agent ``` @@ -99,23 +46,17 @@ per container row. # agent.nix is the # convention entry # point; flake.nix is - # tracked boilerplate - # (manager doesn't edit - # it). + # generated and not + # tracked here. /var/lib/hyperhive/applied// applied — core-only ├── .git/ # tag-rich history -├── flake.nix # tracked, fixed -│ # boilerplate exporting -│ # nixosModules.default +├── .gitignore # ignores flake.nix +├── flake.nix # hive-c0re-generated, +│ # untracked, rewritten +│ # on spawn/rebuild only ├── agent.nix # working tree of main └── # also tracked - -/var/lib/hyperhive/meta/ swarm-wide flake — core -├── .git/ # one commit per successful -│ # deploy -├── flake.nix # generated from agent set -└── flake.lock # pins each agent's sha ``` Why two physical repos: the manager's `/agents//config/` is @@ -124,12 +65,13 @@ proposed tree. The applied repo is never bind-mounted (except the read-only `.git` exposure described below) so a destructive move inside the container cannot reach it. -The container's `--flake` ref is `/var/lib/hyperhive/meta#` -(see "Meta flake" above). The agent's own `applied//flake.nix` -is a fixed boilerplate that exports `nixosModules.default = -import ./agent.nix`; the meta flake imports that module and -wraps it with identity + `HIVE_PORT` / `HIVE_LABEL` / -`HIVE_DASHBOARD_PORT`. +The container's `--flake` ref is `#default`. The +generated `flake.nix` extends +`hyperhive.nixosConfigurations.{agent-base|manager}` with +`./agent.nix` plus an inline module setting +`programs.git.config.user` (committer identity = the agent's name) +and `systemd.services..environment` (`HIVE_PORT`, +`HIVE_LABEL`, `HIVE_DASHBOARD_PORT`). ### Tag state machine @@ -151,63 +93,29 @@ approval id to retry. Because tags are first-class git objects, rejected and failed trees stay browsable forever — `git log --tags` in the applied repo is the audit trail. -### Manager view of applied + meta +### Manager view of applied -The manager container gets three host-side bind mounts via -`set_nspawn_flags`: +`/applied/` is a **read-only bind-mount** of +`/var/lib/hyperhive/applied/` (the entire tree) inside the +manager container. The manager fetches tags into its proposed +clone with `git fetch /applied//.git +'refs/tags/*:refs/tags/applied/*'` and `git show` any +deployed / failed / denied tree to see what actually shipped, +what error blocked the last build, or what note the operator +left on a denial. The RO bind means git plumbing inside the +manager cannot corrupt the applied repos — and a single mount +covers every agent (existing + future) without rebuilding the +manager on each spawn. -- `/var/lib/hyperhive/agents/` → `/agents/` (RW) — proposed - repos. Manager edits + commits per-agent config here. -- `/var/lib/hyperhive/applied/` → `/applied/` (RO) — every - agent's authoritative applied repo, including `.git`. -- `/var/lib/hyperhive/meta/` → `/meta/` (RO) — the swarm-wide - deploy flake. +## Migration from the pre-tag scheme -Each proposed repo (`/agents//config/`) is pre-configured -with `applied` as a git remote pointing at -`/applied//.git`. Useful incantations from inside the -manager: - -```sh -git -C /agents//config fetch applied -git -C /agents//config log applied/main --oneline -git -C /agents//config show applied/refs/tags/deployed/ -git -C /agents//config show applied/refs/tags/failed/ # body = build error -git -C /agents//config show applied/refs/tags/denied/ # body = operator note -git -C /agents//config rebase applied/main # base in-flight work on what's deployed - -git -C /meta log --oneline # swarm-wide deploy history -cat /meta/flake.lock | jq '.nodes | with_entries(select(.key | startswith("agent-")))' -``` - -The RO binds block push at the kernel level, so the manager -can only fetch / read — git plumbing inside the container -cannot corrupt either authoritative repo. - -## Migration from the pre-tag / pre-meta schemes - -Both overhauls (tag-driven flow + meta flake) ship in-place -migrations that run on every hive-c0re startup. Idempotent; -each phase is a no-op once already applied. Behaviour: - -- Tag-driven phase: assumes the operator ran the one-shot - `git tag deployed/0 main` script (see commit history / - earlier docs revisions) once per agent. Tagging is - non-destructive: it doesn't touch live containers, state - dirs, or claude creds. -- Meta-flake phase: rewrites each `applied//flake.nix` to - the module-only boilerplate, wires the `applied` remote in - each proposed repo, bootstraps the meta repo from the - current agent list, and `nixos-container update`s every - container at `meta#`. The expensive last step is - guarded by `/var/lib/hyperhive/.meta-migration-done` so - it only runs once across hive-c0re restarts. Set - `HIVE_SKIP_META_MIGRATION=1` on the service to defer. - -No state loss in either migration. claude creds, /state/ -notes, the events DB, proposed history, and applied history -all survive. The manager keeps its session; sub-agents stay -logged in. +There is no in-place migration. Each existing agent must be +purged and re-spawned: `hive-c0re destroy --purge ` (or +PURG3 on the dashboard), then `request_spawn` and the operator +approves the fresh agent. The new agent starts with `deployed/0` +seeded by hive-c0re; the manager's first config edit becomes +`proposal/1` and walks the tag scheme from there. Pre-overhaul +tombstones lose their config history. ## Manager (`hm1nd`) is hive-c0re-managed diff --git a/docs/persistence.md b/docs/persistence.md index aeb90fda..69cfe6b9 100644 --- a/docs/persistence.md +++ b/docs/persistence.md @@ -67,25 +67,8 @@ Under `/var/lib/hyperhive/agents//`: to `/state` inside the container. Under `/var/lib/hyperhive/applied//` — the hive-c0re-only -applied repo. Tracks `flake.nix` (module-only boilerplate; never -edited after first spawn) + `agent.nix` (the actual config; the -manager's edits land here via the approval flow) + any other -files the manager committed. `.git/` carries the proposal / -approved / building / deployed / failed / denied tag history. - -Under `/var/lib/hyperhive/meta/` — the swarm-wide deploy flake. -Single repo for the whole host; `flake.nix` declares one input -per agent + one `nixosConfigurations.` output per agent; -`flake.lock` is the canonical "what's deployed where." The git -log is the deploy audit trail (one commit per successful -deploy or hyperhive bump). Manager has this RO-mounted at -`/meta/`. - -Marker file `/var/lib/hyperhive/.meta-migration-done` is -written by the startup migration after every container has -been repointed at `meta#`. Removing it forces a re-run on -next hive-c0re start (idempotent — only the actual repoint -step would re-fire). +applied repo (`flake.nix` + `agent.nix`) that the container +actually builds from. ## Destroy vs purge diff --git a/hive-ag3nt/prompts/manager.md b/hive-ag3nt/prompts/manager.md index 123b9596..67f854ce 100644 --- a/hive-ag3nt/prompts/manager.md +++ b/hive-ag3nt/prompts/manager.md @@ -14,18 +14,14 @@ Tools (hyperhive surface): Approval boundary: lifecycle ops on *existing* sub-agents (`kill`, `start`, `restart`) are at your discretion — no operator approval. *Creating* a new agent (`request_spawn`) and *changing* any agent's config (`request_apply_commit`) still go through the approval queue. The operator only signs off on changes; you run the day-to-day. -Your own editable config lives at `/agents/hm1nd/config/`; every sub-agent's lives at `/agents//config/`. `agent.nix` is a plain NixOS module function — `{ config, pkgs, lib, ... }: { ... }`. Add packages, services, imports, sibling `.nix` files; the whole committed tree gets deployed together. **Do not edit `flake.nix`** — it's a fixed boilerplate that exports `agent.nix` as `nixosModules.default`; the hive-c0re-owned meta flake at `/meta/` provides the NixOS base and wires identity / `HIVE_PORT` / `HIVE_LABEL` itself. +Your own editable config lives at `/agents/hm1nd/config/`; every sub-agent's lives at `/agents//config/`. `agent.nix` is the entry point but you can commit any extra files (modules, overlays, prompt fragments) and the whole tree gets deployed together. Use file/git tools to edit + commit, then `request_apply_commit`. -Each proposed repo has an `applied` git remote pre-configured pointing at the read-only mirror of what's deployed. Useful patterns: +To see what hive-c0re actually deployed (or rejected, or failed to build), there's a read-only mirror of every agent's applied repo at `/applied//.git`. Useful patterns: -- `git -C /agents//config fetch applied` — refresh the local copy of every deployed/failed/denied tag. -- `git -C /agents//config log applied/main --oneline` — every successful deploy of this agent. -- `git -C /agents//config show applied/refs/tags/deployed/` — the tree that was deployed for approval ``. -- `git -C /agents//config show applied/refs/tags/failed/` — annotated tag body is the build error from a rejected rebuild. -- `git -C /agents//config show applied/refs/tags/denied/` — annotated tag body is the operator's reason for denial. -- `git -C /agents//config rebase applied/main` — base your in-flight work on whatever's actually deployed (useful after a failed/denied pile-up). - -System-wide view: `/meta/` is a read-only mirror of the deployed-agents flake. `git -C /meta log --oneline` is the deploy log for every agent across the swarm; `cat /meta/flake.lock` shows which sha each agent is pinned at right now. +- `git -C /agents//config fetch /applied//.git 'refs/tags/*:refs/tags/applied/*'` — mirror all tags into your proposed clone. +- `git -C /agents//config show applied/deployed/` — see the tree that's currently running. +- `git -C /agents//config show applied/failed/` — annotated tag body is the build error from a rejected rebuild. +- `git -C /agents//config show applied/denied/` — annotated tag body is the operator's reason for denial. Tag scheme on every approval id: `proposal → approved → building → deployed | failed`, plus `denied` as a terminal alternative to `approved`. `applied/main` only advances on `deployed/*`, so a failed build does not corrupt the agent — submit a fix as a new commit and a fresh `request_apply_commit`. diff --git a/hive-c0re/assets/app.js b/hive-c0re/assets/app.js index 58ed9899..d000976f 100644 --- a/hive-c0re/assets/app.js +++ b/hive-c0re/assets/app.js @@ -279,11 +279,6 @@ )); } head.append(el('span', { class: 'meta' }, `${c.container} :${c.port}`)); - if (c.deployed_sha) { - head.append(el('span', - { class: 'meta', title: 'sha currently locked in /meta/flake.lock' }, - `deployed:${c.deployed_sha}`)); - } li.append(head); // ── line 2: action buttons ─────────────────────────────────── diff --git a/hive-c0re/src/actions.rs b/hive-c0re/src/actions.rs index 45967434..fc18d292 100644 --- a/hive-c0re/src/actions.rs +++ b/hive-c0re/src/actions.rs @@ -147,7 +147,6 @@ async fn run_apply_commit( ) -> (Result<()>, Option) { let id = approval.id; let proposal_ref = format!("refs/tags/proposal/{id}"); - // Defensive: submit-time should have planted proposal/, but if // the row was migrated from an older schema or the tag got pruned // we fail early with a clear note rather than building a stale @@ -160,14 +159,6 @@ async fn run_apply_commit( None, ); } - - // Capture the currently-deployed sha so we can roll applied/main - // (and the meta lock indirectly) back if the build fails. - let prev_main_sha = match lifecycle::git_rev_parse(applied_dir, "refs/heads/main").await { - Ok(s) => s, - Err(e) => return (Err(anyhow::anyhow!("read applied/main: {e:#}")), None), - }; - if let Err(e) = lifecycle::git_tag(applied_dir, &format!("approved/{id}"), &proposal_ref).await { return (Err(anyhow::anyhow!("plant approved/{id}: {e:#}")), None); @@ -176,71 +167,41 @@ async fn run_apply_commit( { return (Err(anyhow::anyhow!("plant building/{id}: {e:#}")), None); } - - // Fast-forward applied/main to proposal/ + sync the working - // tree. Meta input pins `?ref=main`, so this is what makes nix - // re-lock to the proposal commit on the prepare_deploy step - // below. On build failure we roll main back to prev_main_sha so - // a crash leaves the agent on its last-good tree. - if let Err(e) = - lifecycle::git_update_ref(applied_dir, "refs/heads/main", &proposal_ref).await - { + if let Err(e) = lifecycle::git_read_tree_reset(applied_dir, &proposal_ref).await { return ( - Err(anyhow::anyhow!("ff main to {proposal_ref}: {e:#}")), - None, - ); - } - if let Err(e) = lifecycle::git_read_tree_reset(applied_dir, "refs/heads/main").await { - // main is ahead; working tree didn't sync. Roll main back to - // keep the two consistent before bailing. - let _ = lifecycle::git_update_ref(applied_dir, "refs/heads/main", &prev_main_sha).await; - return ( - Err(anyhow::anyhow!("read-tree to main: {e:#}")), + Err(anyhow::anyhow!("read-tree to {proposal_ref}: {e:#}")), None, ); } - // Phase 1 of the meta two-phase deploy: relock without committing. - if let Err(e) = crate::meta::prepare_deploy(&approval.agent).await { - let _ = lifecycle::git_update_ref(applied_dir, "refs/heads/main", &prev_main_sha).await; - let _ = lifecycle::git_read_tree_reset(applied_dir, "refs/heads/main").await; - return ( - Err(anyhow::anyhow!("meta prepare_deploy: {e:#}")), - None, - ); - } - - // Container-level rebuild against meta#. - let build_result = lifecycle::rebuild_no_meta( + let rebuild_result = lifecycle::rebuild( &approval.agent, + &coord.hyperhive_flake, agent_dir, applied_dir, claude_dir, notes_dir, + coord.dashboard_port, ) .await; - match build_result { + match rebuild_result { Ok(()) => { let tag = format!("deployed/{id}"); if let Err(e) = lifecycle::git_tag(applied_dir, &tag, &proposal_ref).await { tracing::warn!(agent = %approval.agent, %id, error = ?e, "plant deployed tag failed"); } - if let Err(e) = crate::meta::finalize_deploy( - &approval.agent, - approval.fetched_sha.as_deref().unwrap_or(&proposal_ref), - &tag, - ) - .await + if let Err(e) = + lifecycle::git_update_ref(applied_dir, "refs/heads/main", &proposal_ref).await { - // The build itself succeeded — meta lock landed but - // couldn't be committed. Surface as a soft warn so the - // operator can git-commit by hand if they care. - tracing::warn!(agent = %approval.agent, %id, error = ?e, "meta finalize_deploy failed"); + // Working tree already matches proposal/, but main + // didn't advance — surface as a build failure so the + // operator notices the desync. + return ( + Err(anyhow::anyhow!("ff main to {proposal_ref}: {e:#}")), + Some(tag), + ); } - // Don't ignore the coord pointer — keeps the borrow alive - // for future tracing additions without re-plumbing. - let _ = coord; (Ok(()), Some(tag)) } Err(e) => { @@ -251,22 +212,13 @@ async fn run_apply_commit( { tracing::warn!(agent = %approval.agent, %id, error = ?te, "annotate failed tag failed"); } - // Roll main back to last known-good so the on-disk state - // matches what nixos-container last successfully built. - if let Err(re) = - lifecycle::git_update_ref(applied_dir, "refs/heads/main", &prev_main_sha).await - { - tracing::warn!(agent = %approval.agent, %id, error = ?re, "main rollback failed"); - } + // Roll working tree back to last known-good main so the + // on-disk state matches what nixos-container last + // successfully built. main hasn't moved, so this is just + // a content reset. if let Err(re) = lifecycle::git_read_tree_reset(applied_dir, "refs/heads/main").await { tracing::warn!(agent = %approval.agent, %id, error = ?re, "rollback read-tree failed"); } - // Drop the staged meta lock change so the deploy log - // only ever shows successes. - if let Err(ae) = crate::meta::abort_deploy().await { - tracing::warn!(agent = %approval.agent, %id, error = ?ae, "meta abort_deploy failed"); - } - let _ = coord; (Err(e), Some(tag)) } } @@ -309,14 +261,6 @@ pub async fn destroy(coord: &Coordinator, name: &str, purge: bool) -> Result<()> } } } - // Meta flake: drop the agent's input + nixosConfiguration so a - // future spawn under the same name re-seeds cleanly, and so the - // meta lock doesn't reference a vanished applied repo. Log + keep - // going on failure — destroy already succeeded at the - // nixos-container level, the meta repo is just bookkeeping. - if let Err(e) = sync_meta_after_lifecycle(coord).await { - tracing::warn!(error = ?e, %name, "meta sync after destroy failed"); - } let _ = coord.approvals.fail_pending_for_agent( name, if purge { @@ -332,14 +276,6 @@ pub async fn destroy(coord: &Coordinator, name: &str, purge: bool) -> Result<()> Ok(()) } -/// Rerender the meta flake from whatever containers still exist on -/// disk. Called after lifecycle ops that change the agent set (today: -/// destroy). Idempotent — a no-op when nothing changed. -async fn sync_meta_after_lifecycle(coord: &Coordinator) -> Result<()> { - let agents = lifecycle::agents_for_meta_listing().await?; - crate::meta::sync_agents(&coord.hyperhive_flake, coord.dashboard_port, &agents).await -} - pub async fn deny(coord: &Coordinator, id: i64, note: Option<&str>) -> Result<()> { let approval = coord.approvals.get(id)?; coord.approvals.mark_denied(id, note)?; diff --git a/hive-c0re/src/auto_update.rs b/hive-c0re/src/auto_update.rs index ad3db220..bce2871f 100644 --- a/hive-c0re/src/auto_update.rs +++ b/hive-c0re/src/auto_update.rs @@ -159,13 +159,6 @@ pub async fn run(coord: Arc) -> Result<()> { }; tracing::info!(rev = %current_rev, "auto-update: scanning agents"); - // Bump meta's hyperhive input up-front so the per-agent rebuilds - // below build against the new base. Failure here is logged but - // not fatal — individual rebuilds will surface concrete errors. - if let Err(e) = crate::meta::lock_update_hyperhive().await { - tracing::warn!(error = ?e, "auto-update: meta lock_update_hyperhive failed"); - } - let containers = match lifecycle::list().await { Ok(c) => c, Err(e) => { diff --git a/hive-c0re/src/dashboard.rs b/hive-c0re/src/dashboard.rs index 210b9766..298ba3e3 100644 --- a/hive-c0re/src/dashboard.rs +++ b/hive-c0re/src/dashboard.rs @@ -187,12 +187,6 @@ struct ContainerView { /// disable other buttons. #[serde(skip_serializing_if = "Option::is_none")] pending: Option<&'static str>, - /// First 12 chars of the sha the meta flake currently has locked - /// for this agent's input. Reflects what's actually deployed; can - /// differ from `applied//main` only between - /// `meta::prepare_deploy` and `finalize_deploy` (≤ build duration). - #[serde(skip_serializing_if = "Option::is_none")] - deployed_sha: Option, } #[derive(Serialize)] @@ -287,7 +281,6 @@ async fn build_container_views( ) -> (Vec, bool) { let mut out = Vec::new(); let mut any_stale = false; - let locked = read_meta_locked_revs(); for c in raw_containers { let (logical, is_manager) = if c == MANAGER_NAME { (MANAGER_NAME.to_owned(), true) @@ -306,9 +299,6 @@ async fn build_container_views( let pending = transient_snapshot .get(&logical) .map(|st| transient_label(st.kind)); - let deployed_sha = locked - .get(&format!("agent-{logical}")) - .map(|s| s[..s.len().min(12)].to_owned()); out.push(ContainerView { port: lifecycle::agent_web_port(&logical), running: lifecycle::is_running(&logical).await, @@ -318,39 +308,11 @@ async fn build_container_views( needs_update, needs_login, pending, - deployed_sha, }); } (out, any_stale) } -/// Parse `/var/lib/hyperhive/meta/flake.lock` into a map of node name -/// (`agent-`, `hyperhive`) → locked sha. Missing / unparsable lock -/// yields an empty map so the dashboard degrades gracefully when the -/// meta repo hasn't been seeded yet. -fn read_meta_locked_revs() -> std::collections::HashMap { - let mut out = std::collections::HashMap::new(); - let Ok(raw) = std::fs::read_to_string("/var/lib/hyperhive/meta/flake.lock") else { - return out; - }; - let Ok(json) = serde_json::from_str::(&raw) else { - return out; - }; - let Some(nodes) = json.get("nodes").and_then(|v| v.as_object()) else { - return out; - }; - for (name, node) in nodes { - if let Some(rev) = node - .get("locked") - .and_then(|v| v.get("rev")) - .and_then(|v| v.as_str()) - { - out.insert(name.clone(), rev.to_owned()); - } - } - out -} - /// Transient state for agents whose container does NOT yet exist /// (`Spawning`). Lifecycle ops on existing containers surface as /// `ContainerView.pending` inline; this list only catches pre-creation. diff --git a/hive-c0re/src/lifecycle.rs b/hive-c0re/src/lifecycle.rs index f3e72daf..9a1fed3c 100644 --- a/hive-c0re/src/lifecycle.rs +++ b/hive-c0re/src/lifecycle.rs @@ -80,6 +80,17 @@ pub fn is_manager(name: &str) -> bool { name == MANAGER_NAME } +/// The nixosConfiguration in the hyperhive flake the agent's `flake.nix` +/// extends. Manager → `manager`; everyone else → `agent-base`. +#[must_use] +pub fn flake_base(name: &str) -> &'static str { + if is_manager(name) { + "manager" + } else { + "agent-base" + } +} + fn validate(name: &str) -> Result<()> { if name.is_empty() { bail!("agent name must not be empty"); @@ -141,16 +152,18 @@ pub async fn spawn( ); } setup_proposed(proposed_dir, name).await?; - setup_applied(applied_dir, Some(proposed_dir), name).await?; + setup_applied( + applied_dir, + Some(proposed_dir), + name, + hyperhive_flake, + dashboard_port, + ) + .await?; ensure_claude_dir(claude_dir)?; ensure_state_dir(notes_dir)?; - // Meta flake gets the new agent's input + nixosConfiguration - // before `nixos-container create` so the `--flake meta#` - // ref resolves. - let agents = agents_after_spawn(name).await?; - crate::meta::sync_agents(hyperhive_flake, dashboard_port, &agents).await?; let container = container_name(name); - let flake_ref = format!("{}#{name}", crate::meta::meta_dir().display()); + let flake_ref = format!("{}#default", applied_dir.display()); run(&["create", &container, "--flake", &flake_ref]).await?; set_nspawn_flags(&container, agent_dir, claude_dir, notes_dir)?; set_resource_limits(&container)?; @@ -158,55 +171,6 @@ pub async fn spawn( run(&["start", &container]).await } -/// Build the `AgentSpec` list for the meta flake from `nixos-container -/// list` + a hypothetical extra name not yet in the list (for spawn -/// where the new agent's container doesn't exist yet). Pass empty -/// `name_to_add` from rebuild paths where the agent is already in the -/// container list. -async fn agents_for_meta(name_to_add: Option<&str>) -> Result> { - let containers = list().await.unwrap_or_default(); - let mut out: Vec = containers - .into_iter() - .filter_map(|c| { - let (name, is_manager) = if c == MANAGER_NAME { - (MANAGER_NAME.to_owned(), true) - } else if let Some(n) = c.strip_prefix(AGENT_PREFIX) { - (n.to_owned(), false) - } else { - return None; - }; - Some(crate::meta::AgentSpec { - port: agent_web_port(&name), - name, - is_manager, - }) - }) - .collect(); - if let Some(extra) = name_to_add - && !out.iter().any(|a| a.name == extra) - { - out.push(crate::meta::AgentSpec { - name: extra.to_owned(), - is_manager: is_manager(extra), - port: agent_web_port(extra), - }); - } - out.sort_by(|a, b| a.name.cmp(&b.name)); - Ok(out) -} - -async fn agents_after_spawn(name: &str) -> Result> { - agents_for_meta(Some(name)).await -} - -/// Public enumeration of currently-existing agents (whatever -/// `nixos-container list` says), sorted, no extras. For callers -/// outside this module that need to reseed meta after lifecycle -/// changes — destroy, startup reconciliation, etc. -pub async fn agents_for_meta_listing() -> Result> { - agents_for_meta(None).await -} - pub async fn kill(name: &str) -> Result<()> { validate(name)?; let container = container_name(name); @@ -265,32 +229,6 @@ pub async fn rebuild( claude_dir: &Path, notes_dir: &Path, dashboard_port: u16, -) -> Result<()> { - // Sync the meta flake (idempotent — no-op when the rendered - // flake matches disk) so a manual rebuild from the dashboard - // can also recover from a divergent meta repo (e.g. an agent - // got added directly via `nixos-container create` outside - // hive-c0re). - let agents = agents_for_meta(None).await?; - crate::meta::sync_agents(hyperhive_flake, dashboard_port, &agents).await?; - // Then bump just this agent's input — picks up whatever - // `applied//main` currently points at (deployed/). - // Commits the lock if it changed. - crate::meta::lock_update_for_rebuild(name).await?; - rebuild_no_meta(name, agent_dir, applied_dir, claude_dir, notes_dir).await -} - -/// Container-level rebuild without touching the meta repo. Callers -/// that own the meta side themselves (`actions::run_apply_commit` -/// drives meta through the two-phase prepare/finalize/abort flow) -/// use this directly. Public `rebuild` wraps it with idempotent meta -/// sync + lock-bump-and-commit. -pub async fn rebuild_no_meta( - name: &str, - agent_dir: &Path, - applied_dir: &Path, - claude_dir: &Path, - notes_dir: &Path, ) -> Result<()> { validate(name)?; if let Some(other) = port_collision(name).await { @@ -299,11 +237,11 @@ pub async fn rebuild_no_meta( agent_web_port(name) ); } - setup_applied(applied_dir, None, name).await?; + setup_applied(applied_dir, None, name, hyperhive_flake, dashboard_port).await?; ensure_claude_dir(claude_dir)?; ensure_state_dir(notes_dir)?; let container = container_name(name); - let flake_ref = format!("{}#{name}", crate::meta::meta_dir().display()); + let flake_ref = format!("{}#default", applied_dir.display()); set_nspawn_flags(&container, agent_dir, claude_dir, notes_dir)?; set_resource_limits(&container)?; systemd_daemon_reload().await?; @@ -334,84 +272,51 @@ pub async fn list() -> Result> { .collect()) } -/// Initialize the manager-editable proposed repo. Seeds two tracked -/// files: `agent.nix` (the module the manager edits) and `flake.nix` -/// (the boilerplate that lets the meta flake import this repo as an -/// input — meta locks at a specific sha and reads -/// `nixosModules.default`, so `flake.nix` must be in the commit). The -/// manager shouldn't edit `flake.nix` (the prompt says so) but it's -/// visible so they can introspect. -/// -/// Touched by hive-c0re only on first spawn — never again — so the -/// manager can't be surprised by hive-c0re commits or working-tree -/// resets. +/// Initialize the manager-editable proposed repo. Contains only `agent.nix` +/// (the file the manager edits). Touched by hive-c0re only on first spawn — +/// never again — so the manager can't be surprised by hive-c0re commits or +/// working-tree resets. pub async fn setup_proposed(proposed_dir: &Path, name: &str) -> Result<()> { - let fresh = !proposed_dir.join(".git").exists(); - if fresh { - std::fs::create_dir_all(proposed_dir) - .with_context(|| format!("create {}", proposed_dir.display()))?; - let agent_path = proposed_dir.join("agent.nix"); - if !agent_path.exists() { - std::fs::write(&agent_path, initial_agent_nix(name)) - .with_context(|| format!("write {}", agent_path.display()))?; - } - let flake_path = proposed_dir.join("flake.nix"); - if !flake_path.exists() { - std::fs::write(&flake_path, initial_flake_nix()) - .with_context(|| format!("write {}", flake_path.display()))?; - } - git(proposed_dir, &["init", "--initial-branch=main"]).await?; - git(proposed_dir, &["add", "agent.nix", "flake.nix"]).await?; - git_commit(proposed_dir, "hive-c0re init").await?; + if proposed_dir.join(".git").exists() { + return Ok(()); } - // Idempotently wire the `applied` remote — purely for the - // manager's ergonomics. The URL is the path inside the manager - // container (`/applied//.git`), where the RO bind in - // `set_nspawn_flags` makes it real. hive-c0re itself never - // dereferences this remote; the host-side fetch in - // `request_apply_commit` uses absolute host paths. - ensure_applied_remote(proposed_dir, name).await + std::fs::create_dir_all(proposed_dir) + .with_context(|| format!("create {}", proposed_dir.display()))?; + let agent_path = proposed_dir.join("agent.nix"); + if !agent_path.exists() { + std::fs::write(&agent_path, initial_agent_nix(name)) + .with_context(|| format!("write {}", agent_path.display()))?; + } + git(proposed_dir, &["init", "--initial-branch=main"]).await?; + git(proposed_dir, &["add", "agent.nix"]).await?; + git_commit(proposed_dir, "hive-c0re init").await?; + Ok(()) } -async fn ensure_applied_remote(proposed_dir: &Path, name: &str) -> Result<()> { - let want = format!("/applied/{name}/.git"); - let existing = git_command() - .current_dir(proposed_dir) - .args(["remote", "get-url", "applied"]) - .output() - .await - .with_context(|| format!("git remote get-url applied in {}", proposed_dir.display()))?; - if existing.status.success() { - let current = String::from_utf8_lossy(&existing.stdout).trim().to_owned(); - if current == want { - return Ok(()); - } - // URL drifted (path scheme changed, etc.) — re-point it. - return git(proposed_dir, &["remote", "set-url", "applied", &want]).await; - } - git(proposed_dir, &["remote", "add", "applied", &want]).await -} - -/// Set up the applied repo. First-spawn only: init the repo, pull -/// proposed's initial commit in via `git fetch`, tag it `deployed/0`. -/// This is the *only* time hive-c0re reads from `proposed` for an -/// agent — subsequent proposals are fetched at `request_apply_commit` -/// time and tagged `proposal/` (see `actions::approve` for the -/// tag state machine). +/// Set up the applied repo. Two responsibilities: +/// - First-spawn only: init the repo, pull proposed's initial commit +/// in via `git fetch`, tag it `deployed/0`. This is the *only* time +/// hive-c0re reads from `proposed` for an agent — subsequent +/// proposals are fetched at `request_apply_commit` time and tagged +/// `proposal/` (see `actions::approve` for the tag state +/// machine). +/// - Every call: regenerate the untracked `flake.nix` so flake-url / +/// dashboard-port changes pick up on rebuild without churning the +/// git log. /// -/// `proposed_dir` is `None` on rebuild paths where the repo already -/// exists — we just verify it's the right shape and bail otherwise. -/// Unlike the pre-overhaul code path, `flake.nix` is no longer -/// regenerated at the host level: it's tracked in proposed (seeded by -/// `setup_proposed`) and rides along on every fetch. +/// `proposed_dir` is `None` on rebuild paths that just want the flake +/// refreshed. pub async fn setup_applied( applied_dir: &Path, proposed_dir: Option<&Path>, name: &str, + hyperhive_flake: &str, + dashboard_port: u16, ) -> Result<()> { std::fs::create_dir_all(applied_dir) .with_context(|| format!("create {}", applied_dir.display()))?; + // 1. First-spawn git init from proposed (or pre-overhaul detection). if !applied_dir.join(".git").exists() { let Some(proposed) = proposed_dir else { bail!( @@ -434,18 +339,60 @@ pub async fn setup_applied( .await .is_err() { - // Pre-overhaul applied repo — no deployed/* tag scheme, - // flake.nix may be untracked, agent.nix possibly authored by - // hive-c0re directly. The startup auto-migration fixes this - // in place; if it didn't run (or got skipped), surface a - // clear error. + // Pre-overhaul applied repo — agent.nix is tracked directly, + // commits authored by hive-c0re, no deployed/* tag scheme. + // No in-place migration; fail loudly so the operator purges. bail!( - "applied repo at {} predates the meta-flake layout. \ - Restart hive-c0re to let the auto-migration run, or \ - destroy --purge {name} and re-spawn.", + "applied repo at {} predates the tag-driven config flow. \ + Run `hive-c0re destroy --purge {name}` and re-spawn.", applied_dir.display() ); } + + // 2. (Re)write the untracked wrapper flake. Tracked files in the + // working tree (agent.nix and anything the manager committed) are + // untouched. + let port = agent_web_port(name); + let base = flake_base(name); + let service = if is_manager(name) { + "hive-m1nd" + } else { + "hive-ag3nt" + }; + let description = if is_manager(name) { + format!("hyperhive manager {name}") + } else { + format!("hyperhive sub-agent {name}") + }; + let flake_body = format!( + r#"{{ + description = "{description}"; + inputs.hyperhive.url = "{hyperhive_flake}"; + outputs = + {{ hyperhive, ... }}: + {{ + nixosConfigurations.default = hyperhive.nixosConfigurations.{base}.extendModules {{ + modules = [ + ./agent.nix + {{ + programs.git.config.user = {{ + name = "{name}"; + email = "{name}@hyperhive"; + }}; + systemd.services.{service}.environment = {{ + HIVE_PORT = "{port}"; + HIVE_LABEL = "{name}"; + HIVE_DASHBOARD_PORT = "{dashboard_port}"; + }}; + }} + ]; + }}; + }}; +}} +"#, + ); + std::fs::write(applied_dir.join("flake.nix"), flake_body) + .with_context(|| format!("write {}/flake.nix", applied_dir.display()))?; Ok(()) } @@ -476,18 +423,10 @@ fn ensure_state_dir(notes_dir: &Path) -> Result<()> { fn initial_agent_nix(name: &str) -> String { format!( - "{{ config, pkgs, lib, ... }}:\n{{\n # Per-agent overrides for {name}. This is a regular NixOS module\n # — add packages, services, modules, imports as needed.\n #\n # imports = [ ./extra-module.nix ];\n # environment.systemPackages = with pkgs; [ ];\n}}\n", + "{{ ... }}:\n{{\n # Per-agent overrides for {name}. The manager edits this\n # file (and commits) to customise the agent's NixOS config.\n}}\n", ) } -/// Module-only flake exposed by every agent's repo. Consumed by the -/// hive-c0re-owned meta flake at `/var/lib/hyperhive/meta/` as a flake -/// input. Identity injection (`HIVE_PORT` / `HIVE_LABEL` / dashboard -/// port / git committer) lives in the meta flake's wrapper, not here. -pub fn initial_flake_nix() -> &'static str { - "{\n description = \"hyperhive agent\";\n inputs = { };\n outputs = { self }: {\n nixosModules.default = import ./agent.nix;\n };\n}\n" -} - async fn git_commit(dir: &Path, message: &str) -> Result<()> { git( dir, @@ -678,10 +617,6 @@ const HOST_AGENTS_ROOT: &str = "/var/lib/hyperhive/agents"; /// `APPLIED_STATE_ROOT` in coordinator.rs. const HOST_APPLIED_ROOT: &str = "/var/lib/hyperhive/applied"; -/// On-host meta repo root, mirrored RO into the manager. Matches -/// `meta::meta_dir()` but duplicated here so lifecycle stays a leaf. -const HOST_META_ROOT: &str = "/var/lib/hyperhive/meta"; - fn set_nspawn_flags( container: &str, runtime_dir: &Path, @@ -697,14 +632,6 @@ fn set_nspawn_flags( notes = notes_dir.display(), ); if container == MANAGER_NAME { - use std::fmt::Write as _; - // systemd-nspawn refuses to start a container whose bind - // source doesn't exist. The meta repo is created by the - // startup migration, but make sure the directory is there - // before the manager comes up in case set_nspawn_flags fires - // first (e.g. cold start with no agents). - std::fs::create_dir_all(HOST_META_ROOT) - .with_context(|| format!("create {HOST_META_ROOT}"))?; // Manager edits sub-agent proposed/ repos and its own. RW so it can // git-commit. Sub-agents see only their own /run/hive socket and // /root/.claude (no /agents or /applied). @@ -715,11 +642,7 @@ fn set_nspawn_flags( // denied tags into its proposed clones and diff against // what's actually deployed. RO bind makes destructive git // plumbing inside the container unable to corrupt applied. - // - // /meta is a third RO mount exposing the system-wide deploy - // flake (`git log /meta --oneline` shows every deploy across - // every agent; `cat /meta/flake.lock` resolves which sha each - // agent is pinned at right now). + use std::fmt::Write as _; let _ = write!( binds, " --bind={HOST_AGENTS_ROOT}:{CONTAINER_MANAGER_AGENTS_MOUNT}", @@ -728,11 +651,6 @@ fn set_nspawn_flags( binds, " --bind-ro={HOST_APPLIED_ROOT}:{CONTAINER_MANAGER_APPLIED_MOUNT}", ); - let _ = write!( - binds, - " --bind-ro={HOST_META_ROOT}:{mount}", - mount = crate::meta::CONTAINER_MANAGER_META_MOUNT, - ); } let bind_flag = format!("EXTRA_NSPAWN_FLAGS=\"{binds}\""); let mut lines: Vec = original diff --git a/hive-c0re/src/main.rs b/hive-c0re/src/main.rs index e6b5893c..c16410d1 100644 --- a/hive-c0re/src/main.rs +++ b/hive-c0re/src/main.rs @@ -17,8 +17,6 @@ mod dashboard; mod events_vacuum; mod lifecycle; mod manager_server; -mod meta; -mod migrate; mod operator_questions; mod server; @@ -98,15 +96,6 @@ async fn main() -> Result<()> { } => { let coord = Arc::new(Coordinator::open(&db, hyperhive_flake, dashboard_port)?); manager_server::start(coord.clone())?; - // Idempotent pre-flight: rewrite pre-meta-layout applied - // repos, ensure proposed repos carry the `applied` - // remote, bootstrap the meta repo, repoint containers at - // `meta#` (one-shot, guarded by a marker file). - // Runs before manager auto-spawn so the new manager is - // built against meta from the first attempt. - if let Err(e) = migrate::run(&coord).await { - tracing::warn!(error = ?e, "startup migration failed"); - } // Auto-create the manager container if it isn't there yet. Block // on this — without hm1nd the system has no manager harness. // Failures are logged but allowed: a broken auto-spawn shouldn't diff --git a/hive-c0re/src/meta.rs b/hive-c0re/src/meta.rs deleted file mode 100644 index e434d244..00000000 --- a/hive-c0re/src/meta.rs +++ /dev/null @@ -1,278 +0,0 @@ -//! Single hive-c0re-owned flake at `/var/lib/hyperhive/meta/` that -//! consumes every agent's applied repo as a flake input and exports one -//! `nixosConfiguration` per agent. Containers run against -//! `--flake /var/lib/hyperhive/meta#`; lifecycle ops here drive the -//! lock file so meta's git log is the system-wide deploy audit trail. -//! -//! Flow: -//! - `sync_agents` (idempotent) — render `flake.nix` for the current -//! agent set, init the repo on first call, relock if the rendered -//! contents changed, commit. Used by spawn / destroy / startup -//! migration. -//! - `prepare_deploy` + `finalize_deploy` / `abort_deploy` — two-phase -//! for the `request_apply_commit` path so a failed -//! `nixos-container update` leaves no orphan commit in meta. Prepare -//! writes the new lock without committing; finalize commits with the -//! deploy message; abort `git restore`s the lock back. -//! - `lock_update_hyperhive` — one-shot for the auto-update path. - -use std::path::{Path, PathBuf}; - -use anyhow::{Context, Result, bail}; -use tokio::process::Command; - -use crate::lifecycle; - -const META_ROOT: &str = "/var/lib/hyperhive/meta"; -const APPLIED_ROOT: &str = "/var/lib/hyperhive/applied"; -const GIT_NAME: &str = "hive-c0re"; -const GIT_EMAIL: &str = "hive-c0re@hyperhive"; - -/// Where the manager sees this directory inside its container (RO bind). -#[allow(dead_code)] // wired up by set_nspawn_flags in a follow-up commit -pub const CONTAINER_MANAGER_META_MOUNT: &str = "/meta"; - -#[derive(Debug, Clone)] -pub struct AgentSpec { - pub name: String, - pub is_manager: bool, - pub port: u16, -} - -#[must_use] -pub fn meta_dir() -> PathBuf { - PathBuf::from(META_ROOT) -} - -/// Idempotently reconcile the meta repo with the current agent set. -/// First call inits the git repo, runs `nix flake lock`, and lands a -/// seed commit. Subsequent calls only touch `flake.nix` when the -/// rendered contents differ from disk; an unchanged `flake.nix` is a -/// no-op. -#[allow(dead_code)] // first caller lands in a later commit -pub async fn sync_agents( - hyperhive_flake: &str, - dashboard_port: u16, - agents: &[AgentSpec], -) -> Result<()> { - let dir = meta_dir(); - std::fs::create_dir_all(&dir).with_context(|| format!("create {}", dir.display()))?; - - let new_flake = render_flake(hyperhive_flake, dashboard_port, agents); - let flake_path = dir.join("flake.nix"); - let on_disk = std::fs::read_to_string(&flake_path).unwrap_or_default(); - let initial = !dir.join(".git").exists(); - - if !initial && on_disk == new_flake { - return Ok(()); - } - - std::fs::write(&flake_path, &new_flake) - .with_context(|| format!("write {}", flake_path.display()))?; - - if initial { - git(&dir, &["init", "--initial-branch=main"]).await?; - } - nix(&dir, &["flake", "lock"]).await?; - git(&dir, &["add", "-A"]).await?; - let msg = if initial { - format!("seed meta from {} agent(s)", agents.len()) - } else { - "regenerate meta flake".to_owned() - }; - git_commit(&dir, &msg).await?; - Ok(()) -} - -/// Phase 1 of an apply-commit deploy. Updates the locked rev of -/// `agent-` to whatever `applied//main` currently points -/// at. **Doesn't commit** — caller must follow with -/// `finalize_deploy` on build success or `abort_deploy` on failure. -#[allow(dead_code)] // wired up by actions::run_apply_commit in a later commit -pub async fn prepare_deploy(name: &str) -> Result<()> { - let dir = meta_dir(); - let input = format!("agent-{name}"); - nix(&dir, &["flake", "lock", "--update-input", &input]).await -} - -/// Phase 2-success. Commits the staged `flake.lock` change with a -/// deploy-shaped message. No-op (clean working tree) is tolerated — -/// some lock-updates resolve to the same rev that's already locked. -#[allow(dead_code)] -pub async fn finalize_deploy(name: &str, sha: &str, tag: &str) -> Result<()> { - let dir = meta_dir(); - if git_is_clean(&dir).await? { - return Ok(()); - } - git(&dir, &["add", "flake.lock"]).await?; - let short = &sha[..sha.len().min(12)]; - git_commit(&dir, &format!("deploy {name} {tag} {short}")).await -} - -/// Phase 2-failure. Drops the uncommitted `flake.lock` change so meta -/// stays pinned at the previously-deployed shas. The failed proposal -/// is still captured in `applied/`'s annotated `failed/` tag — -/// meta's history only carries successful deploys. -#[allow(dead_code)] -pub async fn abort_deploy() -> Result<()> { - let dir = meta_dir(); - git(&dir, &["restore", "flake.lock"]).await -} - -/// One-shot used by the manual-rebuild path: relock just one -/// agent's input and commit the lock change if any. Single-phase -/// (no separate finalize) because rebuild has no failure-revert -/// semantics — it always wants the latest main. -#[allow(dead_code)] // wired up by lifecycle::rebuild in this commit -pub async fn lock_update_for_rebuild(name: &str) -> Result<()> { - let dir = meta_dir(); - let input = format!("agent-{name}"); - nix(&dir, &["flake", "lock", "--update-input", &input]).await?; - if !git_is_clean(&dir).await? { - git(&dir, &["add", "flake.lock"]).await?; - git_commit(&dir, &format!("rebuild {name}: lock update")).await?; - } - Ok(()) -} - -/// One-shot used by the auto-update path: pin the latest hyperhive -/// rev, commit if the lock changed. Cheaper than `sync_agents` -/// because the per-agent inputs aren't touched. -#[allow(dead_code)] -pub async fn lock_update_hyperhive() -> Result<()> { - let dir = meta_dir(); - nix(&dir, &["flake", "lock", "--update-input", "hyperhive"]).await?; - if !git_is_clean(&dir).await? { - git(&dir, &["add", "flake.lock"]).await?; - git_commit(&dir, "bump hyperhive").await?; - } - Ok(()) -} - -fn render_flake(hyperhive_flake: &str, dashboard_port: u16, agents: &[AgentSpec]) -> String { - use std::fmt::Write as _; - let mut out = String::new(); - out.push_str("{\n description = \"hyperhive deployed agents\";\n inputs = {\n"); - let _ = writeln!(out, " hyperhive.url = \"{hyperhive_flake}\";"); - for spec in agents { - let _ = writeln!( - out, - " agent-{}.url = \"git+file://{APPLIED_ROOT}/{}\";", - spec.name, spec.name, - ); - } - out.push_str(" };\n outputs =\n { self, hyperhive, ... }@inputs:\n let\n"); - let _ = writeln!( - out, - " dashboardPort = {dashboard_port};\n mkAgent = {{ name, isManager, port }}:" - ); - out.push_str( - r#" let - base = if isManager - then hyperhive.nixosConfigurations.manager - else hyperhive.nixosConfigurations.agent-base; - input = inputs."agent-${name}"; - service = if isManager then "hive-m1nd" else "hive-ag3nt"; - in - base.extendModules { - modules = [ - input.nixosModules.default - { - programs.git.config.user = { - name = name; - email = "${name}@hyperhive"; - }; - systemd.services.${service}.environment = { - HIVE_PORT = toString port; - HIVE_LABEL = name; - HIVE_DASHBOARD_PORT = toString dashboardPort; - }; - } - ]; - }; - in - { - nixosConfigurations = { -"#, - ); - for spec in agents { - let _ = writeln!( - out, - " {} = mkAgent {{ name = \"{}\"; isManager = {}; port = {}; }};", - spec.name, - spec.name, - if spec.is_manager { "true" } else { "false" }, - spec.port, - ); - } - out.push_str(" };\n };\n}\n"); - out -} - -async fn git_is_clean(dir: &Path) -> Result { - let out = lifecycle::git_command() - .current_dir(dir) - .args(["status", "--porcelain"]) - .output() - .await - .with_context(|| format!("git status in {}", dir.display()))?; - Ok(out.stdout.iter().all(u8::is_ascii_whitespace)) -} - -async fn git(dir: &Path, args: &[&str]) -> Result<()> { - let out = lifecycle::git_command() - .current_dir(dir) - .args(args) - .output() - .await - .with_context(|| format!("git {} in {}", args.join(" "), dir.display()))?; - if !out.status.success() { - bail!( - "git {} failed ({}): {}", - args.join(" "), - out.status, - String::from_utf8_lossy(&out.stderr).trim() - ); - } - Ok(()) -} - -async fn git_commit(dir: &Path, message: &str) -> Result<()> { - git( - dir, - &[ - "-c", - &format!("user.name={GIT_NAME}"), - "-c", - &format!("user.email={GIT_EMAIL}"), - "commit", - "-m", - message, - ], - ) - .await -} - -async fn nix(dir: &Path, args: &[&str]) -> Result<()> { - // `--extra-experimental-features` belt-and-suspenders for hosts - // that haven't set this in nix.conf. The hyperhive module's - // deploy guide assumes flakes are already enabled, but the cost - // of being defensive is one extra argv each call. - let mut all = vec!["--extra-experimental-features", "nix-command flakes"]; - all.extend(args); - let out = Command::new("nix") - .current_dir(dir) - .args(&all) - .output() - .await - .with_context(|| format!("nix {} in {}", args.join(" "), dir.display()))?; - if !out.status.success() { - bail!( - "nix {} failed ({}): {}", - args.join(" "), - out.status, - String::from_utf8_lossy(&out.stderr).trim() - ); - } - Ok(()) -} diff --git a/hive-c0re/src/migrate.rs b/hive-c0re/src/migrate.rs deleted file mode 100644 index aab8d039..00000000 --- a/hive-c0re/src/migrate.rs +++ /dev/null @@ -1,183 +0,0 @@ -//! Startup auto-migration from the pre-meta layout. Runs before -//! `auto_update::run` and consists of four phases, each idempotent: -//! -//! 1. Per-agent applied repo: rewrite `flake.nix` to the module-only -//! boilerplate if it isn't already, commit, relocate `deployed/0` -//! to HEAD so `setup_applied`'s existence check passes. -//! 2. Per-agent proposed repo: ensure the `applied` git remote -//! points at `/applied//.git` (re-runs `setup_proposed`'s -//! `ensure_applied_remote` indirectly via a host-side git call). -//! 3. Meta repo: `meta::sync_agents` over the current agent list — -//! init the repo on first call, rerender + relock if anything -//! drifted. -//! 4. Container repoint: for every existing container, run -//! `nixos-container update --flake meta#` so it -//! activates against the meta flake. Guarded by a marker file -//! so the (expensive) phase 4 only runs once across hive-c0re -//! restarts. -//! -//! Env kill-switch: `HIVE_SKIP_META_MIGRATION=1` skips the whole -//! migration. Use when smoke-testing one agent at a time by hand. - -use std::path::{Path, PathBuf}; -use std::sync::Arc; - -use anyhow::{Context, Result}; -use tokio::process::Command; - -use crate::coordinator::Coordinator; -use crate::lifecycle::{self, AGENT_PREFIX, MANAGER_NAME}; -use crate::meta; - -const KILL_SWITCH: &str = "HIVE_SKIP_META_MIGRATION"; - -/// Marker for phase 4. Once present, container repoint is skipped on -/// future restarts. -fn repoint_marker() -> PathBuf { - PathBuf::from("/var/lib/hyperhive/.meta-migration-done") -} - -const MODULE_FLAKE_MARKER: &str = "nixosModules.default = import ./agent.nix"; - -pub async fn run(coord: &Arc) -> Result<()> { - if std::env::var(KILL_SWITCH).is_ok() { - tracing::info!("migration: {KILL_SWITCH} set — skipping"); - return Ok(()); - } - let names = enumerate_agents().await; - tracing::info!(count = names.len(), "migration: scanning"); - - // Phase 1 + 2: per-agent applied + proposed. - for name in &names { - if let Err(e) = migrate_applied_repo(name).await { - tracing::warn!(%name, error = ?e, "migration: applied repo rewrite failed"); - } - if let Err(e) = lifecycle::setup_proposed(&Coordinator::agent_proposed_dir(name), name) - .await - { - tracing::warn!(%name, error = ?e, "migration: setup_proposed failed"); - } - } - - // Phase 3: meta repo. - let agents = lifecycle::agents_for_meta_listing().await.unwrap_or_default(); - if let Err(e) = - meta::sync_agents(&coord.hyperhive_flake, coord.dashboard_port, &agents).await - { - tracing::warn!(error = ?e, "migration: meta sync_agents failed"); - } - - // Phase 4: container repoint, guarded by marker. - if repoint_marker().exists() { - tracing::debug!("migration: phase 4 marker present, skipping repoint"); - return Ok(()); - } - let mut all_ok = true; - for name in &names { - if let Err(e) = repoint_container(name).await { - tracing::warn!(%name, error = ?e, "migration: container repoint failed"); - all_ok = false; - } - } - if all_ok && !names.is_empty() - && let Err(e) = std::fs::write(repoint_marker(), b"done\n") - { - tracing::warn!(error = ?e, "migration: write repoint marker failed"); - } - Ok(()) -} - -async fn enumerate_agents() -> Vec { - let containers = lifecycle::list().await.unwrap_or_default(); - containers - .into_iter() - .filter_map(|c| { - if c == MANAGER_NAME { - Some(MANAGER_NAME.to_owned()) - } else { - c.strip_prefix(AGENT_PREFIX).map(str::to_owned) - } - }) - .collect() -} - -async fn migrate_applied_repo(name: &str) -> Result<()> { - let dir = Coordinator::agent_applied_dir(name); - if !dir.join(".git").exists() { - return Ok(()); - } - let flake_path = dir.join("flake.nix"); - let cur = std::fs::read_to_string(&flake_path).unwrap_or_default(); - if cur.contains(MODULE_FLAKE_MARKER) { - return Ok(()); - } - let want = lifecycle::initial_flake_nix(); - std::fs::write(&flake_path, want) - .with_context(|| format!("write {}", flake_path.display()))?; - raw_git( - &dir, - &[ - "-c", - "user.name=hive-c0re", - "-c", - "user.email=hive-c0re@hyperhive", - "add", - "flake.nix", - ], - ) - .await?; - raw_git( - &dir, - &[ - "-c", - "user.name=hive-c0re", - "-c", - "user.email=hive-c0re@hyperhive", - "commit", - "-m", - "migration: module-only flake", - ], - ) - .await?; - // Relocate deployed/0 to the migration commit so - // setup_applied's existence check passes. - raw_git(&dir, &["tag", "-f", "deployed/0", "HEAD"]).await?; - tracing::info!(%name, "migration: applied repo migrated to module-only flake"); - Ok(()) -} - -async fn repoint_container(name: &str) -> Result<()> { - let container = lifecycle::container_name(name); - let flake_ref = format!("{}#{name}", meta::meta_dir().display()); - let out = Command::new("nixos-container") - .args(["update", &container, "--flake", &flake_ref]) - .output() - .await - .with_context(|| format!("nixos-container update {container}"))?; - if !out.status.success() { - anyhow::bail!( - "nixos-container update {container} exited {}: {}", - out.status, - String::from_utf8_lossy(&out.stderr).trim() - ); - } - tracing::info!(%name, %container, "migration: container repointed at meta"); - Ok(()) -} - -async fn raw_git(dir: &Path, args: &[&str]) -> Result<()> { - let out = lifecycle::git_command() - .current_dir(dir) - .args(args) - .output() - .await - .with_context(|| format!("git {} in {}", args.join(" "), dir.display()))?; - if !out.status.success() { - anyhow::bail!( - "git {} failed: {}", - args.join(" "), - String::from_utf8_lossy(&out.stderr).trim() - ); - } - Ok(()) -}