Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
14aa7c7acc | ||
|
|
2f6ecc4dc0 | ||
|
|
691057d2d3 | ||
|
|
59a89314f0 | ||
|
|
87016cd567 | ||
|
|
06fdbac1ac | ||
|
|
22f35def8f | ||
|
|
4cb529351e | ||
|
|
8f94e4379a | ||
|
|
c42ad1330c | ||
|
|
3d14ddeb7d | ||
|
|
92822efe16 | ||
|
|
5b5a93e0c6 | ||
|
|
a1cfb60fd0 |
13 changed files with 1020 additions and 210 deletions
69
CLAUDE.md
69
CLAUDE.md
|
|
@ -31,10 +31,18 @@ hive-c0re/ host daemon + CLI (one binary, subcommand-dispatched)
|
||||||
src/coordinator.rs shared state (broker/approvals/questions/transient/
|
src/coordinator.rs shared state (broker/approvals/questions/transient/
|
||||||
sockets) + tombstone enumeration + kick_agent
|
sockets) + tombstone enumeration + kick_agent
|
||||||
src/actions.rs approve/deny/destroy (transient-aware)
|
src/actions.rs approve/deny/destroy (transient-aware)
|
||||||
src/auto_update.rs startup rebuild scan + ensure_manager
|
src/auto_update.rs startup rebuild scan + ensure_manager +
|
||||||
src/lifecycle.rs `nixos-container` shellouts, per-agent flake generator
|
meta::lock_update_hyperhive bump
|
||||||
|
src/lifecycle.rs `nixos-container` shellouts; per-agent applied
|
||||||
|
+ proposed git repo seeding; tag plumbing
|
||||||
|
src/meta.rs single hive-c0re-owned flake at /var/lib/
|
||||||
|
hyperhive/meta/ — sync_agents, two-phase
|
||||||
|
prepare/finalize/abort, lock_update_*
|
||||||
|
src/migrate.rs startup auto-migration from pre-meta layout
|
||||||
|
(idempotent, marker-guarded phase 4)
|
||||||
src/dashboard.rs axum HTTP: static shell + /api/state JSON + actions
|
src/dashboard.rs axum HTTP: static shell + /api/state JSON + actions
|
||||||
+ journald viewer + bind-with-retry (SO_REUSEADDR)
|
+ journald viewer + bind-with-retry (SO_REUSEADDR)
|
||||||
|
+ deployed_sha chip per container
|
||||||
assets/ index.html, dashboard.css, app.js (include_str!)
|
assets/ index.html, dashboard.css, app.js (include_str!)
|
||||||
|
|
||||||
hive-ag3nt/ in-container harness crate; produces TWO binaries
|
hive-ag3nt/ in-container harness crate; produces TWO binaries
|
||||||
|
|
@ -114,29 +122,40 @@ read them à la carte.
|
||||||
In-flight or recent context that hasn't earned a section yet.
|
In-flight or recent context that hasn't earned a section yet.
|
||||||
Prune freely.
|
Prune freely.
|
||||||
|
|
||||||
- **Just landed:** tag-driven config-apply overhaul. Two-repo
|
- **Just landed:** meta-flake overhaul. Each agent's applied
|
||||||
split kept (proposed = manager RW, applied = core-only) for
|
repo is a tiny module-only flake (`nixosModules.default =
|
||||||
safety. New flow: at `request_apply_commit` time hive-c0re
|
import ./agent.nix`); `agent.nix` is a plain NixOS module
|
||||||
fetches the manager's commit into applied and pins it as
|
function — no extendModules, no hyperhive input visible to
|
||||||
`proposal/<id>`; the manager-side repo is then irrelevant
|
the manager. A single hive-c0re-owned repo at
|
||||||
for that approval. Approve / deny / build walk through more
|
`/var/lib/hyperhive/meta/` declares one input per agent
|
||||||
tags (`approved/`, `building/`, `deployed/`, `failed/`,
|
(pointing at that agent's applied repo via `git+file://`)
|
||||||
`denied/`) on the same commit; `applied/main` only
|
and one `nixosConfigurations.<n>` output per agent,
|
||||||
fast-forwards on `deployed/`. `failed/` and `denied/` are
|
wrapping `inputs.agent-<n>.nixosModules.default` with the
|
||||||
annotated — body is the build error or the operator's deny
|
identity + `HIVE_PORT` / `HIVE_LABEL` /
|
||||||
note respectively. Manager has `/applied` bind-mounted RO
|
`HIVE_DASHBOARD_PORT` injection. Containers run against
|
||||||
(whole tree) so `git fetch /applied/<n>/.git
|
`meta#<n>`. Every approve runs `nix flake lock
|
||||||
'refs/tags/*:refs/tags/applied/*'` mirrors every relevant
|
--update-input agent-<n>` (two-phase: prepare on the
|
||||||
tag into its proposed clone. `agent.nix` stays the entry
|
build path, finalize/abort on the result) — meta's git
|
||||||
point; the whole tracked tree is now preserved
|
log is the system-wide deploy audit trail; failures and
|
||||||
through apply (arbitrary files supported). The wrapper
|
denials live as annotated tags in applied. The manager
|
||||||
`flake.nix` is regenerated by hive-c0re every
|
has `/applied` and `/meta` RO-bound and the `applied`
|
||||||
spawn/rebuild but never tracked, so the applied log is
|
remote pre-wired in every proposed repo so `git fetch
|
||||||
exactly the manager's commits in deploy order. Migration:
|
applied`, `git show applied/refs/tags/deployed/<id>`,
|
||||||
no in-place — pre-overhaul applied dirs are detected via
|
`git -C /meta log --oneline`, `cat /meta/flake.lock`
|
||||||
the missing `deployed/0` tag and `setup_applied` bails
|
all just work. Migration runs idempotently on
|
||||||
with `destroy --purge` instructions. See
|
hive-c0re startup (`HIVE_SKIP_META_MIGRATION=1` skips it):
|
||||||
`docs/approvals.md`.
|
rewrites pre-meta applied flakes to module-only, wires
|
||||||
|
the proposed remote, seeds meta, and repoints every
|
||||||
|
container at `meta#<n>` (guarded by a marker so the
|
||||||
|
expensive phase only runs once).
|
||||||
|
- **Just landed (prior overhaul still underneath):** tag-
|
||||||
|
driven config-apply. Two-repo split (proposed = manager
|
||||||
|
RW, applied = core-only); `request_apply_commit` fetches
|
||||||
|
the manager's commit into applied and pins it as
|
||||||
|
`proposal/<id>`; approve / deny / build walk through
|
||||||
|
tags on the same commit; `applied/main` only fast-
|
||||||
|
forwards on `deployed/`. `failed/` + `denied/` are
|
||||||
|
annotated. See `docs/approvals.md`.
|
||||||
- **Recent (since last compaction):** inline +/- diffs on
|
- **Recent (since last compaction):** inline +/- diffs on
|
||||||
Write/Edit, send full body via collapsed details, operator
|
Write/Edit, send full body via collapsed details, operator
|
||||||
cancel + ttl on questions, deny-with-reason, dashboard
|
cancel + ttl on questions, deny-with-reason, dashboard
|
||||||
|
|
|
||||||
44
README.md
44
README.md
|
|
@ -26,8 +26,9 @@ host (NixOS, runs hive-c0re.service)
|
||||||
└── nixos-containers (each bind-mounts its socket dir → /run/hive,
|
└── nixos-containers (each bind-mounts its socket dir → /run/hive,
|
||||||
│ credentials dir → /root/.claude,
|
│ credentials dir → /root/.claude,
|
||||||
│ durable notes dir → /state;
|
│ durable notes dir → /state;
|
||||||
│ manager additionally gets /agents RW
|
│ manager additionally gets /agents RW,
|
||||||
│ + /applied RO for the deployed-tag mirror)
|
│ /applied RO (deployed-tag mirror),
|
||||||
|
│ /meta RO (swarm-wide deploy flake))
|
||||||
│
|
│
|
||||||
├── hm1nd hive-m1nd serve : claude turn loop +
|
├── hm1nd hive-m1nd serve : claude turn loop +
|
||||||
│ MCP (send / recv / request_spawn / kill / start /
|
│ MCP (send / recv / request_spawn / kill / start /
|
||||||
|
|
@ -54,21 +55,30 @@ load; collapsible inbox + collapsible journald viewer + collapsible
|
||||||
`agent.nix` viewer per agent on the dashboard.
|
`agent.nix` viewer per agent on the dashboard.
|
||||||
|
|
||||||
Config changes flow the other way: manager edits files under
|
Config changes flow the other way: manager edits files under
|
||||||
`/agents/<name>/config/` (`agent.nix` is the entry point, but arbitrary
|
`/agents/<name>/config/` — `agent.nix` is a plain NixOS module function
|
||||||
sibling files in the commit are preserved) → commits → submits the sha
|
`{ config, pkgs, lib, ... }: { ... }`, and arbitrary sibling files in
|
||||||
via `request_apply_commit`. Hive-c0re immediately fetches that commit
|
the commit are preserved → commits → submits the sha via
|
||||||
from the proposed repo into the applied repo and pins it as
|
`request_apply_commit`. Hive-c0re immediately fetches that commit from
|
||||||
`proposal/<id>` — from this moment the proposal is immutable from the
|
the proposed repo into the applied repo and pins it as `proposal/<id>`
|
||||||
manager's side. Operator clicks ◆ APPR0VE on the dashboard → hive-c0re
|
— immutable from the manager's side from then on. Operator clicks
|
||||||
moves the working tree to the proposal, runs `nixos-container update`,
|
◆ APPR0VE → hive-c0re fast-forwards `applied/<n>/main` to the proposal,
|
||||||
and either fast-forwards `applied/main` (tagging `deployed/<id>`) or
|
runs `nix flake lock --update-input agent-<n>` against the host-wide
|
||||||
annotates `failed/<id>` with the build error and rolls back to the
|
meta flake at `/var/lib/hyperhive/meta/`, builds via
|
||||||
previous deployed tree. Denials leave a `denied/<id>` annotated tag
|
`nixos-container update <c> --flake meta#<name>`, and either commits
|
||||||
carrying the operator's note. The manager sees everything that
|
the lock + tags `deployed/<id>` on success or `git restore`s the lock +
|
||||||
shipped (or didn't) via a read-only `/applied/<n>/.git` mirror inside
|
annotates `failed/<id>` with the build error + rolls back
|
||||||
its container; `git show applied/deployed/<id>` etc. is the audit
|
`applied/<n>/main` on failure. Denials leave a `denied/<id>` annotated
|
||||||
trail. See [`docs/approvals.md`](docs/approvals.md) for the full tag
|
tag carrying the operator's note.
|
||||||
state machine.
|
|
||||||
|
Meta's git log is the swarm-wide deploy audit trail (one commit per
|
||||||
|
successful deploy). Per-agent applied repos carry the tag-rich state
|
||||||
|
machine for inside-baseball decisions. The manager sees both — proposed
|
||||||
|
repos ship with an `applied` remote pre-wired, and `/meta/` is RO-bound
|
||||||
|
inside the container — so `git fetch applied`,
|
||||||
|
`git show applied/refs/tags/deployed/<id>`, `git log /meta`,
|
||||||
|
`cat /meta/flake.lock` all just work without constructing paths by
|
||||||
|
hand. See [`docs/approvals.md`](docs/approvals.md) for the full state
|
||||||
|
machine + lock-flow walkthrough.
|
||||||
For decisions the manager needs human signal on, `ask_operator(question,
|
For decisions the manager needs human signal on, `ask_operator(question,
|
||||||
options?, multi?)` queues a free-text/checkbox/radio form on the
|
options?, multi?)` queues a free-text/checkbox/radio form on the
|
||||||
dashboard; the answer arrives later as a `HelperEvent::OperatorAnswered`
|
dashboard; the answer arrives later as a `HelperEvent::OperatorAnswered`
|
||||||
|
|
|
||||||
|
|
@ -37,6 +37,59 @@ step — the operator just sees the name. On approve, hive-c0re
|
||||||
creates the container in a background task while the dashboard
|
creates the container in a background task while the dashboard
|
||||||
shows a spinner.
|
shows a spinner.
|
||||||
|
|
||||||
|
## Meta flake
|
||||||
|
|
||||||
|
The hive-c0re-owned repo at `/var/lib/hyperhive/meta/`
|
||||||
|
declares one flake input per agent (`agent-<n>.url =
|
||||||
|
"git+file:///var/lib/hyperhive/applied/<n>"`) and one
|
||||||
|
`nixosConfigurations.<n>` output per agent. Each output wraps
|
||||||
|
`inputs.agent-<n>.nixosModules.default` with the identity +
|
||||||
|
`HIVE_PORT` / `HIVE_LABEL` / `HIVE_DASHBOARD_PORT` injection
|
||||||
|
module that `setup_applied` used to generate inline.
|
||||||
|
Containers run against `--flake /var/lib/hyperhive/meta#<n>`.
|
||||||
|
|
||||||
|
Per-deploy lock flow (two-phase, owned by
|
||||||
|
`actions::run_apply_commit` → `meta::{prepare,finalize,abort}
|
||||||
|
_deploy`):
|
||||||
|
|
||||||
|
1. `meta::prepare_deploy(name)` runs
|
||||||
|
`nix flake lock --update-input agent-<n>` without
|
||||||
|
committing. Working tree of meta now points the input at
|
||||||
|
`applied/<n>/main` (which `run_apply_commit` already
|
||||||
|
fast-forwarded to `proposal/<id>`).
|
||||||
|
2. `lifecycle::rebuild_no_meta` runs
|
||||||
|
`nixos-container update <c> --flake meta#<name>`. Nix
|
||||||
|
evaluates against the staged lock.
|
||||||
|
3. On success — `meta::finalize_deploy(name, sha, "deployed/
|
||||||
|
<id>")` stages `flake.lock` and commits with
|
||||||
|
`deploy <n> deployed/<id> <sha12>`. Meta's git log gains
|
||||||
|
one entry per successful deploy.
|
||||||
|
4. On failure — `meta::abort_deploy()` runs
|
||||||
|
`git restore flake.lock` so the meta history shows only
|
||||||
|
successes; the failure stays as an annotated `failed/<id>`
|
||||||
|
tag in `applied/<n>`.
|
||||||
|
|
||||||
|
Single-phase variants exist for paths without
|
||||||
|
rollback semantics: `meta::lock_update_for_rebuild(name)` for
|
||||||
|
the manual `↻ R3BU1LD` button (commits if the lock changed)
|
||||||
|
and `meta::lock_update_hyperhive()` for the
|
||||||
|
auto-update flake-rev bump (one shot before per-agent
|
||||||
|
rebuilds, commits if the lock changed).
|
||||||
|
|
||||||
|
`meta::sync_agents(hyperhive_flake, dashboard_port, &agents)`
|
||||||
|
is the idempotent reconciler called by `spawn`, `destroy`,
|
||||||
|
`rebuild`, and the startup migration. Renders `flake.nix`
|
||||||
|
from the agent list; if it differs from disk, runs
|
||||||
|
`nix flake lock` + commits as `regenerate meta flake` (or
|
||||||
|
`seed meta from N agent(s)` on the very first call).
|
||||||
|
|
||||||
|
The manager has `/meta` RO-bound inside its container:
|
||||||
|
`git -C /meta log --oneline` is the swarm-wide deploy log,
|
||||||
|
`cat /meta/flake.lock | jq '.nodes["agent-<n>"].locked'`
|
||||||
|
resolves which sha each agent is pinned at right now.
|
||||||
|
Dashboard surfaces the same info as a `deployed:<sha12>` chip
|
||||||
|
per container row.
|
||||||
|
|
||||||
## Two repos per agent
|
## Two repos per agent
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
@ -46,17 +99,23 @@ shows a spinner.
|
||||||
# agent.nix is the
|
# agent.nix is the
|
||||||
# convention entry
|
# convention entry
|
||||||
# point; flake.nix is
|
# point; flake.nix is
|
||||||
# generated and not
|
# tracked boilerplate
|
||||||
# tracked here.
|
# (manager doesn't edit
|
||||||
|
# it).
|
||||||
|
|
||||||
/var/lib/hyperhive/applied/<name>/ applied — core-only
|
/var/lib/hyperhive/applied/<name>/ applied — core-only
|
||||||
├── .git/ # tag-rich history
|
├── .git/ # tag-rich history
|
||||||
├── .gitignore # ignores flake.nix
|
├── flake.nix # tracked, fixed
|
||||||
├── flake.nix # hive-c0re-generated,
|
│ # boilerplate exporting
|
||||||
│ # untracked, rewritten
|
│ # nixosModules.default
|
||||||
│ # on spawn/rebuild only
|
|
||||||
├── agent.nix # working tree of main
|
├── agent.nix # working tree of main
|
||||||
└── <other manager files> # also tracked
|
└── <other manager files> # also tracked
|
||||||
|
|
||||||
|
/var/lib/hyperhive/meta/ swarm-wide flake — core
|
||||||
|
├── .git/ # one commit per successful
|
||||||
|
│ # deploy
|
||||||
|
├── flake.nix # generated from agent set
|
||||||
|
└── flake.lock # pins each agent's sha
|
||||||
```
|
```
|
||||||
|
|
||||||
Why two physical repos: the manager's `/agents/<n>/config/` is
|
Why two physical repos: the manager's `/agents/<n>/config/` is
|
||||||
|
|
@ -65,13 +124,12 @@ proposed tree. The applied repo is never bind-mounted (except
|
||||||
the read-only `.git` exposure described below) so a destructive
|
the read-only `.git` exposure described below) so a destructive
|
||||||
move inside the container cannot reach it.
|
move inside the container cannot reach it.
|
||||||
|
|
||||||
The container's `--flake` ref is `<applied_dir>#default`. The
|
The container's `--flake` ref is `/var/lib/hyperhive/meta#<name>`
|
||||||
generated `flake.nix` extends
|
(see "Meta flake" above). The agent's own `applied/<n>/flake.nix`
|
||||||
`hyperhive.nixosConfigurations.{agent-base|manager}` with
|
is a fixed boilerplate that exports `nixosModules.default =
|
||||||
`./agent.nix` plus an inline module setting
|
import ./agent.nix`; the meta flake imports that module and
|
||||||
`programs.git.config.user` (committer identity = the agent's name)
|
wraps it with identity + `HIVE_PORT` / `HIVE_LABEL` /
|
||||||
and `systemd.services.<harness>.environment` (`HIVE_PORT`,
|
`HIVE_DASHBOARD_PORT`.
|
||||||
`HIVE_LABEL`, `HIVE_DASHBOARD_PORT`).
|
|
||||||
|
|
||||||
### Tag state machine
|
### Tag state machine
|
||||||
|
|
||||||
|
|
@ -93,29 +151,63 @@ approval id to retry. Because tags are first-class git objects,
|
||||||
rejected and failed trees stay browsable forever — `git log
|
rejected and failed trees stay browsable forever — `git log
|
||||||
--tags` in the applied repo is the audit trail.
|
--tags` in the applied repo is the audit trail.
|
||||||
|
|
||||||
### Manager view of applied
|
### Manager view of applied + meta
|
||||||
|
|
||||||
`/applied/` is a **read-only bind-mount** of
|
The manager container gets three host-side bind mounts via
|
||||||
`/var/lib/hyperhive/applied/` (the entire tree) inside the
|
`set_nspawn_flags`:
|
||||||
manager container. The manager fetches tags into its proposed
|
|
||||||
clone with `git fetch /applied/<n>/.git
|
|
||||||
'refs/tags/*:refs/tags/applied/*'` and `git show` any
|
|
||||||
deployed / failed / denied tree to see what actually shipped,
|
|
||||||
what error blocked the last build, or what note the operator
|
|
||||||
left on a denial. The RO bind means git plumbing inside the
|
|
||||||
manager cannot corrupt the applied repos — and a single mount
|
|
||||||
covers every agent (existing + future) without rebuilding the
|
|
||||||
manager on each spawn.
|
|
||||||
|
|
||||||
## Migration from the pre-tag scheme
|
- `/var/lib/hyperhive/agents/` → `/agents/` (RW) — proposed
|
||||||
|
repos. Manager edits + commits per-agent config here.
|
||||||
|
- `/var/lib/hyperhive/applied/` → `/applied/` (RO) — every
|
||||||
|
agent's authoritative applied repo, including `.git`.
|
||||||
|
- `/var/lib/hyperhive/meta/` → `/meta/` (RO) — the swarm-wide
|
||||||
|
deploy flake.
|
||||||
|
|
||||||
There is no in-place migration. Each existing agent must be
|
Each proposed repo (`/agents/<n>/config/`) is pre-configured
|
||||||
purged and re-spawned: `hive-c0re destroy --purge <name>` (or
|
with `applied` as a git remote pointing at
|
||||||
PURG3 on the dashboard), then `request_spawn` and the operator
|
`/applied/<n>/.git`. Useful incantations from inside the
|
||||||
approves the fresh agent. The new agent starts with `deployed/0`
|
manager:
|
||||||
seeded by hive-c0re; the manager's first config edit becomes
|
|
||||||
`proposal/1` and walks the tag scheme from there. Pre-overhaul
|
```sh
|
||||||
tombstones lose their config history.
|
git -C /agents/<n>/config fetch applied
|
||||||
|
git -C /agents/<n>/config log applied/main --oneline
|
||||||
|
git -C /agents/<n>/config show applied/refs/tags/deployed/<id>
|
||||||
|
git -C /agents/<n>/config show applied/refs/tags/failed/<id> # body = build error
|
||||||
|
git -C /agents/<n>/config show applied/refs/tags/denied/<id> # body = operator note
|
||||||
|
git -C /agents/<n>/config rebase applied/main # base in-flight work on what's deployed
|
||||||
|
|
||||||
|
git -C /meta log --oneline # swarm-wide deploy history
|
||||||
|
cat /meta/flake.lock | jq '.nodes | with_entries(select(.key | startswith("agent-")))'
|
||||||
|
```
|
||||||
|
|
||||||
|
The RO binds block push at the kernel level, so the manager
|
||||||
|
can only fetch / read — git plumbing inside the container
|
||||||
|
cannot corrupt either authoritative repo.
|
||||||
|
|
||||||
|
## Migration from the pre-tag / pre-meta schemes
|
||||||
|
|
||||||
|
Both overhauls (tag-driven flow + meta flake) ship in-place
|
||||||
|
migrations that run on every hive-c0re startup. Idempotent;
|
||||||
|
each phase is a no-op once already applied. Behaviour:
|
||||||
|
|
||||||
|
- Tag-driven phase: assumes the operator ran the one-shot
|
||||||
|
`git tag deployed/0 main` script (see commit history /
|
||||||
|
earlier docs revisions) once per agent. Tagging is
|
||||||
|
non-destructive: it doesn't touch live containers, state
|
||||||
|
dirs, or claude creds.
|
||||||
|
- Meta-flake phase: rewrites each `applied/<n>/flake.nix` to
|
||||||
|
the module-only boilerplate, wires the `applied` remote in
|
||||||
|
each proposed repo, bootstraps the meta repo from the
|
||||||
|
current agent list, and `nixos-container update`s every
|
||||||
|
container at `meta#<n>`. The expensive last step is
|
||||||
|
guarded by `/var/lib/hyperhive/.meta-migration-done` so
|
||||||
|
it only runs once across hive-c0re restarts. Set
|
||||||
|
`HIVE_SKIP_META_MIGRATION=1` on the service to defer.
|
||||||
|
|
||||||
|
No state loss in either migration. claude creds, /state/
|
||||||
|
notes, the events DB, proposed history, and applied history
|
||||||
|
all survive. The manager keeps its session; sub-agents stay
|
||||||
|
logged in.
|
||||||
|
|
||||||
## Manager (`hm1nd`) is hive-c0re-managed
|
## Manager (`hm1nd`) is hive-c0re-managed
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -67,8 +67,25 @@ Under `/var/lib/hyperhive/agents/<name>/`:
|
||||||
to `/state` inside the container.
|
to `/state` inside the container.
|
||||||
|
|
||||||
Under `/var/lib/hyperhive/applied/<name>/` — the hive-c0re-only
|
Under `/var/lib/hyperhive/applied/<name>/` — the hive-c0re-only
|
||||||
applied repo (`flake.nix` + `agent.nix`) that the container
|
applied repo. Tracks `flake.nix` (module-only boilerplate; never
|
||||||
actually builds from.
|
edited after first spawn) + `agent.nix` (the actual config; the
|
||||||
|
manager's edits land here via the approval flow) + any other
|
||||||
|
files the manager committed. `.git/` carries the proposal /
|
||||||
|
approved / building / deployed / failed / denied tag history.
|
||||||
|
|
||||||
|
Under `/var/lib/hyperhive/meta/` — the swarm-wide deploy flake.
|
||||||
|
Single repo for the whole host; `flake.nix` declares one input
|
||||||
|
per agent + one `nixosConfigurations.<n>` output per agent;
|
||||||
|
`flake.lock` is the canonical "what's deployed where." The git
|
||||||
|
log is the deploy audit trail (one commit per successful
|
||||||
|
deploy or hyperhive bump). Manager has this RO-mounted at
|
||||||
|
`/meta/`.
|
||||||
|
|
||||||
|
Marker file `/var/lib/hyperhive/.meta-migration-done` is
|
||||||
|
written by the startup migration after every container has
|
||||||
|
been repointed at `meta#<n>`. Removing it forces a re-run on
|
||||||
|
next hive-c0re start (idempotent — only the actual repoint
|
||||||
|
step would re-fire).
|
||||||
|
|
||||||
## Destroy vs purge
|
## Destroy vs purge
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -14,14 +14,18 @@ Tools (hyperhive surface):
|
||||||
|
|
||||||
Approval boundary: lifecycle ops on *existing* sub-agents (`kill`, `start`, `restart`) are at your discretion — no operator approval. *Creating* a new agent (`request_spawn`) and *changing* any agent's config (`request_apply_commit`) still go through the approval queue. The operator only signs off on changes; you run the day-to-day.
|
Approval boundary: lifecycle ops on *existing* sub-agents (`kill`, `start`, `restart`) are at your discretion — no operator approval. *Creating* a new agent (`request_spawn`) and *changing* any agent's config (`request_apply_commit`) still go through the approval queue. The operator only signs off on changes; you run the day-to-day.
|
||||||
|
|
||||||
Your own editable config lives at `/agents/hm1nd/config/`; every sub-agent's lives at `/agents/<name>/config/`. `agent.nix` is the entry point but you can commit any extra files (modules, overlays, prompt fragments) and the whole tree gets deployed together. Use file/git tools to edit + commit, then `request_apply_commit`.
|
Your own editable config lives at `/agents/hm1nd/config/`; every sub-agent's lives at `/agents/<name>/config/`. `agent.nix` is a plain NixOS module function — `{ config, pkgs, lib, ... }: { ... }`. Add packages, services, imports, sibling `.nix` files; the whole committed tree gets deployed together. **Do not edit `flake.nix`** — it's a fixed boilerplate that exports `agent.nix` as `nixosModules.default`; the hive-c0re-owned meta flake at `/meta/` provides the NixOS base and wires identity / `HIVE_PORT` / `HIVE_LABEL` itself.
|
||||||
|
|
||||||
To see what hive-c0re actually deployed (or rejected, or failed to build), there's a read-only mirror of every agent's applied repo at `/applied/<name>/.git`. Useful patterns:
|
Each proposed repo has an `applied` git remote pre-configured pointing at the read-only mirror of what's deployed. Useful patterns:
|
||||||
|
|
||||||
- `git -C /agents/<name>/config fetch /applied/<name>/.git 'refs/tags/*:refs/tags/applied/*'` — mirror all tags into your proposed clone.
|
- `git -C /agents/<name>/config fetch applied` — refresh the local copy of every deployed/failed/denied tag.
|
||||||
- `git -C /agents/<name>/config show applied/deployed/<id>` — see the tree that's currently running.
|
- `git -C /agents/<name>/config log applied/main --oneline` — every successful deploy of this agent.
|
||||||
- `git -C /agents/<name>/config show applied/failed/<id>` — annotated tag body is the build error from a rejected rebuild.
|
- `git -C /agents/<name>/config show applied/refs/tags/deployed/<id>` — the tree that was deployed for approval `<id>`.
|
||||||
- `git -C /agents/<name>/config show applied/denied/<id>` — annotated tag body is the operator's reason for denial.
|
- `git -C /agents/<name>/config show applied/refs/tags/failed/<id>` — annotated tag body is the build error from a rejected rebuild.
|
||||||
|
- `git -C /agents/<name>/config show applied/refs/tags/denied/<id>` — annotated tag body is the operator's reason for denial.
|
||||||
|
- `git -C /agents/<name>/config rebase applied/main` — base your in-flight work on whatever's actually deployed (useful after a failed/denied pile-up).
|
||||||
|
|
||||||
|
System-wide view: `/meta/` is a read-only mirror of the deployed-agents flake. `git -C /meta log --oneline` is the deploy log for every agent across the swarm; `cat /meta/flake.lock` shows which sha each agent is pinned at right now.
|
||||||
|
|
||||||
Tag scheme on every approval id: `proposal → approved → building → deployed | failed`, plus `denied` as a terminal alternative to `approved`. `applied/main` only advances on `deployed/*`, so a failed build does not corrupt the agent — submit a fix as a new commit and a fresh `request_apply_commit`.
|
Tag scheme on every approval id: `proposal → approved → building → deployed | failed`, plus `denied` as a terminal alternative to `approved`. `applied/main` only advances on `deployed/*`, so a failed build does not corrupt the agent — submit a fix as a new commit and a fresh `request_apply_commit`.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -279,6 +279,11 @@
|
||||||
));
|
));
|
||||||
}
|
}
|
||||||
head.append(el('span', { class: 'meta' }, `${c.container} :${c.port}`));
|
head.append(el('span', { class: 'meta' }, `${c.container} :${c.port}`));
|
||||||
|
if (c.deployed_sha) {
|
||||||
|
head.append(el('span',
|
||||||
|
{ class: 'meta', title: 'sha currently locked in /meta/flake.lock' },
|
||||||
|
`deployed:${c.deployed_sha}`));
|
||||||
|
}
|
||||||
li.append(head);
|
li.append(head);
|
||||||
|
|
||||||
// ── line 2: action buttons ───────────────────────────────────
|
// ── line 2: action buttons ───────────────────────────────────
|
||||||
|
|
|
||||||
|
|
@ -147,6 +147,7 @@ async fn run_apply_commit(
|
||||||
) -> (Result<()>, Option<String>) {
|
) -> (Result<()>, Option<String>) {
|
||||||
let id = approval.id;
|
let id = approval.id;
|
||||||
let proposal_ref = format!("refs/tags/proposal/{id}");
|
let proposal_ref = format!("refs/tags/proposal/{id}");
|
||||||
|
|
||||||
// Defensive: submit-time should have planted proposal/<id>, but if
|
// Defensive: submit-time should have planted proposal/<id>, but if
|
||||||
// the row was migrated from an older schema or the tag got pruned
|
// the row was migrated from an older schema or the tag got pruned
|
||||||
// we fail early with a clear note rather than building a stale
|
// we fail early with a clear note rather than building a stale
|
||||||
|
|
@ -159,6 +160,14 @@ async fn run_apply_commit(
|
||||||
None,
|
None,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Capture the currently-deployed sha so we can roll applied/main
|
||||||
|
// (and the meta lock indirectly) back if the build fails.
|
||||||
|
let prev_main_sha = match lifecycle::git_rev_parse(applied_dir, "refs/heads/main").await {
|
||||||
|
Ok(s) => s,
|
||||||
|
Err(e) => return (Err(anyhow::anyhow!("read applied/main: {e:#}")), None),
|
||||||
|
};
|
||||||
|
|
||||||
if let Err(e) = lifecycle::git_tag(applied_dir, &format!("approved/{id}"), &proposal_ref).await
|
if let Err(e) = lifecycle::git_tag(applied_dir, &format!("approved/{id}"), &proposal_ref).await
|
||||||
{
|
{
|
||||||
return (Err(anyhow::anyhow!("plant approved/{id}: {e:#}")), None);
|
return (Err(anyhow::anyhow!("plant approved/{id}: {e:#}")), None);
|
||||||
|
|
@ -167,41 +176,71 @@ async fn run_apply_commit(
|
||||||
{
|
{
|
||||||
return (Err(anyhow::anyhow!("plant building/{id}: {e:#}")), None);
|
return (Err(anyhow::anyhow!("plant building/{id}: {e:#}")), None);
|
||||||
}
|
}
|
||||||
if let Err(e) = lifecycle::git_read_tree_reset(applied_dir, &proposal_ref).await {
|
|
||||||
|
// Fast-forward applied/main to proposal/<id> + sync the working
|
||||||
|
// tree. Meta input pins `?ref=main`, so this is what makes nix
|
||||||
|
// re-lock to the proposal commit on the prepare_deploy step
|
||||||
|
// below. On build failure we roll main back to prev_main_sha so
|
||||||
|
// a crash leaves the agent on its last-good tree.
|
||||||
|
if let Err(e) =
|
||||||
|
lifecycle::git_update_ref(applied_dir, "refs/heads/main", &proposal_ref).await
|
||||||
|
{
|
||||||
return (
|
return (
|
||||||
Err(anyhow::anyhow!("read-tree to {proposal_ref}: {e:#}")),
|
Err(anyhow::anyhow!("ff main to {proposal_ref}: {e:#}")),
|
||||||
|
None,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if let Err(e) = lifecycle::git_read_tree_reset(applied_dir, "refs/heads/main").await {
|
||||||
|
// main is ahead; working tree didn't sync. Roll main back to
|
||||||
|
// keep the two consistent before bailing.
|
||||||
|
let _ = lifecycle::git_update_ref(applied_dir, "refs/heads/main", &prev_main_sha).await;
|
||||||
|
return (
|
||||||
|
Err(anyhow::anyhow!("read-tree to main: {e:#}")),
|
||||||
None,
|
None,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
let rebuild_result = lifecycle::rebuild(
|
// Phase 1 of the meta two-phase deploy: relock without committing.
|
||||||
|
if let Err(e) = crate::meta::prepare_deploy(&approval.agent).await {
|
||||||
|
let _ = lifecycle::git_update_ref(applied_dir, "refs/heads/main", &prev_main_sha).await;
|
||||||
|
let _ = lifecycle::git_read_tree_reset(applied_dir, "refs/heads/main").await;
|
||||||
|
return (
|
||||||
|
Err(anyhow::anyhow!("meta prepare_deploy: {e:#}")),
|
||||||
|
None,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Container-level rebuild against meta#<name>.
|
||||||
|
let build_result = lifecycle::rebuild_no_meta(
|
||||||
&approval.agent,
|
&approval.agent,
|
||||||
&coord.hyperhive_flake,
|
|
||||||
agent_dir,
|
agent_dir,
|
||||||
applied_dir,
|
applied_dir,
|
||||||
claude_dir,
|
claude_dir,
|
||||||
notes_dir,
|
notes_dir,
|
||||||
coord.dashboard_port,
|
|
||||||
)
|
)
|
||||||
.await;
|
.await;
|
||||||
|
|
||||||
match rebuild_result {
|
match build_result {
|
||||||
Ok(()) => {
|
Ok(()) => {
|
||||||
let tag = format!("deployed/{id}");
|
let tag = format!("deployed/{id}");
|
||||||
if let Err(e) = lifecycle::git_tag(applied_dir, &tag, &proposal_ref).await {
|
if let Err(e) = lifecycle::git_tag(applied_dir, &tag, &proposal_ref).await {
|
||||||
tracing::warn!(agent = %approval.agent, %id, error = ?e, "plant deployed tag failed");
|
tracing::warn!(agent = %approval.agent, %id, error = ?e, "plant deployed tag failed");
|
||||||
}
|
}
|
||||||
if let Err(e) =
|
if let Err(e) = crate::meta::finalize_deploy(
|
||||||
lifecycle::git_update_ref(applied_dir, "refs/heads/main", &proposal_ref).await
|
&approval.agent,
|
||||||
|
approval.fetched_sha.as_deref().unwrap_or(&proposal_ref),
|
||||||
|
&tag,
|
||||||
|
)
|
||||||
|
.await
|
||||||
{
|
{
|
||||||
// Working tree already matches proposal/<id>, but main
|
// The build itself succeeded — meta lock landed but
|
||||||
// didn't advance — surface as a build failure so the
|
// couldn't be committed. Surface as a soft warn so the
|
||||||
// operator notices the desync.
|
// operator can git-commit by hand if they care.
|
||||||
return (
|
tracing::warn!(agent = %approval.agent, %id, error = ?e, "meta finalize_deploy failed");
|
||||||
Err(anyhow::anyhow!("ff main to {proposal_ref}: {e:#}")),
|
|
||||||
Some(tag),
|
|
||||||
);
|
|
||||||
}
|
}
|
||||||
|
// Don't ignore the coord pointer — keeps the borrow alive
|
||||||
|
// for future tracing additions without re-plumbing.
|
||||||
|
let _ = coord;
|
||||||
(Ok(()), Some(tag))
|
(Ok(()), Some(tag))
|
||||||
}
|
}
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
|
|
@ -212,13 +251,22 @@ async fn run_apply_commit(
|
||||||
{
|
{
|
||||||
tracing::warn!(agent = %approval.agent, %id, error = ?te, "annotate failed tag failed");
|
tracing::warn!(agent = %approval.agent, %id, error = ?te, "annotate failed tag failed");
|
||||||
}
|
}
|
||||||
// Roll working tree back to last known-good main so the
|
// Roll main back to last known-good so the on-disk state
|
||||||
// on-disk state matches what nixos-container last
|
// matches what nixos-container last successfully built.
|
||||||
// successfully built. main hasn't moved, so this is just
|
if let Err(re) =
|
||||||
// a content reset.
|
lifecycle::git_update_ref(applied_dir, "refs/heads/main", &prev_main_sha).await
|
||||||
|
{
|
||||||
|
tracing::warn!(agent = %approval.agent, %id, error = ?re, "main rollback failed");
|
||||||
|
}
|
||||||
if let Err(re) = lifecycle::git_read_tree_reset(applied_dir, "refs/heads/main").await {
|
if let Err(re) = lifecycle::git_read_tree_reset(applied_dir, "refs/heads/main").await {
|
||||||
tracing::warn!(agent = %approval.agent, %id, error = ?re, "rollback read-tree failed");
|
tracing::warn!(agent = %approval.agent, %id, error = ?re, "rollback read-tree failed");
|
||||||
}
|
}
|
||||||
|
// Drop the staged meta lock change so the deploy log
|
||||||
|
// only ever shows successes.
|
||||||
|
if let Err(ae) = crate::meta::abort_deploy().await {
|
||||||
|
tracing::warn!(agent = %approval.agent, %id, error = ?ae, "meta abort_deploy failed");
|
||||||
|
}
|
||||||
|
let _ = coord;
|
||||||
(Err(e), Some(tag))
|
(Err(e), Some(tag))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -261,6 +309,14 @@ pub async fn destroy(coord: &Coordinator, name: &str, purge: bool) -> Result<()>
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
// Meta flake: drop the agent's input + nixosConfiguration so a
|
||||||
|
// future spawn under the same name re-seeds cleanly, and so the
|
||||||
|
// meta lock doesn't reference a vanished applied repo. Log + keep
|
||||||
|
// going on failure — destroy already succeeded at the
|
||||||
|
// nixos-container level, the meta repo is just bookkeeping.
|
||||||
|
if let Err(e) = sync_meta_after_lifecycle(coord).await {
|
||||||
|
tracing::warn!(error = ?e, %name, "meta sync after destroy failed");
|
||||||
|
}
|
||||||
let _ = coord.approvals.fail_pending_for_agent(
|
let _ = coord.approvals.fail_pending_for_agent(
|
||||||
name,
|
name,
|
||||||
if purge {
|
if purge {
|
||||||
|
|
@ -276,6 +332,14 @@ pub async fn destroy(coord: &Coordinator, name: &str, purge: bool) -> Result<()>
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Rerender the meta flake from whatever containers still exist on
|
||||||
|
/// disk. Called after lifecycle ops that change the agent set (today:
|
||||||
|
/// destroy). Idempotent — a no-op when nothing changed.
|
||||||
|
async fn sync_meta_after_lifecycle(coord: &Coordinator) -> Result<()> {
|
||||||
|
let agents = lifecycle::agents_for_meta_listing().await?;
|
||||||
|
crate::meta::sync_agents(&coord.hyperhive_flake, coord.dashboard_port, &agents).await
|
||||||
|
}
|
||||||
|
|
||||||
pub async fn deny(coord: &Coordinator, id: i64, note: Option<&str>) -> Result<()> {
|
pub async fn deny(coord: &Coordinator, id: i64, note: Option<&str>) -> Result<()> {
|
||||||
let approval = coord.approvals.get(id)?;
|
let approval = coord.approvals.get(id)?;
|
||||||
coord.approvals.mark_denied(id, note)?;
|
coord.approvals.mark_denied(id, note)?;
|
||||||
|
|
|
||||||
|
|
@ -159,6 +159,13 @@ pub async fn run(coord: Arc<Coordinator>) -> Result<()> {
|
||||||
};
|
};
|
||||||
tracing::info!(rev = %current_rev, "auto-update: scanning agents");
|
tracing::info!(rev = %current_rev, "auto-update: scanning agents");
|
||||||
|
|
||||||
|
// Bump meta's hyperhive input up-front so the per-agent rebuilds
|
||||||
|
// below build against the new base. Failure here is logged but
|
||||||
|
// not fatal — individual rebuilds will surface concrete errors.
|
||||||
|
if let Err(e) = crate::meta::lock_update_hyperhive().await {
|
||||||
|
tracing::warn!(error = ?e, "auto-update: meta lock_update_hyperhive failed");
|
||||||
|
}
|
||||||
|
|
||||||
let containers = match lifecycle::list().await {
|
let containers = match lifecycle::list().await {
|
||||||
Ok(c) => c,
|
Ok(c) => c,
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
|
|
|
||||||
|
|
@ -187,6 +187,12 @@ struct ContainerView {
|
||||||
/// disable other buttons.
|
/// disable other buttons.
|
||||||
#[serde(skip_serializing_if = "Option::is_none")]
|
#[serde(skip_serializing_if = "Option::is_none")]
|
||||||
pending: Option<&'static str>,
|
pending: Option<&'static str>,
|
||||||
|
/// First 12 chars of the sha the meta flake currently has locked
|
||||||
|
/// for this agent's input. Reflects what's actually deployed; can
|
||||||
|
/// differ from `applied/<n>/main` only between
|
||||||
|
/// `meta::prepare_deploy` and `finalize_deploy` (≤ build duration).
|
||||||
|
#[serde(skip_serializing_if = "Option::is_none")]
|
||||||
|
deployed_sha: Option<String>,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Serialize)]
|
#[derive(Serialize)]
|
||||||
|
|
@ -281,6 +287,7 @@ async fn build_container_views(
|
||||||
) -> (Vec<ContainerView>, bool) {
|
) -> (Vec<ContainerView>, bool) {
|
||||||
let mut out = Vec::new();
|
let mut out = Vec::new();
|
||||||
let mut any_stale = false;
|
let mut any_stale = false;
|
||||||
|
let locked = read_meta_locked_revs();
|
||||||
for c in raw_containers {
|
for c in raw_containers {
|
||||||
let (logical, is_manager) = if c == MANAGER_NAME {
|
let (logical, is_manager) = if c == MANAGER_NAME {
|
||||||
(MANAGER_NAME.to_owned(), true)
|
(MANAGER_NAME.to_owned(), true)
|
||||||
|
|
@ -299,6 +306,9 @@ async fn build_container_views(
|
||||||
let pending = transient_snapshot
|
let pending = transient_snapshot
|
||||||
.get(&logical)
|
.get(&logical)
|
||||||
.map(|st| transient_label(st.kind));
|
.map(|st| transient_label(st.kind));
|
||||||
|
let deployed_sha = locked
|
||||||
|
.get(&format!("agent-{logical}"))
|
||||||
|
.map(|s| s[..s.len().min(12)].to_owned());
|
||||||
out.push(ContainerView {
|
out.push(ContainerView {
|
||||||
port: lifecycle::agent_web_port(&logical),
|
port: lifecycle::agent_web_port(&logical),
|
||||||
running: lifecycle::is_running(&logical).await,
|
running: lifecycle::is_running(&logical).await,
|
||||||
|
|
@ -308,11 +318,39 @@ async fn build_container_views(
|
||||||
needs_update,
|
needs_update,
|
||||||
needs_login,
|
needs_login,
|
||||||
pending,
|
pending,
|
||||||
|
deployed_sha,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
(out, any_stale)
|
(out, any_stale)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Parse `/var/lib/hyperhive/meta/flake.lock` into a map of node name
|
||||||
|
/// (`agent-<n>`, `hyperhive`) → locked sha. Missing / unparsable lock
|
||||||
|
/// yields an empty map so the dashboard degrades gracefully when the
|
||||||
|
/// meta repo hasn't been seeded yet.
|
||||||
|
fn read_meta_locked_revs() -> std::collections::HashMap<String, String> {
|
||||||
|
let mut out = std::collections::HashMap::new();
|
||||||
|
let Ok(raw) = std::fs::read_to_string("/var/lib/hyperhive/meta/flake.lock") else {
|
||||||
|
return out;
|
||||||
|
};
|
||||||
|
let Ok(json) = serde_json::from_str::<serde_json::Value>(&raw) else {
|
||||||
|
return out;
|
||||||
|
};
|
||||||
|
let Some(nodes) = json.get("nodes").and_then(|v| v.as_object()) else {
|
||||||
|
return out;
|
||||||
|
};
|
||||||
|
for (name, node) in nodes {
|
||||||
|
if let Some(rev) = node
|
||||||
|
.get("locked")
|
||||||
|
.and_then(|v| v.get("rev"))
|
||||||
|
.and_then(|v| v.as_str())
|
||||||
|
{
|
||||||
|
out.insert(name.clone(), rev.to_owned());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
out
|
||||||
|
}
|
||||||
|
|
||||||
/// Transient state for agents whose container does NOT yet exist
|
/// Transient state for agents whose container does NOT yet exist
|
||||||
/// (`Spawning`). Lifecycle ops on existing containers surface as
|
/// (`Spawning`). Lifecycle ops on existing containers surface as
|
||||||
/// `ContainerView.pending` inline; this list only catches pre-creation.
|
/// `ContainerView.pending` inline; this list only catches pre-creation.
|
||||||
|
|
|
||||||
|
|
@ -80,17 +80,6 @@ pub fn is_manager(name: &str) -> bool {
|
||||||
name == MANAGER_NAME
|
name == MANAGER_NAME
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The nixosConfiguration in the hyperhive flake the agent's `flake.nix`
|
|
||||||
/// extends. Manager → `manager`; everyone else → `agent-base`.
|
|
||||||
#[must_use]
|
|
||||||
pub fn flake_base(name: &str) -> &'static str {
|
|
||||||
if is_manager(name) {
|
|
||||||
"manager"
|
|
||||||
} else {
|
|
||||||
"agent-base"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn validate(name: &str) -> Result<()> {
|
fn validate(name: &str) -> Result<()> {
|
||||||
if name.is_empty() {
|
if name.is_empty() {
|
||||||
bail!("agent name must not be empty");
|
bail!("agent name must not be empty");
|
||||||
|
|
@ -152,18 +141,16 @@ pub async fn spawn(
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
setup_proposed(proposed_dir, name).await?;
|
setup_proposed(proposed_dir, name).await?;
|
||||||
setup_applied(
|
setup_applied(applied_dir, Some(proposed_dir), name).await?;
|
||||||
applied_dir,
|
|
||||||
Some(proposed_dir),
|
|
||||||
name,
|
|
||||||
hyperhive_flake,
|
|
||||||
dashboard_port,
|
|
||||||
)
|
|
||||||
.await?;
|
|
||||||
ensure_claude_dir(claude_dir)?;
|
ensure_claude_dir(claude_dir)?;
|
||||||
ensure_state_dir(notes_dir)?;
|
ensure_state_dir(notes_dir)?;
|
||||||
|
// Meta flake gets the new agent's input + nixosConfiguration
|
||||||
|
// before `nixos-container create` so the `--flake meta#<name>`
|
||||||
|
// ref resolves.
|
||||||
|
let agents = agents_after_spawn(name).await?;
|
||||||
|
crate::meta::sync_agents(hyperhive_flake, dashboard_port, &agents).await?;
|
||||||
let container = container_name(name);
|
let container = container_name(name);
|
||||||
let flake_ref = format!("{}#default", applied_dir.display());
|
let flake_ref = format!("{}#{name}", crate::meta::meta_dir().display());
|
||||||
run(&["create", &container, "--flake", &flake_ref]).await?;
|
run(&["create", &container, "--flake", &flake_ref]).await?;
|
||||||
set_nspawn_flags(&container, agent_dir, claude_dir, notes_dir)?;
|
set_nspawn_flags(&container, agent_dir, claude_dir, notes_dir)?;
|
||||||
set_resource_limits(&container)?;
|
set_resource_limits(&container)?;
|
||||||
|
|
@ -171,6 +158,55 @@ pub async fn spawn(
|
||||||
run(&["start", &container]).await
|
run(&["start", &container]).await
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Build the `AgentSpec` list for the meta flake from `nixos-container
|
||||||
|
/// list` + a hypothetical extra name not yet in the list (for spawn
|
||||||
|
/// where the new agent's container doesn't exist yet). Pass empty
|
||||||
|
/// `name_to_add` from rebuild paths where the agent is already in the
|
||||||
|
/// container list.
|
||||||
|
async fn agents_for_meta(name_to_add: Option<&str>) -> Result<Vec<crate::meta::AgentSpec>> {
|
||||||
|
let containers = list().await.unwrap_or_default();
|
||||||
|
let mut out: Vec<crate::meta::AgentSpec> = containers
|
||||||
|
.into_iter()
|
||||||
|
.filter_map(|c| {
|
||||||
|
let (name, is_manager) = if c == MANAGER_NAME {
|
||||||
|
(MANAGER_NAME.to_owned(), true)
|
||||||
|
} else if let Some(n) = c.strip_prefix(AGENT_PREFIX) {
|
||||||
|
(n.to_owned(), false)
|
||||||
|
} else {
|
||||||
|
return None;
|
||||||
|
};
|
||||||
|
Some(crate::meta::AgentSpec {
|
||||||
|
port: agent_web_port(&name),
|
||||||
|
name,
|
||||||
|
is_manager,
|
||||||
|
})
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
if let Some(extra) = name_to_add
|
||||||
|
&& !out.iter().any(|a| a.name == extra)
|
||||||
|
{
|
||||||
|
out.push(crate::meta::AgentSpec {
|
||||||
|
name: extra.to_owned(),
|
||||||
|
is_manager: is_manager(extra),
|
||||||
|
port: agent_web_port(extra),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
out.sort_by(|a, b| a.name.cmp(&b.name));
|
||||||
|
Ok(out)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn agents_after_spawn(name: &str) -> Result<Vec<crate::meta::AgentSpec>> {
|
||||||
|
agents_for_meta(Some(name)).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Public enumeration of currently-existing agents (whatever
|
||||||
|
/// `nixos-container list` says), sorted, no extras. For callers
|
||||||
|
/// outside this module that need to reseed meta after lifecycle
|
||||||
|
/// changes — destroy, startup reconciliation, etc.
|
||||||
|
pub async fn agents_for_meta_listing() -> Result<Vec<crate::meta::AgentSpec>> {
|
||||||
|
agents_for_meta(None).await
|
||||||
|
}
|
||||||
|
|
||||||
pub async fn kill(name: &str) -> Result<()> {
|
pub async fn kill(name: &str) -> Result<()> {
|
||||||
validate(name)?;
|
validate(name)?;
|
||||||
let container = container_name(name);
|
let container = container_name(name);
|
||||||
|
|
@ -229,6 +265,32 @@ pub async fn rebuild(
|
||||||
claude_dir: &Path,
|
claude_dir: &Path,
|
||||||
notes_dir: &Path,
|
notes_dir: &Path,
|
||||||
dashboard_port: u16,
|
dashboard_port: u16,
|
||||||
|
) -> Result<()> {
|
||||||
|
// Sync the meta flake (idempotent — no-op when the rendered
|
||||||
|
// flake matches disk) so a manual rebuild from the dashboard
|
||||||
|
// can also recover from a divergent meta repo (e.g. an agent
|
||||||
|
// got added directly via `nixos-container create` outside
|
||||||
|
// hive-c0re).
|
||||||
|
let agents = agents_for_meta(None).await?;
|
||||||
|
crate::meta::sync_agents(hyperhive_flake, dashboard_port, &agents).await?;
|
||||||
|
// Then bump just this agent's input — picks up whatever
|
||||||
|
// `applied/<n>/main` currently points at (deployed/<latest>).
|
||||||
|
// Commits the lock if it changed.
|
||||||
|
crate::meta::lock_update_for_rebuild(name).await?;
|
||||||
|
rebuild_no_meta(name, agent_dir, applied_dir, claude_dir, notes_dir).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Container-level rebuild without touching the meta repo. Callers
|
||||||
|
/// that own the meta side themselves (`actions::run_apply_commit`
|
||||||
|
/// drives meta through the two-phase prepare/finalize/abort flow)
|
||||||
|
/// use this directly. Public `rebuild` wraps it with idempotent meta
|
||||||
|
/// sync + lock-bump-and-commit.
|
||||||
|
pub async fn rebuild_no_meta(
|
||||||
|
name: &str,
|
||||||
|
agent_dir: &Path,
|
||||||
|
applied_dir: &Path,
|
||||||
|
claude_dir: &Path,
|
||||||
|
notes_dir: &Path,
|
||||||
) -> Result<()> {
|
) -> Result<()> {
|
||||||
validate(name)?;
|
validate(name)?;
|
||||||
if let Some(other) = port_collision(name).await {
|
if let Some(other) = port_collision(name).await {
|
||||||
|
|
@ -237,11 +299,11 @@ pub async fn rebuild(
|
||||||
agent_web_port(name)
|
agent_web_port(name)
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
setup_applied(applied_dir, None, name, hyperhive_flake, dashboard_port).await?;
|
setup_applied(applied_dir, None, name).await?;
|
||||||
ensure_claude_dir(claude_dir)?;
|
ensure_claude_dir(claude_dir)?;
|
||||||
ensure_state_dir(notes_dir)?;
|
ensure_state_dir(notes_dir)?;
|
||||||
let container = container_name(name);
|
let container = container_name(name);
|
||||||
let flake_ref = format!("{}#default", applied_dir.display());
|
let flake_ref = format!("{}#{name}", crate::meta::meta_dir().display());
|
||||||
set_nspawn_flags(&container, agent_dir, claude_dir, notes_dir)?;
|
set_nspawn_flags(&container, agent_dir, claude_dir, notes_dir)?;
|
||||||
set_resource_limits(&container)?;
|
set_resource_limits(&container)?;
|
||||||
systemd_daemon_reload().await?;
|
systemd_daemon_reload().await?;
|
||||||
|
|
@ -272,51 +334,84 @@ pub async fn list() -> Result<Vec<String>> {
|
||||||
.collect())
|
.collect())
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Initialize the manager-editable proposed repo. Contains only `agent.nix`
|
/// Initialize the manager-editable proposed repo. Seeds two tracked
|
||||||
/// (the file the manager edits). Touched by hive-c0re only on first spawn —
|
/// files: `agent.nix` (the module the manager edits) and `flake.nix`
|
||||||
/// never again — so the manager can't be surprised by hive-c0re commits or
|
/// (the boilerplate that lets the meta flake import this repo as an
|
||||||
/// working-tree resets.
|
/// input — meta locks at a specific sha and reads
|
||||||
|
/// `nixosModules.default`, so `flake.nix` must be in the commit). The
|
||||||
|
/// manager shouldn't edit `flake.nix` (the prompt says so) but it's
|
||||||
|
/// visible so they can introspect.
|
||||||
|
///
|
||||||
|
/// Touched by hive-c0re only on first spawn — never again — so the
|
||||||
|
/// manager can't be surprised by hive-c0re commits or working-tree
|
||||||
|
/// resets.
|
||||||
pub async fn setup_proposed(proposed_dir: &Path, name: &str) -> Result<()> {
|
pub async fn setup_proposed(proposed_dir: &Path, name: &str) -> Result<()> {
|
||||||
if proposed_dir.join(".git").exists() {
|
let fresh = !proposed_dir.join(".git").exists();
|
||||||
return Ok(());
|
if fresh {
|
||||||
|
std::fs::create_dir_all(proposed_dir)
|
||||||
|
.with_context(|| format!("create {}", proposed_dir.display()))?;
|
||||||
|
let agent_path = proposed_dir.join("agent.nix");
|
||||||
|
if !agent_path.exists() {
|
||||||
|
std::fs::write(&agent_path, initial_agent_nix(name))
|
||||||
|
.with_context(|| format!("write {}", agent_path.display()))?;
|
||||||
|
}
|
||||||
|
let flake_path = proposed_dir.join("flake.nix");
|
||||||
|
if !flake_path.exists() {
|
||||||
|
std::fs::write(&flake_path, initial_flake_nix())
|
||||||
|
.with_context(|| format!("write {}", flake_path.display()))?;
|
||||||
|
}
|
||||||
|
git(proposed_dir, &["init", "--initial-branch=main"]).await?;
|
||||||
|
git(proposed_dir, &["add", "agent.nix", "flake.nix"]).await?;
|
||||||
|
git_commit(proposed_dir, "hive-c0re init").await?;
|
||||||
}
|
}
|
||||||
std::fs::create_dir_all(proposed_dir)
|
// Idempotently wire the `applied` remote — purely for the
|
||||||
.with_context(|| format!("create {}", proposed_dir.display()))?;
|
// manager's ergonomics. The URL is the path inside the manager
|
||||||
let agent_path = proposed_dir.join("agent.nix");
|
// container (`/applied/<n>/.git`), where the RO bind in
|
||||||
if !agent_path.exists() {
|
// `set_nspawn_flags` makes it real. hive-c0re itself never
|
||||||
std::fs::write(&agent_path, initial_agent_nix(name))
|
// dereferences this remote; the host-side fetch in
|
||||||
.with_context(|| format!("write {}", agent_path.display()))?;
|
// `request_apply_commit` uses absolute host paths.
|
||||||
}
|
ensure_applied_remote(proposed_dir, name).await
|
||||||
git(proposed_dir, &["init", "--initial-branch=main"]).await?;
|
|
||||||
git(proposed_dir, &["add", "agent.nix"]).await?;
|
|
||||||
git_commit(proposed_dir, "hive-c0re init").await?;
|
|
||||||
Ok(())
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Set up the applied repo. Two responsibilities:
|
async fn ensure_applied_remote(proposed_dir: &Path, name: &str) -> Result<()> {
|
||||||
/// - First-spawn only: init the repo, pull proposed's initial commit
|
let want = format!("/applied/{name}/.git");
|
||||||
/// in via `git fetch`, tag it `deployed/0`. This is the *only* time
|
let existing = git_command()
|
||||||
/// hive-c0re reads from `proposed` for an agent — subsequent
|
.current_dir(proposed_dir)
|
||||||
/// proposals are fetched at `request_apply_commit` time and tagged
|
.args(["remote", "get-url", "applied"])
|
||||||
/// `proposal/<id>` (see `actions::approve` for the tag state
|
.output()
|
||||||
/// machine).
|
.await
|
||||||
/// - Every call: regenerate the untracked `flake.nix` so flake-url /
|
.with_context(|| format!("git remote get-url applied in {}", proposed_dir.display()))?;
|
||||||
/// dashboard-port changes pick up on rebuild without churning the
|
if existing.status.success() {
|
||||||
/// git log.
|
let current = String::from_utf8_lossy(&existing.stdout).trim().to_owned();
|
||||||
|
if current == want {
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
// URL drifted (path scheme changed, etc.) — re-point it.
|
||||||
|
return git(proposed_dir, &["remote", "set-url", "applied", &want]).await;
|
||||||
|
}
|
||||||
|
git(proposed_dir, &["remote", "add", "applied", &want]).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Set up the applied repo. First-spawn only: init the repo, pull
|
||||||
|
/// proposed's initial commit in via `git fetch`, tag it `deployed/0`.
|
||||||
|
/// This is the *only* time hive-c0re reads from `proposed` for an
|
||||||
|
/// agent — subsequent proposals are fetched at `request_apply_commit`
|
||||||
|
/// time and tagged `proposal/<id>` (see `actions::approve` for the
|
||||||
|
/// tag state machine).
|
||||||
///
|
///
|
||||||
/// `proposed_dir` is `None` on rebuild paths that just want the flake
|
/// `proposed_dir` is `None` on rebuild paths where the repo already
|
||||||
/// refreshed.
|
/// exists — we just verify it's the right shape and bail otherwise.
|
||||||
|
/// Unlike the pre-overhaul code path, `flake.nix` is no longer
|
||||||
|
/// regenerated at the host level: it's tracked in proposed (seeded by
|
||||||
|
/// `setup_proposed`) and rides along on every fetch.
|
||||||
pub async fn setup_applied(
|
pub async fn setup_applied(
|
||||||
applied_dir: &Path,
|
applied_dir: &Path,
|
||||||
proposed_dir: Option<&Path>,
|
proposed_dir: Option<&Path>,
|
||||||
name: &str,
|
name: &str,
|
||||||
hyperhive_flake: &str,
|
|
||||||
dashboard_port: u16,
|
|
||||||
) -> Result<()> {
|
) -> Result<()> {
|
||||||
std::fs::create_dir_all(applied_dir)
|
std::fs::create_dir_all(applied_dir)
|
||||||
.with_context(|| format!("create {}", applied_dir.display()))?;
|
.with_context(|| format!("create {}", applied_dir.display()))?;
|
||||||
|
|
||||||
// 1. First-spawn git init from proposed (or pre-overhaul detection).
|
|
||||||
if !applied_dir.join(".git").exists() {
|
if !applied_dir.join(".git").exists() {
|
||||||
let Some(proposed) = proposed_dir else {
|
let Some(proposed) = proposed_dir else {
|
||||||
bail!(
|
bail!(
|
||||||
|
|
@ -339,60 +434,18 @@ pub async fn setup_applied(
|
||||||
.await
|
.await
|
||||||
.is_err()
|
.is_err()
|
||||||
{
|
{
|
||||||
// Pre-overhaul applied repo — agent.nix is tracked directly,
|
// Pre-overhaul applied repo — no deployed/* tag scheme,
|
||||||
// commits authored by hive-c0re, no deployed/* tag scheme.
|
// flake.nix may be untracked, agent.nix possibly authored by
|
||||||
// No in-place migration; fail loudly so the operator purges.
|
// hive-c0re directly. The startup auto-migration fixes this
|
||||||
|
// in place; if it didn't run (or got skipped), surface a
|
||||||
|
// clear error.
|
||||||
bail!(
|
bail!(
|
||||||
"applied repo at {} predates the tag-driven config flow. \
|
"applied repo at {} predates the meta-flake layout. \
|
||||||
Run `hive-c0re destroy --purge {name}` and re-spawn.",
|
Restart hive-c0re to let the auto-migration run, or \
|
||||||
|
destroy --purge {name} and re-spawn.",
|
||||||
applied_dir.display()
|
applied_dir.display()
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
// 2. (Re)write the untracked wrapper flake. Tracked files in the
|
|
||||||
// working tree (agent.nix and anything the manager committed) are
|
|
||||||
// untouched.
|
|
||||||
let port = agent_web_port(name);
|
|
||||||
let base = flake_base(name);
|
|
||||||
let service = if is_manager(name) {
|
|
||||||
"hive-m1nd"
|
|
||||||
} else {
|
|
||||||
"hive-ag3nt"
|
|
||||||
};
|
|
||||||
let description = if is_manager(name) {
|
|
||||||
format!("hyperhive manager {name}")
|
|
||||||
} else {
|
|
||||||
format!("hyperhive sub-agent {name}")
|
|
||||||
};
|
|
||||||
let flake_body = format!(
|
|
||||||
r#"{{
|
|
||||||
description = "{description}";
|
|
||||||
inputs.hyperhive.url = "{hyperhive_flake}";
|
|
||||||
outputs =
|
|
||||||
{{ hyperhive, ... }}:
|
|
||||||
{{
|
|
||||||
nixosConfigurations.default = hyperhive.nixosConfigurations.{base}.extendModules {{
|
|
||||||
modules = [
|
|
||||||
./agent.nix
|
|
||||||
{{
|
|
||||||
programs.git.config.user = {{
|
|
||||||
name = "{name}";
|
|
||||||
email = "{name}@hyperhive";
|
|
||||||
}};
|
|
||||||
systemd.services.{service}.environment = {{
|
|
||||||
HIVE_PORT = "{port}";
|
|
||||||
HIVE_LABEL = "{name}";
|
|
||||||
HIVE_DASHBOARD_PORT = "{dashboard_port}";
|
|
||||||
}};
|
|
||||||
}}
|
|
||||||
];
|
|
||||||
}};
|
|
||||||
}};
|
|
||||||
}}
|
|
||||||
"#,
|
|
||||||
);
|
|
||||||
std::fs::write(applied_dir.join("flake.nix"), flake_body)
|
|
||||||
.with_context(|| format!("write {}/flake.nix", applied_dir.display()))?;
|
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -423,10 +476,18 @@ fn ensure_state_dir(notes_dir: &Path) -> Result<()> {
|
||||||
|
|
||||||
fn initial_agent_nix(name: &str) -> String {
|
fn initial_agent_nix(name: &str) -> String {
|
||||||
format!(
|
format!(
|
||||||
"{{ ... }}:\n{{\n # Per-agent overrides for {name}. The manager edits this\n # file (and commits) to customise the agent's NixOS config.\n}}\n",
|
"{{ config, pkgs, lib, ... }}:\n{{\n # Per-agent overrides for {name}. This is a regular NixOS module\n # — add packages, services, modules, imports as needed.\n #\n # imports = [ ./extra-module.nix ];\n # environment.systemPackages = with pkgs; [ ];\n}}\n",
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Module-only flake exposed by every agent's repo. Consumed by the
|
||||||
|
/// hive-c0re-owned meta flake at `/var/lib/hyperhive/meta/` as a flake
|
||||||
|
/// input. Identity injection (`HIVE_PORT` / `HIVE_LABEL` / dashboard
|
||||||
|
/// port / git committer) lives in the meta flake's wrapper, not here.
|
||||||
|
pub fn initial_flake_nix() -> &'static str {
|
||||||
|
"{\n description = \"hyperhive agent\";\n inputs = { };\n outputs = { self }: {\n nixosModules.default = import ./agent.nix;\n };\n}\n"
|
||||||
|
}
|
||||||
|
|
||||||
async fn git_commit(dir: &Path, message: &str) -> Result<()> {
|
async fn git_commit(dir: &Path, message: &str) -> Result<()> {
|
||||||
git(
|
git(
|
||||||
dir,
|
dir,
|
||||||
|
|
@ -617,6 +678,10 @@ const HOST_AGENTS_ROOT: &str = "/var/lib/hyperhive/agents";
|
||||||
/// `APPLIED_STATE_ROOT` in coordinator.rs.
|
/// `APPLIED_STATE_ROOT` in coordinator.rs.
|
||||||
const HOST_APPLIED_ROOT: &str = "/var/lib/hyperhive/applied";
|
const HOST_APPLIED_ROOT: &str = "/var/lib/hyperhive/applied";
|
||||||
|
|
||||||
|
/// On-host meta repo root, mirrored RO into the manager. Matches
|
||||||
|
/// `meta::meta_dir()` but duplicated here so lifecycle stays a leaf.
|
||||||
|
const HOST_META_ROOT: &str = "/var/lib/hyperhive/meta";
|
||||||
|
|
||||||
fn set_nspawn_flags(
|
fn set_nspawn_flags(
|
||||||
container: &str,
|
container: &str,
|
||||||
runtime_dir: &Path,
|
runtime_dir: &Path,
|
||||||
|
|
@ -632,6 +697,14 @@ fn set_nspawn_flags(
|
||||||
notes = notes_dir.display(),
|
notes = notes_dir.display(),
|
||||||
);
|
);
|
||||||
if container == MANAGER_NAME {
|
if container == MANAGER_NAME {
|
||||||
|
use std::fmt::Write as _;
|
||||||
|
// systemd-nspawn refuses to start a container whose bind
|
||||||
|
// source doesn't exist. The meta repo is created by the
|
||||||
|
// startup migration, but make sure the directory is there
|
||||||
|
// before the manager comes up in case set_nspawn_flags fires
|
||||||
|
// first (e.g. cold start with no agents).
|
||||||
|
std::fs::create_dir_all(HOST_META_ROOT)
|
||||||
|
.with_context(|| format!("create {HOST_META_ROOT}"))?;
|
||||||
// Manager edits sub-agent proposed/ repos and its own. RW so it can
|
// Manager edits sub-agent proposed/ repos and its own. RW so it can
|
||||||
// git-commit. Sub-agents see only their own /run/hive socket and
|
// git-commit. Sub-agents see only their own /run/hive socket and
|
||||||
// /root/.claude (no /agents or /applied).
|
// /root/.claude (no /agents or /applied).
|
||||||
|
|
@ -642,7 +715,11 @@ fn set_nspawn_flags(
|
||||||
// denied tags into its proposed clones and diff against
|
// denied tags into its proposed clones and diff against
|
||||||
// what's actually deployed. RO bind makes destructive git
|
// what's actually deployed. RO bind makes destructive git
|
||||||
// plumbing inside the container unable to corrupt applied.
|
// plumbing inside the container unable to corrupt applied.
|
||||||
use std::fmt::Write as _;
|
//
|
||||||
|
// /meta is a third RO mount exposing the system-wide deploy
|
||||||
|
// flake (`git log /meta --oneline` shows every deploy across
|
||||||
|
// every agent; `cat /meta/flake.lock` resolves which sha each
|
||||||
|
// agent is pinned at right now).
|
||||||
let _ = write!(
|
let _ = write!(
|
||||||
binds,
|
binds,
|
||||||
" --bind={HOST_AGENTS_ROOT}:{CONTAINER_MANAGER_AGENTS_MOUNT}",
|
" --bind={HOST_AGENTS_ROOT}:{CONTAINER_MANAGER_AGENTS_MOUNT}",
|
||||||
|
|
@ -651,6 +728,11 @@ fn set_nspawn_flags(
|
||||||
binds,
|
binds,
|
||||||
" --bind-ro={HOST_APPLIED_ROOT}:{CONTAINER_MANAGER_APPLIED_MOUNT}",
|
" --bind-ro={HOST_APPLIED_ROOT}:{CONTAINER_MANAGER_APPLIED_MOUNT}",
|
||||||
);
|
);
|
||||||
|
let _ = write!(
|
||||||
|
binds,
|
||||||
|
" --bind-ro={HOST_META_ROOT}:{mount}",
|
||||||
|
mount = crate::meta::CONTAINER_MANAGER_META_MOUNT,
|
||||||
|
);
|
||||||
}
|
}
|
||||||
let bind_flag = format!("EXTRA_NSPAWN_FLAGS=\"{binds}\"");
|
let bind_flag = format!("EXTRA_NSPAWN_FLAGS=\"{binds}\"");
|
||||||
let mut lines: Vec<String> = original
|
let mut lines: Vec<String> = original
|
||||||
|
|
|
||||||
|
|
@ -17,6 +17,8 @@ mod dashboard;
|
||||||
mod events_vacuum;
|
mod events_vacuum;
|
||||||
mod lifecycle;
|
mod lifecycle;
|
||||||
mod manager_server;
|
mod manager_server;
|
||||||
|
mod meta;
|
||||||
|
mod migrate;
|
||||||
mod operator_questions;
|
mod operator_questions;
|
||||||
mod server;
|
mod server;
|
||||||
|
|
||||||
|
|
@ -96,6 +98,15 @@ async fn main() -> Result<()> {
|
||||||
} => {
|
} => {
|
||||||
let coord = Arc::new(Coordinator::open(&db, hyperhive_flake, dashboard_port)?);
|
let coord = Arc::new(Coordinator::open(&db, hyperhive_flake, dashboard_port)?);
|
||||||
manager_server::start(coord.clone())?;
|
manager_server::start(coord.clone())?;
|
||||||
|
// Idempotent pre-flight: rewrite pre-meta-layout applied
|
||||||
|
// repos, ensure proposed repos carry the `applied`
|
||||||
|
// remote, bootstrap the meta repo, repoint containers at
|
||||||
|
// `meta#<name>` (one-shot, guarded by a marker file).
|
||||||
|
// Runs before manager auto-spawn so the new manager is
|
||||||
|
// built against meta from the first attempt.
|
||||||
|
if let Err(e) = migrate::run(&coord).await {
|
||||||
|
tracing::warn!(error = ?e, "startup migration failed");
|
||||||
|
}
|
||||||
// Auto-create the manager container if it isn't there yet. Block
|
// Auto-create the manager container if it isn't there yet. Block
|
||||||
// on this — without hm1nd the system has no manager harness.
|
// on this — without hm1nd the system has no manager harness.
|
||||||
// Failures are logged but allowed: a broken auto-spawn shouldn't
|
// Failures are logged but allowed: a broken auto-spawn shouldn't
|
||||||
|
|
|
||||||
278
hive-c0re/src/meta.rs
Normal file
278
hive-c0re/src/meta.rs
Normal file
|
|
@ -0,0 +1,278 @@
|
||||||
|
//! Single hive-c0re-owned flake at `/var/lib/hyperhive/meta/` that
|
||||||
|
//! consumes every agent's applied repo as a flake input and exports one
|
||||||
|
//! `nixosConfiguration` per agent. Containers run against
|
||||||
|
//! `--flake /var/lib/hyperhive/meta#<name>`; lifecycle ops here drive the
|
||||||
|
//! lock file so meta's git log is the system-wide deploy audit trail.
|
||||||
|
//!
|
||||||
|
//! Flow:
|
||||||
|
//! - `sync_agents` (idempotent) — render `flake.nix` for the current
|
||||||
|
//! agent set, init the repo on first call, relock if the rendered
|
||||||
|
//! contents changed, commit. Used by spawn / destroy / startup
|
||||||
|
//! migration.
|
||||||
|
//! - `prepare_deploy` + `finalize_deploy` / `abort_deploy` — two-phase
|
||||||
|
//! for the `request_apply_commit` path so a failed
|
||||||
|
//! `nixos-container update` leaves no orphan commit in meta. Prepare
|
||||||
|
//! writes the new lock without committing; finalize commits with the
|
||||||
|
//! deploy message; abort `git restore`s the lock back.
|
||||||
|
//! - `lock_update_hyperhive` — one-shot for the auto-update path.
|
||||||
|
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
|
||||||
|
use anyhow::{Context, Result, bail};
|
||||||
|
use tokio::process::Command;
|
||||||
|
|
||||||
|
use crate::lifecycle;
|
||||||
|
|
||||||
|
const META_ROOT: &str = "/var/lib/hyperhive/meta";
|
||||||
|
const APPLIED_ROOT: &str = "/var/lib/hyperhive/applied";
|
||||||
|
const GIT_NAME: &str = "hive-c0re";
|
||||||
|
const GIT_EMAIL: &str = "hive-c0re@hyperhive";
|
||||||
|
|
||||||
|
/// Where the manager sees this directory inside its container (RO bind).
|
||||||
|
#[allow(dead_code)] // wired up by set_nspawn_flags in a follow-up commit
|
||||||
|
pub const CONTAINER_MANAGER_META_MOUNT: &str = "/meta";
|
||||||
|
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct AgentSpec {
|
||||||
|
pub name: String,
|
||||||
|
pub is_manager: bool,
|
||||||
|
pub port: u16,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[must_use]
|
||||||
|
pub fn meta_dir() -> PathBuf {
|
||||||
|
PathBuf::from(META_ROOT)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Idempotently reconcile the meta repo with the current agent set.
|
||||||
|
/// First call inits the git repo, runs `nix flake lock`, and lands a
|
||||||
|
/// seed commit. Subsequent calls only touch `flake.nix` when the
|
||||||
|
/// rendered contents differ from disk; an unchanged `flake.nix` is a
|
||||||
|
/// no-op.
|
||||||
|
#[allow(dead_code)] // first caller lands in a later commit
|
||||||
|
pub async fn sync_agents(
|
||||||
|
hyperhive_flake: &str,
|
||||||
|
dashboard_port: u16,
|
||||||
|
agents: &[AgentSpec],
|
||||||
|
) -> Result<()> {
|
||||||
|
let dir = meta_dir();
|
||||||
|
std::fs::create_dir_all(&dir).with_context(|| format!("create {}", dir.display()))?;
|
||||||
|
|
||||||
|
let new_flake = render_flake(hyperhive_flake, dashboard_port, agents);
|
||||||
|
let flake_path = dir.join("flake.nix");
|
||||||
|
let on_disk = std::fs::read_to_string(&flake_path).unwrap_or_default();
|
||||||
|
let initial = !dir.join(".git").exists();
|
||||||
|
|
||||||
|
if !initial && on_disk == new_flake {
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
|
||||||
|
std::fs::write(&flake_path, &new_flake)
|
||||||
|
.with_context(|| format!("write {}", flake_path.display()))?;
|
||||||
|
|
||||||
|
if initial {
|
||||||
|
git(&dir, &["init", "--initial-branch=main"]).await?;
|
||||||
|
}
|
||||||
|
nix(&dir, &["flake", "lock"]).await?;
|
||||||
|
git(&dir, &["add", "-A"]).await?;
|
||||||
|
let msg = if initial {
|
||||||
|
format!("seed meta from {} agent(s)", agents.len())
|
||||||
|
} else {
|
||||||
|
"regenerate meta flake".to_owned()
|
||||||
|
};
|
||||||
|
git_commit(&dir, &msg).await?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Phase 1 of an apply-commit deploy. Updates the locked rev of
|
||||||
|
/// `agent-<name>` to whatever `applied/<name>/main` currently points
|
||||||
|
/// at. **Doesn't commit** — caller must follow with
|
||||||
|
/// `finalize_deploy` on build success or `abort_deploy` on failure.
|
||||||
|
#[allow(dead_code)] // wired up by actions::run_apply_commit in a later commit
|
||||||
|
pub async fn prepare_deploy(name: &str) -> Result<()> {
|
||||||
|
let dir = meta_dir();
|
||||||
|
let input = format!("agent-{name}");
|
||||||
|
nix(&dir, &["flake", "lock", "--update-input", &input]).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Phase 2-success. Commits the staged `flake.lock` change with a
|
||||||
|
/// deploy-shaped message. No-op (clean working tree) is tolerated —
|
||||||
|
/// some lock-updates resolve to the same rev that's already locked.
|
||||||
|
#[allow(dead_code)]
|
||||||
|
pub async fn finalize_deploy(name: &str, sha: &str, tag: &str) -> Result<()> {
|
||||||
|
let dir = meta_dir();
|
||||||
|
if git_is_clean(&dir).await? {
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
git(&dir, &["add", "flake.lock"]).await?;
|
||||||
|
let short = &sha[..sha.len().min(12)];
|
||||||
|
git_commit(&dir, &format!("deploy {name} {tag} {short}")).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Phase 2-failure. Drops the uncommitted `flake.lock` change so meta
|
||||||
|
/// stays pinned at the previously-deployed shas. The failed proposal
|
||||||
|
/// is still captured in `applied/<n>`'s annotated `failed/<id>` tag —
|
||||||
|
/// meta's history only carries successful deploys.
|
||||||
|
#[allow(dead_code)]
|
||||||
|
pub async fn abort_deploy() -> Result<()> {
|
||||||
|
let dir = meta_dir();
|
||||||
|
git(&dir, &["restore", "flake.lock"]).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One-shot used by the manual-rebuild path: relock just one
|
||||||
|
/// agent's input and commit the lock change if any. Single-phase
|
||||||
|
/// (no separate finalize) because rebuild has no failure-revert
|
||||||
|
/// semantics — it always wants the latest main.
|
||||||
|
#[allow(dead_code)] // wired up by lifecycle::rebuild in this commit
|
||||||
|
pub async fn lock_update_for_rebuild(name: &str) -> Result<()> {
|
||||||
|
let dir = meta_dir();
|
||||||
|
let input = format!("agent-{name}");
|
||||||
|
nix(&dir, &["flake", "lock", "--update-input", &input]).await?;
|
||||||
|
if !git_is_clean(&dir).await? {
|
||||||
|
git(&dir, &["add", "flake.lock"]).await?;
|
||||||
|
git_commit(&dir, &format!("rebuild {name}: lock update")).await?;
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One-shot used by the auto-update path: pin the latest hyperhive
|
||||||
|
/// rev, commit if the lock changed. Cheaper than `sync_agents`
|
||||||
|
/// because the per-agent inputs aren't touched.
|
||||||
|
#[allow(dead_code)]
|
||||||
|
pub async fn lock_update_hyperhive() -> Result<()> {
|
||||||
|
let dir = meta_dir();
|
||||||
|
nix(&dir, &["flake", "lock", "--update-input", "hyperhive"]).await?;
|
||||||
|
if !git_is_clean(&dir).await? {
|
||||||
|
git(&dir, &["add", "flake.lock"]).await?;
|
||||||
|
git_commit(&dir, "bump hyperhive").await?;
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
fn render_flake(hyperhive_flake: &str, dashboard_port: u16, agents: &[AgentSpec]) -> String {
|
||||||
|
use std::fmt::Write as _;
|
||||||
|
let mut out = String::new();
|
||||||
|
out.push_str("{\n description = \"hyperhive deployed agents\";\n inputs = {\n");
|
||||||
|
let _ = writeln!(out, " hyperhive.url = \"{hyperhive_flake}\";");
|
||||||
|
for spec in agents {
|
||||||
|
let _ = writeln!(
|
||||||
|
out,
|
||||||
|
" agent-{}.url = \"git+file://{APPLIED_ROOT}/{}\";",
|
||||||
|
spec.name, spec.name,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
out.push_str(" };\n outputs =\n { self, hyperhive, ... }@inputs:\n let\n");
|
||||||
|
let _ = writeln!(
|
||||||
|
out,
|
||||||
|
" dashboardPort = {dashboard_port};\n mkAgent = {{ name, isManager, port }}:"
|
||||||
|
);
|
||||||
|
out.push_str(
|
||||||
|
r#" let
|
||||||
|
base = if isManager
|
||||||
|
then hyperhive.nixosConfigurations.manager
|
||||||
|
else hyperhive.nixosConfigurations.agent-base;
|
||||||
|
input = inputs."agent-${name}";
|
||||||
|
service = if isManager then "hive-m1nd" else "hive-ag3nt";
|
||||||
|
in
|
||||||
|
base.extendModules {
|
||||||
|
modules = [
|
||||||
|
input.nixosModules.default
|
||||||
|
{
|
||||||
|
programs.git.config.user = {
|
||||||
|
name = name;
|
||||||
|
email = "${name}@hyperhive";
|
||||||
|
};
|
||||||
|
systemd.services.${service}.environment = {
|
||||||
|
HIVE_PORT = toString port;
|
||||||
|
HIVE_LABEL = name;
|
||||||
|
HIVE_DASHBOARD_PORT = toString dashboardPort;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
];
|
||||||
|
};
|
||||||
|
in
|
||||||
|
{
|
||||||
|
nixosConfigurations = {
|
||||||
|
"#,
|
||||||
|
);
|
||||||
|
for spec in agents {
|
||||||
|
let _ = writeln!(
|
||||||
|
out,
|
||||||
|
" {} = mkAgent {{ name = \"{}\"; isManager = {}; port = {}; }};",
|
||||||
|
spec.name,
|
||||||
|
spec.name,
|
||||||
|
if spec.is_manager { "true" } else { "false" },
|
||||||
|
spec.port,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
out.push_str(" };\n };\n}\n");
|
||||||
|
out
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn git_is_clean(dir: &Path) -> Result<bool> {
|
||||||
|
let out = lifecycle::git_command()
|
||||||
|
.current_dir(dir)
|
||||||
|
.args(["status", "--porcelain"])
|
||||||
|
.output()
|
||||||
|
.await
|
||||||
|
.with_context(|| format!("git status in {}", dir.display()))?;
|
||||||
|
Ok(out.stdout.iter().all(u8::is_ascii_whitespace))
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn git(dir: &Path, args: &[&str]) -> Result<()> {
|
||||||
|
let out = lifecycle::git_command()
|
||||||
|
.current_dir(dir)
|
||||||
|
.args(args)
|
||||||
|
.output()
|
||||||
|
.await
|
||||||
|
.with_context(|| format!("git {} in {}", args.join(" "), dir.display()))?;
|
||||||
|
if !out.status.success() {
|
||||||
|
bail!(
|
||||||
|
"git {} failed ({}): {}",
|
||||||
|
args.join(" "),
|
||||||
|
out.status,
|
||||||
|
String::from_utf8_lossy(&out.stderr).trim()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn git_commit(dir: &Path, message: &str) -> Result<()> {
|
||||||
|
git(
|
||||||
|
dir,
|
||||||
|
&[
|
||||||
|
"-c",
|
||||||
|
&format!("user.name={GIT_NAME}"),
|
||||||
|
"-c",
|
||||||
|
&format!("user.email={GIT_EMAIL}"),
|
||||||
|
"commit",
|
||||||
|
"-m",
|
||||||
|
message,
|
||||||
|
],
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn nix(dir: &Path, args: &[&str]) -> Result<()> {
|
||||||
|
// `--extra-experimental-features` belt-and-suspenders for hosts
|
||||||
|
// that haven't set this in nix.conf. The hyperhive module's
|
||||||
|
// deploy guide assumes flakes are already enabled, but the cost
|
||||||
|
// of being defensive is one extra argv each call.
|
||||||
|
let mut all = vec!["--extra-experimental-features", "nix-command flakes"];
|
||||||
|
all.extend(args);
|
||||||
|
let out = Command::new("nix")
|
||||||
|
.current_dir(dir)
|
||||||
|
.args(&all)
|
||||||
|
.output()
|
||||||
|
.await
|
||||||
|
.with_context(|| format!("nix {} in {}", args.join(" "), dir.display()))?;
|
||||||
|
if !out.status.success() {
|
||||||
|
bail!(
|
||||||
|
"nix {} failed ({}): {}",
|
||||||
|
args.join(" "),
|
||||||
|
out.status,
|
||||||
|
String::from_utf8_lossy(&out.stderr).trim()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
183
hive-c0re/src/migrate.rs
Normal file
183
hive-c0re/src/migrate.rs
Normal file
|
|
@ -0,0 +1,183 @@
|
||||||
|
//! Startup auto-migration from the pre-meta layout. Runs before
|
||||||
|
//! `auto_update::run` and consists of four phases, each idempotent:
|
||||||
|
//!
|
||||||
|
//! 1. Per-agent applied repo: rewrite `flake.nix` to the module-only
|
||||||
|
//! boilerplate if it isn't already, commit, relocate `deployed/0`
|
||||||
|
//! to HEAD so `setup_applied`'s existence check passes.
|
||||||
|
//! 2. Per-agent proposed repo: ensure the `applied` git remote
|
||||||
|
//! points at `/applied/<n>/.git` (re-runs `setup_proposed`'s
|
||||||
|
//! `ensure_applied_remote` indirectly via a host-side git call).
|
||||||
|
//! 3. Meta repo: `meta::sync_agents` over the current agent list —
|
||||||
|
//! init the repo on first call, rerender + relock if anything
|
||||||
|
//! drifted.
|
||||||
|
//! 4. Container repoint: for every existing container, run
|
||||||
|
//! `nixos-container update <c> --flake meta#<name>` so it
|
||||||
|
//! activates against the meta flake. Guarded by a marker file
|
||||||
|
//! so the (expensive) phase 4 only runs once across hive-c0re
|
||||||
|
//! restarts.
|
||||||
|
//!
|
||||||
|
//! Env kill-switch: `HIVE_SKIP_META_MIGRATION=1` skips the whole
|
||||||
|
//! migration. Use when smoke-testing one agent at a time by hand.
|
||||||
|
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::sync::Arc;
|
||||||
|
|
||||||
|
use anyhow::{Context, Result};
|
||||||
|
use tokio::process::Command;
|
||||||
|
|
||||||
|
use crate::coordinator::Coordinator;
|
||||||
|
use crate::lifecycle::{self, AGENT_PREFIX, MANAGER_NAME};
|
||||||
|
use crate::meta;
|
||||||
|
|
||||||
|
const KILL_SWITCH: &str = "HIVE_SKIP_META_MIGRATION";
|
||||||
|
|
||||||
|
/// Marker for phase 4. Once present, container repoint is skipped on
|
||||||
|
/// future restarts.
|
||||||
|
fn repoint_marker() -> PathBuf {
|
||||||
|
PathBuf::from("/var/lib/hyperhive/.meta-migration-done")
|
||||||
|
}
|
||||||
|
|
||||||
|
const MODULE_FLAKE_MARKER: &str = "nixosModules.default = import ./agent.nix";
|
||||||
|
|
||||||
|
pub async fn run(coord: &Arc<Coordinator>) -> Result<()> {
|
||||||
|
if std::env::var(KILL_SWITCH).is_ok() {
|
||||||
|
tracing::info!("migration: {KILL_SWITCH} set — skipping");
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
let names = enumerate_agents().await;
|
||||||
|
tracing::info!(count = names.len(), "migration: scanning");
|
||||||
|
|
||||||
|
// Phase 1 + 2: per-agent applied + proposed.
|
||||||
|
for name in &names {
|
||||||
|
if let Err(e) = migrate_applied_repo(name).await {
|
||||||
|
tracing::warn!(%name, error = ?e, "migration: applied repo rewrite failed");
|
||||||
|
}
|
||||||
|
if let Err(e) = lifecycle::setup_proposed(&Coordinator::agent_proposed_dir(name), name)
|
||||||
|
.await
|
||||||
|
{
|
||||||
|
tracing::warn!(%name, error = ?e, "migration: setup_proposed failed");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Phase 3: meta repo.
|
||||||
|
let agents = lifecycle::agents_for_meta_listing().await.unwrap_or_default();
|
||||||
|
if let Err(e) =
|
||||||
|
meta::sync_agents(&coord.hyperhive_flake, coord.dashboard_port, &agents).await
|
||||||
|
{
|
||||||
|
tracing::warn!(error = ?e, "migration: meta sync_agents failed");
|
||||||
|
}
|
||||||
|
|
||||||
|
// Phase 4: container repoint, guarded by marker.
|
||||||
|
if repoint_marker().exists() {
|
||||||
|
tracing::debug!("migration: phase 4 marker present, skipping repoint");
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
let mut all_ok = true;
|
||||||
|
for name in &names {
|
||||||
|
if let Err(e) = repoint_container(name).await {
|
||||||
|
tracing::warn!(%name, error = ?e, "migration: container repoint failed");
|
||||||
|
all_ok = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if all_ok && !names.is_empty()
|
||||||
|
&& let Err(e) = std::fs::write(repoint_marker(), b"done\n")
|
||||||
|
{
|
||||||
|
tracing::warn!(error = ?e, "migration: write repoint marker failed");
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn enumerate_agents() -> Vec<String> {
|
||||||
|
let containers = lifecycle::list().await.unwrap_or_default();
|
||||||
|
containers
|
||||||
|
.into_iter()
|
||||||
|
.filter_map(|c| {
|
||||||
|
if c == MANAGER_NAME {
|
||||||
|
Some(MANAGER_NAME.to_owned())
|
||||||
|
} else {
|
||||||
|
c.strip_prefix(AGENT_PREFIX).map(str::to_owned)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn migrate_applied_repo(name: &str) -> Result<()> {
|
||||||
|
let dir = Coordinator::agent_applied_dir(name);
|
||||||
|
if !dir.join(".git").exists() {
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
let flake_path = dir.join("flake.nix");
|
||||||
|
let cur = std::fs::read_to_string(&flake_path).unwrap_or_default();
|
||||||
|
if cur.contains(MODULE_FLAKE_MARKER) {
|
||||||
|
return Ok(());
|
||||||
|
}
|
||||||
|
let want = lifecycle::initial_flake_nix();
|
||||||
|
std::fs::write(&flake_path, want)
|
||||||
|
.with_context(|| format!("write {}", flake_path.display()))?;
|
||||||
|
raw_git(
|
||||||
|
&dir,
|
||||||
|
&[
|
||||||
|
"-c",
|
||||||
|
"user.name=hive-c0re",
|
||||||
|
"-c",
|
||||||
|
"user.email=hive-c0re@hyperhive",
|
||||||
|
"add",
|
||||||
|
"flake.nix",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
raw_git(
|
||||||
|
&dir,
|
||||||
|
&[
|
||||||
|
"-c",
|
||||||
|
"user.name=hive-c0re",
|
||||||
|
"-c",
|
||||||
|
"user.email=hive-c0re@hyperhive",
|
||||||
|
"commit",
|
||||||
|
"-m",
|
||||||
|
"migration: module-only flake",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
// Relocate deployed/0 to the migration commit so
|
||||||
|
// setup_applied's existence check passes.
|
||||||
|
raw_git(&dir, &["tag", "-f", "deployed/0", "HEAD"]).await?;
|
||||||
|
tracing::info!(%name, "migration: applied repo migrated to module-only flake");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn repoint_container(name: &str) -> Result<()> {
|
||||||
|
let container = lifecycle::container_name(name);
|
||||||
|
let flake_ref = format!("{}#{name}", meta::meta_dir().display());
|
||||||
|
let out = Command::new("nixos-container")
|
||||||
|
.args(["update", &container, "--flake", &flake_ref])
|
||||||
|
.output()
|
||||||
|
.await
|
||||||
|
.with_context(|| format!("nixos-container update {container}"))?;
|
||||||
|
if !out.status.success() {
|
||||||
|
anyhow::bail!(
|
||||||
|
"nixos-container update {container} exited {}: {}",
|
||||||
|
out.status,
|
||||||
|
String::from_utf8_lossy(&out.stderr).trim()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
tracing::info!(%name, %container, "migration: container repointed at meta");
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn raw_git(dir: &Path, args: &[&str]) -> Result<()> {
|
||||||
|
let out = lifecycle::git_command()
|
||||||
|
.current_dir(dir)
|
||||||
|
.args(args)
|
||||||
|
.output()
|
||||||
|
.await
|
||||||
|
.with_context(|| format!("git {} in {}", args.join(" "), dir.display()))?;
|
||||||
|
if !out.status.success() {
|
||||||
|
anyhow::bail!(
|
||||||
|
"git {} failed: {}",
|
||||||
|
args.join(" "),
|
||||||
|
String::from_utf8_lossy(&out.stderr).trim()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
Loading…
Reference in a new issue