vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

refactor(#2916): drop the two obsolete startup migrations

Browse source 
Phase 4 (repoint every container onto `meta#<n>`) and phase 5 (rename
the `root` container to `h-root`) were marker-guarded one-shots for
layouts no live hive still has: containers are rendered onto `meta#<n>`
at creation, and the `h-` prefix has been the naming for far longer than
any deployment predates. A one-shot nobody can still trigger is dead
weight, so both are gone along with `repoint_container`,
`rename_manager_container`, `CONTAINER_TIMEOUT` and the two marker paths.

Phase 6 was not obsolete, only misplaced. Ruth's tool groups are now
seeded by `ensure_root_agent` on the one path that creates her, rather
than re-asserted on every hive-c0re boot. The skip-if-already-set guard
survives the move: a destroy+recreate under the same name must not reset
an operator's chosen group set back to MANAGER_DEFAULT.

That also settles a latent bug. Phase 4's marker check was a `return`,
not a skip, so on any hive carrying the marker phases 5 and 6 never ran
at all — the tool-group backfill, whose whole job was preventing a silent
privilege downgrade, has not executed here in a long time. Moving it to
create-time removes the question rather than answering it.

What stays is convergence: three unguarded, idempotent phases that re-run
each boot and no-op once their state is right. The module doc now names
the three categories so the next person can tell which kind they're
adding.
This commit is contained in:
atlas 2026-08-01 21:01:13 +02:00 • committed by mara
commit e02ac1e86e
5 changed files with 70 additions and 228 deletions
Download patch file Download diff file Expand all files Collapse all files

16
docs/approvals.md
View file

@ -478,12 +478,16 @@ each phase is a no-op once already applied. Behaviour:
dirs, or claude creds.
- Meta-flake phase: rewrites each `applied/<n>/flake.nix` to
the module-only boilerplate, wires the `applied` remote in
each proposed repo, bootstraps the meta repo from the
current agent list, and `nixos-container update`s every
container at `meta#<n>`. The expensive last step is
guarded by `/var/lib/hyperhive/.meta-migration-done` so
it only runs once across hive-c0re restarts. Set
`HIVE_SKIP_META_MIGRATION=1` on the service to defer.
each proposed repo, and bootstraps the meta repo from the
current agent list. Set `HIVE_SKIP_META_MIGRATION=1` on the
service to defer.
A further step used to `nixos-container update` every
container onto `meta#<n>`, guarded by a marker file so it
ran once per hive. It is gone: containers have been rendered
onto `meta#<n>` at creation for long enough that no live hive
needs the repoint, and a one-shot nobody can still trigger is
dead weight. Same for the `root` → `h-root` container rename.
No state loss in either migration. claude creds, /state/
notes, the events DB, proposed history, and applied history

10
docs/persistence.md
View file

@ -324,11 +324,11 @@ Contents:
The root agent has the meta dir RO-mounted at `/meta/`.
Marker file `/var/lib/hyperhive/.meta-migration-done` is
written by the startup migration after every container has
been repointed at `meta#<n>`. Removing it forces a re-run on
next hive-c0re start (idempotent — only the actual repoint
step would re-fire).
There is no longer a `.meta-migration-done` marker: the
one-shot container repoint it guarded has been removed, since
containers are rendered onto `meta#<n>` at creation. A stale
marker file left over from an older hive is inert and can be
deleted.
## Destroy vs purge

229
hive-c0re/src/migrate.rs
View file

@ -1,8 +1,22 @@
//! Startup auto-migration. Six idempotent phases: applied repo,
//! proposed repo, meta repo, container repoint, root→h-root rename,
//! and manager tool-groups backfill.
//! Kill-switch: `HIVE_SKIP_META_MIGRATION=1`. Full migration sequence
//! and phase details: `docs/approvals.md::Migration from the pre-tag`.
//! Startup convergence. Three phases, all idempotent and unguarded:
//! harness files, applied + proposed repos, meta repo. They re-run every
//! boot on purpose — each one is a no-op once its state is already
//! correct.
//!
//! Deliberately *not* here, and the distinction is the point:
//!
//! - **One-shot, marker-guarded migrations.** Two used to live here
//! (repointing containers onto the meta flake, renaming `root` to
//! `h-root`); both targeted layouts no live hive still has. Add one
//! only if it cannot be expressed as convergence, and expect to delete
//! it once every hive has passed it.
//! - **Create-time setup.** Ruth's tool groups were backfilled here on
//! every boot; they are now seeded where she is created
//! (`workers::auto_update::ensure_root_agent`). A thing that is true
//! from birth does not need re-asserting each morning.
//!
//! Kill-switch: `HIVE_SKIP_META_MIGRATION=1`. Full sequence and phase
//! details: `docs/approvals.md::Migration from the pre-tag`.
use std::path::Path;
use std::sync::Arc;
@ -14,21 +28,17 @@ use tokio::process::Command;
use crate::coordinator::Coordinator;
use crate::lifecycle::{self, AGENT_PREFIX, MANAGER_CONTAINER, MANAGER_NAME};
use crate::meta;
use crate::tool_groups;
const KILL_SWITCH: &str = "HIVE_SKIP_META_MIGRATION";
/// Per-shellout timeouts for the blocking startup migration. `run` is
/// Per-shellout timeout for the blocking startup convergence. `run` is
/// awaited *before* the daemon starts serving (main.rs), so any child
/// process that wedges here freezes the whole daemon — admin socket +
/// dashboard included — with no diagnostics: a git/container shellout was
/// observed blocked for 86min under a concurrent `nixos-rebuild`. Every
/// shellout now runs under a timeout that kills the child on elapse, so a
/// stuck migration degrades to a logged warning instead of a hung boot.
/// Git ops are quick; `nixos-container update` can legitimately trigger a
/// nix build, so it gets a much longer budget.
/// dashboard included — with no diagnostics: a git shellout was observed
/// blocked for 86min under a concurrent `nixos-rebuild`. Every shellout
/// runs under a timeout that kills the child on elapse, so a stuck phase
/// degrades to a logged warning instead of a hung boot.
const GIT_TIMEOUT: Duration = Duration::from_mins(2);
const CONTAINER_TIMEOUT: Duration = Duration::from_mins(10);
/// Substring that identifies the *current* agent flake boilerplate.
/// Bumped whenever the template changes so the startup migration
@ -95,46 +105,6 @@ pub async fn run(coord: &Arc<Coordinator>) -> Result<()> {
Ok(Ok(())) => {}
}
// Phase 4: container repoint, guarded by marker.
if crate::paths::meta_migration_marker().exists() {
tracing::debug!("migration: phase 4 marker present, skipping repoint");
return Ok(());
}
tracing::debug!("migration: phase 4 (container repoint)");
let mut all_ok = true;
for name in &names {
// Mark Rebuilding so the crash watcher skips this container
// during the brief stop+start window the nixos-container
// update activation triggers. Without this, crash_watch
// would fire ContainerCrash for every agent here and the
// manager would spuriously try to recover them.
// No queue node behind this one — migration repoints containers
// directly — so nothing in the graph marks the stop as intended.
let guard = coord.suppress_crash_watch(name.as_str());
let result = repoint_container(name.as_str()).await;
drop(guard);
if let Err(e) = result {
tracing::warn!(%name, error = ?e, "migration: container repoint failed");
all_ok = false;
}
}
if all_ok
&& !names.is_empty()
&& let Err(e) = std::fs::write(crate::paths::meta_migration_marker(), b"done\n")
{
tracing::warn!(error = ?e, "migration: write repoint marker failed");
}
// Phase 5: rename `root` nixos-container to `h-root` for naming
// consistency with sub-agents. Guarded by marker; skipped on
// fresh installs (conf file absent) and after first successful run.
rename_manager_container(coord).await;
// Phase 6: ensure ruth has explicit tool groups so removing the
// role-based fallback (Role::Manager → MANAGER_DEFAULT) doesn't
// silently strip her privileged tools on next rebuild.
backfill_manager_tool_groups(&names);
Ok(())
}
@ -169,106 +139,6 @@ fn migrate_harness_files(name: &hive_types::Ident) {
}
}
/// Phase 5: rename the `root` nixos-container to `h-root` so the
/// manager container name is consistent with the `h-` prefix used by
/// all sub-agents. Idempotent and marker-guarded. Steps:
///
/// 1. Check `/etc/nixos-containers/root.conf` exists (old name present).
/// 2. Stop the `root` container.
/// 3. Copy `root.conf` → `h-root.conf`.
/// 4. Move `/var/lib/nixos-containers/root/` → `h-root/` (if present).
/// 5. `systemctl daemon-reload` so systemd sees the new unit name.
/// 6. `nixos-container start h-root`.
/// 7. Write the done marker.
///
/// Best-effort: logs warnings on failure. A failed rename leaves both
/// conf files present; on the next hive-c0re start the marker is
/// absent so the phase retries.
async fn rename_manager_container(coord: &Arc<Coordinator>) {
if crate::paths::hroot_rename_marker().exists() {
return;
}
let old_conf = std::path::PathBuf::from("/etc/nixos-containers/root.conf");
let new_conf = std::path::PathBuf::from("/etc/nixos-containers/h-root.conf");
if !old_conf.exists() {
// Fresh install — root container was never created under the old name.
let _ = std::fs::write(crate::paths::hroot_rename_marker(), b"done\n");
return;
}
if new_conf.exists() {
// Already renamed (but marker was lost — write it and return).
tracing::info!("migration phase 5: h-root.conf already present, marking done");
let _ = std::fs::write(crate::paths::hroot_rename_marker(), b"done\n");
return;
}
tracing::info!("migration phase 5: renaming root container to h-root");
// The old container is stopped immediately below, on purpose.
let _guard = coord.suppress_crash_watch(MANAGER_NAME);
// Stop the old container. Abort if stop fails — continuing with a
// running `root` and then starting `h-root` risks two manager
// instances racing for the same broker / state files.
match Command::new("nixos-container")
.args(["stop", "root"])
.status()
.await
{
Ok(s) if s.success() => {}
Ok(s) => {
tracing::warn!(status = %s, "migration phase 5: nixos-container stop root failed — aborting");
return;
}
Err(e) => {
tracing::warn!(error = ?e, "migration phase 5: nixos-container stop root failed — aborting");
return;
}
}
// Copy conf file.
if let Err(e) = std::fs::copy(&old_conf, &new_conf) {
tracing::warn!(error = ?e, "migration phase 5: copy root.conf failed — aborting");
return;
}
// Move rootfs if it exists (may be absent for ephemeral containers).
let old_rootfs = std::path::PathBuf::from("/var/lib/nixos-containers/root");
let new_rootfs = std::path::PathBuf::from("/var/lib/nixos-containers/h-root");
if old_rootfs.exists()
&& !new_rootfs.exists()
&& let Err(e) = std::fs::rename(&old_rootfs, &new_rootfs)
{
tracing::warn!(error = ?e, "migration phase 5: rename rootfs failed (non-fatal)");
}
// Daemon reload so systemd picks up the new container@h-root unit.
if let Err(e) = Command::new("systemctl")
.args(["daemon-reload"])
.status()
.await
{
tracing::warn!(error = ?e, "migration phase 5: systemctl daemon-reload failed");
}
// Start the renamed container.
if let Err(e) = Command::new("nixos-container")
.args(["start", "h-root"])
.status()
.await
{
tracing::warn!(error = ?e, "migration phase 5: nixos-container start h-root failed");
return;
}
tracing::info!("migration phase 5: root container renamed to h-root");
let _ = std::fs::write(crate::paths::hroot_rename_marker(), b"done\n");
// Clean up the old conf file so `nixos-container list` doesn't show
// a stale stopped `root` entry. Best-effort; a failure here is
// harmless — h-root is already running and the marker is written.
if let Err(e) = std::fs::remove_file(&old_conf) {
tracing::warn!(error = ?e, "migration phase 5: remove old root.conf failed (non-fatal)");
}
}
async fn enumerate_agents() -> Vec<hive_types::Ident> {
let containers = lifecycle::list().await.unwrap_or_default();
containers
@ -328,57 +198,6 @@ async fn migrate_applied_repo(name: &str) -> Result<()> {
Ok(())
}
async fn repoint_container(name: &str) -> Result<()> {
let container = lifecycle::container_name(name);
let flake_ref = format!("{}#{name}", crate::paths::meta_root().display());
let mut cmd = Command::new("nixos-container");
cmd.args(["update", &container, "--flake", &flake_ref]);
let out = output_with_timeout(
cmd,
CONTAINER_TIMEOUT,
&format!("nixos-container update {container}"),
)
.await?;
if !out.status.success() {
anyhow::bail!(
"nixos-container update {container} exited {}: {}",
out.status,
String::from_utf8_lossy(&out.stderr).trim()
);
}
tracing::info!(%name, %container, "migration: container repointed at meta");
Ok(())
}
/// Phase 6: if ruth is a deployed agent and has no explicit entry in
/// `tool-groups.json`, set her groups to `MANAGER_DEFAULT` (all groups).
/// Idempotent — skips when entry already present. Prevents a silent tool
/// downgrade when upgrading from a build that relied on the manager-flavor
/// fallback in `effective_tool_groups()`.
fn backfill_manager_tool_groups(names: &[hive_types::Ident]) {
if !names.iter().any(|n| n.as_str() == MANAGER_NAME) {
return; // ruth not deployed — nothing to backfill
}
let existing = tool_groups::groups_for(MANAGER_NAME);
if !existing.is_empty() {
tracing::debug!("migration: ruth already has explicit tool groups — skipping backfill");
return;
}
let all_groups: Vec<String> = hive_sh4re::ToolGroup::MANAGER_DEFAULT
.iter()
.map(|g| g.as_str().to_owned())
.collect();
match tool_groups::set_groups(MANAGER_NAME, &all_groups) {
Ok(()) => tracing::info!(
"migration: backfilled ruth's tool groups to MANAGER_DEFAULT (all groups)"
),
Err(e) => tracing::warn!(
error = ?e,
"migration: failed to backfill ruth's tool groups — she may lose privileged tools on next rebuild"
),
}
}
/// Run a command to completion under a timeout, capturing its output. On
/// timeout the child is killed (`kill_on_drop`) and an error is returned,
/// so a wedged shellout can never freeze startup migration. `what` is a

12
hive-c0re/src/paths.rs
View file

@ -248,18 +248,6 @@ pub fn matrix_register_token() -> PathBuf {
state_root().join("matrix-register-token")
}
/// `.meta-migration-done` — one-shot marker: legacy meta layout migrated.
#[must_use]
pub fn meta_migration_marker() -> PathBuf {
state_root().join(".meta-migration-done")
}
/// `.hroot-rename-done` — one-shot marker: legacy hive-root rename applied.
#[must_use]
pub fn hroot_rename_marker() -> PathBuf {
state_root().join(".hroot-rename-done")
}
/// `/run/hyperhive` — the runtime root (host admin socket + per-agent dirs).
#[must_use]
pub fn runtime_root() -> PathBuf {

31
hive-c0re/src/workers/auto_update.rs
View file

@ -23,6 +23,7 @@ use anyhow::Result;
use crate::coordinator::Coordinator;
use crate::lifecycle::{self, AGENT_PREFIX, MANAGER_NAME};
use crate::tool_groups;
/// Resolve the current rev of `hyperhive_flake`. For a path on disk we
/// canonicalize (following symlinks) so a /etc/hyperhive → /nix/store/...
@ -146,6 +147,7 @@ pub async fn ensure_root_agent(coord: &Arc<Coordinator>) -> Result<()> {
let hive = coord.hive_env();
let paths = Coordinator::agent_paths(MANAGER_NAME, runtime);
lifecycle::spawn(MANAGER_NAME, &hive, &paths).await?;
seed_manager_tool_groups();
if let Err(e) = coord.power.set(MANAGER_NAME, crate::power::Wanted::Up) {
tracing::warn!(error = ?e, "agent_power: set manager wanted=up failed");
}
@ -155,6 +157,35 @@ pub async fn ensure_root_agent(coord: &Arc<Coordinator>) -> Result<()> {
Ok(())
}
/// Give ruth her privileged tool groups on the one path that creates her.
///
/// `effective_tool_groups()` has no manager-flavour fallback, so an agent
/// with no entry in `tool-groups.json` is an agent with no privileged
/// tools. Ruth needs hers from her first turn, and this is the only place
/// she is brought into existence — so it is written once, here, rather
/// than re-checked on every hive-c0re boot.
///
/// Skips a name that already has an entry: a destroy+recreate under the
/// same name must not silently reset an operator's chosen group set back
/// to the default.
fn seed_manager_tool_groups() {
if !tool_groups::groups_for(MANAGER_NAME).is_empty() {
tracing::debug!("manager tool groups already set — leaving as-is");
return;
}
let all_groups: Vec<String> = hive_sh4re::ToolGroup::MANAGER_DEFAULT
.iter()
.map(|g| g.as_str().to_owned())
.collect();
match tool_groups::set_groups(MANAGER_NAME, &all_groups) {
Ok(()) => tracing::info!("seeded ruth's tool groups to MANAGER_DEFAULT (all groups)"),
Err(e) => tracing::warn!(
error = ?e,
"failed to seed ruth's tool groups — she will start without privileged tools"
),
}
}
/// Sort `names` in-place so parents precede their children in the topology.
/// Uses BFS from root agents (depth 0). Agents absent from `topo` sort last,
/// alphabetically within their tier. Stable within each depth tier.