vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

build agents only from internal/meta; delete the hive's local meta flake

Browse source 
The hive no longer renders, locks or commits a meta flake. A rebuild's
MetaSync node picks the internal/meta commit and records it with the
hyperhive and config pins it holds; Prebuild, Swap and Create build from it,
and a head that does not pin the agent's current config fails the build
naming the agent. hive-priv's meta_rev is required.

Removed: meta.rs sync/lock/render, the manual-rebuild relock, the boot
sweep's hyperhive bump (MetaLock), the update-meta-inputs form and its
panel, MetaWindow, the hive's hyperhive_rev setting.

Kept: agent_set::refresh holds what sync_agents also did (topology.json, the
gateway's agent-socket map and nginx include), called from spawn, destroy,
rebuild and the migration.

The boot reconcile and the stale-start upgrade compare the agent's marker
with the hyperhive pin of internal/meta; the dashboard's deployed chip reads
the config marker a successful build writes.
This commit is contained in:
atlas 2026-10-25 13:33:15 +01:00
commit 4aa11b3a94
64 changed files with 1042 additions and 3793 deletions
Download patch file Download diff file Expand all files Collapse all files

109
hive-c0re/src/workers/auto_update.rs
View file

@ -1,20 +1,21 @@
//! Boot reconcile: on `hive-c0re serve` boot, (a) run the config path
//! for agents whose per-agent rev marker is stale — one `Boot` DAG
//! (meta hyperhive lock bump) that grows a `Rebuild` subgraph for each
//! stale agent whose `wanted` power intent is `Up` — and (b)
//! converge every other drifted agent to its persisted `wanted` via
//! `Reconcile` DAGs. Two rules keep boot-time nix work minimal:
//! Boot reconcile: on `hive-c0re serve` boot, (a) rebuild the agents whose
//! per-agent rev marker differs from the hyperhive commit the swarm's
//! `internal/meta` pins now — one `Boot` DAG holding a `Rebuild` subgraph for
//! each stale agent whose `wanted` power intent is `Up` — and (b) converge
//! every other drifted agent to its persisted `wanted` via `Reconcile` DAGs.
//! This is how a hive that missed the swarm's fan-out catches up. Three rules
//! keep boot-time nix work minimal:
//!
//! 1. **Stale but wanted-offline agents** get no rebuild — their
//! rebuild happens the first time they're started (the start
//! submit path upgrades a stale start to rebuild+start). The sweep
//! parent still runs whenever *any* marker is stale so the meta
//! hyperhive lock is bumped for those later start-upgrades.
//! 2. **Agents whose rev marker matches** the current hyperhive flake
//! path are skipped — nothing changed, no nix work to do.
//! submit path upgrades a stale start to rebuild+start).
//! 2. **Agents whose rev marker matches** the pin are skipped — nothing
//! changed, no nix work to do.
//! 3. **A pin the hive cannot read** (no forge token, forge down, no
//! `internal/meta` yet) rebuilds nothing: a rebuild would have no flake
//! to build from either.
//!
//! Booting with no config change performs no meta commit — only
//! reconciles. See `docs/scheduler/coordinator.md::Boot reconcile`.
//! See `docs/scheduler/coordinator.md::Boot reconcile`.
use std::path::Path;
use std::sync::Arc;
@ -39,9 +40,10 @@ pub fn current_flake_rev(hyperhive_flake: &str) -> Option<String> {
}
/// Returns true when the applied repo has commits that have not yet been
/// deployed (i.e. the applied HEAD differs from the sha currently locked in
/// meta's flake.lock). This is the semantic the dashboard `needs_update` chip
/// conveys: "there is a config change ready to apply via rebuild."
/// deployed (i.e. the applied HEAD differs from the config commit the agent's
/// last successful build pinned, `paths::applied_config_marker`). This is the
/// semantic the dashboard `needs_update` chip conveys: "there is a config
/// change ready to apply via rebuild."
///
/// Async on purpose: this runs per agent inside `container_view::build_all`,
/// which fires on the ~10s dashboard sweep, every `AgentStatus` request, and
@ -67,10 +69,9 @@ pub async fn agent_config_pending(name: &str, deployed_sha: Option<&str>) -> boo
}
/// Boot reconcile (see the module doc): classify every agent by rev
/// freshness + persisted `wanted` intent, submit one `Boot` DAG
/// (hyperhive lock bump growing an in-DAG rebuild subgraph per stale
/// wanted-up agent) when anything is stale, and `Reconcile` DAGs for
/// agents whose observed power state drifted from `wanted`. Returns Ok even
/// freshness + persisted `wanted` intent, and submit one `Boot` DAG holding a
/// graceful rebuild subgraph per stale wanted-up agent and a `Reconcile` per
/// agent whose observed power state drifted from `wanted`. Returns Ok even
/// if some submissions failed.
pub async fn run(coord: Arc<Coordinator>) -> Result<()> {
let containers = match lifecycle::list().await {
@ -81,7 +82,7 @@ pub async fn run(coord: Arc<Coordinator>) -> Result<()> {
}
};
let current_rev = current_flake_rev(&coord.hyperhive_flake);
let pin = crate::meta_rev::current_hyperhive_pin().await;
// Resolve container names to logical agent names, then sort. The
// parent field this used to depth-sort by is gone; with no
@ -97,7 +98,6 @@ pub async fn run(coord: Arc<Coordinator>) -> Result<()> {
// Classify. `get_or_seed` doubles as the one-time migration: an
// agent without an `agent_power` row is seeded from its observed
// state (running ⇒ Up), after which the DB is authoritative.
let mut any_stale = false;
// stale ∧ wanted=Up → sweep rebuild. The bool is the agent's observed
// running state, used to order running agents first before submit.
let mut fanout: Vec<(String, bool)> = Vec::new();
@ -113,17 +113,11 @@ pub async fn run(coord: Arc<Coordinator>) -> Result<()> {
None
}
};
let fresh = current_rev.as_ref().is_some_and(|rev| {
std::fs::read_to_string(crate::paths::applied_rev_marker(name))
.is_ok_and(|stored| stored == rev.as_str())
});
let stored = std::fs::read_to_string(crate::paths::applied_rev_marker(name)).ok();
let fresh = crate::meta_rev::marker_is_fresh(pin.as_deref(), stored.as_deref());
match boot_action(wanted, fresh, running) {
BootAction::Rebuild => {
any_stale = true;
fanout.push((name.clone(), running));
}
BootAction::Rebuild => fanout.push((name.clone(), running)),
BootAction::Defer { reconcile } => {
any_stale = true;
n_deferred += 1;
tracing::debug!(%name, "boot reconcile: stale but offline — deferring rebuild to on-start");
if reconcile {
@ -158,7 +152,7 @@ pub async fn run(coord: Arc<Coordinator>) -> Result<()> {
fanout.sort_by_key(|(_, running)| !running);
let fanout: Vec<String> = fanout.into_iter().map(|(name, _)| name).collect();
submit_boot_tree(&coord, any_stale, fanout, drifted, n_deferred, n_skipped);
submit_boot_tree(&coord, &fanout, drifted, n_deferred, n_skipped);
submit_startup_sweep_nodes(&coord);
Ok(())
}
@ -166,7 +160,7 @@ pub async fn run(coord: Arc<Coordinator>) -> Result<()> {
/// What the boot sweep does with one agent.
#[derive(Debug, PartialEq, Eq)]
enum BootAction {
/// Stale ∧ wanted up: rebuild against the post-bump lock. The DAG's
/// Stale ∧ wanted up: rebuild from the pinned `internal/meta`. The DAG's
/// tail `Reconcile` brings the agent (back) up, covering both the
/// running-stale and stopped-but-wanted-up cases.
Rebuild,
@ -233,34 +227,22 @@ fn submit_startup_sweep_nodes(coord: &Arc<Coordinator>) {
/// That split is not cosmetic: this path constructs nodes outside
/// `job_queue/`, so it is the one place a resource declaration can be forgotten
/// without any in-module test noticing. It has happened once already — the
/// sweep `MetaLock` and the boot `Reconcile`s silently declared nothing when
/// kind-derived resources were removed, which drops the agent lease a boot
/// reconcile needs to not race another DAG's container ops.
/// boot `Reconcile`s silently declared nothing when kind-derived resources
/// were removed, which drops the agent lease a boot reconcile needs to not
/// race another DAG's container ops.
pub(crate) fn boot_nodes(
builder: &crate::job_queue::JobBuilder,
any_stale: bool,
fanout: Vec<String>,
fanout: &[String],
drifted: Vec<String>,
) {
use crate::job_queue::NodeKind;
use crate::job_queue::resource::Resource;
// Sweep whenever ANY marker is stale — even when every stale agent is
// wanted-offline: the hyperhive lock bump must land now so their later
// start-upgrade rebuilds build against it. No stale agents ⇒ no MetaLock
// ⇒ no meta commit on a no-change boot. The `fanout` list rides the
// MetaLock into `run_meta_lock`, which appends the rebuild subgraphs.
if any_stale {
let _ = builder
.node(NodeKind::MetaLock {
sweep: true,
fanout: Some(fanout),
// A sweep bumps `hyperhive` alone (`lock_update_hyperhive`),
// so it names no inputs.
inputs: Vec::new(),
})
.needs(Resource::BuildSlot)
.needs(Resource::MetaWindow);
// One graceful rebuild subgraph per stale agent, each an independent
// group root: agents that were mid-turn when the host came up drain rather
// than being cut off, and the rebuilds overlap on their own leases.
for agent in fanout {
let _ = crate::job_queue::templates::graceful_rebuild_nodes(builder, agent, None);
}
// One boot Reconcile per drifted agent — independent roots.
for name in drifted {
@ -273,23 +255,18 @@ pub(crate) fn boot_nodes(
}
/// Submit this boot's work as **one DAG** (no anchor node, no per-agent
/// child DAGs). Node 0 is the sweep `MetaLock` (only when
/// something is stale) — its executor bumps the hyperhive lock, then grows
/// one rebuild subgraph per stale agent into *this same* DAG (rooted on the
/// `MetaLock`, so they build against the post-bump lock; see
/// `exec::run_meta_lock`). Every drifted agent gets a boot `Reconcile` as an
/// independent root — a boot reconcile needs no lock bump, so it converges
/// concurrently with the sweep. No-op when there's nothing to do.
/// child DAGs). It holds one graceful rebuild subgraph per stale agent, and
/// every drifted agent gets a boot `Reconcile` as an independent root, so they
/// converge concurrently with the rebuilds. No-op when there's nothing to do.
fn submit_boot_tree(
coord: &Arc<Coordinator>,
any_stale: bool,
fanout: Vec<String>,
fanout: &[String],
drifted: Vec<String>,
n_deferred: usize,
n_skipped: usize,
) {
// Fully-quiet boot (nothing stale, nothing drifted) inserts nothing.
if !any_stale && drifted.is_empty() {
// Fully-quiet boot (nothing to rebuild, nothing drifted) inserts nothing.
if fanout.is_empty() && drifted.is_empty() {
return;
}
// The summary the sweep used to hand the container as its `reason` is a log
@ -309,7 +286,7 @@ fn submit_boot_tree(
// during their `Swap` (applied at claim time); a reconcile-only boot needs
// no transient.
if let Err(e) = coord.job_queue.insert_job(|b| {
boot_nodes(b, any_stale, fanout, drifted);
boot_nodes(b, fanout, drifted);
Vec::new()
}) {
tracing::warn!(error = ?e, "boot: sweep DAG insert failed");