Watch
0
0
Fork
You've already forked hyperhive
0

the rendered meta flake pins each agent config by forge URL at its forge main

This commit is contained in:
atlas 2026-10-18 22:24:49 +02:00 • committed by mara
commit 09cabf8398
8 changed files with 432 additions and 163 deletions

View file

@ -115,26 +115,24 @@ until you create ruth again from the swarm dashboard.
The hive-c0re-owned repo at `/var/lib/hyperhive/meta/` The hive-c0re-owned repo at `/var/lib/hyperhive/meta/`
declares one flake input per agent (`agent-<n>.url = declares one flake input per agent (`agent-<n>.url =
"git+file://<applied>"`, the agent's local `applied` mirror) and one "git+<forge>/agent-configs/<n>.git?rev=<sha>"`) and one
`nixosConfigurations.<n>` output per agent. Each output wraps `nixosConfigurations.<n>` output per agent. Each output wraps
`inputs.agent-<n>.nixosModules.default` with the identity + `inputs.agent-<n>.nixosModules.default` with the identity +
`HIVE_PORT` / `HIVE_LABEL` / `HIVE_DASHBOARD_PORT` injection module. `HIVE_PORT` / `HIVE_LABEL` / `HIVE_DASHBOARD_PORT` injection module.
Containers run against `--flake /var/lib/hyperhive/meta#<n>`. Containers run against `--flake /var/lib/hyperhive/meta#<n>`.
The declared input url is the agent's **local `applied` mirror** The declared input url is the head of `main` on the agent's forge config repo
(`/var/lib/hyperhive/applied/<n>`) of its forge config repo (`agent-configs/<n>`, the one config PRs merge into), which `sync_agents`
(`agent-configs/<n>`, the one config PRs merge into). The meta flake reads at render time. The flake text depends only on those shas and the
therefore resolves entirely from on-disk state: building, and booting, agent set, so any machine that can reach the forge resolves it. `sync_agents` fetches
never depend on the forge being reachable. hive-c0re fills the mirror each repo's `main` with the hive's read token (`forge::reader_token()`),
by fast-forwarding `applied/<n>/main` to the merged config commit, pins its head, and takes the canonical `follows` lines from the
reading the forge with the hive's read token `flake.lock` at that commit. If the forge is unreachable or a repo has no
(`forge::reader_token()`); the forge repo is the review and audit `main`, `sync_agents` fails for that agent with its name and writes
surface, not a flake input. The rebuild paths re-lock the input with nothing; there is no fallback to a local path. A config that sits only in
`--override-input agent-<n> git+file:///var/lib/hyperhive/applied/<n>`, the hive's `applied/<n>` mirror, not on forge `main`, never deploys.
locking the exact config `applied/<n>/main` was fast-forwarded to. The rebuild paths re-lock the input with `nix flake update agent-<n>` against that
Rebuilds fire on crash-restart and meta bumps, not just config same pinned url, so `flake.nix` and `flake.lock` name no local path.
merges, so the lock stays on that head. `sync_agents` re-renders and
re-locks the persistent input.
Lock updates are single-phase and commit when the lock changed: Lock updates are single-phase and commit when the lock changed:
`meta::lock_update_for_rebuild(name)` relocks one agent's input for a `meta::lock_update_for_rebuild(name)` relocks one agent's input for a

View file

@ -118,11 +118,11 @@ Two things live in the `agent-configs` Forgejo organization:
hive-c0re pushes nothing to this org; it only reads from it. hive-c0re pushes nothing to this org; it only reads from it.
Repos stay private, so an agent can't read another Repos stay private, so an agent can't read another
agent's config. (Agents remain read-only collaborators on `core/meta`.) agent's config. (Agents remain read-only collaborators on `core/meta`.)
The agent's **persistent meta flake input** is the hive's local `applied` The agent's **persistent meta flake input** is this repo's `main` head,
mirror of this repo (`agent-<n>.url = git+file://<applied>`; see pinned by sha (`agent-<n>.url = git+<forge>/agent-configs/<n>.git?rev=<sha>`;
[config-changes.md § Meta flake](../agent-lifecycle/config-changes.md#meta-flake)), see
so building never depends on the forge being reachable. hive-c0re's own git [config-changes.md § Meta flake](../agent-lifecycle/config-changes.md#meta-flake)).
reads of this repo, which fill the mirror, authenticate with the hive's read hive-c0re's own git reads of this repo, which resolve the sha, authenticate with the hive's read
token through `forge::reader_token()` (see token through `forge::reader_token()` (see
[How a hive reads the forge](#how-a-hive-reads-the-forge)). [How a hive reads the forge](#how-a-hive-reads-the-forge)).
- The dashboard links each container's deployed config commit to this - The dashboard links each container's deployed config commit to this

View file

@ -11,6 +11,7 @@ mod repos;
mod users; mod users;
pub use reader::reader_token; pub use reader::reader_token;
pub(crate) use reconcile::FORGE_MAIN_REF;
pub use reconcile::{fetch_forge_main, reconcile_config_apply, reconcile_config_status}; pub use reconcile::{fetch_forge_main, reconcile_config_apply, reconcile_config_status};
pub use repos::{ pub use repos::{
clone_config_into_proposed, ensure_meta_remote, ensure_repo, fast_forward_applied_main, clone_config_into_proposed, ensure_meta_remote, ensure_repo, fast_forward_applied_main,

View file

@ -6,9 +6,8 @@
//! //!
//! `Status` is read-only (fetches forge `main` into a scratch ref, reports //! `Status` is read-only (fetches forge `main` into a scratch ref, reports
//! the divergence). `Apply(Forge)` resets the local applied checkout to //! the divergence). `Apply(Forge)` resets the local applied checkout to
//! forge `main` — the change takes effect on the next deploy (the deploy's //! forge `main`; the meta flake pins forge `main` itself, not this tree. It
//! `--override-input` re-locks against the reset local tree); it does NOT //! does NOT auto-deploy. `Apply(Local)` is not supported: forge `main` is
//! auto-deploy. `Apply(Local)` is not supported: forge `main` is
//! branch-protected, so it only advances through a config-PR merge. //! branch-protected, so it only advances through a config-PR merge.
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
@ -21,7 +20,7 @@ use super::{CONFIG_ORG, forge_git_url, reader_token};
/// Scratch ref the forge `main` is fetched into — outside the normal /// Scratch ref the forge `main` is fetched into — outside the normal
/// branch/tag namespace so it never collides with real refs. /// branch/tag namespace so it never collides with real refs.
const FORGE_MAIN_REF: &str = "refs/hyperhive/forge-config-main"; pub(crate) const FORGE_MAIN_REF: &str = "refs/hyperhive/forge-config-main";
/// Fetch `agent-configs/<agent>` `main` into the applied repo's scratch /// Fetch `agent-configs/<agent>` `main` into the applied repo's scratch
/// ref (read-only, no working-tree change) and return the applied dir. /// ref (read-only, no working-tree change) and return the applied dir.

View file

@ -128,6 +128,42 @@ pub async fn git_rev_parse(dir: &Path, refname: &str) -> Result<String> {
Ok(String::from_utf8_lossy(&out.stdout).trim().to_owned()) Ok(String::from_utf8_lossy(&out.stdout).trim().to_owned())
} }
/// The contents of `path` at commit `rev` in `dir`, `None` when `rev` has no
/// such file. Reads the object database, not the working tree.
pub async fn git_show_file(dir: &Path, rev: &str, path: &str) -> Result<Option<String>> {
let spec = format!("{rev}:{path}");
let listed = git_command()
.current_dir(dir)
.args(["ls-tree", "--name-only", rev, "--", path])
.output()
.await
.with_context(|| format!("git ls-tree {spec} in {}", dir.display()))?;
if !listed.status.success() {
bail!(
"git ls-tree {spec} failed ({}): {}",
listed.status,
String::from_utf8_lossy(&listed.stderr).trim()
);
}
if listed.stdout.iter().all(u8::is_ascii_whitespace) {
return Ok(None);
}
let shown = git_command()
.current_dir(dir)
.args(["show", &spec])
.output()
.await
.with_context(|| format!("git show {spec} in {}", dir.display()))?;
if !shown.status.success() {
bail!(
"git show {spec} failed ({}): {}",
shown.status,
String::from_utf8_lossy(&shown.stderr).trim()
);
}
Ok(Some(String::from_utf8_lossy(&shown.stdout).into_owned()))
}
/// Plant a lightweight tag at `target`. Errors if the tag already /// Plant a lightweight tag at `target`. Errors if the tag already
/// exists — we want loud failures on id reuse, not silent /// exists — we want loud failures on id reuse, not silent
/// overwrites. /// overwrites.

View file

@ -9,7 +9,7 @@ mod tests;
pub use git::{ pub use git::{
git, git_authed, git_command, git_command_authed, git_is_ancestor, git_read_tree_reset, git, git_authed, git_command, git_command_authed, git_is_ancestor, git_read_tree_reset,
git_rev_parse, git_tag, git_update_ref, git_update_ref_cas, git_rev_parse, git_show_file, git_tag, git_update_ref, git_update_ref_cas,
}; };
pub use host_config::{hand_permissions, write_dropins}; pub use host_config::{hand_permissions, write_dropins};
pub use setup::{ pub use setup::{
@ -335,6 +335,7 @@ async fn agents_for_meta(name_to_add: Option<&str>) -> Result<Vec<crate::meta::A
Some(crate::meta::AgentSpec { Some(crate::meta::AgentSpec {
port: agent_web_port(&name), port: agent_web_port(&name),
archetype: archetypes.get(&name).cloned(), archetype: archetypes.get(&name).cloned(),
pin: None,
name, name,
}) })
}) })
@ -345,6 +346,7 @@ async fn agents_for_meta(name_to_add: Option<&str>) -> Result<Vec<crate::meta::A
out.push(crate::meta::AgentSpec { out.push(crate::meta::AgentSpec {
port: agent_web_port(extra), port: agent_web_port(extra),
archetype: archetypes.get(extra).cloned(), archetype: archetypes.get(extra).cloned(),
pin: None,
name: extra.to_owned(), name: extra.to_owned(),
}); });
} }

View file

@ -45,6 +45,21 @@ pub struct AgentSpec {
/// The archetype the agent is built from (`crate::archetypes`), `None` /// The archetype the agent is built from (`crate::archetypes`), `None`
/// for a custom agent built from its own config repo. /// for a custom agent built from its own config repo.
pub archetype: Option<swarm_queue_client::agent_archetype::ArchetypeRef>, pub archetype: Option<swarm_queue_client::agent_archetype::ArchetypeRef>,
/// The pin of a custom agent's config input. `None` until [`sync_agents`]
/// resolves it; an archetype agent has no config input and keeps `None`.
pub pin: Option<AgentPin>,
}
/// What the rendered flake needs to know about one agent's config input, all
/// of it read from the agent's config repo at one commit. The rendered text
/// is a function of this and nothing on the host.
#[derive(Debug, Clone)]
pub struct AgentPin {
/// `git+<forge>/agent-configs/<n>.git?rev=<rev>`, `<rev>` being the head of
/// `main` on the forge when the flake was rendered.
pub url: String,
/// The `CANONICAL_INPUTS` the agent's `flake.lock` declares at that commit.
pub canonical_inputs: Vec<&'static str>,
} }
/// Stage topology.json if it exists: it is regenerated by `reconcile` /// Stage topology.json if it exists: it is regenerated by `reconcile`
@ -74,6 +89,12 @@ pub async fn sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<()> {
let dir = crate::paths::meta_root(); let dir = crate::paths::meta_root();
std::fs::create_dir_all(&dir).with_context(|| format!("create {}", dir.display()))?; std::fs::create_dir_all(&dir).with_context(|| format!("create {}", dir.display()))?;
// One agent whose forge main cannot be read fails the whole
// sync, like a missing forge URL or hyperhive rev above: rendering the
// fleet without it would drop a running agent from the flake.
let pinned = pin_agents(agents).await?;
let agents = pinned.as_slice();
let new_flake = render_hive_flake(hive, agents)?; let new_flake = render_hive_flake(hive, agents)?;
let flake_path = dir.join("flake.nix"); let flake_path = dir.join("flake.nix");
let on_disk = std::fs::read_to_string(&flake_path).unwrap_or_default(); let on_disk = std::fs::read_to_string(&flake_path).unwrap_or_default();
@ -203,26 +224,12 @@ pub async fn sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<()> {
/// One-shot used by the manual-rebuild path: relock just one /// One-shot used by the manual-rebuild path: relock just one
/// agent's input and commit the lock change if any. Single-phase /// agent's input and commit the lock change if any. Single-phase
/// (no separate finalize) because rebuild has no failure-revert /// (no separate finalize) because rebuild has no failure-revert
/// semantics — it always wants the latest main. /// semantics. The input follows the url `sync_agents` rendered just before.
pub async fn lock_update_for_rebuild(name: &str) -> Result<()> { pub async fn lock_update_for_rebuild(name: &str) -> Result<()> {
let _guard = META_LOCK.lock().await; let _guard = META_LOCK.lock().await;
let dir = crate::paths::meta_root(); let dir = crate::paths::meta_root();
let input = format!("agent-{name}"); let input = format!("agent-{name}");
// Re-lock against `applied/<n>/main` explicitly, so the lock is the nix(&dir, &rebuild_relock_args(&input)).await?;
// exact applied head whatever the declared input url says.
let applied = applied_override_url(&crate::paths::applied_dir(name));
nix(
&dir,
&[
"flake",
"update",
&input,
"--override-input",
&input,
&applied,
],
)
.await?;
if !paths_dirty(&dir, &["flake.lock"]).await? { if !paths_dirty(&dir, &["flake.lock"]).await? {
return Ok(()); return Ok(());
} }
@ -235,15 +242,11 @@ pub async fn lock_update_for_rebuild(name: &str) -> Result<()> {
.await .await
} }
/// `--override-input` URL re-locking an agent's config input against its /// `nix` arguments re-locking one agent's config input. The input is updated
/// LOCAL applied mirror (`git+file://<applied_dir>`, current `main` head), /// against the url `sync_agents` rendered, the forge url pinned by `?rev=`:
/// the same url the meta flake declares. The deploy + rebuild paths re-lock /// there is no `--override-input`, so the lock never records another source.
/// from here: the applied tree was already fast-forwarded to the reviewed fn rebuild_relock_args(input: &str) -> [&str; 3] {
/// head, so this builds exactly that config with no forge access. No ["flake", "update", input]
/// `?rev` — `main` head is the reviewed head at deploy time. Pure so it's
/// unit-testable.
fn applied_override_url(applied_dir: &Path) -> String {
format!("git+file://{}", applied_dir.display())
} }
/// Update one or more named inputs in the meta flake and commit /// Update one or more named inputs in the meta flake and commit
@ -348,14 +351,14 @@ fn pinned_git_url(forge_git_url: &str, rev: Option<&str>) -> Result<String> {
/// [`render_flake`] with its inputs taken from `hive`, as [`sync_agents`] /// [`render_flake`] with its inputs taken from `hive`, as [`sync_agents`]
/// renders it. /// renders it.
fn render_hive_flake(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<String> { fn render_hive_flake(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<String> {
Ok(render_hive_flake_at( render_hive_flake_at(hive, &hyperhive_input_url(hive)?, agents)
hive,
&hyperhive_input_url(hive)?,
agents,
))
} }
fn render_hive_flake_at(hive: &HiveEnv, hyperhive_url: &str, agents: &[AgentSpec]) -> String { fn render_hive_flake_at(
hive: &HiveEnv,
hyperhive_url: &str,
agents: &[AgentSpec],
) -> Result<String> {
render_flake( render_flake(
hyperhive_url, hyperhive_url,
hive.claude_code_path.as_deref(), hive.claude_code_path.as_deref(),
@ -364,21 +367,6 @@ fn render_hive_flake_at(hive: &HiveEnv, hyperhive_url: &str, agents: &[AgentSpec
) )
} }
fn render_flake(
hyperhive_url: &str,
claude_code_path: Option<&str>,
dashboard_port: u16,
agents: &[AgentSpec],
) -> String {
render_flake_with_lookup(
hyperhive_url,
claude_code_path,
dashboard_port,
agents,
agent_canonical_inputs,
)
}
/// Canonical inputs meta knows how to dedup. An agent that declares one /// Canonical inputs meta knows how to dedup. An agent that declares one
/// of these as a top-level input in its own `flake.nix` will get a /// of these as a top-level input in its own `flake.nix` will get a
/// `follows = "<name>"` line emitted in meta — collapsing the /// `follows = "<name>"` line emitted in meta — collapsing the
@ -712,16 +700,72 @@ fn ca_embed_state(dir: &std::path::Path) -> (Vec<(String, String)>, bool) {
(desired, changed) (desired, changed)
} }
/// Read an agent's applied `flake.lock` and return the subset of /// Resolve every custom agent's [`AgentPin`] and return `agents` with it set.
/// `CANONICAL_INPUTS` it declares as direct (root-level) inputs. async fn pin_agents(agents: &[AgentSpec]) -> Result<Vec<AgentSpec>> {
/// Returns an empty vec when the lock is missing or unparsable — let mut out = Vec::with_capacity(agents.len());
/// safe degradation, the worst case is no dedup for that agent. for spec in agents {
fn agent_canonical_inputs(name: &str) -> Vec<&'static str> { let mut spec = spec.clone();
let path = crate::paths::applied_dir(name).join("flake.lock"); spec.pin = if spec.archetype.is_some() {
let Ok(raw) = std::fs::read_to_string(&path) else { None
return Vec::new(); } else {
}; Some(
let Ok(json) = serde_json::from_str::<serde_json::Value>(&raw) else { pin_agent(
&spec.name,
&crate::paths::applied_dir(&spec.name),
&crate::forge::forge_git_url(&format!(
"{}/{}",
crate::forge::CONFIG_ORG,
spec.name
)),
crate::forge::FORGE_MAIN_REF,
crate::forge::fetch_forge_main(&spec.name),
)
.await?,
)
};
out.push(spec);
}
Ok(out)
}
/// Pin the config input of agent `name` to the head of `agent-configs/<name>`
/// `main` on the forge. `fetch_forge` fetches that branch into `repo` as
/// `forge_ref` ([`crate::forge::fetch_forge_main`]); `repo` is only the object
/// store that fetch lands in, its own branches and files are never read. The
/// pin is a commit the forge has, so a flake rendered from it resolves off this
/// host. Errors, with no fallback to a local path, when the forge cannot be
/// reached or has no `main`.
async fn pin_agent<F>(
name: &str,
repo: &Path,
forge_git_url: &str,
forge_ref: &str,
fetch_forge: F,
) -> Result<AgentPin>
where
F: std::future::Future<Output = Result<std::path::PathBuf>>,
{
fetch_forge
.await
.with_context(|| format!("agent {name}: read agent-configs/{name} main from the forge"))?;
let rev = lifecycle::git_rev_parse(repo, forge_ref)
.await
.with_context(|| format!("agent {name}: agent-configs/{name} has no main on the forge"))?;
let lock = lifecycle::git_show_file(repo, &rev, "flake.lock").await?;
Ok(AgentPin {
url: pinned_git_url(forge_git_url, Some(&rev))?,
canonical_inputs: lock
.as_deref()
.map(canonical_inputs_in_lock)
.unwrap_or_default(),
})
}
/// The subset of `CANONICAL_INPUTS` that the `flake.lock` text `raw` declares
/// as direct (root-level) inputs. Empty when the lock is unparsable — safe
/// degradation, the worst case is no dedup for that agent.
fn canonical_inputs_in_lock(raw: &str) -> Vec<&'static str> {
let Ok(json) = serde_json::from_str::<serde_json::Value>(raw) else {
return Vec::new(); return Vec::new();
}; };
let Some(nodes) = json.get("nodes").and_then(|v| v.as_object()) else { let Some(nodes) = json.get("nodes").and_then(|v| v.as_object()) else {
@ -744,23 +788,18 @@ fn agent_canonical_inputs(name: &str) -> Vec<&'static str> {
.collect() .collect()
} }
/// Inner render helper accepting a lookup fn so tests can stub the /// Render the meta flake. Errors on a custom agent without a [`AgentPin`].
/// agent flake-lock introspection.
#[allow( #[allow(
clippy::too_many_lines, clippy::too_many_lines,
reason = "templated string-builder for the meta flake — the length is one \ reason = "templated string-builder for the meta flake — the length is one \
contiguous fmt block, splitting it would just hide the shape" contiguous fmt block, splitting it would just hide the shape"
)] )]
fn render_flake_with_lookup<F>( fn render_flake(
hyperhive_url: &str, hyperhive_url: &str,
claude_code_path: Option<&str>, claude_code_path: Option<&str>,
dashboard_port: u16, dashboard_port: u16,
agents: &[AgentSpec], agents: &[AgentSpec],
lookup: F, ) -> Result<String> {
) -> String
where
F: Fn(&str) -> Vec<&'static str>,
{
use std::fmt::Write as _; use std::fmt::Write as _;
let mut out = String::new(); let mut out = String::new();
out.push_str("{\n description = \"hyperhive deployed agents\";\n inputs = {\n"); out.push_str("{\n description = \"hyperhive deployed agents\";\n inputs = {\n");
@ -770,33 +809,36 @@ where
// `agent-<n>.inputs.nixpkgs.follows = "nixpkgs"` resolves to this node. // `agent-<n>.inputs.nixpkgs.follows = "nixpkgs"` resolves to this node.
let _ = writeln!(out, " hyperhive.url = \"{hyperhive_url}\";"); let _ = writeln!(out, " hyperhive.url = \"{hyperhive_url}\";");
out.push_str(" nixpkgs.follows = \"hyperhive/nixpkgs\";\n"); out.push_str(" nixpkgs.follows = \"hyperhive/nixpkgs\";\n");
// Each agent's config input is its LOCAL applied mirror // Each agent's config input is its forge repo at the pinned commit, so
// (`git+file://<applied>`), so the meta flake resolves entirely from // the flake text depends only on (agent, rev) and resolves anywhere that
// on-disk state and boot never depends on the forge being reachable. // can reach the forge.
// The forge `agent-configs/<name>` repos stay the review/audit surface
// (config PRs land there) but are not the flake's build input. deploy +
// rebuild re-lock this input to `applied/<name>`'s current `main` head.
for spec in agents { for spec in agents {
// An archetype agent has no config repo, so no input: `mkAgent` // An archetype agent has no config repo, so no input: `mkAgent`
// takes its module from the archetype. // takes its module from the archetype.
if spec.archetype.is_some() { if spec.archetype.is_some() {
continue; continue;
} }
let pin = spec.pin.as_ref().with_context(|| {
format!(
"agent {}: no pinned rev, refusing to render an unpinned config input",
spec.name
)
})?;
let _ = writeln!( let _ = writeln!(
out, out,
" agent-{name}.url = \"git+file://{applied}\";", " agent-{name}.url = \"{url}\";",
name = spec.name, name = spec.name,
applied = crate::paths::applied_dir(&spec.name).display(), url = pin.url,
); );
// For each canonical input the agent declares in its own // For each canonical input the agent's `flake.lock` declares at the
// `flake.nix` (detected by reading its applied `flake.lock`), // pinned commit, emit
// emit `inputs.agent-<name>.inputs.<canon>.follows = "<canon>"`. // `inputs.agent-<name>.inputs.<canon>.follows = "<canon>"`.
// Collapses otherwise-separate-but-identical nixpkgs nodes // Collapses otherwise-separate-but-identical nixpkgs nodes
// (root + every agent's own nixpkgs) into one. Skipped // (root + every agent's own nixpkgs) into one. Skipped
// silently for agents that don't declare the input — emitting // silently for agents that don't declare the input — emitting
// follows on a non-existent input would error at // follows on a non-existent input would error at
// `nix flake lock` time. // `nix flake lock` time.
for canon in lookup(&spec.name) { for canon in &pin.canonical_inputs {
let _ = writeln!( let _ = writeln!(
out, out,
" agent-{}.inputs.{canon}.follows = \"{canon}\";", " agent-{}.inputs.{canon}.follows = \"{canon}\";",
@ -1032,7 +1074,7 @@ where
); );
} }
out.push_str(" };\n };\n}\n"); out.push_str(" };\n };\n}\n");
out Ok(out)
} }
/// Return the list of file names that are currently staged (index differs /// Return the list of file names that are currently staged (index differs
@ -1240,28 +1282,66 @@ mod tests {
); );
} }
const SAMPLE_REV: &str = "1111111111111111111111111111111111111111";
fn sample_pin(name: &str, rev: &str, canonical_inputs: Vec<&'static str>) -> AgentPin {
AgentPin {
url: format!("git+https://forge.example/agent-configs/{name}.git?rev={rev}"),
canonical_inputs,
}
}
fn sample_spec(name: &str, port: u16) -> AgentSpec { fn sample_spec(name: &str, port: u16) -> AgentSpec {
AgentSpec { AgentSpec {
name: name.to_owned(), name: name.to_owned(),
port, port,
archetype: None, archetype: None,
pin: Some(sample_pin(name, SAMPLE_REV, Vec::new())),
} }
} }
fn render(
hyperhive_url: &str,
claude_code_path: Option<&str>,
dashboard_port: u16,
agents: &[AgentSpec],
) -> String {
render_flake(hyperhive_url, claude_code_path, dashboard_port, agents).expect("render")
}
#[test] #[test]
fn applied_override_url_targets_local_mirror_main_head() { fn rebuild_relock_updates_the_input_without_an_override() {
// Deploy + rebuild re-lock against the local applied mirror's `main` let args = rebuild_relock_args("agent-iris");
// head (no `?rev`) — so a rebuild survives forge unreachability and assert_eq!(args, ["flake", "update", "agent-iris"]);
// builds the fast-forwarded config.
let p = Path::new("/var/lib/hyperhive/agents/iris/applied");
let url = applied_override_url(p);
assert_eq!(url, "git+file:///var/lib/hyperhive/agents/iris/applied");
assert!( assert!(
!url.contains("?rev="), !args
"must lock main head, not a pinned rev: {url}" .iter()
.any(|a| *a == "--override-input" || a.contains("git+file")),
"a rebuild must not lock a local path: {args:?}"
); );
} }
/// The url a rebuild relocks is the one `sync_agents` rendered: the forge,
/// pinned by rev.
#[test]
fn the_input_a_rebuild_relocks_is_a_pinned_forge_url() {
let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
&[sample_spec("iris", 9001)],
);
let url = out
.lines()
.find_map(|l| l.trim().strip_prefix("agent-iris.url = \""))
.expect("agent-iris input is rendered");
assert!(
url.starts_with("git+https://forge.example/agent-configs/iris.git?rev="),
"{url}"
);
assert!(!url.contains("git+file"), "{url}");
}
#[test] #[test]
fn pinned_git_url_carries_the_rev_and_refuses_without_one() { fn pinned_git_url_carries_the_rev_and_refuses_without_one() {
assert_eq!( assert_eq!(
@ -1280,7 +1360,7 @@ mod tests {
#[test] #[test]
fn render_flake_pins_hyperhive_by_url_and_follows_its_nixpkgs() { fn render_flake_pins_hyperhive_by_url_and_follows_its_nixpkgs() {
let out = render_flake( let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None, None,
8000, 8000,
@ -1310,7 +1390,7 @@ mod tests {
#[test] #[test]
fn render_flake_names_each_agent_container_after_the_agent() { fn render_flake_names_each_agent_container_after_the_agent() {
let out = render_flake( let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None, None,
8000, 8000,
@ -1346,7 +1426,7 @@ mod tests {
// the host store, so the build is already reachable; making it an // the host store, so the build is already reachable; making it an
// input would re-copy it as a reference-less `-source` and strip // input would re-copy it as a reference-less `-source` and strip
// the closure the binary actually needs. // the closure the binary actually needs.
let out = render_flake( let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
Some("/nix/store/cccc-claude-code-2.1.220"), Some("/nix/store/cccc-claude-code-2.1.220"),
8000, 8000,
@ -1370,7 +1450,7 @@ mod tests {
// agent module keeps its own nixpkgs' `claude-code` (and keeps it // agent module keeps its own nixpkgs' `claude-code` (and keeps it
// in `environment.systemPackages`, which is what makes the // in `environment.systemPackages`, which is what makes the
// unpinned case self-contained). // unpinned case self-contained).
let out = render_flake( let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None, None,
8000, 8000,
@ -1384,24 +1464,21 @@ mod tests {
#[test] #[test]
fn render_flake_emits_follows_for_agents_declaring_nixpkgs() { fn render_flake_emits_follows_for_agents_declaring_nixpkgs() {
// Stub lookup: pretend `bitburner` + `dmatrix` declare `nixpkgs` // `bitburner` + `dmatrix` declare `nixpkgs` at their root, while
// at their root, while `argus` has no canonical inputs at all. // `argus` has no canonical inputs at all.
let lookup = |name: &str| -> Vec<&'static str> { let with_nixpkgs = |name, port| AgentSpec {
match name { pin: Some(sample_pin(name, SAMPLE_REV, vec!["nixpkgs"])),
"bitburner" | "dmatrix" => vec!["nixpkgs"], ..sample_spec(name, port)
_ => vec![],
}
}; };
let out = render_flake_with_lookup( let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None, None,
8000, 8000,
&[ &[
sample_spec("argus", 9001), sample_spec("argus", 9001),
sample_spec("bitburner", 9002), with_nixpkgs("bitburner", 9002),
sample_spec("dmatrix", 9003), with_nixpkgs("dmatrix", 9003),
], ],
lookup,
); );
// bitburner declares nixpkgs → follows emitted. // bitburner declares nixpkgs → follows emitted.
assert!( assert!(
@ -1421,7 +1498,7 @@ mod tests {
#[test] #[test]
fn render_flake_builds_every_agent_from_agent_base() { fn render_flake_builds_every_agent_from_agent_base() {
let out = render_flake( let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None, None,
8000, 8000,
@ -1445,15 +1522,14 @@ mod tests {
} }
#[test] #[test]
fn render_flake_skips_canonical_follows_when_lookup_returns_empty() { fn render_flake_skips_canonical_follows_when_the_lock_declares_none() {
let out = render_flake_with_lookup( let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None, None,
8000, 8000,
&[sample_spec("alice", 9001)], &[sample_spec("alice", 9001)],
|_| Vec::new(),
); );
// No agent-side follows when the lookup reports nothing // No agent-side follows when the lock declares nothing
// declared — protects agents whose flake.lock can't be read // declared — protects agents whose flake.lock can't be read
// (missing / unparsable) from being broken by a follows on a // (missing / unparsable) from being broken by a follows on a
// non-existent input. // non-existent input.
@ -1474,12 +1550,15 @@ mod tests {
}), }),
..sample_spec("droid", 9002) ..sample_spec("droid", 9002)
}; };
let out = render_flake_with_lookup( let alice = AgentSpec {
pin: Some(sample_pin("alice", SAMPLE_REV, vec!["nixpkgs"])),
..sample_spec("alice", 9001)
};
let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None, None,
8000, 8000,
&[sample_spec("alice", 9001), android], &[alice, android],
|_| vec!["nixpkgs"],
); );
assert!( assert!(
!out.contains("agent-droid"), !out.contains("agent-droid"),
@ -1492,7 +1571,7 @@ mod tests {
), ),
"{out}" "{out}"
); );
assert!(out.contains("agent-alice.url = \"git+file://"), "{out}"); assert!(out.contains("agent-alice.url = \"git+https://"), "{out}");
assert!(out.contains("agent-alice.inputs.nixpkgs.follows"), "{out}"); assert!(out.contains("agent-alice.inputs.nixpkgs.follows"), "{out}");
assert!( assert!(
out.contains("alice = mkAgent { name = \"alice\"; port = 9001; };"), out.contains("alice = mkAgent { name = \"alice\"; port = 9001; };"),
@ -1525,7 +1604,7 @@ mod tests {
unsafe { unsafe {
std::env::set_var("HIVE_FORGE_URL", "http://forge.example.test"); std::env::set_var("HIVE_FORGE_URL", "http://forge.example.test");
} }
let out = render_flake( let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None, None,
8000, 8000,
@ -1566,7 +1645,7 @@ mod tests {
std::env::set_var(VARS[0], name); std::env::set_var(VARS[0], name);
std::env::set_var(VARS[1], domain); std::env::set_var(VARS[1], domain);
} }
render_flake( render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None, None,
8000, 8000,
@ -1761,7 +1840,7 @@ mod tests {
std::env::set_var("HIVE_FORGE_URL", "http://forge.example.test"); std::env::set_var("HIVE_FORGE_URL", "http://forge.example.test");
std::env::set_var("HIVE_MATRIX_URL", "http://matrix.example.test"); std::env::set_var("HIVE_MATRIX_URL", "http://matrix.example.test");
} }
let out = render_flake( let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None, None,
8000, 8000,
@ -1787,30 +1866,183 @@ mod tests {
} }
#[test] #[test]
fn render_flake_agent_input_points_at_local_applied_mirror() { fn render_flake_pins_each_agent_input_to_its_own_forge_rev() {
// The agent config input references the LOCAL applied mirror let at = |name, port, rev: &str| AgentSpec {
// (`git+file://<applied>`), NOT the forge — so the meta flake pin: Some(sample_pin(name, rev, Vec::new())),
// resolves entirely from on-disk state and boot never depends on the ..sample_spec(name, port)
// forge being reachable. The forge `agent-configs/<n>` repos stay the };
// review surface, but they are not the flake's build input. let out = render(
let out = render_flake(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None, None,
8000, 8000,
&[sample_spec("alice", 9001)], &[at("alice", 9001, "aaaa"), at("bob", 9002, "bbbb")],
);
let want = format!(
"agent-alice.url = \"git+file://{}\"",
crate::paths::applied_dir("alice").display()
); );
assert!( assert!(
out.contains(&want), out.contains(
"expected the agent input to point at the local applied mirror ({want}):\n{out}" "agent-alice.url = \"git+https://forge.example/agent-configs/alice.git?rev=aaaa\";"
),
"{out}"
); );
assert!( assert!(
!out.contains("agent-alice.url = \"git+http"), out.contains(
"no forge git+http URL must remain for the agent input:\n{out}" "agent-bob.url = \"git+https://forge.example/agent-configs/bob.git?rev=bbbb\";"
),
"{out}"
); );
assert!(
!out.contains("git+file://"),
"no host-local path may remain in the flake:\n{out}"
);
}
#[test]
fn render_flake_refuses_a_custom_agent_without_a_pin() {
let args = |agents: &[AgentSpec]| {
render_flake(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
agents,
)
};
let unpinned = AgentSpec {
pin: None,
..sample_spec("alice", 9001)
};
let err = args(&[unpinned]).expect_err("an unpinned agent must not render");
assert!(
format!("{err:#}").contains("agent alice: no pinned rev"),
"{err:#}"
);
// Presence control: the same agent with a pin renders.
assert!(args(&[sample_spec("alice", 9001)]).is_ok());
// An archetype agent has no config input and needs no pin.
let android = AgentSpec {
archetype: Some(swarm_queue_client::agent_archetype::ArchetypeRef {
source: "hyperhive".to_owned(),
name: "android-dev".to_owned(),
}),
pin: None,
..sample_spec("droid", 9002)
};
assert!(args(&[android]).is_ok());
}
/// A repo whose commits carry a `flake.lock`; `lock` is its content, `None`
/// for a commit without one. Returns the dir and the commit.
async fn commit_lock(dir: &Path, lock: Option<&str>) -> String {
let _ = std::fs::remove_file(dir.join("flake.lock"));
if let Some(lock) = lock {
std::fs::write(dir.join("flake.lock"), lock).expect("write lock");
}
git(dir, &["add", "-A"]).await.expect("add");
git(
dir,
&[
"-c",
"user.name=t",
"-c",
"user.email=t@t",
"commit",
"-q",
"--allow-empty",
"-m",
"c",
],
)
.await
.expect("commit");
lifecycle::git_rev_parse(dir, "HEAD").await.expect("HEAD")
}
async fn repo() -> tempfile::TempDir {
let dir = tempfile::tempdir().expect("tempdir");
git(dir.path(), &["init", "-q", "--initial-branch=main"])
.await
.expect("init");
dir
}
const NIXPKGS_LOCK: &str = r#"{"nodes":{"root":{"inputs":{"nixpkgs":"nixpkgs"}},"nixpkgs":{}},"root":"root","version":7}"#;
const NO_INPUTS_LOCK: &str = r#"{"nodes":{"root":{"inputs":{}}},"root":"root","version":7}"#;
const TEST_FORGE_REF: &str = "refs/test/forge-main";
const TEST_URL: &str = "https://forge.example/agent-configs/alice.git";
/// `pin_agent` where the forge fetch leaves `forge_ref` at `forge_sha`.
async fn pin_with_forge_at(repo: &Path, forge_sha: Option<&str>) -> Result<AgentPin> {
pin_agent("alice", repo, TEST_URL, TEST_FORGE_REF, async {
if let Some(sha) = forge_sha {
lifecycle::git_update_ref(repo, TEST_FORGE_REF, sha).await?;
}
Ok(repo.to_path_buf())
})
.await
}
#[tokio::test]
async fn pin_agent_pins_forge_main_not_the_local_main() {
let dir = repo().await;
let forge = commit_lock(dir.path(), Some(NIXPKGS_LOCK)).await;
// Local main moves on past the forge's: its commit and lock are never read.
let local = commit_lock(dir.path(), Some(NO_INPUTS_LOCK)).await;
std::fs::write(dir.path().join("flake.lock"), NO_INPUTS_LOCK).expect("edit");
assert_ne!(forge, local);
let pin = pin_with_forge_at(dir.path(), Some(&forge))
.await
.expect("pinned");
assert_eq!(pin.url, format!("git+{TEST_URL}?rev={forge}"));
// The lock at the forge commit declares nixpkgs; local main's does not.
assert_eq!(pin.canonical_inputs, vec!["nixpkgs"]);
}
#[tokio::test]
async fn two_agents_at_different_forge_mains_get_distinct_pins() {
let a = repo().await;
let b = repo().await;
let rev_a = commit_lock(a.path(), None).await;
let rev_b = commit_lock(b.path(), Some(NIXPKGS_LOCK)).await;
assert_ne!(rev_a, rev_b);
let pin_a = pin_with_forge_at(a.path(), Some(&rev_a)).await.expect("a");
let pin_b = pin_with_forge_at(b.path(), Some(&rev_b)).await.expect("b");
assert_ne!(pin_a.url, pin_b.url);
assert!(pin_a.url.ends_with(&rev_a) && pin_b.url.ends_with(&rev_b));
// No lock at a's commit, so no follows; b's lock declares nixpkgs.
assert!(pin_a.canonical_inputs.is_empty());
assert_eq!(pin_b.canonical_inputs, vec!["nixpkgs"]);
}
#[tokio::test]
async fn pin_agent_errors_when_the_forge_is_unreachable_or_has_no_main() {
let dir = repo().await;
let rev = commit_lock(dir.path(), None).await;
let unreachable = pin_agent("alice", dir.path(), TEST_URL, TEST_FORGE_REF, async {
anyhow::bail!("forge down")
})
.await
.expect_err("unreachable forge");
let msg = format!("{unreachable:#}");
assert!(
msg.contains("agent alice") && msg.contains("forge down"),
"{msg}"
);
// The fetch succeeds but leaves no forge main.
let no_main = pin_with_forge_at(dir.path(), None)
.await
.expect_err("no main on the forge");
let msg = format!("{no_main:#}");
assert!(
msg.contains("agent alice") && msg.contains("no main"),
"{msg}"
);
// Presence control: with a forge main, the same call pins it.
let pin = pin_with_forge_at(dir.path(), Some(&rev))
.await
.expect("pinned");
assert!(pin.url.ends_with(&rev));
} }
#[test] #[test]
@ -1833,7 +2065,7 @@ mod tests {
.expect("write temp CA"); .expect("write temp CA");
let render = || { let render = || {
render_flake( render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None, None,
8000, 8000,
@ -1921,7 +2153,7 @@ mod tests {
let _env = crate::test_env::lock(); let _env = crate::test_env::lock();
// SAFETY: serialised by the guard above; restored before returning. // SAFETY: serialised by the guard above; restored before returning.
let render = || { let render = || {
render_flake( render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None, None,
8000, 8000,
@ -1991,7 +2223,7 @@ mod tests {
let _env = crate::test_env::lock(); let _env = crate::test_env::lock();
// SAFETY: serialised by the guard above; restored before returning. // SAFETY: serialised by the guard above; restored before returning.
let render = || { let render = || {
render_flake( render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None, None,
8000, 8000,
@ -2032,12 +2264,14 @@ mod tests {
&hive("4G"), &hive("4G"),
"git+https://forge.example/h.git?rev=abc", "git+https://forge.example/h.git?rev=abc",
&agents, &agents,
); )
.expect("render");
let large = render_hive_flake_at( let large = render_hive_flake_at(
&hive("16G"), &hive("16G"),
"git+https://forge.example/h.git?rev=abc", "git+https://forge.example/h.git?rev=abc",
&agents, &agents,
); )
.expect("render");
assert_eq!( assert_eq!(
small, large, small, large,
"the rendered flake must not depend on the memory cap" "the rendered flake must not depend on the memory cap"

View file

@ -44,11 +44,10 @@ let
# (mid-path globs aren't supported, so per-agent repos can't be listed); # (mid-path globs aren't supported, so per-agent repos can't be listed);
# in practice these processes only ever touch hyperhive's own repos. # in practice these processes only ever touch hyperhive's own repos.
# #
# The agent config inputs are local `git+file` mirrors of the applied repos # The agent config inputs are `git+http` forge URLs pinned by sha (meta.rs
# (meta.rs render), so the meta flake fetches nothing from the forge. The # render). The `[credential]` stanza points git at the `hive-forge` helper
# `[credential]` stanza points git at the `hive-forge` helper below, scoped # below, scoped to the forge host, for any `git+http` fetch from it as the
# to the forge host, for any `git+http` fetch from it as the forge `core` # forge `core` user. hive-c0re's own forge reads use `forge::reader_token()` instead.
# user. hive-c0re's own forge reads use `forge::reader_token()` instead.
safeDirGitconfig = pkgs.writeText "hyperhive-safe-gitconfig" '' safeDirGitconfig = pkgs.writeText "hyperhive-safe-gitconfig" ''
[safe] [safe]
directory = * directory = *