af3a9e5433
Review finding from argus. The new endpoint test carried a SAFETY comment claiming no other test in its module asserts on the variables it perturbs — the wrong boundary. The module is not the unit that shares the environment, the process is: meta.rs's render_flake_injects_otel_when_signalled mutates the same HYPERHIVE_OTEL_ENDPOINT, both land in the one hive-c0re test binary, and cargo runs it at default parallelism with no serialisation anywhere in the crate. Each test independently claimed exclusive ownership of shared global state, which is the instrument-that-looks-solid class the endpoint change's own gate reasoning warns about. Adds test_env with a single ENV_LOCK, taken by both. No new dependency: this is the pattern hive-bash-mcp and hive-agent already use, and hive-bash-mcp's helper records why it has to be crate-wide rather than per-module — two per-module mutexes serialise nothing against each other, which produced a CI-only flake there. The asymmetry that makes this hard to see locally is worth stating: an agent container has the hyperhive variables ambient-set, so a losing race still finds a plausible value and the test passes; the nix sandbox strips them, so only there can one thread delete a variable out from under another. Verified in that shape with `env -u HYPERHIVE_OTEL_ENDPOINT -u OTEL_EXPORTER_OTLP_ENDPOINT`, five consecutive runs green — a sanity check, not a proof, since a race cannot be shown absent by running. What makes it correct is structural: both tests take the same lock. Deliberately scoped to the pair that overlaps. meta.rs has three further env-mutating tests (HIVE_FORGE_URL twice, the TLS CA pair) that race with each other, untouched here and tracked separately, because the fix is not the mechanical one it looks like: std::sync::Mutex is not reentrant, so adding a lock to a test whose helpers also lock deadlocks. That needs reading per test rather than a sweep.
2364 lines
100 KiB
Rust
2364 lines
100 KiB
Rust
//! Single hive-c0re-owned flake at `/var/lib/hyperhive/meta/` that
|
|
//! exports one `nixosConfiguration` per agent and drives the system-wide
|
|
//! deploy audit trail. Flow (`sync_agents`, two-phase `prepare_deploy` /
|
|
//! `finalize_deploy` / `abort_deploy`, `lock_update_hyperhive`):
|
|
//! `docs/approvals.md::Meta flake`.
|
|
|
|
use std::path::Path;
|
|
|
|
use anyhow::{Context, Result, bail};
|
|
use tokio::process::Command;
|
|
use tokio::sync::Mutex;
|
|
|
|
use crate::coordinator::HiveEnv;
|
|
use crate::lifecycle;
|
|
|
|
const GIT_NAME: &str = "c0re";
|
|
const GIT_EMAIL: &str = "c0re@hyperhive.local";
|
|
|
|
/// Single-writer lock around every meta-repo operation. Git isn't
|
|
/// safe to drive from concurrent processes against the same `.git/`
|
|
/// — two simultaneous `git add` / `commit` invocations race on
|
|
/// `.git/index.lock`; if either dies before releasing, the lock
|
|
/// sticks and the next operation hits "another git process seems to
|
|
/// be running" until somebody `rm`s it manually. Holding this mutex
|
|
/// across each public function's git+nix calls makes parallel
|
|
/// rebuilds (`auto_update` + dashboard-triggered + apply-commit)
|
|
/// take turns instead of colliding.
|
|
static META_LOCK: Mutex<()> = Mutex::const_new(());
|
|
|
|
// Exclusivity for meta-repo *windows* that span multiple `META_LOCK`
|
|
// acquisitions — above all the two-phase deploy (`prepare_deploy` stages
|
|
// `flake.lock` uncommitted for the whole container build;
|
|
// `finalize_deploy` / `abort_deploy` resolve it) — is **not** a mutex in
|
|
// this module. `META_LOCK` above serializes individual git ops but cannot
|
|
// keep another op out of that staged window; that window is owned by the
|
|
// job queue instead, as `Resource::MetaWindow`, declared by every
|
|
// meta-mutating node kind (it declares `Resource::MetaWindow`). A resource can
|
|
// be held by a subtree root across its children, which a `MutexGuard`
|
|
// (bounded by one executor fn) cannot — that's what lets the deploy be
|
|
// modelled as sub-nodes rather than one opaque node.
|
|
|
|
/// Where the manager sees this directory inside its container (RO bind).
|
|
pub const CONTAINER_MANAGER_META_MOUNT: &str = "/meta";
|
|
|
|
#[derive(Debug, Clone)]
|
|
pub struct AgentSpec {
|
|
pub name: String,
|
|
pub is_manager: bool,
|
|
pub port: u16,
|
|
}
|
|
|
|
/// Stage every generated meta JSON file that exists: topology.json is
|
|
/// regenerated by `reconcile` whenever the agent set changed;
|
|
/// tool-groups/capabilities/resource-limits/roles are created lazily on
|
|
/// first write (`set_groups`/`set_caps`/`set_limits`/role assignment) —
|
|
/// absent means every agent is on defaults, no file needed. Without
|
|
/// staging, an existing-but-untracked file (e.g. roles.json) shows up as
|
|
/// untracked in the meta repo, which can confuse nix's dirty-tree fetch.
|
|
/// `git add` is a no-op when content is unchanged.
|
|
async fn stage_generated_meta_files(dir: &std::path::Path) -> Result<()> {
|
|
for (path, name) in [
|
|
(crate::topology::topology_path(), "topology.json"),
|
|
(crate::tool_groups::tool_groups_path(), "tool-groups.json"),
|
|
(
|
|
crate::capabilities::capabilities_path(),
|
|
"capabilities.json",
|
|
),
|
|
(
|
|
crate::resource_limits::resource_limits_path(),
|
|
"resource-limits.json",
|
|
),
|
|
(crate::topology::roles_path(), "roles.json"),
|
|
] {
|
|
if path.exists() {
|
|
git(dir, &["add", name]).await?;
|
|
}
|
|
}
|
|
Ok(())
|
|
}
|
|
|
|
/// Idempotently reconcile the meta repo with the current agent set.
|
|
/// First call inits the git repo, runs `nix flake lock`, and lands a
|
|
/// seed commit. Subsequent calls only touch `flake.nix` when the
|
|
/// rendered contents differ from disk; an unchanged `flake.nix` is a
|
|
/// no-op.
|
|
pub async fn sync_agents(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<()> {
|
|
let _guard = META_LOCK.lock().await;
|
|
// Before anything is written: a hive without a forge URL would deploy a
|
|
// whole fleet of agents that silently never log in, because the agent
|
|
// option treats "unset" as "no forge configured" rather than erroring.
|
|
// This is the layer that knows a forge is mandatory, so it says so here.
|
|
require_service_urls(&forwarded_env_vars())?;
|
|
let dir = crate::paths::meta_root();
|
|
std::fs::create_dir_all(&dir).with_context(|| format!("create {}", dir.display()))?;
|
|
|
|
let new_flake = render_flake(
|
|
&hive.hyperhive_flake,
|
|
&hive.hyperhive_docs_flake,
|
|
&hive.nixpkgs_flake,
|
|
hive.claude_code_path.as_deref(),
|
|
hive.dashboard_port,
|
|
&hive.operator_pronouns,
|
|
&hive.context_window_tokens,
|
|
&hive.agent_memory_max,
|
|
agents,
|
|
);
|
|
let flake_path = dir.join("flake.nix");
|
|
let on_disk = std::fs::read_to_string(&flake_path).unwrap_or_default();
|
|
let initial = !dir.join(".git").exists();
|
|
|
|
// Embedded-CA list: keep the `./hive-ca.pem` file at the meta root in
|
|
// lockstep with its host source so the build-time `certificateFiles`
|
|
// list render_flake emits always resolves. Empty when no self-signed
|
|
// hive CA is configured.
|
|
let (ca_files, ca_changed) = ca_embed_state(&dir);
|
|
|
|
// Skip only when both the flake AND the embedded CA are unchanged — a
|
|
// CA rotation with an otherwise-identical flake must still re-commit.
|
|
if !initial && on_disk == new_flake && !ca_changed {
|
|
return Ok(());
|
|
}
|
|
|
|
// Safety guard: refuse to write an empty agent list over a non-empty
|
|
// on-disk flake. An empty `agents` slice is never intentional — it
|
|
// means `nixos-container list` failed and the caller got an empty
|
|
// fallback. Overwriting here would drop every agent from the meta
|
|
// flake and trigger unnecessary (and potentially destructive) cascade
|
|
// rebuilds. Callers that genuinely need to clear the agent list
|
|
// (there are none today) must handle this case explicitly.
|
|
if agents.is_empty() && !initial && !on_disk.is_empty() {
|
|
tracing::warn!(
|
|
"sync_agents: refusing to overwrite non-empty meta flake with empty agent list \
|
|
(nixos-container list may have failed)"
|
|
);
|
|
return Ok(());
|
|
}
|
|
|
|
std::fs::write(&flake_path, &new_flake)
|
|
.with_context(|| format!("write {}", flake_path.display()))?;
|
|
|
|
// Materialise the embedded CA list next to flake.nix + drop any stale
|
|
// CA file; `ca_touched` is every filename written or removed, staged
|
|
// for commit below. Public CA certs only; no private key is embedded.
|
|
let ca_touched = materialise_ca_files(&dir, &ca_files)?;
|
|
|
|
// Reconcile topology.json against the live agent set — adds
|
|
// entries for newly-spawned agents (default: manager as parent,
|
|
// manager itself as root) and drops removed agents. Operator
|
|
// overrides via the write API are preserved because reconcile
|
|
// only fills in missing entries. Idempotent; when nothing changed
|
|
// the file isn't touched.
|
|
let agent_names: Vec<String> = agents.iter().map(|a| a.name.clone()).collect();
|
|
let pending: Vec<String> = crate::coordinator::Coordinator::pending_init_names()
|
|
.into_iter()
|
|
.map(hive_types::Ident::into_string)
|
|
.collect();
|
|
crate::topology::reconcile(&agent_names, &pending)
|
|
.with_context(|| format!("reconcile {}", crate::topology::topology_path().display()))?;
|
|
|
|
// Refresh /var/lib/hyperhive/run/agent-sockets.json — drives the
|
|
// gateway's unix-socket upstreams. Every agent binds a unix socket
|
|
// (`HIVE_WEB_SOCKET`); the gateway routes via this map, falling back
|
|
// to a computed TCP loopback port (`lifecycle::agent_web_port`) only
|
|
// while an agent's socket marker is absent. Best-effort + non-fatal.
|
|
// See `docs/gateway.md::Per-agent unix-socket upstream`.
|
|
if let Err(e) = crate::agent_sockets::write(&agent_names) {
|
|
tracing::warn!(error = ?e, "agent_sockets::write failed (non-fatal)");
|
|
}
|
|
|
|
// Refresh /var/lib/hive-gateway/conf/agents.conf — the nginx include
|
|
// file the gateway reads at runtime. c0re then triggers a reload (or
|
|
// start) of the host's nginx via hive-priv, since c0re is unprivileged.
|
|
// Same best-effort + non-fatal shape.
|
|
if let Err(e) = crate::gateway_nginx::write(&agent_names).await {
|
|
tracing::warn!(error = ?e, "gateway_nginx::write failed (non-fatal)");
|
|
}
|
|
|
|
if initial {
|
|
git(&dir, &["init", "--initial-branch=main"]).await?;
|
|
}
|
|
// Stage flake.nix *before* running nix flake lock. When meta is
|
|
// a git repo, nix treats it as a `git+file://` self-reference;
|
|
// its dirty-tree fetcher includes index entries (tracked +
|
|
// staged) but skips untracked files, so without the stage step
|
|
// an untracked flake.nix surfaces as "source tree does not
|
|
// contain '/flake.nix'". Lock then commit once with both
|
|
// flake.nix and flake.lock — single commit per change.
|
|
git(&dir, &["add", "flake.nix"]).await?;
|
|
// Stage every embedded CA file we wrote or removed (hive CA + peer
|
|
// CAs). `git add <path>` stages a deletion when the path is tracked
|
|
// and now gone; best-effort so the never-tracked-and-absent case
|
|
// (pathspec mismatch) is a harmless no-op.
|
|
for name in &ca_touched {
|
|
let _ = git(&dir, &["add", "--", name]).await;
|
|
}
|
|
stage_generated_meta_files(&dir).await?;
|
|
nix(&dir, &["flake", "lock"]).await?;
|
|
if std::path::Path::new(&dir).join("flake.lock").exists() {
|
|
git(&dir, &["add", "flake.lock"]).await?;
|
|
}
|
|
// Build the commit message from what's actually staged so it
|
|
// reflects reality — and skip the commit entirely when nothing
|
|
// changed (avoids "nothing to commit" errors on redundant syncs).
|
|
let staged = git_staged_names(&dir).await?;
|
|
if staged.is_empty() {
|
|
return Ok(());
|
|
}
|
|
let msg = if initial {
|
|
format!("seed meta from {} agent(s)", agents.len())
|
|
} else {
|
|
// Compose a message that names every file that actually changed,
|
|
// mapping the on-disk filename to a short human label.
|
|
let labels: Vec<&str> = staged
|
|
.iter()
|
|
.filter_map(|f| match f.as_str() {
|
|
"flake.nix" => Some("flake"),
|
|
"flake.lock" => Some("lock"),
|
|
"hive-ca.pem" => Some("hive-ca"),
|
|
// Only ever a REMOVAL now; see `is_embedded_ca_name`.
|
|
f if f.starts_with("peer-ca-") && has_pem_ext(f) => Some("peer-ca"),
|
|
"topology.json" => Some("topology"),
|
|
"capabilities.json" => Some("capabilities"),
|
|
"resource-limits.json" => Some("resource-limits"),
|
|
"tool-groups.json" => Some("tool-groups"),
|
|
"roles.json" => Some("roles"),
|
|
_ => None,
|
|
})
|
|
.collect();
|
|
if labels.is_empty() {
|
|
"meta: update".to_owned()
|
|
} else {
|
|
format!("meta: update {}", labels.join(", "))
|
|
}
|
|
};
|
|
git_commit(&dir, &msg).await?;
|
|
Ok(())
|
|
}
|
|
|
|
/// Phase 1 of an apply-commit deploy. Updates the locked rev of
|
|
/// `agent-<name>` to whatever `applied/<name>/main` currently points
|
|
/// at and **stages** the lock so `nixos-container update --flake
|
|
/// meta#<n>` (which reads via `git+file://`) sees the new rev via
|
|
/// the index. Doesn't commit — `finalize_deploy` commits on build
|
|
/// success, `abort_deploy` drops the staged change on failure so
|
|
/// meta history only carries successful deploys.
|
|
pub async fn prepare_deploy(name: &str, node_id: Option<u64>) -> Result<()> {
|
|
let _guard = META_LOCK.lock().await;
|
|
let dir = crate::paths::meta_root();
|
|
let input = format!("agent-{name}");
|
|
// Re-lock the agent input against the LOCAL applied mirror, not the
|
|
// persistent forge URL declared in the meta flake (see the `## Meta flake`
|
|
// note in docs/approvals.md): the deploy must build the exact reviewed
|
|
// config that `verify_commit` gated and `applied/<n>/main` was
|
|
// fast-forwarded to, and it must keep working when the forge is
|
|
// unreachable (rebuilds fire on crash-restart / meta bumps too, not just
|
|
// config PRs). `--override-input` writes the applied rev into the lock;
|
|
// the forge URL stays the declared, reviewable source of truth.
|
|
let applied = applied_override_url(&crate::paths::applied_dir(name));
|
|
nix_logged(
|
|
&dir,
|
|
&[
|
|
"flake",
|
|
"update",
|
|
&input,
|
|
"--override-input",
|
|
&input,
|
|
&applied,
|
|
],
|
|
name,
|
|
"prepare-deploy",
|
|
node_id,
|
|
)
|
|
.await?;
|
|
// Stage the new lock — git+file://'s dirty-tree fetcher reads
|
|
// index entries, so the upcoming nixos-container update sees the
|
|
// bumped rev without a commit yet.
|
|
git(&dir, &["add", "flake.lock"]).await
|
|
}
|
|
|
|
/// Phase 2-success. Commit the staged lock with the deployed tag +
|
|
/// sha as the message. No-op when the rev was already at the right
|
|
/// place (nothing staged → nothing to commit).
|
|
pub async fn finalize_deploy(name: &str, sha: &str, tag: &str) -> Result<()> {
|
|
let _guard = META_LOCK.lock().await;
|
|
let dir = crate::paths::meta_root();
|
|
if !paths_dirty(&dir, &["flake.lock"]).await? {
|
|
return Ok(());
|
|
}
|
|
let short = &sha[..sha.len().min(12)];
|
|
git_commit_paths(
|
|
&dir,
|
|
&format!("deploy {name} {tag} {short}"),
|
|
&["flake.lock"],
|
|
)
|
|
.await
|
|
}
|
|
|
|
/// Phase 2-failure. Unstage + restore the lock so meta returns to
|
|
/// the previously-committed shas. The failed proposal is still
|
|
/// captured in `applied/<n>`'s annotated `failed/<id>` tag.
|
|
pub async fn abort_deploy() -> Result<()> {
|
|
let _guard = META_LOCK.lock().await;
|
|
let dir = crate::paths::meta_root();
|
|
git(&dir, &["restore", "--staged", "flake.lock"]).await?;
|
|
git(&dir, &["restore", "flake.lock"]).await
|
|
}
|
|
|
|
/// One-shot used by the manual-rebuild path: relock just one
|
|
/// agent's input and commit the lock change if any. Single-phase
|
|
/// (no separate finalize) because rebuild has no failure-revert
|
|
/// semantics — it always wants the latest main.
|
|
pub async fn lock_update_for_rebuild(name: &str) -> Result<()> {
|
|
let _guard = META_LOCK.lock().await;
|
|
let dir = crate::paths::meta_root();
|
|
let input = format!("agent-{name}");
|
|
// Re-lock from the local applied mirror, not the persistent forge URL —
|
|
// same rationale as `prepare_deploy`: build exactly `applied/<n>/main` and
|
|
// stay reproducible when the forge is unreachable.
|
|
let applied = applied_override_url(&crate::paths::applied_dir(name));
|
|
nix(
|
|
&dir,
|
|
&[
|
|
"flake",
|
|
"update",
|
|
&input,
|
|
"--override-input",
|
|
&input,
|
|
&applied,
|
|
],
|
|
)
|
|
.await?;
|
|
if !paths_dirty(&dir, &["flake.lock"]).await? {
|
|
return Ok(());
|
|
}
|
|
git(&dir, &["add", "flake.lock"]).await?;
|
|
git_commit_paths(
|
|
&dir,
|
|
&format!("rebuild {name}: lock update"),
|
|
&["flake.lock"],
|
|
)
|
|
.await
|
|
}
|
|
|
|
/// Build the `--override-input` value pinning an agent's config repo to
|
|
/// an exact revision: `git+file://<applied_dir>?rev=<sha>`. Pure so the
|
|
/// URL shape is unit-testable without a nix shellout.
|
|
fn agent_input_override(applied_dir: &Path, sha: &str) -> String {
|
|
format!("git+file://{}?rev={sha}", applied_dir.display())
|
|
}
|
|
|
|
/// `--override-input` URL re-locking an agent's config input against its
|
|
/// LOCAL applied mirror (`git+file://<applied_dir>`, current `main` head)
|
|
/// instead of the persistent forge URL declared in the meta flake. The
|
|
/// deploy + rebuild paths re-lock from here: the applied tree was already
|
|
/// fast-forwarded to the reviewed head, so this builds exactly that config
|
|
/// and stays reproducible when the forge is unreachable. No `?rev` — `main`
|
|
/// head is the reviewed head at deploy time. Pure so it's unit-testable.
|
|
fn applied_override_url(applied_dir: &Path) -> String {
|
|
format!("git+file://{}", applied_dir.display())
|
|
}
|
|
|
|
/// Non-mutating "would this commit apply?" verify for the PR-based config
|
|
/// flow. Evaluates the agent's nixos configuration with its meta
|
|
/// input overridden to the exact `sha`, WITHOUT moving `applied/main` or
|
|
/// writing the real meta `flake.lock`: `--override-input` pins the input
|
|
/// at eval time only and `--no-write-lock-file` guarantees the on-disk
|
|
/// lock is never touched (a genuinely-needed lock change surfaces as an
|
|
/// error rather than a silent mutation). Confirms the flake at `sha`
|
|
/// evaluates and the meta lock resolves around it. `Ok(())` = would
|
|
/// apply; `Err` carries the eval/lock failure so the operator's approve
|
|
/// is rejected before any live-state change.
|
|
///
|
|
/// Precondition: `sha` must be reachable in `applied_dir`'s object store
|
|
/// — the caller fetches the PR head into `applied_dir` first. Eval-only
|
|
/// (no build): catches nix / eval / module-option errors + lock
|
|
/// resolution at the same cost profile as the legacy pre-merge check;
|
|
/// the real container build still runs (and can still roll back) on the
|
|
/// actual apply, so this is the right "would this apply" gate.
|
|
pub async fn verify_commit(
|
|
name: &str,
|
|
applied_dir: &Path,
|
|
sha: &str,
|
|
node_id: Option<u64>,
|
|
) -> Result<()> {
|
|
let _guard = META_LOCK.lock().await;
|
|
let dir = crate::paths::meta_root();
|
|
let input = format!("agent-{name}");
|
|
let over = agent_input_override(applied_dir, sha);
|
|
let attr = format!(".#nixosConfigurations.{name}.config.system.build.toplevel.drvPath");
|
|
nix_logged(
|
|
&dir,
|
|
&[
|
|
"eval",
|
|
&attr,
|
|
"--override-input",
|
|
&input,
|
|
&over,
|
|
"--no-write-lock-file",
|
|
],
|
|
name,
|
|
"verify",
|
|
node_id,
|
|
)
|
|
.await
|
|
}
|
|
|
|
/// Update one or more named inputs in the meta flake and commit
|
|
/// the resulting lock change with a single combined message.
|
|
/// Used by the dashboard's "update meta inputs" form so the
|
|
/// operator can bulk-bump `hyperhive` + selected agents in one
|
|
/// shot. Each input name is passed verbatim to
|
|
/// Run `nix flake update [inputs...]` on the meta flake and commit the
|
|
/// resulting lock changes. When `inputs` is empty, updates ALL inputs
|
|
/// (bare `nix flake update`). The caller is responsible for picking
|
|
/// real input keys (e.g. via `inputs_view()` snapshotted from the lock
|
|
/// file) when targeting specific inputs.
|
|
pub async fn lock_update(inputs: &[String]) -> Result<()> {
|
|
let _guard = META_LOCK.lock().await;
|
|
let dir = crate::paths::meta_root();
|
|
let mut args: Vec<&str> = vec!["flake", "update"];
|
|
for i in inputs {
|
|
args.push(i.as_str());
|
|
}
|
|
nix(&dir, &args).await?;
|
|
if !paths_dirty(&dir, &["flake.lock"]).await? {
|
|
return Ok(());
|
|
}
|
|
git(&dir, &["add", "flake.lock"]).await?;
|
|
let msg = if inputs.is_empty() {
|
|
"lock update: all inputs".to_string()
|
|
} else if inputs.len() == 1 {
|
|
format!("lock update: {}", inputs[0])
|
|
} else {
|
|
format!("lock update: {}", inputs.join(", "))
|
|
};
|
|
git_commit_paths(&dir, &msg, &["flake.lock"]).await
|
|
}
|
|
|
|
/// One-shot used by the auto-update path: pin the latest hyperhive
|
|
/// rev, commit if the lock changed. Cheaper than `sync_agents`
|
|
/// because the per-agent inputs aren't touched.
|
|
pub async fn lock_update_hyperhive() -> Result<()> {
|
|
let _guard = META_LOCK.lock().await;
|
|
let dir = crate::paths::meta_root();
|
|
nix(&dir, &["flake", "update", "hyperhive"]).await?;
|
|
if !paths_dirty(&dir, &["flake.lock"]).await? {
|
|
return Ok(());
|
|
}
|
|
git(&dir, &["add", "flake.lock"]).await?;
|
|
git_commit_paths(&dir, "bump hyperhive", &["flake.lock"]).await
|
|
}
|
|
|
|
/// Write the tool-groups file for `agent` and commit it atomically
|
|
/// under `META_LOCK`. Ensures the JSON change is staged + committed
|
|
/// before the next `prepare_deploy` or `sync_agents` runs, so the
|
|
/// working tree is never left dirty by an untimely `PermChange` write.
|
|
pub async fn commit_tool_groups(agent: &str, groups: &[String]) -> Result<()> {
|
|
let _guard = META_LOCK.lock().await;
|
|
crate::tool_groups::set_groups(agent, groups)?;
|
|
let dir = crate::paths::meta_root();
|
|
if crate::tool_groups::tool_groups_path().exists() {
|
|
git(&dir, &["add", "tool-groups.json"]).await?;
|
|
}
|
|
if paths_dirty(&dir, &["tool-groups.json"]).await? {
|
|
git_commit_paths(
|
|
&dir,
|
|
&format!("set tool-groups for {agent}"),
|
|
&["tool-groups.json"],
|
|
)
|
|
.await?;
|
|
}
|
|
Ok(())
|
|
}
|
|
|
|
/// Write the capabilities file for `agent` and commit it atomically
|
|
/// under `META_LOCK`. Same rationale as `commit_tool_groups`.
|
|
pub async fn commit_capabilities(agent: &str, caps: &[String]) -> Result<()> {
|
|
let _guard = META_LOCK.lock().await;
|
|
crate::capabilities::set_caps(agent, caps)
|
|
.map_err(|e| anyhow::anyhow!("set capabilities for {agent}: {e}"))?;
|
|
let dir = crate::paths::meta_root();
|
|
if crate::capabilities::capabilities_path().exists() {
|
|
git(&dir, &["add", "capabilities.json"]).await?;
|
|
}
|
|
if paths_dirty(&dir, &["capabilities.json"]).await? {
|
|
git_commit_paths(
|
|
&dir,
|
|
&format!("set capabilities for {agent}"),
|
|
&["capabilities.json"],
|
|
)
|
|
.await?;
|
|
}
|
|
Ok(())
|
|
}
|
|
|
|
/// Write the resource-limits file for `agent` and commit it atomically
|
|
/// under `META_LOCK`. Same rationale as `commit_tool_groups`: the
|
|
/// working tree must never be left dirty for the next `prepare_deploy`
|
|
/// or `sync_agents` to trip over.
|
|
///
|
|
/// Unlike the perm files this one is never injected into the container —
|
|
/// it's a host-side cap *on* the agent — but it lives in the same repo
|
|
/// so a limit change gets the same auditable one-commit-per-change trail.
|
|
///
|
|
/// # Errors
|
|
///
|
|
/// Returns an error if writing the JSON file fails or a git stage/commit
|
|
/// step fails.
|
|
pub async fn commit_resource_limits(
|
|
agent: &str,
|
|
limits: &crate::resource_limits::AgentLimits,
|
|
) -> Result<()> {
|
|
let _guard = META_LOCK.lock().await;
|
|
crate::resource_limits::set_limits(agent, limits)
|
|
.map_err(|e| anyhow::anyhow!("set resource limits for {agent}: {e}"))?;
|
|
let dir = crate::paths::meta_root();
|
|
if crate::resource_limits::resource_limits_path().exists() {
|
|
git(&dir, &["add", "resource-limits.json"]).await?;
|
|
}
|
|
if paths_dirty(&dir, &["resource-limits.json"]).await? {
|
|
git_commit_paths(
|
|
&dir,
|
|
&format!("set resource limits for {agent}"),
|
|
&["resource-limits.json"],
|
|
)
|
|
.await?;
|
|
}
|
|
Ok(())
|
|
}
|
|
|
|
/// Write both perm files for `agent` (whichever are `Some`) and commit
|
|
/// them in a SINGLE git commit under `META_LOCK` — the batch
|
|
/// `POST /api/permissions` path. A `None` field leaves that file
|
|
/// untouched. One commit + (caller does) one rebuild means changing an
|
|
/// agent's caps and tool-groups together no longer triggers two
|
|
/// rebuilds. Mirrors the staging discipline of `commit_tool_groups` /
|
|
/// `commit_capabilities`.
|
|
///
|
|
/// # Errors
|
|
///
|
|
/// Returns an error if writing either JSON file fails, a capability name
|
|
/// is invalid (`set_caps`), or a git stage/commit step fails.
|
|
pub async fn commit_perms(
|
|
agent: &str,
|
|
groups: Option<&[String]>,
|
|
caps: Option<&[String]>,
|
|
) -> Result<()> {
|
|
let _guard = META_LOCK.lock().await;
|
|
let dir = crate::paths::meta_root();
|
|
let mut parts: Vec<&str> = Vec::new();
|
|
if let Some(groups) = groups {
|
|
crate::tool_groups::set_groups(agent, groups)?;
|
|
if crate::tool_groups::tool_groups_path().exists() {
|
|
git(&dir, &["add", "tool-groups.json"]).await?;
|
|
}
|
|
parts.push("tool-groups");
|
|
}
|
|
if let Some(caps) = caps {
|
|
crate::capabilities::set_caps(agent, caps)
|
|
.map_err(|e| anyhow::anyhow!("set capabilities for {agent}: {e}"))?;
|
|
if crate::capabilities::capabilities_path().exists() {
|
|
git(&dir, &["add", "capabilities.json"]).await?;
|
|
}
|
|
parts.push("capabilities");
|
|
}
|
|
let paths = ["tool-groups.json", "capabilities.json"];
|
|
if paths_dirty(&dir, &paths).await? {
|
|
git_commit_paths(
|
|
&dir,
|
|
&format!("set {} for {agent}", parts.join(" + ")),
|
|
&paths,
|
|
)
|
|
.await?;
|
|
}
|
|
Ok(())
|
|
}
|
|
|
|
/// Applies every `(child, new_parent)` move under a single `META_LOCK`
|
|
/// acquisition and creates **one** git commit for all of them — a
|
|
/// single-move call is just a one-element slice, so there's no separate
|
|
/// non-batch entry point. Moves are applied in the order given; the first
|
|
/// validation error short-circuits the whole batch. True atomic write: all
|
|
/// moves are pre-validated against a cumulative in-memory state with
|
|
/// [`crate::topology::apply_set_parent`] before anything touches disk, then
|
|
/// [`crate::topology::write`] is called exactly once. If any move fails
|
|
/// validation the topology file is never modified.
|
|
///
|
|
/// The multi-move commit message uses `moves[0].1` as the destination label.
|
|
/// This is intentional: the dashboard bulk-move UI always sends a single
|
|
/// destination for all selected agents, so the message is always accurate in
|
|
/// practice.
|
|
///
|
|
/// Returns a `Vec` of `(child, old_parent)` pairs for every move that
|
|
/// actually changed the topology (idempotent same-parent moves are skipped),
|
|
/// so the caller can send targeted notifications.
|
|
///
|
|
/// # Errors
|
|
///
|
|
/// Returns a `String` error if any move fails validation (cycle, unknown
|
|
/// agent, etc.) or if the topology file cannot be written. Git-commit failure
|
|
/// is logged as a warning and does not propagate — `sync_agents` will recover.
|
|
pub async fn bulk_commit_topology(
|
|
moves: &[(&str, Option<&str>)],
|
|
) -> std::result::Result<Vec<(String, Option<String>)>, String> {
|
|
if moves.is_empty() {
|
|
return Ok(vec![]);
|
|
}
|
|
let _guard = META_LOCK.lock().await;
|
|
// Snapshot parents before any writes so we can compute the diff.
|
|
let topo_before = crate::topology::read();
|
|
// Validate all moves against a cumulative in-memory state -- no disk
|
|
// writes yet; first error aborts with the topology file untouched.
|
|
let mut next = topo_before.clone();
|
|
for (child, new_parent) in moves {
|
|
next = crate::topology::apply_set_parent(&next, child, *new_parent)?;
|
|
}
|
|
// Only flush to disk if something actually changed.
|
|
if next != topo_before {
|
|
crate::topology::write(&next).map_err(|e| format!("{e:#}"))?;
|
|
}
|
|
// Commit the whole batch as one git operation.
|
|
let dir = crate::paths::meta_root();
|
|
let commit_msg = if moves.len() == 1 {
|
|
let (child, new_parent) = moves[0];
|
|
format!("topology: {} → {}", child, new_parent.unwrap_or("<root>"))
|
|
} else {
|
|
let names: Vec<&str> = moves.iter().map(|(c, _)| *c).collect();
|
|
let dest = moves[0].1.unwrap_or("<root>");
|
|
format!(
|
|
"topology: move {} agents → {} ({})",
|
|
moves.len(),
|
|
dest,
|
|
names.join(", ")
|
|
)
|
|
};
|
|
let stage = async {
|
|
git(&dir, &["add", "topology.json"]).await?;
|
|
if paths_dirty(&dir, &["topology.json"]).await? {
|
|
git_commit_paths(&dir, &commit_msg, &["topology.json"]).await?;
|
|
}
|
|
Ok::<_, anyhow::Error>(())
|
|
};
|
|
if let Err(e) = stage.await {
|
|
tracing::warn!(error = ?e, "bulk_commit_topology: topology written but git commit failed (sync_agents will recover)");
|
|
}
|
|
// Return (child, old_parent) for each move that changed state.
|
|
let changed = moves
|
|
.iter()
|
|
.filter_map(|(child, new_parent)| {
|
|
let old = topo_before.get(*child).cloned().flatten();
|
|
(old.as_deref() != *new_parent).then_some((child.to_string(), old))
|
|
})
|
|
.collect();
|
|
Ok(changed)
|
|
}
|
|
|
|
#[allow(
|
|
clippy::too_many_arguments,
|
|
reason = "many genuine flake inputs (source flakes, port, pronouns, tokens, \
|
|
agents, forge base); a params struct would just move the same fields"
|
|
)]
|
|
fn render_flake(
|
|
hyperhive_flake: &str,
|
|
docs_flake: &str,
|
|
nixpkgs_flake: &str,
|
|
claude_code_path: Option<&str>,
|
|
dashboard_port: u16,
|
|
operator_pronouns: &str,
|
|
context_window_tokens: &std::collections::HashMap<String, u64>,
|
|
hive_memory_max: &str,
|
|
agents: &[AgentSpec],
|
|
) -> String {
|
|
render_flake_with_lookup(
|
|
hyperhive_flake,
|
|
docs_flake,
|
|
nixpkgs_flake,
|
|
claude_code_path,
|
|
dashboard_port,
|
|
operator_pronouns,
|
|
context_window_tokens,
|
|
hive_memory_max,
|
|
agents,
|
|
agent_canonical_inputs,
|
|
)
|
|
}
|
|
|
|
/// Canonical inputs meta knows how to dedup. An agent that declares one
|
|
/// of these as a top-level input in its own `flake.nix` will get a
|
|
/// `follows = "<name>"` line emitted in meta — collapsing the
|
|
/// otherwise-separate-but-identical `nixpkgs_N` nodes into a single
|
|
/// meta-level reference.
|
|
const CANONICAL_INPUTS: &[&str] = &["nixpkgs"];
|
|
|
|
/// Env vars hive-c0re forwards from its own systemd unit env into every
|
|
/// sub-agent's harness service env. Each entry is `(env_var_name,
|
|
/// host_value)`. Empty / unset vars are filtered out so absent options
|
|
/// don't render no-op `FOO = ""` lines into the meta flake.
|
|
///
|
|
/// Returns owned `String` values so the result is `'static`-friendly +
|
|
/// trivial to stub from tests (which build their own slice instead of
|
|
/// touching process-wide env).
|
|
const FORWARDED_VARS: &[&str] = &[
|
|
"HIVE_FORGE_URL",
|
|
"HIVE_FORGE_PUBLIC_URL",
|
|
"HIVE_MATRIX_URL",
|
|
"HYPERHIVE_HIVE_DOMAIN",
|
|
"HYPERHIVE_HIVE_NAME",
|
|
"HYPERHIVE_SWARM_NAME",
|
|
];
|
|
|
|
/// Map of forwarded env var -> the agent option carrying the same value.
|
|
///
|
|
/// Both exist because they're consumed at different times: the option is baked
|
|
/// into scripts at build time (tea-login bakes `FORGE_URL` from it), the env
|
|
/// var is read at runtime. Setting only one leaves the other on its default,
|
|
/// which is how a hive ends up with two disagreeing answers for one value.
|
|
///
|
|
/// The hive/swarm display names are here for exactly that reason, learned the
|
|
/// hard way: they were forwarded as runtime env only, while
|
|
/// `claude-settings.nix` read them from the container's `environment.variables`
|
|
/// at *eval* time — where they were never set. Every agent baked
|
|
/// `hive=unknown,swarm=unknown` into its OTEL resource attributes and shipped
|
|
/// that label on every metric, while the same process's env held the right
|
|
/// answer. Half-wiring this map is not a missing nicety; it is a value that
|
|
/// evaluates fine and is silently wrong.
|
|
const FORWARDED_VAR_OPTIONS: &[(&str, &str)] = &[
|
|
("HIVE_FORGE_URL", "hyperhive.forge.url"),
|
|
("HIVE_MATRIX_URL", "hyperhive.matrix.url"),
|
|
("HYPERHIVE_HIVE_NAME", "hyperhive.hiveName"),
|
|
("HYPERHIVE_SWARM_NAME", "hyperhive.swarmName"),
|
|
];
|
|
|
|
/// Render the forwarded-var option assignments for one agent's module block.
|
|
///
|
|
/// Split out of `render_flake` so it can be tested without touching process
|
|
/// env: the render-level tests have to `set_var`, which makes them race each
|
|
/// other under the default parallel test runner. A pure function over the
|
|
/// already-collected pairs has no such hazard.
|
|
///
|
|
/// A var that isn't present emits nothing rather than a guess. For an optional
|
|
/// service that is the whole point — the agent option defaults to `null`,
|
|
/// meaning "not configured", and the units that would use it aren't generated.
|
|
/// For a service the hive cannot run without, silence would instead produce a
|
|
/// fleet of agents quietly missing an integration, so those are checked by
|
|
/// [`require_service_urls`] before this is called.
|
|
fn push_forwarded_var_options(out: &mut String, vars: &[(&'static str, String)]) {
|
|
use std::fmt::Write as _;
|
|
for (var, val) in vars {
|
|
let Some((_, option)) = FORWARDED_VAR_OPTIONS.iter().find(|(name, _)| name == var) else {
|
|
continue;
|
|
};
|
|
let escaped = val.replace('\\', "\\\\").replace('"', "\\\"");
|
|
let _ = writeln!(out, " {option} = \"{escaped}\";");
|
|
}
|
|
}
|
|
|
|
/// Service URLs a running hive must always supply, checked before rendering.
|
|
///
|
|
/// The forge is not optional on a real hive: `hive-c0re.nix` sets
|
|
/// `HIVE_FORGE_URL` unconditionally, so its absence means this daemon was
|
|
/// started outside the NixOS module. The agent option is nullable — `null`
|
|
/// legitimately means "no forge" when the modules are evaluated on their own —
|
|
/// which is exactly why the hive has to assert its own requirement here rather
|
|
/// than leaning on the module to reject the empty case.
|
|
const REQUIRED_SERVICE_URL_VARS: &[&str] = &["HIVE_FORGE_URL"];
|
|
|
|
/// Fails unless every [`REQUIRED_SERVICE_URL_VARS`] entry is present in
|
|
/// `vars`.
|
|
///
|
|
/// Pure over the already-collected pairs so it can be tested without touching
|
|
/// process env (the same reason [`push_forwarded_var_options`] is split out).
|
|
///
|
|
/// Checked by `sync_agents` — the point at which the hive commits a rendered
|
|
/// flake to disk — rather than inside the renderer. Rendering is a pure string
|
|
/// operation that many tests exercise directly; making *it* env-dependent
|
|
/// would mean every one of those tests either sets a process-wide var (the
|
|
/// parallel-test race this module already avoids) or fails for reasons that
|
|
/// have nothing to do with what it asserts.
|
|
///
|
|
/// # Errors
|
|
///
|
|
/// When a required var is missing. `hive-c0re.nix` sets it unconditionally, so
|
|
/// this means the daemon is running outside the NixOS module; refusing to
|
|
/// write the flake beats writing one whose agents would silently all lack a
|
|
/// forge.
|
|
fn require_service_urls(vars: &[(&'static str, String)]) -> Result<()> {
|
|
for required in REQUIRED_SERVICE_URL_VARS {
|
|
if !vars.iter().any(|(name, _)| name == required) {
|
|
anyhow::bail!(
|
|
"{required} is unset — hive-c0re.nix sets it unconditionally, so this process \
|
|
was started outside the NixOS module. Refusing to write a meta flake whose \
|
|
agents would every one of them have no forge configured."
|
|
);
|
|
}
|
|
}
|
|
Ok(())
|
|
}
|
|
|
|
fn forwarded_env_vars() -> Vec<(&'static str, String)> {
|
|
FORWARDED_VARS
|
|
.iter()
|
|
.filter_map(|&name| {
|
|
std::env::var(name)
|
|
.ok()
|
|
.filter(|v| !v.is_empty())
|
|
.map(|v| (name, v))
|
|
})
|
|
.collect()
|
|
}
|
|
|
|
/// Filename the hive's own trust anchors are embedded under at the
|
|
/// meta-flake root — the only entry of the embedded-CA list
|
|
/// `render_flake` references in `security.pki.certificateFiles` (see
|
|
/// `embedded_ca_files`).
|
|
const HIVE_CA_FILE: &str = "hive-ca.pem";
|
|
|
|
/// Host path of the hive's TLS trust anchors, when self-signed TLS is
|
|
/// active. `hive-tls.nix` sets `HIVE_TLS_CA_PATH` in hive-c0re's service env
|
|
/// (to `<tls.stateDir>/trust-bundle.pem`) whenever the gateway serves a
|
|
/// self-signed, hive-CA-signed leaf. The file holds one *or more* certs —
|
|
/// the hive CA plus the swarm root it is issued under — and is copied
|
|
/// verbatim, which `security.pki.certificateFiles` accepts; nothing here
|
|
/// parses it. Returns `Some(path)` only when the var is set AND
|
|
/// the cert exists on disk — so render + write stay consistent (we never
|
|
/// emit a `certificateFiles` reference to a file we didn't embed). Only the
|
|
/// public cert is ever read here; the CA private key never leaves the host.
|
|
fn hive_ca_source() -> Option<String> {
|
|
let path = std::env::var("HIVE_TLS_CA_PATH").ok()?;
|
|
if path.is_empty() || !std::path::Path::new(&path).is_file() {
|
|
return None;
|
|
}
|
|
Some(path)
|
|
}
|
|
|
|
/// Hive-wide OTEL config injected into every agent's build, read off
|
|
/// hive-c0re's own unit env (set from `services.hyperhive.otel.*` in
|
|
/// `nix/modules/hive-c0re.nix`). A present, non-empty
|
|
/// `HYPERHIVE_OTEL_ENDPOINT` is the enable signal — the host module
|
|
/// asserts the endpoint is set whenever `otel.enable` is true, so
|
|
/// "endpoint present" == "OTEL on". The optional fields map to the
|
|
/// matching host options and are only carried when set.
|
|
struct OtelConfig {
|
|
endpoint: String,
|
|
protocol: String,
|
|
extra_resource_attributes: Option<String>,
|
|
metric_interval_ms: Option<u64>,
|
|
/// `HYPERHIVE_OTEL_DEBUG=1` → `hyperhive.otel.debug = true` →
|
|
/// `CLAUDE_CODE_OTEL_DIAG_STDERR=1` in every agent's env.
|
|
debug: bool,
|
|
}
|
|
|
|
/// Read the hive-wide OTEL config from env, or `None` when OTEL is off.
|
|
/// Mirrors `hive_ca_source` — host state surfaced to the meta renderer
|
|
/// so it can bake build-time `hyperhive.otel.*` config into each agent
|
|
/// (the per-agent options the harness modules consume). Returns `None`
|
|
/// when the endpoint signal is absent so the renderer emits no
|
|
/// `hyperhive.otel.*` lines and agents keep the disabled default.
|
|
fn otel_config() -> Option<OtelConfig> {
|
|
let endpoint = std::env::var("HYPERHIVE_OTEL_ENDPOINT")
|
|
.ok()
|
|
.filter(|v| !v.is_empty())?;
|
|
let protocol = std::env::var("HYPERHIVE_OTEL_PROTOCOL")
|
|
.ok()
|
|
.filter(|v| !v.is_empty())
|
|
.unwrap_or_else(|| "http/protobuf".to_owned());
|
|
let extra_resource_attributes = std::env::var("HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES")
|
|
.ok()
|
|
.filter(|v| !v.is_empty());
|
|
let metric_interval_ms = std::env::var("HYPERHIVE_OTEL_METRIC_INTERVAL_MS")
|
|
.ok()
|
|
.and_then(|v| v.parse::<u64>().ok())
|
|
.filter(|v| *v > 0);
|
|
let debug = std::env::var("HYPERHIVE_OTEL_DEBUG")
|
|
.ok()
|
|
.is_some_and(|v| v == "1");
|
|
Some(OtelConfig {
|
|
endpoint,
|
|
protocol,
|
|
extra_resource_attributes,
|
|
metric_interval_ms,
|
|
debug,
|
|
})
|
|
}
|
|
|
|
/// The ordered set of CA certs embedded next to the meta flake, as
|
|
/// `(filename, host_source_path)` — now just the hive's own trust
|
|
/// anchors as `hive-ca.pem`, when self-signed TLS is active.
|
|
///
|
|
/// That single file already carries the swarm root (`hive-tls.nix`
|
|
/// writes the hive CA *and* the root it is issued under into the trust
|
|
/// bundle), so every hive under the swarm root validates from it. Which
|
|
/// is why the per-peer CAs this used to append are gone: they said the
|
|
/// same thing once per peer.
|
|
///
|
|
/// `render_flake` emits exactly these filenames into
|
|
/// `security.pki.certificateFiles` and `sync_agents` materialises them,
|
|
/// so the rendered reference and the embedded files always agree.
|
|
fn embedded_ca_files() -> Vec<(String, String)> {
|
|
let mut out = Vec::new();
|
|
if let Some(p) = hive_ca_source() {
|
|
out.push((HIVE_CA_FILE.to_owned(), p));
|
|
}
|
|
out
|
|
}
|
|
|
|
/// Write each desired embedded CA file next to `flake.nix` and remove
|
|
/// any stale one (a hive CA turned off, or a peer dropped from config),
|
|
/// so the flake never references a file we didn't write. Returns every
|
|
/// filename written or removed, for the caller to stage. The public CA
|
|
/// certs only; no private key is ever embedded.
|
|
fn materialise_ca_files(dir: &Path, ca_files: &[(String, String)]) -> Result<Vec<String>> {
|
|
let desired: std::collections::HashSet<&str> =
|
|
ca_files.iter().map(|(n, _)| n.as_str()).collect();
|
|
let mut touched: Vec<String> = Vec::new();
|
|
if let Ok(entries) = std::fs::read_dir(dir) {
|
|
for e in entries.flatten() {
|
|
let fname = e.file_name();
|
|
let Some(name) = fname.to_str() else { continue };
|
|
if is_embedded_ca_name(name) && !desired.contains(name) {
|
|
let _ = std::fs::remove_file(dir.join(name));
|
|
touched.push(name.to_owned());
|
|
}
|
|
}
|
|
}
|
|
for (name, content) in ca_files {
|
|
let path = dir.join(name);
|
|
std::fs::write(&path, content).with_context(|| format!("write {}", path.display()))?;
|
|
touched.push(name.clone());
|
|
}
|
|
Ok(touched)
|
|
}
|
|
|
|
/// True for a filename the embedded-CA machinery owns. Lets
|
|
/// `sync_agents` find stale CA files to clean up (a CA dropped from
|
|
/// config) without touching unrelated meta files.
|
|
///
|
|
/// ⚠️ Still matches `peer-ca-<N>.pem`, which `embedded_ca_files` no
|
|
/// longer produces — deliberately. Cleanup is driven by
|
|
/// "recognised but not desired", so this arm is exactly what removes the
|
|
/// peer CAs a hive embedded before per-peer pinning was replaced by the
|
|
/// swarm root. Drop it and those files are orphaned at every meta root
|
|
/// forever, referenced by nothing and cleaned by nobody.
|
|
fn is_embedded_ca_name(name: &str) -> bool {
|
|
name == HIVE_CA_FILE || (name.starts_with("peer-ca-") && has_pem_ext(name))
|
|
}
|
|
|
|
/// True when `name` ends in a `.pem` extension (case-insensitive). Split
|
|
/// out so the embedded-CA filename checks share one spelling and dodge
|
|
/// clippy's case-sensitive-extension lint.
|
|
fn has_pem_ext(name: &str) -> bool {
|
|
Path::new(name)
|
|
.extension()
|
|
.is_some_and(|ext| ext.eq_ignore_ascii_case("pem"))
|
|
}
|
|
|
|
/// Embedded-CA state for the meta repo: `(desired_files, changed)`.
|
|
/// `desired_files` is `(filename, contents)` for every CA that should sit
|
|
/// next to flake.nix. `changed` is true when the on-disk set differs in
|
|
/// any way — the file's contents changed, a CA appeared, or a
|
|
/// previously-embedded CA (`hive-ca.pem` / a legacy `peer-ca-*.pem`) is
|
|
/// no longer wanted (stale, to be removed). Drives both the re-commit
|
|
/// decision and the materialise/cleanup in `sync_agents`, so a CA
|
|
/// rotation re-commits even when the flake itself is byte-identical.
|
|
fn ca_embed_state(dir: &std::path::Path) -> (Vec<(String, String)>, bool) {
|
|
let desired: Vec<(String, String)> = embedded_ca_files()
|
|
.into_iter()
|
|
.filter_map(|(name, path)| std::fs::read_to_string(&path).ok().map(|c| (name, c)))
|
|
.collect();
|
|
let desired_names: std::collections::HashSet<&str> =
|
|
desired.iter().map(|(n, _)| n.as_str()).collect();
|
|
|
|
let mut changed = desired.iter().any(|(name, content)| {
|
|
std::fs::read_to_string(dir.join(name)).unwrap_or_default() != *content
|
|
});
|
|
|
|
// A previously-embedded CA file no longer wanted → stale (removal is
|
|
// a change even when every desired file already matches on disk).
|
|
if !changed && let Ok(entries) = std::fs::read_dir(dir) {
|
|
changed = entries.flatten().any(|e| {
|
|
e.file_name()
|
|
.to_str()
|
|
.is_some_and(|name| is_embedded_ca_name(name) && !desired_names.contains(name))
|
|
});
|
|
}
|
|
(desired, changed)
|
|
}
|
|
|
|
/// Read an agent's applied `flake.lock` and return the subset of
|
|
/// `CANONICAL_INPUTS` it declares as direct (root-level) inputs.
|
|
/// Returns an empty vec when the lock is missing or unparsable —
|
|
/// safe degradation, the worst case is no dedup for that agent.
|
|
fn agent_canonical_inputs(name: &str) -> Vec<&'static str> {
|
|
let path = crate::paths::applied_dir(name).join("flake.lock");
|
|
let Ok(raw) = std::fs::read_to_string(&path) else {
|
|
return Vec::new();
|
|
};
|
|
let Ok(json) = serde_json::from_str::<serde_json::Value>(&raw) else {
|
|
return Vec::new();
|
|
};
|
|
let Some(nodes) = json.get("nodes").and_then(|v| v.as_object()) else {
|
|
return Vec::new();
|
|
};
|
|
let Some(root_name) = json.get("root").and_then(|v| v.as_str()) else {
|
|
return Vec::new();
|
|
};
|
|
let Some(root_inputs) = nodes
|
|
.get(root_name)
|
|
.and_then(|n| n.get("inputs"))
|
|
.and_then(|v| v.as_object())
|
|
else {
|
|
return Vec::new();
|
|
};
|
|
CANONICAL_INPUTS
|
|
.iter()
|
|
.copied()
|
|
.filter(|canon| root_inputs.contains_key(*canon))
|
|
.collect()
|
|
}
|
|
|
|
/// Inner render helper accepting a lookup fn so tests can stub the
|
|
/// agent flake-lock introspection.
|
|
#[allow(
|
|
clippy::too_many_lines,
|
|
clippy::too_many_arguments,
|
|
reason = "templated string-builder for the meta flake — the length is one \
|
|
contiguous fmt block, splitting it would just hide the shape; the \
|
|
args mirror the host-level config the flake is rendered from"
|
|
)]
|
|
fn render_flake_with_lookup<F>(
|
|
hyperhive_flake: &str,
|
|
docs_flake: &str,
|
|
nixpkgs_flake: &str,
|
|
claude_code_path: Option<&str>,
|
|
dashboard_port: u16,
|
|
operator_pronouns: &str,
|
|
context_window_tokens: &std::collections::HashMap<String, u64>,
|
|
hive_memory_max: &str,
|
|
agents: &[AgentSpec],
|
|
lookup: F,
|
|
) -> String
|
|
where
|
|
F: Fn(&str) -> Vec<&'static str>,
|
|
{
|
|
use std::fmt::Write as _;
|
|
let mut out = String::new();
|
|
out.push_str("{\n description = \"hyperhive deployed agents\";\n inputs = {\n");
|
|
// `nixpkgs` is a top-level meta input with an explicit store-path URL.
|
|
// `hyperhive` then follows it via `hyperhive.inputs.nixpkgs.follows`.
|
|
// This cascades through to every agent because
|
|
// `agent-<n>.inputs.nixpkgs.follows = "nixpkgs"` resolves to the same
|
|
// top-level node.
|
|
//
|
|
// Why an explicit `path:` URL instead of
|
|
// `nixpkgs.follows = "hyperhive/nixpkgs"`:
|
|
// meta points to hyperhive's *store path* as its flake input, so nix
|
|
// reads hyperhive's own pinned lock when evaluating that input — the
|
|
// host-level `follows` the operator set never propagates. Injecting the
|
|
// evaluated `pkgs.path` directly at nix-module evaluation time is the
|
|
// only reliable way to honour the host's channel choice.
|
|
//
|
|
// Fallback (flake arg empty): legacy `follows` wiring — used when
|
|
// hive-c0re is not built with this option wired up.
|
|
if nixpkgs_flake.is_empty() {
|
|
// Legacy path: meta defers to hyperhive's own lock.
|
|
let _ = writeln!(out, " hyperhive.url = \"{hyperhive_flake}\";");
|
|
out.push_str(" nixpkgs.follows = \"hyperhive/nixpkgs\";\n");
|
|
} else {
|
|
let _ = writeln!(out, " nixpkgs.url = \"{nixpkgs_flake}\";");
|
|
let _ = writeln!(out, " hyperhive.url = \"{hyperhive_flake}\";");
|
|
out.push_str(" hyperhive.inputs.nixpkgs.follows = \"nixpkgs\";\n");
|
|
}
|
|
// Narrow `docs/` source as its own input so a doc edit only
|
|
// re-locks THIS input instead of re-hashing the whole `hyperhive`
|
|
// source. Threaded to each agent below as `hyperhive.docs.source`.
|
|
// Empty = hive-c0re not built with the option wired up (legacy);
|
|
// agents then keep the harness default (`hyperhive.packages.reference-docs`).
|
|
if !docs_flake.is_empty() {
|
|
// `flake = false`: the docs/ tree is a plain source (no flake.nix),
|
|
// so nix must treat it as raw source, not evaluate it as a flake.
|
|
let _ = writeln!(out, " hyperhive-docs.url = \"{docs_flake}\";");
|
|
out.push_str(" hyperhive-docs.flake = false;\n");
|
|
}
|
|
// Each agent's config input is its LOCAL applied mirror
|
|
// (`git+file://<applied>`), so the meta flake resolves entirely from
|
|
// on-disk state and boot never depends on the forge being reachable.
|
|
// The forge `agent-configs/<name>` repos stay the review/audit surface
|
|
// (config PRs land there) but are not the flake's build input. deploy +
|
|
// rebuild re-lock this input to `applied/<name>`'s current `main` head;
|
|
// `verify_commit` overrides it to a proposed `?rev=<sha>` for eval
|
|
// before that head moves.
|
|
for spec in agents {
|
|
let _ = writeln!(
|
|
out,
|
|
" agent-{name}.url = \"git+file://{applied}\";",
|
|
name = spec.name,
|
|
applied = crate::paths::applied_dir(&spec.name).display(),
|
|
);
|
|
// For each canonical input the agent declares in its own
|
|
// `flake.nix` (detected by reading its applied `flake.lock`),
|
|
// emit `inputs.agent-<name>.inputs.<canon>.follows = "<canon>"`.
|
|
// Collapses otherwise-separate-but-identical nixpkgs nodes
|
|
// (root + every agent's own nixpkgs) into one. Skipped
|
|
// silently for agents that don't declare the input — emitting
|
|
// follows on a non-existent input would error at
|
|
// `nix flake lock` time.
|
|
for canon in lookup(&spec.name) {
|
|
let _ = writeln!(
|
|
out,
|
|
" agent-{}.inputs.{canon}.follows = \"{canon}\";",
|
|
spec.name,
|
|
);
|
|
}
|
|
}
|
|
out.push_str(" };\n outputs =\n { self, hyperhive, ... }@inputs:\n let\n");
|
|
// Free-text operator string — escape backslash + double-quote so a
|
|
// pronouns value like `he/him \ "rare"` round-trips into a valid
|
|
// nix string literal without breaking the flake.
|
|
let pronouns_escaped = operator_pronouns.replace('\\', "\\\\").replace('"', "\\\"");
|
|
let _ = writeln!(
|
|
out,
|
|
" dashboardPort = {dashboard_port};\n operatorPronouns = \"{pronouns_escaped}\";\n mkAgent = {{ name, isManager, port, parent ? null, toolGroups ? null, capabilities ? null, memoryMaxBytes ? null }}:"
|
|
);
|
|
out.push_str(
|
|
r#" let
|
|
base = if isManager
|
|
then hyperhive.nixosConfigurations.ruth
|
|
else hyperhive.nixosConfigurations.agent-base;
|
|
input = inputs."agent-${name}";
|
|
service = "hive-agent";
|
|
parentEnv = if parent == null then {} else { HIVE_PARENT = parent; };
|
|
toolGroupsEnv = if toolGroups == null then {} else { HIVE_TOOL_GROUPS = toolGroups; };
|
|
capabilitiesEnv = if capabilities == null then {} else { HIVE_CAPABILITIES = capabilities; };
|
|
in
|
|
base.extendModules {
|
|
modules = [
|
|
input.nixosModules.default
|
|
{
|
|
"#,
|
|
);
|
|
// Point the in-container docs dir (`$HIVE_DOCS_DIR`) at the narrow
|
|
// `hyperhive-docs` input instead of the harness default
|
|
// (`hyperhive.packages.reference-docs`, built from the now-docs-stripped source).
|
|
// `inputs."hyperhive-docs"` is reachable via the outputs `@inputs`
|
|
// capture. Emitted only when the input exists (docs_flake non-empty).
|
|
if !docs_flake.is_empty() {
|
|
out.push_str(" hyperhive.docs.source = inputs.\"hyperhive-docs\".outPath;\n");
|
|
}
|
|
// The `claude-code` agents run, as a bare store path rather than a
|
|
// flake input: containers share the host's `/nix/store`, so the
|
|
// binary is already there with its closure and has nothing to
|
|
// travel. A string literal is also the only shape that evaluates —
|
|
// `lib.types.package` on a bare path runs `builtins.storePath`,
|
|
// which pure eval rejects. The agent module puts its `bin/` on the
|
|
// harness PATH; the host module holds the gc root, since a path
|
|
// spelled out here is text and references nothing.
|
|
// `None` = no override; agents keep their own nixpkgs' `claude-code`.
|
|
if let Some(path) = claude_code_path {
|
|
let _ = writeln!(out, " hyperhive.claudeCodePath = \"{path}\";");
|
|
}
|
|
// CA trust: embed the hive's trust anchors so each agent validates
|
|
// them at build time — the hive's own self-signed CA when active,
|
|
// together with the swarm root it is issued under (one file; see
|
|
// `hive_ca_source`). `certificateFiles` is build-time, so the certs
|
|
// travel with the flake source: `sync_agents` writes `./hive-ca.pem`
|
|
// next to flake.nix and stages it. Only public CA certs are embedded;
|
|
// no private key ever leaves the host. The filename list matches
|
|
// `sync_agents` exactly (both derive it from `embedded_ca_files`), so
|
|
// we never reference a file we didn't embed; emitted only when the
|
|
// list is non-empty.
|
|
let ca_refs: Vec<String> = embedded_ca_files()
|
|
.into_iter()
|
|
.map(|(name, _)| format!("./{name}"))
|
|
.collect();
|
|
if !ca_refs.is_empty() {
|
|
let _ = writeln!(
|
|
out,
|
|
" security.pki.certificateFiles = [ {} ];",
|
|
ca_refs.join(" ")
|
|
);
|
|
}
|
|
// Hive-wide OTEL stats export (`services.hyperhive.otel.*`): inject the
|
|
// build-time `hyperhive.otel.*` config the harness modules consume (its
|
|
// otelEnv + otelExecStart wrapper + LoadCredential). Host-driven, so
|
|
// the same config lands on every agent; emitted only when enabled.
|
|
// Mirrors the CA-cert injection above — host state -> build-time agent
|
|
// module config.
|
|
if let Some(otel) = otel_config() {
|
|
let esc = |s: &str| s.replace('\\', "\\\\").replace('"', "\\\"");
|
|
out.push_str(" hyperhive.otel.enable = true;\n");
|
|
let _ = writeln!(
|
|
out,
|
|
" hyperhive.otel.endpoint = \"{}\";",
|
|
esc(&otel.endpoint)
|
|
);
|
|
let _ = writeln!(
|
|
out,
|
|
" hyperhive.otel.protocol = \"{}\";",
|
|
esc(&otel.protocol)
|
|
);
|
|
if let Some(attrs) = &otel.extra_resource_attributes {
|
|
let _ = writeln!(
|
|
out,
|
|
" hyperhive.otel.extraResourceAttributes = \"{}\";",
|
|
esc(attrs)
|
|
);
|
|
}
|
|
if let Some(ms) = otel.metric_interval_ms {
|
|
// Int option — emit a bare numeric literal (no quotes). `ms` is a
|
|
// parsed u64, so it can't inject anything into the rendered nix.
|
|
let _ = writeln!(out, " hyperhive.otel.metricIntervalMs = {ms};");
|
|
}
|
|
if otel.debug {
|
|
out.push_str(" hyperhive.otel.debug = true;\n");
|
|
}
|
|
}
|
|
// Agent-facing service URLs (`hyperhive.forge.url`, `hyperhive.matrix.url`):
|
|
// emit the host's real values as build-time agent config, the same
|
|
// host-state -> agent-module shape as the otel block above.
|
|
//
|
|
// These options exist already, but until now *nothing set them*, so every
|
|
// agent silently fell back to their `localhost:<port>` defaults while the
|
|
// true value reached the container only as an env var (below). Two sources
|
|
// of truth that disagree, with the winner decided by which code path
|
|
// happens to read which one --- the option default is baked into scripts
|
|
// at build time, the env var is read at runtime.
|
|
//
|
|
// Rendering them here makes the option the single source, which is the
|
|
// precondition for removing those defaults: a loopback address is only
|
|
// ever correct when the callee shares the caller's netns, and the forge
|
|
// and homeserver are moving to swarm level, possibly on other hosts.
|
|
//
|
|
// Absent vars emit nothing rather than a guess, and the agent option
|
|
// treats "unset" as "this service is not configured" rather than
|
|
// substituting a loopback address — an absent integration instead of a
|
|
// misdirected one.
|
|
//
|
|
// That is the right default for an optional service and the wrong one for
|
|
// the forge, which a running hive always has — so `sync_agents` rejects a
|
|
// missing `HIVE_FORGE_URL` before it writes anything (`require_service_urls`).
|
|
// The check lives there rather than here because rendering is a pure string
|
|
// operation the tests exercise directly.
|
|
push_forwarded_var_options(&mut out, &forwarded_env_vars());
|
|
// GitHub integration is on by default in every agent
|
|
// (`hyperhive.github.enable`); the host turns it off hive-wide via
|
|
// `services.hyperhive.github.enable = false`, surfaced here as the
|
|
// `HYPERHIVE_GITHUB_DISABLED` env on hive-c0re's unit. Only the OFF
|
|
// override is propagated — the enabled default needs no per-agent line.
|
|
if std::env::var_os("HYPERHIVE_GITHUB_DISABLED").is_some() {
|
|
out.push_str(" hyperhive.github.enable = false;\n");
|
|
}
|
|
out.push_str(
|
|
r#" # The harness service inside the container runs as a
|
|
# non-root unix user named after the agent (`damocles`,
|
|
# `iris`, `root`, …). UID auto-assigned by NixOS; the
|
|
# per-agent override here is what makes
|
|
# `hyperhive.user.name` match the agent's identity
|
|
# instead of the harness default of `"agent"`.
|
|
hyperhive.user.name = name;
|
|
hyperhive.claudeMemoryMaxBytes = memoryMaxBytes;
|
|
programs.git.config.user = {
|
|
name = name;
|
|
email = "${name}@hyperhive.local";
|
|
};
|
|
# Container-wide env: every service + co-process daemon can
|
|
# resolve the agent's durable state dir without hard-coding it.
|
|
# `environment.variables` only writes /etc/environment (login
|
|
# shells); `systemd.globalEnvironment` is the analogue for
|
|
# systemd units so tea-login / forge-avatar-sync /
|
|
# hive-matrix-daemon etc. can read `$HYPERHIVE_STATE_DIR`
|
|
# without each service having to redeclare it.
|
|
environment.variables = {
|
|
HIVE_LABEL = name;
|
|
HYPERHIVE_STATE_DIR = "/agents/${name}/state";
|
|
HYPERHIVE_HARNESS_DIR = "/agents/${name}/harness";
|
|
};
|
|
systemd.globalEnvironment = {
|
|
HIVE_LABEL = name;
|
|
HYPERHIVE_STATE_DIR = "/agents/${name}/state";
|
|
HYPERHIVE_HARNESS_DIR = "/agents/${name}/harness";
|
|
"#,
|
|
);
|
|
// Forwarded vars (HIVE_FORGE_URL etc.) also go into globalEnvironment,
|
|
// not just the harness service env below, so EVERY service + shell in
|
|
// the container inherits them — crucially the bash-task runner (where
|
|
// `hive-forge` + `git` actually run), plus the matrix daemon, tea-login
|
|
// and interactive shells. Scoped to the harness service alone they were
|
|
// invisible to bash tasks: harmless in shared netns (the localhost
|
|
// default works) but broken under isolation, where the in-cluster
|
|
// `forge.<domain>` URL is the only reachable path.
|
|
for (var, val) in forwarded_env_vars() {
|
|
let escaped = val.replace('\\', "\\\\").replace('"', "\\\"");
|
|
let _ = writeln!(out, " {var} = \"{escaped}\";");
|
|
}
|
|
out.push_str(
|
|
r" };
|
|
systemd.services.${service}.environment = parentEnv // toolGroupsEnv // capabilitiesEnv // {
|
|
HIVE_PORT = toString port;
|
|
HIVE_LABEL = name;
|
|
HIVE_DASHBOARD_PORT = toString dashboardPort;
|
|
HIVE_OPERATOR_PRONOUNS = operatorPronouns;",
|
|
);
|
|
// Per-model context-window env vars declared in the host-level
|
|
// `services.hive-c0re.contextWindowTokens` option. Use a sorted
|
|
// iterator for deterministic flake output (no spurious git diffs).
|
|
let mut sorted_tokens: Vec<(&String, &u64)> = context_window_tokens.iter().collect();
|
|
sorted_tokens.sort_by_key(|(k, _)| k.as_str());
|
|
for (key, val) in &sorted_tokens {
|
|
let upper_key = key.to_ascii_uppercase();
|
|
let _ = writeln!(
|
|
out,
|
|
" HIVE_CONTEXT_WINDOW_TOKENS_{upper_key} = \"{val}\";"
|
|
);
|
|
}
|
|
// Forwarded env vars — picked up from hive-c0re's own systemd unit
|
|
// env (`services.hyperhive.*` options flow through nix/modules/
|
|
// hive-c0re.nix into the host process). We copy whatever's set into
|
|
// each sub-agent's harness service env so the in-container surfaces
|
|
// (`identity.rs`, `forge_notify`) see a consistent view across the
|
|
// whole hive. Absent host-side env (option not set) → skip emission
|
|
// → in-container accessors fall back to None / defaults gracefully.
|
|
//
|
|
// - HIVE_FORGE_URL: agents poll this for Forgejo notifications.
|
|
// - HYPERHIVE_HIVE_DOMAIN: machine-readable hive DNS.
|
|
// - HYPERHIVE_HIVE_NAME / HYPERHIVE_SWARM_NAME: human display
|
|
// names for hive + swarm.
|
|
for (var, val) in forwarded_env_vars() {
|
|
let escaped = val.replace('\\', "\\\\").replace('"', "\\\"");
|
|
let _ = writeln!(out, " {var} = \"{escaped}\";");
|
|
}
|
|
out.push_str(
|
|
r#" HYPERHIVE_STATE_DIR = "/agents/${name}/state";
|
|
HYPERHIVE_HARNESS_DIR = "/agents/${name}/harness";
|
|
};
|
|
}
|
|
];
|
|
};
|
|
in
|
|
{
|
|
nixosConfigurations = {
|
|
"#,
|
|
);
|
|
// Pull the topology map once and look up each agent's parent. An
|
|
// empty / absent topology.json yields `parent = null` for everyone
|
|
// (every container at root). `meta::sync_agents` seeds the file
|
|
// on first run with manager as root + everyone else under manager.
|
|
let topology = crate::topology::read();
|
|
let tool_groups_map = crate::tool_groups::read();
|
|
let capabilities_map = crate::capabilities::read();
|
|
let resource_limits_map = crate::resource_limits::read();
|
|
for spec in agents {
|
|
let parent_attr = topology
|
|
.get(&spec.name)
|
|
.and_then(|p| p.as_ref())
|
|
.map_or_else(|| "null".to_owned(), |p| format!("\"{p}\""));
|
|
// Emit `toolGroups = "group1,group2"` when the operator has
|
|
// explicitly configured groups for this agent. Absent entry = null
|
|
// = harness falls back to AGENT_DEFAULT (no env var emitted,
|
|
// no rebuild cascade for agents whose groups haven't changed).
|
|
let groups = tool_groups_map.get(&spec.name).cloned().unwrap_or_default();
|
|
let tool_groups_attr = if groups.is_empty() {
|
|
"null".to_owned()
|
|
} else {
|
|
let joined = groups.join(",");
|
|
format!("\"{joined}\"")
|
|
};
|
|
// Emit `capabilities = "cap1,cap2"` when the operator has
|
|
// granted capabilities to this agent. Absent entry = null = no
|
|
// capability env var injected, capability-gated tools hidden.
|
|
let caps = capabilities_map
|
|
.get(&spec.name)
|
|
.cloned()
|
|
.unwrap_or_default();
|
|
let capabilities_attr = if caps.is_empty() {
|
|
"null".to_owned()
|
|
} else {
|
|
let joined = caps.join(",");
|
|
format!("\"{joined}\"")
|
|
};
|
|
// Effective `MemoryMax=` for this agent (per-agent override, else
|
|
// the hive-wide default), turned into a raw byte count so
|
|
// `claude-settings.nix` can derive a JSC heap ceiling from it
|
|
// (see `hyperhive.claudeMemoryMaxBytes`). `null` when the
|
|
// effective value is `"infinity"` or a RAM percentage — no
|
|
// byte count to derive, dependent env var stays unset, same as
|
|
// today's no-cap behavior.
|
|
let memory_max_attr = crate::resource_limits::effective_memory_bytes_from(
|
|
&resource_limits_map,
|
|
&spec.name,
|
|
hive_memory_max,
|
|
)
|
|
.map_or_else(|| "null".to_owned(), |b| b.to_string());
|
|
let _ = writeln!(
|
|
out,
|
|
" {} = mkAgent {{ name = \"{}\"; isManager = {}; port = {}; parent = {}; toolGroups = {}; capabilities = {}; memoryMaxBytes = {}; }};",
|
|
spec.name,
|
|
spec.name,
|
|
if spec.is_manager { "true" } else { "false" },
|
|
spec.port,
|
|
parent_attr,
|
|
tool_groups_attr,
|
|
capabilities_attr,
|
|
memory_max_attr,
|
|
);
|
|
}
|
|
out.push_str(" };\n };\n}\n");
|
|
out
|
|
}
|
|
|
|
/// Return the list of file names that are currently staged (index differs
|
|
/// from HEAD). On the initial commit (`HEAD` doesn't exist yet) falls back
|
|
/// to `git diff --cached --name-only HEAD` failing gracefully by using
|
|
/// `git status --porcelain` and collecting the `A ` / `M ` prefix lines.
|
|
async fn git_staged_names(dir: &Path) -> Result<Vec<String>> {
|
|
// `--diff-filter=ACM` skips deleted entries — we only care about
|
|
// additions and modifications for message-building purposes.
|
|
let out = lifecycle::git_command()
|
|
.current_dir(dir)
|
|
.args(["diff", "--cached", "--name-only", "--diff-filter=ACM"])
|
|
.output()
|
|
.await
|
|
.with_context(|| format!("git diff --cached in {}", dir.display()))?;
|
|
if out.status.success() {
|
|
let names = String::from_utf8_lossy(&out.stdout)
|
|
.lines()
|
|
.map(str::trim)
|
|
.filter(|l| !l.is_empty())
|
|
.map(ToOwned::to_owned)
|
|
.collect();
|
|
return Ok(names);
|
|
}
|
|
// Fallback for initial commit (no HEAD yet): parse `git status --porcelain`.
|
|
let st = lifecycle::git_command()
|
|
.current_dir(dir)
|
|
.args(["status", "--porcelain"])
|
|
.output()
|
|
.await
|
|
.with_context(|| format!("git status in {}", dir.display()))?;
|
|
let names = String::from_utf8_lossy(&st.stdout)
|
|
.lines()
|
|
.filter(|l| l.starts_with("A ") || l.starts_with("M "))
|
|
.filter_map(|l| l.get(3..))
|
|
.map(ToOwned::to_owned)
|
|
.collect();
|
|
Ok(names)
|
|
}
|
|
|
|
async fn git(dir: &Path, args: &[&str]) -> Result<()> {
|
|
let out = lifecycle::git_command()
|
|
.current_dir(dir)
|
|
.args(args)
|
|
.output()
|
|
.await
|
|
.with_context(|| format!("git {} in {}", args.join(" "), dir.display()))?;
|
|
if !out.status.success() {
|
|
bail!(
|
|
"git {} failed ({}): {}",
|
|
args.join(" "),
|
|
out.status,
|
|
String::from_utf8_lossy(&out.stderr).trim()
|
|
);
|
|
}
|
|
Ok(())
|
|
}
|
|
|
|
async fn git_commit(dir: &Path, message: &str) -> Result<()> {
|
|
git(
|
|
dir,
|
|
&[
|
|
"-c",
|
|
&format!("user.name={GIT_NAME}"),
|
|
"-c",
|
|
&format!("user.email={GIT_EMAIL}"),
|
|
"commit",
|
|
"-m",
|
|
message,
|
|
],
|
|
)
|
|
.await?;
|
|
// Best-effort mirror to the bundled forge. No-op when the forge
|
|
// isn't seeded (no core token on disk); push failures log a warn
|
|
// but don't bubble up — a missing mirror shouldn't fail an
|
|
// otherwise successful deploy.
|
|
if let Err(e) = crate::forge::push_meta(dir).await {
|
|
tracing::warn!(error = ?e, "forge: meta push after commit failed (non-fatal)");
|
|
}
|
|
Ok(())
|
|
}
|
|
|
|
/// Path-limited commit: commits ONLY the given paths, so unrelated
|
|
/// staged content — above all a `prepare_deploy`-staged `flake.lock`
|
|
/// — can never be swept into someone else's commit. Every targeted
|
|
/// meta commit (perm files, topology, lock bumps) goes through this;
|
|
/// only `sync_agents` uses the bare [`git_commit`], because its
|
|
/// staged set *is* its intentional commit set.
|
|
async fn git_commit_paths(dir: &Path, message: &str, paths: &[&str]) -> Result<()> {
|
|
let name = format!("user.name={GIT_NAME}");
|
|
let email = format!("user.email={GIT_EMAIL}");
|
|
let mut args = vec!["-c", &name, "-c", &email, "commit", "-m", message, "--"];
|
|
args.extend_from_slice(paths);
|
|
git(dir, &args).await?;
|
|
if let Err(e) = crate::forge::push_meta(dir).await {
|
|
tracing::warn!(error = ?e, "forge: meta push after commit failed (non-fatal)");
|
|
}
|
|
Ok(())
|
|
}
|
|
|
|
/// True when any of `paths` differs between HEAD and the index or
|
|
/// working tree — the path-scoped replacement for whole-tree
|
|
/// `git_is_clean` / `has_staged_changes` guards, which a concurrently
|
|
/// staged deploy lock would otherwise trip.
|
|
async fn paths_dirty(dir: &Path, paths: &[&str]) -> Result<bool> {
|
|
let mut args = vec!["diff", "--quiet", "HEAD", "--"];
|
|
args.extend_from_slice(paths);
|
|
let out = lifecycle::git_command()
|
|
.current_dir(dir)
|
|
.args(&args)
|
|
.output()
|
|
.await
|
|
.with_context(|| format!("git diff --quiet in {}", dir.display()))?;
|
|
// Exit 0 = no differences; 1 = differences; anything else (e.g.
|
|
// no HEAD yet on a fresh repo) → treat as dirty so the commit
|
|
// path runs and surfaces real errors loudly.
|
|
Ok(!out.status.success())
|
|
}
|
|
|
|
/// Full argv for a `nix` invocation, prefixed with the flakes-enabling
|
|
/// experimental-features flag. `--extra-experimental-features` is
|
|
/// belt-and-suspenders for hosts that haven't set this in nix.conf: the
|
|
/// hyperhive module's deploy guide assumes flakes are already enabled,
|
|
/// but the cost of being defensive is one extra argv each call.
|
|
fn nix_argv<'a>(args: &[&'a str]) -> Vec<&'a str> {
|
|
let mut all = vec!["--extra-experimental-features", "nix-command flakes"];
|
|
all.extend_from_slice(args);
|
|
all
|
|
}
|
|
|
|
/// Run `nix <args>` in `dir` (flakes enabled), capturing combined output.
|
|
/// Shared core of [`nix`] and [`nix_logged`].
|
|
async fn nix_output(dir: &Path, args: &[&str]) -> Result<std::process::Output> {
|
|
Command::new("nix")
|
|
.current_dir(dir)
|
|
.args(nix_argv(args))
|
|
// Kill the nix child if the caller's future is dropped (e.g. the
|
|
// startup-migration timeout around `sync_agents` fires) rather than
|
|
// orphaning it against an unreachable forge. No-op on normal exit.
|
|
.kill_on_drop(true)
|
|
.output()
|
|
.await
|
|
.with_context(|| format!("nix {} in {}", args.join(" "), dir.display()))
|
|
}
|
|
|
|
/// Turn a finished nix [`Output`](std::process::Output) into `Result<()>`,
|
|
/// bailing with the trimmed stderr tail on non-zero exit.
|
|
fn nix_check(args: &[&str], out: &std::process::Output) -> Result<()> {
|
|
if !out.status.success() {
|
|
bail!(
|
|
"nix {} failed ({}): {}",
|
|
args.join(" "),
|
|
out.status,
|
|
String::from_utf8_lossy(&out.stderr).trim()
|
|
);
|
|
}
|
|
Ok(())
|
|
}
|
|
|
|
async fn nix(dir: &Path, args: &[&str]) -> Result<()> {
|
|
let out = nix_output(dir, args).await?;
|
|
nix_check(args, &out)
|
|
}
|
|
|
|
/// Like [`nix`] but records the invocation (cmdline + stdout + stderr +
|
|
/// terminal status) into a `build_logs.sqlite` row so a config-approval
|
|
/// eval/deploy step shows up on the dashboard. This is what makes a
|
|
/// failing eval-verify visible: it runs before any container build, so
|
|
/// without a row a rejected config approval leaves the operator with
|
|
/// zero build logs to look at. Best-effort logging: a missing global
|
|
/// handle or a failed `start()` just skips the row — the command still
|
|
/// runs and its exit status is still enforced.
|
|
async fn nix_logged(
|
|
dir: &Path,
|
|
args: &[&str],
|
|
agent: &str,
|
|
kind: &str,
|
|
node_id: Option<u64>,
|
|
) -> Result<()> {
|
|
let cmdline = format!("nix {}", nix_argv(args).join(" "));
|
|
let logs = crate::build_logs::global();
|
|
let log_id = logs.as_ref().and_then(|h| {
|
|
// `node_id` links the row to the queue node that ran it, so the
|
|
// dashboard can reach this log from the node instead of only from the
|
|
// agent+kind+time listing. Both call paths are queue-only today and
|
|
// pass `Some`; it stays an `Option` because these are `meta`'s public
|
|
// API and a future non-queue caller has no node to name.
|
|
h.start(agent, kind, &cmdline, node_id)
|
|
.map_err(|e| {
|
|
tracing::warn!(error = ?e, %kind, "build_logs: start failed (meta log dropped)");
|
|
})
|
|
.ok()
|
|
});
|
|
|
|
let out = nix_output(dir, args).await?;
|
|
|
|
if let (Some(h), Some(id)) = (&logs, log_id) {
|
|
for line in String::from_utf8_lossy(&out.stdout).lines() {
|
|
h.append_stdout(id, line);
|
|
}
|
|
for line in String::from_utf8_lossy(&out.stderr).lines() {
|
|
h.append_stderr(id, line);
|
|
}
|
|
h.finish(
|
|
id,
|
|
if out.status.success() {
|
|
crate::build_logs::BuildStatus::Ok
|
|
} else {
|
|
crate::build_logs::BuildStatus::Fail
|
|
},
|
|
);
|
|
}
|
|
|
|
nix_check(args, &out)
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
/// The regression the deploy-window bug review surfaced: a
|
|
/// path-limited commit must leave an unrelated staged file (the
|
|
/// prepare_deploy-staged `flake.lock`) untouched, so a later
|
|
/// `abort_deploy` still has something to restore.
|
|
#[tokio::test]
|
|
async fn path_limited_commit_leaves_unrelated_staged_file_alone() {
|
|
let tmp = tempfile::tempdir().expect("tempdir");
|
|
let dir = tmp.path();
|
|
git(dir, &["init", "--initial-branch=main"])
|
|
.await
|
|
.expect("git init");
|
|
std::fs::write(dir.join("tool-groups.json"), "{}").expect("write");
|
|
std::fs::write(dir.join("flake.lock"), "v1").expect("write");
|
|
git(dir, &["add", "-A"]).await.expect("add");
|
|
git_commit(dir, "seed").await.expect("seed commit");
|
|
// A deploy stages a new lock (uncommitted)…
|
|
std::fs::write(dir.join("flake.lock"), "v2-staged-by-deploy").expect("write");
|
|
git(dir, &["add", "flake.lock"]).await.expect("stage lock");
|
|
// …and a perm change commits, path-limited.
|
|
std::fs::write(dir.join("tool-groups.json"), r#"{"alice":[]}"#).expect("write");
|
|
git(dir, &["add", "tool-groups.json"]).await.expect("add");
|
|
git_commit_paths(dir, "set tool-groups for alice", &["tool-groups.json"])
|
|
.await
|
|
.expect("path-limited commit");
|
|
// The perm file is committed; the deploy's staged lock is not.
|
|
assert!(
|
|
!paths_dirty(dir, &["tool-groups.json"])
|
|
.await
|
|
.expect("check"),
|
|
"perm file must be committed"
|
|
);
|
|
assert!(
|
|
paths_dirty(dir, &["flake.lock"]).await.expect("check"),
|
|
"staged deploy lock must survive the perm commit"
|
|
);
|
|
}
|
|
|
|
fn sample_spec(name: &str, is_manager: bool, port: u16) -> AgentSpec {
|
|
AgentSpec {
|
|
name: name.to_owned(),
|
|
is_manager,
|
|
port,
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn agent_input_override_pins_exact_rev() {
|
|
let p = Path::new("/var/lib/hyperhive/agents/iris/applied");
|
|
assert_eq!(
|
|
agent_input_override(p, "abc123def"),
|
|
"git+file:///var/lib/hyperhive/agents/iris/applied?rev=abc123def"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn applied_override_url_targets_local_mirror_main_head() {
|
|
// Deploy + rebuild re-lock against the local applied mirror's `main`
|
|
// head (no `?rev`), never the persistent forge URL — so a rebuild
|
|
// survives forge unreachability and builds the fast-forwarded config.
|
|
let p = Path::new("/var/lib/hyperhive/agents/iris/applied");
|
|
let url = applied_override_url(p);
|
|
assert_eq!(url, "git+file:///var/lib/hyperhive/agents/iris/applied");
|
|
assert!(
|
|
!url.contains("?rev="),
|
|
"must lock main head, not a pinned rev: {url}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn render_flake_uses_explicit_nixpkgs_url_when_provided() {
|
|
let out = render_flake(
|
|
"github:example/hyperhive",
|
|
"path:/nix/store/bbbb-hyperhive-docs-source",
|
|
"path:/nix/store/aaaa-nixpkgs-source",
|
|
None,
|
|
8000,
|
|
"she/her",
|
|
&std::collections::HashMap::new(),
|
|
"4G",
|
|
&[sample_spec("alice", false, 9001)],
|
|
);
|
|
// nixpkgs is a top-level input with an explicit URL; hyperhive
|
|
// follows it.
|
|
assert!(
|
|
out.contains("nixpkgs.url = \"path:/nix/store/aaaa-nixpkgs-source\""),
|
|
"expected explicit nixpkgs.url:\n{out}"
|
|
);
|
|
assert!(
|
|
out.contains("hyperhive.inputs.nixpkgs.follows = \"nixpkgs\""),
|
|
"expected hyperhive.inputs.nixpkgs.follows:\n{out}"
|
|
);
|
|
assert!(
|
|
!out.contains("nixpkgs.follows = \"hyperhive"),
|
|
"old-style follows must not appear when flake args are set:\n{out}"
|
|
);
|
|
// the narrow docs source is its own non-flake input, and each
|
|
// agent's docs dir resolves from it rather than hyperhive.packages.reference-docs.
|
|
assert!(
|
|
out.contains("hyperhive-docs.url = \"path:/nix/store/bbbb-hyperhive-docs-source\""),
|
|
"expected hyperhive-docs input url:\n{out}"
|
|
);
|
|
assert!(
|
|
out.contains("hyperhive-docs.flake = false;"),
|
|
"docs source is not a flake, must be flake = false:\n{out}"
|
|
);
|
|
assert!(
|
|
out.contains("hyperhive.docs.source = inputs.\"hyperhive-docs\".outPath;"),
|
|
"expected per-agent docs source wired to the input:\n{out}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn render_flake_omits_docs_input_when_docs_flake_empty() {
|
|
// Legacy / not-wired-up: empty docs_flake emits no docs input and
|
|
// leaves each agent on the harness default (hyperhive.packages.reference-docs).
|
|
let out = render_flake(
|
|
"github:example/hyperhive",
|
|
"",
|
|
"path:/nix/store/aaaa-nixpkgs-source",
|
|
None,
|
|
8000,
|
|
"she/her",
|
|
&std::collections::HashMap::new(),
|
|
"4G",
|
|
&[sample_spec("alice", false, 9001)],
|
|
);
|
|
assert!(
|
|
!out.contains("hyperhive-docs"),
|
|
"no docs input/source when docs_flake is empty:\n{out}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn render_flake_pins_claude_path_without_adding_an_input() {
|
|
// The host-pinned claude travels as a bare store path assigned to
|
|
// an option — deliberately NOT as a flake input. Containers share
|
|
// the host store, so the build is already reachable; making it an
|
|
// input would re-copy it as a reference-less `-source` and strip
|
|
// the closure the binary actually needs.
|
|
let out = render_flake(
|
|
"github:example/hyperhive",
|
|
"path:/nix/store/bbbb-hyperhive-docs-source",
|
|
"path:/nix/store/aaaa-nixpkgs-source",
|
|
Some("/nix/store/cccc-claude-code-2.1.220"),
|
|
8000,
|
|
"she/her",
|
|
&std::collections::HashMap::new(),
|
|
"4G",
|
|
&[sample_spec("alice", false, 9001)],
|
|
);
|
|
assert!(
|
|
out.contains("hyperhive.claudeCodePath = \"/nix/store/cccc-claude-code-2.1.220\";"),
|
|
"claude path assigned as a plain string literal:\n{out}"
|
|
);
|
|
assert!(
|
|
!out.contains("claude-code-2.1.220\".url"),
|
|
"the pinned claude must not become a flake input:\n{out}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn render_flake_omits_claude_path_when_unset() {
|
|
// `None` = no host-level pin: the option is left undefined so the
|
|
// agent module keeps its own nixpkgs' `claude-code` (and keeps it
|
|
// in `environment.systemPackages`, which is what makes the
|
|
// unpinned case self-contained).
|
|
let out = render_flake(
|
|
"github:example/hyperhive",
|
|
"path:/nix/store/bbbb-hyperhive-docs-source",
|
|
"path:/nix/store/aaaa-nixpkgs-source",
|
|
None,
|
|
8000,
|
|
"she/her",
|
|
&std::collections::HashMap::new(),
|
|
"4G",
|
|
&[sample_spec("alice", false, 9001)],
|
|
);
|
|
assert!(
|
|
!out.contains("claudeCodePath"),
|
|
"no claude assignment when unpinned:\n{out}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn render_flake_falls_back_to_follows_when_nixpkgs_flake_empty() {
|
|
// Empty nixpkgs_flake → legacy follows behaviour (backward compat
|
|
// for any code path that can't inject pkgs.path).
|
|
let out = render_flake(
|
|
"github:example/hyperhive",
|
|
"path:/nix/store/bbbb-hyperhive-docs-source",
|
|
"",
|
|
None,
|
|
8000,
|
|
"she/her",
|
|
&std::collections::HashMap::new(),
|
|
"4G",
|
|
&[sample_spec("alice", false, 9001)],
|
|
);
|
|
assert!(
|
|
out.contains("nixpkgs.follows = \"hyperhive/nixpkgs\""),
|
|
"expected fallback follows:\n{out}"
|
|
);
|
|
assert!(
|
|
!out.contains("nixpkgs.url ="),
|
|
"no explicit url should be emitted in fallback mode:\n{out}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn render_flake_emits_follows_for_agents_declaring_nixpkgs() {
|
|
// Stub lookup: pretend `bitburner` + `dmatrix` declare `nixpkgs`
|
|
// at their root, while `argus` has no canonical inputs at all.
|
|
let lookup = |name: &str| -> Vec<&'static str> {
|
|
match name {
|
|
"bitburner" | "dmatrix" => vec!["nixpkgs"],
|
|
_ => vec![],
|
|
}
|
|
};
|
|
let out = render_flake_with_lookup(
|
|
"github:example/hyperhive",
|
|
"path:/nix/store/bbbb-hyperhive-docs-source",
|
|
"path:/nix/store/aaaa-nixpkgs-source",
|
|
None,
|
|
8000,
|
|
"she/her",
|
|
&std::collections::HashMap::new(),
|
|
"4G",
|
|
&[
|
|
sample_spec("argus", false, 9001),
|
|
sample_spec("bitburner", false, 9002),
|
|
sample_spec("dmatrix", false, 9003),
|
|
],
|
|
lookup,
|
|
);
|
|
// bitburner declares nixpkgs → follows emitted.
|
|
assert!(
|
|
out.contains("agent-bitburner.inputs.nixpkgs.follows = \"nixpkgs\""),
|
|
"missing bitburner nixpkgs follows:\n{out}"
|
|
);
|
|
// dmatrix declares nixpkgs → follows emitted.
|
|
assert!(out.contains("agent-dmatrix.inputs.nixpkgs.follows = \"nixpkgs\""));
|
|
// argus declares neither → no follows emitted for it. Asserting
|
|
// ABSENCE is the important bit: emitting a follows on a
|
|
// non-existent input errors at `nix flake lock` time.
|
|
assert!(
|
|
!out.contains("agent-argus.inputs.nixpkgs"),
|
|
"argus shouldn't have nixpkgs follows:\n{out}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn render_flake_skips_canonical_follows_when_lookup_returns_empty() {
|
|
let out = render_flake_with_lookup(
|
|
"github:example/hyperhive",
|
|
"path:/nix/store/bbbb-hyperhive-docs-source",
|
|
"path:/nix/store/aaaa-nixpkgs-source",
|
|
None,
|
|
8000,
|
|
"she/her",
|
|
&std::collections::HashMap::new(),
|
|
"4G",
|
|
&[sample_spec("alice", false, 9001)],
|
|
|_| Vec::new(),
|
|
);
|
|
// No agent-side follows when the lookup reports nothing
|
|
// declared — protects agents whose flake.lock can't be read
|
|
// (missing / unparsable) from being broken by a follows on a
|
|
// non-existent input.
|
|
assert!(
|
|
!out.contains("agent-alice.inputs."),
|
|
"alice shouldn't have any inputs follows:\n{out}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn render_flake_forwards_env_into_global_environment() {
|
|
// Regression for the isolation breakage where forwarded vars
|
|
// (HIVE_FORGE_URL etc.) landed only on the harness service env,
|
|
// so the bash-task runner / matrix daemon / shells defaulted to
|
|
// localhost and couldn't reach the in-cluster gateway. They must
|
|
// also appear in `systemd.globalEnvironment`, which every unit +
|
|
// shell in the container inherits.
|
|
//
|
|
// SAFETY: single-threaded mutation of a process env var the other
|
|
// tests don't assert the absence of; restored before returning.
|
|
unsafe {
|
|
std::env::set_var("HIVE_FORGE_URL", "http://forge.example.test");
|
|
}
|
|
let out = render_flake(
|
|
"github:example/hyperhive",
|
|
"path:/nix/store/bbbb-hyperhive-docs-source",
|
|
"path:/nix/store/aaaa-nixpkgs-source",
|
|
None,
|
|
8000,
|
|
"she/her",
|
|
&std::collections::HashMap::new(),
|
|
"4G",
|
|
&[sample_spec("alice", false, 9001)],
|
|
);
|
|
unsafe {
|
|
std::env::remove_var("HIVE_FORGE_URL");
|
|
}
|
|
// The var must be emitted inside the globalEnvironment block, i.e.
|
|
// before the per-service harness env block that follows it.
|
|
let global_at = out
|
|
.find("systemd.globalEnvironment = {")
|
|
.expect("globalEnvironment block must exist");
|
|
let service_at = out
|
|
.find("systemd.services.${service}.environment")
|
|
.expect("harness service env block must exist");
|
|
let forge_at = out
|
|
.find("HIVE_FORGE_URL = \"http://forge.example.test\"")
|
|
.expect("HIVE_FORGE_URL must be forwarded into the flake");
|
|
assert!(
|
|
forge_at > global_at && forge_at < service_at,
|
|
"HIVE_FORGE_URL must land inside systemd.globalEnvironment, \
|
|
not only the harness service env:\n{out}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn service_url_options_render_from_forwarded_pairs() {
|
|
// Pure over the pairs, deliberately: the render-level tests have to
|
|
// mutate process env, which makes them race each other under the
|
|
// parallel runner — the first version of this test did exactly that
|
|
// and failed for that reason rather than for a real defect.
|
|
let mut out = String::new();
|
|
push_forwarded_var_options(
|
|
&mut out,
|
|
&[
|
|
("HIVE_FORGE_URL", "http://forge.example.test".to_string()),
|
|
("HIVE_MATRIX_URL", "http://matrix.example.test".to_string()),
|
|
// Mapped too, and that is the fix this test now guards: the
|
|
// display names used to be forwarded as runtime env *only*,
|
|
// so the build-time reader fell back to "unknown" and every
|
|
// agent shipped that label on every metric.
|
|
("HYPERHIVE_HIVE_NAME", "pr1ma".to_string()),
|
|
// Forwarded but genuinely unmapped — proves the map is a
|
|
// filter, not a pass-through.
|
|
("HYPERHIVE_HIVE_DOMAIN", "pr1ma.darkest.space".to_string()),
|
|
],
|
|
);
|
|
assert_eq!(
|
|
out,
|
|
" hyperhive.forge.url = \"http://forge.example.test\";\n\
|
|
\x20 hyperhive.matrix.url = \"http://matrix.example.test\";\n\
|
|
\x20 hyperhive.hiveName = \"pr1ma\";\n",
|
|
"expected exactly the mapped options, in input order, indented for the module block"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn require_service_urls_accepts_a_rendered_forge_url() {
|
|
require_service_urls(&[
|
|
("HIVE_FORGE_URL", "http://forge.example.test".to_string()),
|
|
("HYPERHIVE_HIVE_NAME", "pr1ma".to_string()),
|
|
])
|
|
.expect("a forwarded forge URL satisfies the requirement");
|
|
}
|
|
|
|
#[test]
|
|
fn require_service_urls_refuses_to_write_without_a_forge() {
|
|
// The agent option is nullable, so nothing downstream would complain:
|
|
// every agent would simply come up with no forge login and no way to
|
|
// tell that was unintended. The hive asserts its own requirement
|
|
// because it is the only layer that knows a forge is mandatory.
|
|
let err = require_service_urls(&[("HYPERHIVE_HIVE_NAME", "pr1ma".to_string())])
|
|
.expect_err("a missing forge URL must stop the write");
|
|
assert!(
|
|
err.to_string().contains("HIVE_FORGE_URL is unset"),
|
|
"the error must name the missing variable: {err}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn service_url_options_emit_nothing_when_absent() {
|
|
// No guess when the host doesn't say — the agent option stays `null`
|
|
// ("not configured") and the units that would use it aren't generated,
|
|
// so absence is an absent integration rather than a misdirected one.
|
|
// Required services don't reach here: `require_service_urls` rejects
|
|
// them first.
|
|
// `HYPERHIVE_HIVE_DOMAIN` is forwarded as env but carries no option —
|
|
// deliberately picked over a mapped var, since the whole point is a
|
|
// forwarded var the option map does not know about.
|
|
let mut out = String::new();
|
|
push_forwarded_var_options(
|
|
&mut out,
|
|
&[("HYPERHIVE_HIVE_DOMAIN", "pr1ma.darkest.space".to_string())],
|
|
);
|
|
assert!(
|
|
out.is_empty(),
|
|
"no option should be emitted for an unmapped forwarded var, got:\n{out}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn service_url_options_escape_quotes() {
|
|
// The value lands inside a nix string literal; an unescaped quote
|
|
// would end the string and change the surrounding config rather than
|
|
// merely corrupting one value.
|
|
let mut out = String::new();
|
|
push_forwarded_var_options(
|
|
&mut out,
|
|
&[("HIVE_FORGE_URL", "http://x/\"; evil = \"".to_string())],
|
|
);
|
|
assert!(
|
|
out.contains("\\\"") && !out.contains("/\";"),
|
|
"quotes in the value must be escaped:\n{out}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
#[ignore = "mutates process env; races the other render_flake env tests under the parallel runner"]
|
|
fn render_flake_sets_service_url_options_from_forwarded_env() {
|
|
// The forwarded env vars must ALSO become option assignments, because
|
|
// the two are consumed at different times: the option is baked into
|
|
// scripts at build time (tea-login's FORGE_URL), the env var is read at
|
|
// runtime. Emitting only the env var leaves the option on its default,
|
|
// which is how a hive ends up with two disagreeing answers for the same
|
|
// URL.
|
|
//
|
|
// Asserts PLACEMENT, not just presence: an option line rendered outside
|
|
// the per-agent module block would appear in the file and change
|
|
// nothing. It has to land before the environment blocks that close the
|
|
// module out.
|
|
//
|
|
// SAFETY: single-threaded mutation of process env vars, restored
|
|
// before returning.
|
|
unsafe {
|
|
std::env::set_var("HIVE_FORGE_URL", "http://forge.example.test");
|
|
std::env::set_var("HIVE_MATRIX_URL", "http://matrix.example.test");
|
|
}
|
|
let out = render_flake(
|
|
"github:example/hyperhive",
|
|
"path:/nix/store/bbbb-hyperhive-docs-source",
|
|
"path:/nix/store/aaaa-nixpkgs-source",
|
|
None,
|
|
8000,
|
|
"she/her",
|
|
&std::collections::HashMap::new(),
|
|
"4G",
|
|
&[sample_spec("alice", false, 9001)],
|
|
);
|
|
unsafe {
|
|
std::env::remove_var("HIVE_FORGE_URL");
|
|
std::env::remove_var("HIVE_MATRIX_URL");
|
|
}
|
|
let forge_opt_at = out
|
|
.find("hyperhive.forge.url = \"http://forge.example.test\"")
|
|
.expect("hyperhive.forge.url must be rendered from HIVE_FORGE_URL");
|
|
let matrix_opt_at = out
|
|
.find("hyperhive.matrix.url = \"http://matrix.example.test\"")
|
|
.expect("hyperhive.matrix.url must be rendered from HIVE_MATRIX_URL");
|
|
let env_block_at = out
|
|
.find("environment.variables = {")
|
|
.expect("per-agent environment.variables block must exist");
|
|
assert!(
|
|
forge_opt_at < env_block_at && matrix_opt_at < env_block_at,
|
|
"service URL options must land inside the per-agent module block:\n{out}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn render_flake_agent_input_points_at_local_applied_mirror() {
|
|
// The agent config input references the LOCAL applied mirror
|
|
// (`git+file://<applied>`), NOT the forge — so the meta flake
|
|
// resolves entirely from on-disk state and boot never depends on the
|
|
// forge being reachable. The forge `agent-configs/<n>` repos stay the
|
|
// review surface, but they are not the flake's build input.
|
|
let out = render_flake(
|
|
"github:example/hyperhive",
|
|
"path:/nix/store/bbbb-hyperhive-docs-source",
|
|
"path:/nix/store/aaaa-nixpkgs-source",
|
|
None,
|
|
8000,
|
|
"she/her",
|
|
&std::collections::HashMap::new(),
|
|
"4G",
|
|
&[sample_spec("alice", false, 9001)],
|
|
);
|
|
let want = format!(
|
|
"agent-alice.url = \"git+file://{}\"",
|
|
crate::paths::applied_dir("alice").display()
|
|
);
|
|
assert!(
|
|
out.contains(&want),
|
|
"expected the agent input to point at the local applied mirror ({want}):\n{out}"
|
|
);
|
|
assert!(
|
|
!out.contains("git+http"),
|
|
"no forge git+http URL must remain in the rendered flake:\n{out}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn render_flake_embeds_hive_ca_when_signalled() {
|
|
// When hive-tls.nix signals a self-signed hive CA via
|
|
// HIVE_TLS_CA_PATH (and the cert exists), the agent module must
|
|
// trust it at build time via security.pki.certificateFiles. Absent
|
|
// the signal, no reference is emitted (so the flake doesn't point at
|
|
// a file that was never embedded).
|
|
//
|
|
// SAFETY: single-threaded mutation of a process env var the other
|
|
// tests don't assert the absence of; restored before returning.
|
|
let ca_file = std::env::temp_dir().join(format!("hive-ca-test-{}.pem", std::process::id()));
|
|
std::fs::write(
|
|
&ca_file,
|
|
"-----BEGIN CERTIFICATE-----\ntest\n-----END CERTIFICATE-----\n",
|
|
)
|
|
.expect("write temp CA");
|
|
|
|
let render = || {
|
|
render_flake(
|
|
"github:example/hyperhive",
|
|
"path:/nix/store/bbbb-hyperhive-docs-source",
|
|
"path:/nix/store/aaaa-nixpkgs-source",
|
|
None,
|
|
8000,
|
|
"she/her",
|
|
&std::collections::HashMap::new(),
|
|
"4G",
|
|
&[sample_spec("alice", false, 9001)],
|
|
)
|
|
};
|
|
|
|
// A leftover peer-CA file + the env var that used to name it. Both
|
|
// must now be inert: the swarm root rides in the hive's own trust
|
|
// bundle, so nothing per-peer is embedded any more.
|
|
let stale_peer =
|
|
std::env::temp_dir().join(format!("peer-ca0-test-{}.pem", std::process::id()));
|
|
std::fs::write(
|
|
&stale_peer,
|
|
"-----BEGIN CERTIFICATE-----\np0\n-----END CERTIFICATE-----\n",
|
|
)
|
|
.expect("write stale peer CA");
|
|
|
|
// All env mutations are serialised within this one test (no other
|
|
// test asserts on these vars), restored before returning.
|
|
unsafe {
|
|
std::env::remove_var("HIVE_PEER_CA_PATHS");
|
|
std::env::set_var("HIVE_TLS_CA_PATH", &ca_file);
|
|
}
|
|
let with_ca = render();
|
|
unsafe {
|
|
std::env::set_var("HIVE_PEER_CA_PATHS", stale_peer.display().to_string());
|
|
}
|
|
let with_stale_peer_env = render();
|
|
// The stale var alone, with no hive CA: must produce nothing.
|
|
unsafe {
|
|
std::env::remove_var("HIVE_TLS_CA_PATH");
|
|
}
|
|
let stale_peer_env_only = render();
|
|
unsafe {
|
|
std::env::remove_var("HIVE_PEER_CA_PATHS");
|
|
}
|
|
let without_ca = render();
|
|
let _ = std::fs::remove_file(&ca_file);
|
|
let _ = std::fs::remove_file(&stale_peer);
|
|
|
|
assert!(
|
|
with_ca.contains("security.pki.certificateFiles = [ ./hive-ca.pem ]"),
|
|
"CA cert must be wired into certificateFiles when signalled:\n{with_ca}"
|
|
);
|
|
// The regression guard, scoped to the CA list rather than the whole
|
|
// render. Comparing the two flakes wholesale looks stronger and is
|
|
// actually FLAKY: `cargo test` runs these in parallel threads and
|
|
// sibling tests mutate process env (OTEL, forge URLs) between the
|
|
// two `render()` calls, so a whole-output equality assertion fails
|
|
// on changes that have nothing to do with this test.
|
|
assert!(
|
|
with_stale_peer_env.contains("security.pki.certificateFiles = [ ./hive-ca.pem ]"),
|
|
"HIVE_PEER_CA_PATHS must not change the CA list — per-peer CA \
|
|
embedding was replaced by the swarm root inside the hive's own \
|
|
trust bundle:\n{with_stale_peer_env}"
|
|
);
|
|
assert!(
|
|
!with_stale_peer_env.contains("peer-ca-"),
|
|
"no peer-ca-<N>.pem may be embedded, however HIVE_PEER_CA_PATHS \
|
|
is set:\n{with_stale_peer_env}"
|
|
);
|
|
assert!(
|
|
!stale_peer_env_only.contains("security.pki.certificateFiles"),
|
|
"a stale HIVE_PEER_CA_PATHS must not resurrect a certificateFiles \
|
|
reference on its own:\n{stale_peer_env_only}"
|
|
);
|
|
assert!(
|
|
!without_ca.contains("security.pki.certificateFiles"),
|
|
"no certificateFiles reference without any CA signal:\n{without_ca}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn render_flake_injects_otel_when_signalled() {
|
|
// services.hyperhive.otel.* -> HYPERHIVE_OTEL_* on hive-c0re's unit
|
|
// -> injected as build-time hyperhive.otel.* into every agent. With
|
|
// no endpoint signal, no hyperhive.otel lines are emitted (agents
|
|
// keep the the harness modules disabled default).
|
|
//
|
|
// Serialised against every other env-mutating test in the crate.
|
|
// `stats::otel_metrics::tests` perturbs HYPERHIVE_OTEL_ENDPOINT too,
|
|
// and lands in this same test binary — "no other test asserts on
|
|
// these" was true of this module and false of the process.
|
|
let _env = crate::test_env::lock();
|
|
// SAFETY: serialised by the guard above; restored before returning.
|
|
let render = || {
|
|
render_flake(
|
|
"github:example/hyperhive",
|
|
"path:/nix/store/bbbb-hyperhive-docs-source",
|
|
"path:/nix/store/aaaa-nixpkgs-source",
|
|
None,
|
|
8000,
|
|
"she/her",
|
|
&std::collections::HashMap::new(),
|
|
"4G",
|
|
&[sample_spec("alice", false, 9001)],
|
|
)
|
|
};
|
|
unsafe {
|
|
std::env::remove_var("HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES");
|
|
std::env::set_var("HYPERHIVE_OTEL_ENDPOINT", "https://c.example/otel");
|
|
std::env::set_var("HYPERHIVE_OTEL_PROTOCOL", "grpc");
|
|
}
|
|
let on_minimal = render();
|
|
unsafe {
|
|
std::env::set_var(
|
|
"HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES",
|
|
"deployment.environment=prod",
|
|
);
|
|
}
|
|
let on_full = render();
|
|
unsafe {
|
|
std::env::remove_var("HYPERHIVE_OTEL_ENDPOINT");
|
|
std::env::remove_var("HYPERHIVE_OTEL_PROTOCOL");
|
|
std::env::remove_var("HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES");
|
|
}
|
|
let off = render();
|
|
|
|
assert!(
|
|
on_minimal.contains("hyperhive.otel.enable = true;"),
|
|
"otel enable must be injected:\n{on_minimal}"
|
|
);
|
|
assert!(
|
|
on_minimal.contains("hyperhive.otel.endpoint = \"https://c.example/otel\";"),
|
|
"otel endpoint must be injected:\n{on_minimal}"
|
|
);
|
|
assert!(
|
|
on_minimal.contains("hyperhive.otel.protocol = \"grpc\";"),
|
|
"otel protocol must be injected:\n{on_minimal}"
|
|
);
|
|
// Optional fields absent when unset.
|
|
assert!(
|
|
!on_minimal.contains("hyperhive.otel.extraResourceAttributes"),
|
|
"extraResourceAttributes must not appear when unset:\n{on_minimal}"
|
|
);
|
|
|
|
assert!(
|
|
on_full.contains(
|
|
"hyperhive.otel.extraResourceAttributes = \"deployment.environment=prod\";"
|
|
),
|
|
"extraResourceAttributes must be injected when set:\n{on_full}"
|
|
);
|
|
|
|
assert!(
|
|
!off.contains("hyperhive.otel"),
|
|
"no otel lines when disabled:\n{off}"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn render_flake_injects_github_disable_only_when_signalled() {
|
|
// services.hyperhive.github.enable = false -> HYPERHIVE_GITHUB_DISABLED
|
|
// on hive-c0re's unit -> `hyperhive.github.enable = false` injected into
|
|
// every agent. On by default, so nothing is emitted unless disabled.
|
|
//
|
|
// SAFETY: single-threaded mutation of an env var no other test asserts
|
|
// on; restored before returning.
|
|
let render = || {
|
|
render_flake(
|
|
"github:example/hyperhive",
|
|
"path:/nix/store/bbbb-hyperhive-docs-source",
|
|
"path:/nix/store/aaaa-nixpkgs-source",
|
|
None,
|
|
8000,
|
|
"she/her",
|
|
&std::collections::HashMap::new(),
|
|
"4G",
|
|
&[sample_spec("alice", false, 9001)],
|
|
)
|
|
};
|
|
unsafe {
|
|
std::env::remove_var("HYPERHIVE_GITHUB_DISABLED");
|
|
}
|
|
let on_default = render();
|
|
unsafe {
|
|
std::env::set_var("HYPERHIVE_GITHUB_DISABLED", "1");
|
|
}
|
|
let disabled = render();
|
|
unsafe {
|
|
std::env::remove_var("HYPERHIVE_GITHUB_DISABLED");
|
|
}
|
|
assert!(
|
|
!on_default.contains("hyperhive.github.enable"),
|
|
"github.enable must not be emitted by default (agents keep the true default):\n{on_default}"
|
|
);
|
|
assert!(
|
|
disabled.contains("hyperhive.github.enable = false;"),
|
|
"github.enable = false must be injected when the host disables it:\n{disabled}"
|
|
);
|
|
}
|
|
|
|
/// The JSC-heap-ceiling fix needs the effective per-agent memory cap
|
|
/// threaded into the flake as raw bytes, since nspawn hides the real
|
|
/// cgroup cap from inside the container. An
|
|
/// agent with no `resource-limits.json` override falls back to the
|
|
/// hive-wide default passed to `render_flake` (`"4G"` in every test
|
|
/// in this module) — this locks in the byte-count conversion end to
|
|
/// end through the actual render path (not just `parse_bytes`
|
|
/// in isolation).
|
|
#[test]
|
|
fn render_flake_derives_memory_max_bytes_from_hive_default() {
|
|
let out = render_flake(
|
|
"github:example/hyperhive",
|
|
"path:/nix/store/bbbb-hyperhive-docs-source",
|
|
"path:/nix/store/aaaa-nixpkgs-source",
|
|
None,
|
|
8000,
|
|
"she/her",
|
|
&std::collections::HashMap::new(),
|
|
"4G",
|
|
&[sample_spec("alice", false, 9001)],
|
|
);
|
|
let want_bytes = 4u64 * 1024 * 1024 * 1024;
|
|
assert!(
|
|
out.contains(&format!("memoryMaxBytes = {want_bytes};")),
|
|
"memoryMaxBytes must reflect the hive-wide default in bytes:\n{out}"
|
|
);
|
|
}
|
|
}
|