vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

flake: lift cargo test out of buildPackage so prompt edits don't bust the binary cache (#555)

Browse source 
Per mara's review on PR #561: the previous commit kept
`./hive-ag3nt/prompts` in `cleanSrc` because
`hive-ag3nt::prompt::tests` had a compile-time
`include_str!("../prompts/system.md")`. That meant a prompt edit
still busted the cargo cache.

This change:

- Replaces the test-side `include_str!` with a runtime read from
  `$HIVE_ASSETS_DIR/prompts/system.md` (with a CARGO_MANIFEST_DIR
  fallback for plain `cargo test` from a checked-out repo).
- Drops `./hive-ag3nt/prompts` from `cleanSrc` — it's now
  `craneLib.cleanCargoSource ./.` (Cargo.* + *.rs only).
- Sets `doCheck = false` on `packages.default` and lifts
  `cargo test` into a separate `checks.cargo-test` derivation
  that carries the `hyperhive-assets` build input. That scopes the
  asset rebuild blast radius to the test check — `nix flake check`
  still exercises the suite, but the binary derivation no longer
  carries the assets dep.

Verified cache-invariance matrix (via `echo '' >> <f>; nix eval
.#default.outPath`):

| edit                    | default | cargo-test | clippy |
|-------------------------|---------|------------|--------|
| README.md               | stable  | stable     | stable |
| branding/hyperhive.svg  | stable  | CHANGED    | stable |
| nix/modules/*           | stable  | stable     | stable |
| prompts/system.md       | stable  | CHANGED    | stable |
| hive-c0re/src/main.rs   | CHANGED | CHANGED    | CHANGED |

(`cargo-test` CHANGED on prompts/branding is correct — tests
read the production template + need the assets output.)
This commit is contained in:
iris 2026-05-29 10:55:55 +02:00 • committed by Mara
commit b3f675e367
2 changed files with 93 additions and 42 deletions
Download patch file Download diff file Expand all files Collapse all files

72
flake.nix
View file

@ -47,24 +47,23 @@
treefmt-eval = treefmt-nix.lib.evalModule pkgs treefmt-config; treefmt-eval = treefmt-nix.lib.evalModule pkgs treefmt-config;
craneLib = crane.mkLib pkgs; craneLib = crane.mkLib pkgs;
# Narrowed source tree the rust derivations consume. # Narrowed source tree the rust derivations consume.
# `commonCargoSources` is crane's standard "everything cargo # `cleanCargoSource` is crane's standard "everything cargo
# cares about" filter (Cargo.toml/Cargo.lock + *.rs); we # cares about" filter (Cargo.toml/Cargo.lock + *.rs). All
# union it with the one non-rust path the workspace still # non-rust runtime assets — branding + the claude system
# references — `hive-ag3nt/prompts/` — which is read at # prompt template + claude-settings.json — live in the
# compile time by `hive-ag3nt::prompt::tests` via # separate `hyperhive-assets` derivation (#555) and are
# `include_str!`. Branding assets + claude prompts at # loaded by the binaries at runtime from `$HIVE_ASSETS_DIR`.
# runtime live in the `hyperhive-assets` derivation # The unit tests in `hive-ag3nt::prompt` read the same
# (#555), so a tweak to e.g. `branding/hyperhive.svg`, # `prompts/system.md` directly from the workspace tree at
# `README.md`, the `nix/` modules, or the frontend tree # *test* runtime (via `env!("CARGO_MANIFEST_DIR")` — a
# does NOT bust this src hash and the rust derivations # compile-time string, no file open at compile), so the
# stay cached. # prompt template doesn't have to be in this fileset to
cleanSrc = lib.fileset.toSource { # keep `cargo test` honest. Net effect: tweaks to any
root = ./.; # non-`*.rs` / non-`Cargo.*` file (README, branding,
fileset = lib.fileset.unions [ # nix modules, frontend tree, OR `hive-ag3nt/prompts/*`)
(craneLib.fileset.commonCargoSources ./.) # do NOT bust this src hash, so the rust derivations
./hive-ag3nt/prompts # stay fully cached.
]; cleanSrc = craneLib.cleanCargoSource ./.;
};
# Build the workspace's dependency tree once, cached as # Build the workspace's dependency tree once, cached as
# its own derivation. `buildPackage` and `cargoClippy` # its own derivation. `buildPackage` and `cargoClippy`
# both reuse this via `inherit cargoArtifacts;` so a # both reuse this via `inherit cargoArtifacts;` so a
@ -111,12 +110,24 @@
... ...
}: }:
{ {
# Build the workspace binaries without running tests. Tests
# are run as a separate check (`checks.cargo-test`) that
# carries the `hyperhive-assets` build input — `hive-ag3nt::
# prompt::tests` reads the production prompt template at test
# runtime through `$HIVE_ASSETS_DIR`, so wiring the env var
# into the build phase here would make the prompt's hash a
# build input of `default` (defeats #555's cache goal: a
# prompt edit would still bust the binary derivation, even
# though no .rs file changed). Keeping tests in a separate
# check derivation localises the asset-rebuild blast radius
# to that one check — `nix flake check` still exercises them.
default = craneLib.buildPackage { default = craneLib.buildPackage {
src = cleanSrc; src = cleanSrc;
inherit cargoArtifacts nativeBuildInputs; inherit cargoArtifacts nativeBuildInputs;
pname = "hyperhive-workspace"; pname = "hyperhive-workspace";
version = "0.1.0"; version = "0.1.0";
meta.description = "hyperhive workspace (hive-c0re, hive-ag3nt, hive-m1nd)"; meta.description = "hyperhive workspace (hive-c0re, hive-ag3nt, hive-m1nd)";
doCheck = false;
}; };
# Bundled browser assets — see ./nix/frontend.nix. Output is # Bundled browser assets — see ./nix/frontend.nix. Output is
# $out/{dashboard,agent}/ which the Rust binaries serve via # $out/{dashboard,agent}/ which the Rust binaries serve via
@ -125,10 +136,12 @@
branding-svg = ./branding/hyperhive.svg; branding-svg = ./branding/hyperhive.svg;
}; };
# Static runtime assets the rust binaries read via # Static runtime assets the rust binaries read via
# `assets::path()` (#555): branding/* + hive-ag3nt/prompts/*, # `hive_sh4re::assets::*` (#555): branding/* + prompts/*,
# plus the rendered agent-configs.png. Split out of the # plus the rendered agent-configs.png. Split out of the
# rust derivation so a tweak to e.g. system.md doesn't bust # rust derivation so a tweak to e.g. system.md doesn't bust
# the cargo cache. # the cargo cache. Build input of the `cargo-test` check but
# NOT of `packages.default`, so the binary derivation stays
# cached when a prompt edit ripples through.
assets = pkgs.callPackage ./nix/assets.nix { }; assets = pkgs.callPackage ./nix/assets.nix { };
# Pre-built per-container system closures. Exposed as packages # Pre-built per-container system closures. Exposed as packages
# so operators can `nix build .#agent-base-toplevel` (or wire # so operators can `nix build .#agent-base-toplevel` (or wire
@ -266,6 +279,8 @@
checks = forAllSystems ( checks = forAllSystems (
{ {
pkgs,
system,
treefmt-eval, treefmt-eval,
craneLib, craneLib,
cleanSrc, cleanSrc,
@ -290,6 +305,23 @@
version = "0.1.0"; version = "0.1.0";
cargoClippyExtraArgs = "--workspace --all-targets -- -D warnings"; cargoClippyExtraArgs = "--workspace --all-targets -- -D warnings";
}; };
# `cargo test --workspace` lifted out of `buildPackage` so the
# `hyperhive-assets` dep (which `hive-ag3nt::prompt::tests`
# needs via `HIVE_ASSETS_DIR` to assert against the actual
# production prompt template) is scoped to this one check
# instead of bleeding into the binary derivation's input
# hash. Net: editing `hive-ag3nt/prompts/system.md` still
# rebuilds this test check (correct — the tests assert
# against its wording), but `packages.default` and the
# per-container toplevels stay fully cached.
cargo-test = craneLib.cargoTest {
src = cleanSrc;
inherit cargoArtifacts nativeBuildInputs;
pname = "hyperhive-workspace";
version = "0.1.0";
cargoTestExtraArgs = "--workspace";
HIVE_ASSETS_DIR = "${self.packages.${system}.assets}/share/hyperhive";
};
} }
); );
}; };

63
hive-ag3nt/src/prompt.rs
View file

@ -117,18 +117,19 @@ fn parse_close_marker(line: &str) -> Option<&str> {
/// # Errors /// # Errors
/// ///
/// Returns an error if the system prompt file cannot be written. /// Returns an error if the system prompt file cannot be written.
pub async fn write_system_prompt( pub async fn write_system_prompt(socket: &Path, label: &str, flavor: Flavor) -> Result<PathBuf> {
socket: &Path,
label: &str,
flavor: Flavor,
) -> Result<PathBuf> {
let parent = socket.parent().unwrap_or_else(|| Path::new("/run/hive")); let parent = socket.parent().unwrap_or_else(|| Path::new("/run/hive"));
tokio::fs::create_dir_all(parent).await.ok(); tokio::fs::create_dir_all(parent).await.ok();
let pronouns = std::env::var("HIVE_OPERATOR_PRONOUNS").unwrap_or_else(|_| "she/her".to_owned()); let pronouns = std::env::var("HIVE_OPERATOR_PRONOUNS").unwrap_or_else(|_| "she/her".to_owned());
let template_path = hive_sh4re::assets::prompt_template(); let template_path = hive_sh4re::assets::prompt_template();
let template = tokio::fs::read_to_string(&template_path) let template = tokio::fs::read_to_string(&template_path)
.await .await
.with_context(|| format!("read claude system prompt template from {}", template_path.display()))?; .with_context(|| {
format!(
"read claude system prompt template from {}",
template_path.display()
)
})?;
let body = render(&template, flavor, label, &pronouns); let body = render(&template, flavor, label, &pronouns);
let path = parent.join("claude-system-prompt.md"); let path = parent.join("claude-system-prompt.md");
tokio::fs::write(&path, body).await?; tokio::fs::write(&path, body).await?;
@ -139,18 +140,36 @@ pub async fn write_system_prompt(
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use std::sync::LazyLock;
// #555: production reads the system-prompt template from // #555: the production template lives at
// `$HIVE_ASSETS_DIR/prompts/system.md` at startup. The test module // `$HIVE_ASSETS_DIR/prompts/system.md` and is loaded at runtime.
// still `include_str!`s it directly because: // The unit tests below want to assert against the actual production
// 1. the "real template still substitutes / still filters" tests // wording (so the renderer + tool surface stay honest), so they
// below need the actual production wording to be honest; // resolve the same path at test runtime via two fallbacks:
// 2. embedding it at compile time keeps `cargo test --workspace` // 1. `$HIVE_ASSETS_DIR/prompts/system.md` — the runtime contract
// runnable without setting `HIVE_ASSETS_DIR`; // production uses. The flake's `checks.cargo-test` derivation
// 3. this is the ONLY remaining compile-time reference to // sets this to the `hyperhive-assets` output so `cargo test`
// `prompts/system.md` from the rust workspace — production // inside the nix sandbox finds the file without needing
// code loads it at runtime. // `prompts/` in the cargo source tree. `packages.default`
const PRODUCTION_TEMPLATE: &str = include_str!("../prompts/system.md"); // explicitly does NOT carry the assets dep, so a prompt edit
// doesn't bust the binary derivation — only this test check.
// 2. `env!("CARGO_MANIFEST_DIR")/prompts/system.md` — for plain
// `cargo test --workspace` from a checked-out repo where the
// env var isn't set; `env!` is a compile-time string lookup,
// no file open at compile, so this still doesn't pull
// `prompts/` into the build hash.
// The combined effect is that the flake's `cleanSrc` no longer
// unions `./hive-ag3nt/prompts` — tweaks to system.md don't bust
// the cargo cache anymore.
static PRODUCTION_TEMPLATE: LazyLock<String> = LazyLock::new(|| {
let path = match std::env::var("HIVE_ASSETS_DIR") {
Ok(v) if !v.is_empty() => format!("{v}/prompts/system.md"),
_ => concat!(env!("CARGO_MANIFEST_DIR"), "/prompts/system.md").to_owned(),
};
std::fs::read_to_string(&path)
.unwrap_or_else(|e| panic!("read production prompt template at {path}: {e}"))
});
const SAMPLE: &str = "\ const SAMPLE: &str = "\
shared opener shared opener
@ -251,7 +270,7 @@ shared closer
// Real template's first agent line — keeps the renderer // Real template's first agent line — keeps the renderer
// honest about the {label} / {operator_pronouns} pair the // honest about the {label} / {operator_pronouns} pair the
// harness already relied on. // harness already relied on.
let rendered = render(PRODUCTION_TEMPLATE, Flavor::Agent, "alice", "they/them"); let rendered = render(&PRODUCTION_TEMPLATE, Flavor::Agent, "alice", "they/them");
assert!(rendered.contains("hyperhive agent `alice`")); assert!(rendered.contains("hyperhive agent `alice`"));
assert!(rendered.contains("**they/them** pronouns")); assert!(rendered.contains("**they/them** pronouns"));
assert!(!rendered.contains("{label}")); assert!(!rendered.contains("{label}"));
@ -264,7 +283,7 @@ shared closer
// kill, schedule_*) MUST NOT appear in the agent's rendered // kill, schedule_*) MUST NOT appear in the agent's rendered
// prompt. Drift between flavor and tool surface bites every // prompt. Drift between flavor and tool surface bites every
// time it happens (cf. #511 missing-allow-list bug). // time it happens (cf. #511 missing-allow-list bug).
let rendered = render(PRODUCTION_TEMPLATE, Flavor::Agent, "alice", "she/her"); let rendered = render(&PRODUCTION_TEMPLATE, Flavor::Agent, "alice", "she/her");
assert!(!rendered.contains("request_init_config")); assert!(!rendered.contains("request_init_config"));
assert!(!rendered.contains("request_apply_commit")); assert!(!rendered.contains("request_apply_commit"));
assert!(!rendered.contains("get_logs")); assert!(!rendered.contains("get_logs"));
@ -275,7 +294,7 @@ shared closer
#[test] #[test]
fn render_manager_includes_manager_only_tools() { fn render_manager_includes_manager_only_tools() {
let rendered = render(PRODUCTION_TEMPLATE, Flavor::Manager, "hm1nd", "she/her"); let rendered = render(&PRODUCTION_TEMPLATE, Flavor::Manager, "hm1nd", "she/her");
assert!(rendered.contains("request_init_config")); assert!(rendered.contains("request_init_config"));
assert!(rendered.contains("request_apply_commit")); assert!(rendered.contains("request_apply_commit"));
assert!(rendered.contains("get_logs")); assert!(rendered.contains("get_logs"));
@ -287,9 +306,9 @@ shared closer
#[test] #[test]
fn render_uses_correct_role_opener() { fn render_uses_correct_role_opener() {
let agent = render(PRODUCTION_TEMPLATE, Flavor::Agent, "alice", "she/her"); let agent = render(&PRODUCTION_TEMPLATE, Flavor::Agent, "alice", "she/her");
assert!(agent.starts_with("You are hyperhive agent")); assert!(agent.starts_with("You are hyperhive agent"));
let manager = render(PRODUCTION_TEMPLATE, Flavor::Manager, "hm1nd", "she/her"); let manager = render(&PRODUCTION_TEMPLATE, Flavor::Manager, "hm1nd", "she/her");
assert!(manager.starts_with("You are the hyperhive manager")); assert!(manager.starts_with("You are the hyperhive manager"));
} }
} }