diff --git a/docs/agent-lifecycle/config-changes.md b/docs/agent-lifecycle/config-changes.md index ac36494a..cd5862d4 100644 --- a/docs/agent-lifecycle/config-changes.md +++ b/docs/agent-lifecycle/config-changes.md @@ -118,10 +118,7 @@ declares one flake input per agent (`agent-.url = "git+/agent-configs/.git?rev="`) and one `nixosConfigurations.` output per agent. Each output wraps `inputs.agent-.nixosModules.default` with the identity + -`HIVE_PORT` / `HIVE_LABEL` injection module. Values that differ per host -(the dashboard port, the pinned claude-code, extra OTEL labels) aren't in it: -`hive-priv` writes them to the container's `/etc/hyperhive-hive-facts` at -start (see `docs/process/conventions.md`). +`HIVE_PORT` / `HIVE_LABEL` / `HIVE_DASHBOARD_PORT` injection module. Containers run against `--flake /var/lib/hyperhive/meta#`. The declared input url is the head of `main` on the agent's forge config repo @@ -198,7 +195,8 @@ The container's `--flake` ref is `/var/lib/hyperhive/meta#` (see "Meta flake" above). The agent's own `applied//flake.nix` is a fixed boilerplate that exports `nixosModules.default = import ./agent.nix`; the meta flake imports that module and -wraps it with identity + `HIVE_PORT` / `HIVE_LABEL`. +wraps it with identity + `HIVE_PORT` / `HIVE_LABEL` / +`HIVE_DASHBOARD_PORT`. ### Tag state machine diff --git a/docs/process/conventions.md b/docs/process/conventions.md index 2ab12c09..adc9ec66 100644 --- a/docs/process/conventions.md +++ b/docs/process/conventions.md @@ -39,15 +39,11 @@ Four env vars cover the identity surface, read by name. Inside an agent container, the hive's name and domain don't come from the -agent's build, and neither do the hive's other per-host values: the dashboard -port (`HIVE_DASHBOARD_PORT`), the pinned claude-code -(`HYPERHIVE_CLAUDE_CODE_PATH`) and extra OTEL labels -(`HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`). `hive-priv` writes them to -`/etc/hyperhive-hive-facts` before every container start, leaving out any the -hive doesn't set; the agent then uses its own default. The agent's environment -generator (`nix/agent-modules/hive-facts.nix`) passes them to every unit, -including `hivectl choom` sessions, and `/etc/set-environment` exports them in login -shells that no unit starts (`nixos-container root-login`). None is part of +agent's build. `hive-priv` writes them to `/etc/hyperhive-hive-facts` before +every container start. The agent's environment generator +(`nix/agent-modules/hive-facts.nix`) passes them to every unit, including +`hivectl choom` sessions, and `/etc/set-environment` exports them in login +shells that no unit starts (`nixos-container root-login`). Neither is part of an agent's closure, and an agent started on another hive reports that hive. `hive_name` + `swarm_name` are **distinct** from diff --git a/docs/process/gotchas.md b/docs/process/gotchas.md index 4f16ca6a..343b7a61 100644 --- a/docs/process/gotchas.md +++ b/docs/process/gotchas.md @@ -145,9 +145,9 @@ and every agent runs that build. What crosses is the **store path**, not the derivation. Containers share the host's `/nix/store`, so the binary is already reachable -inside them with its whole closure. `hive-priv` writes the path into each -container's `/etc/hyperhive-hive-facts` at start, and the agent's `claude` -wrapper on PATH runs its `bin/claude`. Two things rule out the natural-seeming alternatives: a +inside them with its whole closure; hive-c0re writes the path into each +agent's flake as a string literal and the agent module symlinks +`bin/claude` onto PATH. Two things rule out the natural-seeming alternatives: a `path:/nix/store/` flake input is re-copied into the store as a reference-less `-source` (so the runtime closure never arrives), and `lib.types.package` fed a bare path runs `builtins.storePath`, which @@ -171,7 +171,7 @@ expected. -The catch is that a path handed to the container at start is text, not a +The catch is that a path written into a generated flake is text, not a reference — the container's closure doesn't keep the binary alive. The **host** does: it interpolates the package into `/etc/hyperhive/serve.json`, so it lands in the host's system closure @@ -180,7 +180,7 @@ string context, which is the load-bearing detail; discard the context anywhere on that path and `nix-collect-garbage` will eventually take the hive's `claude` out from under it. The price of the root is that `nix-collect-garbage` can't reclaim an old `claude-code` until every -agent has restarted past it and the old generations are gone. +agent has rebuilt past it and the old generations are gone. diff --git a/docs/scheduler/observability.md b/docs/scheduler/observability.md index 01678185..9151eb51 100644 --- a/docs/scheduler/observability.md +++ b/docs/scheduler/observability.md @@ -190,7 +190,7 @@ two-tier architecture, the security model, and how the options interact. The harness sets the OTLP variables (`OTEL_EXPORTER_OTLP_ENDPOINT`, `_PROTOCOL`, `OTEL_RESOURCE_ATTRIBUTES`, the temporality preference) **container -wide**, for every unit and in `/etc/profile`, so every +wide** — in systemd's `DefaultEnvironment` and in `/etc/profile` — so every process in an agent container exports to the hive's collector without any per-tool wiring. That covers Claude Code, `hive-metric`, and anything you run yourself from a tool call or a shell you open in the container. @@ -209,9 +209,8 @@ the push (see [Authenticated ingest](#authenticated-ingest)). The container doesn't set it. Append additional labels for one hive via `extraResourceAttributes` (see -option reference above). Agents get them at container start, so a change -reaches an agent at its next start, not through a rebuild. Pass custom -per-data-point labels with `hive-metric --labels` (see below). +option reference above); pass custom per-data-point labels with +`hive-metric --labels` (see below). ## Log severity diff --git a/docs/turn-loop/claude-invocation.md b/docs/turn-loop/claude-invocation.md index 00cb3a15..21539961 100644 --- a/docs/turn-loop/claude-invocation.md +++ b/docs/turn-loop/claude-invocation.md @@ -33,13 +33,11 @@ harness unit's PATH. By default that's the `claude-code` in the agent's own nixpkgs (the meta flake's `nixpkgs` input) via `environment.systemPackages`. Since that's typically a release channel and this package moves fast, the operator can pin one per hive with -`services.hyperhive.deploy.hive-controller.claudeCodePackage`. `hive-priv` -writes its store path into the container's `/etc/hyperhive-hive-facts` as -`HYPERHIVE_CLAUDE_CODE_PATH` before every start. `claude` on PATH is a wrapper -that runs that build when the path exists, and the container's own -`claude-code` when the variable is unset or the path no longer exists, saying -so on stderr in the second case. A bump reaches an agent at its next container -start, not at a rebuild. See docs/process/gotchas.md::`claude-code` is unfree. +`services.hyperhive.deploy.hive-controller.claudeCodePackage`: hive-c0re writes its +store path into each agent's flake, and `claude` on PATH becomes a symlink to it +instead of the container's own `claude-code` — so there's only ever one +`claude` in the container. Agents pick up a new build on their +next rebuild, not live. See docs/process/gotchas.md::`claude-code` is unfree. Hive-enforced settings ship at `/etc/claude-code/managed-settings.json` (claude-code's canonical managed-settings path — precedence #1, diff --git a/hive-c0re/src/coordinator.rs b/hive-c0re/src/coordinator.rs index bcb72ed4..d752f947 100644 --- a/hive-c0re/src/coordinator.rs +++ b/hive-c0re/src/coordinator.rs @@ -43,8 +43,8 @@ pub struct Coordinator { /// Git rev of the hyperhive checkout this host runs. Pins the meta /// flake's `hyperhive` input; `None` makes the meta render fail. pub hyperhive_rev: Option, - /// Store path of the `claude-code` build every agent runs, handed to - /// each container at start as `HYPERHIVE_CLAUDE_CODE_PATH`. Set by + /// Store path of the `claude-code` build every agent runs, written + /// into each per-agent flake as `services.hyperhive.agent.claudeCodePath`. Set by /// the NixOS module option /// `services.hyperhive.deploy.hive-controller.claudeCodePackage` /// (which resolves the package and hands us its path). `None` = every @@ -55,9 +55,9 @@ pub struct Coordinator { /// closure and all. The host module is what keeps it from being /// garbage-collected — see that option. pub claude_code_path: Option, - /// TCP port the host's hive-c0re dashboard listens on. Handed to each - /// container at start as `HIVE_DASHBOARD_PORT`, so the agent's web UI - /// can build the rebuild-button URL pointing back at the dashboard. + /// TCP port the host's hive-c0re dashboard listens on. Inlined into + /// each per-agent flake so the agent's web UI can build the right + /// rebuild-button URL pointing back at the dashboard. pub dashboard_port: u16, /// Per-agent systemd `CPUQuota=` value (e.g. `"200%"`). Written into /// the `container@h-.service.d/` drop-in on every spawn/rebuild. @@ -193,7 +193,7 @@ pub struct HiveEnv { pub hyperhive_rev: Option, /// Store path of the `claude-code` agents run, or `None` for "each /// agent keeps the one out of its own nixpkgs". Travels into the - /// container as `HYPERHIVE_CLAUDE_CODE_PATH` — a plain string, kept + /// container as `services.hyperhive.agent.claudeCodePath` — a plain string, kept /// alive host-side by the module that resolved it. pub claude_code_path: Option, pub dashboard_port: u16, diff --git a/hive-c0re/src/lifecycle/host_config.rs b/hive-c0re/src/lifecycle/host_config.rs index bb20df7e..4204ede5 100644 --- a/hive-c0re/src/lifecycle/host_config.rs +++ b/hive-c0re/src/lifecycle/host_config.rs @@ -25,7 +25,7 @@ use super::{ pub async fn write_dropins(name: &str, hive: &HiveEnv, paths: &AgentPaths) -> Result<()> { validate(name)?; let container = container_name(name); - set_nspawn_flags(&container, hive, &paths.agent, &paths.claude, &paths.notes).await?; + set_nspawn_flags(&container, &paths.agent, &paths.claude, &paths.notes).await?; hand_permissions(name, &paths.agent); if let Some((cpu_quota, memory_max)) = stage_limits( name, @@ -496,7 +496,6 @@ fn queue_agent_credentials(agent_name: &str, dir: Option<&Path>) -> Vec Result network_isolation_from_vars(bridge.as_deref(), subnet.as_deref()) } -/// The hive's facts every container is handed at runtime, so that no agent -/// build carries a value that differs between hosts. The name, domain and -/// OTEL resource labels come from hive-c0re's own unit environment -/// (`nix/host-modules/hive-c0re/environment.nix`), where an unset or empty -/// variable is `None`; the dashboard port and pinned claude from `hive`. -fn hive_facts(hive: &HiveEnv) -> hive_priv_sock::HiveFacts { - hive_facts_from(hive, |name| std::env::var(name).ok()) -} - -/// [`hive_facts`] over a given environment lookup, for tests. -fn hive_facts_from( - hive: &HiveEnv, - var: impl Fn(&str) -> Option, -) -> hive_priv_sock::HiveFacts { - let read = |name: &str| var(name).filter(|v| !v.is_empty()); +/// The hive's facts every container is handed at runtime, from hive-c0re's +/// own unit environment (`nix/host-modules/hive-c0re/environment.nix`). An +/// unset or empty variable is `None`. +fn hive_facts_from_env() -> hive_priv_sock::HiveFacts { + let read = |name: &str| std::env::var(name).ok().filter(|v| !v.is_empty()); hive_priv_sock::HiveFacts { name: read("HYPERHIVE_HIVE_NAME"), domain: read("HYPERHIVE_HIVE_DOMAIN"), - dashboard_port: Some(hive.dashboard_port), - claude_code_path: hive.claude_code_path.clone(), - otel_extra_resource_attributes: read("HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES"), } } diff --git a/hive-c0re/src/lifecycle/tests.rs b/hive-c0re/src/lifecycle/tests.rs index b68ef27e..0ea0d190 100644 --- a/hive-c0re/src/lifecycle/tests.rs +++ b/hive-c0re/src/lifecycle/tests.rs @@ -527,39 +527,3 @@ async fn run_with_deadline_returns_output_of_a_fast_child() { .expect("echo finishes"); assert_eq!(String::from_utf8_lossy(&out.stdout).trim(), "hi"); } - -/// The three per-host values reach the container as facts: the dashboard -/// port and pinned claude from the hive's config, the OTEL labels from the -/// daemon's environment. -#[test] -fn hive_facts_carry_the_per_host_values() { - let hive = HiveEnv { - dashboard_port: 7123, - claude_code_path: Some("/nix/store/0abc-claude-code".to_owned()), - ..HiveEnv::default() - }; - let facts = hive_facts_from(&hive, |name| { - (name == "HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES").then(|| "team=a".to_owned()) - }); - assert_eq!(facts.dashboard_port, Some(7123)); - assert_eq!( - facts.claude_code_path.as_deref(), - Some("/nix/store/0abc-claude-code") - ); - assert_eq!( - facts.otel_extra_resource_attributes.as_deref(), - Some("team=a") - ); -} - -/// Unset and empty both leave a fact out, so the agent falls back to its -/// own default rather than reading an empty value. -#[test] -fn hive_facts_leave_unset_values_out() { - let facts = hive_facts_from(&HiveEnv::default(), |name| { - (name == "HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES").then(String::new) - }); - assert_eq!(facts.claude_code_path, None); - assert_eq!(facts.otel_extra_resource_attributes, None); - assert_eq!(facts.name, None); -} diff --git a/hive-c0re/src/meta.rs b/hive-c0re/src/meta.rs index 430c65ae..715886b6 100644 --- a/hive-c0re/src/meta.rs +++ b/hive-c0re/src/meta.rs @@ -431,12 +431,23 @@ fn pinned_git_url(forge_git_url: &str, rev: Option<&str>) -> Result { Ok(format!("git+{forge_git_url}?rev={rev}")) } -/// [`render_flake`] with the hyperhive pin taken from `hive`, as -/// [`sync_agents`] renders it. Nothing else in `hive` reaches the flake: the -/// values that differ per host go to the container at start -/// (`lifecycle::hive_facts`). +/// [`render_flake`] with its inputs taken from `hive`, as [`sync_agents`] +/// renders it. fn render_hive_flake(hive: &HiveEnv, agents: &[AgentSpec]) -> Result { - render_flake(&hyperhive_input_url(hive)?, agents) + render_hive_flake_at(hive, &hyperhive_input_url(hive)?, agents) +} + +fn render_hive_flake_at( + hive: &HiveEnv, + hyperhive_url: &str, + agents: &[AgentSpec], +) -> Result { + render_flake( + hyperhive_url, + hive.claude_code_path.as_deref(), + hive.dashboard_port, + agents, + ) } /// Canonical inputs meta knows how to dedup. An agent that declares one @@ -618,6 +629,19 @@ fn forwarded_env_vars() -> Vec<(&'static str, String)> { .collect() } +/// Extra `OTEL_RESOURCE_ATTRIBUTES` entries for every agent, read off +/// hive-c0re's own unit env +/// (`services.hyperhive.deploy.hive-controller.otel.extraResourceAttributes`). +/// `None` when unset or empty. This is the only OTEL value the renderer +/// writes into the flake: agents export to the hive collector by default and +/// find it by name, so there is no enable flag, endpoint or protocol to +/// carry. +fn otel_extra_resource_attributes() -> Option { + std::env::var("HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES") + .ok() + .filter(|v| !v.is_empty()) +} + /// Remove the CA files earlier renders embedded next to `flake.nix` /// (`hive-ca.pem`, and the per-peer `peer-ca-.pem` before it), returning /// every filename removed so the caller stages the deletion. Nothing writes @@ -740,7 +764,12 @@ fn canonical_inputs_in_lock(raw: &str) -> Vec<&'static str> { reason = "templated string-builder for the meta flake — the length is one \ contiguous fmt block, splitting it would just hide the shape" )] -fn render_flake(hyperhive_url: &str, agents: &[AgentSpec]) -> Result { +fn render_flake( + hyperhive_url: &str, + claude_code_path: Option<&str>, + dashboard_port: u16, + agents: &[AgentSpec], +) -> Result { use std::fmt::Write as _; let mut out = String::new(); out.push_str("{\n description = \"hyperhive deployed agents\";\n inputs = {\n"); @@ -787,8 +816,10 @@ fn render_flake(hyperhive_url: &str, agents: &[AgentSpec]) -> Result { ); } } - out.push_str( - " };\n outputs =\n { self, hyperhive, ... }@inputs:\n let\n mkAgent = { name, port, archetype ? null }:\n", + out.push_str(" };\n outputs =\n { self, hyperhive, ... }@inputs:\n let\n"); + let _ = writeln!( + out, + " dashboardPort = {dashboard_port};\n mkAgent = {{ name, port, archetype ? null }}:" ); out.push_str( r#" let @@ -811,6 +842,32 @@ fn render_flake(hyperhive_url: &str, agents: &[AgentSpec]) -> Result { { "#, ); + // The `claude-code` agents run, as a bare store path rather than a + // flake input: containers share the host's `/nix/store`, so the + // binary is already there with its closure and has nothing to + // travel. A string literal is also the only shape that evaluates — + // `lib.types.package` on a bare path runs `builtins.storePath`, + // which pure eval rejects. The agent module puts its `bin/` on the + // harness PATH; the host module holds the gc root, since a path + // spelled out here is text and references nothing. + // `None` = no override; agents keep their own nixpkgs' `claude-code`. + if let Some(path) = claude_code_path { + let _ = writeln!( + out, + " services.hyperhive.agent.claudeCodePath = \"{path}\";" + ); + } + // Extra OTEL resource labels (`deploy.hive-controller.otel.*`) are the one + // OTEL value rendered here; enablement and the collector endpoint are + // agent-module defaults. + if let Some(attrs) = otel_extra_resource_attributes() { + let esc = |s: &str| s.replace('\\', "\\\\").replace('"', "\\\""); + let _ = writeln!( + out, + " services.hyperhive.agent.otel.extraResourceAttributes = \"{}\";", + esc(&attrs) + ); + } // Agent-facing service URLs (`services.hyperhive.agent.forge.url`, `services.hyperhive.agent.matrix.url`): // emit the host's real values as build-time agent config, the same // host-state -> agent-module shape as the CA block above. @@ -902,6 +959,7 @@ fn render_flake(hyperhive_url: &str, agents: &[AgentSpec]) -> Result { systemd.services.${service}.environment = { HIVE_PORT = toString port; HIVE_LABEL = name; + HIVE_DASHBOARD_PORT = toString dashboardPort; ", ); // Forwarded env vars — picked up from hive-c0re's own systemd unit @@ -1224,8 +1282,13 @@ mod tests { } } - fn render(hyperhive_url: &str, agents: &[AgentSpec]) -> String { - render_flake(hyperhive_url, agents).expect("render") + fn render( + hyperhive_url: &str, + claude_code_path: Option<&str>, + dashboard_port: u16, + agents: &[AgentSpec], + ) -> String { + render_flake(hyperhive_url, claude_code_path, dashboard_port, agents).expect("render") } #[test] @@ -1246,6 +1309,8 @@ mod tests { fn the_input_a_rebuild_relocks_is_a_pinned_forge_url() { let out = render( "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", + None, + 8000, &[sample_spec("iris", 9001)], ); let url = out @@ -1279,6 +1344,8 @@ mod tests { fn render_flake_pins_hyperhive_by_url_and_follows_its_nixpkgs() { let out = render( "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", + None, + 8000, &[sample_spec("alice", 9001)], ); assert!( @@ -1307,6 +1374,8 @@ mod tests { fn render_flake_names_each_agent_container_after_the_agent() { let out = render( "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", + None, + 8000, &[sample_spec("alice", 9001)], ); assert!( @@ -1332,6 +1401,49 @@ mod tests { ); } + #[test] + fn render_flake_pins_claude_path_without_adding_an_input() { + // The host-pinned claude travels as a bare store path assigned to + // an option — deliberately NOT as a flake input. Containers share + // the host store, so the build is already reachable; making it an + // input would re-copy it as a reference-less `-source` and strip + // the closure the binary actually needs. + let out = render( + "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", + Some("/nix/store/cccc-claude-code-2.1.220"), + 8000, + &[sample_spec("alice", 9001)], + ); + assert!( + out.contains( + "services.hyperhive.agent.claudeCodePath = \"/nix/store/cccc-claude-code-2.1.220\";" + ), + "claude path assigned as a plain string literal:\n{out}" + ); + assert!( + !out.contains("claude-code-2.1.220\".url"), + "the pinned claude must not become a flake input:\n{out}" + ); + } + + #[test] + fn render_flake_omits_claude_path_when_unset() { + // `None` = no host-level pin: the option is left undefined so the + // agent module keeps its own nixpkgs' `claude-code` (and keeps it + // in `environment.systemPackages`, which is what makes the + // unpinned case self-contained). + let out = render( + "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", + None, + 8000, + &[sample_spec("alice", 9001)], + ); + assert!( + !out.contains("claudeCodePath"), + "no claude assignment when unpinned:\n{out}" + ); + } + #[test] fn render_flake_emits_follows_for_agents_declaring_nixpkgs() { // `bitburner` + `dmatrix` declare `nixpkgs` at their root, while @@ -1342,6 +1454,8 @@ mod tests { }; let out = render( "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", + None, + 8000, &[ sample_spec("argus", 9001), with_nixpkgs("bitburner", 9002), @@ -1368,6 +1482,8 @@ mod tests { fn render_flake_builds_every_agent_from_agent_base() { let out = render( "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", + None, + 8000, &[sample_spec("alice", 9001), sample_spec("ruth", 9002)], ); assert_eq!( @@ -1391,6 +1507,8 @@ mod tests { fn render_flake_skips_canonical_follows_when_the_lock_declares_none() { let out = render( "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", + None, + 8000, &[sample_spec("alice", 9001)], ); // No agent-side follows when the lock declares nothing @@ -1420,6 +1538,8 @@ mod tests { }; let out = render( "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", + None, + 8000, &[alice, android], ); assert!( @@ -1468,6 +1588,8 @@ mod tests { } let out = render( "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", + None, + 8000, &[sample_spec("alice", 9001)], ); unsafe { @@ -1507,6 +1629,8 @@ mod tests { } render( "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", + None, + 8000, &[sample_spec("alice", 9001)], ) }; @@ -1700,6 +1824,8 @@ mod tests { } let out = render( "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", + None, + 8000, &[sample_spec("alice", 9001)], ); unsafe { @@ -1729,6 +1855,8 @@ mod tests { }; let out = render( "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", + None, + 8000, &[at("alice", 9001, "aaaa"), at("bob", 9002, "bbbb")], ); assert!( @@ -1754,6 +1882,8 @@ mod tests { let args = |agents: &[AgentSpec]| { render_flake( "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", + None, + 8000, agents, ) }; @@ -1908,6 +2038,8 @@ mod tests { let render = || { render( "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", + None, + 8000, &[sample_spec("alice", 9001)], ) }; @@ -1968,12 +2100,11 @@ mod tests { } #[test] - fn render_flake_carries_no_otel_config_and_no_per_host_value() { + fn render_flake_carries_no_otel_enablement() { // The agent module defaults otel on and finds the collector by name, so // the flake carries no enable, endpoint or protocol line however the - // host's OTEL env is set. The resource labels, dashboard port and - // pinned claude differ per host and reach the agent at container start - // (`lifecycle::hive_facts`), so none of them is rendered either. + // host's OTEL env is set. Only the per-hive resource labels are + // rendered, and only when set. // // Serialised against every other env-mutating test in the crate. // `stats::otel_metrics::tests` perturbs OTEL env too, and lands in this @@ -1984,6 +2115,8 @@ mod tests { let render = || { render( "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", + None, + 8000, &[sample_spec("alice", 9001)], ) }; @@ -2006,11 +2139,10 @@ mod tests { std::env::remove_var("HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES"); } - // Presence controls: the render is the real flake, carrying the pin and - // the harness env block the dashboard port used to sit in. + // Presence control: the render is the real flake, carrying the pin. assert!( - plain.contains("rev=0123abcd") && plain.contains("HIVE_PORT = toString port;"), - "the hyperhive pin and the harness env must be rendered:\n{plain}" + plain.contains("rev=0123abcd"), + "the hyperhive pin line must be rendered:\n{plain}" ); for (name, flake) in [("plain", &plain), ("labelled", &labelled)] { for needle in [ @@ -2018,11 +2150,6 @@ mod tests { "agent.otel.endpoint", "agent.otel.protocol", "c.example", - "extraResourceAttributes", - "deployment.environment", - "claudeCodePath", - "dashboardPort", - "HIVE_DASHBOARD_PORT", ] { assert!( !flake.contains(needle), @@ -2030,6 +2157,16 @@ mod tests { ); } } + assert!( + !plain.contains("agent.otel.extraResourceAttributes"), + "extraResourceAttributes must not appear when unset:\n{plain}" + ); + assert!( + labelled.contains( + "services.hyperhive.agent.otel.extraResourceAttributes = \"deployment.environment=prod\";" + ), + "extraResourceAttributes must be rendered when set:\n{labelled}" + ); } #[test] fn render_flake_injects_github_disable_only_when_signalled() { @@ -2044,6 +2181,8 @@ mod tests { let render = || { render( "git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd", + None, + 8000, &[sample_spec("alice", 9001)], ) }; @@ -2067,6 +2206,33 @@ mod tests { "github.enable = false must be injected when the host disables it:\n{disabled}" ); } + + /// The agent reads its memory cap at boot from the copy + /// `lifecycle::write_dropins` hands it, so the cap is no build input. + #[test] + fn render_hive_flake_is_identical_across_memory_caps() { + let hive = |cap: &str| HiveEnv { + agent_memory_max: cap.to_owned(), + ..HiveEnv::default() + }; + let agents = [sample_spec("alice", 9001)]; + let small = render_hive_flake_at( + &hive("4G"), + "git+https://forge.example/h.git?rev=abc", + &agents, + ) + .expect("render"); + let large = render_hive_flake_at( + &hive("16G"), + "git+https://forge.example/h.git?rev=abc", + &agents, + ) + .expect("render"); + assert_eq!( + small, large, + "the rendered flake must not depend on the memory cap" + ); + } } #[cfg(test)] diff --git a/hive-c0re/src/test_env.rs b/hive-c0re/src/test_env.rs index 1be54a5a..f280eae9 100644 --- a/hive-c0re/src/test_env.rs +++ b/hive-c0re/src/test_env.rs @@ -11,8 +11,8 @@ //! ⚠️ **The lock has to be crate-wide, not per-module.** `hive-bash-mcp`'s //! sibling helper records why: two separate per-module mutexes serialise //! nothing against each other, and that produced a CI-only flake. The pair that -//! motivated this one is `meta::tests` (which renders a flake with -//! `HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES` set) and `stats::otel_metrics::tests` (which asserts which +//! motivated this one is `meta::tests` (which renders a flake from +//! `HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`) and `stats::otel_metrics::tests` (which asserts which //! variable enables the exporter) — different modules, same process env, same //! binary. //! diff --git a/hive-priv-sock/src/lib.rs b/hive-priv-sock/src/lib.rs index 331c50d2..86e3679f 100644 --- a/hive-priv-sock/src/lib.rs +++ b/hive-priv-sock/src/lib.rs @@ -187,15 +187,6 @@ pub struct HiveFacts { pub name: Option, /// `HYPERHIVE_HIVE_DOMAIN`: the hive's DNS domain. pub domain: Option, - /// `HIVE_DASHBOARD_PORT`: the port this hive's dashboard listens on. - pub dashboard_port: Option, - /// `HYPERHIVE_CLAUDE_CODE_PATH`: store path of the `claude-code` the - /// hive pins for its agents. The host's system closure keeps it alive; - /// the container only gets the path. - pub claude_code_path: Option, - /// `HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`: comma-separated `k=v` - /// entries the agent appends to its `OTEL_RESOURCE_ATTRIBUTES`. - pub otel_extra_resource_attributes: Option, } /// Wall-clock bound hive-priv puts on one [`PrivRequest::MeasureAgentDisk`] diff --git a/hive-priv/src/container_config.rs b/hive-priv/src/container_config.rs index 0fc3e297..64352042 100644 --- a/hive-priv/src/container_config.rs +++ b/hive-priv/src/container_config.rs @@ -367,54 +367,24 @@ fn container_rootfs(container: &str) -> PathBuf { /// Render [`HiveFacts`] as the systemd environment file the agent loads, /// one `KEY=value` line per present fact. /// -/// Each value is refused, never escaped, unless it is made only of the -/// characters its fact can hold. None of those sets admits a newline (a -/// second assignment), whitespace, a quote, `\` or `$`: the agent hands each -/// line to systemd and its shells, which would read those as syntax. +/// A value is refused unless it is DNS-name characters only, so it can carry +/// neither a newline (a second assignment) nor anything an environment file +/// or a shell would interpret. fn render_hive_facts(facts: &HiveFacts) -> Result { use std::fmt::Write as _; - let dns = |b: u8| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'-' | b'_'); - let port = facts.dashboard_port.map(|p| p.to_string()); let mut out = String::new(); - for (key, value, max_len, allowed) in [ - ( - "HYPERHIVE_HIVE_NAME", - facts.name.as_ref(), - 253, - &dns as &dyn Fn(u8) -> bool, - ), - ("HYPERHIVE_HIVE_DOMAIN", facts.domain.as_ref(), 253, &dns), - ("HIVE_DASHBOARD_PORT", port.as_ref(), 5, &|b: u8| { - b.is_ascii_digit() - }), - // A store path's name characters, plus the `/` of its prefix; the - // prefix itself is checked below. - ( - "HYPERHIVE_CLAUDE_CODE_PATH", - facts.claude_code_path.as_ref(), - 256, - &|b: u8| dns(b) || matches!(b, b'+' | b'?' | b'=' | b'/'), - ), - // `k=v,k=v`, values percent-encoded as the OTEL spec has them. - ( - "HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES", - facts.otel_extra_resource_attributes.as_ref(), - 1024, - &|b: u8| dns(b) || matches!(b, b'=' | b',' | b'/' | b':' | b'%' | b'+' | b'@'), - ), + for (key, value) in [ + ("HYPERHIVE_HIVE_NAME", &facts.name), + ("HYPERHIVE_HIVE_DOMAIN", &facts.domain), ] { let Some(value) = value else { continue }; - if value.is_empty() || value.len() > max_len || !value.bytes().all(allowed) { - bail!( - "invalid {key} value {value:?}: must be 1-{max_len} chars of its fact's character set" - ); - } - if key == "HYPERHIVE_CLAUDE_CODE_PATH" - && !value - .strip_prefix("/nix/store/") - .is_some_and(|name| !name.is_empty() && !name.contains('/')) + if value.is_empty() + || value.len() > 253 + || !value + .bytes() + .all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'-' | b'_')) { - bail!("invalid {key} value {value:?}: must be a top-level /nix/store path"); + bail!("invalid {key} value {value:?}: must be 1-253 chars of [A-Za-z0-9._-]"); } let _ = writeln!(out, "{key}={value}"); } @@ -741,15 +711,14 @@ mod tests { let both = HiveFacts { name: Some("pr1ma".to_owned()), domain: Some("pr1ma.darkest.space".to_owned()), - ..HiveFacts::default() }; assert_eq!( render_hive_facts(&both).unwrap(), "HYPERHIVE_HIVE_NAME=pr1ma\nHYPERHIVE_HIVE_DOMAIN=pr1ma.darkest.space\n" ); let domain_only = HiveFacts { + name: None, domain: Some("h1.example.test".to_owned()), - ..HiveFacts::default() }; assert_eq!( render_hive_facts(&domain_only).unwrap(), @@ -765,66 +734,7 @@ mod tests { for bad in ["", "pr1ma\nLD_PRELOAD=/x", "a b", "a\"b", "$(x)"] { let facts = HiveFacts { name: Some(bad.to_owned()), - ..HiveFacts::default() - }; - assert!( - render_hive_facts(&facts).is_err(), - "{bad:?} must be refused" - ); - } - } - - /// The three per-host values agents used to get at build time arrive - /// here, one line each. - #[test] - fn hive_facts_render_the_per_host_values() { - let facts = HiveFacts { - dashboard_port: Some(7123), - claude_code_path: Some("/nix/store/0abc-claude-code-2.1.220".to_owned()), - otel_extra_resource_attributes: Some( - "deployment.environment=prod,team=a%2Fb".to_owned(), - ), - ..HiveFacts::default() - }; - assert_eq!( - render_hive_facts(&facts).unwrap(), - "HIVE_DASHBOARD_PORT=7123\n\ - HYPERHIVE_CLAUDE_CODE_PATH=/nix/store/0abc-claude-code-2.1.220\n\ - HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES=deployment.environment=prod,team=a%2Fb\n" - ); - } - - /// The claude path is a top-level store path or nothing, and neither - /// value may carry generator syntax. - #[test] - fn hive_facts_refuse_bad_claude_paths_and_resource_attributes() { - for bad in [ - "/nix/store/", - "/nix/store/abc-claude/bin/claude", - "/tmp/claude", - "nix/store/abc-claude", - "/nix/store/abc-claude\nLD_PRELOAD=/x", - ] { - let facts = HiveFacts { - claude_code_path: Some(bad.to_owned()), - ..HiveFacts::default() - }; - assert!( - render_hive_facts(&facts).is_err(), - "{bad:?} must be refused" - ); - } - for bad in [ - "a=b\nPATH=/x", - "a=${HOME}", - "a=b c", - "a=\"b\"", - "a=b\\c", - "", - ] { - let facts = HiveFacts { - otel_extra_resource_attributes: Some(bad.to_owned()), - ..HiveFacts::default() + domain: None, }; assert!( render_hive_facts(&facts).is_err(), @@ -842,7 +752,10 @@ mod tests { bridge: "hive0".to_owned(), gateway_ip: "10.42.0.1".to_owned(), }; - let facts = HiveFacts::default(); + let facts = HiveFacts { + name: None, + domain: None, + }; for name in ["hive-forge", "hive-matrix"] { let errs = [ handle_write_nspawn_flags(name, &[], &isolation, &[], &facts).unwrap_err(), diff --git a/nix/agent-modules/default.nix b/nix/agent-modules/default.nix index 6f4f5705..6047ad5c 100644 --- a/nix/agent-modules/default.nix +++ b/nix/agent-modules/default.nix @@ -28,33 +28,6 @@ let which the hive writes before every start, so no agent build sets it. Remove the setting. ''; - claudeCodePathRemoved = '' - An agent reads the claude-code its hive pins at runtime, from - HYPERHIVE_CLAUDE_CODE_PATH in /etc/hyperhive-hive-facts, so no agent build - sets it. Pin it host-side with - services.hyperhive.deploy.hive-controller.claudeCodePackage. - ''; - - # `claude` on PATH. A hive that pins a claude-code - # (`services.hyperhive.deploy.hive-controller.claudeCodePackage`) hands its - # store path over at container start as `HYPERHIVE_CLAUDE_CODE_PATH`, and - # the host's system closure keeps it alive: nothing in this container - # references it. With no pin, or a path that is not there, this container's - # own claude-code runs; a pin that is set but gone (collected after a bump - # while this container still holds the old facts) says so on stderr, since - # the two builds can differ. `exec -a claude` keeps the bare name in argv[0], - # which is how the harness finds its claude child (`find_claude_child`). - ownClaude = import ../claude-code-no-alsa.nix lib pkgs.claude-code; - claude = pkgs.writeShellScriptBin "claude" '' - pinned=''${HYPERHIVE_CLAUDE_CODE_PATH:-} - if [ -n "$pinned" ] && [ -x "$pinned/bin/claude" ]; then - exec -a claude "$pinned/bin/claude" "$@" - fi - if [ -n "$pinned" ]; then - echo "claude: $pinned/bin/claude, the hive's pinned claude-code, is missing; running this container's own" >&2 - fi - exec -a claude ${ownClaude}/bin/claude "$@" - ''; in { imports = [ @@ -95,12 +68,6 @@ in "agent" "claudeMemoryMaxBytes" ] memoryMaxBytesRemoved) - (lib.mkRemovedOptionModule [ - "services" - "hyperhive" - "agent" - "claudeCodePath" - ] claudeCodePathRemoved) ]; options.services.hyperhive.agent.icon = lib.mkOption { @@ -120,6 +87,33 @@ in ''; }; + options.services.hyperhive.agent.claudeCodePath = lib.mkOption { + type = lib.types.nullOr lib.types.str; + default = null; + example = "/nix/store/…-claude-code-2.1.220"; + description = '' + Store path of the `claude-code` this agent runs, or `null` (the + default) to use the `claude-code` from the container's own + nixpkgs. + + Set by the generated meta flake when the operator sets + `services.hyperhive.deploy.hive-controller.claudeCodePackage` host-side, so a hive + can run a claude built from a *different* nixpkgs than the one + its agents evaluate against — a release channel can trail + unstable by weeks on this one package. It arrives as a path and + not a package because agents share the host's `/nix/store`, so + the build is already reachable here with its whole closure and + has nothing to travel. + + When set, `claude` on PATH is a symlink to this path's + `bin/claude` and the container's own `claude-code` is dropped, so + there is only ever one claude in the container. Note that neither + the symlink nor anything else in the container's closure *refers* + to the target — keeping it alive is the host's job, see + `services.hyperhive.deploy.hive-controller.claudeCodePackage`. + ''; + }; + config = { assertions = [ # Guard the inputs-routed-as-output pattern: the agent flake.nix is @@ -246,7 +240,28 @@ in config.services.hyperhive.agent.packages.hive-metric ] ++ [ - claude + ( + if config.services.hyperhive.agent.claudeCodePath == null then + import ../claude-code-no-alsa.nix lib pkgs.claude-code + else + # Host-pinned claude: a symlink farm around a path the + # container was handed as text. It has to be a derivation — + # `environment.systemPackages` coerces a store-path *string* + # with `toDerivation`, i.e. `builtins.storePath`, which pure + # evaluation rejects (`systemd.services.*.path` does the same, + # which is why the harness gets this via PATH like everything + # else rather than a unit-level entry). Interpolating the path + # into the builder is just text, so it evaluates anywhere. + # + # The symlink registers no store reference — the target isn't + # among this derivation's inputs, so nothing here keeps the + # binary alive. That is deliberate and it is the host's job: + # see `services.hyperhive.deploy.hive-controller.claudeCodePackage`. + pkgs.runCommandLocal "claude-code-pinned" { } '' + mkdir -p "$out/bin" + ln -s ${config.services.hyperhive.agent.claudeCodePath}/bin/claude "$out/bin/claude" + '' + ) ] ++ (with pkgs; [ bashInteractive diff --git a/nix/agent-modules/hive-facts.nix b/nix/agent-modules/hive-facts.nix index 864965bb..c5e33e06 100644 --- a/nix/agent-modules/hive-facts.nix +++ b/nix/agent-modules/hive-facts.nix @@ -2,17 +2,8 @@ # agent's closure does not carry it. # # hive-priv writes `/etc/hyperhive-hive-facts` into the container before every -# start, one `KEY=value` line each: -# -# - `HYPERHIVE_HIVE_NAME`, `HYPERHIVE_HIVE_DOMAIN`; -# - `HIVE_DASHBOARD_PORT`, read by the harness's web UI (7000 when absent); -# - `HYPERHIVE_CLAUDE_CODE_PATH`, the claude-code the hive pins (`claude` in -# ./default.nix runs this container's own when absent); -# - `HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`, appended to -# `OTEL_RESOURCE_ATTRIBUTES` by ./otel.nix (no extra labels when absent). -# -# Any of them may be absent: a hive-c0re older than the agent's build writes -# fewer. Two readers hand the lines on: +# start (`HYPERHIVE_HIVE_NAME`, `HYPERHIVE_HIVE_DOMAIN`, one `KEY=value` line +# each). Two readers hand those lines on: # # - an environment generator, for the container's systemd: it passes them to # every unit it starts, `machinectl shell` sessions (`hivectl choom`) @@ -32,8 +23,8 @@ let userName = config.services.hyperhive.agent.user.name; - # Shell builtins only: nothing is assumed on PATH this early. Only the known - # keys are passed on. `|| [ -n "$key" ]` keeps a last line that has + # Shell builtins only: nothing is assumed on PATH this early. Only the two + # known keys are passed on. `|| [ -n "$key" ]` keeps a last line that has # no trailing newline. generator = pkgs.writeShellScript "hyperhive-hive-facts" '' hive=unknown @@ -41,8 +32,7 @@ let while IFS='=' read -r key value || [ -n "$key" ]; do case "$key" in HYPERHIVE_HIVE_NAME) hive=$value ;; - HYPERHIVE_HIVE_DOMAIN | HIVE_DASHBOARD_PORT) ;; - HYPERHIVE_CLAUDE_CODE_PATH | HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES) ;; + HYPERHIVE_HIVE_DOMAIN) ;; *) continue ;; esac printf '%s=%s\n' "$key" "$value" diff --git a/nix/agent-modules/otel.nix b/nix/agent-modules/otel.nix index 9ddbdbf8..d0f9046b 100644 --- a/nix/agent-modules/otel.nix +++ b/nix/agent-modules/otel.nix @@ -42,17 +42,9 @@ let # agent, and their metric names already tell them apart. No `hive`: the # swarm collector stamps it from the receiver that accepted the hive's # push (nix/host-modules/swarm-otel.nix), so the build names no hive. - resourceAttributes = "service.name=hyperhive-agent,agent=${userName},swarm=${swarmDisplayName}"; - - # Prints `OTEL_RESOURCE_ATTRIBUTES`: the labels above, then the hive's own - # (`HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`), which differ per host and so - # arrive at container start. That variable comes from ./hive-facts.nix's - # generator, which runs before this one and is absent when the hive sets no - # labels. - resourceGenerator = pkgs.writeShellScript "hyperhive-otel-resource" '' - printf 'OTEL_RESOURCE_ATTRIBUTES=%s%s\n' ${lib.escapeShellArg resourceAttributes} \ - "''${HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES:+,$HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES}" - ''; + resourceAttributes = + "service.name=hyperhive-agent,agent=${userName},swarm=${swarmDisplayName}" + + lib.optionalString (cfg.extraResourceAttributes != "") ",${cfg.extraResourceAttributes}"; # The variables an OTEL SDK reads on its own, in any language, in any # process here. @@ -70,6 +62,7 @@ let # which Prometheus/Mimir-family backends (incl. grafana-lgtm) # silently drop without a deltatocumulative processor. OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE = "cumulative"; + OTEL_RESOURCE_ATTRIBUTES = resourceAttributes; }; # The exporter's NAME picks the wire protocol, so `protocol` has to be @@ -125,6 +118,19 @@ in ''; }; + extraResourceAttributes = lib.mkOption { + type = lib.types.str; + default = ""; + internal = true; + description = '' + Extra comma-separated entries appended to + `OTEL_RESOURCE_ATTRIBUTES` after the built-in + `service.name` / `agent` / `swarm` labels. + Host-driven via + `services.hyperhive.deploy.hive-controller.otel.extraResourceAttributes`. + ''; + }; + debug = lib.mkOption { type = lib.types.bool; default = false; @@ -161,14 +167,6 @@ in systemd.globalEnvironment = producerEnv; environment.variables = producerEnv; - # `OTEL_RESOURCE_ATTRIBUTES` is in neither: a unit's own `Environment=` - # would override what the generator computes at boot. Units get it from - # the manager's environment; login shells run the same generator after - # ./hive-facts.nix's snippet has exported the hive's labels. - environment.etc."systemd/system-environment-generators/60-hyperhive-otel-resource".source = - resourceGenerator; - environment.extraInit = lib.mkAfter ''export "$(${resourceGenerator})"''; - # The journal this container writes is reachable from inside it and # from nowhere else, which is why the forwarder runs here rather than # the hive reading the container's journal from outside: the host-side diff --git a/nix/host-modules/hive-c0re/default.nix b/nix/host-modules/hive-c0re/default.nix index 0f2d39a0..9b38328f 100644 --- a/nix/host-modules/hive-c0re/default.nix +++ b/nix/host-modules/hive-c0re/default.nix @@ -110,9 +110,9 @@ let # in whole rather than interpolated — that is also the ONLY thing # gc-rooting it: the resulting string carries store context, so this # /etc entry genuinely references the package and the host's system - # closure holds it alive. Nothing container-side can: hive-c0re hands - # the path to each container as text (`HYPERHIVE_CLAUDE_CODE_PATH`), - # and text is not a reference. Hence the assertion below: do + # closure holds it alive. Nothing container-side can — meta.rs writes + # the path into each agent's generated flake as a plain string + # literal, and text is not a reference. Hence the assertion below: do # NOT discard this context, and do not route the path through # anything that drops it. The failure mode is a garbage-collected # `claude` and a hive that can't take a turn, weeks after the commit diff --git a/nix/host-modules/hive-c0re/options.nix b/nix/host-modules/hive-c0re/options.nix index 17267da8..ebabd39e 100644 --- a/nix/host-modules/hive-c0re/options.nix +++ b/nix/host-modules/hive-c0re/options.nix @@ -326,9 +326,8 @@ }; }; - # What this hive hands its own agents at container start, through - # `/etc/hyperhive-hive-facts`. Per host, so two hives may set them - # differently. + # What this hive hands its own agents through the meta-flake render. Per + # host, so two hives may set them differently. options.services.hyperhive.deploy.hive-controller = { claudeCodePackage = lib.mkOption { type = lib.types.nullOr lib.types.package; @@ -351,23 +350,22 @@ What travels into the container is the **store path**, not the derivation: agents share the host's `/nix/store`, so the binary and its full closure are already reachable there — nothing - needs rebuilding or copying. hive-c0re hands the path to each - container at start as `HYPERHIVE_CLAUDE_CODE_PATH`, and the agent's - `claude` on PATH runs it, or the container's own `claude-code` when - the variable is absent or names nothing. A change reaches an agent - at its next container start, not through a rebuild. + needs rebuilding or copying. hive-c0re writes the path into + each agent's generated flake as a plain string literal (a bare + path fed to `lib.types.package` would run `builtins.storePath`, + which is illegal under pure evaluation) and the agent module + puts its `bin/` on the harness's PATH. The flip side of a plain string is that nothing in the agent's own closure refers to it, so the container cannot keep it alive. The **host** does that instead: this package is interpolated into `/etc/hyperhive/serve.json`, which puts it in - the host's system closure, so it is gc-rooted by the running - generation. ⚠️ An agent started under an older generation keeps - that generation's path until its next start, and only the old - generation roots it: collect old generations after restarting - the agents, or an agent still running the old path falls back to - its own `claude-code` at its next `claude` spawn, saying so on - stderr. + the host's system closure — so it is gc-rooted by the running + generation for exactly as long as that generation is the one + the agents were rendered from. The cost is that + `nix-collect-garbage` cannot reclaim an old `claude-code` until + every agent has been rebuilt past it and the old generations + are gone. The swarm's `swarm-claude-login` and `swarm-review-classifier` containers, where this host runs them, follow the same pin and @@ -379,11 +377,7 @@ dashboardPort = lib.mkOption { type = lib.types.port; default = 7000; - description = '' - TCP port this hive's hive-c0re dashboard listens on. Agents get it - at container start as `HIVE_DASHBOARD_PORT`, for their web UI's - links back to the dashboard. - ''; + description = "TCP port this hive's hive-c0re dashboard listens on."; }; }; } diff --git a/nix/host-modules/otel.nix b/nix/host-modules/otel.nix index e2379eb9..b162f8b8 100644 --- a/nix/host-modules/otel.nix +++ b/nix/host-modules/otel.nix @@ -236,7 +236,7 @@ in }; # Read by this hive's hive-c0re, which stamps it on its own export and - # hands it to its agents at container start, so it is a per-host choice. + # renders it into its agents', so it is a per-host choice. options.services.hyperhive.deploy.hive-controller.otel.extraResourceAttributes = lib.mkOption { type = lib.types.str; default = ""; diff --git a/nix/module-eval/agent-otel.nix b/nix/module-eval/agent-otel.nix index d92614a1..a9df766f 100644 --- a/nix/module-eval/agent-otel.nix +++ b/nix/module-eval/agent-otel.nix @@ -123,25 +123,6 @@ let && lib.all (e: s.exporters.${e}.endpoint == endpoint) s.service.pipelines.logs.exporters && !(agentNoOtel.systemd.globalEnvironment ? OTEL_EXPORTER_OTLP_ENDPOINT); } - { - # The hive's extra labels arrive at container start, so the build - # cannot hold the whole `OTEL_RESOURCE_ATTRIBUTES`: a boot-time - # generator computes it, and a unit-level `Environment=` copy would - # override that. The endpoint beside it is the presence control for the - # two absence lookups; `agentNoOtel` is the control for the generator. - name = "an agent's OTEL resource labels come from a boot-time generator, not the build"; - ok = - let - generator = - m: m.environment.etc ? "systemd/system-environment-generators/60-hyperhive-otel-resource"; - in - agentDefault.systemd.globalEnvironment ? OTEL_EXPORTER_OTLP_ENDPOINT - && !(agentDefault.systemd.globalEnvironment ? OTEL_RESOURCE_ATTRIBUTES) - && agentDefault.environment.variables ? OTEL_EXPORTER_OTLP_ENDPOINT - && !(agentDefault.environment.variables ? OTEL_RESOURCE_ATTRIBUTES) - && generator agentDefault - && !(generator agentNoOtel); - } { # Presence control for the two cases above: with the switch off there # is no collector in the container at all, so their passing is about diff --git a/nix/module-eval/swarm-services-switch.nix b/nix/module-eval/swarm-services-switch.nix index 40d38dcb..5b319322 100644 --- a/nix/module-eval/swarm-services-switch.nix +++ b/nix/module-eval/swarm-services-switch.nix @@ -282,7 +282,7 @@ let { # The CLI is the one the agents run, not the container's own nixpkgs'. # Unpinned is the control: it falls back to the container's own `claude-code`. - # serve.json is what hive-c0re hands every agent's `HYPERHIVE_CLAUDE_CODE_PATH` from. + # serve.json is what hive-c0re renders every agent's `claudeCodePath` from. name = "the agents, the claude login minter and the review classifier run the claude-code this host pins"; ok = let diff --git a/nix/script-tests/agent-hive-facts.nix b/nix/script-tests/agent-hive-facts.nix index 560c705e..82c1a2c1 100644 --- a/nix/script-tests/agent-hive-facts.nix +++ b/nix/script-tests/agent-hive-facts.nix @@ -1,9 +1,7 @@ # `checks.script-test-agent-hive-facts` — runs the two readers of # `/etc/hyperhive-hive-facts` in ../agent-modules/hive-facts.nix: the systemd # environment generator and the login-shell snippet `/etc/set-environment` -# sources. Also the two consumers of what they pass on: ../agent-modules/otel.nix's -# `OTEL_RESOURCE_ATTRIBUTES` generator and ../agent-modules/default.nix's `claude`. -# The cases are in ./agent-hive-facts.sh. +# sources. The cases are in ./agent-hive-facts.sh. # # What runs is the rendered generator and snippet. The one edit is the facts # file's path, moved under the build directory, and the snippet's reference to @@ -28,11 +26,6 @@ let ; machine = agentWith { services.hyperhive.agent.user.name = "a1"; }; - - # As text: building it would build the claude-code it falls back to. - claude = lib.findFirst ( - p: (p.name or "") == "claude" - ) (throw "no `claude` in systemPackages") machine.environment.systemPackages; in pkgs.runCommand "hyperhive-script-test-agent-hive-facts" { @@ -40,9 +33,6 @@ pkgs.runCommand "hyperhive-script-test-agent-hive-facts" machine.environment.etc."systemd/system-environment-generators/50-hyperhive-hive-facts".source; PROFILE = machine.environment.etc."hyperhive/hive-facts.sh".source; EXTRA_INIT = machine.environment.extraInit; - OTEL_GENERATOR = - machine.environment.etc."systemd/system-environment-generators/60-hyperhive-otel-resource".source; - CLAUDE_WRAPPER = builtins.unsafeDiscardStringContext claude.text; SHELLS = "${pkgs.dash}/bin/dash ${pkgs.bash}/bin/sh"; } '' diff --git a/nix/script-tests/agent-hive-facts.sh b/nix/script-tests/agent-hive-facts.sh index 58f57ac7..cddb1bf7 100644 --- a/nix/script-tests/agent-hive-facts.sh +++ b/nix/script-tests/agent-hive-facts.sh @@ -40,7 +40,7 @@ check() { for sh in $SHELLS; do # Only what the snippet exported reaches the child's environment. No PATH: # the snippet must not need one. - got=$(env -i "$sh" -c ". '$dir/profile'; exec '$envbin'" | grep -E '^(HYPERHIVE_|HIVE_DASHBOARD_PORT=|CLAUDE_|LD_PRELOAD=|__hyperhive)' | sort) + got=$(env -i "$sh" -c ". '$dir/profile'; exec '$envbin'" | grep -E '^(HYPERHIVE_|CLAUDE_|LD_PRELOAD=|__hyperhive)' | sort) if [ "$got" != "$(printf '%s\n' "$want" | sort)" ]; then fail "$sh: a child of the sourcing shell saw:"$'\n'"$got" fi @@ -61,73 +61,11 @@ check "a last line without a newline is kept" "HYPERHIVE_HIVE_DOMAIN=pr1ma.examp HYPERHIVE_HIVE_NAME=pr1ma CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=pr1ma-a1" -printf 'HIVE_DASHBOARD_PORT=7123\nHYPERHIVE_CLAUDE_CODE_PATH=/nix/store/0abc-claude-code\nHYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES=team=a,env=prod\n' >"$facts" -check "the per-host values, a value holding = kept whole" "HIVE_DASHBOARD_PORT=7123 -HYPERHIVE_CLAUDE_CODE_PATH=/nix/store/0abc-claude-code -HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES=team=a,env=prod -CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=unknown-a1" - : >"$facts" check "an empty file (no facts)" "CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=unknown-a1" rm -f "$facts" check "no file" "CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=unknown-a1" -# OTEL_RESOURCE_ATTRIBUTES, for units (the generator, run with what the -# hive-facts generator printed) and for login shells (the extraInit line, after -# the snippet). The base labels come from the build; the hive's are appended. -base="service.name=hyperhive-agent,agent=a1,swarm=unknown" -otel_check() { - case=$1 - count=$((count + 1)) - local want="OTEL_RESOURCE_ATTRIBUTES=$2" got sh passed - mapfile -t passed < <(env -i "$dir/generator") - got=$(env -i "${passed[@]}" "$OTEL_GENERATOR") - if [ "$got" != "$want" ]; then fail "generator printed: $got, wanted: $want"; fi - for sh in $SHELLS; do - got=$(env -i "$sh" -c ". '$dir/profile'; $otel_init; exec '$envbin'" | grep '^OTEL_RESOURCE_ATTRIBUTES=') - if [ "$got" != "$want" ]; then fail "$sh: a login shell saw: $got"; fi - done -} - -case="otel setup" -otel_init=$(printf '%s\n' "$EXTRA_INIT" | grep -F "$OTEL_GENERATOR") -if [ -z "$otel_init" ]; then fail "/etc/set-environment does not run the OTEL resource generator"; fi -if [ "$(printf '%s\n' "$EXTRA_INIT" | grep -n . | grep -F "$OTEL_GENERATOR" | cut -d: -f1)" -lt \ - "$(printf '%s\n' "$EXTRA_INIT" | grep -nxF '. /etc/hyperhive/hive-facts.sh' | cut -d: -f1)" ]; then - fail "/etc/set-environment runs the OTEL resource generator before the hive-facts snippet" -fi - -printf 'HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES=team=a,env=prod\n' >"$facts" -otel_check "the hive's labels are appended" "$base,team=a,env=prod" -rm -f "$facts" -otel_check "no hive labels: the base alone, no trailing comma" "$base" - -# `claude`: the hive's pinned build when it is there, this container's own -# otherwise. Both builds are stand-ins that print which one ran. -mkdir -p "$dir/own/bin" "$dir/pinned/bin" -printf '#!%s\necho own "$@"\n' "$BASH" >"$dir/own/bin/claude" -printf '#!%s\necho pinned "$@"\n' "$BASH" >"$dir/pinned/bin/claude" -chmod +x "$dir/own/bin/claude" "$dir/pinned/bin/claude" -printf '%s' "$CLAUDE_WRAPPER" | sed -E "s|/nix/store/[^ ]+/bin/claude|$dir/own/bin/claude|" >"$dir/claude" -case="claude setup" -if ! grep -qF "$dir/own/bin/claude" "$dir/claude"; then fail "the rewrite did not reach the fallback claude"; fi -# claude_check -claude_check() { - case=$1 - count=$((count + 1)) - local got said pin=() - if [ -n "$2" ]; then pin=("HYPERHIVE_CLAUDE_CODE_PATH=$2"); fi - # Under this shell: the shebang names the build's bash, not in this sandbox. - got=$(env -i "${pin[@]}" "$BASH" "$dir/claude" -p hi 2>"$dir/stderr") - said=$(cat "$dir/stderr") - if [ "$got" != "$3 -p hi" ]; then fail "ran: $got, wanted: $3 -p hi"; fi - if [ "$said" != "$4" ]; then fail "said on stderr: '$said', wanted: '$4'"; fi -} -claude_check "the pinned claude when the hive hands one over, silently" "$dir/pinned" pinned "" -claude_check "the container's own claude with no pin, silently" "" own "" -claude_check "the container's own claude when the pinned path is gone, saying so" "$dir/gone" own \ - "claude: $dir/gone/bin/claude, the hive's pinned claude-code, is missing; running this container's own" - echo "$count cases, $failures failed" [ "$failures" -eq 0 ]