vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

Compare commits

base: vinzenz:9b1fe857065255bf9fcef923b6ca6f01655f0bbc
Branches Tags
vinzenz:main
..
compare: vinzenz:a32c144b80d17ee42b6bb1f0037e76bdbeb6edc8
Branches Tags
vinzenz:main
23 changed files with 335 additions and 418 deletions
Download patch file Download diff file Expand all files Collapse all files

8
docs/agent-lifecycle/config-changes.md
View file

@ -118,10 +118,7 @@ declares one flake input per agent (`agent-<n>.url =
"git+<forge>/agent-configs/<n>.git?rev=<sha>"`) and one
`nixosConfigurations.<n>` output per agent. Each output wraps
`inputs.agent-<n>.nixosModules.default` with the identity +
`HIVE_PORT` / `HIVE_LABEL` injection module. Values that differ per host
(the dashboard port, the pinned claude-code, extra OTEL labels) aren't in it:
`hive-priv` writes them to the container's `/etc/hyperhive-hive-facts` at
start (see `docs/process/conventions.md`).
`HIVE_PORT` / `HIVE_LABEL` / `HIVE_DASHBOARD_PORT` injection module.
Containers run against `--flake /var/lib/hyperhive/meta#<n>`.
The declared input url is the head of `main` on the agent's forge config repo
@ -198,7 +195,8 @@ The container's `--flake` ref is `/var/lib/hyperhive/meta#<name>`
(see "Meta flake" above). The agent's own `applied/<n>/flake.nix`
is a fixed boilerplate that exports `nixosModules.default =
import ./agent.nix`; the meta flake imports that module and
wraps it with identity + `HIVE_PORT` / `HIVE_LABEL`.
wraps it with identity + `HIVE_PORT` / `HIVE_LABEL` /
`HIVE_DASHBOARD_PORT`.
### Tag state machine

14
docs/process/conventions.md
View file

@ -39,15 +39,11 @@ Four env vars cover the identity surface, read by
name.
Inside an agent container, the hive's name and domain don't come from the
agent's build, and neither do the hive's other per-host values: the dashboard
port (`HIVE_DASHBOARD_PORT`), the pinned claude-code
(`HYPERHIVE_CLAUDE_CODE_PATH`) and extra OTEL labels
(`HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`). `hive-priv` writes them to
`/etc/hyperhive-hive-facts` before every container start, leaving out any the
hive doesn't set; the agent then uses its own default. The agent's environment
generator (`nix/agent-modules/hive-facts.nix`) passes them to every unit,
including `hivectl choom` sessions, and `/etc/set-environment` exports them in login
shells that no unit starts (`nixos-container root-login`). None is part of
agent's build. `hive-priv` writes them to `/etc/hyperhive-hive-facts` before
every container start. The agent's environment generator
(`nix/agent-modules/hive-facts.nix`) passes them to every unit, including
`hivectl choom` sessions, and `/etc/set-environment` exports them in login
shells that no unit starts (`nixos-container root-login`). Neither is part of
an agent's closure, and an agent started on another hive reports that hive.
`hive_name` + `swarm_name` are **distinct** from

10
docs/process/gotchas.md
View file

@ -145,9 +145,9 @@ and every agent runs that build.
What crosses is the **store path**, not the derivation. Containers
share the host's `/nix/store`, so the binary is already reachable
inside them with its whole closure. `hive-priv` writes the path into each
container's `/etc/hyperhive-hive-facts` at start, and the agent's `claude`
wrapper on PATH runs its `bin/claude`. Two things rule out the natural-seeming alternatives: a
inside them with its whole closure; hive-c0re writes the path into each
agent's flake as a string literal and the agent module symlinks
`bin/claude` onto PATH. Two things rule out the natural-seeming alternatives: a
`path:/nix/store/<pkg>` flake input is re-copied into the store as a
reference-less `-source` (so the runtime closure never arrives), and
`lib.types.package` fed a bare path runs `builtins.storePath`, which
@ -171,7 +171,7 @@ expected.
<!-- vale write-good.Passive = NO -->
The catch is that a path handed to the container at start is text, not a
The catch is that a path written into a generated flake is text, not a
reference — the container's closure doesn't keep the binary alive.
The **host** does: it interpolates the package into
`/etc/hyperhive/serve.json`, so it lands in the host's system closure
@ -180,7 +180,7 @@ string context, which is the load-bearing detail; discard the context
anywhere on that path and `nix-collect-garbage` will eventually take
the hive's `claude` out from under it. The price of the root is that
`nix-collect-garbage` can't reclaim an old `claude-code` until every
agent has restarted past it and the old generations are gone.
agent has rebuilt past it and the old generations are gone.
<!-- vale write-good.Passive = YES -->

7
docs/scheduler/observability.md
View file

@ -190,7 +190,7 @@ two-tier architecture, the security model, and how the options interact.
The harness sets the OTLP variables (`OTEL_EXPORTER_OTLP_ENDPOINT`, `_PROTOCOL`,
`OTEL_RESOURCE_ATTRIBUTES`, the temporality preference) **container
wide**, for every unit and in `/etc/profile`, so every
wide** — in systemd's `DefaultEnvironment` and in `/etc/profile` — so every
process in an agent container exports to the hive's collector without any
per-tool wiring. That covers Claude Code, `hive-metric`, and anything you run
yourself from a tool call or a shell you open in the container.
@ -209,9 +209,8 @@ the push (see [Authenticated ingest](#authenticated-ingest)). The container
doesn't set it.
Append additional labels for one hive via `extraResourceAttributes` (see
option reference above). Agents get them at container start, so a change
reaches an agent at its next start, not through a rebuild. Pass custom
per-data-point labels with `hive-metric --labels` (see below).
option reference above); pass custom per-data-point labels with
`hive-metric --labels` (see below).
## Log severity

12
docs/turn-loop/claude-invocation.md
View file

@ -33,13 +33,11 @@ harness unit's PATH. By default that's the `claude-code` in the agent's
own nixpkgs (the meta flake's `nixpkgs` input) via
`environment.systemPackages`. Since that's typically a release channel and
this package moves fast, the operator can pin one per hive with
`services.hyperhive.deploy.hive-controller.claudeCodePackage`. `hive-priv`
writes its store path into the container's `/etc/hyperhive-hive-facts` as
`HYPERHIVE_CLAUDE_CODE_PATH` before every start. `claude` on PATH is a wrapper
that runs that build when the path exists, and the container's own
`claude-code` when the variable is unset or the path no longer exists, saying
so on stderr in the second case. A bump reaches an agent at its next container
start, not at a rebuild. See docs/process/gotchas.md::`claude-code` is unfree.
`services.hyperhive.deploy.hive-controller.claudeCodePackage`: hive-c0re writes its
store path into each agent's flake, and `claude` on PATH becomes a symlink to it
instead of the container's own `claude-code` — so there's only ever one
`claude` in the container. Agents pick up a new build on their
next rebuild, not live. See docs/process/gotchas.md::`claude-code` is unfree.
Hive-enforced settings ship at `/etc/claude-code/managed-settings.json`
(claude-code's canonical managed-settings path — precedence #1,

12
hive-c0re/src/coordinator.rs
View file

@ -43,8 +43,8 @@ pub struct Coordinator {
/// Git rev of the hyperhive checkout this host runs. Pins the meta
/// flake's `hyperhive` input; `None` makes the meta render fail.
pub hyperhive_rev: Option<String>,
/// Store path of the `claude-code` build every agent runs, handed to
/// each container at start as `HYPERHIVE_CLAUDE_CODE_PATH`. Set by
/// Store path of the `claude-code` build every agent runs, written
/// into each per-agent flake as `services.hyperhive.agent.claudeCodePath`. Set by
/// the NixOS module option
/// `services.hyperhive.deploy.hive-controller.claudeCodePackage`
/// (which resolves the package and hands us its path). `None` = every
@ -55,9 +55,9 @@ pub struct Coordinator {
/// closure and all. The host module is what keeps it from being
/// garbage-collected — see that option.
pub claude_code_path: Option<String>,
/// TCP port the host's hive-c0re dashboard listens on. Handed to each
/// container at start as `HIVE_DASHBOARD_PORT`, so the agent's web UI
/// can build the rebuild-button URL pointing back at the dashboard.
/// TCP port the host's hive-c0re dashboard listens on. Inlined into
/// each per-agent flake so the agent's web UI can build the right
/// rebuild-button URL pointing back at the dashboard.
pub dashboard_port: u16,
/// Per-agent systemd `CPUQuota=` value (e.g. `"200%"`). Written into
/// the `container@h-<name>.service.d/` drop-in on every spawn/rebuild.
@ -193,7 +193,7 @@ pub struct HiveEnv {
pub hyperhive_rev: Option<String>,
/// Store path of the `claude-code` agents run, or `None` for "each
/// agent keeps the one out of its own nixpkgs". Travels into the
/// container as `HYPERHIVE_CLAUDE_CODE_PATH` — a plain string, kept
/// container as `services.hyperhive.agent.claudeCodePath` — a plain string, kept
/// alive host-side by the module that resolved it.
pub claude_code_path: Option<String>,
pub dashboard_port: u16,

5
hive-c0re/src/lifecycle/host_config.rs
View file

@ -25,7 +25,7 @@ use super::{
pub async fn write_dropins(name: &str, hive: &HiveEnv, paths: &AgentPaths) -> Result<()> {
validate(name)?;
let container = container_name(name);
set_nspawn_flags(&container, hive, &paths.agent, &paths.claude, &paths.notes).await?;
set_nspawn_flags(&container, &paths.agent, &paths.claude, &paths.notes).await?;
hand_permissions(name, &paths.agent);
if let Some((cpu_quota, memory_max)) = stage_limits(
name,
@ -496,7 +496,6 @@ fn queue_agent_credentials(agent_name: &str, dir: Option<&Path>) -> Vec<Credenti
)]
async fn set_nspawn_flags(
container: &str,
hive: &HiveEnv,
runtime_dir: &Path,
claude_dir: &Path,
notes_dir: &Path,
@ -670,7 +669,7 @@ async fn set_nspawn_flags(
&binds,
isolation,
&load_creds,
super::hive_facts(hive),
super::hive_facts_from_env(),
)
.await
}

23
hive-c0re/src/lifecycle/mod.rs
View file

@ -169,27 +169,14 @@ pub fn network_isolation_from_env() -> Result<hive_priv_sock::NetworkIsolation>
network_isolation_from_vars(bridge.as_deref(), subnet.as_deref())
}
/// The hive's facts every container is handed at runtime, so that no agent
/// build carries a value that differs between hosts. The name, domain and
/// OTEL resource labels come from hive-c0re's own unit environment
/// (`nix/host-modules/hive-c0re/environment.nix`), where an unset or empty
/// variable is `None`; the dashboard port and pinned claude from `hive`.
fn hive_facts(hive: &HiveEnv) -> hive_priv_sock::HiveFacts {
hive_facts_from(hive, |name| std::env::var(name).ok())
}
/// [`hive_facts`] over a given environment lookup, for tests.
fn hive_facts_from(
hive: &HiveEnv,
var: impl Fn(&str) -> Option<String>,
) -> hive_priv_sock::HiveFacts {
let read = |name: &str| var(name).filter(|v| !v.is_empty());
/// The hive's facts every container is handed at runtime, from hive-c0re's
/// own unit environment (`nix/host-modules/hive-c0re/environment.nix`). An
/// unset or empty variable is `None`.
fn hive_facts_from_env() -> hive_priv_sock::HiveFacts {
let read = |name: &str| std::env::var(name).ok().filter(|v| !v.is_empty());
hive_priv_sock::HiveFacts {
name: read("HYPERHIVE_HIVE_NAME"),
domain: read("HYPERHIVE_HIVE_DOMAIN"),
dashboard_port: Some(hive.dashboard_port),
claude_code_path: hive.claude_code_path.clone(),
otel_extra_resource_attributes: read("HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES"),
}
}

36
hive-c0re/src/lifecycle/tests.rs
View file

@ -527,39 +527,3 @@ async fn run_with_deadline_returns_output_of_a_fast_child() {
.expect("echo finishes");
assert_eq!(String::from_utf8_lossy(&out.stdout).trim(), "hi");
}
/// The three per-host values reach the container as facts: the dashboard
/// port and pinned claude from the hive's config, the OTEL labels from the
/// daemon's environment.
#[test]
fn hive_facts_carry_the_per_host_values() {
let hive = HiveEnv {
dashboard_port: 7123,
claude_code_path: Some("/nix/store/0abc-claude-code".to_owned()),
..HiveEnv::default()
};
let facts = hive_facts_from(&hive, |name| {
(name == "HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES").then(|| "team=a".to_owned())
});
assert_eq!(facts.dashboard_port, Some(7123));
assert_eq!(
facts.claude_code_path.as_deref(),
Some("/nix/store/0abc-claude-code")
);
assert_eq!(
facts.otel_extra_resource_attributes.as_deref(),
Some("team=a")
);
}
/// Unset and empty both leave a fact out, so the agent falls back to its
/// own default rather than reading an empty value.
#[test]
fn hive_facts_leave_unset_values_out() {
let facts = hive_facts_from(&HiveEnv::default(), |name| {
(name == "HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES").then(String::new)
});
assert_eq!(facts.claude_code_path, None);
assert_eq!(facts.otel_extra_resource_attributes, None);
assert_eq!(facts.name, None);
}

212
hive-c0re/src/meta.rs
View file

@ -431,12 +431,23 @@ fn pinned_git_url(forge_git_url: &str, rev: Option<&str>) -> Result<String> {
Ok(format!("git+{forge_git_url}?rev={rev}"))
}
/// [`render_flake`] with the hyperhive pin taken from `hive`, as
/// [`sync_agents`] renders it. Nothing else in `hive` reaches the flake: the
/// values that differ per host go to the container at start
/// (`lifecycle::hive_facts`).
/// [`render_flake`] with its inputs taken from `hive`, as [`sync_agents`]
/// renders it.
fn render_hive_flake(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<String> {
render_flake(&hyperhive_input_url(hive)?, agents)
render_hive_flake_at(hive, &hyperhive_input_url(hive)?, agents)
}
fn render_hive_flake_at(
hive: &HiveEnv,
hyperhive_url: &str,
agents: &[AgentSpec],
) -> Result<String> {
render_flake(
hyperhive_url,
hive.claude_code_path.as_deref(),
hive.dashboard_port,
agents,
)
}
/// Canonical inputs meta knows how to dedup. An agent that declares one
@ -618,6 +629,19 @@ fn forwarded_env_vars() -> Vec<(&'static str, String)> {
.collect()
}
/// Extra `OTEL_RESOURCE_ATTRIBUTES` entries for every agent, read off
/// hive-c0re's own unit env
/// (`services.hyperhive.deploy.hive-controller.otel.extraResourceAttributes`).
/// `None` when unset or empty. This is the only OTEL value the renderer
/// writes into the flake: agents export to the hive collector by default and
/// find it by name, so there is no enable flag, endpoint or protocol to
/// carry.
fn otel_extra_resource_attributes() -> Option<String> {
std::env::var("HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES")
.ok()
.filter(|v| !v.is_empty())
}
/// Remove the CA files earlier renders embedded next to `flake.nix`
/// (`hive-ca.pem`, and the per-peer `peer-ca-<N>.pem` before it), returning
/// every filename removed so the caller stages the deletion. Nothing writes
@ -740,7 +764,12 @@ fn canonical_inputs_in_lock(raw: &str) -> Vec<&'static str> {
reason = "templated string-builder for the meta flake — the length is one \
contiguous fmt block, splitting it would just hide the shape"
)]
fn render_flake(hyperhive_url: &str, agents: &[AgentSpec]) -> Result<String> {
fn render_flake(
hyperhive_url: &str,
claude_code_path: Option<&str>,
dashboard_port: u16,
agents: &[AgentSpec],
) -> Result<String> {
use std::fmt::Write as _;
let mut out = String::new();
out.push_str("{\n description = \"hyperhive deployed agents\";\n inputs = {\n");
@ -787,8 +816,10 @@ fn render_flake(hyperhive_url: &str, agents: &[AgentSpec]) -> Result<String> {
);
}
}
out.push_str(
" };\n outputs =\n { self, hyperhive, ... }@inputs:\n let\n mkAgent = { name, port, archetype ? null }:\n",
out.push_str(" };\n outputs =\n { self, hyperhive, ... }@inputs:\n let\n");
let _ = writeln!(
out,
" dashboardPort = {dashboard_port};\n mkAgent = {{ name, port, archetype ? null }}:"
);
out.push_str(
r#" let
@ -811,6 +842,32 @@ fn render_flake(hyperhive_url: &str, agents: &[AgentSpec]) -> Result<String> {
{
"#,
);
// The `claude-code` agents run, as a bare store path rather than a
// flake input: containers share the host's `/nix/store`, so the
// binary is already there with its closure and has nothing to
// travel. A string literal is also the only shape that evaluates —
// `lib.types.package` on a bare path runs `builtins.storePath`,
// which pure eval rejects. The agent module puts its `bin/` on the
// harness PATH; the host module holds the gc root, since a path
// spelled out here is text and references nothing.
// `None` = no override; agents keep their own nixpkgs' `claude-code`.
if let Some(path) = claude_code_path {
let _ = writeln!(
out,
" services.hyperhive.agent.claudeCodePath = \"{path}\";"
);
}
// Extra OTEL resource labels (`deploy.hive-controller.otel.*`) are the one
// OTEL value rendered here; enablement and the collector endpoint are
// agent-module defaults.
if let Some(attrs) = otel_extra_resource_attributes() {
let esc = |s: &str| s.replace('\\', "\\\\").replace('"', "\\\"");
let _ = writeln!(
out,
" services.hyperhive.agent.otel.extraResourceAttributes = \"{}\";",
esc(&attrs)
);
}
// Agent-facing service URLs (`services.hyperhive.agent.forge.url`, `services.hyperhive.agent.matrix.url`):
// emit the host's real values as build-time agent config, the same
// host-state -> agent-module shape as the CA block above.
@ -902,6 +959,7 @@ fn render_flake(hyperhive_url: &str, agents: &[AgentSpec]) -> Result<String> {
systemd.services.${service}.environment = {
HIVE_PORT = toString port;
HIVE_LABEL = name;
HIVE_DASHBOARD_PORT = toString dashboardPort;
",
);
// Forwarded env vars — picked up from hive-c0re's own systemd unit
@ -1224,8 +1282,13 @@ mod tests {
}
}
fn render(hyperhive_url: &str, agents: &[AgentSpec]) -> String {
render_flake(hyperhive_url, agents).expect("render")
fn render(
hyperhive_url: &str,
claude_code_path: Option<&str>,
dashboard_port: u16,
agents: &[AgentSpec],
) -> String {
render_flake(hyperhive_url, claude_code_path, dashboard_port, agents).expect("render")
}
#[test]
@ -1246,6 +1309,8 @@ mod tests {
fn the_input_a_rebuild_relocks_is_a_pinned_forge_url() {
let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
&[sample_spec("iris", 9001)],
);
let url = out
@ -1279,6 +1344,8 @@ mod tests {
fn render_flake_pins_hyperhive_by_url_and_follows_its_nixpkgs() {
let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
&[sample_spec("alice", 9001)],
);
assert!(
@ -1307,6 +1374,8 @@ mod tests {
fn render_flake_names_each_agent_container_after_the_agent() {
let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
&[sample_spec("alice", 9001)],
);
assert!(
@ -1332,6 +1401,49 @@ mod tests {
);
}
#[test]
fn render_flake_pins_claude_path_without_adding_an_input() {
// The host-pinned claude travels as a bare store path assigned to
// an option — deliberately NOT as a flake input. Containers share
// the host store, so the build is already reachable; making it an
// input would re-copy it as a reference-less `-source` and strip
// the closure the binary actually needs.
let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
Some("/nix/store/cccc-claude-code-2.1.220"),
8000,
&[sample_spec("alice", 9001)],
);
assert!(
out.contains(
"services.hyperhive.agent.claudeCodePath = \"/nix/store/cccc-claude-code-2.1.220\";"
),
"claude path assigned as a plain string literal:\n{out}"
);
assert!(
!out.contains("claude-code-2.1.220\".url"),
"the pinned claude must not become a flake input:\n{out}"
);
}
#[test]
fn render_flake_omits_claude_path_when_unset() {
// `None` = no host-level pin: the option is left undefined so the
// agent module keeps its own nixpkgs' `claude-code` (and keeps it
// in `environment.systemPackages`, which is what makes the
// unpinned case self-contained).
let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
&[sample_spec("alice", 9001)],
);
assert!(
!out.contains("claudeCodePath"),
"no claude assignment when unpinned:\n{out}"
);
}
#[test]
fn render_flake_emits_follows_for_agents_declaring_nixpkgs() {
// `bitburner` + `dmatrix` declare `nixpkgs` at their root, while
@ -1342,6 +1454,8 @@ mod tests {
};
let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
&[
sample_spec("argus", 9001),
with_nixpkgs("bitburner", 9002),
@ -1368,6 +1482,8 @@ mod tests {
fn render_flake_builds_every_agent_from_agent_base() {
let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
&[sample_spec("alice", 9001), sample_spec("ruth", 9002)],
);
assert_eq!(
@ -1391,6 +1507,8 @@ mod tests {
fn render_flake_skips_canonical_follows_when_the_lock_declares_none() {
let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
&[sample_spec("alice", 9001)],
);
// No agent-side follows when the lock declares nothing
@ -1420,6 +1538,8 @@ mod tests {
};
let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
&[alice, android],
);
assert!(
@ -1468,6 +1588,8 @@ mod tests {
}
let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
&[sample_spec("alice", 9001)],
);
unsafe {
@ -1507,6 +1629,8 @@ mod tests {
}
render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
&[sample_spec("alice", 9001)],
)
};
@ -1700,6 +1824,8 @@ mod tests {
}
let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
&[sample_spec("alice", 9001)],
);
unsafe {
@ -1729,6 +1855,8 @@ mod tests {
};
let out = render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
&[at("alice", 9001, "aaaa"), at("bob", 9002, "bbbb")],
);
assert!(
@ -1754,6 +1882,8 @@ mod tests {
let args = |agents: &[AgentSpec]| {
render_flake(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
agents,
)
};
@ -1908,6 +2038,8 @@ mod tests {
let render = || {
render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
&[sample_spec("alice", 9001)],
)
};
@ -1968,12 +2100,11 @@ mod tests {
}
#[test]
fn render_flake_carries_no_otel_config_and_no_per_host_value() {
fn render_flake_carries_no_otel_enablement() {
// The agent module defaults otel on and finds the collector by name, so
// the flake carries no enable, endpoint or protocol line however the
// host's OTEL env is set. The resource labels, dashboard port and
// pinned claude differ per host and reach the agent at container start
// (`lifecycle::hive_facts`), so none of them is rendered either.
// host's OTEL env is set. Only the per-hive resource labels are
// rendered, and only when set.
//
// Serialised against every other env-mutating test in the crate.
// `stats::otel_metrics::tests` perturbs OTEL env too, and lands in this
@ -1984,6 +2115,8 @@ mod tests {
let render = || {
render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
&[sample_spec("alice", 9001)],
)
};
@ -2006,11 +2139,10 @@ mod tests {
std::env::remove_var("HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES");
}
// Presence controls: the render is the real flake, carrying the pin and
// the harness env block the dashboard port used to sit in.
// Presence control: the render is the real flake, carrying the pin.
assert!(
plain.contains("rev=0123abcd") && plain.contains("HIVE_PORT = toString port;"),
"the hyperhive pin and the harness env must be rendered:\n{plain}"
plain.contains("rev=0123abcd"),
"the hyperhive pin line must be rendered:\n{plain}"
);
for (name, flake) in [("plain", &plain), ("labelled", &labelled)] {
for needle in [
@ -2018,11 +2150,6 @@ mod tests {
"agent.otel.endpoint",
"agent.otel.protocol",
"c.example",
"extraResourceAttributes",
"deployment.environment",
"claudeCodePath",
"dashboardPort",
"HIVE_DASHBOARD_PORT",
] {
assert!(
!flake.contains(needle),
@ -2030,6 +2157,16 @@ mod tests {
);
}
}
assert!(
!plain.contains("agent.otel.extraResourceAttributes"),
"extraResourceAttributes must not appear when unset:\n{plain}"
);
assert!(
labelled.contains(
"services.hyperhive.agent.otel.extraResourceAttributes = \"deployment.environment=prod\";"
),
"extraResourceAttributes must be rendered when set:\n{labelled}"
);
}
#[test]
fn render_flake_injects_github_disable_only_when_signalled() {
@ -2044,6 +2181,8 @@ mod tests {
let render = || {
render(
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
None,
8000,
&[sample_spec("alice", 9001)],
)
};
@ -2067,6 +2206,33 @@ mod tests {
"github.enable = false must be injected when the host disables it:\n{disabled}"
);
}
/// The agent reads its memory cap at boot from the copy
/// `lifecycle::write_dropins` hands it, so the cap is no build input.
#[test]
fn render_hive_flake_is_identical_across_memory_caps() {
let hive = |cap: &str| HiveEnv {
agent_memory_max: cap.to_owned(),
..HiveEnv::default()
};
let agents = [sample_spec("alice", 9001)];
let small = render_hive_flake_at(
&hive("4G"),
"git+https://forge.example/h.git?rev=abc",
&agents,
)
.expect("render");
let large = render_hive_flake_at(
&hive("16G"),
"git+https://forge.example/h.git?rev=abc",
&agents,
)
.expect("render");
assert_eq!(
small, large,
"the rendered flake must not depend on the memory cap"
);
}
}
#[cfg(test)]

4
hive-c0re/src/test_env.rs
View file

@ -11,8 +11,8 @@
//! ⚠️ **The lock has to be crate-wide, not per-module.** `hive-bash-mcp`'s
//! sibling helper records why: two separate per-module mutexes serialise
//! nothing against each other, and that produced a CI-only flake. The pair that
//! motivated this one is `meta::tests` (which renders a flake with
//! `HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES` set) and `stats::otel_metrics::tests` (which asserts which
//! motivated this one is `meta::tests` (which renders a flake from
//! `HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`) and `stats::otel_metrics::tests` (which asserts which
//! variable enables the exporter) — different modules, same process env, same
//! binary.
//!

9
hive-priv-sock/src/lib.rs
View file

@ -187,15 +187,6 @@ pub struct HiveFacts {
pub name: Option<String>,
/// `HYPERHIVE_HIVE_DOMAIN`: the hive's DNS domain.
pub domain: Option<String>,
/// `HIVE_DASHBOARD_PORT`: the port this hive's dashboard listens on.
pub dashboard_port: Option<u16>,
/// `HYPERHIVE_CLAUDE_CODE_PATH`: store path of the `claude-code` the
/// hive pins for its agents. The host's system closure keeps it alive;
/// the container only gets the path.
pub claude_code_path: Option<String>,
/// `HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`: comma-separated `k=v`
/// entries the agent appends to its `OTEL_RESOURCE_ATTRIBUTES`.
pub otel_extra_resource_attributes: Option<String>,
}
/// Wall-clock bound hive-priv puts on one [`PrivRequest::MeasureAgentDisk`]

123
hive-priv/src/container_config.rs
View file

@ -367,54 +367,24 @@ fn container_rootfs(container: &str) -> PathBuf {
/// Render [`HiveFacts`] as the systemd environment file the agent loads,
/// one `KEY=value` line per present fact.
///
/// Each value is refused, never escaped, unless it is made only of the
/// characters its fact can hold. None of those sets admits a newline (a
/// second assignment), whitespace, a quote, `\` or `$`: the agent hands each
/// line to systemd and its shells, which would read those as syntax.
/// A value is refused unless it is DNS-name characters only, so it can carry
/// neither a newline (a second assignment) nor anything an environment file
/// or a shell would interpret.
fn render_hive_facts(facts: &HiveFacts) -> Result<String> {
use std::fmt::Write as _;
let dns = |b: u8| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'-' | b'_');
let port = facts.dashboard_port.map(|p| p.to_string());
let mut out = String::new();
for (key, value, max_len, allowed) in [
(
"HYPERHIVE_HIVE_NAME",
facts.name.as_ref(),
253,
&dns as &dyn Fn(u8) -> bool,
),
("HYPERHIVE_HIVE_DOMAIN", facts.domain.as_ref(), 253, &dns),
("HIVE_DASHBOARD_PORT", port.as_ref(), 5, &|b: u8| {
b.is_ascii_digit()
}),
// A store path's name characters, plus the `/` of its prefix; the
// prefix itself is checked below.
(
"HYPERHIVE_CLAUDE_CODE_PATH",
facts.claude_code_path.as_ref(),
256,
&|b: u8| dns(b) || matches!(b, b'+' | b'?' | b'=' | b'/'),
),
// `k=v,k=v`, values percent-encoded as the OTEL spec has them.
(
"HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES",
facts.otel_extra_resource_attributes.as_ref(),
1024,
&|b: u8| dns(b) || matches!(b, b'=' | b',' | b'/' | b':' | b'%' | b'+' | b'@'),
),
for (key, value) in [
("HYPERHIVE_HIVE_NAME", &facts.name),
("HYPERHIVE_HIVE_DOMAIN", &facts.domain),
] {
let Some(value) = value else { continue };
if value.is_empty() || value.len() > max_len || !value.bytes().all(allowed) {
bail!(
"invalid {key} value {value:?}: must be 1-{max_len} chars of its fact's character set"
);
}
if key == "HYPERHIVE_CLAUDE_CODE_PATH"
&& !value
.strip_prefix("/nix/store/")
.is_some_and(|name| !name.is_empty() && !name.contains('/'))
if value.is_empty()
|| value.len() > 253
|| !value
.bytes()
.all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'-' | b'_'))
{
bail!("invalid {key} value {value:?}: must be a top-level /nix/store path");
bail!("invalid {key} value {value:?}: must be 1-253 chars of [A-Za-z0-9._-]");
}
let _ = writeln!(out, "{key}={value}");
}
@ -741,15 +711,14 @@ mod tests {
let both = HiveFacts {
name: Some("pr1ma".to_owned()),
domain: Some("pr1ma.darkest.space".to_owned()),
..HiveFacts::default()
};
assert_eq!(
render_hive_facts(&both).unwrap(),
"HYPERHIVE_HIVE_NAME=pr1ma\nHYPERHIVE_HIVE_DOMAIN=pr1ma.darkest.space\n"
);
let domain_only = HiveFacts {
name: None,
domain: Some("h1.example.test".to_owned()),
..HiveFacts::default()
};
assert_eq!(
render_hive_facts(&domain_only).unwrap(),
@ -765,66 +734,7 @@ mod tests {
for bad in ["", "pr1ma\nLD_PRELOAD=/x", "a b", "a\"b", "$(x)"] {
let facts = HiveFacts {
name: Some(bad.to_owned()),
..HiveFacts::default()
};
assert!(
render_hive_facts(&facts).is_err(),
"{bad:?} must be refused"
);
}
}
/// The three per-host values agents used to get at build time arrive
/// here, one line each.
#[test]
fn hive_facts_render_the_per_host_values() {
let facts = HiveFacts {
dashboard_port: Some(7123),
claude_code_path: Some("/nix/store/0abc-claude-code-2.1.220".to_owned()),
otel_extra_resource_attributes: Some(
"deployment.environment=prod,team=a%2Fb".to_owned(),
),
..HiveFacts::default()
};
assert_eq!(
render_hive_facts(&facts).unwrap(),
"HIVE_DASHBOARD_PORT=7123\n\
HYPERHIVE_CLAUDE_CODE_PATH=/nix/store/0abc-claude-code-2.1.220\n\
HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES=deployment.environment=prod,team=a%2Fb\n"
);
}
/// The claude path is a top-level store path or nothing, and neither
/// value may carry generator syntax.
#[test]
fn hive_facts_refuse_bad_claude_paths_and_resource_attributes() {
for bad in [
"/nix/store/",
"/nix/store/abc-claude/bin/claude",
"/tmp/claude",
"nix/store/abc-claude",
"/nix/store/abc-claude\nLD_PRELOAD=/x",
] {
let facts = HiveFacts {
claude_code_path: Some(bad.to_owned()),
..HiveFacts::default()
};
assert!(
render_hive_facts(&facts).is_err(),
"{bad:?} must be refused"
);
}
for bad in [
"a=b\nPATH=/x",
"a=${HOME}",
"a=b c",
"a=\"b\"",
"a=b\\c",
"",
] {
let facts = HiveFacts {
otel_extra_resource_attributes: Some(bad.to_owned()),
..HiveFacts::default()
domain: None,
};
assert!(
render_hive_facts(&facts).is_err(),
@ -842,7 +752,10 @@ mod tests {
bridge: "hive0".to_owned(),
gateway_ip: "10.42.0.1".to_owned(),
};
let facts = HiveFacts::default();
let facts = HiveFacts {
name: None,
domain: None,
};
for name in ["hive-forge", "hive-matrix"] {
let errs = [
handle_write_nspawn_flags(name, &[], &isolation, &[], &facts).unwrap_err(),

83
nix/agent-modules/default.nix
View file

@ -28,33 +28,6 @@ let
which the hive writes before every start, so no agent build sets it.
Remove the setting.
'';
claudeCodePathRemoved = ''
An agent reads the claude-code its hive pins at runtime, from
HYPERHIVE_CLAUDE_CODE_PATH in /etc/hyperhive-hive-facts, so no agent build
sets it. Pin it host-side with
services.hyperhive.deploy.hive-controller.claudeCodePackage.
'';
# `claude` on PATH. A hive that pins a claude-code
# (`services.hyperhive.deploy.hive-controller.claudeCodePackage`) hands its
# store path over at container start as `HYPERHIVE_CLAUDE_CODE_PATH`, and
# the host's system closure keeps it alive: nothing in this container
# references it. With no pin, or a path that is not there, this container's
# own claude-code runs; a pin that is set but gone (collected after a bump
# while this container still holds the old facts) says so on stderr, since
# the two builds can differ. `exec -a claude` keeps the bare name in argv[0],
# which is how the harness finds its claude child (`find_claude_child`).
ownClaude = import ../claude-code-no-alsa.nix lib pkgs.claude-code;
claude = pkgs.writeShellScriptBin "claude" ''
pinned=''${HYPERHIVE_CLAUDE_CODE_PATH:-}
if [ -n "$pinned" ] && [ -x "$pinned/bin/claude" ]; then
exec -a claude "$pinned/bin/claude" "$@"
fi
if [ -n "$pinned" ]; then
echo "claude: $pinned/bin/claude, the hive's pinned claude-code, is missing; running this container's own" >&2
fi
exec -a claude ${ownClaude}/bin/claude "$@"
'';
in
{
imports = [
@ -95,12 +68,6 @@ in
"agent"
"claudeMemoryMaxBytes"
] memoryMaxBytesRemoved)
(lib.mkRemovedOptionModule [
"services"
"hyperhive"
"agent"
"claudeCodePath"
] claudeCodePathRemoved)
];
options.services.hyperhive.agent.icon = lib.mkOption {
@ -120,6 +87,33 @@ in
'';
};
options.services.hyperhive.agent.claudeCodePath = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
example = "/nix/store/…-claude-code-2.1.220";
description = ''
Store path of the `claude-code` this agent runs, or `null` (the
default) to use the `claude-code` from the container's own
nixpkgs.
Set by the generated meta flake when the operator sets
`services.hyperhive.deploy.hive-controller.claudeCodePackage` host-side, so a hive
can run a claude built from a *different* nixpkgs than the one
its agents evaluate against — a release channel can trail
unstable by weeks on this one package. It arrives as a path and
not a package because agents share the host's `/nix/store`, so
the build is already reachable here with its whole closure and
has nothing to travel.
When set, `claude` on PATH is a symlink to this path's
`bin/claude` and the container's own `claude-code` is dropped, so
there is only ever one claude in the container. Note that neither
the symlink nor anything else in the container's closure *refers*
to the target — keeping it alive is the host's job, see
`services.hyperhive.deploy.hive-controller.claudeCodePackage`.
'';
};
config = {
assertions = [
# Guard the inputs-routed-as-output pattern: the agent flake.nix is
@ -246,7 +240,28 @@ in
config.services.hyperhive.agent.packages.hive-metric
]
++ [
claude
(
if config.services.hyperhive.agent.claudeCodePath == null then
import ../claude-code-no-alsa.nix lib pkgs.claude-code
else
# Host-pinned claude: a symlink farm around a path the
# container was handed as text. It has to be a derivation —
# `environment.systemPackages` coerces a store-path *string*
# with `toDerivation`, i.e. `builtins.storePath`, which pure
# evaluation rejects (`systemd.services.*.path` does the same,
# which is why the harness gets this via PATH like everything
# else rather than a unit-level entry). Interpolating the path
# into the builder is just text, so it evaluates anywhere.
#
# The symlink registers no store reference — the target isn't
# among this derivation's inputs, so nothing here keeps the
# binary alive. That is deliberate and it is the host's job:
# see `services.hyperhive.deploy.hive-controller.claudeCodePackage`.
pkgs.runCommandLocal "claude-code-pinned" { } ''
mkdir -p "$out/bin"
ln -s ${config.services.hyperhive.agent.claudeCodePath}/bin/claude "$out/bin/claude"
''
)
]
++ (with pkgs; [
bashInteractive

20
nix/agent-modules/hive-facts.nix
View file

@ -2,17 +2,8 @@
# agent's closure does not carry it.
#
# hive-priv writes `/etc/hyperhive-hive-facts` into the container before every
# start, one `KEY=value` line each:
#
# - `HYPERHIVE_HIVE_NAME`, `HYPERHIVE_HIVE_DOMAIN`;
# - `HIVE_DASHBOARD_PORT`, read by the harness's web UI (7000 when absent);
# - `HYPERHIVE_CLAUDE_CODE_PATH`, the claude-code the hive pins (`claude` in
# ./default.nix runs this container's own when absent);
# - `HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`, appended to
# `OTEL_RESOURCE_ATTRIBUTES` by ./otel.nix (no extra labels when absent).
#
# Any of them may be absent: a hive-c0re older than the agent's build writes
# fewer. Two readers hand the lines on:
# start (`HYPERHIVE_HIVE_NAME`, `HYPERHIVE_HIVE_DOMAIN`, one `KEY=value` line
# each). Two readers hand those lines on:
#
# - an environment generator, for the container's systemd: it passes them to
# every unit it starts, `machinectl shell` sessions (`hivectl choom`)
@ -32,8 +23,8 @@
let
userName = config.services.hyperhive.agent.user.name;
# Shell builtins only: nothing is assumed on PATH this early. Only the known
# keys are passed on. `|| [ -n "$key" ]` keeps a last line that has
# Shell builtins only: nothing is assumed on PATH this early. Only the two
# known keys are passed on. `|| [ -n "$key" ]` keeps a last line that has
# no trailing newline.
generator = pkgs.writeShellScript "hyperhive-hive-facts" ''
hive=unknown
@ -41,8 +32,7 @@ let
while IFS='=' read -r key value || [ -n "$key" ]; do
case "$key" in
HYPERHIVE_HIVE_NAME) hive=$value ;;
HYPERHIVE_HIVE_DOMAIN | HIVE_DASHBOARD_PORT) ;;
HYPERHIVE_CLAUDE_CODE_PATH | HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES) ;;
HYPERHIVE_HIVE_DOMAIN) ;;
*) continue ;;
esac
printf '%s=%s\n' "$key" "$value"

36
nix/agent-modules/otel.nix
View file

@ -42,17 +42,9 @@ let
# agent, and their metric names already tell them apart. No `hive`: the
# swarm collector stamps it from the receiver that accepted the hive's
# push (nix/host-modules/swarm-otel.nix), so the build names no hive.
resourceAttributes = "service.name=hyperhive-agent,agent=${userName},swarm=${swarmDisplayName}";
# Prints `OTEL_RESOURCE_ATTRIBUTES`: the labels above, then the hive's own
# (`HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`), which differ per host and so
# arrive at container start. That variable comes from ./hive-facts.nix's
# generator, which runs before this one and is absent when the hive sets no
# labels.
resourceGenerator = pkgs.writeShellScript "hyperhive-otel-resource" ''
printf 'OTEL_RESOURCE_ATTRIBUTES=%s%s\n' ${lib.escapeShellArg resourceAttributes} \
"''${HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES:+,$HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES}"
'';
resourceAttributes =
"service.name=hyperhive-agent,agent=${userName},swarm=${swarmDisplayName}"
+ lib.optionalString (cfg.extraResourceAttributes != "") ",${cfg.extraResourceAttributes}";
# The variables an OTEL SDK reads on its own, in any language, in any
# process here.
@ -70,6 +62,7 @@ let
# which Prometheus/Mimir-family backends (incl. grafana-lgtm)
# silently drop without a deltatocumulative processor.
OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE = "cumulative";
OTEL_RESOURCE_ATTRIBUTES = resourceAttributes;
};
# The exporter's NAME picks the wire protocol, so `protocol` has to be
@ -125,6 +118,19 @@ in
'';
};
extraResourceAttributes = lib.mkOption {
type = lib.types.str;
default = "";
internal = true;
description = ''
Extra comma-separated entries appended to
`OTEL_RESOURCE_ATTRIBUTES` after the built-in
`service.name` / `agent` / `swarm` labels.
Host-driven via
`services.hyperhive.deploy.hive-controller.otel.extraResourceAttributes`.
'';
};
debug = lib.mkOption {
type = lib.types.bool;
default = false;
@ -161,14 +167,6 @@ in
systemd.globalEnvironment = producerEnv;
environment.variables = producerEnv;
# `OTEL_RESOURCE_ATTRIBUTES` is in neither: a unit's own `Environment=`
# would override what the generator computes at boot. Units get it from
# the manager's environment; login shells run the same generator after
# ./hive-facts.nix's snippet has exported the hive's labels.
environment.etc."systemd/system-environment-generators/60-hyperhive-otel-resource".source =
resourceGenerator;
environment.extraInit = lib.mkAfter ''export "$(${resourceGenerator})"'';
# The journal this container writes is reachable from inside it and
# from nowhere else, which is why the forwarder runs here rather than
# the hive reading the container's journal from outside: the host-side

6
nix/host-modules/hive-c0re/default.nix
View file

@ -110,9 +110,9 @@ let
# in whole rather than interpolated — that is also the ONLY thing
# gc-rooting it: the resulting string carries store context, so this
# /etc entry genuinely references the package and the host's system
# closure holds it alive. Nothing container-side can: hive-c0re hands
# the path to each container as text (`HYPERHIVE_CLAUDE_CODE_PATH`),
# and text is not a reference. Hence the assertion below: do
# closure holds it alive. Nothing container-side can — meta.rs writes
# the path into each agent's generated flake as a plain string
# literal, and text is not a reference. Hence the assertion below: do
# NOT discard this context, and do not route the path through
# anything that drops it. The failure mode is a garbage-collected
# `claude` and a hive that can't take a turn, weeks after the commit

34
nix/host-modules/hive-c0re/options.nix
View file

@ -326,9 +326,8 @@
};
};
# What this hive hands its own agents at container start, through
# `/etc/hyperhive-hive-facts`. Per host, so two hives may set them
# differently.
# What this hive hands its own agents through the meta-flake render. Per
# host, so two hives may set them differently.
options.services.hyperhive.deploy.hive-controller = {
claudeCodePackage = lib.mkOption {
type = lib.types.nullOr lib.types.package;
@ -351,23 +350,22 @@
What travels into the container is the **store path**, not the
derivation: agents share the host's `/nix/store`, so the binary
and its full closure are already reachable there — nothing
needs rebuilding or copying. hive-c0re hands the path to each
container at start as `HYPERHIVE_CLAUDE_CODE_PATH`, and the agent's
`claude` on PATH runs it, or the container's own `claude-code` when
the variable is absent or names nothing. A change reaches an agent
at its next container start, not through a rebuild.
needs rebuilding or copying. hive-c0re writes the path into
each agent's generated flake as a plain string literal (a bare
path fed to `lib.types.package` would run `builtins.storePath`,
which is illegal under pure evaluation) and the agent module
puts its `bin/` on the harness's PATH.
The flip side of a plain string is that nothing in the agent's
own closure refers to it, so the container cannot keep it
alive. The **host** does that instead: this package is
interpolated into `/etc/hyperhive/serve.json`, which puts it in
the host's system closure, so it is gc-rooted by the running
generation. ⚠️ An agent started under an older generation keeps
that generation's path until its next start, and only the old
generation roots it: collect old generations after restarting
the agents, or an agent still running the old path falls back to
its own `claude-code` at its next `claude` spawn, saying so on
stderr.
the host's system closure — so it is gc-rooted by the running
generation for exactly as long as that generation is the one
the agents were rendered from. The cost is that
`nix-collect-garbage` cannot reclaim an old `claude-code` until
every agent has been rebuilt past it and the old generations
are gone.
The swarm's `swarm-claude-login` and `swarm-review-classifier`
containers, where this host runs them, follow the same pin and
@ -379,11 +377,7 @@
dashboardPort = lib.mkOption {
type = lib.types.port;
default = 7000;
description = ''
TCP port this hive's hive-c0re dashboard listens on. Agents get it
at container start as `HIVE_DASHBOARD_PORT`, for their web UI's
links back to the dashboard.
'';
description = "TCP port this hive's hive-c0re dashboard listens on.";
};
};
}

2
nix/host-modules/otel.nix
View file

@ -236,7 +236,7 @@ in
};
# Read by this hive's hive-c0re, which stamps it on its own export and
# hands it to its agents at container start, so it is a per-host choice.
# renders it into its agents', so it is a per-host choice.
options.services.hyperhive.deploy.hive-controller.otel.extraResourceAttributes = lib.mkOption {
type = lib.types.str;
default = "";

19
nix/module-eval/agent-otel.nix
View file

@ -123,25 +123,6 @@ let
&& lib.all (e: s.exporters.${e}.endpoint == endpoint) s.service.pipelines.logs.exporters
&& !(agentNoOtel.systemd.globalEnvironment ? OTEL_EXPORTER_OTLP_ENDPOINT);
}
{
# The hive's extra labels arrive at container start, so the build
# cannot hold the whole `OTEL_RESOURCE_ATTRIBUTES`: a boot-time
# generator computes it, and a unit-level `Environment=` copy would
# override that. The endpoint beside it is the presence control for the
# two absence lookups; `agentNoOtel` is the control for the generator.
name = "an agent's OTEL resource labels come from a boot-time generator, not the build";
ok =
let
generator =
m: m.environment.etc ? "systemd/system-environment-generators/60-hyperhive-otel-resource";
in
agentDefault.systemd.globalEnvironment ? OTEL_EXPORTER_OTLP_ENDPOINT
&& !(agentDefault.systemd.globalEnvironment ? OTEL_RESOURCE_ATTRIBUTES)
&& agentDefault.environment.variables ? OTEL_EXPORTER_OTLP_ENDPOINT
&& !(agentDefault.environment.variables ? OTEL_RESOURCE_ATTRIBUTES)
&& generator agentDefault
&& !(generator agentNoOtel);
}
{
# Presence control for the two cases above: with the switch off there
# is no collector in the container at all, so their passing is about

2
nix/module-eval/swarm-services-switch.nix
View file

@ -282,7 +282,7 @@ let
{
# The CLI is the one the agents run, not the container's own nixpkgs'.
# Unpinned is the control: it falls back to the container's own `claude-code`.
# serve.json is what hive-c0re hands every agent's `HYPERHIVE_CLAUDE_CODE_PATH` from.
# serve.json is what hive-c0re renders every agent's `claudeCodePath` from.
name = "the agents, the claude login minter and the review classifier run the claude-code this host pins";
ok =
let

12
nix/script-tests/agent-hive-facts.nix
View file

@ -1,9 +1,7 @@
# `checks.script-test-agent-hive-facts` — runs the two readers of
# `/etc/hyperhive-hive-facts` in ../agent-modules/hive-facts.nix: the systemd
# environment generator and the login-shell snippet `/etc/set-environment`
# sources. Also the two consumers of what they pass on: ../agent-modules/otel.nix's
# `OTEL_RESOURCE_ATTRIBUTES` generator and ../agent-modules/default.nix's `claude`.
# The cases are in ./agent-hive-facts.sh.
# sources. The cases are in ./agent-hive-facts.sh.
#
# What runs is the rendered generator and snippet. The one edit is the facts
# file's path, moved under the build directory, and the snippet's reference to
@ -28,11 +26,6 @@ let
;
machine = agentWith { services.hyperhive.agent.user.name = "a1"; };
# As text: building it would build the claude-code it falls back to.
claude = lib.findFirst (
p: (p.name or "") == "claude"
) (throw "no `claude` in systemPackages") machine.environment.systemPackages;
in
pkgs.runCommand "hyperhive-script-test-agent-hive-facts"
{
@ -40,9 +33,6 @@ pkgs.runCommand "hyperhive-script-test-agent-hive-facts"
machine.environment.etc."systemd/system-environment-generators/50-hyperhive-hive-facts".source;
PROFILE = machine.environment.etc."hyperhive/hive-facts.sh".source;
EXTRA_INIT = machine.environment.extraInit;
OTEL_GENERATOR =
machine.environment.etc."systemd/system-environment-generators/60-hyperhive-otel-resource".source;
CLAUDE_WRAPPER = builtins.unsafeDiscardStringContext claude.text;
SHELLS = "${pkgs.dash}/bin/dash ${pkgs.bash}/bin/sh";
}
''

64
nix/script-tests/agent-hive-facts.sh
View file

@ -40,7 +40,7 @@ check() {
for sh in $SHELLS; do
# Only what the snippet exported reaches the child's environment. No PATH:
# the snippet must not need one.
got=$(env -i "$sh" -c ". '$dir/profile'; exec '$envbin'" | grep -E '^(HYPERHIVE_|HIVE_DASHBOARD_PORT=|CLAUDE_|LD_PRELOAD=|__hyperhive)' | sort)
got=$(env -i "$sh" -c ". '$dir/profile'; exec '$envbin'" | grep -E '^(HYPERHIVE_|CLAUDE_|LD_PRELOAD=|__hyperhive)' | sort)
if [ "$got" != "$(printf '%s\n' "$want" | sort)" ]; then
fail "$sh: a child of the sourcing shell saw:"$'\n'"$got"
fi
@ -61,73 +61,11 @@ check "a last line without a newline is kept" "HYPERHIVE_HIVE_DOMAIN=pr1ma.examp
HYPERHIVE_HIVE_NAME=pr1ma
CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=pr1ma-a1"
printf 'HIVE_DASHBOARD_PORT=7123\nHYPERHIVE_CLAUDE_CODE_PATH=/nix/store/0abc-claude-code\nHYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES=team=a,env=prod\n' >"$facts"
check "the per-host values, a value holding = kept whole" "HIVE_DASHBOARD_PORT=7123
HYPERHIVE_CLAUDE_CODE_PATH=/nix/store/0abc-claude-code
HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES=team=a,env=prod
CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=unknown-a1"
: >"$facts"
check "an empty file (no facts)" "CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=unknown-a1"
rm -f "$facts"
check "no file" "CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=unknown-a1"
# OTEL_RESOURCE_ATTRIBUTES, for units (the generator, run with what the
# hive-facts generator printed) and for login shells (the extraInit line, after
# the snippet). The base labels come from the build; the hive's are appended.
base="service.name=hyperhive-agent,agent=a1,swarm=unknown"
otel_check() {
case=$1
count=$((count + 1))
local want="OTEL_RESOURCE_ATTRIBUTES=$2" got sh passed
mapfile -t passed < <(env -i "$dir/generator")
got=$(env -i "${passed[@]}" "$OTEL_GENERATOR")
if [ "$got" != "$want" ]; then fail "generator printed: $got, wanted: $want"; fi
for sh in $SHELLS; do
got=$(env -i "$sh" -c ". '$dir/profile'; $otel_init; exec '$envbin'" | grep '^OTEL_RESOURCE_ATTRIBUTES=')
if [ "$got" != "$want" ]; then fail "$sh: a login shell saw: $got"; fi
done
}
case="otel setup"
otel_init=$(printf '%s\n' "$EXTRA_INIT" | grep -F "$OTEL_GENERATOR")
if [ -z "$otel_init" ]; then fail "/etc/set-environment does not run the OTEL resource generator"; fi
if [ "$(printf '%s\n' "$EXTRA_INIT" | grep -n . | grep -F "$OTEL_GENERATOR" | cut -d: -f1)" -lt \
"$(printf '%s\n' "$EXTRA_INIT" | grep -nxF '. /etc/hyperhive/hive-facts.sh' | cut -d: -f1)" ]; then
fail "/etc/set-environment runs the OTEL resource generator before the hive-facts snippet"
fi
printf 'HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES=team=a,env=prod\n' >"$facts"
otel_check "the hive's labels are appended" "$base,team=a,env=prod"
rm -f "$facts"
otel_check "no hive labels: the base alone, no trailing comma" "$base"
# `claude`: the hive's pinned build when it is there, this container's own
# otherwise. Both builds are stand-ins that print which one ran.
mkdir -p "$dir/own/bin" "$dir/pinned/bin"
printf '#!%s\necho own "$@"\n' "$BASH" >"$dir/own/bin/claude"
printf '#!%s\necho pinned "$@"\n' "$BASH" >"$dir/pinned/bin/claude"
chmod +x "$dir/own/bin/claude" "$dir/pinned/bin/claude"
printf '%s' "$CLAUDE_WRAPPER" | sed -E "s|/nix/store/[^ ]+/bin/claude|$dir/own/bin/claude|" >"$dir/claude"
case="claude setup"
if ! grep -qF "$dir/own/bin/claude" "$dir/claude"; then fail "the rewrite did not reach the fallback claude"; fi
# claude_check <description> <pin or ""> <which ran> <expected stderr>
claude_check() {
case=$1
count=$((count + 1))
local got said pin=()
if [ -n "$2" ]; then pin=("HYPERHIVE_CLAUDE_CODE_PATH=$2"); fi
# Under this shell: the shebang names the build's bash, not in this sandbox.
got=$(env -i "${pin[@]}" "$BASH" "$dir/claude" -p hi 2>"$dir/stderr")
said=$(cat "$dir/stderr")
if [ "$got" != "$3 -p hi" ]; then fail "ran: $got, wanted: $3 -p hi"; fi
if [ "$said" != "$4" ]; then fail "said on stderr: '$said', wanted: '$4'"; fi
}
claude_check "the pinned claude when the hive hands one over, silently" "$dir/pinned" pinned ""
claude_check "the container's own claude with no pin, silently" "" own ""
claude_check "the container's own claude when the pinned path is gone, saying so" "$dir/gone" own \
"claude: $dir/gone/bin/claude, the hive's pinned claude-code, is missing; running this container's own"
echo "$count cases, $failures failed"
[ "$failures" -eq 0 ]