Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
9b1fe85706 | ||
|
|
9113651c10 |
23 changed files with 418 additions and 335 deletions
|
|
@ -118,7 +118,10 @@ declares one flake input per agent (`agent-<n>.url =
|
|||
"git+<forge>/agent-configs/<n>.git?rev=<sha>"`) and one
|
||||
`nixosConfigurations.<n>` output per agent. Each output wraps
|
||||
`inputs.agent-<n>.nixosModules.default` with the identity +
|
||||
`HIVE_PORT` / `HIVE_LABEL` / `HIVE_DASHBOARD_PORT` injection module.
|
||||
`HIVE_PORT` / `HIVE_LABEL` injection module. Values that differ per host
|
||||
(the dashboard port, the pinned claude-code, extra OTEL labels) aren't in it:
|
||||
`hive-priv` writes them to the container's `/etc/hyperhive-hive-facts` at
|
||||
start (see `docs/process/conventions.md`).
|
||||
Containers run against `--flake /var/lib/hyperhive/meta#<n>`.
|
||||
|
||||
The declared input url is the head of `main` on the agent's forge config repo
|
||||
|
|
@ -195,8 +198,7 @@ The container's `--flake` ref is `/var/lib/hyperhive/meta#<name>`
|
|||
(see "Meta flake" above). The agent's own `applied/<n>/flake.nix`
|
||||
is a fixed boilerplate that exports `nixosModules.default =
|
||||
import ./agent.nix`; the meta flake imports that module and
|
||||
wraps it with identity + `HIVE_PORT` / `HIVE_LABEL` /
|
||||
`HIVE_DASHBOARD_PORT`.
|
||||
wraps it with identity + `HIVE_PORT` / `HIVE_LABEL`.
|
||||
|
||||
### Tag state machine
|
||||
|
||||
|
|
|
|||
|
|
@ -39,11 +39,15 @@ Four env vars cover the identity surface, read by
|
|||
name.
|
||||
|
||||
Inside an agent container, the hive's name and domain don't come from the
|
||||
agent's build. `hive-priv` writes them to `/etc/hyperhive-hive-facts` before
|
||||
every container start. The agent's environment generator
|
||||
(`nix/agent-modules/hive-facts.nix`) passes them to every unit, including
|
||||
`hivectl choom` sessions, and `/etc/set-environment` exports them in login
|
||||
shells that no unit starts (`nixos-container root-login`). Neither is part of
|
||||
agent's build, and neither do the hive's other per-host values: the dashboard
|
||||
port (`HIVE_DASHBOARD_PORT`), the pinned claude-code
|
||||
(`HYPERHIVE_CLAUDE_CODE_PATH`) and extra OTEL labels
|
||||
(`HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`). `hive-priv` writes them to
|
||||
`/etc/hyperhive-hive-facts` before every container start, leaving out any the
|
||||
hive doesn't set; the agent then uses its own default. The agent's environment
|
||||
generator (`nix/agent-modules/hive-facts.nix`) passes them to every unit,
|
||||
including `hivectl choom` sessions, and `/etc/set-environment` exports them in login
|
||||
shells that no unit starts (`nixos-container root-login`). None is part of
|
||||
an agent's closure, and an agent started on another hive reports that hive.
|
||||
|
||||
`hive_name` + `swarm_name` are **distinct** from
|
||||
|
|
|
|||
|
|
@ -145,9 +145,9 @@ and every agent runs that build.
|
|||
|
||||
What crosses is the **store path**, not the derivation. Containers
|
||||
share the host's `/nix/store`, so the binary is already reachable
|
||||
inside them with its whole closure; hive-c0re writes the path into each
|
||||
agent's flake as a string literal and the agent module symlinks
|
||||
`bin/claude` onto PATH. Two things rule out the natural-seeming alternatives: a
|
||||
inside them with its whole closure. `hive-priv` writes the path into each
|
||||
container's `/etc/hyperhive-hive-facts` at start, and the agent's `claude`
|
||||
wrapper on PATH runs its `bin/claude`. Two things rule out the natural-seeming alternatives: a
|
||||
`path:/nix/store/<pkg>` flake input is re-copied into the store as a
|
||||
reference-less `-source` (so the runtime closure never arrives), and
|
||||
`lib.types.package` fed a bare path runs `builtins.storePath`, which
|
||||
|
|
@ -171,7 +171,7 @@ expected.
|
|||
|
||||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
The catch is that a path written into a generated flake is text, not a
|
||||
The catch is that a path handed to the container at start is text, not a
|
||||
reference — the container's closure doesn't keep the binary alive.
|
||||
The **host** does: it interpolates the package into
|
||||
`/etc/hyperhive/serve.json`, so it lands in the host's system closure
|
||||
|
|
@ -180,7 +180,7 @@ string context, which is the load-bearing detail; discard the context
|
|||
anywhere on that path and `nix-collect-garbage` will eventually take
|
||||
the hive's `claude` out from under it. The price of the root is that
|
||||
`nix-collect-garbage` can't reclaim an old `claude-code` until every
|
||||
agent has rebuilt past it and the old generations are gone.
|
||||
agent has restarted past it and the old generations are gone.
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
|
|
|
|||
|
|
@ -190,7 +190,7 @@ two-tier architecture, the security model, and how the options interact.
|
|||
|
||||
The harness sets the OTLP variables (`OTEL_EXPORTER_OTLP_ENDPOINT`, `_PROTOCOL`,
|
||||
`OTEL_RESOURCE_ATTRIBUTES`, the temporality preference) **container
|
||||
wide** — in systemd's `DefaultEnvironment` and in `/etc/profile` — so every
|
||||
wide**, for every unit and in `/etc/profile`, so every
|
||||
process in an agent container exports to the hive's collector without any
|
||||
per-tool wiring. That covers Claude Code, `hive-metric`, and anything you run
|
||||
yourself from a tool call or a shell you open in the container.
|
||||
|
|
@ -209,8 +209,9 @@ the push (see [Authenticated ingest](#authenticated-ingest)). The container
|
|||
doesn't set it.
|
||||
|
||||
Append additional labels for one hive via `extraResourceAttributes` (see
|
||||
option reference above); pass custom per-data-point labels with
|
||||
`hive-metric --labels` (see below).
|
||||
option reference above). Agents get them at container start, so a change
|
||||
reaches an agent at its next start, not through a rebuild. Pass custom
|
||||
per-data-point labels with `hive-metric --labels` (see below).
|
||||
|
||||
## Log severity
|
||||
|
||||
|
|
|
|||
|
|
@ -33,11 +33,13 @@ harness unit's PATH. By default that's the `claude-code` in the agent's
|
|||
own nixpkgs (the meta flake's `nixpkgs` input) via
|
||||
`environment.systemPackages`. Since that's typically a release channel and
|
||||
this package moves fast, the operator can pin one per hive with
|
||||
`services.hyperhive.deploy.hive-controller.claudeCodePackage`: hive-c0re writes its
|
||||
store path into each agent's flake, and `claude` on PATH becomes a symlink to it
|
||||
instead of the container's own `claude-code` — so there's only ever one
|
||||
`claude` in the container. Agents pick up a new build on their
|
||||
next rebuild, not live. See docs/process/gotchas.md::`claude-code` is unfree.
|
||||
`services.hyperhive.deploy.hive-controller.claudeCodePackage`. `hive-priv`
|
||||
writes its store path into the container's `/etc/hyperhive-hive-facts` as
|
||||
`HYPERHIVE_CLAUDE_CODE_PATH` before every start. `claude` on PATH is a wrapper
|
||||
that runs that build when the path exists, and the container's own
|
||||
`claude-code` when the variable is unset or the path no longer exists, saying
|
||||
so on stderr in the second case. A bump reaches an agent at its next container
|
||||
start, not at a rebuild. See docs/process/gotchas.md::`claude-code` is unfree.
|
||||
|
||||
Hive-enforced settings ship at `/etc/claude-code/managed-settings.json`
|
||||
(claude-code's canonical managed-settings path — precedence #1,
|
||||
|
|
|
|||
|
|
@ -43,8 +43,8 @@ pub struct Coordinator {
|
|||
/// Git rev of the hyperhive checkout this host runs. Pins the meta
|
||||
/// flake's `hyperhive` input; `None` makes the meta render fail.
|
||||
pub hyperhive_rev: Option<String>,
|
||||
/// Store path of the `claude-code` build every agent runs, written
|
||||
/// into each per-agent flake as `services.hyperhive.agent.claudeCodePath`. Set by
|
||||
/// Store path of the `claude-code` build every agent runs, handed to
|
||||
/// each container at start as `HYPERHIVE_CLAUDE_CODE_PATH`. Set by
|
||||
/// the NixOS module option
|
||||
/// `services.hyperhive.deploy.hive-controller.claudeCodePackage`
|
||||
/// (which resolves the package and hands us its path). `None` = every
|
||||
|
|
@ -55,9 +55,9 @@ pub struct Coordinator {
|
|||
/// closure and all. The host module is what keeps it from being
|
||||
/// garbage-collected — see that option.
|
||||
pub claude_code_path: Option<String>,
|
||||
/// TCP port the host's hive-c0re dashboard listens on. Inlined into
|
||||
/// each per-agent flake so the agent's web UI can build the right
|
||||
/// rebuild-button URL pointing back at the dashboard.
|
||||
/// TCP port the host's hive-c0re dashboard listens on. Handed to each
|
||||
/// container at start as `HIVE_DASHBOARD_PORT`, so the agent's web UI
|
||||
/// can build the rebuild-button URL pointing back at the dashboard.
|
||||
pub dashboard_port: u16,
|
||||
/// Per-agent systemd `CPUQuota=` value (e.g. `"200%"`). Written into
|
||||
/// the `container@h-<name>.service.d/` drop-in on every spawn/rebuild.
|
||||
|
|
@ -193,7 +193,7 @@ pub struct HiveEnv {
|
|||
pub hyperhive_rev: Option<String>,
|
||||
/// Store path of the `claude-code` agents run, or `None` for "each
|
||||
/// agent keeps the one out of its own nixpkgs". Travels into the
|
||||
/// container as `services.hyperhive.agent.claudeCodePath` — a plain string, kept
|
||||
/// container as `HYPERHIVE_CLAUDE_CODE_PATH` — a plain string, kept
|
||||
/// alive host-side by the module that resolved it.
|
||||
pub claude_code_path: Option<String>,
|
||||
pub dashboard_port: u16,
|
||||
|
|
|
|||
|
|
@ -25,7 +25,7 @@ use super::{
|
|||
pub async fn write_dropins(name: &str, hive: &HiveEnv, paths: &AgentPaths) -> Result<()> {
|
||||
validate(name)?;
|
||||
let container = container_name(name);
|
||||
set_nspawn_flags(&container, &paths.agent, &paths.claude, &paths.notes).await?;
|
||||
set_nspawn_flags(&container, hive, &paths.agent, &paths.claude, &paths.notes).await?;
|
||||
hand_permissions(name, &paths.agent);
|
||||
if let Some((cpu_quota, memory_max)) = stage_limits(
|
||||
name,
|
||||
|
|
@ -496,6 +496,7 @@ fn queue_agent_credentials(agent_name: &str, dir: Option<&Path>) -> Vec<Credenti
|
|||
)]
|
||||
async fn set_nspawn_flags(
|
||||
container: &str,
|
||||
hive: &HiveEnv,
|
||||
runtime_dir: &Path,
|
||||
claude_dir: &Path,
|
||||
notes_dir: &Path,
|
||||
|
|
@ -669,7 +670,7 @@ async fn set_nspawn_flags(
|
|||
&binds,
|
||||
isolation,
|
||||
&load_creds,
|
||||
super::hive_facts_from_env(),
|
||||
super::hive_facts(hive),
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
|
|
|||
|
|
@ -169,14 +169,27 @@ pub fn network_isolation_from_env() -> Result<hive_priv_sock::NetworkIsolation>
|
|||
network_isolation_from_vars(bridge.as_deref(), subnet.as_deref())
|
||||
}
|
||||
|
||||
/// The hive's facts every container is handed at runtime, from hive-c0re's
|
||||
/// own unit environment (`nix/host-modules/hive-c0re/environment.nix`). An
|
||||
/// unset or empty variable is `None`.
|
||||
fn hive_facts_from_env() -> hive_priv_sock::HiveFacts {
|
||||
let read = |name: &str| std::env::var(name).ok().filter(|v| !v.is_empty());
|
||||
/// The hive's facts every container is handed at runtime, so that no agent
|
||||
/// build carries a value that differs between hosts. The name, domain and
|
||||
/// OTEL resource labels come from hive-c0re's own unit environment
|
||||
/// (`nix/host-modules/hive-c0re/environment.nix`), where an unset or empty
|
||||
/// variable is `None`; the dashboard port and pinned claude from `hive`.
|
||||
fn hive_facts(hive: &HiveEnv) -> hive_priv_sock::HiveFacts {
|
||||
hive_facts_from(hive, |name| std::env::var(name).ok())
|
||||
}
|
||||
|
||||
/// [`hive_facts`] over a given environment lookup, for tests.
|
||||
fn hive_facts_from(
|
||||
hive: &HiveEnv,
|
||||
var: impl Fn(&str) -> Option<String>,
|
||||
) -> hive_priv_sock::HiveFacts {
|
||||
let read = |name: &str| var(name).filter(|v| !v.is_empty());
|
||||
hive_priv_sock::HiveFacts {
|
||||
name: read("HYPERHIVE_HIVE_NAME"),
|
||||
domain: read("HYPERHIVE_HIVE_DOMAIN"),
|
||||
dashboard_port: Some(hive.dashboard_port),
|
||||
claude_code_path: hive.claude_code_path.clone(),
|
||||
otel_extra_resource_attributes: read("HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES"),
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -527,3 +527,39 @@ async fn run_with_deadline_returns_output_of_a_fast_child() {
|
|||
.expect("echo finishes");
|
||||
assert_eq!(String::from_utf8_lossy(&out.stdout).trim(), "hi");
|
||||
}
|
||||
|
||||
/// The three per-host values reach the container as facts: the dashboard
|
||||
/// port and pinned claude from the hive's config, the OTEL labels from the
|
||||
/// daemon's environment.
|
||||
#[test]
|
||||
fn hive_facts_carry_the_per_host_values() {
|
||||
let hive = HiveEnv {
|
||||
dashboard_port: 7123,
|
||||
claude_code_path: Some("/nix/store/0abc-claude-code".to_owned()),
|
||||
..HiveEnv::default()
|
||||
};
|
||||
let facts = hive_facts_from(&hive, |name| {
|
||||
(name == "HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES").then(|| "team=a".to_owned())
|
||||
});
|
||||
assert_eq!(facts.dashboard_port, Some(7123));
|
||||
assert_eq!(
|
||||
facts.claude_code_path.as_deref(),
|
||||
Some("/nix/store/0abc-claude-code")
|
||||
);
|
||||
assert_eq!(
|
||||
facts.otel_extra_resource_attributes.as_deref(),
|
||||
Some("team=a")
|
||||
);
|
||||
}
|
||||
|
||||
/// Unset and empty both leave a fact out, so the agent falls back to its
|
||||
/// own default rather than reading an empty value.
|
||||
#[test]
|
||||
fn hive_facts_leave_unset_values_out() {
|
||||
let facts = hive_facts_from(&HiveEnv::default(), |name| {
|
||||
(name == "HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES").then(String::new)
|
||||
});
|
||||
assert_eq!(facts.claude_code_path, None);
|
||||
assert_eq!(facts.otel_extra_resource_attributes, None);
|
||||
assert_eq!(facts.name, None);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -431,23 +431,12 @@ fn pinned_git_url(forge_git_url: &str, rev: Option<&str>) -> Result<String> {
|
|||
Ok(format!("git+{forge_git_url}?rev={rev}"))
|
||||
}
|
||||
|
||||
/// [`render_flake`] with its inputs taken from `hive`, as [`sync_agents`]
|
||||
/// renders it.
|
||||
/// [`render_flake`] with the hyperhive pin taken from `hive`, as
|
||||
/// [`sync_agents`] renders it. Nothing else in `hive` reaches the flake: the
|
||||
/// values that differ per host go to the container at start
|
||||
/// (`lifecycle::hive_facts`).
|
||||
fn render_hive_flake(hive: &HiveEnv, agents: &[AgentSpec]) -> Result<String> {
|
||||
render_hive_flake_at(hive, &hyperhive_input_url(hive)?, agents)
|
||||
}
|
||||
|
||||
fn render_hive_flake_at(
|
||||
hive: &HiveEnv,
|
||||
hyperhive_url: &str,
|
||||
agents: &[AgentSpec],
|
||||
) -> Result<String> {
|
||||
render_flake(
|
||||
hyperhive_url,
|
||||
hive.claude_code_path.as_deref(),
|
||||
hive.dashboard_port,
|
||||
agents,
|
||||
)
|
||||
render_flake(&hyperhive_input_url(hive)?, agents)
|
||||
}
|
||||
|
||||
/// Canonical inputs meta knows how to dedup. An agent that declares one
|
||||
|
|
@ -629,19 +618,6 @@ fn forwarded_env_vars() -> Vec<(&'static str, String)> {
|
|||
.collect()
|
||||
}
|
||||
|
||||
/// Extra `OTEL_RESOURCE_ATTRIBUTES` entries for every agent, read off
|
||||
/// hive-c0re's own unit env
|
||||
/// (`services.hyperhive.deploy.hive-controller.otel.extraResourceAttributes`).
|
||||
/// `None` when unset or empty. This is the only OTEL value the renderer
|
||||
/// writes into the flake: agents export to the hive collector by default and
|
||||
/// find it by name, so there is no enable flag, endpoint or protocol to
|
||||
/// carry.
|
||||
fn otel_extra_resource_attributes() -> Option<String> {
|
||||
std::env::var("HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES")
|
||||
.ok()
|
||||
.filter(|v| !v.is_empty())
|
||||
}
|
||||
|
||||
/// Remove the CA files earlier renders embedded next to `flake.nix`
|
||||
/// (`hive-ca.pem`, and the per-peer `peer-ca-<N>.pem` before it), returning
|
||||
/// every filename removed so the caller stages the deletion. Nothing writes
|
||||
|
|
@ -764,12 +740,7 @@ fn canonical_inputs_in_lock(raw: &str) -> Vec<&'static str> {
|
|||
reason = "templated string-builder for the meta flake — the length is one \
|
||||
contiguous fmt block, splitting it would just hide the shape"
|
||||
)]
|
||||
fn render_flake(
|
||||
hyperhive_url: &str,
|
||||
claude_code_path: Option<&str>,
|
||||
dashboard_port: u16,
|
||||
agents: &[AgentSpec],
|
||||
) -> Result<String> {
|
||||
fn render_flake(hyperhive_url: &str, agents: &[AgentSpec]) -> Result<String> {
|
||||
use std::fmt::Write as _;
|
||||
let mut out = String::new();
|
||||
out.push_str("{\n description = \"hyperhive deployed agents\";\n inputs = {\n");
|
||||
|
|
@ -816,10 +787,8 @@ fn render_flake(
|
|||
);
|
||||
}
|
||||
}
|
||||
out.push_str(" };\n outputs =\n { self, hyperhive, ... }@inputs:\n let\n");
|
||||
let _ = writeln!(
|
||||
out,
|
||||
" dashboardPort = {dashboard_port};\n mkAgent = {{ name, port, archetype ? null }}:"
|
||||
out.push_str(
|
||||
" };\n outputs =\n { self, hyperhive, ... }@inputs:\n let\n mkAgent = { name, port, archetype ? null }:\n",
|
||||
);
|
||||
out.push_str(
|
||||
r#" let
|
||||
|
|
@ -842,32 +811,6 @@ fn render_flake(
|
|||
{
|
||||
"#,
|
||||
);
|
||||
// The `claude-code` agents run, as a bare store path rather than a
|
||||
// flake input: containers share the host's `/nix/store`, so the
|
||||
// binary is already there with its closure and has nothing to
|
||||
// travel. A string literal is also the only shape that evaluates —
|
||||
// `lib.types.package` on a bare path runs `builtins.storePath`,
|
||||
// which pure eval rejects. The agent module puts its `bin/` on the
|
||||
// harness PATH; the host module holds the gc root, since a path
|
||||
// spelled out here is text and references nothing.
|
||||
// `None` = no override; agents keep their own nixpkgs' `claude-code`.
|
||||
if let Some(path) = claude_code_path {
|
||||
let _ = writeln!(
|
||||
out,
|
||||
" services.hyperhive.agent.claudeCodePath = \"{path}\";"
|
||||
);
|
||||
}
|
||||
// Extra OTEL resource labels (`deploy.hive-controller.otel.*`) are the one
|
||||
// OTEL value rendered here; enablement and the collector endpoint are
|
||||
// agent-module defaults.
|
||||
if let Some(attrs) = otel_extra_resource_attributes() {
|
||||
let esc = |s: &str| s.replace('\\', "\\\\").replace('"', "\\\"");
|
||||
let _ = writeln!(
|
||||
out,
|
||||
" services.hyperhive.agent.otel.extraResourceAttributes = \"{}\";",
|
||||
esc(&attrs)
|
||||
);
|
||||
}
|
||||
// Agent-facing service URLs (`services.hyperhive.agent.forge.url`, `services.hyperhive.agent.matrix.url`):
|
||||
// emit the host's real values as build-time agent config, the same
|
||||
// host-state -> agent-module shape as the CA block above.
|
||||
|
|
@ -959,7 +902,6 @@ fn render_flake(
|
|||
systemd.services.${service}.environment = {
|
||||
HIVE_PORT = toString port;
|
||||
HIVE_LABEL = name;
|
||||
HIVE_DASHBOARD_PORT = toString dashboardPort;
|
||||
",
|
||||
);
|
||||
// Forwarded env vars — picked up from hive-c0re's own systemd unit
|
||||
|
|
@ -1282,13 +1224,8 @@ mod tests {
|
|||
}
|
||||
}
|
||||
|
||||
fn render(
|
||||
hyperhive_url: &str,
|
||||
claude_code_path: Option<&str>,
|
||||
dashboard_port: u16,
|
||||
agents: &[AgentSpec],
|
||||
) -> String {
|
||||
render_flake(hyperhive_url, claude_code_path, dashboard_port, agents).expect("render")
|
||||
fn render(hyperhive_url: &str, agents: &[AgentSpec]) -> String {
|
||||
render_flake(hyperhive_url, agents).expect("render")
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
@ -1309,8 +1246,6 @@ mod tests {
|
|||
fn the_input_a_rebuild_relocks_is_a_pinned_forge_url() {
|
||||
let out = render(
|
||||
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
|
||||
None,
|
||||
8000,
|
||||
&[sample_spec("iris", 9001)],
|
||||
);
|
||||
let url = out
|
||||
|
|
@ -1344,8 +1279,6 @@ mod tests {
|
|||
fn render_flake_pins_hyperhive_by_url_and_follows_its_nixpkgs() {
|
||||
let out = render(
|
||||
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
|
||||
None,
|
||||
8000,
|
||||
&[sample_spec("alice", 9001)],
|
||||
);
|
||||
assert!(
|
||||
|
|
@ -1374,8 +1307,6 @@ mod tests {
|
|||
fn render_flake_names_each_agent_container_after_the_agent() {
|
||||
let out = render(
|
||||
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
|
||||
None,
|
||||
8000,
|
||||
&[sample_spec("alice", 9001)],
|
||||
);
|
||||
assert!(
|
||||
|
|
@ -1401,49 +1332,6 @@ mod tests {
|
|||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn render_flake_pins_claude_path_without_adding_an_input() {
|
||||
// The host-pinned claude travels as a bare store path assigned to
|
||||
// an option — deliberately NOT as a flake input. Containers share
|
||||
// the host store, so the build is already reachable; making it an
|
||||
// input would re-copy it as a reference-less `-source` and strip
|
||||
// the closure the binary actually needs.
|
||||
let out = render(
|
||||
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
|
||||
Some("/nix/store/cccc-claude-code-2.1.220"),
|
||||
8000,
|
||||
&[sample_spec("alice", 9001)],
|
||||
);
|
||||
assert!(
|
||||
out.contains(
|
||||
"services.hyperhive.agent.claudeCodePath = \"/nix/store/cccc-claude-code-2.1.220\";"
|
||||
),
|
||||
"claude path assigned as a plain string literal:\n{out}"
|
||||
);
|
||||
assert!(
|
||||
!out.contains("claude-code-2.1.220\".url"),
|
||||
"the pinned claude must not become a flake input:\n{out}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn render_flake_omits_claude_path_when_unset() {
|
||||
// `None` = no host-level pin: the option is left undefined so the
|
||||
// agent module keeps its own nixpkgs' `claude-code` (and keeps it
|
||||
// in `environment.systemPackages`, which is what makes the
|
||||
// unpinned case self-contained).
|
||||
let out = render(
|
||||
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
|
||||
None,
|
||||
8000,
|
||||
&[sample_spec("alice", 9001)],
|
||||
);
|
||||
assert!(
|
||||
!out.contains("claudeCodePath"),
|
||||
"no claude assignment when unpinned:\n{out}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn render_flake_emits_follows_for_agents_declaring_nixpkgs() {
|
||||
// `bitburner` + `dmatrix` declare `nixpkgs` at their root, while
|
||||
|
|
@ -1454,8 +1342,6 @@ mod tests {
|
|||
};
|
||||
let out = render(
|
||||
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
|
||||
None,
|
||||
8000,
|
||||
&[
|
||||
sample_spec("argus", 9001),
|
||||
with_nixpkgs("bitburner", 9002),
|
||||
|
|
@ -1482,8 +1368,6 @@ mod tests {
|
|||
fn render_flake_builds_every_agent_from_agent_base() {
|
||||
let out = render(
|
||||
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
|
||||
None,
|
||||
8000,
|
||||
&[sample_spec("alice", 9001), sample_spec("ruth", 9002)],
|
||||
);
|
||||
assert_eq!(
|
||||
|
|
@ -1507,8 +1391,6 @@ mod tests {
|
|||
fn render_flake_skips_canonical_follows_when_the_lock_declares_none() {
|
||||
let out = render(
|
||||
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
|
||||
None,
|
||||
8000,
|
||||
&[sample_spec("alice", 9001)],
|
||||
);
|
||||
// No agent-side follows when the lock declares nothing
|
||||
|
|
@ -1538,8 +1420,6 @@ mod tests {
|
|||
};
|
||||
let out = render(
|
||||
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
|
||||
None,
|
||||
8000,
|
||||
&[alice, android],
|
||||
);
|
||||
assert!(
|
||||
|
|
@ -1588,8 +1468,6 @@ mod tests {
|
|||
}
|
||||
let out = render(
|
||||
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
|
||||
None,
|
||||
8000,
|
||||
&[sample_spec("alice", 9001)],
|
||||
);
|
||||
unsafe {
|
||||
|
|
@ -1629,8 +1507,6 @@ mod tests {
|
|||
}
|
||||
render(
|
||||
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
|
||||
None,
|
||||
8000,
|
||||
&[sample_spec("alice", 9001)],
|
||||
)
|
||||
};
|
||||
|
|
@ -1824,8 +1700,6 @@ mod tests {
|
|||
}
|
||||
let out = render(
|
||||
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
|
||||
None,
|
||||
8000,
|
||||
&[sample_spec("alice", 9001)],
|
||||
);
|
||||
unsafe {
|
||||
|
|
@ -1855,8 +1729,6 @@ mod tests {
|
|||
};
|
||||
let out = render(
|
||||
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
|
||||
None,
|
||||
8000,
|
||||
&[at("alice", 9001, "aaaa"), at("bob", 9002, "bbbb")],
|
||||
);
|
||||
assert!(
|
||||
|
|
@ -1882,8 +1754,6 @@ mod tests {
|
|||
let args = |agents: &[AgentSpec]| {
|
||||
render_flake(
|
||||
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
|
||||
None,
|
||||
8000,
|
||||
agents,
|
||||
)
|
||||
};
|
||||
|
|
@ -2038,8 +1908,6 @@ mod tests {
|
|||
let render = || {
|
||||
render(
|
||||
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
|
||||
None,
|
||||
8000,
|
||||
&[sample_spec("alice", 9001)],
|
||||
)
|
||||
};
|
||||
|
|
@ -2100,11 +1968,12 @@ mod tests {
|
|||
}
|
||||
|
||||
#[test]
|
||||
fn render_flake_carries_no_otel_enablement() {
|
||||
fn render_flake_carries_no_otel_config_and_no_per_host_value() {
|
||||
// The agent module defaults otel on and finds the collector by name, so
|
||||
// the flake carries no enable, endpoint or protocol line however the
|
||||
// host's OTEL env is set. Only the per-hive resource labels are
|
||||
// rendered, and only when set.
|
||||
// host's OTEL env is set. The resource labels, dashboard port and
|
||||
// pinned claude differ per host and reach the agent at container start
|
||||
// (`lifecycle::hive_facts`), so none of them is rendered either.
|
||||
//
|
||||
// Serialised against every other env-mutating test in the crate.
|
||||
// `stats::otel_metrics::tests` perturbs OTEL env too, and lands in this
|
||||
|
|
@ -2115,8 +1984,6 @@ mod tests {
|
|||
let render = || {
|
||||
render(
|
||||
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
|
||||
None,
|
||||
8000,
|
||||
&[sample_spec("alice", 9001)],
|
||||
)
|
||||
};
|
||||
|
|
@ -2139,10 +2006,11 @@ mod tests {
|
|||
std::env::remove_var("HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES");
|
||||
}
|
||||
|
||||
// Presence control: the render is the real flake, carrying the pin.
|
||||
// Presence controls: the render is the real flake, carrying the pin and
|
||||
// the harness env block the dashboard port used to sit in.
|
||||
assert!(
|
||||
plain.contains("rev=0123abcd"),
|
||||
"the hyperhive pin line must be rendered:\n{plain}"
|
||||
plain.contains("rev=0123abcd") && plain.contains("HIVE_PORT = toString port;"),
|
||||
"the hyperhive pin and the harness env must be rendered:\n{plain}"
|
||||
);
|
||||
for (name, flake) in [("plain", &plain), ("labelled", &labelled)] {
|
||||
for needle in [
|
||||
|
|
@ -2150,6 +2018,11 @@ mod tests {
|
|||
"agent.otel.endpoint",
|
||||
"agent.otel.protocol",
|
||||
"c.example",
|
||||
"extraResourceAttributes",
|
||||
"deployment.environment",
|
||||
"claudeCodePath",
|
||||
"dashboardPort",
|
||||
"HIVE_DASHBOARD_PORT",
|
||||
] {
|
||||
assert!(
|
||||
!flake.contains(needle),
|
||||
|
|
@ -2157,16 +2030,6 @@ mod tests {
|
|||
);
|
||||
}
|
||||
}
|
||||
assert!(
|
||||
!plain.contains("agent.otel.extraResourceAttributes"),
|
||||
"extraResourceAttributes must not appear when unset:\n{plain}"
|
||||
);
|
||||
assert!(
|
||||
labelled.contains(
|
||||
"services.hyperhive.agent.otel.extraResourceAttributes = \"deployment.environment=prod\";"
|
||||
),
|
||||
"extraResourceAttributes must be rendered when set:\n{labelled}"
|
||||
);
|
||||
}
|
||||
#[test]
|
||||
fn render_flake_injects_github_disable_only_when_signalled() {
|
||||
|
|
@ -2181,8 +2044,6 @@ mod tests {
|
|||
let render = || {
|
||||
render(
|
||||
"git+https://forge.example/hyperhive/hyperhive.git?rev=0123abcd",
|
||||
None,
|
||||
8000,
|
||||
&[sample_spec("alice", 9001)],
|
||||
)
|
||||
};
|
||||
|
|
@ -2206,33 +2067,6 @@ mod tests {
|
|||
"github.enable = false must be injected when the host disables it:\n{disabled}"
|
||||
);
|
||||
}
|
||||
|
||||
/// The agent reads its memory cap at boot from the copy
|
||||
/// `lifecycle::write_dropins` hands it, so the cap is no build input.
|
||||
#[test]
|
||||
fn render_hive_flake_is_identical_across_memory_caps() {
|
||||
let hive = |cap: &str| HiveEnv {
|
||||
agent_memory_max: cap.to_owned(),
|
||||
..HiveEnv::default()
|
||||
};
|
||||
let agents = [sample_spec("alice", 9001)];
|
||||
let small = render_hive_flake_at(
|
||||
&hive("4G"),
|
||||
"git+https://forge.example/h.git?rev=abc",
|
||||
&agents,
|
||||
)
|
||||
.expect("render");
|
||||
let large = render_hive_flake_at(
|
||||
&hive("16G"),
|
||||
"git+https://forge.example/h.git?rev=abc",
|
||||
&agents,
|
||||
)
|
||||
.expect("render");
|
||||
assert_eq!(
|
||||
small, large,
|
||||
"the rendered flake must not depend on the memory cap"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
|
|
|||
|
|
@ -11,8 +11,8 @@
|
|||
//! ⚠️ **The lock has to be crate-wide, not per-module.** `hive-bash-mcp`'s
|
||||
//! sibling helper records why: two separate per-module mutexes serialise
|
||||
//! nothing against each other, and that produced a CI-only flake. The pair that
|
||||
//! motivated this one is `meta::tests` (which renders a flake from
|
||||
//! `HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`) and `stats::otel_metrics::tests` (which asserts which
|
||||
//! motivated this one is `meta::tests` (which renders a flake with
|
||||
//! `HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES` set) and `stats::otel_metrics::tests` (which asserts which
|
||||
//! variable enables the exporter) — different modules, same process env, same
|
||||
//! binary.
|
||||
//!
|
||||
|
|
|
|||
|
|
@ -187,6 +187,15 @@ pub struct HiveFacts {
|
|||
pub name: Option<String>,
|
||||
/// `HYPERHIVE_HIVE_DOMAIN`: the hive's DNS domain.
|
||||
pub domain: Option<String>,
|
||||
/// `HIVE_DASHBOARD_PORT`: the port this hive's dashboard listens on.
|
||||
pub dashboard_port: Option<u16>,
|
||||
/// `HYPERHIVE_CLAUDE_CODE_PATH`: store path of the `claude-code` the
|
||||
/// hive pins for its agents. The host's system closure keeps it alive;
|
||||
/// the container only gets the path.
|
||||
pub claude_code_path: Option<String>,
|
||||
/// `HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`: comma-separated `k=v`
|
||||
/// entries the agent appends to its `OTEL_RESOURCE_ATTRIBUTES`.
|
||||
pub otel_extra_resource_attributes: Option<String>,
|
||||
}
|
||||
|
||||
/// Wall-clock bound hive-priv puts on one [`PrivRequest::MeasureAgentDisk`]
|
||||
|
|
|
|||
|
|
@ -367,24 +367,54 @@ fn container_rootfs(container: &str) -> PathBuf {
|
|||
/// Render [`HiveFacts`] as the systemd environment file the agent loads,
|
||||
/// one `KEY=value` line per present fact.
|
||||
///
|
||||
/// A value is refused unless it is DNS-name characters only, so it can carry
|
||||
/// neither a newline (a second assignment) nor anything an environment file
|
||||
/// or a shell would interpret.
|
||||
/// Each value is refused, never escaped, unless it is made only of the
|
||||
/// characters its fact can hold. None of those sets admits a newline (a
|
||||
/// second assignment), whitespace, a quote, `\` or `$`: the agent hands each
|
||||
/// line to systemd and its shells, which would read those as syntax.
|
||||
fn render_hive_facts(facts: &HiveFacts) -> Result<String> {
|
||||
use std::fmt::Write as _;
|
||||
let dns = |b: u8| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'-' | b'_');
|
||||
let port = facts.dashboard_port.map(|p| p.to_string());
|
||||
let mut out = String::new();
|
||||
for (key, value) in [
|
||||
("HYPERHIVE_HIVE_NAME", &facts.name),
|
||||
("HYPERHIVE_HIVE_DOMAIN", &facts.domain),
|
||||
for (key, value, max_len, allowed) in [
|
||||
(
|
||||
"HYPERHIVE_HIVE_NAME",
|
||||
facts.name.as_ref(),
|
||||
253,
|
||||
&dns as &dyn Fn(u8) -> bool,
|
||||
),
|
||||
("HYPERHIVE_HIVE_DOMAIN", facts.domain.as_ref(), 253, &dns),
|
||||
("HIVE_DASHBOARD_PORT", port.as_ref(), 5, &|b: u8| {
|
||||
b.is_ascii_digit()
|
||||
}),
|
||||
// A store path's name characters, plus the `/` of its prefix; the
|
||||
// prefix itself is checked below.
|
||||
(
|
||||
"HYPERHIVE_CLAUDE_CODE_PATH",
|
||||
facts.claude_code_path.as_ref(),
|
||||
256,
|
||||
&|b: u8| dns(b) || matches!(b, b'+' | b'?' | b'=' | b'/'),
|
||||
),
|
||||
// `k=v,k=v`, values percent-encoded as the OTEL spec has them.
|
||||
(
|
||||
"HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES",
|
||||
facts.otel_extra_resource_attributes.as_ref(),
|
||||
1024,
|
||||
&|b: u8| dns(b) || matches!(b, b'=' | b',' | b'/' | b':' | b'%' | b'+' | b'@'),
|
||||
),
|
||||
] {
|
||||
let Some(value) = value else { continue };
|
||||
if value.is_empty()
|
||||
|| value.len() > 253
|
||||
|| !value
|
||||
.bytes()
|
||||
.all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'-' | b'_'))
|
||||
if value.is_empty() || value.len() > max_len || !value.bytes().all(allowed) {
|
||||
bail!(
|
||||
"invalid {key} value {value:?}: must be 1-{max_len} chars of its fact's character set"
|
||||
);
|
||||
}
|
||||
if key == "HYPERHIVE_CLAUDE_CODE_PATH"
|
||||
&& !value
|
||||
.strip_prefix("/nix/store/")
|
||||
.is_some_and(|name| !name.is_empty() && !name.contains('/'))
|
||||
{
|
||||
bail!("invalid {key} value {value:?}: must be 1-253 chars of [A-Za-z0-9._-]");
|
||||
bail!("invalid {key} value {value:?}: must be a top-level /nix/store path");
|
||||
}
|
||||
let _ = writeln!(out, "{key}={value}");
|
||||
}
|
||||
|
|
@ -711,14 +741,15 @@ mod tests {
|
|||
let both = HiveFacts {
|
||||
name: Some("pr1ma".to_owned()),
|
||||
domain: Some("pr1ma.darkest.space".to_owned()),
|
||||
..HiveFacts::default()
|
||||
};
|
||||
assert_eq!(
|
||||
render_hive_facts(&both).unwrap(),
|
||||
"HYPERHIVE_HIVE_NAME=pr1ma\nHYPERHIVE_HIVE_DOMAIN=pr1ma.darkest.space\n"
|
||||
);
|
||||
let domain_only = HiveFacts {
|
||||
name: None,
|
||||
domain: Some("h1.example.test".to_owned()),
|
||||
..HiveFacts::default()
|
||||
};
|
||||
assert_eq!(
|
||||
render_hive_facts(&domain_only).unwrap(),
|
||||
|
|
@ -734,7 +765,66 @@ mod tests {
|
|||
for bad in ["", "pr1ma\nLD_PRELOAD=/x", "a b", "a\"b", "$(x)"] {
|
||||
let facts = HiveFacts {
|
||||
name: Some(bad.to_owned()),
|
||||
domain: None,
|
||||
..HiveFacts::default()
|
||||
};
|
||||
assert!(
|
||||
render_hive_facts(&facts).is_err(),
|
||||
"{bad:?} must be refused"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// The three per-host values agents used to get at build time arrive
|
||||
/// here, one line each.
|
||||
#[test]
|
||||
fn hive_facts_render_the_per_host_values() {
|
||||
let facts = HiveFacts {
|
||||
dashboard_port: Some(7123),
|
||||
claude_code_path: Some("/nix/store/0abc-claude-code-2.1.220".to_owned()),
|
||||
otel_extra_resource_attributes: Some(
|
||||
"deployment.environment=prod,team=a%2Fb".to_owned(),
|
||||
),
|
||||
..HiveFacts::default()
|
||||
};
|
||||
assert_eq!(
|
||||
render_hive_facts(&facts).unwrap(),
|
||||
"HIVE_DASHBOARD_PORT=7123\n\
|
||||
HYPERHIVE_CLAUDE_CODE_PATH=/nix/store/0abc-claude-code-2.1.220\n\
|
||||
HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES=deployment.environment=prod,team=a%2Fb\n"
|
||||
);
|
||||
}
|
||||
|
||||
/// The claude path is a top-level store path or nothing, and neither
|
||||
/// value may carry generator syntax.
|
||||
#[test]
|
||||
fn hive_facts_refuse_bad_claude_paths_and_resource_attributes() {
|
||||
for bad in [
|
||||
"/nix/store/",
|
||||
"/nix/store/abc-claude/bin/claude",
|
||||
"/tmp/claude",
|
||||
"nix/store/abc-claude",
|
||||
"/nix/store/abc-claude\nLD_PRELOAD=/x",
|
||||
] {
|
||||
let facts = HiveFacts {
|
||||
claude_code_path: Some(bad.to_owned()),
|
||||
..HiveFacts::default()
|
||||
};
|
||||
assert!(
|
||||
render_hive_facts(&facts).is_err(),
|
||||
"{bad:?} must be refused"
|
||||
);
|
||||
}
|
||||
for bad in [
|
||||
"a=b\nPATH=/x",
|
||||
"a=${HOME}",
|
||||
"a=b c",
|
||||
"a=\"b\"",
|
||||
"a=b\\c",
|
||||
"",
|
||||
] {
|
||||
let facts = HiveFacts {
|
||||
otel_extra_resource_attributes: Some(bad.to_owned()),
|
||||
..HiveFacts::default()
|
||||
};
|
||||
assert!(
|
||||
render_hive_facts(&facts).is_err(),
|
||||
|
|
@ -752,10 +842,7 @@ mod tests {
|
|||
bridge: "hive0".to_owned(),
|
||||
gateway_ip: "10.42.0.1".to_owned(),
|
||||
};
|
||||
let facts = HiveFacts {
|
||||
name: None,
|
||||
domain: None,
|
||||
};
|
||||
let facts = HiveFacts::default();
|
||||
for name in ["hive-forge", "hive-matrix"] {
|
||||
let errs = [
|
||||
handle_write_nspawn_flags(name, &[], &isolation, &[], &facts).unwrap_err(),
|
||||
|
|
|
|||
|
|
@ -28,6 +28,33 @@ let
|
|||
which the hive writes before every start, so no agent build sets it.
|
||||
Remove the setting.
|
||||
'';
|
||||
claudeCodePathRemoved = ''
|
||||
An agent reads the claude-code its hive pins at runtime, from
|
||||
HYPERHIVE_CLAUDE_CODE_PATH in /etc/hyperhive-hive-facts, so no agent build
|
||||
sets it. Pin it host-side with
|
||||
services.hyperhive.deploy.hive-controller.claudeCodePackage.
|
||||
'';
|
||||
|
||||
# `claude` on PATH. A hive that pins a claude-code
|
||||
# (`services.hyperhive.deploy.hive-controller.claudeCodePackage`) hands its
|
||||
# store path over at container start as `HYPERHIVE_CLAUDE_CODE_PATH`, and
|
||||
# the host's system closure keeps it alive: nothing in this container
|
||||
# references it. With no pin, or a path that is not there, this container's
|
||||
# own claude-code runs; a pin that is set but gone (collected after a bump
|
||||
# while this container still holds the old facts) says so on stderr, since
|
||||
# the two builds can differ. `exec -a claude` keeps the bare name in argv[0],
|
||||
# which is how the harness finds its claude child (`find_claude_child`).
|
||||
ownClaude = import ../claude-code-no-alsa.nix lib pkgs.claude-code;
|
||||
claude = pkgs.writeShellScriptBin "claude" ''
|
||||
pinned=''${HYPERHIVE_CLAUDE_CODE_PATH:-}
|
||||
if [ -n "$pinned" ] && [ -x "$pinned/bin/claude" ]; then
|
||||
exec -a claude "$pinned/bin/claude" "$@"
|
||||
fi
|
||||
if [ -n "$pinned" ]; then
|
||||
echo "claude: $pinned/bin/claude, the hive's pinned claude-code, is missing; running this container's own" >&2
|
||||
fi
|
||||
exec -a claude ${ownClaude}/bin/claude "$@"
|
||||
'';
|
||||
in
|
||||
{
|
||||
imports = [
|
||||
|
|
@ -68,6 +95,12 @@ in
|
|||
"agent"
|
||||
"claudeMemoryMaxBytes"
|
||||
] memoryMaxBytesRemoved)
|
||||
(lib.mkRemovedOptionModule [
|
||||
"services"
|
||||
"hyperhive"
|
||||
"agent"
|
||||
"claudeCodePath"
|
||||
] claudeCodePathRemoved)
|
||||
];
|
||||
|
||||
options.services.hyperhive.agent.icon = lib.mkOption {
|
||||
|
|
@ -87,33 +120,6 @@ in
|
|||
'';
|
||||
};
|
||||
|
||||
options.services.hyperhive.agent.claudeCodePath = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
example = "/nix/store/…-claude-code-2.1.220";
|
||||
description = ''
|
||||
Store path of the `claude-code` this agent runs, or `null` (the
|
||||
default) to use the `claude-code` from the container's own
|
||||
nixpkgs.
|
||||
|
||||
Set by the generated meta flake when the operator sets
|
||||
`services.hyperhive.deploy.hive-controller.claudeCodePackage` host-side, so a hive
|
||||
can run a claude built from a *different* nixpkgs than the one
|
||||
its agents evaluate against — a release channel can trail
|
||||
unstable by weeks on this one package. It arrives as a path and
|
||||
not a package because agents share the host's `/nix/store`, so
|
||||
the build is already reachable here with its whole closure and
|
||||
has nothing to travel.
|
||||
|
||||
When set, `claude` on PATH is a symlink to this path's
|
||||
`bin/claude` and the container's own `claude-code` is dropped, so
|
||||
there is only ever one claude in the container. Note that neither
|
||||
the symlink nor anything else in the container's closure *refers*
|
||||
to the target — keeping it alive is the host's job, see
|
||||
`services.hyperhive.deploy.hive-controller.claudeCodePackage`.
|
||||
'';
|
||||
};
|
||||
|
||||
config = {
|
||||
assertions = [
|
||||
# Guard the inputs-routed-as-output pattern: the agent flake.nix is
|
||||
|
|
@ -240,28 +246,7 @@ in
|
|||
config.services.hyperhive.agent.packages.hive-metric
|
||||
]
|
||||
++ [
|
||||
(
|
||||
if config.services.hyperhive.agent.claudeCodePath == null then
|
||||
import ../claude-code-no-alsa.nix lib pkgs.claude-code
|
||||
else
|
||||
# Host-pinned claude: a symlink farm around a path the
|
||||
# container was handed as text. It has to be a derivation —
|
||||
# `environment.systemPackages` coerces a store-path *string*
|
||||
# with `toDerivation`, i.e. `builtins.storePath`, which pure
|
||||
# evaluation rejects (`systemd.services.*.path` does the same,
|
||||
# which is why the harness gets this via PATH like everything
|
||||
# else rather than a unit-level entry). Interpolating the path
|
||||
# into the builder is just text, so it evaluates anywhere.
|
||||
#
|
||||
# The symlink registers no store reference — the target isn't
|
||||
# among this derivation's inputs, so nothing here keeps the
|
||||
# binary alive. That is deliberate and it is the host's job:
|
||||
# see `services.hyperhive.deploy.hive-controller.claudeCodePackage`.
|
||||
pkgs.runCommandLocal "claude-code-pinned" { } ''
|
||||
mkdir -p "$out/bin"
|
||||
ln -s ${config.services.hyperhive.agent.claudeCodePath}/bin/claude "$out/bin/claude"
|
||||
''
|
||||
)
|
||||
claude
|
||||
]
|
||||
++ (with pkgs; [
|
||||
bashInteractive
|
||||
|
|
|
|||
|
|
@ -2,8 +2,17 @@
|
|||
# agent's closure does not carry it.
|
||||
#
|
||||
# hive-priv writes `/etc/hyperhive-hive-facts` into the container before every
|
||||
# start (`HYPERHIVE_HIVE_NAME`, `HYPERHIVE_HIVE_DOMAIN`, one `KEY=value` line
|
||||
# each). Two readers hand those lines on:
|
||||
# start, one `KEY=value` line each:
|
||||
#
|
||||
# - `HYPERHIVE_HIVE_NAME`, `HYPERHIVE_HIVE_DOMAIN`;
|
||||
# - `HIVE_DASHBOARD_PORT`, read by the harness's web UI (7000 when absent);
|
||||
# - `HYPERHIVE_CLAUDE_CODE_PATH`, the claude-code the hive pins (`claude` in
|
||||
# ./default.nix runs this container's own when absent);
|
||||
# - `HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`, appended to
|
||||
# `OTEL_RESOURCE_ATTRIBUTES` by ./otel.nix (no extra labels when absent).
|
||||
#
|
||||
# Any of them may be absent: a hive-c0re older than the agent's build writes
|
||||
# fewer. Two readers hand the lines on:
|
||||
#
|
||||
# - an environment generator, for the container's systemd: it passes them to
|
||||
# every unit it starts, `machinectl shell` sessions (`hivectl choom`)
|
||||
|
|
@ -23,8 +32,8 @@
|
|||
let
|
||||
userName = config.services.hyperhive.agent.user.name;
|
||||
|
||||
# Shell builtins only: nothing is assumed on PATH this early. Only the two
|
||||
# known keys are passed on. `|| [ -n "$key" ]` keeps a last line that has
|
||||
# Shell builtins only: nothing is assumed on PATH this early. Only the known
|
||||
# keys are passed on. `|| [ -n "$key" ]` keeps a last line that has
|
||||
# no trailing newline.
|
||||
generator = pkgs.writeShellScript "hyperhive-hive-facts" ''
|
||||
hive=unknown
|
||||
|
|
@ -32,7 +41,8 @@ let
|
|||
while IFS='=' read -r key value || [ -n "$key" ]; do
|
||||
case "$key" in
|
||||
HYPERHIVE_HIVE_NAME) hive=$value ;;
|
||||
HYPERHIVE_HIVE_DOMAIN) ;;
|
||||
HYPERHIVE_HIVE_DOMAIN | HIVE_DASHBOARD_PORT) ;;
|
||||
HYPERHIVE_CLAUDE_CODE_PATH | HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES) ;;
|
||||
*) continue ;;
|
||||
esac
|
||||
printf '%s=%s\n' "$key" "$value"
|
||||
|
|
|
|||
|
|
@ -42,9 +42,17 @@ let
|
|||
# agent, and their metric names already tell them apart. No `hive`: the
|
||||
# swarm collector stamps it from the receiver that accepted the hive's
|
||||
# push (nix/host-modules/swarm-otel.nix), so the build names no hive.
|
||||
resourceAttributes =
|
||||
"service.name=hyperhive-agent,agent=${userName},swarm=${swarmDisplayName}"
|
||||
+ lib.optionalString (cfg.extraResourceAttributes != "") ",${cfg.extraResourceAttributes}";
|
||||
resourceAttributes = "service.name=hyperhive-agent,agent=${userName},swarm=${swarmDisplayName}";
|
||||
|
||||
# Prints `OTEL_RESOURCE_ATTRIBUTES`: the labels above, then the hive's own
|
||||
# (`HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES`), which differ per host and so
|
||||
# arrive at container start. That variable comes from ./hive-facts.nix's
|
||||
# generator, which runs before this one and is absent when the hive sets no
|
||||
# labels.
|
||||
resourceGenerator = pkgs.writeShellScript "hyperhive-otel-resource" ''
|
||||
printf 'OTEL_RESOURCE_ATTRIBUTES=%s%s\n' ${lib.escapeShellArg resourceAttributes} \
|
||||
"''${HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES:+,$HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES}"
|
||||
'';
|
||||
|
||||
# The variables an OTEL SDK reads on its own, in any language, in any
|
||||
# process here.
|
||||
|
|
@ -62,7 +70,6 @@ let
|
|||
# which Prometheus/Mimir-family backends (incl. grafana-lgtm)
|
||||
# silently drop without a deltatocumulative processor.
|
||||
OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE = "cumulative";
|
||||
OTEL_RESOURCE_ATTRIBUTES = resourceAttributes;
|
||||
};
|
||||
|
||||
# The exporter's NAME picks the wire protocol, so `protocol` has to be
|
||||
|
|
@ -118,19 +125,6 @@ in
|
|||
'';
|
||||
};
|
||||
|
||||
extraResourceAttributes = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
default = "";
|
||||
internal = true;
|
||||
description = ''
|
||||
Extra comma-separated entries appended to
|
||||
`OTEL_RESOURCE_ATTRIBUTES` after the built-in
|
||||
`service.name` / `agent` / `swarm` labels.
|
||||
Host-driven via
|
||||
`services.hyperhive.deploy.hive-controller.otel.extraResourceAttributes`.
|
||||
'';
|
||||
};
|
||||
|
||||
debug = lib.mkOption {
|
||||
type = lib.types.bool;
|
||||
default = false;
|
||||
|
|
@ -167,6 +161,14 @@ in
|
|||
systemd.globalEnvironment = producerEnv;
|
||||
environment.variables = producerEnv;
|
||||
|
||||
# `OTEL_RESOURCE_ATTRIBUTES` is in neither: a unit's own `Environment=`
|
||||
# would override what the generator computes at boot. Units get it from
|
||||
# the manager's environment; login shells run the same generator after
|
||||
# ./hive-facts.nix's snippet has exported the hive's labels.
|
||||
environment.etc."systemd/system-environment-generators/60-hyperhive-otel-resource".source =
|
||||
resourceGenerator;
|
||||
environment.extraInit = lib.mkAfter ''export "$(${resourceGenerator})"'';
|
||||
|
||||
# The journal this container writes is reachable from inside it and
|
||||
# from nowhere else, which is why the forwarder runs here rather than
|
||||
# the hive reading the container's journal from outside: the host-side
|
||||
|
|
|
|||
|
|
@ -110,9 +110,9 @@ let
|
|||
# in whole rather than interpolated — that is also the ONLY thing
|
||||
# gc-rooting it: the resulting string carries store context, so this
|
||||
# /etc entry genuinely references the package and the host's system
|
||||
# closure holds it alive. Nothing container-side can — meta.rs writes
|
||||
# the path into each agent's generated flake as a plain string
|
||||
# literal, and text is not a reference. Hence the assertion below: do
|
||||
# closure holds it alive. Nothing container-side can: hive-c0re hands
|
||||
# the path to each container as text (`HYPERHIVE_CLAUDE_CODE_PATH`),
|
||||
# and text is not a reference. Hence the assertion below: do
|
||||
# NOT discard this context, and do not route the path through
|
||||
# anything that drops it. The failure mode is a garbage-collected
|
||||
# `claude` and a hive that can't take a turn, weeks after the commit
|
||||
|
|
|
|||
|
|
@ -326,8 +326,9 @@
|
|||
};
|
||||
};
|
||||
|
||||
# What this hive hands its own agents through the meta-flake render. Per
|
||||
# host, so two hives may set them differently.
|
||||
# What this hive hands its own agents at container start, through
|
||||
# `/etc/hyperhive-hive-facts`. Per host, so two hives may set them
|
||||
# differently.
|
||||
options.services.hyperhive.deploy.hive-controller = {
|
||||
claudeCodePackage = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.package;
|
||||
|
|
@ -350,22 +351,23 @@
|
|||
What travels into the container is the **store path**, not the
|
||||
derivation: agents share the host's `/nix/store`, so the binary
|
||||
and its full closure are already reachable there — nothing
|
||||
needs rebuilding or copying. hive-c0re writes the path into
|
||||
each agent's generated flake as a plain string literal (a bare
|
||||
path fed to `lib.types.package` would run `builtins.storePath`,
|
||||
which is illegal under pure evaluation) and the agent module
|
||||
puts its `bin/` on the harness's PATH.
|
||||
needs rebuilding or copying. hive-c0re hands the path to each
|
||||
container at start as `HYPERHIVE_CLAUDE_CODE_PATH`, and the agent's
|
||||
`claude` on PATH runs it, or the container's own `claude-code` when
|
||||
the variable is absent or names nothing. A change reaches an agent
|
||||
at its next container start, not through a rebuild.
|
||||
|
||||
The flip side of a plain string is that nothing in the agent's
|
||||
own closure refers to it, so the container cannot keep it
|
||||
alive. The **host** does that instead: this package is
|
||||
interpolated into `/etc/hyperhive/serve.json`, which puts it in
|
||||
the host's system closure — so it is gc-rooted by the running
|
||||
generation for exactly as long as that generation is the one
|
||||
the agents were rendered from. The cost is that
|
||||
`nix-collect-garbage` cannot reclaim an old `claude-code` until
|
||||
every agent has been rebuilt past it and the old generations
|
||||
are gone.
|
||||
the host's system closure, so it is gc-rooted by the running
|
||||
generation. ⚠️ An agent started under an older generation keeps
|
||||
that generation's path until its next start, and only the old
|
||||
generation roots it: collect old generations after restarting
|
||||
the agents, or an agent still running the old path falls back to
|
||||
its own `claude-code` at its next `claude` spawn, saying so on
|
||||
stderr.
|
||||
|
||||
The swarm's `swarm-claude-login` and `swarm-review-classifier`
|
||||
containers, where this host runs them, follow the same pin and
|
||||
|
|
@ -377,7 +379,11 @@
|
|||
dashboardPort = lib.mkOption {
|
||||
type = lib.types.port;
|
||||
default = 7000;
|
||||
description = "TCP port this hive's hive-c0re dashboard listens on.";
|
||||
description = ''
|
||||
TCP port this hive's hive-c0re dashboard listens on. Agents get it
|
||||
at container start as `HIVE_DASHBOARD_PORT`, for their web UI's
|
||||
links back to the dashboard.
|
||||
'';
|
||||
};
|
||||
};
|
||||
}
|
||||
|
|
|
|||
|
|
@ -236,7 +236,7 @@ in
|
|||
};
|
||||
|
||||
# Read by this hive's hive-c0re, which stamps it on its own export and
|
||||
# renders it into its agents', so it is a per-host choice.
|
||||
# hands it to its agents at container start, so it is a per-host choice.
|
||||
options.services.hyperhive.deploy.hive-controller.otel.extraResourceAttributes = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
default = "";
|
||||
|
|
|
|||
|
|
@ -123,6 +123,25 @@ let
|
|||
&& lib.all (e: s.exporters.${e}.endpoint == endpoint) s.service.pipelines.logs.exporters
|
||||
&& !(agentNoOtel.systemd.globalEnvironment ? OTEL_EXPORTER_OTLP_ENDPOINT);
|
||||
}
|
||||
{
|
||||
# The hive's extra labels arrive at container start, so the build
|
||||
# cannot hold the whole `OTEL_RESOURCE_ATTRIBUTES`: a boot-time
|
||||
# generator computes it, and a unit-level `Environment=` copy would
|
||||
# override that. The endpoint beside it is the presence control for the
|
||||
# two absence lookups; `agentNoOtel` is the control for the generator.
|
||||
name = "an agent's OTEL resource labels come from a boot-time generator, not the build";
|
||||
ok =
|
||||
let
|
||||
generator =
|
||||
m: m.environment.etc ? "systemd/system-environment-generators/60-hyperhive-otel-resource";
|
||||
in
|
||||
agentDefault.systemd.globalEnvironment ? OTEL_EXPORTER_OTLP_ENDPOINT
|
||||
&& !(agentDefault.systemd.globalEnvironment ? OTEL_RESOURCE_ATTRIBUTES)
|
||||
&& agentDefault.environment.variables ? OTEL_EXPORTER_OTLP_ENDPOINT
|
||||
&& !(agentDefault.environment.variables ? OTEL_RESOURCE_ATTRIBUTES)
|
||||
&& generator agentDefault
|
||||
&& !(generator agentNoOtel);
|
||||
}
|
||||
{
|
||||
# Presence control for the two cases above: with the switch off there
|
||||
# is no collector in the container at all, so their passing is about
|
||||
|
|
|
|||
|
|
@ -282,7 +282,7 @@ let
|
|||
{
|
||||
# The CLI is the one the agents run, not the container's own nixpkgs'.
|
||||
# Unpinned is the control: it falls back to the container's own `claude-code`.
|
||||
# serve.json is what hive-c0re renders every agent's `claudeCodePath` from.
|
||||
# serve.json is what hive-c0re hands every agent's `HYPERHIVE_CLAUDE_CODE_PATH` from.
|
||||
name = "the agents, the claude login minter and the review classifier run the claude-code this host pins";
|
||||
ok =
|
||||
let
|
||||
|
|
|
|||
|
|
@ -1,7 +1,9 @@
|
|||
# `checks.script-test-agent-hive-facts` — runs the two readers of
|
||||
# `/etc/hyperhive-hive-facts` in ../agent-modules/hive-facts.nix: the systemd
|
||||
# environment generator and the login-shell snippet `/etc/set-environment`
|
||||
# sources. The cases are in ./agent-hive-facts.sh.
|
||||
# sources. Also the two consumers of what they pass on: ../agent-modules/otel.nix's
|
||||
# `OTEL_RESOURCE_ATTRIBUTES` generator and ../agent-modules/default.nix's `claude`.
|
||||
# The cases are in ./agent-hive-facts.sh.
|
||||
#
|
||||
# What runs is the rendered generator and snippet. The one edit is the facts
|
||||
# file's path, moved under the build directory, and the snippet's reference to
|
||||
|
|
@ -26,6 +28,11 @@ let
|
|||
;
|
||||
|
||||
machine = agentWith { services.hyperhive.agent.user.name = "a1"; };
|
||||
|
||||
# As text: building it would build the claude-code it falls back to.
|
||||
claude = lib.findFirst (
|
||||
p: (p.name or "") == "claude"
|
||||
) (throw "no `claude` in systemPackages") machine.environment.systemPackages;
|
||||
in
|
||||
pkgs.runCommand "hyperhive-script-test-agent-hive-facts"
|
||||
{
|
||||
|
|
@ -33,6 +40,9 @@ pkgs.runCommand "hyperhive-script-test-agent-hive-facts"
|
|||
machine.environment.etc."systemd/system-environment-generators/50-hyperhive-hive-facts".source;
|
||||
PROFILE = machine.environment.etc."hyperhive/hive-facts.sh".source;
|
||||
EXTRA_INIT = machine.environment.extraInit;
|
||||
OTEL_GENERATOR =
|
||||
machine.environment.etc."systemd/system-environment-generators/60-hyperhive-otel-resource".source;
|
||||
CLAUDE_WRAPPER = builtins.unsafeDiscardStringContext claude.text;
|
||||
SHELLS = "${pkgs.dash}/bin/dash ${pkgs.bash}/bin/sh";
|
||||
}
|
||||
''
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ check() {
|
|||
for sh in $SHELLS; do
|
||||
# Only what the snippet exported reaches the child's environment. No PATH:
|
||||
# the snippet must not need one.
|
||||
got=$(env -i "$sh" -c ". '$dir/profile'; exec '$envbin'" | grep -E '^(HYPERHIVE_|CLAUDE_|LD_PRELOAD=|__hyperhive)' | sort)
|
||||
got=$(env -i "$sh" -c ". '$dir/profile'; exec '$envbin'" | grep -E '^(HYPERHIVE_|HIVE_DASHBOARD_PORT=|CLAUDE_|LD_PRELOAD=|__hyperhive)' | sort)
|
||||
if [ "$got" != "$(printf '%s\n' "$want" | sort)" ]; then
|
||||
fail "$sh: a child of the sourcing shell saw:"$'\n'"$got"
|
||||
fi
|
||||
|
|
@ -61,11 +61,73 @@ check "a last line without a newline is kept" "HYPERHIVE_HIVE_DOMAIN=pr1ma.examp
|
|||
HYPERHIVE_HIVE_NAME=pr1ma
|
||||
CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=pr1ma-a1"
|
||||
|
||||
printf 'HIVE_DASHBOARD_PORT=7123\nHYPERHIVE_CLAUDE_CODE_PATH=/nix/store/0abc-claude-code\nHYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES=team=a,env=prod\n' >"$facts"
|
||||
check "the per-host values, a value holding = kept whole" "HIVE_DASHBOARD_PORT=7123
|
||||
HYPERHIVE_CLAUDE_CODE_PATH=/nix/store/0abc-claude-code
|
||||
HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES=team=a,env=prod
|
||||
CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=unknown-a1"
|
||||
|
||||
: >"$facts"
|
||||
check "an empty file (no facts)" "CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=unknown-a1"
|
||||
|
||||
rm -f "$facts"
|
||||
check "no file" "CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=unknown-a1"
|
||||
|
||||
# OTEL_RESOURCE_ATTRIBUTES, for units (the generator, run with what the
|
||||
# hive-facts generator printed) and for login shells (the extraInit line, after
|
||||
# the snippet). The base labels come from the build; the hive's are appended.
|
||||
base="service.name=hyperhive-agent,agent=a1,swarm=unknown"
|
||||
otel_check() {
|
||||
case=$1
|
||||
count=$((count + 1))
|
||||
local want="OTEL_RESOURCE_ATTRIBUTES=$2" got sh passed
|
||||
mapfile -t passed < <(env -i "$dir/generator")
|
||||
got=$(env -i "${passed[@]}" "$OTEL_GENERATOR")
|
||||
if [ "$got" != "$want" ]; then fail "generator printed: $got, wanted: $want"; fi
|
||||
for sh in $SHELLS; do
|
||||
got=$(env -i "$sh" -c ". '$dir/profile'; $otel_init; exec '$envbin'" | grep '^OTEL_RESOURCE_ATTRIBUTES=')
|
||||
if [ "$got" != "$want" ]; then fail "$sh: a login shell saw: $got"; fi
|
||||
done
|
||||
}
|
||||
|
||||
case="otel setup"
|
||||
otel_init=$(printf '%s\n' "$EXTRA_INIT" | grep -F "$OTEL_GENERATOR")
|
||||
if [ -z "$otel_init" ]; then fail "/etc/set-environment does not run the OTEL resource generator"; fi
|
||||
if [ "$(printf '%s\n' "$EXTRA_INIT" | grep -n . | grep -F "$OTEL_GENERATOR" | cut -d: -f1)" -lt \
|
||||
"$(printf '%s\n' "$EXTRA_INIT" | grep -nxF '. /etc/hyperhive/hive-facts.sh' | cut -d: -f1)" ]; then
|
||||
fail "/etc/set-environment runs the OTEL resource generator before the hive-facts snippet"
|
||||
fi
|
||||
|
||||
printf 'HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES=team=a,env=prod\n' >"$facts"
|
||||
otel_check "the hive's labels are appended" "$base,team=a,env=prod"
|
||||
rm -f "$facts"
|
||||
otel_check "no hive labels: the base alone, no trailing comma" "$base"
|
||||
|
||||
# `claude`: the hive's pinned build when it is there, this container's own
|
||||
# otherwise. Both builds are stand-ins that print which one ran.
|
||||
mkdir -p "$dir/own/bin" "$dir/pinned/bin"
|
||||
printf '#!%s\necho own "$@"\n' "$BASH" >"$dir/own/bin/claude"
|
||||
printf '#!%s\necho pinned "$@"\n' "$BASH" >"$dir/pinned/bin/claude"
|
||||
chmod +x "$dir/own/bin/claude" "$dir/pinned/bin/claude"
|
||||
printf '%s' "$CLAUDE_WRAPPER" | sed -E "s|/nix/store/[^ ]+/bin/claude|$dir/own/bin/claude|" >"$dir/claude"
|
||||
case="claude setup"
|
||||
if ! grep -qF "$dir/own/bin/claude" "$dir/claude"; then fail "the rewrite did not reach the fallback claude"; fi
|
||||
# claude_check <description> <pin or ""> <which ran> <expected stderr>
|
||||
claude_check() {
|
||||
case=$1
|
||||
count=$((count + 1))
|
||||
local got said pin=()
|
||||
if [ -n "$2" ]; then pin=("HYPERHIVE_CLAUDE_CODE_PATH=$2"); fi
|
||||
# Under this shell: the shebang names the build's bash, not in this sandbox.
|
||||
got=$(env -i "${pin[@]}" "$BASH" "$dir/claude" -p hi 2>"$dir/stderr")
|
||||
said=$(cat "$dir/stderr")
|
||||
if [ "$got" != "$3 -p hi" ]; then fail "ran: $got, wanted: $3 -p hi"; fi
|
||||
if [ "$said" != "$4" ]; then fail "said on stderr: '$said', wanted: '$4'"; fi
|
||||
}
|
||||
claude_check "the pinned claude when the hive hands one over, silently" "$dir/pinned" pinned ""
|
||||
claude_check "the container's own claude with no pin, silently" "" own ""
|
||||
claude_check "the container's own claude when the pinned path is gone, saying so" "$dir/gone" own \
|
||||
"claude: $dir/gone/bin/claude, the hive's pinned claude-code, is missing; running this container's own"
|
||||
|
||||
echo "$count cases, $failures failed"
|
||||
[ "$failures" -eq 0 ]
|
||||
|
|
|
|||
Loading…
Reference in a new issue