From c87a4d2c1a3ee70430c8bc9664abfc30739df6a2 Mon Sep 17 00:00:00 2001 From: atlas Date: Mon, 12 Oct 2026 04:40:13 +0200 Subject: [PATCH] agents: read the hive's name and domain at runtime, not from the build Part of #5081 (slice S0-1a). An agent's closure no longer depends on the name or domain of the hive that renders it. - hive-priv writes /etc/hyperhive-hive-facts (HYPERHIVE_HIVE_NAME, HYPERHIVE_HIVE_DOMAIN) into the container from the same WriteNspawnFlags call that writes /etc/hyperhive-bridge-dns. hive-c0re passes the values from its own unit env. hive-priv refuses a value outside [A-Za-z0-9._-]. - A systemd environment generator in the agent (hive-facts.nix) loads the file into the container manager's environment, so every unit sees it, as with the systemd.globalEnvironment entries the render used to emit. It also builds CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX (-), which left claude's managed settings. - meta.rs stops forwarding HYPERHIVE_HIVE_{NAME,DOMAIN} and stops setting services.hyperhive.agent.hiveName. The option is removed (both spellings now fail with a removal message). - otel.nix drops the agent's hive= resource label. The swarm collector upserts hive per authenticated receiver (swarm-otel.nix resource/, on both the metrics and logs pipelines), so stored series keep it. The otel first hop, otel enable and the dashboard port are still baked; they are S0-1b. --- docs/process/conventions.md | 7 ++ docs/scheduler/observability.md | 13 ++- docs/turn-loop/claude-invocation.md | 2 +- hive-c0re/src/lifecycle/host_config.rs | 9 +- hive-c0re/src/lifecycle/mod.rs | 11 ++ hive-c0re/src/meta.rs | 92 ++++++++++++----- hive-c0re/src/priv_client.rs | 4 +- hive-priv-sock/src/lib.rs | 16 +++ hive-priv/src/main.rs | 135 ++++++++++++++++++++----- nix/agent-modules/claude-settings.nix | 66 ++++-------- nix/agent-modules/default.nix | 9 ++ nix/agent-modules/hive-facts.nix | 37 +++++++ nix/agent-modules/otel.nix | 30 +++--- nix/agent-modules/renamed-options.nix | 1 - 14 files changed, 304 insertions(+), 128 deletions(-) create mode 100644 nix/agent-modules/hive-facts.nix diff --git a/docs/process/conventions.md b/docs/process/conventions.md index 8cca616d..e6d817e4 100644 --- a/docs/process/conventions.md +++ b/docs/process/conventions.md @@ -38,6 +38,13 @@ Four env vars cover the identity surface, read by federated hives at different DNS domains can share a swarm name. +Inside an agent container, the hive's name and domain don't come from the +agent's build. `hive-priv` writes them to `/etc/hyperhive-hive-facts` before +every container start, and the agent's environment generator +(`nix/agent-modules/hive-facts.nix`) passes them to every unit. Neither is +part of an agent's closure, and an agent started on another hive reports +that hive. + `hive_name` + `swarm_name` are **distinct** from `HYPERHIVE_HIVE_DOMAIN`: the domain may carry the hive name as its leftmost label by convention, but the convention isn't diff --git a/docs/scheduler/observability.md b/docs/scheduler/observability.md index 0fdd7c61..8ca6160a 100644 --- a/docs/scheduler/observability.md +++ b/docs/scheduler/observability.md @@ -193,9 +193,12 @@ automatically: | -------------- | ------------------------------------------------------------ | | `service.name` | `hyperhive-agent` (constant) | | `agent` | agent logical name (for example `iris`) | -| `hive` | hive display name (`services.hyperhive.hiveName`) | | `swarm` | swarm display name (`services.hyperhive.swarm.name`, if set) | +The swarm's collector adds `hive`, the name of the hive whose receiver accepted +the push (see [Authenticated ingest](#authenticated-ingest)). The container +doesn't set it. + Append additional labels via `extraResourceAttributes` (see option reference above); pass custom per-data-point labels with `hive-metric --labels` (see below). @@ -368,10 +371,10 @@ API-reported window, else the model's default. A turn that parsed no usage leaves both where the previous turn put them. The dashboard's **Context window used by agent** panel (health tab) divides one by the other. -Resource attributes (`service.name`, `agent`, `hive`, `swarm`) come from the +Resource attributes (`service.name`, `agent`, `swarm`) come from the same container-wide `OTEL_RESOURCE_ATTRIBUTES` as everything else in this -section — nothing extra to configure. Cadence follows -`HYPERHIVE_OTEL_METRIC_INTERVAL_MS` (default 60s, same variable + default as +section, and the swarm's collector adds `hive` — nothing extra to configure. +Cadence follows `HYPERHIVE_OTEL_METRIC_INTERVAL_MS` (default 60s, same variable + default as `hive-c0re`'s container-resource export above) — that only controls how often the harness flushes the batched points to the collector, not how often it records them (every turn, always). @@ -479,7 +482,7 @@ hive-metric [--type counter|gauge] [--temporality delta|cumulativ itself). - `--labels key=value` — extra per-data-point labels. Repeat the flag for more than one. - `hive-metric` inherits the resource labels (agent, hive, swarm, service.name) + `hive-metric` inherits the resource labels (agent, swarm, service.name) automatically from `OTEL_RESOURCE_ATTRIBUTES` — don't re-specify them. ### Examples diff --git a/docs/turn-loop/claude-invocation.md b/docs/turn-loop/claude-invocation.md index e6d3cf3e..bdbc0742 100644 --- a/docs/turn-loop/claude-invocation.md +++ b/docs/turn-loop/claude-invocation.md @@ -260,7 +260,7 @@ needs to `chown` a bind mount), once at startup: substituted: `{label}` (short agent name), `{qualified_label}` (hive-qualified `name@domain` form), `{operator_pronouns}`, `{hive_identity}` (for example `` on hive `pr1ma` ``; empty when - `services.hyperhive.agent.hiveName` is unset), and `{swarm_identity}` (same + `HYPERHIVE_HIVE_NAME` is unset), and `{swarm_identity}` (same shape for the swarm). Pronouns come from `HIVE_OPERATOR_PRONOUNS` env (set by the meta flake from `services.hyperhive.c0re.operatorPronouns`, default `she/her`). diff --git a/hive-c0re/src/lifecycle/host_config.rs b/hive-c0re/src/lifecycle/host_config.rs index 236413fa..2c70fd1a 100644 --- a/hive-c0re/src/lifecycle/host_config.rs +++ b/hive-c0re/src/lifecycle/host_config.rs @@ -476,7 +476,14 @@ async fn set_nspawn_flags( ); // Delegate the actual conf-file rewrite to hive-priv (runs as root). - crate::priv_client::write_nspawn_flags(container, &binds, isolation, &load_creds).await + crate::priv_client::write_nspawn_flags( + container, + &binds, + isolation, + &load_creds, + super::hive_facts_from_env(), + ) + .await } #[cfg(test)] diff --git a/hive-c0re/src/lifecycle/mod.rs b/hive-c0re/src/lifecycle/mod.rs index 4328b8fa..a513f731 100644 --- a/hive-c0re/src/lifecycle/mod.rs +++ b/hive-c0re/src/lifecycle/mod.rs @@ -177,6 +177,17 @@ pub fn network_isolation_from_env() -> Result network_isolation_from_vars(bridge.as_deref(), subnet.as_deref()) } +/// The hive's facts every container is handed at runtime, from hive-c0re's +/// own unit environment (`nix/host-modules/hive-c0re/environment.nix`). An +/// unset or empty variable is `None`. +fn hive_facts_from_env() -> hive_priv_sock::HiveFacts { + let read = |name: &str| std::env::var(name).ok().filter(|v| !v.is_empty()); + hive_priv_sock::HiveFacts { + name: read("HYPERHIVE_HIVE_NAME"), + domain: read("HYPERHIVE_HIVE_DOMAIN"), + } +} + fn validate(name: &str) -> Result<()> { if name.is_empty() { bail!("agent name must not be empty"); diff --git a/hive-c0re/src/meta.rs b/hive-c0re/src/meta.rs index 1ce14990..917a85e1 100644 --- a/hive-c0re/src/meta.rs +++ b/hive-c0re/src/meta.rs @@ -508,8 +508,6 @@ const FORWARDED_VARS: &[&str] = &[ "HIVE_FORGE_URL", "HIVE_FORGE_PUBLIC_URL", "HIVE_MATRIX_URL", - "HYPERHIVE_HIVE_DOMAIN", - "HYPERHIVE_HIVE_NAME", "HYPERHIVE_SWARM_NAME", ]; @@ -520,14 +518,11 @@ const FORWARDED_VARS: &[&str] = &[ /// var is read at runtime. Setting only one leaves the other on its default, /// which is how a hive ends up with two disagreeing answers for one value. /// -/// The hive/swarm display names are here for exactly that reason, learned the -/// hard way: they were forwarded as runtime env only, while the agent's -/// claude-settings module read them from the container's `environment.variables` -/// at *eval* time — where they were never set. Every agent baked -/// `hive=unknown,swarm=unknown` into its OTEL resource attributes and shipped -/// that label on every metric, while the same process's env held the right -/// answer. Half-wiring this map is not a missing nicety; it is a value that -/// evaluates fine and is silently wrong. +/// The swarm display name is here for that reason: `otel.nix` reads the option +/// at eval time for the `swarm=` resource label, and with the env var alone +/// that label evaluates to `unknown` while the process env holds the right +/// name. Half-wiring this map is a value that evaluates fine and is silently +/// wrong. const FORWARDED_VAR_OPTIONS: &[(&str, &str)] = &[ // Read at build time to decide whether the harness unit imports the hive's // queue credential at all — see `nix/agent-modules/queue.nix`. Both halves @@ -558,7 +553,6 @@ const FORWARDED_VAR_OPTIONS: &[(&str, &str)] = &[ ), ("HIVE_FORGE_URL", "services.hyperhive.agent.forge.url"), ("HIVE_MATRIX_URL", "services.hyperhive.agent.matrix.url"), - ("HYPERHIVE_HIVE_NAME", "services.hyperhive.agent.hiveName"), ("HYPERHIVE_SWARM_NAME", "services.hyperhive.agent.swarmName"), ]; @@ -1164,9 +1158,7 @@ where // → in-container accessors fall back to None / defaults gracefully. // // - HIVE_FORGE_URL: agents poll this for Forgejo notifications. - // - HYPERHIVE_HIVE_DOMAIN: machine-readable hive DNS. - // - HYPERHIVE_HIVE_NAME / HYPERHIVE_SWARM_NAME: human display - // names for hive + swarm. + // - HYPERHIVE_SWARM_NAME: human display name for the swarm. for (var, val) in forwarded_env_vars() { let escaped = val.replace('\\', "\\\\").replace('"', "\\\""); let _ = writeln!(out, " {var} = \"{escaped}\";"); @@ -1770,6 +1762,53 @@ mod tests { ); } + /// The hive's name and domain reach an agent at runtime + /// (`/etc/hyperhive-hive-facts`), so the rendered flake does not change with + /// either. + #[test] + fn render_flake_is_identical_across_hive_names_and_domains() { + const VARS: [&str; 2] = ["HYPERHIVE_HIVE_NAME", "HYPERHIVE_HIVE_DOMAIN"]; + let _env = crate::test_env::lock(); + let saved = VARS.map(std::env::var_os); + let render_on = |name: &str, domain: &str| { + // SAFETY: serialised by the guard above; restored before returning. + unsafe { + std::env::set_var(VARS[0], name); + std::env::set_var(VARS[1], domain); + } + render_flake( + "github:example/hyperhive", + "path:/nix/store/bbbb-hyperhive-docs-source", + "path:/nix/store/aaaa-nixpkgs-source", + None, + 8000, + "she/her", + &std::collections::HashMap::new(), + "4G", + &[sample_spec("alice", false, 9001)], + ) + .expect("render flake") + }; + let alpha = render_on("alpha-hive", "alpha-hive.example.test"); + let beta = render_on("beta-hive", "beta-hive.example.test"); + for (var, value) in VARS.iter().zip(saved) { + // SAFETY: as above. + unsafe { + match value { + Some(v) => std::env::set_var(var, v), + None => std::env::remove_var(var), + } + } + } + for hive in ["alpha-hive", "beta-hive"] { + assert!( + !alpha.contains(hive) && !beta.contains(hive), + "the rendered flake must not name the hive ({hive}):\n{alpha}" + ); + } + assert_eq!(alpha, beta, "the render must not depend on the hive"); + } + #[test] fn service_url_options_render_from_forwarded_pairs() { // Pure over the pairs, deliberately: the render-level tests have to @@ -1782,21 +1821,21 @@ mod tests { &[ ("HIVE_FORGE_URL", "http://forge.example.test".to_string()), ("HIVE_MATRIX_URL", "http://matrix.example.test".to_string()), - // Mapped too, and that is the fix this test now guards: the - // display names used to be forwarded as runtime env *only*, - // so the build-time reader fell back to "unknown" and every - // agent shipped that label on every metric. - ("HYPERHIVE_HIVE_NAME", "pr1ma".to_string()), + // Mapped too: `otel.nix` reads the option at eval time. + ("HYPERHIVE_SWARM_NAME", "constellat1on".to_string()), // Forwarded but genuinely unmapped — proves the map is a // filter, not a pass-through. - ("HYPERHIVE_HIVE_DOMAIN", "pr1ma.darkest.space".to_string()), + ( + "HIVE_FORGE_PUBLIC_URL", + "https://forge.example.test".to_string(), + ), ], ); assert_eq!( out, " services.hyperhive.agent.forge.url = \"http://forge.example.test\";\n\ \x20 services.hyperhive.agent.matrix.url = \"http://matrix.example.test\";\n\ - \x20 services.hyperhive.agent.hiveName = \"pr1ma\";\n", + \x20 services.hyperhive.agent.swarmName = \"constellat1on\";\n", "expected exactly the mapped options, in input order, indented for the module block" ); } @@ -1836,7 +1875,7 @@ mod tests { fn require_service_urls_accepts_a_rendered_forge_url() { require_service_urls(&[ ("HIVE_FORGE_URL", "http://forge.example.test".to_string()), - ("HYPERHIVE_HIVE_NAME", "pr1ma".to_string()), + ("HYPERHIVE_SWARM_NAME", "constellat1on".to_string()), ]) .expect("a forwarded forge URL satisfies the requirement"); } @@ -1847,7 +1886,7 @@ mod tests { // every agent would simply come up with no forge login and no way to // tell that was unintended. The hive asserts its own requirement // because it is the only layer that knows a forge is mandatory. - let err = require_service_urls(&[("HYPERHIVE_HIVE_NAME", "pr1ma".to_string())]) + let err = require_service_urls(&[("HYPERHIVE_SWARM_NAME", "constellat1on".to_string())]) .expect_err("a missing forge URL must stop the write"); assert!( err.to_string().contains("HIVE_FORGE_URL is unset"), @@ -1862,13 +1901,16 @@ mod tests { // so absence is an absent integration rather than a misdirected one. // Required services don't reach here: `require_service_urls` rejects // them first. - // `HYPERHIVE_HIVE_DOMAIN` is forwarded as env but carries no option — + // `HIVE_FORGE_PUBLIC_URL` is forwarded as env but carries no option — // deliberately picked over a mapped var, since the whole point is a // forwarded var the option map does not know about. let mut out = String::new(); push_forwarded_var_options( &mut out, - &[("HYPERHIVE_HIVE_DOMAIN", "pr1ma.darkest.space".to_string())], + &[( + "HIVE_FORGE_PUBLIC_URL", + "https://forge.example.test".to_string(), + )], ); assert!( out.is_empty(), diff --git a/hive-c0re/src/priv_client.rs b/hive-c0re/src/priv_client.rs index 8e1bb4d9..0cd30e75 100644 --- a/hive-c0re/src/priv_client.rs +++ b/hive-c0re/src/priv_client.rs @@ -8,7 +8,7 @@ use anyhow::{Context as _, Result, bail}; use hive_priv_sock::{ - BindMount, CredentialMount, InfraAction, InfraContainer, MEASURE_AGENT_DISK_TIMEOUT, + BindMount, CredentialMount, HiveFacts, InfraAction, InfraContainer, MEASURE_AGENT_DISK_TIMEOUT, NetworkIsolation, PRIV_SOCK, PrivEvent, PrivRequest, PrivResponse, PrivStream, }; use std::os::fd::{AsRawFd as _, OwnedFd, RawFd}; @@ -288,12 +288,14 @@ pub async fn write_nspawn_flags( binds: &[BindMount], isolation: NetworkIsolation, load_credentials: &[CredentialMount], + hive_facts: HiveFacts, ) -> Result<()> { ok(call(&PrivRequest::WriteNspawnFlags { container: container.to_owned(), binds: binds.to_vec(), isolation, load_credentials: load_credentials.to_vec(), + hive_facts, }) .await?) } diff --git a/hive-priv-sock/src/lib.rs b/hive-priv-sock/src/lib.rs index 4b393e4f..24844cf5 100644 --- a/hive-priv-sock/src/lib.rs +++ b/hive-priv-sock/src/lib.rs @@ -232,6 +232,18 @@ pub struct NetworkIsolation { pub gateway_ip: String, } +/// Facts about the hive a container runs on, for `WriteNspawnFlags`. +/// hive-priv writes them into the container's `/etc/hyperhive-hive-facts`, +/// which the agent loads into its environment at boot, so no agent build +/// carries them. A `None` field leaves that variable unset in the agent. +#[derive(Debug, Clone, Default, Serialize, Deserialize)] +pub struct HiveFacts { + /// `HYPERHIVE_HIVE_NAME`: the hive's name (`pr1ma`). + pub name: Option, + /// `HYPERHIVE_HIVE_DOMAIN`: the hive's DNS domain. + pub domain: Option, +} + /// Wall-clock bound hive-priv puts on one [`PrivRequest::MeasureAgentDisk`] /// walk. A caller's own wait has to be longer than this, so a slow walk /// reaches it as hive-priv's timeout error rather than as its own. @@ -306,6 +318,10 @@ pub enum PrivRequest { /// so a hive-priv built before this field deserialises new requests. #[serde(default)] load_credentials: Vec, + /// Written to the container's `/etc/hyperhive-hive-facts` on every + /// call, so a fact the hive no longer has is removed, not kept. + #[serde(default)] + hive_facts: HiveFacts, }, /// Create the agent's socket dir `/run/hive-agent/` as `0751 diff --git a/hive-priv/src/main.rs b/hive-priv/src/main.rs index 12e9dfe5..052ff7aa 100644 --- a/hive-priv/src/main.rs +++ b/hive-priv/src/main.rs @@ -22,10 +22,10 @@ use std::path::{Path, PathBuf}; use anyhow::{Context as _, Result, anyhow, bail}; use hive_priv_sock::{ - AGENT_PREFIX, AGENT_RUNTIME_ROOT, AGENT_STATE_ROOT, BindMount, CredentialMount, InfraAction, - InfraContainer, MEASURE_AGENT_DISK_TIMEOUT, META_DIR, MIGRATE_STAGING_ROOT, NetworkIsolation, - PAUSED_MARKER_FILE, PRIV_SOCK, PrivEvent, PrivRequest, PrivResponse, PrivStream, - PrivStreamLine, SIBLING_CONTAINERS, + AGENT_PREFIX, AGENT_RUNTIME_ROOT, AGENT_STATE_ROOT, BindMount, CredentialMount, HiveFacts, + InfraAction, InfraContainer, MEASURE_AGENT_DISK_TIMEOUT, META_DIR, MIGRATE_STAGING_ROOT, + NetworkIsolation, PAUSED_MARKER_FILE, PRIV_SOCK, PrivEvent, PrivRequest, PrivResponse, + PrivStream, PrivStreamLine, SIBLING_CONTAINERS, }; use tokio::io::{AsyncWriteExt, BufReader}; use tokio::net::unix::OwnedWriteHalf; @@ -377,7 +377,8 @@ async fn exec( ref binds, ref isolation, ref load_credentials, - } => handle_write_nspawn_flags(container, binds, isolation, load_credentials), + ref hive_facts, + } => handle_write_nspawn_flags(container, binds, isolation, load_credentials, hive_facts), PrivRequest::EnsureAgentSocketDir { ref name } => ensure_agent_socket_dir(name), @@ -984,6 +985,7 @@ fn handle_write_nspawn_flags( binds: &[BindMount], isolation: &NetworkIsolation, load_credentials: &[CredentialMount], + hive_facts: &HiveFacts, ) -> Result<(String, String)> { validate_container_system_name(container)?; for bind in binds { @@ -997,7 +999,8 @@ fn handle_write_nspawn_flags( // uses `:` as the name/path separator. validate_bind_path(&cred.host_path)?; } - write_nspawn_flags(container, binds, isolation, load_credentials)?; + let hive_facts = render_hive_facts(hive_facts)?; + write_nspawn_flags(container, binds, isolation, load_credentials, &hive_facts)?; Ok((String::new(), String::new())) } @@ -2870,6 +2873,7 @@ fn write_nspawn_flags( binds: &[BindMount], isolation: &NetworkIsolation, load_credentials: &[CredentialMount], + hive_facts: &str, ) -> Result<()> { use std::fmt::Write as _; use std::os::unix::fs::MetadataExt as _; @@ -2950,36 +2954,78 @@ fn write_nspawn_flags( // and which unit does it: `docs/networking/network.md` § *How the isolated // container gets its resolver*. write_bridge_dns_marker(container, isolation)?; + // Read by the agent's environment generator at boot + // (`nix/agent-modules/hive-facts.nix`). + write_container_etc_file_in(&container_rootfs(container), HIVE_FACTS_FILE, hive_facts)?; Ok(()) } /// Filename of the in-container DNS marker, in the container's own `/etc`. const BRIDGE_DNS_MARKER: &str = "hyperhive-bridge-dns"; +/// Filename of the hive-facts environment file, in the container's own `/etc`. +const HIVE_FACTS_FILE: &str = "hyperhive-hive-facts"; + +/// The container's `/` on the host. +fn container_rootfs(container: &str) -> PathBuf { + PathBuf::from(format!("/var/lib/nixos-containers/{container}")) +} + +/// Render [`HiveFacts`] as the systemd environment file the agent loads, +/// one `KEY=value` line per present fact. +/// +/// A value is refused unless it is DNS-name characters only, so it can carry +/// neither a newline (a second assignment) nor anything an environment file +/// or a shell would interpret. +fn render_hive_facts(facts: &HiveFacts) -> Result { + use std::fmt::Write as _; + let mut out = String::new(); + for (key, value) in [ + ("HYPERHIVE_HIVE_NAME", &facts.name), + ("HYPERHIVE_HIVE_DOMAIN", &facts.domain), + ] { + let Some(value) = value else { continue }; + if value.is_empty() + || value.len() > 253 + || !value + .bytes() + .all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'-' | b'_')) + { + bail!("invalid {key} value {value:?}: must be 1-253 chars of [A-Za-z0-9._-]"); + } + let _ = writeln!(out, "{key}={value}"); + } + Ok(out) +} + /// Write the bridge-DNS marker the `hyperhive-isolated-dns` oneshot keys /// off. The marker file contains just the gateway IP. Always written: /// every container is isolated, so there is no host-netns case that /// wants the marker absent. fn write_bridge_dns_marker(container: &str, isolation: &NetworkIsolation) -> Result<()> { - let rootfs = PathBuf::from(format!("/var/lib/nixos-containers/{container}")); - write_bridge_dns_marker_in(&rootfs, &isolation.gateway_ip) + write_bridge_dns_marker_in(&container_rootfs(container), &isolation.gateway_ip) } -/// Write `/etc/hyperhive-bridge-dns` without following a symlink. +/// Write `/etc/hyperhive-bridge-dns`; see [`write_container_etc_file_in`]. +fn write_bridge_dns_marker_in(rootfs: &Path, gateway_ip: &str) -> Result<()> { + write_container_etc_file_in(rootfs, BRIDGE_DNS_MARKER, &format!("{gateway_ip}\n")) +} + +/// Write `/etc/` without following a symlink. /// /// Root inside the container owns its `/etc`, and this write runs as host /// root, where an absolute symlink planted in the container resolves against /// the host's `/`. So this opens `etc` with `O_DIRECTORY|O_NOFOLLOW` (a -/// symlinked `etc` fails with `ELOOP`) and publishes the marker through a +/// symlinked `etc` fails with `ELOOP`) and publishes the file through a /// [`StagedFile`] relative to that fd: whatever the container planted at the /// leaf, symlink or FIFO or device node, is replaced by the rename, never /// written through or opened. /// /// On a fresh install the container's `/etc` may not exist yet (nixos-container /// materialises the rootfs on the first start), so this creates it first; the -/// dir and the marker persist through that start. Mode 0644: the gateway IP -/// is not a secret. -fn write_bridge_dns_marker_in(rootfs: &Path, gateway_ip: &str) -> Result<()> { +/// dir and the file persist through that start. Mode 0644: nothing written +/// here is a secret. +fn write_container_etc_file_in(rootfs: &Path, name: &str, contents: &str) -> Result<()> { use std::os::unix::fs::OpenOptionsExt as _; let etc_path = rootfs.join("etc"); @@ -3014,14 +3060,7 @@ fn write_bridge_dns_marker_in(rootfs: &Path, gateway_ip: &str) -> Result<()> { } // SAFETY: `etc_fd` is a fresh, valid fd that nothing else owns. let etc = std::fs::File::from(unsafe { OwnedFd::from_raw_fd(etc_fd) }); - StagedFile::write( - &etc, - &etc_path, - BRIDGE_DNS_MARKER, - format!("{gateway_ip}\n").as_bytes(), - 0o644, - )? - .publish() + StagedFile::write(&etc, &etc_path, name, contents.as_bytes(), 0o644)?.publish() } /// Written by older hive-priv builds. Nothing writes it now, but left on disk @@ -3124,13 +3163,14 @@ fn ensure_socket_dir_in(root: &Path, name: &str) -> Result<()> { #[cfg(test)] mod tests { use super::{ - BindMount, BoundedRun, Command, OwnedFd, PAUSED_MARKER_FILE, PrivRequest, StagedFile, - agent_disk_paths, check_fd_agreement, contains_secret_shaped_run, describe_forge_admin, - du_command, du_total, ensure_plain_filename, ensure_socket_dir_in, exec_measure_agent_disk, - git_overlay_flags, limits_dropin_body, open_dir, open_export_dest, parse_du_total, - partial_name, publish_file, redact_secret_line, remove_marker_in, run_bounded, - single_output_path, toplevel_attr, validate_credential_name, validate_snapshot_name, - write_agent_dir_file, write_bridge_dns_marker_in, + BindMount, BoundedRun, Command, HiveFacts, OwnedFd, PAUSED_MARKER_FILE, PrivRequest, + StagedFile, agent_disk_paths, check_fd_agreement, contains_secret_shaped_run, + describe_forge_admin, du_command, du_total, ensure_plain_filename, ensure_socket_dir_in, + exec_measure_agent_disk, git_overlay_flags, limits_dropin_body, open_dir, open_export_dest, + parse_du_total, partial_name, publish_file, redact_secret_line, remove_marker_in, + render_hive_facts, run_bounded, single_output_path, toplevel_attr, + validate_credential_name, validate_snapshot_name, write_agent_dir_file, + write_bridge_dns_marker_in, }; use std::path::PathBuf; use std::sync::atomic::{AtomicU32, Ordering}; @@ -3782,6 +3822,45 @@ mod tests { std::fs::remove_dir_all(&dir).ok(); } + /// Each present fact is one environment-file line; an absent one writes + /// nothing, so the agent sees the variable unset rather than empty. + #[test] + fn hive_facts_render_one_line_per_present_fact() { + let both = HiveFacts { + name: Some("pr1ma".to_owned()), + domain: Some("pr1ma.darkest.space".to_owned()), + }; + assert_eq!( + render_hive_facts(&both).unwrap(), + "HYPERHIVE_HIVE_NAME=pr1ma\nHYPERHIVE_HIVE_DOMAIN=pr1ma.darkest.space\n" + ); + let domain_only = HiveFacts { + name: None, + domain: Some("h1.example.test".to_owned()), + }; + assert_eq!( + render_hive_facts(&domain_only).unwrap(), + "HYPERHIVE_HIVE_DOMAIN=h1.example.test\n" + ); + assert_eq!(render_hive_facts(&HiveFacts::default()).unwrap(), ""); + } + + /// A value that could end its line or be read as syntax is refused, not + /// escaped: a newline would add an assignment of the caller's choosing. + #[test] + fn hive_facts_refuse_values_outside_dns_characters() { + for bad in ["", "pr1ma\nLD_PRELOAD=/x", "a b", "a\"b", "$(x)"] { + let facts = HiveFacts { + name: Some(bad.to_owned()), + domain: None, + }; + assert!( + render_hive_facts(&facts).is_err(), + "{bad:?} must be refused" + ); + } + } + /// Whether `pid` is gone, or only a zombie awaiting its reaper: either /// way it is no longer running. fn is_dead(pid: &str) -> bool { diff --git a/nix/agent-modules/claude-settings.nix b/nix/agent-modules/claude-settings.nix index f77f790c..0322bc8e 100644 --- a/nix/agent-modules/claude-settings.nix +++ b/nix/agent-modules/claude-settings.nix @@ -19,17 +19,6 @@ let # meta.rs `otel_config`). Options declared in `otel.nix`, which also # exports the generic OTLP environment this container's producers read. otelCfg = config.services.hyperhive.agent.otel; - # Hive display name, read from the per-agent option meta.rs renders - # (NOT from `environment.variables` — that carries the same name at - # *runtime* only, so reading it here silently yielded "unknown" on - # every agent while the process env held the right answer). `null` - # means the hive did not name itself; "unknown" is then an honest label - # rather than a guess. - hiveDisplayName = - if config.services.hyperhive.agent.hiveName == null then - "unknown" - else - config.services.hyperhive.agent.hiveName; # Effective per-agent MemoryMax=, in bytes, injected by meta.rs's # per-agent flake render (`services.hyperhive.agent.claudeMemoryMaxBytes`). `null` # when the effective cap is unbounded ("infinity") or a RAM @@ -58,8 +47,6 @@ let CLAUDE_CODE_RESUME_INTERRUPTED_TURN = "1"; # Use the simpler system prompt variant suited to headless operation. CLAUDE_CODE_SIMPLE_SYSTEM_PROMPT = "1"; - # Tag remote-control sessions with "-" for identification. - CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX = "${hiveDisplayName}-${userName}"; } # Bun/JavaScriptCore (the runtime under claude-code's `.claude-wrapped` # binary) sizes its default heap ceiling off the container's visible @@ -97,6 +84,21 @@ let }; in { + options.services.hyperhive.agent.swarmName = lib.mkOption { + type = lib.types.nullOr lib.types.str; + default = null; + internal = true; + description = '' + Human-readable swarm name, rendered per-agent by + `meta.rs::render_flake` from the host's + `services.hyperhive.swarm.name`. Read during evaluation for the + OTEL `swarm` resource label; `HYPERHIVE_SWARM_NAME` carries the + same value at runtime. + + `null` means the hive is not part of a named swarm. + ''; + }; + # Build-time implementation surface for the JSC-heap-ceiling fix: # meta.rs's per-agent flake render injects this from the effective # `MemoryMax=` (per-agent `resource-limits.json` override, else the @@ -108,37 +110,6 @@ in # the agent's *last rebuild* — `set-limits` still applies the # cgroup cap live via a drop-in reload, but this derived heap ceiling # needs a rebuild to pick up a new value. - options.services.hyperhive.agent.hiveName = lib.mkOption { - type = lib.types.nullOr lib.types.str; - default = null; - internal = true; - description = '' - Human-readable hive name, rendered per-agent by - `meta.rs::render_flake` from the host's - `services.hyperhive.hiveName`. Baked into the OTEL resource - attributes at build time, which is why it is an option and not - just the `HYPERHIVE_HIVE_NAME` env var: the env var is read at - runtime, this is read during evaluation, and wiring only one of - the two is how every agent ended up reporting `hive=unknown`. - - `null` means the hive did not name itself. - ''; - }; - - options.services.hyperhive.agent.swarmName = lib.mkOption { - type = lib.types.nullOr lib.types.str; - default = null; - internal = true; - description = '' - Human-readable swarm name, rendered per-agent by - `meta.rs::render_flake` from the host's - `services.hyperhive.swarm.name`. Same build-time/runtime split as - `services.hyperhive.agent.hiveName`. - - `null` means the hive is not part of a named swarm. - ''; - }; - options.services.hyperhive.agent.claudeMemoryMaxBytes = lib.mkOption { type = lib.types.nullOr lib.types.ints.positive; default = null; @@ -279,7 +250,7 @@ in # Hive-enforced settings merged with a per-agent `env` block at BUILD # time via `jq` (not eval-time `readFile`, which would be import-from- # derivation). The `env` block is always present: `baseClaudeEnv` sets - # behaviour flags and the remote-control session prefix for every agent; + # behaviour flags for every agent; # `otelSettingsEnv` is merged on top when OTEL is enabled. claude-code # auto-discovers this file in every context (harness turn-loop AND # `hivectl choom`) so no launch wrapper is needed. @@ -287,9 +258,8 @@ in let baseSettings = "${config.services.hyperhive.agent.packages.assets}/share/hyperhive/prompts/claude-settings.json"; # Merge base env (always) with OTEL env (when enabled). jq is always - # run — `baseClaudeEnv` contains per-agent values (e.g. - # CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX) that can't live in the - # static store asset. + # run — `baseClaudeEnv` can carry per-agent values (e.g. + # BUN_JSC_forceRAMSize) that can't live in the static store asset. allEnv = baseClaudeEnv // lib.optionalAttrs otelCfg.enable otelSettingsEnv diff --git a/nix/agent-modules/default.nix b/nix/agent-modules/default.nix index 7f46fb80..ff833705 100644 --- a/nix/agent-modules/default.nix +++ b/nix/agent-modules/default.nix @@ -36,6 +36,11 @@ let See docs/tools/matrix.md. ''; + hiveNameRemoved = '' + An agent reads its hive's name at runtime from /etc/hyperhive-hive-facts, + which the hive writes into every container, so no agent build sets it. + Remove the setting. + ''; in { imports = [ @@ -51,6 +56,7 @@ in ./frontend.nix ./github.nix ./github-token.nix + ./hive-facts.nix ./logs.nix ./matrix.nix ./mcp.nix @@ -85,6 +91,9 @@ in # breaks evaluation for everyone, removed or not. (lib.mkRemovedOptionModule [ "hyperhive" "matrix" "enable" ] matrixEnableRemoved) (lib.mkRemovedOptionModule [ "services" "hyperhive" "agent" "matrix" "enable" ] matrixEnableRemoved) + # Both spellings, for the reason given above. + (lib.mkRemovedOptionModule [ "hyperhive" "hiveName" ] hiveNameRemoved) + (lib.mkRemovedOptionModule [ "services" "hyperhive" "agent" "hiveName" ] hiveNameRemoved) ]; options.services.hyperhive.agent.icon = lib.mkOption { diff --git a/nix/agent-modules/hive-facts.nix b/nix/agent-modules/hive-facts.nix new file mode 100644 index 00000000..345c4a7a --- /dev/null +++ b/nix/agent-modules/hive-facts.nix @@ -0,0 +1,37 @@ +# The hive this container runs on, read at boot rather than built in, so an +# agent's closure does not carry it. +# +# hive-priv writes `/etc/hyperhive-hive-facts` into the container before every +# start (`HYPERHIVE_HIVE_NAME`, `HYPERHIVE_HIVE_DOMAIN`, one `KEY=value` line +# each). This environment generator hands those lines to the container's +# systemd, which passes them to every unit it starts and to `machinectl shell` +# sessions; a unit's own `Environment=` still overrides them. +{ + config, + lib, + pkgs, + ... +}: +let + userName = config.services.hyperhive.agent.user.name; +in +{ + # Shell builtins only: nothing is assumed on PATH this early. Only the two + # known keys are passed on. + environment.etc."systemd/system-environment-generators/50-hyperhive-hive-facts".source = + pkgs.writeShellScript "hyperhive-hive-facts" '' + hive=unknown + if [ -r /etc/hyperhive-hive-facts ]; then + while IFS='=' read -r key value; do + case "$key" in + HYPERHIVE_HIVE_NAME) hive=$value ;; + HYPERHIVE_HIVE_DOMAIN) ;; + *) continue ;; + esac + printf '%s=%s\n' "$key" "$value" + done < /etc/hyperhive-hive-facts + fi + # Names claude's remote-control sessions `-`. + printf 'CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=%s-%s\n' "$hive" ${lib.escapeShellArg userName} + ''; +} diff --git a/nix/agent-modules/otel.nix b/nix/agent-modules/otel.nix index 359fe81d..f42618ab 100644 --- a/nix/agent-modules/otel.nix +++ b/nix/agent-modules/otel.nix @@ -25,17 +25,10 @@ let cfg = config.services.hyperhive.agent.otel; userName = config.services.hyperhive.agent.user.name; - # Hive/swarm display names, read from the per-agent options meta.rs - # renders (NOT from `environment.variables` — those carry the same - # names at *runtime* only, so reading them here silently yielded - # "unknown" on every agent while the process env held the right - # answer). `null` means the hive did not name itself; "unknown" is then - # an honest label rather than a guess. - hiveDisplayName = - if config.services.hyperhive.agent.hiveName == null then - "unknown" - else - config.services.hyperhive.agent.hiveName; + # Swarm display name, read from the per-agent option meta.rs renders + # (NOT from `environment.variables`, which carries it at *runtime* + # only). `null` means the swarm is unnamed; "unknown" is then an honest + # label rather than a guess. swarmDisplayName = if config.services.hyperhive.agent.swarmName == null then "unknown" @@ -45,9 +38,11 @@ let # Resource labels every producer in this container stamps on what it # emits. `service.name` names the container's role, not one binary # inside it: claude's samples and `hive-metric`'s both come from this - # agent, and their metric names already tell them apart. + # agent, and their metric names already tell them apart. No `hive`: the + # swarm collector stamps it from the receiver that accepted the hive's + # push (nix/host-modules/swarm-otel.nix), so the build names no hive. resourceAttributes = - "service.name=hyperhive-agent,agent=${userName},hive=${hiveDisplayName},swarm=${swarmDisplayName}" + "service.name=hyperhive-agent,agent=${userName},swarm=${swarmDisplayName}" + lib.optionalString (cfg.extraResourceAttributes != "") ",${cfg.extraResourceAttributes}"; # The variables an OTEL SDK reads on its own, in any language, in any @@ -131,7 +126,7 @@ in description = '' Extra comma-separated entries appended to `OTEL_RESOURCE_ATTRIBUTES` after the built-in - `service.name` / `agent` / `hive` / `swarm` labels. + `service.name` / `agent` / `swarm` labels. Host-driven via `services.hyperhive.otel.extraResourceAttributes`. ''; }; @@ -274,10 +269,9 @@ in # Identity the hop above cannot supply. A host-side reader can say # which machine a line came from; only a collector running as the # agent can say which agent. `hive` and `swarm` are deliberately - # NOT stamped here even though the env above carries them for - # in-process producers: the swarm tier upserts `hive` from the - # receiver that accepted the sample, precisely so the label comes - # from something the sender cannot write. + # NOT stamped here: the swarm tier upserts both from the receiver + # that accepted the sample, precisely so the labels come from + # something the sender cannot write. processors.resource.attributes = [ { key = "service.name"; diff --git a/nix/agent-modules/renamed-options.nix b/nix/agent-modules/renamed-options.nix index 8c10140a..f0b00b74 100644 --- a/nix/agent-modules/renamed-options.nix +++ b/nix/agent-modules/renamed-options.nix @@ -112,7 +112,6 @@ [ "hyperhive" "gui" "vncPort" ] [ "services" "hyperhive" "agent" "gui" "vncPort" ] ) - (lib.mkRenamedOptionModule [ "hyperhive" "hiveName" ] [ "services" "hyperhive" "agent" "hiveName" ]) (lib.mkRenamedOptionModule [ "hyperhive" "icon" ] [ "services" "hyperhive" "agent" "icon" ]) (lib.mkRenamedOptionModule [ "hyperhive" "logs" "queryUrl" ]