Watch
0
0
Fork
You've already forked hyperhive
0

agents: read the hive's name and domain at runtime, not from the build

Part of #5081 (slice S0-1a). An agent's closure no longer depends on the
name or domain of the hive that renders it.

- hive-priv writes /etc/hyperhive-hive-facts (HYPERHIVE_HIVE_NAME,
  HYPERHIVE_HIVE_DOMAIN) into the container from the same WriteNspawnFlags
  call that writes /etc/hyperhive-bridge-dns. hive-c0re passes the values
  from its own unit env. hive-priv refuses a value outside [A-Za-z0-9._-].
- A systemd environment generator in the agent (hive-facts.nix) loads the
  file into the container manager's environment, so every unit sees it,
  as with the systemd.globalEnvironment entries the render used to emit.
  It also builds CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX
  (<hive>-<agent>), which left claude's managed settings.
- meta.rs stops forwarding HYPERHIVE_HIVE_{NAME,DOMAIN} and stops setting
  services.hyperhive.agent.hiveName. The option is removed (both
  spellings now fail with a removal message).
- otel.nix drops the agent's hive= resource label. The swarm collector
  upserts hive per authenticated receiver (swarm-otel.nix resource/<h>,
  on both the metrics and logs pipelines), so stored series keep it.

The otel first hop, otel enable and the dashboard port are still baked;
they are S0-1b.
This commit is contained in:
atlas 2026-10-12 04:40:13 +02:00 • committed by mara
commit c87a4d2c1a
14 changed files with 303 additions and 127 deletions

View file

@ -38,6 +38,13 @@ Four env vars cover the identity surface, read by
federated hives at different DNS domains can share a swarm federated hives at different DNS domains can share a swarm
name. name.
Inside an agent container, the hive's name and domain don't come from the
agent's build. `hive-priv` writes them to `/etc/hyperhive-hive-facts` before
every container start, and the agent's environment generator
(`nix/agent-modules/hive-facts.nix`) passes them to every unit. Neither is
part of an agent's closure, and an agent started on another hive reports
that hive.
`hive_name` + `swarm_name` are **distinct** from `hive_name` + `swarm_name` are **distinct** from
`HYPERHIVE_HIVE_DOMAIN`: the domain may carry the hive name as `HYPERHIVE_HIVE_DOMAIN`: the domain may carry the hive name as
its leftmost label by convention, but the convention isn't its leftmost label by convention, but the convention isn't

View file

@ -193,9 +193,12 @@ automatically:
| -------------- | ------------------------------------------------------------ | | -------------- | ------------------------------------------------------------ |
| `service.name` | `hyperhive-agent` (constant) | | `service.name` | `hyperhive-agent` (constant) |
| `agent` | agent logical name (for example `iris`) | | `agent` | agent logical name (for example `iris`) |
| `hive` | hive display name (`services.hyperhive.hiveName`) |
| `swarm` | swarm display name (`services.hyperhive.swarm.name`, if set) | | `swarm` | swarm display name (`services.hyperhive.swarm.name`, if set) |
The swarm's collector adds `hive`, the name of the hive whose receiver accepted
the push (see [Authenticated ingest](#authenticated-ingest)). The container
doesn't set it.
Append additional labels via `extraResourceAttributes` (see option Append additional labels via `extraResourceAttributes` (see option
reference above); pass custom per-data-point labels with reference above); pass custom per-data-point labels with
`hive-metric --labels` (see below). `hive-metric --labels` (see below).
@ -368,10 +371,10 @@ API-reported window, else the model's default. A turn that parsed no usage
leaves both where the previous turn put them. The dashboard's **Context leaves both where the previous turn put them. The dashboard's **Context
window used by agent** panel (health tab) divides one by the other. window used by agent** panel (health tab) divides one by the other.
Resource attributes (`service.name`, `agent`, `hive`, `swarm`) come from the Resource attributes (`service.name`, `agent`, `swarm`) come from the
same container-wide `OTEL_RESOURCE_ATTRIBUTES` as everything else in this same container-wide `OTEL_RESOURCE_ATTRIBUTES` as everything else in this
section — nothing extra to configure. Cadence follows section, and the swarm's collector adds `hive` — nothing extra to configure.
`HYPERHIVE_OTEL_METRIC_INTERVAL_MS` (default 60s, same variable + default as Cadence follows `HYPERHIVE_OTEL_METRIC_INTERVAL_MS` (default 60s, same variable + default as
`hive-c0re`'s container-resource export above) — that only controls how often `hive-c0re`'s container-resource export above) — that only controls how often
the harness flushes the batched points to the collector, not how often it the harness flushes the batched points to the collector, not how often it
records them (every turn, always). records them (every turn, always).
@ -479,7 +482,7 @@ hive-metric <name> <value> [--type counter|gauge] [--temporality delta|cumulativ
itself). itself).
- `--labels key=value` — extra per-data-point labels. Repeat the flag for - `--labels key=value` — extra per-data-point labels. Repeat the flag for
more than one. more than one.
`hive-metric` inherits the resource labels (agent, hive, swarm, service.name) `hive-metric` inherits the resource labels (agent, swarm, service.name)
automatically from `OTEL_RESOURCE_ATTRIBUTES` — don't re-specify them. automatically from `OTEL_RESOURCE_ATTRIBUTES` — don't re-specify them.
### Examples ### Examples

View file

@ -260,7 +260,7 @@ needs to `chown` a bind mount), once at startup:
substituted: `{label}` (short agent name), `{qualified_label}` substituted: `{label}` (short agent name), `{qualified_label}`
(hive-qualified `name@domain` form), `{operator_pronouns}`, (hive-qualified `name@domain` form), `{operator_pronouns}`,
`{hive_identity}` (for example `` on hive `pr1ma` ``; empty when `{hive_identity}` (for example `` on hive `pr1ma` ``; empty when
`services.hyperhive.agent.hiveName` is unset), and `{swarm_identity}` (same `HYPERHIVE_HIVE_NAME` is unset), and `{swarm_identity}` (same
shape for the swarm). Pronouns come from `HIVE_OPERATOR_PRONOUNS` shape for the swarm). Pronouns come from `HIVE_OPERATOR_PRONOUNS`
env (set by the meta flake from env (set by the meta flake from
`services.hyperhive.c0re.operatorPronouns`, default `she/her`). `services.hyperhive.c0re.operatorPronouns`, default `she/her`).

View file

@ -476,7 +476,14 @@ async fn set_nspawn_flags(
); );
// Delegate the actual conf-file rewrite to hive-priv (runs as root). // Delegate the actual conf-file rewrite to hive-priv (runs as root).
crate::priv_client::write_nspawn_flags(container, &binds, isolation, &load_creds).await crate::priv_client::write_nspawn_flags(
container,
&binds,
isolation,
&load_creds,
super::hive_facts_from_env(),
)
.await
} }
#[cfg(test)] #[cfg(test)]

View file

@ -177,6 +177,17 @@ pub fn network_isolation_from_env() -> Result<hive_priv_sock::NetworkIsolation>
network_isolation_from_vars(bridge.as_deref(), subnet.as_deref()) network_isolation_from_vars(bridge.as_deref(), subnet.as_deref())
} }
/// The hive's facts every container is handed at runtime, from hive-c0re's
/// own unit environment (`nix/host-modules/hive-c0re/environment.nix`). An
/// unset or empty variable is `None`.
fn hive_facts_from_env() -> hive_priv_sock::HiveFacts {
let read = |name: &str| std::env::var(name).ok().filter(|v| !v.is_empty());
hive_priv_sock::HiveFacts {
name: read("HYPERHIVE_HIVE_NAME"),
domain: read("HYPERHIVE_HIVE_DOMAIN"),
}
}
fn validate(name: &str) -> Result<()> { fn validate(name: &str) -> Result<()> {
if name.is_empty() { if name.is_empty() {
bail!("agent name must not be empty"); bail!("agent name must not be empty");

View file

@ -508,8 +508,6 @@ const FORWARDED_VARS: &[&str] = &[
"HIVE_FORGE_URL", "HIVE_FORGE_URL",
"HIVE_FORGE_PUBLIC_URL", "HIVE_FORGE_PUBLIC_URL",
"HIVE_MATRIX_URL", "HIVE_MATRIX_URL",
"HYPERHIVE_HIVE_DOMAIN",
"HYPERHIVE_HIVE_NAME",
"HYPERHIVE_SWARM_NAME", "HYPERHIVE_SWARM_NAME",
]; ];
@ -520,14 +518,11 @@ const FORWARDED_VARS: &[&str] = &[
/// var is read at runtime. Setting only one leaves the other on its default, /// var is read at runtime. Setting only one leaves the other on its default,
/// which is how a hive ends up with two disagreeing answers for one value. /// which is how a hive ends up with two disagreeing answers for one value.
/// ///
/// The hive/swarm display names are here for exactly that reason, learned the /// The swarm display name is here for that reason: `otel.nix` reads the option
/// hard way: they were forwarded as runtime env only, while the agent's /// at eval time for the `swarm=` resource label, and with the env var alone
/// claude-settings module read them from the container's `environment.variables` /// that label evaluates to `unknown` while the process env holds the right
/// at *eval* time — where they were never set. Every agent baked /// name. Half-wiring this map is a value that evaluates fine and is silently
/// `hive=unknown,swarm=unknown` into its OTEL resource attributes and shipped /// wrong.
/// that label on every metric, while the same process's env held the right
/// answer. Half-wiring this map is not a missing nicety; it is a value that
/// evaluates fine and is silently wrong.
const FORWARDED_VAR_OPTIONS: &[(&str, &str)] = &[ const FORWARDED_VAR_OPTIONS: &[(&str, &str)] = &[
// Read at build time to decide whether the harness unit imports the hive's // Read at build time to decide whether the harness unit imports the hive's
// queue credential at all — see `nix/agent-modules/queue.nix`. Both halves // queue credential at all — see `nix/agent-modules/queue.nix`. Both halves
@ -558,7 +553,6 @@ const FORWARDED_VAR_OPTIONS: &[(&str, &str)] = &[
), ),
("HIVE_FORGE_URL", "services.hyperhive.agent.forge.url"), ("HIVE_FORGE_URL", "services.hyperhive.agent.forge.url"),
("HIVE_MATRIX_URL", "services.hyperhive.agent.matrix.url"), ("HIVE_MATRIX_URL", "services.hyperhive.agent.matrix.url"),
("HYPERHIVE_HIVE_NAME", "services.hyperhive.agent.hiveName"),
("HYPERHIVE_SWARM_NAME", "services.hyperhive.agent.swarmName"), ("HYPERHIVE_SWARM_NAME", "services.hyperhive.agent.swarmName"),
]; ];
@ -1164,9 +1158,7 @@ where
// → in-container accessors fall back to None / defaults gracefully. // → in-container accessors fall back to None / defaults gracefully.
// //
// - HIVE_FORGE_URL: agents poll this for Forgejo notifications. // - HIVE_FORGE_URL: agents poll this for Forgejo notifications.
// - HYPERHIVE_HIVE_DOMAIN: machine-readable hive DNS. // - HYPERHIVE_SWARM_NAME: human display name for the swarm.
// - HYPERHIVE_HIVE_NAME / HYPERHIVE_SWARM_NAME: human display
// names for hive + swarm.
for (var, val) in forwarded_env_vars() { for (var, val) in forwarded_env_vars() {
let escaped = val.replace('\\', "\\\\").replace('"', "\\\""); let escaped = val.replace('\\', "\\\\").replace('"', "\\\"");
let _ = writeln!(out, " {var} = \"{escaped}\";"); let _ = writeln!(out, " {var} = \"{escaped}\";");
@ -1770,6 +1762,53 @@ mod tests {
); );
} }
/// The hive's name and domain reach an agent at runtime
/// (`/etc/hyperhive-hive-facts`), so the rendered flake does not change with
/// either.
#[test]
fn render_flake_is_identical_across_hive_names_and_domains() {
const VARS: [&str; 2] = ["HYPERHIVE_HIVE_NAME", "HYPERHIVE_HIVE_DOMAIN"];
let _env = crate::test_env::lock();
let saved = VARS.map(std::env::var_os);
let render_on = |name: &str, domain: &str| {
// SAFETY: serialised by the guard above; restored before returning.
unsafe {
std::env::set_var(VARS[0], name);
std::env::set_var(VARS[1], domain);
}
render_flake(
"github:example/hyperhive",
"path:/nix/store/bbbb-hyperhive-docs-source",
"path:/nix/store/aaaa-nixpkgs-source",
None,
8000,
"she/her",
&std::collections::HashMap::new(),
"4G",
&[sample_spec("alice", false, 9001)],
)
.expect("render flake")
};
let alpha = render_on("alpha-hive", "alpha-hive.example.test");
let beta = render_on("beta-hive", "beta-hive.example.test");
for (var, value) in VARS.iter().zip(saved) {
// SAFETY: as above.
unsafe {
match value {
Some(v) => std::env::set_var(var, v),
None => std::env::remove_var(var),
}
}
}
for hive in ["alpha-hive", "beta-hive"] {
assert!(
!alpha.contains(hive) && !beta.contains(hive),
"the rendered flake must not name the hive ({hive}):\n{alpha}"
);
}
assert_eq!(alpha, beta, "the render must not depend on the hive");
}
#[test] #[test]
fn service_url_options_render_from_forwarded_pairs() { fn service_url_options_render_from_forwarded_pairs() {
// Pure over the pairs, deliberately: the render-level tests have to // Pure over the pairs, deliberately: the render-level tests have to
@ -1782,21 +1821,21 @@ mod tests {
&[ &[
("HIVE_FORGE_URL", "http://forge.example.test".to_string()), ("HIVE_FORGE_URL", "http://forge.example.test".to_string()),
("HIVE_MATRIX_URL", "http://matrix.example.test".to_string()), ("HIVE_MATRIX_URL", "http://matrix.example.test".to_string()),
// Mapped too, and that is the fix this test now guards: the // Mapped too: `otel.nix` reads the option at eval time.
// display names used to be forwarded as runtime env *only*, ("HYPERHIVE_SWARM_NAME", "constellat1on".to_string()),
// so the build-time reader fell back to "unknown" and every
// agent shipped that label on every metric.
("HYPERHIVE_HIVE_NAME", "pr1ma".to_string()),
// Forwarded but genuinely unmapped — proves the map is a // Forwarded but genuinely unmapped — proves the map is a
// filter, not a pass-through. // filter, not a pass-through.
("HYPERHIVE_HIVE_DOMAIN", "pr1ma.darkest.space".to_string()), (
"HIVE_FORGE_PUBLIC_URL",
"https://forge.example.test".to_string(),
),
], ],
); );
assert_eq!( assert_eq!(
out, out,
" services.hyperhive.agent.forge.url = \"http://forge.example.test\";\n\ " services.hyperhive.agent.forge.url = \"http://forge.example.test\";\n\
\x20 services.hyperhive.agent.matrix.url = \"http://matrix.example.test\";\n\ \x20 services.hyperhive.agent.matrix.url = \"http://matrix.example.test\";\n\
\x20 services.hyperhive.agent.hiveName = \"pr1ma\";\n", \x20 services.hyperhive.agent.swarmName = \"constellat1on\";\n",
"expected exactly the mapped options, in input order, indented for the module block" "expected exactly the mapped options, in input order, indented for the module block"
); );
} }
@ -1836,7 +1875,7 @@ mod tests {
fn require_service_urls_accepts_a_rendered_forge_url() { fn require_service_urls_accepts_a_rendered_forge_url() {
require_service_urls(&[ require_service_urls(&[
("HIVE_FORGE_URL", "http://forge.example.test".to_string()), ("HIVE_FORGE_URL", "http://forge.example.test".to_string()),
("HYPERHIVE_HIVE_NAME", "pr1ma".to_string()), ("HYPERHIVE_SWARM_NAME", "constellat1on".to_string()),
]) ])
.expect("a forwarded forge URL satisfies the requirement"); .expect("a forwarded forge URL satisfies the requirement");
} }
@ -1847,7 +1886,7 @@ mod tests {
// every agent would simply come up with no forge login and no way to // every agent would simply come up with no forge login and no way to
// tell that was unintended. The hive asserts its own requirement // tell that was unintended. The hive asserts its own requirement
// because it is the only layer that knows a forge is mandatory. // because it is the only layer that knows a forge is mandatory.
let err = require_service_urls(&[("HYPERHIVE_HIVE_NAME", "pr1ma".to_string())]) let err = require_service_urls(&[("HYPERHIVE_SWARM_NAME", "constellat1on".to_string())])
.expect_err("a missing forge URL must stop the write"); .expect_err("a missing forge URL must stop the write");
assert!( assert!(
err.to_string().contains("HIVE_FORGE_URL is unset"), err.to_string().contains("HIVE_FORGE_URL is unset"),
@ -1862,13 +1901,16 @@ mod tests {
// so absence is an absent integration rather than a misdirected one. // so absence is an absent integration rather than a misdirected one.
// Required services don't reach here: `require_service_urls` rejects // Required services don't reach here: `require_service_urls` rejects
// them first. // them first.
// `HYPERHIVE_HIVE_DOMAIN` is forwarded as env but carries no option — // `HIVE_FORGE_PUBLIC_URL` is forwarded as env but carries no option —
// deliberately picked over a mapped var, since the whole point is a // deliberately picked over a mapped var, since the whole point is a
// forwarded var the option map does not know about. // forwarded var the option map does not know about.
let mut out = String::new(); let mut out = String::new();
push_forwarded_var_options( push_forwarded_var_options(
&mut out, &mut out,
&[("HYPERHIVE_HIVE_DOMAIN", "pr1ma.darkest.space".to_string())], &[(
"HIVE_FORGE_PUBLIC_URL",
"https://forge.example.test".to_string(),
)],
); );
assert!( assert!(
out.is_empty(), out.is_empty(),

View file

@ -8,7 +8,7 @@
use anyhow::{Context as _, Result, bail}; use anyhow::{Context as _, Result, bail};
use hive_priv_sock::{ use hive_priv_sock::{
BindMount, CredentialMount, InfraAction, InfraContainer, MEASURE_AGENT_DISK_TIMEOUT, BindMount, CredentialMount, HiveFacts, InfraAction, InfraContainer, MEASURE_AGENT_DISK_TIMEOUT,
NetworkIsolation, PRIV_SOCK, PrivEvent, PrivRequest, PrivResponse, PrivStream, NetworkIsolation, PRIV_SOCK, PrivEvent, PrivRequest, PrivResponse, PrivStream,
}; };
use std::os::fd::{AsRawFd as _, OwnedFd, RawFd}; use std::os::fd::{AsRawFd as _, OwnedFd, RawFd};
@ -288,12 +288,14 @@ pub async fn write_nspawn_flags(
binds: &[BindMount], binds: &[BindMount],
isolation: NetworkIsolation, isolation: NetworkIsolation,
load_credentials: &[CredentialMount], load_credentials: &[CredentialMount],
hive_facts: HiveFacts,
) -> Result<()> { ) -> Result<()> {
ok(call(&PrivRequest::WriteNspawnFlags { ok(call(&PrivRequest::WriteNspawnFlags {
container: container.to_owned(), container: container.to_owned(),
binds: binds.to_vec(), binds: binds.to_vec(),
isolation, isolation,
load_credentials: load_credentials.to_vec(), load_credentials: load_credentials.to_vec(),
hive_facts,
}) })
.await?) .await?)
} }

View file

@ -232,6 +232,18 @@ pub struct NetworkIsolation {
pub gateway_ip: String, pub gateway_ip: String,
} }
/// Facts about the hive a container runs on, for `WriteNspawnFlags`.
/// hive-priv writes them into the container's `/etc/hyperhive-hive-facts`,
/// which the agent loads into its environment at boot, so no agent build
/// carries them. A `None` field leaves that variable unset in the agent.
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct HiveFacts {
/// `HYPERHIVE_HIVE_NAME`: the hive's name (`pr1ma`).
pub name: Option<String>,
/// `HYPERHIVE_HIVE_DOMAIN`: the hive's DNS domain.
pub domain: Option<String>,
}
/// Wall-clock bound hive-priv puts on one [`PrivRequest::MeasureAgentDisk`] /// Wall-clock bound hive-priv puts on one [`PrivRequest::MeasureAgentDisk`]
/// walk. A caller's own wait has to be longer than this, so a slow walk /// walk. A caller's own wait has to be longer than this, so a slow walk
/// reaches it as hive-priv's timeout error rather than as its own. /// reaches it as hive-priv's timeout error rather than as its own.
@ -306,6 +318,10 @@ pub enum PrivRequest {
/// so a hive-priv built before this field deserialises new requests. /// so a hive-priv built before this field deserialises new requests.
#[serde(default)] #[serde(default)]
load_credentials: Vec<CredentialMount>, load_credentials: Vec<CredentialMount>,
/// Written to the container's `/etc/hyperhive-hive-facts` on every
/// call, so a fact the hive no longer has is removed, not kept.
#[serde(default)]
hive_facts: HiveFacts,
}, },
/// Create the agent's socket dir `/run/hive-agent/<name>` as `0751 /// Create the agent's socket dir `/run/hive-agent/<name>` as `0751

View file

@ -22,10 +22,10 @@ use std::path::{Path, PathBuf};
use anyhow::{Context as _, Result, anyhow, bail}; use anyhow::{Context as _, Result, anyhow, bail};
use hive_priv_sock::{ use hive_priv_sock::{
AGENT_PREFIX, AGENT_RUNTIME_ROOT, AGENT_STATE_ROOT, BindMount, CredentialMount, InfraAction, AGENT_PREFIX, AGENT_RUNTIME_ROOT, AGENT_STATE_ROOT, BindMount, CredentialMount, HiveFacts,
InfraContainer, MEASURE_AGENT_DISK_TIMEOUT, META_DIR, MIGRATE_STAGING_ROOT, NetworkIsolation, InfraAction, InfraContainer, MEASURE_AGENT_DISK_TIMEOUT, META_DIR, MIGRATE_STAGING_ROOT,
PAUSED_MARKER_FILE, PRIV_SOCK, PrivEvent, PrivRequest, PrivResponse, PrivStream, NetworkIsolation, PAUSED_MARKER_FILE, PRIV_SOCK, PrivEvent, PrivRequest, PrivResponse,
PrivStreamLine, SIBLING_CONTAINERS, PrivStream, PrivStreamLine, SIBLING_CONTAINERS,
}; };
use tokio::io::{AsyncWriteExt, BufReader}; use tokio::io::{AsyncWriteExt, BufReader};
use tokio::net::unix::OwnedWriteHalf; use tokio::net::unix::OwnedWriteHalf;
@ -377,7 +377,8 @@ async fn exec(
ref binds, ref binds,
ref isolation, ref isolation,
ref load_credentials, ref load_credentials,
} => handle_write_nspawn_flags(container, binds, isolation, load_credentials), ref hive_facts,
} => handle_write_nspawn_flags(container, binds, isolation, load_credentials, hive_facts),
PrivRequest::EnsureAgentSocketDir { ref name } => ensure_agent_socket_dir(name), PrivRequest::EnsureAgentSocketDir { ref name } => ensure_agent_socket_dir(name),
@ -984,6 +985,7 @@ fn handle_write_nspawn_flags(
binds: &[BindMount], binds: &[BindMount],
isolation: &NetworkIsolation, isolation: &NetworkIsolation,
load_credentials: &[CredentialMount], load_credentials: &[CredentialMount],
hive_facts: &HiveFacts,
) -> Result<(String, String)> { ) -> Result<(String, String)> {
validate_container_system_name(container)?; validate_container_system_name(container)?;
for bind in binds { for bind in binds {
@ -997,7 +999,8 @@ fn handle_write_nspawn_flags(
// uses `:` as the name/path separator. // uses `:` as the name/path separator.
validate_bind_path(&cred.host_path)?; validate_bind_path(&cred.host_path)?;
} }
write_nspawn_flags(container, binds, isolation, load_credentials)?; let hive_facts = render_hive_facts(hive_facts)?;
write_nspawn_flags(container, binds, isolation, load_credentials, &hive_facts)?;
Ok((String::new(), String::new())) Ok((String::new(), String::new()))
} }
@ -2870,6 +2873,7 @@ fn write_nspawn_flags(
binds: &[BindMount], binds: &[BindMount],
isolation: &NetworkIsolation, isolation: &NetworkIsolation,
load_credentials: &[CredentialMount], load_credentials: &[CredentialMount],
hive_facts: &str,
) -> Result<()> { ) -> Result<()> {
use std::fmt::Write as _; use std::fmt::Write as _;
use std::os::unix::fs::MetadataExt as _; use std::os::unix::fs::MetadataExt as _;
@ -2950,36 +2954,78 @@ fn write_nspawn_flags(
// and which unit does it: `docs/networking/network.md` § *How the isolated // and which unit does it: `docs/networking/network.md` § *How the isolated
// container gets its resolver*. // container gets its resolver*.
write_bridge_dns_marker(container, isolation)?; write_bridge_dns_marker(container, isolation)?;
// Read by the agent's environment generator at boot
// (`nix/agent-modules/hive-facts.nix`).
write_container_etc_file_in(&container_rootfs(container), HIVE_FACTS_FILE, hive_facts)?;
Ok(()) Ok(())
} }
/// Filename of the in-container DNS marker, in the container's own `/etc`. /// Filename of the in-container DNS marker, in the container's own `/etc`.
const BRIDGE_DNS_MARKER: &str = "hyperhive-bridge-dns"; const BRIDGE_DNS_MARKER: &str = "hyperhive-bridge-dns";
/// Filename of the hive-facts environment file, in the container's own `/etc`.
const HIVE_FACTS_FILE: &str = "hyperhive-hive-facts";
/// The container's `/` on the host.
fn container_rootfs(container: &str) -> PathBuf {
PathBuf::from(format!("/var/lib/nixos-containers/{container}"))
}
/// Render [`HiveFacts`] as the systemd environment file the agent loads,
/// one `KEY=value` line per present fact.
///
/// A value is refused unless it is DNS-name characters only, so it can carry
/// neither a newline (a second assignment) nor anything an environment file
/// or a shell would interpret.
fn render_hive_facts(facts: &HiveFacts) -> Result<String> {
use std::fmt::Write as _;
let mut out = String::new();
for (key, value) in [
("HYPERHIVE_HIVE_NAME", &facts.name),
("HYPERHIVE_HIVE_DOMAIN", &facts.domain),
] {
let Some(value) = value else { continue };
if value.is_empty()
|| value.len() > 253
|| !value
.bytes()
.all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'-' | b'_'))
{
bail!("invalid {key} value {value:?}: must be 1-253 chars of [A-Za-z0-9._-]");
}
let _ = writeln!(out, "{key}={value}");
}
Ok(out)
}
/// Write the bridge-DNS marker the `hyperhive-isolated-dns` oneshot keys /// Write the bridge-DNS marker the `hyperhive-isolated-dns` oneshot keys
/// off. The marker file contains just the gateway IP. Always written: /// off. The marker file contains just the gateway IP. Always written:
/// every container is isolated, so there is no host-netns case that /// every container is isolated, so there is no host-netns case that
/// wants the marker absent. /// wants the marker absent.
fn write_bridge_dns_marker(container: &str, isolation: &NetworkIsolation) -> Result<()> { fn write_bridge_dns_marker(container: &str, isolation: &NetworkIsolation) -> Result<()> {
let rootfs = PathBuf::from(format!("/var/lib/nixos-containers/{container}")); write_bridge_dns_marker_in(&container_rootfs(container), &isolation.gateway_ip)
write_bridge_dns_marker_in(&rootfs, &isolation.gateway_ip)
} }
/// Write `<rootfs>/etc/hyperhive-bridge-dns` without following a symlink. /// Write `<rootfs>/etc/hyperhive-bridge-dns`; see [`write_container_etc_file_in`].
fn write_bridge_dns_marker_in(rootfs: &Path, gateway_ip: &str) -> Result<()> {
write_container_etc_file_in(rootfs, BRIDGE_DNS_MARKER, &format!("{gateway_ip}\n"))
}
/// Write `<rootfs>/etc/<name>` without following a symlink.
/// ///
/// Root inside the container owns its `/etc`, and this write runs as host /// Root inside the container owns its `/etc`, and this write runs as host
/// root, where an absolute symlink planted in the container resolves against /// root, where an absolute symlink planted in the container resolves against
/// the host's `/`. So this opens `etc` with `O_DIRECTORY|O_NOFOLLOW` (a /// the host's `/`. So this opens `etc` with `O_DIRECTORY|O_NOFOLLOW` (a
/// symlinked `etc` fails with `ELOOP`) and publishes the marker through a /// symlinked `etc` fails with `ELOOP`) and publishes the file through a
/// [`StagedFile`] relative to that fd: whatever the container planted at the /// [`StagedFile`] relative to that fd: whatever the container planted at the
/// leaf, symlink or FIFO or device node, is replaced by the rename, never /// leaf, symlink or FIFO or device node, is replaced by the rename, never
/// written through or opened. /// written through or opened.
/// ///
/// On a fresh install the container's `/etc` may not exist yet (nixos-container /// On a fresh install the container's `/etc` may not exist yet (nixos-container
/// materialises the rootfs on the first start), so this creates it first; the /// materialises the rootfs on the first start), so this creates it first; the
/// dir and the marker persist through that start. Mode 0644: the gateway IP /// dir and the file persist through that start. Mode 0644: nothing written
/// is not a secret. /// here is a secret.
fn write_bridge_dns_marker_in(rootfs: &Path, gateway_ip: &str) -> Result<()> { fn write_container_etc_file_in(rootfs: &Path, name: &str, contents: &str) -> Result<()> {
use std::os::unix::fs::OpenOptionsExt as _; use std::os::unix::fs::OpenOptionsExt as _;
let etc_path = rootfs.join("etc"); let etc_path = rootfs.join("etc");
@ -3014,14 +3060,7 @@ fn write_bridge_dns_marker_in(rootfs: &Path, gateway_ip: &str) -> Result<()> {
} }
// SAFETY: `etc_fd` is a fresh, valid fd that nothing else owns. // SAFETY: `etc_fd` is a fresh, valid fd that nothing else owns.
let etc = std::fs::File::from(unsafe { OwnedFd::from_raw_fd(etc_fd) }); let etc = std::fs::File::from(unsafe { OwnedFd::from_raw_fd(etc_fd) });
StagedFile::write( StagedFile::write(&etc, &etc_path, name, contents.as_bytes(), 0o644)?.publish()
&etc,
&etc_path,
BRIDGE_DNS_MARKER,
format!("{gateway_ip}\n").as_bytes(),
0o644,
)?
.publish()
} }
/// Written by older hive-priv builds. Nothing writes it now, but left on disk /// Written by older hive-priv builds. Nothing writes it now, but left on disk
@ -3124,13 +3163,14 @@ fn ensure_socket_dir_in(root: &Path, name: &str) -> Result<()> {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::{ use super::{
BindMount, BoundedRun, Command, OwnedFd, PAUSED_MARKER_FILE, PrivRequest, StagedFile, BindMount, BoundedRun, Command, HiveFacts, OwnedFd, PAUSED_MARKER_FILE, PrivRequest,
agent_disk_paths, check_fd_agreement, contains_secret_shaped_run, describe_forge_admin, StagedFile, agent_disk_paths, check_fd_agreement, contains_secret_shaped_run,
du_command, du_total, ensure_plain_filename, ensure_socket_dir_in, exec_measure_agent_disk, describe_forge_admin, du_command, du_total, ensure_plain_filename, ensure_socket_dir_in,
git_overlay_flags, limits_dropin_body, open_dir, open_export_dest, parse_du_total, exec_measure_agent_disk, git_overlay_flags, limits_dropin_body, open_dir, open_export_dest,
partial_name, publish_file, redact_secret_line, remove_marker_in, run_bounded, parse_du_total, partial_name, publish_file, redact_secret_line, remove_marker_in,
single_output_path, toplevel_attr, validate_credential_name, validate_snapshot_name, render_hive_facts, run_bounded, single_output_path, toplevel_attr,
write_agent_dir_file, write_bridge_dns_marker_in, validate_credential_name, validate_snapshot_name, write_agent_dir_file,
write_bridge_dns_marker_in,
}; };
use std::path::PathBuf; use std::path::PathBuf;
use std::sync::atomic::{AtomicU32, Ordering}; use std::sync::atomic::{AtomicU32, Ordering};
@ -3782,6 +3822,45 @@ mod tests {
std::fs::remove_dir_all(&dir).ok(); std::fs::remove_dir_all(&dir).ok();
} }
/// Each present fact is one environment-file line; an absent one writes
/// nothing, so the agent sees the variable unset rather than empty.
#[test]
fn hive_facts_render_one_line_per_present_fact() {
let both = HiveFacts {
name: Some("pr1ma".to_owned()),
domain: Some("pr1ma.darkest.space".to_owned()),
};
assert_eq!(
render_hive_facts(&both).unwrap(),
"HYPERHIVE_HIVE_NAME=pr1ma\nHYPERHIVE_HIVE_DOMAIN=pr1ma.darkest.space\n"
);
let domain_only = HiveFacts {
name: None,
domain: Some("h1.example.test".to_owned()),
};
assert_eq!(
render_hive_facts(&domain_only).unwrap(),
"HYPERHIVE_HIVE_DOMAIN=h1.example.test\n"
);
assert_eq!(render_hive_facts(&HiveFacts::default()).unwrap(), "");
}
/// A value that could end its line or be read as syntax is refused, not
/// escaped: a newline would add an assignment of the caller's choosing.
#[test]
fn hive_facts_refuse_values_outside_dns_characters() {
for bad in ["", "pr1ma\nLD_PRELOAD=/x", "a b", "a\"b", "$(x)"] {
let facts = HiveFacts {
name: Some(bad.to_owned()),
domain: None,
};
assert!(
render_hive_facts(&facts).is_err(),
"{bad:?} must be refused"
);
}
}
/// Whether `pid` is gone, or only a zombie awaiting its reaper: either /// Whether `pid` is gone, or only a zombie awaiting its reaper: either
/// way it is no longer running. /// way it is no longer running.
fn is_dead(pid: &str) -> bool { fn is_dead(pid: &str) -> bool {

View file

@ -19,17 +19,6 @@ let
# meta.rs `otel_config`). Options declared in `otel.nix`, which also # meta.rs `otel_config`). Options declared in `otel.nix`, which also
# exports the generic OTLP environment this container's producers read. # exports the generic OTLP environment this container's producers read.
otelCfg = config.services.hyperhive.agent.otel; otelCfg = config.services.hyperhive.agent.otel;
# Hive display name, read from the per-agent option meta.rs renders
# (NOT from `environment.variables` — that carries the same name at
# *runtime* only, so reading it here silently yielded "unknown" on
# every agent while the process env held the right answer). `null`
# means the hive did not name itself; "unknown" is then an honest label
# rather than a guess.
hiveDisplayName =
if config.services.hyperhive.agent.hiveName == null then
"unknown"
else
config.services.hyperhive.agent.hiveName;
# Effective per-agent MemoryMax=, in bytes, injected by meta.rs's # Effective per-agent MemoryMax=, in bytes, injected by meta.rs's
# per-agent flake render (`services.hyperhive.agent.claudeMemoryMaxBytes`). `null` # per-agent flake render (`services.hyperhive.agent.claudeMemoryMaxBytes`). `null`
# when the effective cap is unbounded ("infinity") or a RAM # when the effective cap is unbounded ("infinity") or a RAM
@ -58,8 +47,6 @@ let
CLAUDE_CODE_RESUME_INTERRUPTED_TURN = "1"; CLAUDE_CODE_RESUME_INTERRUPTED_TURN = "1";
# Use the simpler system prompt variant suited to headless operation. # Use the simpler system prompt variant suited to headless operation.
CLAUDE_CODE_SIMPLE_SYSTEM_PROMPT = "1"; CLAUDE_CODE_SIMPLE_SYSTEM_PROMPT = "1";
# Tag remote-control sessions with "<hive>-<agent>" for identification.
CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX = "${hiveDisplayName}-${userName}";
} }
# Bun/JavaScriptCore (the runtime under claude-code's `.claude-wrapped` # Bun/JavaScriptCore (the runtime under claude-code's `.claude-wrapped`
# binary) sizes its default heap ceiling off the container's visible # binary) sizes its default heap ceiling off the container's visible
@ -97,6 +84,21 @@ let
}; };
in in
{ {
options.services.hyperhive.agent.swarmName = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
internal = true;
description = ''
Human-readable swarm name, rendered per-agent by
`meta.rs::render_flake` from the host's
`services.hyperhive.swarm.name`. Read during evaluation for the
OTEL `swarm` resource label; `HYPERHIVE_SWARM_NAME` carries the
same value at runtime.
`null` means the hive is not part of a named swarm.
'';
};
# Build-time implementation surface for the JSC-heap-ceiling fix: # Build-time implementation surface for the JSC-heap-ceiling fix:
# meta.rs's per-agent flake render injects this from the effective # meta.rs's per-agent flake render injects this from the effective
# `MemoryMax=` (per-agent `resource-limits.json` override, else the # `MemoryMax=` (per-agent `resource-limits.json` override, else the
@ -108,37 +110,6 @@ in
# the agent's *last rebuild* — `set-limits` still applies the # the agent's *last rebuild* — `set-limits` still applies the
# cgroup cap live via a drop-in reload, but this derived heap ceiling # cgroup cap live via a drop-in reload, but this derived heap ceiling
# needs a rebuild to pick up a new value. # needs a rebuild to pick up a new value.
options.services.hyperhive.agent.hiveName = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
internal = true;
description = ''
Human-readable hive name, rendered per-agent by
`meta.rs::render_flake` from the host's
`services.hyperhive.hiveName`. Baked into the OTEL resource
attributes at build time, which is why it is an option and not
just the `HYPERHIVE_HIVE_NAME` env var: the env var is read at
runtime, this is read during evaluation, and wiring only one of
the two is how every agent ended up reporting `hive=unknown`.
`null` means the hive did not name itself.
'';
};
options.services.hyperhive.agent.swarmName = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
internal = true;
description = ''
Human-readable swarm name, rendered per-agent by
`meta.rs::render_flake` from the host's
`services.hyperhive.swarm.name`. Same build-time/runtime split as
`services.hyperhive.agent.hiveName`.
`null` means the hive is not part of a named swarm.
'';
};
options.services.hyperhive.agent.claudeMemoryMaxBytes = lib.mkOption { options.services.hyperhive.agent.claudeMemoryMaxBytes = lib.mkOption {
type = lib.types.nullOr lib.types.ints.positive; type = lib.types.nullOr lib.types.ints.positive;
default = null; default = null;
@ -279,7 +250,7 @@ in
# Hive-enforced settings merged with a per-agent `env` block at BUILD # Hive-enforced settings merged with a per-agent `env` block at BUILD
# time via `jq` (not eval-time `readFile`, which would be import-from- # time via `jq` (not eval-time `readFile`, which would be import-from-
# derivation). The `env` block is always present: `baseClaudeEnv` sets # derivation). The `env` block is always present: `baseClaudeEnv` sets
# behaviour flags and the remote-control session prefix for every agent; # behaviour flags for every agent;
# `otelSettingsEnv` is merged on top when OTEL is enabled. claude-code # `otelSettingsEnv` is merged on top when OTEL is enabled. claude-code
# auto-discovers this file in every context (harness turn-loop AND # auto-discovers this file in every context (harness turn-loop AND
# `hivectl choom`) so no launch wrapper is needed. # `hivectl choom`) so no launch wrapper is needed.
@ -287,9 +258,8 @@ in
let let
baseSettings = "${config.services.hyperhive.agent.packages.assets}/share/hyperhive/prompts/claude-settings.json"; baseSettings = "${config.services.hyperhive.agent.packages.assets}/share/hyperhive/prompts/claude-settings.json";
# Merge base env (always) with OTEL env (when enabled). jq is always # Merge base env (always) with OTEL env (when enabled). jq is always
# run — `baseClaudeEnv` contains per-agent values (e.g. # run — `baseClaudeEnv` can carry per-agent values (e.g.
# CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX) that can't live in the # BUN_JSC_forceRAMSize) that can't live in the static store asset.
# static store asset.
allEnv = allEnv =
baseClaudeEnv baseClaudeEnv
// lib.optionalAttrs otelCfg.enable otelSettingsEnv // lib.optionalAttrs otelCfg.enable otelSettingsEnv

View file

@ -36,6 +36,11 @@ let
See docs/tools/matrix.md. See docs/tools/matrix.md.
''; '';
hiveNameRemoved = ''
An agent reads its hive's name at runtime from /etc/hyperhive-hive-facts,
which the hive writes into every container, so no agent build sets it.
Remove the setting.
'';
in in
{ {
imports = [ imports = [
@ -51,6 +56,7 @@ in
./frontend.nix ./frontend.nix
./github.nix ./github.nix
./github-token.nix ./github-token.nix
./hive-facts.nix
./logs.nix ./logs.nix
./matrix.nix ./matrix.nix
./mcp.nix ./mcp.nix
@ -85,6 +91,9 @@ in
# breaks evaluation for everyone, removed or not. # breaks evaluation for everyone, removed or not.
(lib.mkRemovedOptionModule [ "hyperhive" "matrix" "enable" ] matrixEnableRemoved) (lib.mkRemovedOptionModule [ "hyperhive" "matrix" "enable" ] matrixEnableRemoved)
(lib.mkRemovedOptionModule [ "services" "hyperhive" "agent" "matrix" "enable" ] matrixEnableRemoved) (lib.mkRemovedOptionModule [ "services" "hyperhive" "agent" "matrix" "enable" ] matrixEnableRemoved)
# Both spellings, for the reason given above.
(lib.mkRemovedOptionModule [ "hyperhive" "hiveName" ] hiveNameRemoved)
(lib.mkRemovedOptionModule [ "services" "hyperhive" "agent" "hiveName" ] hiveNameRemoved)
]; ];
options.services.hyperhive.agent.icon = lib.mkOption { options.services.hyperhive.agent.icon = lib.mkOption {

View file

@ -0,0 +1,37 @@
# The hive this container runs on, read at boot rather than built in, so an
# agent's closure does not carry it.
#
# hive-priv writes `/etc/hyperhive-hive-facts` into the container before every
# start (`HYPERHIVE_HIVE_NAME`, `HYPERHIVE_HIVE_DOMAIN`, one `KEY=value` line
# each). This environment generator hands those lines to the container's
# systemd, which passes them to every unit it starts and to `machinectl shell`
# sessions; a unit's own `Environment=` still overrides them.
{
config,
lib,
pkgs,
...
}:
let
userName = config.services.hyperhive.agent.user.name;
in
{
# Shell builtins only: nothing is assumed on PATH this early. Only the two
# known keys are passed on.
environment.etc."systemd/system-environment-generators/50-hyperhive-hive-facts".source =
pkgs.writeShellScript "hyperhive-hive-facts" ''
hive=unknown
if [ -r /etc/hyperhive-hive-facts ]; then
while IFS='=' read -r key value; do
case "$key" in
HYPERHIVE_HIVE_NAME) hive=$value ;;
HYPERHIVE_HIVE_DOMAIN) ;;
*) continue ;;
esac
printf '%s=%s\n' "$key" "$value"
done < /etc/hyperhive-hive-facts
fi
# Names claude's remote-control sessions `<hive>-<agent>`.
printf 'CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=%s-%s\n' "$hive" ${lib.escapeShellArg userName}
'';
}

View file

@ -25,17 +25,10 @@
let let
cfg = config.services.hyperhive.agent.otel; cfg = config.services.hyperhive.agent.otel;
userName = config.services.hyperhive.agent.user.name; userName = config.services.hyperhive.agent.user.name;
# Hive/swarm display names, read from the per-agent options meta.rs # Swarm display name, read from the per-agent option meta.rs renders
# renders (NOT from `environment.variables` — those carry the same # (NOT from `environment.variables`, which carries it at *runtime*
# names at *runtime* only, so reading them here silently yielded # only). `null` means the swarm is unnamed; "unknown" is then an honest
# "unknown" on every agent while the process env held the right # label rather than a guess.
# answer). `null` means the hive did not name itself; "unknown" is then
# an honest label rather than a guess.
hiveDisplayName =
if config.services.hyperhive.agent.hiveName == null then
"unknown"
else
config.services.hyperhive.agent.hiveName;
swarmDisplayName = swarmDisplayName =
if config.services.hyperhive.agent.swarmName == null then if config.services.hyperhive.agent.swarmName == null then
"unknown" "unknown"
@ -45,9 +38,11 @@ let
# Resource labels every producer in this container stamps on what it # Resource labels every producer in this container stamps on what it
# emits. `service.name` names the container's role, not one binary # emits. `service.name` names the container's role, not one binary
# inside it: claude's samples and `hive-metric`'s both come from this # inside it: claude's samples and `hive-metric`'s both come from this
# agent, and their metric names already tell them apart. # agent, and their metric names already tell them apart. No `hive`: the
# swarm collector stamps it from the receiver that accepted the hive's
# push (nix/host-modules/swarm-otel.nix), so the build names no hive.
resourceAttributes = resourceAttributes =
"service.name=hyperhive-agent,agent=${userName},hive=${hiveDisplayName},swarm=${swarmDisplayName}" "service.name=hyperhive-agent,agent=${userName},swarm=${swarmDisplayName}"
+ lib.optionalString (cfg.extraResourceAttributes != "") ",${cfg.extraResourceAttributes}"; + lib.optionalString (cfg.extraResourceAttributes != "") ",${cfg.extraResourceAttributes}";
# The variables an OTEL SDK reads on its own, in any language, in any # The variables an OTEL SDK reads on its own, in any language, in any
@ -131,7 +126,7 @@ in
description = '' description = ''
Extra comma-separated entries appended to Extra comma-separated entries appended to
`OTEL_RESOURCE_ATTRIBUTES` after the built-in `OTEL_RESOURCE_ATTRIBUTES` after the built-in
`service.name` / `agent` / `hive` / `swarm` labels. `service.name` / `agent` / `swarm` labels.
Host-driven via `services.hyperhive.otel.extraResourceAttributes`. Host-driven via `services.hyperhive.otel.extraResourceAttributes`.
''; '';
}; };
@ -274,10 +269,9 @@ in
# Identity the hop above cannot supply. A host-side reader can say # Identity the hop above cannot supply. A host-side reader can say
# which machine a line came from; only a collector running as the # which machine a line came from; only a collector running as the
# agent can say which agent. `hive` and `swarm` are deliberately # agent can say which agent. `hive` and `swarm` are deliberately
# NOT stamped here even though the env above carries them for # NOT stamped here: the swarm tier upserts both from the receiver
# in-process producers: the swarm tier upserts `hive` from the # that accepted the sample, precisely so the labels come from
# receiver that accepted the sample, precisely so the label comes # something the sender cannot write.
# from something the sender cannot write.
processors.resource.attributes = [ processors.resource.attributes = [
{ {
key = "service.name"; key = "service.name";

View file

@ -112,7 +112,6 @@
[ "hyperhive" "gui" "vncPort" ] [ "hyperhive" "gui" "vncPort" ]
[ "services" "hyperhive" "agent" "gui" "vncPort" ] [ "services" "hyperhive" "agent" "gui" "vncPort" ]
) )
(lib.mkRenamedOptionModule [ "hyperhive" "hiveName" ] [ "services" "hyperhive" "agent" "hiveName" ])
(lib.mkRenamedOptionModule [ "hyperhive" "icon" ] [ "services" "hyperhive" "agent" "icon" ]) (lib.mkRenamedOptionModule [ "hyperhive" "icon" ] [ "services" "hyperhive" "agent" "icon" ])
(lib.mkRenamedOptionModule (lib.mkRenamedOptionModule
[ "hyperhive" "logs" "queryUrl" ] [ "hyperhive" "logs" "queryUrl" ]