agents: read the hive's name and domain at runtime, not from the build
Part of #5081 (slice S0-1a). An agent's closure no longer depends on the name or domain of the hive that renders it. - hive-priv writes /etc/hyperhive-hive-facts (HYPERHIVE_HIVE_NAME, HYPERHIVE_HIVE_DOMAIN) into the container from the same WriteNspawnFlags call that writes /etc/hyperhive-bridge-dns. hive-c0re passes the values from its own unit env. hive-priv refuses a value outside [A-Za-z0-9._-]. - A systemd environment generator in the agent (hive-facts.nix) loads the file into the container manager's environment, so every unit sees it, as with the systemd.globalEnvironment entries the render used to emit. It also builds CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX (<hive>-<agent>), which left claude's managed settings. - meta.rs stops forwarding HYPERHIVE_HIVE_{NAME,DOMAIN} and stops setting services.hyperhive.agent.hiveName. The option is removed (both spellings now fail with a removal message). - otel.nix drops the agent's hive= resource label. The swarm collector upserts hive per authenticated receiver (swarm-otel.nix resource/<h>, on both the metrics and logs pipelines), so stored series keep it. The otel first hop, otel enable and the dashboard port are still baked; they are S0-1b.
This commit is contained in:
atlas
mara
parent
80e8361da9
commit
c87a4d2c1a
14 changed files with 303 additions and 127 deletions
|
|
@ -38,6 +38,13 @@ Four env vars cover the identity surface, read by
|
|||
federated hives at different DNS domains can share a swarm
|
||||
name.
|
||||
|
||||
Inside an agent container, the hive's name and domain don't come from the
|
||||
agent's build. `hive-priv` writes them to `/etc/hyperhive-hive-facts` before
|
||||
every container start, and the agent's environment generator
|
||||
(`nix/agent-modules/hive-facts.nix`) passes them to every unit. Neither is
|
||||
part of an agent's closure, and an agent started on another hive reports
|
||||
that hive.
|
||||
|
||||
`hive_name` + `swarm_name` are **distinct** from
|
||||
`HYPERHIVE_HIVE_DOMAIN`: the domain may carry the hive name as
|
||||
its leftmost label by convention, but the convention isn't
|
||||
|
|
|
|||
|
|
@ -193,9 +193,12 @@ automatically:
|
|||
| -------------- | ------------------------------------------------------------ |
|
||||
| `service.name` | `hyperhive-agent` (constant) |
|
||||
| `agent` | agent logical name (for example `iris`) |
|
||||
| `hive` | hive display name (`services.hyperhive.hiveName`) |
|
||||
| `swarm` | swarm display name (`services.hyperhive.swarm.name`, if set) |
|
||||
|
||||
The swarm's collector adds `hive`, the name of the hive whose receiver accepted
|
||||
the push (see [Authenticated ingest](#authenticated-ingest)). The container
|
||||
doesn't set it.
|
||||
|
||||
Append additional labels via `extraResourceAttributes` (see option
|
||||
reference above); pass custom per-data-point labels with
|
||||
`hive-metric --labels` (see below).
|
||||
|
|
@ -368,10 +371,10 @@ API-reported window, else the model's default. A turn that parsed no usage
|
|||
leaves both where the previous turn put them. The dashboard's **Context
|
||||
window used by agent** panel (health tab) divides one by the other.
|
||||
|
||||
Resource attributes (`service.name`, `agent`, `hive`, `swarm`) come from the
|
||||
Resource attributes (`service.name`, `agent`, `swarm`) come from the
|
||||
same container-wide `OTEL_RESOURCE_ATTRIBUTES` as everything else in this
|
||||
section — nothing extra to configure. Cadence follows
|
||||
`HYPERHIVE_OTEL_METRIC_INTERVAL_MS` (default 60s, same variable + default as
|
||||
section, and the swarm's collector adds `hive` — nothing extra to configure.
|
||||
Cadence follows `HYPERHIVE_OTEL_METRIC_INTERVAL_MS` (default 60s, same variable + default as
|
||||
`hive-c0re`'s container-resource export above) — that only controls how often
|
||||
the harness flushes the batched points to the collector, not how often it
|
||||
records them (every turn, always).
|
||||
|
|
@ -479,7 +482,7 @@ hive-metric <name> <value> [--type counter|gauge] [--temporality delta|cumulativ
|
|||
itself).
|
||||
- `--labels key=value` — extra per-data-point labels. Repeat the flag for
|
||||
more than one.
|
||||
`hive-metric` inherits the resource labels (agent, hive, swarm, service.name)
|
||||
`hive-metric` inherits the resource labels (agent, swarm, service.name)
|
||||
automatically from `OTEL_RESOURCE_ATTRIBUTES` — don't re-specify them.
|
||||
|
||||
### Examples
|
||||
|
|
|
|||
|
|
@ -260,7 +260,7 @@ needs to `chown` a bind mount), once at startup:
|
|||
substituted: `{label}` (short agent name), `{qualified_label}`
|
||||
(hive-qualified `name@domain` form), `{operator_pronouns}`,
|
||||
`{hive_identity}` (for example `` on hive `pr1ma` ``; empty when
|
||||
`services.hyperhive.agent.hiveName` is unset), and `{swarm_identity}` (same
|
||||
`HYPERHIVE_HIVE_NAME` is unset), and `{swarm_identity}` (same
|
||||
shape for the swarm). Pronouns come from `HIVE_OPERATOR_PRONOUNS`
|
||||
env (set by the meta flake from
|
||||
`services.hyperhive.c0re.operatorPronouns`, default `she/her`).
|
||||
|
|
|
|||
|
|
@ -476,7 +476,14 @@ async fn set_nspawn_flags(
|
|||
);
|
||||
|
||||
// Delegate the actual conf-file rewrite to hive-priv (runs as root).
|
||||
crate::priv_client::write_nspawn_flags(container, &binds, isolation, &load_creds).await
|
||||
crate::priv_client::write_nspawn_flags(
|
||||
container,
|
||||
&binds,
|
||||
isolation,
|
||||
&load_creds,
|
||||
super::hive_facts_from_env(),
|
||||
)
|
||||
.await
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
|
|
|||
|
|
@ -177,6 +177,17 @@ pub fn network_isolation_from_env() -> Result<hive_priv_sock::NetworkIsolation>
|
|||
network_isolation_from_vars(bridge.as_deref(), subnet.as_deref())
|
||||
}
|
||||
|
||||
/// The hive's facts every container is handed at runtime, from hive-c0re's
|
||||
/// own unit environment (`nix/host-modules/hive-c0re/environment.nix`). An
|
||||
/// unset or empty variable is `None`.
|
||||
fn hive_facts_from_env() -> hive_priv_sock::HiveFacts {
|
||||
let read = |name: &str| std::env::var(name).ok().filter(|v| !v.is_empty());
|
||||
hive_priv_sock::HiveFacts {
|
||||
name: read("HYPERHIVE_HIVE_NAME"),
|
||||
domain: read("HYPERHIVE_HIVE_DOMAIN"),
|
||||
}
|
||||
}
|
||||
|
||||
fn validate(name: &str) -> Result<()> {
|
||||
if name.is_empty() {
|
||||
bail!("agent name must not be empty");
|
||||
|
|
|
|||
|
|
@ -508,8 +508,6 @@ const FORWARDED_VARS: &[&str] = &[
|
|||
"HIVE_FORGE_URL",
|
||||
"HIVE_FORGE_PUBLIC_URL",
|
||||
"HIVE_MATRIX_URL",
|
||||
"HYPERHIVE_HIVE_DOMAIN",
|
||||
"HYPERHIVE_HIVE_NAME",
|
||||
"HYPERHIVE_SWARM_NAME",
|
||||
];
|
||||
|
||||
|
|
@ -520,14 +518,11 @@ const FORWARDED_VARS: &[&str] = &[
|
|||
/// var is read at runtime. Setting only one leaves the other on its default,
|
||||
/// which is how a hive ends up with two disagreeing answers for one value.
|
||||
///
|
||||
/// The hive/swarm display names are here for exactly that reason, learned the
|
||||
/// hard way: they were forwarded as runtime env only, while the agent's
|
||||
/// claude-settings module read them from the container's `environment.variables`
|
||||
/// at *eval* time — where they were never set. Every agent baked
|
||||
/// `hive=unknown,swarm=unknown` into its OTEL resource attributes and shipped
|
||||
/// that label on every metric, while the same process's env held the right
|
||||
/// answer. Half-wiring this map is not a missing nicety; it is a value that
|
||||
/// evaluates fine and is silently wrong.
|
||||
/// The swarm display name is here for that reason: `otel.nix` reads the option
|
||||
/// at eval time for the `swarm=` resource label, and with the env var alone
|
||||
/// that label evaluates to `unknown` while the process env holds the right
|
||||
/// name. Half-wiring this map is a value that evaluates fine and is silently
|
||||
/// wrong.
|
||||
const FORWARDED_VAR_OPTIONS: &[(&str, &str)] = &[
|
||||
// Read at build time to decide whether the harness unit imports the hive's
|
||||
// queue credential at all — see `nix/agent-modules/queue.nix`. Both halves
|
||||
|
|
@ -558,7 +553,6 @@ const FORWARDED_VAR_OPTIONS: &[(&str, &str)] = &[
|
|||
),
|
||||
("HIVE_FORGE_URL", "services.hyperhive.agent.forge.url"),
|
||||
("HIVE_MATRIX_URL", "services.hyperhive.agent.matrix.url"),
|
||||
("HYPERHIVE_HIVE_NAME", "services.hyperhive.agent.hiveName"),
|
||||
("HYPERHIVE_SWARM_NAME", "services.hyperhive.agent.swarmName"),
|
||||
];
|
||||
|
||||
|
|
@ -1164,9 +1158,7 @@ where
|
|||
// → in-container accessors fall back to None / defaults gracefully.
|
||||
//
|
||||
// - HIVE_FORGE_URL: agents poll this for Forgejo notifications.
|
||||
// - HYPERHIVE_HIVE_DOMAIN: machine-readable hive DNS.
|
||||
// - HYPERHIVE_HIVE_NAME / HYPERHIVE_SWARM_NAME: human display
|
||||
// names for hive + swarm.
|
||||
// - HYPERHIVE_SWARM_NAME: human display name for the swarm.
|
||||
for (var, val) in forwarded_env_vars() {
|
||||
let escaped = val.replace('\\', "\\\\").replace('"', "\\\"");
|
||||
let _ = writeln!(out, " {var} = \"{escaped}\";");
|
||||
|
|
@ -1770,6 +1762,53 @@ mod tests {
|
|||
);
|
||||
}
|
||||
|
||||
/// The hive's name and domain reach an agent at runtime
|
||||
/// (`/etc/hyperhive-hive-facts`), so the rendered flake does not change with
|
||||
/// either.
|
||||
#[test]
|
||||
fn render_flake_is_identical_across_hive_names_and_domains() {
|
||||
const VARS: [&str; 2] = ["HYPERHIVE_HIVE_NAME", "HYPERHIVE_HIVE_DOMAIN"];
|
||||
let _env = crate::test_env::lock();
|
||||
let saved = VARS.map(std::env::var_os);
|
||||
let render_on = |name: &str, domain: &str| {
|
||||
// SAFETY: serialised by the guard above; restored before returning.
|
||||
unsafe {
|
||||
std::env::set_var(VARS[0], name);
|
||||
std::env::set_var(VARS[1], domain);
|
||||
}
|
||||
render_flake(
|
||||
"github:example/hyperhive",
|
||||
"path:/nix/store/bbbb-hyperhive-docs-source",
|
||||
"path:/nix/store/aaaa-nixpkgs-source",
|
||||
None,
|
||||
8000,
|
||||
"she/her",
|
||||
&std::collections::HashMap::new(),
|
||||
"4G",
|
||||
&[sample_spec("alice", false, 9001)],
|
||||
)
|
||||
.expect("render flake")
|
||||
};
|
||||
let alpha = render_on("alpha-hive", "alpha-hive.example.test");
|
||||
let beta = render_on("beta-hive", "beta-hive.example.test");
|
||||
for (var, value) in VARS.iter().zip(saved) {
|
||||
// SAFETY: as above.
|
||||
unsafe {
|
||||
match value {
|
||||
Some(v) => std::env::set_var(var, v),
|
||||
None => std::env::remove_var(var),
|
||||
}
|
||||
}
|
||||
}
|
||||
for hive in ["alpha-hive", "beta-hive"] {
|
||||
assert!(
|
||||
!alpha.contains(hive) && !beta.contains(hive),
|
||||
"the rendered flake must not name the hive ({hive}):\n{alpha}"
|
||||
);
|
||||
}
|
||||
assert_eq!(alpha, beta, "the render must not depend on the hive");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn service_url_options_render_from_forwarded_pairs() {
|
||||
// Pure over the pairs, deliberately: the render-level tests have to
|
||||
|
|
@ -1782,21 +1821,21 @@ mod tests {
|
|||
&[
|
||||
("HIVE_FORGE_URL", "http://forge.example.test".to_string()),
|
||||
("HIVE_MATRIX_URL", "http://matrix.example.test".to_string()),
|
||||
// Mapped too, and that is the fix this test now guards: the
|
||||
// display names used to be forwarded as runtime env *only*,
|
||||
// so the build-time reader fell back to "unknown" and every
|
||||
// agent shipped that label on every metric.
|
||||
("HYPERHIVE_HIVE_NAME", "pr1ma".to_string()),
|
||||
// Mapped too: `otel.nix` reads the option at eval time.
|
||||
("HYPERHIVE_SWARM_NAME", "constellat1on".to_string()),
|
||||
// Forwarded but genuinely unmapped — proves the map is a
|
||||
// filter, not a pass-through.
|
||||
("HYPERHIVE_HIVE_DOMAIN", "pr1ma.darkest.space".to_string()),
|
||||
(
|
||||
"HIVE_FORGE_PUBLIC_URL",
|
||||
"https://forge.example.test".to_string(),
|
||||
),
|
||||
],
|
||||
);
|
||||
assert_eq!(
|
||||
out,
|
||||
" services.hyperhive.agent.forge.url = \"http://forge.example.test\";\n\
|
||||
\x20 services.hyperhive.agent.matrix.url = \"http://matrix.example.test\";\n\
|
||||
\x20 services.hyperhive.agent.hiveName = \"pr1ma\";\n",
|
||||
\x20 services.hyperhive.agent.swarmName = \"constellat1on\";\n",
|
||||
"expected exactly the mapped options, in input order, indented for the module block"
|
||||
);
|
||||
}
|
||||
|
|
@ -1836,7 +1875,7 @@ mod tests {
|
|||
fn require_service_urls_accepts_a_rendered_forge_url() {
|
||||
require_service_urls(&[
|
||||
("HIVE_FORGE_URL", "http://forge.example.test".to_string()),
|
||||
("HYPERHIVE_HIVE_NAME", "pr1ma".to_string()),
|
||||
("HYPERHIVE_SWARM_NAME", "constellat1on".to_string()),
|
||||
])
|
||||
.expect("a forwarded forge URL satisfies the requirement");
|
||||
}
|
||||
|
|
@ -1847,7 +1886,7 @@ mod tests {
|
|||
// every agent would simply come up with no forge login and no way to
|
||||
// tell that was unintended. The hive asserts its own requirement
|
||||
// because it is the only layer that knows a forge is mandatory.
|
||||
let err = require_service_urls(&[("HYPERHIVE_HIVE_NAME", "pr1ma".to_string())])
|
||||
let err = require_service_urls(&[("HYPERHIVE_SWARM_NAME", "constellat1on".to_string())])
|
||||
.expect_err("a missing forge URL must stop the write");
|
||||
assert!(
|
||||
err.to_string().contains("HIVE_FORGE_URL is unset"),
|
||||
|
|
@ -1862,13 +1901,16 @@ mod tests {
|
|||
// so absence is an absent integration rather than a misdirected one.
|
||||
// Required services don't reach here: `require_service_urls` rejects
|
||||
// them first.
|
||||
// `HYPERHIVE_HIVE_DOMAIN` is forwarded as env but carries no option —
|
||||
// `HIVE_FORGE_PUBLIC_URL` is forwarded as env but carries no option —
|
||||
// deliberately picked over a mapped var, since the whole point is a
|
||||
// forwarded var the option map does not know about.
|
||||
let mut out = String::new();
|
||||
push_forwarded_var_options(
|
||||
&mut out,
|
||||
&[("HYPERHIVE_HIVE_DOMAIN", "pr1ma.darkest.space".to_string())],
|
||||
&[(
|
||||
"HIVE_FORGE_PUBLIC_URL",
|
||||
"https://forge.example.test".to_string(),
|
||||
)],
|
||||
);
|
||||
assert!(
|
||||
out.is_empty(),
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@
|
|||
|
||||
use anyhow::{Context as _, Result, bail};
|
||||
use hive_priv_sock::{
|
||||
BindMount, CredentialMount, InfraAction, InfraContainer, MEASURE_AGENT_DISK_TIMEOUT,
|
||||
BindMount, CredentialMount, HiveFacts, InfraAction, InfraContainer, MEASURE_AGENT_DISK_TIMEOUT,
|
||||
NetworkIsolation, PRIV_SOCK, PrivEvent, PrivRequest, PrivResponse, PrivStream,
|
||||
};
|
||||
use std::os::fd::{AsRawFd as _, OwnedFd, RawFd};
|
||||
|
|
@ -288,12 +288,14 @@ pub async fn write_nspawn_flags(
|
|||
binds: &[BindMount],
|
||||
isolation: NetworkIsolation,
|
||||
load_credentials: &[CredentialMount],
|
||||
hive_facts: HiveFacts,
|
||||
) -> Result<()> {
|
||||
ok(call(&PrivRequest::WriteNspawnFlags {
|
||||
container: container.to_owned(),
|
||||
binds: binds.to_vec(),
|
||||
isolation,
|
||||
load_credentials: load_credentials.to_vec(),
|
||||
hive_facts,
|
||||
})
|
||||
.await?)
|
||||
}
|
||||
|
|
|
|||
|
|
@ -232,6 +232,18 @@ pub struct NetworkIsolation {
|
|||
pub gateway_ip: String,
|
||||
}
|
||||
|
||||
/// Facts about the hive a container runs on, for `WriteNspawnFlags`.
|
||||
/// hive-priv writes them into the container's `/etc/hyperhive-hive-facts`,
|
||||
/// which the agent loads into its environment at boot, so no agent build
|
||||
/// carries them. A `None` field leaves that variable unset in the agent.
|
||||
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
|
||||
pub struct HiveFacts {
|
||||
/// `HYPERHIVE_HIVE_NAME`: the hive's name (`pr1ma`).
|
||||
pub name: Option<String>,
|
||||
/// `HYPERHIVE_HIVE_DOMAIN`: the hive's DNS domain.
|
||||
pub domain: Option<String>,
|
||||
}
|
||||
|
||||
/// Wall-clock bound hive-priv puts on one [`PrivRequest::MeasureAgentDisk`]
|
||||
/// walk. A caller's own wait has to be longer than this, so a slow walk
|
||||
/// reaches it as hive-priv's timeout error rather than as its own.
|
||||
|
|
@ -306,6 +318,10 @@ pub enum PrivRequest {
|
|||
/// so a hive-priv built before this field deserialises new requests.
|
||||
#[serde(default)]
|
||||
load_credentials: Vec<CredentialMount>,
|
||||
/// Written to the container's `/etc/hyperhive-hive-facts` on every
|
||||
/// call, so a fact the hive no longer has is removed, not kept.
|
||||
#[serde(default)]
|
||||
hive_facts: HiveFacts,
|
||||
},
|
||||
|
||||
/// Create the agent's socket dir `/run/hive-agent/<name>` as `0751
|
||||
|
|
|
|||
|
|
@ -22,10 +22,10 @@ use std::path::{Path, PathBuf};
|
|||
|
||||
use anyhow::{Context as _, Result, anyhow, bail};
|
||||
use hive_priv_sock::{
|
||||
AGENT_PREFIX, AGENT_RUNTIME_ROOT, AGENT_STATE_ROOT, BindMount, CredentialMount, InfraAction,
|
||||
InfraContainer, MEASURE_AGENT_DISK_TIMEOUT, META_DIR, MIGRATE_STAGING_ROOT, NetworkIsolation,
|
||||
PAUSED_MARKER_FILE, PRIV_SOCK, PrivEvent, PrivRequest, PrivResponse, PrivStream,
|
||||
PrivStreamLine, SIBLING_CONTAINERS,
|
||||
AGENT_PREFIX, AGENT_RUNTIME_ROOT, AGENT_STATE_ROOT, BindMount, CredentialMount, HiveFacts,
|
||||
InfraAction, InfraContainer, MEASURE_AGENT_DISK_TIMEOUT, META_DIR, MIGRATE_STAGING_ROOT,
|
||||
NetworkIsolation, PAUSED_MARKER_FILE, PRIV_SOCK, PrivEvent, PrivRequest, PrivResponse,
|
||||
PrivStream, PrivStreamLine, SIBLING_CONTAINERS,
|
||||
};
|
||||
use tokio::io::{AsyncWriteExt, BufReader};
|
||||
use tokio::net::unix::OwnedWriteHalf;
|
||||
|
|
@ -377,7 +377,8 @@ async fn exec(
|
|||
ref binds,
|
||||
ref isolation,
|
||||
ref load_credentials,
|
||||
} => handle_write_nspawn_flags(container, binds, isolation, load_credentials),
|
||||
ref hive_facts,
|
||||
} => handle_write_nspawn_flags(container, binds, isolation, load_credentials, hive_facts),
|
||||
|
||||
PrivRequest::EnsureAgentSocketDir { ref name } => ensure_agent_socket_dir(name),
|
||||
|
||||
|
|
@ -984,6 +985,7 @@ fn handle_write_nspawn_flags(
|
|||
binds: &[BindMount],
|
||||
isolation: &NetworkIsolation,
|
||||
load_credentials: &[CredentialMount],
|
||||
hive_facts: &HiveFacts,
|
||||
) -> Result<(String, String)> {
|
||||
validate_container_system_name(container)?;
|
||||
for bind in binds {
|
||||
|
|
@ -997,7 +999,8 @@ fn handle_write_nspawn_flags(
|
|||
// uses `:` as the name/path separator.
|
||||
validate_bind_path(&cred.host_path)?;
|
||||
}
|
||||
write_nspawn_flags(container, binds, isolation, load_credentials)?;
|
||||
let hive_facts = render_hive_facts(hive_facts)?;
|
||||
write_nspawn_flags(container, binds, isolation, load_credentials, &hive_facts)?;
|
||||
Ok((String::new(), String::new()))
|
||||
}
|
||||
|
||||
|
|
@ -2870,6 +2873,7 @@ fn write_nspawn_flags(
|
|||
binds: &[BindMount],
|
||||
isolation: &NetworkIsolation,
|
||||
load_credentials: &[CredentialMount],
|
||||
hive_facts: &str,
|
||||
) -> Result<()> {
|
||||
use std::fmt::Write as _;
|
||||
use std::os::unix::fs::MetadataExt as _;
|
||||
|
|
@ -2950,36 +2954,78 @@ fn write_nspawn_flags(
|
|||
// and which unit does it: `docs/networking/network.md` § *How the isolated
|
||||
// container gets its resolver*.
|
||||
write_bridge_dns_marker(container, isolation)?;
|
||||
// Read by the agent's environment generator at boot
|
||||
// (`nix/agent-modules/hive-facts.nix`).
|
||||
write_container_etc_file_in(&container_rootfs(container), HIVE_FACTS_FILE, hive_facts)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Filename of the in-container DNS marker, in the container's own `/etc`.
|
||||
const BRIDGE_DNS_MARKER: &str = "hyperhive-bridge-dns";
|
||||
|
||||
/// Filename of the hive-facts environment file, in the container's own `/etc`.
|
||||
const HIVE_FACTS_FILE: &str = "hyperhive-hive-facts";
|
||||
|
||||
/// The container's `/` on the host.
|
||||
fn container_rootfs(container: &str) -> PathBuf {
|
||||
PathBuf::from(format!("/var/lib/nixos-containers/{container}"))
|
||||
}
|
||||
|
||||
/// Render [`HiveFacts`] as the systemd environment file the agent loads,
|
||||
/// one `KEY=value` line per present fact.
|
||||
///
|
||||
/// A value is refused unless it is DNS-name characters only, so it can carry
|
||||
/// neither a newline (a second assignment) nor anything an environment file
|
||||
/// or a shell would interpret.
|
||||
fn render_hive_facts(facts: &HiveFacts) -> Result<String> {
|
||||
use std::fmt::Write as _;
|
||||
let mut out = String::new();
|
||||
for (key, value) in [
|
||||
("HYPERHIVE_HIVE_NAME", &facts.name),
|
||||
("HYPERHIVE_HIVE_DOMAIN", &facts.domain),
|
||||
] {
|
||||
let Some(value) = value else { continue };
|
||||
if value.is_empty()
|
||||
|| value.len() > 253
|
||||
|| !value
|
||||
.bytes()
|
||||
.all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'-' | b'_'))
|
||||
{
|
||||
bail!("invalid {key} value {value:?}: must be 1-253 chars of [A-Za-z0-9._-]");
|
||||
}
|
||||
let _ = writeln!(out, "{key}={value}");
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Write the bridge-DNS marker the `hyperhive-isolated-dns` oneshot keys
|
||||
/// off. The marker file contains just the gateway IP. Always written:
|
||||
/// every container is isolated, so there is no host-netns case that
|
||||
/// wants the marker absent.
|
||||
fn write_bridge_dns_marker(container: &str, isolation: &NetworkIsolation) -> Result<()> {
|
||||
let rootfs = PathBuf::from(format!("/var/lib/nixos-containers/{container}"));
|
||||
write_bridge_dns_marker_in(&rootfs, &isolation.gateway_ip)
|
||||
write_bridge_dns_marker_in(&container_rootfs(container), &isolation.gateway_ip)
|
||||
}
|
||||
|
||||
/// Write `<rootfs>/etc/hyperhive-bridge-dns` without following a symlink.
|
||||
/// Write `<rootfs>/etc/hyperhive-bridge-dns`; see [`write_container_etc_file_in`].
|
||||
fn write_bridge_dns_marker_in(rootfs: &Path, gateway_ip: &str) -> Result<()> {
|
||||
write_container_etc_file_in(rootfs, BRIDGE_DNS_MARKER, &format!("{gateway_ip}\n"))
|
||||
}
|
||||
|
||||
/// Write `<rootfs>/etc/<name>` without following a symlink.
|
||||
///
|
||||
/// Root inside the container owns its `/etc`, and this write runs as host
|
||||
/// root, where an absolute symlink planted in the container resolves against
|
||||
/// the host's `/`. So this opens `etc` with `O_DIRECTORY|O_NOFOLLOW` (a
|
||||
/// symlinked `etc` fails with `ELOOP`) and publishes the marker through a
|
||||
/// symlinked `etc` fails with `ELOOP`) and publishes the file through a
|
||||
/// [`StagedFile`] relative to that fd: whatever the container planted at the
|
||||
/// leaf, symlink or FIFO or device node, is replaced by the rename, never
|
||||
/// written through or opened.
|
||||
///
|
||||
/// On a fresh install the container's `/etc` may not exist yet (nixos-container
|
||||
/// materialises the rootfs on the first start), so this creates it first; the
|
||||
/// dir and the marker persist through that start. Mode 0644: the gateway IP
|
||||
/// is not a secret.
|
||||
fn write_bridge_dns_marker_in(rootfs: &Path, gateway_ip: &str) -> Result<()> {
|
||||
/// dir and the file persist through that start. Mode 0644: nothing written
|
||||
/// here is a secret.
|
||||
fn write_container_etc_file_in(rootfs: &Path, name: &str, contents: &str) -> Result<()> {
|
||||
use std::os::unix::fs::OpenOptionsExt as _;
|
||||
|
||||
let etc_path = rootfs.join("etc");
|
||||
|
|
@ -3014,14 +3060,7 @@ fn write_bridge_dns_marker_in(rootfs: &Path, gateway_ip: &str) -> Result<()> {
|
|||
}
|
||||
// SAFETY: `etc_fd` is a fresh, valid fd that nothing else owns.
|
||||
let etc = std::fs::File::from(unsafe { OwnedFd::from_raw_fd(etc_fd) });
|
||||
StagedFile::write(
|
||||
&etc,
|
||||
&etc_path,
|
||||
BRIDGE_DNS_MARKER,
|
||||
format!("{gateway_ip}\n").as_bytes(),
|
||||
0o644,
|
||||
)?
|
||||
.publish()
|
||||
StagedFile::write(&etc, &etc_path, name, contents.as_bytes(), 0o644)?.publish()
|
||||
}
|
||||
|
||||
/// Written by older hive-priv builds. Nothing writes it now, but left on disk
|
||||
|
|
@ -3124,13 +3163,14 @@ fn ensure_socket_dir_in(root: &Path, name: &str) -> Result<()> {
|
|||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::{
|
||||
BindMount, BoundedRun, Command, OwnedFd, PAUSED_MARKER_FILE, PrivRequest, StagedFile,
|
||||
agent_disk_paths, check_fd_agreement, contains_secret_shaped_run, describe_forge_admin,
|
||||
du_command, du_total, ensure_plain_filename, ensure_socket_dir_in, exec_measure_agent_disk,
|
||||
git_overlay_flags, limits_dropin_body, open_dir, open_export_dest, parse_du_total,
|
||||
partial_name, publish_file, redact_secret_line, remove_marker_in, run_bounded,
|
||||
single_output_path, toplevel_attr, validate_credential_name, validate_snapshot_name,
|
||||
write_agent_dir_file, write_bridge_dns_marker_in,
|
||||
BindMount, BoundedRun, Command, HiveFacts, OwnedFd, PAUSED_MARKER_FILE, PrivRequest,
|
||||
StagedFile, agent_disk_paths, check_fd_agreement, contains_secret_shaped_run,
|
||||
describe_forge_admin, du_command, du_total, ensure_plain_filename, ensure_socket_dir_in,
|
||||
exec_measure_agent_disk, git_overlay_flags, limits_dropin_body, open_dir, open_export_dest,
|
||||
parse_du_total, partial_name, publish_file, redact_secret_line, remove_marker_in,
|
||||
render_hive_facts, run_bounded, single_output_path, toplevel_attr,
|
||||
validate_credential_name, validate_snapshot_name, write_agent_dir_file,
|
||||
write_bridge_dns_marker_in,
|
||||
};
|
||||
use std::path::PathBuf;
|
||||
use std::sync::atomic::{AtomicU32, Ordering};
|
||||
|
|
@ -3782,6 +3822,45 @@ mod tests {
|
|||
std::fs::remove_dir_all(&dir).ok();
|
||||
}
|
||||
|
||||
/// Each present fact is one environment-file line; an absent one writes
|
||||
/// nothing, so the agent sees the variable unset rather than empty.
|
||||
#[test]
|
||||
fn hive_facts_render_one_line_per_present_fact() {
|
||||
let both = HiveFacts {
|
||||
name: Some("pr1ma".to_owned()),
|
||||
domain: Some("pr1ma.darkest.space".to_owned()),
|
||||
};
|
||||
assert_eq!(
|
||||
render_hive_facts(&both).unwrap(),
|
||||
"HYPERHIVE_HIVE_NAME=pr1ma\nHYPERHIVE_HIVE_DOMAIN=pr1ma.darkest.space\n"
|
||||
);
|
||||
let domain_only = HiveFacts {
|
||||
name: None,
|
||||
domain: Some("h1.example.test".to_owned()),
|
||||
};
|
||||
assert_eq!(
|
||||
render_hive_facts(&domain_only).unwrap(),
|
||||
"HYPERHIVE_HIVE_DOMAIN=h1.example.test\n"
|
||||
);
|
||||
assert_eq!(render_hive_facts(&HiveFacts::default()).unwrap(), "");
|
||||
}
|
||||
|
||||
/// A value that could end its line or be read as syntax is refused, not
|
||||
/// escaped: a newline would add an assignment of the caller's choosing.
|
||||
#[test]
|
||||
fn hive_facts_refuse_values_outside_dns_characters() {
|
||||
for bad in ["", "pr1ma\nLD_PRELOAD=/x", "a b", "a\"b", "$(x)"] {
|
||||
let facts = HiveFacts {
|
||||
name: Some(bad.to_owned()),
|
||||
domain: None,
|
||||
};
|
||||
assert!(
|
||||
render_hive_facts(&facts).is_err(),
|
||||
"{bad:?} must be refused"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether `pid` is gone, or only a zombie awaiting its reaper: either
|
||||
/// way it is no longer running.
|
||||
fn is_dead(pid: &str) -> bool {
|
||||
|
|
|
|||
|
|
@ -19,17 +19,6 @@ let
|
|||
# meta.rs `otel_config`). Options declared in `otel.nix`, which also
|
||||
# exports the generic OTLP environment this container's producers read.
|
||||
otelCfg = config.services.hyperhive.agent.otel;
|
||||
# Hive display name, read from the per-agent option meta.rs renders
|
||||
# (NOT from `environment.variables` — that carries the same name at
|
||||
# *runtime* only, so reading it here silently yielded "unknown" on
|
||||
# every agent while the process env held the right answer). `null`
|
||||
# means the hive did not name itself; "unknown" is then an honest label
|
||||
# rather than a guess.
|
||||
hiveDisplayName =
|
||||
if config.services.hyperhive.agent.hiveName == null then
|
||||
"unknown"
|
||||
else
|
||||
config.services.hyperhive.agent.hiveName;
|
||||
# Effective per-agent MemoryMax=, in bytes, injected by meta.rs's
|
||||
# per-agent flake render (`services.hyperhive.agent.claudeMemoryMaxBytes`). `null`
|
||||
# when the effective cap is unbounded ("infinity") or a RAM
|
||||
|
|
@ -58,8 +47,6 @@ let
|
|||
CLAUDE_CODE_RESUME_INTERRUPTED_TURN = "1";
|
||||
# Use the simpler system prompt variant suited to headless operation.
|
||||
CLAUDE_CODE_SIMPLE_SYSTEM_PROMPT = "1";
|
||||
# Tag remote-control sessions with "<hive>-<agent>" for identification.
|
||||
CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX = "${hiveDisplayName}-${userName}";
|
||||
}
|
||||
# Bun/JavaScriptCore (the runtime under claude-code's `.claude-wrapped`
|
||||
# binary) sizes its default heap ceiling off the container's visible
|
||||
|
|
@ -97,6 +84,21 @@ let
|
|||
};
|
||||
in
|
||||
{
|
||||
options.services.hyperhive.agent.swarmName = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
internal = true;
|
||||
description = ''
|
||||
Human-readable swarm name, rendered per-agent by
|
||||
`meta.rs::render_flake` from the host's
|
||||
`services.hyperhive.swarm.name`. Read during evaluation for the
|
||||
OTEL `swarm` resource label; `HYPERHIVE_SWARM_NAME` carries the
|
||||
same value at runtime.
|
||||
|
||||
`null` means the hive is not part of a named swarm.
|
||||
'';
|
||||
};
|
||||
|
||||
# Build-time implementation surface for the JSC-heap-ceiling fix:
|
||||
# meta.rs's per-agent flake render injects this from the effective
|
||||
# `MemoryMax=` (per-agent `resource-limits.json` override, else the
|
||||
|
|
@ -108,37 +110,6 @@ in
|
|||
# the agent's *last rebuild* — `set-limits` still applies the
|
||||
# cgroup cap live via a drop-in reload, but this derived heap ceiling
|
||||
# needs a rebuild to pick up a new value.
|
||||
options.services.hyperhive.agent.hiveName = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
internal = true;
|
||||
description = ''
|
||||
Human-readable hive name, rendered per-agent by
|
||||
`meta.rs::render_flake` from the host's
|
||||
`services.hyperhive.hiveName`. Baked into the OTEL resource
|
||||
attributes at build time, which is why it is an option and not
|
||||
just the `HYPERHIVE_HIVE_NAME` env var: the env var is read at
|
||||
runtime, this is read during evaluation, and wiring only one of
|
||||
the two is how every agent ended up reporting `hive=unknown`.
|
||||
|
||||
`null` means the hive did not name itself.
|
||||
'';
|
||||
};
|
||||
|
||||
options.services.hyperhive.agent.swarmName = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
internal = true;
|
||||
description = ''
|
||||
Human-readable swarm name, rendered per-agent by
|
||||
`meta.rs::render_flake` from the host's
|
||||
`services.hyperhive.swarm.name`. Same build-time/runtime split as
|
||||
`services.hyperhive.agent.hiveName`.
|
||||
|
||||
`null` means the hive is not part of a named swarm.
|
||||
'';
|
||||
};
|
||||
|
||||
options.services.hyperhive.agent.claudeMemoryMaxBytes = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.ints.positive;
|
||||
default = null;
|
||||
|
|
@ -279,7 +250,7 @@ in
|
|||
# Hive-enforced settings merged with a per-agent `env` block at BUILD
|
||||
# time via `jq` (not eval-time `readFile`, which would be import-from-
|
||||
# derivation). The `env` block is always present: `baseClaudeEnv` sets
|
||||
# behaviour flags and the remote-control session prefix for every agent;
|
||||
# behaviour flags for every agent;
|
||||
# `otelSettingsEnv` is merged on top when OTEL is enabled. claude-code
|
||||
# auto-discovers this file in every context (harness turn-loop AND
|
||||
# `hivectl choom`) so no launch wrapper is needed.
|
||||
|
|
@ -287,9 +258,8 @@ in
|
|||
let
|
||||
baseSettings = "${config.services.hyperhive.agent.packages.assets}/share/hyperhive/prompts/claude-settings.json";
|
||||
# Merge base env (always) with OTEL env (when enabled). jq is always
|
||||
# run — `baseClaudeEnv` contains per-agent values (e.g.
|
||||
# CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX) that can't live in the
|
||||
# static store asset.
|
||||
# run — `baseClaudeEnv` can carry per-agent values (e.g.
|
||||
# BUN_JSC_forceRAMSize) that can't live in the static store asset.
|
||||
allEnv =
|
||||
baseClaudeEnv
|
||||
// lib.optionalAttrs otelCfg.enable otelSettingsEnv
|
||||
|
|
|
|||
|
|
@ -36,6 +36,11 @@ let
|
|||
|
||||
See docs/tools/matrix.md.
|
||||
'';
|
||||
hiveNameRemoved = ''
|
||||
An agent reads its hive's name at runtime from /etc/hyperhive-hive-facts,
|
||||
which the hive writes into every container, so no agent build sets it.
|
||||
Remove the setting.
|
||||
'';
|
||||
in
|
||||
{
|
||||
imports = [
|
||||
|
|
@ -51,6 +56,7 @@ in
|
|||
./frontend.nix
|
||||
./github.nix
|
||||
./github-token.nix
|
||||
./hive-facts.nix
|
||||
./logs.nix
|
||||
./matrix.nix
|
||||
./mcp.nix
|
||||
|
|
@ -85,6 +91,9 @@ in
|
|||
# breaks evaluation for everyone, removed or not.
|
||||
(lib.mkRemovedOptionModule [ "hyperhive" "matrix" "enable" ] matrixEnableRemoved)
|
||||
(lib.mkRemovedOptionModule [ "services" "hyperhive" "agent" "matrix" "enable" ] matrixEnableRemoved)
|
||||
# Both spellings, for the reason given above.
|
||||
(lib.mkRemovedOptionModule [ "hyperhive" "hiveName" ] hiveNameRemoved)
|
||||
(lib.mkRemovedOptionModule [ "services" "hyperhive" "agent" "hiveName" ] hiveNameRemoved)
|
||||
];
|
||||
|
||||
options.services.hyperhive.agent.icon = lib.mkOption {
|
||||
|
|
|
|||
37
nix/agent-modules/hive-facts.nix
Normal file
37
nix/agent-modules/hive-facts.nix
Normal file
|
|
@ -0,0 +1,37 @@
|
|||
# The hive this container runs on, read at boot rather than built in, so an
|
||||
# agent's closure does not carry it.
|
||||
#
|
||||
# hive-priv writes `/etc/hyperhive-hive-facts` into the container before every
|
||||
# start (`HYPERHIVE_HIVE_NAME`, `HYPERHIVE_HIVE_DOMAIN`, one `KEY=value` line
|
||||
# each). This environment generator hands those lines to the container's
|
||||
# systemd, which passes them to every unit it starts and to `machinectl shell`
|
||||
# sessions; a unit's own `Environment=` still overrides them.
|
||||
{
|
||||
config,
|
||||
lib,
|
||||
pkgs,
|
||||
...
|
||||
}:
|
||||
let
|
||||
userName = config.services.hyperhive.agent.user.name;
|
||||
in
|
||||
{
|
||||
# Shell builtins only: nothing is assumed on PATH this early. Only the two
|
||||
# known keys are passed on.
|
||||
environment.etc."systemd/system-environment-generators/50-hyperhive-hive-facts".source =
|
||||
pkgs.writeShellScript "hyperhive-hive-facts" ''
|
||||
hive=unknown
|
||||
if [ -r /etc/hyperhive-hive-facts ]; then
|
||||
while IFS='=' read -r key value; do
|
||||
case "$key" in
|
||||
HYPERHIVE_HIVE_NAME) hive=$value ;;
|
||||
HYPERHIVE_HIVE_DOMAIN) ;;
|
||||
*) continue ;;
|
||||
esac
|
||||
printf '%s=%s\n' "$key" "$value"
|
||||
done < /etc/hyperhive-hive-facts
|
||||
fi
|
||||
# Names claude's remote-control sessions `<hive>-<agent>`.
|
||||
printf 'CLAUDE_REMOTE_CONTROL_SESSION_NAME_PREFIX=%s-%s\n' "$hive" ${lib.escapeShellArg userName}
|
||||
'';
|
||||
}
|
||||
|
|
@ -25,17 +25,10 @@
|
|||
let
|
||||
cfg = config.services.hyperhive.agent.otel;
|
||||
userName = config.services.hyperhive.agent.user.name;
|
||||
# Hive/swarm display names, read from the per-agent options meta.rs
|
||||
# renders (NOT from `environment.variables` — those carry the same
|
||||
# names at *runtime* only, so reading them here silently yielded
|
||||
# "unknown" on every agent while the process env held the right
|
||||
# answer). `null` means the hive did not name itself; "unknown" is then
|
||||
# an honest label rather than a guess.
|
||||
hiveDisplayName =
|
||||
if config.services.hyperhive.agent.hiveName == null then
|
||||
"unknown"
|
||||
else
|
||||
config.services.hyperhive.agent.hiveName;
|
||||
# Swarm display name, read from the per-agent option meta.rs renders
|
||||
# (NOT from `environment.variables`, which carries it at *runtime*
|
||||
# only). `null` means the swarm is unnamed; "unknown" is then an honest
|
||||
# label rather than a guess.
|
||||
swarmDisplayName =
|
||||
if config.services.hyperhive.agent.swarmName == null then
|
||||
"unknown"
|
||||
|
|
@ -45,9 +38,11 @@ let
|
|||
# Resource labels every producer in this container stamps on what it
|
||||
# emits. `service.name` names the container's role, not one binary
|
||||
# inside it: claude's samples and `hive-metric`'s both come from this
|
||||
# agent, and their metric names already tell them apart.
|
||||
# agent, and their metric names already tell them apart. No `hive`: the
|
||||
# swarm collector stamps it from the receiver that accepted the hive's
|
||||
# push (nix/host-modules/swarm-otel.nix), so the build names no hive.
|
||||
resourceAttributes =
|
||||
"service.name=hyperhive-agent,agent=${userName},hive=${hiveDisplayName},swarm=${swarmDisplayName}"
|
||||
"service.name=hyperhive-agent,agent=${userName},swarm=${swarmDisplayName}"
|
||||
+ lib.optionalString (cfg.extraResourceAttributes != "") ",${cfg.extraResourceAttributes}";
|
||||
|
||||
# The variables an OTEL SDK reads on its own, in any language, in any
|
||||
|
|
@ -131,7 +126,7 @@ in
|
|||
description = ''
|
||||
Extra comma-separated entries appended to
|
||||
`OTEL_RESOURCE_ATTRIBUTES` after the built-in
|
||||
`service.name` / `agent` / `hive` / `swarm` labels.
|
||||
`service.name` / `agent` / `swarm` labels.
|
||||
Host-driven via `services.hyperhive.otel.extraResourceAttributes`.
|
||||
'';
|
||||
};
|
||||
|
|
@ -274,10 +269,9 @@ in
|
|||
# Identity the hop above cannot supply. A host-side reader can say
|
||||
# which machine a line came from; only a collector running as the
|
||||
# agent can say which agent. `hive` and `swarm` are deliberately
|
||||
# NOT stamped here even though the env above carries them for
|
||||
# in-process producers: the swarm tier upserts `hive` from the
|
||||
# receiver that accepted the sample, precisely so the label comes
|
||||
# from something the sender cannot write.
|
||||
# NOT stamped here: the swarm tier upserts both from the receiver
|
||||
# that accepted the sample, precisely so the labels come from
|
||||
# something the sender cannot write.
|
||||
processors.resource.attributes = [
|
||||
{
|
||||
key = "service.name";
|
||||
|
|
|
|||
|
|
@ -112,7 +112,6 @@
|
|||
[ "hyperhive" "gui" "vncPort" ]
|
||||
[ "services" "hyperhive" "agent" "gui" "vncPort" ]
|
||||
)
|
||||
(lib.mkRenamedOptionModule [ "hyperhive" "hiveName" ] [ "services" "hyperhive" "agent" "hiveName" ])
|
||||
(lib.mkRenamedOptionModule [ "hyperhive" "icon" ] [ "services" "hyperhive" "agent" "icon" ])
|
||||
(lib.mkRenamedOptionModule
|
||||
[ "hyperhive" "logs" "queryUrl" ]
|
||||
|
|
|
|||
Loading…
Reference in a new issue