otel: move the generic OTLP environment out of claude's settings
The endpoint, protocol, temporality preference and resource labels were shipped only inside claude's managed settings json, so they applied to claude's own process. hive-bash-daemon, hive-mcp-http, hive-matrix-daemon and hive-forge-notify are systemd *siblings* of claude rather than its children, so nothing shipped there could ever reach them: `hive-metric` invoked from a tool call exited with "OTEL_EXPORTER_OTLP_ENDPOINT not set", which is the honest failure of a value it structurally could not see. Declare those variables container-wide in a new agent module instead — systemd.globalEnvironment for every unit PID 1 starts, environment.variables for login shells. Both are needed and neither implies the other; NIX_REMOTE is set both ways for the same reason. Claude keeps only what is genuinely its own: the telemetry master flag, the feedback-survey flag, the version label, and which signals it exports. A different producer in the same container may legitimately emit only metrics. The hyperhive.otel.* options move across with them. They have more than one consumer now, so their home is the OTEL module rather than the claude one.
This commit is contained in:
atlas
mara
parent
328ba19595
commit
8b14d959d6
4 changed files with 204 additions and 116 deletions
|
|
@ -1,7 +1,11 @@
|
|||
# Everything that shapes claude-code's own configuration inside the
|
||||
# container: the managed settings json (base env + OTEL), the
|
||||
# onboarding/trust seed, the runtime OTEL auth-header injection, and
|
||||
# the plugin/marketplace install lists the harness reads at boot.
|
||||
# container: the managed settings json (base env + claude's own
|
||||
# telemetry switches), the onboarding/trust seed, and the
|
||||
# plugin/marketplace install lists the harness reads at boot.
|
||||
#
|
||||
# The generic OTLP environment — endpoint, protocol, resource labels —
|
||||
# is NOT here: it belongs to every producer in the container, not to
|
||||
# claude, and lives in `otel.nix`.
|
||||
{
|
||||
pkgs,
|
||||
lib,
|
||||
|
|
@ -12,18 +16,17 @@ let
|
|||
userName = config.hyperhive.user.name;
|
||||
homeDir = "/home/${userName}";
|
||||
# Hive-wide OpenTelemetry config (host-driven; baked in per-agent by
|
||||
# meta.rs `otel_config`).
|
||||
# meta.rs `otel_config`). Options declared in `otel.nix`, which also
|
||||
# exports the generic OTLP environment this container's producers read.
|
||||
otelCfg = config.hyperhive.otel;
|
||||
# Hive/swarm display names, read from the per-agent options meta.rs
|
||||
# renders (NOT from `environment.variables` — those carry the same names
|
||||
# at *runtime* only, so reading them here silently yielded "unknown" on
|
||||
# every agent while the process env held the right answer). `null` means
|
||||
# the hive did not name itself; "unknown" is then an honest label rather
|
||||
# than a guess.
|
||||
# Hive display name, read from the per-agent option meta.rs renders
|
||||
# (NOT from `environment.variables` — that carries the same name at
|
||||
# *runtime* only, so reading it here silently yielded "unknown" on
|
||||
# every agent while the process env held the right answer). `null`
|
||||
# means the hive did not name itself; "unknown" is then an honest label
|
||||
# rather than a guess.
|
||||
hiveDisplayName =
|
||||
if config.hyperhive.hiveName == null then "unknown" else config.hyperhive.hiveName;
|
||||
swarmDisplayName =
|
||||
if config.hyperhive.swarmName == null then "unknown" else config.hyperhive.swarmName;
|
||||
# Effective per-agent MemoryMax=, in bytes, injected by meta.rs's
|
||||
# per-agent flake render (`hyperhive.claudeMemoryMaxBytes`). `null`
|
||||
# when the effective cap is unbounded ("infinity") or a RAM
|
||||
|
|
@ -68,121 +71,29 @@ let
|
|||
// lib.optionalAttrs (memoryMaxBytes != null) {
|
||||
BUN_JSC_forceRAMSize = toString (memoryMaxBytes * 75 / 100);
|
||||
};
|
||||
# OTEL environment Claude Code reads to export metrics/logs/traces.
|
||||
# Shipped via the managed claude settings json (below), which claude
|
||||
# auto-discovers for BOTH the harness turn-loop and `hivectl choom` —
|
||||
# so telemetry parity is declarative, with no launch wrapper.
|
||||
#
|
||||
# There is no auth header here, and no mechanism to add one. An agent
|
||||
# exports to the hive's own collector, which is the only thing holding
|
||||
# a credential for anything upstream; nothing an agent can read is a
|
||||
# secret to the swarm. An earlier revision forwarded the operator's
|
||||
# upstream token into this container and merged it into the agent's own
|
||||
# `~/.claude/settings.json` — which handed every agent the hive's
|
||||
# credential, and was removed with the direct-export path it served.
|
||||
# Claude Code's own telemetry switches — what it emits, and whether it
|
||||
# emits at all. Everything an OTEL SDK reads generically (endpoint,
|
||||
# protocol, temporality, resource labels) is deliberately NOT here: it
|
||||
# lives in `otel.nix` as container environment, because claude is one
|
||||
# producer in this container and not the owner of the pipe. Shipping
|
||||
# those in claude's managed settings put them on claude's process only,
|
||||
# and `hive-metric` — a sibling of claude, not a child — could not see
|
||||
# the endpoint at all.
|
||||
otelSettingsEnv = {
|
||||
CLAUDE_CODE_ENABLE_TELEMETRY = "1";
|
||||
# Attach feedback-survey data to the OTEL pipeline.
|
||||
CLAUDE_CODE_ENABLE_FEEDBACK_SURVEY_FOR_OTEL = "1";
|
||||
# Which signals claude exports. A different producer in this same
|
||||
# container may legitimately emit only metrics, so this stays a
|
||||
# claude decision rather than container-wide config.
|
||||
OTEL_METRICS_EXPORTER = "otlp";
|
||||
OTEL_LOGS_EXPORTER = "otlp";
|
||||
OTEL_TRACES_EXPORTER = "otlp";
|
||||
OTEL_EXPORTER_OTLP_PROTOCOL = otelCfg.protocol;
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT = otelCfg.endpoint;
|
||||
# Force CUMULATIVE temporality — Claude Code defaults to DELTA,
|
||||
# which Prometheus/Mimir-family backends (incl. grafana-lgtm)
|
||||
# silently drop without a deltatocumulative processor.
|
||||
OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE = "cumulative";
|
||||
OTEL_RESOURCE_ATTRIBUTES =
|
||||
"service.name=hyperhive-agent,agent=${userName},hive=${hiveDisplayName},swarm=${swarmDisplayName}"
|
||||
+ lib.optionalString (otelCfg.extraResourceAttributes != "") ",${otelCfg.extraResourceAttributes}";
|
||||
# Include the Claude Code version label in emitted metrics.
|
||||
OTEL_METRICS_INCLUDE_VERSION = "1";
|
||||
}
|
||||
// lib.optionalAttrs (otelCfg.metricIntervalMs != null) {
|
||||
OTEL_METRIC_EXPORT_INTERVAL = toString otelCfg.metricIntervalMs;
|
||||
};
|
||||
in
|
||||
{
|
||||
# OTEL stats export is configured ONCE at host level via
|
||||
# `services.hyperhive.otel.*` (see nix/host-modules/hive-c0re.nix) and
|
||||
# injected into every agent's build by the meta-flake renderer
|
||||
# (`hive-c0re/src/meta.rs::otel_config`). These per-agent options are
|
||||
# the build-time implementation surface that injection writes into;
|
||||
# they are not meant to be set directly in an agent.nix. Marked
|
||||
# `internal` so the host option is the only documented operator knob.
|
||||
options.hyperhive.otel = {
|
||||
enable = lib.mkOption {
|
||||
type = lib.types.bool;
|
||||
default = false;
|
||||
internal = true;
|
||||
description = ''
|
||||
Export this agent's Claude Code stats (token usage, cost, tool
|
||||
calls) to an OTLP endpoint via Claude Code's built-in
|
||||
OpenTelemetry. Each agent's harness exports directly to the
|
||||
collector, so it keeps working even when hive-c0re is down.
|
||||
Host-driven: set `services.hyperhive.otel.enable` instead.
|
||||
'';
|
||||
};
|
||||
|
||||
endpoint = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
default = "";
|
||||
internal = true;
|
||||
description = ''
|
||||
OTLP collector endpoint, set as `OTEL_EXPORTER_OTLP_ENDPOINT`.
|
||||
Host-driven via `services.hyperhive.otel.endpoint`.
|
||||
'';
|
||||
};
|
||||
|
||||
protocol = lib.mkOption {
|
||||
type = lib.types.enum [
|
||||
"http/protobuf"
|
||||
"http/json"
|
||||
"grpc"
|
||||
];
|
||||
default = "http/protobuf";
|
||||
internal = true;
|
||||
description = ''
|
||||
OTLP wire protocol, set as `OTEL_EXPORTER_OTLP_PROTOCOL`.
|
||||
Host-driven via `services.hyperhive.otel.protocol`.
|
||||
'';
|
||||
};
|
||||
|
||||
extraResourceAttributes = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
default = "";
|
||||
internal = true;
|
||||
description = ''
|
||||
Extra comma-separated entries appended to
|
||||
`OTEL_RESOURCE_ATTRIBUTES` after the built-in
|
||||
`service.name` / `agent` / `hive` / `swarm` labels.
|
||||
Host-driven via `services.hyperhive.otel.extraResourceAttributes`.
|
||||
'';
|
||||
};
|
||||
|
||||
metricIntervalMs = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.ints.positive;
|
||||
default = null;
|
||||
internal = true;
|
||||
description = ''
|
||||
Metric export interval in milliseconds, set as
|
||||
`OTEL_METRIC_EXPORT_INTERVAL`. Null leaves Claude Code's 60s
|
||||
default. Host-driven via `services.hyperhive.otel.metricIntervalMs`.
|
||||
'';
|
||||
};
|
||||
|
||||
debug = lib.mkOption {
|
||||
type = lib.types.bool;
|
||||
default = false;
|
||||
internal = true;
|
||||
description = ''
|
||||
Emit OTEL SDK diagnostics to stderr (`CLAUDE_CODE_OTEL_DIAG_STDERR=1`).
|
||||
Host-driven via `services.hyperhive.otel.debug`.
|
||||
'';
|
||||
};
|
||||
};
|
||||
|
||||
# Build-time implementation surface for the JSC-heap-ceiling fix:
|
||||
# meta.rs's per-agent flake render injects this from the effective
|
||||
# `MemoryMax=` (per-agent `resource-limits.json` override, else the
|
||||
|
|
|
|||
Loading…
Reference in a new issue