diff --git a/docs/integrations/github.md b/docs/integrations/github.md index a63ef0cd..f973afb1 100644 --- a/docs/integrations/github.md +++ b/docs/integrations/github.md @@ -19,17 +19,17 @@ needed — an agent gains GitHub by having a PAT stored for it. -To turn it off for the whole hive, set the host option: +To turn it off for the whole swarm, set the swarm option: ```nix -services.hyperhive.github.enable = false; +services.hyperhive.swarm.github.enable = false; ``` hive-c0re's meta-flake renderer then injects `services.hyperhive.agent.github.enable = false` into every agent, so no agent ships the `gh` wrapper, the credential helper or the token fetch. (`services.hyperhive.agent.github.enable` also exists per-agent for completeness, but the -hive-wide host switch is the intended control.) +swarm-wide switch is the intended control.) github.com only. The token **value** never touches nix — the agent writes it to `/github-token` at runtime (see [Provisioning](#provisioning)). diff --git a/docs/networking/gateway.md b/docs/networking/gateway.md index 81250056..ee186dc5 100644 --- a/docs/networking/gateway.md +++ b/docs/networking/gateway.md @@ -129,7 +129,7 @@ The name sits under the swarm domain so authelia's session cookie reaches it, wh nginx is the one external entry point. The per-agent web-port range (`8100`–`8999`) stays closed: agents serve their UI on a unix socket ([Per-agent unix-socket upstream](#per-agent-unix-socket-upstream)), and the hashed TCP port (`lifecycle::agent_web_port`) is only a fallback bind for an agent missing `HIVE_WEB_SOCKET` — the gateway never proxies through it. -The dashboard port (`services.hyperhive.c0re.dashboardPort`, default 7000) binds `127.0.0.1` only, so remote dashboard access goes through the gateway. The dashboard can stop, rebuild and destroy agents; don't expose it without a reverse proxy in front. +The dashboard port (`services.hyperhive.deploy.hive-controller.dashboardPort`, default 7000) binds `127.0.0.1` only, so remote dashboard access goes through the gateway. The dashboard can stop, rebuild and destroy agents; don't expose it without a reverse proxy in front. nginx serves its `stub_status` counters on `127.0.0.1:` (`services.hyperhive.gateway.statusPort`, default 8085) at `/stub_status`, from a server of its own outside the vhost tree. The hive's collector reads them with its `nginx` receiver, and the `gateway` Grafana dashboard plots them. The receiver only dials `http://`, which is why this is loopback TCP rather than a unix socket. diff --git a/docs/process/conventions.md b/docs/process/conventions.md index 2f083c0a..347131ba 100644 --- a/docs/process/conventions.md +++ b/docs/process/conventions.md @@ -11,7 +11,7 @@ exist because something already went wrong without them. - `MAX_AGENT_NAME` in `hive-c0re/src/lifecycle/mod.rs` enforces the cap. - Per-agent web UI port = `WEB_PORT_BASE + FNV1a(name) % WEB_PORT_RANGE` (8100..8999) for every agent; dashboard - `services.hyperhive.c0re.dashboardPort` (default 7000). + `services.hyperhive.deploy.hive-controller.dashboardPort` (default 7000). ## Hive identity (label + domain + display names) diff --git a/docs/process/gotchas.md b/docs/process/gotchas.md index 3021ef10..6f55d2d1 100644 --- a/docs/process/gotchas.md +++ b/docs/process/gotchas.md @@ -140,7 +140,7 @@ _different_ nixpkgs by itself: a container only ever sees the one nixpkgs the meta flake injects, so an `agent.nix` naming the host's `nixpkgs-unstable` has nothing to name. A release channel can trail unstable by weeks on this package, which is what -`services.hyperhive.c0re.claudeCodePackage` is for — set it host-side +`services.hyperhive.deploy.hive-controller.claudeCodePackage` is for — set it host-side and every agent runs that build. What crosses is the **store path**, not the derivation. Containers @@ -228,7 +228,7 @@ agent (different hash → different port) and rebuilding. No state file, no probing, no port-allocation drift — the value is reproducible from just the name. Every agent hashes into 8100..8999 via the same FNV-1a; dashboard -at `services.hyperhive.c0re.dashboardPort` (default 7000). +at `services.hyperhive.deploy.hive-controller.dashboardPort` (default 7000). ### Restart races on TCP bind diff --git a/docs/scheduler/observability.md b/docs/scheduler/observability.md index 3a84cb93..f50eb853 100644 --- a/docs/scheduler/observability.md +++ b/docs/scheduler/observability.md @@ -179,9 +179,12 @@ To let agents reach some other host-local service of your own → The nix module (`nix/host-modules/otel.nix`) generates every `services.hyperhive.otel.*` option's full type, default, description and example into the [options reference](https://hyperhive.darkest.space/options/) -(host options — `nix build .#docs-host` for a local render). That's the -reference; this page covers what a per-option listing can't: the two-tier -architecture, the security model, and how the options interact. +(host options — `nix build .#docs-host` for a local render). The same module +declares the collector port every hive and its agents share, +`services.hyperhive.swarm.hiveOtel.collectorPort`, and each hive's own +`services.hyperhive.deploy.hive-controller.otel.extraResourceAttributes`. +That's the reference; this page covers what a per-option listing can't: the +two-tier architecture, the security model, and how the options interact. ## Built-in resource labels @@ -205,8 +208,8 @@ The swarm's collector adds `hive`, the name of the hive whose receiver accepted the push (see [Authenticated ingest](#authenticated-ingest)). The container doesn't set it. -Append additional labels via `extraResourceAttributes` (see option -reference above); pass custom per-data-point labels with +Append additional labels for one hive via `extraResourceAttributes` (see +option reference above); pass custom per-data-point labels with `hive-metric --labels` (see below). ## Log severity @@ -319,8 +322,9 @@ exists, so off-the-shelf OTEL/Grafana container dashboards work. Resource The `hyperhive.`-prefixed metrics have no semconv equivalent (memory limit + peak, on-disk footprint, and an instantaneous cpu percent kept -alongside the spec `container.cpu.time` counter for convenience). Hive -labels (`hive`, `swarm`, …) ride on the resource via +alongside the spec `container.cpu.time` counter for convenience). The `swarm` +label rides on the resource, and the swarm collector adds `hive` from the +receiver that accepted the sample. Any other label goes in the hive's `extraResourceAttributes`. diff --git a/docs/swarm/claude-login.md b/docs/swarm/claude-login.md index dd3bb921..f06d0115 100644 --- a/docs/swarm/claude-login.md +++ b/docs/swarm/claude-login.md @@ -7,8 +7,9 @@ agent's harness installs them from there. Where it runs and how it's enabled: [`services.md`](services.md#claude-login-minter). -The service runs the `claude-code` that `services.hyperhive.c0re.claudeCodePackage` -pins for agents, and its container's own nixpkgs' where the hive pins none. +The service runs the `claude-code` that its host's +`services.hyperhive.deploy.hive-controller.claudeCodePackage` +pins for agents, and its container's own nixpkgs' where the host pins none. ## What happens on a login diff --git a/docs/swarm/review-classifier.md b/docs/swarm/review-classifier.md index f562d969..f2d1e1ca 100644 --- a/docs/swarm/review-classifier.md +++ b/docs/swarm/review-classifier.md @@ -145,8 +145,9 @@ config) and an empty directory of its own. The prompt reaches it on stdin as one user message in `--input-format stream-json`, so that it can carry content blocks (see [Shared context](#shared-context)). -The `claude` is the one `services.hyperhive.c0re.claudeCodePackage` pins for -agents, or its container's own nixpkgs' where the hive pins none. +The `claude` is the one its host's +`services.hyperhive.deploy.hive-controller.claudeCodePackage` pins for +agents, or its container's own nixpkgs' where the host pins none. On the ACP runtime, each classification is one prompt on a new `opencode acp` session. Its config and opencode's state live in a directory of its own, diff --git a/docs/turn-loop/claude-invocation.md b/docs/turn-loop/claude-invocation.md index 28dc5548..c1441c99 100644 --- a/docs/turn-loop/claude-invocation.md +++ b/docs/turn-loop/claude-invocation.md @@ -32,8 +32,8 @@ to the bus. harness unit's PATH. By default that's the `claude-code` in the agent's own nixpkgs (the meta flake's `nixpkgs` input) via `environment.systemPackages`. Since that's typically a release channel and -this package moves fast, the operator can pin one hive-wide with -`services.hyperhive.c0re.claudeCodePackage`: hive-c0re writes its +this package moves fast, the operator can pin one per hive with +`services.hyperhive.deploy.hive-controller.claudeCodePackage`: hive-c0re writes its store path into each agent's flake, and `claude` on PATH becomes a symlink to it instead of the container's own `claude-code` — so there's only ever one `claude` in the container. Agents pick up a new build on their diff --git a/hive-c0re/src/coordinator.rs b/hive-c0re/src/coordinator.rs index c14b5d53..5351c473 100644 --- a/hive-c0re/src/coordinator.rs +++ b/hive-c0re/src/coordinator.rs @@ -54,7 +54,8 @@ pub struct Coordinator { pub nixpkgs_flake: String, /// Store path of the `claude-code` build every agent runs, written /// into each per-agent flake as `services.hyperhive.agent.claudeCodePath`. Set by - /// the NixOS module option `services.hyperhive.c0re.claudeCodePackage` + /// the NixOS module option + /// `services.hyperhive.deploy.hive-controller.claudeCodePackage` /// (which resolves the package and hands us its path). `None` = every /// agent keeps the `claude-code` from its own nixpkgs. /// diff --git a/hive-c0re/src/meta.rs b/hive-c0re/src/meta.rs index 6c46f7bf..edb33c24 100644 --- a/hive-c0re/src/meta.rs +++ b/hive-c0re/src/meta.rs @@ -551,8 +551,9 @@ fn hive_ca_source() -> Option { } /// Hive-wide OTEL config injected into every agent's build, read off -/// hive-c0re's own unit env (set from `services.hyperhive.otel.*`). A -/// present, non-empty +/// hive-c0re's own unit env (set from `services.hyperhive.otel.*`, +/// `services.hyperhive.swarm.hiveOtel.*` and +/// `services.hyperhive.deploy.hive-controller.otel.*`). A present, non-empty /// `HYPERHIVE_OTEL_ENDPOINT` is the enable signal — the host module /// asserts the endpoint is set whenever `otel.enable` is true, so /// "endpoint present" == "OTEL on". The optional fields map to the @@ -922,8 +923,8 @@ where // operation the tests exercise directly. push_forwarded_var_options(&mut out, &forwarded_env_vars()); // GitHub integration is on by default in every agent - // (`services.hyperhive.agent.github.enable`); the host turns it off hive-wide via - // `services.hyperhive.github.enable = false`, surfaced here as the + // (`services.hyperhive.agent.github.enable`); the host turns it off swarm-wide via + // `services.hyperhive.swarm.github.enable = false`, surfaced here as the // `HYPERHIVE_GITHUB_DISABLED` env on hive-c0re's unit. Only the OFF // override is propagated — the enabled default needs no per-agent line. if std::env::var_os("HYPERHIVE_GITHUB_DISABLED").is_some() { @@ -1986,7 +1987,7 @@ mod tests { #[test] fn render_flake_injects_github_disable_only_when_signalled() { - // services.hyperhive.github.enable = false -> HYPERHIVE_GITHUB_DISABLED + // services.hyperhive.swarm.github.enable = false -> HYPERHIVE_GITHUB_DISABLED // on hive-c0re's unit -> `services.hyperhive.agent.github.enable = false` injected into // every agent. On by default, so nothing is emitted unless disabled. // diff --git a/hive-c0re/src/stats/otel_metrics.rs b/hive-c0re/src/stats/otel_metrics.rs index 36ae37bb..fa01c52e 100644 --- a/hive-c0re/src/stats/otel_metrics.rs +++ b/hive-c0re/src/stats/otel_metrics.rs @@ -347,8 +347,9 @@ fn attrs(c: &ContainerResource) -> Vec { /// Resource: `service.name = hyperhive-c0re`, the host architecture, the /// running hyperhive revision, plus whatever the operator set in -/// `extraResourceAttributes` (where the hive `hive` / `swarm` labels live, same -/// channel the agent-side export uses). +/// `deploy.hive-controller.otel.extraResourceAttributes` (same channel the +/// agent-side export uses; the `hive` label is stamped by the swarm +/// collector, not set here). /// /// These belong on the resource rather than on each data point for the same /// reason `hive` / `swarm` do: they are constant across one `hive-c0re`. Only diff --git a/nix/agent-modules/default.nix b/nix/agent-modules/default.nix index 47f37fa3..b8949cc5 100644 --- a/nix/agent-modules/default.nix +++ b/nix/agent-modules/default.nix @@ -96,7 +96,7 @@ in nixpkgs. Set by the generated meta flake when the operator sets - `services.hyperhive.c0re.claudeCodePackage` host-side, so a hive + `services.hyperhive.deploy.hive-controller.claudeCodePackage` host-side, so a hive can run a claude built from a *different* nixpkgs than the one its agents evaluate against — a release channel can trail unstable by weeks on this one package. It arrives as a path and @@ -109,7 +109,7 @@ in there is only ever one claude in the container. Note that neither the symlink nor anything else in the container's closure *refers* to the target — keeping it alive is the host's job, see - `services.hyperhive.c0re.claudeCodePackage`. + `services.hyperhive.deploy.hive-controller.claudeCodePackage`. ''; }; @@ -255,7 +255,7 @@ in # The symlink registers no store reference — the target isn't # among this derivation's inputs, so nothing here keeps the # binary alive. That is deliberate and it is the host's job: - # see `services.hyperhive.c0re.claudeCodePackage`. + # see `services.hyperhive.deploy.hive-controller.claudeCodePackage`. pkgs.runCommandLocal "claude-code-pinned" { } '' mkdir -p "$out/bin" ln -s ${config.services.hyperhive.agent.claudeCodePath}/bin/claude "$out/bin/claude" diff --git a/nix/agent-modules/github.nix b/nix/agent-modules/github.nix index bab686ec..94085849 100644 --- a/nix/agent-modules/github.nix +++ b/nix/agent-modules/github.nix @@ -55,8 +55,8 @@ in a compromised agent can act within the token's scopes --- scope is the real blast-radius limiter. - On by default. Host-driven: set `services.hyperhive.github.enable = false` - to turn the integration off hive-wide (meta.rs propagates the override + On by default. Host-driven: set `services.hyperhive.swarm.github.enable = false` + to turn the integration off swarm-wide (meta.rs propagates the override into every agent). ''; }; diff --git a/nix/agent-modules/otel.nix b/nix/agent-modules/otel.nix index f080c8ea..90af2d6a 100644 --- a/nix/agent-modules/otel.nix +++ b/nix/agent-modules/otel.nix @@ -125,7 +125,8 @@ in Extra comma-separated entries appended to `OTEL_RESOURCE_ATTRIBUTES` after the built-in `service.name` / `agent` / `swarm` labels. - Host-driven via `services.hyperhive.otel.extraResourceAttributes`. + Host-driven via + `services.hyperhive.deploy.hive-controller.otel.extraResourceAttributes`. ''; }; diff --git a/nix/host-modules/deploy.nix b/nix/host-modules/deploy.nix index 72213240..f3370360 100644 --- a/nix/host-modules/deploy.nix +++ b/nix/host-modules/deploy.nix @@ -112,6 +112,21 @@ in [ "services" "hyperhive" "deploy" "bao" "servicesIssuerClientKeyFile" ] ) + # What hive-c0re hands this hive's agents: a per-host choice, so it + # sits with the controller's other deployment options. + (lib.mkRenamedOptionModule + [ "services" "hyperhive" "c0re" "claudeCodePackage" ] + [ "services" "hyperhive" "deploy" "hive-controller" "claudeCodePackage" ] + ) + (lib.mkRenamedOptionModule + [ "services" "hyperhive" "c0re" "dashboardPort" ] + [ "services" "hyperhive" "deploy" "hive-controller" "dashboardPort" ] + ) + (lib.mkRenamedOptionModule + [ "services" "hyperhive" "otel" "extraResourceAttributes" ] + [ "services" "hyperhive" "deploy" "hive-controller" "otel" "extraResourceAttributes" ] + ) + # The homeserver's port and its firewall switch, removed rather than # renamed: tuwunel listens on a unix socket only the gateway reaches, so # there is no host port to choose or open. diff --git a/nix/host-modules/hive-c0re/default.nix b/nix/host-modules/hive-c0re/default.nix index 82799991..e919af48 100644 --- a/nix/host-modules/hive-c0re/default.nix +++ b/nix/host-modules/hive-c0re/default.nix @@ -12,6 +12,7 @@ }: let cfg = config.services.hyperhive.c0re; + controllerCfg = config.services.hyperhive.deploy.hive-controller; # `/run/hyperhive`'s mode. Held in one place because whichever of the two # units below activates first creates the directory, so they cannot be @@ -110,8 +111,8 @@ let # anything that drops it. The failure mode is a garbage-collected # `claude` and a hive that can't take a turn, weeks after the commit # that caused it. - claude_code_path = cfg.claudeCodePackage; - dashboard_port = cfg.dashboardPort; + claude_code_path = controllerCfg.claudeCodePackage; + dashboard_port = controllerCfg.dashboardPort; agent_cpu_quota = cfg.agentCpuQuota; agent_memory_max = cfg.agentMemoryMax; agent_cpu_weight = cfg.agentCpuWeight; @@ -163,9 +164,10 @@ in # `claude`, at whatever unrelated moment the gc runs. Cheap # enough to just check. assertion = - cfg.claudeCodePackage == null || builtins.hasContext (builtins.toJSON cfg.claudeCodePackage); + controllerCfg.claudeCodePackage == null + || builtins.hasContext (builtins.toJSON controllerCfg.claudeCodePackage); message = '' - services.hyperhive.c0re.claudeCodePackage lost its store + services.hyperhive.deploy.hive-controller.claudeCodePackage lost its store context on the way into /etc/hyperhive/serve.json, so the package is no longer gc-rooted by the system closure and `nix-collect-garbage` may delete the claude every agent runs. diff --git a/nix/host-modules/hive-c0re/environment.nix b/nix/host-modules/hive-c0re/environment.nix index d70b54b0..e01c404e 100644 --- a/nix/host-modules/hive-c0re/environment.nix +++ b/nix/host-modules/hive-c0re/environment.nix @@ -65,27 +65,30 @@ in // lib.optionalAttrs (config.services.hyperhive.swarm.name != null) { HYPERHIVE_SWARM_NAME = config.services.hyperhive.swarm.name; } -// lib.optionalAttrs (!config.services.hyperhive.github.enable) { +// lib.optionalAttrs (!config.services.hyperhive.swarm.github.enable) { # GitHub integration is on by default; only signal the OFF override to # meta.rs, which then injects `services.hyperhive.agent.github.enable = false` into - # every agent. See services.hyperhive.github.enable. + # every agent. See services.hyperhive.swarm.github.enable. HYPERHIVE_GITHUB_DISABLED = "1"; } // lib.optionalAttrs config.services.hyperhive.otel.enable ( - # Hive-wide OTEL config -> read by meta.rs::otel_config and - # injected as build-time `services.hyperhive.agent.otel.*` into every agent. + # OTEL config (`swarm.hiveOtel`, `deploy.hive-controller.otel`, gated on + # this hive's `otel.enable`) -> read by meta.rs::otel_config and injected + # as build-time `services.hyperhive.agent.otel.*` into every agent. # Endpoint presence is the enable signal on the meta side; the # optional fields are only emitted when set so absent values # don't render no-op env lines. let otel = config.services.hyperhive.otel; + hiveOtel = config.services.hyperhive.swarm.hiveOtel; + controllerOtel = config.services.hyperhive.deploy.hive-controller.otel; # The first hop, this hive's own collector, under the two spellings its # two kinds of caller need. Agents get the swarm-wide name, so their # builds carry nothing hive-specific; this daemon gets the bridge # address, because on the host the name may resolve to loopback # (`gateway.localHostsEntry`), where the receiver does not listen. - agentFirstHop = "http://${otel.collector.host}:${toString otel.collector.port}"; - hostFirstHop = "http://${config.services.hyperhive.network.bridgeIp}:${toString otel.collector.port}"; + agentFirstHop = "http://${otel.collector.host}:${toString hiveOtel.collectorPort}"; + hostFirstHop = "http://${config.services.hyperhive.network.bridgeIp}:${toString hiveOtel.collectorPort}"; in { # `otel.endpoint` means "where telemetry ultimately goes" and keeps @@ -113,8 +116,8 @@ in # what has to honour it (see nix/host-modules/otel.nix). HYPERHIVE_OTEL_PROTOCOL = "http/protobuf"; } - // lib.optionalAttrs (otel.extraResourceAttributes != "") { - HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES = otel.extraResourceAttributes; + // lib.optionalAttrs (controllerOtel.extraResourceAttributes != "") { + HYPERHIVE_OTEL_EXTRA_RESOURCE_ATTRIBUTES = controllerOtel.extraResourceAttributes; } # HYPERHIVE_OTEL_HEADERS_CREDENTIAL is deliberately NOT emitted, and # its absence is the security half of this design: that variable would diff --git a/nix/host-modules/hive-c0re/options.nix b/nix/host-modules/hive-c0re/options.nix index f53cc448..6abf7a29 100644 --- a/nix/host-modules/hive-c0re/options.nix +++ b/nix/host-modules/hive-c0re/options.nix @@ -3,7 +3,9 @@ # `nixosModules.default` wires to its own outputs (they carry no # in-module defaults; see ../../../flake.nix). The read-only # `servedFrontend` option lives in ./theme.nix with the stylix wiring -# that computes it. +# that computes it. The hive's own values the meta-flake render hands +# its agents are declared here too, under +# `services.hyperhive.deploy.hive-controller`. { pkgs, lib, @@ -119,56 +121,6 @@ of the host's channel. ''; }; - claudeCodePackage = lib.mkOption { - type = lib.types.nullOr lib.types.package; - default = null; - example = lib.literalExpression "inputs.nixpkgs-unstable.legacyPackages.x86_64-linux.claude-code"; - description = '' - The `claude-code` build every agent runs, or `null` (the - default) to leave each agent on the `claude-code` from its own - nixpkgs — i.e. whatever `nixpkgsFlake` resolves to. - - This is the one binary the whole hive is built around, and it - moves fast enough that a release channel routinely trails - unstable by weeks on it. An agent cannot fix that for itself: - agents evaluate against the single nixpkgs hive-c0re injects, - so an `agent.nix` has no other tree to reach for. Set this from - a second nixpkgs in the host flake and every agent follows, - without moving the nixpkgs the rest of the container is built - from. - - What travels into the container is the **store path**, not the - derivation: agents share the host's `/nix/store`, so the binary - and its full closure are already reachable there — nothing - needs rebuilding or copying. hive-c0re writes the path into - each agent's generated flake as a plain string literal (a bare - path fed to `lib.types.package` would run `builtins.storePath`, - which is illegal under pure evaluation) and the agent module - puts its `bin/` on the harness's PATH. - - The flip side of a plain string is that nothing in the agent's - own closure refers to it, so the container cannot keep it - alive. The **host** does that instead: this package is - interpolated into `/etc/hyperhive/serve.json`, which puts it in - the host's system closure — so it is gc-rooted by the running - generation for exactly as long as that generation is the one - the agents were rendered from. The cost is that - `nix-collect-garbage` cannot reclaim an old `claude-code` until - every agent has been rebuilt past it and the old generations - are gone. - - The swarm's `swarm-claude-login` and `swarm-review-classifier` - containers, where this host runs them, follow the same pin and - keep their own nixpkgs' `claude-code` when it is `null`. They are - not agents, so pinning an older build for an agent-side - regression pins them too. - ''; - }; - dashboardPort = lib.mkOption { - type = lib.types.port; - default = 7000; - description = "TCP port the hive-c0re dashboard listens on."; - }; preBuildAgentTemplates = lib.mkOption { type = lib.types.bool; default = false; @@ -377,4 +329,59 @@ ''; }; }; + + # What this hive hands its own agents through the meta-flake render. Per + # host, so two hives may set them differently. + options.services.hyperhive.deploy.hive-controller = { + claudeCodePackage = lib.mkOption { + type = lib.types.nullOr lib.types.package; + default = null; + example = lib.literalExpression "inputs.nixpkgs-unstable.legacyPackages.x86_64-linux.claude-code"; + description = '' + The `claude-code` build every agent on this hive runs, or `null` (the + default) to leave each agent on the `claude-code` from its own + nixpkgs — i.e. whatever `c0re.nixpkgsFlake` resolves to. + + This is the one binary the whole hive is built around, and it + moves fast enough that a release channel routinely trails + unstable by weeks on it. An agent cannot fix that for itself: + agents evaluate against the single nixpkgs hive-c0re injects, + so an `agent.nix` has no other tree to reach for. Set this from + a second nixpkgs in the host flake and every agent follows, + without moving the nixpkgs the rest of the container is built + from. + + What travels into the container is the **store path**, not the + derivation: agents share the host's `/nix/store`, so the binary + and its full closure are already reachable there — nothing + needs rebuilding or copying. hive-c0re writes the path into + each agent's generated flake as a plain string literal (a bare + path fed to `lib.types.package` would run `builtins.storePath`, + which is illegal under pure evaluation) and the agent module + puts its `bin/` on the harness's PATH. + + The flip side of a plain string is that nothing in the agent's + own closure refers to it, so the container cannot keep it + alive. The **host** does that instead: this package is + interpolated into `/etc/hyperhive/serve.json`, which puts it in + the host's system closure — so it is gc-rooted by the running + generation for exactly as long as that generation is the one + the agents were rendered from. The cost is that + `nix-collect-garbage` cannot reclaim an old `claude-code` until + every agent has been rebuilt past it and the old generations + are gone. + + The swarm's `swarm-claude-login` and `swarm-review-classifier` + containers, where this host runs them, follow the same pin and + keep their own nixpkgs' `claude-code` when it is `null`. They are + not agents, so pinning an older build for an agent-side + regression pins them too. + ''; + }; + dashboardPort = lib.mkOption { + type = lib.types.port; + default = 7000; + description = "TCP port this hive's hive-c0re dashboard listens on."; + }; + }; } diff --git a/nix/host-modules/hive-gateway/options.nix b/nix/host-modules/hive-gateway/options.nix index beb3a5e9..2d4c0c70 100644 --- a/nix/host-modules/hive-gateway/options.nix +++ b/nix/host-modules/hive-gateway/options.nix @@ -62,7 +62,7 @@ in description = '' TCP port the gateway proxies non-static requests to. Defaults to `7000` (hive-c0re's out-of-the-box dashboard port). Operators - who change `services.hyperhive.c0re.dashboardPort` should set + who change `services.hyperhive.deploy.hive-controller.dashboardPort` should set `upstreamPort` to match — kept as a hardcoded default rather than a cross-reference to keep this module's options eval independent of c0re's option tree shape. diff --git a/nix/host-modules/hive-network.nix b/nix/host-modules/hive-network.nix index 52c5659e..21264644 100644 --- a/nix/host-modules/hive-network.nix +++ b/nix/host-modules/hive-network.nix @@ -29,7 +29,7 @@ let networkBase = builtins.bitAnd (ipToInt cfg.bridgeIp) (4294967295 - hostCount + 1); # Every switch that runs a hyperhive service on this host. Left out: - # `github.enable` (on by default, runs nothing by itself), the gateway's + # `swarm.github.enable` (on by default, runs nothing by itself), the gateway's # and matrix's sub-features, and `allSwarmServices`/`singleHostSwarm`, # which only take effect through the switches listed here. anyServiceEnabled = diff --git a/nix/host-modules/hyperhive.nix b/nix/host-modules/hyperhive.nix index 23083851..44d682c6 100644 --- a/nix/host-modules/hyperhive.nix +++ b/nix/host-modules/hyperhive.nix @@ -17,6 +17,10 @@ in ruth from the swarm dashboard like any other agent; a ruth container that already exists keeps starting with the hive. '') + (lib.mkRenamedOptionModule + [ "services" "hyperhive" "github" "enable" ] + [ "services" "hyperhive" "swarm" "github" "enable" ] + ) ]; # Canonical hive DNS domain shared by every subsystem that needs a @@ -192,15 +196,15 @@ in # the values it asserts. It is a deployment mode rather than a setting # this module's options read, so it lives with its consequences. - options.services.hyperhive.github.enable = lib.mkOption { + options.services.hyperhive.swarm.github.enable = lib.mkOption { type = lib.types.bool; default = true; example = false; description = '' - Hive-wide switch for the per-agent GitHub integration (the `gh` CLI + Swarm-wide switch for the per-agent GitHub integration (the `gh` CLI wrapper + git credential helper, per `services.hyperhive.agent.github.enable`). On by default: every agent gets the integration, inert until an operator links - a PAT for it in the swarm UI. Set `false` to turn it off for the whole hive --- the + a PAT for it in the swarm UI. Set `false` to turn it off for the whole swarm --- the meta-flake renderer (`hive-c0re/src/meta.rs`) then injects `services.hyperhive.agent.github.enable = false` into every agent. Exposed to hive-c0re as `HYPERHIVE_GITHUB_DISABLED` (set only when the integration is off). diff --git a/nix/host-modules/otel.nix b/nix/host-modules/otel.nix index 5ab1e6de..c527498f 100644 --- a/nix/host-modules/otel.nix +++ b/nix/host-modules/otel.nix @@ -57,6 +57,10 @@ in hostUnit = true; enable = config.services.hyperhive.otel.enable; }) + (lib.mkRenamedOptionModule + [ "services" "hyperhive" "otel" "collector" "port" ] + [ "services" "hyperhive" "swarm" "hiveOtel" "collectorPort" ] + ) ]; options.services.hyperhive.otel = { @@ -147,17 +151,6 @@ in ''; }; - extraResourceAttributes = lib.mkOption { - type = lib.types.str; - default = ""; - example = "deployment.environment=prod"; - description = '' - Extra comma-separated entries appended to - `OTEL_RESOURCE_ATTRIBUTES` after the built-in - `service.name` / `agent` / `hive` / `swarm` labels. - ''; - }; - collector.upstreamHeaderName = lib.mkOption { type = lib.types.str; default = "Authorization"; @@ -178,21 +171,6 @@ in ''; }; - collector.port = lib.mkOption { - type = lib.types.port; - default = 4318; - description = '' - Port the collector's OTLP/HTTP receiver listens on, at - `services.hyperhive.network.bridgeIp`. 4318 is the OTLP/HTTP - default. - - The port is contributed to - `services.hyperhive.network.exposeHostPorts`, which opens it on - the bridge interface only — so it is reachable from agent - containers and not from the outside world. - ''; - }; - collector.host = lib.mkOption { type = lib.types.str; readOnly = true; @@ -296,14 +274,48 @@ in }; }; + # Shared by every hive's otel tier and its agents, so an agent's build + # carries the same port whichever hive renders it. + options.services.hyperhive.swarm.hiveOtel = { + collectorPort = lib.mkOption { + type = lib.types.port; + default = 4318; + description = '' + Port of every hive collector's OTLP/HTTP receiver, at that hive's + `services.hyperhive.network.bridgeIp`: the port agents dial at + `services.hyperhive.otel.collector.host`. 4318 is the OTLP/HTTP + default. + + The port is contributed to + `services.hyperhive.network.exposeHostPorts`, which opens it on + the bridge interface only — so it is reachable from agent + containers and not from the outside world. + ''; + }; + }; + + # Read by this hive's hive-c0re, which stamps it on its own export and + # renders it into its agents', so it is a per-host choice. + options.services.hyperhive.deploy.hive-controller.otel.extraResourceAttributes = lib.mkOption { + type = lib.types.str; + default = ""; + example = "deployment.environment=prod"; + description = '' + Extra comma-separated entries appended to + `OTEL_RESOURCE_ATTRIBUTES` of this hive's hive-c0re and its agents, + after the built-in `service.name` / `agent` / `hive` / `swarm` labels. + ''; + }; + config = lib.mkIf config.services.hyperhive.otel.enable ( let otel = config.services.hyperhive.otel; autheliaCfg = config.services.hyperhive.swarm.authelia; deployCfg = config.services.hyperhive.deploy; swarmOtelCfg = config.services.hyperhive.swarm.otel; + hiveOtelCfg = config.services.hyperhive.swarm.hiveOtel; hiveName = config.services.hyperhive.hiveName; - listen = "${config.services.hyperhive.network.bridgeIp}:${toString otel.collector.port}"; + listen = "${config.services.hyperhive.network.bridgeIp}:${toString hiveOtelCfg.collectorPort}"; swarmName = "otlphttp/swarm"; gatewayCfg = config.services.hyperhive.gateway; authName = "oauth2client/swarm"; @@ -351,7 +363,7 @@ in { # Reachable from agent containers and nowhere else: this opens # the port on the bridge interface only. - services.hyperhive.network.exposeHostPorts = [ otel.collector.port ]; + services.hyperhive.network.exposeHostPorts = [ hiveOtelCfg.collectorPort ]; # The name agents dial, answered with the bridge IP the receiver binds. services.hyperhive.gateway.localNames = [ otel.collector.host ]; diff --git a/nix/host-modules/swarm-claude-login.nix b/nix/host-modules/swarm-claude-login.nix index 4b04d864..7d9f8bf7 100644 --- a/nix/host-modules/swarm-claude-login.nix +++ b/nix/host-modules/swarm-claude-login.nix @@ -23,12 +23,12 @@ let hyperhiveCfg = config.services.hyperhive; # The `claude-code` the agents run, so this container does not trail them on - # the CLI; `fallback`, the container's own nixpkgs', only where the hive pins + # the CLI; `fallback`, the container's own nixpkgs', only where this host pins # none. claudeCode = fallback: - if hyperhiveCfg.c0re.claudeCodePackage != null then - hyperhiveCfg.c0re.claudeCodePackage + if hyperhiveCfg.deploy.hive-controller.claudeCodePackage != null then + hyperhiveCfg.deploy.hive-controller.claudeCodePackage else fallback; cfg = hyperhiveCfg.deploy.claude-login; diff --git a/nix/host-modules/swarm-otel-service.nix b/nix/host-modules/swarm-otel-service.nix index 79b8bcf3..a6fc945a 100644 --- a/nix/host-modules/swarm-otel-service.nix +++ b/nix/host-modules/swarm-otel-service.nix @@ -67,7 +67,7 @@ in with it. ⚠️ **Deliberately not 4318**, the OTLP/HTTP default, because the - hive tier already uses it (`services.hyperhive.otel.collector.port`) + hive tier already uses it (`services.hyperhive.swarm.hiveOtel.collectorPort`) and every swarm container shares the host's network namespace. Two listeners claiming one port on one host is not a build failure — it is a runtime coin toss over which one gets it, with nothing in diff --git a/nix/host-modules/swarm-otel.nix b/nix/host-modules/swarm-otel.nix index 5a1a73a0..95d1b0ca 100644 --- a/nix/host-modules/swarm-otel.nix +++ b/nix/host-modules/swarm-otel.nix @@ -779,7 +779,7 @@ in others = [ cfg.telemetryPort cfg.producerPort - otelCfg.collector.port + config.services.hyperhive.swarm.hiveOtel.collectorPort ] ++ map (p: p.port) (lib.attrValues serviceProducers); all = derived ++ others; diff --git a/nix/host-modules/swarm-review-classifier.nix b/nix/host-modules/swarm-review-classifier.nix index 54ba25e6..e4d8496e 100644 --- a/nix/host-modules/swarm-review-classifier.nix +++ b/nix/host-modules/swarm-review-classifier.nix @@ -24,12 +24,12 @@ let hyperhiveCfg = config.services.hyperhive; # The `claude-code` the agents run, so this container does not trail them on - # the CLI; `fallback`, the container's own nixpkgs', only where the hive pins + # the CLI; `fallback`, the container's own nixpkgs', only where this host pins # none. claudeCode = fallback: - if hyperhiveCfg.c0re.claudeCodePackage != null then - hyperhiveCfg.c0re.claudeCodePackage + if hyperhiveCfg.deploy.hive-controller.claudeCodePackage != null then + hyperhiveCfg.deploy.hive-controller.claudeCodePackage else fallback; cfg = hyperhiveCfg.deploy.review-classifier; @@ -100,7 +100,7 @@ let otelEnv = lib.optionalAttrs otelCfg.enable { CLAUDE_CODE_ENABLE_TELEMETRY = "1"; OTEL_METRICS_EXPORTER = "otlp"; - OTEL_EXPORTER_OTLP_ENDPOINT = "http://${networkCfg.bridgeIp}:${toString otelCfg.collector.port}"; + OTEL_EXPORTER_OTLP_ENDPOINT = "http://${networkCfg.bridgeIp}:${toString hyperhiveCfg.swarm.hiveOtel.collectorPort}"; OTEL_EXPORTER_OTLP_PROTOCOL = "http/protobuf"; # Claude Code defaults to delta, which the metrics store drops. OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE = "cumulative"; diff --git a/nix/module-eval/swarm-services-switch.nix b/nix/module-eval/swarm-services-switch.nix index a4a86a10..0bf2343a 100644 --- a/nix/module-eval/swarm-services-switch.nix +++ b/nix/module-eval/swarm-services-switch.nix @@ -99,11 +99,11 @@ let deploy.builder.enable = false; }; - # Same, with the claude-code build the agents run pinned on the hive. Any + # Same, with the claude-code build the agents run pinned on this host. Any # package serves: only the path the containers are pointed at is read. swarmServicesPinnedClaude = hiveWithController { deploy.allSwarmServices = true; - c0re.claudeCodePackage = pkgs.hello; + deploy.hive-controller.claudeCodePackage = pkgs.hello; }; # Same, with one of those services placed on another host. Every derivation @@ -283,16 +283,25 @@ let { # The CLI is the one the agents run, not the container's own nixpkgs'. # Unpinned is the control: it falls back to the container's own `claude-code`. - name = "the claude login minter and the review classifier run the claude-code the hive pins"; + # serve.json is what hive-c0re renders every agent's `claudeCodePath` from. + name = "the agents, the claude login minter and the review classifier run the claude-code this host pins"; ok = let claudeOf = host: container: unit: var: host.containers.${container}.config.systemd.services.${unit}.environment.${var}; pinned = "${pkgs.hello}/bin/claude"; + servedClaude = + host: + (builtins.fromJSON ( + builtins.unsafeDiscardStringContext host.environment.etc."hyperhive/serve.json".text + )).claude_code_path; in - claudeOf swarmServicesPinnedClaude "swarm-claude-login" "swarm-claude-login" - "SWARM_CLAUDE_LOGIN_CLAUDE" == pinned + servedClaude swarmServicesPinnedClaude == "${pkgs.hello}" + && servedClaude swarmServicesHere == null + && + claudeOf swarmServicesPinnedClaude "swarm-claude-login" "swarm-claude-login" + "SWARM_CLAUDE_LOGIN_CLAUDE" == pinned && claudeOf swarmServicesPinnedClaude "swarm-review-classifier" "swarm-review-classifier" "SWARM_REVIEW_CLASSIFIER_CLAUDE" == pinned