diff --git a/docs/network.md b/docs/network.md index 5ec8cd3b..1e40c4b2 100644 --- a/docs/network.md +++ b/docs/network.md @@ -177,11 +177,8 @@ namespace. ### Reaching host services (`exposeHostPorts`) By default agents can only reach the host on 80/443 (+53 DNS), so a -host-side service on another port — e.g. a dev OTLP collector you want -agents to reach directly — is unreachable. (hyperhive's own telemetry -needs none of this: `otel.enable` opens its collector's port itself, and -`otel.endpoint` is the *upstream*, which no agent ever dials. See -`docs/observability.md`.) +host-side service on another port — e.g. a dev OTEL collector for +`services.hyperhive.otel.endpoint` (see `docs/observability.md`) — is unreachable. `services.hyperhive.network.exposeHostPorts = [ 4318 ];` opens each listed TCP port `P` on the bridge-interface `allowedTCPPorts`, so an diff --git a/docs/observability.md b/docs/observability.md index 19daedc6..5e4f5b01 100644 --- a/docs/observability.md +++ b/docs/observability.md @@ -16,25 +16,18 @@ services.hyperhive.otel = { }; ``` -`enable` is the single gate. `endpoint` is where telemetry ends up after it -leaves the swarm — optional, because the swarm's own metrics store -(`swarm.victoriametrics.enable`) is a destination in its own right. With both, -telemetry goes to both. See +`enable` is the single gate. `endpoint` is where telemetry goes upstream — +required when enabled *unless* this host runs the swarm's own metrics store +(`swarm.victoriametrics.enable`), which is a destination in its own right. With +both, telemetry goes to both. See [`swarm/services.md`](swarm/services.md#metrics-victoriametrics--grafana). **There is exactly one way telemetry leaves a hive: through the collector that `enable` starts on the host.** Agents never talk to `endpoint` themselves — they export unauthenticated to a bridge address only their own containers can -reach. That collector forwards to the swarm's -([`swarm/services.md`](swarm/services.md#telemetry-collector-otel)), which is -the single process holding the upstream credential and the only writer to the -swarm's store. No agent holds a copy, and neither does this hive. - -The hive collector reaches the swarm collector by its gateway name -(`swarm.otel.domain`, default `otel.`) — the same DNS-and-CA-trust -shape every hive-to-swarm-service hop uses, not a URL an operator has to point -anywhere. A hive that does not run the swarm's services still resolves that -name through the gateway; nothing here needs setting for the split-host case. +reach, and the collector forwards upstream with the auth header. So the +upstream credential exists in one place, on the host, and no agent ever holds +a copy. ⚠️ **The collector is therefore in the path of all telemetry.** It runs on the same host as the agents and restarts on failure, and telemetry is not the @@ -49,8 +42,8 @@ port on the bridge interface only. So "unauthenticated to a bridge address" means *reachable from an agent container*, not *presents a credential*. The consequence, stated because it is a choice rather than an oversight: **any -agent can push arbitrary OTLP, and it is forwarded on under the operator's -credential.** Neither tier can tell a container's genuine Claude Code stats +agent can push arbitrary OTLP, and the collector forwards it upstream under the +operator's credential.** It cannot tell a container's genuine Claude Code stats from anything else shaped like OTLP arriving on that port — including data smuggled out in resource attributes on an otherwise-legitimate export. @@ -68,44 +61,33 @@ closed by this design, and nothing here should be read as closing it. Master switch. When true, all other options below take effect. -### `services.hyperhive.otel.endpoint` — string, default `""` +### `services.hyperhive.otel.endpoint` — string, required when enabled unless the swarm store runs here -Upstream OTLP endpoint URL, read by the swarm's collector. Example: -`"https://collector.example.com/otel"`. +Upstream OTLP endpoint URL. Set as `OTEL_EXPORTER_OTLP_ENDPOINT` for every +agent. Example: `"https://collector.example.com/otel"`. -Leave it empty on a swarm running its own metrics store — that store is then -the destination. With neither, the swarm collector is refused at eval: -telemetry with nowhere to go is a misconfiguration, not a quiet no-op. - -Not what agents are handed. Their endpoint is this hive's own collector, -derived from the bridge address, so setting this changes where telemetry -*ends up* and never what a container is told. +Leave it empty **only** on a host running `swarm.victoriametrics.enable` — the +local store is then the destination and the collector writes there instead. +With neither, `enable` is refused at eval: telemetry with nowhere to go is a +misconfiguration, not a quiet no-op. ### `services.hyperhive.otel.protocol` — enum, default `"http/protobuf"` -Wire protocol for the **upstream** link, honoured by the swarm collector's -exporter. Accepted values: +OTLP wire protocol, passed as `OTEL_EXPORTER_OTLP_PROTOCOL`. Accepted values: - `"http/protobuf"` (default) - `"http/json"` - `"grpc"` -Agents are not affected: their first hop is this hive's collector, whose -OTLP/HTTP receiver takes protobuf whatever the upstream wants. - ### `services.hyperhive.otel.headersCredential` — string or null, default `null` Absolute path to a secret file on the host holding the upstream auth header as `NAME=value` (e.g. `Authorization=Bearer `). -**Only the swarm's collector reads it** — the one tier that talks to the -upstream. It arrives as an `EnvironmentFile` on that unit, so the value is -never read by nix, never copied into the store or the generated config, never -passed in argv — and reaches **neither an agent container nor a hive's own -collector**. An agent cannot read the upstream credential because it is never -given one. - -Set it on the host running the swarm's services; a hive that only forwards has -no use for it. +**Only the host collector reads it.** It arrives as an `EnvironmentFile` on the +collector's unit, so the value is never read by nix, never copied into the +store or the generated config, never passed in argv — and **never forwarded +into an agent container**. An agent cannot read the hive's upstream credential +because it is never given one. Leave `null` if the upstream needs no auth header; the collector then sends none rather than an empty one. @@ -148,56 +130,46 @@ metrics on process exit, so interval tuning is not required for metrics to be exported. A lower value gives more frequent intermediate flushes within long-running turns — cosmetic, not a correctness knob. -## The two collectors +## The host collector -Telemetry crosses two collectors, and which one you configure depends on what -the host is: - -| | runs where | receives from | does | -|---|---|---|---| -| **hive tier** — `otel.enable` | every hive with agents | that hive's agents, on the bridge | forwards to the swarm tier. Holds no credential, picks no destination | -| **swarm tier** — `swarm.otel.enable` | once per swarm | every hive's collector | writes the swarm's store and exports upstream | - -An all-local host runs both, and needs nothing said about the hop between them. +`enable` starts an OpenTelemetry collector on the host. It is not optional and +there is no second path — that is the whole point: ```nix services.hyperhive.otel = { enable = true; endpoint = "https://collector.example.com/otel"; # the upstream - headersCredential = "/run/secrets/otel-headers"; # only the swarm tier reads it + headersCredential = "/run/secrets/otel-headers"; # only the host reads it }; ``` -**Why the hive tier isn't optional.** Exporting straight to `endpoint` means -every agent needs the credential to authenticate — and the harness delivers -that token into the agent's own `~/.claude/settings.json`, a file the agent can -read. `0600` protects it from other containers, not from the agent itself. As -long as the direct path stays *selectable*, that hole stays selectable; an -option that can reintroduce it is a hole with extra steps. +**Why it isn't a knob.** Exporting straight to `endpoint` means every agent +needs the credential to authenticate — and the harness delivers that token into +the agent's own `~/.claude/settings.json`, a file the agent can read. `0600` +protects it from other containers, not from the agent itself. As long as the +direct path stays *selectable*, that hole stays selectable; an option that can +reintroduce it is a hole with extra steps. -**Why the tiers stay separate on one box.** They are not collapsed when -co-located: an all-local hive is a statement about *where* processes run, not -about the shape of the deployment. A boundary that disappears locally is one -the local deployment stops testing. - -**`endpoint` keeps meaning "where telemetry goes upstream."** Neither tier -redefines it — the agent-facing value is *derived* +**`endpoint` keeps meaning "where telemetry goes upstream."** The collector +does not redefine it — the agent-facing value is *derived* (`http://:`), so an existing deployment's `endpoint` keeps working unchanged. The bridge port is contributed to `exposeHostPorts` automatically; there is nothing to open by hand. +What the collector *added* is a second destination: on a host running the +swarm's metrics store it writes there too, so `endpoint` is no longer the only +place telemetry can land — and no longer the only way to have one. + ### `services.hyperhive.otel.collector.port` — port, default `4318` -The OTLP/HTTP port the hive tier listens on, bound to the bridge IP only. The -swarm tier has its own (`swarm.otel.port`, default `4319`) — they share a -network namespace when co-located, so the two must differ. +The OTLP/HTTP port the collector listens on, bound to the bridge IP only. ### `services.hyperhive.otel.collector.upstreamHeaderName` — string, default `"Authorization"` -Name of the header the swarm tier sends upstream. The **value** comes from the +Name of the header the collector sends upstream. The **value** comes from the credential file at runtime (`EnvironmentFile` → `${env:}`), never from nix — so header names are config and header values are secrets, which is the -only split a static header map can express. +only split the collector's static header map can express. ⚠️ **`endpoint` must be valid for `protocol`.** The upstream exporter follows `otel.protocol` (`grpc` → the gRPC exporter, otherwise OTLP/HTTP), and the gRPC diff --git a/docs/swarm/services.md b/docs/swarm/services.md index bb5dc93e..8608cb6c 100644 --- a/docs/swarm/services.md +++ b/docs/swarm/services.md @@ -93,42 +93,15 @@ login form is switched off whenever SSO is configured. If you enable Grafana on a host with no authelia, the form stays on and Grafana's default `admin`/`admin` applies; change it before exposing that host. -**Where the data comes from.** The swarm's OTEL collector, below. +**Where the data comes from.** With `otel.enable` on, the hive's OTEL +collector writes into this store as well as to any upstream endpoint — +both, not one or the other, since a local store is for looking at this +swarm and an upstream is for whoever aggregates across swarms. That also +means `otel.endpoint` is no longer required when the store runs here: a +hive with a local store already has somewhere for telemetry to go. See +[`../observability.md`](../observability.md). Neither container is reachable except through the gateway: both bind loopback, and VictoriaMetrics' write endpoint takes no credential, so the collector is the only intended writer. -### Telemetry collector (OTEL) - -The swarm's collector receives from every hive's own collector and is the -only process that decides where telemetry goes: it writes the store above -and exports to `otel.endpoint`, doing both when both are configured. It -also holds the upstream credential, which is why no hive and no agent -needs one. - -It follows `swarm.enableRequiredServices` like the services above, in a -`swarm-otel` container. Its `swarm.otel.port` defaults to `4319` rather -than OTLP's usual `4318`, which the hive tier already uses — swarm -containers share the host's network namespace, so two collectors on one -port is a coin toss at runtime rather than an error at build time. - -Every hive's own collector reaches this one by its gateway name, -`swarm.otel.domain` (default `otel.`) — the same -by-domain-through-the-gateway shape every other swarm service uses, not a -loopback URL an operator has to redirect. There is nothing to set on a hive -that does not run the swarm's services; the name resolves through the -gateway either way. - -| Option | When you'd touch it | -|---|---| -| `swarm.otel.domain` | Only to rename it — the default already resolves correctly for every hive in the swarm. | -| `swarm.otel.port` | Only if something else on the services host already claims `4319`. | - -With neither `otel.endpoint` nor the store enabled, this collector is -refused at eval — a tier that receives samples and drops them looks -healthy while losing data. - -Agent-side configuration, and what a hive's own collector does, are in -[`../observability.md`](../observability.md). - diff --git a/frontend/packages/swarm-ui/src/pages/CreateAgentPage.css b/frontend/packages/swarm-ui/src/pages/CreateAgentPage.css index 394ad05c..789a6403 100644 --- a/frontend/packages/swarm-ui/src/pages/CreateAgentPage.css +++ b/frontend/packages/swarm-ui/src/pages/CreateAgentPage.css @@ -1,19 +1,16 @@ /* — the create-agent form. Input/button chrome now - comes from the shared `ui/` form kit (`TextField`/`SelectField`/ - `Button`); this file only owns the page's own layout + copy. Column, - not row: with a second field (hive) added, a row layout put fields of - different natural widths on one baseline and looked misaligned — - mara: "make it a col". Reuses the same base16 slots (../theme.css) - every other component draws from. */ + comes from the shared `ui/` form kit (`TextField`/`Button`); this + file only owns the page's own layout + copy. Reuses the same base16 + slots (../theme.css) every other component draws from. */ .create-agent-intro { margin: 0 0 1.5em; color: var(--muted); } .create-agent-form { display: flex; - flex-direction: column; - align-items: flex-start; + align-items: flex-end; gap: 0.75em; + flex-wrap: wrap; } .create-agent-result { margin-top: 1em; diff --git a/frontend/packages/swarm-ui/src/pages/CreateAgentPage.tsx b/frontend/packages/swarm-ui/src/pages/CreateAgentPage.tsx index 5d6d7634..35fc4722 100644 --- a/frontend/packages/swarm-ui/src/pages/CreateAgentPage.tsx +++ b/frontend/packages/swarm-ui/src/pages/CreateAgentPage.tsx @@ -16,35 +16,19 @@ // one path segment deep — a real bug, filed separately rather than // fixed here, but reason enough to avoid a nested route today. // -// `hive` field added because agent creation had no way to record which -// hive an agent runs on. `POST /api/agents` now requires it, so the -// roster fetched off `GET /api/hives` backs a required select here -// rather than a free-text field — a spawn target has to be one of the -// swarm's actual hives, same reasoning `Ident.parse` gets client-side -// pattern validation for `name`. -// -// Its name field, hive select, and submit button all come from the -// shared `ui/` form kit (`TextField`/`SelectField`/`Button`) rather than -// page-scoped input/button chrome — see that kit's own comments for why. -import { useEffect, useState } from 'preact/hooks'; +// First real form in this package — its name field + submit button now +// come from the shared `ui/` form kit (`TextField`/`Button`) rather than +// page-scoped input/button chrome, so a hive-picker soon landing on this +// same page has something to reuse instead of copying this page's CSS. +import { useState } from 'preact/hooks'; import { Link } from 'wouter-preact'; import { ApiErrorPanel } from '@hive/shared/api-error-panel.js'; import { readApiError, type ProblemDetails } from '@hive/shared/api-error.js'; import { Panel } from '../ui/panel/Panel.js'; import { TextField } from '../ui/text-field/TextField.js'; -import { SelectField, type SelectOption } from '../ui/select-field/SelectField.js'; import { Button } from '../ui/button/Button.js'; import './CreateAgentPage.css'; -// Mirrors swarm-controller's `HiveEntry` — same shape `OverviewPage` -// consumes off `/api/hives/status`, but this page hits the plain -// `/api/hives` roster (no status/freshness needed, just "what hives -// exist to spawn into"). -interface HiveEntry { - name: string; - domain: string; -} - interface CreateAgentResponse { node_id: number; } @@ -72,37 +56,8 @@ const NAME_PATTERN = '[a-z0-9\\-]{1,63}'; export function CreateAgentPage() { const [name, setName] = useState(''); - const [hive, setHive] = useState(''); - // `null` = still loading, `[]` = loaded but empty (a real, if unusual, - // swarm state) — distinct from "not fetched yet" so the placeholder - // option's label can tell the two apart. - const [hives, setHives] = useState(null); - const [hivesError, setHivesError] = useState(null); const [result, setResult] = useState({ status: 'idle' }); - // Loaded once, not re-fetched on submit: the roster changes rarely - // enough (a nix-level swarm config change) that staleness within one - // page visit isn't worth a second round-trip per keystroke/submit. - useEffect(() => { - (async () => { - const r = await fetch('/api/hives'); - if (!r.ok) { - setHivesError(await readApiError(r)); - return; - } - const data = (await r.json()) as HiveEntry[]; - setHives(data); - // Single-hive swarms are the common case — default it rather than - // making the operator pick the only option. Multi-hive swarms get - // no default (the placeholder stays selected), so `required` on - // the select below forces an explicit choice per mara's scope. - if (data.length === 1) setHive(data[0].name); - })().catch((e: unknown) => setHivesError({ detail: String(e) })); - }, []); - - const hiveOptions: SelectOption[] = (hives ?? []).map((h) => ({ value: h.name, label: h.name })); - const hivesReady = hives !== null && hives.length > 0; - async function submit(e: Event) { e.preventDefault(); setResult({ status: 'submitting' }); @@ -110,7 +65,7 @@ export function CreateAgentPage() { const r = await fetch('/api/agents', { method: 'POST', headers: { 'content-type': 'application/json' }, - body: JSON.stringify({ name, hive }), + body: JSON.stringify({ name }), }); if (!r.ok) { setResult({ status: 'error', problem: await readApiError(r) }); @@ -130,7 +85,6 @@ export function CreateAgentPage() { Create a new agent's swarm-level identity. This only queues the job — check{' '} jobs to watch it settle.

- {hivesError && }
- - diff --git a/frontend/packages/swarm-ui/src/ui/select-field/SelectField.tsx b/frontend/packages/swarm-ui/src/ui/select-field/SelectField.tsx index d9f66305..982b0fbd 100644 --- a/frontend/packages/swarm-ui/src/ui/select-field/SelectField.tsx +++ b/frontend/packages/swarm-ui/src/ui/select-field/SelectField.tsx @@ -18,8 +18,6 @@ export function SelectField({ onChange, options, required, - disabled, - placeholder, }: { id: string; label: string; @@ -27,13 +25,6 @@ export function SelectField({ onChange: (value: string) => void; options: SelectOption[]; required?: boolean; - disabled?: boolean; - // Rendered as a disabled, always-first `value=""` option — for a - // required select with no default (e.g. a multi-hive roster: no - // single right answer to pre-select), so the control shows something - // other than silently landing on whatever option happens to be first. - // Omit when the field always has a real default (`value` never `''`). - placeholder?: string; }) { return ( @@ -42,14 +33,8 @@ export function SelectField({ class="ui-form-control" value={value} required={required} - disabled={disabled} onChange={(e) => onChange((e.target as HTMLSelectElement).value)} > - {placeholder !== undefined && ( - - )} {options.map((o) => (