The endpoint, protocol, temporality preference and resource labels were
shipped only inside claude's managed settings json, so they applied to
claude's own process. hive-bash-daemon, hive-mcp-http, hive-matrix-daemon
and hive-forge-notify are systemd *siblings* of claude rather than its
children, so nothing shipped there could ever reach them: `hive-metric`
invoked from a tool call exited with "OTEL_EXPORTER_OTLP_ENDPOINT not
set", which is the honest failure of a value it structurally could not
see.
Declare those variables container-wide in a new agent module instead —
systemd.globalEnvironment for every unit PID 1 starts, environment.variables
for login shells. Both are needed and neither implies the other; NIX_REMOTE
is set both ways for the same reason.
Claude keeps only what is genuinely its own: the telemetry master flag,
the feedback-survey flag, the version label, and which signals it
exports. A different producer in the same container may legitimately
emit only metrics.
The hyperhive.otel.* options move across with them. They have more than
one consumer now, so their home is the OTEL module rather than the
claude one.
Per mara's follow-up: the doc shouldn't read as 'no dynamic colour at
all' — mixing/lighter/darker derivation is fine and matches Material's
own approach, the constraint is that every derived colour still
traces back to the base16/stylix contract, not an independent source.
Addresses all 7 line comments from her REQUEST_CHANGES review:
- drop the issue-#3444 history pointer and any issue-number tracking
refs throughout (a design guide states expectations, it isn't a
change log or a status report)
- stop naming specific rejected technical solutions (SSE/WS) for a
design constraint — state the chosen shape only
- flip the component-first heuristic: build the primitive first/
alongside its first real caller, not after — the point is giving the
next thing built ready-made blocks, not lagging behind usage
- drop the 'Open questions' section entirely — that's what the issue
thread is for, not a doc
- stop leading Visual language with 'basis: Material Design' and then
immediately carving out big exceptions — lead with what we actually
want, mention Material as a minor closing influence instead
- drop 'future work'/timeline framing everywhere (motion override,
PWA-as-future-slice, 'not shipped yet' theming caveats) — state the
target design as the expectation, not its current build status
- reworded the stylix-wins theming bullet to drop 'build time', which
reads wrong from a frontend dev's perspective (stylix supplies the
palette separately, it isn't decided by the frontend's own build)
- matrix-rain reference is in packages/dashboard/src/home.js, not
swarm-ui — cite it correctly (dashboard package) rather than implying
it lives inside this doc's own scope.
- drop FormField from the visible component-first inventory (it has no
/components demo, deliberately — a real exception to the 'every new
ui/ component gets a demo' rule stated two paragraphs later) and note
the exception explicitly instead of leaving the contradiction.
Distills the design-language discussion on #3444 into a durable
reference: visual language, motion, theming policy, data-freshness/
refresh, error UX, empty states, layout/viewport, component-first
design, and attention — plus an open-questions table pointing at the
still-unbuilt issues (#3452/#3453/#3454/#3456) instead of asserting
them as settled.
Mirrors css-vars.md's shape and links to it + to /components rather
than duplicating either. First draft for comment, not a final spec.
Drops swarm.otel.url (a loopback default an operator had to override on a
split host) in favor of swarm.otel.domain -- the same
gateway.localNames + nginx-vhost-through-the-gateway shape every other
swarm service (authelia, grafana, victoriametrics, ui) already uses. The
hive tier's exporter now reaches it as https://<domain> unconditionally,
resolved locally by dnsmasq on a co-located host and over the real network
otherwise, instead of a config knob nobody sets until they hit the silent
drop.
Costs CA trust on the hive tier: otel.nix wires
lib/hive-ca-trust.nix's trustBundle with hostUnit = true on the
opentelemetry-collector host unit, the same flag #3441/#3442 added for
swarm-controller and hive-c0re.
mara, #3125 comment 58363: "go c".
observability.md described a single collector holding the upstream
credential. It also said endpoint and protocol are what agents are
handed; agents get the derived first hop, which has been true since the
collector was introduced.
The swarm tier is documented beside its sibling swarm services rather
than here, and the one line an operator must not miss - swarm.otel.url
on a hive that does not run them - is called out in both places, since
leaving it unset loses telemetry silently.
mara: "pls remove historical wording, only present pls".
The correction was written as a diff against what the docs used to claim
-- "this used to say X", "where this is going", "the intended state for
now". That is a changelog, and a reader arriving cold has to reconstruct
the current truth from it. The reasoning about why the old shape was
wrong belongs in the PR that changed it, not in the file.
Now says what is true: the controller interprets a delivery and emits a
semantic message; receipt is all that is wired today because the
swarm->hive channel does not exist yet.
mara asked for the docs and comments to be fixed in one PR before the
implementation, since a half-corrected description is what confuses a
reader coming back after a context compact.
Three more places said or implied "forward the payload":
- post_webhook_forge's doc ("relays the delivery to every hive ... the
payload is never parsed here")
- its OpenAPI request_body description ("the delivery can be relayed
unmodified") -- this one is published in the spec, so it was the most
externally visible of them
- DeliveryKind::as_str, which called the hook kind the event's routing
key; the message is semantic, and which hook it arrived on is an input
to deriving it rather than the thing sent
docs/swarm/README.md gains the direction next to the interim state, so
"two hooks, one of which only logs" reads as a step rather than a design.
argus's optional note. Nothing in the doc was inaccurate, but an operator
who sees the activation-time warning has no thread back to the page that
explains why pinning matters -- and this section is the explanation.
States the one property they need from it: it never fails the rebuild, so
acting on it is theirs to do before the ids are minted.
The endpoint landed inert: nothing pointed at it, so the only way to see
it work was to mint an HMAC by hand. Register the two swarm-wide hooks
at startup so a real forge event produces a journal line.
Registered ALONGSIDE the per-hive hooks, not instead of them. Every hive
keeps receiving and acting on its own deliveries; the controller gets a
copy and logs it. Moving the registration is a later step and has to be:
fan-out swarm->hive does not exist yet, so a hook moved now would point
at a receiver that forwards nowhere, silently on both sides.
Deliberately no stale-hook deletion arm, unlike the two per-hive
registrars this otherwise mirrors: theirs delete hooks matching their own
path with a foreign base, and the hives' hooks are not stale.
The route prefix is what keeps this safe. Both hive-side registrars
delete any hook ending in /webhook/knowledge or /webhook/config-pr with a
different base, so a swarm hook under those paths would be deleted by
every hive on every boot. Serving them under /webhook/forge/ avoids it,
and a test pins it -- there is nothing else that can.
SWARM_CONTROLLER_PUBLIC_URL is set only where the swarm vhost is served,
because a hook whose target_url nothing answers is worse than no hook.
The crate's module doc claimed no persistence and no writes, which this
change makes false, and docs/persistence.md had no swarm-controller entry
at all — the webhook secret is the daemon's first persisted state.
Six places asserted the old design as fact, and none of them mention the
change by name -- the class of doc breakage that is found by asking what
a diff made untrue, not by grepping for a feature:
- swarmctl/README.md and swarm-authelia-bridge/README.md both described
their own private canonical store. The bridge's "known limitation"
section described the seam as unsolved; it is what this fixes, so it
becomes what both writers must uphold instead.
- docs/swarm/{sso,ui,secrets}.md described a rendered artifact.
- The repo CLAUDE.md entry for swarmctl said the same.
- docs/tools/swarmctl-cli.md is regenerated (CI diffs it against the
clap tree), picking up the removed --store flag.
Operator-facing where it is read: the hand-editing consequence (values
survive a rewrite, comments do not) is stated in sso.md, where an
operator is being told to edit the file, rather than only in a module doc.
Its three error paths returned a bare string with a status code, which forces
a caller to treat the whole body as prose. hive-c0re converted some time ago,
so swarm-controller was the last backend on the old shape -- and it is the one
behind the hive status page's 503, where the body is frequently the entire
diagnosis rather than a summary.
Adds the commitment to docs/conventions.md, since it was implied by the code
in one daemon and written down nowhere: an endpoint of ours answering with a
bare string is a bug to file, not something callers work around.
The test asserts the rendered response -- media type plus an addressable
detail -- rather than the problem_details value, because a handler that built
the value correctly and then returned it as a string would satisfy a test
written against the type alone.
authelia now watches the users file, so the restart is redundant -- and it
was the wrong shape twice over. It could fail: a login was refused for a
user whose record was already correct on disk, with nothing in either log
implicating the reload. And it only ever worked for this writer --
swarm-authelia-bridge writes the same file and cannot restart anything,
since running unprivileged inside the container is the whole reason it may
write it at all. A reload that depends on which process did the writing is
not a reload.
--machine/--unit and their two env vars existed solely to name a
systemctl -M target, so they go with it. That drops two required settings
from the operator surface.
The three objections previously recorded against watch are all answered
now, and are kept next to the decision rather than deleted: the key is
verified against the pinned build (validate-config accepts it and rejects
a misspelling), the watch is on the directory so a rename is observed, and
partial reads are structurally impossible because every writer of this
file goes through write_atomic.
The error names the protocol and never the sandbox, so it reads like a
network fault. Records the two easy-to-miss families (AF_INET6 alongside
AF_INET, and AF_NETLINK for getaddrinfo) and the reason the directive rots:
it is a claim about what the program does, and nothing re-checks it when a
client is added.
Review catch: this PR relaxed the `otel.endpoint` assertion and staled the
canonical OTEL reference in the same stroke — `docs/observability.md` is
what CLAUDE.md points readers at for "what OTEL options are available",
and it still said required-full-stop while the new swarm/services.md
section said a local store satisfies it.
Also corrects the option's own description in otel.nix, which said the
same thing and renders into the generated options doc. Grepping the
reviewer's phrasing did not find that one; grepping the claim did.
Records the second destination where the "endpoint is where telemetry
ultimately goes" paragraph makes its claim, rather than only in the new
section a reader may not reach.
Review feedback: the metrics pair had nothing writing into it, and it sat
outside the switch that turns on every other swarm-wide service.
The collector now exports to VictoriaMetrics as well as upstream — a
fan-out, not a choice: a local store is for looking at this swarm, an
upstream is for whoever aggregates across swarms.
That makes a local store a complete destination on its own, so
`otel.endpoint` is no longer required when it runs here; a hive with
neither is still refused. The assertion only ever relaxes, so every
config that evaluated before still does.
`enableRequiredServices` now derives both halves, alongside matrix,
authelia and nats. They derive together because a store with no UI is
unreadable and a UI with no store is empty.
New @hive/shared/jobq-rollup.js (JobqRollup.tsx + jobq-rollup.css),
mirroring JobqGraph's shape exactly: JSX use plus an imperative
mountJobqRollup(container, props) for a plain-.js call site. Fetches
Vec<hive_jobq_wire::StateCount> off `endpoint`, sums Running+Finishing
roots as "running" and Pending roots as "queued", renders nothing when
both are zero. Optional `queueHref` adds a "view queue -> " link.
Swapped dashboard's hand-rolled queue-summary banner (swarm.js) over to
this component instead of keeping two parallel implementations - same
"one shared component" pattern JobqGraph already set for the rebuild
queue tree view. Mounted once into a new #jobq-rollup-section, kept as
a sibling of (not inside) #containers-section since that section gets
replaceChildren()-wiped on every container-state render, which would
tear down and remount a Preact tree on every tick. Refreshed via the
mount handle's .refresh() on rebuild_queue_changed, same as builds.js's
JobqGraph handle.
Also mounted in swarm-ui's /jobs page, above JobqGraph, with no
queueHref (a link back to the page you're already on is noise) - the
literal ask on hyperhive#3364.
Verified: npm run build (whole workspace) and swarm-ui typecheck both
clean, comment-block + issue-ref lints run manually, headless-chromium
screenshots of both the dashboard SW4RM tab and swarm-ui's /jobs page
against mocked /api/jobq/rollup payloads - banner renders identically
in both, with and without the queue link as expected.
Review: the public keys are not an exception, the paragraph was
incomplete. It is private key material and access tokens that are
file-based so they stay out of the nix store; public material in a
public store is fine.
Fixes the rule itself, which removes the need for the special case the
previous commit added.
secrets.md is the page that answers "who mints this and where does it
go" per topology, and the callout nkeys had no row. Adds one, plus the
`nk` invocations an operator needs on every topology that is not
all-local.
The note exists because the change provokes a question the page would
otherwise answer wrongly: this page opens with "a secret is a path,
never a value", and the queue takes two public keys as values. A reader
who meets those after reading that line concludes the page is stale.
fmt_args_generic's generic array handling collapsed `ids: [4]` — the
count — since mark_todos_done had no dedicated match arm. Added one,
matching the file's existing per-tool pattern (extracted into its own
helper to stay under the 100-line clippy limit on fmt_hyperhive_tool).
Ports the shadow-DOM <hive-jobq-graph> custom element
(frontend/packages/shared/src/jobq-graph/) to a Preact component
(JobqGraph.js) shared by the dashboard and swarm-ui, per hyperhive#3310.
- JobqGraph.js: written with plain h() calls (no JSX) so the same file
compiles unmodified under both the dashboard's text-loader CSS config
and swarm-ui's JSX config. Exports `JobqGraph` for JSX use and
`mountJobqGraph(container, props)` for the dashboard's non-JSX
imperative mount, returning a `{refresh(), update()}` handle matching
the old custom element's public surface. Same rendering contract as
before: indented state tree, payload.label verbatim, payload.data as
a generic key/value list, "waits on: <label>" text for Node-kind deps,
per-state filter checkboxes, optional cancel button.
- jobq-graph.css: light-DOM adaptation of the old shadow-scoped
stylesheet (:host -> .jg-root, otherwise unchanged).
- dashboard/src/builds.js: local mountJobqGraph() renamed to
mountRebuildQueue() to avoid colliding with the newly-imported shared
mountJobqGraph; cancel handling is now a plain onCancel callback
instead of a DOM CustomEvent listener (no shadow boundary to cross
anymore).
- dashboard + shared package.json: added preact as a dependency (matches
swarm-ui's existing pin, 10.29.8) - the dashboard was a vanilla-JS MPA
with no Preact/JSX pipeline before this.
- Removed the old hive-jobq-graph.js/.css entirely (confirmed via grep
it had exactly one consumer, dashboard/src/builds.js, so this is a
clean swap, not parallel maintenance of two implementations).
- Updated stale doc-comment references to the old element name in
builds.html, tabs.js, swarm.js, docs/web-ui/dashboard.md, and
hive-c0re/src/job_queue/mod.rs.
Verified: npm run build (whole frontend workspace) and npm run
typecheck (swarm-ui) both clean; cargo build/clippy/test -p hive-c0re
all clean (331 tests, 0 failures); headless-chromium screenshot of
/builds.html against a mock GET /api/jobq/graph payload confirms full
visual/behavioral parity with the old custom element (tree, filter
checkboxes, cancel buttons, error text, waits-on line, data list, live
build log panel).
This covers the dashboard-replacement half of hyperhive#3310 only. The
swarm-ui half (rendering the CreateAgent DAG on the agent-creation page)
is downstream of hyperhive#3306/#3124 landing - no swarm-ui page exists
yet to mount it in.
The section carried a "nothing publishes yet" note that is now false, and
said nothing about the one thing an operator has to do.
Written to the reader's question rather than the author's: what to set,
what defaults on an all-in-one host, what has to be carried by hand to a
hive that is not the swarm host, and where to look when a hive goes quiet.
The identity and the publish cadence are stated because they constrain the
staleness threshold an operator picks; the mechanism behind them is not.
Follow-up to the plumbing removal: docs/swarm/README.md gets the
biggest rewrite (drops the whole "Fingerprint format" section, fixes
the runtime-effects list, the WireGuard config example + "what the
mesh does" bullet), docs/gateway.md and hive-gateway/options.nix drop
4 "needs no certFingerprint" mentions, swarm-peers-removed.nix's
migration-warning text no longer tells an upgrading operator to carry
a field over that no longer exists, swarm.nix/swarm-wireguard.nix/
swarm-controller.nix/swarm-controller's main.rs get comment fixes
where they described the now-removed HYPERHIVE_PEERS shape. Also
caught one more stale "peer hives" mention in docs/web-ui/README.md's
SW4RM tab description that the first pass on this issue missed.
Follow-up to the peer-hives removal: docs/web-ui/dashboard.md drops
the whole "P33R H1V3S" section + its stale #peers hash and tab-strip
mentions; docs/swarm/README.md's runtime-effects list and
cross-references point at swarm-ui's roster page instead. Nix comments
in hive-c0re/environment.nix, swarm.nix and swarm-wireguard.nix that
described HYPERHIVE_PEERS/certFingerprint as dashboard-consumed are
updated to say what's actually still true (agent identity only).
Deliberately NOT removed: the `certFingerprint` nix option itself.
It has no remaining Rust consumer, but deleting a documented,
potentially already-set option is a bigger, breaking change than
removing dead display code — flagged in docs/swarm/README.md with a
note instead of silently dropped, so an operator or reviewer can make
that call explicitly rather than have it made for them here.
Per mara on the PR: less implementation detail, more operator focus. The
section argued why the design is shaped as it is - a cache would
disagree with itself, freshness must be derived rather than stored, the
timestamp is the server's - which is reasoning a reader of this page does
not need and cannot act on.
It now answers what an operator actually asks: what each freshness value
means they should do, what to set the threshold to and why that number,
what a 503 tells them, and that nothing publishes yet so an empty board
is expected rather than broken.
The design rationale is not lost, it is just homed where it belongs: the
status module's own doc comment, next to the code it constrains.
The controller connects to the swarm queue as its own client and serves
what each hive last said about itself at GET /api/hives/status.
THE QUEUE IS THE STORE. A hive publishes into the `hive-status` JetStream
KV bucket (history 1) and the controller reads it per request, keeping no
copy. A cache here would be a second answer to the same question, free to
disagree with the first, and the disagreement surfaces as a hive reading
healthy on a dashboard while the bucket says otherwise. Whichever side
arrives first creates the bucket; both want the same shape.
Rows come from the roster rather than from the bucket, so an empty bucket
renders as a swarm nobody has heard from instead of a healthy one, and
`never_reported` stays distinct from `stale` - went quiet is a fault,
never spoke is usually a deployment that has not happened. Freshness is
derived at read time and never stored as a flag, because a stored
`healthy` boolean goes stale silently the moment nothing arrives, which
is the failure this endpoint is designed against. The timestamp is the
NATS server's, applied when the value landed, so a publisher cannot make
itself look fresher than it is.
Authentication is per connection attempt, not per process. Authelia
issues `client_credentials` tokens that expire in 3599s, and auth happens
at CONNECT, so a long-lived connection is fine but a reconnect an hour
later needs a token minted an hour later. `with_auth_callback` is re-run
by async-nats for each attempt, which handles expiry by construction
rather than by a timer - the alternative fails in the way this subsystem
exists to prevent, with the controller still serving while its data
quietly stops updating.
Three failure shapes are deliberate:
- A half-set environment is fatal; an absent one is not. Silently
behaving like an unconfigured host is how every hive ends up reading
`never_reported` with nothing to point at.
- The endpoint answers 503 rather than an empty list when the store
cannot be read. "I cannot reach the store" and "every hive is silent"
are different answers, and rendering the second turns a local fault
into an apparent swarm-wide outage.
- `retry_on_initial_connect` makes the daemon and the queue bootable in
either order, and the status handler refuses when the client is not
Connected rather than issuing a request into it - a request made in
that window does not fail, it waits, so every poll would hang and
learn nothing. `Pending` is the state a never-connected client is in,
which is why the test is `!= Connected` and not `== Disconnected`.
The rendering rules are a pure function over a map, so the semantics are
tested against a table rather than against a running server. The KV read,
the credential rotation and the 503 paths are covered behaviourally
instead: a real NATS server with a rotating token endpoint, asserting
that the controller recovers only when the credential rotates, and
mutation-tested by holding the credential wrong for the same window.
Single 🔗 button in the Shell header (direction 1 of 3 proposed on the
issue) — one fixed element regardless of link count, so it can't
clutter as the list grows, and visible on every route since Shell
wraps all of them. Opens a popover listing whatever GET /api/links
returns (icon + label, opens in a new tab); hidden entirely when the
list is empty, same "don't render a dead affordance" rule the old
dashboard's H0M3 tiles follow for Forge/Matrix.
Docs: docs/swarm/ui.md gets a short section on the feature + the
contribute-your-own-entry idiom.
Verified: npm run build + npm run typecheck clean, headless-chromium
screenshots of both the closed and (temporarily forced open for the
screenshot only) open states, pixel-sampled to confirm the popover is
actually using the dark theme vars and not just looking that way in a
downscaled preview.
Same bug class the facts sweep hunted for across everything else —
hive-c0re.nix is a module directory now
(nix/host-modules/hive-c0re/), environment.nix is where
HYPERHIVE_HIVE_DOMAIN is actually set. Caught doing a final grep
across the merged sweep for the known bad patterns before pushing.
- MAX_AGENT_NAME lives in lifecycle/mod.rs, not bare lifecycle.rs
- agent_server::handle_send -> socket_server::handle_send (no agent_server
module exists)
- AgentMeta response is missing matrix_accounts in the field list
- tool-group table missing list_containers/ack_until (added since written)
and the forge/web_tools groups entirely
- #[tool] fns live on AgentServer in hive-agent-mcp/src/mcp/mod.rs, not a
HiveServer in a nonexistent hive-ag3nt/src/mcp.rs; ToolGroup::tools()
lives in hive-sh4re/src/permissions.rs, not lib.rs
- Capability + as_str also moved to permissions.rs; capability-check
handler list was stale (agent_server.rs/mcp.rs don't exist)
- async-forms listener is bindAsyncForms in frontend/packages/shared,
not assets/tabs.js + assets/app.js
- 'rebuild is the reconcile verb' described a monolithic lifecycle::rebuild
that no longer exists — it's job_queue::templates::rebuild building a DAG,
and flake.nix is no longer regenerated host-side on rebuild at all
- meta::sync_agents signature (now takes &HiveEnv, not 3 loose params)
- lifecycle.rs -> lifecycle/host_config.rs bare-file ref
- nixosConfigurations.manager -> .ruth (no 'manager' name in flake.nix)
- manager socket path/owner: no /run/hyperhive/manager/ or manager_server,
it's socket_server::start_manager on the standard per-agent runtime dir
- Spawn approval also fires ApprovalResolved (finish_approval always does,
for every kind), not push_todo instead of a HelperEvent
- /agents/<n>/applied.git -> /applied/<n>/.git (matches the actual RO bind)
- lifecycle::rebuild / auto_update::rebuild_agent don't exist anymore;
the reconcile verb is job_queue::templates::rebuild
