Registering the OIDC login source makes an outbound HTTPS call - the CLI
fetches <issuer>/.well-known/openid-configuration to validate the
provider before writing the row. That URL is a swarm service name served
under the swarm CA, which the default system store has never heard of.
SSL_CERT_FILE was set on forgejo.service and not on
forgejo-sso-source.service, so the web service trusted the chain and the
registration one-shot did not. Same binary, same host, different unit.
The result was a 100% reproducible failure that no restart could fix:
nothing about restarting a unit gives it an environment variable it
never had. Every recorded run since the feature landed has failed with
x509: certificate signed by unknown authority.
The trust belongs to every process that makes the call, not to the
service that happens to be the obvious consumer.
Follow-up to the plumbing removal: docs/swarm/README.md gets the
biggest rewrite (drops the whole "Fingerprint format" section, fixes
the runtime-effects list, the WireGuard config example + "what the
mesh does" bullet), docs/gateway.md and hive-gateway/options.nix drop
4 "needs no certFingerprint" mentions, swarm-peers-removed.nix's
migration-warning text no longer tells an upgrading operator to carry
a field over that no longer exists, swarm.nix/swarm-wireguard.nix/
swarm-controller.nix/swarm-controller's main.rs get comment fixes
where they described the now-removed HYPERHIVE_PEERS shape. Also
caught one more stale "peer hives" mention in docs/web-ui/README.md's
SW4RM tab description that the first pass on this issue missed.
Mara wanted the underlying plumbing gone too, not just the dashboard
display. Traced every consumer before cutting:
- certFingerprint (services.hyperhive.swarm.hives.<name>.certFingerprint):
removed the nix option entirely. Its only consumer was the dashboard
code removed in the previous commits.
- HYPERHIVE_PEERS: removed entirely — the env var itself, the whole
block that built it in hive-c0re/environment.nix, and its entry in
meta.rs's FORWARDED_VARS (which forwarded it into every agent
container). Turned out to have zero real consumers, not just one:
the docs claimed hive-agent::identity::peers() read it for qualified
agent labels, but no such function exists — identity.rs only
qualifies THIS agent's own label with HYPERHIVE_HIVE_DOMAIN, nothing
peer-list-related. Grepped the whole hive-agent crate to confirm
before removing.
services.hyperhive.swarm.peerHives (the nix option HYPERHIVE_PEERS was
built from) is untouched — swarm-wireguard.nix reads it directly for
the wg-hive mesh, a real and unrelated consumer.
Verified: cargo build/clippy/test -p hive-c0re -p swarm-controller all
clean (needed nix develop -c per the usual -lsqlite3 gap), all touched
nix files pass nix-instantiate --parse, and a throwaway nixosSystem
eval confirms the wireguard mesh still configures a peer's
wireguardAddress into wg-hive correctly with certFingerprint gone.
Follow-up to the peer-hives removal: docs/web-ui/dashboard.md drops
the whole "P33R H1V3S" section + its stale #peers hash and tab-strip
mentions; docs/swarm/README.md's runtime-effects list and
cross-references point at swarm-ui's roster page instead. Nix comments
in hive-c0re/environment.nix, swarm.nix and swarm-wireguard.nix that
described HYPERHIVE_PEERS/certFingerprint as dashboard-consumed are
updated to say what's actually still true (agent identity only).
Deliberately NOT removed: the `certFingerprint` nix option itself.
It has no remaining Rust consumer, but deleting a documented,
potentially already-set option is a bigger, breaking change than
removing dead display code — flagged in docs/swarm/README.md with a
note instead of silently dropped, so an operator or reviewer can make
that call explicitly rather than have it made for them here.
Removes the SW4RM tab's "P33R H1V3S" headline section
(#peers-block/#peers-section in dashboard.html, renderPeerHives in
swarm.js, its call site + import in tabs.js). Superseded by swarm-ui's
own hive roster page.
Verified: npm run build clean (would have failed to resolve
renderPeerHives if the import/export pair were left mismatched).
Removes the per-hive dashboard's "peer hives" display support:
`peer_hives` field on StateSnapshot, the `PeerHiveView` struct,
`parse_peer_hives()`, and `validate_cert_fingerprint()`. That surface
moved to swarm-ui's own hive roster page — no longer needed at the
hive level.
HYPERHIVE_PEERS itself is untouched: hive-agent::identity::peers()
still reads it for qualified agent labels, and the nix module still
forwards it to agent containers. Only this crate's dashboard-only
consumption is gone.
Verified: cargo build/clippy/test -p hive-c0re clean, grepped the
whole tree for stray peer_hives/PeerHiveView/parse_peer_hives
references after the removal — none left.
Per mara on the PR: less implementation detail, more operator focus. The
section argued why the design is shaped as it is - a cache would
disagree with itself, freshness must be derived rather than stored, the
timestamp is the server's - which is reasoning a reader of this page does
not need and cannot act on.
It now answers what an operator actually asks: what each freshness value
means they should do, what to set the threshold to and why that number,
what a 503 tells them, and that nothing publishes yet so an empty board
is expected rather than broken.
The design rationale is not lost, it is just homed where it belongs: the
status module's own doc comment, next to the code it constrains.
Both from argus's review on the PR, both non-blocking, both real.
The HTTP client had no timeout, and it runs INSIDE the auth callback: a
token endpoint that accepts the connection and then never answers would
hang the callback and the connection attempt that invoked it, with no
retry and nothing in the log to say why. That is the same hang class the
status endpoint's connection-state check exists to prevent, one layer up.
Failing fast lets async-nats back off and try again, which it already
does well.
And the startup log said "connected to the swarm queue" at a point where
`retry_on_initial_connect` guarantees no connection has been established
yet - so the journal would read "connected", then 503 "not connected"
moments later, and a reader would rightly distrust the second line rather
than the first.
The controller connects to the swarm queue as its own client and serves
what each hive last said about itself at GET /api/hives/status.
THE QUEUE IS THE STORE. A hive publishes into the `hive-status` JetStream
KV bucket (history 1) and the controller reads it per request, keeping no
copy. A cache here would be a second answer to the same question, free to
disagree with the first, and the disagreement surfaces as a hive reading
healthy on a dashboard while the bucket says otherwise. Whichever side
arrives first creates the bucket; both want the same shape.
Rows come from the roster rather than from the bucket, so an empty bucket
renders as a swarm nobody has heard from instead of a healthy one, and
`never_reported` stays distinct from `stale` - went quiet is a fault,
never spoke is usually a deployment that has not happened. Freshness is
derived at read time and never stored as a flag, because a stored
`healthy` boolean goes stale silently the moment nothing arrives, which
is the failure this endpoint is designed against. The timestamp is the
NATS server's, applied when the value landed, so a publisher cannot make
itself look fresher than it is.
Authentication is per connection attempt, not per process. Authelia
issues `client_credentials` tokens that expire in 3599s, and auth happens
at CONNECT, so a long-lived connection is fine but a reconnect an hour
later needs a token minted an hour later. `with_auth_callback` is re-run
by async-nats for each attempt, which handles expiry by construction
rather than by a timer - the alternative fails in the way this subsystem
exists to prevent, with the controller still serving while its data
quietly stops updating.
Three failure shapes are deliberate:
- A half-set environment is fatal; an absent one is not. Silently
behaving like an unconfigured host is how every hive ends up reading
`never_reported` with nothing to point at.
- The endpoint answers 503 rather than an empty list when the store
cannot be read. "I cannot reach the store" and "every hive is silent"
are different answers, and rendering the second turns a local fault
into an apparent swarm-wide outage.
- `retry_on_initial_connect` makes the daemon and the queue bootable in
either order, and the status handler refuses when the client is not
Connected rather than issuing a request into it - a request made in
that window does not fail, it waits, so every poll would hang and
learn nothing. `Pending` is the state a never-connected client is in,
which is why the test is `!= Connected` and not `== Disconnected`.
The rendering rules are a pure function over a map, so the semantics are
tested against a table rather than against a running server. The KV read,
the credential rotation and the 503 paths are covered behaviourally
instead: a real NATS server with a rotating token endpoint, asserting
that the controller recovers only when the credential rotates, and
mutation-tested by holding the credential wrong for the same window.
Second slice of #3274. Each entry in
`services.hyperhive.swarm.hives` gets an authelia client
`hive-<name>`, so a hive can authenticate to swarm services as
itself.
The clients are derived here rather than declared by a consuming
service, because a hive's identity belongs to the directory and not to
whichever service happens to use it first. mara's constraint is that a
hive's credentials "derive from the same identity" — one hive holds
one client and mints a different token per service from it — so the
queue declaring this list would mean the next consumer collides on the
same client id, and only at the moment it landed.
`oidc.hiveIdentities` defaults to whether the message queue is
enabled, that being the first service to need a hive to prove who it
is. It is an option rather than a hard-coded condition so the swarm
telemetry collector can turn it on without the queue.
No audience and no `access_token_signed_response_alg` here: the queue
learns which hive presented a token from the introspection response
(`{"active":true,"client_id":"hive-alpha",...}`, measured), so it needs
neither. The collector does need both, and they land with it.
Also asserts client ids are unique — newly reachable now that part of
the list is derived, since a hive named `x` and a service client named
`hive-x` never met before.
Refs #3274.
Single 🔗 button in the Shell header (direction 1 of 3 proposed on the
issue) — one fixed element regardless of link count, so it can't
clutter as the list grows, and visible on every route since Shell
wraps all of them. Opens a popover listing whatever GET /api/links
returns (icon + label, opens in a new tab); hidden entirely when the
list is empty, same "don't render a dead affordance" rule the old
dashboard's H0M3 tiles follow for Forge/Matrix.
Docs: docs/swarm/ui.md gets a short section on the feature + the
contribute-your-own-entry idiom.
Verified: npm run build + npm run typecheck clean, headless-chromium
screenshots of both the closed and (temporarily forced open for the
screenshot only) open states, pixel-sampled to confirm the popover is
actually using the dark theme vars and not just looking that way in a
downscaled preview.
New `services.hyperhive.swarm.controller.links` option (listOf {label,
icon, url}, same shape as the per-agent hyperhive.dashboardLinks) plus
a new GET /api/links route serving it, same pattern as the existing
hives/GET /api/hives.
Rather than one central hardcoded list, each service's own module
contributes its own entry when actually enabled on the controller's
host: swarm-authelia.nix, hive-matrix.nix (gated on gui.enable too,
since / on that vhost only serves fluffychat then) and
hive-forge/default.nix (gated on behindGateway) each push one entry,
the same list-merge idiom services.hyperhive.gateway.localNames
already uses. swarm-ui.nix contributes a static entry for its own
same-origin swagger docs. Adding a future service's link is a nix-only
change to that service's own module.
Verified: cargo build/clippy/test -p swarm-controller clean, a
throwaway nixosSystem eval confirms all 4 entries merge correctly into
SWARM_CONTROLLER_LINKS, nix build .#swarm-controller succeeds.
`renderClient` could only emit the authorization-code shape, so a
daemon client was expressed as an interactive one with an empty
redirect list. Authelia permits only the grants a client names, and an
omitted `grant_types` means authorization-code alone — so that shape
cannot obtain a token at all.
Measured against authelia 4.39.20, rendering exactly what this module
produced for `swarm-nats`:
client_secret_basic → unauthorized_client: The OAuth 2.0 Client is
not allowed to use authorization grant
'client_credentials'
introspection → {"active":false} (works)
Introspection is all the queue's responder needs today, which is why
nothing was visibly broken while the comment in `swarm-nats.nix`
described a grant that was never configured.
Adds `kind = "interactive" | "machine"` rather than inferring from an
empty `redirectUris`, because the two differ in what authelia permits
and not merely in what is populated. `openid` is dropped from a machine
client's scopes because authelia refuses that combination outright — a
daemon receives an access token and never an id-token.
An assertion rejects redirect URIs on a machine client: they are not
harmlessly unused, they mean the author believed a browser was
involved.
Refs #3274.
New /components route showing every ui primitive (Panel, StatusChip,
Table) in each mode it supports: Panel with/without a title,
StatusChip's four tones, Table populated and empty. Sample data only,
no API calls, so it renders identically regardless of backend state.
Linked from the Shell nav next to "overview".
Verified with npm run build + npm run typecheck, and a headless
chromium screenshot of the built dist.
Same bug class the facts sweep hunted for across everything else —
hive-c0re.nix is a module directory now
(nix/host-modules/hive-c0re/), environment.nix is where
HYPERHIVE_HIVE_DOMAIN is actually set. Caught doing a final grep
across the merged sweep for the known bad patterns before pushing.
- MAX_AGENT_NAME lives in lifecycle/mod.rs, not bare lifecycle.rs
- agent_server::handle_send -> socket_server::handle_send (no agent_server
module exists)
- AgentMeta response is missing matrix_accounts in the field list
- tool-group table missing list_containers/ack_until (added since written)
and the forge/web_tools groups entirely
- #[tool] fns live on AgentServer in hive-agent-mcp/src/mcp/mod.rs, not a
HiveServer in a nonexistent hive-ag3nt/src/mcp.rs; ToolGroup::tools()
lives in hive-sh4re/src/permissions.rs, not lib.rs
- Capability + as_str also moved to permissions.rs; capability-check
handler list was stale (agent_server.rs/mcp.rs don't exist)
- async-forms listener is bindAsyncForms in frontend/packages/shared,
not assets/tabs.js + assets/app.js
- 'rebuild is the reconcile verb' described a monolithic lifecycle::rebuild
that no longer exists — it's job_queue::templates::rebuild building a DAG,
and flake.nix is no longer regenerated host-side on rebuild at all
- meta::sync_agents signature (now takes &HiveEnv, not 3 loose params)
- lifecycle.rs -> lifecycle/host_config.rs bare-file ref
- nixosConfigurations.manager -> .ruth (no 'manager' name in flake.nix)
- manager socket path/owner: no /run/hyperhive/manager/ or manager_server,
it's socket_server::start_manager on the standard per-agent runtime dir
- Spawn approval also fires ApprovalResolved (finish_approval always does,
for every kind), not push_todo instead of a HelperEvent
- /agents/<n>/applied.git -> /applied/<n>/.git (matches the actual RO bind)
- lifecycle::rebuild / auto_update::rebuild_agent don't exist anymore;
the reconcile verb is job_queue::templates::rebuild
Verified against nix/host-modules/hive-gateway/*.nix + hive-matrix.nix +
hive-forge/default.nix:
- services.hyperhive.gateway.enable was removed (gateway is now
unconditional) — fixes the Dashboard link shape + Firewall posture
sections, which described a gateway-off mode that no longer exists.
- gateway.selfSignedTls is a removed option (fails eval), not a
deprecated-and-ignored one.
- The forge rootUrl auto-derivation now uses https:// behind the
gateway (httpsPort), not http://.
- Vhost map + discovery-flow + local-dev sections used matrix.<hive> /
forge.<hive>; current defaults are chat.<swarm-domain> and
forge.<swarm-domain> (both swarm-domain-rooted, per hive-matrix.nix's
gatewayHost and hive-forge's domain option).
- hive-c0re.nix is now a directory (nix/host-modules/hive-c0re/); pointed
the HIVE_FORGE_URL reference at the actual file, environment.nix.
Verified against nix/host-modules/hive-matrix.nix: gatewayHost's current
default is chat.<swarm-domain>, not matrix.<hive>, so the GUI mount point
and fluffychat client line were stale. Also rewrote the Assertion
rationale section, which named an assertion (hyperhiveDomain/serverName)
that doesn't exist in this module and missed the three SSO assertions
that do.
Verified against nix/host-modules/hive-gateway/dnsmasq.nix + gateway.localNames
contributors (hive-forge, hive-matrix, swarm-authelia): forge and the matrix
gateway vhost are swarm-domain names (forge.<swarm-domain>, chat.<swarm-domain>),
not sub-domains of the hive's own domain as the doc previously claimed.
Verified against hive-agent/src/stream_enrich.rs and frontend/packages/agent/src/app.js:
- per-tool icon/summary/category classification moved server-side to
stream_enrich.rs::enrich() a while ago; fmtToolUse/fmtArgsGeneric/toolIcon no
longer exist in app.js -- the client now just dispatches on the backend-stamped
_icon/_summary/_category/_body/_body_type fields. Rewrote the 'Renderer dispatch'
and 'Salient-arg formatting' (was 'fmtToolUse patterns') sections to match, and
retargeted the per-tool table's 'Everything else' row to fmt_args_generic.
- Write is no longer a rich diff row -- is_rich_tool() doesn't include it, and
fmt_builtin_tool formats it identically to Read (flat, no count). Only Edit
still gets the rich diff.
- TodoWrite and built-in Bash aren't in ALLOWED_BUILTIN_TOOLS (mcp_config.rs) any
more than Task is -- flagged both as dead paths for consistency with the
existing Task annotation.
- fixed stray 'ask* has an inline answer form' table note that contradicted the
doc's own later 'Inline ask-operator answer' section (no such form exists).
- terminal.css path: frontend/packages/shared/src/terminal/terminal.css, not
.../shared/src/terminal.css; marked npm dep is v18.x, not v4.x.
Verified against hive-agent/src/web_ui/state.rs: the struct is AgentLink
(AgentLinkKind for kind), not NavLink -- NavLink doesn't exist anywhere in
the Rust or frontend source.
Verified against frontend/packages/dashboard/build.mjs, hive-c0re/src/dashboard/mod.rs,
and hive-agent/src/web_ui/mod.rs:
- dashboard ships 9 page CSS bundles + colors/theme/common, not 4
- dashboard SSE routes are /api/dashboard/stream + /api/dashboard/history, not
/dashboard/stream + /dashboard/history
- per-agent bind_with_retry has a MAX_BIND_ATTEMPTS=12 cap (gives up), unlike the
dashboard's genuinely uncapped retry loop
- assets/swarm.js, assets/call.js -> swarm.js, call.js (no assets/ dir exists)
- server-warnings banner: was described as disk-pressure-only; it's now
a push-based registry with multiple producers (forge/CI boot
failures, agent-state warnings), and shown on more standalone pages
than listed
- browser tab title: actual default/format is "hyperhive // h1ve-c0re",
not "hive / c0re"
- SW4RM container tree: ContainerView.parent comes from
agent_config/topology.rs, not the ambiguous dashboard/topology.rs
(which only holds set-parent endpoints)
- T00L GR0UPS: tool-group enumeration was missing "forge"; root agent's
default (MANAGER_DEFAULT) is not literally "all groups" -- it excludes
forge and web_tools
- ST4TS moved to its own /stats.html page a while back; section still
called it a dashboard tab
- P33RS is a headline block within the SW4RM tab, not its own tab --
contradicted the doc's own chrome-header description
- active model badge: reads hyperhive-harness.json's active_model
field, not a bare harness/hyperhive-model file
- themed dialogs: modal.js lives in the shared package
(@hive/shared/modal.js), and themedToast takes (message, opts) not
a single options object
- graceful stop: backend uses a Signal/Drain job-queue node pair, not
a "GracefulStop" transient kind; hivectl flag is `stop --graceful`,
not a bare `--graceful`
- container_state_changed event: named a nonexistent
auto_update::rebuild_agent as an emit site; actual sites are
actions::approve and the job queue's own node execution
Verified against hive-c0re/src (dashboard/, stats/, job_queue/,
container_view.rs), hive-sh4re/src/permissions.rs, and the dashboard
frontend package (tabs.js, stats.js, swarm.js, builds.js, modal.js).
No structural/prose changes beyond what the fact corrections required.
Verified against hive-agent/src/paths.rs: harness-internal files
(hyperhive-events.sqlite, hyperhive-turn-stats.sqlite, hyperhive-model)
live in the separate harness dir, specifically kept out of the
agent-visible state dir.
Verified against hive-sh4re/src/manager.rs and docs/approvals.md: only
ApprovalResolved/ContainerCrash/NeedsUpdate/QuestionAsked/QuestionAnswered
are HelperEvents delivered as inbox messages. spawned/rebuilt/killed/
destroyed/needs_login/logged_in/config_ready aren't HelperEvent variants
at all — they deliver via Coordinator::push_todo/push_todo_submitter
(the in-container todo socket), not the broker inbox.
Verified against hive-c0re/src/agent_config/topology.rs: there is no
parent_of function; <parent> sentinel resolution happens in
topology::resolve_recipient.
Verified against nix/agent-modules/claude-settings.nix (claudeMarketplaces
and claudePlugins defaults both gained a second entry — the hive's own
claude-plugins marketplace and base@hyperhive plugin — that the doc never
picked up) and hive-agent/src/web_ui/state.rs (dashboardLinks is read by
the harness's own per-agent web UI, not hive-c0re / the operator
dashboard).
Verified against Cargo.lock (rmcp resolves to 2.2.0, Cargo.toml pins
"2") and hive-core-agent-sock/src/lib.rs (the Wake variant lives on
Request there; hive_sh4re::AgentRequest doesn't exist).
Verified against hive-agent/src/harness_state.rs (context_window_tokens
lives there, only imported into events.rs) and hive-agent/src/paths.rs
(config_dir() defaults to /run/hive-config, deliberately separate from
/run/hive/ which holds the per-agent socket).
Verified against hive-agent/src/main.rs and hive-core-agent-sock/src/lib.rs:
there is no AgentRequest/AgentResponse or ManagerRequest/ManagerResponse
alias anywhere in the tree — hive-agent uses hive_core_agent_sock's
unified Request/Response enum directly.
Verified against hive-c0re/src/lifecycle/host_config.rs (function lives
in the host_config submodule, not bare lifecycle::) and
nix/host-modules/hive-matrix.nix (the working Accept-header SPA
fallback lives there, serving fluffychat at the vhost root /index.html,
not a /matrix/ path prefix in a nonexistent hive-gateway.nix).
hive-c0re/src/dashboard.rs became a module directory
(hive-c0re/src/dashboard/) a while back; three more references to it
as a single file survived the #3281 sweep (conventions.md x2,
web-ui/shape.md's StateSnapshot pointer). Same bug class the sweep
was hunting for, argus caught it in review.
Two follow-ups the per-file sweep workers flagged but couldn't fix
themselves (either not their assignment, or not visible from their
worktree):
- web-ui/dashboard.md's P33RS tab named dashboard.rs, which no longer
exists (hive-c0re/src/dashboard/ is a module dir now); the function
moved to state_snapshot.rs.
- approvals.md and conventions.md used hive_ag3nt:: as a Rust
module-path prefix. hive-agent is a bin-only crate (no lib target),
so that prefix never resolved to anything; also fixed the
hive-ag3nt/prompts/system.md path to the real hive-agent/prompts/.
All three verified directly against the current source tree, not
guessed.
