Per mara: the mirror seeding belongs in hive-c0re's forge provisioning
sweep, where the core admin token + org-ensure already live — not a parallel
host-side nix oneshot.
- forge.rs: ensure_mirrors() reads HYPERHIVE_FORGE_MIRRORS (JSON list of
{upstream,dest}), ensures each dest org (reuse ensure_org) + creates the
pull-mirror via the migrate API (reuse forge_http, serde_json::json! body,
409/existing = success). Called in ensure_all() right after the SEEDED_ORGS
loop (token in scope, warn-and-continue like the other ensure_* steps).
- hive-forge.nix: forward effectiveMirrors to c0re via
systemd.services.hive-c0re.environment.HYPERHIVE_FORGE_MIRRORS; drop the
forgejo-seed-mirrors.service + its script + the host-side core-token read.
Keep the forge.mirrors option, DEFAULT_ACTIONS_URL=self (CI-gated), and the
dest-shape / no-c0re-namespace-collision assertions.
Verified locally: nix parse + treefmt (incl rustfmt) clean; serde/serde_json
patterns mirror dashboard.rs. cargo build runs in CI (no cc in my container).
nixfmt wanted the ${...} on a single line (verified locally:
nix build .#checks.x86_64-linux.formatting passes). The earlier CI 'docs'
failure was a transient muede-pc2 build hiccup — the docs check builds clean
locally on the same drv.
- drop the three cross-ref tracker tags from comments/description (prose only)
- build the orgs + migrate JSON bodies with jq -n --arg (an upstream URL
containing a quote no longer corrupts the request)
- don't auto-append the actions/checkout mirror when the operator already
declared that dest (avoids a duplicate effectiveMirrors entry when CI is on)
General-purpose mirror mechanism for the internal forge, per mara's call on
#2074 (real Forgejo pull-mirrors, nix-configured — not a pushed clone).
- services.hyperhive.forge.mirrors: list of { upstream, dest } pull-mirrors,
any repo. Each is created as a real Forgejo pull-mirror (re-syncs from
upstream), dest = <owner>/<repo> in its own org.
- When forge.ci.enable is set: an actions/checkout mirror is auto-appended +
forgejo DEFAULT_ACTIONS_URL is pointed at this instance, so CI
'uses: actions/checkout@vN' resolves on loopback — immune to a host-resolver
blip that previously reded every checkout (the seed/re-sync needs external
DNS, but that's off the CI critical path).
- forgejo-seed-mirrors.service: host-side oneshot (the core admin token never
enters a container), modelled on hive-ci-prefetch — waits <=60s for the core
token, then idempotently ensures each dest org + creates the pull-mirror via
the migrate API. partOf the forge container so it re-ensures on restart.
- assertions: dest must be <owner>/<repo>; mirror orgs can't shadow the
c0re-managed namespaces (config/shared/agents/core) so the seed never races
hive-c0re's own provisioning.
Supersedes #2074 (the raw-clone stopgap) as the durable #2072 fix.
General-purpose Forgejo pull-mirror config (services.hyperhive.forge.mirrors:
list of {upstream, dest}). When CI is enabled, auto-append an actions/checkout
mirror + point forgejo DEFAULT_ACTIONS_URL at this instance so CI's
actions/checkout@vN resolves on loopback (immune to host-resolver blips, #2072).
Seed oneshot (creates the dest orgs + pull-mirrors via the migrate API) is the
next commit.
Per mara: a secret in the nix store is not acceptable. The non-secret
OTEL config (telemetry-enable, endpoint, protocol, resource attributes)
stays in the world-readable managed settings json; the auth header is
handled separately at runtime so it never touches the store.
New hive-otel-header oneshot (only when otel.enable && headersCredential
is set): inherits the forwarded otel-headers systemd credential via
LoadCredential, reads it at start, and merges OTEL_EXPORTER_OTLP_HEADERS
into the agent's 0600 ~/.claude/settings.json env block via jq. claude
layers the user env on top of the managed settings, so both the harness
turn-loop and hivectl choom (same agent user) export with auth. The
token is read from disk at start and never copied into the nix store or
the world-readable managed file.
Ordering is best-effort (before=, not a hard dep): a failure leaves the
harness running and telemetry exporting unauthenticated. headersCredential
option description updated to reflect it's now wired.
nix fmt clean.
The option description still claimed the credential is loaded via
systemd LoadCredential, but this PR removed that path. Clarify that the
option is currently inert (only the unauthenticated OTEL export is
implemented) and that runtime header injection is a planned follow-up,
so configuring it doesn't silently no-op without explanation.
Per mara: configure OTEL in the generated claude settings json (what the
Claude Code docs suggest), not a launch wrapper or /etc shell file.
claude-code auto-discovers /etc/claude-code/managed-settings.json in
every context — the harness turn loop AND hivectl choom — so putting the
OTEL env there gives telemetry parity declaratively, with no wrapper and
no --settings plumbing.
- managed-settings.json: was a static shared .source; now, when OTEL is
enabled, a per-agent build-time jq merge of the base asset + an env
block (jq at build, not eval-time readFile, to avoid IFD). OTEL off =
the static asset verbatim.
- otelSettingsEnv carries the static OTEL knobs + OTEL_RESOURCE_ATTRIBUTES
with the agent name (build-time) and the hive/swarm names forwarded by
meta.rs into environment.variables (mara: forward host config into
agent config where needed).
- removed the hive-serve-otel ExecStart wrapper, the per-unit otelEnv,
and the otel-headers LoadCredential from the harness service — the
harness binary emits no OTEL itself; only claude does, and it now reads
the settings json directly.
Known follow-ups (noted in code): the auth header (otel.headersCredential,
opt-in/default-null) is a secret and can't live in the world-readable
settings file — authenticated collectors need a runtime mechanism; this
PR covers the unauthenticated default.
nix fmt clean.
Follow-up to the harness fix: any unit whose ExecStart is a generated
wrapper (a NixOS `script =` block or a writeShellScript) gets its journal
SyslogIdentifier derived from the wrapper's store-path basename — an
opaque, unstable name. Pin an explicit stable identifier on every unit
hyperhive declares.
script/wrapper units (leaked store paths): forgejo-gpg-init,
hive-gateway-self-signed-cert, hive-tls-ca, hive-ci-prefetch,
hyperhive-isolated-dns, tea-login, forge-avatar-sync,
matrix-avatar-sync, hive-dashboard-links.
direct-binary units (basename already sane; explicit for tidiness +
future-proofing): hive-c0re, hive-priv, hive-matrix-daemon,
hive-bash-daemon, weston.
Only changes the displayed line identifier; journalctl -u <unit>
filtering (keys on _SYSTEMD_UNIT) is unaffected.
Short-lived leaf bounds the blast radius of a key compromise. Doc notes
the renewal caveat: re-sign is activation-only today, so a hive that
stays up >30 days without a rebuild would need a periodic re-sign timer
to avoid leaf expiry (follow-up).
A 10-year leaf is pointless given the leaf auto-rotates (re-signed
within 30 days of expiry) and is long enough to make browsers warn.
397 days sits just under the 398-day CA/Browser-Forum leaf cap (so
browsers accept it) while bounding the blast radius of a leaf-key
compromise. The re-sign runs at service activation (hive-tls-ca is a
oneshot, no timer), so 397 stays comfortably above any gateway
rebuild gap.
The dashboard vhost split static-vs-backend on the request Accept
header (map $http_accept $dashboard_spa_target), so the same URL
behaved differently by content-type — e.g. /api/state fetched with
Accept: text/html wrongly returned index.html.
Now that all hive-c0re routes live under /api/ plus the single
/webhook/knowledge endpoint, route by path instead: /api/ and
/webhook/ proxy to c0re (SSE settings on /api/), everything else
serves the dist with try_files $uri /index.html. Drops the
dashboard Accept-header map and the @c0re named location.
Updates docs/gateway.md accordingly.
`nixos-container update <agent>` (run by the hive-priv root helper) invokes
nix, which shells out by bare name to `git` whenever it has to fetch or
re-resolve a git-source flake input, and to `ssh` to dispatch builds to
remote builders. The hive-priv service `path` had neither, so an agent whose
config carries a git-source input — or a stale flake.lock whose node URL no
longer matches the flake's declared input, forcing nix to re-resolve at eval
— fails the build with `executing "git": No such file or directory`, and
remote-builder dispatch fails with `Could not find executable 'ssh'`.
Add `pkgs.gitMinimal` + `pkgs.openssh` to the path. Latent-bug fix: nix
legitimately needs both for any git-source input / remote build. Surfaced by
the bitburner agent's build failing on a git-source input re-resolution.
The internal forge is the canonical store for the meta flake, every
agent's config repo, and the internal/* repos, so it can no longer be
optional. Remove the services.hyperhive.forge.enable toggle:
- hive-forge.nix: drop the `enable` option; the forge config now
deploys gated on `services.hyperhive.enable` (it ships with hyperhive).
- hive-c0re.nix: HIVE_FORGE_URL env unconditional; forge-public-URL gate
drops the enable check (keeps behindGateway).
- hive-gateway.nix: local /etc/hosts forge entry keyed on behindGateway.
- hive-ci.nix: drop the now-moot `forge.ci.enable requires forge.enable`
assertion (forge is always present); reword the option doc.
- nix/docs/default.nix: drop the `forge.enable = mkForce false` stub
(option gone); the options-doc eval stays light via hyperhive.enable.
- hive-c0re forge.rs / hivectl.rs: reword 'forge.enable = true' error
text to 'wait for hive-c0re to start the container' (the runtime
token-absent path is unchanged — it's a bootstrap-timing check, not
the opt-out).
- docs/approvals.md, docs/ci.md: drop stale forge.enable references.
Migration: configs that set `services.hyperhive.forge.enable = false`
must drop the line — the forge is now mandatory.
Prereq/companion to #1838 (PR-based config flow, which assumes the forge
is always present).
Add swarm.peers.<domain>.caCert (path to a peer hive's root CA PEM),
trusted everywhere the hive's own internal CA is — so a self-signed
peer hive can federate (matrix) and any in-hive consumer validates its
certs.
Mechanism (reuses the existing hive-CA embedding): the meta-flake
renderer embeds a LIST of CA files next to each agent's flake —
hive-ca.pem (the hive's own self-signed CA, when active) plus each peer
caCert as peer-ca-<N>.pem — and emits them all in
security.pki.certificateFiles, so every agent trusts them at build
time. The matrix container trusts the same peer CAs for federation TLS.
Nothing is installed in the host trust store; the certs live in the nix
store (no mutable host file).
- meta.rs: embedded_ca_files() = hive CA + peer CAs (from new
HIVE_PEER_CA_PATHS env); ca_embed_state() tracks the list (content +
add/remove); sync_agents materialises + stages the list; render emits
the multi-entry certificateFiles. Tests cover hive-only / hive+peers
/ peers-only / none.
- hive-c0re.nix: HIVE_PEER_CA_PATHS service env (colon-joined caCerts);
caCert / certFingerprint option docs updated to the hive-wide scope.
- hive-matrix.nix + docs/swarm.md: scope + comment updates.
certFingerprint stays the c0re-only leaf-pin path.
nginx proxied `<hive>/` straight to hive-c0re:7000, and hive-c0re served the
dashboard dist itself via `tower_http::ServeDir` (from `HIVE_STATIC_DIR` baked
into its service env). So a frontend-only change rebuilt the hive-c0re unit and
restarted the core daemon — every operator session dropped its SSE stream for a
pure CSS/JS change.
The gateway nginx now static-serves the dashboard dist directly; hive-c0re's
dashboard router is API-only. The split uses the Accept-header SPA fallback (the
same `map $http_accept` pattern the matrix/agent vhosts already use), so no
backend prefix has to be enumerated: a browser navigation (Accept: text/html)
whose path is not an on-disk asset gets the SPA index.html; everything else
(every /api route, the bare action/mutation routes, the two SSE streams, the
knowledge webhook — all Accept != text/html) falls through `try_files` to the
`@c0re` named location and is reverse-proxied to hive-c0re. A new c0re route
needs no gateway change.
- hive-c0re.nix: expose the themed dist as a new internal read-only option
`services.hyperhive.c0re.servedFrontend`; drop `HIVE_STATIC_DIR` from the
service env (the router no longer serves files).
- hive-gateway.nix: read that option in host-module scope (dashboardDist),
static-serve `dashboard/` with the Accept-header `try_files ... @c0re` split;
`@c0re` carries `proxy_buffering off` + a 1d read timeout for the SSE streams
and a duplicated auth_basic block (named locations do not inherit it). The
dashboard map is unconditional; the matrix map stays gated on the matrix GUI.
- dashboard.rs: drop the ServeDir fallback + the HIVE_STATIC_DIR resolution; the
router 404s unmatched paths (the gateway only proxies non-static requests).
- hive-c0re/Cargo.toml: drop the now-unused tower-http dependency.
- docs/gateway.md: document the dashboard static split + the `@c0re` fall-through.
The store path is reachable inside the gateway nspawn container (shared
/nix/store), mirroring how HIVE_AGENT_FRONTEND_DIR already exposes the per-agent
UIs. The gateway and c0re changes must land together (atomic cutover) or the
dashboard 404s — this needs a watched gateway + c0re rebuild.
The CI container set `nix.settings.sandbox-fallback` but never
`nix.settings.fallback`, so a cache-miss build with an unreachable
remote build machine had no local fallback and hard-failed. A single
down or DNS-unresolvable `nix.buildMachines` host therefore turned every
fresh `nix flake check` red hive-wide, in ~30s, even for diffs that
can't affect the nix build (pure JS/markdown). Add
`nix.settings.fallback = true` so CI degrades to a slower local build
instead of failing.
