The external export is a setting of the swarm collector, so it moves to
services.hyperhive.deploy.swarm-otel.upstream. Renamed:
- otel.endpoint -> upstream.endpoint
- otel.protocol -> upstream.protocol
- otel.headersCredential -> upstream.headersCredential
- otel.collector.upstreamHeaderName -> upstream.headerName
The unreleased otel.upstreamSignals is declared as upstream.signals.
Hosts keep working through the rename warnings. Eval test covers the old
paths reaching the new options. Docs and stale comments point at the
new paths.
Adds services.hyperhive.otel.upstreamSignals, a list of "metrics" and/or
"logs" naming the signals the swarm collector sends to otel.endpoint. The
swarm's own stores always receive everything. The upstream exporter is
only defined when at least one signal uses it, since otelcol refuses an
exporter no pipeline references.
On deploy: nothing changes until the option is set; the default is
[ "metrics" "logs" ], today's behaviour. To stop the 404 flood from an
upstream that has no /v1/logs, set services.hyperhive.otel.upstreamSignals
= [ "metrics" ].
swarm-bao-granter-role exits 4 when the store is unsealed but refuses the
bootstrap token, and Restart=on-failure retried that every 30s for as long
as the token file existed. RestartPreventExitStatus=4 makes that permanent
failure leave the unit failed once. Exit 1 (store sealed or unreachable)
keeps retrying.
On deploy: the unit goes red once instead of looping every 30s. The stale
token on muede-lpt2 still needs the operator step (remove it or mint a
fresh one); this does not clear it.
nixos-rebuild switch no longer hangs when nginx is (re)started in the
same transaction: the cert unit is Before=nginx, so its synchronous
reload and nginx's queued start job waited on each other. The reload is
now queued with --no-block try-reload-or-restart, as the nixpkgs ACME
module does.
The logs/self pipeline shipped the collector's own journal to the upstream
exporter too. When the upstream failed, the collector logged the failure,
journald/self read it, and logs/self sent it upstream again: ~8 lines/s.
logs/self now exports only to otlphttp/victorialogs, and a filter processor
drops the collector's failure lines that name that exporter, so the store
exporter cannot rebuild the loop one hop later. Other collector lines,
including upstream failures, still reach the store.
On deploy, swarm-otel's store rows drop from ~8/s of error spam to its real
output.
hive-core's .gitconfig now resets credential.helper (globally and for the
forge URL) before adding hive-forge. git runs every matching helper, system
config first, so a host-level `oauth` helper blocked `nix flake update`'s
git ls-remote of agent-configs forever.
On deploy: the boot sweep's meta_lock completes and rebuilds run.
The collector's container ran with --link-journal=host and the host's
journal bound over /var/log/journal, so its own journal was never
written where anything read it. It now keeps its journal in the guest,
reads it with a second journald receiver, and gets the host's journal
at /run/host-journal.
On deploy swarm-otel rows start arriving in the store, stamped
service.name swarm-otel.
no behaviour change: a pure move of items and tests into new modules, plus
visibility edits (pub(crate)) and path updates in sibling modules that named
crate-root items. same routes, same OpenAPI, 561 tests before and after.
why: #5470, new features stop landing in one 6.4k-line file by default.
files outside swarm-controller/src: docs/agent-lifecycle/agent-roster.md,
swarm-ui JobsPage.tsx and AgentsPage.tsx, nix/host-modules/swarm-controller.nix
and swarm-grafana.nix hold comments that named main.rs for items that moved.
The agent's state/AGENTS.md stays the truth and its harness the only
writer. The harness mirrors the file into a new `agents-md` KV bucket,
and swarm-controller serves that copy, so a read works while the agent is
stopped. An edit is a unified diff plus the revision it was made against,
sent over the agent's existing control subject; the harness refuses a
stale base, applies the diff with `git apply --check` then `git apply` to
a temp copy, and writes the file only when both pass. Fixes#5519.
Files:
- swarm-queue-client/src/agents_md.rs, lib.rs: the bucket name and
subject, the mirror value, the `ApplyAgentsMd` control command and its
reply. `AgentControl` loses `Copy` because the new variant carries text.
- hive-agent/src/swarm_agents_md.rs, main.rs: the mirror task (start,
10 s re-read, immediate republish after an edit) and the apply.
- hive-agent/src/swarm_input.rs: the control loop answers both commands.
- hive-agent/src/claude_md_watch.rs: its AGENTS.md path is shared.
- hive-agent/Cargo.toml, Cargo.lock: sha1 for the git blob id (already in
the lock), tempfile moves from dev to runtime for the apply directory.
- swarm-controller/src/agents_md.rs, main.rs: the bucket reader and
GET/PUT /api/agents/{name}/agents-md; test AppState literals get the
new field.
- swarm-controller/src/agent_control.rs: the sink hands back raw reply
bytes, so each command decodes its own reply type.
- swarm-controller-api/src/agents_md.rs: the route bodies, shared with
swarmctl.
- swarmctl/src/agents_md.rs, main.rs: `swarmctl agent agents-md get|apply`.
- nix/host-modules/swarm-nats.nix, nix/module-eval/nats-authelia.nix: the
agent's publish grant on its own `$KV.agents-md.<agent>` key.
git needs no new package: agent containers enable programs.git, and the
harness unit's PATH includes /run/current-system/sw.
Claude Code (>= 2.1.277) and opencode both read AGENTS.md, but Claude
Code reads it only when no CLAUDE.md exists in its cwd or above. So
this is a plain rename with no CLAUDE.md stub left behind.
- hive-agent/src/agents_md_migrate.rs: new. At harness start, before
the first turn, renames state/CLAUDE.md to state/AGENTS.md when only
the former exists; logs an error naming both paths when both exist
and touches neither; does nothing otherwise. Idempotent.
- hive-agent/src/main.rs: calls it in serve_main; checkpoint prompt
and a comment name AGENTS.md.
- hive-agent/src/claude_md_watch.rs: the size watch reads AGENTS.md;
todo text, doc comments and tests follow. Module name, todo
subsystem key ("claude_md") and metric name
(hyperhive.agent.claude_md.lines) are unchanged so existing todo
rows and dashboard history carry over.
- hive-agent/src/turn.rs, hive-agent/prompts/system.md: prompts tell
the agent its file is AGENTS.md.
- hive-agent/src/{otel_turn_metrics,stats}.rs,
frontend/packages/agent/src/stats.js, hive-subagent-mcp/src/*.rs:
comments and a tool parameter description naming the agent's file.
- nix/host-modules/swarm-grafana/dashboards/agents.json: panel title
and description.
- claude-plugins skills claude-subagents, state-hygiene: they mean the
agent's own state file.
- CLAUDE.md -> AGENTS.md at the repo root; docs/crates/README.md link
(checked by check-doc-refs.sh), hive-c0re/README.md,
.prettierignore and the nix/sources.nix comment follow.
- docs/: every prose mention of the agent's file; persistence.md
documents the boot-time rename, claude-invocation.md the shadowing.
Fixes#5509
The poller's forge, authelia and collector calls go to leaves issued by
the swarm's own CA, and its reqwest/rustls roots come from SSL_CERT_FILE,
which the unit never set. Every pass since it was deployed fails
"invalid peer certificate: UnknownIssuer" before pushing, so the CI runs
dashboard has no hyperhive.ci.run.* samples (#5438).
- nix/host-modules/swarm-ci-metrics.nix: import lib/hive-ca-trust.nix and
register a host trustBundle for the unit, as swarm-controller does; it
adds swarm-ci-metrics-ca-bundle.service and sets SSL_CERT_FILE.
- nix/module-eval/ci-metrics.nix: a case asserting the unit's
SSL_CERT_FILE is its bundle path when self-signed, with the bundle
unit's ordering, and absent where the poller is off.
- docs/swarm/ci-metrics.md: how the poller trusts the swarm's CA.
The hive reads the agent-archetype key when it provisions and rebuilds an
agent and keeps it in swarm-agent-archetypes.json. For an archetype agent
the meta flake has no agent-<name> input and mkAgent imports
hyperhive.hyperhiveArchetypes.<name>.module; provisioning, rebuilds, the
meta-input cascade, a deploy's rev and the startup migration skip the
config repos. The harness gets HIVE_ARCHETYPE and the system prompt says
the agent has no config repo. The create form sends the picked archetype.
Adds the android-dev built-in.
Refs #5386
The dashboard vhost, the hive domain vhost and the dashboard's local
name move to hive-c0re/vhosts.nix; the matrix .well-known discovery and
the legacy /matrix/ redirect move to glue-matrix-hive-discovery.nix,
which adds them to the hive domain vhost. Both declare through
gateway.vhosts. hive-gateway/vhosts.nix keeps the catch-all only, and
the gateway no longer reads deploy.matrix or c0re.servedFrontend, nor
deploy.hive-controller.enable for its vhosts.
Part of #3742.
These three are per-host choices that hive-c0re reads and renders into
its agents, so they belong under the hive controller's deploy namespace
rather than swarm-wide:
- c0re.dashboardPort -> deploy.hive-controller.dashboardPort
- c0re.claudeCodePackage -> deploy.hive-controller.claudeCodePackage
(the claude-login and review-classifier containers on the same host
keep following it)
- otel.extraResourceAttributes -> deploy.hive-controller.otel.extraResourceAttributes
The old paths stay as mkRenamedOptionModule aliases, now in deploy.nix
with the namespace's other renames. github.enable and the collector port
stay swarm-wide.
Five host options whose values hive-c0re renders into every agent's
build move under services.hyperhive.swarm, so the swarm states each of
them once instead of every hive carrying its own copy:
c0re.dashboardPort -> swarm.dashboardPort
c0re.claudeCodePackage -> swarm.claudeCodePackage
github.enable -> swarm.github.enable
otel.collector.port -> swarm.hiveOtel.collectorPort
otel.extraResourceAttributes -> swarm.hiveOtel.extraResourceAttributes
Every old path is a mkRenamedOptionModule alias, so a host that sets one
keeps its value and gets an eval warning naming the new path. Types and
defaults are unchanged, so the rendered meta flake, serve.json and
hive-c0re's environment are the same as before on a host that sets
none of them.
The readers move with them: serve.json and its gc-root assertion, the
hive-c0re environment, the hive collector's listen address and bridge
port, the swarm collector's port-collision assertion, and the
claude-login and review-classifier containers' claude pin. The
swarm-services-switch fixture sets the swarm path and now also checks
that serve.json carries the pinned claude.
Part of #4857 (slice 2).
The builder's hyperhive input is the docs-stripped flake source, so an
agent config with docs.enable = true failed to evaluate: docs.source
defaults to reference-docs, built from the missing docs/. Pass the host's
c0re.hyperhiveDocs as a hyperhive-docs input and set docs.source from it,
as the hive's meta flake does.
Fixes#5411
The agent read the operator-profiles bucket with a keys() listing, an
ordered consumer whose grant (STREAM.INFO, CONSUMER.CREATE) and flow-control
reply had never run against a permissioned queue. The controller now writes
the whole list as one JSON array under one key, only when it changed, and
the agent reads it with one DIRECT.GET the way agent_inference::read_own
reads its key. The agent grant shrinks to that one subject.
swarm-controller reads each member of authelia's operator group's forge
profile every five minutes and keeps one key per operator in the new
operator-profiles bucket. Every agent may list and read that bucket, and
hive-agent renders it into the system prompt at harness start, falling
back to today's prompt when the bucket is empty or doesn't answer within
five seconds.
Fixes#5383
The reviewer now has messaging, inbox and execution. execution is what
gives an agent the bash server, without which it cannot run hive-forge
or git to read a PR and post a review. meta is dropped: get_agent_meta
is not part of reviewing a PR.
Refs #5386
A flake declares agent archetypes in its `hyperhiveArchetypes` output,
`<name> = { module; meta; }`. The hyperhive flake ships `reviewer`
(default permissions without `execution`).
swarm-builder answers `$SWARM.builder.archetypes` with the `meta` of
every archetype its own hyperhive source declares. swarm-controller asks
once the queue connects and every 10 minutes, checks each archetype as
`POST /api/agents` checks its settings, and stores the result in the
`agent-archetypes` KV bucket when it changed. `GET /api/archetypes`
serves it, and the create form's archetype select prefills permissions,
resource caps and inference from it.
The agent created is an ordinary agent with its own config repo; the
archetype's module is not applied yet.
Refs #5386
Each `hive` fixture imported its own nixpkgs. Pass the flake's pkgs
through nixpkgs.pkgs with nixpkgs' read-only module instead. On
module-eval-swarm-services-switch this drops nixpkgs instances from 52
to 23 and eval max RSS from 4.83 GiB to 3.28 GiB, with the same drvPath.
Agent fixtures keep their own nixpkgs: agent-modules sets
nixpkgs.config.allowUnfreePredicate, which a shared pkgs can't take.
swarm-metrics reads the swarm's VictoriaMetrics from inside an agent
container: query, query-range, series, labels and label-values, each one
GET against the store's Prometheus read API through the gateway, printing
the store's JSON. It authenticates the way swarm-logs does, with the
agent's own queue secret read from bao under the agent's leaf, sent as
a swarm-agent bearer.
On the metrics store's host the read API's auth map gains an agent arm
ahead of the Grafana one. It routes to swarm-metrics-agent-auth, a
swarm-logs-auth instance logged in under the existing swarm-metrics-auth
leaf, whose bao grant now includes swarm/agents/+/queue read. The check is
rate-limited per address like the log store's
(deploy.victoriametrics.agentAuth{Rate,Burst,VerdictTtl}).
nix/agent-modules/metrics.nix installs the wrapped binary when the hive
forwards HIVE_AGENT_METRICS_URL (from swarm.victoriametrics.domain) and a
store address, the same gate as swarm-logs. Adds the base:swarm-metrics
skill and the CLI reference with its freshness check.
Fixes#5395
The collector's oidc/swarm authenticator verifies the bearer offline as a
JWT. The controller's authelia client was registered without
accessTokenSignedResponseAlg, so authelia mints it an opaque
authelia_at_ handle, which that authenticator cannot verify: by config,
no push from the vcs or jobq exporter can be accepted. Register it with
RS256 like every other client whose token lands on an oidc/* receiver.
The SDK logs a failed export at debug only, so a refused push never
reached the journal. AuthenticatedHttpClient now warns on a failed token
mint, a transport error, or a non-2xx answer (status plus a body
snippet), once per kind of failure and again every 10 minutes with a
count, and logs once when a push succeeds after failing.
Refs #5393
Delete nix/templates/ruth.nix, the ruth nixosConfiguration, module and
toplevel, the hive-c0re managerToplevel option, and the meta flake's
isManager switch with lifecycle::MANAGER_NAME. Every agent, ruth
included, now builds from nix/templates/agent.nix. The template's only
content beyond agent.nix was docs.enable = mkDefault true; docs.enable
is off by default for every agent and ruth's own agent.nix turns it on.
Refs #5377
hh-swarm-queue-connect-failed matched "swarm queue connect failed; this
agent publishes nothing upward", which nothing has logged since the
harness started retrying its first connect and exiting (235bc481b). The
rule now matches the error the harness logs when every attempt has
failed, CONNECT_EXHAUSTED in swarm_queue.rs, and its comment and summary
describe the retry-then-exit behaviour.
A failed MintAgentIdentity node logged nothing: the error reached only
the job graph. mint_identity now warns with the agent and the error, and
hh-agent-identity-mint-failed fires on that line.
Refs #4651
Not every swarm's operator is she/her. The prompt now names no pronouns
and tells agents to say "the operator"; pronouns will come from a
per-operator profile setting. The operatorPronouns removal warning and
claude-invocation.md say the same.
