no behaviour change: a pure move of items and tests into new modules, plus
visibility edits (pub(crate)) and path updates in sibling modules that named
crate-root items. same routes, same OpenAPI, 561 tests before and after.
why: #5470, new features stop landing in one 6.4k-line file by default.
files outside swarm-controller/src: docs/agent-lifecycle/agent-roster.md,
swarm-ui JobsPage.tsx and AgentsPage.tsx, nix/host-modules/swarm-controller.nix
and swarm-grafana.nix hold comments that named main.rs for items that moved.
The agent's state/AGENTS.md stays the truth and its harness the only
writer. The harness mirrors the file into a new `agents-md` KV bucket,
and swarm-controller serves that copy, so a read works while the agent is
stopped. An edit is a unified diff plus the revision it was made against,
sent over the agent's existing control subject; the harness refuses a
stale base, applies the diff with `git apply --check` then `git apply` to
a temp copy, and writes the file only when both pass. Fixes#5519.
Files:
- swarm-queue-client/src/agents_md.rs, lib.rs: the bucket name and
subject, the mirror value, the `ApplyAgentsMd` control command and its
reply. `AgentControl` loses `Copy` because the new variant carries text.
- hive-agent/src/swarm_agents_md.rs, main.rs: the mirror task (start,
10 s re-read, immediate republish after an edit) and the apply.
- hive-agent/src/swarm_input.rs: the control loop answers both commands.
- hive-agent/src/claude_md_watch.rs: its AGENTS.md path is shared.
- hive-agent/Cargo.toml, Cargo.lock: sha1 for the git blob id (already in
the lock), tempfile moves from dev to runtime for the apply directory.
- swarm-controller/src/agents_md.rs, main.rs: the bucket reader and
GET/PUT /api/agents/{name}/agents-md; test AppState literals get the
new field.
- swarm-controller/src/agent_control.rs: the sink hands back raw reply
bytes, so each command decodes its own reply type.
- swarm-controller-api/src/agents_md.rs: the route bodies, shared with
swarmctl.
- swarmctl/src/agents_md.rs, main.rs: `swarmctl agent agents-md get|apply`.
- nix/host-modules/swarm-nats.nix, nix/module-eval/nats-authelia.nix: the
agent's publish grant on its own `$KV.agents-md.<agent>` key.
git needs no new package: agent containers enable programs.git, and the
harness unit's PATH includes /run/current-system/sw.
Claude Code (>= 2.1.277) and opencode both read AGENTS.md, but Claude
Code reads it only when no CLAUDE.md exists in its cwd or above. So
this is a plain rename with no CLAUDE.md stub left behind.
- hive-agent/src/agents_md_migrate.rs: new. At harness start, before
the first turn, renames state/CLAUDE.md to state/AGENTS.md when only
the former exists; logs an error naming both paths when both exist
and touches neither; does nothing otherwise. Idempotent.
- hive-agent/src/main.rs: calls it in serve_main; checkpoint prompt
and a comment name AGENTS.md.
- hive-agent/src/claude_md_watch.rs: the size watch reads AGENTS.md;
todo text, doc comments and tests follow. Module name, todo
subsystem key ("claude_md") and metric name
(hyperhive.agent.claude_md.lines) are unchanged so existing todo
rows and dashboard history carry over.
- hive-agent/src/turn.rs, hive-agent/prompts/system.md: prompts tell
the agent its file is AGENTS.md.
- hive-agent/src/{otel_turn_metrics,stats}.rs,
frontend/packages/agent/src/stats.js, hive-subagent-mcp/src/*.rs:
comments and a tool parameter description naming the agent's file.
- nix/host-modules/swarm-grafana/dashboards/agents.json: panel title
and description.
- claude-plugins skills claude-subagents, state-hygiene: they mean the
agent's own state file.
- CLAUDE.md -> AGENTS.md at the repo root; docs/crates/README.md link
(checked by check-doc-refs.sh), hive-c0re/README.md,
.prettierignore and the nix/sources.nix comment follow.
- docs/: every prose mention of the agent's file; persistence.md
documents the boot-time rename, claude-invocation.md the shadowing.
Fixes#5509
The poller's forge, authelia and collector calls go to leaves issued by
the swarm's own CA, and its reqwest/rustls roots come from SSL_CERT_FILE,
which the unit never set. Every pass since it was deployed fails
"invalid peer certificate: UnknownIssuer" before pushing, so the CI runs
dashboard has no hyperhive.ci.run.* samples (#5438).
- nix/host-modules/swarm-ci-metrics.nix: import lib/hive-ca-trust.nix and
register a host trustBundle for the unit, as swarm-controller does; it
adds swarm-ci-metrics-ca-bundle.service and sets SSL_CERT_FILE.
- nix/module-eval/ci-metrics.nix: a case asserting the unit's
SSL_CERT_FILE is its bundle path when self-signed, with the bundle
unit's ordering, and absent where the poller is off.
- docs/swarm/ci-metrics.md: how the poller trusts the swarm's CA.
The hive reads the agent-archetype key when it provisions and rebuilds an
agent and keeps it in swarm-agent-archetypes.json. For an archetype agent
the meta flake has no agent-<name> input and mkAgent imports
hyperhive.hyperhiveArchetypes.<name>.module; provisioning, rebuilds, the
meta-input cascade, a deploy's rev and the startup migration skip the
config repos. The harness gets HIVE_ARCHETYPE and the system prompt says
the agent has no config repo. The create form sends the picked archetype.
Adds the android-dev built-in.
Refs #5386
The dashboard vhost, the hive domain vhost and the dashboard's local
name move to hive-c0re/vhosts.nix; the matrix .well-known discovery and
the legacy /matrix/ redirect move to glue-matrix-hive-discovery.nix,
which adds them to the hive domain vhost. Both declare through
gateway.vhosts. hive-gateway/vhosts.nix keeps the catch-all only, and
the gateway no longer reads deploy.matrix or c0re.servedFrontend, nor
deploy.hive-controller.enable for its vhosts.
Part of #3742.
These three are per-host choices that hive-c0re reads and renders into
its agents, so they belong under the hive controller's deploy namespace
rather than swarm-wide:
- c0re.dashboardPort -> deploy.hive-controller.dashboardPort
- c0re.claudeCodePackage -> deploy.hive-controller.claudeCodePackage
(the claude-login and review-classifier containers on the same host
keep following it)
- otel.extraResourceAttributes -> deploy.hive-controller.otel.extraResourceAttributes
The old paths stay as mkRenamedOptionModule aliases, now in deploy.nix
with the namespace's other renames. github.enable and the collector port
stay swarm-wide.
Five host options whose values hive-c0re renders into every agent's
build move under services.hyperhive.swarm, so the swarm states each of
them once instead of every hive carrying its own copy:
c0re.dashboardPort -> swarm.dashboardPort
c0re.claudeCodePackage -> swarm.claudeCodePackage
github.enable -> swarm.github.enable
otel.collector.port -> swarm.hiveOtel.collectorPort
otel.extraResourceAttributes -> swarm.hiveOtel.extraResourceAttributes
Every old path is a mkRenamedOptionModule alias, so a host that sets one
keeps its value and gets an eval warning naming the new path. Types and
defaults are unchanged, so the rendered meta flake, serve.json and
hive-c0re's environment are the same as before on a host that sets
none of them.
The readers move with them: serve.json and its gc-root assertion, the
hive-c0re environment, the hive collector's listen address and bridge
port, the swarm collector's port-collision assertion, and the
claude-login and review-classifier containers' claude pin. The
swarm-services-switch fixture sets the swarm path and now also checks
that serve.json carries the pinned claude.
Part of #4857 (slice 2).
The builder's hyperhive input is the docs-stripped flake source, so an
agent config with docs.enable = true failed to evaluate: docs.source
defaults to reference-docs, built from the missing docs/. Pass the host's
c0re.hyperhiveDocs as a hyperhive-docs input and set docs.source from it,
as the hive's meta flake does.
Fixes#5411
The agent read the operator-profiles bucket with a keys() listing, an
ordered consumer whose grant (STREAM.INFO, CONSUMER.CREATE) and flow-control
reply had never run against a permissioned queue. The controller now writes
the whole list as one JSON array under one key, only when it changed, and
the agent reads it with one DIRECT.GET the way agent_inference::read_own
reads its key. The agent grant shrinks to that one subject.
swarm-controller reads each member of authelia's operator group's forge
profile every five minutes and keeps one key per operator in the new
operator-profiles bucket. Every agent may list and read that bucket, and
hive-agent renders it into the system prompt at harness start, falling
back to today's prompt when the bucket is empty or doesn't answer within
five seconds.
Fixes#5383
The reviewer now has messaging, inbox and execution. execution is what
gives an agent the bash server, without which it cannot run hive-forge
or git to read a PR and post a review. meta is dropped: get_agent_meta
is not part of reviewing a PR.
Refs #5386
A flake declares agent archetypes in its `hyperhiveArchetypes` output,
`<name> = { module; meta; }`. The hyperhive flake ships `reviewer`
(default permissions without `execution`).
swarm-builder answers `$SWARM.builder.archetypes` with the `meta` of
every archetype its own hyperhive source declares. swarm-controller asks
once the queue connects and every 10 minutes, checks each archetype as
`POST /api/agents` checks its settings, and stores the result in the
`agent-archetypes` KV bucket when it changed. `GET /api/archetypes`
serves it, and the create form's archetype select prefills permissions,
resource caps and inference from it.
The agent created is an ordinary agent with its own config repo; the
archetype's module is not applied yet.
Refs #5386
Each `hive` fixture imported its own nixpkgs. Pass the flake's pkgs
through nixpkgs.pkgs with nixpkgs' read-only module instead. On
module-eval-swarm-services-switch this drops nixpkgs instances from 52
to 23 and eval max RSS from 4.83 GiB to 3.28 GiB, with the same drvPath.
Agent fixtures keep their own nixpkgs: agent-modules sets
nixpkgs.config.allowUnfreePredicate, which a shared pkgs can't take.
swarm-metrics reads the swarm's VictoriaMetrics from inside an agent
container: query, query-range, series, labels and label-values, each one
GET against the store's Prometheus read API through the gateway, printing
the store's JSON. It authenticates the way swarm-logs does, with the
agent's own queue secret read from bao under the agent's leaf, sent as
a swarm-agent bearer.
On the metrics store's host the read API's auth map gains an agent arm
ahead of the Grafana one. It routes to swarm-metrics-agent-auth, a
swarm-logs-auth instance logged in under the existing swarm-metrics-auth
leaf, whose bao grant now includes swarm/agents/+/queue read. The check is
rate-limited per address like the log store's
(deploy.victoriametrics.agentAuth{Rate,Burst,VerdictTtl}).
nix/agent-modules/metrics.nix installs the wrapped binary when the hive
forwards HIVE_AGENT_METRICS_URL (from swarm.victoriametrics.domain) and a
store address, the same gate as swarm-logs. Adds the base:swarm-metrics
skill and the CLI reference with its freshness check.
Fixes#5395
The collector's oidc/swarm authenticator verifies the bearer offline as a
JWT. The controller's authelia client was registered without
accessTokenSignedResponseAlg, so authelia mints it an opaque
authelia_at_ handle, which that authenticator cannot verify: by config,
no push from the vcs or jobq exporter can be accepted. Register it with
RS256 like every other client whose token lands on an oidc/* receiver.
The SDK logs a failed export at debug only, so a refused push never
reached the journal. AuthenticatedHttpClient now warns on a failed token
mint, a transport error, or a non-2xx answer (status plus a body
snippet), once per kind of failure and again every 10 minutes with a
count, and logs once when a push succeeds after failing.
Refs #5393
Delete nix/templates/ruth.nix, the ruth nixosConfiguration, module and
toplevel, the hive-c0re managerToplevel option, and the meta flake's
isManager switch with lifecycle::MANAGER_NAME. Every agent, ruth
included, now builds from nix/templates/agent.nix. The template's only
content beyond agent.nix was docs.enable = mkDefault true; docs.enable
is off by default for every agent and ruth's own agent.nix turns it on.
Refs #5377
hh-swarm-queue-connect-failed matched "swarm queue connect failed; this
agent publishes nothing upward", which nothing has logged since the
harness started retrying its first connect and exiting (235bc481b). The
rule now matches the error the harness logs when every attempt has
failed, CONNECT_EXHAUSTED in swarm_queue.rs, and its comment and summary
describe the retry-then-exit behaviour.
A failed MintAgentIdentity node logged nothing: the error reached only
the job graph. mint_identity now warns with the agent and the error, and
hh-agent-identity-mint-failed fires on that line.
Refs #4651
Not every swarm's operator is she/her. The prompt now names no pronouns
and tells agents to say "the operator"; pronouns will come from a
per-operator profile setting. The operatorPronouns removal warning and
claude-invocation.md say the same.
Nothing sets the variable once otel.metricIntervalMs is gone, so the
turn-metrics, jobq-metrics and vcs-metrics exporters export at a fixed
60s, as hive-c0re's container-resource exporter already does. The
observability doc and the removal warning now say so.
Both are ignored with a warning through the same shim, now
nix/host-modules/hive-options-removed.nix.
- otel.metricIntervalMs: claude in every agent and hive-c0re's
container-resource exporter run at the 60s default. The agent-level
services.hyperhive.agent.otel.metricIntervalMs, set only by the meta
render, goes too (warning shim in nix/agent-modules/otel-removed.nix).
- otel.debug: services.hyperhive.agent.otel.debug stays, documented and
set in an agent's own config; the meta render no longer emits it.
Both options are now ignored with a warning (the swarm-peers-removed.nix
shape), so a host that still sets one keeps evaluating.
- operatorPronouns: the agents' system prompt names the pronouns as
she/her, the value the harness fell back to and the option's default.
The --operator-pronouns flag, HiveEnv field, meta-flake let binding
and HIVE_OPERATOR_PRONOUNS env var are gone.
- contextWindowTokens: the per-family defaults (haiku 200000, sonnet
1000000, opus 1000000) move into hive-agent's harness_state as a
table; the window the API reports after a turn still wins. The
--context-window-tokens flag, HiveEnv field and the rendered
HIVE_CONTEXT_WINDOW_TOKENS_<KEY> env vars are gone.
swarm-secret-publish and its path unit now run inside authelia's container,
beside the mint, and read each client secret at the path authelia writes it.
The host unit and host path unit, which read the secrets through the
container's root on the host, are gone, and so is
deploy.authelia.hostClientSecretDir, which only they read. The publisher keeps
its store identity, the secret-publisher leaf. The host hands the leaf and the
store's CA into the container as nspawn system credentials, and the unit loads
them; where this host mints the leaf, the container waits for swarm-bao-pki.
Authelia does not wait for the publisher, which retries every 30 seconds while
the store is down.
The container also runs the journal forwarder the other service containers
run, pushing to a receiver of its own on the swarm collector (serviceProducers
swarm-authelia) as a new machine OIDC client (swarm.authelia.otel.clientId)
that authelia registers for itself and the publisher carries to the store. The
fetch unit reads it back under the publisher's leaf, also handed to it as a
system credential, so the swarm-secret-publisher policy gains read on that one
leaf; its stanza keeps create and update, which the exact path would otherwise
take from the services/* prefix. Without the leaf the forwarder is off.
The container drops --link-journal=host, and the host collector stops reading
it. The services' "nothing at <path>" errors name the publisher in authelia's
container.
start_manager bound the same path register_agent binds. The migrate branch, hive-priv-sock::MANAGER_NAME and MANAGER_CONTAINER were no-ops or dead. The send allow-list docs promised a manager exemption the code never had. Tests that used ruth as the manager fixture use plain names.
ruth is created from the swarm dashboard like any agent. An existing ruth container starts at boot through the boot reconcile, like every other agent. A definition of services.hyperhive.ruthless now fails evaluation with a removal message.
A swarm-ci-metrics host unit on a 5-minute timer lists every forge
repo's Actions runs as the new read-only ci-metrics account and pushes
each finished run's duration and queue wait over OTLP, stamped at the
run's stop time. The first pass backfills from run 1. swarm-controller
mints the account's token into bao and keeps a ci-metrics team with
read on the Actions unit in every organisation.
Closes#5376
The dashboard filtered by hive and agent only, so the review classifier's
API-key spend could not be told apart from the agents' subscription usage.
Add a service variable (agents or the review classifier) and apply it to
every panel and to the agent variable's own query.
Drop the H0M3 tile hub and the core page. The dashboard is now the index;
kept state is its fifth tab. A pages menu in the chrome links flow, stats,
API docs and the swarm UI. The swarm apex URL comes from
services.hyperhive.swarm.domain via HIVE_SWARM_PUBLIC_URL and the new
swarm_public_url field of the state snapshot.
A pki start refused by its start limit failed every reader that
Requires= it with `dependency`. That leaves a unit inactive, not
failed, so Restart=on-failure never fired and readers without a
refresh timer never ran again (#5381: grafana 502 for ~43 min).
Readers and granting units now Wants= + After= pki; one whose leaf is
still missing fails its login with exit 1 into its existing retry.
pki gets StartLimitIntervalSec=0: it is deliberately not
RemainAfterExit, so each reader's start re-runs it.
`forgejo-swarm-controller-account`, in the forge's container, now puts
the swarm-controller's forge token into the swarm secret store itself,
under the forge publisher's leaf. The container already receives that
leaf as nspawn system credentials for its journal forwarder; the mint
loads the same three. The token reaches bao on stdin (`value=-`) and
the forge's API through a header file, never as an argument, and never
touches disk.
The mint's guard is a marker holding the published token's name, at
/var/lib/forgejo/swarm-controller-token-name, written last: after the
put, and after every other `swarm-controller-boot-*` token of the
account is revoked through forgejo's admin API, authenticated as the
new token. So a put that failed, or a marker lost while the store kept
an older token, leaves no live token behind once a mint completes. The
mint logs in to the store and checks that the account is a site admin
before it mints, so a store that is down or an account the admin API
would refuse mints nothing on each retry. Failures retry under
store-retry; forgejo does not wait for the mint. No bao read and no
grant change: the forge publisher keeps create/update on the token.
Removed: the host's `swarm-bao-forge-publisher` service and path unit,
which read the token file through the container's root, and their
entries in the store and policy ordering glue. The forge publisher's
policy unit stays; `swarm-bao-forge-oidc` still uses the leaf.
On deploy, an existing forge host has no marker, so the mint runs once:
it mints a new token, publishes it, revokes the old one, and deletes
the old /var/lib/forgejo/swarm-controller-token. The controller's next
forge call gets a 401, re-reads the store and carries on.
Adds LockPersonality, RestrictRealtime, ProtectKernelLogs and
ProtectKernelModules to the hive-priv unit. Nothing in hive-priv's
sources or its children uses personality(2), real-time scheduling,
/dev/kmsg, /proc/kmsg or explicit module loading.
A module-eval case pins the four directives on the rendered unit.
The metrics store's container now runs the journal forwarder the other
service containers run, pushing to a receiver of its own on the swarm
collector (serviceProducers swarm-victoriametrics) as a new machine OIDC
client (swarm.victoriametrics.otel.clientId), registered by a glue file
wherever authelia runs and published to the store by the secret publisher.
The container keeps its loopback-only netns. The collector cannot dial a
unix socket, so swarm-victoriametrics-egress-relay, a socket-activated
systemd-socket-proxyd inside the container, takes its connections on
127.0.0.1:4380 and passes them to /run/swarm-victoriametrics-egress/egress.sock.
The host's nginx listens there and sends /swarm-victoriametrics/ to the swarm
collector and /api/oidc/token to authelia, both by name and verified, and
refuses every other path. nginx gets ReadWritePaths on that directory and
removes a stale socket before it starts.
The container cannot reach the store, so the client secret is fetched on the
host by swarm-victoriametrics-forwarder-oidc under swarm-metrics-auth's leaf,
the shape swarm-bao-forwarder-oidc has, and bound into the container
read-only. The forwarder module gains hostSecretFile for that: set, it renders
no fetch unit. swarm-metrics-auth's policy gains one read stanza on the
forwarder's client path.
The container drops --link-journal=host, and the host collector stops reading
it.
Also lists the forge publisher's and matrix-ctl's leaves in the forwarder
secrets row of docs/swarm/secrets.md, which the hive-forge/hive-matrix change
missed.
