per #5471, verifiers stop depending on the queue client. the module is
store-free and needs only thiserror, so swarm-agent-verify gains one
dependency and loses its swarm-queue-client edge. consumers import it
from swarm_agent_verify; swarm-logs and swarm-metrics, whose only use of
swarm-queue-client was this module, drop that dependency, as does
swarm-logs-auth.
The module is a rename (97% similar); everything else in the diff follows
from the move:
- stored_secret.rs: private `chain()` helper, the 8-line copy of the queue
crate's `chain` the module called as `crate::chain`; the queue crate's
public `chain` stays.
- stored_secret.rs: `Duration::from_mins(5)` in `MAX_REREAD_DELAY` is
unchanged moved code, not a new use.
- swarm-secret-client Cargo.toml: gains futures-util, tokio and tracing as
dependencies and tokio `test-util` plus tracing-subscriber as dev
dependencies; the module and its tests use them.
- swarm-queue-client Cargo.toml: drops futures-util and the tracing-subscriber
dev dependency, which only the moved module used; its tokio dev comment is
reworded to the tests that remain.
- Cargo.lock: the same dependency moves, nothing else.
- swarm-queue-client lib.rs: imports `swarm_secret_client::stored_secret`,
and its tests use a small local `Store` in place of the moved module's
test-only `Fake`, which is no longer reachable.
- callers (hive-c0re, swarm-builder, swarm-ci-metrics, swarm-claude-login,
swarm-controller, swarm-nats-auth, swarm-review-classifier): import path
only; no re-export is left behind.
The rustls provider install is process setup every swarm binary runs, and
swarm-secret-client depends on no other swarm crate, so a binary that only
needs the provider and the store no longer links the queue client.
- swarm-secret-client/src/lib.rs, Cargo.toml: the function, its test, a
`//!` line on why it lives there, and the rustls dependency; rustls
`ring` as a dev-dependency feature, so the test builds both providers as
the binaries do and fails if the install is dropped.
- swarm-queue-client/src/lib.rs, Cargo.toml: function and test removed;
`connect()` calls the secret client's; rustls dropped (no other use),
swarm-secret-client added.
- swarm-metrics-auth/Cargo.toml: drop swarm-queue-client, whose only use
was the provider; the binary no longer links async-nats.
- swarm-logs-auth, swarm-logs, swarm-metrics, swarm-ci-metrics Cargo.toml:
the dep comment no longer names the provider (they keep the dep for
agent_token / the OIDC mint).
- Cargo.toml: the rustls comment names the provider's new home.
- 14 call sites (hive-agent, hive-c0re, hive-subagent-mcp x2,
swarm-builder, swarm-ci-metrics, swarm-claude-login, swarm-controller,
swarm-logs-auth, swarm-logs, swarm-metrics-auth, swarm-metrics,
swarm-nats-auth, swarm-review-classifier): one-line path change.
- Cargo.lock: the dependency edges above.
A subagent's terminal opened directly showed nothing until the subagent
published again: the relay was a core subscription, though its rows sit
in term-sub-<agent> for 24 hours. It now goes through the agent route's
replay-then-tail, so it opens with the subagent's newest 100 rows and
resumes after Last-Event-ID.
- term_stream.rs: the replay takes a stream name and a filter subject
instead of an agent, and `replay_or_live` (replay, else the live core
subscription) is the one fn both routes call. When the filter is one of
several subjects in the stream, a fresh reader's consumer starts at the
oldest row and drops all but the newest REPLAY_ROWS it counts as
pending, since the stream's newest sequences may belong to sibling
subagents. A single-subject stream (term-main-<agent>) picks its start
with replay_start exactly as before. Tests for both subagent cases.
- subagent_term.rs: the relay calls replay_or_live on the subagent's
subject in term-sub-<agent> and reads Last-Event-ID; module doc and
OpenAPI text no longer say "live only".
- main.rs: the handler test passes the new headers argument.
- swarm-queue-client agent_term.rs: replay_consumer_config takes the
filter subject instead of the agent.
- swarm-queue-client subagent_term.rs: consumer_create_grant, the
extended CONSUMER.CREATE form with a subagent subject as filter.
- swarm-nats-auth policy.rs: the controller holds that grant; the agent
grant's `$SWARM.term.*` filter is two tokens short of a subagent
subject, and the test that refused it now admits it.
- docs/swarm/ui.md, README.md: a subagent terminal opens with its newest
rows.
Fixes#5527
Each request now runs in its own task, at most CONCURRENT_REQUESTS (32)
at once, instead of being awaited inline by the subscription loop. A
request whose exp (nats-server sets it to send time plus its
authorization.timeout) has passed is skipped without a lookup or reply.
Fixes#5434.
The user JWT the callout issues carried no exp, so nats-server armed no
expiry timer and a connection kept its grant until the client dropped it:
a deleted or re-minted agent queue secret, a removed OIDC client or an
expired access token never reached a live connection.
Every issued user JWT now expires at the presented OIDC token's exp
(read from the introspection answer) or an hour after issue, whichever
is sooner; the agent queue-secret path has no exp of its own and gets
the hour. Every queue client reconnects through an auth callback that
re-reads or re-mints its credential, so the forced reconnect presents a
current one.
Fixes#5432
POST /api/agents takes an archetype the stored list holds, records it in
the new agent-archetype bucket, and queues no CreateRepo, AddRepoMember or
InitAgentConfigRepo for it. A destroy clears the key. Hives may read the
bucket.
Refs #5386
The agent read the operator-profiles bucket with a keys() listing, an
ordered consumer whose grant (STREAM.INFO, CONSUMER.CREATE) and flow-control
reply had never run against a permissioned queue. The controller now writes
the whole list as one JSON array under one key, only when it changed, and
the agent reads it with one DIRECT.GET the way agent_inference::read_own
reads its key. The agent grant shrinks to that one subject.
swarm-controller reads each member of authelia's operator group's forge
profile every five minutes and keeps one key per operator in the new
operator-profiles bucket. Every agent may list and read that bucket, and
hive-agent renders it into the system prompt at harness start, falling
back to today's prompt when the bucket is empty or doesn't answer within
five seconds.
Fixes#5383
A flake declares agent archetypes in its `hyperhiveArchetypes` output,
`<name> = { module; meta; }`. The hyperhive flake ships `reviewer`
(default permissions without `execution`).
swarm-builder answers `$SWARM.builder.archetypes` with the `meta` of
every archetype its own hyperhive source declares. swarm-controller asks
once the queue connects and every 10 minutes, checks each archetype as
`POST /api/agents` checks its settings, and stores the result in the
`agent-archetypes` KV bucket when it changed. `GET /api/archetypes`
serves it, and the create form's archetype select prefills permissions,
resource caps and inference from it.
The agent created is an ordinary agent with its own config repo; the
archetype's module is not applied yet.
Refs #5386
swarm-secret-publish and its path unit now run inside authelia's container,
beside the mint, and read each client secret at the path authelia writes it.
The host unit and host path unit, which read the secrets through the
container's root on the host, are gone, and so is
deploy.authelia.hostClientSecretDir, which only they read. The publisher keeps
its store identity, the secret-publisher leaf. The host hands the leaf and the
store's CA into the container as nspawn system credentials, and the unit loads
them; where this host mints the leaf, the container waits for swarm-bao-pki.
Authelia does not wait for the publisher, which retries every 30 seconds while
the store is down.
The container also runs the journal forwarder the other service containers
run, pushing to a receiver of its own on the swarm collector (serviceProducers
swarm-authelia) as a new machine OIDC client (swarm.authelia.otel.clientId)
that authelia registers for itself and the publisher carries to the store. The
fetch unit reads it back under the publisher's leaf, also handed to it as a
system credential, so the swarm-secret-publisher policy gains read on that one
leaf; its stanza keeps create and update, which the exact path would otherwise
take from the services/* prefix. Without the leaf the forwarder is off.
The container drops --link-journal=host, and the host collector stops reading
it. The services' "nothing at <path>" errors name the publisher in authelia's
container.
mara on #5170: "dont split caps and perms. its all perms. / dont include
the now empty tool groups / nothing set -> default / also add perms to
agent creation screen".
- hive-sh4re: one `Permission` enum (messaging, meta, inbox, execution,
web_tools, manage_root_agent) replaces `ToolGroup` and `Capability`;
`lifecycle`/`forge` are gone. `Permission::default_for` is the default
for an agent the swarm sets nothing for.
- The record is `{ "permissions": [...] }`. Missing key, unset field or
no successful read yet = default; `[]` = none; a failed read after a
good one keeps the last value.
- hive-agent reads its own key every 10s and builds each turn's --tools,
--allowedTools and MCP config (rewritten before every turn) from it;
the subagent daemon reads it on each start/continue. Agents get a
read-only grant on their own key.
- hive-c0re restarts an agent only when its manage_root_agent changes.
The import now merges meta/capabilities.json and meta/tool-groups.json
into one list per agent and deletes both; the "import pending, don't
read" decision is `ImportOutcome::read_allowed`, table-tested.
- swarm-controller: `GET /api/permissions` (names + default) and an
optional `permissions` on `POST /api/agents`, stored before the job is
queued. swarm-ui's create-agent form gets a permissions field.
- Deleted: HIVE_TOOL_GROUPS and its forward to the subagent daemon,
tool_groups.rs, ruth's tool-group seed, the permissions rebuild job
(PermPayload/WritePermFile/perm_change), the dashboard permissions tab
and its stale-entries panel.
Part of #5170.
An agent's capabilities move from the hive's meta repo
(`meta/capabilities.json`, injected as `HIVE_CAPABILITIES`) into the
`capabilities` field of its key in a new `agent-permissions` KV bucket,
which swarm-controller alone writes.
- swarm-queue-client: `agent_permissions` (bucket, key, `AgentPermissions`,
direct-get read, the import subject and its wire types).
- swarm-controller: creates the bucket at start; `GET|PUT|DELETE
/api/agents/{name}/permissions` (deny_unknown_fields and unknown
capability ⇒ 422, duplicate ⇒ 400, agent must be declared by a hive);
answers each hive's one-shot import, writing a key only where there is
none and only for agents that hive's wanted state declares.
- swarmctl: `agent permissions get|set|clear`.
- swarm-nats-auth: a hive reads every key of the bucket and sends its
import on its own subject; the controller subscribes to the imports and
may answer them (`responses`).
- hive-c0re: reads each of its agents' keys every 30s into
/var/lib/hyperhive/swarm-agent-permissions.json and restarts an agent,
gracefully, when its capabilities change. Unset is the compiled-in
default (ruth: manage_root_agent; everyone else: none). A failed read
keeps the stored value. On first boot it seeds that file from
meta/capabilities.json, sends the grants to the controller, and deletes
the file once the controller answers.
Deleted: the capabilities.json reader/writer, `HIVE_CAPABILITIES` in the
rendered meta flake, ruth's capability seed, the capabilities half of the
permissions rebuild job (`PermPayload::Capabilities`/`Combined`), the
dashboard's capabilities table and `capabilities_changed` event, and the
now-unused `Capability::description`.
Part of #5170.
A new swarm service, `swarm-claude-login`, in its own container on the host
that sets `deploy.claude-login.enable` (defaulted on by `allSwarmServices`
where the host has the minter's store leaf, like the review classifier).
- swarm-ui: an agent card showing "login needed" gets a "log in" button.
Its dialog starts the login, shows the URL claude printed, and takes the
pasted code.
- swarm-controller: `POST/DELETE /api/agents/{name}/claude-login` and
`POST /api/agents/{name}/claude-login/code`, behind the swarm UI's
authelia-gated `/api/`, relay the steps to the minter as requests on
`$SWARM.claude-login.request`.
- swarm-claude-login: runs `claude auth login` over piped stdio with HOME
and CLAUDE_CONFIG_DIR in a per-login scratch dir, scrapes the first
https URL, writes the code to stdin, and stores the `.credentials.json`
at `swarm/agents/<agent>/claude-credentials` with a KV v2 write, a new
version over any earlier one. The scratch dir goes on every exit path; a
login waits 15 minutes for its code.
- hive-agent: while parked in `wait_for_login`, reads that leaf every 30 s
under the agent's own cert and installs it only when its KV version is
newer than the one recorded in `<harness>/claude-credentials-version`.
An equal or older version is never put back, because claude rotates the
refresh token and the stored copy is stale after the first refresh.
- swarm-secret-client: `claude_login` paths and `SecretStore::read_versioned`.
- Credentials: a `claude-login` store leaf minted by glue-bao-tls, a
`swarm-claude-login` policy (create/update on
`swarm/agents/+/claude-credentials`, read on its own queue client
secret), an authelia machine client, and a `--claude-login-client`
grant in the queue's auth-callout responder. The unit runs as a static
system user.
The hive's own web-UI login is unchanged.
Refs #5268
A separate $SWARM.control.<agent> subject carries commands, apart from
the text subject. swarm-controller exposes POST /api/agents/{name}/cancel
as a NATS request; the harness answers whether a turn was actually
stopped, through the same cancel path as the dashboard button. The agent
credential gets subscribe on its control subject and allow_responses to
answer; only the controller may publish there.
Refs #3803
A new `swarm-builder` service, in its own container on the host that sets
`deploy.builder.enable` (defaulted on by `allSwarmServices` where the host
has the builder's store leaf, like the review classifier).
When an `agent-configs/<agent>` PR is opened, reopened or synchronized,
swarm-controller queues a `build_agent` jobq node. The node sets a
`swarm-builder` commit status to pending on the PR head, sends
`{agent, rev, pr}` to the builder over NATS request-reply
(`$SWARM.builder.build`), and writes success (store path), failure (last
line nix printed) or error (no builder / no reply) on the head. The node
fails when the build does, so the Jobs page matches the PR.
The builder builds nix/builder/check-flake's `check` with every input
overridden: hyperhive and nixpkgs to the host's own
`c0re.hyperhiveFlake`/`c0re.nixpkgsFlake` (what a hive there renders into
its meta flake), and the agent to the PR head on the forge. Evaluation is
pure and runs in the container; builds go to the host nix daemon as an
untrusted client. The private config repo is fetched with the token of a
new read-only `builder` forge account (in the `hives` team), which the
controller mints into `swarm/services/builder/forge/token`; git gets it
from a credential helper reading the child's environment, so it is in no
argument or URL.
Credentials: a `builder` store leaf minted by glue-bao-tls on the store
host, a `swarm-builder` reader policy (forge token + its own OIDC client
secret), an authelia machine client, and a `--builder-client` grant in the
queue's auth-callout responder. The controller gains a grant on the token
path and the secret publisher a deny on it.
This is the agent-config build check atlas scoped on #5081 (slice 3):
nothing is pushed to a cache, nothing is written to KV, and a red status
gates no deploy. The check composes agent-base + the config without the
values a hive adds (CA, docs, claude-code path, ports, input follows), so
it can disagree with a hive's own build.
The responder re-read its secret on any non-2xx introspection answer.
With the re-read backoff added in the previous commit, an IdP outage
(503s, 502s from nginx, 429s) made each re-read find the unchanged
secret, so the delay grew to minutes. A real 401 after the rotation
then landed inside that window, and the responder denied every new
NATS connect until it ran out.
`Answer::Refused` now carries the status. `OidcSecret::introspect`
re-reads only when `stored_secret::is_credential_refusal` says the
status is 400 or 401, the same predicate `ClientSecret::refused` uses on
the client path. Every other non-2xx is a denial with no re-read.
The client-path test for a 429 now runs the mint against a loopback
endpoint answering 429 and 503, instead of calling `refused()` directly.
swarm-queue-client's README no longer names `ClientSecret::Value`.
Part of #5017 (slice Q).
hive-c0re (`hive-<name>`), swarm-controller and swarm-review-classifier
read their queue client secret from the store once and held it as
`ClientSecret::Value` for the life of the process. After authelia
re-minted a secret and swarm-secret-publish stored the new one, every
later token mint presented the old value and was refused until the
daemon restarted.
`ClientSecret::Stored` replaces `Value`: a `StoredSecret` holds the
secret in memory with a caller-supplied read callback, so the crate
takes no store dependency. When the token endpoint answers 400 or 401,
`mint_token` (and its blocking sibling, used by the controller's OTLP
exporters) re-reads it and, if the store holds a different secret,
mints once more with it. One re-read runs at a time and is spawned so
it outlives a caller that gives up. A re-read that fails or finds the
refused value delays the next one, 5 s doubling to 5 min, so a secret
that stays wrong costs one store read per delay. Logs name the client
id and never the secret.
swarm-nats-auth's `oidc_secret` was the template; its re-read machinery
now sits on `StoredSecret`, so the tree has one copy. The responder
gains the same re-read backoff.
The controller keeps its `IDENTITY` OnceLock: it now holds a
`QueueConfig` whose secret is re-read, and the lazily built vcs-metrics
exporter reads the identity from it.
agent_renewal.rs's module doc said a re-minted agent queue secret only
reaches the agent at its next start; hive-agent reads the store on every
connection attempt after its first, so the doc now says that.
Part of #5017 (slice Q).
`CredentialSource`, `Store` and `verify` move from swarm-nats-auth's
`agent_token` module into a small library crate, so a second verifier
can make the same check: the secret in a `swarm-agent.<agent>.<secret>`
token equals the one stored at `swarm/agents/<agent>/queue`, read under
the verifier's own certificate.
`verify` now takes its timeout as an argument; swarm-nats-auth passes
`INTROSPECTION_TIMEOUT` as before. No behaviour change. The store-denial
tests moved with the code; swarm-nats-auth keeps the ones that go
through its policy and its bound.
Part of #4459.
The grant test covered the bare consumer-create subject only, so a grant
of `$JS.API.CONSUMER.CREATE.KV_agent-limits.>` would have passed it.
It now also checks a named consumer, bare and with a key filter.
Raised by the #4268 verify pass.
Every 30 seconds `workers::swarm_limits` reads the `agent-limits` key of
each agent this hive has and stores the result in
swarm-resource-limits.json, where the drop-in, the heap ceiling and the
dashboard read it. A cap therefore follows its agent to whichever hive
runs it.
Each value goes through the hive's own validate_cpu_quota /
validate_memory_max before it is stored, since hive-priv renders it
verbatim into the container's drop-in, and one bad value refuses the whole
set (argus on #4268, carried over from the wanted-state consumer this
replaces). A failed read, or a bucket that does not answer, leaves the
stored caps as they are; three failures in a row raise a banner.
swarm-nats-auth grants every hive `$JS.API.DIRECT.GET.KV_agent-limits.>`:
read-only, and every agent's key, not just the hive's own. Scoping it to
the hive's agents would need a roster in the auth responder, which
policy.rs argues against. mara chose this on #4268 (c95293); narrowing it
is a possible low-priority follow-up, not planned. A test pins that the
grant reads every key, writes none, and reaches no other bucket.
Contributes to #3865.
swarm-queue-client gains review_classifier: the try-out subject, the
request and reply that travel on it, and the deadlines both ends share.
The controller's reader grant may publish the subject; a client named
with --review-classifier-client may subscribe to it and its inbox, and
answers under NATS allow_responses (resp in the user JWT) within the
run deadline. Every other client id is still denied.
swarm-controller now creates `term-main-<agent>` for every live agent, in
the same one-minute pass and with the same config style as
`term-sub-<agent>` (#4897): subjects `$SWARM.term.<agent>` only, max_age
24h, max_bytes 64 MiB, discard Old, max_message_size from the queue's
max_payload. An existing stream is opened as it is. The agent keeps its
plain publish on `$SWARM.term.<agent>` and gets no JetStream subject.
`GET /api/agents/{name}/term/stream` creates an ephemeral push consumer on
that stream starting 100 rows back, so a swarm-ui terminal opens with
recent history and then follows new rows, in order, from one
subscription. Each SSE event carries the row's stream sequence as its id,
and a reconnect's Last-Event-ID resumes after it. Without the stream the
route falls back to the live core subscription it used before.
The stream is `term-main-<agent>`, not `term-<agent>`: an agent may be
named `sub-x`, whose `term-<agent>` would be agent `x`'s subagent stream.
Its one literal subject cannot match `$SWARM.term.<agent>.sub.>`, which
needs more tokens, so the two streams never overlap.
The reader gains `$JS.API.CONSUMER.CREATE.*.*.$SWARM.term.*`, the
extended create form whose last tokens are the consumer's filter. Checked
against nats-server 2.14.4, the trailing `*` also admits a literal `*` or
`>` filter, so a consumer under it reaches any terminal row, main or
subagent; those are rows the reader already subscribes to and reads with
`STREAM.MSG.GET.*`.
Closes#4826
swarm-controller keeps each agent's inference settings in a new
`agent-inference` KV bucket, one key per agent, and is its only writer
(`GET`/`PUT`/`DELETE /api/agents/{name}/inference`; `swarmctl agent
inference get|set|clear`). Every field is optional. Each agent reads its
own key every 10 s through a direct-get grant on that key alone, and a
set `model` or `effort` replaces the Bus value, so the next turn runs on
it and the badge follows.
The swarm value is applied without persisting it: `Bus::set_model` and
`set_effort` are split into a persisting setter and a non-persisting
`apply_*`, and the poller uses the latter. Otherwise the swarm value
would overwrite `hyperhive-model`/`hyperhive-effort`, and clearing it, or
a restart, would no longer find the agent's own value. An unset field
changes nothing; a field cleared after being applied goes back to the
local value (`harness_state::local_model`/`local_effort`, the same
resolution `Bus::new` uses), read again at the clear.
`subagent_models` is validated, stored and shown by `get`; no agent
applies it yet.
Refs #5034
- swarm-controller webhook: regenerate the secret only when the file is
absent. Any other read error is returned with the path; the caller
already logs "webhook secret unavailable" and the endpoint answers 503.
Overwriting meant every registered hook kept the old secret and every
delivery failed HMAC, with nothing re-registering them.
- swarm-queue-client wanted: open_read_only/watch return Result; only
STREAM_NOT_FOUND on the bucket maps to None. hive-c0re's wanted pull
fails the DAG node with the cause instead of logging "nothing has been
declared"; the watch warns with the error chain.
- swarm-nats-auth: warn when the OIDC secret re-read task dies.
- swarm-controller forge: listing a repo's config PRs failing is warn,
not debug.
Closes#5056
The reader's subscribe allow-list spelled `$SWARM.term.*`,
`$SWARM.term.*.sub.*` and `$SWARM.agent-state.*` as literals, copies of
private prefixes in hive-agent and swarm-controller. A rename on either side
left the policy behind and the stream stopped with only a connection-level
server error.
swarm-queue-client now owns `term_subject`, `agent_state_subject` and their
`*_wildcard` forms, and `subagent_term::subject_wildcard`; hive-agent's
publishers, swarm-controller's relays and the policy all build from them.
Subjects are byte-identical.
A hive's queue credential and the swarm controller's could subscribe to
every subject in the swarm account, so a hive could read every other
hive's deploy requests and every agent's terminal, state, input and
request replies.
Their grants now carry a subscribe allow-list:
- a hive `hive-<h>`: `$SWARM.knowledge`, `$SWARM.deploy.<h>` and its
reply inbox `_INBOX.client.hive-<h>.>`
- the reader (swarm-controller): `$SWARM.term.*`, `$SWARM.term.*.sub.*`,
`$SWARM.agent-state.*` and its reply inbox
`_INBOX.client.<client-id>.>`
`swarm_queue_client::connect`, which hive-c0re and swarm-controller both
use, now connects with `_INBOX.client.<client-id>` as its inbox prefix,
the same shape agents got for their own credential. In async-nats 0.50
every request reply and every ordered consumer (the hive's wanted-state
watch, the controller's KV key listings) delivers under that prefix, so
the one inbox grant covers them all.
A hive or controller still on a build without the prefix has every
JetStream reply refused once the responder runs this policy, so the
clients need to be rebuilt before, or with, swarm-nats-auth.
Closes#4945
When the store re-read failed or timed out, introspect fell through to the
unchanged-secret branch and logged "the stored OIDC client secret is the one
the IdP refused" after the re-read's own failure line. The re-read now says
whether it read the store, and a failed one denies without the second line.
The verdict is unchanged.
The re-read of the responder's own OIDC client secret had no timeout:
neither swarm-secret-client nor vaultrs nor reqwest sets one by default,
and while a re-read runs every later refusal joins it rather than
starting another. A store connection whose peer stays alive but silent
therefore held off rotation pickup indefinitely. The spawned re-read
now gives up after 10 s, logs that it did (without the secret), keeps
the secret in use, and frees the slot for the next refusal's re-read.
The link from a non-2xx introspection (identify_caller's
`Answer::Refused`) to that re-read had no test: every existing test
handed `OidcSecret::introspect` an `Answer` directly. A new test runs
the real `identify_caller` against a loopback IdP that answers 401 to
the old secret and admits the new one, so mapping a non-2xx to
anything but `Refused` turns it red. axum is a dev-dependency for that
stub, as in swarm-controller's forge tests.
The responder reads its OIDC client secret at
`swarm/services/<client-id>/oidc/client` with the store login it already
opens for the callout keys, and holds it in memory. `--client-secret-file`
is removed.
When introspection answers non-2xx, the responder re-reads the secret from
the store once and, if it changed, retries the introspection once. Both fit
inside the request's 1.5s introspection budget, under the server's 2s
callout timeout: the budget is now one deadline for the whole request. A
re-read that does not finish by then keeps running in the background, the
request is denied, and the next request introspects with what it read. One
re-read runs at a time; a refusal while one runs waits on it.
In swarm-nats.nix the `swarm-bao-nats-auth-oidc` fetch unit, the flag and
the `oidc-client.secret` LoadCredential entry are gone, along with the
unit's policy ordering in glue-bao-readers-policy-order.nix. A tmpfiles `r`
rule removes the copy a container still holds at
/var/lib/nixos-containers/swarm-nats/var/lib/swarm-nats-auth/oidc-client.secret;
nothing reads it.
No policy change: the `swarm-nats-auth` role already reads this path
(`natsAuthReaders` in swarm-bao.nix), which is the role the removed fetch
unit logged in as.
Deploy order: none, and no operator step. The store's grant already exists,
so the queue host can be deployed before, after or without the store's host
and authelia's. Rolling back re-creates the fetch unit and the file; the
store leaf is untouched.
The responder loads the issuer and callout-user seeds from the secret
store under its own role, beside the xkey, and no longer takes
--issuer-seed-file / --user-seed-file.
`ensure-xkey` becomes `ensure-callout-keys`: it ensures the xkey as before,
and for each nkey creates the seed create-only when the store holds none,
from --import-{issuer,user}-seed-file when that file is present, so a key
kept on disk keeps its public half. Once a read of the store returns a seed,
the import file is removed; when that seed differs from the file the
store's is kept and a warning names the file. The public halves are
published create-only at swarm/public/swarm-nats/callout-{issuer,user}.
The KeyStore trait and its test double move to `keystore` so both key
modules share them.
Every nats-server this module starts renders `auth_callout.xkey` or does
not start, and the responder serves only that server, in the same
container and generation. A request without `Nats-Server-Xkey` therefore
comes from no supported server, so the plaintext framing is gone: such a
request is logged at warn and gets no reply, and the server that sent it
times out the connect as a denial. A signed denial would have had to go
out unsealed, which is the path this removes.
Also scopes two claims to what is true: the xkey's public half is read by
the swarm-nats-auth policy only, and the metadata-delete gap in the
replace-key procedure is about the two xkey paths, not all of
`secret/metadata/`.
Writing bao's `default` policy needed a new stanza in the granter's own
policy, which only the bootstrap token can upload, so deploying it took
a manual root-token step. Nothing needed it: the one reader of the
callout xkey's public half is `swarm-bao-nats-callout-xkey`, which logs
in as the `swarm-nats-auth` role, and that role's policy already names
the path. `swarm-bao-nats-auth-policy` writes that policy as the granter
on every deploy, under the granter's existing `sys/policies/acl/swarm-*`
grant.
Removed: the `swarm-bao-public-read-policy` unit, its marker block, the
granter's `sys/policies/acl/default` stanza, the granter-role restart of
that unit, the bao.md section and the module-eval cases pinning them.
The responder reads the callout xkey before it connects, so a responder
with no secret store no longer starts, and its "deny every agent token,
serve OIDC" mode could not run. That mode is gone: `agent_token::Store`
is built from the settings the xkey load already read, and `authorize`
takes a source rather than an `Option` of one.
On the nix side, `deploy.nats.authBaoClientCertFile` and
`authBaoClientKeyFile` are now asserted at eval when the queue is
enabled. The units' no-identity branches, the copy unit's empty-file
fallback and its "denies agent tokens" message are removed; an absent
leaf file fails `swarm-nats-auth-secrets` like an absent seed does.
When nats-server has an auth_callout xkey it seals each request to that
key and names its own public xkey in the Nats-Server-Xkey header. The
responder now loads the callout xkey seed from the secret store before
it connects, opens a sealed request with it, and seals its reply (grant
or deny) back to the header's key. A request with no header is answered
in plaintext, as before.
A new `ensure-xkey` verb creates the pair when the store holds no seed:
the seed at swarm/services/swarm-nats/callout/xkey-seed, the public half
at swarm/public/swarm-nats/callout-xkey. Each write is a create-only
(cas=0) write, and a run whose create fails reads the stored value back,
so two concurrent runs end up with one key. The responder itself only
reads the seed and refuses to start without it.
swarm-secret-client gains SecretStore::create, the cas=0 write.
Docs (review on #4948): the harness's restart settings are not useful to
a reader, so the `Restart`/`RestartSec` bullet in docs/turn-loop and the
"30 seconds later, no start limit" wording in docs/swarm are removed.
What stays is the behaviour: the harness exits and comes back until the
queue accepts it.
Grant: with the harness's shared-client fallback gone, nothing presents
the hive's shared agent client (`hive-<hive>-agent`) to the queue;
swarm-logs uses that OIDC client over HTTP only. swarm-nats-auth now
refuses it outright instead of granting the configured
`$SWARM.term.{hive}.>` / `$SWARM.agent-state.{hive}.>` publish and its
`_INBOX.client.<id>.>` subscribe, so an agent holding it can no longer
publish as a sibling. The agent arm still runs before the hive arm, so
that id never falls through to its hive's grant.
Removed with it: the `--agent-publish-subject` flag and its two uses in
swarm-nats.nix, `Policy`'s agent subject list, and
swarm-queue-client's `client_inbox_prefix`. The module-eval now asserts
the flag is absent; the policy tests assert the refusal and that the
hive itself is still admitted.
GET /api/agents/{name}/subagents reads each subagent's newest row with
get_last_raw_message_by_subject, which publishes
$JS.API.STREAM.MSG.GET.term-sub-<agent>. The controller's NATS grant had
no STREAM.MSG.GET, so every lookup was refused and waited the 10s request
timeout, one subagent after another; six subagents passed nginx's 60s.
Grant `$JS.API.STREAM.MSG.GET.*`. A wildcard matches a whole subject
token only, so a `term-sub-*` spelling would name one stream literally;
`*` is the narrowest grant that covers the per-agent streams, the same
width as the controller's STREAM.INFO.* grant.
The lookups now run concurrently (16 at a time) with a 2s timeout each.
A lookup that fails or times out lists its subagent with a null
last_row_unix, and one warn per request names how many failed and the
first error.
An agent's queue credential could subscribe to every subject in the swarm
account, so any agent could read every other agent's terminal, turn state,
operator input and request replies. Agents are the least trusted clients
on the queue (mara's ruling on #4901), so their grants now carry a
subscribe allow-list:
- an agent's own credential (the harness and its subagent daemon):
`_INBOX.agent.<agent>.>` and `$SWARM.input.<agent>`
- the hive's shared agent client: `_INBOX.client.<client-id>.>`
Hive and swarm-controller grants are unchanged (no `sub` claim, so still
every subject).
The clients connect with the matching inbox prefix, so the replies they
rely on still arrive: the icon write's JetStream ack and the schedule
DIRECT.GET. `connect_with_token` takes the agent name and sets the prefix
itself; the hive-shared fallback uses the new
`connect_with_inbox_prefix`.
The harness no longer subscribes to its input subject when it fell back
to the hive's shared client: that identity names no agent, so it cannot
be granted one agent's input without all of them. It logs that and
skips, the same as the schedule reader on that path.
Refs #4901
swarm-controller keeps every scheduled prompt (id, name, agents, body,
anchor, interval, paused) as one document under one key of a new
`schedule-definitions` KV bucket, so each change is a single
compare-and-set and a prompt exists for all of its agents or none. It
writes each agent's key of a new `agent-schedules` bucket as the prompts
naming that agent, without names or other agents, after every change and
on a one-minute reconcile pass. It is the only writer of both, behind
GET/POST /api/schedules and PATCH/DELETE /api/schedules/{id}.
`swarmctl schedule add|list|edit|pause|resume|rm` drives those routes over
the controller socket.
An agent's own queue credential is granted one more subject, the direct
get of its own key in `agent-schedules`. A KV watch is not granted: its
consumer create subject carries no key, so it would cover every agent's.
hive-agent's new schedule ticker reads that key every 10 seconds and fires
due slots as the same `schedule:<id>` todo hive-c0re's worker pushes. Fire
times are anchor + k*interval; the last fired slot per schedule id is kept
in the state db, so a body edit or list refresh never moves or refires a
slot, downtime fires the latest missed slot once, a restart refires
nothing, a one-shot fires once, and paused slots are skipped.
Refs #4914
The agent view in the swarm UI gets a message box. The text reaches the
agent's harness straight from swarm-controller over the swarm queue and
becomes an inbox turn from the operator's authelia username. hive-c0re is
not in the path.
- gateway: `/api/` forwards authelia's `Remote-User`, replacing any the
client sent.
- swarm-controller: `POST /api/agents/{name}/input` takes that header as
the sender (401 without it) and sends `{from, body}` as a NATS request
on `$SWARM.input.<agent>`. No subscriber is a 503.
- swarm-nats-auth: the controller is granted publish on `$SWARM.input.*`;
no hive or agent grant names the family, so the harness can trust
`from`.
- hive-agent: a task subscribes to the agent's own input subject and
pushes each message into the serve loop's existing reminder channel,
renamed `local_msg_*` since it now carries both.
The harness does not reply to the request: no agent grant covers a reply
subject. The controller therefore answers 202 once the request times out
(2s) and 503 on the server's no-responders status.
Closes#4828
