Watch
0
0
Fork
You've already forked hyperhive
0
Commit graph

5,377 commits

Author SHA1 Message Date
atlas
22ce8b815a hive-subagent-mcp: a refused continue leaves the session's records alone
`continue_` cleared the stop reason, reset the turn budget and remembered
the caller's `dir` before the session lookup ran, and nothing restored
them when the lookup refused. With the strict lookup a refusal is an
expected outcome (the caller omitted `dir`), so every such retry erased
why the session stopped from `status`, gave a capped goal session its
budget back, and made the wrong `dir` the remembered one for the next
bare `continue`.

The lookup now runs first, against the dir peeked rather than committed;
only a session that is found gets `State::begin_continue`, which does
all three. Same for the ACP path's recorded-session check.
2026-10-07 13:18:07 +02:00
atlas
0e015197b6 hive-subagent-mcp: refuse a continue whose session is not in the dir searched
resume_target no longer falls back to the bare name when the project dir
holds no session titled it: continue (and a goal run's next turn) refuses
before anything spawns, naming the directory searched. A store that cannot
be resolved refuses too, with the io error. The fallback notice is gone.
2026-10-07 12:28:25 +02:00
atlas
d03f10822c hive-subagent-mcp: say so when a resume falls back to the bare name
resume_target logs the fallback and returns a notice naming the project
dir searched; continue appends it to its reply and to a failed resume's
error, so a caller hitting claude's 'matches 2 sessions' knows to pass dir.
The test's final assertion now pins the whole attach list so the continue
site is covered on its own, and the fallback branch has a test.
2026-10-07 12:28:25 +02:00
atlas
8f2069486d hive-subagent-mcp: resume a subagent by session id, not by name
`continue`, and every later turn of a goal run, passed `--resume <name>`.
claude 2.1.287 resolves a title across the project dirs of every git
worktree of the repo and refuses when two sessions match, while `start`
archives a prior session only in its own cwd's project dir. A name first
started under one worktree and then started again under a sibling stayed
live in both, and every resume of it failed with `--resume "<name>"
matches 2 sessions`. That is the #5000 incident: build-4921a/b were first
run under /home/atlas/wt/4901-build, then under their own worktrees.

Both resume sites now look up the live session titled `name` in the
config's own project dir (the same `SessionStore::find_by_title` lookup
`start` archives with) and pass its id. The id comes off disk at each
resume, so it survives a daemon restart. When that dir holds no session
under the name, the name is passed as before and claude's answer stands.

The test drives a fake claude that keeps sessions on disk and resolves
titles across project dirs the way claude does. Its script is written by
a child process, so no write fd on it exists in the test binary for a
concurrent fork to inherit (the ETXTBSY of #4898).
2026-10-07 12:28:25 +02:00
atlas
40687ab4a5 swarm-nats-auth: a failed re-read no longer logs that the stored secret was refused
When the store re-read failed or timed out, introspect fell through to the
unchanged-secret branch and logged "the stored OIDC client secret is the one
the IdP refused" after the re-read's own failure line. The re-read now says
whether it read the store, and a failed one denies without the second line.
The verdict is unchanged.
2026-10-06 16:01:53 +02:00
atlas
f83030281e nats-callout-removed: remove the container's OIDC secret copy here, ungated
The tmpfiles `r` rule for the stale `oidc-client.secret` in the
swarm-nats container sat in swarm-nats.nix under `deploy.nats.enable`,
so a host that turned the queue off kept the copy. It now sits in the
ungated removal shim next to the seed copies' rules, which is where the
S2 plan put it and the one file to delete when the deprecation window
closes. The module-eval case asserts the rule on a host with the queue
off as well.
2026-10-06 15:25:29 +02:00
atlas
32e28e6096 swarm-nats-auth: bound the OIDC secret re-read, and test the 401 that starts it
The re-read of the responder's own OIDC client secret had no timeout:
neither swarm-secret-client nor vaultrs nor reqwest sets one by default,
and while a re-read runs every later refusal joins it rather than
starting another. A store connection whose peer stays alive but silent
therefore held off rotation pickup indefinitely. The spawned re-read
now gives up after 10 s, logs that it did (without the secret), keeps
the secret in use, and frees the slot for the next refusal's re-read.

The link from a non-2xx introspection (identify_caller's
`Answer::Refused`) to that re-read had no test: every existing test
handed `OidcSecret::introspect` an `Answer` directly. A new test runs
the real `identify_caller` against a loopback IdP that answers 401 to
the old secret and admits the new one, so mapping a non-2xx to
anything but `Refused` turns it red. axum is a dev-dependency for that
stub, as in swarm-controller's forge tests.
2026-10-06 15:25:29 +02:00
atlas
6646329da5 swarm-nats-auth: read its own OIDC client secret from the store
The responder reads its OIDC client secret at
`swarm/services/<client-id>/oidc/client` with the store login it already
opens for the callout keys, and holds it in memory. `--client-secret-file`
is removed.

When introspection answers non-2xx, the responder re-reads the secret from
the store once and, if it changed, retries the introspection once. Both fit
inside the request's 1.5s introspection budget, under the server's 2s
callout timeout: the budget is now one deadline for the whole request. A
re-read that does not finish by then keeps running in the background, the
request is denied, and the next request introspects with what it read. One
re-read runs at a time; a refusal while one runs waits on it.

In swarm-nats.nix the `swarm-bao-nats-auth-oidc` fetch unit, the flag and
the `oidc-client.secret` LoadCredential entry are gone, along with the
unit's policy ordering in glue-bao-readers-policy-order.nix. A tmpfiles `r`
rule removes the copy a container still holds at
/var/lib/nixos-containers/swarm-nats/var/lib/swarm-nats-auth/oidc-client.secret;
nothing reads it.

No policy change: the `swarm-nats-auth` role already reads this path
(`natsAuthReaders` in swarm-bao.nix), which is the role the removed fetch
unit logged in as.

Deploy order: none, and no operator step. The store's grant already exists,
so the queue host can be deployed before, after or without the store's host
and authelia's. Rolling back re-creates the fetch unit and the file; the
store leaf is untouched.
2026-10-06 14:54:19 +02:00
atlas
a66670be87 hive-tls: the root fetch presents the hive identity, as on main
On a hive fronting no swarm name, swarm-services-cert's token-less
GET pki/ca/pem presents baoHiveIdentity{Cert,Key}File again. The fetch
never logs in and grants nothing; the client certificate only completes
the store listener's mTLS. A hive host without the store therefore needs
no copy of the services-issuer pair.

The issue path, which logs in to pki/issue/swarm-services, still presents
only deploy.bao.servicesIssuerClient{Cert,Key}File.

module-eval-hive-tls: the root-fetch cases pin the hive identity and the
absence of the services-issuer pair (a fixture holding both, and one
holding only the hive identity); the issue-path case pins the pair on a
hive host and on a host without a hive; a gateway without the pair
evaluates and its issue path names both options.

Root-fetch identity: option (a), mara on PR #5002 (comment 92591).

Refs #4921
2026-10-06 14:54:07 +02:00
atlas
6a9c819136 hive-tls: a gateway host without the services-issuer pair evaluates and goes red at runtime
Drop the eval assertion b78190e1 added. It refused every self-signed
gateway host that does not run the store, which is the default hive
shape, and contradicts docs/swarm/secrets.md's supported no-store
deployment. The approved #4921 plan has such a host evaluate and fail
at runtime; `requireServicesIssuer` already exits 1 naming both
options, on the root-fetch branch and before issuing.

The module-eval case that pinned `hiveWithController { }` as refused
now pins it, and a hive-less gateway with no pair, as evaluating with
the unit's refusal in the script. The fixture pairs the assertion
forced into forge-placement, nats-tls and swarm-otel-identity are
reverted.

Refs #4921
2026-10-06 14:54:07 +02:00
atlas
5761398d60 hive-tls: the gateway leaf uses the services-issuer identity, not a hive's
swarm-services-cert issues the swarm-services leaf a self-signed gateway
serves, and on a host fronting no swarm name fetches the store's pki root.
It is gated on the gateway, not on a hive, yet its leaf pair was named
under deploy.hive-controller.tls and defaulted from the hive reader's cert
directory, and the root fetch presented the hive's own identity. A gateway
host with no hive had no such identity, so that branch was red by
construction.

- The pair moves to deploy.bao.servicesIssuerClient{Cert,Key}File, beside
  the store's other host-unit readers, and glue-bao-tls.nix defaults it on
  the store's host from the leaf it already mints. The old
  deploy.hive-controller.tls.baoClient{Cert,Key}File paths are
  mkRenamedOptionModule aliases. glue-services-issuer-bao-identity.nix,
  which derived the default from deploy.bao.clientCertFile, is gone.
- The no-swarm-name root fetch presents the services-issuer pair. The hive
  identity is now only hive-tls-resign's.
- A self-signed gateway with no pair fails evaluation, naming both options.
  A pair named but unreadable still fails the unit, naming the paths.

Refs #4921
2026-10-06 14:54:07 +02:00
atlas
c3ef18ad09 module-eval: assert every removed callout option fails in both namespaces
The removal case set 6 of the 10 option x namespace pairs. Build the pairs
from one option table over both namespaces and check there are 10.
2026-10-06 14:53:33 +02:00
atlas
c10cc0e018 swarm-nats: take the callout issuer and user keys from the store
`swarm-bao-nats-callout-xkey` runs `ensure-callout-keys`, importing
/var/lib/swarm-nats-callout/{issuer,callout-user}.seed when present so the
keys keep their public halves, then fetches all three public halves from
the store. The verb removes each seed file once a read of the store returns
a seed; when the store's seed differs, it is kept and a warning names the
file. No tmpfiles rule touches those files, since one could run before the
import with the store sealed and leave the next run to mint a new key.

The server renders the issuer and user public halves from the fetched files,
checked for shape as the xkey's is, and does not start without them. The
responder reads both seeds from the store, so the copies into the container
and their LoadCredential entries are gone, and a tmpfiles rule removes the
copies a container still holds. `swarm-nats-callout-keys` is removed.

On the store's host the fetch unit is also ordered after
`swarm-bao-secret-publisher-policy`, so the publisher's denial of the seed
paths is written before the unit creates or keeps a seed there.

`deploy.nats.autoGenerateCallout`, `calloutUserPublicKey`,
`calloutIssuerPublicKey`, `calloutUserSeedFile` and `calloutIssuerSeedFile`
are removed with `mkRemovedOptionModule`, under `swarm.nats` too, each
message saying the keys live in the store.
2026-10-06 14:53:33 +02:00
atlas
66ac034949 swarm-bao: grant the queue responder the callout issuer and user keys' leaves
The `swarm-nats-auth` policy grants create and read, never update, on the
seed and public half of the callout issuer and user keys, beside the xkey's
two leaves: `ensure-callout-keys` creates them under that role, and the
responder reads both seeds as it starts.

The `swarm-secret-publisher` policy, which may write the rest of
`swarm/services/`, is denied both seed paths exactly, as it is the xkey's.
A planted issuer seed would be the authority to admit anyone to the queue.

Both policies are written by the granter on every deploy; nothing about the
granter's own policy changes.
2026-10-06 14:53:33 +02:00
atlas
8a4bb97377 swarm-nats-auth: read the callout issuer and user seeds from the store
The responder loads the issuer and callout-user seeds from the secret
store under its own role, beside the xkey, and no longer takes
--issuer-seed-file / --user-seed-file.

`ensure-xkey` becomes `ensure-callout-keys`: it ensures the xkey as before,
and for each nkey creates the seed create-only when the store holds none,
from --import-{issuer,user}-seed-file when that file is present, so a key
kept on disk keeps its public half. Once a read of the store returns a seed,
the import file is removed; when that seed differs from the file the
store's is kept and a warning names the file. The public halves are
published create-only at swarm/public/swarm-nats/callout-{issuer,user}.

The KeyStore trait and its test double move to `keystore` so both key
modules share them.
2026-10-06 14:53:33 +02:00
atlas
1b18f6dcbd swarm-nats, matrix: the queue's and matrix-ctl's store identity paths are their own, not the hive's
glue-nats-bao-identity.nix and glue-matrix-ctl-bao-identity.nix defaulted
deploy.nats.baoClient{Cert,Key}File and deploy.matrix.ctlBaoClient{Cert,Key}File
to siblings of deploy.bao.clientCertFile, the hive's own store reader. That
pointed a store host whose reader was moved at files nobody mints, and on a
queue or homeserver host without the store that held a placed hive reader
invented a nats.pem / matrix-ctl.pem path beside it.

Both glues are now gated on the store running on this host
(deploy.bao.enable, the condition glue-bao-tls.nix mints under) and name the
directory the minting unit writes to, through nix/host-modules/lib/bao-pki-dir.nix.

Host shapes:
- Store host, default reader: unchanged. The evaluated paths stay
  /var/lib/swarm-bao-pki/nats{,-key}.pem and
  /var/lib/swarm-bao-pki/matrix-ctl{,-key}.pem, and the rendered units are
  the same.
- Store host whose hive reader was moved: now gets the leaf minted there.
- Queue or homeserver host without the store, holding a hive reader and not
  naming these options: no default any more. It still evaluates (neither
  option has an eval assertion). swarm-bao-nats-tls exits 1 naming
  deploy.nats.baoClient{Cert,Key}File when the queue's certificate is missing
  or due for renewal; swarm-bao-matrix-oidc exits 1 naming
  ctlBaoClient{Cert,Key}File, and the swarm appservice render/publish units
  and the PKI bind mount no longer render. If that host had placed nats.pem
  or matrix-ctl.pem beside its hive reader, it worked before and now needs
  both options named.
- Hosts without the store and without a hive reader, and hosts naming the
  options: unchanged.

Both refusals and option descriptions now say the default comes from
deploy.bao.enable.

Refs #4921
2026-10-06 12:04:48 +02:00
atlas
3c79c5eb5d swarm-nats: the auth responder's store identity path is its own, not the hive's
glue-nats-auth-bao-identity.nix defaulted deploy.nats.authBaoClient{Cert,Key}File
to a sibling of deploy.bao.clientCertFile, which is the hive's own store
reader. That tied a swarm service to a hive identity, pointed a store host
whose reader was moved at a file nobody mints, and on a queue host without
the store that held only a placed hive reader invented a nats-auth.pem path
that passed the eval assertion and failed at runtime.

The glue is now gated on the store running on this host (deploy.bao.enable,
the condition glue-bao-tls.nix mints under) and names the directory the
minting unit writes to, shared through nix/host-modules/lib/bao-pki-dir.nix.
A queue host without the store gets no default and the eval assertion
refuses it until the responder's own leaf is named; the assertion message
now names deploy.bao.enable as where the default comes from.

On a default store host the evaluated paths are unchanged:
/var/lib/swarm-bao-pki/nats-auth{,-key}.pem.

Refs #4921
2026-10-06 10:04:30 +02:00
flake-bot
95b28268ba nix flake update 2026-10-06 09:36:21 +02:00
atlas
9184df555b swarm-nats-auth: module doc says an unsealed request gets no reply 2026-10-05 23:25:36 +02:00
atlas
923554cd62 swarm-nats-auth: refuse an auth request the server did not seal
Every nats-server this module starts renders `auth_callout.xkey` or does
not start, and the responder serves only that server, in the same
container and generation. A request without `Nats-Server-Xkey` therefore
comes from no supported server, so the plaintext framing is gone: such a
request is logged at warn and gets no reply, and the server that sent it
times out the connect as a denial. A signed denial would have had to go
out unsealed, which is the path this removes.

Also scopes two claims to what is true: the xkey's public half is read by
the swarm-nats-auth policy only, and the metadata-delete gap in the
replace-key procedure is about the two xkey paths, not all of
`secret/metadata/`.
2026-10-05 23:25:36 +02:00
atlas
1ac34a4ad6 docs: callout xkey needs the store at every responder start; split-host order; replacing it
secrets.md now says that the responder reads the seed at every start, so
a store outage plus a responder restart refuses every client; that on a
split deployment the store's host deploys first; and that replacing the
key takes `bao kv metadata delete` on both paths, since a soft delete
keeps the metadata and the create is then refused. It also drops the
claim that every token reads `swarm/public/`.
2026-10-05 23:25:36 +02:00
atlas
6c4a459773 swarm-nats: test that the server's rendered config carries the callout xkey
`checks.script-test-nats-render-config` runs the queue server's
`ExecStartPre` render against a scratch runtime directory: a fetched key
lands in `auth_callout.xkey` and the wrapper includes the fragment, and
a missing or malformed key file fails the start and writes no config.

module-eval only sees the render as a store path, so dropping `xkey`
from the template, or its substitution, passed every check while the
server would have run unsealed.
2026-10-05 23:25:36 +02:00
atlas
531d44d34b swarm-bao: deny the secret publisher the callout xkey's seed
`swarm-secret-publisher` holds create and update on
`secret/data/swarm/services/*`, which covers the callout xkey's seed at
`swarm/services/swarm-nats/callout/xkey-seed`. It never writes that
path, but a write there would replace the key the queue's server seals
every callout request to. An exact-path `deny` outranks the prefix.

A create-only grant would not do: KV v2 honours create versus update, so
it would stop an overwrite, but it would still let the publisher plant a
seed of its choosing whenever the path is empty, on a first boot or
after a key replacement. The publisher needs no access to the path at
all.
2026-10-05 23:25:36 +02:00
atlas
06784e55fa swarm-bao: drop the default-policy grant; swarm/public/ is read by name
Writing bao's `default` policy needed a new stanza in the granter's own
policy, which only the bootstrap token can upload, so deploying it took
a manual root-token step. Nothing needed it: the one reader of the
callout xkey's public half is `swarm-bao-nats-callout-xkey`, which logs
in as the `swarm-nats-auth` role, and that role's policy already names
the path. `swarm-bao-nats-auth-policy` writes that policy as the granter
on every deploy, under the granter's existing `sys/policies/acl/swarm-*`
grant.

Removed: the `swarm-bao-public-read-policy` unit, its marker block, the
granter's `sys/policies/acl/default` stanza, the granter-role restart of
that unit, the bao.md section and the module-eval cases pinning them.
2026-10-05 23:25:36 +02:00
atlas
52c931c3a7 swarm-nats: require the responder's store identity, and drop its no-store mode
The responder reads the callout xkey before it connects, so a responder
with no secret store no longer starts, and its "deny every agent token,
serve OIDC" mode could not run. That mode is gone: `agent_token::Store`
is built from the settings the xkey load already read, and `authorize`
takes a source rather than an `Option` of one.

On the nix side, `deploy.nats.authBaoClientCertFile` and
`authBaoClientKeyFile` are now asserted at eval when the queue is
enabled. The units' no-identity branches, the copy unit's empty-file
fallback and its "denies agent tokens" message are removed; an absent
leaf file fails `swarm-nats-auth-secrets` like an absent seed does.
2026-10-05 23:25:36 +02:00
atlas
9a241a1df1 swarm-nats: fetch the callout xkey before the server starts and seal callout traffic
swarm-bao-nats-callout-xkey runs on the queue's host before the
swarm-nats container, under the responder's store leaf: it runs
`swarm-nats-auth ensure-xkey`, reads the public half back from the
store, and writes it where the container's runtime config dir is. On a
change it restarts the server.

nats-server now always starts through the runtime wrapper, in manual
mode too. A privileged ExecStartPre renders auth_callout with the xkey
(and, in auto mode, the two generated nkeys) at every start, and fails
the server when no xkey has been fetched. The auto-mode key unit now
only mints seeds and writes their public halves.

The responder is wanted by nats.service, so a server started late, once
its key arrived, brings the responder up with it.
2026-10-05 23:25:36 +02:00
atlas
bb571285fd swarm-bao: the responder may create the callout xkey; every token reads swarm/public/
The swarm-nats-auth role gains create+read on exactly the callout xkey's
two leaves, and no update, so a stored pair is never replaced.

swarm/public/ is readable by every token through bao's default policy.
swarm-bao-public-read-policy reads that policy, replaces only its own
marker-delimited block, and writes it back when it differs; it never
writes a policy it read back empty. Writing `default` is a new grant on
the granter's own policy, so on an existing store it lands only after
the bootstrap-token step re-runs; until then the unit says so and exits
0, and swarm-bao-granter-role restarts it on success.
2026-10-05 23:25:36 +02:00
atlas
1e0b10bb7d swarm-nats-auth: open sealed callout requests and seal the replies
When nats-server has an auth_callout xkey it seals each request to that
key and names its own public xkey in the Nats-Server-Xkey header. The
responder now loads the callout xkey seed from the secret store before
it connects, opens a sealed request with it, and seals its reply (grant
or deny) back to the header's key. A request with no header is answered
in plaintext, as before.

A new `ensure-xkey` verb creates the pair when the store holds no seed:
the seed at swarm/services/swarm-nats/callout/xkey-seed, the public half
at swarm/public/swarm-nats/callout-xkey. Each write is a create-only
(cas=0) write, and a run whose create fails reads the stored value back,
so two concurrent runs end up with one key. The responder itself only
reads the seed and refuses to start without it.

swarm-secret-client gains SecretStore::create, the cas=0 write.
2026-10-05 23:25:36 +02:00
atlas
4a289c62e7 docs/scheduler/ci: say it in the active voice where the forwarded enable is refused 2026-10-05 21:52:22 +02:00
atlas
65505f5235 deploy: forward swarm.forge.ci to deploy.ci-runner, say what the forwarded enable needs
swarm.forge.ci.* were per-host options that 4423da48 had already moved to deploy.forgejo.ci.*, so they rename straight to deploy.ci-runner like the forgejo.ci ones; ciRunnerRemoved goes with its last caller.

The forwarded enable now trips deploy.ci-runner's store-identity assertion on a host with no deploy.bao.enable. The assertion names both options and the default, the doc states the requirement, and a module-eval case pins the refusal.
2026-10-05 21:36:11 +02:00
atlas
b114c6b165 deploy: rename the old forgejo.ci options to ci-runner instead of removing them
deploy.forgejo.ci.{enable,name,concurrency,labels,package,jobTimeout} each
have a same-typed, same-meaning counterpart under deploy.ci-runner on the
same host, so they are mkRenamedOptionModule entries again and a config that
still sets them evaluates with the standard rename warning. The swarm.forge.ci
spellings stay removals.

docs/scheduler/ci.md states the rename, that the old hive-ci runner's entry
is deleted by hand in the forge admin panel, and that actionsFromSelf
defaults on where the controller runs.
2026-10-05 21:31:38 +02:00
atlas
b6f5cb0479 swarm CA: say when tmpfiles deletes the old CA files
`systemd-tmpfiles` removes them at boot and when a switch changes the
host's `tmpfiles.d` rules, not on every deploy.
2026-10-05 21:17:10 +02:00
atlas
626790a464 hive CA: catch a deleted CA file however its path is spelled
The CA-file assertion compared the option's raw string, so
`/var/lib//swarm-ca/root.pem` or `/var/lib/hive-tls/./ca.pem` got past it.
Both sides are normalised lexically now: empty and `.` components drop out,
`..` removes the one before it, a trailing `/` goes. A symlink is not
resolved at eval time, so a path through one still passes; docs/swarm/ca.md
says so.
2026-10-05 21:17:10 +02:00
atlas
1a4dc16053 hive TLS: reload the trust bundle's unit on a switch, name the missing root when it's empty
A switch that changed `hive-tls-ca` stopped and started it, and the stop
propagated through `Requires=` to the store's container (sealed again under
the shamir seal), every container binding the bundle, and every
`<name>-ca-bundle` unit with its consumers. `reloadIfChanged` with the same
script as `ExecReload` rewrites the bundle in place and restarts none of them.
`Requires=` stays, so a failed assembly still keeps them from starting.

With `swarm-services-root.pem` missing, `hive-tls-ca` writes an empty bundle
and every `<name>-ca-bundle` unit fails. Its log line now names the root's
path. docs/swarm/ca.md describes both.
2026-10-05 21:17:10 +02:00
atlas
cf7d357f17 swarm CA: describe the store's pki root as the only anchor in docs and comments
The docs, option descriptions and comments that named the swarm root, the
swarm CA or the hive CA as an existing trust anchor now name the secret
store's pki root or the trust bundle that holds it:

- docs/swarm/README.md, secrets.md, bao.md
- deploy.bao.clientCaFile's and gateway.verifiedProxyTo's descriptions
- comments in hive-c0re/src/meta.rs, hive-agent/src/swarm_queue.rs,
  swarm-logs/src/auth.rs, swarm-controller/src/vcs_metrics.rs,
  hive-forge, swarm-authelia, swarm-ci-runner, glue-bao-tls, swarm-bao,
  swarm-grafana, swarm-otel, otel, hive-c0re, lib/hive-ca-trust.nix,
  hive-forge/Cargo.toml and the flake-update workflow

Nothing behaves differently: every edit is a nix, Rust or TOML comment, an
option description, or a test assertion message. The Rust edits change the
workspace source, which this PR already rebuilds for hivectl.

Refs #4588
2026-10-05 21:17:10 +02:00
atlas
15f7d4d0cf hive TLS: re-issue a services leaf that does not chain to the store's root, delete the hive CA's leftovers
- swarm-services-cert (and its daily renewal) treats a services leaf that
  `openssl verify` rejects against swarm-services-root.pem as invalid and
  asks the store for a new one, as hive-tls-resign already does for the
  gateway leaf. The trust bundle is that root alone, so a leaf under any
  other issuer verifies for no consumer. With the store unreachable the
  unit fails, and where the gateway waits for the leaf, the import fails
  with it.
- Every host also deletes the old hive CA's ca.srl and .swarm-ca-adopted
  in deploy.hive-controller.tls.stateDir, with the same tmpfiles `r`
  rules as ca.pem/ca-previous.pem/ca-key.pem. Nothing reads either.
- docs/swarm/ca.md: the services leaf's re-issue conditions, the two
  extra deleted files, and that the deletion is irreversible: rolling back
  to a generation with swarm-ca.service mints a new, unrelated root that
  nothing issues under.

module-eval hive-tls: a case for the services-leaf check in the boot
issuance and the renewal (red with the line removed), and the deletion
case covers the two extra files.

Refs #4588
2026-10-05 21:17:10 +02:00
atlas
4e5a642928 hive CA: delete ca.pem, ca-previous.pem and ca-key.pem, the bundle is the store's root
The gateway leaf comes from the store's pki mount, so a hive CA's
certificates anchor nothing and its key is a secret outside the store.

- hive-tls-ca writes trust-bundle.pem from swarm-services-root.pem alone;
  the ca.pem and ca-previous.pem slots are gone.
- Every host deletes ca.pem, ca-previous.pem and ca-key.pem in
  deploy.hive-controller.tls.stateDir with tmpfiles `r` rules that follow
  the option. The removal shim, now host-ca-removed.nix, holds these
  beside the /var/lib/swarm-ca rule and the swarm.ca.* removals.
- deploy.bao.clientCaFile, deploy.bao.serverCaFile and
  deploy.swarm-controller.hiveClientCaFile naming /var/lib/swarm-ca or one
  of the deleted hive CA files fail evaluation, naming the store's mTLS CA
  (/var/lib/swarm-bao-pki/ca.pem on the store's host) as the replacement.

module-eval hive-tls covers the single-anchor bundle, the deletion (incl.
a relocated stateDir) and the assertion on all three options.

Refs #4588
2026-10-05 21:17:10 +02:00
atlas
93f1a76ea7 swarm CA: retire the host-held root CA, the store holds the only anchor
The secret store's pki root issues every hive's gateway leaf and the
swarm's service names, so the root swarm-ca.service generated on the
host anchors nothing and its key is a secret outside the store.

- swarm-ca.nix is gone: swarm-ca.service, root.pem and root-key.pem,
  and the swarm.ca.{autoConfigure,stateDir,validityDays} options. A
  definition of any of them fails evaluation with a removal message
  (swarm-ca-removed.nix), and a tmpfiles rule deletes /var/lib/swarm-ca
  on every host. singleHostSwarm no longer sets a swarm.* option.
- hive-tls-ca drops the $root slot: trust-bundle.pem holds the store's
  pki root and a hive CA's public ca.pem/ca-previous.pem only.
- hivectl peer-config no longer prints the scp line for the swarm root;
  the store's pki root is the one step.
- The gateway's cert import no longer stands a hive leaf or a throwaway
  in for a missing swarm-services leaf where it Requires= the issuing
  unit: it fails, naming swarm-services-cert.service. The stand-in stays
  where the gateway does not wait (no swarm name, or the store's UI
  only, whose passthrough must start with the store sealed).

Comments and docs that named swarm-ca.nix, swarm.ca.* or the host root
now name the store's pki root.

Refs #4588
2026-10-05 21:17:10 +02:00
atlas
5d6ebda3e8 swarm-otel: name the port-collision assertion instead of pointing below
The three port descriptions in swarm-otel-service.nix said "the
assertion(s) below", but the assertion lives in swarm-otel.nix, on the
host that runs the collector. Describe it in this file's terms, and drop
the release anecdote from the port option's description.

Refs #3742
2026-10-05 19:22:55 +02:00
atlas
dc5c2d8d83 nix: follow-ups from the service/deploy split
- swarm.victoriametrics.port: the description states why the listener is
  loopback-only instead of pointing "below" at text in another file.
- authelia: the deploy half derives its unit and instance names from
  swarm.authelia.unit, so the instance is spelled once, in
  swarm-authelia-service.nix.
- authelia: the tokenEndpointAuthMethod and bearerAuthz descriptions name
  the obligations the authelia host asserts instead of "below"; the
  swarm-authelia.nix header describes that file only.
- module-eval-swarm-otel-core: swarm.otel.audience must contain both store
  push URLs the collector's exporters use and request tokens for.

Refs #3742
2026-10-05 19:12:46 +02:00
atlas
960cf9ce30 container stats: bound hive-c0re's wait for the disk measurement, no total for an agent with no paths
hive-c0re waited on hive-priv's answer with no timeout, so a hung
hive-priv would stall the disk sampler indefinitely. The wait is now
hive-priv's own 10 min bound plus a minute (the bound is shared as
MEASURE_AGENT_DISK_TIMEOUT); a timeout is an error, so the agent shows
unknown.

An agent with none of state/, work/ or the rootfs has no measurable
size. hive-priv now answers with an error for it instead of a total of
0, which read as a real measurement.
2026-10-05 18:56:46 +02:00
atlas
98ccc31e2b container stats: measure agent disk usage through hive-priv, unknown on failure
The disk sampler ran `du -sxb` as the unprivileged hive-core user over
each agent's state/, work/ and container rootfs. hive-core cannot read
the rootfs or all of the agent-owned files, so `du` failed and each
failed path silently counted as 0: the dashboard showed a few MB for
agents using over 13 GB.

Add a hive-priv `MeasureAgentDisk { agent_name }` op. hive-priv
validates the name, derives the three paths itself (the request carries
no path), skips the ones that don't exist, and runs one
`du -s -x -b -c -P` as root with a 10 min timeout. A timeout, a non-zero
exit or output without a `total` line is an error.

hive-c0re calls it every 30 min instead of running `du` every 5 min:
each pass walks every agent's full tree as root on the disk the agents
work on, and the sizes move slowly. A failed measurement is cached as
None, so `disk_bytes` is null, the OTel storage.usage sample is omitted
and the dashboard shows "unknown", never 0.

`Coordinator::agent_work_dir` had no other caller and is removed.

Closes #4956
2026-10-05 18:56:46 +02:00
atlas
8ffceea8e3 swarm-ci-runner: load the registration token in a register unit of its own
systemd hands a unit's credentials to every process of that unit, and a
host job is a process of the runner daemon, so the `LoadCredential=` on
gitea-runner-swarm still let any workflow read the registration token
from `$CREDENTIALS_DIRECTORY`.

swarm-ci-runner-register.service, a oneshot under the daemon's dynamic
user and state dir, now holds the only `LoadCredential=` and runs the
precond and `act_runner register` (only when `.runner` is missing). The
daemon unit has no credential, no `EnvironmentFile` and no `TOKEN`; its
pre-start step starts the register unit, so a missing token fails the
daemon's start and `Restart=on-failure` retries it.

The precond compares `.runner`'s labels in the form the daemon rewrites
them to (`nix` -> `nix:host`); compared verbatim, a scheme-less label
re-registered the runner on every start.

tmpfiles removes the unused `registration-token.env` and upstream's
`.token-hash`/`.labels` markers.
2026-10-05 16:33:24 +02:00
atlas
0302b29ca4 swarm-ci-runner: give the registration token only to the register step
The runner got its registration token through an `EnvironmentFile`
(`TOKEN=…`), so the daemon carried it for its whole life, and for a
`host` label act_runner copies its environment into every job step:
any workflow could read a token that registers runners on the forge.

The host unit now writes the bare token, and the runner unit loads it
with `LoadCredential=`. The upstream pre-start step, which registered
from `$TOKEN`, is replaced by one register step that reads the token
from `$CREDENTIALS_DIRECTORY` and runs `act_runner register` only when
`.runner` is missing, with the instance URL, name, labels and config
file upstream used. The precond now also removes `.runner` when its
labels differ from the configured ones, which upstream's step covered.
A changed token alone no longer re-registers.

Host jobs can still read `.runner` itself (#4979).

Closes #4974
2026-10-05 16:06:52 +02:00
atlas
84347f122c forgejo: actionsFromSelf defaults to whether this host runs the swarm-controller
The controller seeds the actions/checkout mirror from its own host's
mirror list, so a forge host without it got DEFAULT_ACTIONS_URL = self
and no mirror. The default now follows
deploy.swarm-controller.enable; the module-eval check pins forge-only
(false, no warning) and forge+controller (true, mirror seeded).
2026-10-05 13:55:37 +02:00
atlas
80bc92fb49 forgejo: actionsFromSelf defaults to true 2026-10-05 13:55:37 +02:00
atlas
e2da5509ae swarm-ci-runner: run the forge CI runner as a swarm service
The CI runner was a hive deploy module: hive-ci.nix ran it beside the
forge, and hive-c0re registered it through the forge's admin API with
the `core` token, writing the registration token through hive-priv. It
could only run on the forge's host, and it kept an admin-scoped forge
token on a hive.

Now the runner is a swarm service:

- swarm-ci-runner.nix runs it in a `swarm-ci-runner` container on the
  host that sets deploy.ci-runner.enable, reaching the forge by name.
  It needs neither the forge nor a hive there.
- swarm-controller reads the forge's runner registration token with its
  own admin token and writes it to swarm/services/ci-runner/
  registration-token whenever it differs from what it last wrote. Its
  grant on that path is create/update only.
- The runner's own leaf (CN swarm-ci-runner, role and policy of the same
  name) reads that one path. swarm-bao-ci-runner.service fetches it into
  the runner's EnvironmentFile and restarts the runner on a change.
- A pre-start step removes .runner when the daemon's journal since its
  last start holds Forgejo's "unregistered runner" refusal (the daemon
  exits 1 when its startup Declare gets it), or when .runner names
  another forge address; upstream's pre-start step then registers again.

Removed from the hive side: hive-ci.nix and deploy.forgejo.ci.*,
forge/ci_runner.rs, PrivRequest::RegisterCiRunner and its handler and
ReadWritePaths entry, InfraContainer::Ci with hivectl's --ci and the
dashboard's hive-ci journal entry, HYPERHIVE_FORGE_CI_ENABLED, and
read:admin,write:admin from CORE_TOKEN_SCOPES: nothing else in hive-c0re
calls /api/v1/admin/*. The old ci options, under both swarm.forge.ci and
deploy.forgejo.ci, now fail evaluation naming deploy.ci-runner.

The forge's half of the old ci.enable, the actions/checkout mirror and
DEFAULT_ACTIONS_URL = self, is deploy.forgejo.actionsFromSelf.

Closes #4954
2026-10-05 13:55:37 +02:00
atlas
3d43e9588c agents: shorten bind_child_agent_dirs doc under the comment-block limit 2026-10-05 13:24:36 +02:00
atlas
d2ea50031e agents: work/ in the disk-metric docs, holder never sees work/, chown guard comment 2026-10-05 13:23:39 +02:00
atlas
6ff348d88b agents: define the work dir name once and test its creation 2026-10-05 13:10:18 +02:00