New GET /api/swarm on swarm-controller, backed by
services.hyperhive.swarm.name (SWARM_CONTROLLER_NAME env var, same
optionalAttrs-gated-on-option-resolving shape queueEnv/forgeEnv/etc.
already use). swarm-ui's <Shell> fetches it once and sets both
document.title and the header's brand text; falls back to the
existing static "hyperhive swarm" label when the operator never set
a name or the fetch fails.
Extracted the swarm-queue connect block out of main() into its own
connect_status_reader() fn to keep main() under clippy's line-count
lint after adding the new field wiring — no behavior change, same
comments moved as-is.
The controller's forge client speaks TLS to `https://<forge domain>`,
which the gateway serves with a leaf signed by the hive CA. That CA is
generated at runtime, so nothing build-time can name it and it is not in
the system store -- and `reqwest`/`rustls` resolves roots through
`rustls-native-certs`, whose `SSL_CERT_FILE` *replaces* the store rather
than adding to it. With no bundle wired, every forge call failed
`invalid peer certificate: UnknownIssuer` and agent creation died at its
first step.
`lib/hive-ca-trust.nix` already solved this, but only for containers: it
sources the CA through `/run/hive-ca/trust-bundle.pem`, a bind mount that
does not exist on the host. `hostUnit` makes it read the host copy and
wait on `hive-tls-ca.service` itself -- one flag driving both, because a
host source without that ordering is a race.
`enable` is the other half, and it is the sharp edge: a container caller
imports this into the container's module set, so it disappears with the
container. A host caller imports it at the host's top level, where
`imports` is unconditional -- without the flag, a hive with the
controller turned off would get a bundle oneshot and a `swarm-controller`
service conjured by `genAttrs`, holding an `SSL_CERT_FILE` and no
`ExecStart`.
Forgejo returns a bare `null` body (not `[]`) for a reactions list when
nothing has reacted yet. issue_reactions/comment_reactions deserialized
straight into forgejo-api's typed Vec<Reaction>, which has no null
tolerance, so issue show / pr show / view failed on every item with zero
reactions - i.e. nearly everything.
Generalize pr_status's existing null_as_empty (same quirk, hit earlier
on combined-status statuses) into a shared helper in verbs/mod.rs, and
fetch reactions via the raw JSON path (Client::get_api_json) with a
NullableVec<T> wrapper instead of the typed client's bare Vec<Reaction>.
Also names the failing request in errors going forward, since
get_api_json's error context includes the URL - closes the gap the
issue itself flagged (the old error said what didn't parse but not
what was fetched).
mara: "pls remove historical wording, only present pls".
The correction was written as a diff against what the docs used to claim
-- "this used to say X", "where this is going", "the intended state for
now". That is a changelog, and a reader arriving cold has to reconstruct
the current truth from it. The reasoning about why the old shape was
wrong belongs in the PR that changed it, not in the file.
Now says what is true: the controller interprets a delivery and emits a
semantic message; receipt is all that is wired today because the
swarm->hive channel does not exist yet.
mara asked for the docs and comments to be fixed in one PR before the
implementation, since a half-corrected description is what confuses a
reader coming back after a context compact.
Three more places said or implied "forward the payload":
- post_webhook_forge's doc ("relays the delivery to every hive ... the
payload is never parsed here")
- its OpenAPI request_body description ("the delivery can be relayed
unmodified") -- this one is published in the spec, so it was the most
externally visible of them
- DeliveryKind::as_str, which called the hook kind the event's routing
key; the message is semantic, and which hook it arrived on is an input
to deriving it rather than the thing sent
docs/swarm/README.md gains the direction next to the interim state, so
"two hooks, one of which only logs" reads as a step rather than a design.
mara's ruling: the controller should not forward received events, it
should create specific messages from them ("knowledge repo changed",
"deploy agent foo to rev abc123") and send those to whoever needs them.
The module doc said the opposite as settled design.
The argument it made was wrong in a specific way worth keeping: it
counted this daemon as a *second* place deciding what a delivery means,
while assuming the hives stay interpreters. They do not -- the parsing
moves rather than duplicating, so the end state has one interpreter, not
two.
Doc-only. The behaviour is unchanged and still receipt-only; what changes
is that the next reader is not told the wrong direction.
argus's optional note. Nothing in the doc was inaccurate, but an operator
who sees the activation-time warning has no thread back to the page that
explains why pinning matters -- and this section is the explanation.
States the one property they need from it: it never fails the rebuild, so
acting on it is theirs to do before the ids are minted.
`serverName` is baked irrevocably into every user and room id, so a hive
that rebuilds onto a new default is a *different homeserver*, not a
renamed one: existing accounts and rooms are stranded, and reverting the
config does not undo it. Its neighbours (`gatewayHost`, the forge domain)
are routing, rediscovered through `.well-known` and fixed by editing them
back. Same diff shape, three orders of magnitude apart in blast radius --
which is an asymmetry a module should carry rather than an operator.
An activation script and not `warnings`, which is where this obviously
belongs and does not work: the condition needs the host filesystem, and
`nixos-rebuild switch --flake` evaluates purely, where
`builtins.pathExists "/var/lib/..."` answers false rather than throwing.
A `warnings` entry gated on it would evaluate, deploy, and print nothing
on every real deployment.
Rendered only when `serverName` is null, so a pinned hive has no script
rather than a script that stays quiet -- a guard that cries wolf at a
correctly-configured deployment makes the next real one read as noise.
Never fails the activation: it warns about a choice that cannot be
undone, and refusing the rebuild of a hive that already chose
deliberately is the opposite of helping.
The probed path is read out of the container's own evaluated config
rather than hardcoded. A guessed path resolves cleanly and silently never
matches, which is the same failure this guard exists to catch one level
up.
mara asked whether SWARM_CONTROLLER_PUBLIC_URL should also be set when the
controller does not run on the same host, and then said the two read as one
service to her. Both are worth answering in the file rather than only in the
thread: that split is not representable today, and the reason is two hops
away from this line.
The UI's /api/ location proxies http://unix:<socketPath> -- a path that
resolves nowhere else -- and this daemon binds no TCP address at all. So the
vhost and the daemon are co-located by construction, and ui.enable is the
flag that declares the vhost rather than a guess about some host serving it.
Also names the tripwire: the day the daemon grows a TCP listener, this line
silently stops being right (env unset, registration quietly skipped, no
error). That is when an explicit publicUrl option becomes correct -- not
before, while there is exactly one derivable answer.
The endpoint landed inert: nothing pointed at it, so the only way to see
it work was to mint an HMAC by hand. Register the two swarm-wide hooks
at startup so a real forge event produces a journal line.
Registered ALONGSIDE the per-hive hooks, not instead of them. Every hive
keeps receiving and acting on its own deliveries; the controller gets a
copy and logs it. Moving the registration is a later step and has to be:
fan-out swarm->hive does not exist yet, so a hook moved now would point
at a receiver that forwards nowhere, silently on both sides.
Deliberately no stale-hook deletion arm, unlike the two per-hive
registrars this otherwise mirrors: theirs delete hooks matching their own
path with a foreign base, and the hives' hooks are not stale.
The route prefix is what keeps this safe. Both hive-side registrars
delete any hook ending in /webhook/knowledge or /webhook/config-pr with a
different base, so a swarm hook under those paths would be deleted by
every hive on every boot. Serving them under /webhook/forge/ avoids it,
and a test pins it -- there is nothing else that can.
SWARM_CONTROLLER_PUBLIC_URL is set only where the swarm vhost is served,
because a hook whose target_url nothing answers is worse than no hook.
A forge webhook is a machine POST carrying an HMAC and no session cookie,
so it cannot pass the authelia auth-request subrequest every other location
on this vhost uses. This location deliberately omits it; the HMAC check in
the controller is what guards the path.
Scoped to /webhook/forge/ rather than /webhook/ so a future endpoint under
the same prefix does not inherit the bypass.
The crate's module doc claimed no persistence and no writes, which this
change makes false, and docs/persistence.md had no swarm-controller entry
at all — the webhook secret is the daemon's first persisted state.
A Forgejo webhook has one target_url, so every hive registering the same
swarm-wide hooks is last-writer-wins rather than idempotent. The controller
is the only swarm-wide thing in the deployment, so it becomes the receiver.
It verifies the HMAC and treats the payload as opaque bytes keyed by the hook
kind in the URL path; it deliberately does not parse the payload, because the
hives' existing handlers already decide what a delivery means.
Nothing is registered against the endpoint yet. The replacement path is built
and observable before anything takes the old one away, so the swarm's single
target_url never points at a receiver that forwards nowhere.
Its last call sites were the JSON store's to_string_pretty/from_str,
which this branch replaced with serde_norway. Nothing in swarmctl/src
names it any more.
Caught by argus, and it is the second time: the same reviewer found a
dead reqwest in swarm-controller after a move earlier this week. No lint
in this workspace sees an unused dependency, so both were green locally
and in CI -- a move has two sides and my gate only looked at one. My
local gate now carries an advisory check for it.
Six places asserted the old design as fact, and none of them mention the
change by name -- the class of doc breakage that is found by asking what
a diff made untrue, not by grepping for a feature:
- swarmctl/README.md and swarm-authelia-bridge/README.md both described
their own private canonical store. The bridge's "known limitation"
section described the seam as unsolved; it is what this fixes, so it
becomes what both writers must uphold instead.
- docs/swarm/{sso,ui,secrets}.md described a rendered artifact.
- The repo CLAUDE.md entry for swarmctl said the same.
- docs/tools/swarmctl-cli.md is regenerated (CI diffs it against the
clap tree), picking up the removed --store flag.
Operator-facing where it is read: the hand-editing consequence (values
survive a rewrite, comments do not) is stated in sso.md, where an
operator is being told to edit the file, rather than only in a module doc.
#3414 fixed the missing-email defect in swarmctl only, so every identity
created by the bridge landed in users.yml with no `email`. A relying
party that asks for the claim does not degrade, it fails -- grafana's
OIDC login is the measured case (#3393).
Two writers of one file disagreeing about a field one of them treats as
required is not a difference worth keeping, and with the file shared the
result also depended on which tool wrote last.
Mutation-checked: removing the fill turns the new test red and nothing
else.
The second half of making one file canonical. swarmctl kept its own
private JSON store and rendered users.yml from it, against the same
physical file the bridge wrote -- the seam that made `swarm agent create`
refuse to run.
- users.yml is read before it is written, so users another writer added
are loaded rather than treated as a file to refuse or clobber. The
overwrite guard and the seed check go with the second store: they
existed to police two stores that could disagree.
- serde_norway replaces the hand-rolled emitter. Unknown top-level and
per-user keys round-trip through `extra`, so two writers cannot delete
each other's fields.
- The synthetic email moves from render time to the write path and is
stored. With the file as the store, "rendered but not persisted" has
nowhere left to live, and mara ruled the stored address correct.
- `--store` is removed rather than deprecated: a flag whose only
remaining effect is nothing reads as accepted and does nothing.
Three tests asserted the emitter's exact bytes, and one asserted the
guard. Rewritten rather than deleted -- as round-trips for the former,
and inverted for the latter, since "a populated file is READ" is the
behaviour this change is for and deleting its test would leave it
unpinned.
The bridge no longer has a private canonical store, so the env var
naming one is gone rather than repointed. What is left is the single
users file it now reads and writes directly.
That line is what made `swarm agent create` fail on this hive: the
bridge found no store at its own JSON path, saw users already in
`users.yml`, and refused to overwrite a file it had not written --
correctly, given two things claimed to be canonical for one file.
`swarm agent create` failed with "no user store at
swarm-authelia-bridge-users.json but users.yml already holds users" on
any hive that had users. The guard was correct; what was wrong is that
two files both claimed to be canonical for one physical file.
The bridge kept a private users.json and rendered users.yml from it,
while swarmctl kept its own pair against the same users.yml. A writer
whose own JSON was absent could not tell "nothing here yet" from
"someone else's users", so it refused to write at all.
users.yml becomes the store: read before write, through serde_norway
rather than a hand-rolled emitter. Unknown top-level and per-user keys
round-trip through `extra`, or whichever process writes second would
silently delete what the first added. Validation stays on the write path
-- a bare to_string(&store) serialises perfectly and drops the "no
control character ever reaches this file" guarantee silently.
The seed constant goes with the guard: nothing writes a seed now, an
absent file is an empty store, and left as a pub constant it read as if
the seed dance were still load-bearing.
swarm-authelia-bridge answered one 401 for four causes, including its own
client credentials being rejected by authelia — telling a caller its token
was invalid when the fault was ours. That cost a real diagnostic round-trip,
because the only thing separating the two was a log line inside a container.
Introspection now returns a three-way Verdict, and one classify() maps it to
either admission, a 401 (the caller's credential) or a 503 (ours). Admission
is unchanged: Ok is reachable from exactly one variant.
Closes#3423.
Modern browsers validate an <input pattern> attribute's regex in
Unicode-set ('v') mode, which is stricter about hyphen placement than
classic mode: `[a-z0-9-]` throws "Invalid character class" under 'v'
mode even though a trailing hyphen is unambiguous (and valid) in classic
regex. Reproduced directly: `new RegExp('[a-z0-9-]', 'v')` throws,
`new RegExp('[a-z0-9\\-]', 'v')` doesn't. Escaping the hyphen fixes it
without changing what the pattern matches.
atlas, on the issue: swarm-controller's problem+json conversion merged
(58588a68, closing hyperhive#3412 which this comment referenced as an
in-flight fix). Every first-party API is RFC 9457 now, so the non-JSON
fallback path is not backend-transition compatibility anymore — it's for
bodies this repo didn't author (a caller pointed elsewhere, or an
nginx-synthesised 502/504 that never reaches our handlers). Comment
updated to say that instead of pointing at a transition that's already
finished.
mara, on review: "i want each component to import its own css file
itself[;] if you need a bunch of extra css externally, its not a proper
component ... you may need to migrate other components that you would
want to use first".
WarnBanner is the Preact-component successor to the shadow-DOM
<hive-warn> custom element (same three-tier info/warning/error visual
language, colours copied faithfully from hive-warn.css). ApiErrorPanel
now composes it instead of owning a copy of the border/colour/pulse
rules itself — its own CSS is back down to just the layout that's
actually specific to it (heading, copy button, detail text).
Re-verified with a fresh mock-server screenshot: same rendered output as
before, now via composition instead of a duplicated banner shape.
Closes#3410.
`ApiErrorPanel` renders a ProblemDetails (RFC 9457) error nicely, with a
copy button so the full text can be pasted straight into a bug report.
No truncation of `detail` — on the #3363 incident that motivated this
issue, that string was the entire diagnosis.
`readApiError`/`problemMessage`/`ProblemDetails` are promoted out of
credentials.js's original `readErrorBody` into `@hive/shared/api-error.js`
(comment rewritten: the RFC 9457 rework has already landed everywhere
except swarm-controller's status route, #3412 in flight, so the raw-text
fallback is a compat shim for that one gap, not a general transition).
credentials.js's 4 call sites switch to the shared reader (kept as
one-line messages there, its result slots are single-line aria-live
regions, not a panel context).
Wired ApiErrorPanel into OverviewPage.tsx (the issue's own worked
example) and CreateAgentPage.tsx (second real call site).
Not built on <hive-warn> despite matching its visual language — that
custom element's CSS-as-text import only works under a build with
loader: 'text' for .css (dashboard's), and silently renders unstyled
under swarm-ui's default css loader (filed separately as #3415).
ApiErrorPanel is a self-contained light-DOM component instead, per
mara's own suggestion to keep it independent of the old UI's shapes.
Plugin management is server-admin scoped, and on an SSO hive nobody holds that
role: auto_assign_org_role grants an org role, and the built-in local admin
that does hold server admin cannot log in because the login form is disabled
whenever SSO is configured. Two individually-correct decisions leaving no path
to the plugin UI at all.
Declarative is the way through rather than a workaround for it -- plugins land
in the store and in git, survive a rebuild and a state reset, and the container
needs no runtime egress to grafana.com.
mkIf rather than passing the list through: upstream's default is null, while an
empty list is a real value pointing the plugin path at an empty store dir, so a
hive that sets nothing must keep seeing null.
Authelia serves no `email` claim for a user without one, and a relying party
that wants that claim does not degrade -- grafana falls through to
`<api_url>/emails`, a GitHub-ism authelia does not implement, and the login
dies with InternalError naming nothing useful. So an absent address is a
broken login rather than a sparse profile.
Uses `<username>@hyperhive.local`, the domain hive-c0re already gives every
agent's forge account and every hyperhive-authored commit. Not deployment-
derived: that would have to be plumbed in from config, and an operator already
supplying a domain may as well supply the whole address.
Synthesised in the renderer, never in the store: users.json stays honest that
none was supplied, an operator who later sets a real one is not fighting an
invented value, and existing users are fixed by the next render with no
migration step. A supplied address always wins.
The old test asserting an absent email is omitted pinned exactly the behaviour
that broke the login; it is split so the group half keeps its meaning and the
email half states the new contract.
Its three error paths returned a bare string with a status code, which forces
a caller to treat the whole body as prose. hive-c0re converted some time ago,
so swarm-controller was the last backend on the old shape -- and it is the one
behind the hive status page's 503, where the body is frequently the entire
diagnosis rather than a summary.
Adds the commitment to docs/conventions.md, since it was implied by the code
in one daemon and written down nowhere: an endpoint of ours answering with a
bare string is a bug to file, not something callers work around.
The test asserts the rendered response -- media type plus an addressable
detail -- rather than the problem_details value, because a handler that built
the value correctly and then returned it as a string would satisfy a test
written against the type alone.
`store.keys()` creates an ephemeral ordered consumer, whose create subject
ends at the stream name. `>` matches one or more tokens and never zero, so
the `.>` form alone never covered it: the server refused
`$JS.API.CONSUMER.CREATE.KV_hive-status`, the refusal reached the client as a
timeout, and the operator saw a 503 on the hive status page.
The test asserted only the `.>` form, which reads as covering the bare one, so
the suite stayed green while every list timed out in production. It now names
the bare subject separately and first.
Last of the three modules that hand-rolled the same concat with wantedBy +
before and no requires, so a failed assembly left the consumer running against
a missing file and trusting nothing -- every outbound TLS call fails while the
unit looks healthy.
The forge is the one with two consumers: forgejo-sso-source fetches the
issuer's discovery document over the swarm CA and once shipped without the
trust its sibling had. It only exists when SSO is on, so the consumer list is
conditional -- naming an absent unit would define a serviceless one and order
nothing.
Removes the now-dead useSelfSigned and caContainerPath bindings (nix does not
warn) and retargets three comments the deletion orphaned, including the helper
header that still named this module as the per-call-site concat.
Both hand-rolled the same concat with wantedBy + before and no requires,
so a failed assembly let the consumer start against a missing file and
trust nothing at all -- which fails every outbound TLS call while the unit
looks healthy. The helper puts requires on the consumer and verifies the
assembled bundle before moving it into place.
Removes each module's now-dead useSelfSigned binding: its only use was the
mkIf on the deleted block, and nix does not warn about an unused let
binding. Also corrects two comments the deletion orphaned -- one described
the removed path binding, the other pointed at a bundle service that no
longer exists.
A loopback literal encodes 'authelia is in my netns' at the call site,
and authelia's OIDC endpoints are https-only in effect: reached directly
they answer 400, because the forwarded headers nginx injects for every
other consumer are what let it determine its own issuer. Going by name
deletes the need for those headers rather than reproducing them, and
converges on the URL swarm-nats-auth already uses.
Depends on the CA trust added for the same container in #3407 -- without
it this swaps a 400 for an UnknownIssuer.
The null-url assertion joins the module's existing list: interpolating a
null would surface as a nix coercion error several files from its cause.
Same class-B defect as the queue responder: swarm-authelia-bridge
introspects authelia by name over https and the container trusted no swarm
CA, so the call could only ever fail UnknownIssuer.
This is also the prerequisite for the by-domain introspection change on
#3391 -- flipping that URL without the trust half would swap one failure
for another.
The responder introspects authelia over https by name. It had no CA trust
at all, so the handshake failed UnknownIssuer, introspection failed, and it
denied every client -- surfacing at the controller as a 60s
authorization-violation loop, two layers from the cause.
Adds a shared trustBundle helper to lib/hive-ca-trust.nix rather than a
fifth hand-rolled concat. Four containers were each assembling this
themselves, which is how they came to share one defect: wantedBy + before
express ordering but not success, so a failed assembly let the consumer
start against a missing file and trust nothing at all.
The helper fixes both halves of that. requires goes on the consumer, so a
failed bundle stops it and the dependency is visible in systemctl status
where someone debugging a TLS failure looks. And the script assembles to a
temp path, checks the result actually contains a certificate, and only then
moves it into place -- cat of an empty bind exits 0, so set -e does not
catch it and a partial bundle must never appear under the final name.
Returns a module rather than bare services: a caller that already writes
systemd.services.<consumer> cannot also write systemd.services in the same
attrset.
Keeps why the setting exists (the bridge writes this file and cannot
restart authelia) and drops the incident narrative and the answered
objections -- that history belongs in the PR discussion, not in three
comment blocks a future reader has to scroll past.
authelia now watches the users file, so the restart is redundant -- and it
was the wrong shape twice over. It could fail: a login was refused for a
user whose record was already correct on disk, with nothing in either log
implicating the reload. And it only ever worked for this writer --
swarm-authelia-bridge writes the same file and cannot restart anything,
since running unprivileged inside the container is the whole reason it may
write it at all. A reload that depends on which process did the writing is
not a reload.
--machine/--unit and their two env vars existed solely to name a
systemctl -M target, so they go with it. That drops two required settings
from the operator surface.
The three objections previously recorded against watch are all answered
now, and are kept next to the decision rather than deleted: the key is
verified against the pinned build (validate-config accepts it and rejects
a misspelling), the watch is on the directory so a rename is observed, and
partial reads are structurally impossible because every writer of this
file goes through write_atomic.
Authelia reads authentication_backend.file.path once at startup. Without
watch, an identity the bridge creates is real on disk and invisible to the
running authelia until something unrelated bounces the unit.
swarm-authelia-bridge cannot restart authelia -- running unprivileged
inside the container is the whole reason it may write that file, so
restarting the unit is exactly the privilege it was designed not to hold.
swarmctl does restart it, but that is a systemctl -M shellout that can
fail, and when it did the symptom was a login refused for a user whose
record was already correct on disk. This makes correctness stop depending
on that restart succeeding.
The auth-callout responder decides what an admitted client may publish
from two strings: the prefix marking a hive client, and the client id
allowed to read every hive's key. Both were literals in three places --
swarm-authelia.nix mints "hive-${name}", swarm-controller.nix defines
"swarm-controller", and the responder carried its own copies as clap
defaults because swarm-nats.nix passed neither.
Each producer now publishes its value as a readOnly option and the
responder's ExecStart reads them, so the agreement is one evaluation
rather than three strings that happen to be equal. Same pattern the
module already uses for `--account`, and the same argument
swarm-authelia.nix gives for publishing `machine` and `unit`.
Worth the change because the failure is silent and misattributed:
rename either principal and the responder starts denying the one that
stopped matching, a denial reaches a NATS client as a timeout rather
than an error, and a hive that is refused looks exactly like a hive
that has not reported yet.
The module doc had grown a 37-line preamble carrying three separate
arguments, which the comment-block lint refuses. Splitting it is the
better fix than raising the limit: the reader who is about to widen a
subject list meets the reason not to at the list, not seven screens up.
Also corrects one claim I had no measurement for. The CREATE note said
creating an existing stream with a different config "is an error rather
than a rewrite" — asserted, not observed. What is observed is narrower
and enough: a hive holding this grant leaves the stream config untouched
and never publishes $JS.API.STREAM.UPDATE at all.
