The secrets page discussed 'the telemetry collector' as a reader needing no
delivery, but there are two: the hive's is a host unit and reads authelia's
file in place, while the swarm's runs in a container and gets a copy placed by
a host oneshot.
States plainly that the container one has no operator-provided variant, which
is a consequence of it running beside authelia rather than a gap.
nixpkgs' collector unit already sets SupplementaryGroups to systemd-journal
unconditionally, with a comment saying why. Systemd list options concatenate,
so this module's copy rendered ["systemd-journal" "systemd-journal"] and made
this a second owner of a fact upstream may later change.
The bind mount stays, since that half is genuinely ours.
The logs half was gated on the swarm's log store being enabled, so turning
that store off stopped collection entirely rather than leaving the upstream
export. Logs now fan out exactly as metrics do: the store when it runs, the
operator's upstream when one is configured, both when both.
The receiver, the journal mount and the group grant follow whether there is
anywhere to send logs, not whether the local store exists. Tying the mount to
the store instead would render a receiver that can read nothing.
Deferred until the collector had a logs pipeline writing to it: a store
nothing writes to starts, answers queries and returns nothing, so the first
person to look concludes there were no logs rather than that nothing was
collecting them.
The journald receiver leaves the OTLP body empty and carries the entry as a
map of journal fields, so VictoriaLogs had no message to index and wrote a
placeholder into _msg on every record. Ingest returned 200, every field was
present, and a plain search for a line sitting in the store found nothing.
_msg_field names the field that holds the text. _stream_fields is the
difference between one stream for the whole host and one per unit per
machine; both are set by journald itself and both are low-cardinality.
A journald receiver reading the host's journal directory, a logs pipeline
stamped with the swarm tier's own resource processor, and an otlphttp
exporter pointed at VictoriaLogs. All four parts are conditional on the log
store being enabled, so a swarm without one renders exactly as before.
The host's directory is enough to see every container: nspawn links a
non-ephemeral container's journal guest-side, so the files live on the host
under the container's machine-id, and journalctl descends into those
subdirectories. Measured, along with the fixed systemd-journal gid that makes
the group grant meaningful across the bind mount.
An agent can verify that a unit was launched and never that it is
working: container journals are not reachable, so a diagnosis stops at
the first broken component -- which is precisely the component whose own
instrument is least likely to be legible. This is the store half of
collecting logs centrally so the question becomes answerable.
Mirrors swarm-victoriametrics deliberately: same container shape, same
loopback pin, same self-scrape arrangement. Two differences, both
intentional.
No gateway vhost. VictoriaLogs' ingest and query endpoints carry no
authentication of their own, exactly like the metrics store's -- and the
metrics store IS published under a resolvable name, which is an open
question rather than a settled design. Publishing this one the same way
would repeat that before the first instance is decided.
Retention defaults to 30d against the metrics store's 5y. Logs are
orders of magnitude larger per unit of time and their value decays much
faster: a log line answers what happened during an incident, a metric
answers whether this is worse than last quarter.
Not wired into enableRequiredServices yet -- that lands with the
collector pipeline, so we do not start a store nothing writes to.
prometheus-nats-exporter has never served a metric. Upstream's module
renders `-addr … -port … ${extraFlags} ${url}` and defaults extraFlags
to the empty list, but the binary refuses to start without at least one
collector: it exits 1 with "no Collectors specified". So the unit logged
Started, the process was gone milliseconds later, and every scrape was
refused -- up=0 continuously, scrape_duration 0.6ms, zero samples.
Verified by running the exact argv both ways: without a collector it
exits 1, with -varz it stays up and logs the listener. A bogus flag is
rejected with exit 2, so the check measures acceptance rather than
tolerance.
varz is the server itself, connz makes a client that will not stay
connected visible, and jsz covers JetStream, which this swarm uses for
the status KV. The rest describe a clustered deployment we do not have.
An empty list is not a neutral default when the program requires a flag,
and nothing about it is visible to evaluation -- which is why the gate
now reads the rendered ExecStart rather than only asserting enable.
The swarm-services board was mostly authelia, so it becomes its own page
and is trimmed out of that one (mara, #3494). Eleven panels: uptime,
authentications and failures, authorization decisions, requests and
verdicts by status code, request and OIDC latency, and the three Go
process signals.
OIDC latency gets its own panel rather than being folded into the
general one because authelia keeps a separate duration family for it,
and every machine-to-machine credential in this swarm is minted through
those endpoints -- averaging the two produces a number describing
neither.
Counters are counted over the dashboard range rather than rated. At this
volume a rate window contains no requests, so rate() returns zero and
draws a flat line, which is indistinguishable from a broken query; the
quantile panels are worse, because a quantile over all-zero buckets is
NaN and renders empty rather than zero.
Part of #3591 (mara: "pop ups and menus appearing should animate").
Each popover mounts fresh on open ({open ? <div> : null}, not a state
transition), so a keyframe animation on the popover element itself is
the right tool -- same shape as Shell.css's own shell-page-enter
(fade + a slight translate/scale settle), including the identical
three-rule motion-guard (base rule, prefers-reduced-motion media
query, data-motion=reduce/allow explicit overrides).
Scope: LinksMenu, SettingsMenu, UserMenu -- the three header popovers.
Not included here (posted findings on the issue instead of guessing):
the refresh-interval picker's dropdown (native <select>, whose open
popup is OS/browser chrome outside CSS reach in current browsers --
"not themed" is a platform limitation, not a bug in this component's
own styling) and the jobs graph's node animations (JobqGraph is a
@hive/shared component consumed by both swarm-ui and the per-hive
dashboard, real design/implementation work on shared infra, not a
same-shape mechanical extension of an existing pattern).
Screenshot-verified the settled (post-animation) state renders
correctly; a static screenshot cannot show an in-flight CSS animation,
so this leans on exact structural parity with the already-shipped
Shell.css pattern for the animation's own correctness.
The option's description claimed declaring an entry from the service's
own module put 'the scraper and the target on the same host by
construction rather than by luck'. It does not. It constrains where the
target is; nothing in it places the collector, and the two enable flags
are co-located by a shared lib.mkDefault rather than by construction.
Split across hosts, a target is silently never scraped — the service's
host declares an entry no local collector reads, the collector's host
never enabled the service. No error surfaces, and no assertion can
catch it: separate hosts are separate evaluations with no shared
context, so the doc telling the truth is the only mechanism there is.
The same paragraph already warned co-location was not a guarantee, four
lines below the sentence claiming it was; a reader arriving for
permission stopped at the permission. This one did.
swarm-nats carries the concrete caveat for its own contribution.
NATS has no Prometheus format of its own. It serves a JSON monitoring
endpoint, and prometheus-nats-exporter translates that — so this is two
changes in order, not one: without the monitoring endpoint the exporter
starts cleanly and scrapes nothing, which is the inert-config shape the
scrape work exists to avoid.
Both listeners are loopback and the exporter is the monitoring
endpoint's only intended reader: it is unauthenticated and /connz names
every connected client, so the address it binds is the whole access
control.
The scrape target is declared here rather than in the collector's
module, gated on a collector existing to read it — an entry exists only
where the service that named it runs.
mara, on review: "i thought we just swap a css file via nginx
config?" -- right instinct. The package-copy overlay (cp -r + install)
only made sense for hive-c0re's servedFrontend, which backs multiple
serve points (dashboard root + every per-agent gateway route) from one
swapped tree. swarm-ui has exactly one location serving cfg.package, so
an = /static/colors.css exact-match override -- the same idiom every
other single-path override on this vhost already uses (/api/whoami,
/api/docs) -- replaces the whole derivation with one location block.
Verified with the same throwaway nixosSystem eval as the previous
commit: unthemed case has no colors.css location and the / root is
cfg.package unchanged.
mara: "swarm dash is in my own colors, but tab colors on swarm ui are
the default catpucchin one." Root cause: hive-c0re/theme.nix's stylix
overlay only ever wrote a themed colors.css onto the dashboard/agent
frontend subtrees -- swarm-ui, served from its own separate package,
was never in scope.
Extracts the stylix-detection + colors.css-generation logic (previously
inline in theme.nix) into a shared nix/host-modules/stylix-theme.nix,
imported by both theme.nix and swarm-ui.nix -- one source instead of a
second copy that has to agree by inspection. swarm-ui.nix gains its own
themedPackage overlay (same shape as theme.nix's themedFrontend: copy
the package, overwrite static/colors.css) and serves that instead of
cfg.package directly when stylix is active; a clean passthrough
otherwise.
Verified with a throwaway nixosSystem eval (this repo's own flake
checks do not exercise gateway-module wiring) confirming the unthemed
path resolves cfg.package unchanged.
The scrape config names a client_secret_file; this is what puts a file
there. A host oneshot copies authelia's minted secret between the two
container trees — a copy and not a bind mount, because the secret does
not exist until authelia's first boot and nixos-container refuses to
start on a missing bind source, which on a fresh swarm is a permanent
stall presenting as broken metrics.
The collector runs under DynamicUser and the prometheus receiver opens
client_secret_file itself, as that user, so there is no uid to hand the
file to. LoadCredential reads it as root before the sandbox exists and
re-exposes it under a path that does not depend on which uid the unit
got; the scrape config points there. Both spellings derive from one
binding, since a mismatch is a file the collector cannot open and
nothing but a runtime 401 would say so.
The published targets now render as prometheus scrape jobs, so the
option's name is true: the collector reaches them by name over https,
using prometheus-native oauth2 with the audience set to the target's own
url. Registered is not the same as requested — a client that does not
ask for an audience gets a token with an empty one however complete its
registration looks.
The url is split into scheme, target and metrics_path rather than asked
for three times: two spellings of one address is a mismatch waiting to
happen, and the failure it produces is a valid token refused at the
target. A malformed or non-https url is an assertion rather than a null
dereference from inside the renderer.
`collectorAudiences` was a list of URLs services contributed so the
collector's client could be registered for them. Slice C needs the same
URLs as scrape jobs, and a job needs a name the audience list has no
room for — so services would have contributed to two options that must
agree.
They now contribute `publishedScrapeTargets` once, as `<job> = "<url>"`,
and the client's audiences derive from it. The drift that would have
needed maintaining is gone, and its failure mode was the quiet one: a
target whose audience was forgotten authenticates against nothing and
reads as a broken scrape rather than a missing registration.
Adds an assertion for the one collision the module system cannot catch.
Two definitions of the same key within one option are already refused
(measured); across the two scrape options nothing arbitrates, and both
entries would render into a single scrape_configs list under one
job_name.
Review found that nothing stopped `bearerAuthz = true` on an interactive
client. `renderClient` reads the flag only in the machine branch, so the
scope is never emitted: the client authenticates, is authorised for
nothing, and the build is green. `kind` defaults to `interactive`, so it
is reached by forgetting a field rather than by writing a wrong one —
this module's own failure mode one level up.
The other half of the review asked to relax the method assertion to
accept `null`, on the strength of the option's doc calling `null`
authelia's default. Measured instead: under `authelia.bearer.authz`
authelia refuses the omission outright, so the assertion was right and
the DOC was wrong. The doc now carries the exception, and the assertion
message says null is refused rather than leaving a reader to infer it.
The forge's `/metrics` is published behind the gateway and denied to
everyone, waiting on a client to allow. This is that client.
An audience is a URL: authelia validates a bearer token against the
address being requested, and a client may only request an audience it is
registered for, so registration is the authorisation. The URLs are owned
by the services that publish them while audiences attach to one client,
so services contribute to a list and this module builds the single entry
— the `gateway.localNames` split, forced here by client definitions
concatenating rather than merging into a shared entry.
The access-control rule asks the client list whether the collector is
registered rather than re-deriving the conditions that register it. The
two drifting is not a build failure: authelia refuses a rule naming an
unknown client in its startup validator, so SSO fails to restart.
A scraper reaches a service published behind the gateway by presenting an
access token to authelia's authz endpoint, which requires the client to
carry authelia.bearer.authz. Nothing could express that: scopes are
derived from kind, and a machine client rendered an empty list.
A named capability rather than a free-form scopes list, for the reason
the derivation exists — authelia refuses some scope/grant combinations
outright, openid with client_credentials among them, and a list makes
those expressible again.
The two assertions carry their weight: authelia checks the same
obligations, but in its preStart validator, so a violation builds and
deploys cleanly and then fails to restart with swarm SSO attached to it.
The collector names components `<kind>/<owner>` — a hive name for the
per-hive pipelines, the literal `swarm` for the swarm tier's own. Both
land in one attrset via `//`, so a hive named `swarm` replaced the swarm
tier's parts and lost its own: its receiver kept accepting pushes into a
pipeline that routed nowhere, and its samples lost the `hive` stamp that
makes attribution unforgeable. Zero failed assertions.
The reserved name is now bound once and interpolated at each swarm-tier
use, so the guard checks the same string the config emits rather than a
copy of it. A second swarm-tier pipeline joins the list and inherits the
check without touching the assertion.
There are two otel options one word apart: swarm.otel is the swarm's
collector, hyperhive.otel is the per-hive tier that ships upstream and
never reads scrapeTargets. The gate went through a let binding declared
800 lines from its use, so which one it referred to was not visible where
it mattered — mara had to ask.
Gating the wrong one is not a build error. It is a target that is either
always declared or never declared, and both look like working config.
The comment claimed the option's rule keeps scraper and target on one host
by construction. It does not. Both services default from
enableRequiredServices via mkDefault, which is an invitation to override
rather than a guarantee, so co-location is a property of the auto-deployed
topology and not of the module.
Gating the target on the collector's own enable makes the loopback address
honest: a host running authelia without a collector no longer declares a
target nothing can read. That absence was the part worth fixing, because it
is silent — no error, no metrics, nothing in a log to notice.
This does not make authelia scrapeable from another host. That needs the
endpoint published under a name with a certificate and an audience, which
is separate work; the option's docs now say so where someone splitting the
two would read it.
The endpoint was off, so nothing reported on the swarm's own SSO. Enabling
it alone would have added no data — the scraper that reads it only landed
with the swarm-tier prometheus receiver.
Loopback only, like the main listener and for a stronger reason: this
endpoint authenticates nothing and reports request volumes and outcomes for
every login on the swarm.
metricsPort is an option rather than a literal because every swarm container
shares the host netns, so two services picking the same port do not conflict
at build time — one loses at runtime with nothing in any log. 9959 is
upstream's default and is unclaimed across nix/.
mara: "should be https://auth.constellation.darkest.space/settings in
profile pic menu" -- the UserMenu link was pointing at the plain
authelia domain root, which lands on the portal rather than the
account settings page. Appends /settings client-side, same base-URL
source as before (GET /api/links Authelia entry).
Two review findings, folded together.
mara: a swarm integrated auto deployed forge always has metrics, so the
toggle is gone. The endpoint follows behindGateway instead, which is the
swarm-integrated shape and the condition the protected location lives
under. Serving it without that location would put it on a listener
openFirewall can expose with nothing in front.
argus: /metrics matched no access_control rule, so default_policy
one_factor governed it. That is any authenticated subject, which today
means any operator and tomorrow any agent. My audience argument covered
the Bearer path only; the same endpoint also accepts CookieSession, and a
cookie carries no audience at all, so the audience was never what stood
between a browser session and this data.
The rule is deny rather than a client-scoped allow because the collector's
client does not exist yet. Authelia refuses a subject naming an
unregistered client, and does so in a preStart validator rather than at
build time, so naming one early yields a green nixos-rebuild and dead
swarm SSO on the next restart. Denying until the client is registered
makes publishing the endpoint safe on its own; registering it is a
one-line change from deny to that allow.
Rule order is load-bearing: authelia takes the first match.
The gateway authenticates scrapers so services do not each grow a static
bearer of their own, but authelia's auth_request endpoint ran its default
strategies, which are cookie-only. A scraper's OAuth2 access token was
refused no matter how it was minted.
CookieSession is listed explicitly because authn_strategies replaces the
defaults rather than extending them. Omitting it evaluates, renders and
starts, and silently ends every operator session on the swarm UI, which
uses this same endpoint.
Unconditional rather than keyed to whichever service is scraped today:
this makes a scheme available, not an authorisation. Authelia refuses a
token carrying no audience for the requested URL and only issues a client
audiences it is registered for, so nothing passes until a client is
registered against a specific URL.
Forgejo can serve prometheus metrics but nothing turned them on, and
turning them on alone would have published them: forgejo serves
`/metrics` on its normal listener and the gateway vhost proxies `/` to
that listener, so the existing catch-all would have carried the endpoint
to anyone. The option is therefore one switch for both halves, and the
`= /metrics` location is an exact match so it outranks that prefix.
Authentication is the gateway's rather than forgejo's own `[metrics]
TOKEN`: a scraper presents an audience-scoped authelia token which nginx
checks via auth_request, so the swarm keeps one identity system instead
of gaining a static bearer per service.
The subrequest deliberately omits the `error_page 401 =302` that
swarm-ui uses. That redirect sends a browser to a login page; a scraper
would follow it and parse HTML as metrics.
Mechanical addition to the existing bulk-action framework
(renderSelectionBar/addBulkButton, swarm.js) -- two per-agent actions
already exist (POST /api/pause/, /api/resume/, see hive-agent-menu.js),
this just gives the selection bar the same all-or-nothing enablement
rule the other six bulk buttons already use (pause enabled only when
none of the selection is already paused, and vice versa for resume).
.btn-pause reuses .badge-paused's yellow so the trigger and the
resulting state pill read as one colour; .btn-resume is green like
.btn-start (both are "go" actions). Docs updated to list both in the
Selection bar reference.
Table gains an optional emptyMessage prop, rendered as a single
full-width row in place of a bare empty tbody; AgentsPage is the first
consumer. Screenshot-verified against a mock server returning an empty
roster.
No sitewide box-sizing: border-box reset exists, so .ui-dialog's
padding + border were adding on top of width: 90vw instead of being
carved out of it -- 401px rendered against a 390px viewport, real
horizontal overflow. Caught via a real 390x844 screenshot per mara's
"phone size?" ask on the PR, not assumed.
Three doc comments elsewhere in the tree still named the old
CreateAgentPage identifier/page framing after the rename in this
branch -- TextField.tsx, Panel.tsx, FormField.css (x3). None of these
files are touched by the rest of the diff, which is exactly how the
staleness happened.
Adds a generic ui/dialog/Dialog primitive (native <dialog>, no
third-party modal lib and no shadow-DOM custom element per the esbuild
gap on those) and wires a "+ agent" button into AgentsPage that opens
the existing create-agent form inside it, content unchanged from its
former life as a standalone /create-agent route/nav item.
Removes the "new agent" top-level nav entry and the /create-agent
route entirely -- creation now only reachable from the roster that
gets populated by it. CreateAgentPage.tsx/css renamed to
CreateAgentForm.tsx/css to match its new role as a mounted component
rather than a page.
Screenshot-verified the dialog open/closed states against a mock
server.
Adds an "/api/whoami" same-origin nginx proxy to authelia's own
GET /api/user/info (session-cookie authenticated, no swarm-controller
code needed) and a new UserMenu header component: a generated initials
avatar (first letter of display name, coloured from the same seven
base16 chromatic slots the nav accent already cycles through) opening a
popover with the signed-in name, a link to authelia settings, and log
out — both reusing the existing "Authelia" entry from GET /api/links
rather than a second source of the domain.
Per mara's call on the open avatar-mechanism question: initials now,
a real uploaded photo (authelia's settings UI implies pics are
settable) is an explicit future item, not blocking this.
Dashboard 2 of the set the operator asked for: how Claude is being used
rather than which agent is using it, so the axes are model, effort, token
type and query source. The only deliberate overlap with the agents page
is the cost/token headline.
Every panel was run against the live store before this landed. Three
panels were dropped rather than shipped, because their label has exactly
one live value today and a page of single-bar charts is the same silent
failure as an empty one.
Turn count and turn length are PROXIES and say so in their descriptions:
nothing exports turn stats, so a session record stands in for a turn,
which holds because each turn runs a new claude process.
New AgentsPage at /agents: fetches GET /api/agents (roster names) and
GET /api/config-prs (bulk config-PR status) and merges them into one
table, one row per agent. Reuses the existing Panel/Table/StatusChip/
RefreshIntervalPicker components exactly as HivesPage does — the roster
page and the config-PR panel turned out to be the same page rather than
two separate pieces of UI.
Adds a nav entry (green accent, the next unused base16 chromatic slot)
between hives and new agent.
The operator asked for this when the dashboard was first reviewed and it was
deferred, not declined: the container metric family carried no hive label, so
selecting a hive emptied every container panel while the "All" default hid the
problem completely -- a `.*` matcher matches series where the label is absent.
That family now carries hive and swarm, verified against the live store rather
than inferred from the fix having merged: a hive selection returns the same 7
agents as the All default, at every window out to 168h, with a nonexistent
hive returning zero.
The agent list chains off the hive selection, so picking a hive narrows the
agent dropdown rather than leaving entries in it that resolve to nothing.
