Forgejo can serve prometheus metrics but nothing turned them on, and
turning them on alone would have published them: forgejo serves
`/metrics` on its normal listener and the gateway vhost proxies `/` to
that listener, so the existing catch-all would have carried the endpoint
to anyone. The option is therefore one switch for both halves, and the
`= /metrics` location is an exact match so it outranks that prefix.
Authentication is the gateway's rather than forgejo's own `[metrics]
TOKEN`: a scraper presents an audience-scoped authelia token which nginx
checks via auth_request, so the swarm keeps one identity system instead
of gaining a static bearer per service.
The subrequest deliberately omits the `error_page 401 =302` that
swarm-ui uses. That redirect sends a browser to a login page; a scraper
would follow it and parse HTML as metrics.
Last of the three modules that hand-rolled the same concat with wantedBy +
before and no requires, so a failed assembly left the consumer running against
a missing file and trusting nothing -- every outbound TLS call fails while the
unit looks healthy.
The forge is the one with two consumers: forgejo-sso-source fetches the
issuer's discovery document over the swarm CA and once shipped without the
trust its sibling had. It only exists when SSO is on, so the consumer list is
conditional -- naming an absent unit would define a serviceless one and order
nothing.
Removes the now-dead useSelfSigned and caContainerPath bindings (nix does not
warn) and retargets three comments the deletion orphaned, including the helper
header that still named this module as the per-call-site concat.
Three host units poll up to 120s for a secret authelia mints on its first
boot, and all three are `Type=oneshot` with no `TimeoutStartSec`. systemd's
`DefaultTimeoutStartSec` is 90s, so it kills them at 90 — before the script
reaches its own `exit 1` and names the file that never appeared.
The wait itself is fine; what's lost is the diagnosis. On a fresh hive the
operator gets a bare start-timeout instead of "authelia has not minted
<path>", several layers from the container that was actually slow.
Found while writing the same unit for Grafana, where the timeout is set —
so this is the existing three catching up with it, not a new pattern.
Registering the OIDC login source makes an outbound HTTPS call - the CLI
fetches <issuer>/.well-known/openid-configuration to validate the
provider before writing the row. That URL is a swarm service name served
under the swarm CA, which the default system store has never heard of.
SSL_CERT_FILE was set on forgejo.service and not on
forgejo-sso-source.service, so the web service trusted the chain and the
registration one-shot did not. Same binary, same host, different unit.
The result was a 100% reproducible failure that no restart could fix:
nothing about restarting a unit gives it an environment variable it
never had. Every recorded run since the feature landed has failed with
x509: certificate signed by unknown authority.
The trust belongs to every process that makes the call, not to the
service that happens to be the obvious consumer.
New `services.hyperhive.swarm.controller.links` option (listOf {label,
icon, url}, same shape as the per-agent hyperhive.dashboardLinks) plus
a new GET /api/links route serving it, same pattern as the existing
hives/GET /api/hives.
Rather than one central hardcoded list, each service's own module
contributes its own entry when actually enabled on the controller's
host: swarm-authelia.nix, hive-matrix.nix (gated on gui.enable too,
since / on that vhost only serves fluffychat then) and
hive-forge/default.nix (gated on behindGateway) each push one entry,
the same list-merge idiom services.hyperhive.gateway.localNames
already uses. swarm-ui.nix contributes a static entry for its own
same-origin swagger docs. Adding a future service's link is a nix-only
change to that service's own module.
Verified: cargo build/clippy/test -p swarm-controller clean, a
throwaway nixosSystem eval confirms all 4 entries merge correctly into
SWARM_CONTROLLER_LINKS, nix build .#swarm-controller succeeds.
Moves `forgeVhost` out of the gateway's vhosts.nix and the forge's
`address=` rule out of dnsmasq.nix, into nix/host-modules/hive-forge —
the module that already owns everything else about the forge.
The gateway keeps what is gateway knowledge (the listen set, which
issuer covers a name, the header block) and loses the last reason it
had to read `swarm.forge` at all: `forgeCfg` is gone from both files
and from the module's `let`.
Both halves stay gated on `behindGateway` — with it off the operator
fronts forgejo themselves, so this hive must neither claim the vhost nor
answer DNS for the name.
The login source still failed after the argv fix, with
dial tcp: lookup auth.constellation.darkest.space: no such host
The hive's dnsmasq is authoritative for the swarm service names, but
only containers whose resolv.conf points at the bridge ask it — agent
containers do, via an explicit unit written for that reason
(nix/agent-modules/network.nix). hive-forge resolves through the host's
resolvers instead, and the swarm domain has no public records, so
discovery fails for a name that resolves fine one container over.
Publish it in the container's own hosts file, mapped to 127.0.0.1:
sharing the host netns, loopback is the host, where nginx already serves
that vhost. TLS still validates - the CA trust bundle is bind-mounted
and the leaf covers the name. Gated on authelia being local, since a
remote provider's name belongs to another machine.
Gate (state/eval-3149-hosts.sh) asserts the RENDERED /etc/hosts rather
than the option, plus an absence probe with SSO off and a check that the
discovery URL names the same host the entry publishes.
The login source was never registered on any boot since it landed:
forgejo-sso-source[422]: Command error: unexpected arguments: profile, email, groups'
The flags were built as a shell STRING and word-split at the call site.
Splitting happens after quote removal, so the quotes inside the value are
just characters: --scopes 'openid profile email groups' reached forgejo
as four words, three of them unexpected and one carrying an apostrophe.
escapeShellArg interpolated into a string that is later word-split is a
no-op that looks exactly like protection.
An array carries the argument boundaries instead of re-deriving them from
whitespace, and the shellcheck disable goes with it.
Also assert the effect: the unit now fails if the source is absent from
'admin auth list' afterwards. The old failure exited non-zero and was
still invisible to every check that read the rendered script rather than
its result.
The all-local case from the delivery ruling: when one host runs both the
forge and the swarm's authelia, nothing should need an operator.
Two containers, one secret, and the awkward part is that they share this
host's network namespace but not its filesystem. They reach each other on
127.0.0.1, which makes them feel co-located — the forge still cannot open
a path inside authelia's tree. The host is the only place both are
addressable, so the copy runs there, and `hostClientSecretDir` publishes
the outside view of the inside path exactly as `hostUsersFile` already
does for the users database.
Deliberately a copy rather than a `bindMounts` entry. nixos-container
refuses to start when a bind source is missing, and this secret does not
exist until authelia's first boot has minted it — binding it would make
the forge wait on a file that waits on a container that starts after it.
On a fresh hive that is a permanent stall presenting as "the forge is
broken", several layers from its cause.
The owning uid is discovered from the forge container's own state dir
rather than assumed. Whatever uid maps to forgejo inside that container
already owns the directory it was created with; writing a number here
would be a second place for it to be wrong.
The client entry is contributed to authelia's list by the forge module
itself, from the same source-name constant the registration uses, so the
redirect URI authelia allows and the one forgejo sends cannot drift.
A mismatch there is a rejected login with no error text worth reading.
Additive, never exclusive: forgejo keeps its local password database
and gains a second way in. An identity provider that can take the forge
offline when it hiccups is a worse forge than one with two doors.
A login source in forgejo is a database ROW, not an `app.ini` key, so
this is a unit rather than config. It is ordered AFTER forgejo — unlike
its neighbour `forgejo-gpg-init`, which runs before — because on a fresh
hive that database does not exist until forgejo has started and
migrated; running first would either fail or initialise a schema behind
the server's back.
Idempotency is by query (`admin auth list`), not by a stamp file: the
same reasoning already written down for the GPG key next to it, that a
stamp outlives a state wipe and then suppresses the repair.
Two assertions rather than defaults, both firing at eval: SSO needs a
secret path, and it needs somewhere to discover the provider. Either
one missing produces a login button that always fails — a runtime
symptom several layers from its cause, which is exactly the trade an
eval error is worth making.
The secret is read from a path and passed on argv for one exec, because
`--secret` is the only input forgejo offers — no `--secret-file`, no env
var, though its sibling `forgejo-cli actions register` has both. Inside
this container the value is already at rest in the login-source row and
the only principals are root and forgejo, so argv widens its readership
to nobody new. Accepted deliberately, not overlooked.
A swarm runs one forge and every hive in it reaches the same host, so
the name belongs to the swarm rather than to whichever hive happens to
run it: `forge.<swarm.domain>` instead of `forge.<hive domain>`.
A deployment that was running before this keeps its current name with
one line -- `swarm.forge.domain = "forge.<hive domain>"` -- which is
exactly what the old default rendered, so pinning it is a no-op today
and freezes it against the new default.
Certificates follow either way: the swarm-services sub-CA is
name-constrained to the CONFIGURED names rather than to a fixed tree, so
a pinned legacy name is as issuable as the new default. DNS follows too
-- dnsmasq already lists `forge.domain` explicitly, precisely so a
cross-domain override stays routed.
The default stays total on a null swarm domain (`forge.invalid`) so the
required-domain assertion in hive-network.nix is what an operator sees,
rather than a coercion error naming this option.
The required-domain assertion in hive-network.nix could not be reached:
`forge.<domain>` and `matrix.<domain>` are evaluated while the
assertion list is, so an unset domain threw `cannot coerce null to a
string` naming one of those options instead of printing the message
that says which option to set.
Both defaults now fall back to a name under the reserved `.invalid`
TLD, which the assertion refuses to let out the door.
mara: "remove the extra deprecatios file, less comments". The alias now
sits in hive-forge/default.nix next to the options it renames, and the
comment is three lines instead of eighteen.
Better than tidier: there is no shared file for the sibling migrations
to queue behind. Matrix's rename goes in hive-matrix.nix, authelia's in
its own module, and none of them touch each other. hive-ci.nix declares
swarm.forge.ci.* from a different module and stays covered, because the
rename is on the namespace rather than on the declaring file.
Re-ran the check both ways this time, per argus on the last round: old
paths set and new paths read back 3999 / 7, and a config that only uses
the new paths produces no warnings at all.
Forgejo is a swarm-global service, so its operator-facing host options
move to services.hyperhive.swarm.forge (and .swarm.forge.ci) as the
first of the namespace consolidation.
Existing hive configs keep evaluating: swarm-renames.nix maps every
moved leaf with mkRenamedOptionModule, which also emits a deprecation
warning naming both the old and new path, so an operator is told what to
rename rather than discovering it from a failed eval.
The per-agent hyperhive.forge.url does NOT move. It is a client pointer
at whatever forge an agent talks to - it shares a word with the service
and nothing else, and the two are already documented as separate option
surfaces.
Verified by evaluating the host module, since no Rust gate evaluates
nix: setting the old paths and reading the new ones yields the values
(httpPort 3999, ci.concurrency 7), and config.warnings carries the
rename notice.
Adds services.hyperhive.forge.publicUrl (defaults to the gateway vhost
URL when behindGateway=true, null otherwise). HIVE_FORGE_PUBLIC_URL is
now sourced from it instead of hardcoding https://${forge.domain}
whenever behindGateway is on.
The 4 frontend call sites that built a forge link from
state.forge_public_url now hide the link when that's absent, rather
than guessing http://<browser-hostname>:3000 — a guess that's only
correct by accident once the operator isn't on plain localhost. Fixes
the dashboard H0M3 tile, per-agent-row forge links + agent menu, the
approval-queue PR link, and the per-agent page's own meta-nav forge
link (found during this pass, same defect, not in the original
3-site inventory).
Docs + doc-comments updated to match.
hive-ci and hive-forge both bind the runtime-generated hive CA cert
read-only and order their container@ unit after hive-tls-ca.service so
the bind source exists before nspawn sets the mount up — the same
bind-mount + ordering + rationale duplicated verbatim in two modules.
Extract that language-agnostic half into a pure helper,
nix/host-modules/lib/hive-ca-trust.nix, taking a container name and
returning { useSelfSigned, caContainerPath, bindMount, containerOrdering }.
The per-runtime consumption stays at each call site (hive-ci's additive
NODE_EXTRA_CA_CERTS, hive-forge's Go SSL_CERT_FILE concat). hive-ci folds
containerOrdering into its existing mkMerge alongside the TimeoutStartSec
bump.
The helper is a pure function, not a module: host-modules/default.nix is
an explicit aggregator (not a glob) and the docs eval imports that same
aggregator, so the lib/ file is never picked up as a module. A third
outbound-TLS-trusting container no longer means a third copy-paste.
Per mara's feedback on PR #2407 ("better: you can also provide url in
dashboard, same as with matrix, no host config"), drops
services.hyperhive.extraForges and the admin-API mint/revoke flow
entirely. The operator now creates a token on the external forge
themselves and pastes a label + base URL + access token into the
dashboard's FORGES tab, the same shape as the GitHub PAT flow plus the
base-URL field from the matrix extra-account flow. hive-c0re only ever
writes/deletes two local files per account (forge-<label>-token,
forge-<label>.json sidecar for the URL) via hive-priv — no remote
account creation, no admin token, no revoke-on-the-remote-side, no nix
config to enumerate.
- nix/host-modules/hive-forge/default.nix: removed the extraForges
option, its label-format assertion, and the HYPERHIVE_EXTRA_FORGES
env forwarding.
- hive-c0re/src/forge/extra.rs: deleted (REST admin-API provisioning,
no longer needed).
- hive-c0re/src/dashboard/extra_forges.rs: GET /api/extra-forges?
agent= lists an agent's stored forges by scanning its state dir
(mirrors matrix_accounts.rs's filename-scan listing), POST
/api/extra-forge-account (agent/label/base_url/token/
action=add|remove) stores or removes an account.
- hive-sh4re/priv_proto.rs + hive-priv/main.rs: new
WriteAgentExtraForgeAccount/DeleteAgentExtraForgeAccount priv
requests (adds base_url, writes/deletes a JSON sidecar alongside the
token).
- hive-c0re/src/priv_client.rs: matching wrapper functions.
- frontend/packages/dashboard/src/credentials.{html,js}: FORGES tab is
a per-agent list + add-account paste form (label/base_url/token), no
grant/revoke-from-catalog UI.
- docs/web-ui/dashboard.md: FORGES tab section rewritten.
Supersedes the design in PR #2407 (already approved+green on the old
admin-API model) — opening as a fresh PR against the same issues
rather than force-pushing over the approved one.
