vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

forge: hand the OIDC client secret to forgejo from /run, not from the container's disk

Browse source 
swarm-bao-forge-oidc fetched the forge's OIDC client secret from bao and wrote
it to /var/lib/nixos-containers/hive-forge/var/lib/forgejo-oidc/<id>.secret,
on the container's persistent root, where it outlived a bao rotation and was
in every backup of that tree.

Now the fetch writes it to /run/hive-forge-oidc/<id>.secret on the host
(tmpfs, root 0400). The directory is bound read-only into the container at
the same path, and forgejo-sso-source loads it through LoadCredential. A
tmpfiles rule creates the directory at boot, so the bind source always
exists, and removes the old persistent copy.

After a reboot the file is absent until the fetch succeeds, and
forgejo-sso-source fails to start until then; the fetch starts it again once
it writes the file. Logins keep working meanwhile: forgejo holds the secret
in its login-source row.

deploy.forgejo.sso.clientSecretFile (and its old swarm.forge.sso name) is
removed: it only chose where in the container the fetched file landed. The
fetch no longer needs the container's forgejo uid, so it no longer fails on a
fresh host before the container has created its state dir.

Part of #4998.
This commit is contained in:
atlas 2026-10-15 19:24:23 +02:00 • committed by mara
commit 09ad7386b2
8 changed files with 106 additions and 130 deletions
Download patch file Download diff file Expand all files Collapse all files

14
docs/swarm/secrets.md
View file

@ -248,13 +248,13 @@ such as the root token.
## Hive-level — one of each per hive
| secret | generated by | lives at |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------- |
| hive leaf certs | `hive-tls.nix`, issued by the secret store's `pki` mount | `<deploy.hive-controller.tls.stateDir>/<name>.pem` |
| the hive's own store identity | `glue-bao-tls.nix` on the store's host; placed by hand on every other host ([`ca.md`](ca.md#the-gateway-leaf-comes-from-the-secret-store)) | `deploy.hive-controller.tls.baoHiveIdentityCertFile` / `baoHiveIdentityKeyFile` |
| the forge's copy of its OIDC secret | `swarm-bao-forge-oidc.service` reads it out of the swarm secret store under the forge publisher's leaf, wherever authelia runs | `/var/lib/forgejo-oidc/<id>.secret` inside the forge container |
| the homeserver's copy of its OIDC secret | `swarm-bao-matrix-oidc.service` reads it out of the swarm secret store under matrix-ctl's leaf, wherever authelia runs | `/var/lib/tuwunel-oidc/<id>.secret`, handed to tuwunel through `LoadCredential` |
| the agent containers' queue credential | authelia, published to the store by `swarm-secret-publish` | `<deploy.hive-controller.queue.agentCredentialDir>/secret` (`0600`) and `/client_id` (`0644`) |
| secret | generated by | lives at |
| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| hive leaf certs | `hive-tls.nix`, issued by the secret store's `pki` mount | `<deploy.hive-controller.tls.stateDir>/<name>.pem` |
| the hive's own store identity | `glue-bao-tls.nix` on the store's host; placed by hand on every other host ([`ca.md`](ca.md#the-gateway-leaf-comes-from-the-secret-store)) | `deploy.hive-controller.tls.baoHiveIdentityCertFile` / `baoHiveIdentityKeyFile` |
| the forge's copy of its OIDC secret | `swarm-bao-forge-oidc.service` reads it out of the swarm secret store under the forge publisher's leaf, wherever authelia runs | `/run/hive-forge-oidc/<id>.secret` on the host (tmpfs), bound into the forge container, handed to `forgejo-sso-source` through `LoadCredential` |
| the homeserver's copy of its OIDC secret | `swarm-bao-matrix-oidc.service` reads it out of the swarm secret store under matrix-ctl's leaf, wherever authelia runs | `/var/lib/tuwunel-oidc/<id>.secret`, handed to tuwunel through `LoadCredential` |
| the agent containers' queue credential | authelia, published to the store by `swarm-secret-publish` | `<deploy.hive-controller.queue.agentCredentialDir>/secret` (`0600`) and `/client_id` (`0644`) |
The forge and the homeserver read their copies out of the store, the way
Grafana does below: the forge under the forge publisher's leaf, the

16
docs/swarm/sso.md
View file

@ -126,8 +126,8 @@ first boot, and `swarm-secret-publish` on its host writes it to the swarm
secret store at `swarm/services/<client id>/oidc/client`. A host-side unit
then reads it from the store and writes it into the service's container:
`swarm-bao-forge-oidc` for the forge, under the forge publisher's store
leaf, into `/var/lib/forgejo-oidc/<client id>.secret` (the file
`forgejo-sso-source` reads); `swarm-bao-matrix-oidc` for matrix, under
leaf, into `/run/hive-forge-oidc/<client id>.secret` on the host, which
`forgejo-sso-source` loads as a credential; `swarm-bao-matrix-oidc` for matrix, under
matrix-ctl's. A unit that can't read the secret yet fails and retries
every 30 seconds for a day, so there is no start order to get right. The
service's own module contributes its client entry — callback URL
@ -141,10 +141,14 @@ rejected login with no error text worth reading.
<!-- vale write-good.Passive = NO -->
⚠️ The secret is written into the container's state directory, not bound
in with `bindMounts`: nixos-container refuses to start a container whose
bind source is missing, and this secret doesn't exist until authelia's
first boot has run and the publisher has written it.
⚠️ nixos-container refuses to start a container whose bind source is
missing, and this secret doesn't exist until authelia's first boot has run
and the publisher has written it. The forge therefore binds the **directory**
`/run/hive-forge-oidc`, which a tmpfiles rule creates at boot, never the
file. The directory is tmpfs, so the secret is gone after a reboot until
the fetch runs again; forgejo keeps it in its login-source row, so logins
still work meanwhile. Matrix's secret is written into the container's state
directory instead.
<!-- vale write-good.Passive = YES -->