vinzenz/hyperhive
vinzenz/hyperhive
Watch
0
0
Fork
You've already forked hyperhive
 0
Code Activity

hive telemetry collector: fetch its OIDC secret from the store under the hive's own leaf

Browse source 
The hive collector's `oauth2client` secret came from a `clientSecretFile`
defaulting to authelia's own minted file on the IdP's host, with a
`hive-otel-client-secret` oneshot that only waited up to 120s for that
file to exist; any other hive had to copy the file across by hand.

`swarm-bao-hive-otel-oidc` now logs in with the hive's leaf
(`deploy.bao.clientCertFile`), reads `swarm/hives/<hive>/oidc/client`
(written by the publisher, granted by the hive leaf's
`swarm/hives/<hive>/*` read), writes it root 0400 when it differs, and
restarts the collector when the file is newer than its last start.
Retries every 30s for 24h (`lib/store-retry.nix`), like the matrix and
forge fetches. `clientSecretFile` defaults to that file wherever the hive
leaf is set and stays as an override; an overridden file renders no fetch.

otelcol-contrib 0.151.0's oauth2client re-reads `client_secret_file` on
every token fetch, but the path is `LoadCredential`'s start-time copy, so a
changed secret still needs a restart. `lib/refresh-consumer.nix` gains
`refresh_unit` for a host unit; `refresh_consumer` now calls it with
`--machine`.

nats-authelia's "consumer of the host client-secret dir" case pointed at
the collector's old default; it now reads the publisher's path unit.

Refs #5015
This commit is contained in:
atlas 2026-10-10 01:09:20 +02:00
commit 7c50568db1
7 changed files with 282 additions and 114 deletions
Download patch file Download diff file Expand all files Collapse all files

3
docs/getting-started/setup.md
View file

@ -9,8 +9,7 @@ Every command runs as **root on the host**.
> **Not all-local?** Each step says which host it runs on. A split swarm
> also has credentials you can't generate where they're read — each
> host's mTLS identity at the secret store, and each hive's telemetry
> ingest secret. Read [`swarm/secrets.md`](../swarm/secrets.md) first; it
> host's mTLS identity at the secret store. Read [`swarm/secrets.md`](../swarm/secrets.md) first; it
> says which files you place, and where.
## 1 · Secret store

15
docs/scheduler/observability.md
View file

@ -129,11 +129,18 @@ from whichever receiver accepted a sample. A hive therefore can't report
metrics as another hive, and can't relabel its own by editing what it sends —
the label isn't taken from the payload at all.
**On an all-local swarm there is nothing to set.** Each hive already has an
identity, and its collector reads the secret that host's own authelia minted.
**Nothing to set where the hive holds its store identity.** The hive's collector
reads its secret from the swarm's secret store under the hive's own leaf
(`deploy.bao.clientCertFile`): `swarm-bao-hive-otel-oidc.service` fetches
`swarm/hives/<hive>/oidc/client`, which the publisher on authelia's host
writes. It runs at boot and retries every 30s until the store answers; when a
run lands a changed value it restarts the collector. It doesn't run on a timer,
so a secret rotated later arrives the next time the unit runs. On a hive that
runs the store the leaf is already there; elsewhere it's the credential you
place — see [secret placement](../swarm/secrets.md).
**On a hive that doesn't host the swarm's services**, the secret has to arrive
somehow — copy it across and name it:
To deliver the secret some other way, name the file, and the fetch unit isn't
rendered:
```nix
services.hyperhive.otel.clientSecretFile = "/run/secrets/hive-telemetry.secret";

20
docs/swarm/secrets.md
View file

@ -47,12 +47,13 @@ neither is a renaming of the other.
| authelia session, JWT and storage-encryption keys | authelia's first-boot unit, in-container | `/var/lib/authelia-swarm/{session,jwt,storage-encryption}.key` | generated in place; nothing outside that container reads them |
| authelia OIDC HMAC key | same unit | `/var/lib/authelia-swarm/oidc-hmac.key` | same |
| authelia OIDC issuer key (RSA) | same unit | `/var/lib/authelia-swarm/oidc-issuer.key` | same — relying parties verify against the **public** half at `/jwks.json` |
| OIDC client secret, plaintext half | `authelia crypto hash generate --random` | `/var/lib/authelia-swarm/oidc-clients/<id>.secret` | operator provides the file and names it in whichever option reads it — `sso.clientSecretFile` for a service, `otel.clientSecretFile` for the hive's telemetry collector |
| OIDC client secret, plaintext half | `authelia crypto hash generate --random` | `/var/lib/authelia-swarm/oidc-clients/<id>.secret` | operator provides the file and names it in whichever option reads it — `sso.clientSecretFile` for a service |
| OIDC client secret, digest half | the same mint | `oidc-clients/<id>.digest` | authelia's own half; merged at runtime via `settingsFiles` |
| the swarm collector's copy of its OIDC secret | `swarm-bao-otel-oidc.service` reads it out of the swarm secret store, **on every host that runs the collector and holds a store identity** | `/var/lib/swarm-otel-oidc/<id>.secret` inside the `swarm-otel` container | same unit, same path — one route, co-located or not. A collector with no store identity has `clientSecretFile == null`, its already-supported unauthenticated-push degrade — see below |
| Grafana's copy of its OIDC secret | `swarm-bao-grafana-oidc.service` reads it out of the swarm secret store, **on every host that runs Grafana** | `/var/lib/grafana-oidc/<id>.secret` inside the `swarm-grafana` container | same unit, same path — one route, co-located or not. Nothing for an operator to place beyond this host's store leaf, see below |
| the swarm controller's copy of its OIDC secret | `swarm-controller` reads it out of the swarm secret store at start, under its own leaf | the daemon's memory only; never on disk | same read, co-located or not. Nothing for an operator to place beyond the controller's store leaf (`deploy.swarm-controller.baoClientCertFile`) |
| a hive's copy of its own OIDC secret | hive-c0re reads it out of the swarm secret store when it first connects to the queue, under the hive's own leaf | the daemon's memory only; never on disk | same read, co-located or not. Nothing for an operator to place beyond the hive's store leaf (`deploy.bao.clientCertFile`) |
| the hive collector's copy of its hive's OIDC secret | `swarm-bao-hive-otel-oidc.service` reads it out of the swarm secret store under the hive's own leaf, **on every hive that runs the collector and holds that leaf** | `/var/lib/hive-otel-oidc/hive-<name>.secret` on the hive's host, 0400 root, loaded by `LoadCredential` | same unit, same path, co-located or not. Nothing for an operator to place beyond the hive's store leaf (`deploy.bao.clientCertFile`) |
| the swarm controller's forge token | the forge's `forgejo-swarm-controller-account`, in-container, once | `/var/lib/forgejo/swarm-controller-token` inside the forge container, the minter's record that it has minted; `swarm-bao-forge-publisher` copies it into the swarm secret store at `swarm/controller/swarm-controller/forge/token` each time the forge container starts and each time the mint writes the file; the controller holds it in memory only | nothing to place beyond the controller's store leaf. A forge host without the store needs the publisher's leaf placed (`deploy.bao.forgePublisherClientCertFile`) |
| the queue responder's copy of its OIDC secret | the responder reads it out of the swarm secret store itself as it starts, under its own leaf, and again when authelia refuses it | the daemon's memory only; never on disk | same read, wherever authelia runs. Nothing for an operator to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) |
| authelia subject store | `swarmctl` and `swarm-authelia-bridge` | `users.yml` — one file, read and written by both | `swarmctl`, on the host that runs authelia |
@ -67,16 +68,17 @@ because nothing outside that container ever reads them. **That's the test worth
applying to any secret added here** — and the client secret's plaintext half is
the one row that fails it, which is the entire reason a delivery step exists.
Three telemetry collectors exist, and they don't land on the same side of
that test.
Three telemetry collectors exist, and each one fails that test: each reads its
own secret out of the store, under its own identity.
The **hive's** collector needs no delivery step. It authenticates to the swarm's
collector as its own hive, and it's a host unit rather than a container, so on
an all-local swarm it reads authelia's file where it lies and nothing makes a
second copy. On any other topology it's an ordinary "operator provides the file"
case — see `services.hyperhive.otel.clientSecretFile`.
The **hive's** collector authenticates to the swarm's collector as its own hive,
so it reads its secret as the hive: `swarm-bao-hive-otel-oidc.service` fetches
`swarm/hives/<hive>/oidc/client` under the hive's own leaf
(`deploy.bao.clientCertFile`), on every topology. A hive with no leaf has no
default, and the build stops at the collector's identity assertion unless
`services.hyperhive.otel.clientSecretFile` names a file delivered some other way.
The **swarm's** collector does need one, and its delivery route is the fourth
The **swarm's** collector reads its own under a leaf of its own, the fourth
store reader — the same shape as Grafana's, described in full below. In short:
`swarm-bao-otel-oidc.service` reads `swarm/services/<client-id>/oidc/client`
out of the store on every host that runs the collector and holds a store