grafana: mint the admin password into the store and apply it on every start
Grafana's built-in admin kept the nixpkgs default password `admin`, and Grafana's API takes basic auth whatever `auth.disable_login_form` says, so the public vhost gave server-admin to `admin:admin`. `swarm-bao-grafana-admin` creates a random password in the store under Grafana's existing leaf when none is stored (`create` without `update`, so the first one is kept), then copies it into the container. `security.admin_password` reads it for a new database, and a `grafana cli admin reset-admin-password` in `grafana.service`'s `preStart` applies it to an existing one on every start. Closes #5138
This commit is contained in:
atlas
mara
parent
cf4c4cde87
commit
993f5fd99d
7 changed files with 251 additions and 48 deletions
|
|
@ -41,27 +41,28 @@ neither is a renaming of the other.
|
||||||
|
|
||||||
## Swarm-level — one of each per swarm
|
## Swarm-level — one of each per swarm
|
||||||
|
|
||||||
| secret | generated by | lives at | hive elsewhere |
|
| secret | generated by | lives at | hive elsewhere |
|
||||||
| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
| --------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||||
| swarm-services root CA (cert + key) | `swarm-bao.nix`'s granting unit, inside the store's `pki` mount | the **key** never leaves the mount; the **cert** lands at `/var/lib/swarm-bao-services-pki/services-root.pem` (0644) and at `/var/lib/hive-tls/swarm-services-root.pem` on every hive | nothing to copy — every hive receives the cert with the leaf it asks the store for, or from `pki/ca/pem` where it fronts no swarm name and needs a store identity |
|
| swarm-services root CA (cert + key) | `swarm-bao.nix`'s granting unit, inside the store's `pki` mount | the **key** never leaves the mount; the **cert** lands at `/var/lib/swarm-bao-services-pki/services-root.pem` (0644) and at `/var/lib/hive-tls/swarm-services-root.pem` on every hive | nothing to copy — every hive receives the cert with the leaf it asks the store for, or from `pki/ca/pem` where it fronts no swarm name and needs a store identity |
|
||||||
| authelia session, JWT and storage-encryption keys | authelia's first-boot unit, in-container | `/var/lib/authelia-swarm/{session,jwt,storage-encryption}.key` | generated in place; nothing outside that container reads them |
|
| authelia session, JWT and storage-encryption keys | authelia's first-boot unit, in-container | `/var/lib/authelia-swarm/{session,jwt,storage-encryption}.key` | generated in place; nothing outside that container reads them |
|
||||||
| authelia OIDC HMAC key | same unit | `/var/lib/authelia-swarm/oidc-hmac.key` | same |
|
| authelia OIDC HMAC key | same unit | `/var/lib/authelia-swarm/oidc-hmac.key` | same |
|
||||||
| authelia OIDC issuer key (RSA) | same unit | `/var/lib/authelia-swarm/oidc-issuer.key` | same — relying parties verify against the **public** half at `/jwks.json` |
|
| authelia OIDC issuer key (RSA) | same unit | `/var/lib/authelia-swarm/oidc-issuer.key` | same — relying parties verify against the **public** half at `/jwks.json` |
|
||||||
| OIDC client secret, plaintext half | `authelia crypto hash generate --random` | `/var/lib/authelia-swarm/oidc-clients/<id>.secret` | operator provides the file and names it in whichever option reads it — `sso.clientSecretFile` for a service; the forge and, by default, the hive collector need none — `swarm-bao-forge-oidc` and `swarm-bao-hive-otel-oidc` fetch it from the store (below) |
|
| OIDC client secret, plaintext half | `authelia crypto hash generate --random` | `/var/lib/authelia-swarm/oidc-clients/<id>.secret` | operator provides the file and names it in whichever option reads it — `sso.clientSecretFile` for a service; the forge and, by default, the hive collector need none — `swarm-bao-forge-oidc` and `swarm-bao-hive-otel-oidc` fetch it from the store (below) |
|
||||||
| OIDC client secret, digest half | the same mint | `oidc-clients/<id>.digest` | authelia's own half; merged at runtime via `settingsFiles` |
|
| OIDC client secret, digest half | the same mint | `oidc-clients/<id>.digest` | authelia's own half; merged at runtime via `settingsFiles` |
|
||||||
| the swarm collector's copy of its OIDC secret | `swarm-bao-otel-oidc.service` reads it out of the swarm secret store, **on every host that runs the collector and holds a store identity** | `/var/lib/swarm-otel-oidc/<id>.secret` inside the `swarm-otel` container | same unit, same path — one route, co-located or not. A collector with no store identity has `clientSecretFile == null`, its already-supported unauthenticated-push degrade — see below |
|
| the swarm collector's copy of its OIDC secret | `swarm-bao-otel-oidc.service` reads it out of the swarm secret store, **on every host that runs the collector and holds a store identity** | `/var/lib/swarm-otel-oidc/<id>.secret` inside the `swarm-otel` container | same unit, same path — one route, co-located or not. A collector with no store identity has `clientSecretFile == null`, its already-supported unauthenticated-push degrade — see below |
|
||||||
| Grafana's copy of its OIDC secret | `swarm-bao-grafana-oidc.service` reads it out of the swarm secret store, **on every host that runs Grafana** | `/var/lib/grafana-oidc/<id>.secret` inside the `swarm-grafana` container | same unit, same path — one route, co-located or not. Nothing for an operator to place beyond this host's store leaf, see below |
|
| Grafana's copy of its OIDC secret | `swarm-bao-grafana-oidc.service` reads it out of the swarm secret store, **on every host that runs Grafana** | `/var/lib/grafana-oidc/<id>.secret` inside the `swarm-grafana` container | same unit, same path — one route, co-located or not. Nothing for an operator to place beyond this host's store leaf, see below |
|
||||||
| the swarm controller's copy of its OIDC secret | `swarm-controller` reads it out of the swarm secret store at start, under its own leaf | the daemon's memory only; never on disk | same read, co-located or not. Nothing for an operator to place beyond the controller's store leaf (`deploy.swarm-controller.baoClientCertFile`) |
|
| Grafana's admin password | `swarm-bao-grafana-admin.service` creates it in the swarm secret store at `swarm/services/<grafana client id>/admin/password` when the store holds none, then copies it out, **on every host that runs Grafana**. Grafana's leaf may create that leaf but not update it, so the store keeps the first value | `/var/lib/grafana-admin/password` inside the `swarm-grafana` container: `security.admin_password` for a new database, and `grafana cli admin reset-admin-password` in `grafana.service`'s `preStart` for an existing one, on every start | same unit, same path. Grafana doesn't start until the file exists, so it stays down rather than falling back to the upstream default `admin` |
|
||||||
| a hive's copy of its own OIDC secret | hive-c0re reads it out of the swarm secret store when it first connects to the queue, under the hive's own leaf | the daemon's memory only; never on disk | same read, co-located or not. Nothing for an operator to place beyond the hive's store leaf (`deploy.bao.clientCertFile`) |
|
| the swarm controller's copy of its OIDC secret | `swarm-controller` reads it out of the swarm secret store at start, under its own leaf | the daemon's memory only; never on disk | same read, co-located or not. Nothing for an operator to place beyond the controller's store leaf (`deploy.swarm-controller.baoClientCertFile`) |
|
||||||
| the hive collector's copy of its hive's OIDC secret | `swarm-bao-hive-otel-oidc.service` reads it out of the swarm secret store under the hive's own leaf, **on every hive that runs the collector and holds that leaf** | `/var/lib/hive-otel-oidc/hive-<name>.secret` on the hive's host, 0400 root, loaded by `LoadCredential` | same unit, same path, co-located or not. Nothing for an operator to place beyond the hive's store leaf (`deploy.bao.clientCertFile`) |
|
| a hive's copy of its own OIDC secret | hive-c0re reads it out of the swarm secret store when it first connects to the queue, under the hive's own leaf | the daemon's memory only; never on disk | same read, co-located or not. Nothing for an operator to place beyond the hive's store leaf (`deploy.bao.clientCertFile`) |
|
||||||
| the swarm controller's forge token | the forge's `forgejo-swarm-controller-account`, in-container, once | `/var/lib/forgejo/swarm-controller-token` inside the forge container, the minter's record that it has minted; `swarm-bao-forge-publisher` copies it into the swarm secret store at `swarm/controller/swarm-controller/forge/token` each time the forge container starts and each time the mint writes the file; the controller holds it in memory only | nothing to place beyond the controller's store leaf. A forge host without the store needs the publisher's leaf placed (`deploy.bao.forgePublisherClientCertFile`) |
|
| the hive collector's copy of its hive's OIDC secret | `swarm-bao-hive-otel-oidc.service` reads it out of the swarm secret store under the hive's own leaf, **on every hive that runs the collector and holds that leaf** | `/var/lib/hive-otel-oidc/hive-<name>.secret` on the hive's host, 0400 root, loaded by `LoadCredential` | same unit, same path, co-located or not. Nothing for an operator to place beyond the hive's store leaf (`deploy.bao.clientCertFile`) |
|
||||||
| the queue responder's copy of its OIDC secret | the responder reads it out of the swarm secret store itself as it starts, under its own leaf, and again when authelia refuses it | the daemon's memory only; never on disk | same read, wherever authelia runs. Nothing for an operator to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) |
|
| the swarm controller's forge token | the forge's `forgejo-swarm-controller-account`, in-container, once | `/var/lib/forgejo/swarm-controller-token` inside the forge container, the minter's record that it has minted; `swarm-bao-forge-publisher` copies it into the swarm secret store at `swarm/controller/swarm-controller/forge/token` each time the forge container starts and each time the mint writes the file; the controller holds it in memory only | nothing to place beyond the controller's store leaf. A forge host without the store needs the publisher's leaf placed (`deploy.bao.forgePublisherClientCertFile`) |
|
||||||
| authelia subject store | `swarmctl` and `swarm-authelia-bridge` | `users.yml` — one file, read and written by both | `swarmctl`, on the host that runs authelia |
|
| the queue responder's copy of its OIDC secret | the responder reads it out of the swarm secret store itself as it starts, under its own leaf, and again when authelia refuses it | the daemon's memory only; never on disk | same read, wherever authelia runs. Nothing for an operator to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) |
|
||||||
| wireguard private key | **the operator** — `wg genkey` | whatever `deploy.wireguard.privateKeyFile` names | always operator-provided; nothing generates this for you |
|
| authelia subject store | `swarmctl` and `swarm-authelia-bridge` | `users.yml` — one file, read and written by both | `swarmctl`, on the host that runs authelia |
|
||||||
| queue auth-callout nkeys (user seed + account seed) | `swarm-nats-auth ensure-callout-keys`, run by `swarm-bao-nats-callout-xkey` on the queue's host when the store holds no seed; the verb stores a seed it finds at `/var/lib/swarm-nats-callout/{callout-user,issuer}.seed` as is, then removes the file | the swarm secret store: the seeds at `swarm/services/swarm-nats/callout/{user,issuer}-seed`, read by the `swarm-nats-auth` role alone; the public halves at `swarm/public/swarm-nats/callout-{user,issuer}` | nothing to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) |
|
| wireguard private key | **the operator** — `wg genkey` | whatever `deploy.wireguard.privateKeyFile` names | always operator-provided; nothing generates this for you |
|
||||||
| queue auth-callout xkey (curve seed) | `swarm-nats-auth ensure-callout-keys`, run by `swarm-bao-nats-callout-xkey` on the queue's host when the store holds no seed | the swarm secret store: the seed at `swarm/services/swarm-nats/callout/xkey-seed`, read by the `swarm-nats-auth` role alone; the public half at `swarm/public/swarm-nats/callout-xkey` | nothing to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) |
|
| queue auth-callout nkeys (user seed + account seed) | `swarm-nats-auth ensure-callout-keys`, run by `swarm-bao-nats-callout-xkey` on the queue's host when the store holds no seed; the verb stores a seed it finds at `/var/lib/swarm-nats-callout/{callout-user,issuer}.seed` as is, then removes the file | the swarm secret store: the seeds at `swarm/services/swarm-nats/callout/{user,issuer}-seed`, read by the `swarm-nats-auth` role alone; the public halves at `swarm/public/swarm-nats/callout-{user,issuer}` | nothing to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) |
|
||||||
| the secret store's own contents | openbao, on first `bao operator init` — **an operator action, not a unit** | inside the `swarm-bao` container, at its own `/var/lib/openbao`, kept across rebuilds by `ephemeral = false`. ⚠️ **Not a host path**: `nixos-container destroy swarm-bao` takes the raft data with it, so back up the container's tree, not `/var/lib/`. Only the store's TLS material (`/var/lib/swarm-bao-tls`) and its PKCS11 token (`/var/lib/swarm-bao-token`) are host-level | n/a — there is one store; a hive elsewhere is a _client_ of it and holds none of this |
|
| queue auth-callout xkey (curve seed) | `swarm-nats-auth ensure-callout-keys`, run by `swarm-bao-nats-callout-xkey` on the queue's host when the store holds no seed | the swarm secret store: the seed at `swarm/services/swarm-nats/callout/xkey-seed`, read by the `swarm-nats-auth` role alone; the public half at `swarm/public/swarm-nats/callout-xkey` | nothing to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) |
|
||||||
| the secret store's unseal material | the HSM/TPM under `deploy.bao.seal = "pkcs11"`; openbao itself under `"shamir"` | in the token; or held by whoever ran `bao operator init`, which is what `"shamir"` means and why it's stated rather than inferred | n/a — only the host running the store seals anything |
|
| the secret store's own contents | openbao, on first `bao operator init` — **an operator action, not a unit** | inside the `swarm-bao` container, at its own `/var/lib/openbao`, kept across rebuilds by `ephemeral = false`. ⚠️ **Not a host path**: `nixos-container destroy swarm-bao` takes the raft data with it, so back up the container's tree, not `/var/lib/`. Only the store's TLS material (`/var/lib/swarm-bao-tls`) and its PKCS11 token (`/var/lib/swarm-bao-token`) are host-level | n/a — there is one store; a hive elsewhere is a _client_ of it and holds none of this |
|
||||||
|
| the secret store's unseal material | the HSM/TPM under `deploy.bao.seal = "pkcs11"`; openbao itself under `"shamir"` | in the token; or held by whoever ran `bao operator init`, which is what `"shamir"` means and why it's stated rather than inferred | n/a — only the host running the store seals anything |
|
||||||
|
|
||||||
Authelia mints the three keys for itself, in-container, precisely
|
Authelia mints the three keys for itself, in-container, precisely
|
||||||
because nothing outside that container ever reads them. **That's the test worth
|
because nothing outside that container ever reads them. **That's the test worth
|
||||||
|
|
@ -392,17 +393,17 @@ next controller restart.
|
||||||
Bao matches a cert-auth role on the certificate's subject, so a certificate is an
|
Bao matches a cert-auth role on the certificate's subject, so a certificate is an
|
||||||
identity and sharing one merges the identities. Each of these seven leaves has
|
identity and sharing one merges the identities. Each of these seven leaves has
|
||||||
a role and a policy of its own, and reaches one path, except the review
|
a role and a policy of its own, and reaches one path, except the review
|
||||||
classifier's and the forge host's, which reach two:
|
classifier's, the forge host's and Grafana's, which reach two each:
|
||||||
|
|
||||||
| unit | option pair under `deploy.bao.` | reads, or writes |
|
| unit | option pair under `deploy.bao.` | reads, or writes |
|
||||||
| --------------------------------------------------- | ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------ |
|
| --------------------------------------------------- | ------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| `swarm-bao-queue-agent` | `queueAgentClientCertFile` / `queueAgentClientKeyFile` | `swarm/hives/<hive>/queue/agent` |
|
| `swarm-bao-queue-agent` | `queueAgentClientCertFile` / `queueAgentClientKeyFile` | `swarm/hives/<hive>/queue/agent` |
|
||||||
| `swarm-bao-grafana-oidc` | `grafanaOidcClientCertFile` / `grafanaOidcClientKeyFile` | `swarm/services/<grafana client id>/oidc/client` |
|
| `swarm-bao-grafana-oidc`, `swarm-bao-grafana-admin` | `grafanaOidcClientCertFile` / `grafanaOidcClientKeyFile` | reads `swarm/services/<grafana client id>/oidc/client`; creates and reads `swarm/services/<grafana client id>/admin/password` |
|
||||||
| `swarm-bao-otel-oidc` | `otelOidcClientCertFile` / `otelOidcClientKeyFile` | `swarm/services/<collector client id>/oidc/client` |
|
| `swarm-bao-otel-oidc` | `otelOidcClientCertFile` / `otelOidcClientKeyFile` | `swarm/services/<collector client id>/oidc/client` |
|
||||||
| `swarm-bao-forwarder-oidc` | `forwarderOidcClientCertFile` / `forwarderOidcClientKeyFile` | `swarm/services/<store forwarder client id>/oidc/client` |
|
| `swarm-bao-forwarder-oidc` | `forwarderOidcClientCertFile` / `forwarderOidcClientKeyFile` | `swarm/services/<store forwarder client id>/oidc/client` |
|
||||||
| `swarm-bao-ci-runner` | `ciRunnerClientCertFile` / `ciRunnerClientKeyFile` | `swarm/services/ci-runner/registration-token` |
|
| `swarm-bao-ci-runner` | `ciRunnerClientCertFile` / `ciRunnerClientKeyFile` | `swarm/services/ci-runner/registration-token` |
|
||||||
| `swarm-review-classifier`, in its container | `reviewClassifierClientCertFile` / `reviewClassifierClientKeyFile` | `swarm/services/review-classifier/anthropic-api-key` and `swarm/services/<classifier client id>/oidc/client` |
|
| `swarm-review-classifier`, in its container | `reviewClassifierClientCertFile` / `reviewClassifierClientKeyFile` | `swarm/services/review-classifier/anthropic-api-key` and `swarm/services/<classifier client id>/oidc/client` |
|
||||||
| `swarm-bao-forge-publisher`, `swarm-bao-forge-oidc` | `forgePublisherClientCertFile` / `forgePublisherClientKeyFile` | writes `swarm/controller/swarm-controller/forge/token`; reads `swarm/services/<forge client id>/oidc/client` |
|
| `swarm-bao-forge-publisher`, `swarm-bao-forge-oidc` | `forgePublisherClientCertFile` / `forgePublisherClientKeyFile` | writes `swarm/controller/swarm-controller/forge/token`; reads `swarm/services/<forge client id>/oidc/client` |
|
||||||
|
|
||||||
The first exists **per hive**, because the path it reads carries a hive name
|
The first exists **per hive**, because the path it reads carries a hive name
|
||||||
and every hive runs its own reader. Its subject is
|
and every hive runs its own reader. Its subject is
|
||||||
|
|
|
||||||
|
|
@ -153,10 +153,11 @@ See `retentionPeriod` below before leaving it at its default.
|
||||||
<!-- vale write-good.Passive = NO -->
|
<!-- vale write-good.Passive = NO -->
|
||||||
|
|
||||||
**Logging in.** Grafana is behind swarm SSO, so the accounts are the
|
**Logging in.** Grafana is behind swarm SSO, so the accounts are the
|
||||||
authelia ones — there is no separate Grafana password, and this module
|
authelia ones, and the local login form is always off. Grafana's API still
|
||||||
switches off the local login form whenever SSO is configured. If you enable
|
accepts HTTP basic auth for its built-in `admin` account, so that account's
|
||||||
Grafana on a host with no authelia, the form stays on and Grafana's
|
password is a random one minted into the swarm secret store
|
||||||
default `admin`/`admin` applies; change it before exposing that host.
|
(`swarm/services/<grafana client id>/admin/password`) and applied on every
|
||||||
|
Grafana start — see [`secrets.md`](secrets.md).
|
||||||
|
|
||||||
<!-- vale write-good.Passive = YES -->
|
<!-- vale write-good.Passive = YES -->
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -66,6 +66,12 @@ let
|
||||||
"swarm-bao-secret-publisher-policy"
|
"swarm-bao-secret-publisher-policy"
|
||||||
];
|
];
|
||||||
};
|
};
|
||||||
|
# ./swarm-grafana.nix, under Grafana's leaf, which may create the admin
|
||||||
|
# password only once that policy is written.
|
||||||
|
swarm-bao-grafana-admin = {
|
||||||
|
here = deployCfg.grafana.enable;
|
||||||
|
policies = [ "swarm-bao-grafana-oidc-policy" ];
|
||||||
|
};
|
||||||
# ./hive-matrix.nix, under matrix-ctl's leaf.
|
# ./hive-matrix.nix, under matrix-ctl's leaf.
|
||||||
swarm-bao-matrix-oidc = {
|
swarm-bao-matrix-oidc = {
|
||||||
here = deployCfg.matrix.enable;
|
here = deployCfg.matrix.enable;
|
||||||
|
|
|
||||||
|
|
@ -850,11 +850,17 @@ let
|
||||||
# Singletons, so a list of one rather than a second shape: the granting unit
|
# Singletons, so a list of one rather than a second shape: the granting unit
|
||||||
# below takes a list either way and the units stay identical apart from
|
# below takes a list either way and the units stay identical apart from
|
||||||
# the objects they write.
|
# the objects they write.
|
||||||
|
#
|
||||||
|
# Grafana's leaf also creates its admin password (`swarm-bao-grafana-admin`
|
||||||
|
# in ./swarm-grafana.nix). `create` without `update`, so a stored password is
|
||||||
|
# never replaced.
|
||||||
grafanaOidcReaders = [
|
grafanaOidcReaders = [
|
||||||
{
|
{
|
||||||
name = "swarm-grafana-oidc";
|
name = "swarm-grafana-oidc";
|
||||||
cn = baoDeploy.grafanaOidcCommonName;
|
cn = baoDeploy.grafanaOidcCommonName;
|
||||||
policyText = readStanza "${credentialMountPath}/data/swarm/services/${hyperhiveCfg.swarm.grafana.oidc.clientId}/oidc/client";
|
policyText =
|
||||||
|
readStanza "${credentialMountPath}/data/swarm/services/${hyperhiveCfg.swarm.grafana.oidc.clientId}/oidc/client"
|
||||||
|
+ createStanza "${credentialMountPath}/data/swarm/services/${hyperhiveCfg.swarm.grafana.oidc.clientId}/admin/password";
|
||||||
}
|
}
|
||||||
];
|
];
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -253,6 +253,15 @@ let
|
||||||
# the OIDC secret above, whose other reader is authelia's container.
|
# the OIDC secret above, whose other reader is authelia's container.
|
||||||
secretKeyPath = "/var/lib/grafana-secret/secret_key";
|
secretKeyPath = "/var/lib/grafana-secret/secret_key";
|
||||||
|
|
||||||
|
# The built-in `admin` account's password. Grafana's API takes basic auth
|
||||||
|
# whatever the login form says, so this account is reachable from the public
|
||||||
|
# vhost and its password cannot be the upstream default. Minted once into the
|
||||||
|
# store by `swarm-bao-grafana-admin` below, which `swarm-bao.nix`'s
|
||||||
|
# `grafanaOidcReaders` grants create and read on this one leaf.
|
||||||
|
adminPasswordStorePath = "secret/swarm/services/${cfg.oidc.clientId}/admin/password";
|
||||||
|
adminPasswordPath = "/var/lib/grafana-admin/password";
|
||||||
|
hostAdminPasswordPath = "/var/lib/nixos-containers/${cfg.machine}${adminPasswordPath}";
|
||||||
|
|
||||||
socketPath = "${deployCfg.grafana.socketDir}/grafana.sock";
|
socketPath = "${deployCfg.grafana.socketDir}/grafana.sock";
|
||||||
|
|
||||||
# Both static NixOS ids, and both checked rather than assumed: nginx has
|
# Both static NixOS ids, and both checked rather than assumed: nginx has
|
||||||
|
|
@ -656,6 +665,97 @@ in
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|
||||||
|
# The admin password: created in the store if it holds none, then copied
|
||||||
|
# into the container, under Grafana's own leaf. Same shape as
|
||||||
|
# `swarm-bao-nats-callout-xkey` in ./swarm-nats.nix. The grant is
|
||||||
|
# `create` without `update`, so the first password stored is the one every
|
||||||
|
# later run reads, and a race between two runs leaves one value.
|
||||||
|
#
|
||||||
|
# Grafana does not start without the file (`$__file{}` on a missing path is
|
||||||
|
# a startup error), so a store that is unreachable on first boot keeps
|
||||||
|
# Grafana down rather than up with the upstream default.
|
||||||
|
systemd.services.swarm-bao-grafana-admin = {
|
||||||
|
description = "create Grafana's admin password in the swarm secret store if absent and fetch it";
|
||||||
|
after = lib.optionals baoDeploy.enable [
|
||||||
|
"swarm-bao-pki.service"
|
||||||
|
"container@${baoCfg.machine}.service"
|
||||||
|
];
|
||||||
|
wants = lib.optionals baoDeploy.enable [ "container@${baoCfg.machine}.service" ];
|
||||||
|
requires = lib.optionals baoDeploy.enable [ "swarm-bao-pki.service" ];
|
||||||
|
before = [ "container@${cfg.machine}.service" ];
|
||||||
|
wantedBy = [
|
||||||
|
"multi-user.target"
|
||||||
|
"container@${cfg.machine}.service"
|
||||||
|
];
|
||||||
|
path = [
|
||||||
|
baoDeploy.package
|
||||||
|
pkgs.coreutils
|
||||||
|
pkgs.openssl
|
||||||
|
pkgs.systemd
|
||||||
|
];
|
||||||
|
inherit (storeRetry) startLimitBurst startLimitIntervalSec;
|
||||||
|
serviceConfig = storeRetry.serviceConfig // {
|
||||||
|
Type = "oneshot";
|
||||||
|
RemainAfterExit = true;
|
||||||
|
SyslogIdentifier = "swarm-bao-grafana-admin";
|
||||||
|
TimeoutStartSec = 30;
|
||||||
|
};
|
||||||
|
environment = {
|
||||||
|
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
|
||||||
|
BAO_CLIENT_CERT = baoDeploy.grafanaOidcClientCertFile;
|
||||||
|
BAO_CLIENT_KEY = baoDeploy.grafanaOidcClientKeyFile;
|
||||||
|
}
|
||||||
|
// lib.optionalAttrs (baoDeploy.serverCaFile != null) {
|
||||||
|
BAO_CACERT = baoDeploy.serverCaFile;
|
||||||
|
};
|
||||||
|
script = ''
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
${atomicWriteSecret}
|
||||||
|
${refreshConsumer}
|
||||||
|
|
||||||
|
err="$(mktemp)"
|
||||||
|
trap 'rm -f "$err"' EXIT
|
||||||
|
|
||||||
|
if ! BAO_TOKEN="$(bao login -method=cert -token-only 2>"$err")"; then
|
||||||
|
echo "could not log in to swarm-bao with Grafana's certificate; leaving Grafana's admin password file as it is." >&2
|
||||||
|
cat "$err" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
export BAO_TOKEN
|
||||||
|
|
||||||
|
# `bao kv get` exits 2 both for "nothing stored" and for a refusal, so
|
||||||
|
# a failed read always tries the create, and the second read decides.
|
||||||
|
# The value reaches `bao` on stdin, never in an argument.
|
||||||
|
if ! password="$(bao kv get -field=value ${lib.escapeShellArg adminPasswordStorePath} 2>"$err")"; then
|
||||||
|
minted="$(openssl rand -hex 32)"
|
||||||
|
if ! printf '%s' "$minted" | bao kv put -cas=0 ${lib.escapeShellArg adminPasswordStorePath} value=- >/dev/null 2>"$err"; then
|
||||||
|
echo "could not create ${adminPasswordStorePath}; reading it back in case another run created it." >&2
|
||||||
|
cat "$err" >&2
|
||||||
|
fi
|
||||||
|
if ! password="$(bao kv get -field=value ${lib.escapeShellArg adminPasswordStorePath} 2>"$err")"; then
|
||||||
|
echo "swarm-bao did not return ${adminPasswordStorePath}; Grafana does not start without it." >&2
|
||||||
|
cat "$err" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
if [ -z "$password" ]; then
|
||||||
|
echo "swarm-bao returned an empty ${adminPasswordStorePath}; leaving the file as it is." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Owner and mode as for the OIDC secret above, for its reasons.
|
||||||
|
install -d -m 0755 ${lib.escapeShellArg (builtins.dirOf hostAdminPasswordPath)}
|
||||||
|
if secret_differs ${lib.escapeShellArg hostAdminPasswordPath} "$password"; then
|
||||||
|
atomic_write_secret 0400 ${lib.escapeShellArg "${toString config.ids.uids.grafana}:0"} ${lib.escapeShellArg hostAdminPasswordPath} "$password"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Grafana's `preStart` applies the file to the database, so a file that
|
||||||
|
# changed after it started needs a restart.
|
||||||
|
refresh_consumer ${lib.escapeShellArg cfg.machine} grafana.service ${lib.escapeShellArg hostAdminPasswordPath}
|
||||||
|
'';
|
||||||
|
};
|
||||||
|
|
||||||
containers.${cfg.machine} = {
|
containers.${cfg.machine} = {
|
||||||
autoStart = true;
|
autoStart = true;
|
||||||
ephemeral = false;
|
ephemeral = false;
|
||||||
|
|
@ -681,7 +781,7 @@ in
|
||||||
// caTrust.bindMount;
|
// caTrust.bindMount;
|
||||||
|
|
||||||
config =
|
config =
|
||||||
{ ... }:
|
{ config, ... }:
|
||||||
{
|
{
|
||||||
imports = [
|
imports = [
|
||||||
../container-modules/swarm-container.nix
|
../container-modules/swarm-container.nix
|
||||||
|
|
@ -752,6 +852,32 @@ in
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|
||||||
|
# Upstream applies `security.admin_password` only when it creates the
|
||||||
|
# database, so a database created before that setting keeps the
|
||||||
|
# password it was created with. This resets it from the delivered
|
||||||
|
# file on every start, before the server listens.
|
||||||
|
#
|
||||||
|
# `grafana cli` must open the database the server opens, which the
|
||||||
|
# server's config.ini names. nixpkgs keeps that file private, so it
|
||||||
|
# is rendered again from the same settings with nixpkgs' formatter.
|
||||||
|
systemd.services.grafana.preStart =
|
||||||
|
let
|
||||||
|
grafana = config.services.grafana;
|
||||||
|
configFile =
|
||||||
|
(pkgs.formats.ini {
|
||||||
|
listToValue = lib.concatMapStringsSep " " (lib.generators.mkValueStringDefault { });
|
||||||
|
mkKeyValue = lib.generators.mkKeyValueDefault {
|
||||||
|
mkValueString = v: if v == null then "" else lib.generators.mkValueStringDefault { } v;
|
||||||
|
} "=";
|
||||||
|
}).generate
|
||||||
|
"config.ini"
|
||||||
|
grafana.settings;
|
||||||
|
in
|
||||||
|
lib.mkAfter ''
|
||||||
|
${lib.getExe' grafana.package "grafana"} cli --homepath ${grafana.dataDir} --config ${configFile} \
|
||||||
|
admin reset-admin-password --password-from-stdin < ${adminPasswordPath}
|
||||||
|
'';
|
||||||
|
|
||||||
# The group Grafana chgrps its socket to. Declared here because a
|
# The group Grafana chgrps its socket to. Declared here because a
|
||||||
# container has its own user database — the host knowing gid 60 as
|
# container has its own user database — the host knowing gid 60 as
|
||||||
# `nginx` means nothing in here, only the number crosses.
|
# `nginx` means nothing in here, only the number crosses.
|
||||||
|
|
@ -837,10 +963,14 @@ in
|
||||||
# whole host fails to build. The unit below generates it.
|
# whole host fails to build. The unit below generates it.
|
||||||
security.secret_key = "$__file{${secretKeyPath}}";
|
security.secret_key = "$__file{${secretKeyPath}}";
|
||||||
|
|
||||||
# No local password path, ever. This is not tidiness: Grafana
|
# The password a newly created database gets; the `preStart`
|
||||||
# ships an `admin`/`admin` account and this vhost is on the
|
# above gives an existing one the same. Written by
|
||||||
# public gateway, so SSO is the only way in — not merely the
|
# `swarm-bao-grafana-admin` on the host.
|
||||||
# preferred one.
|
security.admin_password = "$__file{${adminPasswordPath}}";
|
||||||
|
|
||||||
|
# No password box on the public login page: people log in through
|
||||||
|
# SSO. This hides the form only; the API still takes basic auth,
|
||||||
|
# which is why the admin password above is minted.
|
||||||
#
|
#
|
||||||
# Not conditional on anything, and the block below is not
|
# Not conditional on anything, and the block below is not
|
||||||
# either, which is what keeps the two from ever disagreeing.
|
# either, which is what keeps the two from ever disagreeing.
|
||||||
|
|
|
||||||
|
|
@ -557,13 +557,28 @@ let
|
||||||
# for this one is the OTHER service's: `services/*` would have granted
|
# for this one is the OTHER service's: `services/*` would have granted
|
||||||
# both, and the two are separate principals precisely because a
|
# both, and the two are separate principals precisely because a
|
||||||
# dashboard is not entitled to a collector's credential.
|
# dashboard is not entitled to a collector's credential.
|
||||||
name = "the Grafana OIDC reader's grant is Grafana's own client secret and nothing else";
|
#
|
||||||
|
# The admin password is the one leaf it may create, and without `update`:
|
||||||
|
# a grant that could replace it would let a second run rotate the
|
||||||
|
# password Grafana's database already holds.
|
||||||
|
name = "the Grafana reader reads its own client secret and creates its admin password, and nothing else";
|
||||||
ok =
|
ok =
|
||||||
let
|
let
|
||||||
s = baoGrantHere.systemd.services.swarm-bao-grafana-oidc-policy.script;
|
s = baoGrantHere.systemd.services.swarm-bao-grafana-oidc-policy.script;
|
||||||
in
|
in
|
||||||
lib.hasInfix "path \"secret/data/swarm/services/swarm-grafana/oidc/client\" {" s
|
grantsIn s == [
|
||||||
&& lib.hasInfix "capabilities = [\"read\"]" s
|
{
|
||||||
|
path = "secret/data/swarm/services/swarm-grafana/oidc/client";
|
||||||
|
caps = [ "read" ];
|
||||||
|
}
|
||||||
|
{
|
||||||
|
path = "secret/data/swarm/services/swarm-grafana/admin/password";
|
||||||
|
caps = [
|
||||||
|
"create"
|
||||||
|
"read"
|
||||||
|
];
|
||||||
|
}
|
||||||
|
]
|
||||||
&& !(lib.hasInfix "secret/data/swarm/agents" s)
|
&& !(lib.hasInfix "secret/data/swarm/agents" s)
|
||||||
&& !(lib.hasInfix "secret/data/swarm/hives" s)
|
&& !(lib.hasInfix "secret/data/swarm/hives" s)
|
||||||
&& !(lib.hasInfix "secret/data/swarm/services/*" s)
|
&& !(lib.hasInfix "secret/data/swarm/services/*" s)
|
||||||
|
|
@ -827,6 +842,20 @@ let
|
||||||
&& !(noReaders ? swarm-bao-matrix-oidc)
|
&& !(noReaders ? swarm-bao-matrix-oidc)
|
||||||
&& !(noReaders ? swarm-bao-nats-callout-xkey);
|
&& !(noReaders ? swarm-bao-nats-callout-xkey);
|
||||||
}
|
}
|
||||||
|
{
|
||||||
|
# The admin password is created under Grafana's leaf, so its unit waits
|
||||||
|
# for the unit writing that leaf's role and policy; before it, the create
|
||||||
|
# is refused. The second arm is the absence control.
|
||||||
|
name = "grafana's admin password unit is ordered after grafana's policy unit";
|
||||||
|
ok =
|
||||||
|
let
|
||||||
|
u = baoGrantWithConsumers.systemd.services.swarm-bao-grafana-admin;
|
||||||
|
in
|
||||||
|
lib.elem "swarm-bao-grafana-oidc-policy.service" u.after
|
||||||
|
&& lib.elem "swarm-bao-grafana-oidc-policy.service" u.wants
|
||||||
|
&& !(lib.elem "swarm-bao-grafana-oidc-policy.service" u.requires)
|
||||||
|
&& !(baoGrantNoReaders.systemd.services ? swarm-bao-grafana-admin);
|
||||||
|
}
|
||||||
{
|
{
|
||||||
# The ordering is set apart from each reader's own definition, so it can
|
# The ordering is set apart from each reader's own definition, so it can
|
||||||
# define a reader by itself: `after` on a unit nothing else declares is a
|
# define a reader by itself: `after` on a unit nothing else declares is a
|
||||||
|
|
@ -959,6 +988,7 @@ let
|
||||||
[
|
[
|
||||||
"swarm-services-cert"
|
"swarm-services-cert"
|
||||||
"swarm-bao-forwarder-oidc"
|
"swarm-bao-forwarder-oidc"
|
||||||
|
"swarm-bao-grafana-admin"
|
||||||
]
|
]
|
||||||
++ policyReaders
|
++ policyReaders
|
||||||
);
|
);
|
||||||
|
|
|
||||||
|
|
@ -123,9 +123,8 @@ let
|
||||||
{
|
{
|
||||||
# This fixture enables grafana and NOT authelia, so the login-form toggle
|
# This fixture enables grafana and NOT authelia, so the login-form toggle
|
||||||
# has to follow "grafana requires SSO" rather than "both services are on
|
# has to follow "grafana requires SSO" rather than "both services are on
|
||||||
# this host". Grafana ships an
|
# this host". Its vhost is on the public gateway, so a password box there
|
||||||
# `admin`/`admin` account and its vhost is on the public gateway, so a
|
# is a way in whatever the topology.
|
||||||
# password box there is a way in whatever the topology.
|
|
||||||
name = "grafana disables its local login form even where authelia is not on this host";
|
name = "grafana disables its local login form even where authelia is not on this host";
|
||||||
ok =
|
ok =
|
||||||
grafanaOldPath.containers.swarm-grafana.config.services.grafana.settings.auth.disable_login_form;
|
grafanaOldPath.containers.swarm-grafana.config.services.grafana.settings.auth.disable_login_form;
|
||||||
|
|
@ -196,6 +195,36 @@ let
|
||||||
&& !(grafanaRefusedFor grafanaRemoteAuthelia "services.hyperhive.swarm.authelia.url")
|
&& !(grafanaRefusedFor grafanaRemoteAuthelia "services.hyperhive.swarm.authelia.url")
|
||||||
&& !(grafanaRefusedFor grafanaRemoteAuthelia "deploy.bao.grafanaOidcClientCertFile");
|
&& !(grafanaRefusedFor grafanaRemoteAuthelia "deploy.bao.grafanaOidcClientCertFile");
|
||||||
}
|
}
|
||||||
|
{
|
||||||
|
# Grafana's API takes basic auth whatever the login form says, so the
|
||||||
|
# built-in admin's password is the only thing between the public vhost
|
||||||
|
# and server-admin. Upstream sets it only when it creates the database,
|
||||||
|
# so both halves are pinned: the setting a new database reads, and the
|
||||||
|
# reset every start applies to whichever database exists. Without the
|
||||||
|
# reset, a database created before the setting keeps `admin`.
|
||||||
|
#
|
||||||
|
# The reset's `--config` must be the file the server reads, or it resets
|
||||||
|
# a different database: compared by store path against the server's own
|
||||||
|
# command line.
|
||||||
|
name = "grafana's admin password is minted into the store and applied to new and existing databases";
|
||||||
|
ok =
|
||||||
|
let
|
||||||
|
unit = grafanaRemoteAuthelia.systemd.services.swarm-bao-grafana-admin;
|
||||||
|
grafana = grafanaRemoteAuthelia.containers.swarm-grafana.config.services.grafana;
|
||||||
|
service = grafanaRemoteAuthelia.containers.swarm-grafana.config.systemd.services.grafana;
|
||||||
|
resetConfig = lib.head (
|
||||||
|
lib.head (lib.filter lib.isList (builtins.split "--config ([^ ]+)" service.preStart))
|
||||||
|
);
|
||||||
|
in
|
||||||
|
lib.hasInfix "bao kv put -cas=0 secret/swarm/services/swarm-grafana/admin/password value=-" unit.script
|
||||||
|
&& lib.hasInfix "openssl rand -hex 32" unit.script
|
||||||
|
&& lib.hasInfix "atomic_write_secret 0400 196:0 /var/lib/nixos-containers/swarm-grafana/var/lib/grafana-admin/password" unit.script
|
||||||
|
&& lib.elem "container@swarm-grafana.service" unit.before
|
||||||
|
&& unit.environment.BAO_CLIENT_CERT == "/etc/pki/bao-grafana-oidc.pem"
|
||||||
|
&& grafana.settings.security.admin_password == "$__file{/var/lib/grafana-admin/password}"
|
||||||
|
&& lib.hasInfix "admin reset-admin-password --password-from-stdin < /var/lib/grafana-admin/password" service.preStart
|
||||||
|
&& lib.hasInfix "-config ${resetConfig}" service.script;
|
||||||
|
}
|
||||||
{
|
{
|
||||||
# Same 403-not-a-miss reason as the matrix and queue arms below: the
|
# Same 403-not-a-miss reason as the matrix and queue arms below: the
|
||||||
# reader's grant covers the `services` prefix, so a path outside it is
|
# reader's grant covers the `services` prefix, so a path outside it is
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue