Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
ad47664e01 | ||
|
|
eecd2b62e8 | ||
|
|
993f5fd99d |
8 changed files with 299 additions and 55 deletions
|
|
@ -41,27 +41,28 @@ neither is a renaming of the other.
|
|||
|
||||
## Swarm-level — one of each per swarm
|
||||
|
||||
| secret | generated by | lives at | hive elsewhere |
|
||||
| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| swarm-services root CA (cert + key) | `swarm-bao.nix`'s granting unit, inside the store's `pki` mount | the **key** never leaves the mount; the **cert** lands at `/var/lib/swarm-bao-services-pki/services-root.pem` (0644) and at `/var/lib/hive-tls/swarm-services-root.pem` on every hive | nothing to copy — every hive receives the cert with the leaf it asks the store for, or from `pki/ca/pem` where it fronts no swarm name and needs a store identity |
|
||||
| authelia session, JWT and storage-encryption keys | authelia's first-boot unit, in-container | `/var/lib/authelia-swarm/{session,jwt,storage-encryption}.key` | generated in place; nothing outside that container reads them |
|
||||
| authelia OIDC HMAC key | same unit | `/var/lib/authelia-swarm/oidc-hmac.key` | same |
|
||||
| authelia OIDC issuer key (RSA) | same unit | `/var/lib/authelia-swarm/oidc-issuer.key` | same — relying parties verify against the **public** half at `/jwks.json` |
|
||||
| OIDC client secret, plaintext half | `authelia crypto hash generate --random` | `/var/lib/authelia-swarm/oidc-clients/<id>.secret` | operator provides the file and names it in whichever option reads it — `sso.clientSecretFile` for a service; the forge and, by default, the hive collector need none — `swarm-bao-forge-oidc` and `swarm-bao-hive-otel-oidc` fetch it from the store (below) |
|
||||
| OIDC client secret, digest half | the same mint | `oidc-clients/<id>.digest` | authelia's own half; merged at runtime via `settingsFiles` |
|
||||
| the swarm collector's copy of its OIDC secret | `swarm-bao-otel-oidc.service` reads it out of the swarm secret store, **on every host that runs the collector and holds a store identity** | `/var/lib/swarm-otel-oidc/<id>.secret` inside the `swarm-otel` container | same unit, same path — one route, co-located or not. A collector with no store identity has `clientSecretFile == null`, its already-supported unauthenticated-push degrade — see below |
|
||||
| Grafana's copy of its OIDC secret | `swarm-bao-grafana-oidc.service` reads it out of the swarm secret store, **on every host that runs Grafana** | `/var/lib/grafana-oidc/<id>.secret` inside the `swarm-grafana` container | same unit, same path — one route, co-located or not. Nothing for an operator to place beyond this host's store leaf, see below |
|
||||
| the swarm controller's copy of its OIDC secret | `swarm-controller` reads it out of the swarm secret store at start, under its own leaf | the daemon's memory only; never on disk | same read, co-located or not. Nothing for an operator to place beyond the controller's store leaf (`deploy.swarm-controller.baoClientCertFile`) |
|
||||
| a hive's copy of its own OIDC secret | hive-c0re reads it out of the swarm secret store when it first connects to the queue, under the hive's own leaf | the daemon's memory only; never on disk | same read, co-located or not. Nothing for an operator to place beyond the hive's store leaf (`deploy.bao.clientCertFile`) |
|
||||
| the hive collector's copy of its hive's OIDC secret | `swarm-bao-hive-otel-oidc.service` reads it out of the swarm secret store under the hive's own leaf, **on every hive that runs the collector and holds that leaf** | `/var/lib/hive-otel-oidc/hive-<name>.secret` on the hive's host, 0400 root, loaded by `LoadCredential` | same unit, same path, co-located or not. Nothing for an operator to place beyond the hive's store leaf (`deploy.bao.clientCertFile`) |
|
||||
| the swarm controller's forge token | the forge's `forgejo-swarm-controller-account`, in-container, once | `/var/lib/forgejo/swarm-controller-token` inside the forge container, the minter's record that it has minted; `swarm-bao-forge-publisher` copies it into the swarm secret store at `swarm/controller/swarm-controller/forge/token` each time the forge container starts and each time the mint writes the file; the controller holds it in memory only | nothing to place beyond the controller's store leaf. A forge host without the store needs the publisher's leaf placed (`deploy.bao.forgePublisherClientCertFile`) |
|
||||
| the queue responder's copy of its OIDC secret | the responder reads it out of the swarm secret store itself as it starts, under its own leaf, and again when authelia refuses it | the daemon's memory only; never on disk | same read, wherever authelia runs. Nothing for an operator to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) |
|
||||
| authelia subject store | `swarmctl` and `swarm-authelia-bridge` | `users.yml` — one file, read and written by both | `swarmctl`, on the host that runs authelia |
|
||||
| wireguard private key | **the operator** — `wg genkey` | whatever `deploy.wireguard.privateKeyFile` names | always operator-provided; nothing generates this for you |
|
||||
| queue auth-callout nkeys (user seed + account seed) | `swarm-nats-auth ensure-callout-keys`, run by `swarm-bao-nats-callout-xkey` on the queue's host when the store holds no seed; the verb stores a seed it finds at `/var/lib/swarm-nats-callout/{callout-user,issuer}.seed` as is, then removes the file | the swarm secret store: the seeds at `swarm/services/swarm-nats/callout/{user,issuer}-seed`, read by the `swarm-nats-auth` role alone; the public halves at `swarm/public/swarm-nats/callout-{user,issuer}` | nothing to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) |
|
||||
| queue auth-callout xkey (curve seed) | `swarm-nats-auth ensure-callout-keys`, run by `swarm-bao-nats-callout-xkey` on the queue's host when the store holds no seed | the swarm secret store: the seed at `swarm/services/swarm-nats/callout/xkey-seed`, read by the `swarm-nats-auth` role alone; the public half at `swarm/public/swarm-nats/callout-xkey` | nothing to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) |
|
||||
| the secret store's own contents | openbao, on first `bao operator init` — **an operator action, not a unit** | inside the `swarm-bao` container, at its own `/var/lib/openbao`, kept across rebuilds by `ephemeral = false`. ⚠️ **Not a host path**: `nixos-container destroy swarm-bao` takes the raft data with it, so back up the container's tree, not `/var/lib/`. Only the store's TLS material (`/var/lib/swarm-bao-tls`) and its PKCS11 token (`/var/lib/swarm-bao-token`) are host-level | n/a — there is one store; a hive elsewhere is a _client_ of it and holds none of this |
|
||||
| the secret store's unseal material | the HSM/TPM under `deploy.bao.seal = "pkcs11"`; openbao itself under `"shamir"` | in the token; or held by whoever ran `bao operator init`, which is what `"shamir"` means and why it's stated rather than inferred | n/a — only the host running the store seals anything |
|
||||
| secret | generated by | lives at | hive elsewhere |
|
||||
| --------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| swarm-services root CA (cert + key) | `swarm-bao.nix`'s granting unit, inside the store's `pki` mount | the **key** never leaves the mount; the **cert** lands at `/var/lib/swarm-bao-services-pki/services-root.pem` (0644) and at `/var/lib/hive-tls/swarm-services-root.pem` on every hive | nothing to copy — every hive receives the cert with the leaf it asks the store for, or from `pki/ca/pem` where it fronts no swarm name and needs a store identity |
|
||||
| authelia session, JWT and storage-encryption keys | authelia's first-boot unit, in-container | `/var/lib/authelia-swarm/{session,jwt,storage-encryption}.key` | generated in place; nothing outside that container reads them |
|
||||
| authelia OIDC HMAC key | same unit | `/var/lib/authelia-swarm/oidc-hmac.key` | same |
|
||||
| authelia OIDC issuer key (RSA) | same unit | `/var/lib/authelia-swarm/oidc-issuer.key` | same — relying parties verify against the **public** half at `/jwks.json` |
|
||||
| OIDC client secret, plaintext half | `authelia crypto hash generate --random` | `/var/lib/authelia-swarm/oidc-clients/<id>.secret` | operator provides the file and names it in whichever option reads it — `sso.clientSecretFile` for a service; the forge and, by default, the hive collector need none — `swarm-bao-forge-oidc` and `swarm-bao-hive-otel-oidc` fetch it from the store (below) |
|
||||
| OIDC client secret, digest half | the same mint | `oidc-clients/<id>.digest` | authelia's own half; merged at runtime via `settingsFiles` |
|
||||
| the swarm collector's copy of its OIDC secret | `swarm-bao-otel-oidc.service` reads it out of the swarm secret store, **on every host that runs the collector and holds a store identity** | `/var/lib/swarm-otel-oidc/<id>.secret` inside the `swarm-otel` container | same unit, same path — one route, co-located or not. A collector with no store identity has `clientSecretFile == null`, its already-supported unauthenticated-push degrade — see below |
|
||||
| Grafana's copy of its OIDC secret | `swarm-bao-grafana-oidc.service` reads it out of the swarm secret store, **on every host that runs Grafana** | `/var/lib/grafana-oidc/<id>.secret` inside the `swarm-grafana` container | same unit, same path — one route, co-located or not. Nothing for an operator to place beyond this host's store leaf, see below |
|
||||
| Grafana's admin password | `swarm-bao-grafana-admin.service` creates it in the swarm secret store at `swarm/services/<grafana client id>/admin/password` when the store holds none, then copies it out, **on every host that runs Grafana**. Grafana's leaf may create that leaf but not update it, so the store keeps the first value, and the secret publisher can't write it. To rotate, run `bao kv metadata delete secret/swarm/services/<grafana client id>/admin/password` with a token that may, such as the root token (Grafana's, the publisher's and the controller's can't), then `systemctl restart swarm-bao-grafana-admin`; a plain `bao kv delete` isn't enough, see [the callout xkey's section](#the-queues-callout-xkey) | `/var/lib/grafana-admin/password` inside the `swarm-grafana` container: `security.admin_password` for a new database, and `grafana cli admin reset-admin-password` in `grafana.service`'s `preStart` for an existing one, on every start | same unit, same path. Grafana doesn't start until the file exists, so it stays down rather than falling back to the upstream default `admin` |
|
||||
| the swarm controller's copy of its OIDC secret | `swarm-controller` reads it out of the swarm secret store at start, under its own leaf | the daemon's memory only; never on disk | same read, co-located or not. Nothing for an operator to place beyond the controller's store leaf (`deploy.swarm-controller.baoClientCertFile`) |
|
||||
| a hive's copy of its own OIDC secret | hive-c0re reads it out of the swarm secret store when it first connects to the queue, under the hive's own leaf | the daemon's memory only; never on disk | same read, co-located or not. Nothing for an operator to place beyond the hive's store leaf (`deploy.bao.clientCertFile`) |
|
||||
| the hive collector's copy of its hive's OIDC secret | `swarm-bao-hive-otel-oidc.service` reads it out of the swarm secret store under the hive's own leaf, **on every hive that runs the collector and holds that leaf** | `/var/lib/hive-otel-oidc/hive-<name>.secret` on the hive's host, 0400 root, loaded by `LoadCredential` | same unit, same path, co-located or not. Nothing for an operator to place beyond the hive's store leaf (`deploy.bao.clientCertFile`) |
|
||||
| the swarm controller's forge token | the forge's `forgejo-swarm-controller-account`, in-container, once | `/var/lib/forgejo/swarm-controller-token` inside the forge container, the minter's record that it has minted; `swarm-bao-forge-publisher` copies it into the swarm secret store at `swarm/controller/swarm-controller/forge/token` each time the forge container starts and each time the mint writes the file; the controller holds it in memory only | nothing to place beyond the controller's store leaf. A forge host without the store needs the publisher's leaf placed (`deploy.bao.forgePublisherClientCertFile`) |
|
||||
| the queue responder's copy of its OIDC secret | the responder reads it out of the swarm secret store itself as it starts, under its own leaf, and again when authelia refuses it | the daemon's memory only; never on disk | same read, wherever authelia runs. Nothing for an operator to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) |
|
||||
| authelia subject store | `swarmctl` and `swarm-authelia-bridge` | `users.yml` — one file, read and written by both | `swarmctl`, on the host that runs authelia |
|
||||
| wireguard private key | **the operator** — `wg genkey` | whatever `deploy.wireguard.privateKeyFile` names | always operator-provided; nothing generates this for you |
|
||||
| queue auth-callout nkeys (user seed + account seed) | `swarm-nats-auth ensure-callout-keys`, run by `swarm-bao-nats-callout-xkey` on the queue's host when the store holds no seed; the verb stores a seed it finds at `/var/lib/swarm-nats-callout/{callout-user,issuer}.seed` as is, then removes the file | the swarm secret store: the seeds at `swarm/services/swarm-nats/callout/{user,issuer}-seed`, read by the `swarm-nats-auth` role alone; the public halves at `swarm/public/swarm-nats/callout-{user,issuer}` | nothing to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) |
|
||||
| queue auth-callout xkey (curve seed) | `swarm-nats-auth ensure-callout-keys`, run by `swarm-bao-nats-callout-xkey` on the queue's host when the store holds no seed | the swarm secret store: the seed at `swarm/services/swarm-nats/callout/xkey-seed`, read by the `swarm-nats-auth` role alone; the public half at `swarm/public/swarm-nats/callout-xkey` | nothing to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) |
|
||||
| the secret store's own contents | openbao, on first `bao operator init` — **an operator action, not a unit** | inside the `swarm-bao` container, at its own `/var/lib/openbao`, kept across rebuilds by `ephemeral = false`. ⚠️ **Not a host path**: `nixos-container destroy swarm-bao` takes the raft data with it, so back up the container's tree, not `/var/lib/`. Only the store's TLS material (`/var/lib/swarm-bao-tls`) and its PKCS11 token (`/var/lib/swarm-bao-token`) are host-level | n/a — there is one store; a hive elsewhere is a _client_ of it and holds none of this |
|
||||
| the secret store's unseal material | the HSM/TPM under `deploy.bao.seal = "pkcs11"`; openbao itself under `"shamir"` | in the token; or held by whoever ran `bao operator init`, which is what `"shamir"` means and why it's stated rather than inferred | n/a — only the host running the store seals anything |
|
||||
|
||||
Authelia mints the three keys for itself, in-container, precisely
|
||||
because nothing outside that container ever reads them. **That's the test worth
|
||||
|
|
@ -298,7 +299,7 @@ a host reads everything else with them.
|
|||
Two things follow, and `swarm-grafana.nix` asserts both rather than degrading:
|
||||
running Grafana requires `swarm.authelia.url` (this module disables its local
|
||||
login form unconditionally, so SSO isn't a feature of some topologies — it's the
|
||||
only way in), and it requires this host's `deploy.bao.grafanaOidcClientCertFile`
|
||||
only way into the UI), and it requires this host's `deploy.bao.grafanaOidcClientCertFile`
|
||||
/ `grafanaOidcClientKeyFile`. Each refusal names the option to set. Both used to be silent:
|
||||
a null URL dropped the OIDC block, a missing leaf produced a warning, and
|
||||
either one left a Grafana with no SSO and no password box — nothing failed, so
|
||||
|
|
@ -316,6 +317,13 @@ the option pair, when that leaf is unset — neither the responder nor the
|
|||
homeserver starts without its secret, and the forge registers no login source
|
||||
without its.
|
||||
|
||||
Grafana's admin password rides the same leaf and the same store. Where Grafana
|
||||
and the store run on different hosts, deploy the store's host first:
|
||||
`swarm-bao-grafana-oidc-policy` there writes the grant that lets Grafana create
|
||||
the password, and until it has, the store refuses `swarm-bao-grafana-admin` on
|
||||
the Grafana host, which retries, while Grafana stays down because its config
|
||||
reads that file.
|
||||
|
||||
The **third reader** is the swarm collector's own OIDC client secret, the
|
||||
same shape one level down: `swarm-bao-otel-oidc.service` reads
|
||||
`swarm/services/<client-id>/oidc/client` out of the store on every host that
|
||||
|
|
@ -392,17 +400,17 @@ next controller restart.
|
|||
Bao matches a cert-auth role on the certificate's subject, so a certificate is an
|
||||
identity and sharing one merges the identities. Each of these seven leaves has
|
||||
a role and a policy of its own, and reaches one path, except the review
|
||||
classifier's and the forge host's, which reach two:
|
||||
classifier's, the forge host's and Grafana's, which reach two each:
|
||||
|
||||
| unit | option pair under `deploy.bao.` | reads, or writes |
|
||||
| --------------------------------------------------- | ------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------ |
|
||||
| `swarm-bao-queue-agent` | `queueAgentClientCertFile` / `queueAgentClientKeyFile` | `swarm/hives/<hive>/queue/agent` |
|
||||
| `swarm-bao-grafana-oidc` | `grafanaOidcClientCertFile` / `grafanaOidcClientKeyFile` | `swarm/services/<grafana client id>/oidc/client` |
|
||||
| `swarm-bao-otel-oidc` | `otelOidcClientCertFile` / `otelOidcClientKeyFile` | `swarm/services/<collector client id>/oidc/client` |
|
||||
| `swarm-bao-forwarder-oidc` | `forwarderOidcClientCertFile` / `forwarderOidcClientKeyFile` | `swarm/services/<store forwarder client id>/oidc/client` |
|
||||
| `swarm-bao-ci-runner` | `ciRunnerClientCertFile` / `ciRunnerClientKeyFile` | `swarm/services/ci-runner/registration-token` |
|
||||
| `swarm-review-classifier`, in its container | `reviewClassifierClientCertFile` / `reviewClassifierClientKeyFile` | `swarm/services/review-classifier/anthropic-api-key` and `swarm/services/<classifier client id>/oidc/client` |
|
||||
| `swarm-bao-forge-publisher`, `swarm-bao-forge-oidc` | `forgePublisherClientCertFile` / `forgePublisherClientKeyFile` | writes `swarm/controller/swarm-controller/forge/token`; reads `swarm/services/<forge client id>/oidc/client` |
|
||||
| unit | option pair under `deploy.bao.` | reads, or writes |
|
||||
| --------------------------------------------------- | ------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `swarm-bao-queue-agent` | `queueAgentClientCertFile` / `queueAgentClientKeyFile` | `swarm/hives/<hive>/queue/agent` |
|
||||
| `swarm-bao-grafana-oidc`, `swarm-bao-grafana-admin` | `grafanaOidcClientCertFile` / `grafanaOidcClientKeyFile` | reads `swarm/services/<grafana client id>/oidc/client`; creates and reads `swarm/services/<grafana client id>/admin/password` |
|
||||
| `swarm-bao-otel-oidc` | `otelOidcClientCertFile` / `otelOidcClientKeyFile` | `swarm/services/<collector client id>/oidc/client` |
|
||||
| `swarm-bao-forwarder-oidc` | `forwarderOidcClientCertFile` / `forwarderOidcClientKeyFile` | `swarm/services/<store forwarder client id>/oidc/client` |
|
||||
| `swarm-bao-ci-runner` | `ciRunnerClientCertFile` / `ciRunnerClientKeyFile` | `swarm/services/ci-runner/registration-token` |
|
||||
| `swarm-review-classifier`, in its container | `reviewClassifierClientCertFile` / `reviewClassifierClientKeyFile` | `swarm/services/review-classifier/anthropic-api-key` and `swarm/services/<classifier client id>/oidc/client` |
|
||||
| `swarm-bao-forge-publisher`, `swarm-bao-forge-oidc` | `forgePublisherClientCertFile` / `forgePublisherClientKeyFile` | writes `swarm/controller/swarm-controller/forge/token`; reads `swarm/services/<forge client id>/oidc/client` |
|
||||
|
||||
The first exists **per hive**, because the path it reads carries a hive name
|
||||
and every hive runs its own reader. Its subject is
|
||||
|
|
|
|||
|
|
@ -153,10 +153,11 @@ See `retentionPeriod` below before leaving it at its default.
|
|||
<!-- vale write-good.Passive = NO -->
|
||||
|
||||
**Logging in.** Grafana is behind swarm SSO, so the accounts are the
|
||||
authelia ones — there is no separate Grafana password, and this module
|
||||
switches off the local login form whenever SSO is configured. If you enable
|
||||
Grafana on a host with no authelia, the form stays on and Grafana's
|
||||
default `admin`/`admin` applies; change it before exposing that host.
|
||||
authelia ones, and the local login form is always off. Grafana's API still
|
||||
accepts HTTP basic auth for its built-in `admin` account, so that account's
|
||||
password is a random one minted into the swarm secret store
|
||||
(`swarm/services/<grafana client id>/admin/password`) and applied on every
|
||||
Grafana start — see [`secrets.md`](secrets.md).
|
||||
|
||||
<!-- vale write-good.Passive = YES -->
|
||||
|
||||
|
|
|
|||
|
|
@ -300,12 +300,18 @@ gateway refuses its token too. See [`swarmctl/README.md`](../../swarmctl/README.
|
|||
with two ways in, and making authelia the only path there is a separate,
|
||||
reversible switch (tuwunel's `login_with_password`, forgejo's own setting).
|
||||
⚠️ **Grafana is the exception, and not a reversible one.** Its module sets
|
||||
`auth.disable_login_form` unconditionally, because Grafana ships an
|
||||
`admin`/`admin` account and its vhost is on the public gateway — so a
|
||||
password box there is a way in whatever the deployment. SSO is the only
|
||||
door, and a dead provider locks everyone out, the built-in admin
|
||||
included. That's why `swarm.grafana.oidc.role` defaults to `Admin`; see
|
||||
[`services.md`](services.md#metrics-victoriametrics--grafana).
|
||||
`auth.disable_login_form` unconditionally: its vhost is on the public
|
||||
gateway, so a password box there would be a way in whatever the
|
||||
deployment. The form is all this hides. Grafana's API still takes HTTP
|
||||
basic auth for the built-in `admin` account, whose password is a random
|
||||
one minted into the swarm secret store, not the upstream `admin`. A
|
||||
dead provider locks out the SSO accounts but not that one: an operator
|
||||
reads the password from `swarm/services/<grafana client id>/admin/password`
|
||||
in the store (or, as root on the Grafana host, from
|
||||
`/var/lib/grafana-admin/password` inside the `swarm-grafana` container)
|
||||
and calls the API with it. See [`services.md`](services.md#metrics-victoriametrics--grafana)
|
||||
and [`secrets.md`](secrets.md). SSO logins are the everyday way in, which
|
||||
is why `swarm.grafana.oidc.role` defaults to `Admin`.
|
||||
- **It doesn't provision users.** `swarmctl` creates and destroys agents
|
||||
continuously today, so the subject set belongs to a program rather
|
||||
than to a config file.
|
||||
|
|
|
|||
|
|
@ -66,6 +66,12 @@ let
|
|||
"swarm-bao-secret-publisher-policy"
|
||||
];
|
||||
};
|
||||
# ./swarm-grafana.nix, under Grafana's leaf, which may create the admin
|
||||
# password only once that policy is written.
|
||||
swarm-bao-grafana-admin = {
|
||||
here = deployCfg.grafana.enable;
|
||||
policies = [ "swarm-bao-grafana-oidc-policy" ];
|
||||
};
|
||||
# ./hive-matrix.nix, under matrix-ctl's leaf.
|
||||
swarm-bao-matrix-oidc = {
|
||||
here = deployCfg.matrix.enable;
|
||||
|
|
|
|||
|
|
@ -482,6 +482,9 @@ let
|
|||
# prefix, are denied. A write to the xkey's would replace the key every
|
||||
# queue request is sealed to; the issuer's is the authority to admit anyone
|
||||
# to the queue, and the user's is the one identity exempt from callout.
|
||||
# Grafana's admin password is denied the same way: a write to it sets the
|
||||
# password `grafana.service` applies to the server-admin account at its next
|
||||
# start.
|
||||
secretPublisherPolicyText = ''
|
||||
path "${credentialMountPath}/data/swarm/hives/*" {
|
||||
capabilities = ["create", "update"]
|
||||
|
|
@ -503,6 +506,10 @@ let
|
|||
capabilities = ["deny"]
|
||||
}
|
||||
|
||||
path "${credentialMountPath}/data/${grafanaAdminLeaf}" {
|
||||
capabilities = ["deny"]
|
||||
}
|
||||
|
||||
path "${credentialMountPath}/data/${controllerClientLeaf}" {
|
||||
capabilities = ["create", "update"]
|
||||
}
|
||||
|
|
@ -850,11 +857,17 @@ let
|
|||
# Singletons, so a list of one rather than a second shape: the granting unit
|
||||
# below takes a list either way and the units stay identical apart from
|
||||
# the objects they write.
|
||||
#
|
||||
# Grafana's leaf also creates its admin password (`swarm-bao-grafana-admin`
|
||||
# in ./swarm-grafana.nix). `create` without `update`, so a stored password is
|
||||
# never replaced.
|
||||
grafanaOidcReaders = [
|
||||
{
|
||||
name = "swarm-grafana-oidc";
|
||||
cn = baoDeploy.grafanaOidcCommonName;
|
||||
policyText = readStanza "${credentialMountPath}/data/swarm/services/${hyperhiveCfg.swarm.grafana.oidc.clientId}/oidc/client";
|
||||
policyText =
|
||||
readStanza "${credentialMountPath}/data/swarm/services/${hyperhiveCfg.swarm.grafana.oidc.clientId}/oidc/client"
|
||||
+ createStanza "${credentialMountPath}/data/${grafanaAdminLeaf}";
|
||||
}
|
||||
];
|
||||
|
||||
|
|
@ -916,6 +929,10 @@ let
|
|||
userSeedLeaf = "swarm/services/swarm-nats/callout/user-seed";
|
||||
userPublicLeaf = "swarm/public/swarm-nats/callout-user";
|
||||
|
||||
# Grafana's admin password, which only Grafana's leaf creates. The nix half
|
||||
# of `adminPasswordStorePath` in ./swarm-grafana.nix.
|
||||
grafanaAdminLeaf = "swarm/services/${hyperhiveCfg.swarm.grafana.oidc.clientId}/admin/password";
|
||||
|
||||
createStanza = path: ''
|
||||
path "${path}" {
|
||||
capabilities = ["create", "read"]
|
||||
|
|
|
|||
|
|
@ -253,6 +253,15 @@ let
|
|||
# the OIDC secret above, whose other reader is authelia's container.
|
||||
secretKeyPath = "/var/lib/grafana-secret/secret_key";
|
||||
|
||||
# The built-in `admin` account's password. Grafana's API takes basic auth
|
||||
# whatever the login form says, so this account is reachable from the public
|
||||
# vhost and its password cannot be the upstream default. Minted once into the
|
||||
# store by `swarm-bao-grafana-admin` below, which `swarm-bao.nix`'s
|
||||
# `grafanaOidcReaders` grants create and read on this one leaf.
|
||||
adminPasswordStorePath = "secret/swarm/services/${cfg.oidc.clientId}/admin/password";
|
||||
adminPasswordPath = "/var/lib/grafana-admin/password";
|
||||
hostAdminPasswordPath = "/var/lib/nixos-containers/${cfg.machine}${adminPasswordPath}";
|
||||
|
||||
socketPath = "${deployCfg.grafana.socketDir}/grafana.sock";
|
||||
|
||||
# Both static NixOS ids, and both checked rather than assumed: nginx has
|
||||
|
|
@ -656,6 +665,97 @@ in
|
|||
'';
|
||||
};
|
||||
|
||||
# The admin password: created in the store if it holds none, then copied
|
||||
# into the container, under Grafana's own leaf. Same shape as
|
||||
# `swarm-bao-nats-callout-xkey` in ./swarm-nats.nix. The grant is
|
||||
# `create` without `update`, so the first password stored is the one every
|
||||
# later run reads, and a race between two runs leaves one value.
|
||||
#
|
||||
# Grafana does not start without the file (`$__file{}` on a missing path is
|
||||
# a startup error), so a store that is unreachable on first boot keeps
|
||||
# Grafana down rather than up with the upstream default.
|
||||
systemd.services.swarm-bao-grafana-admin = {
|
||||
description = "create Grafana's admin password in the swarm secret store if absent and fetch it";
|
||||
after = lib.optionals baoDeploy.enable [
|
||||
"swarm-bao-pki.service"
|
||||
"container@${baoCfg.machine}.service"
|
||||
];
|
||||
wants = lib.optionals baoDeploy.enable [ "container@${baoCfg.machine}.service" ];
|
||||
requires = lib.optionals baoDeploy.enable [ "swarm-bao-pki.service" ];
|
||||
before = [ "container@${cfg.machine}.service" ];
|
||||
wantedBy = [
|
||||
"multi-user.target"
|
||||
"container@${cfg.machine}.service"
|
||||
];
|
||||
path = [
|
||||
baoDeploy.package
|
||||
pkgs.coreutils
|
||||
pkgs.openssl
|
||||
pkgs.systemd
|
||||
];
|
||||
inherit (storeRetry) startLimitBurst startLimitIntervalSec;
|
||||
serviceConfig = storeRetry.serviceConfig // {
|
||||
Type = "oneshot";
|
||||
RemainAfterExit = true;
|
||||
SyslogIdentifier = "swarm-bao-grafana-admin";
|
||||
TimeoutStartSec = 30;
|
||||
};
|
||||
environment = {
|
||||
BAO_ADDR = "https://${baoCfg.domain}:${toString baoCfg.port}";
|
||||
BAO_CLIENT_CERT = baoDeploy.grafanaOidcClientCertFile;
|
||||
BAO_CLIENT_KEY = baoDeploy.grafanaOidcClientKeyFile;
|
||||
}
|
||||
// lib.optionalAttrs (baoDeploy.serverCaFile != null) {
|
||||
BAO_CACERT = baoDeploy.serverCaFile;
|
||||
};
|
||||
script = ''
|
||||
set -euo pipefail
|
||||
|
||||
${atomicWriteSecret}
|
||||
${refreshConsumer}
|
||||
|
||||
err="$(mktemp)"
|
||||
trap 'rm -f "$err"' EXIT
|
||||
|
||||
if ! BAO_TOKEN="$(bao login -method=cert -token-only 2>"$err")"; then
|
||||
echo "could not log in to swarm-bao with Grafana's certificate; leaving Grafana's admin password file as it is." >&2
|
||||
cat "$err" >&2
|
||||
exit 1
|
||||
fi
|
||||
export BAO_TOKEN
|
||||
|
||||
# `bao kv get` exits 2 both for "nothing stored" and for a refusal, so
|
||||
# a failed read always tries the create, and the second read decides.
|
||||
# The value reaches `bao` on stdin, never in an argument.
|
||||
if ! password="$(bao kv get -field=value ${lib.escapeShellArg adminPasswordStorePath} 2>"$err")"; then
|
||||
minted="$(openssl rand -hex 32)"
|
||||
if ! printf '%s' "$minted" | bao kv put -cas=0 ${lib.escapeShellArg adminPasswordStorePath} value=- >/dev/null 2>"$err"; then
|
||||
echo "could not create ${adminPasswordStorePath}; reading it back in case another run created it." >&2
|
||||
cat "$err" >&2
|
||||
fi
|
||||
if ! password="$(bao kv get -field=value ${lib.escapeShellArg adminPasswordStorePath} 2>"$err")"; then
|
||||
echo "swarm-bao did not return ${adminPasswordStorePath}; Grafana does not start without it." >&2
|
||||
cat "$err" >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
if [ -z "$password" ]; then
|
||||
echo "swarm-bao returned an empty ${adminPasswordStorePath}; leaving the file as it is." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Owner and mode as for the OIDC secret above, for its reasons.
|
||||
install -d -m 0755 ${lib.escapeShellArg (builtins.dirOf hostAdminPasswordPath)}
|
||||
if secret_differs ${lib.escapeShellArg hostAdminPasswordPath} "$password"; then
|
||||
atomic_write_secret 0400 ${lib.escapeShellArg "${toString config.ids.uids.grafana}:0"} ${lib.escapeShellArg hostAdminPasswordPath} "$password"
|
||||
fi
|
||||
|
||||
# Grafana's `preStart` applies the file to the database, so a file that
|
||||
# changed after it started needs a restart.
|
||||
refresh_consumer ${lib.escapeShellArg cfg.machine} grafana.service ${lib.escapeShellArg hostAdminPasswordPath}
|
||||
'';
|
||||
};
|
||||
|
||||
containers.${cfg.machine} = {
|
||||
autoStart = true;
|
||||
ephemeral = false;
|
||||
|
|
@ -681,7 +781,7 @@ in
|
|||
// caTrust.bindMount;
|
||||
|
||||
config =
|
||||
{ ... }:
|
||||
{ config, ... }:
|
||||
{
|
||||
imports = [
|
||||
../container-modules/swarm-container.nix
|
||||
|
|
@ -752,6 +852,32 @@ in
|
|||
'';
|
||||
};
|
||||
|
||||
# Upstream applies `security.admin_password` only when it creates the
|
||||
# database, so a database created before that setting keeps the
|
||||
# password it was created with. This resets it from the delivered
|
||||
# file on every start, before the server listens.
|
||||
#
|
||||
# `grafana cli` must open the database the server opens, which the
|
||||
# server's config.ini names. nixpkgs keeps that file private, so it
|
||||
# is rendered again from the same settings with nixpkgs' formatter.
|
||||
systemd.services.grafana.preStart =
|
||||
let
|
||||
grafana = config.services.grafana;
|
||||
configFile =
|
||||
(pkgs.formats.ini {
|
||||
listToValue = lib.concatMapStringsSep " " (lib.generators.mkValueStringDefault { });
|
||||
mkKeyValue = lib.generators.mkKeyValueDefault {
|
||||
mkValueString = v: if v == null then "" else lib.generators.mkValueStringDefault { } v;
|
||||
} "=";
|
||||
}).generate
|
||||
"config.ini"
|
||||
grafana.settings;
|
||||
in
|
||||
lib.mkAfter ''
|
||||
${lib.getExe' grafana.package "grafana"} cli --homepath ${grafana.dataDir} --config ${configFile} \
|
||||
admin reset-admin-password --password-from-stdin < ${adminPasswordPath}
|
||||
'';
|
||||
|
||||
# The group Grafana chgrps its socket to. Declared here because a
|
||||
# container has its own user database — the host knowing gid 60 as
|
||||
# `nginx` means nothing in here, only the number crosses.
|
||||
|
|
@ -837,10 +963,14 @@ in
|
|||
# whole host fails to build. The unit below generates it.
|
||||
security.secret_key = "$__file{${secretKeyPath}}";
|
||||
|
||||
# No local password path, ever. This is not tidiness: Grafana
|
||||
# ships an `admin`/`admin` account and this vhost is on the
|
||||
# public gateway, so SSO is the only way in — not merely the
|
||||
# preferred one.
|
||||
# The password a newly created database gets; the `preStart`
|
||||
# above gives an existing one the same. Written by
|
||||
# `swarm-bao-grafana-admin` on the host.
|
||||
security.admin_password = "$__file{${adminPasswordPath}}";
|
||||
|
||||
# No password box on the public login page: people log in through
|
||||
# SSO. This hides the form only; the API still takes basic auth,
|
||||
# which is why the admin password above is minted.
|
||||
#
|
||||
# Not conditional on anything, and the block below is not
|
||||
# either, which is what keeps the two from ever disagreeing.
|
||||
|
|
|
|||
|
|
@ -462,6 +462,23 @@ let
|
|||
"update"
|
||||
];
|
||||
}
|
||||
{
|
||||
# Grafana's admin password sits under the publisher's `services/*` write
|
||||
# too, and a planted value is the password the next Grafana start applies
|
||||
# to the server-admin account. The OIDC leaf beside it is the control:
|
||||
# still the prefix's.
|
||||
name = "the publisher may not write Grafana's admin password";
|
||||
ok =
|
||||
let
|
||||
grants = grantsIn baoGrantHere.systemd.services.swarm-bao-secret-publisher-policy.script;
|
||||
in
|
||||
(grantFor grants "secret/data/swarm/services/swarm-grafana/admin/password").caps == [ "deny" ]
|
||||
&&
|
||||
(grantFor grants "secret/data/swarm/services/swarm-grafana/oidc/client").caps == [
|
||||
"create"
|
||||
"update"
|
||||
];
|
||||
}
|
||||
{
|
||||
# The ordering is load-bearing and invisible at runtime: the controller's
|
||||
# unit creates the KV and cert-auth mounts this one writes into, so
|
||||
|
|
@ -557,13 +574,28 @@ let
|
|||
# for this one is the OTHER service's: `services/*` would have granted
|
||||
# both, and the two are separate principals precisely because a
|
||||
# dashboard is not entitled to a collector's credential.
|
||||
name = "the Grafana OIDC reader's grant is Grafana's own client secret and nothing else";
|
||||
#
|
||||
# The admin password is the one leaf it may create, and without `update`:
|
||||
# a grant that could replace it would let a second run rotate the
|
||||
# password Grafana's database already holds.
|
||||
name = "the Grafana reader reads its own client secret and creates its admin password, and nothing else";
|
||||
ok =
|
||||
let
|
||||
s = baoGrantHere.systemd.services.swarm-bao-grafana-oidc-policy.script;
|
||||
in
|
||||
lib.hasInfix "path \"secret/data/swarm/services/swarm-grafana/oidc/client\" {" s
|
||||
&& lib.hasInfix "capabilities = [\"read\"]" s
|
||||
grantsIn s == [
|
||||
{
|
||||
path = "secret/data/swarm/services/swarm-grafana/oidc/client";
|
||||
caps = [ "read" ];
|
||||
}
|
||||
{
|
||||
path = "secret/data/swarm/services/swarm-grafana/admin/password";
|
||||
caps = [
|
||||
"create"
|
||||
"read"
|
||||
];
|
||||
}
|
||||
]
|
||||
&& !(lib.hasInfix "secret/data/swarm/agents" s)
|
||||
&& !(lib.hasInfix "secret/data/swarm/hives" s)
|
||||
&& !(lib.hasInfix "secret/data/swarm/services/*" s)
|
||||
|
|
@ -827,6 +859,20 @@ let
|
|||
&& !(noReaders ? swarm-bao-matrix-oidc)
|
||||
&& !(noReaders ? swarm-bao-nats-callout-xkey);
|
||||
}
|
||||
{
|
||||
# The admin password is created under Grafana's leaf, so its unit waits
|
||||
# for the unit writing that leaf's role and policy; before it, the create
|
||||
# is refused. The second arm is the absence control.
|
||||
name = "grafana's admin password unit is ordered after grafana's policy unit";
|
||||
ok =
|
||||
let
|
||||
u = baoGrantWithConsumers.systemd.services.swarm-bao-grafana-admin;
|
||||
in
|
||||
lib.elem "swarm-bao-grafana-oidc-policy.service" u.after
|
||||
&& lib.elem "swarm-bao-grafana-oidc-policy.service" u.wants
|
||||
&& !(lib.elem "swarm-bao-grafana-oidc-policy.service" u.requires)
|
||||
&& !(baoGrantNoReaders.systemd.services ? swarm-bao-grafana-admin);
|
||||
}
|
||||
{
|
||||
# The ordering is set apart from each reader's own definition, so it can
|
||||
# define a reader by itself: `after` on a unit nothing else declares is a
|
||||
|
|
@ -959,6 +1005,7 @@ let
|
|||
[
|
||||
"swarm-services-cert"
|
||||
"swarm-bao-forwarder-oidc"
|
||||
"swarm-bao-grafana-admin"
|
||||
]
|
||||
++ policyReaders
|
||||
);
|
||||
|
|
|
|||
|
|
@ -123,9 +123,8 @@ let
|
|||
{
|
||||
# This fixture enables grafana and NOT authelia, so the login-form toggle
|
||||
# has to follow "grafana requires SSO" rather than "both services are on
|
||||
# this host". Grafana ships an
|
||||
# `admin`/`admin` account and its vhost is on the public gateway, so a
|
||||
# password box there is a way in whatever the topology.
|
||||
# this host". Its vhost is on the public gateway, so a password box there
|
||||
# is a way in whatever the topology.
|
||||
name = "grafana disables its local login form even where authelia is not on this host";
|
||||
ok =
|
||||
grafanaOldPath.containers.swarm-grafana.config.services.grafana.settings.auth.disable_login_form;
|
||||
|
|
@ -196,6 +195,36 @@ let
|
|||
&& !(grafanaRefusedFor grafanaRemoteAuthelia "services.hyperhive.swarm.authelia.url")
|
||||
&& !(grafanaRefusedFor grafanaRemoteAuthelia "deploy.bao.grafanaOidcClientCertFile");
|
||||
}
|
||||
{
|
||||
# Grafana's API takes basic auth whatever the login form says, so the
|
||||
# built-in admin's password is the only thing between the public vhost
|
||||
# and server-admin. Upstream sets it only when it creates the database,
|
||||
# so both halves are pinned: the setting a new database reads, and the
|
||||
# reset every start applies to whichever database exists. Without the
|
||||
# reset, a database created before the setting keeps `admin`.
|
||||
#
|
||||
# The reset's `--config` must be the file the server reads, or it resets
|
||||
# a different database: compared by store path against the server's own
|
||||
# command line.
|
||||
name = "grafana's admin password is minted into the store and applied to new and existing databases";
|
||||
ok =
|
||||
let
|
||||
unit = grafanaRemoteAuthelia.systemd.services.swarm-bao-grafana-admin;
|
||||
grafana = grafanaRemoteAuthelia.containers.swarm-grafana.config.services.grafana;
|
||||
service = grafanaRemoteAuthelia.containers.swarm-grafana.config.systemd.services.grafana;
|
||||
resetConfig = lib.head (
|
||||
lib.head (lib.filter lib.isList (builtins.split "--config ([^ ]+)" service.preStart))
|
||||
);
|
||||
in
|
||||
lib.hasInfix "bao kv put -cas=0 secret/swarm/services/swarm-grafana/admin/password value=-" unit.script
|
||||
&& lib.hasInfix "openssl rand -hex 32" unit.script
|
||||
&& lib.hasInfix "atomic_write_secret 0400 196:0 /var/lib/nixos-containers/swarm-grafana/var/lib/grafana-admin/password" unit.script
|
||||
&& lib.elem "container@swarm-grafana.service" unit.before
|
||||
&& unit.environment.BAO_CLIENT_CERT == "/etc/pki/bao-grafana-oidc.pem"
|
||||
&& grafana.settings.security.admin_password == "$__file{/var/lib/grafana-admin/password}"
|
||||
&& lib.hasInfix "admin reset-admin-password --password-from-stdin < /var/lib/grafana-admin/password" service.preStart
|
||||
&& lib.hasInfix "-config ${resetConfig}" service.script;
|
||||
}
|
||||
{
|
||||
# Same 403-not-a-miss reason as the matrix and queue arms below: the
|
||||
# reader's grant covers the `services` prefix, so a path outside it is
|
||||
|
|
|
|||
Loading…
Reference in a new issue