swarm-authelia: run the secret publisher and a journal forwarder inside the container
swarm-secret-publish and its path unit now run inside authelia's container, beside the mint, and read each client secret at the path authelia writes it. The host unit and host path unit, which read the secrets through the container's root on the host, are gone, and so is deploy.authelia.hostClientSecretDir, which only they read. The publisher keeps its store identity, the secret-publisher leaf. The host hands the leaf and the store's CA into the container as nspawn system credentials, and the unit loads them; where this host mints the leaf, the container waits for swarm-bao-pki. Authelia does not wait for the publisher, which retries every 30 seconds while the store is down. The container also runs the journal forwarder the other service containers run, pushing to a receiver of its own on the swarm collector (serviceProducers swarm-authelia) as a new machine OIDC client (swarm.authelia.otel.clientId) that authelia registers for itself and the publisher carries to the store. The fetch unit reads it back under the publisher's leaf, also handed to it as a system credential, so the swarm-secret-publisher policy gains read on that one leaf; its stanza keeps create and update, which the exact path would otherwise take from the services/* prefix. Without the leaf the forwarder is off. The container drops --link-journal=host, and the host collector stops reading it. The services' "nothing at <path>" errors name the publisher in authelia's container.
This commit is contained in:
atlas
mara
parent
a2dca32bb8
commit
b68a8ef290
36 changed files with 753 additions and 363 deletions
|
|
@ -54,6 +54,7 @@ neither is a renaming of the other.
|
|||
| Grafana's admin password | `swarm-bao-grafana-admin.service` creates it in the swarm secret store at `swarm/services/<grafana client id>/admin/password` when the store holds none, then copies it out, **on every host that runs Grafana**. Grafana's leaf may create that leaf but not update it, so the store keeps the first value, and the secret publisher can't write it. To rotate, run `bao kv metadata delete secret/swarm/services/<grafana client id>/admin/password` with a token that may, such as the root token (Grafana's, the publisher's and the controller's can't), then `systemctl restart swarm-bao-grafana-admin`; a plain `bao kv delete` isn't enough, see [the callout xkey's section](#the-queues-callout-xkey) | `/run/swarm-grafana-secrets/admin-password` on the host's tmpfs, bind-mounted read-only into the `swarm-grafana` container, never on persistent disk: `security.admin_password` for a new database, and `grafana cli admin reset-admin-password` in `grafana.service`'s `preStart` for an existing one, on every start | same unit, same path. After a reboot the unit fetches the file again, and Grafana doesn't start until it exists, so it stays down rather than falling back to the upstream default `admin` |
|
||||
| Grafana's metrics bearer | `swarm-bao-grafana-metrics-bearer.service` creates it in the swarm secret store at `swarm/services/<grafana client id>/metrics/bearer` when the store holds none, then copies it out, **on every host that runs Grafana**. Same grant as the admin password, create without update. To rotate, run `bao kv metadata delete secret/swarm/services/<grafana client id>/metrics/bearer` with a token that may, then `systemctl restart swarm-bao-grafana-metrics-bearer` | `/run/swarm-grafana-secrets/metrics-bearer` on the host's tmpfs, bind-mounted read-only into the `swarm-grafana` container, holding `Bearer <value>`: the metrics datasource's `Authorization` header. The logs datasource sends the same header. `swarm-metrics-auth` on the metrics store's host, and a second instance on the log store's host (`swarm-logs-grafana-auth`), read the stored value themselves, each under its own leaf | same unit, same path. The unit restarts Grafana when the file changes |
|
||||
| the metrics store responder's own store leaf (cert + key) | `signLeaf` in `glue-bao-tls.nix` (`swarm-bao-pki.service`), on the host that runs the secret store, once, subject `deploy.bao.metricsAuthCommonName`. `swarm-bao-metrics-auth-policy.service` writes the `swarm-metrics-auth` role and its read grants on the bearer leaf and on the metrics store forwarder's client secret, nothing else | `/var/lib/swarm-bao-pki/metrics-auth.pem` and `metrics-auth-key.pem` on the store's host, handed to `swarm-metrics-auth` through `LoadCredential` (paths in argv, never values) and read by path by `swarm-victoriametrics-forwarder-oidc` | **the operator** places a leaf with that subject on the metrics store's host and names it in `deploy.victoriametrics.authBaoClientCertFile` / `authBaoClientKeyFile`; evaluation fails naming both until then. If the store is on another host, deploy that host first: the role must exist before the responder's login succeeds, and until it does the gateway refuses every Grafana query (fail closed) |
|
||||
| the secret publisher's store leaf (cert + key) | `signLeaf` in `glue-bao-tls.nix` (`swarm-bao-pki.service`), on the host that runs the secret store, once, subject `deploy.bao.secretPublisherCommonName`. `swarm-bao-secret-publisher-policy.service` writes the `swarm-secret-publisher` role: create and update under `swarm/hives/*`, `swarm/services/*` and the controller's client, and read on authelia's forwarder's client secret alone | `/var/lib/swarm-bao-pki/secret-publisher.pem` and `secret-publisher-key.pem` on the host, handed into authelia's container as nspawn system credentials, which `swarm-secret-publish` and the forwarder's `swarm-journal-forwarder-oidc` load with `LoadCredential`. nspawn won't start authelia's container while a named file is missing | **the operator** copies the leaf to authelia's host, beside the hive's own store leaf, where `glue-secret-publisher-bao-identity.nix` looks, or names it in `deploy.swarm-secret-publisher.baoClientCertFile` / `baoClientKeyFile` |
|
||||
| the swarm controller's copy of its OIDC secret | `swarm-controller` reads it out of the swarm secret store at start, under its own leaf | the daemon's memory only; never on disk | same read, co-located or not. Nothing for an operator to place beyond the controller's store leaf (`deploy.swarm-controller.baoClientCertFile`) |
|
||||
| a hive's copy of its own OIDC secret | hive-c0re reads it out of the swarm secret store when it first connects to the queue, under the hive's own leaf | the daemon's memory only; never on disk | same read, co-located or not. Nothing for an operator to place beyond the hive's store leaf (`deploy.bao.clientCertFile`) |
|
||||
| the hive collector's copy of its hive's OIDC secret | `swarm-bao-hive-otel-oidc.service` reads it out of the swarm secret store under the hive's own leaf, **on every hive that runs the collector and holds that leaf** | `/var/lib/hive-otel-oidc/hive-<name>.secret` on the hive's host, 0400 root, loaded by `LoadCredential` | same unit, same path, co-located or not. Nothing for an operator to place beyond the hive's store leaf (`deploy.bao.clientCertFile`) |
|
||||
|
|
@ -66,6 +67,7 @@ neither is a renaming of the other.
|
|||
| the metrics store container's journal forwarder's OIDC secret | `swarm-victoriametrics-forwarder-oidc.service` on the store's host reads its forwarder client's secret (`swarm.victoriametrics.otel.clientId`) out of the swarm secret store under the metrics store responder's leaf, and again every 10 minutes: the container has no network to reach the store | `/run/swarm-victoriametrics-forwarder-oidc/<id>.secret` on the host (tmpfs), root `0400`, never on persistent disk; bound into the container read-only, where `LoadCredential` hands it to the collector | same unit, same path. Nothing for an operator to place beyond that leaf (`deploy.victoriametrics.authBaoClientCertFile`) |
|
||||
| the CI run poller's forge token | `swarm-controller`, with the forge's admin API, for its `ci-metrics` account, at start and in a five-minute pass | the swarm secret store at `swarm/services/ci-metrics/forge/token`; `swarm-ci-metrics` reads it once per pass under its own leaf and holds it in memory for that pass. When the store's token no longer matches the forge's, the controller's next pass mints a new one; until then the poller's forge calls answer 401, the unit fails and no cursor moves | nothing to place beyond the poller's leaf (`deploy.bao.ciMetricsClientCertFile`), on a host without the store |
|
||||
| the CI run poller's copy of its OIDC secret | authelia; `swarm-secret-publish` copies it into the swarm secret store | `swarm-ci-metrics` reads `swarm/services/<swarm.ci-metrics.clientId>/oidc/client` once per pass under its own leaf, and again when authelia refuses it; memory only. While it differs from authelia's, the token request fails, nothing gets pushed and no cursor moves | same leaf, nothing else |
|
||||
| authelia's container's journal forwarder's OIDC secret | authelia's mint, in its container, like every client's plaintext half (`swarm.authelia.otel.clientId`). `swarm-secret-publish` copies it to `swarm/services/<id>/oidc/client`, and `swarm-journal-forwarder-oidc.service` in the same container reads it back under the secret publisher's leaf, at start and every 10 minutes. The two copies can differ while the publisher hasn't caught up, for example while the store is down after a re-mint. Authelia then refuses the forwarder's token request, and the collector retries until the next read lands the new value and restarts it | `/run/swarm-journal-forwarder-oidc/client-secret` on the container's tmpfs, root `0400`; `LoadCredential` hands it to the collector. The mint's own copy stays in `/var/lib/authelia-swarm/oidc-clients/<id>.secret` | none: the forwarder runs only in authelia's container, and is off there without the publisher's leaf |
|
||||
| authelia subject store | `swarmctl` and `swarm-authelia-bridge` | `users.yml` — one file, read and written by both | `swarmctl`, on the host that runs authelia |
|
||||
| wireguard private key | **the operator** — `wg genkey` | whatever `deploy.wireguard.privateKeyFile` names | always operator-provided; nothing generates this for you |
|
||||
| queue auth-callout nkeys (user seed + account seed) | `swarm-nats-auth ensure-callout-keys`, run by `swarm-bao-nats-callout-xkey` on the queue's host when the store holds no seed; the verb stores a seed it finds at `/var/lib/swarm-nats-callout/{callout-user,issuer}.seed` as is, then removes the file | the swarm secret store: the seeds at `swarm/services/swarm-nats/callout/{user,issuer}-seed`, read by the `swarm-nats-auth` role alone; the public halves at `swarm/public/swarm-nats/callout-{user,issuer}` | nothing to place beyond the responder's store leaf (`deploy.nats.authBaoClientCertFile`) |
|
||||
|
|
@ -262,8 +264,8 @@ such as the root token.
|
|||
The forge and the homeserver read their copies out of the store, the way
|
||||
Grafana does below: the forge under the forge publisher's leaf, the
|
||||
homeserver under matrix-ctl's. Each **fails loudly** and retries until the
|
||||
publisher on authelia's host has written the secret, rather than skipping. A
|
||||
silent skip produces a service whose login button always fails, which is a
|
||||
publisher in authelia's container has written the secret, rather than skipping.
|
||||
A silent skip produces a service whose login button always fails, which is a
|
||||
symptom many layers from its cause.
|
||||
|
||||
The store's **first reader** is the agent containers' queue credential:
|
||||
|
|
@ -272,7 +274,7 @@ the client id it authenticates, because that's how a queue client reads them —
|
|||
the secret by path, the id by value. No local fallback exists, and none is
|
||||
possible, so absent files mean an agent built before `swarm-logs` read its own
|
||||
queue secret has no working `swarm-logs`, which is what a swarm looks like
|
||||
before the publisher on the authelia host has run.
|
||||
before the publisher in authelia's container has run.
|
||||
The reader runs before `hive-c0re.service`, which wants it (doesn't require
|
||||
it), so an agent container never renders ahead of the credential; an
|
||||
unreachable store delays the daemon's start rather than failing it.
|
||||
|
|
@ -296,8 +298,8 @@ The **second reader** is Grafana's OIDC client secret, and it's the first one
|
|||
that's a _swarm service's_ own credential rather than a hive's. It has exactly
|
||||
one delivery route: `swarm-bao-grafana-oidc.service` reads
|
||||
`swarm/services/<client-id>/oidc/client` out of the store, on **every** host
|
||||
that runs Grafana. Where authelia is doesn't enter into it — the publisher on
|
||||
authelia's host writes that path whether the reader is a network away or in the
|
||||
that runs Grafana. Where authelia is doesn't enter into it — the publisher in
|
||||
authelia's container writes that path whether the reader is a network away or in the
|
||||
container next door.
|
||||
|
||||
That's a deliberate refusal of the shortcut. Copying the plaintext
|
||||
|
|
|
|||
Loading…
Reference in a new issue