After a failed or timed-out store read the responder denies for LOOKUP_FAILURE_WINDOW (3 s) without a new lookup, so a burst of requests during an outage costs one lookup instead of one per request.
Three edits besides the code in main.rs belong to this change:
- swarm-metrics-auth/Cargo.toml gains tokio's test-util dev-dependency because the burst test runs on paused time (start_paused) to let a hanging store sleep past LOOKUP_TIMEOUT without a real 5 s wait.
- docs/swarm/services.md gains the sentence that a failed or timed-out read is denied for three seconds without a retry, which documents the window this change adds.
- docs/swarm/credentials.md gains the clause "every three seconds while the store fails" on the Grafana bearer row, which documents the same window in the secrets table.
The metrics vhost forwarded the store's read API (query, series, labels,
export, vmui, ...) with no credential, so anything that could reach
metrics.<swarm> could read every hive's and agent's metrics. mara ruled on
checked by an auth_request responder, and operators use their authelia
session.
- swarm-metrics-auth (new crate): auth_request responder on the metrics
store's host. Compares the presented bearer, in constant time, with the
one stored at swarm/services/<grafana client id>/metrics/bearer, read
under its own bao leaf and cached for 60s. Static user, the post-#5264
swarm-logs-auth unit shape.
- swarm-bao-grafana-metrics-bearer: on Grafana's host, creates that bearer
in bao if absent (create without update, Grafana's existing leaf) and
writes "Bearer <value>" into the container; the datasource sends it as
its Authorization header via $__file{}.
- The vhost: read locations take auth_request through a map (a bearer goes
to swarm-metrics-auth, anything else to authelia, no login redirect);
vmui takes an authelia session with the login redirect; forceSSL like
the logs vhost. The authelia subrequest location is renamed from
/__metrics_push_authz to /__metrics_authelia since it now serves reads.
- Authelia: the collector keeps every path; everyone else is refused the
collector's push and /metrics paths; group admins is admitted to the
rest; then deny.
- bao: a swarm-metrics-auth cert role and policy (read on the one leaf),
its leaf minted by glue-bao-tls, the CN reserved as a hive name, and
Grafana's grant gains create on the bearer leaf.
Fixes#3596
