add swarm-metrics, an agent CLI for the metrics store
swarm-metrics reads the swarm's VictoriaMetrics from inside an agent
container: query, query-range, series, labels and label-values, each one
GET against the store's Prometheus read API through the gateway, printing
the store's JSON. It authenticates the way swarm-logs does, with the
agent's own queue secret read from bao under the agent's leaf, sent as
a swarm-agent bearer.
On the metrics store's host the read API's auth map gains an agent arm
ahead of the Grafana one. It routes to swarm-metrics-agent-auth, a
swarm-logs-auth instance logged in under the existing swarm-metrics-auth
leaf, whose bao grant now includes swarm/agents/+/queue read. The check is
rate-limited per address like the log store's
(deploy.victoriametrics.agentAuth{Rate,Burst,VerdictTtl}).
nix/agent-modules/metrics.nix installs the wrapped binary when the hive
forwards HIVE_AGENT_METRICS_URL (from swarm.victoriametrics.domain) and a
store address, the same gate as swarm-logs. Adds the base:swarm-metrics
skill and the CLI reference with its freshness check.
Fixes #5395
This commit is contained in:
parent
1e8c398132
commit
e92f118bda
30 changed files with 1417 additions and 88 deletions
|
|
@ -1,3 +1,7 @@
|
|||
| `deploy.victoriametrics.agentAuthRate` | Default `50r/s` per client address, for agent reads checked by `swarm-metrics-agent-auth`; over it and `agentAuthBurst`, the gateway answers 403. |
|
||||
| `deploy.victoriametrics.agentAuthBurst` | Default `500` requests above `agentAuthRate` before the 403s start. |
|
||||
| `deploy.victoriametrics.agentAuthVerdictTtl` | Default `60` seconds that `swarm-metrics-agent-auth` keeps granting an agent secret it verified. A replaced or deleted secret keeps working at the metrics store for up to that long. |
|
||||
|
||||
# Swarm-wide services
|
||||
|
||||
Some things exist once per **swarm** rather than once per hive: the forge,
|
||||
|
|
@ -201,11 +205,14 @@ VictoriaMetrics authenticates nothing itself; the `metrics.<swarm-domain>`
|
|||
vhost does it:
|
||||
|
||||
- **The read API** (the Prometheus query, series, label and metadata
|
||||
routes): Grafana's datasource or an operator. Grafana sends a bearer that
|
||||
`swarm-bao-grafana-metrics-bearer` creates in the swarm secret store, and
|
||||
`swarm-metrics-auth` on the store's host checks it against the same store
|
||||
path. A request without a bearer needs an operator's authelia session
|
||||
(group `admins`); without one it gets a 401, not a login redirect.
|
||||
routes): Grafana's datasource, an agent, or an operator. Grafana sends a
|
||||
bearer that `swarm-bao-grafana-metrics-bearer` creates in the swarm secret
|
||||
store, and `swarm-metrics-auth` on the store's host checks it against the
|
||||
same store path. An agent's `swarm-metrics` sends its
|
||||
`swarm-agent.<agent>.<secret>` bearer, which `swarm-metrics-agent-auth` on
|
||||
the store's host checks against `swarm/agents/<agent>/queue`. A request
|
||||
without a bearer needs an operator's authelia session (group `admins`);
|
||||
without one it gets a 401, not a login redirect.
|
||||
- **vmui**: an operator's authelia session, with a redirect to the login.
|
||||
The bare domain redirects to vmui.
|
||||
- **The collector's OTLP push, and its scrape of the store's own
|
||||
|
|
@ -213,13 +220,16 @@ vhost does it:
|
|||
any other session or client there, operators included.
|
||||
- **Every other path**, the store's write and admin API included: 403.
|
||||
|
||||
`swarm-metrics-auth` reads the store with a leaf of its own,
|
||||
`deploy.victoriametrics.authBaoClientCertFile` / `authBaoClientKeyFile`.
|
||||
`swarm-metrics-auth` and `swarm-metrics-agent-auth` read the store with one
|
||||
leaf, `deploy.victoriametrics.authBaoClientCertFile` / `authBaoClientKeyFile`.
|
||||
Where the secret store runs on the same host, these default to the leaf it
|
||||
mints. On any other host you place a leaf whose CN is
|
||||
`deploy.bao.metricsAuthCommonName`, and evaluation refuses the config until
|
||||
you set both. It reads the stored bearer at most once a minute, so a
|
||||
replaced bearer starts working up to a minute later.
|
||||
you set both. `swarm-metrics-auth` reads the stored bearer at most once a
|
||||
minute, so a replaced bearer starts working up to a minute later.
|
||||
`swarm-metrics-agent-auth` is the log store's `swarm-logs-auth` binary, with
|
||||
the same grant TTL and rate limit, under `deploy.victoriametrics.agentAuth*`
|
||||
(table above).
|
||||
|
||||
The store's container has no network, so its journal forwarder leaves through
|
||||
a relay. Inside the container, `swarm-victoriametrics-egress-relay` listens on
|
||||
|
|
|
|||
Loading…
Reference in a new issue