---
name: swarm-metrics
description: Query the swarm metrics store with `swarm-metrics` when you need to know whether a service's metrics reach VictoriaMetrics, which metric names or labels exist, or what a counter or gauge did over a window. Covers how to match dotted OTLP metric names, how to bound a query, and how to tell an empty answer from a refused request. Use this before running `swarm-metrics` for the first time in a session, and before you report a metric as missing.
---

# swarm-metrics

`swarm-metrics` reads the swarm's VictoriaMetrics through the gateway as you,
the agent. Every verb prints the store's JSON answer, so pipe it into `jq`.

## Find out what exists first

Metric names from OTLP keep their dots (`jobq.nodes`,
`hyperhive.vcs.commit.count`), and so do OTLP attribute labels
(`service.name`). Match a dotted metric by `__name__`, and quote a dotted
label name:

```console
$ swarm-metrics label-values __name__ --match '{__name__=~"jobq\\..*"}' | jq -r '.data[]'
$ swarm-metrics label-values service.name | jq -r '.data[]'
$ swarm-metrics series '{"service.name"="nats"}' --start -1h --limit 20
```

`series`, `labels` and `label-values` cover the last day from 00:00 UTC
unless you pass `--start`/`--end`. Use `--limit` on a broad selector.

## Query

```console
$ swarm-metrics query 'count by (__name__) ({__name__=~"hyperhive\\.vcs\\..*"})'
$ swarm-metrics query-range 'sum(rate({__name__="jobq.nodes"}[5m]))' --start -6h --step 5m
```

Times take RFC 3339, a unix timestamp or a relative duration (`-1h`).
`query-range` requires `--start`, so a range is always bounded. Inside the
expression, a regex needs its dots escaped twice (`\\.`), once for the
PromQL string and once for the regex.

## Telling "no series" from "refused"

- **No matching series**: exit 0, and `.data.result` (or `.data`) is `[]`.
- **Refused** (auth, bad expression, rate limit): non-zero exit, with the
  HTTP status and the body on stderr. A 403 is the gateway's rate limit.

## Before you report a metric missing, run a control

An empty answer fits a wrong name, a missing label value, or a window too
narrow just as well as a metric that never arrived. Query something that must
exist, such as `label-values service.name`, and widen `--start`. If the
control is also empty, the query is wrong, not the store.