Watch
0
0
Fork
You've already forked hyperhive
0
hyperhive/nix/collector-own-export-failures.nix

21 lines
1,017 B
Nix

# The `filter` processor every collector's logs pipeline carries so that a
# collector never ships its own export-failure lines.
#
# A collector that cannot reach its exporter logs one line per failed batch
# (`Exporting failed. Will retry the request after interval.`, `Exporting
# failed. Dropping data.`, and the 503 body `sending queue is full`) into the
# journal its own journald receiver reads. Shipping those lines is more
# export traffic during the outage that caused them.
#
# Only the collector's own unit, and only exporter failures: its startup,
# config and receiver errors still ship. The failure stays visible as the
# collector's `otelcol_exporter_send_failed_*` metrics.
#
# `body` is the journal entry as a map. A record whose body is not a map
# raises an OTTL error, which `ignore` turns into "kept".
{
error_mode = "ignore";
logs.log_record = [
''body["_SYSTEMD_UNIT"] == "opentelemetry-collector.service" and IsMatch(body["MESSAGE"], "Exporting failed|sending queue is full")''
];
}