ci: coverage on manual dispatch, no threshold
Adds `cargo llvm-cov` to the devshell and a coverage workflow that runs only when someone asks for it. Manual dispatch rather than nightly or per-PR, per the discussion on the issue: a coverage run needs its own instrumented build and cannot reuse the normal test artifacts, so it roughly doubles a test job. "Do we have glaring holes" is a question someone asks occasionally, not a gate every PR pays for, and not a number worth spending farm time on every night whether or not anyone reads it. Manual dispatch costs nothing until the answer is wanted. Its own workflow file rather than a job in ci.yml: that file already has a `workflow_dispatch` trigger so `hive-forge ci-rerun` can retrigger without an empty commit, and a trigger there fires EVERY job in the file -- a coverage job added there would run on every pull request. No threshold and no --fail-under-lines. A coverage gate mostly teaches people to write assertion-free tests that execute lines; the report is the deliverable and the number is for a human to read. The devshell needs LLVM_COV / LLVM_PROFDATA set explicitly: cargo llvm-cov expects rustup's `llvm-tools-preview` beside the toolchain and nixpkgs has no such component, so without them it aborts with "failed to find llvm-tools-preview" -- which reads like a missing install rather than a path the shell has to name. Verified by running it: `cargo llvm-cov --package hive-types` produces a real report (3 tests, 85.44% regions). Both the variable names and the package were wrong on the first attempt and only running it said so -- the names take no `_PATH` suffix, and the binaries are in `llvmPackages.llvm`, not `llvmPackages.bintools`, which is the linker wrapper and ships neither.
This commit is contained in:
parent
1aa35974ee
commit
1e3347fb46
2 changed files with 61 additions and 0 deletions
40
.forgejo/workflows/coverage.yml
Normal file
40
.forgejo/workflows/coverage.yml
Normal file
|
|
@ -0,0 +1,40 @@
|
|||
name: coverage
|
||||
|
||||
# Manual dispatch ONLY — deliberately not on `pull_request`, and
|
||||
# deliberately not nightly.
|
||||
#
|
||||
# A coverage run needs its own instrumented build: it cannot reuse the
|
||||
# normal test artifacts, so it roughly doubles a test job. Coverage here
|
||||
# answers "do we have glaring holes", which is a question someone asks
|
||||
# occasionally — not a gate every PR pays for, and not a number worth
|
||||
# spending farm time on every night whether or not anyone reads it.
|
||||
# Manual dispatch costs nothing until someone wants the answer.
|
||||
#
|
||||
# Its own file rather than a job in ci.yml: that workflow already has a
|
||||
# `workflow_dispatch` trigger (so `hive-forge ci-rerun` can retrigger
|
||||
# without an empty commit), and a trigger there fires EVERY job in the
|
||||
# file. A coverage job added to ci.yml would run on every pull request.
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
coverage:
|
||||
name: cargo llvm-cov
|
||||
runs-on: [hive-ci]
|
||||
# Above the 30 min `nix flake check` allows, because instrumented
|
||||
# builds are slower than the ordinary ones and this starts from a
|
||||
# cache that the normal jobs never populate.
|
||||
timeout-minutes: 60
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- name: coverage
|
||||
# No threshold and no `--fail-under-lines`: a coverage gate
|
||||
# mostly teaches people to write assertion-free tests that
|
||||
# execute lines. The report is the deliverable; the number is
|
||||
# for a human to look at.
|
||||
#
|
||||
# `--workspace` because the interesting holes are in the crates
|
||||
# nobody runs directly, and the per-crate default would hide
|
||||
# exactly those behind a green summary for the one crate someone
|
||||
# happened to name.
|
||||
run: nix develop -c cargo llvm-cov --workspace --summary-only
|
||||
Loading…
Reference in a new issue