The test dashboard

One static HTML page about the quality of a test suite: what the tests cover, which paths they reach, which mutants they kill, what is duplicated, and how the module behaves on CPAN Testers. Nothing is served by a program — it is files you can publish anywhere.

The distribution's own dashboard is generated this way on every push.

What it shows

Section	Content
Coverage	Per-file statement, branch, condition and subroutine percentages, with a Total row, a Δ column against the previous run, and a sparkline of recent history for each file.
Trend chart	Aggregate coverage per commit, read from the archived snapshots in `coverage_history/`, plus the current run appended as the last point.
Mutation	Per-file killed, survived and skipped counts, kill score, cyclomatic complexity, TER1/TER2/TER3 path metrics, and a duplication percentage.
Per-file mutation pages	A heatmap of the source with each mutant, whether it survived, and the exact change that was made.
Duplication	Copy-paste blocks over ten lines, and per-file drill-down pages with duplicated lines highlighted and cross-links between the copies.
CPAN Testers	Recent FAIL and UNKNOWN reports, with a confidence badge, root-cause grouping, and a downloadable reproduction script per failure.

Where the numbers come from

Input	Produced by
`cover_html/cover.json`	`cover -report json` after a Devel::Cover test run.
`cover_html/lcsaj_hits.json`	The LCSAJ runtime debugger during a second test run, merged from per-process hit files.
`cover_html/mutation_html/lib/*/.lcsaj.json`	`instrument-modules`: the control-flow paths themselves.
`mutation.json`	`test-generator-mutate --json mutation.json`.
`cover_html/cpd.json`	Copy-paste detection data.
`coverage_history/*.json`	Archived snapshots, one per commit, for the trend and the delta column.

The paths are fixed rather than configurable, so a local run and the CI run cannot disagree about where the data lives.

TER1, TER2 and TER3

Statement and branch coverage both have blind spots: a branch can be taken and still leave a later combination of conditions unexercised. LCSAJ — linear code sequence and jump — counts whole paths through a routine.

Metric	Meaning
TER1	Every statement executes at least once. The weakest of the three.
TER2	Every jump is taken. Roughly branch coverage at the path level.
TER3	Every path from a jump to the next jump is followed. This is what a linear code sequence and jump analysis adds.

instrument-modules derives the paths; the runtime debugger records which ones a test run actually took; the dashboard joins the two. The debugger is loaded as Devel::App::Test::Generator::LCSAJ::Runtime and needs PERL5LIB set separately from PERL5OPT, so the module can be found before the debugger initialises.

Generating it

# the whole pipeline: coverage, LCSAJ, mutation, HTML
bin/generate-test-dashboard

That is a bash script that runs the suite twice — once under Devel::Cover, once under the LCSAJ debugger — then mutation testing, then test-generator-index. It takes no options; the environment supplies the repository name, which decides the -select pattern.

If the individual steps already exist, test-generator-index on its own rebuilds the HTML from them in a second.

Putting it in another repository

deploy-workflows --target /path/to/My-Widget --dry-run
deploy-workflows --target /path/to/My-Widget

Two workflow files arrive in .github/workflows/:

dashboard.yml — builds, runs the suite twice under instrumentation, runs mutation testing, generates the dashboard, commits the fresh data and publishes to GitHub Pages. Its commits are guarded on github.event_name != 'pull_request', so a pull request produces a report without pushing it.
mutate.yml — a shorter mutation-only run, for a faster signal on a schedule.

Both are self-contained: no script outside the file, no path into the ATG source tree. That is deliberate — a dependent repository receives only these two files, so anything they referenced elsewhere would fail at runtime. Third-party actions are pinned to commit SHAs, not tags.

To publish, enable Pages for the gh_pages branch in the repository settings. deploy-workflows prints the git add, commit and push lines to run afterwards.

Reducing the cost

The pipeline is thorough and therefore slow. Three levers keep a run reasonable:

Network access. The CPAN Testers section fetches report pages. The shipped workflow sets NO_NETWORK_TESTING=1, which disables all of it — the version-cliff notice is still computed, because it only needs report metadata.
Fuzz guard. t/fuzz.t stays behind its extended-test guard. Under cover -test, every nested prove and every property trial also runs under Devel::Cover, which turns minutes into hours.
Changed files. test-generator-mutate --changed_only --base_sha SHA mutates only what a pull request touched.

The reproduction scripts

For every CPAN Testers FAIL report it can resolve, the dashboard writes cover_html/reproduce/reproduce-GUID.sh and links it from the failure table. The script pins every module installed in the failing environment at its exact version with cpanm --notest Module@version, then opens the distribution at the failing version with cpanm --look. Reproducing a smoker failure usually means guessing at a dependency version; this removes the guess.

The script is a shell file built from third-party report data, so the fields it interpolates are stripped of control characters before they are written — a newline in a reporter name would otherwise end a comment line and make the rest executable.

Using it as a page

The output is plain HTML, CSS and a little inline JavaScript: sortable tables, drag-to-reorder columns, and a light/dark toggle that remembers your choice in localStorage. It works from file:// as well as over HTTP, so a local run can be opened with a double click.

A dashboard is a prompt, not a score

The useful thing on the page is the shortest column: the files with surviving mutants, the lines no path reaches, the failure that nobody reproduced. Coverage percentages drift upward on their own; the list of survivors does not.