Keep the coverage pipeline honest: a frontend unit suite, and guards against silently-empty coverage #72

New Issue

2026-08-20T10:15:56-05:00

bermudalamb commented

2026-08-20 10:15:56 -05:00

Follow-on from #61, which made SonarQube's coverage number real — 0.0% to 69.6%. This issue is about keeping it real, and about the one weakness #61 knowingly shipped.

The contract the pipeline must now keep is written up in docs/ci/coverage-pipeline-contract.md, including what specifically breaks if each requirement lapses. This issue tracks the parts that are not done.

1. Frontend coverage measures execution, not testing

The frontend has no unit tests. Its 71% comes entirely from Playwright driving an istanbul-instrumented dev server, and istanbul marks a line covered when the browser executes it. A component rendered during an end-to-end test therefore reports as covered with nothing asserting anything about it.

Backend coverage does not work this way — it comes from tests that assert on responses, so a covered line is usually a checked line. Averaging the two into one project number hides the difference.

The practical consequence is that the 80% gate on new code is easier to clear on frontend changes than backend ones, which is precisely backwards from where the risk sits: #60 found 30 unhandled promise rejections and a permanent-spinner bug in the frontend, none of which a coverage number would have flagged.

This was accepted deliberately in #61 rather than overlooked — the alternative, instrumenting nothing on the frontend, left every frontend pull request failing a gate it could never satisfy. But the honest fix is a frontend unit suite (Vitest plus Testing Library), starting with the components that carry real logic: CustomerAuthContext, Cart, ItemCard, and the filters helpers.

2. Nothing yet stops coverage from silently going to zero

frontend/scripts/coverage-report.js fails when .nyc_output holds no samples, which catches the common case of an uninstrumented dev server. It does not catch a partial collapse — one workspace's report going missing, or a sonar.javascript.lcov.reportPaths entry being dropped, would just move the number down.

That matters because a coverage drop is ambiguous by nature: it looks identical whether tests were deleted or collection broke. Worth considering a floor — fail the scan if total coverage falls more than a few points below the last analysis — so the ambiguous case becomes a loud one.

3. The integration suite still hangs

It runs with --forceExit and a job timeout, which is a workaround, not a fix. Jest reports "did not exit one second after the test run has completed", so something keeps the event loop alive: most likely a pool, an interval, or a cron handle that is never closed. --detectOpenHandles would name it.

Until that is root-caused the suite stays workflow_dispatch-only in tests.yml, which means the only place it runs automatically is now the SonarQube job.

Why one issue rather than three

All three are the same underlying problem: the pipeline can be wrong in ways that look right. A flattering coverage number, a silently smaller one, and a suite that only appears to finish are all instances of a tool succeeding while telling you nothing. This project has met that shape three times already — #67, #60, and coverage itself.

Split them out if any becomes real work on its own; the frontend unit suite in particular is a project rather than a task.

Severity

Medium. Nothing is broken. The risk is a metric that looks trustworthy being used to make decisions it cannot support.

Follow-on from #61, which made SonarQube's coverage number real — 0.0% to 69.6%. This issue is about keeping it real, and about the one weakness #61 knowingly shipped. The contract the pipeline must now keep is written up in `docs/ci/coverage-pipeline-contract.md`, including what specifically breaks if each requirement lapses. This issue tracks the parts that are not done. ## 1. Frontend coverage measures execution, not testing The frontend has **no unit tests**. Its 71% comes entirely from Playwright driving an istanbul-instrumented dev server, and istanbul marks a line covered when the browser executes it. A component rendered during an end-to-end test therefore reports as covered with nothing asserting anything about it. Backend coverage does not work this way — it comes from tests that assert on responses, so a covered line is usually a checked line. Averaging the two into one project number hides the difference. The practical consequence is that the **80% gate on new code is easier to clear on frontend changes than backend ones**, which is precisely backwards from where the risk sits: #60 found 30 unhandled promise rejections and a permanent-spinner bug in the frontend, none of which a coverage number would have flagged. This was accepted deliberately in #61 rather than overlooked — the alternative, instrumenting nothing on the frontend, left every frontend pull request failing a gate it could never satisfy. But the honest fix is a frontend unit suite (Vitest plus Testing Library), starting with the components that carry real logic: `CustomerAuthContext`, `Cart`, `ItemCard`, and the `filters` helpers. ## 2. Nothing yet stops coverage from silently going to zero `frontend/scripts/coverage-report.js` fails when `.nyc_output` holds no samples, which catches the common case of an uninstrumented dev server. It does not catch a partial collapse — one workspace's report going missing, or a `sonar.javascript.lcov.reportPaths` entry being dropped, would just move the number down. That matters because a coverage drop is ambiguous by nature: it looks identical whether tests were deleted or collection broke. Worth considering a floor — fail the scan if total coverage falls more than a few points below the last analysis — so the ambiguous case becomes a loud one. ## 3. The integration suite still hangs It runs with `--forceExit` and a job timeout, which is a workaround, not a fix. Jest reports "did not exit one second after the test run has completed", so something keeps the event loop alive: most likely a pool, an interval, or a cron handle that is never closed. `--detectOpenHandles` would name it. Until that is root-caused the suite stays `workflow_dispatch`-only in `tests.yml`, which means the only place it runs automatically is now the SonarQube job. ## Why one issue rather than three All three are the same underlying problem: the pipeline can be wrong in ways that look right. A flattering coverage number, a silently smaller one, and a suite that only appears to finish are all instances of a tool succeeding while telling you nothing. This project has met that shape three times already — #67, #60, and coverage itself. Split them out if any becomes real work on its own; the frontend unit suite in particular is a project rather than a task. ## Severity Medium. Nothing is broken. The risk is a metric that looks trustworthy being used to make decisions it cannot support.

bermudalamb referenced this issue

2026-08-20 10:16:36 -05:00

SonarQube analyses the code but imports no test coverage, so its quality gate is measuring nothing #61

bermudalamb added this to the Code Quality and Hardening project 2026-08-20 11:01:13 -05:00

bermudalamb moved this to Review in Code Quality and Hardening on 2026-08-20 11:01:17 -05:00

bermudalamb referenced this issue

2026-08-20 11:25:14 -05:00

new_coverage is 71.2% against an 80% gate — and "new code" is silently the entire codebase #79

bermudalamb referenced this issue

2026-08-20 13:52:31 -05:00

Clear the 85 minutes of technical debt — four of the fourteen "smells" are real React defects #81

bermudalamb referenced this issue