tests.yml duplicates the suites sonarqube.yml already runs: fold its reporting in and delete it #123

New Issue

2026-08-22T10:57:29-05:00

bermudalamb commented

2026-08-22 10:57:29 -05:00

tests.yml and sonarqube.yml have identical triggers, both firing on every push to main and every pull request. They then run the same tests.

sonarqube.yml executes a strict superset. Where tests.yml runs test:unit:json and playwright test --reporter=json, sonarqube.yml runs test:unit:cov and test:e2e:cov: the same two suites, differing only in reporter and instrumentation. On top of those it runs the integration suite, both TypeScript builds, the Sonar tsconfig drift check, the coverage merge and the scan itself.

The cost is not just tidiness. Every pull request installs dependencies twice, migrates twice, builds and starts the backend twice, installs a Playwright browser twice and runs the full end-to-end suite twice. On a single self-hosted runner the second copy does not even run in parallel, it queues.

What tests.yml has that sonarqube.yml does not

Two things, and both are worth keeping.

The summaries. scripts/summarize-jest.js and scripts/summarize-playwright.js render pass and fail counts into the job summary. sonarqube.yml produces none, so a failure there means reading raw jest and Playwright output.

The continue-on-error: true and explicit fail-at-the-end pattern, which is what allows those summaries to render on a failing run at all. sonarqube.yml aborts on the first failing step, so a broken unit test there means no end-to-end results, no coverage and no scan: one fact per run, when the run costs many minutes.

The change

Fold both into sonarqube.yml. Give the unit and end-to-end steps an id and continue-on-error: true, add the two summarize steps with if: always(), and end the job with a step that fails when either suite failed. Then delete tests.yml.

The trade-off, stated rather than buried

This gives up fast feedback. Today a broken unit test fails tests.yml in a couple of minutes while the Sonar run grinds on; afterwards, the same break is only reported once the whole pipeline has finished.

That is judged worth it here because there is one runner. The fast job is not actually fast when it is queued behind the slow one, and a single run that reports unit results, end-to-end results, coverage and the scan together beats two runs that each report a fragment. If the runner count changes, this is worth revisiting.

Related: the integration suite is not as quarantined as the comments suggest

backend-integration.yml says the suite was split out and made manual after it held the runner for 3h12m on 2026-08-18, and the header comment in tests.yml says the same. Both are true of tests.yml. Neither is true of sonarqube.yml, which runs test:integration:cov on every push and pull request.

It survives because that script passes --forceExit, which papers over the post-run hang, and because the job carries timeout-minutes: 30. But the comments give the impression that the integration suite never runs automatically, and it has been running automatically the whole time. Whatever wording survives this change should say so, rather than repeating a quarantine that only ever applied to one of the two files.

Verification

Parse the remaining workflows and check the job inventory, the same way the lint split in #115 was checked. sonarqube.yml should declare exactly one job. tests.yml should be gone. No job that ran before should have stopped running: the unit and end-to-end suites both still execute, now once instead of twice.

Then confirm on a real run that a deliberately failing unit test still fails the job, and that both summaries appear. The continue-on-error pattern is exactly the kind of change that can silently turn a failing suite into a passing job, so it must be seen to fail before it is trusted.

`tests.yml` and `sonarqube.yml` have identical triggers, both firing on every push to main and every pull request. They then run the same tests. `sonarqube.yml` executes a strict superset. Where `tests.yml` runs `test:unit:json` and `playwright test --reporter=json`, `sonarqube.yml` runs `test:unit:cov` and `test:e2e:cov`: the same two suites, differing only in reporter and instrumentation. On top of those it runs the integration suite, both TypeScript builds, the Sonar tsconfig drift check, the coverage merge and the scan itself. The cost is not just tidiness. Every pull request installs dependencies twice, migrates twice, builds and starts the backend twice, installs a Playwright browser twice and runs the full end-to-end suite twice. On a single self-hosted runner the second copy does not even run in parallel, it queues. ## What tests.yml has that sonarqube.yml does not Two things, and both are worth keeping. The summaries. `scripts/summarize-jest.js` and `scripts/summarize-playwright.js` render pass and fail counts into the job summary. `sonarqube.yml` produces none, so a failure there means reading raw jest and Playwright output. The `continue-on-error: true` and explicit fail-at-the-end pattern, which is what allows those summaries to render on a failing run at all. `sonarqube.yml` aborts on the first failing step, so a broken unit test there means no end-to-end results, no coverage and no scan: one fact per run, when the run costs many minutes. ## The change Fold both into `sonarqube.yml`. Give the unit and end-to-end steps an `id` and `continue-on-error: true`, add the two summarize steps with `if: always()`, and end the job with a step that fails when either suite failed. Then delete `tests.yml`. ## The trade-off, stated rather than buried This gives up fast feedback. Today a broken unit test fails `tests.yml` in a couple of minutes while the Sonar run grinds on; afterwards, the same break is only reported once the whole pipeline has finished. That is judged worth it here because there is one runner. The fast job is not actually fast when it is queued behind the slow one, and a single run that reports unit results, end-to-end results, coverage and the scan together beats two runs that each report a fragment. If the runner count changes, this is worth revisiting. ## Related: the integration suite is not as quarantined as the comments suggest `backend-integration.yml` says the suite was split out and made manual after it held the runner for 3h12m on 2026-08-18, and the header comment in `tests.yml` says the same. Both are true of `tests.yml`. Neither is true of `sonarqube.yml`, which runs `test:integration:cov` on every push and pull request. It survives because that script passes `--forceExit`, which papers over the post-run hang, and because the job carries `timeout-minutes: 30`. But the comments give the impression that the integration suite never runs automatically, and it has been running automatically the whole time. Whatever wording survives this change should say so, rather than repeating a quarantine that only ever applied to one of the two files. ## Verification Parse the remaining workflows and check the job inventory, the same way the lint split in #115 was checked. `sonarqube.yml` should declare exactly one job. `tests.yml` should be gone. No job that ran before should have stopped running: the unit and end-to-end suites both still execute, now once instead of twice. Then confirm on a real run that a deliberately failing unit test still fails the job, and that both summaries appear. The `continue-on-error` pattern is exactly the kind of change that can silently turn a failing suite into a passing job, so it must be seen to fail before it is trusted.

bermudalamb referenced this issue from a commit

2026-08-22 11:09:19 -05:00

ci: fold the Tests workflow into SonarQube Analysis and delete it (#123)

bermudalamb referenced a pull request that will close this issue

2026-08-22 11:10:42 -05:00

ci: fold the Tests workflow into SonarQube Analysis and delete it (#123) #127

bermudalamb self-assigned this 2026-08-22 11:11:02 -05:00

bermudalamb added this to the Code Quality and Hardening 2 project 2026-08-22 11:11:14 -05:00

bermudalamb moved this to Review in Code Quality and Hardening 2 on 2026-08-22 11:11:19 -05:00

bermudalamb added reference feature/123-consolidate-test-workflows

2026-08-22 11:11:25 -05:00

bermudalamb closed this issue