ci: let the summarisers summarise and the gate do the failing (#142)

A failing end-to-end run reported itself like this:

    Run node scripts/summarize-playwright.js frontend/playwright-results.json
      ❌  Failure - Main Summarize end-to-end tests
    exitcode '1': failure

which reads as a broken summary script. It was not — it was the summariser correctly reporting that tests had failed, with nothing said about it.

Two things combined to produce that.

The summariser doubled as the gate. It ended with `process.exit(stats.unexpected > 0 ? 1 : 0)`, so it failed the job itself. The workflow already has a step written for exactly that — `Fail if either suite failed`, whose comment explains it exists so `continue-on-error` on the suites cannot turn a failing suite into a passing job. That step was being skipped while its condition was true, because a step whose `if:` omits `always()` still implicitly requires its predecessors to have succeeded, and the summariser had already failed the job one step earlier. The gate written to be the place the job fails was dead code.

And the explanation went where the log is not. `report()` writes to GITEA_STEP_SUMMARY when it is set, which under Actions is always, so the counts and the list of failing tests landed in the Summary tab while the log showed a bare exit with no output at all.

So both scripts now exit 0 whatever they find, both print one line of counts to stdout as well as the markdown to the summary, and the gate carries `always() &&` so it actually runs. A failing suite now fails at a step named for what failed, and the log says how many.

Verified by executing both scripts against crafted results rather than by reading them: failing counts, passing counts, and a missing results file, each with and without GITEA_STEP_SUMMARY set. All four exit 0, the headline appears on stdout in every case, and the step summary still receives the full table and the failure list.

Not covered: nothing in this repository runs the scripts under `scripts/`. Jest's testMatch is scoped to backend/tests/unit, so a permanent regression test would need a runner these files do not have. Worth its own issue rather than widening the backend suite's roots to reach the repository root.

Closes #142

scripts/summarize-jest.js

@@ -8,6 +8,7 @@ const [, , resultsPath, label] = process.argv;
 // fails on the real step, and a stack trace here would only bury it.
 if (!fs.existsSync(resultsPath)) {
   report([`## ${label || 'Test'} Results`, '', `No results file at \`${resultsPath}\` — the test step failed before writing one.`, '']);
+  console.log(`${label || 'Test'}: no results file at ${resultsPath} — the test step failed before writing one`);
   process.exit(0);
 }
 
@@ -44,7 +45,20 @@ if (data.numFailedTests > 0) {
 
 report(lines);
 
-process.exit(data.numFailedTests > 0 ? 1 : 0);
+headline();
+
+// Exits 0 whatever it found, for the same reason as summarize-playwright.js:
+// summarising and gating are two jobs, and the workflow's `Fail if either suite
+// failed` step is the one that should fail a run.
+process.exit(0);
+
+/** One line of counts on stdout, so the log says what happened, not only the Summary tab. */
+function headline() {
+  const counts =
+    `${label || 'Test'}: ${data.numPassedTests} passed, ${data.numFailedTests} failed, ` +
+    `${data.numPendingTests} skipped`;
+  console.log(data.numFailedTests > 0 ? `${counts} — see the job summary for which` : counts);
+}
 
 function report(out) {
   const summaryPath = process.env.GITEA_STEP_SUMMARY || process.env.GITHUB_STEP_SUMMARY;

scripts/summarize-playwright.js

@@ -8,6 +8,7 @@ const [, , resultsPath] = process.argv;
 // fails on the real step, and a stack trace here would only bury it.
 if (!fs.existsSync(resultsPath)) {
   report([`## Playwright E2E Results`, '', `No results file at \`${resultsPath}\` — the test step failed before writing one.`, '']);
+  console.log(`Playwright: no results file at ${resultsPath} — the test step failed before writing one`);
   process.exit(0);
 }
 
@@ -55,7 +56,32 @@ if (failures.length) {
 
 report(lines);
 
-process.exit((stats.unexpected || 0) > 0 ? 1 : 0);
+headline();
+
+// Exits 0 whatever it found. Summarising and gating are two jobs, and this
+// script doing both meant the workflow failed at a step named "Summarize
+// end-to-end tests", which reads as a broken summary script rather than as
+// failing tests. The workflow's own `Fail if either suite failed` step fails the
+// job now. It was written for exactly that and was being skipped, because a step
+// whose `if:` omits always() still requires its predecessors to have succeeded,
+// and this script had already failed the job before it could run.
+process.exit(0);
+
+/**
+ * One line of counts on stdout, alongside the markdown.
+ *
+ * `report` writes to GITEA_STEP_SUMMARY when it is set, which under Actions is
+ * always — so the table and the failure list landed in the Summary tab while the
+ * log showed a bare exit with no output at all. This puts the numbers in the log
+ * too, at the point the run stops.
+ */
+function headline() {
+  const failed = stats.unexpected || 0;
+  const counts =
+    `Playwright: ${stats.expected || 0} passed, ${failed} failed, ` +
+    `${stats.flaky || 0} flaky, ${stats.skipped || 0} skipped`;
+  console.log(failed > 0 ? `${counts} — see the job summary for which` : counts);
+}
 
 function report(out) {
   const summaryPath = process.env.GITEA_STEP_SUMMARY || process.env.GITHUB_STEP_SUMMARY;