A failing end-to-end run reported itself like this:
Run node scripts/summarize-playwright.js frontend/playwright-results.json
❌ Failure - Main Summarize end-to-end tests
exitcode '1': failure
which reads as a broken summary script. It was not — it was the summariser correctly reporting that tests had failed, with nothing said about it.
Two things combined to produce that.
The summariser doubled as the gate. It ended with `process.exit(stats.unexpected > 0 ? 1 : 0)`, so it failed the job itself. The workflow already has a step written for exactly that — `Fail if either suite failed`, whose comment explains it exists so `continue-on-error` on the suites cannot turn a failing suite into a passing job. That step was being skipped while its condition was true, because a step whose `if:` omits `always()` still implicitly requires its predecessors to have succeeded, and the summariser had already failed the job one step earlier. The gate written to be the place the job fails was dead code.
And the explanation went where the log is not. `report()` writes to GITEA_STEP_SUMMARY when it is set, which under Actions is always, so the counts and the list of failing tests landed in the Summary tab while the log showed a bare exit with no output at all.
So both scripts now exit 0 whatever they find, both print one line of counts to stdout as well as the markdown to the summary, and the gate carries `always() &&` so it actually runs. A failing suite now fails at a step named for what failed, and the log says how many.
Verified by executing both scripts against crafted results rather than by reading them: failing counts, passing counts, and a missing results file, each with and without GITEA_STEP_SUMMARY set. All four exit 0, the headline appears on stdout in every case, and the step summary still receives the full table and the failure list.
Not covered: nothing in this repository runs the scripts under `scripts/`. Jest's testMatch is scoped to backend/tests/unit, so a permanent regression test would need a runner these files do not have. Worth its own issue rather than widening the backend suite's roots to reach the repository root.
Closes #142
192 lines
7.5 KiB
YAML
Executable File
192 lines
7.5 KiB
YAML
Executable File
name: SonarQube Analysis
|
|
|
|
# The only workflow that runs the test suites. tests.yml used to run the unit
|
|
# and end-to-end suites as well, on identical triggers, so every pull request
|
|
# installed, migrated, built, started the backend and ran the whole end-to-end
|
|
# suite twice. With one runner the second copy did not run in parallel, it
|
|
# queued. It was deleted and its two summarize steps folded in here. See #123.
|
|
#
|
|
# Note that the integration suite DOES run here, on every push and pull request,
|
|
# despite backend-integration.yml describing it as manual. That quarantine only
|
|
# ever applied to tests.yml. It is survivable here because test:integration:cov
|
|
# passes --forceExit, which papers over the post-run hang, and because of the
|
|
# timeout below.
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
pull_request:
|
|
types: [opened, synchronize, reopened]
|
|
workflow_dispatch:
|
|
|
|
jobs:
|
|
sonarqube:
|
|
runs-on: ubuntu-latest
|
|
# The scanner needs every coverage report in one workspace, so the suites run
|
|
# here rather than being passed between jobs as artifacts. That makes this the
|
|
# long job. The timeout is a stop, not a budget: the integration suite has
|
|
# hung after completing before (see backend-integration.yml) and cost three
|
|
# hours of runner time.
|
|
timeout-minutes: 30
|
|
services:
|
|
postgres:
|
|
image: postgres:16
|
|
env:
|
|
POSTGRES_USER: redefined_test
|
|
POSTGRES_PASSWORD: redefined_test
|
|
POSTGRES_DB: redefined_test
|
|
options: >-
|
|
--health-cmd "pg_isready -U redefined_test"
|
|
--health-interval 5s
|
|
--health-timeout 5s
|
|
--health-retries 10
|
|
env:
|
|
# Consumed by the integration suite's setup files.
|
|
TEST_PGHOST: postgres
|
|
TEST_PGPORT: 5432
|
|
TEST_PGUSER: redefined_test
|
|
TEST_PGPASSWORD: redefined_test
|
|
TEST_PGDATABASE: redefined_test
|
|
# Consumed by the backend process the end-to-end run drives.
|
|
PGHOST: postgres
|
|
PGPORT: 5432
|
|
PGUSER: redefined_test
|
|
PGPASSWORD: redefined_test
|
|
PGDATABASE: redefined_test
|
|
PORT: 3000
|
|
DEMO_MODE: 'true'
|
|
UPLOADS_DIR: /tmp/redefined-uploads
|
|
# NODE_ENV is deliberately unset: `npm install` omits devDependencies when
|
|
# NODE_ENV=production, which strips tsc/vite/@playwright/test and breaks the
|
|
# build. It would also flip the session cookie to Secure, which the e2e run
|
|
# serves over plain http.
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
|
|
- name: Setup Node
|
|
uses: actions/setup-node@v4
|
|
with:
|
|
node-version: '20'
|
|
|
|
- name: Install backend deps
|
|
run: npm install
|
|
working-directory: backend
|
|
|
|
- name: Install frontend deps
|
|
run: npm install
|
|
working-directory: frontend
|
|
|
|
- name: TypeScript build check (backend)
|
|
run: npm run build
|
|
working-directory: backend
|
|
|
|
- name: TypeScript build check (frontend)
|
|
run: npm run build
|
|
working-directory: frontend
|
|
|
|
- name: Check the Sonar tsconfig has not drifted
|
|
run: node scripts/check-sonar-tsconfig.js
|
|
|
|
- name: Run migrations
|
|
run: node migrate.js up
|
|
working-directory: backend
|
|
|
|
# --json/--outputFile appended rather than baked into the script: the
|
|
# coverage run and the results file are wanted together here, and nowhere
|
|
# else. summarize-jest.js below reads that file.
|
|
- name: Backend unit tests with coverage
|
|
id: unit
|
|
continue-on-error: true
|
|
run: npm run test:unit:cov -- --json --outputFile=unit-results.json
|
|
working-directory: backend
|
|
|
|
# Covers everything in src/routes, which the unit suite does not touch —
|
|
# without this the backend reports around 11% rather than the ~73% it
|
|
# actually has.
|
|
- name: Backend integration tests with coverage
|
|
run: npm run test:integration:cov
|
|
working-directory: backend
|
|
|
|
- name: Start backend for the end-to-end run
|
|
run: |
|
|
mkdir -p /tmp/redefined-uploads
|
|
node dist/server.js > /tmp/backend.log 2>&1 &
|
|
for i in $(seq 1 30); do
|
|
if node -e "require('http').get('http://localhost:3000/api/config', r => process.exit(r.statusCode === 200 ? 0 : 1)).on('error', () => process.exit(1))"; then
|
|
echo "Backend ready after ${i}s"
|
|
exit 0
|
|
fi
|
|
sleep 1
|
|
done
|
|
echo "Backend did not become ready within 30s:"
|
|
cat /tmp/backend.log
|
|
exit 1
|
|
working-directory: backend
|
|
|
|
- name: Install Playwright browsers
|
|
run: npx playwright install --with-deps chromium
|
|
working-directory: frontend
|
|
|
|
# Runs against an istanbul-instrumented dev server, which is what produces
|
|
# window.__coverage__ for the fixture to collect.
|
|
- name: Frontend end-to-end tests with coverage
|
|
id: e2e
|
|
continue-on-error: true
|
|
env:
|
|
PLAYWRIGHT_JSON_OUTPUT_NAME: playwright-results.json
|
|
# list as well as json, so the log still shows which test failed rather
|
|
# than only a file nobody reads until the summary step.
|
|
run: npm run test:e2e:cov -- --reporter=list,json
|
|
working-directory: frontend
|
|
|
|
# Keyed off the step outcome rather than failure(). continue-on-error above
|
|
# means the job is not in a failed state at this point, so failure() would
|
|
# never fire and the log that explains an end-to-end failure would go
|
|
# unprinted precisely when it is wanted.
|
|
- name: Backend log
|
|
if: steps.e2e.outcome == 'failure'
|
|
run: cat /tmp/backend.log
|
|
|
|
# Fails when nothing was collected rather than writing an empty report. An
|
|
# uninstrumented dev server lets every test pass while gathering nothing,
|
|
# and the resulting 0% reads as "the tests stopped covering things".
|
|
- name: Merge frontend coverage
|
|
run: npm run coverage:report
|
|
working-directory: frontend
|
|
|
|
- name: SonarQube Scan
|
|
uses: sonarsource/sonarqube-scan-action@v4
|
|
env:
|
|
SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }}
|
|
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
|
|
|
|
# Readable pass and fail counts, which the raw jest and Playwright output
|
|
# does not give at a glance. Both run with if: always() so a failing suite
|
|
# is still summarised — which is the case they exist for.
|
|
- name: Summarize unit tests
|
|
if: always()
|
|
run: node scripts/summarize-jest.js backend/unit-results.json "Backend Unit Test"
|
|
|
|
- name: Summarize end-to-end tests
|
|
if: always()
|
|
run: node scripts/summarize-playwright.js frontend/playwright-results.json
|
|
|
|
# Last, so a failing suite still produces coverage, a scan and both
|
|
# summaries first. Without this step continue-on-error above would turn a
|
|
# failing suite into a passing job, which is the one way this change could
|
|
# do real damage.
|
|
#
|
|
# always() is load-bearing. A step whose `if:` omits it still implicitly
|
|
# requires every previous step to have succeeded, so this was skipped in
|
|
# exactly the case it exists for: the summarise step above used to exit 1
|
|
# on a failing suite, which failed the job first and left this gate as dead
|
|
# code. The job then reported its failure under a name that describes
|
|
# summarising rather than testing. The summarisers exit 0 now; this is what
|
|
# fails the run. See #142.
|
|
- name: Fail if either suite failed
|
|
if: always() && (steps.unit.outcome == 'failure' || steps.e2e.outcome == 'failure')
|
|
run: exit 1
|