Files
redefined-designs/.gitea/workflows/sonarqube.yml
T
bermudalamb 491c2652f3
Linting / lint (pull_request) Successful in 2m10s
SonarQube Analysis / sonarqube (pull_request) Failing after 5m53s
ci: let the summarisers summarise and the gate do the failing (#142)
A failing end-to-end run reported itself like this:

    Run node scripts/summarize-playwright.js frontend/playwright-results.json
        Failure - Main Summarize end-to-end tests
    exitcode '1': failure

which reads as a broken summary script. It was not — it was the summariser correctly reporting that tests had failed, with nothing said about it.

Two things combined to produce that.

The summariser doubled as the gate. It ended with `process.exit(stats.unexpected > 0 ? 1 : 0)`, so it failed the job itself. The workflow already has a step written for exactly that — `Fail if either suite failed`, whose comment explains it exists so `continue-on-error` on the suites cannot turn a failing suite into a passing job. That step was being skipped while its condition was true, because a step whose `if:` omits `always()` still implicitly requires its predecessors to have succeeded, and the summariser had already failed the job one step earlier. The gate written to be the place the job fails was dead code.

And the explanation went where the log is not. `report()` writes to GITEA_STEP_SUMMARY when it is set, which under Actions is always, so the counts and the list of failing tests landed in the Summary tab while the log showed a bare exit with no output at all.

So both scripts now exit 0 whatever they find, both print one line of counts to stdout as well as the markdown to the summary, and the gate carries `always() &&` so it actually runs. A failing suite now fails at a step named for what failed, and the log says how many.

Verified by executing both scripts against crafted results rather than by reading them: failing counts, passing counts, and a missing results file, each with and without GITEA_STEP_SUMMARY set. All four exit 0, the headline appears on stdout in every case, and the step summary still receives the full table and the failure list.

Not covered: nothing in this repository runs the scripts under `scripts/`. Jest's testMatch is scoped to backend/tests/unit, so a permanent regression test would need a runner these files do not have. Worth its own issue rather than widening the backend suite's roots to reach the repository root.

Closes #142
2026-08-24 09:46:54 -05:00

192 lines
7.5 KiB
YAML
Executable File

name: SonarQube Analysis
# The only workflow that runs the test suites. tests.yml used to run the unit
# and end-to-end suites as well, on identical triggers, so every pull request
# installed, migrated, built, started the backend and ran the whole end-to-end
# suite twice. With one runner the second copy did not run in parallel, it
# queued. It was deleted and its two summarize steps folded in here. See #123.
#
# Note that the integration suite DOES run here, on every push and pull request,
# despite backend-integration.yml describing it as manual. That quarantine only
# ever applied to tests.yml. It is survivable here because test:integration:cov
# passes --forceExit, which papers over the post-run hang, and because of the
# timeout below.
on:
push:
branches: [main]
pull_request:
types: [opened, synchronize, reopened]
workflow_dispatch:
jobs:
sonarqube:
runs-on: ubuntu-latest
# The scanner needs every coverage report in one workspace, so the suites run
# here rather than being passed between jobs as artifacts. That makes this the
# long job. The timeout is a stop, not a budget: the integration suite has
# hung after completing before (see backend-integration.yml) and cost three
# hours of runner time.
timeout-minutes: 30
services:
postgres:
image: postgres:16
env:
POSTGRES_USER: redefined_test
POSTGRES_PASSWORD: redefined_test
POSTGRES_DB: redefined_test
options: >-
--health-cmd "pg_isready -U redefined_test"
--health-interval 5s
--health-timeout 5s
--health-retries 10
env:
# Consumed by the integration suite's setup files.
TEST_PGHOST: postgres
TEST_PGPORT: 5432
TEST_PGUSER: redefined_test
TEST_PGPASSWORD: redefined_test
TEST_PGDATABASE: redefined_test
# Consumed by the backend process the end-to-end run drives.
PGHOST: postgres
PGPORT: 5432
PGUSER: redefined_test
PGPASSWORD: redefined_test
PGDATABASE: redefined_test
PORT: 3000
DEMO_MODE: 'true'
UPLOADS_DIR: /tmp/redefined-uploads
# NODE_ENV is deliberately unset: `npm install` omits devDependencies when
# NODE_ENV=production, which strips tsc/vite/@playwright/test and breaks the
# build. It would also flip the session cookie to Secure, which the e2e run
# serves over plain http.
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Setup Node
uses: actions/setup-node@v4
with:
node-version: '20'
- name: Install backend deps
run: npm install
working-directory: backend
- name: Install frontend deps
run: npm install
working-directory: frontend
- name: TypeScript build check (backend)
run: npm run build
working-directory: backend
- name: TypeScript build check (frontend)
run: npm run build
working-directory: frontend
- name: Check the Sonar tsconfig has not drifted
run: node scripts/check-sonar-tsconfig.js
- name: Run migrations
run: node migrate.js up
working-directory: backend
# --json/--outputFile appended rather than baked into the script: the
# coverage run and the results file are wanted together here, and nowhere
# else. summarize-jest.js below reads that file.
- name: Backend unit tests with coverage
id: unit
continue-on-error: true
run: npm run test:unit:cov -- --json --outputFile=unit-results.json
working-directory: backend
# Covers everything in src/routes, which the unit suite does not touch —
# without this the backend reports around 11% rather than the ~73% it
# actually has.
- name: Backend integration tests with coverage
run: npm run test:integration:cov
working-directory: backend
- name: Start backend for the end-to-end run
run: |
mkdir -p /tmp/redefined-uploads
node dist/server.js > /tmp/backend.log 2>&1 &
for i in $(seq 1 30); do
if node -e "require('http').get('http://localhost:3000/api/config', r => process.exit(r.statusCode === 200 ? 0 : 1)).on('error', () => process.exit(1))"; then
echo "Backend ready after ${i}s"
exit 0
fi
sleep 1
done
echo "Backend did not become ready within 30s:"
cat /tmp/backend.log
exit 1
working-directory: backend
- name: Install Playwright browsers
run: npx playwright install --with-deps chromium
working-directory: frontend
# Runs against an istanbul-instrumented dev server, which is what produces
# window.__coverage__ for the fixture to collect.
- name: Frontend end-to-end tests with coverage
id: e2e
continue-on-error: true
env:
PLAYWRIGHT_JSON_OUTPUT_NAME: playwright-results.json
# list as well as json, so the log still shows which test failed rather
# than only a file nobody reads until the summary step.
run: npm run test:e2e:cov -- --reporter=list,json
working-directory: frontend
# Keyed off the step outcome rather than failure(). continue-on-error above
# means the job is not in a failed state at this point, so failure() would
# never fire and the log that explains an end-to-end failure would go
# unprinted precisely when it is wanted.
- name: Backend log
if: steps.e2e.outcome == 'failure'
run: cat /tmp/backend.log
# Fails when nothing was collected rather than writing an empty report. An
# uninstrumented dev server lets every test pass while gathering nothing,
# and the resulting 0% reads as "the tests stopped covering things".
- name: Merge frontend coverage
run: npm run coverage:report
working-directory: frontend
- name: SonarQube Scan
uses: sonarsource/sonarqube-scan-action@v4
env:
SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
# Readable pass and fail counts, which the raw jest and Playwright output
# does not give at a glance. Both run with if: always() so a failing suite
# is still summarised — which is the case they exist for.
- name: Summarize unit tests
if: always()
run: node scripts/summarize-jest.js backend/unit-results.json "Backend Unit Test"
- name: Summarize end-to-end tests
if: always()
run: node scripts/summarize-playwright.js frontend/playwright-results.json
# Last, so a failing suite still produces coverage, a scan and both
# summaries first. Without this step continue-on-error above would turn a
# failing suite into a passing job, which is the one way this change could
# do real damage.
#
# always() is load-bearing. A step whose `if:` omits it still implicitly
# requires every previous step to have succeeded, so this was skipped in
# exactly the case it exists for: the summarise step above used to exit 1
# on a failing suite, which failed the job first and left this gate as dead
# code. The job then reported its failure under a name that describes
# summarising rather than testing. The summarisers exit 0 now; this is what
# fails the run. See #142.
- name: Fail if either suite failed
if: always() && (steps.unit.outcome == 'failure' || steps.e2e.outcome == 'failure')
run: exit 1