bermudalambandClaude Opus 5
0a300f3c4b
main went red on 86095cb with "Playwright: 157 passed, 3 failed" while every other step in the job passed. The three are the background-removal cases from #281, and they fail for the most boring reason available: REMBG_URL appears nowhere in .gitea/workflows, so isRembgConfigured() is false, both endpoints report backgroundRemoval as false, and neither the submitter's checkbox nor the review queue's per-photo button is rendered for the tests to find. The application is right and the tests are right; the environment they run in was missing one variable.
Deliberately a URL that does not resolve. Nothing in the suite reaches the sidecar — the worker only cuts a background out after a draft is written, and drafting needs an ANTHROPIC_API_KEY this job does not have. Standing up a real rembg in CI would cost a 4.24 GB image and about forty seconds of startup to prove that a control is on screen.
How it got in is worth recording, because the missing line is the symptom. #281 added the variable to scripts/start-local.ps1 and stopped there, and its three end-to-end tests were merged unrun — verified locally against a stack that did have the variable, then handed over as outstanding rather than run before the merge. A test that has never been run is not evidence of anything, and two of the three would have failed on the first honest execution.
The deeper gap is that a variable which changes rendered behaviour now lives in two places nothing compares. composeEnvironment.test.ts exists for exactly this failure and caught it twice before, in #107 and #118, but it guards the deploying compose files rather than the CI workflow. Whether it should reach into .gitea/workflows is a decision rather than another hand-added line, and #287 leaves it open.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
309 lines
14 KiB
YAML
Executable File
309 lines
14 KiB
YAML
Executable File
name: SonarQube Analysis
|
|
|
|
# The only workflow that runs the test suites. tests.yml used to run the unit
|
|
# and end-to-end suites as well, on identical triggers, so every pull request
|
|
# installed, migrated, built, started the backend and ran the whole end-to-end
|
|
# suite twice. With one runner the second copy did not run in parallel, it
|
|
# queued. It was deleted and its two summarize steps folded in here. See #123.
|
|
#
|
|
# Note that the integration suite DOES run here, on every push and pull request,
|
|
# despite backend-integration.yml describing it as manual. That quarantine only
|
|
# ever applied to tests.yml. It is survivable here because test:integration:cov
|
|
# passes --forceExit, which papers over the post-run hang, and because of the
|
|
# timeout below.
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
pull_request:
|
|
types: [opened, synchronize, reopened]
|
|
workflow_dispatch:
|
|
|
|
jobs:
|
|
sonarqube:
|
|
runs-on: ubuntu-latest
|
|
# The scanner needs every coverage report in one workspace, so the suites run
|
|
# here rather than being passed between jobs as artifacts. That makes this the
|
|
# long job. The timeout is a stop, not a budget: the integration suite has
|
|
# hung after completing before (see backend-integration.yml) and cost three
|
|
# hours of runner time.
|
|
timeout-minutes: 30
|
|
services:
|
|
postgres:
|
|
image: postgres:16
|
|
env:
|
|
POSTGRES_USER: redefined_test
|
|
POSTGRES_PASSWORD: redefined_test
|
|
POSTGRES_DB: redefined_test
|
|
options: >-
|
|
--health-cmd "pg_isready -U redefined_test"
|
|
--health-interval 5s
|
|
--health-timeout 5s
|
|
--health-retries 10
|
|
env:
|
|
# Consumed by the integration suite's setup files.
|
|
TEST_PGHOST: postgres
|
|
TEST_PGPORT: 5432
|
|
TEST_PGUSER: redefined_test
|
|
TEST_PGPASSWORD: redefined_test
|
|
TEST_PGDATABASE: redefined_test
|
|
# Consumed by the backend process the end-to-end run drives.
|
|
PGHOST: postgres
|
|
PGPORT: 5432
|
|
PGUSER: redefined_test
|
|
PGPASSWORD: redefined_test
|
|
PGDATABASE: redefined_test
|
|
PORT: 3000
|
|
DEMO_MODE: 'true'
|
|
UPLOADS_DIR: /tmp/redefined-uploads
|
|
# Set so the background-removal controls render at all: both the
|
|
# submission page's checkbox and the review queue's per-photo button are
|
|
# hidden unless the server reports the feature as configured, and three
|
|
# end-to-end tests assert they are there (#287).
|
|
#
|
|
# Deliberately a URL that does not resolve. Nothing in the suite reaches
|
|
# the sidecar — the worker only cuts a background out after a draft is
|
|
# written, and drafting needs an ANTHROPIC_API_KEY this job does not have.
|
|
# A real rembg here would mean a 4.24 GB image and forty seconds of
|
|
# startup to prove a control is on screen.
|
|
REMBG_URL: http://127.0.0.1:7000
|
|
# NODE_ENV is deliberately unset: `npm install` omits devDependencies when
|
|
# NODE_ENV=production, which strips tsc/vite/@playwright/test and breaks the
|
|
# build. It would also flip the session cookie to Secure, which the e2e run
|
|
# serves over plain http.
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@v4
|
|
with:
|
|
fetch-depth: 0
|
|
|
|
- name: Setup Node
|
|
uses: actions/setup-node@v4
|
|
with:
|
|
node-version: '20'
|
|
|
|
- name: Install backend deps
|
|
run: npm install
|
|
working-directory: backend
|
|
|
|
- name: Install frontend deps
|
|
run: npm install
|
|
working-directory: frontend
|
|
|
|
- name: TypeScript build check (backend)
|
|
run: npm run build
|
|
working-directory: backend
|
|
|
|
- name: TypeScript build check (frontend)
|
|
run: npm run build
|
|
working-directory: frontend
|
|
|
|
# The frontend's build was the only thing this workspace ran, so the unit
|
|
# suite #188 added over the filter dimensions was run by nothing but the
|
|
# author's terminal. A suite CI never runs decays into a record of what
|
|
# the code used to do, and its value is highest exactly here: chips() is
|
|
# pure, and the end-to-end run reaches it only through a browser.
|
|
#
|
|
# Guarded and named in the gate like every suite: a failing test should
|
|
# fail the job at the end, not abort it and take the scan with it.
|
|
- name: Frontend unit tests
|
|
id: frontend_unit
|
|
continue-on-error: true
|
|
run: npm run test:unit
|
|
working-directory: frontend
|
|
|
|
- name: Check the Sonar tsconfig has not drifted
|
|
run: node scripts/check-sonar-tsconfig.js
|
|
|
|
- name: Run migrations
|
|
run: node migrate.js up
|
|
working-directory: backend
|
|
|
|
# --json/--outputFile appended rather than baked into the script: the
|
|
# coverage run and the results file are wanted together here, and nowhere
|
|
# else. summarize-jest.js below reads that file.
|
|
- name: Backend unit tests with coverage
|
|
id: unit
|
|
continue-on-error: true
|
|
run: npm run test:unit:cov -- --json --outputFile=unit-results.json
|
|
working-directory: backend
|
|
|
|
# Covers everything in src/routes, which the unit suite does not touch —
|
|
# without this the backend reports around 11% rather than the ~73% it
|
|
# actually has.
|
|
#
|
|
# Guarded like the other two suites. It was the only one that was not, so
|
|
# it was the only one whose failure aborted the job — taking the scan, the
|
|
# end-to-end run and the coverage merge with it. That is what #154 has
|
|
# cost on every push since it started: not a degraded analysis, none at
|
|
# all. See #174.
|
|
- name: Backend integration tests with coverage
|
|
id: integration
|
|
continue-on-error: true
|
|
run: npm run test:integration:cov -- --json --outputFile=integration-results.json
|
|
working-directory: backend
|
|
|
|
# Guarded because the scan does not depend on it. A backend that will not
|
|
# start fails the end-to-end run below on its own, and the gate records
|
|
# both — there is no reason for it to also cost the analysis.
|
|
- name: Start backend for the end-to-end run
|
|
id: backend
|
|
continue-on-error: true
|
|
run: |
|
|
mkdir -p /tmp/redefined-uploads
|
|
node dist/server.js > /tmp/backend.log 2>&1 &
|
|
for i in $(seq 1 30); do
|
|
if node -e "require('http').get('http://localhost:3000/api/config', r => process.exit(r.statusCode === 200 ? 0 : 1)).on('error', () => process.exit(1))"; then
|
|
echo "Backend ready after ${i}s"
|
|
exit 0
|
|
fi
|
|
sleep 1
|
|
done
|
|
echo "Backend did not become ready within 30s:"
|
|
cat /tmp/backend.log
|
|
exit 1
|
|
working-directory: backend
|
|
|
|
# A network fetch, and so the least interesting way to lose an analysis.
|
|
- name: Install Playwright browsers
|
|
id: browsers
|
|
continue-on-error: true
|
|
run: npx playwright install --with-deps chromium
|
|
working-directory: frontend
|
|
|
|
# Runs against an istanbul-instrumented dev server, which is what produces
|
|
# window.__coverage__ for the fixture to collect.
|
|
- name: Frontend end-to-end tests with coverage
|
|
id: e2e
|
|
continue-on-error: true
|
|
env:
|
|
PLAYWRIGHT_JSON_OUTPUT_NAME: playwright-results.json
|
|
# list as well as json, so the log still shows which test failed rather
|
|
# than only a file nobody reads until the summary step.
|
|
run: npm run test:e2e:cov -- --reporter=list,json
|
|
working-directory: frontend
|
|
|
|
# Keyed off the step outcome rather than failure(). continue-on-error above
|
|
# means the job is not in a failed state at this point, so failure() would
|
|
# never fire and the log that explains an end-to-end failure would go
|
|
# unprinted precisely when it is wanted.
|
|
- name: Backend log
|
|
if: steps.e2e.outcome == 'failure' || steps.backend.outcome == 'failure'
|
|
run: cat /tmp/backend.log
|
|
|
|
# Fails when nothing was collected rather than writing an empty report. An
|
|
# uninstrumented dev server lets every test pass while gathering nothing,
|
|
# and the resulting 0% reads as "the tests stopped covering things".
|
|
- name: Merge frontend coverage
|
|
id: coverage
|
|
continue-on-error: true
|
|
run: npm run coverage:report
|
|
working-directory: frontend
|
|
|
|
# Guarded so that a scanner error does not take the summaries below with
|
|
# it. The gate still records it, so a failed scan fails the job.
|
|
#
|
|
# If a coverage report is missing — the end-to-end run collecting nothing,
|
|
# say — this scans anyway and SonarQube reports those files as uncovered,
|
|
# which reads as a regression rather than as a missing input. Accepted:
|
|
# jest still writes coverage for a suite whose tests fail, so the failure
|
|
# actually occurring produces all three reports; there is no
|
|
# sonar.qualitygate.wait, so a degraded run marks the dashboard and is
|
|
# overwritten by the next good one rather than blocking anything; and the
|
|
# job fails regardless, so no run in this state reads as clean.
|
|
# Not on pull requests. SonarQube Community has no branch analysis: every
|
|
# scan published under a project key replaces that project's single
|
|
# analysis, whatever revision it came from. So a pull request scan
|
|
# overwrote the dashboard's picture of main with the branch, silently, and
|
|
# the new-code period, gate result, coverage and hotspot list then all
|
|
# described whatever was scanned last with nothing saying which revision
|
|
# that was. #197 caught it in the act — the dashboard describing a feature
|
|
# branch while reporting hotspot line numbers that landed on a blank line
|
|
# in main.
|
|
#
|
|
# scripts/scan-local.sh already refuses to do this, defaulting to a
|
|
# scratch key for exactly this reason. CI walked into the hazard that
|
|
# script guards against; now the two tell the same story.
|
|
#
|
|
# The suites above still run on pull requests, which is where their value
|
|
# is. Only publishing is restricted.
|
|
- name: SonarQube Scan
|
|
id: scan
|
|
if: github.event_name != 'pull_request'
|
|
continue-on-error: true
|
|
uses: sonarsource/sonarqube-scan-action@v4
|
|
env:
|
|
SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }}
|
|
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
|
|
|
|
# Readable pass and fail counts, which the raw jest and Playwright output
|
|
# does not give at a glance. Both run with if: always() so a failing suite
|
|
# is still summarised — which is the case they exist for.
|
|
- name: Summarize unit tests
|
|
if: always()
|
|
run: node scripts/summarize-jest.js backend/unit-results.json "Backend Unit Test"
|
|
|
|
# The suite this workflow has been failing on for weeks, and the only one
|
|
# that had no summary — so it presented as 36 assertion errors about
|
|
# categories and price filters rather than as a count. #154 records how
|
|
# expensive that misdirection was to read.
|
|
- name: Summarize integration tests
|
|
if: always()
|
|
run: node scripts/summarize-jest.js backend/integration-results.json "Backend Integration Test"
|
|
|
|
- name: Summarize end-to-end tests
|
|
if: always()
|
|
run: node scripts/summarize-playwright.js frontend/playwright-results.json
|
|
|
|
# The measures, printed into the log because that is the only place this
|
|
# project can read them. SonarQube 9.9 Community has no Bearer auth, so the
|
|
# official MCP cannot connect, and the host is a CI secret — so security
|
|
# hotspots, duplication, debt and coverage lived solely on a dashboard, and
|
|
# "reduce the debt" was an instruction nobody could act on without opening a
|
|
# browser. The scanner masks the URL and token; the measures are not secret.
|
|
#
|
|
# Deliberately not guarded with continue-on-error. The script exits 0 on
|
|
# every path, so it cannot fail the job anyway, and guarding it would
|
|
# oblige it to appear in the gate below — which exists to fail the job,
|
|
# the opposite of what a report should do. See #261.
|
|
# Skipped alongside the scan on pull requests. With nothing published, this
|
|
# would report main's numbers under a pull request's log, which is noise
|
|
# at best and misread as the branch's own at worst.
|
|
- name: Report SonarQube measures
|
|
if: always() && github.event_name != 'pull_request'
|
|
env:
|
|
SONAR_HOST_URL: ${{ secrets.SONAR_HOST_URL }}
|
|
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
|
|
run: node scripts/summarize-sonar.js
|
|
|
|
# Last, so a failing step still produces coverage, a scan and every
|
|
# summary first. Without this step continue-on-error above would turn a
|
|
# failing suite into a passing job, which is the one way this change could
|
|
# do real damage.
|
|
#
|
|
# Every guarded step is listed. That is the invariant — a step carrying
|
|
# continue-on-error and missing from here cannot fail the job at all — and
|
|
# tests/unit/workflowGate.test.ts asserts it, because the integration
|
|
# suite going unlisted is exactly how #174 happened.
|
|
#
|
|
# always() is load-bearing. A step whose `if:` omits it still implicitly
|
|
# requires every previous step to have succeeded, so this was skipped in
|
|
# exactly the case it exists for: the summarise step above used to exit 1
|
|
# on a failing suite, which failed the job first and left this gate as dead
|
|
# code. The job then reported its failure under a name that describes
|
|
# summarising rather than testing. The summarisers exit 0 now; this is what
|
|
# fails the run. See #142.
|
|
- name: Fail if any guarded step failed
|
|
if: >-
|
|
always() && (
|
|
steps.frontend_unit.outcome == 'failure' ||
|
|
steps.unit.outcome == 'failure' ||
|
|
steps.integration.outcome == 'failure' ||
|
|
steps.backend.outcome == 'failure' ||
|
|
steps.browsers.outcome == 'failure' ||
|
|
steps.e2e.outcome == 'failure' ||
|
|
steps.coverage.outcome == 'failure' ||
|
|
steps.scan.outcome == 'failure'
|
|
)
|
|
run: exit 1
|