test(e2e): make the resend allowance failure say why (#257) #319

Merged
bermudalamb merged 1 commits from fix/257-resend-verification-flake into main 2026-09-08 14:01:10 -05:00
Owner

This does not fix the flake, and is not meant to. #257 asks for the mechanism to be found before a fix is attempted rather than guessed at. Nothing here touches the limiter or its store. What changes is that the next failure will say which mechanism it was.

Why the old assertion could not answer the question

The test failed roughly one full-suite run in three with a strict-mode violation — getByText(/already sent several/) resolving to three refusal toasts where one was expected — and two investigations could not establish why.

The second investigation corrected the arithmetic the first relied on: antd toasts auto-dismiss, so the number visible at the moment of an assertion is a lower bound on how many refusals happened, not a count. Three visible refusals is equally consistent with four where the first had already faded.

That removed the only evidence for the original theory — that the customer's bucket held two hits before the test clicked anything — and left the issue with a symptom nobody could read. It also means toast-counting was the wrong instrument twice over: it is timing-dependent, and toBeVisible() on a multi-match locator fails in strict mode even when the behaviour was correct.

What it asserts now

The sequence of response statuses from /api/customers/resend-verification, recorded from the first navigation onward. Toasts are a lossy rendering of the behaviour; the responses are the behaviour.

A failure now reports what actually happened:

Recorded Means
[204, 204, 204, 429] Correct — three sends, one refusal
[429, 429, 429, 429] The bucket really did carry hits from somewhere else
More than four entries The UI sent more requests than there were clicks

Either failure identifies the mechanism from a single failing run, where previously it needed a temporary probe re-added and the suite run until it failed again. That is what the issue's second comment said would settle it, made permanent instead of temporary.

One ordering change

Each click now waits for its own response. The previous await expect(resend).toBeEnabled() looked like pacing but was a no-op — the button is never disabled, so it returned immediately and left four requests racing. Removing that means a failure here cannot be explained by ordering, which eliminates one candidate without guessing at it.

The copy assertion stays, because it is what the test is named for, but scoped with .first() so strict mode does not treat several identical toasts as ambiguous.

Verification

Typecheck (tsconfig.test.json) and lint clean, 0 errors.

Not verified by running it. The e2e suite needs a database and a browser this machine cannot provide — no Docker, and the active Node is too old for Playwright. Whether this passes is for CI to say. That is the same gap that let an integration regression through earlier today, so it is worth watching rather than assuming.

Worth knowing

#307's comment recorded a second sighting of the same signaturemailOutcome.test.ts passing in isolation and failing once in a full unit run, not reproduced since. Different suite, same shape. If the recorded statuses here come back clean while failures continue, that pairing is the next thread.

Refs #257

🤖 Generated with Claude Code

**This does not fix the flake, and is not meant to.** #257 asks for the mechanism to be found before a fix is attempted rather than guessed at. Nothing here touches the limiter or its store. What changes is that the next failure will say which mechanism it was. ## Why the old assertion could not answer the question The test failed roughly one full-suite run in three with a strict-mode violation — `getByText(/already sent several/)` resolving to three refusal toasts where one was expected — and two investigations could not establish why. The second investigation corrected the arithmetic the first relied on: **antd toasts auto-dismiss, so the number visible at the moment of an assertion is a lower bound on how many refusals happened, not a count.** Three visible refusals is equally consistent with four where the first had already faded. That removed the only evidence for the original theory — that the customer's bucket held two hits before the test clicked anything — and left the issue with a symptom nobody could read. It also means toast-counting was the wrong instrument twice over: it is timing-dependent, and `toBeVisible()` on a multi-match locator fails in strict mode even when the behaviour was correct. ## What it asserts now The sequence of response statuses from `/api/customers/resend-verification`, recorded from the first navigation onward. Toasts are a lossy rendering of the behaviour; the responses are the behaviour. A failure now reports what actually happened: | Recorded | Means | | --- | --- | | `[204, 204, 204, 429]` | Correct — three sends, one refusal | | `[429, 429, 429, 429]` | The bucket really did carry hits from somewhere else | | More than four entries | The UI sent more requests than there were clicks | Either failure identifies the mechanism **from a single failing run**, where previously it needed a temporary probe re-added and the suite run until it failed again. That is what the issue's second comment said would settle it, made permanent instead of temporary. ## One ordering change Each click now waits for its own response. The previous `await expect(resend).toBeEnabled()` looked like pacing but was a no-op — the button is never disabled, so it returned immediately and left four requests racing. Removing that means a failure here cannot be explained by ordering, which eliminates one candidate without guessing at it. The copy assertion stays, because it is what the test is named for, but scoped with `.first()` so strict mode does not treat several identical toasts as ambiguous. ## Verification Typecheck (`tsconfig.test.json`) and lint clean, 0 errors. **Not verified by running it.** The e2e suite needs a database and a browser this machine cannot provide — no Docker, and the active Node is too old for Playwright. Whether this passes is for CI to say. That is the same gap that let an integration regression through earlier today, so it is worth watching rather than assuming. ## Worth knowing #307's comment recorded a **second sighting of the same signature** — `mailOutcome.test.ts` passing in isolation and failing once in a full unit run, not reproduced since. Different suite, same shape. If the recorded statuses here come back clean while failures continue, that pairing is the next thread. Refs #257 🤖 Generated with [Claude Code](https://claude.com/claude-code)
bermudalamb added 1 commit 2026-09-08 12:48:38 -05:00
test(e2e): make the resend allowance failure say why (#257)
SonarQube Analysis / sonarqube (pull_request) Failing after 24m53s
Linting / lint (pull_request) Successful in 3m2s
218be6d298
The test asserted on toasts, and toasts were the wrong instrument twice over. It failed roughly one full-suite run in three with a strict mode violation — getByText(/already sent several/) resolving to three refusal toasts where one was expected — and two investigations could not establish the mechanism.

The second investigation corrected the arithmetic the first depended on: antd toasts auto-dismiss, so the number visible at the moment of an assertion is a lower bound on how many refusals happened rather than a count. Three visible refusals is equally consistent with four where the first had already faded. That removed the only evidence anyone had for the original theory that the customer's bucket held two hits before the test clicked anything, which left the issue with a symptom and no way to read it.

So this asserts the sequence of response statuses instead. Toasts are a lossy, timing-dependent rendering of the thing the test is actually about, and the responses are the behaviour itself. A failure now reports what happened: four 429s means the bucket really did carry hits from somewhere else, while more than four entries means the UI sent more requests than there were clicks. Either reading identifies the mechanism from one failing run, where before it needed a temporary probe re-added and the suite run until it failed again.

Each click now waits for its own response. The previous "await expect(resend).toBeEnabled()" looked like pacing but was a no-op, since the button is never disabled, so four requests raced. Removing that ordering means a failure cannot be blamed on it. The copy assertion stays, because it is what the test is named for, but scoped with .first() so strict mode does not treat several identical toasts as ambiguous.

This does not fix the underlying flake, and is not meant to. The issue asks for the mechanism to be found before a fix is attempted rather than guessed at, and nothing here changes the limiter or the store.

Verified by typecheck and lint only. The e2e suite needs a database and a browser this machine cannot run, so whether this passes is for CI to say — the same gap that let the integration regression through earlier today.

Refs #257

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
bermudalamb force-pushed fix/257-resend-verification-flake from 271078ff20 to 218be6d298 2026-09-08 12:48:38 -05:00 Compare
bermudalamb merged commit 0ec6064391 into main 2026-09-08 14:01:10 -05:00
bermudalamb deleted branch fix/257-resend-verification-flake 2026-09-08 14:01:10 -05:00
Sign in to join this conversation.
No Reviewers
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: bermudalamb/redefined-designs#319