bermudalamb/redefined-designs
1
0
Fork 0
Code Issues 2 Pull Requests 1 Actions Packages Projects 3 Releases Wiki Activity

e2e: the resend-verification allowance test fails intermittently in full runs but never in isolation #257

New Issue
Closed
opened 2026-09-01 10:03:47 -05:00 by bermudalamb · 5 comments
bermudalamb commented 2026-09-01 10:03:47 -05:00
Owner
Copy Link
Copy Source

resend-verification.spec.ts:25 — "says something useful once the allowance runs out" — fails in roughly one full-suite run in three. It has never failed in isolation.

Measured today on main (after #254 and #253): three full runs, one failed and two were 155 of 155. Three consecutive isolated runs of the spec, all passed. It also failed once in an earlier parallel run during #241.

The failure is not the one the test guards against. The refusal message appears — three times:

Error: strict mode violation: getByText(/already sent several/) resolved to 3 elements:
    1) <span>we have already sent several verification emails …</span>
    2) <span>we have already sent several verification emails …</span>
    3) <span>we have already sent several verification emails …</span>

Three sibling toasts, all refusals. The test clicks resend four times against a limit of three, so exactly one refusal is expected. Three means only the first click was allowed — the customer's bucket already held two hits before the test clicked anything.

Ruled out, with the checks:

  • Not parallel contention. It reproduces with workers: 1, serially, on a freshly created database and a freshly started backend. #241's fix does not touch it.
  • Not a mis-keyed limiter. verificationResendLimiter is mounted after requireCustomer in routes/customers.ts:252, deliberately and with a comment, so req.customerId is populated and the key is not falling back to customer:anonymous.
  • Not a shared fixture. customer is test-scoped and calls uniqueEmail(), so each test registers its own customer and should get its own bucket.
  • Not identity reuse from a test helper. tests/e2e/support/db.ts only reads a password-reset token; nothing in the e2e suite truncates or resets sequences.

Still open: how a freshly created customer's bucket comes to hold two hits. verificationResendStore is a process-wide MemoryStore with a one-hour window, so anything that causes a customer id to repeat within an hour would explain it — but nothing found so far does. rateLimit.ts:88-91 already documents exactly this hazard for the integration suite, where resetDb recycles ids; the same shape of bug in e2e is the obvious suspect and I could not locate the mechanism.

Worth fixing rather than tolerating: #241's whole premise is that a red run should mean something, and this is a remaining counterexample. Exporting the store for e2e to clear, or keying the bucket on something that cannot repeat, are both plausible; the mechanism should be found first so the fix is not a guess.

`resend-verification.spec.ts:25` — "says something useful once the allowance runs out" — fails in roughly one full-suite run in three. It has never failed in isolation. Measured today on `main` (after #254 and #253): three full runs, one failed and two were 155 of 155. Three consecutive isolated runs of the spec, all passed. It also failed once in an earlier parallel run during #241. **The failure is not the one the test guards against.** The refusal message appears — three times: ``` Error: strict mode violation: getByText(/already sent several/) resolved to 3 elements: 1) <span>we have already sent several verification emails …</span> 2) <span>we have already sent several verification emails …</span> 3) <span>we have already sent several verification emails …</span> ``` Three sibling toasts, all refusals. The test clicks resend four times against a limit of three, so exactly one refusal is expected. Three means only the first click was allowed — the customer's bucket already held two hits before the test clicked anything. **Ruled out, with the checks:** - *Not parallel contention.* It reproduces with `workers: 1`, serially, on a freshly created database and a freshly started backend. #241's fix does not touch it. - *Not a mis-keyed limiter.* `verificationResendLimiter` is mounted after `requireCustomer` in `routes/customers.ts:252`, deliberately and with a comment, so `req.customerId` is populated and the key is not falling back to `customer:anonymous`. - *Not a shared fixture.* `customer` is test-scoped and calls `uniqueEmail()`, so each test registers its own customer and should get its own bucket. - *Not identity reuse from a test helper.* `tests/e2e/support/db.ts` only reads a password-reset token; nothing in the e2e suite truncates or resets sequences. **Still open:** how a freshly created customer's bucket comes to hold two hits. `verificationResendStore` is a process-wide `MemoryStore` with a one-hour window, so anything that causes a customer id to repeat within an hour would explain it — but nothing found so far does. `rateLimit.ts:88-91` already documents exactly this hazard for the integration suite, where `resetDb` recycles ids; the same shape of bug in e2e is the obvious suspect and I could not locate the mechanism. Worth fixing rather than tolerating: #241's whole premise is that a red run should mean something, and this is a remaining counterexample. Exporting the store for e2e to clear, or keying the bucket on something that cannot repeat, are both plausible; the mechanism should be found first so the fix is not a guess.
bermudalamb commented 2026-09-02 11:37:01 -05:00
Author
Owner
Copy Link
Copy Source

Found it, and reproduced it deterministically. The mechanism is outside the suite entirely, which is why the four things this issue ruled out were all correctly ruled out.

The mechanism

start-local.ps1 said "something is already listening on 3000; leaving it alone" and carried on. That is safe only while the database has not changed underneath that process. When it has — -Fresh, a recreated container, or now -E2eDb — the old backend is serving a database it no longer owns, and its in-memory state describes rows that no longer exist.

verificationResendStore is a process-wide MemoryStore with a one-hour window, keyed on customer id. A recreated database restarts ids at 1, so a brand-new customer inherits a previous run's spent allowance. That is exactly the "how does a freshly created customer's bucket come to hold two hits" this issue could not answer: the customer is fresh, the bucket is not, and nothing inside the suite can see that.

rateLimit.ts:88-91 already documents this hazard for the integration suite, where resetDb recycles ids. This issue guessed the same shape of bug was the suspect. It was — one process further out.

Reproduced, with a control

Run Setup Result
1 Fresh database, fresh backend 3 of 3 pass
2 Recreate the database only, same backend process fails — resolved to 3 elements, the exact reported symptom
3 Restart the ids again, but restart the backend too 3 of 3 pass

So the variable is the process outliving the database, not the id restart on its own.

Why it was one run in three

It needed a leftover backend from an earlier session to still be listening when a new database appeared. Nothing made that happen reliably, and nothing announced it — the script's own message read like a convenience.

The fix

start-local.ps1 now refuses when it has just created a database and something it does not own is listening on the API port, naming the reason and saying to run -Stop. It still leaves a backend alone when the database is unchanged, which is the case that message was written for.

This matters beyond the limiter. A stale listener on 3000 has produced two wrong measurements in this project already: a rate-limiter reading that was really the old process's exhausted store, and an e2e run reported as "23 passed, 53 did not run" while the backend was talking to a database that had been deleted. Both were mine. A run that stops loudly is recoverable; one that quietly tests the wrong thing is not.

Deliberately not done

Keying the limiter on something that cannot recycle. Email would work and is arguably more apt for a resend limiter, but requireCustomer only puts customerId on the request, so it would cost a query per call on a rate-limited path — a production cost to fix a harness-level hazard. Worth revisiting only if this recurs for a reason the guard does not cover.

Verified by AST-parsing the script; every reference to the new flag is script-scoped, since a function-local read would see $null and the guard would never fire. The script is deliberately never executed from an agent shell, so it is worth running -Stop and then start-local.ps1 -E2eDb once to confirm the guard behaves.

Closes #257

Found it, and reproduced it deterministically. The mechanism is outside the suite entirely, which is why the four things this issue ruled out were all correctly ruled out. ## The mechanism `start-local.ps1` said *"something is already listening on 3000; leaving it alone"* and carried on. That is safe only while the database has not changed underneath that process. When it has — `-Fresh`, a recreated container, or now `-E2eDb` — the old backend is serving a database it no longer owns, and **its in-memory state describes rows that no longer exist**. `verificationResendStore` is a process-wide `MemoryStore` with a one-hour window, keyed on customer id. A recreated database restarts ids at 1, so a brand-new customer inherits a previous run's spent allowance. That is exactly the "how does a freshly created customer's bucket come to hold two hits" this issue could not answer: the customer is fresh, the *bucket* is not, and nothing inside the suite can see that. `rateLimit.ts:88-91` already documents this hazard for the integration suite, where `resetDb` recycles ids. This issue guessed the same shape of bug was the suspect. It was — one process further out. ## Reproduced, with a control | Run | Setup | Result | |---|---|---| | 1 | Fresh database, fresh backend | 3 of 3 pass | | 2 | **Recreate the database only**, same backend process | **fails — `resolved to 3 elements`**, the exact reported symptom | | 3 | Restart the ids again, but restart the backend too | 3 of 3 pass | So the variable is the process outliving the database, not the id restart on its own. ## Why it was one run in three It needed a leftover backend from an earlier session to still be listening when a new database appeared. Nothing made that happen reliably, and nothing announced it — the script's own message read like a convenience. ## The fix `start-local.ps1` now refuses when it has just created a database and something it does not own is listening on the API port, naming the reason and saying to run `-Stop`. It still leaves a backend alone when the database is unchanged, which is the case that message was written for. This matters beyond the limiter. A stale listener on 3000 has produced two wrong measurements in this project already: a rate-limiter reading that was really the old process's exhausted store, and an e2e run reported as "23 passed, 53 did not run" while the backend was talking to a database that had been deleted. Both were mine. A run that stops loudly is recoverable; one that quietly tests the wrong thing is not. ## Deliberately not done Keying the limiter on something that cannot recycle. Email would work and is arguably more apt for a resend limiter, but `requireCustomer` only puts `customerId` on the request, so it would cost a query per call on a rate-limited path — a production cost to fix a harness-level hazard. Worth revisiting only if this recurs for a reason the guard does not cover. Verified by AST-parsing the script; every reference to the new flag is script-scoped, since a function-local read would see `$null` and the guard would never fire. The script is deliberately never executed from an agent shell, so **it is worth running `-Stop` and then `start-local.ps1 -E2eDb` once to confirm the guard behaves**. Closes #257
bermudalamb commented 2026-09-03 17:20:41 -05:00
Author
Owner
Copy Link
Copy Source

Investigated, not reproduced, and four candidate mechanisms eliminated with evidence. No fix, because the mechanism is still not known and guessing is what this issue asks not to do.

Did not reproduce

One valid full-suite run on main (168 tests): resend-verification.spec.ts:25 passed. A temporary probe spec running inside that same parallel suite recorded the requests its four clicks actually produced:

PROBE requests=4 statuses=[204,204,204,429]
PROBE allowed=3 refused=1 visibleRefusalToasts=1

Exactly the intended behaviour. The probe has been removed.

Eliminated

customer:anonymous cannot happen on this route. requireCustomer answers 401 and returns when req.customerId is missing, so nothing without an id ever reaches the limiter. The mount order in routes/customers.ts is requireCustomer, verificationResendLimiter, as the comment there claims.

A fresh customer's bucket is genuinely empty. Registered a new customer against the running backend and called the endpoint five times: 204, 204, 204, 429, 429. The full allowance is present for a new id, so the store is not carrying hits into a new bucket.

Four clicks make four requests. The probe above counts them. The browser is not double-firing, and the loop's expect(resend).toBeEnabled() — which returns immediately, since the button is never disabled — does not cause extra requests.

Cross-run id reuse via the throwaway database is not what was being observed. That mechanism is real in principle: the store is process-wide with a one-hour window, and -E2eDb recreates its database each run so ids restart at 1, which would let a later run inherit an earlier run's buckets. But no redefined-designs-e2e-db container exists on this machine, so the runs that produced the reported failures used the development database, where ids only ever increase — currently 954 customers, ids 1..954. Worth keeping in mind for anyone who does start using -E2eDb, but it does not explain the failures actually seen.

One correction to the issue text

The arithmetic that "three refusal toasts means the bucket already held two hits" is not sound. antd toasts auto-dismiss after a few seconds, so the number visible at the moment of the assertion is a lower bound on how many refusals occurred, not an exact count. Three visible refusals is equally consistent with four refusals where the first had already faded. That does not make the failure less real, but it does mean the "two prior hits" figure should not be used to narrow the search.

What would settle it

The probe spec is the right instrument and it needs to run on a failing run rather than a passing one. Re-adding it temporarily and running the suite until it fails would give the request count and status sequence at the moment of failure, which distinguishes "the bucket had hits" from "more requests than clicks" definitively. That is a cheap thing to do the next time this is seen failing.

Unrelated, and worth knowing

Seven tests in admin-disable-customer.spec.ts and admin-reserved-items.spec.ts failed in a later run, all through expect(res.ok(), 'registering …') in the customer fixture. That was the backend having stopped partway through, not a defect — curl to /api/config returned nothing and port 3000 had no listener. The fixture's own error message identified it immediately, which is the fixture doing exactly what its comment says it is for.

Investigated, **not reproduced**, and four candidate mechanisms eliminated with evidence. No fix, because the mechanism is still not known and guessing is what this issue asks not to do. ## Did not reproduce One valid full-suite run on `main` (168 tests): `resend-verification.spec.ts:25` passed. A temporary probe spec running inside that same parallel suite recorded the requests its four clicks actually produced: ``` PROBE requests=4 statuses=[204,204,204,429] PROBE allowed=3 refused=1 visibleRefusalToasts=1 ``` Exactly the intended behaviour. The probe has been removed. ## Eliminated **`customer:anonymous` cannot happen on this route.** `requireCustomer` answers 401 and returns when `req.customerId` is missing, so nothing without an id ever reaches the limiter. The mount order in `routes/customers.ts` is `requireCustomer, verificationResendLimiter`, as the comment there claims. **A fresh customer's bucket is genuinely empty.** Registered a new customer against the running backend and called the endpoint five times: `204, 204, 204, 429, 429`. The full allowance is present for a new id, so the store is not carrying hits into a new bucket. **Four clicks make four requests.** The probe above counts them. The browser is not double-firing, and the loop's `expect(resend).toBeEnabled()` — which returns immediately, since the button is never disabled — does not cause extra requests. **Cross-run id reuse via the throwaway database is not what was being observed.** That mechanism is real in principle: the store is process-wide with a one-hour window, and `-E2eDb` recreates its database each run so ids restart at 1, which would let a later run inherit an earlier run's buckets. But no `redefined-designs-e2e-db` container exists on this machine, so the runs that produced the reported failures used the development database, where ids only ever increase — currently 954 customers, ids 1..954. Worth keeping in mind for anyone who does start using `-E2eDb`, but it does not explain the failures actually seen. ## One correction to the issue text The arithmetic that "three refusal toasts means the bucket already held two hits" is not sound. antd toasts auto-dismiss after a few seconds, so the number visible at the moment of the assertion is a lower bound on how many refusals occurred, not an exact count. Three visible refusals is equally consistent with four refusals where the first had already faded. That does not make the failure less real, but it does mean the "two prior hits" figure should not be used to narrow the search. ## What would settle it The probe spec is the right instrument and it needs to run on a failing run rather than a passing one. Re-adding it temporarily and running the suite until it fails would give the request count and status sequence at the moment of failure, which distinguishes "the bucket had hits" from "more requests than clicks" definitively. That is a cheap thing to do the next time this is seen failing. ## Unrelated, and worth knowing Seven tests in `admin-disable-customer.spec.ts` and `admin-reserved-items.spec.ts` failed in a later run, all through `expect(res.ok(), 'registering …')` in the `customer` fixture. That was the backend having stopped partway through, not a defect — `curl` to `/api/config` returned nothing and port 3000 had no listener. The fixture's own error message identified it immediately, which is the fixture doing exactly what its comment says it is for.
bermudalamb commented 2026-09-08 12:48:55 -05:00
Author
Owner
Copy Link
Copy Source

Instrumented rather than fixed, in PR #319 (218be6d). The mechanism is still unknown and nothing here changes the limiter or its store — this issue asks for the cause to be found before a fix is attempted, so what changes is only that the next failure will say which cause it was.

Why the old assertion could not answer the question

The comment above corrected the arithmetic the issue text depends on: antd toasts auto-dismiss, so the number visible at the moment of an assertion is a lower bound on how many refusals occurred, not a count. Three visible refusals is equally consistent with four where the first had already faded.

That correction removed the only evidence for the original theory — that the bucket already held two hits — and left a symptom nobody could read. It also means toast-counting was the wrong instrument twice over:

  • It is timing-dependent, so the recorded number is not the number that happened.
  • toBeVisible() on a multi-match locator fails in strict mode even when the behaviour was correct, which is a defect in the test independent of the flake.

What it asserts now

The sequence of response statuses from /api/customers/resend-verification, recorded from the first navigation onward. The toasts are a lossy rendering of the behaviour; the responses are the behaviour.

Recorded Reading
[204, 204, 204, 429] Correct — three sends, one refusal
[429, 429, 429, 429] The bucket really did carry hits from somewhere else
More than four entries The UI sent more requests than there were clicks

The failure message carries the sequence and names the customer, since whose bucket it was is the open question.

This is exactly what the previous comment said would settle it — "the probe spec is the right instrument and it needs to run on a failing run" — made permanent instead of temporary. It no longer needs anyone to re-add a probe and run the suite until it fails.

One candidate eliminated rather than guessed at

Each click now waits for its own response. The previous await expect(resend).toBeEnabled() looked like pacing but was a no-op, because the button is never disabled — so four requests raced. That ordering can no longer explain a failure. Whether it ever did is unknown; it is simply no longer a variable.

The copy assertion stays, scoped with .first(), because the useful message is what this test is named for.

Not verified by running it

Typecheck and lint are clean, but the e2e suite needs a database and a browser this machine cannot provide — no Docker, and the active Node is too old for Playwright. Whether it passes is for CI to say. Worth watching rather than assuming: the same gap let an integration regression through earlier today, caught only by CI.

The other thread

#307's comment recorded a second sighting of the same signature — mailOutcome.test.ts passing in isolation and failing once in a full unit run, never reproduced. Different suite, same shape: passes alone, fails in company. If the statuses here come back clean while failures continue, that pairing is where to look next, because it would mean the cause is not specific to this limiter at all.

Instrumented rather than fixed, in PR #319 (`218be6d`). **The mechanism is still unknown and nothing here changes the limiter or its store** — this issue asks for the cause to be found before a fix is attempted, so what changes is only that the next failure will say which cause it was. ## Why the old assertion could not answer the question The comment above corrected the arithmetic the issue text depends on: antd toasts auto-dismiss, so the number visible at the moment of an assertion is a **lower bound** on how many refusals occurred, not a count. Three visible refusals is equally consistent with four where the first had already faded. That correction removed the only evidence for the original theory — that the bucket already held two hits — and left a symptom nobody could read. It also means toast-counting was the wrong instrument twice over: - It is timing-dependent, so the recorded number is not the number that happened. - `toBeVisible()` on a multi-match locator fails in strict mode **even when the behaviour was correct**, which is a defect in the test independent of the flake. ## What it asserts now The sequence of response statuses from `/api/customers/resend-verification`, recorded from the first navigation onward. The toasts are a lossy rendering of the behaviour; the responses are the behaviour. | Recorded | Reading | | --- | --- | | `[204, 204, 204, 429]` | Correct — three sends, one refusal | | `[429, 429, 429, 429]` | The bucket really did carry hits from somewhere else | | More than four entries | The UI sent more requests than there were clicks | The failure message carries the sequence and names the customer, since whose bucket it was is the open question. This is exactly what the previous comment said would settle it — "the probe spec is the right instrument and it needs to run on a failing run" — made permanent instead of temporary. It no longer needs anyone to re-add a probe and run the suite until it fails. ## One candidate eliminated rather than guessed at Each click now waits for its own response. The previous `await expect(resend).toBeEnabled()` looked like pacing but was a no-op, because the button is never disabled — so four requests raced. That ordering can no longer explain a failure. Whether it ever did is unknown; it is simply no longer a variable. The copy assertion stays, scoped with `.first()`, because the useful message is what this test is named for. ## Not verified by running it Typecheck and lint are clean, but the e2e suite needs a database and a browser this machine cannot provide — no Docker, and the active Node is too old for Playwright. Whether it passes is for CI to say. Worth watching rather than assuming: the same gap let an integration regression through earlier today, caught only by CI. ## The other thread #307's comment recorded a **second sighting of the same signature** — `mailOutcome.test.ts` passing in isolation and failing once in a full unit run, never reproduced. Different suite, same shape: passes alone, fails in company. If the statuses here come back clean while failures continue, that pairing is where to look next, because it would mean the cause is not specific to this limiter at all.
bermudalamb commented 2026-09-08 15:35:24 -05:00
Author
Owner
Copy Link
Copy Source

The rewritten spec passed CI

Run 881, on 218be6d, ran all three resend tests green — including the rewritten one:

✓ 158 resend-verification.spec.ts:49:7 › says something useful once the allowance runs out (5.4s)

So the status-sequence assertion works against a real run. It was previously unverified; it no longer is. The other failures in that run were the #56 fallout, unrelated to this spec and fixed in #320.

A data point that narrows this issue, from two CI runs

The flaky test is not the same test twice:

Run Flaky spec Failed attempt Retry
875 auth.spec.ts:96 — the logged-out header survives a reload — passed
881 account-details.spec.ts:85 — refuses an email change when the password is wrong 31.5s 7.1s

Two things follow, and both narrow the search:

It rotates between unrelated specs. Neither of these is the resend test, neither touches the verification limiter, and they have nothing in common with each other. Whatever this is, it is not specific to verificationResendLimiter — which is consistent with the mailOutcome.test.ts sighting recorded on #307, in a different suite entirely.

The failing attempt took four times as long as the retry. 31.5s against 7.1s is a timeout being hit, not a wrong answer being computed. That is a load or contention signature, not a state one.

What that rules out

CI runs against a fresh Postgres service container every time, so:

  • The accumulated-junk theory in project-context.md — "when failures start rotating between unrelated specs, reset the database" — cannot apply here. There is nothing accumulated to reset.
  • The recycled-id theory from this issue's first comment cannot apply either, for the same reason.

Both were about the local database. The rotation happens in CI regardless, which means the cause survives a clean database.

Where that leaves it

The remaining shape is timing under parallel workers: assertions waiting on a round trip that is slow enough, often enough, to cross a timeout — the hazard project-context.md already describes for bcrypt-bound registration, where "the failure surfaces on whichever test lost the race, so it looks like an unrelated flake that moves between runs."

That is a good description of what these two runs show. It is not proof, and this issue still asks for the mechanism rather than a guess, so nothing is being changed on the strength of it. But it is worth recording that the two theories this issue was built on are now both excluded by evidence rather than by argument.

The instrumentation stays either way: if the resend test is the one that fails next, the status sequence will say whether it was slow or wrong.

## The rewritten spec passed CI [Run 881](https://gitea.bermudalamb.synology.me/bermudalamb/redefined-designs/actions/runs/881), on `218be6d`, ran all three resend tests green — including the rewritten one: ``` ✓ 158 resend-verification.spec.ts:49:7 › says something useful once the allowance runs out (5.4s) ``` So the status-sequence assertion works against a real run. It was previously unverified; it no longer is. The other failures in that run were the #56 fallout, unrelated to this spec and fixed in #320. ## A data point that narrows this issue, from two CI runs The flaky test **is not the same test twice**: | Run | Flaky spec | Failed attempt | Retry | | --- | --- | --- | --- | | [875](https://gitea.bermudalamb.synology.me/bermudalamb/redefined-designs/actions/runs/875) | `auth.spec.ts:96` — the logged-out header survives a reload | — | passed | | [881](https://gitea.bermudalamb.synology.me/bermudalamb/redefined-designs/actions/runs/881) | `account-details.spec.ts:85` — refuses an email change when the password is wrong | **31.5s** | **7.1s** | Two things follow, and both narrow the search: **It rotates between unrelated specs.** Neither of these is the resend test, neither touches the verification limiter, and they have nothing in common with each other. Whatever this is, it is not specific to `verificationResendLimiter` — which is consistent with the `mailOutcome.test.ts` sighting recorded on #307, in a different suite entirely. **The failing attempt took four times as long as the retry.** 31.5s against 7.1s is a timeout being hit, not a wrong answer being computed. That is a load or contention signature, not a state one. ## What that rules out CI runs against a **fresh Postgres service container every time**, so: - The accumulated-junk theory in `project-context.md` — "when failures start rotating between unrelated specs, reset the database" — cannot apply here. There is nothing accumulated to reset. - The recycled-id theory from this issue's first comment cannot apply either, for the same reason. Both were about the local database. The rotation happens in CI regardless, which means the cause survives a clean database. ## Where that leaves it The remaining shape is timing under parallel workers: assertions waiting on a round trip that is slow enough, often enough, to cross a timeout — the hazard `project-context.md` already describes for bcrypt-bound registration, where "the failure surfaces on whichever test lost the race, so it looks like an unrelated flake that moves between runs." That is a good description of what these two runs show. It is not proof, and this issue still asks for the mechanism rather than a guess, so nothing is being changed on the strength of it. But it is worth recording that the two theories this issue was built on are now both excluded by evidence rather than by argument. The instrumentation stays either way: if the resend test is the one that fails next, the status sequence will say whether it was slow or wrong.
bermudalamb commented 2026-09-09 09:22:46 -05:00
Author
Owner
Copy Link
Copy Source

Closing. The instrumentation is merged and verified; the underlying flake is not solved, and this records what is known so a recurrence starts from evidence rather than from scratch.

What shipped

resend-verification.spec.ts asserts the sequence of response statuses rather than counting toasts. Verified passing in run 881 — all three tests green, the rewritten one in 5.4s.

If it fails again, the failure message says which mechanism it was:

Recorded Reading
[204, 204, 204, 429] Correct
[429, 429, 429, 429] The bucket carried hits from elsewhere
More than four entries The UI sent more requests than there were clicks

What was ruled out, and why the issue's own framing was wrong

Both theories this issue was built on are excluded by evidence:

  • Recycled customer ids — CI runs a fresh Postgres service container every time, so ids cannot carry between runs.
  • Accumulated local database junk — same reason. The rotation happens in CI regardless.

And the arithmetic the issue opened with does not hold: antd toasts auto-dismiss, so three visible refusals is a lower bound, not a count. "The bucket already held two hits" was never a sound inference from the screenshot.

What the evidence actually points at

Two CI runs, two different flaky specs, neither touching the verification limiter:

Run Spec Failed Retry
875 auth.spec.ts:96 — passed
881 account-details.spec.ts:85 31.5s 7.1s

Four times slower on the failing attempt is a timeout being crossed, not a wrong answer computed — a load signature, not a state one. Combined with the mailOutcome.test.ts sighting on #307, in a different suite entirely, this is not specific to verificationResendLimiter at all.

The shape that fits is timing under parallel workers, which project-context.md already describes for bcrypt-bound registration: "the failure surfaces on whichever test lost the race, so it looks like an unrelated flake that moves between runs."

That is a hypothesis, not a finding, and nothing was changed on the strength of it — this issue asked for the mechanism rather than a guess, and the mechanism is still unproven.

Reopen this if the resend test fails again; the recorded statuses will settle it in one run. If a different spec flakes instead, that belongs in a new issue about suite-wide timing rather than here.

Closing. The instrumentation is merged and verified; the underlying flake is not solved, and this records what is known so a recurrence starts from evidence rather than from scratch. ## What shipped `resend-verification.spec.ts` asserts the **sequence of response statuses** rather than counting toasts. Verified passing in [run 881](https://gitea.bermudalamb.synology.me/bermudalamb/redefined-designs/actions/runs/881) — all three tests green, the rewritten one in 5.4s. If it fails again, the failure message says which mechanism it was: | Recorded | Reading | | --- | --- | | `[204, 204, 204, 429]` | Correct | | `[429, 429, 429, 429]` | The bucket carried hits from elsewhere | | More than four entries | The UI sent more requests than there were clicks | ## What was ruled out, and why the issue's own framing was wrong Both theories this issue was built on are excluded by evidence: - **Recycled customer ids** — CI runs a fresh Postgres service container every time, so ids cannot carry between runs. - **Accumulated local database junk** — same reason. The rotation happens in CI regardless. And the arithmetic the issue opened with does not hold: antd toasts auto-dismiss, so three visible refusals is a **lower bound**, not a count. "The bucket already held two hits" was never a sound inference from the screenshot. ## What the evidence actually points at Two CI runs, two different flaky specs, neither touching the verification limiter: | Run | Spec | Failed | Retry | | --- | --- | --- | --- | | 875 | `auth.spec.ts:96` | — | passed | | 881 | `account-details.spec.ts:85` | **31.5s** | **7.1s** | Four times slower on the failing attempt is a **timeout being crossed, not a wrong answer computed** — a load signature, not a state one. Combined with the `mailOutcome.test.ts` sighting on #307, in a different suite entirely, this is not specific to `verificationResendLimiter` at all. The shape that fits is timing under parallel workers, which `project-context.md` already describes for bcrypt-bound registration: "the failure surfaces on whichever test lost the race, so it looks like an unrelated flake that moves between runs." That is a hypothesis, not a finding, and nothing was changed on the strength of it — this issue asked for the mechanism rather than a guess, and the mechanism is still unproven. **Reopen this if the resend test fails again**; the recorded statuses will settle it in one run. If a *different* spec flakes instead, that belongs in a new issue about suite-wide timing rather than here.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
No items
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
calamb (Chelsea Lamb) bermudalamb (Thom Lamb)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: bermudalamb/redefined-designs#257