Fixing a flaky test without reaching for retry

A retry hides the flake instead of removing it. Here is the order I work through before I let a test run twice.

The fastest way to make a flaky test green is to retry it. It is also the fastest way to lose trust in the whole suite, because a retry does not fix the flake, it just gives the flake a second chance to pass. Do that across a suite and a red run stops meaning anything.

So before I add a retry, I work through the same short list.

Is it waiting on the right thing

Most flakiness is a timing assumption in disguise. The test acts before the app is ready, and it only fails when the machine is slow. Fixed sleeps are the usual culprit. Web-first assertions that wait for a real condition remove almost all of this.

// Flaky: races the app.
await page.waitForTimeout(500);
expect(await page.locator('.total').textContent()).toBe('$42.00');

// Stable: waits for the condition, not the clock.
await expect(page.locator('.total')).toHaveText('$42.00');

Is the test sharing state

If a test passes alone but fails in the suite, it is reading state that another test wrote. Shared logins, a seeded record someone else deletes, a counter that assumes it ran first. The fix is isolation, not a retry: each test sets up what it needs and does not depend on order.

Is it the app or the test

Sometimes the flake is real. The app has a genuine race condition and the test is the only thing honest enough to catch it. This is the case worth protecting. If you retry it away, you have used your test suite to hide a production bug from yourself.

When a retry is actually fine

There is one honest use: a dependency you do not control, like a third party that is occasionally slow, where a single retry reflects how a real user would experience it. Even then I scope the retry to that spec and leave a comment saying why, so the next person does not read it as permission to retry everything.

The rule I keep coming back to: a test suite is only worth running if you believe it when it goes red.