Turning an ignored E2E suite back into a deploy gate

The end-to-end suite was red often enough that the team had quietly stopped believing it, so a real failure looked the same as the usual noise.

A suite that is red half the time is not a safety net, it is background noise. This is the shape of getting one back to a state where a red run actually stops a release. Swap in your own numbers.

The constraint

The Playwright suite existed and had good coverage on paper. In practice it failed so often that the team merged through it. Once people learn to ignore a signal, adding more tests to that signal does nothing.

What I tried first

I started by fixing individual failing tests as they came up. That was treating symptoms. The failures came back in different tests because the causes were shared: timing assumptions, tests leaning on each other’s data, and a handful of genuinely slow third-party calls.

What it became

I stopped fixing tests one at a time and fixed the categories. Fixed waits became web-first assertions. Tests that shared state got their own setup. The few real external-dependency flakes got a scoped, commented retry, and nothing else was allowed one. Then I made the pass rate visible so the trend could not be ignored.

Where it landed

• The suite moved from failing on most runs to passing reliably, so red went back to meaning broken.
• The team let it gate deploys again, which is the only real measure of a suite people trust.
• A shared vocabulary for flakiness meant new tests started stable instead of joining the noise.

The lesson I keep: trust in a suite is a number that goes down fast and climbs back slowly. Protecting it is most of the job.