Flake it till you make it: how to detect and deal with flaky tests (Ep. 528)

But it works on my machine at least half the time!

Article hero image

On today's episode we chat with Prof. Gregory Kapfhammer, Associate Professor in the Department of Computer Science at Allegheny College, and a specialist in the subject of flaky tests. He breaks down his definition of what makes a test flaky, how you can solve for it, and automated solutions that take some of the guesswork and busywork out of avoiding them altogether.

Episode Notes

There is a ton of great research to be found on Prof. Kapfhammer's website, including:

  • Flaky Tests: Finding and fixing unpredictable and harmful test cases
  • Database Testing: Automatically testing relational database schemas
  • Web Testing: Detecting and repairing poor responsive web page layout

We've written a bit about how Stack Overflow is upping its unit testing game and how you can evaluate multiple assertions in a single test.

Thanks to our lifeboat badge winner of the week, Survivor, for answering the question: Is it possible to find out if a value exists twice in an arraylist?

And thanks to Andre for writing in to let us know that yes, there are folks who code on mobile. He suggests trying: - Termux https://wiki.termux.com/wiki/Main_Page can be used as a terminal emulator. I have all I need here: Vim, git, node.js, Python - Markor https://gsantner.net/project/markor.html has a HTML view


Login with your stackoverflow.com account to take part in the discussion.