Avoiding Flaky Tests: Focused Test Design and Proper Use of Mocks

Flaky tests make development harder by generating noisy failures, desensitising developers to real problems, and causing unfair blame for code changes; many factors can make a test unstable.

For example, a unit test that reads a file assumes the path /leases/gap exists and contains data; when the test testCreateGapLease runs in parallel with another test using the same file, the assumption may be violated and the test fails.

Clearing the file at the start of the test reduces fragility, but if the path is on NFS or another test writes to it, failures can still occur, so tests should own exclusive resources.

Refactor the code to inject the file path (e.g., via a creategaplease method) so production code calls the real method while tests pass a temporary path to CreateGapLease , isolating the resource.

To keep tests fast, use a mock file system that completely shields disk access.

Tests should focus on a single scenario; a test that checks three cases—having $5, overdraft rejection, and allowed overdraft—should be split into three separate tests, each with a clear name.

This approach yields clearer logic, simpler setup, no side effects between scenarios, independent failure handling, and descriptive test names.

An indicator of multiple scenarios in one test is asserting a call’s output and then making another call on the system under test.

Method calls can be classified as state‑changing (e.g., sendEmail() , saveRecord() ) or non‑state‑changing (e.g., getUser() , readFile() ). Verifying non‑state‑changing calls is usually unnecessary and makes tests brittle.

Instead, use non‑state‑changing methods to set up conditions, then assert on return values or on the state‑changing method’s effect; only verify non‑state‑changing calls when they are the only observable behaviour (e.g., caching an RPC call).

For more details see "Growing Object‑Oriented Software, Guided by Tests", which uses the terms “command” and “query” for state‑changing and non‑state‑changing methods.