Tag

flaky tests

1 views collected around this technical thread.

Test Development Learning Exchange
Test Development Learning Exchange
Sep 19, 2024 · Fundamentals

How to Use pytest‑rerunfailures to Rerun Flaky Tests

This guide explains how to install, configure, and apply the pytest‑rerunfailures plugin—including basic –r usage, example test files, configuration files, advanced custom rerun conditions, and the @pytest.mark.flaky decorator—to improve stability of intermittent Python test failures.

Pythonautomationflaky tests
0 likes · 6 min read
How to Use pytest‑rerunfailures to Rerun Flaky Tests
Continuous Delivery 2.0
Continuous Delivery 2.0
Apr 11, 2022 · Fundamentals

Root Causes of Unstable Test Cases

This article explains that unstable test cases stem from four main sources—test cases themselves, the testing framework, the system under test and its dependencies, and the underlying operating system and hardware—detailing typical causes and offering a classification guide for developers.

automationflaky testssoftware testing
0 likes · 5 min read
Root Causes of Unstable Test Cases
Continuous Delivery 2.0
Continuous Delivery 2.0
Apr 6, 2022 · Fundamentals

How to Prevent Flaky Unit Tests by Managing Test Preconditions

Unstable test cases, caused by shared resources and unchecked assumptions, can mislead developers, hide real bugs, and lead to unfair blame, so this article explains why preconditions must be verified, demonstrates flaky test examples, and offers practical strategies such as resource cleanup, unique test resources, and mock file systems to achieve reliable testing.

Unit Testingflaky testsmocking
0 likes · 4 min read
How to Prevent Flaky Unit Tests by Managing Test Preconditions
Continuous Delivery 2.0
Continuous Delivery 2.0
Mar 28, 2022 · Fundamentals

Analyzing the Origins of Flaky Tests: Size, Tooling, and Instability at Google

This article examines why some tests become flaky, showing that larger test binaries and higher RAM usage strongly correlate with instability, while the choice of testing tools has a smaller effect, and offers recommendations for reducing flaky tests in large‑scale continuous integration environments.

Google testingcontinuous integrationflaky tests
0 likes · 9 min read
Analyzing the Origins of Flaky Tests: Size, Tooling, and Instability at Google
Continuous Delivery 2.0
Continuous Delivery 2.0
Mar 23, 2022 · Operations

Why CTOs Must Pay Attention to Unstable Automated Tests: Lessons from Aviation and Google

The article draws parallels between pilots ignoring cockpit alarms and engineers overlooking flaky automated test failures, explains why unstable tests threaten software quality, presents data from Boeing 737 incidents and Google's testing practices, and outlines mitigation strategies for CTOs to improve reliability.

CTODevOpsGoogle
0 likes · 10 min read
Why CTOs Must Pay Attention to Unstable Automated Tests: Lessons from Aviation and Google
Continuous Delivery 2.0
Continuous Delivery 2.0
May 22, 2020 · Fundamentals

Avoiding Flaky Tests: Focused Test Design and Proper Use of Mocks

This article explains why flaky tests harm development, demonstrates how shared resources cause instability, and provides practical guidelines—such as using exclusive resources, refactoring for dependency injection, writing one‑scenario tests, and avoiding verification of non‑state‑changing calls—to create reliable, focused unit tests.

Unit Testingflaky testsmocking
0 likes · 7 min read
Avoiding Flaky Tests: Focused Test Design and Proper Use of Mocks