Boost Regression Testing Efficiency with the Kudiffy Platform

Background Introduction

System refactoring, splitting, and underlying dependency changes cause large impact ranges, making test regression redundant and complex; how can regression testing be performed efficiently?

To address this, we combined internal resources with excellent open‑source tools to develop the Kudiffy platform.

Open‑Source Tools

Diffy

Twitter's open‑source automated testing tool that distributes requests to different server versions and compares results for API regression without manual verification. See GitHub .

Goreplay

An open‑source traffic replay tool that records and replays HTTP requests. See GitHub Wiki .

Using these tools improves efficiency, but we encountered issues such as onboarding time cost and poor service maintainability.

Kudiffy Platform

System Overview

Overall Architecture:

Features

Platform‑wide Diffy service management.

Integration of Goreplay and Diffy for real‑time traffic replay and result comparison.

Connection to internal DevOps to obtain SOA and API information.

Permission control via SSO and CMDB.

Automatic restart of Diffy services after each deployment.

Integration with API automation platforms for automatic result verification.

Usage

1. Diffy service creation: zero‑cost onboarding, linking internal service permissions, and filtering servers/IPs accessible to the logged‑in user.

2. Platform displays Diffy service health; if killed, it can be manually restarted.

3. View running results directly on the page.

4. Combine Diffy with API automation frameworks for automatic result validation without manual coding.

Integration of Goreplay and Diffy:

1. Traffic diversion service onboarding with specific API filtering.

2. Page display of integrated service.

3. After traffic diversion, traffic on the target server is recorded and sent to the corresponding Diffy service for automatic result comparison.

Future Plans

Kudiffy is already used in multiple business groups, significantly improving regression efficiency. Future improvements include:

Mock functionality to stabilize Diffy results for long call chains and database mock to prevent data pollution in POST replay.

Customizable adaptations based on business characteristics.

Process optimization, such as encouraging developers to use the tool for self‑testing and adding Diffy results to quality checkpoints.

Contributors: Xiao Bang, Xia Ling, San Li, Xi Han, Shao An, Si Si.