Page Difference Detection for Automated Regression Testing in Mobile Apps

Background: Regression testing verifies that code changes do not introduce new bugs. Traditional automated regression testing requires manually written scripts to capture and compare view trees, which becomes cumbersome when features iterate frequently. Directly comparing recorded and replayed page images can quickly locate defects.

Challenges in the Xianyu app: The page consists of a fixed top area, a scrollable middle area, and a fixed bottom area. Differences caused by scrolling or cursor presence should be ignored, while semantic differences (e.g., changed icons) must be detected.

Algorithm Overview: The method includes layout segmentation, scroll‑area alignment, similarity calculation, and post‑processing.

1. Layout Segmentation: Convert recorded and replayed images to grayscale, then split them into top, middle, and bottom regions. For scrollable middle areas, detect the upper and lower bounds by scanning rows and computing the Structural Similarity Index (SSIM) until the similarity falls below a threshold.

2. Scroll‑Area Alignment: Detect sparse feature points in both images using ORB (Oriented FAST and Rotated BRIEF). Match points with Hamming distance and brute‑force search, then compute a 3×3 transformation matrix via least‑squares to align the scrollable region.

3. Similarity Calculation: Compute SSIM for each aligned region (or the whole image if no scrollable area). Threshold the SSIM map with Otsu’s method, extract contours, and draw bounding boxes around differences.

4. Post‑Processing: Discard boxes that lie within a configurable vertical range around the scrollable area or whose area is below a size threshold. Additionally, a CNN binary classifier (three conv‑pool‑ReLU layers, 128‑node dense layer, softmax output) filters out differences caused by cursors or texture‑rich icons that are semantically identical.

Results: Experiments on Xianyu pages show that the pipeline correctly identifies semantic differences while ignoring irrelevant scroll‑induced changes. Some false positives remain when pixel‑level variations affect SSIM in non‑scrolling regions.

Conclusion: The proposed approach—layout segmentation, ORB‑based alignment, SSIM similarity, and CNN‑based filtering—provides an effective, reusable solution for automated UI regression testing of mobile applications.