Kuaishou’s Self‑Developed Green‑Screen Matting Algorithm and Its Deployment in Kuaiying, Live Companion, and Cloud Editing

In sci‑fi movies and live broadcasts, green‑screen matting enables striking visual effects, and Kuaishou’s audio‑video team has built a self‑developed algorithm that powers the Kuaiying app, Kuaishou Live Companion, and Kuaishou Cloud Editing.

What is green‑screen matting? The technique separates foreground subjects from a uniformly colored background. Green is preferred because it rarely appears in human skin tones and cameras capture it with the highest signal‑to‑noise ratio, making it easier to isolate.

Practical issues include uneven lighting, green‑colored objects in the foreground, edge artifacts, color spill, and background reflections, all of which degrade the quality of the composited video.

Algorithm challenges

Fine‑detail matting : Simple hue‑thresholding in HSV leaves harsh edges and loses fine structures such as hair. The team models each pixel’s opacity (α) using weighted blending of foreground (F) and background (B) colors and solves an inverse problem to obtain a smooth α‑mask.

Color spill handling : Pixels identified as green (high G channel, hue within the green range) have their saturation reduced, mitigating green halos around subjects.

Green‑reflection removal : A gray‑world white‑balance method estimates a neutral gray value K from the average R, G, B channels and applies per‑channel scaling to neutralize background reflections.

The pipeline first converts the RGB image to HSV and YCbCr, computes a combined color distance (disH from HSV, disY from YCbCr), and maps this distance to opacity using a Gaussian function. Shadow regions are handled separately by also weighting the V (value) channel.

To smooth the α‑mask without blurring the background, a guided‑filter (edge‑preserving) is applied, which is computationally efficient for mobile devices.

Deployment

For real‑time mobile use, the algorithm runs on the GPU. Video frames are decoded, processed frame‑by‑frame, and the results are integrated via platform‑specific SDKs: OpenGL ES for Kuaiying, Direct3D 11 for Live Companion, and a C++ SDK for Cloud Editing.

The solution achieves hair‑level detail, smooth edges, and effective spill and reflection removal, outperforming competing products in both completeness and visual quality.

All referenced images from the original article are retained below:

… (additional illustrative images omitted for brevity) …

Conclusion

The green‑screen matting feature is now fully launched in Kuaishou’s video products, delivering superior edge precision, spill control, and reflection removal, and is available for developers and end‑users alike.