JianYi: AI‑Powered Image Segmentation and Matting System for Taobao Home‑Decoration

This article presents JianYi, an in‑house image segmentation and matting system designed for the Taobao home‑decoration industry. JianYi focuses on three core functions—product segmentation, human segmentation, and panoramic segmentation—while providing three interaction modes (single‑image, image‑text, and image‑box) for efficient multi‑modal user experiences.

The system processes millions of images daily, delivering high accuracy, real‑time speed, and robust performance across complex indoor scenes. It is integrated into two AI applications: "Jiazuo" (which offers intelligent one‑click matting, selective item matting, and indoor studio preprocessing) and "Fang Wo Jia" (which enables panoramic segmentation for scene reconstruction, soft‑decoration layering, and foreground‑background separation).

Technically, JianYi employs a BiRefNet‑based salient segmentation backbone optimized with multi‑source data (3D‑rendered, manually annotated, and synthetic datasets) and combines closed‑set detection (YOLO11) with open‑set detection (Grounding DINO) to generate precise bounding boxes. Post‑processing replaces traditional morphological operations with connected‑component merging and minimal‑set extraction to reduce noise and preserve fine details.

Extensive evaluations show JianYi outperforms external matting tools, achieving cleaner edges, complete object extraction, and higher DAU growth (e.g., a 196% increase). Comparative visual results and quantitative metrics are provided, demonstrating superior performance in both online real‑time and offline batch processing scenarios.

The paper also discusses model fine‑tuning for specific challenges such as human‑object grouping and furniture separation, and outlines future directions for AI model improvement within the home‑decoration domain.