MimicMotion: A Controllable Video Generation Framework for High-Quality Human Motion Synthesis

MimicMotion is an advanced controllable video generation framework designed to address the limitations of current video generation technologies, including short output duration, weak controllability, and lack of dynamic detail. The framework enables the creation of smooth, high-quality human motion videos by utilizing skeletal action guidance, making it particularly valuable for advertising applications where specific character actions are required.

The technology has been implemented by Tencent Advertising's Miaosi platform, offering a "Human Motion Video" feature that allows users to upload a single frontal standing image of a person and select from various action templates to generate advertising-ready motion videos. This functionality has been adopted across social media, education, and e-commerce industries, helping clients reduce video production costs and improve advertising efficiency.

Technically, MimicMotion employs a spatiotemporal diffusion model with U-Net architecture operating in latent space. The system incorporates three key innovations: confidence-aware pose guidance that prioritizes reliable skeletal information, hand detail enhancement using confidence-based masking to reduce deformation, and progressive latent feature fusion that enables generation of infinitely long videos while maintaining temporal coherence. These advancements result in videos with superior frame-to-frame consistency, reduced flickering, and enhanced detail preservation, particularly in hand regions.

Extensive evaluations demonstrate MimicMotion's superiority over existing methods like MagicPose, Moore-AnimateAnyone, and MuseV across multiple metrics including FID-VID and FVD scores. User studies with 36 participants showed strong preference for MimicMotion-generated videos in terms of image quality, temporal smoothness, and overall visual appeal. The framework is open-source and available on GitHub, with comprehensive documentation and technical reports published.

Looking forward, MimicMotion represents a significant breakthrough in controllable video generation, with potential applications extending beyond advertising to various industries requiring high-quality human motion synthesis. The technology's ability to generate long, smooth, and detailed videos while maintaining character consistency positions it as a valuable tool for creative content production and digital marketing.