How I2V-Adapter Turns Images into Videos with Minimal Training

Research Background

Generating video from a single static image (I2V) is challenging because the model must infer temporal dynamics while preserving the identity and semantics of the input image. Existing I2V methods often require extensive modifications to text‑to‑video (T2V) diffusion models and large amounts of training data, leading to high computational cost and limited compatibility.

Research Approach

The proposed I2V‑Adapter is a lightweight adaptation module for Stable Diffusion‑based video diffusion models. It injects the input image as the first video frame and as a parallel noise stream. In the spatial self‑attention block, every frame queries the key/value from the clean first frame, and the output is added to the original attention output. The added mapping matrices are zero‑initialized, allowing the model to retain the pretrained T2V weights while only training the new adapter parameters.

To improve stability, a Frame Similarity Prior assumes that, at low noise levels, consecutive frames of a video are similar. The noisy input image is used as a prior for subsequent frames, with Gaussian blur and random masking applied to suppress high‑frequency noise.

Experimental Results

The I2V‑Adapter was evaluated on four quantitative metrics: DoverVQA (aesthetic score), CLIPTemp (first‑frame consistency), FlowScore (motion magnitude), and WarpingError (motion error). It achieved the highest aesthetic score, superior first‑frame consistency, and the best trade‑off between motion intensity and error, outperforming prior methods while training only 1% of the parameters and using 18% of the data.

Business Applications

Fast‑forward’s AI team partnered with MediaTek to integrate I2V‑Adapter into the Dimensity platform, enabling on‑device conversion of photos into dynamic videos with customizable styles, background music, and text overlays. The solution also works with personalized T2I models and ControlNet, offering highly controllable and stylized I2V generation.

Future Outlook

I2V‑Adapter’s plug‑and‑play design, decoupled attention mechanisms, and compatibility with DreamBooth, LoRA, and ControlNet make it a versatile foundation for future research and commercial products in image‑to‑video generation.