Chaplin Project: Template‑Based Dynamic Image‑to‑Video Rendering

Background : Mobile internet evolution (1G‑4G) has driven short‑video growth. Xianyu, a C2C marketplace, wants to improve browsing experience by turning product images into video‑like presentations.

Direct image‑to‑video synthesis was rejected because strong guidance harms publishing success, weak guidance is ineffective, and the solution would waste resources on push notifications and cannot handle the massive existing product pool.

Solution : Use a “dynamic effect template” approach. Materials (images) are separated from visual effects (OpenGL shaders) and combined at playback time via templates.

Chaplin Project : Inspired by Charlie Chaplin’s silent films, the project abstracts video editing into reusable templates. Templates are managed centrally, support versioned publishing, and are delivered to clients.

Design Components :

Template editor – visual tool for designers, extensible effect library built with OpenGL shaders.

Resource management platform – handles music, effect, and template libraries; supports feature tagging and playback control.

Client side – incremental template sync, local template cache, shader and asset management.

Template recommendation engine – analyzes product content and image attributes to personalize template suggestions.

Playback Architecture :

Multiple threads: control queue, GL rendering, resource loading, UI refresh (30 FPS target).

Multi‑track (non‑linear) playback merges tracks in real time; resource pre‑loading and texture preprocessing reduce stutter.

Template Protocol : All coordinates are normalized to [0,1]. Effects are defined by shader URLs and uniform parameters. Example effect definition:

{
  "name":"放大",
  "type":"image",
  "duration":2.0,
  "shader":{
    "vertexURL":"https://xianyu-chaplin-bucket.oss-cn-beijing.aliyuncs.com/ShaderV2/ImageEffect_Move2_V2/ImageEffect_Move2_V2.vert",
    "fragmentURL":"https://xianyu-chaplin-bucket.oss-cn-beijing.aliyuncs.com/ShaderV2/ImageEffect_Move2_V2/ImageEffect_Move2_V2.frag",
    "uniforms":{
      "u_FirstScaleBegin":1.00,
      "u_FirstScaleRange":0.04,
      "u_FirstMoveBeginX":0.00,
      "u_FirstMoveRangeX":0.00,
      "u_FirstMoveBeginY":0.00,
      "u_FirstMoveRangeY":0.00
    }
  }
}

Transition effect example:

{
  "name":"溶解",
  "type":"transition",
  "duration":0.5,
  "shader":{
    "vertexURL":"https://xianyu-chaplin-bucket.oss-cn-beijing.aliyuncs.com/ShaderV2/TransitionEffect_Mix2_V2/TransitionEffect_Mix2_V2.vert",
    "fragmentURL":"https://xianyu-chaplin-bucket.oss-cn-beijing.aliyuncs.com/ShaderV2/TransitionEffect_Mix2_V2/TransitionEffect_Mix2_V2.frag",
    "uniforms":{
      "u_FirstScaleBegin":1.04,
      "u_FirstScaleRange":0.01,
      "u_SecondScaleBegin":1.00,
      "u_SecondScaleRange":0.01,
      "u_FirstMoveBeginX":0.00,
      "u_FirstMoveRangeX":0.00,
      "u_FirstMoveBeginY":0.00,
      "u_FirstMoveRangeY":0.00,
      "u_SecondMoveBeginX":0.00,
      "u_SecondMoveRangeX":0.00,
      "u_SecondMoveBeginY":0.00,
      "u_SecondMoveRangeY":0.00
    }
  }
}

Loop Template Design : Reduces template count and size, supports start/end with image nodes, optional transition nodes, and “once” execution flags to avoid redundant processing.

Summary & Outlook : The template‑based approach successfully replaces naïve image‑to‑video synthesis, delivering richer user experience while conserving resources. Future work includes intelligent template generation, AI‑driven subject detection for effect placement, and extending the solution to UGC creation scenarios.