Relation-Aware Diffusion Models for Automated Poster Layout and Product Background Generation

In 2023, JD Advertising introduced a series of AI-driven methods to automate e‑commerce poster creation, addressing the inefficiencies of manual design by leveraging a relation‑aware diffusion model that incorporates visual‑text and geometric relationships.

The model uses a Visual‑Text Relation Awareness Module (VTRAM) to align image and textual features via cross‑attention, and a Geometric Relation Awareness Module (GRAM) to encode relative positions of Regions of Interest, enabling controllable layout generation.

To achieve scalable and personalized backgrounds, a category‑common generator extracts generic background cues from product images, while a personalized generator learns style from reference images; both are integrated into Stable Diffusion.

A planning‑and‑rendering framework (P&R) combines a PlanNet that predicts element layouts from product visuals and text, and a RenderNet that fuses layout, visual, and spatial information through a spatial‑fusion module and ControlNet to produce final posters.

The paper concludes with a technical roadmap summarizing the three‑stage solution and outlines future research directions such as controllability, multimodal integration, and personalized advertising generation.