How Ubisoft Embraces Generative AI for 2D Image Generation in AAA Game Development

In this presentation, Ubisoft China shares how the studio is adopting generative AI and applying 2D image generation techniques to AAA game development.

Ubisoft, a global leader in game development known for titles such as Assassin’s Creed and Far Cry, has operated in China since 1996 and now runs two studios with a talent pool of over a thousand people.

Agenda

Speaker introduction – Alexis Rolland

The generative AI revolution

Impact of generative AI on 2D art creation

Ubisoft’s strategic initiatives

Q&A

Generative AI Revolution

Since 2015, image‑generation research has accelerated, especially after the release of Text‑guided Diffusion, Disco Diffusion (2021), DALL‑E 2 (2022), and the open‑source Stable Diffusion models (2022). The rapid improvement in image quality and compute efficiency, highlighted by releases such as Stable Diffusion XL and DALL‑E 3, has sparked a new wave of creativity.

Key points include the open‑source nature of these models, their ability to run on consumer hardware, and the emergence of a vibrant ecosystem of papers and tools (e.g., DreamBooth, LoRA, ControlNets, Text‑to‑Image Adapters) that accelerate AI‑driven image creation.

Impact on 2D Art Creation

Concept art in games—whether environments for Assassin’s Creed: Mirage or character studies for Valhalla—traditionally requires extensive manual effort. Generative AI can accelerate the ideation and production phases by quickly producing high‑quality visual variants, reducing the number of iterations needed and allowing artists to focus on creative direction.

The typical pipeline consists of an Ideation Phase (driven by creative leads) followed by a Production Phase where artists generate assets. AI assists by providing rapid reference images, sketches, and variations, improving quantity, diversity, and iteration speed.

Ubisoft’s Strategic Initiatives

The team decided to fine‑tune diffusion models on the Assassin’s Creed IP, leveraging a massive internal 2D asset library (≈36 000 images) collected over years of game releases. Goals include building internal AI capability, creating high‑quality, transparent datasets, and adhering to ethical guidelines.

Data preparation involves a custom tool called CAPTION STUDIO that formats image‑caption pairs, de‑duplicates images using CLIP embeddings, and enables manual classification (concept art, renders, sketches, etc.). Automated tagging is performed with a CLIP + BLIP pipeline, followed by human review.

Fine‑tuning experiments used Stable Diffusion 1.5 and Stable Diffusion XL as base models. Early trials with ~4 500 images showed that low‑quality, automatically generated captions hurt performance, reinforcing the importance of a curated small dataset over a large noisy one.

Model selection relied on “pairwise comparison” studies where participants chose the better image from two model outputs, allowing the team to compute preference scores and pick the best model.

Sample prompts and generated images demonstrate that the fine‑tuned Assassin’s Creed model captures the series’ artistic style better than generic models, producing richer brush strokes and more coherent compositions.

Q&A Highlights

Q1: On the relevance of NeRF and emerging techniques like Gaussian Splatting for games – the answer highlighted that implicit representations may eventually aid texture and material compression but are not yet ready for full‑pipeline integration.

Q2: On using large language models to create game agents – the response noted a global trend toward LLM‑driven NPCs and interactive characters, with Ubisoft actively researching this area.

Overall, Ubisoft views AI as a tool that empowers artists rather than replaces them, aiming to speed up pipelines, improve quality, and maintain creative control while adhering to ethical standards.