How Virtual Digital Humans Are Shaping the Future of Entertainment and Tech

Virtual characters (also called digital humans) are a broad concept encompassing various virtual personas across industries. The term traces back to the 1989 Visible Human Project in the US and was introduced in China in 2001 during the Xiangshan Science Conference.

The definition includes three essential elements:

Form – a human‑like appearance with specific facial features.

Movement – behaviors expressed through language, facial expressions, and gestures.

Spirit – cognitive abilities to perceive the environment and interact with people.

These elements form a progressive hierarchy for character completeness.

Industry Status

Market – Virtual characters have penetrated e‑commerce, finance, film, gaming, and other sectors. For example, China’s virtual idol market reached ¥3.46 billion in 2020 and is projected to hit ¥6.22 billion in 2021.

Historical milestones include Japan’s first virtual singer in 1962 and the globally renowned Hatsune Miku, created by Crypton. Advances in CG, motion capture, and AI have reduced production costs and improved realism, driving rapid industry growth.

Industry Chain

The chain is divided into three layers:

Foundation layer – hardware (displays, optics, sensors, chips) and software (modeling tools, rendering engines).

Platform layer – integrated hardware‑software systems, production service platforms, and AI capability platforms.

Application layer – domain‑specific solutions that embed virtual characters into various business scenarios.

Representative Products

Only a few representative examples are listed.

IP‑operated

Samsung Neon – AI‑driven digital human with realistic facial expressions.

Lil Miquela – CGI character used for fashion marketing.

Hatsune Miku – Popular virtual singer with open‑source voice synthesis tools.

CodeMiko – Virtual YouTuber.

Luotianyi – Shanghai HENIAN’s virtual singer.

Deer Cry – MiHoYo’s virtual “girlfriend” avatar.

Solution‑oriented

VOCALOID – Yamaha’s singing synthesis technology.

MetaHuman – Unreal Engine’s high‑fidelity digital human creation tool.

Tencent IP Virtual Human, AvatarX, NetEase Youling – End‑to‑end services covering platform, cloud AI, and data layers for virtual anchors, teachers, assistants, and more.

Game‑oriented

Zepeto, Shine Shining Warm Heart – Popular avatar‑customization games.

What We Are Doing

In early 2023, Alibaba’s Front‑End Committee launched a virtual‑character group to explore technologies and applications across games, video, and live streaming. Five teams contributed works that align with industry hotspots.

Game

Virtual characters are standard in games, especially “avatar‑customization” titles where players design their own characters.

Taobao Life

A lightweight mobile game inside the Taobao app offering facial modeling, dressing, beauty filters, and photo features, with cross‑game avatar sharing.

Yang Koala

An interactive pet‑raising mini‑app in the Kaola shopping app, built with Web technologies and featuring dressing and feeding mechanics.

Video

Short videos with virtual characters provide consistent user experiences and generate incremental revenue. Production leverages motion capture, facial recognition, and AI‑generated content to achieve lifelike behavior.

Live Streaming

Virtual characters in live streams are still experimental, serving as assistants, co‑hosts, or AI‑driven sales agents across platforms like Taobao Live, Youku Live, and Ant Financial Live.

Our Technical Research

Research spans engineering and algorithmic solutions, tailored to each scenario.

Art Production

Technology assists both 2D and 3D pipelines. In 3D, custom tools and plugins integrate with various engines to streamline workflows and maintain quality.

Expression and Motion

Realistic facial and body motion is achieved using Blendshape for expressions and physics‑based bone rigs (including collision boxes) for motion; over 50 blendshapes enable a wide range of expressions.

Rendering and Style

Key rendering considerations include:

Engines – Unity, self‑developed EVA Figure (Web, based on Hilo3D), Oasis3D (Web), ApisXEngine (cross‑platform).

Materials – PBR workflow; experiments with ultra‑realistic styles, e.g., the sign‑language assistant “Xiao Mo”.

Cloud Rendering – Unity‑based rendering with live streaming; collaboration with Alibaba Cloud and Node teams to explore Web‑engine + Puppeteer for non‑real‑time cloud rendering.

Scene Orchestration

A “director system” enables flexible scene composition for live streams, games, and video scripts, allowing timed actions, product introductions, and template‑based video generation.

Big Data and Algorithms

Long‑term development relies on big‑data‑driven algorithms to generate diverse expressions and motions, using training data from video analysis and key‑frame extraction to overcome engineering capacity limits.

D2 Preview

We will present a comprehensive talk titled “The Birth of Virtual Idols – Industry and Technology Exploration” at the D2 Front‑End Technology Forum on December 18‑19.