Digital Human Technology on the Soul Platform: Architecture, Key Techniques, and Application Scenarios

Background

Soul defines a social metaverse that runs in real‑time parallel to the physical world, allowing users to interact through avatars driven by AI, AR/VR, and computer‑vision technologies. The platform targets young mobile users and therefore optimizes for on‑device compute and memory constraints.

Key Technologies

1. Technical Roadmap – The digital‑human pipeline evolves from head‑only to half‑body and full‑body capture, with future integration of speech and NLP for real‑time multimodal control.

2. Recognition & Driving

Head : Face detection, 3D reconstruction, fine‑grained expression recognition (including eyebrows, eyes, mouth, and tongue). The driver combines 74 shape coefficients for facial sculpting and 56 expression coefficients, plus accessories and bone animation.

Half‑body : Hand detection → 3D keypoints → IK‑based skeletal inference, using hand and head positions to estimate shoulder and wrist joints.

Full‑body : 2D heat‑map pose estimation → depth regression → 3D skeleton, followed by mapping to a cartoon‑style avatar and applying joint limits.

3. Rendering – The N⋀W⋀ engine supports laser, translucent, leather, and real‑time shadow effects, as well as edge lighting for enhanced depth perception.

4. Design Resources – A rich library of youth‑oriented clothing, accessories, and visual assets enables rapid avatar customization.

Application Scenarios

Face‑morph & chat – Pre‑made or photo‑generated avatars with one‑to‑one or multi‑person video chat effects.

Immersive spaces – Virtual squares where users can explore, socialize, and co‑create content.

Summary & Outlook

The digital‑human solution follows a progressive roadmap from head to full‑body, integrating AI, graphics, and design pipelines in a highly self‑developed SDK. Future work will focus on multimodal (image, voice, text) driving, cross‑platform avatar asset interoperability, and deployment in AR/VR hardware.

—