3D Rendering Techniques for Huajiao Live Streaming: Layered Rendering, Avatar Customization, and Physically‑Based Lighting

As Huajiao expands its live‑streaming services, the demand for richer, three‑dimensional visual experiences has grown beyond traditional 2D rendering. To meet this need, a 3D‑based virtual avatar system was developed, focusing on eight key aspects: layered rendering, outfit and accessory swapping, bone‑based shape editing, HSV‑based color tinting, depth handling, transparent hair rendering, shadow effects, and physically‑based lighting.

1. Layered Rendering

Live streams consist of multiple rendering layers; the lower layer may contain 3D particle effects while the upper layer displays virtual avatars and gifts. Each layer operates as an independent 3D instance with its own lifecycle, requiring support for multiple windows and instances. On Android, UI layers such as SurfaceView or TextureView can host several windows, each containing multiple rendering layers (e.g., particles, avatars). This architecture enables independent lifecycle control, flexible composition, and easy capture or recording of specific layers.

2. Outfit and Accessory Swapping

Outfit swapping replaces the mesh vertices of a base skeleton (e.g., changing a traditional dress mesh0 to a denim jacket mesh1) while preserving the original bone animation. Accessories are treated as child objects that can also carry bone animation; they inherit transformations from the parent avatar and support the same operations as the avatar itself (skin, color, bone editing). This design allows extensive customization of virtual characters.

3. Bone‑Based Shape Editing ("Doll‑like" Adjustments)

Users can modify facial features, body proportions, and footwear by adjusting bone transforms. For example, converting flat shoes to high heels involves changing the mesh vertices (as in outfit swapping) and applying a bone‑offset matrix to raise the foot height and adjust the skeleton accordingly. Similar techniques are used for chest deformation.

4. HSV‑Based Color Tinting

Avatar components such as skin, eyes, clothing, and accessories can be recolored using the HSV color space. By converting RGB to HSV, adjusting only the hue while keeping saturation and value constant, and converting back to RGB, dynamic color changes are achieved without affecting material properties.

5. Depth Handling (Z‑Buffer and Z‑Fighting)

Accurate depth buffering ensures correct occlusion of opaque meshes. The article discusses common Z‑fighting issues and offers four mitigation strategies: avoid placing meshes too close together, adjust the near‑plane distance, keep the camera at an appropriate distance, and increase Z‑buffer precision while balancing performance.

6. Transparent Hair Rendering

Hair and foliage are rendered as transparent objects, which introduces ordering problems. The solution adopted uses a two‑pass approach: first render opaque geometry, then render transparent geometry without writing to the depth buffer, performing color blending. This method balances visual quality and performance on mobile devices.

7. Shadow Techniques

Both hard shadows (stencil volume) and soft shadows (shadow maps) are examined. Soft shadows are implemented with a two‑pass shadow‑map algorithm, followed by percentage‑closer filtering (PCF) to smooth edges and reduce aliasing artifacts.

8. Physically‑Based Rendering (PBR) Lighting

The system adopts a PBR pipeline, separating direct (point, directional, spot) and indirect (HDR environment) lighting. Key PBR components include the Normal Distribution Function, Fresnel equation, and Geometry function. Multiple texture maps—albedo, roughness, metallic, normal, ambient‑occlusion, and emissive—are used to achieve realistic material appearance.

Overall, the described techniques combine to deliver high‑quality, interactive 3D avatars suitable for mobile live‑streaming scenarios, requiring a solid foundation in graphics, GPU programming, and mathematical physics.