Key Technologies of Immersive Media in the 5G Era: 3D, Point Cloud, and Compression

Currently, the majority of network traffic—especially on mobile terminals—is occupied by video. With the arrival of 5G, video traffic on smartphones is expected to reach 74% or higher by 2024, further accelerating the development of multimedia technologies.

1. 5G Meets Multimedia The article explains how 5G will boost multimedia innovation, citing the rapid growth of video consumption and the increasing human demand for richer information.

2. Tencent Multimedia Lab’s Participation in Standards and Industry Systems The lab contributes to media, network, system, and other industry standards, turning these standards into concrete products across various technical modules.

3. Overview of Immersive Media Immersive media includes acquisition, compression, transmission, rendering, and interaction. It enables more realistic 3D scene perception and interaction, enhancing user experience.

4. Important Technologies of Immersive Media

4.1 3D Representation 3D content can be projected onto a sphere with the viewer at the center, allowing free rotation to view different angles. Compression leverages existing 2D video codecs by projecting 3D data onto 2D planes (ERP) before applying standard video compression.

4.2 3D Data Acquisition (VR360) Using dual fisheye lenses, 360° coverage is achieved by stitching two 180° images after projecting them onto ERP and aligning them via translation‑rotation matrices.

The lab reduces ERP redundancy by about 25% through optimized projection, improving transmission efficiency.

4.3 Media System Implementation Traditional multimedia containers are extended for immersive media (e.g., OMAF) to describe projection type, initial view angles, etc. HEVC’s tile‑based processing enables chunked transmission, allowing users to download only visible regions, saving up to 50% bandwidth.

5. Point Cloud Technology Point clouds represent 3D objects as collections of points (x, y, z, r, g, b, etc.). Compression methods include V‑PCC (video‑based projection) and G‑PCC (geometry‑based octree coding). V‑PCC can compress 100k points at 30 fps from 360 MB to ~1 MB; G‑PCC can reduce 110 MB to 24 MB for 10 fps data.

The lab contributes to international standards (V‑PCC, G‑PCC) and leads Chinese national point‑cloud standardization efforts.

6. Applications Point‑cloud compression is applied in autonomous driving, reducing storage to 1/6 of the original. It also supports broadcast viewpoints that combine 360° video for distant scenes and point clouds for close‑range detail.

7. Future Outlook With 5G infrastructure, immersive media (VR/AR, XR, cloud gaming) will see rapid growth across sports, tourism, entertainment, education, news, and commerce, delivering new user experiences.