Technical Overview of Low‑Latency Interactive Live Streaming and MLVBLiveRoom Solutions

On June 29, the Audio‑Video and Fusion Communication Technology Salon concluded, featuring Tencent Cloud experts who presented the latest low‑latency technologies and new commercial live‑streaming solutions.

Speaker Introduction: Jiang Lei, Senior Engineer at Tencent Cloud, works in the Terminal R&D Center and is responsible for the client SDK of Tencent Cloud Video Service. He has previously worked at NetEase and Alibaba Cloud on real‑time audio‑video, live streaming, VOD, CDN, and IM technologies.

Live streaming generally falls into two scenarios: ordinary streaming (one host, many viewers, typically using RTMP and CDN distribution) and multi‑host ("link‑mic") streaming, where two or more hosts interact, often categorized as a main host and sub‑hosts. This multi‑host model is common in entertainment, education, and e‑commerce.

Four key challenges in link‑mic live streaming:

Latency – delay between speakers degrades interaction.

Echo – acoustic feedback caused by audio looping.

Mixing – combining multiple video/audio streams for a single audience.

Room management – handling user states, host‑sub‑host coordination, and business logic.

Latency sources and mitigation:

1. Transcoding adds hundreds of milliseconds.

2. CDN introduces 1‑3 seconds due to GOP‑based cache fetching.

3. Player jitter buffer (typically 500‑1000 ms) adds further delay; for link‑mic we reduce it to ~200 ms but must correct accumulated jitter.

Solution: bypass CDN for host‑to‑host communication using RTMP‑ACC nodes with UDP acceleration, achieving sub‑500 ms latency between main and sub hosts.

Echo cancellation (AEC): Acoustic echo is eliminated by comparing playback audio with microphone capture and applying inverse waveforms. Tencent Cloud integrates AEC directly into LiteAVSDK, requiring only a parameter toggle.

Video mixing: On the client side, each host renders both local preview and remote streams, requiring a multi‑instance player. On the cloud side, a mixing module combines up to 16 input streams (audio, video, canvas, images) before CDN distribution, greatly reducing client load.

Beyond these, developers must address device compatibility, codec tuning, network QoS, player optimization, host‑mic management, message handling, and cost‑effectiveness.

MLVBLiveRoom Architecture: Users push RTMP streams to accelerated nodes; the nodes replace TCP with UDP (RTMP over UDP) to keep host‑to‑host latency below 500 ms. After acceleration, streams are sent to a cloud mixing service, mixed, and then delivered via standard CDN to ordinary viewers. This adds minimal cost while providing high‑quality, low‑latency link‑mic.

Room management is provided as a cloud service, handling user entry/exit, host‑mic switching, and optional AI face‑effect features. An open‑source demo app ("Xiao Zhibo") is available for developers to customize.

Monitoring & Metrics: LiteAVSDK exposes onNetStatus callbacks with low‑level network data (jitter, packet loss, bitrate). Tencent Cloud console aggregates these metrics and offers a scoring system for stream health.

Remaining challenge: switching a regular viewer from CDN (second‑level latency) to UDP (sub‑second latency) introduces a latency gap. Tencent’s TRTC low‑latency large‑room solution places all participants in a UDP‑based room, supporting up to 100 k users, with host‑to‑host latency ~100 ms and viewer latency < 1 s, enabling seamless mic‑on/off.

Q&A:

Q: How many users can TRTC support for interactive sessions? A: Upstream is limited to 10 users by default (can be increased via ticket); downstream is recommended under 10 k users, but no hard limit.

Q: What is the reliability of converting RTMP to UDP? A: The UDP acceleration is built on QUIC, which provides TCP‑like reliability. It has been running in production for years with stable performance.