Mastering WebRTC QoS: NACK, FEC, SVC, and More Explained

WebRTC offers a rich set of QoS strategies to improve real‑time audio‑video transmission, but its large, fast‑evolving codebase makes learning challenging. This guide summarizes the core ideas and algorithmic implementations of those strategies.

Overview

Key QoS techniques in WebRTC include NACK, FEC, SVC, JitterBuffer, IDR Request, Pacer, Sender‑Side BWE/REMB, Probe, VFR (dynamic frame‑rate), AVSync, and dynamic resolution adjustment. The diagram below shows their placement within the WebRTC architecture.

1. NACK

NACK (Negative Acknowledgement) works opposite to ACK: when a packet is missing, the receiver sends a NACK to request retransmission. The sender uses the sequence number in the NACK to locate the lost packet in its buffer and resend it, as defined in RFC 5104.

2. FEC

Forward Error Correction (FEC) packs previous packets into new ones so that lost data can be recovered from redundancy.

WebRTC implements FEC in three ways:

RED : RFC 2198 redundancy, where previous packets are inserted into the new packet.

ULPFEC : Uses the VPX encoder's SVC temporal layer Level 0 frames as FEC payload.

FLEXFEC : Extends ULPFEC with additional XOR operations for stronger loss protection.

Examples of sender‑side packaging, network loss, and recovery are illustrated below:

3. SVC

Scalable Video Coding (SVC) extends H.264/MPEG‑4 AVC to provide temporal, spatial, and quality scalability. While spatial and quality scalability increase bitrate and complexity, temporal scalability is widely used in unstable networks.

WebRTC’s VPX encoder employs temporal scalability, protecting the base layer (T0) with FEC + NACK, while higher layers can be dropped to maintain smooth playback.

4. JitterBuffer

JitterBuffer stores incoming RTP packets and reorders them based on timestamps or sequence numbers, smoothing out packet loss and jitter. It can operate in static mode (fixed size) or dynamic mode (size adapts to network delay).

5. IDR Request

Instantaneous Decoder Refresh (IDR) frames are keyframes that can be decoded without reference to previous frames. When packet loss is severe, a receiver can request an IDR frame via RTCP FIR, RTCP PLI, or SIP INFO.

6. Pacer

Pacer smooths packet transmission by spacing out RTP packets belonging to the same video frame, preventing bursty traffic that could cause congestion.

7. Sender‑Side BWE / REMB

The sender adjusts its bitrate based on feedback from the receiver (loss rate, delay). When over‑use is detected, bitrate is reduced; when under‑use is detected, bitrate is increased. The receiver conveys the target bitrate via RTCP REMB messages.

8. Probe

During startup, WebRTC must quickly estimate an appropriate sending bitrate. The GCC algorithm adapts bitrate slowly upward, so a probing mechanism is used to rapidly discover the network’s capacity.

9. Dynamic Frame‑Rate Adjustment

Based on the sender‑side bandwidth estimate, the encoder can lower the frame rate when network conditions degrade, preserving video smoothness without excessive bitrate.

10. AVSync

Audio and video streams travel separate RTP paths, leading to potential desynchronization. AVSync aligns playback by using RTCP NTP timestamps and buffering each stream to match capture and render times.

11. Dynamic Resolution Adjustment

When network quality drops or CPU usage rises, the encoder reduces resolution to maintain real‑time performance; it increases resolution again when conditions improve.

12. Adaptive QoS Selection (HARQ)

WebRTC dynamically selects QoS strategies (NACK, FEC, SVC) based on current network latency and loss characteristics, using NACK alone in low‑delay, low‑loss scenarios and combining multiple techniques in harsher conditions.