How Simulcast Boosts WebRTC Video Quality and Scales Large Conferences

What is Simulcast?

Simulcast is a standardized technique in WebRTC that allows a client to send multiple versions of the same video, each encoded at different resolution and frame rate. Receivers with higher bandwidth can get higher‑quality streams, while low‑bandwidth receivers get lower‑quality streams, ensuring smooth experience for all participants.

Why use Simulcast?

In poor network conditions, participants with limited bandwidth can degrade overall conference quality, causing video quality drop, network congestion, and a “key‑frame amplification” effect where lost packets trigger frequent key‑frame requests, increasing bandwidth demand.

Common solutions

Transcoding : Generates alternate streams on the server, but adds heavy CPU load and cost at scale.

SVC : Single stream with multiple bitrates, but requires decoder support that many devices lack.

Simulcast : Client encodes multiple layers and sends them to the SFU, which forwards the appropriate layer. It imposes little server load but adds some client encoding and uplink cost.

SFU support for Simulcast

The SFU receives separate RTP streams for each layer, each with its own SSRC. To switch layers seamlessly, the SFU rewrites SSRC, RTP sequence numbers, and timestamps so that the receiver sees a continuous stream.

RTP header rewriting

SSRC: Unified across layers when forwarding.

Sequence number: Adjusted to maintain continuity across layers.

Timestamp: Aligned to a common clock base.

The SFU also buffers the mapping between original and rewritten packets to support retransmission.

Congestion detection and bandwidth estimation

The SFU uses TWCC feedback to monitor packet delay and loss, defining five congestion states (no congestion, pre‑congestion, congestion, pre‑congestion recovery, congestion recovery) and transitions based on signals and timeouts.

State‑machine logic determines when to downgrade or upgrade video layers.

Automated layer orchestration

Based on congestion state, the SFU dynamically allocates video layers, prioritizing screen sharing and the presenter, then distributing remaining bandwidth fairly.

Three allocation strategies are described:

Unified downgrade in congestion : Quickly drop to lower layers, then gradually restore.

Periodic upgrade attempts : Try to raise one layer at a time when bandwidth permits.

Bandwidth redistribution for new subscriptions : Reallocate bandwidth from lower‑priority tracks to accommodate new tracks, or pause streams if insufficient.

Client‑side optimizations

Uplink (publisher) side

The SFU can signal the publisher to enable or pause encoding of specific layers based on subscription changes, saving uplink bandwidth.

Downlink (subscriber) side

Clients report the size of video elements they are rendering; the SFU can stop sending high‑resolution layers for invisible tracks, reducing downlink bandwidth.

In weak networks, the client can increase the jitter buffer target (up to 4000 ms) to smooth playback.