WebRTC MediaStream and RTCPeerConnection API Overview and Usage Guide

Introduction

WebRTC is a key technology for real‑time communication, enabling low‑latency audio, video, and data transmission for scenarios like remote collaboration, online education, telemedicine, and IoT. This guide focuses on API usage and important considerations.

MediaStream API

The MediaStream API consists of MediaStreamTrack and MediaStream . A MediaStreamTrack represents a single media type (audio or video) produced by a source and consumed by a sink. Multiple tracks are bundled into a MediaStream , which can contain both audio and video tracks that must stay synchronized during playback.

Each track is composed of a source that provides data and a sink that consumes it.

Source and Sink

In the browser, a source generates media resources (e.g., microphone audio, camera video, static files), while a sink consumes them (e.g., rendering via <video> tags or sending through RTCPeerConnection ). The peer connection can act as both source and sink.

Device Detection

The MediaDevices interface allows enumeration of available input and output devices via enumerateDevices() , which returns a Promise that resolves to an array of MediaDeviceInfo objects.

Capturing Local Media

MediaDevices.getUserMedia() prompts the user for permission and returns a Promise that resolves to a MediaStream containing the requested audio and/or video tracks. If permission is denied, the promise rejects with PermissionDeniedError or NotFoundError .

Video Constraints

getUserMedia() accepts a MediaStreamConstraints object to specify required tracks and optional constraints such as video and audio flags, resolution, frame rate, and device ID.

Resolution can be set using width and height (treated as “ideal”), or refined with min , max , and exact keywords.

Bitrate can be adjusted during a call by modifying RTCRtpEncodingParameters.maxBitrate (units in bps).

Compatibility Issues

Older browsers may lack support for navigator.mediaDevices.getUserMedia ; polyfills or vendor‑specific prefixes are required. Safari does not support multiple tabs using getUserMedia simultaneously, and its screen‑sharing capabilities are limited to the entire screen.

Screen Sharing

MediaDevices.getDisplayMedia() prompts the user to share a screen, window, or tab, returning a MediaStream similar to getUserMedia . Safari only allows full‑screen sharing.

Device Monitoring

The devicechange event on navigator.mediaDevices enables real‑time detection of media device plug‑and‑play changes.

Video Quality Adaptation Strategies

MAINTAIN_FRAMERATE – keep frame rate, lower resolution (video mode).

MAINTAIN_RESOLUTION – keep resolution, lower frame rate (screen sharing or document mode).

BALANCED – balance frame rate and resolution; requires enabling WebRTC-Video-BalancedDegradation .

Content hints can be set via the contentHint property of a MediaStreamTrack to guide the encoder:

Audio Content Hints

"speech" – optimize for spoken voice.

"speech-recognition" – optimize for transcription.

"music" – optimize for music playback.

Video Content Hints

"motion" – prioritize frame rate.

"detail" – prioritize resolution (default for screen sharing).

"text" – prioritize text clarity, useful with AV1 text mode.

RTCPeerConnection API

The RTCPeerConnection interface represents a WebRTC connection between a local and a remote peer, providing methods to create, maintain, monitor, and close the connection.

Configuration is supplied via an optional RTCConfiguration object, which includes:

iceServers – list of STUN/TURN servers.

iceTransportPolicy – "all", "public" (deprecated), or "relay".

rtcpMuxPolicy – "negotiate" or "require".

bundlePolicy – "balanced", "max-compat", or "max-bundle".

Common methods include addIceCandidate , addTrack , createOffer , createAnswer , setLocalDescription , setRemoteDescription , getStats , and close . Important events are ontrack , onicecandidate , ondatachannel , onsignalingstatechange , etc.

Typical Connection Workflow

Create an RTCPeerConnection instance and add local MediaStreamTrack s via addTrack .

Exchange SDP offers/answers through a signaling server to negotiate codecs, parameters, and transport.

Exchange ICE candidates using addIceCandidate on both peers.

After these steps, a basic WebRTC connection is established.