Optimizing First-Frame Latency in Mobile Live Streaming Using ijkplayer and FFmpeg

The live‑streaming market is becoming increasingly competitive, and the first‑screen (first‑frame) latency is a critical user‑experience metric. This article details a comprehensive optimization of Meipai live‑streaming that reduced the first‑screen time by more than 50% to around 500 ms.

Key factors affecting first‑screen time

Loading of UI elements (avatars, gift lists, etc.) after entering a live room.

Mobile network bandwidth limitations (typically ~1 Mbps).

Live‑player pull‑stream speed and buffering strategy.

CDN cache availability and policies.

Streaming protocol (RTMP vs. HTTP‑FLV). HTTP‑FLV reduces protocol handshake overhead by ~300‑400 ms.

The total latency can be broken down into five components: DNS resolution, TCP connection, HTTP response, audio‑video stream probing, and buffer waiting.

1. DNS Resolution

DNS lookup is performed via getaddrinfo in FFmpeg. Without cache, a lookup takes ≥300 ms; with cache it is a few milliseconds.

hints.ai_family = AF_UNSPEC;

Using AF_UNSPEC triggers both A and AAAA queries, which adds unnecessary delay when only IPv4 is needed. Switching to AF_INET can cut the lookup to <100 ms.

Statistics are collected in tcp.c :

int64_t start = av_gettime();
if (!hostname[0])
    ret = getaddrinfo(NULL, portstr, &hints, &ai);
else
    ret = getaddrinfo(hostname, portstr, &hints, &ai);
int64_t end = av_gettime();

Optimization: combine HTTPDNS with a local DNS SDK that pre‑resolves and caches domain IPs, and fall back to HTTPDNS when the cached IP is slow or blocked.

2. TCP Connection

TCP connect time (three‑way handshake) is usually <50 ms on Wi‑Fi and offers little room for improvement.

Statistics (FFmpeg tcp.c ):

int64_t start = av_gettime();
if ((ret = ff_listen_connect(fd, cur_ai->ai_addr, cur_ai->ai_addrlen,
    s->open_timeout / 1000, h, !!cur_ai->ai_next)) < 0) {
    if (ret == AVERROR_EXIT)
        goto fail1;
    else
        goto fail;
}
int64_t end = av_gettime();

Optimization focuses on CDN node selection and HTTPDNS‑driven DNS results to route users to faster edge nodes.

3. HTTP Response

HTTP response time is the interval between sending the request and receiving the response header. Cached CDN responses are typically ≤100 ms; uncached (cold) streams can exceed 400 ms.

Statistics (FFmpeg http.c ):

int64_t start = av_gettime();
ret = http_open_cnx(h, options);
int64_t end = av_gettime();

Optimization: improve CDN cache hit rate and use multi‑CDN strategies.

4. Audio‑Video Stream Probing

Implemented by avformat_find_stream_info . By default FFmpeg reads at least 20 frames, causing ~1 s delay. Reducing the probe size speeds up first‑frame rendering.

av_dict_set_int(&ffp->format_opts, "fpsprobesize", 0, 0);

Result: probe time drops to <100 ms.

5. Buffering

Buffering waits until a configurable number of audio/video frames are queued. Two key parameters:

BUFFERING_CHECK_PER_MILLISECONDS – interval for buffer‑check callbacks.

MIN_MIN_FRAMES – minimum frames required before playback starts.

Original code (FFmpeg ff_ffplay.c ):

#define BUFFERING_CHECK_PER_MILLISECONDS     (300)
if (ffp->packet_buffering) {
    io_tick_counter = SDL_GetTickHR();
    if (abs((int)(io_tick_counter - prev_io_tick_counter)) > BUFFERING_CHECK_PER_MILLISECONDS){
        prev_io_tick_counter = io_tick_counter;
        ffp_check_buffering_l(ffp);
    }
}

Changing the interval to 50 ms reduces buffering latency by ~150‑200 ms.

Minimum frame settings:

#define MIN_MIN_FRAMES      10
if (is->buffer_indicator_queue && is->buffer_indicator_queue->nb_packets > 0) {
    if ((is->audioq.nb_packets > MIN_MIN_FRAMES || is->audio_stream < 0 || is->audioq.abort_request)
        && (is->videoq.nb_packets > MIN_MIN_FRAMES || is->video_stream < 0 || is->videoq.abort_request)) {
        printf("ffp_check_buffering_l buffering end \n");
        ffp_toggle_buffering(ffp, 0);
    }
}

Reducing MIN_MIN_FRAMES from 10 to 5 cuts first‑screen time by ~300 ms with only a slight increase in stutter.

CDN Edge Optimizations

GOP‑based caching and fast‑start techniques deliver enough initial frames quickly, further shaving ~100 ms off latency.

Results

Combined optimizations bring the first‑screen latency from >1 s to ~500 ms, a reduction of >50 %. Most requests now fall within the 0‑500 ms and 500‑1000 ms ranges, achieving near‑instant playback for popular streams.

Conclusion

Accurate instrumentation of each latency component is essential for targeted improvements. Ongoing work will address stability, stall rate, and stall duration.