Live Broadcast Stress Testing Platform and Performance Optimization for Mobile Apps

Author : Lu Kai, Senior Test Development Engineer at Bilibili, responsible for live streaming quality assurance and innovation projects.

Background and Challenges : Bilibili's live streaming has grown rapidly, with high‑traffic interactive programs demanding ever‑greater client performance. Real‑time interactions (danmu, gifts, entrance alerts, etc.) generate thousands of messages per second, creating heavy rendering load on the client. Traditional testing methods involve long, multi‑device workflows and cannot easily simulate abnormal broadcast data, leading to low testing efficiency and difficulty reusing test assets.

Solution Overview : A platformized broadcast stress‑testing system was built, consisting of five modules: (1) Broadcast message metadata collection and management, (2) Stress‑test task configuration and scenario orchestration, (3) Pressure‑generation service, (4) Performance data collection, and (5) Reporting service.

1. Broadcast Message Metadata Collection : Metadata is gathered from live‑message listeners, a Python‑based BliveClient, and documentation. Fields can be edited to create abnormal cases, which are archived with custom tags for reuse.

2. Stress‑Test Task Configuration : Scenarios are defined by mapping the CMD field of messages to queue IDs. Multiple cases can be placed in each queue, allowing flexible QPS configuration. A composition layer enables reuse of over 350 configurations and 1000+ cases.

3. Pressure‑Generation Service : A custom service runs a 1‑second loop per queue, pulls a random batch of cases according to the configured QPS, and forwards them via the broadcast‑proxy to clients. The current implementation has coarse QPS accuracy; future work includes integrating tools like JMeter or Locust.

4. Performance Data Collection : The Perfdog tool is used on both iOS and Android to capture comprehensive performance metrics, with custom report upload support.

5. Reporting Service : Collected data are uploaded, archived, and visualized to provide baseline statistics, detect performance regressions, and support scenario comparisons.

Practical Experience : For the 2020 New Year live event, performance goals were set for high‑end and low‑end devices (2‑hour smooth playback without crashes). Device models were tiered, and test cases were tuned (higher QPS for danmu, additional gift combos) to mimic real traffic. Memory and FPS analysis revealed leaks and excessive thread switches; optimizations included merging GCD queues, disabling danmu logging on low‑end devices, adjusting socket rates, and avoiding ViewModel references that delayed release.

Log Replay Optimization : An initial replay algorithm accumulated timing errors (up to minutes). A new algorithm corrects each packet’s timestamp based on the offset from the first packet, limiting per‑packet error to <10 ms regardless of total volume.

Results : The platform has accumulated >500 test cases, >300 configurations, and executed >4000 stress‑test tasks. It uncovered >100 critical performance bugs and >450 memory leaks, ensuring stable operation for major events such as the “Most Beautiful Night” New Year’s gala. Performance metrics on both iOS and Android improved by several folds under high‑load broadcast scenarios.

Future work will extend these practices to web clients, which still carry a large share of live‑stream traffic.