Real-Time Audio/Video in Mini-Program Interactive Education: Challenges and Solutions

Author Liu Lianxiang, founder of Yi Qi Wan Xue Technology, has been researching WebRTC since 2013 and focuses on real‑time audio/video applications in online education and healthcare.

The talk introduces the use of mini‑program real‑time audio/video for interactive education, outlining common problems faced when building such a product:

No camera on some PCs, requiring users to purchase external devices.

Cluttered background that discourages video usage.

Diverse terminal devices (iPad, smartphones, various PCs) causing adaptation challenges.

Device debugging and remote support burdens for C‑end users.

Multiple teaching‑management systems that are hard to integrate.

Network latency and video stutter, with no perfect solution yet.

Key technical questions include audio latency (smooth) versus video lag (stutter).

Solution overview for mini‑program audio/video:

Zero‑knowledge rapid integration via JS API components, no deep audio/video expertise required.

High‑quality audio/video: end‑to‑end latency < 400 ms, network jitter < 800 ms, 30 % packet loss tolerance.

Cross‑platform interoperability: the same SDK works on mini‑programs, Android, iOS, and Web.

Advantages of the unified WeChat OS platform:

Unified account system : users already have avatars and complete profiles.

Standardized interfaces : simple “up” and “down” APIs.

Minimalist API .

Hardware‑level adaptation : Tencent handles diverse chipsets (Qualcomm, Samsung, Intel) for Android.

Noise reduction and echo cancellation capabilities.

Consistent operation flow across all mini‑programs.

The ecosystem supports four educational stages—teaching, practice, assessment, and management—through integrated tools such as recorded/live courses, homework, quizzes, and group communication.

Technical stack details:

Framework: Vue.js + WePY for front‑end, Socket.IO for signaling, TypeScript for both front‑ and back‑end, Redis for real‑time data storage.

Audio/video components: pusher, player, room management, 1‑to‑6 interaction (one teacher, up to six students).

UI constraints: native video component limits complex overlays; simple layouts are recommended.

Whiteboard support: PPT/Doc/PDF converted to images via OpenOffice headless mode; real‑time annotation synchronization.

Cloud recording: RTMP‑based, optional paid service.

Content moderation: screenshot-based anti‑pornography is considered necessary.

Mixing & recording: server‑side video mixing supported; custom solutions for PPT‑audio‑video merging using ffmpeg/moviepy.

Browser‑side mixing and screen recording as a supplement to server mixing.

Identified limitations:

Native video component cannot handle complex interactions or overlays.

Category restrictions (only four categories currently open).

No pad support; large‑screen use requires PC alternatives.

Lack of emulator debugging; remote debugging is painful.

File upload limits imposed by the OS, not the mini‑program.

Q&A highlights:

High‑concurrency video streaming (hundreds of thousands of concurrent users) is challenging; typical large platforms handle tens of thousands.

Dependency on Tencent Cloud may increase costs at scale; strategic product differentiation and eventual migration are advised.

Technical differentiation is essential despite low barriers; leveraging unique features and strong service integration can create competitive advantage.

Promotional notice: Tencent Cloud Community invites submissions, technical articles, and participation in offline events. Contact via WeChat (andyhu) for contributions and group joining.