Asynchronous Architecture Optimization for High‑Concurrency Video Playback: Thread Pool, In‑Memory Queue, MQ, and Agent + MQ Solutions

The author, who recently completed a Spring Cloud Alibaba video tutorial covering middleware, OAuth2, social logins, gray release, and distributed transactions, uses a real‑world incident to discuss why asynchronous processing is crucial in high‑concurrency environments.

1. Business Scenario

In an educational platform, a teacher logs in, views a list of courses, and selects a video. The system performs two core actions: (1) reading video metadata from Redis and returning it to the front‑end, and (2) recording the teacher’s watch behavior every three seconds by inserting records into a MySQL table. As concurrent users increase, the write frequency grows exponentially, leading to thread blockage on DAO methods and degraded response times.

Typical scaling ideas—optimizing SQL, upgrading hardware, or sharding—are either costly or insufficient because occasional write delays and minor data loss are acceptable.

The optimization goal therefore becomes “reduce write latency and increase write concurrency.”

2. Thread‑Pool Mode

Drawing from the author’s experience at a travel website, write requests are handed off to a dedicated thread pool, allowing the controller to return immediately while the pool processes the activation logic asynchronously. This simple approach stabilized the system and eliminated timeout issues.

3. Local Memory + Scheduled Task

Inspired by an open‑source solution that increments view counters in memory, the proposal stores watch‑record data in a LinkedBlockingQueue in the controller, then a background thread every minute batches the queued items and writes them to MySQL using Jdbc batchUpdate . This improves throughput without altering existing business logic, though memory‑overflow risks must be managed.

4. MQ Mode

Message queues provide asynchronous decoupling. The flow is: controller converts the watch record into a message, sends it to the MQ, immediately acknowledges the front‑end, and a consumer service pulls messages from the queue to perform batch database writes. MQs offer high availability, persistence, and batch consumption, but introduce additional components and complexity.

5. Agent Service + MQ Mode

A separate agent process watches a directory for files written by the business service (e.g., JSON logs). The agent reads the files, pushes their contents to the MQ, and a consumer persists the data to MySQL. This architecture keeps the business service lightweight, avoids embedding MQ libraries, and is used in many performance‑monitoring and log‑analysis platforms.

6. Summary

The article outlines three layers of asynchronous thinking: (1) identifying scenarios where massive writes strain resources, (2) understanding the four async patterns—thread pool, local memory + scheduled task, MQ, and agent + MQ—and their shared principle of queuing write commands to respond instantly, and (3) recognizing async as a fine‑grained resource‑usage strategy that must be bounded to avoid over‑consumption.