Design and Implementation of a High‑Throughput WeChat Red‑Envelope System Simulating 10 Billion Requests

The author revisits a 2015 article about building a reliable "Spring Festival Gala" red‑envelope system and creates a personal prototype to simulate 10 billion shake‑red‑envelope requests using a single server.

Background knowledge explains QPS, PPS, the shake‑red‑envelope request flow, and the red‑envelope distribution process.

Goals include supporting one million concurrent users, achieving a peak of 60 000 QPS per machine, and handling both shake and send red‑envelope operations at specified rates (e.g., 83 shakes / s, 200 sends / s).

Hardware and software used are Golang 1.8r3, shell, Python, Ubuntu 12.04 on a Dell R2950 (8‑core, 16 GB RAM) for the server and Debian 5.0 VMs (4 cores, 5 GB RAM) for 17 client machines, each establishing 60 000 connections.

Implementation details cover:

Achieving one million connections by partitioning them into multiple independent sets (SETs) to limit goroutine count.

Client‑side QPS coordination using NTP‑synchronised timestamps and a modulo‑based algorithm to evenly distribute request bursts.

Server‑side request counting, network monitoring via a Python‑ethtool script, and simple lock‑free bucket design to reduce contention.

Monitoring integration using a custom module from the author’s fakewechat project.

Key code snippet for counting established connections:

Alias ss2=Ss –ant | grep 1025 | grep EST | awk –F: "{print $8}" | sort | uniq –c

Practice phases :

Start server and monitoring, launch 17 client VMs to create one million connections, verify with the above command.

Increase client QPS to 30 000, observe stable network and server metrics, and run a red‑envelope generator at 200 / s.

Raise client QPS to 60 000, repeat the generation and sending tests, noting increased variance in QPS and network packet loss.

Data analysis shows three QPS regions (≈30 k, ≈60 k, and peak) with expected fluctuations due to goroutine scheduling, network latency, and occasional packet loss; server‑side QPS mirrors client behaviour, and GC pauses stay under 10 ms on the legacy hardware.

Conclusion confirms that the prototype meets the design targets—supporting one million users and 30 k–60 k QPS per machine—demonstrating that a 600‑node deployment could process 10 billion shake‑red‑envelope requests in roughly seven minutes, while also highlighting areas for further optimisation.