Adaptive Degradation and Recovery for JD Alliance Recommendation System under High‑Frequency Traffic Spikes

During JD's 618 promotion, the JD Alliance marketing platform experiences explosive traffic growth that stresses the recommendation system, especially because of numerous marketing activities, varying campaign intensities, hundreds of external traffic sources, and flash‑sale style red‑packet events that cause traffic to rise ninefold within seconds and then drop sharply.

Challenges include difficulty in accurately forecasting traffic, heterogeneous recommendation strategies across scenarios, and the need for second‑level response to massive, instantaneous traffic fluctuations, which can lead to system crashes if resources are insufficient.

Existing solutions such as simple rate limiting, pre‑written degradation plans, and automatic scaling are insufficient for recommendation workloads because they either sacrifice personalization, are coarse‑grained, or depend on upstream services that cannot scale within the required sub‑second window.

Redefined problem – the system must gain an "adaptive" capability that can differentiate control per scenario, operate fully automatically, sense traffic changes in real time, recover smoothly after peaks, and minimize recommendation quality loss.

Proposed adaptive solution consists of five key abilities:

Scenario‑aware identification and tiered handling, allowing critical paths to receive higher priority.

Fully automated degradation and recovery driven by intelligent monitoring and decision logic.

Real‑time traffic monitoring with dynamic adjustment of degradation policies.

Automatic restoration to full recommendation once traffic subsides.

Precise degradation that preserves recommendation relevance for high‑value users.

Implementation details :

Real‑time performance perception : configure per‑scenario timeout thresholds and run guardian coroutines on each recommendation instance to collect per‑second response times and timeout rates.

Wilson confidence interval correction : apply the Wilson formula (z = 1.96 for 95% confidence) to adjust the observed timeout rate, reducing statistical error during low‑traffic periods.

Scenario‑differentiated control : collect latency per scenario and enforce the configured thresholds.

Fine‑grained traffic slicing : only a portion of traffic is marked as "degraded" based on current timeout ratios; user segmentation (e.g., KMFP tags) determines degradation intensity.

Dynamic linear programming of the recommendation chain : model each recall path, coarse‑ranking, fine‑ranking, and re‑ranking module with its contribution (E) and latency (T); solve a linear program that maximizes business benefit under latency constraints, yielding the optimal set of active modules (binary variables W).

Real‑time pipeline orchestration : generate the call‑graph pipeline from the optimal W set and schedule execution dynamically.

Small‑traffic probing and stepwise recovery : periodically test a tiny slice of degraded traffic; if the probe succeeds, gradually expand the restored traffic until full recommendation resumes.

Business‑agnostic API : expose generic interfaces for profit and latency inputs, timeout configuration, and degradation flags, enabling low‑cost migration to other services.

Combined with existing rate‑limiting and serverless auto‑scaling, the adaptive module enables the JD Alliance recommendation system to maintain stability and recommendation quality during massive traffic bursts.

Results during the promotion:

Traffic loss reduced by over 90% compared with traditional manual degradation.

System achieved second‑level adaptive degradation and minute‑level automatic recovery.

No reliance on upstream peak‑traffic estimates.

Multiple traffic spikes were handled without upstream degradation protection.

Zero manual interventions and zero incidents.

The solution demonstrates a scalable, low‑cost approach to safeguarding recommendation effectiveness under extreme, unpredictable traffic conditions.