How Live-Streaming Platforms Build Scalable Recommendation Systems

1. Recommendation System Architecture Design

With the growth of internet technology, information overload makes it hard for users to find content they like; recommendation systems address this by suggesting items based on users' historical behavior rather than explicit queries.

The system consists of three parts: overall architecture, user profiling, and evaluation.

Data from user actions (views, purchases, etc.) is collected, cached, and used to train models for later recommendation.

Common algorithms include popularity ranking, content‑based filtering, collaborative filtering (UserCF, ItemCF, ModelCF), and graph‑based or social‑relation methods.

Recommendation System Architecture - Ranking

Ranking algorithms are simple, easy to implement, and help with cold‑start problems, but they lack personalization and can be vulnerable to manipulation.

The ranking score is calculated as

P / (T+2)^G

, where P is votes, T is hours since publication, and G is a gravity factor that pulls items down.

Bayesian average adjusts scores based on the number of votes, moving toward the global average for low‑vote items and toward the item’s own average for many votes.

Recommendation System Architecture - Content‑Based

Content‑based recommendation matches user interest features with item features; it is easy to implement and works for new items without ratings, but it cannot discover surprising items and suffers from cold‑start for new users.

Recommendation System Architecture - Collaborative Filtering

Collaborative filtering finds users with similar behavior and recommends items liked by that group. It provides personalized, automated recommendations and can uncover hidden interests, but suffers from data sparsity, cold‑start, and limited explainability.

Recommendation System Architecture - Similarity Calculation

Similarity measures include Jaccard similarity for boolean vectors, Euclidean distance for numeric vectors, and cosine similarity for vector angles, each suited to different data types and ranging from -1 to 1.

Recommendation System Architecture - Matrix Factorization

Matrix factorization decomposes the user‑item rating matrix into low‑rank user and item latent factor matrices, enabling prediction of missing scores; it is more computationally efficient than full matrix methods but still requires model updates for new users/items.

Recommendation System Architecture – Multi‑Algorithm Fusion

After recall, results from different algorithms are merged using cross‑mixing (simple union), switch‑mixing (choose algorithm per scenario), or hierarchical mixing (prioritize stronger algorithms).

Recommendation System Architecture – Sorting

Sorting combines user behavior features with a model (e.g., LR+GBDT) to assign a probability score to each candidate, then ranks items from high to low probability for final display.

2. User Profiling System

User profiling aggregates raw logs (view, comment, gift, etc.) to build detailed user attributes such as basic demographics, activity patterns, and interest tags, which support product, operation, and push‑notification services.

3. Evaluating Recommendation Systems

Algorithmic metrics include accuracy, recall, and coverage; product‑level metrics include user retention, payment rate, and watch time. Evaluation follows offline experiments, A/B testing, and iterative tuning, supported by simulation and AB‑test platforms.