Overview of Toutiao's Recommendation System: Architecture, Content Analysis, User Tagging, Evaluation, and Content Safety

The presentation introduces Toutiao's recommendation system, outlining its overall architecture and the principles behind content analysis, user tagging, evaluation, and content safety.

1. System Overview

The recommendation model fits a function y = F(X_content, X_user, X_context) that predicts a user's satisfaction with a piece of content. The three dimensions are:

Content : diverse media types (articles, videos, UGC short videos, Q&A, micro‑posts) each with specific features that must be extracted.

User : explicit interests, demographics (age, gender, occupation) and implicit interests derived from behavior models.

Context : environment features such as location, time, and device, reflecting the mobile‑first usage patterns.

The model outputs an estimated suitability score for a given user‑content‑context tuple. Besides quantifiable goals (CTR, dwell time, likes, comments, shares), the system also incorporates non‑metric objectives like ad frequency control, special‑content handling, and content‑ecosystem interventions (e.g., low‑quality suppression, top‑news weighting).

Feature engineering covers four major categories:

Relevance features – keyword, category, source, and topic matching.

Environment features – geographic and temporal bias.

Popularity features – global, category, topic, and keyword hotness.

Collaborative features – similarity across users (click, interest, topic, vector similarity) to mitigate recommendation narrowing.

Training is performed in near‑real‑time using a Storm cluster. User actions are streamed into Kafka, consumed by Storm workers, and fed back to a custom high‑performance parameter server for online model updates. The pipeline processes billions of raw features and vectors, with latency dominated by user‑action feedback.

Recall is handled by an inverted‑index based strategy that must return candidate items within 50 ms. Offline‑built indexes (by category, topic, entity, source) are filtered online using user interest tags, freshness, and hotness scores.

2. Content Analysis

Content analysis encompasses text, image, and video processing, with a focus on textual features for interest modeling. Explicit semantic tags are manually defined, while implicit features include topics and keywords derived from probabilistic models. Similarity detection, spatio‑temporal relevance, and quality assessment (e.g., porn, vulgarity, click‑bait) are also critical.

3. User Tagging

User tags include interest categories, topics, keywords, source, clustered interest groups, and demographic attributes (gender, age, location). Early versions computed tags in batch on Hadoop; later, a Storm‑based streaming system updates tags in near‑real‑time, reducing CPU usage by ~80 % and supporting tens of millions of daily updates. Some stable attributes (gender, age, residence) remain on daily batch pipelines.

4. Evaluation and Experimentation

Effective evaluation requires a multi‑metric framework beyond simple CTR or dwell time. Toutiao employs a robust A/B testing platform that partitions users into buckets, assigns traffic, and collects real‑time action logs. The system automatically generates statistical confidence, comparative results, and optimization suggestions, while acknowledging that some user‑experience aspects still need manual analysis.

5. Content Safety

Content safety is a top priority. Dedicated moderation teams and AI models (deep‑learning based porn, profanity, and low‑quality detectors) filter both PGC and UGC streams. Models favor high recall (e.g., >95 % for profanity) with acceptable precision, and suspicious items undergo secondary human review. Ongoing research with academic partners targets rumor detection and further quality improvements.

Overall, the talk shares practical insights into building and operating a large‑scale, real‑time recommendation system that balances algorithmic performance, engineering efficiency, and social responsibility.