Technical Architecture Overview of Toutiao: Data Processing, User Modeling, and Recommendation System

Toutiao, founded in March 2012, grew from a handful of engineers to over 200 staff within four years, expanding product lines from jokes to news, movies, and e‑commerce.

The platform now serves over 500 million registered users, with daily active users reaching 48 million and daily page views exceeding 5 billion, handling massive article and video traffic.

Content acquisition relies on crawlers to fetch roughly 10 000 original news items daily, followed by manual sensitive‑content review and automated text analysis for classification, tagging, and topic extraction.

User modeling captures real‑time logs using Scribe, Flume, and Kafka, processes data with Hadoop and Storm, and stores models in MySQL/MongoDB (with read/write separation) and Memcached/Redis, covering dimensions such as subscriptions, tags, and article push preferences.

For new‑user cold‑start, Toutiao identifies device, OS, and social‑login information, leveraging friends, followers, and activity to build an initial profile.

The recommendation system combines automatic and semi‑automatic pipelines: automatic candidate generation, user matching, and push task creation; semi‑automatic selection based on user actions, with personalization across frequency, content, region, and interests.

Data storage utilizes MySQL or MongoDB for persistence, Memcached/Redis for caching, and distributes images via CDN; message push boosts DAU by about 20 % and is measured by click‑through rates and uninstall metrics.

Toutiao's architecture is illustrated with multi‑layer diagrams showing a split‑into‑micro‑services approach, a common abstraction layer for code reuse, and a three‑tier virtualization PaaS platform (IaaS, SaaS, App engine).

The core components include data generation and collection, transmission via Kafka, storage in databases and data warehouses, and computation using batch, MPP, and cube processing models to support efficient analytics.