Multi‑Level Deep Model Fusion for Fake News Detection Using BERT – Winning Solution of WSDM Cup 2019

WSDM (Web Search and Data Mining) is a top‑tier information‑retrieval conference. In the 12th WSDM Cup 2019, the Travel team from Meituan’s NLP Center achieved second place in the "Fake News Identification" task and presented their solution at the conference.

1. Background The rapid growth of online information has led to a surge of fake news, which harms public opinion and social stability. The competition aims to develop accurate methods for detecting such misinformation.

2. Data Analysis The provided dataset contains over 320,000 training samples and 80,000 test samples, each consisting of a pair of news headlines labeled as Agreed, Disagreed, or Unrelated. The authors observed a severe class imbalance (≈70% Unrelated) and analyzed title length distributions, which were similar across classes and mostly between 20–100 characters.

3. Data Pre‑processing and Augmentation To reduce noise, the team converted traditional Chinese characters to simplified, removed stop‑words, and performed label‑propagation based data augmentation. They also expanded the training set by swapping the order of the two headlines in each pair.

4. Base Model The team selected BERT, a bidirectional Transformer‑based language model, as the backbone because of its strong text representation capabilities, which are essential for the natural‑language‑inference formulation of the task.

5. Multi‑Level Deep Model Fusion Framework To improve performance while keeping computational cost reasonable, the team used a three‑stage fusion strategy. First, 25 BERT variants were fine‑tuned (Blending). Second, a 5‑fold stacking layer combined these models using traditional classifiers (SVM, LR, KNN, NB). Third, a linear LR model performed a final blending. This hierarchical design increased model diversity and depth, yielding superior results.

6. Experiments The weighted accuracy metric was used to address class imbalance. The single best BERT model achieved 0.8675 accuracy; averaging 25 BERT models raised it to 0.8770, and the proposed multi‑level fusion reached 0.88156, demonstrating a significant performance boost.

7. Conclusion and Outlook The solution highlights the effectiveness of data augmentation, careful preprocessing, and multi‑level model fusion for fake‑news classification. Future work includes further pre‑training BERT on news‑domain data to better capture domain‑specific language.