Deep Learning Applications in Recommender Systems: A Hybrid Collaborative Filtering Model

In recent years deep learning has achieved remarkable breakthroughs in speech recognition, image processing, and natural language processing, while its research and application in recommender systems remain in an early stage. Ctrip’s BI team, led by senior algorithm engineer Dong Xin, presented at the Ctrip Personalized Recommendation and AI Meetup and published a hybrid collaborative‑filtering model with deep structure at AAAI 2017.

1. Recommendation System Overview – A recommender system aims to present users with personalized items by predicting missing ratings in a large, sparse user‑item matrix. The matrix consists of rows (users), columns (items), and values (ratings), and the system’s output is a ranked list of items for each user.

The problem can be formalized as predicting the unknown entries ("?") of the matrix based on observed ratings ("+"). This leads to two sub‑tasks: rating prediction and recommendation.

2. Collaborative Filtering – Traditional collaborative filtering is divided into memory‑based (user‑based, item‑based) and model‑based approaches, the latter most commonly using matrix factorization. Matrix factorization decomposes the rating matrix R into two low‑rank matrices U and V, learning latent user and item vectors by minimizing a loss on observed entries only.

Key challenges include data sparsity and cold‑start, often mitigated by incorporating side information (user demographics, item content). Collective Matrix Factorization (CMF) extends the idea by jointly factorizing multiple related matrices while sharing latent factors.

3. Deep Learning in Recommender Systems – Deep models can learn richer latent representations. Autoencoders (AE) reconstruct input rating vectors, predicting missing values. Denoising Autoencoders (DAE) add noise to inputs, forcing the network to learn robust features. Stacked Denoising Autoencoders (SDAE) stack multiple DAEs, and Bayesian SDAE introduces probabilistic priors on parameters.

The Ctrip team improved these ideas by proposing an Additional Stacked Denoising Autoencoder (aSDAE) that takes both rating vectors and side‑information at each hidden layer, inspired by sequence‑to‑sequence models. Combining two aSDAEs (one for users, one for items) with matrix factorization yields a hybrid collaborative‑filtering model that jointly optimizes reconstruction loss and rating prediction loss via stochastic gradient descent.

Experimental evaluation on several public datasets shows the hybrid model achieves lower RMSE and higher recall compared with baseline methods.

Other notable deep‑learning‑based recommender approaches include YouTube’s DNN model (embedding user behavior and side information) and convolutional matrix factorization that learns item embeddings from document text via CNNs.

4. Conclusion – Deep learning models such as DNNs, autoencoders, and CNNs can effectively enhance recommendation accuracy, but different domains may require tailored architectures. As deep learning continues to evolve, it is expected to become a core technology for recommender systems.

References

[1] Ajit P. Singh, Geoffrey J. Gordon. “Relational Learning via Collective Matrix Factorization”, KDD 2008.

[2] Suvash Sedhain et al. “AutoRec: Autoencoders Meet Collaborative Filtering”, WWW 2015.

[3] Hao Wang, Naiyan Wang, Dit-Yan Yeung. “Collaborative Deep Learning for Recommender Systems”, KDD 2015.

[4] Xin Dong et al. “A Hybrid Collaborative Filtering Model with Deep Structure for Recommender Systems”, AAAI 2017.

[5] Paul Covington, Jay Adams, Emre Sargin. “Deep Neural Networks for YouTube Recommendations”, RecSys 2016.

[6] Donghyun Kim et al. “Convolutional Matrix Factorization for Document Context‑Aware Recommendation”, RecSys 2016.