Sliding Spectrum Decomposition for Diversified Recommendation in Feed Systems

Abstract Researchers from Xiaohongshu propose a Sliding Spectrum Decomposition (SSD) method to capture users' perception of diversity when browsing long item sequences in a feed. The approach is validated through theoretical analysis, offline experiments, and online A/B tests.

Background Diversified recommendation is crucial for both user experience and platform health. It helps users discover new interests and prevents the “filter bubble”. In Xiaohongshu’s Explore feed, diversity, quality, and fairness are three key factors. The paper studies diversity in the context of continuous feed browsing, where a sliding window size influences the user experience.

The work is based on the KDD’21 paper “Sliding Spectrum Decomposition for Diversified Recommendation”. Users view the feed as a one‑dimensional time series of items. By decomposing the item sequence into independent components (analogous to price‑trend decomposition in time‑series analysis), the method aims to measure and improve diversity.

SSD Method The feed sequence is transformed into a tensor. For a simple case with two items (w = T = 2), orthogonal directions define independent components. The overlap and orthogonal parts are quantified; larger orthogonal parts indicate higher diversity. In higher dimensions, the volume of a window’s item vectors serves as a diversity metric. Since tensor determinants are undefined, the product of singular values (via SVD) is used as a proxy for volume, extending the concept to tensors.

The overall objective combines quality (exploitation) and diversity (exploration) using an EE (exploitation‑exploration) trade‑off. SSD is compared with Determinantal Point Processes (DPP); SSD can be seen as a sequential extension of DPP, handling windows explicitly and offering a more efficient greedy approximation.

CB2CF (Content‑Based to Collaborative Filtering) To obtain item vectors, two strategies are discussed: content‑based supervision and collaborative‑filtering (CF) embeddings. Both have drawbacks, so a hybrid CB2CF model is introduced, which predicts CF vectors from content features, enabling cold‑item recommendations while preserving fairness.

Experimental Results Offline experiments show CB2CF outperforms pure CF on long‑tail categories. Online A/B tests comparing SSD with state‑of‑the‑art DPP demonstrate gains in time spent, engagement, inter‑item distance (ILAD), and average category count (MRT). The paper includes links to the full arXiv version.

References 1. Huang et al., “Sliding Spectrum Decomposition for Diversified Recommendation,” KDD 2021. 2. Barkan et al., “CB2CF: a neural multiview content‑to‑collaborative filtering model for completely cold item recommendations,” RecSys 2019. 3. Chen et al., “Fast greedy MAP inference for DPP to improve recommendation diversity,” NeurIPS 2018. 4. Broomhead & King, “Extracting qualitative dynamics from experimental data,” Physica D 1986.