COBRA: Unified Generative Recommendations with Cascaded Sparse-Dense Representations

Recent advances in generative AI have sparked interest in applying large language models (LLMs) to recommendation systems. Baidu's advertising recommendation team presents a new generative retrieval framework called COBRA (Cascaded Organized Bi‑Represented Generative Retrieval), which combines sparse ID generation with dense vector representations.

The paper titled Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations (arXiv:2503.02453) introduces four main contributions: (1) a cascaded sparse‑dense retrieval architecture that alternates between generating sparse IDs and dense vectors, (2) end‑to‑end learnable dense representations via a trainable encoder, (3) a coarse‑to‑fine generation process that first predicts a sparse ID and then refines it into a dense vector, and (4) extensive empirical validation showing state‑of‑the‑art performance on multiple benchmark datasets.

COBRA builds on earlier generative retrieval methods such as Google’s TIGER. It first encodes item features into dense vectors, then quantizes them into hierarchical ID tuples using a residual‑quantized VAE (RQ‑VAE). During inference, a causal transformer predicts the next sparse ID, which is fed back to generate a fine‑grained dense vector. The framework also incorporates a BeamFusion mechanism that balances relevance and diversity.

Extensive experiments on public datasets (Amazon Beauty, Sports & Outdoors, Toys & Games) demonstrate significant improvements over TIGER: up to 24.5% gain in Recall@10 and 19.2% gain in NDCG@10. Industrial evaluations on Baidu’s advertising logs (500 M users, 200 M ads) show a 3.6% increase in conversion rate and a 4.15% rise in ARPU when deployed in production.

Overall, COBRA offers a novel paradigm for generative recommendation by tightly integrating sparse and dense representations, enabling more accurate and diverse recommendations across various scenarios.