LLMRG: Improving Recommendations through Large Language Model Reasoning Graphs

The paper LLMRG: Improving Recommendations through Large Language Model Reasoning Graphs was accepted as a manuscript at AAAI 2024, one of the top international conferences in artificial intelligence.

Traditional recommendation systems often rely on shallow machine‑learning features or knowledge graphs, which provide rich relational information but lack deep reasoning about user behavior and interests. Large language models (LLMs) have demonstrated strong reasoning, analogy, and causal inference capabilities, offering a new opportunity for more intelligent recommender systems.

The authors ask whether LLMs can endow recommender systems with stronger reasoning and insight capabilities.

To answer this, they propose the LLMRG framework, which builds a personalized reasoning graph for each user using an LLM. The framework consists of four core modules: (1) Chained Graph Reasoning that constructs reasoning chains from a user's historical actions and attributes; (2) Self‑verification and Scoring that generates masks, validates the logical correctness of each chain, and filters out low‑confidence paths; (3) Divergent Extension that expands verified chains into a broader reasoning graph capturing higher‑order intent; and (4) Knowledge‑base Self‑improvement that stores high‑confidence reasoning chains in a knowledge store for retrieval‑augmented generation, improving efficiency.

The resulting reasoning graph is encoded (e.g., via a graph neural network) and used as an additional feature for existing recommendation models, providing both enhanced accuracy and interpretability.

Extensive experiments on three public benchmark datasets—MovieLens‑1M, Amazon Beauty, and Amazon Clothing—show that LLMRG (implemented with GPT‑3.5 or GPT‑4) significantly outperforms several baselines, achieving notable performance gains without requiring any extra user or item information. Ablation studies demonstrate the importance of each module, especially the self‑verification component, which mitigates hallucination‑induced noise from LLMs.

Future directions include exploring more advanced reasoning strategies such as counterfactual and causal modeling, extending LLMRG to other domains like social‑network and advertising recommendation, integrating multimodal AI techniques (e.g., computer vision), and developing more efficient graph construction and encoding algorithms to enable large‑scale deployment.