Generative Adversarial User Model for Reinforcement Learning‑Based Recommendation Systems

Reinforcement learning (RL) offers a promising way to incorporate long‑term user satisfaction into recommendation systems, but practical deployment faces two major challenges: the reward function and environment dynamics are unknown, and model‑free RL requires prohibitive amounts of online interaction.

To address these issues, the authors propose a generative adversarial network (GAN) that jointly learns a user behavior model (the transition dynamics) and the reward function from offline data. The learned user model serves as a simulated environment for RL, allowing offline training before online fine‑tuning.

Within this simulated environment, they introduce a Cascading‑DQN algorithm that selects a set of items rather than a single item, reducing the combinatorial search space to linear complexity with respect to the candidate pool.

Experiments on real‑world datasets demonstrate that the GAN‑based user model predicts user behavior more accurately than baselines, and the resulting RL policy yields higher long‑term cumulative reward and click‑through rate while dramatically lowering the number of required online interactions.

The paper’s main contributions are: (i) a GAN framework that simultaneously learns user dynamics and reward; (ii) a Cascading‑DQN architecture for efficient combinatorial recommendation; and (iii) empirical evidence of improved long‑term performance and sample efficiency.