How LLM-Auction Lets Large Language Models Learn to Auction Marketing Content Within Answers

Overview

With ChatGPT, Perplexity and similar LLM‑driven applications reshaping information access, traditional "slot‑based" online advertising—where marketing cards are inserted at fixed positions—fails to exploit LLM reasoning and generation capabilities. The authors propose LLM‑Native advertising, where marketing content is integrated naturally into LLM responses, turning the allocation target from discrete slots to the LLM's output distribution.

Background

Two core challenges arise: (1) classic position auctions become invalid because there is no clear discrete placement, and (2) existing methods either auction before generation (low inference cost but cannot model externalities) or generate then auction (captures context but incurs linear LLM inference cost with the number of advertisers). The paper introduces LLM‑Auction, which lets the LLM itself learn to auction, modeling allocation and generation as a preference‑alignment problem.

LLM‑Auction Framework

The framework addresses three questions: what is being allocated, how the LLM should generate to satisfy the mechanism, and how to price the allocation. Allocation is defined as the LLM’s output distribution conditioned on user query, user profile, candidate ads, and advertiser bids. The mechanism objective combines expected effect metrics (CTR, CVR) with constraints (ad count, format, KL‑divergence to the pretrained model).

Preference‑Alignment Training

A reward model estimates the expected effect of each ad within a full response, using offline‑trained effect‑prediction parameters. This model provides feedback for DPO‑style preference alignment, but direct training causes distribution shift. To mitigate this, the authors propose Iterative Reward‑Preference Optimization (IRPO), alternating between updating the reward model with real user feedback and fine‑tuning the LLM.

Mechanism Properties

After alignment, the learned generative allocation satisfies monotonicity (higher bids never reduce expected effect) and continuity (small bid changes lead to smooth changes in allocation), unlike the stepwise outcomes of traditional slot auctions. These properties support a single‑price billing rule where advertisers pay bid × expected effect.

Experimental Evaluation

Simulation Environment

A large‑scale LLM‑as‑a‑judge simulator creates 15 k user queries from 3 k synthetic user profiles and 100 real ads from Taobao. Ads are inserted using the format @AdTitle@[AdID]. The environment includes an Ad‑LLM (the trained LLM) and a User‑LLM that generates click feedback.

Allocation Efficiency Comparison

LLM‑Auction is compared against four baselines: (1) a pretrained Qwen3‑4B LLM, (2) RAG‑Auction (auction‑then‑generate), (3) MOSAIC (generate‑then‑auction), and (4) an Oracle version of LLM‑Auction with perfect effect estimates. LLM‑Auction outperforms all baselines, achieving a 59.1 % revenue increase over MOSAIC and a reward boost from –15.27 to 81.86, without extra inference cost.

Mechanism Property Verification

Monotonicity is validated by perturbing bids on a random ad and measuring click‑based allocation; the correlation strengthens over training epochs, achieving monotonic allocation after three epochs. Continuity is verified by fixing a query and ad, varying the bid from 1 to 100, sampling 200 responses per bid, and observing smooth click‑rate changes, confirming the single‑price rule.

Ablation studies show that (i) using the full LLM response as input improves allocation, and (ii) IRPO mitigates distribution shift, stabilizing performance. Qualitative cases demonstrate better semantic matching and higher placement priority for high‑bid ads.

Conclusion and Future Work

LLM‑Auction demonstrates that integrating auction design with LLM alignment yields a unified, efficient, and user‑friendly AI‑native advertising mechanism that adds no extra inference overhead. Future directions include building larger, more realistic benchmarks, exploring learned dynamic pricing, and extending the reward function to incorporate answer credibility and ad fidelity.