Designing Incentive Strategies for Two‑Sided Market Experiments

Guide – This article introduces experimental design for incentive strategies in two‑sided markets.

It will cover four points:

1. Problem background

2. Challenges of incentive strategies

3. Possible solutions

4. Building a comprehensive plan

Problem Background

Two‑sided markets consist of producers and consumers that mutually reinforce each other, e.g., creators and viewers on Kuaishou. A bilateral experiment groups producers and consumers separately, allowing simultaneous measurement of a new strategy’s impact on both sides.

Advantages of bilateral experiments include:

Measuring effects on both product DAU and content uploads, capturing cross‑side network effects.

Detecting spillover and transfer effects.

Understanding mechanisms beyond simple A/B results, requiring richer metrics and designs.

Example – A live‑beauty‑filter experiment illustrates how combining producer (anchor) and consumer (viewer) groups reveals spillover when only some anchors receive the filter.

Challenges of Incentive Strategies

Three typical scenarios drive incentive policies:

Introducing high‑quality creators without knowing their performance.

Targeting specific creator types with macro‑level flow support.

Strategic platform directions that reshape flow allocation.

These are not typical machine‑learning problems but macro‑level flow controls, requiring long‑term observation of production and interaction effects.

Key issues include:

Author‑side “crowding out” where boosted creators gain exposure at the expense of control‑group creators.

Violation of the SUTVA assumption because users on one side can be influenced by the other side’s treatment.

Complex ranking algorithms that produce different exposure patterns for various traffic percentages.

Possible Solutions

1. Gradual scaling – Expand the experimental traffic step‑by‑step; the exposure gap narrows as the control‑group’s flow shrinks, reducing crowding‑out effects.

2. Small‑world partition – Completely isolate experiment and control groups so that each side only sees its own content, avoiding cross‑side crowding‑out, though it incurs higher computational cost.

3. Analytical correction – Adjust results post‑hoc using network‑effect models or linear assumptions when experimental design cannot fully eliminate bias.

Comprehensive Plan

Build a ranking‑fusion system that keeps the ordering of a partial‑traffic experiment (RT_a%) consistent with the full‑traffic version (RT_100%). The workflow:

Run both the control ranking (RC) and the boosted ranking (RT) in parallel and record item order.

For experiment‑group creators, present the fused order of RC and RT to users.

When traffic is low, keep non‑experiment items in their original order; when fully rolled out, use the RT order entirely.

If experiment and control items compete for the same slot, resolve it randomly – the probability of conflict is low (≈3.3% for top‑10 with 2% traffic).

Evaluation Metrics

Author‑side: number of works, active creators.

Content‑view metrics: CTR, EVTR, exposure uplift.

Reader‑side: single‑side experiment validation.

All solutions have trade‑offs; the strong spillover in two‑sided markets makes a single fix insufficient, so a combination of gradual scaling, isolation, analytical correction, and ranking fusion is recommended.

Thank you for reading, and click the link below to watch the video replay.