Experiment-Driven Advertising and User Operations in Game Growth: Causal Inference, Uplift Modeling, and Practical Pitfalls

01 Game Experiment Scenarios

Games can be divided into six user‑lifecycle stages—potential, new, active, declining, at‑risk, and churned—each with distinct development and operations priorities. Traditional expert intuition or simple analytics struggle to handle the myriad micro‑decisions, making experiment‑driven growth essential.

02 Experiment‑Driven Advertising

The case study shows how a naïve response‑model approach selects high‑probability churners for ad exposure, but A/B tests reveal that only users in the mid‑range predicted return probability benefit from ads. This leads to an uplift‑model formulation that directly predicts the incremental return probability caused by advertising.

Four uplift‑model families are introduced: Meta‑learner, Tree‑Based, Deep, and Transform models. Applying uplift modeling identifies three user groups: sleeping dog (ad‑negative), sure thing/lost cost (no effect), and persuadable (positive lift), increasing per‑user gain from 1.4% to 2%.

02.2 Privacy‑Aware Exposure De‑biasing

Because overseas ad platforms hide individual exposure data, users are clustered and cohort‑level exposure rates are requested. Large exposure‑rate variance across cohorts proves exposure bias, which skews uplift predictions toward naturally returning users. Re‑weighting samples by exposure probability removes this bias, raising lift from 0.74% to 1.5%.

02.3 Bid Optimization

Standard eCPM bidding (target CPA × predicted conversion probability) favors users already likely to return. By reformulating eCPM to bid on estimated incremental users (incremental return × willingness to pay), bids are increased for ad‑sensitive low‑return users and decreased for ad‑insensitive high‑return users.

03 Experiment‑Driven User Operations

Typical operational decisions—which users to target, which tasks or gifts to push, and how to design activity pages—are tested via random traffic splits. Common pitfalls include improper account segmentation, low experiment sensitivity, inability to test balance‑critical changes, and contamination from user sharing.

Solutions: use hash‑with‑seed for orthogonal traffic splits; increase sample size or reduce metric variance (e.g., CUPED, log transforms) to boost sensitivity; employ country/server‑level clusters or synthetic controls when user‑level tests are unsafe; detect and mitigate network effects by clustering users and assigning whole clusters to treatment or control.

04 Personal Insights

Key takeaways: balance cost and benefit, avoid over‑simplifying a vast decision space, and recognize that experiments are most valuable in mid‑to‑late growth phases rather than early product launch.

05 Q&A Highlights

Sample size is determined by statistical formulas balancing power, error rates, and expected lift.

High experiment failure rates are expected; they reveal hidden costs.

When persuadable users are few, adjust thresholds or accept lower ROI.

Extreme events in synthetic control groups often require rebuilding the control.

AA test imbalances stem from timing, traffic allocation, or residual effects from prior experiments.