online experiments

DataFunTalk

Jul 22, 2024 · Fundamentals

A/B Testing and Causal Inference: Evolution of Sampling, Metric Evaluation, and Statistical Inference

The article reviews the development of online A/B testing, covering sampling and traffic‑splitting techniques, metric computation improvements, statistical inference advances, and current challenges such as interference, real‑time inference, and large‑scale metric computation, while referencing recent research papers.

A/B TestingMetric Evaluationcausal inference

0 likes · 10 min read

A/B Testing and Causal Inference: Evolution of Sampling, Metric Evaluation, and Statistical Inference

DataFunTalk

May 12, 2024 · Artificial Intelligence

Paired Data Based A/B Experiments: Causal Inference in Network Experiments

The DataFun Data Science Summit on May 25 will feature Tencent data scientist Li Yilin presenting a comprehensive overview of paired‑data A/B experiments, covering causal inference challenges, unbiased estimators under various randomization designs, theoretical analysis, and practical insights for network‑based online experiments.

A/B Testingcausal inferencenetwork experiments

0 likes · 5 min read

Paired Data Based A/B Experiments: Causal Inference in Network Experiments

DataFunTalk

Mar 28, 2024 · Artificial Intelligence

Multi-Task and Multi-Scenario Algorithms for Recommendation Systems: Methods, Challenges, and Applications

This article presents a comprehensive overview of multi‑task and multi‑scenario recommendation algorithms, detailing background challenges, algorithm classifications such as TAML, CausalInt, and DFFM, their modular designs, experimental validations, and practical Q&A insights for large‑scale advertising systems.

Recommendation systemsadvertising algorithmsmachine learning

0 likes · 19 min read

Multi-Task and Multi-Scenario Algorithms for Recommendation Systems: Methods, Challenges, and Applications

DataFunSummit

Oct 17, 2023 · Artificial Intelligence

DataFunSummit2023: Deep Learning‑Driven Multi‑Experiment Causal Inference and Distributed Causal Tools

The DataFunSummit2023 online conference brings together experts from Tencent and Kuaishou to present cutting‑edge research on causal inference for large‑scale A/B testing, including deep‑learning‑based multi‑experiment effect estimation, a distributed causal inference framework (Fast‑Causal‑Inference), and strategies for evaluating long‑term policy impacts.

A/B TestingDeep Learningcausal inference

0 likes · 7 min read

DataFunSummit2023: Deep Learning‑Driven Multi‑Experiment Causal Inference and Distributed Causal Tools

DataFunTalk

Oct 4, 2023 · Big Data

Insights into Regional Differences in Overseas A/B Experiments

The presentation explains how to detect, analyze, and leverage regional variations in overseas A/B test results to make more informed product decisions, using a systematic experimental analysis framework grounded in causal inference and online experimentation methods.

A/B Testingcausal inferencegame data science

0 likes · 2 min read

Insights into Regional Differences in Overseas A/B Experiments

DataFunTalk

Feb 6, 2023 · Product Management

A Comprehensive Guide to A/B Testing: Principles, Methods, and Applications

This article explains the scientific foundations, historical origins, statistical principles, implementation techniques, and practical applications of A/B testing as a data‑driven growth tool for product optimization, algorithm iteration, and marketing decisions in modern internet companies.

A/B TestingProduct Optimizationdata-driven growth

0 likes · 26 min read

A Comprehensive Guide to A/B Testing: Principles, Methods, and Applications

DataFunTalk

Feb 20, 2022 · Artificial Intelligence

Distilled Reinforcement Learning Framework for Recommendation (DRL-Rec): Design, Modules, and Experimental Evaluation

This article presents DRL-Rec, a distilled reinforcement learning framework for recommendation that integrates an exploring‑filtering module and confidence‑guided distillation to compress RL‑based recommenders while improving accuracy, and reports significant offline and online performance gains on a large‑scale system.

Recommendation systemsknowledge distillationmodel compression

0 likes · 16 min read

Distilled Reinforcement Learning Framework for Recommendation (DRL-Rec): Design, Modules, and Experimental Evaluation

DataFunTalk

Nov 1, 2021 · Product Management

Online Experiment Design and Analysis: Practices, Case Studies, and Guidelines from Tencent Data Platform

This article presents a comprehensive overview of online experiment design and analysis, covering basic definitions, AB testing principles, complex experiment types, real-world case studies from Tencent's information flow platform, and practical guidelines for reliable experiment evaluation and product decision‑making.

A/B TestingRecommendation systemscausal inference

0 likes · 21 min read

Online Experiment Design and Analysis: Practices, Case Studies, and Guidelines from Tencent Data Platform

Alimama Tech

Jul 14, 2021 · Big Data

A/B Testing Framework for Online Experiments: Design, Implementation, Analysis, and Decision Making

The paper presents a comprehensive A/B testing framework for online experiments that guides practitioners through four stages—designing objectives and metrics, implementing random traffic allocation with robustness checks, evaluating effects using descriptive statistics and hypothesis testing, and making rollout decisions based on multidimensional significance and attribution analyses.

A/B TestingExperimental designdata analysis

0 likes · 22 min read

A/B Testing Framework for Online Experiments: Design, Implementation, Analysis, and Decision Making

DataFunTalk

Jul 11, 2021 · Fundamentals

Understanding Online Experiments: Origins, Types, and Applications

This article explains the concept, history, and various forms of online experiments such as AB testing, ABn, AA, and multivariate tests, highlighting their role in causal inference, value evaluation, risk control, and product optimization within modern internet businesses.

AB testingProduct Optimizationcausal inference

0 likes · 16 min read

Understanding Online Experiments: Origins, Types, and Applications

Alimama Tech

Jul 8, 2021 · Product Management

Understanding Online Experiments: Origins, Development, Types, and Applications

Online experiments, rooted in biomedical randomized controlled trials, have become essential for internet businesses to achieve data‑driven growth by providing causal inference, quantifying value, and managing risk through various designs such as AB, ABn, AA, multivariate and quasi‑experimental tests.

AB testingProduct Optimizationcausal inference

0 likes · 18 min read

Understanding Online Experiments: Origins, Development, Types, and Applications

DataFunTalk

Jun 13, 2021 · Artificial Intelligence

HRL-Rec: A Hierarchical Reinforcement Learning Framework for Integrated Recommendation

This article presents HRL-Rec, a hierarchical reinforcement learning model that jointly learns user preferences at the item and channel levels for integrated recommendation systems, and demonstrates its superior offline and online performance, stability, and scalability through extensive experiments on the WeChat "See" platform.

channel selectorhierarchical reinforcement learningintegrated recommendation

0 likes · 12 min read

HRL-Rec: A Hierarchical Reinforcement Learning Framework for Integrated Recommendation

DataFunTalk

Aug 15, 2020 · Artificial Intelligence

Dynamic Knapsack Optimization for Multi‑Channel Sequential Advertising Using Long‑Term Value

The article presents a novel multi‑channel sequential advertising framework that models budget‑constrained GMV optimization as a dynamic knapsack problem, introduces a long‑term value‑based RL solution (MSBCB), and validates its superiority through extensive offline and online experiments showing up to 10% ROI improvement.

advertisingbudget optimizationdynamic knapsack

0 likes · 16 min read

Dynamic Knapsack Optimization for Multi‑Channel Sequential Advertising Using Long‑Term Value