Interview with Wang Xiaobo (Yongshu) on Large‑Scale Machine Learning, Recommendation Systems, and AutoML at Alibaba and Youku

At this year’s AI Pioneer Conference, CSDN invited Wang Xiaobo (known as “Yongshu”), the head of Alibaba’s Commercial Machine Intelligence Algorithm Team and Youku’s Algorithm Technology Team, for a Q&A on the practical deployment of machine learning.

Background: Wang holds a Ph.D. earned in 2010 and has ten years of experience in data mining. He previously worked on advertising strategy algorithms at Baidu and Sogou, focusing on display and search ads. He joined Alibaba in March 2015, initially responsible for Taobao’s homepage recommendation (“You Might Like”). He now leads two algorithm teams: the Commercial Machine Intelligence team (B2B and B2C recommendation) and the Youku algorithm platform (search, recommendation, content understanding, intelligent marketing, and cloud engine).

Current research focus at Youku:

1. Content understanding – moving from holistic to element‑level comprehension to narrow the information gap between machines and humans after watching a video.

2. Multi‑modal video search and recommendation.

3. Combining few‑shot learning with domain knowledge.

Why large‑scale distributed machine learning? Wang’s Ph.D. work was on complex networks (graph mining). When he joined Baidu’s ad‑strategy team, he was thrust into click‑through‑rate (CTR) prediction, a massive binary classification problem with huge data volume. The business need to improve CTR accuracy pushed him into large‑scale machine learning.

Role of machine learning in AI: Machine learning provides a practical pathway for AI deployment. Since the late 1990s, statistical machine learning has become the dominant paradigm, enabling AI to integrate deeply with any industry, regardless of data size.

Deep learning perspective: Deep learning is an extension of neural networks dating back to the 1960s‑70s. It transforms complex nonlinear functions into scalable models, where each neuron acts like a simple logistic classifier. By stacking many weak learners (similar to boosting), deep networks achieve strong expressive power. Initially successful in vision and speech, deep learning now flourishes in natural language processing and serves as a powerful sub‑set of machine learning, especially when paired with abundant hardware resources.

Cold‑start problem in recommendation: The classic issue of recommending to users with little or no data is tackled in two ways. First, expand user information collection (e.g., questionnaires, leveraging unified accounts across Alibaba apps, location data, and transfer learning from other domains). Second, employ exploration‑and‑exploitation strategies or reinforcement learning, using random probes and reward‑guided online learning to reduce exploration loss.

Differences between e‑commerce and video recommendation: E‑commerce items have well‑structured, standardized attributes, while video content lacks such uniform features, leading to high heterogeneity. Video recommendation must handle non‑standardized metadata, diverse user intents, and a reverse ratio of items to users compared to e‑commerce.

AutoML discussion: Alibaba built the XPS (eXtreme Parameter Server) platform to support billions of features. While many teams adopt TensorFlow, its runtime optimizations are insufficient for trillion‑sample, hundred‑billion‑feature scenarios. AutoML aims to automate network architecture search, dynamic feature‑embedding sizing, and hyper‑parameter tuning, reducing manual effort. Future directions include network compression for mobile deployment (e.g., NPU‑accelerated models) and shrinking large models from gigabytes to tens of megabytes with minimal accuracy loss.

Multi‑modal search during large‑scale promotion events: Alibaba’s “Double‑11” promotion is divided into hype, pre‑heat, and transaction phases, each with distinct optimization goals (impressions, add‑to‑cart, GMV). Search ranking must adapt to varying traffic patterns, user contexts (e.g., relaxed vs. fragmented time slots), and price elasticity, requiring dynamic reward functions and real‑time learning.

Transfer learning for Taobao’s “see‑again” page: To overcome limited candidate pools for a single merchant, Alibaba introduced a “neighboring store” module, forming alliances across non‑competing merchants. Transfer learning leverages user behavior from one domain (e.g., fashion) to another (e.g., electronics), expanding the recommendation space.

Short‑video search challenges: Sparse metadata forces a labeling approach. Massive label spaces (millions of tags) and long‑tail tags require few‑shot learning combined with domain knowledge. Ranking shifts from static click‑through‑rate optimization to interactive, multi‑objective learning, often using reinforcement learning to model user actions (scroll, click, like, etc.) in a long‑form feed.

Job market and AI outlook: While top‑tier AI talent remains scarce, many AI engineers focused only on parameter tuning are becoming saturated. Future demand will favor specialists who can solve fundamental research problems (few‑shot learning, domain adaptation) and senior algorithm engineers who can translate business problems into technical solutions. Leaders must improve their understanding of algorithm capabilities to avoid misusing black‑box models.