How eBay’s Data+AI Platform Leverages Ray for Faster Model Development and Deployment

Background and Challenges

Traditional workflows at eBay required researchers to write models in Python while production teams rewrote the same code in Java, consuming about 50% of development time. Inefficient data loading and model path handling kept GPU utilization around 10%, far below the hardware’s potential.

Solution: Building a Data+AI Platform on Ray

Ray, a flexible distributed‑computing framework, was introduced to provide a unified API that lets researchers develop and deploy models entirely in Python. This eliminated the need for language conversion, reduced code‑base complexity, and enabled automatic cluster scaling via Ray Notebook, raising GPU utilization to over 75%.

Key Highlight 1 – Ray Notebook Auto‑Scaling

Ray Notebook monitors resource demand and automatically expands or contracts GPU clusters. The mechanism allows seamless switching between development and production environments, improving resource scheduling flexibility and delivering large‑model real‑time and batch inference efficiency.

Key Highlight 2 – GPU Utilization in Batch Inference

By combining large batch sizes, Triton, and Ray’s auto‑scaling, eBay reduced I/O bottlenecks, increasing GPU utilization by 65% for batch inference. Ray’s data‑flow architecture also optimized streaming processing, making inference smoother and more accurate.

Key Highlight 3 – Near‑Real‑Time Inference Architecture

The Pythonic API lets researchers integrate model development and production pipelines in a single environment, simplifying coordination between GPU and CPU tasks. This architecture maintains model performance while supporting rapid iteration, enabling fast responses to business needs.

Results and Insights

The Ray‑enabled platform cut model development and deployment time by nearly half and lifted GPU utilization from roughly 10% to over 75%. Automated scaling and high‑availability features also improved platform stability and scalability, offering a reference model for other enterprises building efficient Data+AI platforms.

Future Outlook

eBay plans to enhance Ray cluster high‑availability and security integration, expand support for larger models, and continue deepening AI infrastructure to drive broader and deeper AI applications across the industry.