Cloud Native 3 min read

Optimizing I/O for Data‑Intensive Analytics in Cloud‑Native Environments: Insights from Uber’s Presto Deployment

This whitepaper examines the trend of moving data‑intensive analytics workloads to cloud‑native platforms, revealing how cloud storage cost models affect I/O optimization, and using Uber’s Presto production data to show that traditional I/O strategies overlook costly storage API calls, leading to high expenses.

DataFunTalk
DataFunTalk
DataFunTalk
Optimizing I/O for Data‑Intensive Analytics in Cloud‑Native Environments: Insights from Uber’s Presto Deployment

This whitepaper explores the industry trend of migrating data‑intensive analytics applications from on‑premises environments to cloud‑native platforms, emphasizing the unique cost model of cloud storage and its implications for performance optimization.

Through an empirical study of Uber’s production Presto workload, the authors discover that more than 50 % of data accesses are smaller than 10 KB and over 90 % are under 1 MB, indicating a highly fragmented access pattern that incurs significant storage API call costs in the cloud.

The paper presents a case‑study‑driven analysis of traditional I/O optimization techniques, demonstrating that they often ignore the financial overhead of storage API interactions, which can lead to unexpectedly high expenses when applications are migrated to cloud environments.

Based on these findings, the authors propose a set of cloud‑aware I/O design principles and strategies aimed at improving cost‑performance ratios for data‑intensive workloads in cloud‑native settings.

Readers will gain a concrete framework for designing efficient I/O solutions tailored to cloud storage economics, providing a foundation for further research and practical implementation.

case studycloud nativeI/O optimizationprestodata-intensivecost model
DataFunTalk
Written by

DataFunTalk

Dedicated to sharing and discussing big data and AI technology applications, aiming to empower a million data scientists. Regularly hosts live tech talks and curates articles on big data, recommendation/search algorithms, advertising algorithms, NLP, intelligent risk control, autonomous driving, and machine learning/deep learning.

0 followers
Reader feedback

How this landed with the community

login Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.