Tag

Data Orchestration

1 views collected around this technical thread.

Alibaba Cloud Infrastructure
Alibaba Cloud Infrastructure
Jun 3, 2024 · Cloud Native

Fluid 1.0 Release: Cloud‑Native Data Orchestration for AI and Big Data

Fluid 1.0 introduces a cloud‑native data orchestration platform that abstracts dataset management, affinity scheduling, custom data processing, and data flow pipelines for AI and big‑data workloads on Kubernetes, backed by extensive production testing, open‑source contributions, and a roadmap for future enhancements.

AIBig DataCloud Native
0 likes · 13 min read
Fluid 1.0 Release: Cloud‑Native Data Orchestration for AI and Big Data
DataFunSummit
DataFunSummit
May 5, 2024 · Big Data

Alluxio in Lakehouse Architecture: Benefits, Challenges, and Real‑World Use Cases

This article explains how Alluxio enables a unified lake‑warehouse architecture by decoupling compute and storage, outlines its core capabilities, evaluates the cost‑saving and performance benefits, discusses the technical challenges, and presents several practical deployment scenarios in finance and AI workloads.

AlluxioBig DataCloud Native
0 likes · 15 min read
Alluxio in Lakehouse Architecture: Benefits, Challenges, and Real‑World Use Cases
DataFunTalk
DataFunTalk
Feb 9, 2024 · Big Data

Alluxio’s Role in Lakehouse Architecture: Benefits, Challenges, and Real‑World Use Cases

This article explains how Alluxio enables lake‑warehouse integration by providing a data orchestration layer that caches data near compute, reduces storage‑compute separation costs, improves performance, and addresses challenges such as security, scalability, and multi‑cloud deployment, illustrated with several industry case studies.

AIAlluxioBig Data
0 likes · 16 min read
Alluxio’s Role in Lakehouse Architecture: Benefits, Challenges, and Real‑World Use Cases
DataFunTalk
DataFunTalk
Feb 3, 2024 · Big Data

Alluxio: Introduction, Architecture, and Practical Experience for Big Data Construction

This article introduces Alluxio as an open‑source data orchestration layer, explains its architecture and core features such as unified namespace, caching strategies, and cloud‑native deployment, and shares practical experiences on using Alluxio to simplify data lakehouse construction, migration, and hot‑cold data separation in complex big‑data environments.

AlluxioBig DataData Lakehouse
0 likes · 13 min read
Alluxio: Introduction, Architecture, and Practical Experience for Big Data Construction
DataFunTalk
DataFunTalk
Nov 26, 2023 · Big Data

Data Orchestration in Hybrid Storage Architectures with Alluxio

This article explains how Alluxio, an open‑source data orchestration system, improves data access efficiency in hybrid multi‑cloud and multi‑storage environments by providing caching, a unified namespace, interface translation, automated data management, and federation capabilities for modern big‑data workloads.

AlluxioData FederationData Orchestration
0 likes · 18 min read
Data Orchestration in Hybrid Storage Architectures with Alluxio
DataFunTalk
DataFunTalk
Feb 17, 2023 · Big Data

Tencent Alluxio (DOP) Deployment and Optimization in Financial Data Analytics

This article describes how Tencent's Alluxio-based Data Orchestration Platform (DOP) was applied to financial analytics, detailing the business background, challenges of large‑scale OLAP workloads, the Alluxio architecture and usage modes, performance results, and the series of optimizations and tuning performed to achieve significant speedups.

AlluxioBig DataData Orchestration
0 likes · 15 min read
Tencent Alluxio (DOP) Deployment and Optimization in Financial Data Analytics
DataFunTalk
DataFunTalk
Jan 19, 2023 · Big Data

Tencent Alluxio: Accelerating the Next Generation of Big Data and AI

This article presents a comprehensive overview of Tencent's Alluxio project, covering the evolution of big‑data architecture, recent Alluxio research progress, typical deployment cases, and future work, while highlighting performance improvements, integration with cloud and AI workloads, and community contributions.

AIAlluxioBig Data
0 likes · 21 min read
Tencent Alluxio: Accelerating the Next Generation of Big Data and AI
DataFunTalk
DataFunTalk
Sep 4, 2022 · Big Data

Alluxio 2.8 New Features Overview

This article summarizes the Alluxio 2.8 release, detailing enhancements in API support, enterprise‑grade security features, and data‑movement capabilities, while also covering new encryption options, master‑proxy S3 token handling, OPA integration, and various performance and observability optimizations.

APIAlluxioBig Data
0 likes · 9 min read
Alluxio 2.8 New Features Overview
DataFunTalk
DataFunTalk
Aug 31, 2022 · Big Data

Alluxio Data Orchestration and Cache Acceleration in China Unicom: Use Cases and Performance Gains

This article presents Zhang Ce's detailed overview of Alluxio's deployment at China Unicom, covering cache acceleration, compute‑storage separation, mixed‑load workloads, and lightweight analysis, and demonstrates how these strategies dramatically improve performance, scalability, and cost efficiency for big data processing.

AlluxioBig DataCache Acceleration
0 likes · 19 min read
Alluxio Data Orchestration and Cache Acceleration in China Unicom: Use Cases and Performance Gains
Architecture Digest
Architecture Digest
Dec 8, 2021 · Cloud Native

Implementing Compute-Storage Separation for Large-Scale Retrieval Systems Using Fluid

This article describes the challenges of operating massive, TB‑scale retrieval clusters at Zuoyebang, and presents a Fluid‑based compute‑storage separation architecture that improves data distribution, update efficiency, scalability, and stability, enabling containerized search services to be managed like regular stateless workloads.

Compute-Storage SeparationData OrchestrationFluid
0 likes · 13 min read
Implementing Compute-Storage Separation for Large-Scale Retrieval Systems Using Fluid
Big Data Technology Architecture
Big Data Technology Architecture
Aug 15, 2020 · Big Data

Alluxio: Open‑Source Data Orchestration Platform – Overview, Benefits, Innovations, and Getting‑Started Resources

Alluxio is an open‑source, memory‑centric data orchestration layer that bridges compute frameworks such as Spark, Presto, and TensorFlow with diverse storage systems, offering high‑speed I/O, unified namespace, multi‑level caching, and easy deployment, while providing extensive documentation, download links, and community resources for rapid adoption.

AlluxioBig DataData Orchestration
0 likes · 7 min read
Alluxio: Open‑Source Data Orchestration Platform – Overview, Benefits, Innovations, and Getting‑Started Resources