metadata synchronization

DataFunTalk

Oct 23, 2022 · Backend Development

Design and Implementation of a Lightweight Asynchronous Message Processing Framework for Data Catalog

This article describes the motivation, requirements, design, and implementation of a lightweight asynchronous message processing framework built by ByteDance to handle near‑real‑time metadata changes for DataLeap's Data Catalog, detailing its architecture, thread model, state management, delay handling, monitoring, and operational experiences.

Backend FrameworkKafkaMessage Queue

0 likes · 11 min read

Design and Implementation of a Lightweight Asynchronous Message Processing Framework for Data Catalog

DataFunTalk

Jun 25, 2022 · Big Data

Alluxio Metadata and Data Synchronization: Design, Implementation, and Optimization

This article provides a comprehensive overview of Alluxio's metadata and data synchronization mechanisms, covering its unified namespace, mounting strategies, consistency models, various write modes, read workflows, metadata sync techniques, performance optimizations, and recommended configurations for different deployment scenarios.

AlluxioPerformance Optimizationbig data

0 likes · 26 min read

Alluxio Metadata and Data Synchronization: Design, Implementation, and Optimization