BestHub
Sign in
Tagged articles
214 articles
Page 2 of 3
Baidu Geek Talk
Baidu Geek Talk
Jul 1, 2022 · Big Data

Evolution of Data Platform Technology: From Data Warehouse to Lakehouse Architecture

The article traces the evolution of data platforms from early data warehouses—using schema‑on‑write, columnar storage, and MPP engines—to data lakes that retain raw data with schema‑on‑read, and finally to lakehouse architectures that merge storage and compute, offering unified metadata, versioning, and support for BI, big‑data, AI, and HPC workloads.

Data ArchitectureLakehouseOLAP
0 likes · 25 min read
Evolution of Data Platform Technology: From Data Warehouse to Lakehouse Architecture
Architects Research Society
Architects Research Society
Jun 25, 2022 · Fundamentals

Data Architecture: Definition, Goals, Principles, Components, and Best Practices

This article explains data architecture as the transformation of business needs into data and system requirements, outlines its objectives, core principles, essential components, the relationship with data modeling, relevant frameworks, and modern best‑practice guidelines for building scalable, cloud‑native, AI‑enabled architectures.

AI integrationCloud NativeData Architecture
0 likes · 10 min read
Data Architecture: Definition, Goals, Principles, Components, and Best Practices
Architecture Digest
Architecture Digest
May 16, 2022 · Databases

Evolution of JD Baitiao’s Data Architecture: From MySQL to Apache ShardingSphere

This article chronicles JD Baitiao’s journey from a monolithic MySQL setup through NoSQL and DBRep to a mature ShardingSphere‑based sharding solution, highlighting the technical motivations, architectural decoupling strategies, evaluation criteria, performance comparison, and the operational benefits achieved for a high‑traffic financial service.

Apache ShardingSphereData Architecturedatabases
0 likes · 14 min read
Evolution of JD Baitiao’s Data Architecture: From MySQL to Apache ShardingSphere
DataFunSummit
DataFunSummit
Apr 9, 2022 · Big Data

Impala Deployment and Optimization: Practical Experience with Sensor Data Multi‑dimensional Analysis Platform

This article presents a comprehensive technical walkthrough of Sensor Data's multi‑dimensional analysis platform, covering product architecture, an Impala‑based real‑time query engine, query performance tuning, resource‑estimation strategies, and future plans, with concrete diagrams, test results, and community contributions.

Big DataData ArchitectureImpala
0 likes · 19 min read
Impala Deployment and Optimization: Practical Experience with Sensor Data Multi‑dimensional Analysis Platform
dbaplus Community
dbaplus Community
Apr 1, 2022 · Databases

How iQIYI Built a Scalable OLTP Data Center to Eliminate Data Silos

This article details iQIYI's design and implementation of a unified OLTP data center that consolidates data across business lines, solves data‑island issues, ensures strong consistency between MongoDB and Elasticsearch, and provides high‑availability, massive‑scale storage for billions of records.

Data ArchitectureElasticsearchMongoDB
0 likes · 12 min read
How iQIYI Built a Scalable OLTP Data Center to Eliminate Data Silos
DataFunTalk
DataFunTalk
Feb 19, 2022 · Big Data

Fundamentals of Data Middle Platform: Logic, Principles, and Practice

This article explains what a data middle platform is, why organizations need it, its core principles, technical architecture, and practical implementation guidelines, highlighting how it solves issues like inconsistent metrics, duplicate data construction, low query efficiency, poor data quality, and high development costs.

Big DataData ArchitectureData Middle Platform
0 likes · 14 min read
Fundamentals of Data Middle Platform: Logic, Principles, and Practice
DataFunSummit
DataFunSummit
Jan 23, 2022 · Big Data

MobTech's Integrated Data Governance Practices and Architecture

This article presents MobTech's comprehensive data governance and security practices, covering the necessity of governance, challenges in large‑scale data environments, the full‑link governance chain, modular architecture, and specific implementations for financial risk‑control scenarios.

Big DataData ArchitectureData Management
0 likes · 19 min read
MobTech's Integrated Data Governance Practices and Architecture
Architecture Digest
Architecture Digest
Jan 21, 2022 · Big Data

Building a Real-Time Data Warehouse with Flink: Architecture, Core Concepts, and Practical Implementation

This article explains how to build a unified stream‑batch real‑time data warehouse using FlinkSQL, covering prerequisite knowledge, five core concepts, two implementation approaches, a comparison of traditional versus real‑time architectures, and a comprehensive hands‑on example, illustrated with diagrams.

Batch ProcessingData ArchitectureFlink
0 likes · 6 min read
Building a Real-Time Data Warehouse with Flink: Architecture, Core Concepts, and Practical Implementation
Architects' Tech Alliance
Architects' Tech Alliance
Dec 26, 2021 · Big Data

Understanding Data Lakes: Concepts, Architecture, Vendor Solutions, and Implementation Practices

This comprehensive article explains what a data lake is, its core characteristics, reference architecture, major cloud vendor implementations, typical use cases such as advertising and gaming, step‑by‑step construction guidance, and future trends in cloud‑native big‑data platforms.

Data ArchitectureData ManagementVendor Solutions
0 likes · 51 min read
Understanding Data Lakes: Concepts, Architecture, Vendor Solutions, and Implementation Practices
High Availability Architecture
High Availability Architecture
Dec 23, 2021 · Fundamentals

Master Data Management Architecture and Practices for Baidu Smart Mini Programs

This article presents a comprehensive overview of master data management concepts, maturity levels, and the challenges faced by Baidu smart mini‑programs, followed by a detailed practical architecture design—including domain modeling, high‑availability microservice implementation, performance optimization, and data synchronization—while also discussing future extensions and team capability building.

Baidu Mini ProgramsData ArchitectureMaster Data Management
0 likes · 14 min read
Master Data Management Architecture and Practices for Baidu Smart Mini Programs
21CTO
21CTO
Nov 8, 2021 · Big Data

How Baidu iFanFan Built a Real-Time Big Data Platform: Challenges & Lessons

Facing rapid business iteration, Baidu’s iFanFan data team designed a unified real‑time and offline big‑data platform, tackling business, technical, and organizational challenges through Lambda/Kappa architectures, data integration, storage, computation, governance, and scalable analytics to deliver timely, accurate, and valuable data products.

Big DataData ArchitectureData Warehouse
0 likes · 33 min read
How Baidu iFanFan Built a Real-Time Big Data Platform: Challenges & Lessons
Big Data Technology & Architecture
Big Data Technology & Architecture
Oct 14, 2021 · Big Data

Overview of Big Data Architecture Trends and Curated Resources

This article, discovered on the Yunqi community site, provides a system‑architecture perspective overview of current big‑data architecture hotspots, development trajectories, emerging trends, and unresolved challenges, while highlighting the field’s rapid evolution and recommending a curated list of in‑depth resources for further study.

Data ArchitectureResourcesdata engineering
0 likes · 5 min read
Overview of Big Data Architecture Trends and Curated Resources
Java High-Performance Architecture
Java High-Performance Architecture
Oct 12, 2021 · Big Data

Unpacking the Core Technologies Behind Modern Big Data Platforms

This article breaks down a typical big data platform architecture into its four layers—data collection, storage and analysis, sharing, and real‑time computation—detailing the essential tools such as Flume, HDFS, Hive, Spark, DataX, and task scheduling systems that enable scalable, low‑latency data processing and delivery.

Big DataData ArchitectureDataX
0 likes · 8 min read
Unpacking the Core Technologies Behind Modern Big Data Platforms
Architecture Digest
Architecture Digest
Oct 11, 2021 · Big Data

Core Technologies and Architecture of a Big Data Platform

This article explains the typical architecture of a big‑data platform, detailing its four core layers—data collection, storage & analysis, data sharing, and application—and describing the key technologies such as Flume, DataX, HDFS, Hive, Spark, Spark Streaming, and task scheduling components.

Big DataData ArchitectureDataX
0 likes · 8 min read
Core Technologies and Architecture of a Big Data Platform
Architects' Tech Alliance
Architects' Tech Alliance
Sep 2, 2021 · Big Data

Core Technologies and Architecture of a Big Data Platform

The article outlines a typical big data platform architecture, detailing its core layers—data collection, storage and analysis, sharing, application, real-time computation, and task scheduling—while describing key technologies such as Flume, DataX, HDFS, Hive, Spark, Spark Streaming, and Redis.

Data ArchitectureHadoopReal-time Processing
0 likes · 9 min read
Core Technologies and Architecture of a Big Data Platform
Alibaba Cloud Developer
Alibaba Cloud Developer
Sep 2, 2021 · Databases

From LAMP to Cloud‑Native: Evolving Application Data Architecture and Best Practices

This article traces two decades of application data architecture evolution, comparing traditional single‑system LAMP designs with modern multi‑component cloud‑native stacks, and offers practical guidance on scaling, component selection, CDC‑based data derivation, and cloud‑native implementations such as Tablestore.

CDCData Architecturedatabases
0 likes · 22 min read
From LAMP to Cloud‑Native: Evolving Application Data Architecture and Best Practices
IT Architects Alliance
IT Architects Alliance
Sep 1, 2021 · Big Data

Understanding Data Middle Platform Architecture and Its Core Components

The article explains the concept of a data middle platform, describing its architecture, the essential big‑data foundation, metadata management, data service components such as BI and tag systems, and how these layers together enable unified data access, governance, and business intelligence across enterprises.

Data ArchitectureTag Managementbusiness intelligence
0 likes · 14 min read
Understanding Data Middle Platform Architecture and Its Core Components
Efficient Ops
Efficient Ops
Aug 31, 2021 · Cloud Computing

Why Object Storage Is the New Backbone of Cloud Data Management

This article explains how object storage emerged as a cloud-native solution that surpasses traditional DAS, SAN, and NAS architectures by offering virtually unlimited capacity, robust metadata handling, and simple RESTful APIs for modern applications and large‑scale data workloads.

Data ArchitectureObject StorageScalability
0 likes · 11 min read
Why Object Storage Is the New Backbone of Cloud Data Management
JD Retail Technology
JD Retail Technology
Aug 12, 2021 · Big Data

Design and Implementation of JD Mini‑Program Custom Data Analysis Service

This article presents the technical solution and key processes of JD's mini‑program custom data analysis service, covering business background, ClickHouse‑based storage design, real‑time processing pipelines, dynamic rule parsing, table architecture, monitoring mechanisms, and future outlook for large‑scale data analytics.

ClickHouseCustom Data AnalysisData Architecture
0 likes · 13 min read
Design and Implementation of JD Mini‑Program Custom Data Analysis Service
Qingyun Technology Community
Qingyun Technology Community
Aug 3, 2021 · Cloud Computing

How QingStor’s Object Storage Architecture Powers Massive Data Scalability

This article explains QingStor's object storage concepts, core advantages, global data model, subsystem design, massive small‑file optimizations, key features like lifecycle management and cross‑region replication, and showcases a traffic‑industry use case, highlighting its scalability, reliability, and ease of integration.

Data ArchitectureObject StorageQingStor
0 likes · 20 min read
How QingStor’s Object Storage Architecture Powers Massive Data Scalability
Airbnb Technology Team
Airbnb Technology Team
Jul 29, 2021 · Big Data

Airbnb’s Data Quality Improvement Plan: Organizational, Architectural, and Governance Practices

Airbnb’s 2019 Data Quality Improvement Plan reorganized its data‑engineering workforce, introduced a dedicated data‑engineer role, adopted a decentralized Minerva‑based architecture with Spark pipelines, instituted rigorous testing, governance, and certification processes, and established SLAs and monitoring to ensure timely, trustworthy, well‑documented data across the enterprise.

AirbnbBig DataData Architecture
0 likes · 13 min read
Airbnb’s Data Quality Improvement Plan: Organizational, Architectural, and Governance Practices
IT Architects Alliance
IT Architects Alliance
Jul 20, 2021 · Big Data

Understanding Data Middle Platform: Layers, Architecture, and Implementation Methodology

The article explains the concept of a data middle platform, detailing its three-layer structure—data model, data service, and data development—illustrates how data modeling enables cross-domain integration, how services encapsulate data for flexible consumption, and how development tools support customized data applications, using a telecom operator example.

Big DataData ArchitectureData Platform
0 likes · 2 min read
Understanding Data Middle Platform: Layers, Architecture, and Implementation Methodology
dbaplus Community
dbaplus Community
Jun 2, 2021 · Databases

How to Build a Mature Data Warehouse: 7 Essential Steps and Best Practices

This article explains why data warehouses are critical for decision‑making, outlines the challenges of immature warehouses, and provides a step‑by‑step framework—including goal setting, technology selection, problem identification, domain modeling, layer design, modeling principles, and governance standards—to help teams build a robust, maintainable data warehouse.

Big DataData ArchitectureData Warehouse
0 likes · 22 min read
How to Build a Mature Data Warehouse: 7 Essential Steps and Best Practices
Big Data Technology & Architecture
Big Data Technology & Architecture
May 27, 2021 · Databases

Database Selection and TiDB Implementation in NetEase Interactive Entertainment Billing Group

This article details the billing group's challenges with single‑node MySQL, the evaluation of alternative databases such as TiDB and CockroachDB, performance testing, migration strategies, operational best practices, and the final decision to adopt TiDB for scalable, high‑availability data services.

Cloud NativeData ArchitectureDistributed SQL
0 likes · 15 min read
Database Selection and TiDB Implementation in NetEase Interactive Entertainment Billing Group
Architects Research Society
Architects Research Society
May 23, 2021 · Big Data

Data Architecture Trends: From Chaos to an Organized Era – Insights from Anthony J. Algmin

The article reviews Anthony J. Algmin’s reflections on past data‑architecture predictions, current hot topics such as cloud, AI/ML, data governance, and real‑time analytics, and forecasts future trends including metadata management, blockchain, and the evolving role of data architects within enterprises.

Artificial IntelligenceBig DataData Architecture
0 likes · 13 min read
Data Architecture Trends: From Chaos to an Organized Era – Insights from Anthony J. Algmin
ITFLY8 Architecture Home
ITFLY8 Architecture Home
May 22, 2021 · Fundamentals

How Meituan Scaled Data Governance: Practical Lessons for Enterprise Data Management

This article outlines Meituan's journey in data governance, detailing the challenges of data quality, cost, security, standardization and efficiency, and presenting a three‑stage roadmap—passive, proactive, and automated governance—along with concrete technical and organizational solutions.

Data ArchitectureInformation Securitydata governance
0 likes · 9 min read
How Meituan Scaled Data Governance: Practical Lessons for Enterprise Data Management
Programmer DD
Programmer DD
May 22, 2021 · Big Data

What Is a Data Lake? Origins, Architecture, and How It Powers Modern Big Data

This article explains the concept of a data lake—its origin in 2011, how it differs from traditional databases and data warehouses, its core characteristics such as raw data storage, on‑demand computing, and schema‑on‑read, as well as its advantages, challenges, architectural components, and future outlook within the big‑data ecosystem.

Big DataData ArchitectureETL
0 likes · 20 min read
What Is a Data Lake? Origins, Architecture, and How It Powers Modern Big Data
Architects Research Society
Architects Research Society
May 15, 2021 · Big Data

Data Warehouse vs Data Lake: Definitions, Differences, and Architectural Considerations

Data warehouses store structured data centrally for reporting and analysis, while data lakes retain raw data in various formats, offering flexible, low‑cost, schema‑on‑read processing; the article explains their definitions, key differences, common misconceptions, and why many organizations now combine both to enable self‑service big‑data analytics.

AnalyticsBig DataData Architecture
0 likes · 21 min read
Data Warehouse vs Data Lake: Definitions, Differences, and Architectural Considerations
Architects Research Society
Architects Research Society
May 9, 2021 · Big Data

Data Lakes vs. Data Warehouses: Key Differences and Choosing the Right Approach

This article explains the fundamental distinctions between data lakes and data warehouses, outlines five critical differences—including data retention, type support, user support, adaptability, and insight speed—and offers guidance on selecting the appropriate solution based on organizational needs and technology options.

AnalyticsBig DataData Architecture
0 likes · 12 min read
Data Lakes vs. Data Warehouses: Key Differences and Choosing the Right Approach
IT Architects Alliance
IT Architects Alliance
Mar 24, 2021 · Fundamentals

How BA, DA, AA, and TA Interlock: A Practical Guide to Enterprise Architecture

This article clarifies the relationships among Business Architecture (BA), Data Architecture (DA), Application Architecture (AA) and Technology Architecture (TA), explains their roles within strategic, business, and solution layers, and walks through a concrete stock‑purchase example to illustrate end‑to‑end design and implementation.

Architecture ProcessData ArchitectureEnterprise Architecture
0 likes · 11 min read
How BA, DA, AA, and TA Interlock: A Practical Guide to Enterprise Architecture
Efficient Ops
Efficient Ops
Mar 17, 2021 · Operations

How CMB’s Real‑Time Business Flow Monitoring Platform Transforms Banking Operations

This article examines CMB’s award‑winning real‑time business‑flow monitoring and operations platform, detailing how map‑and‑navigation‑based architecture transforms banking IT, enhances customer experience, enables fine‑grained intelligent operations, and outlines a future driven by big‑data analytics.

Data ArchitectureFinTechbusiness operations
0 likes · 11 min read
How CMB’s Real‑Time Business Flow Monitoring Platform Transforms Banking Operations
Architects' Tech Alliance
Architects' Tech Alliance
Feb 21, 2021 · Big Data

Data Warehouse and Data Lake: Concepts, Architecture, and Comparison

This article provides an extensive overview of data warehouse and data lake concepts, their architectures, differences, components, and implementation considerations, covering topics such as OLTP/OLAP, ETL processes, data quality, cloud solutions, and the role of data platforms in modern enterprises.

Cloud ComputingData ArchitectureETL
0 likes · 92 min read
Data Warehouse and Data Lake: Concepts, Architecture, and Comparison
Big Data Technology & Architecture
Big Data Technology & Architecture
Jan 20, 2021 · Big Data

Understanding Data Warehouse, Data Lake, and Data Middle Platform: Concepts, Differences, and Applications

This article provides a comprehensive overview of data warehouses, data lakes, and data middle platforms, explaining their definitions, architectures, functions, differences, and the value they bring to enterprises, while also addressing common misconceptions and related concepts such as data marts and data swamps.

Data ArchitectureData Warehousedata lake
0 likes · 37 min read
Understanding Data Warehouse, Data Lake, and Data Middle Platform: Concepts, Differences, and Applications
Suning Technology
Suning Technology
Jan 19, 2021 · Databases

Scaling Citus Clusters via Logical Replication – Lessons from PostgreSQL China Conference

The 10th PostgreSQL China Technical Conference in January 2021 featured expert talks on scaling Citus clusters with logical replication and on applying PostgreSQL to Suning’s retail digitalization, highlighting architecture, challenges, technology choices, and future directions for database-driven business transformation.

CitusData ArchitectureLogical Replication
0 likes · 2 min read
Scaling Citus Clusters via Logical Replication – Lessons from PostgreSQL China Conference
Big Data Technology & Architecture
Big Data Technology & Architecture
Dec 31, 2020 · Big Data

Data Lake vs Data Warehouse: Evolution, Comparison, and Alibaba Cloud Lakehouse Integration

This article examines the 20‑year evolution of big data architectures, contrasts data lakes and data warehouses, explores their respective strengths and challenges, and details Alibaba Cloud’s lake‑warehouse (lakehouse) solution that unifies storage, metadata, and compute for enterprise‑grade analytics and AI workloads.

Cloud ComputingData ArchitectureData Warehouse
0 likes · 30 min read
Data Lake vs Data Warehouse: Evolution, Comparison, and Alibaba Cloud Lakehouse Integration
dbaplus Community
dbaplus Community
Dec 27, 2020 · Big Data

How ClickHouse Powers a 700 B‑Row Real‑Time Data Platform at Ctrip

This article details how Ctrip's senior engineering manager leveraged ClickHouse to build a high‑availability, sub‑second response data platform handling nearly 700 billion rows, describing the motivations, architecture, data synchronization processes, performance gains, challenges, and practical recommendations for large‑scale analytics.

Big DataClickHouseData Architecture
0 likes · 28 min read
How ClickHouse Powers a 700 B‑Row Real‑Time Data Platform at Ctrip
dbaplus Community
dbaplus Community
Nov 26, 2020 · Big Data

Silicon Valley's Data Middle Platform Secrets: EA, Twitter, Airbnb, Uber

This article examines how leading Silicon Valley companies such as EA, Twitter, Airbnb, and Uber design and operate data middle platforms—detailing their architectures, data collection pipelines, standardization efforts, real‑time and batch processing, and the business impact of shared data capabilities.

Big DataData ArchitectureData Platform
0 likes · 25 min read
Silicon Valley's Data Middle Platform Secrets: EA, Twitter, Airbnb, Uber
DataFunSummit
DataFunSummit
Nov 15, 2020 · Big Data

Evolution of 58.com Commercial Data Warehouse: From 0‑1 to 3.0 Using Hadoop, Flume, Kafka, Spark, and Flink

This article details the three‑stage evolution of 58.com’s commercial data warehouse, describing its massive scale, four‑layer architecture, technical challenges, migrations from MapReduce to Hive and Flink, real‑time streaming upgrades, and the resulting improvements in stability, accuracy, and timeliness.

Big DataData ArchitectureData Warehouse
0 likes · 10 min read
Evolution of 58.com Commercial Data Warehouse: From 0‑1 to 3.0 Using Hadoop, Flume, Kafka, Spark, and Flink
Big Data Technology & Architecture
Big Data Technology & Architecture
Aug 15, 2020 · Big Data

Understanding Data Lakes: Concepts, Architecture, Vendor Solutions, and Practical Use Cases

This comprehensive article explains what a data lake is, outlines its core characteristics and reference architecture, compares major cloud providers' data‑lake offerings, presents typical advertising and gaming use cases, and proposes a practical, agile process for building and operating a data lake.

Big DataCloud NativeData Architecture
0 likes · 50 min read
Understanding Data Lakes: Concepts, Architecture, Vendor Solutions, and Practical Use Cases
Efficient Ops
Efficient Ops
Aug 5, 2020 · Cloud Computing

Why Object Storage Is the Next Big Thing in Cloud Computing

This article explains the fundamentals of object storage, compares it with block and file storage, outlines its architecture, components, advantages, use cases, and limitations, showing why it has become the dominant storage model in modern cloud environments.

Data ArchitectureObject StorageScalability
0 likes · 11 min read
Why Object Storage Is the Next Big Thing in Cloud Computing
Big Data and Microservices
Big Data and Microservices
Jun 28, 2020 · Big Data

Data Warehouse vs Data Lake vs Data Platform vs Data Middle Platform: Which Fits Your Business?

This article compares data warehouse, data lake, data platform, and data middle platform, explaining their definitions, architectures, strengths, limitations, and use‑case differences, and provides tables that highlight how each solution handles structured and unstructured data, governance, flexibility, and business value.

Big DataData ArchitectureData Middle Platform
0 likes · 12 min read
Data Warehouse vs Data Lake vs Data Platform vs Data Middle Platform: Which Fits Your Business?
Big Data and Microservices
Big Data and Microservices
Jun 24, 2020 · Industry Insights

What Is a Data Middle Platform and How It Boosts Business Agility

The article explains what a data middle platform is, why it differs from a traditional big‑data platform, the efficiency, collaboration and talent challenges it addresses, its definition as a data‑driven innovation layer built on big data, cloud and AI, and outlines its logical architecture centered on data APIs.

Artificial IntelligenceBig DataCloud Computing
0 likes · 6 min read
What Is a Data Middle Platform and How It Boosts Business Agility
Big Data Technology Architecture
Big Data Technology Architecture
Jun 7, 2020 · Big Data

Comprehensive Overview of Data Lake Concepts, Architectures, Vendor Solutions, and Use Cases

This article provides an in‑depth, English‑language overview of data lakes, covering their definition, core characteristics, reference architectures, major cloud‑vendor implementations (AWS, Huawei, Alibaba Cloud, Azure), typical industry applications such as advertising and gaming, as well as practical guidance on building and evolving a data lake in a cloud‑native, big‑data environment.

AnalyticsData ArchitectureLakehouse
0 likes · 50 min read
Comprehensive Overview of Data Lake Concepts, Architectures, Vendor Solutions, and Use Cases
dbaplus Community
dbaplus Community
Apr 26, 2020 · Big Data

Evolving from Data Warehouses to Data Middle Platforms: Architecture & Practices

This talk reviews China's big‑data evolution from early enterprise data warehouses to modern data middle platforms, outlines core architectural components, technology selections, data development practices, lifecycle and quality management, and shares practical Q&A insights for building scalable, cost‑effective data infrastructures.

Big DataData ArchitectureData Middle Platform
0 likes · 28 min read
Evolving from Data Warehouses to Data Middle Platforms: Architecture & Practices
ITPUB
ITPUB
Apr 6, 2020 · Big Data

How to Build a Data Lake Quickly: Strategies, Tools, and Real‑World Cases

This article explains the origins and market growth of data lakes, compares them with traditional data warehouses, showcases major implementations like Amazon Galaxy and Club Factory, and provides practical guidance on choosing open‑source or commercial cloud solutions to construct a data lake efficiently while minimizing risk.

Big DataCloud ComputingData Architecture
0 likes · 10 min read
How to Build a Data Lake Quickly: Strategies, Tools, and Real‑World Cases
Meituan Technology Team
Meituan Technology Team
Mar 12, 2020 · Big Data

Data Governance Practices in Meituan Delivery: Architecture, Standards, and Security

Meituan Delivery’s data‑governance framework combines a four‑layer warehouse architecture with comprehensive business, technical, security, and resource‑management standards, continuous metadata and security controls, and tools such as Wherehows and QuickSight, delivering standardized, secure, and easily shareable data while guiding future optimization and emerging‑technology adoption.

Big DataData ArchitectureData Security
0 likes · 27 min read
Data Governance Practices in Meituan Delivery: Architecture, Standards, and Security
Tencent Cloud Developer
Tencent Cloud Developer
Feb 13, 2020 · Big Data

Data Middle Platform: Vision, Architecture, and Business Value

The Data Middle Platform, described by Shi Kai, is a service‑oriented architecture that transforms raw enterprise data into reusable, real‑time APIs for business applications, bridging the gap between traditional warehouses and front‑end systems, accelerating digital transformation through unified governance, rapid development, and direct business value.

Big DataData ArchitectureData Middle Platform
0 likes · 26 min read
Data Middle Platform: Vision, Architecture, and Business Value
Youzan Coder
Youzan Coder
Jan 8, 2020 · Cloud Native

Youzan Retail Finance Middle Platform Architecture Design and Practice

The article outlines Youzan's retail finance middle platform design, detailing the business background of complex SaaS retail, a structured analysis using layered business, application, data, and technology architectures, and a step‑by‑step implementation that emphasizes reusable domain capabilities and long‑term, standardized middle‑platform development.

Data ArchitectureDomain-Driven DesignEnterprise Architecture
0 likes · 20 min read
Youzan Retail Finance Middle Platform Architecture Design and Practice
Architecture Digest
Architecture Digest
Dec 24, 2019 · Big Data

Design Architecture and Technical Strategies for Big Data Products

This article systematically outlines the architecture and technical strategy of big‑data product design, detailing a five‑step process from front‑end data collection and ETL to data warehousing, modeling, algorithm design, and personalized user‑centric delivery, while highlighting common platform challenges and future deep‑learning enhancements.

Data ArchitectureETLuser profiling
0 likes · 14 min read
Design Architecture and Technical Strategies for Big Data Products
Programmer DD
Programmer DD
Dec 11, 2019 · Big Data

Big Data Architecture Secrets: Storage-Compute Separation & Spark in Action

This article explores how enterprises can tackle the explosive growth of data by adopting modern big‑data architectures, including storage‑compute separation, data‑driven workflows, risk‑control frameworks, and real‑world Spark optimizations, offering practical guidance for scalable, high‑performance analytics.

Big DataData ArchitectureSpark
0 likes · 12 min read
Big Data Architecture Secrets: Storage-Compute Separation & Spark in Action
UCloud Tech
UCloud Tech
Dec 4, 2019 · Big Data

How to Evolve Big Data Architectures for ZB‑Scale Analytics and Real‑World Use Cases

This article reviews the challenges of handling Zettabyte‑scale data, outlines practical big‑data processing architectures, discusses storage‑compute separation, data‑driven workflows, risk‑control frameworks, and shares concrete Spark implementations at MobTech, offering actionable insights for modern data engineers.

Data ArchitectureSparkStorage Compute Separation
0 likes · 13 min read
How to Evolve Big Data Architectures for ZB‑Scale Analytics and Real‑World Use Cases
21CTO
21CTO
Nov 27, 2019 · Big Data

How Xiaohongshu Scales Real‑Time Personalized Recommendations with Flink

The article summarizes Guo Yi’s 2019 Alibaba Cloud conference talk, outlining Xiaohongshu’s personalized recommendation architecture, detailing the data stack from ingestion to warehouse, and showcasing a Flink‑based real‑time multi‑dimensional user behavior aggregation use case, followed by a vision for the next year’s data architecture evolution.

Data ArchitectureFlinkReal-time Streaming
0 likes · 3 min read
How Xiaohongshu Scales Real‑Time Personalized Recommendations with Flink
Architecture Digest
Architecture Digest
Nov 5, 2019 · Big Data

Architecture Overview of Taobao, Meituan, and Didi Big Data Platforms

This article examines the big‑data architectures of three leading Chinese internet companies—Taobao, Meituan, and Didi—detailing their data sources, synchronization mechanisms, batch and streaming processing layers, and the common scheduling components that unify their Hadoop‑based ecosystems.

Big DataData ArchitectureDidi
0 likes · 7 min read
Architecture Overview of Taobao, Meituan, and Didi Big Data Platforms
Big Data Technology & Architecture
Big Data Technology & Architecture
Oct 28, 2019 · Big Data

Big Data Technology and Architecture: Leveraging Spark and HBase for Real‑Time and Offline Processing

This article outlines the challenges of various big‑data scenarios such as financial risk control, recommendation systems, and social feeds, explains why Spark is chosen over alternatives, describes a one‑stop data platform architecture with Spark‑HBase integration, and shares best‑practice tips and case studies.

Big DataData ArchitectureHBase
0 likes · 7 min read
Big Data Technology and Architecture: Leveraging Spark and HBase for Real‑Time and Offline Processing
dbaplus Community
dbaplus Community
Oct 27, 2019 · Product Management

What Skills Do Data Product Managers Need in a Data Middle Platform?

The article explains the concept of a data middle platform, why it matters for rapid demand response and resource integration, and outlines the distinct responsibilities and required skill sets of data product managers and data platform product managers within such ecosystems.

Data ArchitectureData Middle PlatformData Product
0 likes · 11 min read
What Skills Do Data Product Managers Need in a Data Middle Platform?
dbaplus Community
dbaplus Community
Oct 22, 2019 · Big Data

How Weibo Built a Billion‑Log Real‑Time Data Platform with Flink

This article details how Weibo’s advertising team designed and implemented a real‑time data platform capable of processing over a hundred billion daily logs, covering technology selection, Flink advantages, architecture evolution, data processing pipelines, component libraries, fault‑tolerance strategies, and the construction of a multi‑layer real‑time data warehouse.

Big DataCheckpointData Architecture
0 likes · 25 min read
How Weibo Built a Billion‑Log Real‑Time Data Platform with Flink
Architects' Tech Alliance
Architects' Tech Alliance
Oct 17, 2019 · Big Data

Understanding Alibaba's Data Middle Platform: Concepts, Architecture, and Differences from Data Warehouses and Data Lakes

The article explains Alibaba's data middle platform—its definition, methodology, organizational structure, key tools, and how it differs from traditional data warehouses and data lakes—while highlighting its role in supporting scalable, business‑centric data services and digital transformation.

AlibabaBig DataData Architecture
0 likes · 16 min read
Understanding Alibaba's Data Middle Platform: Concepts, Architecture, and Differences from Data Warehouses and Data Lakes
Meituan Technology Team
Meituan Technology Team
Oct 17, 2019 · Big Data

OneData Methodology: Building a Unified Data Warehouse Architecture and Governance Framework

By adapting Alibaba’s OneData methodology, the project establishes a unified data‑warehouse architecture, standards, and governance framework—including consolidated business intake, standardized design layers, naming conventions, and delivery metrics—that resolves data‑quality issues, enhances scalability and reusability, and delivers faster, reliable data support for evolving business needs.

Big DataData ArchitectureData Warehouse
0 likes · 15 min read
OneData Methodology: Building a Unified Data Warehouse Architecture and Governance Framework
iQIYI Technical Product Team
iQIYI Technical Product Team
Sep 12, 2019 · Big Data

iQIYI's Big Data Architecture Evolution and Adoption of Druid

iQIYI upgraded its big‑data stack by adopting Druid as the core engine for free‑time queries and ElasticSearch for pre‑computed fixed‑time queries, overcoming early API, security and scaling challenges through monthly segment granularity, parallel sub‑queries, Redis caching and failover, cutting typical query latency from over two seconds to about 150 ms and reaching 99.9 % service success.

Bitmap IndexData ArchitectureElasticsearch
0 likes · 12 min read
iQIYI's Big Data Architecture Evolution and Adoption of Druid
Alibaba Cloud Developer
Alibaba Cloud Developer
Sep 4, 2019 · Big Data

How Structured Big Data Storage Powers Modern Data Systems

This article explores the core components of data systems, the evolution toward lightweight, intelligent big data architectures, the distinction between primary and secondary storage, challenges of data replication, and how Alibaba Cloud's Tablestore implements advanced features such as storage‑compute separation, CDC, and multi‑model indexing for scalable, cost‑effective structured big data storage.

Big DataCDCCloud Services
0 likes · 24 min read
How Structured Big Data Storage Powers Modern Data Systems
37 Interactive Technology Team
37 Interactive Technology Team
Mar 28, 2019 · Big Data

Approaches to Building a Basic Data Platform

To handle terabytes of daily data and diverse business needs, the company built a three‑layer basic data platform—collection/computation/storage, unified data management, and API‑driven services—augmented by a standardized collection system, a robust Domino scheduler, and a self‑service analysis tool, aiming to evolve into a full data‑middle‑office for end‑to‑end intelligence.

Data ArchitectureSchedulingSelf-Service Analytics
0 likes · 8 min read
Approaches to Building a Basic Data Platform
DataFunTalk
DataFunTalk
Feb 18, 2019 · Big Data

Hulu’s Big Data Architecture and Sophon OLAP Cache Layer Overview

This article presents an in‑depth overview of Hulu’s big‑data platform, detailing its multi‑layer architecture, the design and functionality of the Sophon OLAP cache layer, and how Impala is employed for high‑performance query processing and integration with cloud‑native engines.

Data ArchitectureHuluImpala
0 likes · 16 min read
Hulu’s Big Data Architecture and Sophon OLAP Cache Layer Overview
JD Tech
JD Tech
Jan 28, 2019 · Big Data

Technical Overview of JD Marketing 360: 4A Consumer Asset Model, 4E Marketing Framework, and Big Data Architecture

The article presents a comprehensive technical analysis of JD's Marketing 360 project, detailing the three industry pain points, the 4A consumer‑asset model, the 4E marketing methodology, and the underlying big‑data, AI‑driven architecture that enables real‑time analytics, multi‑modal model training, and performance optimizations.

Artificial IntelligenceCustomer Asset ModelData Architecture
0 likes · 14 min read
Technical Overview of JD Marketing 360: 4A Consumer Asset Model, 4E Marketing Framework, and Big Data Architecture
ITPUB
ITPUB
Oct 23, 2018 · Big Data

How Meituan Built a Scalable Real‑Time Data Warehouse with Flink

This article explains how Meituan tackled growing real‑time data demands by redesigning its streaming platform, adopting a layered real‑time data warehouse architecture, selecting storage and compute technologies such as Cellar, Elasticsearch, Druid and Flink, and sharing practical tips on dimension expansion, joins, and aggregation to achieve higher throughput and lower latency.

Data ArchitectureFlinkMeituan
0 likes · 15 min read
How Meituan Built a Scalable Real‑Time Data Warehouse with Flink
DataFunTalk
DataFunTalk
Sep 9, 2018 · Big Data

Druid Principles and Their Application in Insurance Data Analytics

This article summarizes a presentation by Ping An Insurance data engineers on Druid’s architecture, core concepts, node roles, tuning strategies, and real-world deployment for insurance analytics, illustrating how Druid enables sub‑second, high‑cardinality OLAP queries and supports both real‑time and batch processing.

Data ArchitectureDruidInsurance
0 likes · 11 min read
Druid Principles and Their Application in Insurance Data Analytics
Big Data and Microservices
Big Data and Microservices
Sep 4, 2018 · Big Data

Exploring Five Big Data Architectures—from Traditional to Unified AI Designs

The article examines the evolution of big‑data processing by comparing five prevalent architectures—traditional Hadoop‑based stacks, streaming‑only designs, Kappa, Lambda, and the unified Unifield model—highlighting their strengths, weaknesses, and suitable scenarios while discussing the limitations of classic BI systems and the role of distributed storage, computation, and machine‑learning integration.

Big DataData ArchitectureHadoop
0 likes · 14 min read
Exploring Five Big Data Architectures—from Traditional to Unified AI Designs
Architects' Tech Alliance
Architects' Tech Alliance
Jul 27, 2018 · Backend Development

Designing Data Architecture for Microservices: Principles, Patterns, and Database Choices

This article explains how to design data architecture for microservice systems, covering microservice fundamentals, advantages, decoupling, lightweight APIs, continuous delivery, database per service versus shared databases, polyglot persistence, scaling dimensions, sharding strategies, and why MongoDB is a suitable choice.

Data ArchitectureMongoDBScalability
0 likes · 15 min read
Designing Data Architecture for Microservices: Principles, Patterns, and Database Choices
58 Tech
58 Tech
Jun 27, 2018 · Big Data

Overview of the 58 User Profile System Architecture and Data Processing

The article describes the design, data integration, ID mapping, tag generation, and application scenarios of the 58 user profiling platform, which aggregates billions of user IDs across multiple business lines to provide online and offline persona data for personalization, analytics, and AI modeling.

Big DataData ArchitectureID-Mapping
0 likes · 12 min read
Overview of the 58 User Profile System Architecture and Data Processing
High Availability Architecture
High Availability Architecture
May 21, 2018 · Big Data

Interview with Baidu’s Chief Big Data Architect Ma Ruyue on OLAP, HTAP, and Emerging Big Data Technologies

In this interview, Baidu’s senior big‑data architect Ma Ruyue discusses his career transition from Hadoop to online databases, the design philosophy behind Baidu’s Palo ROLAP system, the future of HTAP, and his views on the evolving big‑data ecosystem including Spark, AI, and containerization.

Data ArchitectureDistributed computingHTAP
0 likes · 11 min read
Interview with Baidu’s Chief Big Data Architect Ma Ruyue on OLAP, HTAP, and Emerging Big Data Technologies
Architecture Digest
Architecture Digest
Mar 27, 2018 · Backend Development

Data Architecture Design in Microservice Development

This article explains the multi‑layer data architecture design for microservice systems, covering concepts such as data usability, primary and secondary data decoupling, sharding, multi‑source data adaptation and caching, and introduces data marts to improve scalability and maintainability.

Data ArchitectureData CachingData Mart
0 likes · 10 min read
Data Architecture Design in Microservice Development
Alibaba Cloud Developer
Alibaba Cloud Developer
Nov 3, 2017 · Big Data

How Alibaba Built an EB-Scale, Real-Time Big Data Platform

Alibaba’s senior data expert Yao Bin Hui explains how the company constructed a standardized, end-to-end big-data ecosystem—from low-level data collection and AI algorithms to data services and product platforms—enabling petabyte-scale integration and second-level response times that power both internal operations and millions of external users.

AlibabaBig DataData Architecture
0 likes · 10 min read
How Alibaba Built an EB-Scale, Real-Time Big Data Platform
21CTO
21CTO
Jul 3, 2017 · Big Data

Inside the World’s Best Data Architectures: Netflix, Facebook, Airbnb, Pinterest

This article explores the cutting‑edge data pipelines of Netflix, Facebook, Airbnb and Pinterest, detailing the massive event volumes they handle, the core technologies such as Kafka, Spark, Presto and Hadoop, and how these giants design scalable, real‑time analytics infrastructures.

AirbnbBig DataData Architecture
0 likes · 6 min read
Inside the World’s Best Data Architectures: Netflix, Facebook, Airbnb, Pinterest
Architecture Digest
Architecture Digest
May 25, 2017 · Big Data

Designing Data Warehouse Layers: Principles, Models, and Practical Practices

This article explains why data warehouses should be layered, describes the classic ODS‑DW‑APP model, details each layer’s purpose and implementation techniques, presents an improved layering scheme with dimension and temporary tables, and answers common questions about parallel DWS and DWD processing.

Big DataData ArchitectureData Warehouse
0 likes · 17 min read
Designing Data Warehouse Layers: Principles, Models, and Practical Practices
dbaplus Community
dbaplus Community
Jan 8, 2017 · Big Data

How to Build a Cost‑Effective Data Platform for Small‑to‑Medium Enterprises

This article explains why data platforms are essential for modern SMEs, defines what a data platform is, outlines a four‑step methodology (source definition, analysis theme, ETL processing, and reporting), and shares architectural choices, team structures, common pitfalls, and practical advice for rapid, iterative implementation.

Data ArchitectureData PlatformData Warehouse
0 likes · 15 min read
How to Build a Cost‑Effective Data Platform for Small‑to‑Medium Enterprises
Tencent Cloud Developer
Tencent Cloud Developer
Jan 6, 2017 · Game Development

Challenges and Design Considerations for Game Server Data Systems

Game server development suffers from generic client‑communication tools and inadequate data stores, leading to duplicated, latency‑heavy code, so a purpose‑built, memory‑resident distributed cache that persists locally and eliminates serialization boiler‑plate is essential for real‑time, low‑latency gameplay.

CachingData Architecturegame server
0 likes · 12 min read
Challenges and Design Considerations for Game Server Data Systems
Liulishuo Tech Team
Liulishuo Tech Team
Dec 16, 2016 · Cloud Computing

Key Takeaways from AWS re:Invent 2016: New Services, Data Architecture, and Operational Insights

The article shares a comprehensive recap of AWS re:Invent 2016, highlighting over a thousand new features across compute, storage, networking, security and tools, discussing serverless concepts, the Modern Data Architecture framework, and practical lessons learned by the English‑Fluent‑Talk engineering team.

Cloud ComputingData ArchitectureServerless
0 likes · 11 min read
Key Takeaways from AWS re:Invent 2016: New Services, Data Architecture, and Operational Insights
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Dec 13, 2016 · Big Data

Umeng’s Mobile Big Data Platform: Architecture, Challenges & Insights

The article details Umeng’s mobile big‑data platform architecture, describing its Lambda‑style hybrid design, data ingestion pipeline with dual Kafka clusters, offline and real‑time processing using Hadoop, Spark, Storm, and storage layers such as HDFS, HBase, MongoDB and Elasticsearch, while also discussing challenges in data collection, cleaning, computation, security, and value‑added services.

Data ArchitectureHadoopKafka
0 likes · 13 min read
Umeng’s Mobile Big Data Platform: Architecture, Challenges & Insights
StarRing Big Data Open Lab
StarRing Big Data Open Lab
Nov 18, 2016 · Big Data

Unveiling Modern Big Data Architecture: Key Technologies and Trends

This article reviews a comprehensive big‑data lecture covering traditional databases, Hadoop ecosystems, commercial big‑data platforms, computing models, analysis techniques, visualization, and leading vendors, highlighting how these technologies shape today’s data‑driven enterprises.

Big DataData AnalysisData Architecture
0 likes · 14 min read
Unveiling Modern Big Data Architecture: Key Technologies and Trends