Distributed Streaming

Big Data Technology Architecture

Mar 25, 2025 · Big Data

Kafka 4.0 Release: KRaft Architecture, Consumer Group Optimizations, and New Queue Features

Kafka 4.0 marks a milestone release that replaces ZooKeeper with the KRaft consensus engine, improves scalability and performance, introduces a server‑side consumer‑group protocol, adds shared‑group queue capabilities, and updates Java requirements and documentation, delivering a more robust and flexible streaming platform.

Distributed StreamingJava11KRaft

0 likes · 6 min read

Kafka 4.0 Release: KRaft Architecture, Consumer Group Optimizations, and New Queue Features

JD Tech

Nov 7, 2024 · Big Data

JDQ Kafka Bandwidth Control and Rate Limiting Architecture

This article analyzes the limitations of native Apache Kafka rate‑limiting, introduces JDQ's multi‑dimensional, dynamic, priority‑aware bandwidth control framework, and demonstrates its effectiveness through partition‑level, single‑machine, and hierarchical elastic throttling experiments for large‑scale e‑commerce data streams.

Distributed StreamingElastic ThrottlingKafka

0 likes · 18 min read

JDQ Kafka Bandwidth Control and Rate Limiting Architecture

JD Tech

Jun 16, 2023 · Big Data

Comprehensive Introduction to Apache Kafka: Architecture, Features, and Best Practices

This article provides a detailed overview of Apache Kafka, covering its distributed streaming architecture, storage mechanisms, replication, consumer groups, compression techniques, exactly‑once semantics, configuration tips, and performance optimizations for building reliable high‑throughput data pipelines.

Distributed StreamingKafkaMessage Queue

0 likes · 19 min read

Comprehensive Introduction to Apache Kafka: Architecture, Features, and Best Practices

IT Architects Alliance

Feb 13, 2022 · Big Data

Comprehensive Overview of Apache Kafka Architecture and Core Concepts

This article provides an in‑depth introduction to Apache Kafka, covering its distributed streaming platform fundamentals, message‑queue models, topic and partition design, broker and cluster roles, producer partitioning logic, reliability guarantees, consumer group assignors, offset management, and performance optimizations such as sequential disk writes and zero‑copy techniques.

Apache KafkaDistributed StreamingMessage Queues

0 likes · 25 min read

Comprehensive Overview of Apache Kafka Architecture and Core Concepts

IT Architects Alliance

Dec 3, 2021 · Big Data

Comprehensive Overview of Apache Kafka Architecture and Core Concepts

This article provides an in‑depth technical guide to Apache Kafka, covering its distributed streaming architecture, core concepts such as topics, partitions, brokers, producers and consumers, reliability guarantees, storage mechanisms, configuration parameters, and consumer assignment strategies, supplemented with Java code examples.

Apache KafkaConsumerDistributed Streaming

0 likes · 24 min read

Java Architect Essentials

Jun 15, 2021 · Big Data

Comprehensive Guide to Apache Kafka: Concepts, Installation, Configuration, and Usage

This article provides a thorough overview of Apache Kafka, covering its core streaming concepts, key components such as topics, partitions, producers and consumers, common use cases, step‑by‑step installation and multi‑broker configuration, fault‑tolerance testing, and an introduction to Kafka Connect for data import/export.

ConsumerDistributed StreamingInstallation

0 likes · 24 min read

Comprehensive Guide to Apache Kafka: Concepts, Installation, Configuration, and Usage

Top Architect

May 26, 2021 · Big Data

Comprehensive Introduction to Apache Kafka: Concepts, Architecture, Installation, and Usage

This article provides a comprehensive guide to Apache Kafka, covering its core concepts, architecture, key APIs, topics and partitions, deployment steps, multi‑broker clustering, fault tolerance, and data integration using Kafka Connect, with detailed command‑line examples.

ConsumerDistributed StreamingInstallation

0 likes · 26 min read

Comprehensive Introduction to Apache Kafka: Concepts, Architecture, Installation, and Usage

vivo Internet Technology

Jan 13, 2021 · Big Data

An Introduction to Apache Kafka: Architecture, Concepts, and Operations

Apache Kafka is a distributed, fault‑tolerant streaming platform that uses broker clusters, topic partitions, and replicated logs to provide publish/subscribe and queue messaging, configurable retention, strong ordering within partitions, producer acknowledgments, consumer groups, and performance optimizations such as batching and zero‑copy for real‑time data pipelines.

Apache KafkaConsumerDistributed Streaming

0 likes · 26 min read

An Introduction to Apache Kafka: Architecture, Concepts, and Operations

Big Data Technology Architecture

Jul 12, 2019 · Big Data

Why Kafka Is So Popular: Features, Use Cases, and Architecture Overview

This article explains why Apache Kafka has become a cornerstone of modern big‑data pipelines by detailing its high‑throughput, fault‑tolerant publish‑subscribe architecture, real‑time processing capabilities, extensive language support, scalability mechanisms, and the wide range of use cases adopted by leading enterprises.

Distributed StreamingKafkaMessage Queue

0 likes · 9 min read

Why Kafka Is So Popular: Features, Use Cases, and Architecture Overview

System Architect Go

Apr 11, 2019 · Big Data

Introduction to Apache Kafka: Core Concepts, Message Delivery, Partition Storage, and Consumption

This article introduces Apache Kafka as a distributed streaming platform, explaining its three core capabilities, key concepts such as producers, topics, brokers, partitions and consumers, and detailing how messages are delivered, stored in partitions, and consumed by consumer groups.

Distributed StreamingKafkaMessage Queue

0 likes · 8 min read

Introduction to Apache Kafka: Core Concepts, Message Delivery, Partition Storage, and Consumption

Architecture Digest

Aug 29, 2017 · Big Data

Introduction to Apache Kafka: Concepts, Architecture, and Core APIs

This article provides a comprehensive overview of Apache Kafka, explaining its role in real‑time data pipelines and stream processing, describing key concepts such as topics, partitions, logs, producers, consumers, replication, guarantees, and how Kafka functions as both a messaging and storage system.

Consumer APIDistributed StreamingKafka

0 likes · 13 min read

Introduction to Apache Kafka: Concepts, Architecture, and Core APIs