1 views collected around this technical thread.
This article examines the typical reasons Kafka messages are lost across producers, brokers, and consumers, and provides detailed configuration recommendations and best‑practice solutions to significantly reduce the risk of data loss in distributed streaming systems.
This article explains how RocketMQ implements long‑polling for message consumption, detailing the pull‑based model, the broker and consumer timeout settings, the internal suspension of pull requests, and the processing loop that resumes suspended requests to improve efficiency.
The article thoroughly explains RocketMQ’s architecture—including stateless NameServers for routing, master‑slave Brokers with sequential CommitLog storage, stateless Producers and pull‑or‑push Consumers, message flow, indexing, and synchronous double‑write reliability—showing how these components deliver high‑throughput, scalable, fault‑tolerant messaging for distributed systems.
This article analyzes the challenges of scaling automated quality‑inspection hardware, introduces IoT fundamentals, evaluates application‑layer protocols, MQTT broker options, QoS levels, and compares cloud, local, and edge deployment architectures to recommend a low‑latency, high‑reliability solution for the smart inspection platform.
This article presents a detailed case study of RocketMQ's fast‑failure mechanism, explains the root causes of a massive message‑write surge, analyzes broker thread models, and proposes topic‑level rate‑limiting solutions with concrete implementation and code examples.
This article explains Kafka's message delivery semantics, analyzes potential data‑loss scenarios across producer, broker, and consumer components, and provides concrete configuration and coding practices—such as idempotent producers, proper ACK settings, replication factors, and manual offset commits—to maximize message durability and reliability.
This article provides a comprehensive overview of Kafka’s core components—including brokers, topics, partitions, producers, and consumers—explaining their roles, relationships, replication, leader/follower dynamics, and architectural diagrams, while also highlighting configuration details and operational considerations for building reliable distributed streaming systems.
This article provides a detailed overview of message queue fundamentals, covering producers, consumers, brokers, queue and topic models, delivery guarantees, acknowledgment mechanisms, ordering, persistence, high availability, and selection criteria, helping readers quickly grasp essential MQ concepts for distributed systems.
This article explains how RocketMQ reduces message loss and duplicate consumption through synchronous sending, retry mechanisms, multi‑master brokers, synchronous disk flushing, master‑slave replication, at‑least‑once delivery, and idempotent processing techniques.
This article provides a comprehensive overview of Kafka's core components—including brokers, topics, partitions, producers, and consumers—explaining their roles, how they interact within a Kafka cluster, and illustrating the architecture with diagrams while also noting recent version changes and related disclaimer information.
This article provides a comprehensive overview of Kafka's core components—including brokers, topics, partitions, producers, and consumers—explaining their roles, relationships, replication strategies, and how they work together within a Kafka cluster to ensure reliable, scalable message streaming.
This article thoroughly analyzes Kafka broker's high‑throughput network architecture, tracing its evolution from simple sequential handling to a multi‑selector Reactor model, detailing Acceptor and Processor thread implementations, request‑handling pipelines, and practical tuning parameters for optimal performance.
The Apache Pulsar 2.10.0 release introduces automatic cluster failover, lazy‑loading producers, new TableView support, enhanced broker interceptors, enriched client authentication, Etcd metadata storage, and numerous bug fixes, offering developers and operators a more flexible and performant messaging platform.
mica‑mqtt, built on t‑io, offers a simple, low‑latency, high‑performance open‑source MQTT component that integrates easily into existing services, supports multiple protocols and deployment scenarios, and includes Spring Boot starter, GraalVM compilation, and Redis‑based clustering for robust IoT solutions.
This article explains how Apache Pulsar brokers track consumer acknowledgment using cursors, describes persistent and non‑persistent subscription differences, details cursor metadata stored in ZooKeeper and BookKeeper, and outlines optimizations for handling message gaps, including RangeSet improvements and a new LRU‑based storage design.
RSocket is a binary, back‑pressure‑aware protocol that supports four interaction models—fire‑and‑forget, request‑response, request‑stream, and request‑channel—using multiplexed frames with odd/even stream IDs for true bidirectional communication, and provides a reactive, resumable API suitable for microservices, IoT, and broker‑based architectures.
This article provides a comprehensive overview of Knative Eventing's Broker and Trigger model, detailing background concepts, event routing patterns, data‑plane and control‑plane workflows, and includes practical YAML and command‑line examples for deploying and managing the components in a Kubernetes environment.
This article explains why Kafka can lose messages at the broker, producer, and consumer levels, describes the underlying mechanisms such as page cache flushing and acknowledgment settings, and provides practical code examples and mitigation strategies to improve reliability.
The article thoroughly explains Kafka’s message‑queue reliability design and implementation, covering use‑case scenarios, core concepts, storage format, producer acknowledgment settings, broker replication mechanisms (ISR, HW, LEO), consumer delivery semantics, the epoch solution for synchronization, and practical configuration guidelines for various consistency and availability requirements.
This article documents the supported features, deprecated services, and specific limitations of a range of MQTT brokers, providing a concise reference for developers to understand each broker's capabilities and constraints within IoT and backend messaging systems.