BestHub
search
Sign in
Tag

Off-heap

1 views collected around this technical thread.

Beike Product & Technology
Beike Product & Technology
Jul 1, 2021 · Big Data

Oak Off‑Heap Key‑Value Map and Its Application in Apache Druid for Real‑Time and Batch Ingestion

The article introduces Oak, an off‑heap concurrent key‑value map, explains its design and performance benefits over ConcurrentSkipListMap, and details extensive offline and real‑time ingestion experiments in Apache Druid that demonstrate reduced memory usage, lower CPU consumption, and faster data loading.

Apache DruidIncremental IndexOak
0 likes · 10 min read
Oak Off‑Heap Key‑Value Map and Its Application in Apache Druid for Real‑Time and Batch Ingestion
Big Data Technology Architecture
Big Data Technology Architecture
Apr 16, 2020 · Databases

Memory Management Optimizations in HBase MemStore: SkipList, MemStoreLAB, ChunkPool, Off‑heap and CCSMap

The article systematically explains how HBase's MemStore uses a SkipList‑based model and introduces successive memory‑management optimizations—MemStoreLAB, ChunkPool, off‑heap chunks, CompactingMemStore and the CCSMap data structure—to reduce object overhead, GC pressure and improve throughput.

HBaseMemory OptimizationMemstore
0 likes · 20 min read
Memory Management Optimizations in HBase MemStore: SkipList, MemStoreLAB, ChunkPool, Off‑heap and CCSMap
Aikesheng Open Source Community
Aikesheng Open Source Community
Oct 10, 2019 · Databases

Source Code Analysis of DBLE Memory Management Module

This article provides a detailed analysis of DBLE's memory management module, explaining its structure, configuration parameters, allocation and recycling logic, and includes annotated source code snippets for classes such as DirectByteBufferPool and ByteBufferPage, illustrating how off‑heap and on‑heap memory are handled.

ByteBufferDBLEOff-heap
0 likes · 13 min read
Source Code Analysis of DBLE Memory Management Module
Big Data Technology Architecture
Big Data Technology Architecture
Aug 19, 2019 · Big Data

Understanding Spark Unified Memory Management and Dynamic Allocation

This article explains Apache Spark's memory architecture, covering the shift from static to unified memory management, the roles of on‑heap and off‑heap memory, configurable parameters, dynamic memory sharing between execution and storage, and the legacy mode introduced in Spark 1.6.

Apache SparkExecutorOff-heap
0 likes · 7 min read
Understanding Spark Unified Memory Management and Dynamic Allocation
Big Data Technology Architecture
Big Data Technology Architecture
Apr 27, 2019 · Big Data

Understanding Spark Memory Management: On‑Heap and Off‑Heap Planning and Allocation

This article explains Spark's memory management architecture, covering on‑heap and off‑heap memory planning, the MemoryManager interface, static versus unified memory allocation strategies, and how dynamic borrowing improves resource utilization for Spark executors.

Off-heapOn-HeapSpark
0 likes · 11 min read
Understanding Spark Memory Management: On‑Heap and Off‑Heap Planning and Allocation
Qunar Tech Salon
Qunar Tech Salon
Apr 9, 2018 · Big Data

Analysis of Apache Spark 2.2.1 Memory Management Model

This article examines Spark's unified memory manager in version 2.2.1, detailing on‑heap and off‑heap memory regions, the four on‑heap memory pools, dynamic execution‑storage memory sharing, task memory accounting, and provides concrete calculation examples to explain UI discrepancies and runtime memory limits.

ExecutorOff-heapPerformance Tuning
0 likes · 13 min read
Analysis of Apache Spark 2.2.1 Memory Management Model
Vipshop Quality Engineering
Vipshop Quality Engineering
May 31, 2017 · Backend Development

How to Detect and Fix Netty Direct Memory Leaks in Janus Service

This article explains how a Janus service gateway ran out of memory due to unreleased Netty off‑heap ByteBufs, how enabling Netty's leak detection and adjusting JVM parameters helped reproduce the issue, and how to fix the leak by explicitly releasing the buffers.

Leak DetectionNettyOff-heap
0 likes · 5 min read
How to Detect and Fix Netty Direct Memory Leaks in Janus Service
Architect
Architect
May 25, 2016 · Big Data

How Flink Manages Memory to Overcome JVM Limitations

The article explains how Flink tackles JVM memory challenges by using proactive memory management, a custom serialization framework, cache‑friendly binary operations, and off‑heap memory techniques to reduce GC pressure, avoid OOM, and improve performance in big‑data workloads.

FlinkOff-heapSerialization
0 likes · 17 min read
How Flink Manages Memory to Overcome JVM Limitations