Tag

Unified Memory

0 views collected around this technical thread.

Data Thinking Notes

Oct 31, 2022 · Big Data

Mastering Spark’s Unified Memory Management: A Deep Dive into On‑Heap & Off‑Heap Tuning

This article explains Spark's unified memory manager, detailing on‑heap and off‑heap memory regions, dynamic memory sharing, task memory allocation, and practical tuning techniques to optimize performance and avoid common out‑of‑memory errors.

Memory ManagementPerformance TuningSpark

0 likes · 13 min read

Mastering Spark’s Unified Memory Management: A Deep Dive into On‑Heap & Off‑Heap Tuning

Full-Stack Internet Architecture

Jul 17, 2020 · Big Data

Understanding Apache Spark Unified (Dynamic) Memory Management

This article explains Spark's transition from static to unified memory management, detailing on‑heap and off‑heap memory regions, key configuration parameters, dynamic allocation behavior, and legacy mode, helping users optimize executor memory usage for both batch and streaming workloads.

ExecutorMemory ManagementSpark

0 likes · 7 min read

Understanding Apache Spark Unified (Dynamic) Memory Management