0 views collected around this technical thread.
mmap shared memory lets multiple processes access the same physical memory, which can break process isolation, expose permission misconfigurations like PROT_EXEC, and cause cross‑process crashes or code‑injection attacks, making it far riskier than heap allocations with malloc that remain confined to a single process.
This comprehensive guide explores Linux's memory management ecosystem, detailing user‑space allocators like malloc, kernel‑space mechanisms such as kmalloc, vmalloc, and mmap, and core kernel algorithms including the Buddy system, CMA, slab allocator, and memory pools, while providing code examples and practical usage tips.
This article explains how mmap maps files into a process's virtual memory to eliminate double data copies and reduce system‑call overhead, offering performance gains, a simpler programming model, and discusses its limitations such as address‑space constraints and page‑fault latency.
This article explains how the C functions malloc and free allocate and release memory, covering stack vs heap, the brk and mmap system calls, fragmentation, header metadata, and why both allocation strategies are needed for efficient memory management.
This article breaks down zero‑copy data transfer techniques—write+read, mmap+write, sendfile, sendfile + SG‑DMA, and splice—showing how they reduce CPU copies and context switches to boost I/O performance in modern operating systems.
This article explains the concept of zero‑copy in Linux, compares the four main system calls—sendfile, mmap, splice and tee—describes their APIs, internal mechanisms, performance characteristics, typical use‑cases and provides practical code examples for high‑performance network programming.
This article explains how mmap and zero‑copy mechanisms, combined with DMA and shared‑memory APIs, can dramatically reduce CPU involvement, context switches, and data copies during file and network I/O, thereby improving system performance for high‑throughput applications.
Zero‑copy eliminates CPU‑mediated data copies between user and kernel spaces by using DMA and memory‑mapping, dramatically improving I/O performance, reducing context switches, and enabling high‑throughput applications such as file servers, Kafka brokers, and Java networking frameworks.
This article explains the concept of zero‑copy, compares traditional I/O copying with mmap, sendfile, DMA scatter/gather and splice techniques, and shows how Java NIO leverages mmap and sendfile to achieve high‑performance data transfer while minimizing CPU involvement.
The article explains how RocketMQ’s reliance on mmap‑based zero‑copy leads to more data copies and system‑call overhead compared to Kafka’s sendfile approach, resulting in lower throughput, and discusses when to choose each message‑queue system based on functional needs and performance requirements.
This article examines mmap's historical origins, its performance benefits and drawbacks, and analyzes how Prometheus' time‑series database employs memory‑mapped files, revealing why mmap does not degrade Prometheus performance despite known kernel‑level issues such as TLB misses and lock contention.
This article explains the key techniques behind Kafka's high‑throughput performance, including batch sending, message compression, sequential disk reads/writes, efficient use of PageCache, zero‑copy transfers, and memory‑mapped index files, with code examples illustrating each mechanism.
This article explains Linux's mmap mechanism, its relationship with shared memory, the advantages and disadvantages of mmap versus traditional shared memory, and provides practical code examples and detailed steps for using mmap in interprocess communication and large‑file handling.
Zarr provides a modern, chunked and compressed storage format that lets you treat massive NumPy arrays like in‑memory objects, offering on‑demand loading, flexible back‑ends (disk, S3, zip), automatic caching, resizing, parallel reads/writes, and superior performance compared to traditional mmap‑based memmap files.
This article explains how Linux processes can share large data efficiently by creating a memory file with memfd_create, mapping it with mmap, and transferring the file descriptor over a Unix Domain Socket, while detailing the kernel mechanisms that enable true cross‑process memory sharing.
This article explains the fundamentals of memory‑mapped files in Linux, covering the mmap/munmap interfaces, mapping types, protection flags, error handling, virtual‑memory structures, kernel implementation details, driver examples, and practical use‑cases such as shared memory and high‑performance I/O.
This article explains Linux's flexible memory allocation system, covering user‑space malloc, kernel‑space kmalloc, vmalloc for large buffers, mmap for file and anonymous mappings, page allocation functions, the slab allocator, and memory pools, with detailed code examples and operational insights.
This article explains Linux memory mapping (mmap), covering its purpose, API parameters, different mapping types, internal kernel implementation, page‑fault handling, copy‑on‑write semantics, practical use cases, and includes a complete Objective‑C example demonstrating file mapping and manipulation.
This article explains the zero‑copy technique in Java, covering I/O fundamentals, kernel read/write flow, mmap+write and Sendfile methods, virtual memory mapping, MappedByteBuffer and DirectByteBuffer usage, channel‑to‑channel transfer, Netty composite buffers, and how frameworks like RocketMQ and Kafka leverage zero‑copy for performance.
Facing a challenge of managing roughly one million server-side images and 180 client images, the TOOSIMPLE team built a high-performance backend using fingerprinting, parallel processing, mmap-SSE2 acceleration, and sparsemap indexing, achieving sub-second response times while ensuring correct ordered display.