Understanding Disk I/O: Read, Write, DMA, Page Cache, mmap and Performance Optimizations

Developers often hear that improving program performance starts with optimizing I/O. When network, hardware, and application logic are already optimal, the next focus is disk and network I/O, specifically how data moves between disk, CPU buffers, and memory.

What is Disk I/O? When a program accesses data not present in memory, it issues a system call that triggers the operating system to read the required data from the disk into memory – a classic disk I/O operation.

IO Interrupt Process

1. The user process calls read (or similar) and blocks. 2. The OS forwards the request to the disk. 3. The disk driver reads data into its internal buffer and signals an interrupt when the buffer is full. 4. The kernel copies data from the driver buffer to a kernel buffer. 5. If the kernel buffer does not contain enough data, steps 3‑4 repeat. 6. Finally, the CPU copies data from the kernel buffer to the user buffer and returns from the system call.

Because the CPU participates at several stages, reducing its involvement can greatly improve efficiency.

DMA (Direct Memory Access)

DMA offloads the data transfer from the CPU. The OS sends the I/O request to a DMA controller, which moves data directly from the disk to memory, generating an interrupt only when the transfer is complete.

Key Concepts

What is DMA? DMA allows hardware devices to transfer data to/from memory without continuous CPU intervention, reducing CPU load.

What is page cache? The Linux page cache stores file data in memory (typically 4 KB pages). It speeds up reads/writes, shares data among processes, and writes back dirty pages to disk at appropriate times.

What is mmap? Memory‑mapped files map a file directly into a process’s address space, allowing the program to read/write via normal memory accesses; the kernel synchronizes changes back to the file without explicit read / write calls.

Practical Java Examples

Unbuffered I/O (continuous out.write(data) ) is much slower than buffered I/O, where a BufferedOutputStream accumulates data before invoking a system call. The JVM’s default buffer size is 8 KB.

public class OSFileIO {
    static byte[] data = "123456789\n".getBytes();
    static String path = "/root/testfileio/out.txt";
    public static void testBasicFileIO() throws Exception {
        File file = new File(path);
        FileOutputStream out = new FileOutputStream(file);
        while (true) {
            out.write(data);
        }
    }
}

public class OSFileIO2 {
    static byte[] data = "123456789\n".getBytes();
    static String path = "/root/testfileio/out1.txt";
    public static void testBufferedFileIO() throws Exception {
        File file = new File(path);
        BufferedOutputStream out = new BufferedOutputStream(new FileOutputStream(file));
        while (true) {
            out.write(data);
        }
    }
}

Using ByteBuffer (direct allocation) demonstrates typical buffer operations:

public void whatByteBuffer() {
    ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
    System.out.println("position: " + buffer.position());
    System.out.println("limit: " + buffer.limit());
    System.out.println("capacity: " + buffer.capacity());
    buffer.put("123".getBytes());
    buffer.flip();
    buffer.get();
    buffer.compact();
    buffer.clear();
}

Performance observations:

Unbuffered I/O << buffered I/O << mmap in terms of write speed.

Buffering reduces the number of system calls, allowing the CPU to perform other work.

Linux Kernel Parameters Affecting I/O

vm.dirty_background_ratio = 0   # % of memory before background writeback starts
vm.dirty_background_bytes = 1048576
vm.dirty_ratio = 0               # % of memory before foreground writeback blocks
vm.dirty_bytes = 1048576
vm.dirty_writeback_centisecs = 5000
vm.dirty_expire_centisecs = 30000

These settings control when dirty pages are flushed to disk, influencing latency and data safety.

Summary

Disk I/O performance hinges on minimizing data movement and kernel involvement. Techniques such as DMA, buffering, page cache, and mmap each reduce CPU work to varying degrees, but none can guarantee absolute data durability; the OS balances reliability against performance. Understanding these mechanisms enables developers to write more efficient I/O‑intensive applications.