Understanding Linux Kernel Packet Reception Path and NAPI

As a business SRE, the services we operate are often provided as Linux + TCP/UDP daemons, so we must understand the complete packet‑receive path in the kernel to diagnose issues quickly.

Overall packet‑receive flow : the NIC registers itself at boot and allocates a memory ring buffer. When a packet arrives, the NIC places it in this buffer and triggers a hardware interrupt. The kernel copies the packet from the NIC buffer into the kernel’s TCP/IP stack, which eventually places the data into a socket buffer that applications read via read() .

The path can be divided into three layers:

NIC layer – packet reception and DMA transfer to host memory.

Kernel layer – TCP/IP stack processing.

Application layer – read() from the socket buffer.

What is NAPI? At boot the driver allocates a ring buffer of packet descriptors (ready/used) that point to sk_buff structures. DMA fills these buffers, and the NIC raises an IRQ. To avoid an IRQ per packet, Linux uses NAPI (New API), which combines interrupt handling with polling to batch‑process packets, reducing CPU overhead.

NIC‑side reception steps :

Driver allocates sk_buffers and registers them in the RX ring.

Driver notifies the NIC of available descriptors.

The NIC fetches a descriptor, obtains the buffer address, and DMA‑writes incoming packets into the sk_buffer .

After DMA, the NIC raises an IRQ.

The kernel’s interrupt handler wakes a soft‑IRQ that polls the ring, passes packets to the IP layer, and re‑enables the NIC interrupt.

Tools like ethtool -g eth1 can view ring settings, and ethtool -G eth1 rx 4096 tx 4096 can adjust them.

Interrupt handling is split into an upper half (quick hardware acknowledgment) and a lower half (soft‑IRQ that polls the ring). Modern NICs also support RSS (Receive Side Scaling) and RPS (Receive Packet Steering) to distribute processing across CPUs, improving scalability.

TCP/IP stack processing :

Connection establishment follows the three‑way handshake, involving a SYN queue and an accept queue.

After the connection is established, data is received into the socket’s receive buffer.

Linux can automatically tune the receive buffer size via net.ipv4.tcp_moderate_rcvbuf=1 and the net.ipv4.tcp_rmem triple (min, default, max). Disabling the auto‑tuning or setting SO_RCVBUF overrides this behavior, and the maximum is limited by net.core.rmem_max .

Overall optimization summary : ensure NAPI is enabled, balance IRQ affinity, tune ring buffer sizes, enable receive‑buffer auto‑scaling, and align net.ipv4.tcp_rmem max with net.core.rmem_max to achieve optimal packet‑receive performance.