netcap: An eBPF‑Based Next‑Generation Kernel Network Capture Tool

Background: Traditional tools like tcpdump are limited for deep kernel network debugging. The ByteDance STE team created netcap, an open‑source eBPF‑based packet capture tool (GitHub: https://github.com/bytedance/netcap) that can trace almost the entire kernel network stack using skb as a context.

Usage examples

Example 1 – Trace ICMP packets for a specific host:

netcap skb -f icmp_rcv@1 -i eth0 -e "host 10.227.0.45" -t "-nnv"

Example 2 – Locate packet loss for TCP port 9000:

netcap skb -f tracepoint:skb:kfree_skb -e "tcp port 9000" -S

To simulate loss, an iptables rule can be added:

iptables -A INPUT -p tcp --dport 9000 -j DROP

Multi‑trace aggregation can be performed, for instance:

netcap skb -f tracepoint:net:netif_receive_skb,ip_local_deliver@1,ip_local_deliver_finish@3,icmp_rcv@1 -e "host 10.227.0.72 and icmp" -i eth0 --gather --gather-output-color cyan

Design and implementation

netcap hooks functions via kprobe/tracepoint, extracts skb and sock structures, and passes packet data to user space through BPF maps. It reuses tcpdump for filtering (cBPF compiled via Cloudflare’s cbpfc library) and for displaying packet contents.

The tool reconstructs packets from skb, even when headers are incomplete, by inferring missing fields from the associated sock structure.

Extended functionality

Users can provide custom filter and action C files. Example filter:

static inline int xcap_user_filter(void *ctx, void *pkt, u16 trace_index) {
    return 1;
}

Example action defining a custom output structure:

struct xcap_user_extend {
    int a;          // format: 0x%x
    uint32_t b;
    int64_t c;
    uint8_t x1;      // format: %c
    uint8_t x2;      // format: 0x%x
    uint16_t x3;     // format: 0x%x
};
static inline int xcap_user_action(void *ctx, void *pkt, u32 pkt_len, struct xcap_user_extend *user, u16 trace_index) {
    user->a = 0x12345678;
    user->b = 1000;
    user->c = 2002;
    user->x1 = 'M';
    user->x2 = 0x11;
    user->x3 = 0xabcd;
    return 1;
}

Future outlook

The team plans to add better DPDK support, unify multi‑kernel version compatibility, and resolve output interleaving issues when custom actions produce large amounts of data.