Understanding and Handling Redis Bigkey Issues

Redis bigkey refers to a key‑value pair that occupies a large amount of memory in a Redis database, such as a string larger than 10 KB or non‑string types (hash, list, set, sorted set) with more than 5 000 elements.

Bigkeys can cause timeout blocking due to Redis's single‑threaded nature, lead to cluster node imbalance, and make backup and recovery difficult because of the high memory and bandwidth consumption.

They are usually created by poor program design or inaccurate data‑size estimation; avoiding them requires proper data‑structure selection, splitting large strings, compressing data, and regular size checks.

How to Discover Bigkeys

Using Redis Built‑in Commands

$ redis-cli --bigkeys
# Scanning the entire keyspace to find biggest keys as well as average sizes per key type.
# Use -i 0.01 to sleep 0.01 sec per SCAN command (not usually needed).
[00.00%] Biggest string found so far 'key-419' with 3 bytes
[05.14%] Biggest list   found so far 'mylist' with 100004 items
... (output truncated) ...

Running --bigkeys scans the whole database and may affect performance; it is recommended to run it on a replica.

Scanning with SCAN and TYPE

# redis-cli will pause 0.1 sec after every 100 scans
./redis-cli --bigkeys -i 0.1

Use SCAN to iterate over keys, TYPE to get the key type, and then measure the value size according to the type.

String: STRLEN key

Set/List/Hash/ZSet: use SCARD , LLEN , HLEN , ZCARD respectively (or MEMORY USAGE key for unknown sizes).

Analyzing RDB Files

Export the dataset to an RDB file and inspect it with tools such as od -A x -t x1c -v dump.rdb or redis‑rdb‑tool to generate memory reports or convert the dump to JSON.

rdb --command json dump.rdb
[{"hello":"redis"}]

The RDB format consists of a header, the data section (all key‑value pairs), and a footer with a checksum. Each entry includes a type code that determines how the value is decoded.

How to Handle Bigkeys

When a bigkey is identified, do not delete it directly; instead notify the caller to refactor the data structure, split large strings, or apply compression.

If Redis performance degrades, use the following checklist:

Avoid high‑complexity commands or full‑scan queries.

Eliminate operations on bigkeys.

Prevent massive key expirations.

Monitor memory usage and maxmemory limits.

Prefer persistent connections over short‑lived ones.

Be aware that large datasets increase RDB/AOF rewrite time.

Adjust AOF write‑back policy if set to always .

Ensure the host has enough RAM to avoid swapping.

Bind processes to appropriate CPUs.

Disable transparent huge pages if they cause latency.

Check network interface load.

Additional Tools

The redis‑rdb‑tool can generate memory reports, convert RDB files to JSON, and compare two dump files, providing a non‑intrusive way to analyze Redis data.

By analyzing RDB files or using the above commands, you can monitor and mitigate bigkey issues without impacting the live Redis server.

Recruitment Notice

The article concludes with a recruitment invitation from the Zhengcai Cloud Technology team, encouraging interested engineers to contact [email protected] .