Optimizing Redis String Storage: JSON vs MessagePack and Compression Techniques

Redis strings are one of the most commonly used (and often abused) data structures in Redis, offering binary safety that allows any type of binary data to be stored. In practice, many users serialize objects to JSON strings before storing them in Redis.

You might ask what the problem is?

JSON serialization/deserialization is CPU‑intensive and inefficient.

It consumes more storage space, which is costly in an in‑memory database like Redis.

It adds overall service latency without providing real benefits.

Storing data as JSON in Redis increases latency and resource usage without any genuine advantage.

An alternative "simple" optimization is compression, which involves a trade‑off among space, latency, and CPU usage. Algorithms such as ZSTD or LZ4 can achieve minimal CPU overhead while significantly reducing storage size.

The following experiments show the performance gains when converting JSON to a binary format like MessagePack, including serialization/deserialization times and the impact of compression on latency and memory usage.

Images below illustrate benchmark results for different array sizes and compression methods:

As demonstrated, switching from JSON to MessagePack can reduce latency by more than three times without any real side effects. Below is a Python example that performs Redis SET and GET operations using both JSON and MessagePack:

import msgpack
import redis
import json

data = {}  # normal python dictionary with any values
r = redis.Redis(host='REDIS_HOST', port=6379, db=0)

# using json
r.set('foo_json', json.dumps(data))
json.loads(r.get('foo_json').decode('utf-8'))

# using msgpack
r.set('foo_msgpack', msgpack.packb(data))
msgpack.unpackb(r.get('foo_msgpack'))

MessagePack also has a Go library: https://github.com/vmihailenco/msgpack .

The article concludes with a note about a curated list of interview questions from major tech companies (BAT) and various promotional links, which are unrelated to the technical content.