Using Redis Bloom Filter to Prevent Cache Penetration and Understanding Redis Memory Eviction Policies

1. Scenario Example

Assume a backend API GET /userinfo/100 where the database only contains users up to ID 100. Requests with IDs within 100 are valid, but a request like 10000 is abnormal and would still query the DB, wasting resources – this is cache penetration.

2. Cache Knowledge

For readability a mind map is provided (image omitted).

2.1 Three Scenarios of Cache Expiration

2.2 Cache Warm-up

3. Redis Memory Reclamation Strategies

When a Redis instance reaches its memory limit, it triggers one of several eviction policies.

3.1 When used as a cache:

allkeys-lru: Keeps most recently used keys; removes least recently used (LRU) keys

allkeys-lfu: Keeps frequently used keys; removes least frequently used (LFU) keys

volatile-lru: Removes least recently used keys with an expire field set

volatile-lfu: Removes least frequently used keys with an expire field set

allkeys-random: Randomly removes keys to make space for new data

volatile-random: Randomly removes keys with expire field set

volatile-ttl: Removes keys with expire field set and the shortest remaining TTL

3.2 When used as a database (ensure data is not lost)

Noeviction (default) stops write requests and returns an error, except for DEL and some special commands.

4. Redis Module – Bloom Filter (RedisBloom)

To identify illegal user IDs (e.g., >100 or <0), a Bloom filter can be used to record and check existence.

Introduction

Bloom Filter, proposed by Bloom in 1970, is a long binary vector with multiple hash functions.

It can test membership of an element in a set, offering high space efficiency and fast queries, at the cost of false positives and difficulty deleting elements.

Features

Bloom filter is not exact; two keys may map to the same bits. A positive result does not guarantee the element exists in the database, while a negative result guarantees non‑existence.

Standard Bloom filter cannot delete elements (Cuckoo filter can).

Principle

Inserting a key hashes it k times to set k bits in the bitmap. Querying checks those bits; any zero means the element is absent.

4.1 Redis Module Overview

Redis provides an extension point for modules to add functionality.

Official link: https://redis.io/resources/modules

Common modules are listed below.

4.2 RedisBloom

Repository: https://github.com/RedisBloom/RedisBloom

Documentation: https://redis.io/docs/stack/bloom

RedisBloom offers Bloom filter and Cuckoo filter; use cases:

Bloom filter: good insertion performance and scalability

Cuckoo filter: better query performance and supports deletion

5. Example

Prerequisite steps (download, compile, load, restart Redis) are omitted.

# redis-cli
# BF means Bloom filter; a new user registers, the business adds userid filter with value 101
127.0.0.1:6379> BF.ADD userid 101
(integer) 1
# EXISTS checks existence: 1 means present
127.0.0.1:6379> BF.EXISTS userid 101
(integer) 1
# 0 means absent
127.0.0.1:6379> BF.EXISTS userid 102
(integer) 0
# BF.INFO shows filter statistics
127.0.0.1:6379> BF.INFO userid
1) Capacity
(integer) 100
3) Size
(integer) 296
5) Number of filters
(integer) 1
7) Number of items inserted
(integer) 1
9) Expansion rate
(integer) 2
# More commands: https://redis.io/commands/?name=bf

When a Bloom filter is empty, all requests are rejected; cache warm‑up (section 2.1) can mitigate this.

RedisBloom provides three configuration parameters:

Parameter

Dynamic Adjustment

Default

Description

ERROR_RATE

No

0.1

Fault tolerance ratio; lower values need more space

INITIAL_SIZE

No

100

Default capacity

CF_MAX_EXPANSIONS

No

32

Maximum expansions for Cuckoo filter

6. Summary

Bloom or Cuckoo filters can solve cache penetration.

Remember to synchronize data (add new user IDs to the filter) and warm up the cache before startup.

Redis modules offer many useful features that can be configured as needed.