Implementing a Dynamic IP Blacklist with Nginx, Lua, and Redis

To block malicious crawlers or abusive users, a dynamic IP blacklist can be created that rejects requests from listed IPs and supports configurable expiration times.

Environment Preparation

Linux (CentOS 7 / Ubuntu, etc.)

Redis 5.0.5

Nginx (OpenResty)

Design Options

Three main approaches are considered:

OS level (iptables) – simple but requires manual updates.

Nginx level (deny or Lua plugin) – allows dynamic blocking via Redis and time‑based rules, but needs Lua knowledge.

Application level – implement checks in code; easy to maintain but may affect performance under high concurrency.

The chosen solution combines Nginx, Lua, and Redis for a lightweight, distributed blacklist.

Configure nginx.conf

location / {
    # access_by_lua_file /usr/local/lua/access_limit.lua;
    access_by_lua_file /usr/local/lua/access_limit.lua;
    alias /usr/local/web/;
    index index.html index.htm;
}

Lua Script ( /usr/local/lua/access_limit.lua )

local pool_max_idle_time = 10000
local pool_size = 100
local redis_connection_timeout = 100
local redis_host = "your redis host ip"
local redis_port = "your redis port"
local redis_auth = "your redis authpassword"
local ip_block_time = 120
local ip_time_out = 1
local ip_max_count = 3

local function errlog(msg, ex)
    ngx.log(ngx.ERR, msg, ex)
end

local function close_redis(red)
    if not red then return end
    local ok, err = red:set_keepalive(pool_max_idle_time, pool_size)
    if not ok then
        ngx.say("redis connct err:", err)
        return red:close()
    end
end

local redis = require "resty.redis"
local client = redis:new()
local ok, err = client:connect(redis_host, redis_port)
if not ok then
    close_redis(client)
    ngx.exit(ngx.HTTP_INTERNAL_SERVER_ERROR)
end
client:set_timeout(redis_connection_timeout)

local connCount, err = client:get_reused_times()
if 0 == connCount then
    ok, err = client:auth(redis_auth)
    if not ok then errlog("failed to auth: ", err); return end
elseif err then
    errlog("failed to get reused times: ", err); return
end

local function getIp()
    local clientIP = ngx.req.get_headers()["X-Real-IP"]
    if not clientIP then clientIP = ngx.req.get_headers()["x_forwarded_for"] end
    if not clientIP then clientIP = ngx.var.remote_addr end
    return clientIP
end

local cliendIp = getIp()
local incrKey = "limit:count:" .. cliendIp
local blockKey = "limit:block:" .. cliendIp

local is_block = client:get(blockKey)
if tonumber(is_block) == 1 then
    ngx.exit(ngx.HTTP_FORBIDDEN)
    close_redis(client)
end

local ip_count = client:incr(incrKey)
if tonumber(ip_count) == 1 then client:expire(incrKey, ip_time_out) end
if tonumber(ip_count) > tonumber(ip_max_count) then
    client:set(blockKey, 1)
    client:expire(blockKey, ip_block_time)
end

close_redis(client)

Summary

Simple, lightweight configuration with minimal performance impact.

Multiple servers can share the blacklist via a common Redis instance.

Dynamic updates allow manual or automated management of blocked IPs.

Extensions

Application Scenarios

Prevent malicious access (brute‑force, SQL injection, XSS).

Block crawlers and data abuse.

Mitigate DDoS attacks.

Rate‑limit specific IPs to stop abuse.

Advanced Features

Automatic anomaly detection and blocking.

Whitelist mechanism for trusted IPs.

CAPTCHA verification for suspicious traffic.

Statistics and analysis of blacklist usage.

Continuous improvements to the IP blacklist enhance server security and reliability.