Implementing Token‑Bucket Rate Limiting with Guava RateLimiter in Java

During peak traffic periods such as the annual Double 11 shopping event, systems must avoid overload; the recommended strategy is to add capacity, degrade non‑essential services, or apply traffic throttling to protect core functionality.

The company's middleware includes a single‑machine rate‑limiting framework that supports two modes: rate control (using token‑bucket algorithms) and concurrency control (similar to semaphores).

According to Wikipedia, the token‑bucket algorithm works as follows: each second r tokens are added to the bucket (or one token every 1/r seconds); the bucket can hold at most b tokens, discarding excess; a packet of n bytes consumes n tokens; if insufficient tokens are available, the packet is buffered or dropped.

In practice, the token bucket limits the amount of data transmitted within a time window, which corresponds to API‑level metrics such as QPS or TPS measured over a 1‑second interval.

For Java projects, Guava's RateLimiter offers a convenient token‑bucket implementation. Its default single‑bucket version, called SmoothBursty , is suitable for most web‑application scenarios, while the SmoothWarmingUp variant implements a leaky‑bucket style algorithm.

SmoothBursty can accumulate tokens for N time windows, allowing the system to handle bursts up to N times the configured rate; setting N = 1 limits the burst to a single window.

An interesting feature of RateLimiter is the “borrow‑and‑repay” behavior: a request may take more tokens than currently available, but subsequent requests will be delayed until the deficit is replenished.

While RateLimiter serves as a solid core component for a rate‑control framework, a complete solution also requires a generic API, interceptor hooks, and a backend for online configuration of limits.

Below is a simple Java traffic‑shaping API that demonstrates how to create, update, and enforce per‑resource rate limits using Guava's RateLimiter :

public class TrafficShaper {
    public static class RateLimitException extends Exception {
        private static final long serialVersionUID = 1L;
        private String resource;
        public String getResource() { return resource; }
        public RateLimitException(String resource) {
            super(resource + " should not be visited so frequently");
            this.resource = resource;
        }
        @Override
        public synchronized Throwable fillInStackTrace() { return this; }
    }
    private static final ConcurrentMap
resourceLimiterMap = Maps.newConcurrentMap();
    public static void updateResourceQps(String resource, double qps) {
        RateLimiter limiter = resourceLimiterMap.get(resource);
        if (limiter == null) {
            limiter = RateLimiter.create(qps);
            RateLimiter putByOtherThread = resourceLimiterMap.putIfAbsent(resource, limiter);
            if (putByOtherThread != null) { limiter = putByOtherThread; }
        }
        limiter.setRate(qps);
    }
    public static void removeResource(String resource) { resourceLimiterMap.remove(resource); }
    public static void enter(String resource) throws RateLimitException {
        RateLimiter limiter = resourceLimiterMap.get(resource);
        if (limiter == null) { return; }
        if (!limiter.tryAcquire()) { throw new RateLimitException(resource); }
    }
    public static void exit(String resource) { /* no action needed for RateLimiter */ }
}

Source: http://blog.jamespan.me/2015/10/19/traffic-shaping-with-token-bucket/