Master API Rate Limiting in SpringBoot with Caffeine Cache and Custom Annotations

Environment

SpringBoot 2.7.16

1. Introduction

Interface rate limiting protects system stability by restricting the number of requests within a time window, preventing overload, handling burst traffic, buggy callers, or malicious attacks. It can reject, queue, wait, or degrade requests when limits are reached.

2. Caffeine Overview

Caffeine is a high‑performance Java 8 cache library offering near‑optimal hit rates. It differs from ConcurrentMap by automatically evicting entries to limit memory usage. It provides flexible builders for features such as automatic loading, size‑based eviction, time‑based eviction, asynchronous refresh, weak/soft references, removal notifications, write‑through, and statistics.

Key Features

Automatic loading (including async)

Size‑based eviction using near‑frequency algorithm

Time‑based eviction based on last access or write

Asynchronous refresh of expired entries

Keys stored as weak references

Values stored as weak or soft references

Removal notifications

Write‑through to external data source

Statistics collection

Integration

Caffeine provides adapters for JSR‑107 JCache and Guava, enabling easy migration.

3. Simple Usage Example

<code>Cache&lt;String, Integer&gt; cache = Caffeine.newBuilder()
    .maximumSize(1)
    // Keep entry alive while accessed within 1 s; otherwise expire
    .expireAfterAccess(Duration.ofSeconds(1))
    // .expireAfterWrite(Duration.ofSeconds(111))
    .scheduler(Scheduler.systemScheduler())
    .build();
cache.put("a", 666);
System.out.println(cache.getIfPresent("a"));
</code>

4. Cache Deletion

<code>// Invalidate a single key
cache.invalidate(key);
// Invalidate multiple keys
cache.invalidateAll(keys);
// Invalidate all keys
cache.invalidateAll();
</code>

5. Cache Refresh

<code>Cache&lt;String, AccessCount&gt; cache = Caffeine.newBuilder()
    .maximumSize(10_000)
    .refreshAfterWrite(1, TimeUnit.MINUTES)
    .build();
</code>

6. Cache Statistics

<code>Cache&lt;String, AccessCount&gt; cache = Caffeine.newBuilder()
    .maximumSize(10_000)
    .recordStats()
    .build();
CacheStats stats = cache.stats();
stats.hitRate();          // cache hit rate
stats.evictionCount();    // number of evicted entries
stats.averageLoadPenalty(); // average load time for new values
</code>

7. Rate Limiting Implementation

Custom Annotation

<code>@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@Inherited
@Documented
public @interface PackLimiter {
    /** requests per second */
    int count() default 5;
    /** cache key */
    String key() default "";
    /** fallback method name */
    String fallbackMethod() default "";
}
</code>

Aspect Logic

<code>@Component
@Aspect
public class PackLimiterAspect {
    private static final Logger logger = LoggerFactory.getLogger(PackLimiterAspect.class);
    public static final Cache<String, AccessCount> LIMITRATE = Caffeine.newBuilder()
        .maximumSize(10_000)
        .expireAfterAccess(Duration.ofSeconds(1))
        .scheduler(Scheduler.systemScheduler())
        .build();
    private static final Map<String, Object> FALLBACK_METHOD = new ConcurrentHashMap<>();
    private static final String DEFAULT_RET_DATA = "429 - Too Many Requests";
    private static final Function<? super String, ? extends AccessCount> INIT_COUNT = AccessCount::new;

    @Resource
    private HttpServletRequest request;

    @Pointcut("@annotation(limiter)")
    private void access(PackLimiter limiter) {}

    @Around("access(limiter)")
    public Object limiter(ProceedingJoinPoint pjp, PackLimiter limiter) throws Throwable {
        int count = limiter.count();
        String key = limiter.key();
        String fallbackMethod = limiter.fallbackMethod();
        MethodSignature ms = (MethodSignature) pjp.getSignature();
        Class<?> target = ms.getDeclaringType();
        Method method = ms.getMethod();
        if (!StringUtils.hasLength(key)) {
            key = getKey(target, method);
        }
        logger.info("Cache key: {}", key);
        AccessCount ac = LIMITRATE.get("a", INIT_COUNT);
        if (ac.isValid(count)) {
            return pjp.proceed();
        } else {
            if (!FALLBACK_METHOD.containsKey(key)) {
                if (StringUtils.hasLength(fallbackMethod)) {
                    try {
                        Method fallback = target.getDeclaredMethod(fallbackMethod);
                        FALLBACK_METHOD.put(key, fallback.invoke(pjp.getTarget()));
                    } catch (Exception e) {
                        throw new RuntimeException(e);
                    }
                } else {
                    FALLBACK_METHOD.put(key, DEFAULT_RET_DATA);
                }
            }
            return FALLBACK_METHOD.get(key);
        }
    }

    private String getKey(Class<?> target, Method method) {
        StringBuilder sb = new StringBuilder();
        sb.append(target.getSimpleName()).append('#').append(method.getName()).append('(');
        for (Class<?> p : method.getParameterTypes()) {
            sb.append(p.getSimpleName()).append(',');
        }
        if (method.getParameterTypes().length > 0) {
            sb.deleteCharAt(sb.length() - 1);
        }
        return (InetUtils.getIp() + sb.append(')').toString()).replaceAll("[^a-zA-Z0-9]", "");
    }

    public void intt(List<String> list) {}
}
</code>

Controller Example

<code>@GetMapping("/{id}")
@PackLimiter(count = 3, fallbackMethod = "fallbackIndex")
public Object index(@PathVariable("id") Integer id) {
    return "success";
}
public Object fallbackIndex() {
    return "访问太快了";
}
</code>

Conclusion

By combining custom annotations, AspectJ, and Caffeine, developers can implement lightweight, high‑performance API rate limiting to safeguard system stability.