KLimiter: Adaptive Rate Limiter for Downstream Resource Protection

With the rapid growth of internet services, system architectures become increasingly complex and protecting downstream resources such as databases is essential for system stability. Traditional rate‑limiting methods rely on manually set fixed thresholds, which cannot cope with dynamic business demands and often lead to resource waste or overload.

Typical problems include a growing number of upstream interfaces whose peak times are hard to coordinate, and agreements that allow peak overlaps, causing the downstream resource water‑level to exceed safe limits. Common work‑arounds—lowering thresholds for all interfaces or over‑provisioning downstream resources—result in long‑term inefficiency.

KLimiter is introduced as an adaptive limiter that monitors the water‑level of downstream resources and dynamically adjusts traffic of multiple upstream entrances according to their priorities, thereby protecting the resource while optimizing upstream flow.

Key capabilities of KLimiter are: (1) second‑level monitoring of multiple downstream resources; (2) adaptive adjustment of entrance traffic to keep the resource water‑level below a configured threshold; (3) per‑entrance priority handling, where lower‑priority entrances are reduced first.

Important terminology includes: limit entry (an upstream interface to be throttled), resource baseline (a downstream resource instance with a water‑level value), impact coefficient (the normalized influence of an entry on a baseline), traffic ratio (the proportion of traffic allowed for an entry), and entry priority (the maximum reduction ratio an entry may experience).

The core algorithm computes impact coefficients by solving a system of linear equations derived from historical QPS matrices and baseline water‑levels. LU decomposition (Gaussian elimination) is used to obtain the coefficients, which are then normalized to represent each entry’s share of influence on a baseline.

For traffic reduction, KLimiter first calculates the required baseline reduction ratio, then distributes the reduction among entries based on their priorities and impact coefficients. Each iteration limits reduction to a maximum of 5 % to avoid abrupt changes, and a “zero‑flow protection” ensures ratios never drop below 1 % when QPS is very low.

Traffic increase follows a similar principle but uses Newton‑Raphson iteration to find the nearest feasible increase while respecting the monotonicity of the underlying function. A “slow‑start” mechanism mimics TCP congestion control: the allowed increase grows with the current traffic ratio.

Additional fallback strategies include rate‑capping, slow‑start, and a simple 5‑second moving‑average filter for QPS and water‑level values to smooth out fluctuations.

Evaluation demonstrates that KLimiter can keep a baseline water‑level around the target (e.g., 60) while respecting entry priorities (e.g., entry 2 limited to 50 % before entry 1 is further reduced). When the baseline threshold is raised, traffic ratios recover accordingly, confirming stable and efficient behavior compared to static limiting.