Traffic Governance and High‑Availability Strategies for Microservice Systems

Availability (Availability = MTBF / (MTBF + MTTR) * 100%) measures the proportion of time a system remains operational; higher MTBF and lower MTTR lead to better availability.

Traffic governance aims to keep the system’s three‑high goals—high performance, high availability, and easy scalability—by balancing data flow, optimizing network usage, and ensuring fault tolerance.

Circuit breakers prevent cascading failures. Traditional breakers switch between Closed, Open, and Half‑Open states based on error thresholds. Google SRE introduces client‑side adaptive throttling, where the request‑rejection probability p = max(0, (requests – K·accepts) / (requests + 1)) controls local request dropping.

Isolation strategies limit fault propagation: dynamic/static content separation, read/write segregation (CQRS), core vs. non‑core services, hotspot caching, user‑level tenancy, process, thread, cluster, and data‑center isolation.

Retry mechanisms include synchronous and asynchronous retries, configurable max attempts, and backoff policies (linear, jittered, exponential, exponential‑jitter). To avoid retry storms, techniques such as limiting per‑service retries, retry windows, and chain‑level controls are recommended.

Degradation reduces functionality under overload: automatic degradation based on thresholds, manual tiered degradation, and execution flow diagrams illustrate how non‑critical features are shed to preserve core services.

Timeout handling uses fixed limits or EMA‑based dynamic timeouts. EMA adjusts the timeout based on average response latency, with parameters INITIAL_BACKOFF, MULTIPLIER, JITTER, MAX_BACKOFF, and MIN_CONNECT_TIMEOUT. Propagating remaining timeout across RPC calls prevents wasteful downstream processing.

/* Pseudo‑code */
ConnectWithBackoff()
current_backoff = INITIAL_BACKOFF
current_deadline = now() + INITIAL_BACKOFF
while (TryConnect(Max(current_deadline, now() + MIN_CONNECT_TIMEOUT)) != SUCCESS) {
    SleepUntil(current_deadline)
    current_backoff = Min(current_backoff * MULTIPLIER, MAX_BACKOFF)
    current_deadline = now() + current_backoff + UniformRandom(-JITTER * current_backoff, JITTER * current_backoff)
}

Rate limiting protects services from traffic spikes. Client‑side limits allocate quotas per caller, while server‑side limits (sliding window, token bucket, leaky bucket) drop or delay excess requests based on resource usage, success rate, or response latency.

Combining these mechanisms—circuit breaking, isolation, retries, degradation, timeout control, and rate limiting—creates a resilient, high‑performance, and easily scalable microservice architecture capable of handling diverse network conditions and failures.