Service Governance in Cloud‑Native Architecture: Rate Limiting and Circuit Breaking with Istio

1. Project Background

The previous article introduced platform‑based monitoring and alerting for Autohome's cloud‑native service‑mesh transformation. This article focuses on service governance, specifically rate limiting and circuit breaking, which are the most frequently used governance scenarios for business teams.

Rate limiting protects services by rejecting excess traffic when request volume exceeds system capacity, preventing resource exhaustion. Service circuit breaking works like an electrical fuse: when failure or timeout thresholds are crossed, the circuit opens, causing subsequent calls to fail immediately, giving the faulty service time to recover and preventing cascade failures.

Traditional micro‑service architectures embed rate‑limiting and circuit‑breaking logic directly in SDKs, leading to strong coupling, higher development and maintenance costs, and language‑specific fragmentation. By moving these capabilities to the sidecar layer of a service mesh, the business code remains clean and the operational complexity is isolated.

2. Service Rate Limiting

Istio provides two types of rate limiting: local (per‑sidecar) and global (cluster‑wide). Local rate limiting is implemented via an EnvoyFilter that adds the envoy.filters.http.local_ratelimit filter. Example configuration:

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: filter-local-ratelimit-svc
spec:
  workloadSelector:
    labels:
      app: productpage
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      listener:
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
    patch:
      operation: INSERT_BEFORE
      value:
        name: envoy.filters.http.local_ratelimit
        typed_config:
          "@type": type.googleapis.com/udpa.type.v1.TypedStruct
          type_url: type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
          value:
            stat_prefix: http_local_rate_limiter
            token_bucket:
              max_tokens: 10
              tokens_per_fill: 10
              fill_interval: 60s
            filter_enabled:
              runtime_key: local_rate_limit_enabled
              default_value:
                numerator: 100
                denominator: HUNDRED
            filter_enforced:
              runtime_key: local_rate_limit_enforced
              default_value:
                numerator: 100
                denominator: HUNDRED
            response_headers_to_add:
            - append: false
              header:
                key: x-local-rate-limit
                value: 'true'

Local rate limiting works at the sidecar level using a token‑bucket algorithm, but it cannot enforce a global limit across all instances. Therefore a global rate‑limiting solution is required.

Global rate limiting in Istio integrates a gRPC ratelimit service. The service reads configuration from a file or an xDS management server, caches keys, and interacts with Redis to make decisions. Example file‑based configuration:

domain: ratelimit_demo01
descriptors:
- key: demoKey
  value: users
  rate_limit:
    unit: second
    requests_per_unit: 500
- key: demoKey
  value: default
  rate_limit:
    unit: second
    requests_per_unit: 500

Autohome uses the xDS Management Server approach. The server (built with go‑control‑plane, Gin, Gorm) provides an HTTP API to create policies, which are then pushed to the ratelimit service via gRPC. Example policy creation request:

{
  "domain": "ratelimit_demo01",
  "descriptors": [
    {
      "key": "demoKey",
      "value": "users",
      "rate_limit": { "unit": 1, "requests_per_unit": 500 }
    },
    {
      "key": "Remote_IP",
      "value": "default",
      "rate_limit": { "unit": 1, "requests_per_unit": 500 }
    }
  ]
}

A WASM plugin is also used to normalize user‑IP extraction from different client entry points, splitting the x-forwarded-for header and the :path header into separate variables for consistent rate‑limit key generation.

Deployment examples for the global rate‑limit filter and its associated cluster are provided via additional EnvoyFilter resources.

3. Service Circuit Breaking

Traditional SDK‑based circuit breaking (e.g., Hystrix, Sentinel) suffers from the same coupling issues as rate limiting. Istio enables non‑intrusive circuit breaking through DestinationRule and EnvoyFilter configurations. The outlierDetection field in TrafficPolicy defines failure detection thresholds.

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: dotnet-car-automesh
spec:
  host: dotnet-car-automesh-10001.autohome.com
  trafficPolicy:
    outlierDetection:
      consecutive5xxErrors: 10
      interval: 5s
      baseEjectionTime: 5s
      maxEjectionPercent: 100

Alternatively, an EnvoyFilter can apply the same outlier detection to a specific workload selector, allowing fine‑grained circuit‑breaking at the service level.

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: match
spec:
  workloadSelector:
    labels:
      app_service: dotnet-car-automesh
  configPatches:
  - applyTo: CLUSTER
    match:
      cluster:
        name: outbound|8080||dotnet-car-automesh-10001.autohome.com
    patch:
      operation: MERGE
      value:
        outlierDetection:
          consecutive5xxErrors: 10
          interval: 5s
          baseEjectionTime: 5s
          maxEjectionPercent: 100

These strategies detect unhealthy instances every five seconds and eject them after ten consecutive 5xx errors, protecting the overall system. However, both approaches are limited to domain‑ or service‑level granularity. To achieve path‑level circuit breaking, Autohome plans to develop a WASM plugin that leverages Sentinel’s circuit‑breaking algorithm within the sidecar.

4. Summary

The article introduced service governance concepts in a cloud‑native environment and demonstrated Autohome’s practical migration using Istio’s native capabilities. By abstracting governance functions into the service mesh, developers can focus on business logic while the platform provides reliable rate limiting and circuit‑breaking mechanisms.