Introduction to Apache Dubbo and iQIYI’s Practical Experience, Optimizations, and Future Outlook

Apache Dubbo is an open‑source RPC (Remote Procedure Call) framework that offers simple‑to‑use, high‑performance RPC capabilities, flexible extensibility, powerful service governance, and a rich open‑source ecosystem. It currently supports multiple languages including Java, Go, JavaScript, and Python.

Dubbo also serves as a service‑governance framework, providing static and dynamic routing configurations, cluster fault‑tolerance, visual operation‑and‑maintenance, monitoring, and traffic scheduling.

iQIYI’s Practice on Dubbo

Since 2019, iQIYI has adopted Dubbo and built a custom plugin system based on Dubbo’s powerful SPI (Service Provider Interface) mechanism to meet internal requirements. The first version focused on adapting the company’s infrastructure, container runtime, and integrating basic tracing with iQIYI’s monitoring system, enabling downstream business teams to develop services efficiently.

Subsequent versions added adaptations for the registration center, configuration center, and metadata center:

1. Registration Center : Continued use of Zookeeper (e.g., iqiyi-zk://idc1-cluster ) to simplify user configuration and enable transparent operations.

2. Configuration Center : Enhanced in version 2.7.x to provide dynamic configuration push capabilities, supporting runtime service governance and dynamic parameter updates.

3. Metadata Center : Introduced in version 2.7.x, storing method metadata separately from the registration URL, reducing payload size and load on the registry. Redis is used as the storage backend.

Later releases integrated Dubbo Admin functions into iQIYI’s internal management platform, offering instance visualization, monitoring dashboards, basic alarm subscriptions, service interface inspection, online testing, and governance configuration distribution.

Dubbo Optimizations

Improving Availability

iQIYI introduced two dimensions of availability:

• Instance‑level availability : Detect unhealthy instances not captured by the registry’s health check (e.g., containers with full disks causing timeouts). A strategy was added to temporarily blacklist such instances based on error/timeout ratios, with periodic re‑probing to restore healthy instances.

• Service‑level availability : Integrated Sentinel for circuit‑breaking, rate‑limiting, and degradation. Enhanced the Sentinel‑Dubbo adapter to support asynchronous calls and fine‑grained control based on service group/version.

Configuration is enabled via parameters, for example (illustrated in the image below):

Enhancing RPC Efficiency

To reduce cross‑region latency, iQIYI designed a region‑aware nearest‑routing mechanism. The SDK fetches instance metadata (region, availability zone) from an internal CMDB at startup and registers this information with the registry, allowing the client to prefer same‑region instances.

When a region lacks enough healthy instances, the routing expands to other regions, and health thresholds can be customized via SPI.

Strengthening Service Security

For security‑sensitive services (e.g., payment), iQIYI added an AK/SK based authentication mechanism. The workflow includes:

Clients obtain Access Key ID and Secret Key from an authentication service center.

Requests are signed with the secret key; the server verifies the signature.

All communication with the auth service center uses mutual TLS for confidentiality.

The solution has been contributed to the Dubbo open‑source community and merged, with extensible SPI hooks for custom authentication plugins.

Future Outlook

1. Evaluate alternatives to Zookeeper for service registration due to its CP model limitations and plan a smooth migration.

2. Further enrich the management platform with advanced governance capabilities such as dynamic rate‑limit distribution, stress testing, and remote debugging.

3. Explore Dubbo’s deployment in cloud‑native environments and Service Mesh scenarios.

Summary

In just six months, iQIYI has deployed thousands of Dubbo instances, contributed 18 pull requests (15 merged) to the Dubbo and Sentinel ecosystems, and cultivated a Dubbo committer. The internal Dubbo services run stably without major incidents. iQIYI will continue to enhance Dubbo’s governance features, promote its adoption, and feed back practical experience to the open‑source community.