Evolution of JD.com Product Architecture: From Database Replication to Platformized Service Architecture (V1.0‑V3.0)

Author: You Fengkai, JD.com R&D leader for the transaction platform and product development. Joined JD in 2010, contributed to early monitoring, messaging, and EDM systems, and later led the product middle‑platform and componentization efforts.

Product data is the core of e‑commerce; accurate, reliable product services are essential for the entire supply chain, from merchant publishing to order settlement and after‑sales.

Initially JD used a multi‑level SQL Server replication setup. Replication delays could exceed 12 hours, causing severe user‑experience issues. Multiple systems (ERP, POP, BIP) wrote directly to the same tables, leading to data inconsistency and inability to handle large promotional traffic.

In March 2012 the product team split from the website team and launched an architecture upgrade. The 618 shopping festival highlighted the need for a more robust solution. Early 2012, Redis was introduced as an in‑memory store; a custom client based on Jedis was built, adding a sharded connection pool and pipelined command execution, dramatically improving read performance.

Architecture 1.0 – Read Service

All product data was cached in Redis (master‑slave mode) with incremental updates and a full‑refresh fallback. An HTTP‑based notification system (Rcat‑server) received change events from the SQL Server primary, updated Redis, and an asynchronous worker periodically reconciled missed updates.

During the 2012 618 event, the read‑service upgrade performed flawlessly, prompting other systems to adopt Rcat.

Architecture 2.0 – Full Service

Both POP and self‑operated product systems originally wrote to Oracle, then synced to SQL Server. To avoid the cost and DBA dependency of Oracle, the team migrated all product data to SQL Server, planning a later migration to a MySQL cluster.

The full‑service roadmap included:

Comprehensive read service: Redis cache plus Solr indexes for search queries.

Write service: Unified product write API, supporting both self‑operated and POP models, with a unified SKU data model.

Elimination of Oracle ("去O") to reduce latency.

Asynchronous engine: Unified notification to downstream systems (WMS, search, etc.) via configurable MQ messages.

Storage upgrade: From a single SQL Server instance to a sharded MySQL cluster with vertical and horizontal partitioning, custom middleware for routing and failover.

These changes alleviated write‑service and downstream notification bottlenecks, but coupling between write and dispatch services still caused occasional performance spikes.

Architecture 3.0 – Platformization & Precision

Starting mid‑2015, the team pursued a decoupled, simplified platform. Jingobus was built to read binlog changes, push notifications, and refresh Redis clusters. Configurable message routing reduced unnecessary traffic (e.g., WMS traffic cut by >80%).

A second‑level monitoring platform was created to collect, aggregate, and alert on memory‑level metrics within three minutes, ensuring rapid incident response.

Read service continued to scale: billions of SKUs required multiple Redis clusters, a NIO‑based client with unified sharding management, automatic failover, and load balancing. A three‑tier cache hierarchy placed hot data in application memory, warm data in a smaller Redis cluster, and cold data in a larger cluster, reducing overall OPS.

JD.com’s product system will keep advancing in business innovation, data intelligence, performance, and stability, embodying the vision of technology improving life.