JD Retail Big Data Cloud‑Native Platform Practice

In this talk, JD architect Liu Zhongwei (with editorial support from Zhang Mingyu) shares JD Retail’s practical experience of building a cloud‑native platform for big data, organized by the DataFunTalk community.

The speaker first clarifies the definition of cloud‑native, citing the CNCF description and noting that definitions evolve over time; Pivotal’s 2019 definition adds DevOps, continuous delivery, micro‑services, and containers. He emphasizes that cloud‑native should be understood as “born in cloud, grow in cloud,” distinguishing the cloud (infrastructure services) from the native (design‑time considerations).

The evolution of cloud‑native technologies is traced from Docker (2013) and early Kubernetes (2014) through CNCF’s formation, the rise of Istio, Knative, and recent projects such as KubeVela, illustrating the shift from containerization to service mesh and serverless paradigms.

JD’s big‑data cloud‑native practice is then detailed. After evaluating Knative, service mesh, and Operators, the team chose a PaaS layer built on Kubernetes with a unified model rather than per‑product Operators. The business model is split into three layers: components (e.g., Yarn NodeManager), application templates (combinations of components), and system templates (full big‑data stacks such as HBase, Spark, etc.).

The platform provides many capabilities via sidecar containers, abstracts configuration and scripts within components, and translates a custom Application CRD into native Kubernetes resources (Pods, Services, ConfigMaps, Volumes). A declarative controller coordinates workflows, exposing hooks for custom actions and supporting multi‑shard deployments with affinity rules.

Finally, the speaker reviews the development of cloud‑native application platforms, mentioning Heroku, Cloud Foundry, OpenShift, domestic projects like Rainbond and Kubesphere, and KubeVela, and explains why JD’s solution differs by focusing on a platform rather than a development framework. He encourages attendees to consider how their own cloud‑native platforms should be “born in cloud” and continue to “grow in cloud.”p>