How a Leading Bank Evolved Its Big Data Platform Architecture

Big data is frequently mentioned in banking and has become a development trend. For banks, big data makes customers more transparent. Our big‑data system started construction in 2014 and is now entering a mature development stage.

1. Big Data Positioning

Since 2014 we have built eight big‑data products that make customers more transparent and improve risk control. The products have already generated tangible benefits.

We are building a customer‑centric financial service platform based on big data and AI. All eight products are completed, enabling precise marketing and real‑time risk identification.

The platform is now mature and will further develop toward AI.

2. Big Data System Planning

As business and demand grew, we encountered three main problems:

Business demand is urgent but software delivery lags expectations.

Existing system architecture is complex and hard to scale during peak periods.

Rapid emergence of new technologies raises team collaboration and skill requirements.

We addressed these by improving system architecture, adopting agile delivery, and establishing an efficient talent organization.

Optimized architecture became high‑concurrency, easily extensible, and highly available. Agile development achieved high‑quality software delivery.

Technical strategy includes distributed architecture, micro‑service front‑end transformation, and layered back‑end data architecture.

3.1 Platform Architecture Optimization

We split the overall architecture into front‑end application architecture and back‑end data architecture. The front‑end consists of data services and data products; the back‑end follows a data‑factory model, processing data and exposing results via a middle layer.

After the split, data flows 24/7, supporting all channels (online banking, mobile, WeChat) in real time.

We transformed monolithic applications into micro‑services, introduced an API gateway, and layered services into component, common, and business services.

Micro‑services run in containers on an eight‑node cluster behind an F5 load balancer; traditional deployment remains as a fallback.

We built base‑image, application‑image, and version‑image pipelines to support flexible deployment across environments.

Data is now layered and processed through a data pipeline, enabling consistent quality control and delivering ready‑to‑use data products to front‑end applications.

3.2 Delivery Model Improvement

To accelerate delivery we introduced a continuous delivery pipeline covering code management, baseline control, automated build review, automated testing, and environment deployment.

Code submissions undergo review; resulting WAR packages follow two paths: one to traditional test and production environments, another to container environments where they are tested and deployed.

We use tools such as FindBugs, JUnit, Jenkins for integration testing, and SonarQube for quality reporting.

Kanban visualizes demand flow, resource allocation, and progress, helping identify bottlenecks.

3.3 Team Building

Beyond technical improvements we reorganized the team following DevOps principles, grouping members by product line and breaking down silos between business, development, operations, big‑data, and modeling teams.

Product managers act as service owners, coordinating requirements and version iterations every two weeks.

We refined roles into seven distinct positions and introduced capability assessment models to match engineers with appropriate growth paths.

4. Application Cases

These evolutions have yielded positive impacts, such as a risk‑control case where big‑data analysis and relationship mining enable locating previously unreachable credit‑card customers, improving collection and repayment.

Another case involves operational analysis that detects risky transactions, intrusion attempts, and password attacks by analyzing network traffic and data streams.