Ant Financial's Open Computing Architecture for Data Intelligence and AI

Ant Financial reflects on fifteen years of technology innovation that reshaped payments for over 1.2 billion users and introduces a series of talks from the 2019 Hangzhou Cloud Conference, now published on the "Ant Financial Technology" public account.

The company tackles two core technical challenges: moving money between accounts at massive scale while ensuring security, availability, and disaster‑recovery through a multi‑site distributed architecture, and leveraging data‑driven intelligence in the new digital finance era.

Financial data intelligence demands differ from traditional big data: high real‑time requirements, complex and diverse compute scenarios (rules, graph, machine learning), long data pipelines with low debugging efficiency, high‑availability storage and compute, and strict security, compliance, and risk‑control measures.

As compute technologies evolved from batch data warehouses to real‑time streaming and interactive analysis, new challenges emerged, such as multiple compute modes, diverse storage costs, and complex disaster‑recovery and security requirements, prompting the need for a more open compute architecture.

The proposed open computing architecture provides three unifications: a unified storage layer that connects various storage systems for data sharing and custom optimizations; a unified data‑security specification that offers consistent metadata management, lineage, authentication, access control, and privacy protection; and a unified programming model based on standard SQL (with extensions) that abstracts underlying compute modes, allowing developers to write concise, data‑centric code.

AI capabilities are integrated via SQLFlow, which translates SQL statements into machine‑learning tasks, and ElasticDL, an open‑source TensorFlow‑based elastic training engine that ensures efficient, fault‑tolerant AI model training even under resource constraints.

For graph computing, Ant Financial built GeaBase, a financial‑grade distributed graph database with strong consistency and high capacity, along with large‑scale full‑graph processing using adaptive partitioning, a high‑performance graph cache, and the AntGraph platform that unifies access through Graph SQL.

Fusion computing, built on the Ray engine, combines multiple compute modes (stream, graph, machine learning) into a single engine, delivering sub‑second latency for dynamic graph inference, online decision‑making, and online machine‑learning updates across various financial scenarios.

The overall vision, termed "Big Data Base," aims for a plug‑and‑play, interoperable ecosystem where storage, compute engines, and data access patterns are standardized, enabling seamless integration of future technologies and advancing financial data intelligence to the next stage.