Deep Dive into Ant Financial’s Service Mesh Data Plane SOFAMosn

Preface

The article introduces the deep technical analysis of Ant Financial’s Service Mesh data‑plane SOFAMosn, a Golang‑based proxy designed to replace Envoy in their large‑scale microservice environment.

Background

Ant Financial adopts Service Mesh to address four key needs: full microservice and cloud‑native adoption, advanced traffic scheduling for operations, strict security requirements due to financial attributes, and the high cost of multi‑language integration.

SOFAMesh Overall Architecture

The presented diagram follows the Istio architecture, substituting Envoy with SOFAMosn and adding Ant Financial‑specific enhancements. Version 0.1.0 supports xDS V0.4, the SOFARPC protocol, and basic HTTP/1.1 and HTTP/2.0 functionality.

Core Design Philosophy

SOFAMosn processes inbound traffic through four layers: Network I/O, Binary Protocol, Protocol Flow, and Forwarding/Routing; outbound traffic follows the reverse order.

The four layers provide:

I/O layer – encapsulated read/write with extensible event subscription.

Protocol layer – serialization/deserialization for various protocols.

Streaming layer – protocol consistency, stream lifecycle management, and client‑side stream pooling.

Proxy layer – routing, load‑balancing, and stream forwarding.

Module Division

Beyond the four core layers, additional modules handle routing, backend management, and health checks. The diagram highlights current (blue) and planned (red dashed) functionalities, inviting community contributions.

Core Capabilities

SOFAMosn offers programmable I/O interfaces, a built‑in TCP proxy, TLS support (leveraging Golang’s implementation), TProxy mode via iptables, and smooth reload/upgrade capabilities.

Protocol support includes SOFARPC (used in production), HTTP/1.1 (FastHTTP), HTTP/2.0 (Golang net/http2), with future plans for Dubbo and HSF. All supported protocols can use Mesh‑level TLS encryption.

Routing features align with Envoy: virtual host matching, route matching, and subset load‑balancing. Backend management provides basic load‑balancing algorithms and active health checks.

Additional highlights: X‑PROTOCOL for lightweight custom RPC forwarding, programmable codec extensions, protocol‑agnostic connection migration, and Istio 0.8 Pilot V0.4 API dynamic configuration.

Performance

The article details single‑core performance optimizations for TCP and SOFARPC forwarding, including CPU pinning, memory slab‑style recycling, and careful management of Golang’s GC and runtime scheduling.

IO tuning recommendations cover minimizing SetReadDeadline calls, appropriate write buffering, and balancing read/write frequencies across goroutines.

Benchmark results show SOFAMosn 0.1.0’s single‑core TCP and SOFARPC forwarding performance comparable to Envoy 1.7, with ongoing multi‑core and memory optimizations.

TLS performance tests reveal Golang’s native ECDHE‑P256 implementation outperforms Go‑with‑boring‑SSL and approaches Nginx/OpenSSL performance.

Roadmap

Planned releases include:

August 2020 – SOFAMesh 0.1.0: core proxy, xDS V0.4, SOFARPC.

Late August 2020 – 0.2.0: X‑PROTOCOL extensions, Dubbo/HSF support, K8s operator integration, HTTP/2.0 enhancements.

September 2020 – 0.3.0: Mixer integration, quota, reporting, circuit breaking, and rate limiting.

Continuous efforts will focus on multi‑core performance, memory optimization, code quality, and testing.

Conclusion

SOFAMosn is an early‑stage, high‑throughput, programmable data‑plane for Service Mesh, designed with modularity, extensibility, and cloud‑native principles, and the project welcomes contributions from the open‑source community.