Comparing NewSQL Databases with Middleware‑Based Sharding: Advantages, Trade‑offs, and Selection Guidance

Recently, the author has been asked many times about how to choose between sharding (splitting databases and tables) and distributed NewSQL databases. This article aims to objectively compare the two approaches by analyzing their key characteristics, implementation principles, advantages, disadvantages, and suitable scenarios.

What makes NewSQL databases advanced?

According to the paper pavlo-newsql-sigmodrec , NewSQL architectures can be classified into first‑generation new‑architecture types (e.g., Spanner, TiDB, OceanBase) and second‑generation middleware solutions such as Sharding‑Sphere , Mycat , DRDS . The author argues that middleware + traditional relational databases (sharding) also constitute a distributed architecture because storage is distributed and horizontal scaling is possible, but it may be considered a “pseudo” distributed database due to duplicated SQL parsing, execution‑plan generation, and B+Tree‑based storage engines.

NewSQL databases differ from middleware‑based sharding in several ways:

Traditional databases are disk‑oriented, while NewSQL makes more efficient use of memory.

Middleware repeats SQL parsing and optimization, leading to lower efficiency.

NewSQL optimizes distributed transactions compared with XA, achieving higher performance.

NewSQL stores data using Paxos (or Raft) multi‑replica protocols, providing true high availability (RTO < 30 s, RPO = 0).

NewSQL natively supports automatic sharding, data migration, and scaling without requiring application‑level sharding keys.

The article then delves into each of these points in detail.

Distributed Transactions

This is a double‑edged sword.

CAP Limitation

Many NoSQL databases originally omitted distributed transactions due to the CAP theorem, which forces a trade‑off between consistency, availability, and partition tolerance. NewSQL does not break CAP; for example, Google Spanner claims to be effectively CA by operating on a private global network that minimizes partitions.

Completeness

Two‑phase commit (2PC) can struggle to guarantee strict ACID properties under failures; recovery mechanisms can only ensure eventual consistency after faults. Some NewSQL products still have incomplete transaction support, as observed in real‑world tests.

Performance

Traditional relational databases use XA, which incurs high network overhead and blocking time, making it unsuitable for high‑concurrency OLTP. NewSQL often implements optimized 2PC models such as Google Percolator, using a Timestamp Oracle (TSO) with MVCC and Snapshot Isolation, plus primary/secondary locks to make part of the commit asynchronous, thereby improving performance over XA.

SI is optimistic locking; in hot‑spot scenarios it may cause many aborts, and its isolation level differs from Repeatable Read.

Nevertheless, the extra GID acquisition, network cost, and log persistence in 2PC still cause noticeable performance loss, especially when many nodes participate.

HA and Multi‑Active Deployment

Traditional master‑slave replication (even semi‑synchronous) can lose data under failure. Modern solutions adopt Paxos or Raft multi‑replica protocols (e.g., Google Spanner, TiDB, CockroachDB, OceanBase) to achieve automatic leader election, high reliability, and fast failover. Some vendors also retrofit MySQL with Group Replication to achieve similar goals.

Implementing production‑grade consensus protocols requires multi‑Paxos or multi‑Raft optimizations such as batching and asynchronous I/O.

While Paxos‑based multi‑active setups are theoretically possible, they demand low inter‑region latency; otherwise, the added delay makes true active‑active OLTP impractical.

Scale (Horizontal Expansion) and Sharding Mechanism

NewSQL databases embed automatic sharding; they monitor region load (e.g., TiDB splits a region at 64 MiB) and migrate data transparently. In contrast, middleware‑based sharding requires upfront design of sharding keys, routing rules, and manual scaling procedures, increasing application complexity.

Online scaling for sharding can be achieved via asynchronous replication, read‑only switches, and routing updates, but it still depends on coordinated middleware and database actions.

However, built‑in sharding strategies may not align with domain models, potentially causing distributed transactions.

Distributed SQL Support

Both approaches handle single‑shard SQL well. NewSQL offers richer cross‑shard capabilities (joins, aggregations) thanks to global statistics and cost‑based optimization (CBO). Middleware typically relies on rule‑based optimization (RBO) and may lack efficient cross‑shard query support.

Storage Engine

Traditional engines use B+Tree, optimized for disk reads but suffering from random‑write overhead. NewSQL often adopts LSM‑tree, converting random writes into sequential writes, improving write throughput at the cost of more complex reads. Additional techniques (SSD, bloom filters) mitigate read penalties.

Maturity and Ecosystem

NewSQL is still evolving, with strong adoption in internet companies but less proven in high‑risk industries. Traditional RDBMS benefit from decades of stability, extensive tooling, and broader DBA talent pools. Choice depends on growth pressure, willingness to adopt new tech, and the need for transparent scaling.

Conclusion

If you answer “yes” to several of the following questions—strong consistency needed at the database layer, unpredictable data growth, frequent scaling beyond DBA capacity, throughput over latency, and desire for application transparency—consider a NewSQL solution despite its learning curve. Otherwise, middleware‑based sharding remains a lower‑risk, lower‑cost option that leverages mature relational ecosystems.

Both paths have trade‑offs; NewSQL is not a silver bullet, and sharding remains a viable, high‑availability strategy for many traditional enterprises.