Choosing Between Commercial and Open‑Source Databases: Insights from a Veteran DBA

This article records a discussion from the “Efficient Operations” WeChat group, where Lu Dongming, a veteran database professional, answers a question about the characteristics of commercial versus open‑source databases and how to choose among them.

Q1: From Lu’s perspective, what are the characteristics of commercial and open‑source databases, and how should one select a commercial database product?

Database’s First Spring (1995‑2005)

The early 1990s saw relational databases, led by Oracle, reshape the industry, spawning many competing vendors.

Sybase introduced the Client/Server architecture, popularizing network databases.

Informix (later merged with RedBrick) contributed parallel processing (MPP) and massive data handling.

Microsoft acquired SQL Server source from Sybase and built Microsoft SQL Server for the NT platform.

Other notable players included IBM DB2, DEC RDB, Teradata, and Tandem Non‑Stop SQL.

The competition focused on OLTP workloads such as stock trading and airline ticketing.

Database’s Second Spring

In the following decade, the market consolidated, with Oracle emerging dominant while other vendors either merged or shifted focus.

As OLTP challenges were largely solved, enterprise ERP deployments caused data volumes to explode, giving rise to “big data” needs that commercial vendors struggled to address.

Open‑source technologies, especially Hadoop, gained prominence, leading to the NoSQL movement (later rebranded as NewSQL) and intensifying competition.

Key milestones of this second spring include:

Column‑store databases such as Sybase IQ delivering superior analytical performance.

Row‑store optimizations like Oracle Exadata integrating storage to handle hybrid transactional/analytical workloads.

MPP technologies exemplified by Teradata, Greenplum, and Netezza advancing large‑scale data processing.

Open‑source big‑data platforms such as Hadoop and BigTable powering major internet companies.

Commercial vs Open‑Source Databases

Lu’s view on the current state of the competition:

It is healthy.

It is roughly balanced.

Both camps have strengths and weaknesses; there is no clear winner.

When selecting a database or big‑data architecture, one must calmly assess data characteristics, business requirements, and choose the appropriate product and design.

Advantages of Commercial Databases

Pros: mature development and testing processes, comprehensive training, service, and implementation support, higher product maturity, and lower risk of errors.

Cons: slower innovation cycles, lengthy decision chains, limited internal advocacy, and potential monopolistic tendencies that can stifle new technology adoption.

Advantages of Open‑Source Databases

Pros: rapid iteration, democratic decision‑making, strong fit for novel scenarios, especially in internet‑scale environments.

Cons: fragmented architecture, lack of holistic design, niche focus that may require highly skilled DBAs, and challenges scaling to enterprise‑level problems.

“Facebook uses a technology, so I use it; LinkedIn uses a technology, so I use it; Google uses a technology, so I use it.”

Lu cautions against blindly copying technologies without considering unique workload requirements.

Key Selection Criteria

OLTP workloads: prioritize total cost of ownership (TCO) when dealing with high‑throughput transactional systems.

OLAP workloads: choose between column‑store solutions with advanced indexing or in‑memory analytics (e.g., SAP HANA, Oracle’s in‑memory options).

MPP and cloud‑native solutions: open‑source often leads, with offerings from AWS, Alibaba Cloud, and large enterprises providing integrated services.

Complex Event Processing (CEP) and Event Streaming: technologies such as Sybase ESP, Apama, StreamBase, Apache Storm, and Spark Streaming deserve attention.