An Overview of Greenplum Database Architecture and Core Components

1. Introduction to Greenplum

Greenplum Database (GPDB) is an advanced open‑source distributed database designed for large‑scale data analysis, data warehousing, OLAP, and data mining. Since its open‑source release in October 2015, it has attracted wide attention.

2. Greenplum Architecture

2.1 Platform Architecture

GPDB follows a four‑layer architecture (hardware, interconnect, storage, service). The platform includes an MPP core, advanced optimizers (PostgreSQL planner‑based and the ORCA optimizer), polymorphic storage, and a software switch for high‑performance data flow.

GPDB is a massive shared‑nothing parallel processing system.

It supports two optimizers: the traditional PostgreSQL planner and the newer ORCA optimizer.

Polymorphic storage automatically selects the best storage format (row, column, or external) based on access patterns.

Parallel data‑flow engine provides redistribution and broadcast operators.

The software switch implements reliable UDP communication between nodes.

Scatter/Gather engine handles parallel data loading and export.

2.2 Service Layer

GPDB offers multi‑level fault tolerance and high availability: standby master for master failover, mirrored segment nodes with filerep, and network redundancy with multiple NICs and switches. It also supports online expansion, task management, and resource monitoring.

2.3 Core Features

Full ANSI SQL 2008 and SQL‑OLAP 2003 support, with ODBC/JDBC APIs.

Distributed ACID transactions.

Linear scalability to hundreds of nodes.

Enterprise‑grade deployment in finance, government, logistics, retail, etc.

Derived from PostgreSQL 8.2, with roughly 1.3 million lines of source code.

Rich ecosystem integrations (SAS, Cognos, Tableau, Pentaho, Talend, etc.).

Polymorphic storage (row, column, external tables).

Multiple compression methods, partitioning, indexes, and authentication (LDAP, Kerberos, ACL).

Extensible with languages such as Python, R, Java, Perl, C/C++.

Geospatial support via PostGIS.

Built‑in data‑mining algorithms (MADLib) and full‑text search (GPText).

2.4 Client Access and Tools

Clients can connect via psql, ODBC, JDBC, OLEDB, or libpq. Management tools include the graphical Greenplum Command Center (GPCC) and the Greenplum Workload Manager for rule‑based resource control.

2.5 Parallel Query Planning and Execution

Queries are parsed, optimized (by ORCA or planner), and dispatched from the master (QD) to segment executors (QE). Execution slices are coordinated as gangs, with data flowing upward through the interconnect before results are returned to the client.

2.6 Polymorphic Storage

GPDB stores data using row storage, column storage, or external tables (e.g., HDFS), selecting the optimal format per table or per data segment.

2.7 Massive Parallel Data Loading

GPDB provides high‑throughput parallel loading (DCA) supporting various sources (Hadoop, file systems, databases) and formats (text, CSV, Parquet, Avro).

3. Core Components

Parser – lexical and syntactic analysis of SQL.

Optimizer – selects the best execution plan (ORCA).

Scheduler (QD) – distributes plans to segment executors.

Executor (QE) – performs scans, joins, aggregates, etc.

Interconnect – handles node‑to‑node data transfer.

System catalogs – store metadata on each node.

Distributed transaction manager – implements two‑phase commit.

4. Open‑Source Release

Greenplum was open‑sourced in October 2015 under the Apache 2.0 license. The project’s website, source code repository, sandbox tutorials, and mailing lists are publicly available for community contributions.

Website: http://greenplum.org

Source code: https://github.com/greenplum-db/gpdb