An Overview of ChatGPT’s Software Architecture and Technology Stack

ChatGPT has ignited a wave of general AI, comparable to past revolutions in agriculture, industry, and computing, and may herald a new AI technology revolution.

While the industry focuses on ChatGPT’s AI algorithms, OpenAI has evolved into a platform serving hundreds of millions of users, maintaining an approximate 99% SLA over the past three months, which highlights the importance of its overall technical architecture and reliability.

Because OpenAI has not publicly disclosed a detailed architecture diagram, this article attempts to reconstruct the stack using public internet information, recent job postings, incident reports, and GitHub code.

OpenAI began as an AI startup in the cloud era, initially building its services on public‑cloud AWS with extensive use of AWS services, as shown in early OpenAI documentation.

In recent years, after a large investment from Microsoft, OpenAI is transitioning to a multi‑cloud strategy centered on Azure, while still employing Terraform for cloud‑resource management.

The core business data of ChatGPT is stored in a relational PostgreSQL database, with PgBouncer used as a connection‑pooling service; a February 2023 incident report confirmed PostgreSQL issues.

Redis clusters serve as caching layers, and a March 2023 security incident involving a Redis‑py bug exposed user conversations.

OpenAI’s job listings also mention experience with Azure CosmosDB (a multi‑model NoSQL service) and hint at possible use of Cassandra APIs, though this remains unconfirmed.

Additionally, ChatGPT leverages Snowflake as a cloud‑native data warehouse and Tableau for analytics, facilitating cross‑cloud data migration.

On the front‑end, the product uses TypeScript and the React framework; mobile apps are under development, as indicated by recent iOS and Android engineering hires.

Python is the primary language for platform services, with Flask and OpenAPI components forming the core of the backend.

The AI stack centers on PyTorch and possibly TensorFlow, while OpenAI’s own Triton framework—implemented in C++ and Python—replaces CUDA for more efficient GPU programming.

For deployment and operations, ChatGPT runs on Kubernetes, monitors with Prometheus (requiring PromQL expertise), logs with Splunk, and includes services written in Golang and Python.

References include OpenAI job descriptions, the February 2023 incident analysis, the Triton GitHub repository, and a LinkedIn discussion about potential Cassandra usage.