DBOS – Database‑Oriented Operating System

In 2021 Matei Zaharia, founder of Databricks and creator of Spark, highlighted a scalability problem: managing millions of Spark tasks on PostgreSQL for scheduling and state tracking.

PostgreSQL, while mature, struggled with the high write load of tracking task states across a distributed cluster.

Inspired by this, Zaharia and Michael Stonebrake proposed a new architecture where the operating system is built on top of a distributed database, termed DBOS (Database‑Oriented Operating System).

DBOS consists of four layers: a microkernel for low‑level services; a high‑performance multi‑node distributed database; OS services (distributed file system, scheduler, IPC) implemented on the DBMS; and user applications such as Spark.

Key characteristics of DBOS are that all system and application state is stored in database tables and accessed solely via transactional operations.

This design offers benefits including simple scaling of task tracking, strong consistency and durability via ACID transactions, comprehensive logging for debugging, and a reduced attack surface.

An example workflow demonstrates DBOS’s ability to reliably execute delayed actions even across crashes:

@DBOS.workflow()
def reminder_workflow(email: str, time_to_sleep: int):
    send_confirmation_email(email)
    DBOS.sleep(time_to_sleep)
    send_reminder_email(email)

The article also references earlier attempts to integrate databases with operating systems, such as IBM’s AS/400 and Microsoft’s WinFS, and discusses why DBOS may represent a future direction for cloud‑native operating systems.