Choosing Between Ceph and Swift for OpenStack Storage: Architectural Considerations

Deep Dive

Swift originated early in OpenStack development about five years ago and is a core project proven to be robust and stable. However, its design leads to sub‑optimal transfer speed and latency because all traffic passes through proxy servers. Additionally, Swift does not support block or file storage, and its eventual consistency model can cause readers to see stale data after an update.

Ceph, while popular, has its own challenges in cloud environments. Its multi‑region support is often marketed as an advantage but actually follows a master‑slave model; replication occurs only from master to slave, resulting in uneven load distribution when more than two regions are involved. The two‑region design also permits writes only on the master, risking cluster instability if slaves write. Security is another concern: the RADOS client uses the same network for client‑to‑server communication and unencrypted replication traffic, so a compromised client can capture all storage network traffic. Ceph requires quorum for synchronous writes, and in a two‑region scenario with high latency, write operations can be delayed significantly.

In a comparable two‑region setup, Swift writes locally first and then replicates to the remote region based on its consistency design. Although Swift also requires a write quorum, it can be configured with write_affinity to force the quorum to the local region, allowing the write to return as soon as the local copy succeeds.

How to Choose?

If the deployment is single‑region with no plans for expansion, Ceph is a solid choice, and Mirantis OpenStack can use Glance or Cinder on top of it. For large‑scale, multi‑region deployments, Swift’s design offers better performance and consistency. When security is the primary concern, Swift’s closed replication network is advantageous; if the underlying cloud infrastructure is already secure, Ceph may be preferable.

A hybrid approach can also be employed: using Ceph for high‑performance local storage and Swift as a multi‑region Glance backend, though this incurs higher cost and complexity, often leading to a decision between the two. Consulting services such as Mirantis’s architecture design assessment can help evaluate requirements and guide the selection.