Container‑Defined Storage: Enabling Lightweight Data Centers and Cloud‑Native Architectures

Container technology has become one of the fastest‑adopted innovations in data centers, with Docker downloads approaching 3 billion since 2013, fundamentally changing application deployment while infrastructure management lags behind.

This article examines the application of container technology to storage. Beyond EMC Unity’s use of containers for NAS and replication, emerging startups are offering Container‑Defined Storage (CDS) products. The first part focuses on how containers transform deployment and why CDS can achieve a "lightweight" data center; the second part will explore a specific CDS product from Portworx.

Portworx, a US‑based storage startup, created the industry’s first container‑defined storage system that separates control and data planes, delivering enterprise‑grade features with performance comparable to bare metal.

Modern SDS and SBS solutions make it easy to acquire storage capacity from commercial servers, enabling converged architectures where compute and storage coexist. However, these solutions lack a newly designed software stack for deploying applications and containers that fully leverages commoditized x86 compute and micro‑service trends.

Traditional enterprise storage requires deep expertise in FC, SAN switches, iSCSI, NFS, CIFS, FCoE, etc., making it difficult for teams without specialized knowledge to obtain enterprise‑grade services.

Visuals illustrate the complexity of legacy storage. In contrast, modern data centers like Google and Facebook build distributed, fault‑tolerant storage on scale‑out x86 servers, providing easy‑to‑use services.

New application models (cloud‑native, DevOps, micro‑services) impose fresh storage requirements, including service‑oriented storage provisioning, finer‑grained and larger numbers of volumes, multi‑layered local storage, hyper‑convergence of compute and storage, and service‑level storage operations across distributed nodes.

Current hardware in data centers consists of heterogeneous servers from multiple vendors, each with different storage tiers. To support many thin services, storage must be able to place data intelligently across containers and be aware of distributed x86 clusters.

Metadata should be distributed and decentralized, using load‑balancing or gossip protocols to achieve high‑availability and self‑healing without a central metadata server.

The gossip protocol spreads node state changes rapidly through a peer‑to‑peer broadcast, ensuring efficient communication.

To make storage container‑aware, storage schedulers must integrate with container engines. Modern schedulers such as Kubernetes, Mesosphere, Swarm, and Spark automate software lifecycle management, aligning with DevOps practices.

Deep integration of container orchestrators with software‑defined storage and container‑defined storage enables data centers to become truly lightweight.

Containers run multiple applications on a single OS, offering low resource consumption, fast startup, and easy deployment, reducing hardware and space requirements. However, containers remain largely stateless, limiting their adoption for stateful data‑center workloads because existing containers are optimized for stateless workloads and lack robust state management.

Designing distributed, extensible storage architectures must address service‑oriented data‑center needs, supporting stateless services, stateful database services, and file/object storage for stateful applications.

Today’s cloud‑native applications are built on Docker‑based infrastructures rather than being migrated from legacy physical systems, demanding storage that matches micro‑service and container deployment patterns.

The article concludes by previewing the next part, which will dive into Portworx’s container‑defined storage technology and product details.