Overview of Lustre Parallel File System Architecture and Performance Characteristics

Lustre is a scalable, high‑performance parallel file system that runs on Linux and offers a POSIX‑compatible UNIX interface, making it suitable for large‑scale clusters with thousands of clients and petabyte‑level storage.

The architecture comprises a Management Server (MGS), Metadata Servers (MDS) with Metadata Targets (MDT), Object Storage Servers (OSS) with Object Storage Targets (OST), and Lustre clients connected via the custom LNet network API.

Key features include POSIX compliance, support for high‑speed RDMA networks, active‑active high availability, ACL‑based security, object‑based storage, fine‑grained locking, quotas, controlled file layout, network data integrity, MPI I/O, NFS/CIFS export, online distributed filesystem check (LFSCK), performance monitoring, and an open‑source GPL‑2.0 license.

Since Lustre 2.0, a 128‑bit File Identifier (FID) replaces traditional inode numbers, and the system supports configurable striping (stripe count and size) across multiple OSTs, enabling large file bandwidth aggregation and flexible capacity growth.

Performance analysis shows that Lustre excels with large sequential I/O workloads, offering superior write performance due to asynchronous client writes and zero‑copy mechanisms, while small‑file workloads suffer from metadata overhead and limited read caching.

Optimization recommendations for small‑file scenarios include aggregating files into larger archives, using O_DIRECT I/O, ensuring sequential access patterns, deploying SSD‑backed OSTs, employing RAID 1+0, disabling unnecessary LNet debugging, increasing client dirty cache size, and leveraging loopback mounts for scratch spaces.