Why Does du Show 9 GB While ls Shows 100 GB? Understanding Linux Sparse Files

Yesterday a developer asked for a log file; using

du -hs smartorder.log

I saw 9.0G, compressed it to 744M, and sent it. Later the developer showed a screenshot indicating the file was actually 100 GB.

<code># du -hs smartorder.log
9.0G smartorder.log</code>

<code># du -hs smartorder.log.tar.gz
744M smartorder.log.tar.gz</code>

Checking with

ls -l --block-size=G smartorder.log

revealed the true size of 103 GB.

<code># ls -l --block-size=G smartorder.log
-rw-r--r-- 1 root root 103G Oct 21 09:00 smartorder.log</code>

The discrepancy is due to Linux sparse files, which allocate disk space lazily. A sparse file initially occupies no physical blocks; space is allocated only as data is written, typically in 64 KB increments. This results in a logical size far larger than the actual disk usage.

To copy sparse files efficiently, use

cp --sparse=WHEN

where

WHEN

can be

auto

,

always

, or

never

. Other tools that support sparse files include

tar

,

cpio

, and

rsync

. For example:

<code># tar cSf smartorder.log.tar smartorder.log
# ls -l --block-size=G smartorder.log.tar
-rw-r--r-- 1 root root 10G Oct 21 09:57 smartorder.log.tar</code>

To detect whether a file is sparse, the

find

command can report the ratio of allocated blocks to file size using the

%S

format:

<code># find ./smartorder.log -type f -printf "%S\t%p\n"
0.0886597 ./smartorder.log</code>

A value less than 1.0 indicates a sparse file. To locate all sparse files on a filesystem:

<code>find / -type f -printf "%S\t%p\n" | gawk '$1 < 1.0 {print}'</code>

Understanding sparse files helps avoid misleading size reports and ensures proper handling during backup or transfer operations.