Mastering rsync: Efficient Ways to Exclude Files and Directories

Introduction

Rsync is a fast, versatile command‑line utility that synchronizes files and directories between two locations over a remote shell. It can mirror data, create incremental backups, and copy files across systems, while allowing you to exclude specific files or directories.

Preparation

We use the -a option, which recursively syncs directories, preserves symbolic links, timestamps, groups, owners, and permissions. When excluding items, you must provide their paths relative to the source directory.

Two primary options let you specify exclusions:

Use --exclude directly on the command line.

Use --exclude-from to read exclusions from a file.

Excluding Specific Files

Pass the relative path of the file to --exclude. Example:

rsync -a --exclude 'file.txt' src_directory/ dst_directory/

Excluding Specific Directories

Provide the directory’s relative path to --exclude. Example:

rsync -a --exclude 'dir1' src_directory/ dst_directory/

To exclude the contents of a directory but keep the directory itself, use 'dir1/*' instead of 'dir1':

rsync -a --exclude 'dir1/*' src_directory/ dst_directory/

Excluding Multiple Files or Directories

Specify multiple --exclude options:

rsync -a --exclude 'file1.txt' --exclude 'dir1/*' --exclude 'dir2' src_directory/ dst_directory/

Or use a single --exclude with a brace list:

rsync -a --exclude{'file1.txt','dir1/*','dir2'} src_directory/ dst_directory/

If you have many items to exclude, create a file and use --exclude-from:

rsync -a --exclude-from='exclude-file.txt' src_directory/ dst_directory/

Contents of exclude-file.txt:

file1.txt</code>
<code>dir1/*</code>
<code>dir2

Excluding by Pattern (Regex)

You can exclude files matching a pattern, e.g., all .jpg files:

rsync -a --exclude '*.jpg*' src_directory/ dst_directory/

To include only .jpg files while excluding everything else, combine include and exclude options:

rsync -a -m --include='*.jpg' --include='*/' --exclude='*' src_directory/ dst_directory/

Explanation of the options: --include='*.jpg' – first includes all .jpg files. --include='*/' – then includes all directories so the .jpg files inside them are reachable. -m – removes empty directories.

Another method is to pipe the output of find into rsync, which can be easier for operations staff:

find src_directory/ -name "*.jpg" -printf '%P\0
' | rsync -a --files-from=- src_directory/ dst_directory/

-printf %P\0

strips the leading source directory from the paths, and --files-from=- tells rsync to read the file list from standard input.

Conclusion

After mastering these exclusion techniques, rsync often becomes the preferred tool over scp for routine file and directory synchronization between local machines and servers.