Mastering Rsync: Fast File Sync Across LAN/WAN with Daemon and Inotify

1. Introduction

1.1 What is Rsync

Rsync (remote synchronize) is a remote data synchronization tool that can quickly sync files between multiple hosts over LAN/WAN. It uses the “rsync algorithm” which transfers only the differences between files, making it very fast.

Rsync runs on most Unix‑like systems (Linux, Solaris, BSD) and also has Windows ports such as cwRsync and Sync2NAS.

2. How Rsync Works

Originally a replacement for rcp, rsync is maintained at rsync.samba.org and its configuration file rsync.conf follows the Samba format.

It can be invoked via rsh or ssh, or run as a daemon. In daemon mode the server listens on port 873.

During the first connection the whole file is transferred; subsequent runs transfer only incremental changes.

3. Key Features

Can mirror entire directory trees and file systems.

Preserves permissions, timestamps, and symbolic/hard links.

Installs without special privileges.

Optimized transfer pipeline for high efficiency.

Supports rsh, ssh, or direct socket connections.

Allows anonymous transfers.

4. SSH Mode Examples

4.1 Local Synchronization

<code># mkdir src
# touch src/{1,2,3,4}
# mkdir dest
# rsync -av src/ dest/   # sync contents of src to dest (excluding src itself)
# rsync -av src dest/    # sync src directory itself to dest
# rsync -avR src/ dest/  # same effect when src ends with /</code>

4.2 LAN Synchronization

<code># mkdir src
# touch src/{a,b,c,d}
# mkdir dest
# rsync -av 172.16.22.12:/data/test/src/ dest/   # pull from remote to local (requires root password)
# rsync -av src/ 172.16.22.12:/data/test/dest/   # push local files to remote
# rsync -av src 172.16.22.12:/data/test/dest/   # sync whole directory
# rm -rf src/d
# rsync -av --delete src/ 172.16.22.12:/data/test/dest/   # delete files on remote that no longer exist locally</code>

4.3 Specifying Remote User

<code># useradd george
# passwd george
# mkdir /home/george/test
# touch /home/george/test/g{1,2,3,4}
# rsync -av src -e 'ssh -l george' 172.16.22.12:/home/george
# rsync -av 172.16.22.12:/home/george/test/g* -e 'ssh -l george -p 22' dest/</code>

5. Daemon Mode

5.1 Starting the Service

For heavy‑load servers run rsync as a standalone daemon:

<code># yum install rsync xinetd
# /usr/bin/rsync --daemon</code>

For light‑load servers use xinetd:

<code># yum install rsync xinetd
# edit /etc/xinetd.d/rsync, set "disable = no"
# service xinetd start
# chkconfig rsync on
# netstat -ntpl | grep 873   # verify the daemon is listening</code>

5.2 Configuration File (rsyncd.conf)

The file follows the same syntax as Samba’s main config. Global parameters include address, port (default 873), motd file, pid file, log file, syslog facility, socket options, lockfile, and timeout (default 600 s).

Module parameters define which directories are shared. A module is declared as

[module]

and must specify at least a

path

. Optional parameters include

comment

,

use chroot

,

uid

,

gid

,

max connections

,

read only

,

write only

,

ignore errors

,

exclude

/

include

rules, authentication users, secrets file, host allow/deny lists, and logging options.

5.3 Example Module Configuration

<code># vim /etc/rsyncd.conf
uid = root
gid = root
use chroot = no
max connections = 5
timeout = 600
pid file = /var/run/rsyncd.pid
lockfile = /var/run/rsyncd.lock
log file = /var/log/rsyncd.log

[web1]
  path = /data/test/src
  ignore errors = yes
  read only = no
  write only = no
  hosts allow = 192.168.22.12
  hosts deny = *
  list = yes
  auth users = web
  secrets file = /etc/web.passwd</code>

6. Testing the Setup

6.1 Client Preparation

<code># yum -y install rsync
# mkdir /data/test</code>

6.2 Common Rsync Options

-a (archive) = -rlptgoD, -r recursive, -l copy symlinks, -i list files, -p preserve permissions, -t preserve times, -g preserve group, -o preserve owner, -D preserve devices, -z compress, -P show progress, -v verbose.

6.3 Sample Commands

<code># rsync -avzP [email protected]::web1 /data/test/
# rsync -avzP --delete [email protected]::web1 /data/test/
# rsync -avzP --delete /data/test/ [email protected]::web1
# rsync -avzP --delete /data/test/ [email protected]::web1/george
# rsync -ir --password-file=/tmp/rsync.password [email protected]::web1
# rsync -avzP --exclude="*3*" --password-file=/tmp/rsync.password [email protected]::web1 /data/test/</code>

6.4 Using a Password File

<code># echo "123" > /tmp/rsync.password
# chmod 600 /tmp/rsync.password
# rsync -avzP --delete --password-file=/tmp/rsync.password [email protected]::web1 /data/test/</code>

6.5 Automating with Cron

<code># crontab -e
10 0 * * * rsync -avzP --delete --password-file=/tmp/rsync.password [email protected]::web1 /data/test/</code>

7. Real‑Time Sync with Inotify‑tools

Inotify‑tools provide C libraries and command‑line utilities (inotifywait, inotifywatch) to monitor filesystem events on Linux.

7.1 Installation

<code># yum install -y gcc
# mkdir /usr/local/inotify
# tar -xf inotify-tools-3.14.tar.gz
# cd inotify-tools-3.14
# ./configure --prefix=/usr/local/inotify/
# make && make install</code>

7.2 Environment Setup

<code># vim /root/.bash_profile
export PATH=/usr/local/inotify/bin/:$PATH</code>

<code># source /root/.bash_profile
# echo '/usr/local/inotify/lib' >> /etc/ld.so.conf
# ldconfig
# ln -s /usr/local/inotify/include /usr/include/inotify</code>

7.3 Common Options

-m keep listening

-r recursive

-q quiet output

-e define events (access, modify, attrib, open, delete, create, move)

--fromfile read watch list from file

--timefmt specify time format

--format custom output

--exclude / --excludei for pattern filtering

7.4 Example Real‑Time Sync Script

<code># mkdir /data/test/src /data/test/dest
# rsync -av --delete /data/test/src/ 192.168.22.12:/data/test/dest   # initial test
# vim /data/test/test.sh
#!/bin/bash
/usr/local/inotify/bin/inotifywait -mrq -e modify,create,move,delete,attrib /data/test/src |
while read events; do
  rsync -a --delete /data/test/src/ 192.168.22.12:/data/test/dest
  echo "`date +'%F %T'` Event: $events" >> /tmp/rsync.log 2>&1
done
# chmod 755 /data/test/test.sh
# /data/test/test.sh &   # run in background
# echo '/data/test/test.sh &' >> /etc/rc.local   # start on boot</code>

A symmetric script on the destination can push changes back, achieving bidirectional synchronization.