Mastering Rsync: Fast File Sync, Daemon Setup, and Real‑Time Inotify Integration

1. Introduction

1.1 Overview

Rsync (remote synchronize) is a remote data synchronization tool that can quickly sync files between multiple hosts over LAN/WAN. It uses the “Rsync algorithm” which transfers only the differences between files, making it fast.

Rsync supports most Unix‑like systems, including Linux, Solaris, BSD, and also has Windows versions such as cwRsync and Sync2NAS.

2. Principle

Rsync was originally created to replace rcp and is now maintained at rsync.samba.org; its configuration file rsync.conf follows the Samba format.

Rsync can be used via rsh or ssh, or run as a daemon. In daemon mode the rsync server listens on port 873.

During the first connection the full file set is transferred; subsequent runs transfer only incremental changes.

3. Features

Can mirror entire directory trees and file systems.

Preserves permissions, timestamps, and symbolic/hard links.

Installs without special privileges.

Optimized workflow for high transfer efficiency.

Supports rsh, ssh, or direct socket connections.

Allows anonymous transfers.

2. SSH Mode

1. Local-to-Local Sync

<code># mkdir src
# touch src/{1,2,3,4}
# mkdir dest
# rsync -av src/ dest/   # sync contents of src to dest (excluding src itself)
# rsync -av src dest/    # sync src directory itself to dest
# rsync -avR src/ dest/  # same effect when src ends with /</code>

2. LAN Sync

<code># mkdir src
# touch src/{a,b,c,d}
# mkdir dest
# rsync -av 172.16.22.12:/data/test/src/ dest/   # pull from remote to local (requires root password)
# rsync -av src/ 172.16.22.12:/data/test/dest/   # push local files to remote
# rsync -av src 172.16.22.12:/data/test/dest/   # sync whole directory
# rm -rf src/d                                 # delete a file
# rsync -av --delete src/ 172.16.22.12:/data/test/dest/   # delete extraneous files on remote</code>

3. LAN Sync with Specified User

On host 172.16.22.12:

<code># useradd george
# passwd george
# mkdir /home/george/test
# touch /home/george/test/g{1,2,3,4}</code>

On host 172.16.22.11:

<code># rsync -av src '-e ssh -l george' 172.16.22.12:/home/george   # local to remote as user george
# rsync -av 172.16.22.12:/home/george/test/g* '-e ssh -l george -p 22' dest/</code>

3. Daemon Mode

1. Service Startup

1.1 Stand‑alone daemon for heavy load

<code># yum install rsync xinetd
# /usr/bin/rsync --daemon</code>

1.2 xinetd managed daemon for light load

<code># yum install rsync xinetd
# vim /etc/xinetd.d/rsync   # set "disable = no"
# /etc/init.d/xinetd start
# chkconfig rsync on
# netstat -ntpl | grep 873   # verify service is listening</code>

2. Configuration Details

Both modes use

rsyncd.conf

, which has a format similar to Samba’s main config file.

Global parameters (apply to all modules unless overridden): address, port, motd file, pid file, log file, syslog facility, socket options, lockfile, timeout, etc.

Module parameters define which directory is shared. Example:

<code>[web1]
path = /data/test/src
ignore errors = yes
read only = no
write only = no
hosts allow = 192.168.22.12
hosts deny =
list = yes
auth users = web
secrets file = /etc/web.passwd</code>

Key module options include

path

,

comment

,

use chroot

,

uid

,

gid

,

max connections

,

lock file

,

list

,

read only

,

write only

,

ignore errors

,

dont compress

,

exclude

/

include

, authentication (

auth users

,

secrets file

,

strict modes

), host access control (

hosts allow

,

hosts deny

), and logging (

transfer logging

,

log format

).

3. Server‑side Configuration

<code># vim /etc/rsyncd.conf   # create configuration file
uid = root
gid = root
use chroot = no
max connections = 5
timeout = 600
pid file = /var/run/rsyncd.pid
lockfile = /var/run/rsyncd.lock
log file = /var/log/rsyncd.log

[web1]
path = /data/test/src
ignore errors = yes
read only = no
write only = no
hosts allow = 192.168.22.12
auth users = web
secrets file = /etc/web.passwd</code>

<code># mkdir /data/test/src
# mkdir /data/test/src/george
# touch /data/test/src/{1,2,3}
# echo "web:123" > /etc/web.passwd
# chmod 600 /etc/web.passwd
# service xinetd restart</code>

4. Client Tests

1. Install client

<code># yum -y install rsync
# mkdir /data/test</code>

2. Basic command examples

<code># rsync -avzP [email protected]::web1 /data/test/   # pull module web1
# rsync -avzP --delete [email protected]::web1 /data/test/   # sync and delete extraneous files
# rsync -avzP --delete /data/test/ [email protected]::web1   # push local to server
# rsync -avzP --delete /data/test/ [email protected]::web1/george   # push to specific subdirectory
# rsync -ir --password-file=/tmp/rsync.password [email protected]::web1   # list files recursively
# rsync -avzP --exclude="*3*" --password-file=/tmp/rsync.password [email protected]::web1 /data/test/   # exclude pattern</code>

3. Sync using password file

<code># echo "123" > /tmp/rsync.password
# chmod 600 /tmp/rsync.password
# rsync -avzP --delete --password-file=/tmp/rsync.password [email protected]::web1 /data/test/</code>

4. Automated sync with cron

<code># crontab -e
10 0 * * * rsync -avzP --delete --password-file=/tmp/rsync.password [email protected]::web1 /data/test/
# crontab -l</code>

5. Real‑time Sync with Inotify‑tools

1. About inotify‑tools

inotify‑tools provides C libraries and command‑line utilities (inotifywait, inotifywatch) for monitoring filesystem events on Linux.

2. Installation

<code># yum install -y gcc
# mkdir /usr/local/inotify
# tar -xf inotify-tools-3.14.tar.gz
# cd inotify-tools-3.14
# ./configure --prefix=/usr/local/inotify/
# make && make install</code>

3. Environment setup

<code># vim /root/.bash_profile   # add PATH
export PATH=/usr/local/inotify/bin/:$PATH
# source /root/.bash_profile
# echo '/usr/local/inotify/lib' >> /etc/ld.so.conf
# ldconfig
# ln -s /usr/local/inotify/include /usr/include/inotify</code>

4. Common inotifywait options

-m : keep listening

-r : recursive

-q : quiet output

-e : specify events (access, modify, attrib, open, delete, create, move, …)

--exclude, --excludei : pattern‑based exclusion

--timefmt, --format : customize output

5. Test script for bidirectional sync

<code># mkdir /data/test/dest/   # on destination host
# mkdir /data/test/src/    # on source host
# rsync -av --delete /data/test/src/ 192.168.22.12:/data/test/dest   # initial sync
# cat > /data/test/test.sh <<'EOF'
#!/bin/bash
/usr/local/inotify/bin/inotifywait -mrq -e modify,create,move,delete,attrib /data/test/src |
while read events; do
    rsync -a --delete /data/test/src/ 192.168.22.12:/data/test/dest
    echo "`date +'%F %T'` Event: $events" >> /tmp/rsync.log 2>&1
done
EOF
# chmod 755 /data/test/test.sh
# /data/test/test.sh &
# echo '/data/test/test.sh &' >> /etc/rc.local   # run at boot</code>

A similar script can be placed on the remote host to achieve two‑way synchronization.