scp vs rsync: Detailed Usage, Parameters, and When to Choose Each for Server File Transfers

Background

System administrators often need to move files between a local machine and remote servers or between two remote servers. The two most common Linux tools are scp (secure copy) and rsync . scp is simple, works over SSH, and is suited for occasional small transfers. rsync provides incremental synchronization, compression, and resume support, making it ideal for large files, frequent backups, and periodic syncs.

scp Detailed Explanation

Principle

scp runs locally, opens an SSH connection to the remote host, copies the data over the encrypted channel, and then exits.

Common Options

-r

: copy directories recursively -p: preserve timestamps, access times and permissions -q: quiet mode (no progress output) -v: verbose/debug output -C: enable compression -P port: specify a non‑default SSH port (uppercase P) -i identity_file: specify a private key -l limit: limit bandwidth (Kbit/s) -o ssh_option: pass additional SSH options -F ssh_config: use a custom SSH config file

Practical Scenarios

Upload a single file

scp /path/to/local/file.txt [email protected]:/remote/path/

Upload with a custom SSH port

scp -P 2222 /path/to/local/file.txt [email protected]:/remote/path/

Upload using a private key

scp -i ~/.ssh/id_rsa /path/to/local/file.txt [email protected]:/remote/path/

Preserve file attributes

scp -p /path/to/local/file.txt [email protected]:/remote/path/

Download a single file

scp [email protected]:/remote/path/file.txt /local/path/

Download an entire directory

scp -r [email protected]:/remote/path/directory /local/path/

Transfer directly between two remote hosts

scp user1@host1:/path/to/file user2@host2:/path/to/destination/

Batch upload with wildcards

scp /local/path/*.txt [email protected]:/remote/path/

Limit bandwidth to 500 KB/s

scp -l 4000 /largefile.tar.gz [email protected]:/remote/path/

Note: -l uses Kbit/s, so 500 KB/s corresponds to 4000 Kbit/s.

Common Errors and Fixes

Connection refused

# Verify SSH port
ssh -p 2222 [email protected]
# Use the correct port with scp
scp -P 2222 file.txt [email protected]:/path/

Permission denied (publickey)

# Ensure private key permissions
chmod 600 ~/.ssh/id_rsa
# Specify the correct key
scp -i ~/.ssh/id_rsa file.txt [email protected]:/path/

Not a regular file

# Copy a directory
scp -r /local/directory [email protected]:/remote/path/
# Ensure target path ends with '/' for a directory
scp file.txt [email protected]:/remote/path/

No such file or directory

# Create the remote directory first
ssh [email protected] "mkdir -p /remote/path"
# Then copy
scp file.txt [email protected]:/remote/path/

Security Considerations

Private‑key files must have 600 permissions; otherwise SSH rejects them.

Never embed passwords in commands; use key‑based authentication.

Be careful with wildcards to avoid unintentionally copying sensitive files.

scp overwrites existing files without prompting; verify target paths.

For large files, prefer rsync because scp lacks resume support.

rsync Detailed Explanation

Principle

rsync transfers only the differences between source and destination (incremental sync) using a rolling checksum algorithm, dramatically reducing network traffic.

Basic Syntax

# Local sync
rsync [options] source_path target_path
# Remote sync via SSH
rsync [options] source_path user@host:target_path
# Remote sync via rsync daemon
rsync [options] source_path user@host::module/target_path

Key Option Groups

Synchronization control -r: recursive (does not preserve permissions) -a: archive mode (equivalent to -rlptgoD) -z: compress data during transfer -P: show progress and keep partially transferred files (same as --partial --progress) --delete: delete files in the destination that are absent from the source (mirror sync) --exclude=PATTERN: exclude matching files --include=PATTERN: include matching files (must appear before --exclude) --filter=RULE: custom filter rules

Behavior -n or --dry-run: simulate the operation without making changes -v: verbose output -q: quiet mode (errors only) --stats: display transfer statistics --itemize-changes: detailed per‑file change list

File attributes -t / --times: preserve timestamps -p / --perms: preserve permissions -o / --owner: preserve owner (requires root) -g / --group: preserve group --links: preserve symbolic links --devices: preserve device files (requires root) --specials: preserve special files

Transfer parameters -e: specify remote shell (e.g., ssh) -l: copy symlinks as symlinks -L: copy the referent of symlinks -W: disable incremental checks (full file transfer, useful on LAN) --bwlimit=RATE: limit bandwidth (KB/s) --partial: keep partially transferred files for resume --progress: show progress bar

Practical Scenarios

Local directory sync

rsync -av /source/directory/ /target/directory/

Trailing slash on the source copies only its contents; omitting the slash copies the directory itself.

Remote sync via SSH (upload)

rsync -avz -e ssh /local/directory/ [email protected]:/remote/directory/

Remote sync via SSH (download)

rsync -avz -e ssh [email protected]:/remote/directory/ /local/directory/

Incremental sync (core advantage)

# First run copies everything; subsequent runs transfer only changed parts
rsync -avz /source/directory/ [email protected]:/remote/directory/

Mirror sync (dangerous)

rsync -avz --delete /source/directory/ [email protected]:/remote/directory/

This makes the destination an exact copy; extra files are deleted.

Exclude specific files

# Exclude log files
rsync -avz --exclude='*.log' /source/directory/ [email protected]:/remote/directory/
# Exclude multiple patterns
rsync -avz \
    --exclude='*.log' \
    --exclude='*.tmp' \
    --exclude='node_modules' \
    /source/directory/ [email protected]:/remote/directory/

Include then exclude (order matters)

rsync -avz --include='*.php' --include='*/' --exclude='*' /source/ remote:/target/

Only .php files and all directories are transferred.

Dry‑run preview

rsync -avzn --delete /source/directory/ [email protected]:/remote/directory/

Bandwidth limit (1 MB/s)

rsync -avz --bwlimit=1024 /large/directory/ [email protected]:/remote/directory/

Resume interrupted transfer

rsync -avzP --partial /large/directory/ [email protected]:/remote/directory/

Common Errors and Fixes

Skipping directory (missing trailing slash)

# Correct usage with trailing slash
rsync -av /source/directory/ /target/directory/

Permission denied (publickey)

# Test SSH authentication
ssh -v [email protected]
# Debug rsync over SSH
rsync -avz -e "ssh -v" /source [email protected]:/target

Connection unexpectedly closed (daemon not running)

# Check daemon status
systemctl status rsync
# Increase SSH timeout
rsync -avz --timeout=600 /source remote:/target

rsync: safe_read failed (unstable network)

# Increase timeout and use resume
rsync -avz --timeout=600 /source remote:/target
rsync -avzP /source remote:/target

Security Considerations

Prefer SSH mode for production; daemon mode is for trusted internal networks.

Use SSH keys; avoid passwords in scripts.

Restrict daemon access with hosts allow directives.

Exclude sensitive files such as .env, *.key, *.pem.

Always preview --delete actions with -n first.

scp vs rsync Comparison

Transfer mode : scp copies the full file set; rsync transfers only differences (incremental).

Resume support : scp does not support resume; rsync supports it via --partial.

Compression : scp uses -C; rsync uses -z.

Exclude files : not supported by scp; supported by rsync with --exclude.

Dry‑run : unavailable in scp; available in rsync with -n.

Speed (full transfer) : both are fast.

Speed (incremental) : scp is slow because it copies everything; rsync is fast because it copies only changed parts.

Syntax complexity : scp is simple; rsync has more options.

Typical use case : scp for small, one‑off transfers; rsync for large files, periodic syncs, and backups.

Selection Guidance

Use scp when you need to copy a few small files quickly, the network is stable, and incremental sync or advanced filtering is unnecessary.

Use rsync for large or many files, when you need incremental updates, exclusion patterns, resume capability, or a preview before applying changes.

Performance Illustration

Assume a source directory contains 1 000 files, of which only 10 have changed.

scp transfers all 1 000 files.

rsync transfers only the 10 changed files, saving bandwidth and time.

For a single 10 GB file where only a few hundred megabytes differ, scp would resend the entire 10 GB, whereas rsync would send only the delta.

Transfer Speed Optimizations

scp Optimizations

# Enable compression
scp -C file.tar.gz remote:/path/
# For very large files, compress locally then split (requires external tools)

rsync Optimizations

# Enable compression
rsync -avz source/ remote:/target/
# Use whole‑file mode for fast LAN transfers
rsync -avzW source/ remote:/target/
# Adjust block size for huge files
rsync -avz --block-size=8192 source/ remote:/target/

Production‑Level Real‑World Cases

Case 1 – Large Log File Migration

Scenario : Move 500 GB of logs from an old server to a new one over a 100 Mbps link without impacting production traffic.

Design :

Use rsync incremental sync to copy most data first.

Perform the final sync during a low‑traffic window.

Apply --bwlimit to cap bandwidth.

#!/bin/bash
SOURCE_USER=root
SOURCE_HOST=192.168.1.50
SOURCE_PATH=/var/log/myapp
TARGET_USER=root
TARGET_HOST=192.168.1.100
TARGET_PATH=/var/log/myapp
BW_LIMIT=10240   # 10 MB/s ≈ 80 Mbps
EXCLUDE_FILE=/tmp/rsync_exclude.txt
cat > "$EXCLUDE_FILE" <<'EOF'
*.current.log
*.tmp
EOF
# First (full) sync with bandwidth limit
rsync -avzP \
    --bwlimit=$BW_LIMIT \
    --exclude-from="$EXCLUDE_FILE" \
    -e "ssh -p 22" $SOURCE_USER@$SOURCE_HOST:"$SOURCE_PATH/" \
    $TARGET_USER@$TARGET_HOST:"$TARGET_PATH/"
# Second (incremental) sync
rsync -avzP \
    --bwlimit=$BW_LIMIT \
    --exclude-from="$EXCLUDE_FILE" \
    -e "ssh -p 22" $SOURCE_USER@$SOURCE_HOST:"$SOURCE_PATH/" \
    $TARGET_USER@$TARGET_HOST:"$TARGET_PATH/"

Verification steps include counting files ( find … | wc -l) and comparing sizes ( du -sh).

Case 2 – Zero‑Downtime Code Deployment

Scenario : Deploy new web‑app code without service interruption, using a test directory for validation before switching.

#!/bin/bash
set -e
APP_USER=deploy
APP_HOST=192.168.1.100
APP_PATH=/var/www/myapp
TEST_PATH=/var/www/myapp_test
SOURCE_PATH=/home/deploy/releases/$(date +%Y%m%d_%H%M%S)
# Prepare release directory
mkdir -p "$SOURCE_PATH"
cp -r /home/deploy/app/* "$SOURCE_PATH/"
# Sync to test environment
rsync -avz --exclude='.env' --exclude='.git' --exclude='node_modules' \
    -e "ssh -p 22" "$SOURCE_PATH/" $APP_USER@$APP_HOST:"$TEST_PATH/"
# Manual or automated verification
ssh -p 22 $APP_USER@$APP_HOST "ls -la $TEST_PATH | head -20"
read -p "Confirm switch to production? (yes/no): " CONFIRM
if [[ "$CONFIRM" != "yes" ]]; then echo "Deployment cancelled"; exit 0; fi
# Sync to production
rsync -avz --exclude='.env' --exclude='.git' --exclude='node_modules' \
    -e "ssh -p 22" "$SOURCE_PATH/" $APP_USER@$APP_HOST:"$APP_PATH/"
# Restart service
ssh -p 22 $APP_USER@$APP_HOST "systemctl restart myapp"
# Health check
sleep 3
ssh -p 22 $APP_USER@$APP_HOST "curl -s http://localhost/health"
echo "=== Deployment completed ==="

Case 3 – Database Backup Synchronization

Scenario : Daily MySQL dumps are compressed and synchronized to an off‑site backup server.

#!/bin/bash
set -e
DB_HOST=localhost
DB_USER=backup
DB_PASS='password123'
DB_NAME=myapp
BACKUP_DIR=/backup/db
REMOTE_USER=backup
REMOTE_HOST=192.168.1.200
REMOTE_PATH=/backup/myapp
RETENTION_DAYS=30
mkdir -p "$BACKUP_DIR"
BACKUP_FILE="$BACKUP_DIR/${DB_NAME}_$(date +%Y%m%d_%H%M%S).sql"
mysqldump -h "$DB_HOST" -u "$DB_USER" -p"$DB_PASS" \
    --single-transaction --routines --triggers --events "$DB_NAME" > "$BACKUP_FILE"
gzip "$BACKUP_FILE"
BACKUP_FILE="${BACKUP_FILE}.gz"
rsync -avzP --bwlimit=5120 -e "ssh -p 22" "$BACKUP_FILE" $REMOTE_USER@$REMOTE_HOST:"$REMOTE_PATH/"
# Cleanup old backups
find "$BACKUP_DIR" -name '*.sql.gz' -mtime +$RETENTION_DAYS -delete
# Show stats
echo "Local backups: $(find "$BACKUP_DIR" -name '*.sql.gz' | wc -l)"
echo "Total size: $(du -sh "$BACKUP_DIR" | cut -f1)"

Best Practices and Checklists

Pre‑Transfer Checklist

[ ] Verify source file paths
[ ] Verify target paths and write permissions
[ ] Test SSH connectivity (ssh remote_user@host)
[ ] Ensure sufficient disk space on destination (df -h)
[ ] Confirm bandwidth usage will not impact services
[ ] Ensure no sensitive data is unintentionally transferred

Post‑Transfer Checklist

[ ] Compare file counts (find | wc -l)
[ ] Compare total sizes (du -sh)
[ ] Sample MD5 checksums (md5sum)
[ ] Review transfer logs for errors
[ ] Verify applications can read the transferred files

Automated Inspection Script (cron‑daily)

# /etc/cron.daily/check_sync.sh
LOG_FILE="/var/log/sync_check.log"
REMOTE_USER=deploy
REMOTE_HOST=192.168.1.100
SOURCE_PATH=/data
REMOTE_PATH=/backup/data
# Log header
echo "=== Sync Inspection $(date) ===" >> "$LOG_FILE"
# Check remote directory existence
if ssh $REMOTE_USER@$REMOTE_HOST "test -d $REMOTE_PATH"; then
    echo "[OK] Remote directory exists" >> "$LOG_FILE"
else
    echo "[FAIL] Remote directory missing" >> "$LOG_FILE"
fi
# Source size
SOURCE_SIZE=$(du -sh "$SOURCE_PATH" 2>/dev/null | cut -f1)
echo "Source size: $SOURCE_SIZE" >> "$LOG_FILE"
# Remote size
REMOTE_SIZE=$(ssh $REMOTE_USER@$REMOTE_HOST "du -sh $REMOTE_PATH" 2>/dev/null | cut -f1)
echo "Remote size: $REMOTE_SIZE" >> "$LOG_FILE"
# Size comparison
if [ "$SOURCE_SIZE" = "$REMOTE_SIZE" ]; then
    echo "[OK] Sync complete" >> "$LOG_FILE"
else
    echo "[WARN] Size mismatch – may need resync" >> "$LOG_FILE"
fi

echo "" >> "$LOG_FILE"

Selection Summary

One‑off small file transfers : use scp.

Large files, frequent syncs, backups : use rsync.

Need preview before destructive actions : use rsync -n.

Unstable network : use rsync --partial.

Exclude specific files or directories : use rsync --exclude.

High‑throughput internal network : consider rsync daemon mode.

Key Takeaways

Prefer SSH key authentication for both tools.

Use --bwlimit to protect production bandwidth.

Exclude sensitive files such as .env, *.key, *.pem when syncing code or backups.

Always dry‑run --delete operations with -n first.

Log transfers for audit and troubleshooting; verify results with file counts, sizes, or checksums.