Building a High‑Availability SFTP Service with Keepalived, Rsync, and Inotify

SFTP (SSH File Transfer Protocol) is a secure file‑transfer protocol that runs as a subsystem of SSH, using encrypted authentication and data transfer. Unlike FTP, it provides encrypted communication but may be slower due to encryption overhead.

1. Deploying SFTP with Keepalived High Availability

Create an sftp group and a dedicated user (e.g., mysftp) with a locked shell.

Set the user’s home directory to /data/sftp/mysftp and adjust ownership and permissions:

# mkdir -p /data/sftp/mysftp
# usermod -d /data/sftp/mysftp mysftp
# chown root:sftp /data/sftp/mysftp
# chmod 755 /data/sftp/mysftp

Configure /etc/ssh/sshd_config to use the internal SFTP server and chroot the sftp group:

#Subsystem sftp internal-sftp
Match Group sftp
  ChrootDirectory /data/sftp/%u
  ForceCommand internal-sftp
  AllowTcpForwarding no
  X11Forwarding no

Create a writable upload directory for the SFTP user:

# mkdir /data/sftp/mysftp/upload
# chown mysftp:sftp /data/sftp/mysftp/upload
# chmod 755 /data/sftp/mysftp/upload

Disable the firewall and SELinux temporarily, then restart the SSH daemon.

# /etc/init.d/iptables stop
# setenforce 0
# service sshd restart

Verify the SFTP connection (e.g., sftp [email protected]) and ensure the upload directory is visible.

2. One‑Way Real‑Time Sync Between Two Nodes (rsync + inotify)

Install rsync and xinetd on both servers, configure /etc/rsyncd.conf, and create a password file with mode 600.

log file = /var/log/rsyncd.log
pidfile = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
secrets file = /etc/rsync.pass
motd file = /etc/rsyncd.Motd

[sftp_upload]
  path = /data/sftp
  comment = sftp_upload
  uid = root
  gid = sftp
  port = 873
  use chroot = no
  read only = no
  list = no
  max connections = 200
  timeout = 600
  auth users = RSYNC_USER
  hosts allow = 172.16.51.191

Start the xinetd service and verify the rsync daemon is listening on port 873.

Inotify Installation

# yum install make gcc gcc-c++
# cd /usr/local/src/
# wget http://github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz
# tar zxvf inotify-tools-3.14.tar.gz
# cd inotify-tools-3.14
# ./configure --prefix=/usr/local/inotify
# make && make install
# echo 'export PATH=$PATH:/usr/local/inotify/bin' >> /etc/profile
# source /etc/profile
# echo '/usr/local/inotify/lib' >> /etc/ld.so.conf
# ldconfig

Increase kernel inotify limits:

# sysctl -w fs.inotify.max_queued_events=99999999
# sysctl -w fs.inotify.max_user_watches=99999999
# sysctl -w fs.inotify.max_user_instances=65535

Initial Full Sync

# rsync -avH --port=873 --progress --delete /data/sftp/ [email protected]::sftp_upload --password-file=/etc/rsync.pass

Real‑Time Sync Script (source node)

#!/bin/bash
SRCDIR=/data/sftp/
USER=RSYNC_USER
IP=172.16.51.192
DESTDIR=sftp_upload
/usr/local/inotify/bin/inotifywait -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f%e' -e close_write,modify,delete,create,attrib,move $SRCDIR |
while read file; do
  /usr/bin/rsync -avH --port=873 --progress --delete-before $SRCDIR $USER@$IP::$DESTDIR --password-file=/etc/rsync.pass
  echo "${file} was rsynced" >> /tmp/rsync.log 2>&1
done

Make the script executable, run it in the background, and verify the inotifywait process is active.

3. Adding Keepalived for Active‑Passive Failover

# cd /usr/local/src/
# wget http://www.keepalived.org/software/keepalived-1.3.2.tar.gz
# tar -zvxf keepalived-1.3.2.tar.gz
# cd keepalived-1.3.2
# ./configure && make && make install
# cp keepalived/etc/init.d/keepalived /etc/rc.d/init.d/
# cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
# mkdir /etc/keepalived
# cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/

Edit /etc/keepalived/keepalived.conf on the master node:

global_defs {
  notification_email { [email protected] }
  notification_email_from [email protected]
  smtp_server smtp.wangshibo.com
  smtp_connect_timeout 30
  router_id master-node
}

vrrp_script chk_sftp_port {
  script "/data/chk_sftp.sh"
  interval 2
  weight -5
  fall 2
  rise 1
}

vrrp_instance VI_1 {
  state MASTER
  interface eth0
  mcast_src_ip 172.16.51.191
  virtual_router_id 51
  priority 101
  advert_int 1
  authentication {
    auth_type PASS
    auth_pass 1111
  }
  virtual_ipaddress { 172.16.51.193 }
}

track_script { chk_sftp_port }

On the backup node, set state BACKUP and adjust priority to a lower value (e.g., 99). Add the same vrrp_script and track_script sections.

Create /data/chk_sftp.sh to monitor the SSH daemon and stop Keepalived if SSH cannot be started:

#!/bin/bash
counter=$(/etc/init.d/sshd status|grep running|wc -l)
if [ "$counter" = "0" ]; then
  /etc/init.d/sshd start
  sleep 2
  counter=$(/etc/init.d/sshd status|grep running|wc -l)
  if [ "$counter" = "0" ]; then
    /etc/init.d/keepalived stop
  fi
fi

Start Keepalived on both nodes; the virtual IP (VIP) will move automatically when the master fails.

4. Revised Synchronization Strategy (VIP‑Based One‑Way Sync)

Replace the bidirectional inotify script with a VIP‑aware script that runs rsync only on the node currently holding the VIP.

#!/bin/bash
while true; do
  NUM=$(ip addr|grep 172.16.51.193|wc -l)
  if [ $NUM -eq 1 ]; then
    /usr/bin/rsync -e "ssh -p22" -avpgolr --progress --delete-before /data/sftp/mysftp/ [email protected]:/data/sftp/mysftp/
  fi
done

Deploy the script on both servers, make it executable, and run it in the background.

5. Creating a Read‑Only SFTP Account

# useradd -g sftp -s /bin/false readftp
# passwd readftp
# mkdir /data/sftp/readftp
# usermod -d /data/sftp/readftp readftp
# chown root:sftp /data/sftp/readftp
# chmod 755 /data/sftp/readftp
# mkdir /data/sftp/readftp/upload
# chown mysftp:sftp /data/sftp/readftp/upload

Synchronize the writable mysftp upload directory to the read‑only directory using a simple rsync loop:

#!/bin/bash
while true; do
  /usr/bin/rsync -avpgolr --delete-before /data/sftp/mysftp/upload/ /data/sftp/readftp/upload/
done

Ensure the source upload directory always has 755 permissions so the read‑only user can see the files.

6. Common SFTP Maintenance Commands

cd <em>remote_path</em>          # change remote directory
lcd <em>local_path</em>           # change local directory
chgrp <em>group</em> <em>path</em>      # change group ownership
chmod <em>mode</em> <em>path</em>       # change permissions
chown <em>owner</em> <em>path</em>       # change owner
exit                         # leave sftp session
help                         # show help
get <em>remote_file</em>          # download file
ln <em>existing</em> <em>link</em>    # create symlink
ls [options] [path]          # list remote files
lls [options] [path]        # list local files
mkdir <em>path</em>               # create remote directory
lmkdir <em>path</em>              # create local directory
mv <em>old</em> <em>new</em>          # rename/move remote file
open <em>user@host[:port]</em>   # connect to host
put <em>local_file</em>           # upload file
pwd                          # print remote working directory
lpwd                         # print local working directory
quit                         # exit sftp
rmdir <em>path</em>              # remove remote directory
lrmdir <em>path</em>             # remove local directory
rm <em>path</em>                 # delete remote file
lrm <em>path</em>                # delete local file
symlink <em>target</em> <em>link</em># create remote symlink
version                      # show protocol version

Typical connection commands:

sftp [email protected]          # default SSH port 22
sftp -o port=6666 [email protected]  # non‑standard port

By following these steps, you can deploy a secure, chrooted SFTP service with high availability, real‑time one‑way synchronization, and separate read‑only access for auditors or downstream systems.