How to call rsync client so that is detects that server has gone away?

Bjorn Madsen bjorn.madsen at operationsresearchgroup.com
Wed Jul 13 14:17:11 MDT 2011


Hi Larry,
Your tech details are way beyond my knowledge, but maybe python can do
the job for you too? I use this python script (from the python
cookbook) and crontab to keep my sheep in the dry:

#!/usr/bin/env python
#wraps up rsync to synchronize two directories
# ! -- should contain check of disk space using rsync -n option for drytest !
from subprocess import call
import sys
import time

"""this motivated rsync tries to synchronise forever"""

source = "/media/rsync_gb01/movies/" #note the trailing slash!
target = "bjorn at srv01:/home/bjorn/movies"
rsync = "rsync"
arguments = "-avz --timeout=60 --delete"
cmd = "%s %s %s %s" % (rsync, arguments, source, target)

def sync():
	while True:
		ret = call(cmd, shell=True)
		if ret !=0:
			print "resubmitting rsync"
			time.sleep(30)
		else:
			print "rsync was succesful"
			#subprocess.call("MAIL -s 'jobs done'
bjorn.madsen at operationsresearchgroup.com", shell=True)
			sys.exit(0)
sync()

# Running in crontab since 29/6/2011 10:07
# Dry-run Testline: rsync -avzn --stats --timeout=60
/media/rsync_gb01/movies/ bjorn at srv01:/home/bjorn/movies



---------- Forwarded message ----------
From: Larry Irwin <mkitwrk at gmail.com>
To: rsync at lists.samba.org
Date: Tue, 12 Jul 2011 14:50:01 -0400
Subject: How to call rsync client so that is detects that server has gone away?
 Hi All,
I have an rsync client that is not re-starting and/or timing out when
the server processes die/go away...
The client is rsync  version 3.0.4  protocol version 30, natively
compiled on Debian 6, kernel 2.6.32.5-amd64.
The server is a QNAP NAS device running rsync version 3.0.6 protocol version 30.
(I can change the client version if I need to, but not the version on
the QNAP device)
I am trying to rsync ~700gb to the QNAP device, hopefully once per day.
There are ~40,000,000 files, many of them rsync-snapshot hard-links.
It takes a long time, so I'm using LVM2 snapshots to get static views
of the data partitions.
I need to preserve the links or the destination size will multiply by
a factor of 6 or so...
The command I am using on the client is:
nice -n 19 rsync -ave "ssh -l ${LOGIN} -p ${PORT}" --links
--hard-links --perms --times --owner --group --delete --stats
--exclude-from=${RSHOME}/cfg/rsyncbackup.exclude
--link-dest=${LINKDEST} / ${LOGIN}@${IPADDRESS}:..${DEST} 2>&1 | gzip
-9 > ${LOGNAME}
The client and the QNAP are connected on the LAN with 1Gb NIC's on Gb switches.
When/if the server's rsync processes die (for an unknown reason at
this point), I'd love for the client to recognize this fact and
jump-start the server processes to pick up where they left off.
If not that, at least to quit and exit in a reasonable fashion...
At this moment, it is 2:45pm: The rsync started at 9:00am - the server
processes went away at 12:33 PM - the client processes are still
running...
Any ideas would be greatly appreciated.
Thanks,
Larry Irwin


More information about the rsync mailing list