rsync is flaky going to Penang

[Tony at ServaCorp.com]

The internet connection is probably flaky and erratic.
What we do between interior of China and US
In China is a box whose only purpose is an rsync host.
>From China to that box is at LAN speeds and is dependable.
To or from that box to the US is at bad internet speeds and is not
dependable.
Within the US transfers are controlled from whereever I can log in and get a
reasonable conection.
(I've got more or less up-to-date info at 2 or 3 different locations
stateside)

You may be able get something from the following.
The important thing is to not expose your primary sources and destinations
to the vagaries of the internet. Disk is cheap.


[edited and sanitized] (and no apologies for bad code herein)
[root at sys001 root]# time ./rsync-XXX-dwg ; date ; date -u
real 0m13.706s
Thu Mar 31 12:10:57 CST 2005 C as in China, not Central
Thu Mar 31 04:10:57 UTC 2005
[root at sys001 root]# cat rsync-XXX-dwg
#!/bin/bash
# rsync-XXX-dwg XXX_Drawings/ title/
mkdir -p /tmp/rsync ; echo `hostname` > /tmp/rsync/OPENED
rsync -a --password-file=/etc/rsync.secrets/XXX-dwg --timeout=750 \
/tmp/rsync/OPENED rsync-XXX-dwg at 10.0.0.xxx::rsync-XXX-dwg/
for name in title XXX_Drawings ; do
rsync -a --password-file=/etc/rsync.secrets/XXX-dwg --timeout=750 \
/home/dwg/$name rsync-XXX-dwg at 10.0.0.xxx::rsync-XXX-dwg/
done
mkdir -p /tmp/rsync ; echo `hostname` > /tmp/rsync/CLOSED
rsync -a --password-file=/etc/rsync.secrets/XXX-dwg --timeout=750 \
/tmp/rsync/CLOSED rsync-XXX-dwg at 10.0.0.xxx::rsync-XXX-dwg/
[root at sys001 root]# du -s /home/dwg/XXX_Drawings/
12618920 /home/dwg/XXX_Drawings

[root at xx1 /root]# time ./rsync-XXX-dwg ; date ; date -u
real 1m12.449s <-- IDE to same IDE (lots of seeks?)
receiving file list ...
50246 files to consider
CLOSED
7 100% 6.84kB/s 0:00:00 (1, 0.0% of 50246)
OPENED
7 100% 0.49kB/s 0:00:00 (2, 0.0% of 50246)
[snip]
sent 420 bytes received 2101460 bytes 2355.05 bytes/sec
total size is 12642359798 speedup is 6014.79
real 14m52.839s <-- This is over a BAD internet connection*
Wed Mar 30 22:32:16 CST 2005 C as in Central, not China
Thu Mar 31 04:32:16 UTC 2005
*BAD Internet. 2 tries to SSH directly, both timed out trying to connect.
(SSH'd to a box at big boss's home (DSL) and then from there. Different
routes)
This is acceptable transfer over a sporadically available connection!


-----Original Message-----
From: rsync-bounces+tony=servacorp.com at lists.samba.org
[mailto:rsync-bounces+tony=servacorp.com at lists.samba.org]On Behalf Of
jackie_wright at agilent.com
Sent: Monday, April 04, 2005 5:41 PM
To: agilent_ux at it.lists.it.agilent.com; rsync at samba.org
Subject: rsync is flaky going to Penang


Hello,

We are experiencing flaky behavior from rsync when attempting to rsync
directories/files to a server in Penang.   Several times the job seems to
‘hang’ and never completes.   Penang then is therefore missing a lot of
required cad files.    Have any of you experienced the same thing and what
did you do to fix the problem?   Does anyone know of a better tool to use?
We chose rsync after rdist gave us problems.

Perhaps we’re missing an important flag in our rsync commands but I think it
’s set up correctly.
Thanks and Best Regards,
Jackie Wright
Agilent Technologies - WSD R&D IT Engineer
Telnet:  435-6653 or (408) 435-6653
Cell:  (510) 825-7638
Fax: (408) 435-4803