Windows 2003, Cygwin, and rsync

[John Oliver]

On Wed, Jun 21, 2006 at 05:06:22PM +1000, Christian Hack wrote:
> > 
> > I just installed Cygwin / OpenSSH / rsync on two Dell PowerVault 745N
> > NASes running Windows 2003 Appliance Edition.  My rsync daemons are
> > running, ssh works, and in theory all is well.  But... I'm getting an
> > average of maybe 15 Mb/s rsyncing between them.
> > 
> > Now, I know I have an issue in the way they're connected... one is
> > attached to a Cisco 2970 (1 Gb/s), which is attached to a NetGear
> > 100Mb/s switch, which has the other NAS attached to it.  Yes, the
> > NetGear is going to go away :-)  And the NASes are going to 
> > wind up with
> > a crossover cable between them for a straight, dedicated 1Gb/s  But it
> > certainly looks like rsync is not using more than a fraction of the
> > (theoretically) available 100Mb/s  There are several other hosts
> > attached to the NetGear switch, and there is no "slowness" about them
> > (which disproved my initial hypothesis of a saturated backplane)
> > 
> > Is this a known rsync and/or Cygwin issue?
> 
> As for you speed issues. From a Dual Xeon 3.2GHz to a dual P3 1GHz (both
> Win2k3 server) using a single 500MB file for transfer I was able to get
> 5.4MB/sec over a 100Mb LAN through at least 1 switch. Which is not too far
> off the theoretical limit. That was from an active file server during the
> middle of the day too.
> 
> Using the -W whole file option didn't really help although it might with
> lots smaller files. Turning on compression (-z) didn't help either and
> should probably be turned off over a LAN anyway.
> 
> Maybe try doing a simple file copy to see what throughput you can get

I creted a ~250MB test file and was able to scp it between these NASes
at ~40mb/s

rsync still was transferring at about 12 mb/s over a direct 1Gb/s link.

sent 665653991 bytes  received 78242 bytes  395210.59 bytes/sec total
size is 62501572674  speedup is 93.88 rsync error: some files could not
be transferred (code 23) at
/home/lapo/packaging/tmp/rsync-2.6.6/main.c(791)

real    31m47.484s
user    1m49.609s
sys     2m15.998s

I am dealing with a lot of small files... about 50 GB of ~180,000
someodd files.

I'll check out that -W, but is there anything else that might improve
this seemingly-pathetic performance?

-- 
***********************************************************************
* John Oliver                             http://www.john-oliver.net/ *
*                                                                     *
***********************************************************************