efficiency issue with rsync.....

[Rogier Wolff]

Hi rsync team, 

I thought that rsync would try to overlap computing and IO on both
machines. 

I'm rsyncing a large tree (18G) and am keeping an eye on that. 
Suddenly my side completely stopped. No IO visible, no CPU
time spent. The otherside was doing 100% CPU. Then the other
side started to do disk IO. Then suddenly the activities moved
over to my side, and I saw things moving again in the "-v --progress"
output. 

The transfer happened to hit my "spam" mailbox, 400Mb of mostly-stable
data. I probably rewrote the mailbox yesterday, having changed some
flags on mails somewhere in the middle. 

Note: This is NOT a request: "Please go and fix". Just an "I noticed
that it might be possible to make it more efficient". Feel free to 
keep this in mind when performing further development. 

(Still rsync is a very good tool, transferring the 17G of data in 
a couple of hours....)

Oh, another thing: When two files are almost the same, (e.g. I just
added something to the end of a mailbox) the bandwidth of the link
is not fully used while the counters are running quickly. Is this
unavoidable "the machine simply won't generate enough work to
keep the link busy" or is there a bug in the "limit bandwidht to XX"
code? (I limit to 30k per second, and I see rsync doing lots of small 
sleeps. If you try to sleep for 100 usec, you'll actually be woken up 
by the kernel after a whopping 20 msec.)

			Roger. 

-- 
** R.E.Wolff at BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
* The Worlds Ecosystem is a stable system. Stable systems may experience *
* excursions from the stable situation. We are currently in such an      * 
* excursion: The stable situation does not include humans. ***************