stalling during delta processing

Wallace Matthews wmatthews at sepaton.com
Mon Jun 14 19:49:33 GMT 2004


I have a 29 Gig full backup on a remote server (lets call if fedor) that is called Kbup_1.
I have a 1.3 Gig incremental backup on my local server.

I have rsync 2.6.2 on both servers. Both are RedHat Linux 9.1 on i-686 hardware platforms.

I issue the command "time rsync -avv --rsh=rsh --stats /test/Kibbutz/Kbup_1 fedor://test/Kibbutz". The synch takes ~5 minutes of real time and about ~2 minutes of user time and it finds 1087632 bytes of match data.

I copy the 29 Gig full backup back into fedor//test/Kibbutz and issue the command "time rsync -avv --rsh=rsh --stats --block-size=90636 /test/Kibbutz/Kbup_1 fedor://test/Kibbutz" and it takes about ~5 minutes of real time and about ~2 minutes of user time and it finds 90636 of match data.

I copy the 29 Gig full backup back into fedor//test/Kibbutz and issue the command "time rsync -avv --rsh=rsh --stats --block-size=181272 /test/Kibbutz/Kbup_1 fedor://test/Kibbutz" and it CRAWLS during delta generation/transmittal at about 1 Megabyte per second.

I have repeated the experiment 3 times; same result each time. 

The only thing that is different is --block-size= option. First, time it isnt specified and I get a predictable answer. Second time, I give it a block
size that is about 1/2 of square root of (29 Gig) and that is ok. But, explicitly give it something that is approximately the square root of the 
29 Gig and it CRAWLS. 

When I cancel the command, the real time is 86 minutes and the user time is 84 minutes. This is similar to the issue I reported on Friday that Chris suggested I remove the --write-batch= option and that seemed to fix the CRAWL.

Now, it appears that the --block-size= option is the culprit.

Can someone who knows rsync internals explain what is happening??

wally


More information about the rsync mailing list