performance with >50GB files

René Rebe rene at
Tue Jan 10 22:31:08 GMT 2006


On Tuesday 10 January 2006 21:47, Wayne Davison wrote:
> On Tue, Jan 10, 2006 at 09:02:14PM +0100, Ren? Rebe wrote:
> > So far just increasing the block-size significantly (10-20MB) bumps
> > the speed by magnitudes into useful regions.
> That's good.  For some reason I was thinking that the block size was
> nearly maxxed out for a 50GB file, but I can see that the current
> algorithm for selecting the block size ends up nowhere near the
> maximum block-size limit.

Yes, I think this could be fixed up quite easily. Also I found the current
code does decide from the receiving-side file what blocksize to use. Now
with the case where a receiver file is way smaller than the reference
one, it would pick a rather tiny blocksize due to the available tiny file
and then take nearly a day due to the tiny blocksize. The same applies
when the sync was aborted and restarted.

But getting the sent size to the code selecting the blocksize is not a too tiny


René Rebe - Rubensstr. 64 - 12157 Berlin (Europe / Germany)
            +49 (0)30  255 897 45

More information about the rsync mailing list