performance with >50GB files
rene at exactcode.de
Tue Jan 10 22:31:08 GMT 2006
On Tuesday 10 January 2006 21:47, Wayne Davison wrote:
> On Tue, Jan 10, 2006 at 09:02:14PM +0100, Ren? Rebe wrote:
> > So far just increasing the block-size significantly (10-20MB) bumps
> > the speed by magnitudes into useful regions.
> That's good. For some reason I was thinking that the block size was
> nearly maxxed out for a 50GB file, but I can see that the current
> algorithm for selecting the block size ends up nowhere near the
> maximum block-size limit.
Yes, I think this could be fixed up quite easily. Also I found the current
code does decide from the receiving-side file what blocksize to use. Now
with the case where a receiver file is way smaller than the reference
one, it would pick a rather tiny blocksize due to the available tiny file
and then take nearly a day due to the tiny blocksize. The same applies
when the sync was aborted and restarted.
But getting the sent size to the code selecting the blocksize is not a too tiny
René Rebe - Rubensstr. 64 - 12157 Berlin (Europe / Germany)
http://www.exactcode.de | http://www.t2-project.org
+49 (0)30 255 897 45
More information about the rsync