Large file - match process taking days
wayned at samba.org
Thu Jul 31 02:29:46 GMT 2008
On Wed, Jul 30, 2008 at 04:33:04PM +0300, Shachar Shemesh wrote:
> Rsync does have some CPU inefficient behavior for especially large files.
> However, it should not happen at the block size you are using (assuming the
> files are fairly identical). Try increasing it a little further, to 1638400
> (80% utilization on the hash table), and see if things are any better.
Rsync 3.0.x automatically sizes the hash table to keep things around 80%
utilization, so the block size shouldn't make any difference in that
regard (using a slightly tweaked version of your patch). Hmm, this
improvement does not seem to be in the NEWS for 3.0.0, so I added it
into the OLDNEWS file and the on-line version.
More information about the rsync