Nice little performance improvement

[Matt McCutchen]

On Sat, 2009-10-17 at 12:13 -0700, Mike Connell wrote:
> > Interesting.  If you're not using incremental recursion (the default in
> > rsync >= 3.0.0), I can see that the "du" would help by forcing the
> > destination I/O to overlap the file-list building in time.  But with
> > incremental recursion, the "du" shouldn't be necessary because rsync
> > actually overlaps the checking of destination files with the file-list
> > building on the source.
> 
> Ignoring incremental recursion for a moment.

Don't ignore it, it makes a difference.

> It seems to me that anything
> that can warm up the file cache before it is needed would be beneficial?

I didn't reason it out carefully enough; let's try again...

Warming up the destination file cache decreases the amount of time the
generator spends blocked on I/O.  So the answer is yes, provided that
the generator is the bottleneck.

If incremental recursion is not used, that's almost certainly the case
during the main phase of the rsync run, since the generator is checking
all the destination files but the sender is only processing the small
number of source files that need a transfer.  But with incremental
recursion, the sender and generator are checking files in parallel, so
the sender may be the bottleneck depending on the relative speeds or
disk configurations of the machines.  (I take it that your rsync run is
local.  For remote runs, the network could be the bottleneck.)

-- 
Matt