Rsync: Re: patch to enable faster mirroring of large filesystems

[Dave Dykstra]

On Tue, Nov 20, 2001 at 11:45:44AM +0000, Lachlan Cranswick wrote:
> 
> Is there any chance this can be added into the distribution as it sounds
> really nifty.

I exchanged some off-list email with the patch author and besides the fact
that it adds too many options I object to it because it only supports
copying from the local side to remote, not also from remote to local.

His option is essentially the same as the --files-from option that was
discussed last January.  See the thread in the archives beginning at

     http://lists.samba.org/pipermail/rsync/2001-January/003368.html


In summary, he can do pretty much what he wants by making an --include-from
list that lists all the parent directories of the files he wants plus all
the files he wants and end it with an --exclude '*', but before rsync 2.4.0
I had an optimization (which I put in when I officially maintained rsync)
that would directly read the included files in that situation rather than
recurse through all the directories.  The author of rsync Andrew Tridgell
took that optimization out in 2.4.0 because he thought it was confusing
that the optimization didn't require explicitly listing the parent
directories like an --exclude '*' otherwise does, and I couldn't prove that
recursing through the directories made a significant performance impact.
Later people argued that a new option --files-from would be worth doing
just for convenience even if not for performance, but I said I still wanted
people to do some performance testing before I'd implement it.  I wanted
people to run version 2.3.2 on their systems and compare the time
difference between running with and without my optimization, which you can
force by simply putting in a single wildcard in one included filename.

I still want to write a --files-from option sometime, and I'm still waiting
for somebody who has an application that could use it to do some
performance measurements with rsync 2.3.2.  I agree that --files-from has
value on its own without performance implications, but somebody has to want
it badly enough to put it in a little effort if they'd like me to implement
it.

- Dave Dykstra