Rsync'ing lists of files

Alberto Accomazzi aaccomazzi at
Tue Jun 11 07:12:01 EST 2002

Just so that we don't forget the lessons from the past, let me point
out that we had discussion and testing done on this subject back in
November, with mixed results (i.e. YMMV):
I think the consensus from that experiment was that implementing the
option using the include/exclude mechanism was not the way to go
(correct me if I'm wrong Dave).

Andrew Schorr's patch does this differently but from what I can tell
it would only work when uploading files to a server (which is the 
opposite of what my experiments with --files-from were):

Since it seems that different people want this option for different
purposes, we need to make sure that it gets implemented in a sensible
way, with some testing being done to ensure that we still have decent
performance and that it works in all cases (sending/receiving files).

So I think our strategy should be to bug Dave Dykstra until he gives
up and writes the patch :-)

-- Alberto

Stephane Paltani wrote:

> Dave Dykstra wrote:
> > 
> > Sigh, another request for the --files-from I promised to write over 6
> > months ago, but I've been so overloaded at work lately that I don't know if
> > I'm ever going to get to it.  Perhaps someone else will have to do it.
> He he, happy to see a general consensus for this feature!
> > It turns out that back in rsync 2.3.2 and earlier there was an optimization
> > (which I wrote and actually was the primary reason that I volunteered to be
> > maintainer of rsync for a while) that kicked in when there was list of
> > includes with no wildcards followed by an --exclude '*', and there was no
> > --delete.  Instead of recursing through the files and doing comparisons, it
> > would just directly open the files in the include list.  It only had to be
> on the sending side, you might want to try 2.3.2 on your sending side to
> > see if you get a significant performance boost.  Andrew Tridgell took it
> > out in 2.4.0 because he didn't like how it changed the usual semantics of
> > requiring all parent directories to be explicitly listed in the include
> > list.
> Whoops! That did the trick for me! It took me 6 minutes to transfer 250 GB!
> Too bad it has been turned down. I have the impression it would satisfy
> most, if not all, "--files-from" lobbyists.

Alberto Accomazzi                          mailto:aaccomazzi at
NASA Astrophysics Data System            
Harvard-Smithsonian Center for Astrophysics
60 Garden Street, MS 83, Cambridge, MA 02138 USA   

More information about the rsync mailing list