Rsync: Re: patch to enable faster mirroring of large filesystems

[Dave Dykstra]

On Thu, Nov 29, 2001 at 11:02:07AM -0500, Alberto Accomazzi wrote:
...
> These numbers show that reading the filenames this way rather than using
> the code in place to deal with the include/exclude list cuts the startup
> time down to 0 (from 1hr).  The actual sending of the filenames is down
> from 2h 15m to 1h 40m.  The reason this isn't better is due to the fact
> that turning buffering on only helps the client, while the server still
> has to do unbuffered reads because of the way the list is sent across. 

Are you sure about that?  I don't see any unbuffered reads.

2.3.2 did have the read_check() hack which was there to prevent SSH pipes
from getting stuck, maybe that's what you're seeing.  That was taken out
in 2.4.0 so maybe that would greatly speed it up.

> As far as I can tell there is no way to get around the buffering without
> a protocol change or a different approach to sending this list.
> 
> Given the data above, I think implementing --files-from this way would
> be the wrong way to go, for a number of reasons:

I've been starting to think along those lines too.  It should be a protocol
change to just send the files and not treat it like excludes.  In fact,
the file list is normally sent from the sender to the receiver, but if
the client is the receiver maybe we could figure out a way to have
--files-from only send the list in the other direction.

- Dave Dykstra