[Bug 5124] Parallelize the rsync run using multiple threads and/or connections

[samba-bugs at samba.org]

https://bugzilla.samba.org/show_bug.cgi?id=5124

--- Comment #5 from Andrew J. Kroll <forge at dr.ea.ms> 2014-01-19 03:10:35 UTC ---
Actually having two or three TCP streams at the same time has proven to be
faster, because it can scan ahead. It is proven that if you download one large
file, while downloading several smaller ones, that the entire transfer is
faster because the handshake turn-around is hidden. It has nothing to do with
getting around per-connection bandwidth limiters, although in some cases it can
help with that too. Another proven case is your typical modern web browser.
There is a very good reason why multiple connections are used to load in those
pretty pictures you see. It is all about getting around the latency by using
TCP as a double buffer. What is needed is the ability to be scanning on one
side while transferring a file, and if we have a match as we go with the other
process, start sending it on a second stream. Again, look at how lftp does it.
the concept simply works fantastic. You get multiple dir scans in parallel, and
data when it is to be updated, while still scanning.

UDP? Interesting idea but, not needed. Just do > 1 scan and send process.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.