[Bug 10518] rsync hangs (100% cpu)

[samba-bugs at samba.org]

https://bugzilla.samba.org/show_bug.cgi?id=10518

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED

--- Comment #5 from Wayne Davison <wayned at samba.org> 2014-03-28 16:57:07 UTC ---
The latest git will have the same issue.  Looks like your HUGE file has created
a very inefficient hash table of chunks, which the sender is spending a lot of
CPU time trying to search through.

The only fix for this is to either try to make a really large hash more
efficient (i.e. drop duplicates if they get out of hand) or to break the file
up into more manageable chunks.

I've been thinking about the latter possibility, and I think that it would be
pretty easy to change the sender to have it stop reading checksums at a certain
point in a file and just have it start generating content, at which point it
would clear the hash table and go 'round again.  The hardest bit is that the
code order doesn't support such a looping, so it might take a bit of work to
get that going.  It shouldn't require any change in protocol, though, as the
receiver would not notice that the sender had decided to chunk-ify the file,
and the generator wouldn't have needed to send anything different.

Rsync has never been very good at really big files, but a change such as the
above should help it.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.