Problem rsyncing 450GB file to my NAS: 'connection unexpectedly closed'

Libor Klepáč libor.klepac at bcom.cz
Tue Oct 16 04:16:09 MDT 2012


Hello, 
i was dealing with same thing , I coudln't find a sollution using rsyncd.
Using rsync+ssh did the trick ...
Problem was (probably) with slow NAS, taking ages to compute (initial?) 
checksums of target file.
When using --progress or --verbose, it was just sitting there without output, 
but spinning CPU hard on NAS side, then timeout (even with timeout set to 2 
hours)
Rsync over ssh does the same thing, but it doesn't timeout ... 

https://lists.samba.org/archive/rsync/2012-August/027769.html

With regards
Libor

On Saturday 13 October 2012 14:03:06 rsyncml.frucade at spamgourmet.com wrote:
> I can't believe it -- but I DO fiddle around with _wireshark_ to get my
> _backup_ done. Strange age of the 21th century.
> 
> Ok - here is what I was able to observe: At first everything seems to be
> fine. After handshake the rsyncd is continuously sending every 1.7s
> packets to the client which the client immediately and silently ACKs.
> 
> This goes until some random point in time, where the rsyncd sends within
> the same ms first a regular packet, then a new packet with the timeout
> message and hangsup the connection. Duh!
> 
> Even with -vvvvv the logs do not really provide more helpful pointers. A
> quick google search revealed, that really many others share the same
> issue with huge files, though w/o a solution.
> 
> The directory contains only 5 files and I was able to obvserve a
> continuous stream of packets, so probably its not about IO operations.
> Maybe some kernel/socket/network stack issue?
> 
> I assume that at this point everybody, else is on this list is just as
> puzzled and clueless as I am now?!
> 
> - Ben
> 
> Am 12.10.2012 22:57, schrieb Justin T Pryzby - justinp at norchemlab.com:
> > You can tcpdump it to see which side is closing the connection.  My
> > understanding is that "the network" isn't timing out, but one end of
> > the rsync, due to the other end doing slow things at various points in
> > the connection.  In the immediate case, one side is probably timing
> > out while the other side is still building hashes of the 450G file.
> > In other cases, it may be due to scanning a large directory or (pre
> > 3.0 or with --no-inc-recur, a large file list across many directories.
> > 
> > Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20121016/43b4dff8/attachment.html>


More information about the rsync mailing list