Problem rsyncing 450GB file to my NAS: 'connection unexpectedly closed'

[Justin T Pryzby]

You can tcpdump it to see which side is closing the connection.  My
understanding is that "the network" isn't timing out, but one end of
the rsync, due to the other end doing slow things at various points in
the connection.  In the immediate case, one side is probably timing
out while the other side is still building hashes of the 450G file.
In other cases, it may be due to scanning a large directory or (pre
3.0 or with --no-inc-recur, a large file list across many directories.

Justin

On Fri, Oct 12, 2012 at 09:41:16PM +0200, rsyncml.frucade at spamgourmet.com wrote:
> Damn! Just in the moment I clicked send it aborted again.
> 
> This is the tail of my rsyncd.log:
> 2012/10/12 19:36:06 [7325] chunk[261276] offset=34245967872
> len=131072 sum1=db0593fa
> 2012/10/12 19:36:06 [7325] [receiver] _exit_cleanup(code=30,
> file=io.c, line=137): entered
> 2012/10/12 19:36:06 [7325] chunk[261277] offset=34246098944
> len=131072 sum1=2153f6c2
> .... more
> 2012/10/12 19:36:06 [7325] chunk[261300] offset=34249113600
> len=131072 sum1=3989ce96
> 2012/10/12 19:36:06 [7325] rsync error: timeout in data send/receive
> (code 30) at io.c(137) [receiver=3.0.7]
> 2012/10/12 19:36:06 [7325] [receiver] _exit_cleanup(code=30,
> file=io.c, line=137): about to call exit(30)
> 2012/10/12 19:36:06 [7325] rsync: connection unexpectedly closed
> (322 bytes received so far) [generator]
> 2012/10/12 19:36:06 [7325] [generator] _exit_cleanup(code=12,
> file=io.c, line=601): entered
> 2012/10/12 19:36:06 [7325] rsync error: error in rsync protocol data
> stream (code 12) at io.c(601) [generator=3.0.7]
> 2012/10/12 19:36:06 [7325] [generator] _exit_cleanup(code=12,
> file=io.c, line=601): about to call exit(12)
> 
> On the client side the last chunk is cb753436.
> 
> Does the client do a hangup?
> 
> On 12.10.2012 21:36, Ben wrote:
> >Wow: Thanks for your fast responses, Justin and Karl!
> >
> >Yet the NAS is still located in my home network. So the network
> >connection shouldn't be the problem.
> >
> >The pointer to the --inplace argument was really helpful. This is what I
> >really wanted in this particular use case.
> >
> >I also tried to enabled file logging on the target NAS:
> >2012/10/12 18:48:17 [6390] rsync error: timeout in data send/receive
> >(code 30) at io.c(137) [receiver=3.0.7]
> >
> >Oha! But the rsyncd.conf on the NAS did not contain a timeout entry?! I
> >also stumbled over an RLimitRate entry not listed in the rsyncd man.
> >Maybe this rsyncd is a patched by QNap and timeout no longer defaults to 0?
> >
> >So I added "timeout = 0" and started a new run with --inplace and -vvvv:
> >Until now its still running fine!
> >
> >I'll report back when it finishes....
> >So long - thanks for the quick and helpful pointers!
> >
> >- Ben
> >
> >On 12.10.2012 20:06, Justin T Pryzby - justinp at norchemlab.com wrote:
> >>Not sure, but some ideas:
> >>
> >>-P means to retain partial files, and doesn't have anything to do with
> >>/tmp; I wonder if you mean --inplace (not sure)?
> >>
> >>It sounds like a daemon may be timing out; is there a timeout
> >>specified in rsyncd.conf? Is there a remote logfile with any useful
> >>content?
> >>
> >>Justin
> >>
> >>On Fri, Oct 12, 2012 at 07:29:58PM +0200,
> >>rsyncml.frucade at spamgourmet.com wrote:
> >>>Hi list!
> >>>
> >>>I'm trying to use rsync to do a regular remote backup of a 450GB
> >>>sized container file located on my squeeze server onto a colocated
> >>>QNap NAS device.
> >>>
> >>>Both use rsync 3.0.7. While this setup works for most files, it
> >>>fails on this large file (see log below).
> >>>
> >>>I'm already trying to help rsync using "-P" to indicate that rsync
> >>>should reuse the current copy and not trying to create a copy on the
> >>>/tmp parition of the NAS.
> >>>
> >>>Any hints what might be the problem?
> >>>
> >>>Thanks
> >>>- Ben
> >
> 
> 
> -- 
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>