various "rsync: connection unexpectedly closed" errors on debian

[wearitdown at myrealbox.com]

jw schultz wrote:

>On Sun, Jun 22, 2003 at 11:12:56PM +0800, wearitdown at myrealbox.com wrote:
>  
>
>>>>When I sync'ed against other servers, my rsync client process always 
>>>>ran without --compress.  Nevertheless, I've tried increasing 
>>>>verbosity by 3, and this is what I get:
>>>>        
>>>>
>>>Check the rsync log file. That is where the error message will be.
>>>      
>>>
>>Hi.  Okay, on the server I am downloading from, it shows the following 
>>(doesn't seem very detailed, if more details are needed, how do I ask 
>>the admin to increase it?):
>>
>>   2003/06/21 18:07:03 [2020] rsync: name lookup failed for
>>   203.208.246.57: Name or service not known
>>   2003/06/21 18:07:04 [2020] rsync on gentoo-portage/ from UNKNOWN
>>   (203.208.246.57)
>>   2003/06/21 18:12:16 [2020] rsync error: timeout in data send/receive
>>   (code 30) at io.c(103)
>>    
>>
>Ahh, there it is.  A timeout.  You must be overriding the
>default and setting it too short for the work you want done..
>  
>

That's interesting.  At that time, my client did indeed run --timeout 
300 (the 5 minute gap fits the above interval almost exactly).  So it 
does indeed look like the timeout was the most pressing cause of the 
disconnection.

However, correct me if I'm wrong but, a 5 minute timeout during which 
there is no activity is quite unusual.  When I sync against other 
servers, the entire sync will normally only last 1-2 minutes.  I thought 
this odd, so I removed the --timeout from my client, and obtained errors 
similar to the below (
"error in rsync protocol data stream (code 12) at io.c(165)").

So, I asked the site admin to send me logs again.  The following is 
indicative of the logs.

2003/06/24 23:30:02 [10939] rsync on gentoo-portage from UNKNOWN (203.208.241.235)
2003/06/24 23:34:57 [10939] rsync: read error: Connection reset by peer
2003/06/24 23:34:57 [10939] rsync error: error in rsync protocol data stream (code 12) at io.c(177)


He also said he noticed that when clients fail to specify -z or 
--compress, their sync's against him fail  (he runs the server with 
--compress.  He suggested that I run my syncs with -z, and lo and 
behold, he was right and that sync worked..

>>Whereas on my client, it shows something like this (this clip not taken 
>>from the same session, but likely to be similar).
>>
>>   [snip]
>>   x11-wm/xpde/xpde-0.3.0.ebuild is uptodate
>>   generate_files phase=1
>>   rsync: connection unexpectedly closed (1657342 bytes read so far)
>>   rsync error: error in rsync protocol data stream (code 12) at io.c(165)
>>   _exit_cleanup(code=12, file=io.c, line=165): about to call exit(12)
>>   rsync: connection unexpectedly closed (1493478 bytes read so far)
>>   rsync error: error in rsync protocol data stream (code 12) at io.c(165)
>>   _exit_cleanup(code=12, file=io.c, line=165): about to call exit(12)
>>
>>Not being much of a programmer, I can only hazard a guess that something 
>>in io.c on the server and client don't agree, possibly due to a mistake 
>>on my part.  (The server in this case ran "/usr/bin/rsync --daemon 
>>--no-detach --safe-links --compress --timeout=1800", but I definitely 
>>ran without -z or --compress on my client, so compression should not 
>>have been enabled, right?)
>>    
>>
>
>--timeout=1800
>
>Evidently 3 minutes isn't long enough.
>
>  
>


Okay, I'm pretty confused now.  Here's my attempt to summarise the 
problems I faced, and what I think is happening.

1) Require "--compress" when sync'ing aginst rsyncd's that run with 
--compress.
I consistently (100% of the time) fail to sync against some gentoo 
mirrors (rsync1.jp and rsync1.us) when my client doesn't specify 
--compress.  When I do specify --compress, the sync appers to complete 
successfully.  Is this -- requiring the client to turn on compression if 
the server does -- by design, as I haven't seen anything about this in 
the man pages?

2) timeout in data send/receive at io.c
Even when 1) is not causing problems, I still frequently fail to sync 
due to "timeout in data send/receive at io.c".  I had previously set 
--timeout 300 on my client, but have since removed that parameter.  I'm 
quite stumped by what is causing this timeout in the first place.  Is 
there any way to increase logging to find out?