webdav timeout

[Matt McCutchen]

Helge, please CC rsync at lists.samba.org in your replies so your messages
are archived for others' future benefit.

On Wed, 2009-01-21 at 19:48 +0100, Kraak Helge wrote:
> Thanks for your reply Matt!
> 
> I now tested two different commands with two different versions of  
> rsync. Once I used the timeout option and once I didn't.
> 
> When I don't use the timeout option I don't get any error and the  
> upload to the webdav server works for all files (with 2.6.9 and with  
> 3.0.5), but there's a long break (I guess about 60-90 minutes)  
> between the large files with size of 50 Megabytes and 100 Megabyte.

I see.  Then the underlying problem is the slowness of the operation;
obviously, if you set a timeout, rsync will give up after being stalled
for that amount of time.  We can still try to determine whether the
slowness is the fault of rsync or the WebDAV client.

> I uploaded all system call traces to this address as I don't know  
> which parts of the files are relevant for you: http://kraak.info/rsync/

rsync_system_call_trace_3.0.5.txt is the case I care about (recent
version of rsync, no timeout).  Could you obtain a new trace that shows
the time (either relative or absolute) of each line?  Without that, it's
impossible to tell where the 60-minute delay is.

Incidentally, I wonder if the WebDAV client is performing the final
rename in a silly way, such as by downloading and re-uploading the file
instead of using the WebDAV MOVE method.  If that's the problem, you
could work around it by passing --inplace to rsync.  Or it could be that
the WebDAV client buffers all the data rsync writes to a destination
file and does not actually upload anything until rsync closes the file
(like AFS); in that case, rsync would see a long delay and there's
really nothing that can be done about it.

> Without timeout:
> /usr/local/bin/rsync  --size-only --human-readable --progress -- 
> recursive  --delete  --bwlimit=50 --times --stats   /Users/Helge/ 
> temp  /Volumes/webdav.macbay.de/backup/temp  >> /users/Helge/Pictures/ 
> rsync_Pictures_` date +%Y_%m_%d__%H_%M`.log
> 
> As said: No error messages when I don't use a timeout but it takes  
> several hours to upload the files.

-- 
Matt