matt at mattmccutchen.net
Thu Jan 22 04:04:18 GMT 2009
Helge, please CC rsync at lists.samba.org in your replies so your messages
are archived for others' future benefit.
On Wed, 2009-01-21 at 19:48 +0100, Kraak Helge wrote:
> Thanks for your reply Matt!
> I now tested two different commands with two different versions of
> rsync. Once I used the timeout option and once I didn't.
> When I don't use the timeout option I don't get any error and the
> upload to the webdav server works for all files (with 2.6.9 and with
> 3.0.5), but there's a long break (I guess about 60-90 minutes)
> between the large files with size of 50 Megabytes and 100 Megabyte.
I see. Then the underlying problem is the slowness of the operation;
obviously, if you set a timeout, rsync will give up after being stalled
for that amount of time. We can still try to determine whether the
slowness is the fault of rsync or the WebDAV client.
> I uploaded all system call traces to this address as I don't know
> which parts of the files are relevant for you: http://kraak.info/rsync/
rsync_system_call_trace_3.0.5.txt is the case I care about (recent
version of rsync, no timeout). Could you obtain a new trace that shows
the time (either relative or absolute) of each line? Without that, it's
impossible to tell where the 60-minute delay is.
Incidentally, I wonder if the WebDAV client is performing the final
rename in a silly way, such as by downloading and re-uploading the file
instead of using the WebDAV MOVE method. If that's the problem, you
could work around it by passing --inplace to rsync. Or it could be that
the WebDAV client buffers all the data rsync writes to a destination
file and does not actually upload anything until rsync closes the file
(like AFS); in that case, rsync would see a long delay and there's
really nothing that can be done about it.
> Without timeout:
> /usr/local/bin/rsync --size-only --human-readable --progress --
> recursive --delete --bwlimit=50 --times --stats /Users/Helge/
> temp /Volumes/webdav.macbay.de/backup/temp >> /users/Helge/Pictures/
> rsync_Pictures_` date +%Y_%m_%d__%H_%M`.log
> As said: No error messages when I don't use a timeout but it takes
> several hours to upload the files.
More information about the rsync