retransfer fail of large files with inplace and broken pipe

[Tony Abernethy]

tom raschel wrote:
> Hi,
> 
> i have to tranfer large files each 5-100 GB (mo-fri) over dsl line.
> unfortunately dsl lines are often not very stable and i got a broken
> pipe error.
> (dsl lines are getting a new ip if they are broken or at least after a
> reconnect every 24 hours)
> 
> i had a script which detect the rsync error and restart the
> transmission.
> this means that if a file has transfered e.g. 80 % i start again from
> beginning.
> 
> using partial and partial-dir was no solution to resync because rsync
> cut the
> original file (e.g. from 20 GB to 15 GB) which means that i have to
> transfer
> the whole rest of 5 GB.
> 
> so i had a look at --inplace which I thougt could do the trick, but
> inplace is
> updating the timestamp and if the script start a retransfer after a
> broken pipe
> it fails because the --inplace file is newer than the original file of
> the
> sender.
> 
> using ignore-times could be a solution but slow down the whole process
> to much.
> 
> is there a option to tell rsync not to change the time of a --inplace
> transfered file, or maybe preserve the mtime and do a comparison of
> mtime
> instead of ctime.
> 
> Thx
> Tom
> 
> 
> 
> --
> Please use reply-all for most replies to avoid omitting the mailing
> list.
> To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-
> questions.html


First: Transfer or re-transfer. I think, particularly with bad connections,
you need to treat those VERY differently.

For the initial transfer, --partial should help.

For retransfers, where stuff in the middle has changed, I would expect the 
necessary state information to exist ONLY in the two running processes, and
that information is lost if the connection goes down.
This includes the connection dying because both sides are going through the 
file and have nothing worthwhile to say to each other.

As usual, flames invited if I've got any of this wrong.