retransfer fail of large files with inplace and broken pipe

Tony Abernethy tony at servasoftware.com
Sun Dec 13 01:10:12 MST 2009


Tom wrote:
> to make things more clear
> 
> 1.)
> first transfer is done either a initial setup or with a usb hdd to get
> sender and receiver in sync.
> 
> 2.)
> transfer does not stop because rsync had a timeout, it stops because
> the dsl
> line is broken (which i could see at dyndns)
> 
> 3)
> if dsl line is stable the transfer is successfull (which works
> furtunately
> most of the time)
> 
> 4.)
> i am searching for a way to reduce the time to retransfer the file or
> in
> other words to resume the filetransfer after a broken pipe
> (e.g. if you download a 4.4 GB Centos Image it is comfortable to resume
> the
> transfer of a 99 % transfered file instead to download all from
> scratch)
> 
> Tom


But you already have 100% of the image, only it is an older version of the image.
The only thing I've found that works (and this is ONLY on something UNIXy) is
to monitor the temporary file on the target and if it is "big enough", rename it
to the intended target file before the target rsync destroys it. 
For real disasters, you can attempt to automate this process.


> > First: Transfer or re-transfer. I think, particularly with bad
> > connections,
> > you need to treat those VERY differently.
> >
> > For the initial transfer, --partial should help.
> >
> > For retransfers, where stuff in the middle has changed, I would
> expect the
> > necessary state information to exist ONLY in the two running
> processes,
> > and
> > that information is lost if the connection goes down.
> > This includes the connection dying because both sides are going
> through
> > the
> > file and have nothing worthwhile to say to each other.
> >
> > As usual, flames invited if I've got any of this wrong.
> > --
> > Please use reply-all for most replies to avoid omitting the mailing
> list.
> > To unsubscribe or change options:
> > https://lists.samba.org/mailman/listinfo/rsync
> > Before posting, read: http://www.catb.org/~esr/faqs/smart-
> questions.html
> >
> 
> 
> 
> --
> Please use reply-all for most replies to avoid omitting the mailing
> list.
> To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-
> questions.html


More information about the rsync mailing list