Suggested chnage to "--partial" usage.

[Chris Shoemaker]

On Wed, Jun 16, 2004 at 08:30:16PM +0800, Jason Potter wrote:
> Hi There,
> 
>  
> 
> This post is brought about due to the following two:
> 
> http://www.mail-archive.com/rsync@lists.samba.org/msg10702.html
> 
> http://www.mail-archive.com/rsync@lists.samba.org/msg10709.html
> 
>  
> 
> I have a situation where I need to upload large files over an unstable link
> (resuming is a requirement) and only when they are complete can they be
> renamed and hence replace the original file.  The process is an automated
> one so I don't have the ability to just walk over to the machine and see if
> it is finished by manually comparing the file size at the source and
> destination.
> 
>  
> 
> I have looked at the source code and have an alternative suggestion to the
> way -partial works.  I am looking for comment on if people think this is a
> valid suggestion or am I missing something.
> 
>  
> 
> 1)       In cleanup.c the function _exit_cleanup will call finish_transfer
> regardless of the code the function receives as a parameter. (see errcode.h
> for the codes it could receive)
> 
> 2)       Suggest a new function is written to adjust the name of the file
> the temp file is written to, to be the correct name plus a know extension.
> This achieves the requirement that the original file does not get over
> written if the file is not complete.
> 
> 3)       If the above was implemented you would then just have to adjust the
> scyncing code to check for these temporary files and use these to resume the
> connections.
> 
> 4)       When the file is complete _exit_cleanup gets a code value of 0
> (zero) and this can be used to call the currently written finish_transfer.
> 
>  
> 
> So what do you all think, will it work. 
> 

I can see the usefulness of such a feature, but ...

what if the portion of the source or destination file that was
already transferred (and which was stored in the temporary file) is modified
between subsequent attempts?  (I don't mean the temp file changes, I mean
the actual src or dest.)  ISTM, you'd have to at least _check_ for this
condition.  That means rechecksumming the source file on the sender from the
beginning.  On the receiver side, it means checksumming both the
(possibly modified) destination file and the previously saved temp file.
Then you have 3-way compare:
   sumA = checksum from source file
   sumB = checksum from possibly modified dest
   sumT = checksum from previously saved temp

   if sumA == sumB, do nothing.
   if sumA != sumB && sumA == sumT,
      then no retransmit needed, use block from tempfile
   if sumA != sumB && sumA != sumT, 
      then retransmit anyway, throw away temp block.


Interesting.  I haven't delved into the core rsync algorithm enough to
say for sure that this is possible, but I don't know that it's
_im_possible.  :-)

-chris




 
>  
> 
> I look forward to your responses.
> 
>  
> 
> Cheers
> 
> Jason
> 
>  
> 
> -- 
> To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html