Question about --partial-dir and aborted transfers of large files

[Wayne Davison]

On Fri, Aug 10, 2012 at 9:03 AM, T.J. Crowder <tj at crowdersoftware.com>wrote:

> 1. Am I correct in inferring that when rsync sees data for a file in the
> --partial-dir directory, it applies its delta transfer algorithm to the
> partial file?
>
2. And that this is _instead of_ applying it to the real target file? (Not
> a nifty three-way combination.)
>

Yes.  The current code behaves the same as if you had specified --partial
(as far as the next transfer goes), just without actually being destructive
of the destination file.

I have imagined making the code pretend that the partial file and any
destination file are concatenated together for the purpose of generating
checksums.  That would allow content references to both files, but rsync
would need to be enhanced to open both files in both the generator and the
receiver and be able to figure out what read goes where (which shouldn't be
too hard).  I'd suggest that the code read the partial file first, padding
out the end of its data to an even checksum-sized unit so that the
destination file starts on a even checksum boundary (so that the code never
needs to combine data from two files in a single checksum or copy
reference).

If so, it would appear that this means a large amount of unnecessary data
> may end up being transferred in the second sync of a large file if you
> interrupt the first sync.
>

It all depends on where you interrupt it and how much data matches in the
remaining portion of the destination file.  It does give you the option of
discarding the partial data if it is too short to be useful, or possibly
doing your own concatenation of the whole (or trailing portion) of the
destination file onto the partial file, should you want to tweak things
before resuming the transfer.

..wayne..
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20120812/1da0e5c6/attachment.html>