[Bug 13645] New: Improve efficiency when resuming transfer of large files

[samba-bugs at samba.org]

https://bugzilla.samba.org/show_bug.cgi?id=13645

            Bug ID: 13645
           Summary: Improve efficiency when resuming transfer of large
                    files
           Product: rsync
           Version: 3.0.9
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P5
         Component: core
          Assignee: wayned at samba.org
          Reporter: pe1chl at amsat.org
        QA Contact: rsync-qa at samba.org

When transferring large files over a slow network, we interrupt rsync at the
beginning of business hours leaving the transfer unfinished.

The command used is: rsync -av --inplace --bwlimit=400 hostname::module /dest

When restarting the transfer, a lot of time is "wasted" while first the local
system is reading the partially transferred file and sends the checksums to the
remote, which only then starts to read the source file until it finds something
to transfer.  So nothing happens until 2 times the time required to read the
partial transfer from the disks!  When the partial file is many many GB, this
can take hours.

Suggestions:
1. when the source is larger than the destination, immediately begin to
transfer from the offset in the source equal to the size of the destination. 
it is already known that this part will have to be transferred.
2. try to do the reading of the partial file at the destination and the same
part of the source in parallel (so the time is halved), and preferably also in
parallel to 1.

Of course these optimizations (at least #2) may actually decrease performance
when the transfer is local (not over slow network) and the disk read rate is
negatively affected by reading at two different places in parallel.  So #2
should only be attempted when the transfer is over a network.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.