rsync speed limited by read chunk size?

CM Fields cmfileds at gmail.com
Fri Jul 12 15:46:29 MDT 2013


Wayne,

We see almost double the speed using rsync 3.1.0dev. 314MB/sec compared to
160MB/sec.

~/rsync/rsync-HEAD-20130623-1942GMT# ./rsync --version
rsync  version 3.1.0dev  protocol version 31.PR14

./rsync -avP /data1/Fedora-Live-Desktop.iso  /data2/
 Fedora-Live-Desktop.iso
    997,195,776 100%  314.06MB/s    0:00:03 (xfr#1, to-chk=0/1)

Can you tell me where in the source I might be able to test larger
boundaries then 1K or larger then 16K chunks? These machines should be able
to move data even faster and I would be happy to do testing and post our
results back to the rsync mailing list.





On Fri, Jul 12, 2013 at 1:35 PM, Wayne Davison <wayned at samba.org> wrote:

> On Tue, Jul 9, 2013 at 9:46 AM, CM Fields <cmfileds at gmail.com> wrote:
>
>> Is rsync speed limited to 160MB/sec read speeds due to the chunk
>> read size? Or, are we seeing just a coincidence and the limitation
>> is somewhere else?
>>
>
> The fileio routines are usually reading 256K chunks except for the
> receiver, which tries to read 16K chunks (since it may be randomly
> accessing the file data in some cases).
>
> Give the 3.1.0dev code a try -- it aligns its reads on 1K boundaries.
>  That might help the speed.  It also has better I/O throughput between
> processes.
>
> ..wayne..
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20130712/25db5e67/attachment.html>


More information about the rsync mailing list