RAM speedup

[Rupert Gallagher]

On Friday 26 June 2020 21:58, Rupert Gallagher via rsync <rsync at lists.samba.org> wrote:

> Hello,
>
> As disks are slow and rsync reads and writes so much that for the bus this is the equivalent of context switching galore, would it be possible to use RAM as a buffer? Say, you have 10GB of spare RAM, rsync uses the bus to its peak for reading 10GB, then again for writing it down. This would be more efficient than lot of small read/write operations.
>
> Thank you


Current task: rsync 752 GB

source disk
Writing speed: 77 MB/s
Reading speed: 97 MB/s

target disk
Writing speed: 117 MB/s
Reading speed: 99 MB/s

Actual time: 380 min (6.3 hours) to copy 648 GB
Actual speed: 28 MB/s (648/380 = 1.7 GB min =~ 1700MB min / 60 min = 28MB sec)

Potential speed

Since both disks are on same bus,
first we saturate the bus when reading,
then we saturate the bus when writing.

reading time: 752 GB =~ 752000 MB / 97 MB/s =~ 7752 sec = 2.2 hours
writing time: 752 GB =~ 752000 MB / 117 MB/s =~ 6427 sec = 1.7 hours

In summary, the task could be completed in 4 hours instead of 8.

If the disks were on a different bus, and RAM would be refilled while writing on disk,
the task could be completed in 2 hours.