Tip for 'nocache' users

Holger Hoffstätte holger.hoffstaette at googlemail.com
Wed Apr 30 06:37:12 MDT 2014


Hi,

Many people are using nocache (https://github.com/Feh/nocache) to prevent
excessive buffer cache pollution by large rsync jobs. If you do so,
you may have started to notice an *insane* slowdown on small-file workloads;
turns out that nocache adds (IMHO unnecessary) fdatasync() calls after
writing each file, making rsync excessively slow and a terrible I/O
torture, especially on single disks.
See: https://github.com/Feh/nocache/issues/22

To restore regular performance for rsync jobs with many small files,
consider either fixing your nocache source or - much easier -
additionally use libeatmydata to undo the fdatasync hit:
https://www.flamingspork.com/projects/libeatmydata/

If you don't know what any of this means, carry on. ;)

hope this helps someone.

Holger



More information about the rsync mailing list