[RFC PATCH] fadvise support in rsync

Ben Gamari bgamari.foss at gmail.com
Tue Nov 23 09:20:03 MST 2010

Sending the patchset leader to the list again since the first message is
being held approval for the unbelievably braindead reason of there being
too many recipients. For the patches themselves, see the lkml or prod
the moderators to approve the posts.

Moderators: Could you please approve the first post and disable this
unbelievably counterproductive "feature"? Thanks!


- Ben

On Tue, 23 Nov 2010 09:49:49 -0500, Ben Gamari <bgamari.foss at gmail.com> wrote:
> Warning for kernel folks: I'm not much of an mm person; let me know if I got
> anything horribly wrong.
> Many folks use rsync in their nightly backup jobs. In these applications, speed
> is of minimal concern and should be sacrificed in order to minimize the effect
> of rsync on the rest of the machine. When rsync is working on a large directory
> it can quickly fill the page cache with written data, displacing the rest of
> the system's working set. The solution for this is to inform the kernel that
> our written pages will no longer be needed and should be expelled to disk. The
> POSIX interface for this, posix_fadvise, has existed for some time, but there
> has been no useable implementation in any of the major operating systems.
> Attempts have been made in the past[1] to use the fadvise interface, but kernel
> limitations have made this quite messy. In particular, the kernel supports
> FADV_DONTNEED as a single-shot hint; i.e. if the page is clean when the hint is
> given it will be freed, otherwise the hint is ignored. For this reason it is
> necessary to fdatasync() against dirtied pages before giving the hint. This,
> however, requires that rsync do some accounting, calling fdatasync() and
> fadvise() only after giving the kernel an opportunity to flush the data itself.
> Moreover, fadvise(DONTNEED) frees pages regardless of whether the hinting
> process is the only referrer. For this reason, the previous fadvise patch also
> used mincore to identify which pages are needed by other processes. Altogether,
> this makes using fadvise very expensive from a complexity standpoint. This is
> very unfortunately since the interface could be quite usable with a few minor
> changes.
> I recently asked about this on the LKML[2], where Minchan Kim was nice enough
> to put together a patch improving support for the FADV_DONTNEED hint. His patch
> adds invalidated flagged pages to the inactive list. This obviates the need for
> fdatasync() since the page will be reclaimed by the kernel in the standard
> inactive reclaim path. Moreover, by adding hinted pages to the head of the
> inactive list, other processes are given ample time to call the pages back to
> the active list, eliminating the need for the previous mincore() hack.
> Here is my attempt at adding fadvise support to rsync (against v3.0.7). I do
> this in both the sender (hinting after match_sums()) and the receiver (hinting
> after receive_data()). In principle we could get better granularity if this was
> hooked up within match_sums() (or even the map_ptr() interface) and the receive
> loop in receive_data(), but I wanted to keep things simple at first (any
> comments on these ideas?) . At the moment is for little more than testing.
> Considering the potential negative effects of using FADV_DONTNEED on older
> kernels, it is likely we will want this functionality off by default with a
> command line flag to enable.
> Cheers,
> - Ben
> [1] http://insights.oetiker.ch/linux/fadvise.html
> [2] http://lkml.org/lkml/2010/11/21/59

More information about the rsync mailing list