O_DIRECT, avoiding system cache?

Jamie Lokier jamie at shareable.org
Sun Mar 14 21:16:16 MDT 2010


Tomasz Chmielewski wrote:
> Is it possible (planned?) to make rsync avoid going through system cache 
> and use direct IO?
> 
> Right now, if you decide to backup your desktop system (but it's not 
> only about desktop systems; rather more about one-time-only data 
> transfers) to external disk, you will notice that your X session lags 
> terribly, mainly because:
> 
> - system caches everything what rsync reads from the original drive,
> - system caches everything what rsync writes to the destination drive,
> - while doing the above, current cache is being empties.
> 
> 
> I've seen some discussions on rsync mailing list in the past about such 
> a feature (avoiding using system cache), but I'm not sure if there were 
> any conclusions out of it.

The details of how to use O_DIRECT are quite non-portable.
Even on a single OS, the details vary between different filesystem types
and different kernel versions.

In theory, there are posix_madvise() or madvise() flags to accomplish
the goal described above, but how well they work varies a lot.  But at
least those functions, especially posix_madvise(), should be portable
enough to use as an option in rsync.

-- Jamie



More information about the rsync mailing list