O_DIRECT, avoiding system cache?
jamie at shareable.org
Sun Mar 14 21:16:16 MDT 2010
Tomasz Chmielewski wrote:
> Is it possible (planned?) to make rsync avoid going through system cache
> and use direct IO?
> Right now, if you decide to backup your desktop system (but it's not
> only about desktop systems; rather more about one-time-only data
> transfers) to external disk, you will notice that your X session lags
> terribly, mainly because:
> - system caches everything what rsync reads from the original drive,
> - system caches everything what rsync writes to the destination drive,
> - while doing the above, current cache is being empties.
> I've seen some discussions on rsync mailing list in the past about such
> a feature (avoiding using system cache), but I'm not sure if there were
> any conclusions out of it.
The details of how to use O_DIRECT are quite non-portable.
Even on a single OS, the details vary between different filesystem types
and different kernel versions.
In theory, there are posix_madvise() or madvise() flags to accomplish
the goal described above, but how well they work varies a lot. But at
least those functions, especially posix_madvise(), should be portable
enough to use as an option in rsync.
More information about the rsync