Suggestion: rsync and direct IO
dwd at drdykstra.us
Thu Dec 5 21:04:00 EST 2002
On Wed, Dec 04, 2002 at 03:41:33PM -0800, David Sisson wrote:
> We have some cases when copying indexes that we'd like rsync to avoid
> the system cache when copying a chunk of data from some other machine.
> I am probably going to modify our own copy of rsync to do this, but if
> we could write directly to the disk using direct-io (or from for that
> matter) we could avoid polluting the operating system's buffer cache
> before we're ready to use the new data. Obviously this feature isn't
> tied to Direct-IO, so I'd call the new feature something like
> -avoid-system-cache or something. I will probably make the feature an
> optional one because I can imagine having a situation where sometimes
> I'd want it to hit the system filecache and sometimes I wouldn't.
Yeah, that sounds like a very good idea.
> On another topic while I'm thinking about it -- does any one know of a
> reason that you can't set socket options in rsync like you can in rsyncd
> -- if you have a push going in the opposite direction to the server it'd
> be nice to have a larger outgoing buffer size (useful for cross country
> pushes). We have a local version that has been modified to set the
> buffer size and it performs 3 times faster than the unmodified version.
Probably just an oversight, although I would have thought the buffer size
would be up to the transport program (rsh or ssh).
On Thu, Dec 05, 2002 at 10:01:08AM -0800, Jos Backus wrote:
> On Thu, Dec 05, 2002 at 07:39:53AM -0500, Green, Paul wrote:
> > But is there an OS-independent way to set this attribute? I haven't
> > memorized the POSIX standard, but I don't recall seeing a way to set or get
> > such an attribute.
> Fyi: Solaris has directio(3C), FreeBSD has open(..., O_DIRECT).
What about Linux? There's an O_DIRECT but the comment in asm/fcntl.h says
it is currently ignored.
- Dave Dykstra
More information about the rsync