Suggestion: rsync and direct IO

[Dave Dykstra]

On Wed, Dec 04, 2002 at 03:41:33PM -0800, David Sisson wrote:
> 	We have some cases when copying indexes that we'd like rsync to avoid
> the system cache when copying a chunk of data from some other machine. 
> I am probably going to modify our own copy of rsync to do this, but if
> we could write directly to the disk using direct-io (or from for that
> matter) we could avoid polluting the operating system's buffer cache
> before we're ready to use the new data.  Obviously this feature isn't
> tied to Direct-IO, so I'd call the new feature something like
> -avoid-system-cache or something.  I will probably make the feature an
> optional one because I can imagine having a situation where sometimes
> I'd want it to hit the system filecache and sometimes I wouldn't.

Yeah, that sounds like a very good idea.

> 	On another topic while I'm thinking about it -- does any one know of a
> reason that you can't set socket options in rsync like you can in rsyncd
> -- if you have a push going in the opposite direction to the server it'd
> be nice to have a larger outgoing buffer size (useful for cross country
> pushes).  We have a local version that has been modified to set the
> buffer size and it performs 3 times faster than the unmodified version.

Probably just an oversight, although I would have thought the buffer size
would be up to the transport program (rsh or ssh).



On Thu, Dec 05, 2002 at 10:01:08AM -0800, Jos Backus wrote:
> On Thu, Dec 05, 2002 at 07:39:53AM -0500, Green, Paul wrote:
> > But is there an OS-independent way to set this attribute?  I haven't
> > memorized the POSIX standard, but I don't recall seeing a way to set or get
> > such an attribute.
>
> Fyi: Solaris has directio(3C), FreeBSD has open(..., O_DIRECT).

What about Linux?  There's an O_DIRECT but the comment in asm/fcntl.h says
it is currently ignored.

- Dave Dykstra