Suggestion: rsync and direct IO

[Green, Paul]

From: David Sisson [mailto:David.Sisson at av.com] asks:
> [...]
> If we could write directly to the disk using direct-io (or from for that
> matter) we could avoid polluting the operating system's buffer cache
> before we're ready to use the new data.  Obviously this feature isn't
> tied to Direct-IO, so I'd call the new feature something like
> -avoid-system-cache or something.  I will probably make the feature an
> optional one because I can imagine having a situation where sometimes
> I'd want it to hit the system filecache and sometimes I wouldn't.

I think such a feature would be highly desireable. Here at Stratus our disk
cache manager has a feature the caller can set called the "transient file
attribute" that does exactly this; it discards the data block from the disk
cache as soon as the application is done with it (the default is to hold a
referenced data block in the disk cache for several minutes).  We encourage
our customers to enable this bit whenever they are referencing data in a
"single use" mode, such as when they are scanning files to prepare reports.
As you say, it prevents such activities from interfering with the online
transaction processing system's data.

But is there an OS-independent way to set this attribute?  I haven't
memorized the POSIX standard, but I don't recall seeing a way to set or get
such an attribute.

My general feeling is that rsync should, by default, avoid the system cache,
given that it will reference any given block only once. It is just chance
that data that it brings in would be useful to another app.

PG
--
Paul Green, Senior Technical Consultant, Stratus Computer, Inc.
Voice: +1 978-461-7557; FAX: +1 978-461-3610; Video on request.
Speaking from Stratus not for Stratus