Solved (sort of): Strange VFS performance problem

David Collier-Brown David.Collier-Brown at Sun.COM
Thu Feb 17 20:51:37 GMT 2005



Terry Griffin wrote:

> The final culprit was AIO, or rather the absence of any mechanism
> for a user-space application to "instantly" respond to the completion
> of an AIO operation. The 4K sizes sent to VFS pwrite() via the
> Linux/CIFS client made things worse but was not the root cause.
> 
> The original sequence in my VFS modules's pwrite() was:
> 
> - aio_write()
> - Do some other useful stuff with the data
> - Polling loop: aio_error()/nanosleep()
> - aio_return()
> 
> The problem is nanosleep(). (See the BUGS section of the man page.)
> It sleeps way too long for this to be efficient when the data is
> coming in over 1Gb Ethernet.
> 
> One workaround is:
> 
> - aio_write()
> - Do some other useful stuff with the data
> - nice(1)
> - Polling loop: aio_error(), but no sleeping!
> - nice(-1)
> - aio_return()
> 
> This gives me a ten-fold improvement in throughput with the
> Linux/CIFS 4K-writes. The nice(1)/nice(-1) thing is required so the
> polling loop doesn't take CPU away from the aio_write() operation.
> 
> What I really need is a blocking aio_error() with instant return
> on completion of the AIO operation. A real nanosleep() might also
> do it.

	Look and see how the academic lisp community implement
	"futures", which is a mechanism to express exactly what
	you describe...

	(set fx  (future(some fucntion call)))
	(do something else)
	(set y (future x))

	Google pointed me at http://www.ipsj.or.jp/members/SIGNotes/Eng/22/1994/078/article002.html

--dave
> 
> Terry

-- 
David Collier-Brown,      | Always do right. This will gratify
Sun Microsystems, Toronto | some people and astonish the rest
davecb at canada.sun.com     |                      -- Mark Twain


More information about the samba-technical mailing list