[PATCHES] - Statlite Support in Samba/GPFS vfs

[Shekhar Amlekar]

Hi Ira, All,

I'd like to present the statlite patchset again with following
changes from the previous one -

-introducing only one vfs call for statlite, as gpfs supports
(l)statlite but not fstatlite. If required, a fstatlite vfs call can
be added later without changing this one.

-used statlite in few more places in Samba.

-statlite handling in some vfs modules (mostly cut-paste from
stat/lstat handlers)

-added a config option to turn on/off statlite in gpfs vfs module.

-testing results.

> From: Ira Cooper <ira at samba.org>

> Hello,
> I'm sorry I didn't catch this thread earlier.
> Jeremy asked about other systems that implement similar functionality.

> Ceph: Not sure.  I strongly suspect the answer is no.
> Gluster:  I'm fairly sure that Gluster doesn't do this.  (85%+)  At 
> the least we don't expose it at the user API level.

There are some traces of statlite support in ceph here -
https://github.com/ceph/ceph/blob/master/src/include/statlite.h
However, it seems to be not exposed in the library header file,
libcephfs.h

> As far as the change goes: Changes like this are *highly* 
> implementation dependent, IMHO.  The performance implications will 
> be all over the map for various systems.

> Can you provide some benchmarks supporting this change, and telling 
> us how much it is helping you?  Supporting data is a huge part of 
> making an argument like this....

I've a 4-node cluster with gpfs writing to a file on a node
and Samba issuing stat (or statlite) calls on the same file
from other nodes. (In my test, the stat calls were because
of windows clients reading ACL on the file, but it
could also be because of other activity). Here are the results -

-The speed of writer without any stat
interruption-- 470 Mbps
-The speed of writer with interfering stat's from 3
nodes -- ~290 Mbps
-The speed of writer with interfering statlite's
from 3 nodes -- ~460 Mbps

There is 2-3% tolerance in these numbers in different runs.
So, I'm seeing a speed improvement of upto 60% for the
writer in this particular test.

As you noted, since the use of statlite is scattered in
different places, it offers some benefit in different test
cases instead of being immensely useful in one particular
test. More the inode lock contention caused by stat calls
on other nodes (higher stat frequency), more would be
the performance gain.

> The discussion usually goes: "No, that sounds like a bad idea..." 
> "Here is my data supporting the change, isn't it cool?  1000% better
> performance and 5 puppies saved." "Pushed."

> Now, the talk about the architectural issues around xstat, etc... 
> Those are also valid.  I'd like to know why the kernel declined the 
> interface.  So I don't expect a "Pushed" immediately here... but it 
> at least motivates us to actually consider a change like this.

I tried to dig up a bit on this. It seems other than offering
a lite-stat, xstat/fstat interface was also about being able to
retrieve additional metadata for a file - that can probably
also be retrieved using getxattr. Maybe that was the reason?
I didn't follow the complete discussion on this.

Comments welcome. Thanks.

regards,
shekhar.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: stlite-patch
Type: application/octet-stream
Size: 52937 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20141204/e9c8a610/attachment.obj>