fcntl spinlock in Linux?

[Stefan (metze) Metzmacher]

Am 24.01.2013 13:00, schrieb Volker Lendecke:
> Hi!
> 
> For a while I am involved in a scalability test for Samba.
> We are running into situations over and over again where the
> system load goes through the roof. Load of 2000 or so can
> easily be reproduced. One factor that we suspect to be a
> problem is fcntl locks, see for example my effort towards
> tdb mutex support. I initially suspected that we have a
> problem if many fcntl locks on a single file come together
> and line up. But recently I came across a case where we are
> doing busy fcntl locks on many (thousands) of files. The
> async smb echo handler does this. It seems that this also
> contributes to the silly load values.
> 
> My assumption was that doing fcntl locks on independent
> files scales. When taking a look at linux/fs/locks.c I find
> the following lines:
> 
> static DEFINE_SPINLOCK(file_lock_lock);
> 
> /*
>  * Protects the two list heads above, plus the
>  * inode->i_flock list
>  */
> void lock_flocks(void)
> {
>         spin_lock(&file_lock_lock);
> }
> EXPORT_SYMBOL_GPL(lock_flocks);
> 
> void unlock_flocks(void)
> {
>         spin_unlock(&file_lock_lock);
> }
> EXPORT_SYMBOL_GPL(unlock_flocks);
> 
> My untrained eye interprets this as a single global spinlock
> covering all fcntl operations happening on any file.
> 
> If that interpretation is right, then I guess with many
> cores in a NUMA configuration fcntl locks are just a VERY,
> VERY bad idea at all if you want to scale.
> 
> Please tell me that I am wrong :-)

Reading through
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=blob;f=Documentation/DocBook/kernel-locking.tmpl
indicates to me that this could be solved using RCU to protect the
global lock list.

metze

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20130124/0e64a221/attachment.pgp>