fcntl spinlock in Linux?

[Volker Lendecke]

Hi!

For a while I am involved in a scalability test for Samba.
We are running into situations over and over again where the
system load goes through the roof. Load of 2000 or so can
easily be reproduced. One factor that we suspect to be a
problem is fcntl locks, see for example my effort towards
tdb mutex support. I initially suspected that we have a
problem if many fcntl locks on a single file come together
and line up. But recently I came across a case where we are
doing busy fcntl locks on many (thousands) of files. The
async smb echo handler does this. It seems that this also
contributes to the silly load values.

My assumption was that doing fcntl locks on independent
files scales. When taking a look at linux/fs/locks.c I find
the following lines:

static DEFINE_SPINLOCK(file_lock_lock);

/*
 * Protects the two list heads above, plus the
 * inode->i_flock list
 */
void lock_flocks(void)
{
        spin_lock(&file_lock_lock);
}
EXPORT_SYMBOL_GPL(lock_flocks);

void unlock_flocks(void)
{
        spin_unlock(&file_lock_lock);
}
EXPORT_SYMBOL_GPL(unlock_flocks);

My untrained eye interprets this as a single global spinlock
covering all fcntl operations happening on any file.

If that interpretation is right, then I guess with many
cores in a NUMA configuration fcntl locks are just a VERY,
VERY bad idea at all if you want to scale.

Please tell me that I am wrong :-)

Thanks,

Volker

-- 
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
http://www.sernet.de, mailto:kontakt at sernet.de

**********************************************************
visit us at CeBIT: March 5th - 9th 2013, hall 6, booth E15
all about SAMBA and verinice, firewalls, Linux and Windows
free tickets available via email here : cebit at sernet.com !
**********************************************************