Q: Increasing samba lock performance in a ctdb 1.2 setup?

[Amitay Isaacs]

Hi Rainer,

On Fri, Nov 9, 2012 at 10:03 PM, Rainer Krienke <krienke at uni-koblenz.de> wrote:
> Hello,
>
> I am running a CTDB samba cluster with 4 nodes installed with SLES 11SP2
> using samba 3.6.8 and ctdb version 1.2.52. The cluster nodes are virtual
> machines running on a Citrix Xen Server 6.
>
> The Cluster is basically working fine. What is causing me some headache
> is however a problem that shows up when the total number of active samba
> users rises to a value of about 300 (smbstatus -b|wc -l). Most of the
> users access their windows profiles as well as a software share that
> contains commonly used software like firefox that most user will start.
>
> When there are many active windows users the effect is that starting
> especially firefox (until the window shows up) takes up to 5 minutes
> where usually it only takes about  20 seconds. This seems not to be due
> to network congestion since there is not really much network traffic on
> the samba nodes at this time  (say about 10MB/sec) on a 1 gigabit
> network infrastructure. Copying files from another share via the same
> CTDB cluster works really fast, big files with up to 90MByte/sec.  On
> the samba cluster nodes I see that the ctdb process is eating up 40-60%
> of CPU time on one CPU. Each node has a total of 6 CPUs to there is
> enough CPU time for smb processes left that do not use more that 10%.
> Each cluster node has 16GB of RAM at the moment.
>
> Running smbstatus -L shows that there is a total of about 15000 locks in
> the peak times and about 5000 that are related to firefox files.
>
> stracing some smb-processes we found that there are many many byte range
> locks executed on this software share. So what we did first was to set
> "fake oplocks = Yes" for this software share. Globally in smb.conf we
> configured:
>
>         kernel oplocks = yes
>         oplocks = yes
>         posix locking = no
>
> The fake oplocks for the software share helped already a lot making
> firefox on windows start up faster (max about 1 minute) which is far
> better that 5 minutes or more.

How is firefox configured on windows clients? Is it using share name
"software" to share the installation of firefox among windows clients?

> I now wonder what I could do else to increase performance for windows
> locking which seems to be the bottle neck to me.  I found information
> that suggests to use a ramdisk to store some ctdb databases that are now
> stored in /var/lib/ctdb on a local (virtual) disk on each cluster node.
>
> Can anyone tell me *which tdb files* exactly I should place on the
> ramdisk? Is it ok to place a symlink in /var/lib/ctdb/<linkname> to the
> file in the ramdisk or how else should I do this?

It might be premature to use tdb files from ramdisk without figuring
out what's causing the lock contention.

> Any other ideas what I could do to find out more that might be related
> to my problemsd are of course also welcome.
>
> Thank you very much for any help in advance
> Rainer Krienke
> --
> Rainer Krienke, Uni Koblenz, Rechenzentrum, A22, Universitaetsstrasse  1
> 56070 Koblenz, http://userpages.uni-koblenz.de/~krienke, Tel: +49261287 1312
> PGP: http://userpages.uni-koblenz.de/~krienke/mypgp.html,Fax: +49261287
> 1001312
>

Amitay.