Q: Increasing samba lock performance in a ctdb 1.2 setup?
amitay at gmail.com
Fri Nov 9 05:37:39 MST 2012
On Fri, Nov 9, 2012 at 10:03 PM, Rainer Krienke <krienke at uni-koblenz.de> wrote:
> I am running a CTDB samba cluster with 4 nodes installed with SLES 11SP2
> using samba 3.6.8 and ctdb version 1.2.52. The cluster nodes are virtual
> machines running on a Citrix Xen Server 6.
> The Cluster is basically working fine. What is causing me some headache
> is however a problem that shows up when the total number of active samba
> users rises to a value of about 300 (smbstatus -b|wc -l). Most of the
> users access their windows profiles as well as a software share that
> contains commonly used software like firefox that most user will start.
> When there are many active windows users the effect is that starting
> especially firefox (until the window shows up) takes up to 5 minutes
> where usually it only takes about 20 seconds. This seems not to be due
> to network congestion since there is not really much network traffic on
> the samba nodes at this time (say about 10MB/sec) on a 1 gigabit
> network infrastructure. Copying files from another share via the same
> CTDB cluster works really fast, big files with up to 90MByte/sec. On
> the samba cluster nodes I see that the ctdb process is eating up 40-60%
> of CPU time on one CPU. Each node has a total of 6 CPUs to there is
> enough CPU time for smb processes left that do not use more that 10%.
> Each cluster node has 16GB of RAM at the moment.
> Running smbstatus -L shows that there is a total of about 15000 locks in
> the peak times and about 5000 that are related to firefox files.
> stracing some smb-processes we found that there are many many byte range
> locks executed on this software share. So what we did first was to set
> "fake oplocks = Yes" for this software share. Globally in smb.conf we
> kernel oplocks = yes
> oplocks = yes
> posix locking = no
> The fake oplocks for the software share helped already a lot making
> firefox on windows start up faster (max about 1 minute) which is far
> better that 5 minutes or more.
How is firefox configured on windows clients? Is it using share name
"software" to share the installation of firefox among windows clients?
> I now wonder what I could do else to increase performance for windows
> locking which seems to be the bottle neck to me. I found information
> that suggests to use a ramdisk to store some ctdb databases that are now
> stored in /var/lib/ctdb on a local (virtual) disk on each cluster node.
> Can anyone tell me *which tdb files* exactly I should place on the
> ramdisk? Is it ok to place a symlink in /var/lib/ctdb/<linkname> to the
> file in the ramdisk or how else should I do this?
It might be premature to use tdb files from ramdisk without figuring
out what's causing the lock contention.
> Any other ideas what I could do to find out more that might be related
> to my problemsd are of course also welcome.
> Thank you very much for any help in advance
> Rainer Krienke
> Rainer Krienke, Uni Koblenz, Rechenzentrum, A22, Universitaetsstrasse 1
> 56070 Koblenz, http://userpages.uni-koblenz.de/~krienke, Tel: +49261287 1312
> PGP: http://userpages.uni-koblenz.de/~krienke/mypgp.html,Fax: +49261287
More information about the samba-technical