[Samba] Too many open files, Bug 3342

[Eric Boehm]

On Mon, Feb 06, 2006 at 08:39:34AM -0800, David Highley wrote:
>>>>> "David" == David Highley <dhighley at highley-recommended.com> writes:

    David> RedHat Advanced Server 3 update 3 Linux quinault
    David> 2.4.21-27.0.2.ELsmp #1 SMP Wed Jan 12 23:35:44 EST 2005
    David> i686 i686 i386 GNU/Linux Samba version=3.0.6-2E

    Eric> I see you are using Samba 3.0.6. There were a lot of bug
    Eric> fixes after that release. I would start with version
    Eric> 3.0.14a. I've been using that version with ClearCase
    Eric> (although with Solaris 8).

    Eric> I would not use 3.0.20anything or even the latest 3.0.21b.

    Eric> It might be useful to see your smb.conf as well as your
    Eric> client MVFS settings. However, ClearCase is not within the
    Eric> scope of this mailing list.

It might be helpful to see how you've defined the share.

Here's a template that I've recommended for ClearCase.

#
# Duplicate the following block for each share you wish to
# define. ClearCase related storage areas must have oplocks = no. For
# other applications, you may wish to set oplocks = yes.
#
[<share-name>] 
        comment         = <your-comment-here>
        path            = <path-to-storage>
        # admin users   =
        # hosts allow   = @<netgroup>, host, ip_address
        # valid users   = @<netgroup>, userid
        create mask     = 0775
        directory mask  = 0775 
        # If guest ok = yes, then anyone will be able to connect
        # without authentication. If that is the desired configuration,
        # you may want to make the share read-only (i.e., writeable = no)
        guest ok        = no
        map archive     = no 
        oplocks         = no 
        writeable       = yes

    David> We have tuned the client MVFS setting down to 200 for
    David> Maximum number of mnodes to kee on the VOB free list and
    David> Maximum number of mnodes to keep for cleartext free list.

You might be surprised but these limits are not always observed.
I have seen clients open more than 200 files even when MVFS settings
are tuned down. Is any of this Java code? That's where I've seen this
problem occur (exceeding the mnodes setting).

FWIW, we don't tune down MVFS. We have the scaling factor set to 4 and
let mnodes be 4500 (VOB free) and 1800 (cleartext free). We've been
running this way for years at multiple locations around the globe. We
have anywhere from 20-600+ clients at a location.

    David> Were not looking for ClearCase support, we believe that
    David> this issue maybe a symptom of some other issue but so far
    David> we have not been able to make the connection. Because of
    David> our operating environment, security, it is very hard to
    David> make changes to platform configurations.

I understand your constraints but given that your environment is
having problems, isn't that enough to warrant upgrading? I've looked
at the release notes since 3.0.6 and there are lots of fixes for
memory leaks and problems in the code that handles files. I think
3.0.14a is your next best bet. 3.0.6 is about 1.5 years old.

I would also expect that the Samba team isn't going to be interested
in pursuing the bug unless you can reproduce it against the latest
version (3.0.21b).

Your security setting might be influencing the number of open
connections you have (not necessarily the number of file opens). Do
you have 'security = domain' or 'security = server' in your smb.conf?

    David> What information might be available about the samba
    David> connections that might lead us to determine what triggers
    David> this event?

You might look at the Samba client logs for any strange error
messages. Does the problem occur at a predictable time -- such as
after a large build? or a particular build?

You said that you were using NAS. If that's the case, why aren't you
using CIFS through the NAS? You might not even need Samba.

Lastly, what are 

/proc/sys/fs/file-max
/proc/sys/fs/file-nr

set to?

-- 
Eric M. Boehm                  /"\  ASCII Ribbon Campaign
boehm at nortel.com               \ /  No HTML or RTF in mail
                                X   No proprietary word-processing
Respect Open Standards         / \  files in mail