[Samba] Too many open files, Bug 3342
David Highley
dhighley at highley-recommended.com
Mon Feb 6 19:54:16 GMT 2006
"Eric Boehm wrote:"
>
> On Mon, Feb 06, 2006 at 08:39:34AM -0800, David Highley wrote:
> >>>>> "David" == David Highley <dhighley at highley-recommended.com> writes:
>
> David> RedHat Advanced Server 3 update 3 Linux quinault
> David> 2.4.21-27.0.2.ELsmp #1 SMP Wed Jan 12 23:35:44 EST 2005
> David> i686 i686 i386 GNU/Linux Samba version=3.0.6-2E
>
> Eric> I see you are using Samba 3.0.6. There were a lot of bug
> Eric> fixes after that release. I would start with version
> Eric> 3.0.14a. I've been using that version with ClearCase
> Eric> (although with Solaris 8).
>
> Eric> I would not use 3.0.20anything or even the latest 3.0.21b.
>
> Eric> It might be useful to see your smb.conf as well as your
> Eric> client MVFS settings. However, ClearCase is not within the
> Eric> scope of this mailing list.
>
> It might be helpful to see how you've defined the share.
[global]
workgroup = NW
server string = Samba Server
security = DOMAIN
password server = adc-nw-03 adc-nw-01
log file = /var/log/samba/%m.log
max log size = 0
socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192
os level = 0
preferred master = No
local master = No
domain master = No
dns proxy = No
ldap ssl = no
create mask = 0664
cups options = raw
use sendfile = no
[export]
comment = ClearCase Vobs and Views
path = /export
valid users = +nw_usawacs
read only = No
oplocks = No
level2 oplocks = No
>
> Here's a template that I've recommended for ClearCase.
>
> #
> # Duplicate the following block for each share you wish to
> # define. ClearCase related storage areas must have oplocks = no. For
> # other applications, you may wish to set oplocks = yes.
> #
> [<share-name>]
> comment = <your-comment-here>
> path = <path-to-storage>
> # admin users =
> # hosts allow = @<netgroup>, host, ip_address
> # valid users = @<netgroup>, userid
> create mask = 0775
> directory mask = 0775
> # If guest ok = yes, then anyone will be able to connect
> # without authentication. If that is the desired configuration,
> # you may want to make the share read-only (i.e., writeable = no)
> guest ok = no
> map archive = no
> oplocks = no
> writeable = yes
>
> David> We have tuned the client MVFS setting down to 200 for
> David> Maximum number of mnodes to kee on the VOB free list and
> David> Maximum number of mnodes to keep for cleartext free list.
>
> You might be surprised but these limits are not always observed.
> I have seen clients open more than 200 files even when MVFS settings
> are tuned down. Is any of this Java code? That's where I've seen this
> problem occur (exceeding the mnodes setting).
We do all the Java builds on the Linux platform as it is much faster. We
push our C++ builds by running parallel builds and we build all
platforms in the same view so we generate a pretty high file system and
CPU load. Java builds are run serial of course as ANT is not compatible
with parallel building. We also switched to GNU make as clearmake does
not throttle across sub make processes when doing parallel builds.
Some of our builds on the PCs will have over 1800 threads, 600-700
processes, and 20,000+ file handles open. Our build process has been
monitored and IBM has been in and reviewed the ClearCase aspects.
> FWIW, we don't tune down MVFS. We have the scaling factor set to 4 and
> let mnodes be 4500 (VOB free) and 1800 (cleartext free). We've been
> running this way for years at multiple locations around the globe. We
> have anywhere from 20-600+ clients at a location.
Scaling factor is 1. We also remotely monitor the client settings.
>
> David> Were not looking for ClearCase support, we believe that
> David> this issue maybe a symptom of some other issue but so far
> David> we have not been able to make the connection. Because of
> David> our operating environment, security, it is very hard to
> David> make changes to platform configurations.
>
> I understand your constraints but given that your environment is
> having problems, isn't that enough to warrant upgrading? I've looked
> at the release notes since 3.0.6 and there are lots of fixes for
> memory leaks and problems in the code that handles files. I think
> 3.0.14a is your next best bet. 3.0.6 is about 1.5 years old.
>
> I would also expect that the Samba team isn't going to be interested
> in pursuing the bug unless you can reproduce it against the latest
> version (3.0.21b).
>
> Your security setting might be influencing the number of open
> connections you have (not necessarily the number of file opens). Do
> you have 'security = domain' or 'security = server' in your smb.conf?
security = DOMAIN
>
> David> What information might be available about the samba
> David> connections that might lead us to determine what triggers
> David> this event?
>
> You might look at the Samba client logs for any strange error
> messages. Does the problem occur at a predictable time -- such as
> after a large build? or a particular build?
No, that is the biggest issue in trying to track this thing down. We
have turned on some server monitor, but how do you know what is going on
at any moment in time with the 400 plus PC clients or the network for
that matter.
> You said that you were using NAS. If that's the case, why aren't you
> using CIFS through the NAS? You might not even need Samba.
It depends on the storage units. Wish we had NetApps every where...
>
> Lastly, what are
>
> /proc/sys/fs/file-max
393203
> /proc/sys/fs/file-nr
18161 5719 393203
>
> set to?
>
> --
> Eric M. Boehm /"\ ASCII Ribbon Campaign
> boehm at nortel.com \ / No HTML or RTF in mail
> X No proprietary word-processing
> Respect Open Standards / \ files in mail
>
More information about the samba
mailing list