[Samba] Clearcase, Samba, and mnode values

[Kathy]

Hi all --

I recently temporarily deployed an upgraded Clearcase server (7.1.2.3)
on 64 bit RHEL 5.6.  With it I was using Redhat's release of Samba for
that version of the OS -- 3.0.33.    The system has 32 GB of memory
and within 12 hours, it would be down into swap.  Maximum active
licenses used is 36 but a more typical load is 18-25 users having a
Clearcase license at once.  Clearcase clients are on Windows.  This
server is the whole shebang  -- license server, view and vob server.
All disks are local, no NAS.  Most of the time our developers are
using snapshot views but merge into the dynamic views.

It looks to me like I have a memory leak, and IBM is blaming Samba,
seeing that I was using an unblessed version of the tool.  Apparently
3.0.32 is blessed and 3.0.33 is not.  Although I'm skeptical, I'm
willing to entertain the idea that a 0.01 release change could be the
problem, so I recently updated to 3.5.8, which apparently is blessed
to work with 7.1.2.3.  This system is now a test server so we can test
it without impacting production use.  We still need to do some load
testing on it.  The original server is on different hardware (still
Dell, though), running 32 bit RHEL 5.2 and Clearcase 7.0.1.3, and
Samba 3.0.28 (which also apparently isn't blessed by IBM but has never
had any problems).  I am using the samba smb.conf file for both
releases/servers.

For those of you who are familiar with how Clearcase and Samba work
together, my questions are the following:

1.  What is typical memory usage with Clearcase and Samba?  Is it
typical for it to cache all memory, eat into a little bit of swap (say
312k) and then sit there like that and work just fine?  Or should I
expect it to eat up initially a few gigs of memory and then stay
relatively stable, recognizing that over time the running system will
slowly eat up memory (but not all in 12 hours).

2.  IBM has suggested making sure these conditions are met for mnode
values on Windows clients and the MAX_OPEN_FILES is big enough on the
Samba side:   https://www-304.ibm.com/support/docview.wss?uid=swg21169548

Basically saying that for a 64 bit Clearcase/Samba server, the max
mnode values should be set to 800 and not the default 1800.
Anyone with experience with this?  We have not seen the particular
errors mentioned in this article, but IBM support is telling me about
it as a guideline.  Also mentioned in the article is MAX_OPEN_FILES.
My current value for that is 16,384.   How do you know what is an
optimal value for that for your system?

Thanks for any input.  I have asked IBM to describe typical memory
usage but so far I'm not getting any solid answers.

Kathy