[Samba] Samba tuning advise

[Jeremy Allison]

On Thu, Sep 14, 2006 at 08:08:07PM +0200, Fabrice.Davaine at AltisSemiconductor.com wrote:
> Helo,
> 
> We use a big fileserver running Linux/Samba as fileshare. 
> We are facing performances problems for which we'd like some advise :
> 
> Clients are OS2(LanManager)  and windows 2000 reading and writing lots of 
> files in the same directory on the fileshare.
> We currently have more than 80 000 files in the directory (files size is 
> about 100 Bytes).
>  
> Could you please advise us some Samba tuning you would apply in a such 
> situation. I'm sure you also have some
> recommendations on Linux parameters like filsystem type, general 
> kernel/network settings.
> 
> Feel free to forward this to whoever who could help us.

Here's a copy of an old mail of mine....

> So here's how to set up an application that needs large
> number of files per directory in a way that doesn't damage
> performance.
>
> Firstly, you need to canonicalize all the files in the
> directory to have one case, upper or lower - take your
> pick (I chose upper as all my files were already upper
> case names). Then set up a new custom share for the
> application as follows:
>
> [bigshare]
>         path = /home/jeremy/tmp/manyfilesdir
>         read only = no
> 	case sensitive = True
>         default case = upper
>         preserve case = no
>         short preserve case = no
>
> Of course, use your own path and settings, but set the
> case options to match the case of all the files in your
> directory. The path should point at the large directory
> needed for the application - any new files created in
> there and in any paths under it will be forced by smbd
> into upper case - but smbd will no longer have to scan
> the directory for names - it knows that if a file doesn't
> exist in upper case then it doesn't exist at all.
>
> The secret to this is really in the "case sensitive = True"
> line - it tells smbd never to scan for case-insensitive
> versions of names. So if an application asks for a file
> called "FOO", and it can't be found by a simple stat call,
> then smbd will return file not found immediately without
> scanning the containing directory for a version of a different
> case. The other "xxx case xxx" lines make this work by forcing
> a consistent case on all files created by smbd.
>
> Remember, all files and directories under the "path" directory
> must be in upper case with this smb.conf stanza as smbd won't
> be able to find lower case filenames with these settings. Also
> note this is done on a per-share basis, allowing this to be set
> only for a share servicing an application with this problematic
> behaviour (using large numbers of entries in a directory) - the
> rest of your smbd shares don't need to be affected.
>
> This makes smbd *much* faster when dealing with large directories.
> My test case has over 100,000 files and smbd now deals with this
> very efficiently.
>
> So please give this a test if you have problems with
> Samba and large sized directories. Remember this is in SVN code
> only, it isn't in the 3.0.11 pre releases or rc candidates,
> as we need to ensure this new code is correct. If you
> can help me test it it'll be in 3.0.12 (security problems
> notwithstanding :-).
>
> Cheers,
>
> 	Jeremy.