[Samba] Samba Performance question

[paul.r.schenk at accenture.com]

We had the same problem here and I traced it to how Samba pretends to be a
Windows server.

Basically Samba does this:

1) build an in-memory list of a directory's contents, with 8.3 mangled
names
2) When asked for a file, look through the list created in step 1) trying
to find a match.  It tries an exact match and then an 8.3 match.

With large numbers of files in directories (I have one with about 650000
files), 1) creates a huge list and 2) takes forever and pegs the CPU at
100%.

In cases with large numbers of files, Windows wins hands-down, because the
8.3-stupid-stuff is handled by the filesystem.

I solved this by making a modified version of a few routines.
1) I make the routines that create the directory list abort after 100 files
and pretend there are no more files.
2) I modified the file opening routine (trans2_readdir, I think) to attempt
to open the file using the filesystem first, bypassing all
case-insenitive-8.3-mangling code. If that fails, I let it try the
look-up-in-a-list method (except for a hard-coded directory where I return
file-not-found if the direct attempt failed).
3) I set 'dont descend' on the big directories, to help users who
mistakenly try to browse the directory with explorer, although mod 1) would
mean they'd only see 100 files anyway.

Making these changes allows my HP9000-D380/2 to outperform a Windows NT 4
Pentium 2 when dealing with directories of over 600000 files. Stock Samba
compiled from source (or the depot from itrc) served files from this
directory at about the rate of 5 min/file, with the CPU pegged at 100%. NT
can handle this in less than 1 second. Now I have over 400 people opening
files in this directory all day, and the CPU doesn't even work up a sweat.

The mods I made break what I understand SMB to be. The broken-ness would
only affect old clients (Win 3.1) and clients that try to open 'AFILE.DOC'
and expect to get 'afile.doc'. Since I control what the client requests, I
could get around this. YMMV.

Hope this helps. Does anybody know if changes to address this problem are
in Samba 3?

All the best,
Paul



                                                                                                                                  
              "Belgardt, Wolfgang"                                                                                                
              <Wolfgang.Belgardt at hp.co         To:      <samba at samba.org>                                                         
              m>                               cc:                                                                                
              Sent by:                         Subject: [Samba] Samba Performance question                                        
              samba-admin at lists.samba.                                                                                            
              org                                                                                                                 
                                                                                                                                  
                                                                                                                                  
              05/12/2002 04:45 PM                                                                                                 
                                                                                                                                  
                                                                                                                                  



Dear all,





I have a difficult Problem with samba 2.2.5, I hope everyone can help me.


My customer has samba 2.2.5 running on a HP Alpha Server ES40 Cluster with
Tru64 V5.1. The share on this Server has  3.1 million files in  16000
directories.


Some one this directories have 45000 files on it.


The problem is: if we try a search a file from this  big directory  via an
NT Client the response time is to large for the the customer.


He has run an similar application on a NT File server. NT responded after 1
sec  and samba need 6 sec.


Can someone explain me what I can do to increase the performance, please?






Kind Regards / Mit freundlichen Grüssen


Wolfgang Belgardt
Customer Support Consultant

Hewlett-Packard GmbH
Customer Support
Bonsiepen 5
D-45136 Essen
Phone: ++49 (0) 201 2663 258
Fax:     ++49 (0) 201 2663 200
mobil:   +49 (0171 3357 256)
E-mail:  Wolfgang.Belgardt at hp.com
http://www.hp.com/de
__________________________________________________________________________________
Hewlett-Packard GmbH
Geschäftsführer: Jörg Menno Harms (Vorsitzender), Jürgen Banhardt, Wolfram
Fischer,
 Rainer Kaczmarczyk, Bärbel Schmidt, Fritz Schuller, Regine Stachelhaus
Vorsitzender des Aufsichtsrats: Heribert Schmitz
Sitz der Gesellschaft: Böblingen, Amtsgericht Böblingen HRB 4081









This message is for the designated recipient only and may contain
privileged, proprietary, or otherwise private information.  If you have
received it in error, please notify the sender immediately and delete the
original.  Any other use of the email by you is prohibited.