[Samba] Samba Performance question

Mon Dec 9 13:20:01 GMT 2002

Hello Wolfgang,

First, to answer your question 1), Samba builds the directory list on the
server.

Now, given what you describe in 2), Samba is not the tool for you. It
handles the situation you describe very poorly.

All is not lost though, since you control the filenames, and presumably the
code that looks for them, I'd rip-out all the samba 8.3-case-insensitive
code in trans2.c (get_lanman2_dir_entry, trans2_find_first, ....) and
replace it with Unix opendir and fopen calls etc. This will give you
something that speaks SMB on the wire, but does not implement the
filesystem correctly. It will be fast though.

All the best,
Paul

              "Belgardt, Wolfgang"                                                                                                
              <Wolfgang.Belgardt at hp.co         To:      <paul.r.schenk at accenture.com>                                             
              m>                               cc:      <samba at samba.org>                                                         
              Sent by:                         Subject: RE: [Samba] Samba Performance question                                    
              samba-admin at lists.samba.                                                                                            
              org                                                                                                                 

              06/12/2002 04:51 PM                                                                                                 

Hello Paul,

Thanks for your explanations of samba doings in this case.

1) where is samba build the in-memory list?  On server  or on the client? I
think on the server, right?
1a) All file  are 8.3 named files;  installed via a NT client
.
2) To your informations: I believe  the customer software is searching the
files with wildcards. What is the customer doing? He read on a NT client
music CDs and build from every Track on this CD a 30 sec MP3 file.  The
software is  automatic create a 8 character long directory and write  for
all tracks,of the CD,  an mp3 file with an 8.3 name. Only the 3 character
exts is different; to recognize the tracks. The write is not a Problem,
because the time is not relevant for the customer.

Kind Regards / Grüsse

Wolfgang

-----Original Message-----
From: paul.r.schenk at accenture.com [mailto:paul.r.schenk at accenture.com]
Sent: Friday, December 06, 2002 19:33
To: Belgardt, Wolfgang
Cc: samba at samba.org
Subject: Re: [Samba] Samba Performance question

We had the same problem here and I traced it to how Samba pretends to be a
Windows server.

Basically Samba does this:

1) build an in-memory list of a directory's contents, with 8.3 mangled
names
2) When asked for a file, look through the list created in step 1) trying
to find a match.  It tries an exact match and then an 8.3 match.

With large numbers of files in directories (I have one with about 650000
files), 1) creates a huge list and 2) takes forever and pegs the CPU at
100%.

In cases with large numbers of files, Windows wins hands-down, because the
8.3-stupid-stuff is handled by the filesystem.

I solved this by making a modified version of a few routines.
1) I make the routines that create the directory list abort after 100 files
and pretend there are no more files.
2) I modified the file opening routine (trans2_readdir, I think) to attempt
to open the file using the filesystem first, bypassing all
case-insenitive-8.3-mangling code. If that fails, I let it try the
look-up-in-a-list method (except for a hard-coded directory where I return
file-not-found if the direct attempt failed).
3) I set 'dont descend' on the big directories, to help users who
mistakenly try to browse the directory with explorer, although mod 1) would
mean they'd only see 100 files anyway.

Making these changes allows my HP9000-D380/2 to outperform a Windows NT 4
Pentium 2 when dealing with directories of over 600000 files. Stock Samba
compiled from source (or the depot from itrc) served files from this
directory at about the rate of 5 min/file, with the CPU pegged at 100%. NT
can handle this in less than 1 second. Now I have over 400 people opening
files in this directory all day, and the CPU doesn't even work up a sweat.

The mods I made break what I understand SMB to be. The broken-ness would
only affect old clients (Win 3.1) and clients that try to open 'AFILE.DOC'
and expect to get 'afile.doc'. Since I control what the client requests, I
could get around this. YMMV.

Hope this helps. Does anybody know if changes to address this problem are
in Samba 3?

All the best,
Paul

              "Belgardt, Wolfgang"

              <Wolfgang.Belgardt at hp.co         To:      <samba at samba.org>

              m>                               cc:

              Sent by:                         Subject: [Samba] Samba
Performance question
              samba-admin at lists.samba.

              org

              05/12/2002 04:45 PM

Dear all,

I have a difficult Problem with samba 2.2.5, I hope everyone can help me.

My customer has samba 2.2.5 running on a HP Alpha Server ES40 Cluster with
Tru64 V5.1. The share on this Server has  3.1 million files in  16000
directories.

Some one this directories have 45000 files on it.

The problem is: if we try a search a file from this  big directory  via an
NT Client the response time is to large for the the customer.

He has run an similar application on a NT File server. NT responded after 1
sec  and samba need 6 sec.

Can someone explain me what I can do to increase the performance, please?

Kind Regards / Mit freundlichen Grüssen

Wolfgang Belgardt
Customer Support Consultant

Hewlett-Packard GmbH
Customer Support
Bonsiepen 5
D-45136 Essen
Phone: ++49 (0) 201 2663 258
Fax:     ++49 (0) 201 2663 200
mobil:   +49 (0171 3357 256)
E-mail:  Wolfgang.Belgardt at hp.com
http://www.hp.com/de
__________________________________________________________________________________

Hewlett-Packard GmbH
Geschäftsführer: Jörg Menno Harms (Vorsitzender), Jürgen Banhardt, Wolfram
Fischer,
 Rainer Kaczmarczyk, Bärbel Schmidt, Fritz Schuller, Regine Stachelhaus
Vorsitzender des Aufsichtsrats: Heribert Schmitz
Sitz der Gesellschaft: Böblingen, Amtsgericht Böblingen HRB 4081

This message is for the designated recipient only and may contain
privileged, proprietary, or otherwise private information.  If you have
received it in error, please notify the sender immediately and delete the
original.  Any other use of the email by you is prohibited.
--
To unsubscribe from this list go to the following URL and read the
instructions:  http://lists.samba.org/mailman/listinfo/samba

This message is for the designated recipient only and may contain
privileged, proprietary, or otherwise private information.  If you have
received it in error, please notify the sender immediately and delete the
original.  Any other use of the email by you is prohibited.