Samba Speed Question - Please Help!

[Rob Tanner]

Poor UNIX performance with large directories (e.g., 10,000 files) is 
principally due to the overhead of searching the directory space.  In this 
case it would appear that the copy request was based on a wildcard which 
meant the directory had to be searched.  But a second factor in this case 
is the time it takes to set up all the directory pointers (I don't know 
from what was said below with files were being written to read from the 
SAMBA server).  You will also find that on must UNIX systems a tar archive 
containing a single 1Gb file can be untarred in just a few seconds, 
basically at about the transfer rate of the disk.  However, with a tar 
archive of 10,000 10Kb files, you might as well take a lunch break.  I 
would guess that the time the system takes to create 10,000 file pointers 
and allocate space (one file at a time) is going to exceed the cost of 
wildcard searching (as in "copy *")

-- Rob


--On 10/04/00 10:05:29 AM +0200 Robert Dahlem <Robert.Dahlem at gmx.net> wrote:

> Justen,
>
> On Wed, 4 Oct 2000 13:50:30 +1000, Justen Marshall wrote:
>
>> I'm currently experiencing an excessive amount of CPU usage by my
>> smbd daemons, and was wondering what I could do to reduce it.
>
>>  - Experimented with file transfers and discovered something
>>    interesting... the copy of a 1Gb file proceeded at around 7.5Mb
>>    per second, which is in line with the 9Mbps we achieved with FTP
>>    (which is always expected to be a bit faster than a more complex
>>    protocol). However, when copying 10,000 files of 10k each (only
>>    100Mb) it took AGES, and CPU use went towards 100% for the smbd
>>    thread handling the requests.
>
> Well, most unixes do perform really bad when it comes to directories
> of such size.
>
>> If you could please offer me any advice on how to tune Samba for high
>> frequency, small size file access,
>
> I guess you checked the standard options for such cases:
>
> no "hide [dot] files"
> no "veto files"
> no "dont descend"
>
> wide links = yes
> getwd cache = yes
> change notify timeout = <as much as possible>
> debug level = <as low as possible>
>
> Do you have a lot of locks? Dir you consider setting "lock directory"
> to a virtual disk in memory?
>
> I don't know your OS but at least one of the OSs I used in the past
> had a tuning option in itself to cache more directory entries. Might
> be worth to check with your "local dealer".
>
> Regards,
>         Robert
>
>
>
> --
> ---------------------------------------------------------------
> Robert.Dahlem at gmx.net           Fax +49-69-432647
> ---------------------------------------------------------------
>
> Sent using PMMail (http://www.pmmail2000.com) - fast, decent, email
> software; far better than Outlook. Try it sometime.
>
>
>




       _ _ _ _           _    _ _ _ _ _
      /\_\_\_\_\        /\_\ /\_\_\_\_\_\
     /\/_/_/_/_/       /\/_/ \/_/_/_/_/_/  QUIDQUID LATINE DICTUM SIT,
    /\/_/__\/_/ __    /\/_/    /\/_/          PROFUNDUM VIDITUR
   /\/_/_/_/_/ /\_\  /\/_/    /\/_/
  /\/_/ \/_/  /\/_/_/\/_/    /\/_/         (Whatever is said in Latin
  \/_/  \/_/  \/_/_/_/_/     \/_/              appears profound)

  Rob Tanner
  UNIX and Networks Manager
  Linfield College, McMinnville OR
  (503) 434-2558 <rtanner at linfield.edu>