Samba CPU Usage with large directories ...

[Andrew Bartlett]

On Fri, 2002-12-20 at 02:03, Green, Paul wrote:
> Scott Taylor [mailto:scottt at methodstudios.com] wrote:
> 
> > We have a samba server running version 2.2.5 on kernel 2.4.18 with the SGI
> XFS patch.
> > The shared volume consists of an XFS partition on a 3-ware raid5
> controller. The
> > network connection is via a 4 port bonded pipe to the switch.
> 
> > We notice that the samba CPU usage during write operations increases
> dramatically
> > once a directory contains more than a certian number of files - thought to
> be
> > somewhere around the 1500 to 2000 mark.
> 
> > We have tried allowing samba more memory, which did not seem to help - and
> have
> > had little or no success finding any information on the web, hence this
> post.
> 
> My guess (and that's all it is) is that this is an operating system issue. I
> presume you are using Linux 2.4.18 although you didn't say.  Try writing a
> small C benchmark program that just does straight fopen/fread/frwrite/fclose
> operations, and time them, and see how you fare.  I'll bet you find that the
> system calls (esp. the open call) take a lot longer on the big directories.
> 
> Make sure your benchmark program uses the same file naming conventions as
> your real code, in case the problem has something to do with the efficiency
> of hashing or searching the specific names.

Samba doesn't like large directories, when it has to work out 'mangled'
file names.  Because the is no way to ask the OS 'give me the file with
this mangled name', we have to ask it for every file in that directory,
mangle it, and check it for a match.  This can certainly get slow on
large directories.  However, the penalty should only be payed on the
open(), not on each write.  (And we do cache etc)

Andrew Bartlett

-- 
Andrew Bartlett                                 abartlet at pcug.org.au
Manager, Authentication Subsystems, Samba Team  abartlet at samba.org
Student Network Administrator, Hawker College   abartlet at hawkerc.net
http://samba.org     http://build.samba.org     http://hawkerc.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.samba.org/archive/samba-technical/attachments/20021229/5ab61715/attachment.bin