[Samba] massive performance problems if transferring many sma ll files

[Simon.Harris at nationwide.co.uk]

-----BEGIN PGP SIGNED MESSAGE-----

********************************************************************
The contents of this email are intended exclusively for the
addressee. If you are not the addressee you must not read
use or disclose the email contents ; you should notify us
immediately [ by clicking "Reply" ] and delete this email.

Nationwide monitors e-mails to ensure its systems operate
effectively and to minimise the risk of viruses.  Whilst it has
taken reasonable steps to scan this email, it does not
accept liability for any virus that may be contained in it.
********************************************************************

Having had similar problems with not much success I would be interested in
the outcome. 

Seems that Solaris with Samba with large number of files in a single
directory is not a good mix! 

Doing a network trace the, time it takes samba to respond goes up
significantly with the more files in that directory. In tests with an
application doing basically a findfirst for every file gave the following
results:
Directory with 3000 files
Reply for 'file found': 16ms
Reply for 'file not found': 47ms
Directory with 9000 files
Reply for 'file found': 31 - 62ms
Reply for 'file not found': 125ms
Directory with ~17000 files
Reply for 'file found': 100 - 200ms
Reply for 'file not found': 250ms

These were rough timings but you can see the trend.

Also, can't find option 'mangle method'.

Cheers,

Simon

- -----Original Message-----
From: steffen.k at gmx.at [mailto:steffen.k at gmx.at] 
Sent: 05 February 2004 10:29
To: abartlet at samba.org
Cc: samba at lists.samba.org
Subject: Re: [Samba] massive performance problems if transferring many small
files

> On Thu, 2004-02-05 at 20:12, steffen.k at gmx.at wrote:
> > Hi,
> > > > If I try to transfer about 27.000 small HTML-Files from my
Windows-PC to
> our
> > central SAMBA-Server I'll get  massive performance problems. First the
> > performance looks ok but after a time period of 3 minutes it goes slow
> do
> wn -
> > appr. one file per second will be transferred. Has anyone an idea what I
> can tune
> > to avoid this annoying behaviour.
> > > - samba version 2.2.8a (64bit compiled)
> > Are all/many of these files in the same directory?

Yes, it's a flat directory hierarchy.

> I would suggest that Samba 3.0 might handle this situation better, or in
> 2.2 set 'mangle method = hash2'.  

I'll try it.... 

Are there any Solaris kernel variables they interfere with this behaviour
and needs to be adjusted?

Regards,
Steffen

- -- To unsubscribe from this list go to the following URL and read the
instructions:  http://lists.samba.org/mailman/listinfo/samba


-----BEGIN PGP SIGNATURE-----
Version: PGPsdk version 1.1.1 (C) 1997 Pretty Good Privacy, Inc.

iQEVAwUBQCKQjm4wJNdvwuyFAQG3Ygf/UECbdjJ89Dlqn/6uvxNmgbgvIZ03QRHY
vMBfqc/WfGyH9xJ/0YDbKr+zDppPeBx4vyDIhCVMxt3khQgceFoT0NwE3JRA5Rqx
m4yZ9ccosMNcWCJgFFyKRUPy2kcH+he/Aooi+UXyISbAluGTEAGpmrrbiEotCuQC
kFyXlCQBxwJfcM8VSgUvBtUXvXay6taEpvQmLSWGG9MMRtOZnRYiDFtpvEOsrerX
wHjwWidLKLyhfEcLFtHu93THfiRDSK59rb8xdko3Rn58Tg9jgjUMbWWgBfRT38nR
TWJp7kd1m4sm/BQcDHt9fU4LiDh6hVi2FJysOR6I5P4gJEWG4Kosfg==
=oqA1
-----END PGP SIGNATURE-----