[Samba] massive performance problems if transferring many small files

[Andrew Bartlett]

On Thu, 2004-02-05 at 21:34, Steffen Kauka wrote:
> > On Thu, 2004-02-05 at 20:12, steffen.k at gmx.at wrote:
> > > Hi,
> > > 
> > > If I try to transfer about 27.000 small HTML-Files from my Windows-PC to
> > our
> > > central SAMBA-Server I'll get  massive performance problems. First the
> > > performance looks ok but after a time period of 3 minutes it goes slow
> > do
> > wn -
> > > appr. one file per second will be transferred. Has anyone an idea what I
> > can tune
> > > to avoid this annoying behaviour.
> > 
> > > - samba version 2.2.8a (64bit compiled)
> > 
> > Are all/many of these files in the same directory?
> 
> Yes, it's a flat directory hierarchy.

Because unix is case sensitive, and windows is not, this is almost a
pathalogical case for Samba.

Samba must scan the *entire* directory, to see if there is a matching
file (of potentially different case), so it can say 'sorry, file by that
name already'.  Naturally, this isn't exactly fast as you approach 27000
files...

> > I would suggest that Samba 3.0 might handle this situation better, or in
> > 2.2 set 'mangle method = hash2'.  
> 
> I'll try it.... 

The hash2 method is not only much faster, it has a much lower collision
rate.  This helps if for some reason, the 8.3 names are being used.

If you can put the files into a hierarchy, then things will be *much*
better.  Or, for this copy only, you might want to turn 'case sensitive
= yes' on in your smb.conf - however the implications of that are nasty
for normal windows operations (and perhaps even the copy, depending on
what you use).

Another quick hack might simply be to zip the files up, and run 'unzip'
on the server.  

Andrew Bartlett

-- 
Andrew Bartlett                                 abartlet at pcug.org.au
Manager, Authentication Subsystems, Samba Team  abartlet at samba.org
Student Network Administrator, Hawker College   abartlet at hawkerc.net
http://samba.org     http://build.samba.org     http://hawkerc.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.samba.org/archive/samba/attachments/20040206/216f5e93/attachment.bin