[Samba] massive performance problems if transferring
many small files
abartlet at samba.org
Thu Feb 5 20:35:33 GMT 2004
On Thu, 2004-02-05 at 21:34, Steffen Kauka wrote:
> > On Thu, 2004-02-05 at 20:12, steffen.k at gmx.at wrote:
> > > Hi,
> > >
> > > If I try to transfer about 27.000 small HTML-Files from my Windows-PC to
> > our
> > > central SAMBA-Server I'll get massive performance problems. First the
> > > performance looks ok but after a time period of 3 minutes it goes slow
> > do
> > wn -
> > > appr. one file per second will be transferred. Has anyone an idea what I
> > can tune
> > > to avoid this annoying behaviour.
> > > - samba version 2.2.8a (64bit compiled)
> > Are all/many of these files in the same directory?
> Yes, it's a flat directory hierarchy.
Because unix is case sensitive, and windows is not, this is almost a
pathalogical case for Samba.
Samba must scan the *entire* directory, to see if there is a matching
file (of potentially different case), so it can say 'sorry, file by that
name already'. Naturally, this isn't exactly fast as you approach 27000
> > I would suggest that Samba 3.0 might handle this situation better, or in
> > 2.2 set 'mangle method = hash2'.
> I'll try it....
The hash2 method is not only much faster, it has a much lower collision
rate. This helps if for some reason, the 8.3 names are being used.
If you can put the files into a hierarchy, then things will be *much*
better. Or, for this copy only, you might want to turn 'case sensitive
= yes' on in your smb.conf - however the implications of that are nasty
for normal windows operations (and perhaps even the copy, depending on
what you use).
Another quick hack might simply be to zip the files up, and run 'unzip'
on the server.
Andrew Bartlett abartlet at pcug.org.au
Manager, Authentication Subsystems, Samba Team abartlet at samba.org
Student Network Administrator, Hawker College abartlet at hawkerc.net
http://samba.org http://build.samba.org http://hawkerc.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.samba.org/archive/samba/attachments/20040206/216f5e93/attachment.bin
More information about the samba