[Samba] Performance issue on AIX when deleting files in a directory with a large number of files

[Claus Lund]

> >
> > We have an application that creates a large number of files in a single
> > directory. At peak times the number of files in that directory
> get up around
> > 150,000. The files are around 10-200KB in size. When I try to
> delete files
> > out of that directory then things get really slow ... up to 2-3
> seconds per
> > file ... and the CPU utilization on the server goes up near 100%.
> > If I delete files out of that directory directly on the box then it goes
> > very quickly so it seems like this is definitely a Samba issue.
>
> This may be the client rescanning the directory after a delete.
> If the application is braindamaged enough to create 150k files
> in a directory it's stupid enough to rescan also.
>

Well, then Windows Explorer is braindamaged as well (which is a claim I
won't contest ;) )... because it's slow deleting files right through
Explorer as well.
Unfortunately I have no control over the app that's creating all these
files. It's part of a big package used for scanning and doing OCR/hand
processing of paperforms.


> > I tried the suggestions on this page:
> > http://mirrors.uol.com.br/pub/samba/HOWTO/Samba-LargeDirectory-HOWTO
> >
> > But that didn't make any difference either.
> >
> > I also put together a Linux test box (1.5GHz P4, Ubuntu Dapper
> Beta 7, Samba
> > 3.0.22, ext3 file system) and the performance issue is not
> present on that
> > platform.
>
> Hmmm. So the problem doesn't occur on a Linux+ext3 box, but does
> on an AIX+jfs box - with the same version of Samba....
>
> I can see a pointer to the problem here, and it doesn't look like
> Samba to me :-).
>
> Jeremy.


Well, it looks like a Samba problem to me (or maybe a compiler/compiler
optimization problem?). I'm trying to get Samba compiled using IBM's XLC
instead of GCC in the hopes that XLC might produce faster running binaries
... but unfortunately that seems like a somewhat involved task (Samba
doesn't compile cleanly out of the box with XLC).

-Claus