[Samba] Performance issue on AIX when deleting files in a directory with a large number of files

[William Jojo]

----- Original Message ----- 
From: "Jeremy Allison" <jra at samba.org>
To: "James Peach" <jorgar at gmail.com>
Cc: "Jeremy Allison" <jra at samba.org>; "Claus Lund" <clund at tax.state.vt.us>;
<samba at lists.samba.org>; <jojowil at samba.org>
Sent: Thursday, May 11, 2006 11:30 AM
Subject: Re: [Samba] Performance issue on AIX when deleting files in a
directory with a large number of files


> On Fri, May 12, 2006 at 12:50:37AM +1000, James Peach wrote:
> > On 5/11/06, Jeremy Allison <jra at samba.org> wrote:
> > >On Wed, May 10, 2006 at 12:29:53PM -0400, Claus Lund wrote:
> > >>
> > >> Well, then Windows Explorer is braindamaged as well (which is a claim
I
> > >> won't contest ;) )... because it's slow deleting files right through
> > >> Explorer as well.
> > >
> > >Yep - I completely agree Explorer is braindamaged. That's probably
> > >the worst of the lot :-).
> > >
> > >> Well, it looks like a Samba problem to me (or maybe a
compiler/compiler
> > >> optimization problem?). I'm trying to get Samba compiled using IBM's
XLC
> > >> instead of GCC in the hopes that XLC might produce faster running
> > >binaries
> > >> ... but unfortunately that seems like a somewhat involved task (Samba
> > >> doesn't compile cleanly out of the box with XLC).
> > >
> > >If it only happens on AIX, it's not a Samba code logic problem, that's
> > >what I mean. I doubt modifying compiler flags will do anything about
> > >actions taking seconds to resolve - that's almost certainly a system
> > >problem (ie. JFS on AIX).
> >
> > sounds like samba is recreating the directory hash for change notifies
>
> It only does that every 30 seconds or so - so I don't think that is
> the problem (although turning if off might prove me wrong).
>

I'm looking into this here. Are you referring to "stat cache"? I've
confirmed that 150k statx's are being done after each delete request. I
mocked it up here on JFS2 and JFS. I'm mocking is up now in FC3/4 on ext3 to
see if this is normal and just FS throughput issue on AIX.

What's interesting is the client requests the initial list of files and you
can see the exchange between Samba/client:

statx("bigdir/file61702", 0x2FF20618, 128, 010) = 0
statx("bigdir/file61703", 0x2FF20618, 128, 010) = 0
kwrite(26, "\0\0 @18FF S M B 2\0\0\0".., 16412) = 16412
_select(27, 0x2FF209E8, 0x00000000, 0x00000000, 0x2FF229E8) = 1
kread(26, "\0\0\0 d", 4)                        = 4
kread(26, "FF S M B 2\0\0\0\01807D8".., 100)    = 100
statx("bigdir/file61704", 0x2FF20618, 128, 010) = 0
statx("bigdir/file61705", 0x2FF20618, 128, 010) = 0
statx("bigdir/file61706", 0x2FF20618, 128, 010) = 0


But, between deletes:

statx("bigdir/file74272", 0x2FF22290, 128, 010) = 0
statx("bigdir/file74273", 0x2FF22290, 128, 010) = 0
getdirent64(21, 0x20184A28, 4096)               = 4096
statx("bigdir/file74274", 0x2FF22290, 128, 010) = 0
statx("bigdir/file74275", 0x2FF22290, 128, 010) = 0
statx("bigdir/file74276", 0x2FF22290, 128, 010) = 0


I haven't verified this with ethereal yet, but I don't think the client is
asking for an update while the files are still being processed...


Cheers,

Bill

> Jeremy.
>