[Samba] Samba and Seq. Read ahead.

[William Jojo]

Please don't shoot me for cross posting, but I wanted to share this with
everyone.

Just thought you'd like to know that I've discovered a very interesting
thing about AIX and Samba with respect to Sequential Read Ahead.

I had been tracking a number of issues related to performance on my Samba
2.2.7a server and didn't have any issues other than lengthy profiles when
I was running 4.3.3.

Now that I'm at 5.1, the 32-bit and 64-bit kernel seem to do a *lot* of
thinking about the history of your reading.

When I realized that my cswitch was 10000 and syscalls were 150000 per 2
second vmstat interval, I knew I was doing some serious work, but I always
seemed to have a fair amount of cpu wait.

So I read *many* pages on the IBM site about tuning the VMM and none
seemed to help until after some PMR's with IBM and some personal
investigation.

By defeating the read ahead mechanism with "vmtune -r 0", our throughput
has gone through the roof! Now the only thing we see is Kernel and User
time no Wait at all! And the disks have been happier than ever.

As I see it, we were agressively caching things that may be bumped out of
memory before we ask for it since there are 550 WS's asking for roaming
profiles at login time.

Turns out my reads although sequential on the part of the smbd were really
random in the sense that there was an onslaught of calls to the disks
at one time.

System is 6H1 with 6-RS64-III at 668MHz with 6GB memory. Disks are SSA
36.4 10k in 2 6+P/HS raid-5.

Now it seems that single tuning has allowed Samba to work even better than
it has!

Also the addition of O_DIRECT and/or O_NOCACHE to the fd_open() call in
source/smbd/open.c had no significant increase in performance.

I thought someone might get to use this as well and share success with the
Samba Team not related to a *bug*!

Thanks for listening.

PS: There are further tunings I'm looking into wrt to not allowing Gb
ethernet interrupts to be serviced by CPU0, turns out the Gb card has a
higher int service level than the system timer (so I'm told anyway) and
can result in poor performance under *heavy* load. Anyone interested in
that can look at "bindintcpu".


Bill