[Samba] ArcView + Samba: Performance nightmare under Linux,ok
under Solaris or HP-UX
Andreas Haumer
andreas at xss.co.at
Wed Jul 19 14:00:00 GMT 2006
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi!
It's quite a few days since my last message to the Samba mailing
list and there were no useful replies to my previous messages since.
I took me several days of hard work but it seems now I have found
the reason for the problem and so I'll reply to my own message just
in case someone else has a similar problem and searches the archives
for a solution...
Executive summary: kernel oplocks have to be considered evil...
Andreas Haumer schrieb:
[...]
>
> The main question IMHO still ist: why does the application
> request the files differently when changing the Samba server
> operating system (and the Samba servers itself are configured
> identically)?
During the past week I did a lot of tests with different
samba versions and different server operating systems, just
to get a larger picture of what was going on.
For all tests I used the following stripped down Samba configuration
file:
<smb.conf>
[global]
workgroup = WG_TECH
log level = 0
security = user
encrypt passwords = yes
socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192
[testav]
path = /work/samba/testav
</smb.conf>
With this configuration file I can perfectly reproduce
the problem we originally had on the production server
(which uses a much more complicated configuration file
with lots of samba shares, LDAP account management, PDC
settings and so on)
I did compile all Samba versions to be tested from source
using the following simple procedure:
cd /work
tar xvzf samba-x.y.z.tar.gz
cd samba-x.y.z/source
./configure --prefix=/usr/local/samba-x.y.z
make install
I did run all Samba versions to be tested using the following
procedure:
cd /usr/local/samba-x.y.z
cp /tmp/smb.conf ./lib
./sbin/smbd -D
./sbin/nmbd -D
./bin/smbpasswd -a testuser
The client system was the same for all tests (WinXP + ArcView),
as well as the data files in /work/samba/testav
Here are the results from my tests:
Samba-1.9.18p10
Solaris 10: slow
BLD-5.3: slow
SUSE-9.3: slow
Samba-2.0.10
Solaris 10: slow
BLD-5.3: slow
SUSE-9.3: slow
Samba-2.2.0
Solaris 10: fast
BLD-5.3: fast
SUSE-9.3: fast
Samba-2.2.8a
Solaris 10: does not compile
BLD-5.3: fast
SUSE-9.3: fast
Samba-2.2.12
Solaris 10: does not compile
BLD-5.3: fast
SUSE-9.3: fast
Samba-3.0.0
Solaris 10: did not test
BLD-5.3: slow
SUSE-9.3: slow
Samba-3.0.11
Solaris 10: fast
BLD-5.3: did not test
SUSE-9.3: did not test
Samba-3.0.12 (SuSE 9.3 original)
Solaris 10: did not test
BLD-5.3: did not test
SUSE-9.3: slow
Samba-3.0.22
Solaris 10: fast
BLD-5.3: slow
SUSE-9.3: slow
Samba-3.0.23
Solaris 10: fast
BLD-5.3: slow
SUSE-9.3: slow
(Note: BLD-5.3 is a Linux system using a 2.4.32 kernel)
Summary:
All Samba versions before 2.2.0 were slow on all operating systems.
All Samba versions 2.2.x were fast on all operating systems
All Samba versions 3.x were fast on Solaris but slow on Linux systems.
I also noticed that on all tests which performend fast, the client
had LEVEL2 Oplocks on the "*.adf" files (these are the files which
the clients reads in a very ineffective way when the tests are slow).
On tests which performed slow, there were no oplocks on those files.
This pattern made me check the releasenotes and other documentation
as well as the sources (using diff). I found that "kernel oplocks"
and "level2 oplocks" interact in a problematic way!
The smb.conf(5) manual page says this about "level2 oplocks":
<quote>
Currently, if kernel oplocks are supported then level2 oplocks are not
granted (even if this parameter is set to yes). Note also, the oplocks
parameter must be set to yes on this share in order for this parameter
to have any effect.
</quote>
This is also evident if one looks into the sources of smbd/oplock.c,
function process_oplock_break_message() (for Samba-3.0.23)
<code>
if ((global_client_caps & CAP_LEVEL_II_OPLOCKS) &&
!(msg.op_type & FORCE_OPLOCK_BREAK_TO_NONE) &&
!koplocks && /* NOTE: we force levelII off for kernel oplocks -
* this will change when it is supported */
lp_level2_oplocks(SNUM(fsp->conn))) {
break_to_level2 = True;
}
</code>
Global variable "koplocks" get's initialized in function
init_oplocks() if configuration parameter "kernel oplocks"
is set to "yes" (the default) and is NULL for all operating
systems which do not support kernel oplocks but contains some
valid address for the others.
So this is the point: Linux does support kernel oplocks by
default and so level2 oplocks automatically get disabled
by the current (3.x) Samba sources.
Solaris does not support kernel oplocks at all, so level2
oplocks are enabled here.
The ArcView application seems to have problems with this
and falls back to a really insane way of reading a file
if level2 oplocks are not available: using a very small
blocksize and performing repeated reads of the same blocks
scattered all over the file.
If I do instead disable kernel oplocks by setting
kernel oplocks = no
in the global section of smb.conf and enable level2 oplocks
by setting
oplocks = yes
level2 oplocks = yes
in the share section, resulting in this minimal Samba
configuration file:
<smb.conf>
[global]
workgroup = WG_TECH
log level = 0
security = user
encrypt passwords = yes
socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192
kernel oplocks = no
[testav]
oplocks = yes
level2 oplocks = yes
path = /work/samba/testav
</smb.conf>
even the Linux Samba server works fast! We get a reduction
of application execution time by a factor of more than 10
compared to the situation with kernel oplocks enabled.
The Linux server is now even about 25% faster than the
Solaris server as it has better hardware.
Problem solved, call closed, sysadmin happy... ;-)
Any comments?
- - andreas
- --
Andreas Haumer | mailto:andreas at xss.co.at
*x Software + Systeme | http://www.xss.co.at/
Karmarschgasse 51/2/20 | Tel: +43-1-6060114-0
A-1100 Vienna, Austria | Fax: +43-1-6060114-71
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFEvjrdxJmyeGcXPhERAqZFAJ4vBtX59Bo+BhPdLZ7n3oQ4p61A0gCdGaEz
uY7ll1tJJrZRzEwRky0HL2g=
=bwmU
-----END PGP SIGNATURE-----
More information about the samba
mailing list