[Samba] Samba 4 + Windows XP very slow - especially noticeable with many files

[Bram Matthys]

Hi,

After our upgrade from Samba 3.x to 4.1.6 we are having major 
performance issues. Today, I've been able to nail it down to a simple 
test case.
What strikes me is that Windows XP takes about 25 seconds for one job, 
while it only costs 3 seconds on Windows 7.

I have two tests:

1000 files of 10k each (total ~10Mb):
* Linux on localhost or a remote host (1GE or 10GE): 5.5 seconds
* Windows XP - Machine #1: 24.5 seconds
* Windows XP - Machine #2: 60 sec
* Windows 7 - Identical hardware as XP machine #1: 3 seconds

One 500MB file:
* Expected result: 4-5 seconds on 1Gbit (100-125Mbyte/s)
* Linux on localhost: 0.5 sec
* Other Linux machine with 10GE connect: 0.9 sec
* Remote Linux machine (1Gbit): 4.6 sec
* Windows XP - Machine #1: 10.4 sec
* Windows XP - Machine #2: 11 sec
* Windows 7 - Identical hardware as XP machine #1: 7 sec

The setup is a Samba 4.1.6 server on a high school serving 200 - 250 
computers, (at the moment) all Windows XP clients. Server runs 
virtualized in a KVM and has a 10GE connection to the switch.
Traffic stats: OUT: 110Mbit peak (5m avg) and 30Mbit average
Traffic stats: IN: 20Mbit peak (5m avg) and <5Mbit average

The slower-than-usual-speed for the 500M file I can live with for now, 
but the "many small files" case is a real problem. User profiles and 
desktops are full of small files so this causes a slow login. In fact, 
when like 30 computers log in at the same time it becomes so slow it 
takes many minutes for just a 10Mb profile (some even freezing forever), 
students can't start their lesson, etc... it's quite a problem.
Previously, with the same Windows XP clients the performance on the 
Samba 3.x server was good. Users were able to log in quickly. We had no 
issues whatsoever.

The files are served from Samba 4.1.6 on a share which points to 
'tmpfs' storage (ramdisk).
All clients in this test have ssd's. In case of the 500M file I copied 
the file to 'NUL' to avoid storage altogether.
Test files .tar.gz can be found on 
http://www.vulnscan.org/tmp/samba-testcase-slow-files.tar.gz
Test code I used was:
echo.|time
copy \\green\xyz\500M nul
echo.|time
And for the small files case:
echo.|time
xcopy \\green\ramdisk\10k-1000x /S/E/K/H c:\local\10k-1000x\ >nul
echo.|time
Explorer gives very similar timings by the way. I repeated the tests on 
all clients multiple times.
I also tried the "100 files in 10 directories" test case to see if 
maybe the problem was many files in one directory, this made no 
difference so was ruled out.

I can provide log files on request. Sorry, I didn't want to post them 
in public. I have log level 3 ones available plus packet dumps. If 
that's not sufficient I can re-run the test case on a higher log level, 
just let me know.

I already saw a number of 'Samba 4 is slow' threads here, didn't see 
much of a resolution, though. Also makes me wonder if these are also XP.

Finally, here's my smb.conf:
[global]
         workgroup = XXXXXXXX
         realm = jnet.xxxxxxxxxxxx.nl
         netbios name = GREEN
         server role = active directory domain controller
         idmap_ldb:use rfc2307 = yes
         dns forwarder = 8.8.8.8
         netbios aliases = STATLER ATHENA THORTON
         interfaces = jnet
         logon script = connectu.bat
         log file = /usr/local/samba/var/log/samba.log
         log level = 1
         max log size = 250000
         debug prefix timestamp = yes
         debug uid = yes
         reset on zero vc = yes
#       socket options = IPTOS_LOWDELAY TCP_NODELAY
         socket options = IPTOS_LOWDELAY TCP_NODELAY SO_KEEPALIVE 
TCP_KEEPIDLE=120 TCP_KEEPINTVL=10 TCP_KEEPCNT=5 SO_SNDBUF=65536 
SO_RCVBUF=65536
         kernel oplocks = no
         kernel share modes = no
         mangling method         = hash2
         mangle prefix           = 6
         lanman auth             = yes
         time server = yes
         admin users = admin bram root xyz fog
         guest account = nobody
         map to guest            = Never
         # ON OR OFF??? (default = no)
         winbind enum users = yes
         winbind enum groups = yes
         wins support            = yes
         deadtime = 20
         # Make wide symlinks work
         unix extensions = no
         wide links = yes

[ramdisk]
         comment = "Ramdisk - for testing"
         read only = no
         browseable = no
         public = no
         guest ok = no
         valid users = bram, xyz, zzz, admin
         path = /data/ramdisk


I already tried a number of configuration changes, like not having any 
socket options, or only LOWDELAY NODELAY, etc.. no difference (on XP at 
least).
Also commented out various options to see if that made a difference... 
not that I could find.

Hopefully someone can help us out here. We've always been proud to use 
Samba at our school instead of Windows servers, but right now it's not 
really showing the best of it ;).

Thanks in advance, and like I said.. if I need to provide anything.. 
let me know,

Bram Matthys.