[Samba] Huge number of small files performance regression from 3.5.16 to 4.6.5 with identical smb.conf

[awl1]

Hello again, Jeremy,

first of all, I am terribly sorry for my late reply. I tried to send my 
posting many times, but my mail has always been silently discarded by 
the Samba mail servers due to my main mail provider (GMX - a very large 
German mail provider with millions of customers) having been blacklisted 
by SORBS.

For the time being, SORBS is still unwilling to delist them for unknown 
reasons (which I consider a clear malpractice by SORBS, as GMX has 
sophisticated spam/abuse management in place), so I had to switch to 
another mail provider just in order to be able to post again on the 
Samba list... :-(


Am 13.06.2017 um 19:00 schrieb awl1:
> Am 13.06.2017 um 18:36 schrieb Jeremy Allison:
>> Can you get comparitive wireshark traces for the two cases ?
>>
>> That would help discover what the bottleneck is.
> I am not at all a network guy, but I hope that - maybe with a little 
> more help from your part once I have tried to do so in practice - I 
> should be able to do so...

OK, so it looks like I have been able to successfully produce Wireshark 
capture files for the four scenarios... :-)

As I am almost certain that these packet captures will contain at least 
some sensitive information from my environment - such as e.g. user, 
share and machine names, IP addresses (possibly in the old SMB dialect 
1.5 even the clear-text password for the share?) - I will only send the 
link to the captures ZIP file stored in my cloud space to you via 
private mail. So please keep the packet dumps confidential, and only 
share them with other Samba developers after getting my explicit consent!

The ZIP file contains four Wireshark captures for the two scenarios 
(write to and read from share) and the two Samba/SMB versions 
(4.6.5/SMB2/dialect 3.1.1 and 3.5.16/SMB/dialect 1.5) in "pcapng" format:

  * smb311_write - Win10 client writing to Samba 4.6.5 using SMB2
    protocol (dialect 3.1.1), copying ~ 1000 files from local hard disk
    onto the share, documenting the issue with very slow throughput of
    below 10 kB/sec (especially in the range of file 300-600, most
    interestingly throughput improved again after some time)
  * smb15_write - Win10 client writing to Samba 3.5.16 using SMB
    protocol (dialect 1.5), copying ~ 1000 files from local hard disk
    onto the share, with much better throughput than in smb311_write

  * smb311_read - Win10 client reading from Samba 4.6.5 using SMB2
    protocol (dialect 3.1.1), copying ~ 2000 files from the share to
    local hard disk, with acceptable throughput, but consistently slower
    than in smb15_read
  * smb15_read - Win10 client reading from Samba 3.5.16 using SMB
    protocol (dialect 1.5), copying ~ 2000 files from the share to local
    hard disk, with consistently better throughput than in smb311_read

Fingers crossed that you will be able to determine why 4.6.5 is slower 
in both scenarios, and especially so much slower when writing to the 
share (smb311_write) and one more time, thanks a million for digging 
into these packet dumps...

Best regards
Andreas