[Samba] Windows SMB2 client doing excessive, inefficient SMB2 Find (and other) requests

[awl1]

Hello Ralph,

Am 21.09.2017 um 18:38 schrieb Ralph Böhme via samba-technical:
> Andreas,
>
> On Mon, Sep 18, 2017 at 04:02:43PM +0000, awl1 wrote:
>> Am 18.09.2017 um 01:55 schrieb Ralph Böhme:
>>> I just added an event to my calendar for Tuesday to remind me of
>>> talking about this with MS engineers while attending an MS interop event this
>>> week.
>> that's great!
> ECANTREPRODUCE...
>
> I'm copying 100 files (actually selecting the 100 files for dragn'drop) from a
> Windows 10 client to a Win 2016 server or a Samba server. Both traces show that
> the client is *not* enumerating the destination directory for every file.
>
> If you like to take a look, here's the trace:
> <https://www.samba.org/~slow/copy100files-win10-2-samba.pcapng>
hmm, I think it's rather 
"EREPRODUCINGPARTLYANDWONDERINGWHATMIGHTBEDIFFERENT", and we need to 
find out why:

If I read your trace file correctly, it does reproduce the issue (though 
I admit that your traces, particularly the service response times, look 
much better than in my traces).

When you filter your trace with "smb2.cmd == 14", your trace shows no 
less 198 Find requests of type ("Info Level") 37: 
"SMB2_FIND_ID_BOTH_DIRECTORY_INFO" with Search Pattern: "*".

1) As you have copied only 100 (or most probably, 99) files from the 
client to the target share, why do we see 198 (i.e. two times the number 
of files) Find requests? This successfully reproduces my finding, and I 
assume 100 Find requests would besufficient to copy 100 files.

2) Why is the Find request of type ("info level") 
SMB2_FIND_ID_BOTH_DIRECTORY_INFO with Pattern "*"? This reproduces my 
finding, and is completely unnecessary. While it might be needed to 
check that a file with the same name (case [in]sensitive depending on 
Samba parameters) is not already existing, it clearly is not necessary 
to enumerate all files in the current directory, which is what Pattern 
"*" would be expected to do (and does in my testing).

3) I am wondering why in your testing, the Find response to 
SMB2_FIND_ID_BOTH_DIRECTORY_INFO with Pattern "*" seemingly always stays 
the same in all 198 responses:

Find Response (0x0e)
     [Info Level: SMB2_FIND_ID_BOTH_DIRECTORY_INFO (37)]
     StructureSize: 0x0009
     Info: 7000000000000000a35363ccf432d30130b364ccf432d301...
         Offset: 0x00000048
         Length: 334
         FileIdBothDirectoryInfo: .
         FileIdBothDirectoryInfo: ..
         FileIdBothDirectoryInfo: dir

The difference between your and my testing here is as follows:

When writing the 100 files to the share, why don't they end up in 
subsequent Find responses?

I would expect to see your files (dir\file1, ..., dir\file100) to end up 
in the Find response packets as FileIdBothDirectoryInfo as well.

In my testing as done so far, this rather looks like the following:

Find Response (0x0e)
     [Info Level: SMB2_FIND_ID_BOTH_DIRECTORY_INFO (37)]
     StructureSize: 0x0009
     Info: 7000000000000000a0f5d991fe1bd3010070d1ff911bd301...
         Offset: 0x00000048
         Length: 1120
         FileIdBothDirectoryInfo: .
         FileIdBothDirectoryInfo: ..
         FileIdBothDirectoryInfo: file0001.rnd
         FileIdBothDirectoryInfo: file0002.rnd
(...)
         FileIdBothDirectoryInfo: file<n>.rnd

i.e. the size of the Find Response grows with every single file 
successfully copied onto the share, and the current Find Response always 
contains the names of all the n (with n running between 0 and 2000) 
files that have been successully copied to the share so far.

This results in a much larger trace file: In my testing, the trace file 
size for copying 2000 files from a Win10 machine with the buggy client 
to a Samba server is ~ 31 MB and contains no less than 2627 Find 
requests, while using a Linux client in the exact same scenario to copy 
those 2000 files onto the same share, the session trace file is only ~ 6 
MB in size and it does not contain a single Find request (!!!).

Please find and compare my two pcapng trace files here:
http://home.mnet-online.de/awl1/win10_write_smb3_to_samba.pcapng
http://home.mnet-online.de/awl1/linux_write_smb3_to_samba.pcapng

*** First and foremost, we need to find out why here, your testing from 
the Windows client does indeed show different results from my testing. ***

I really have no clue why in your SMB2_FIND_ID_BOTH_DIRECTORY_INFO 
responses, the files just written to the share will not be 
found/returned (I think they really should!). Do you have an idea why 
this would be the case?
Can you try putting all your n (whether 100 or 2000) files into an extra 
directory and then copy this whole directory to the share with Windows 
explorer rather than explicitly selecting 100 files (which is an 
uncommon scenario when copying larger file trees)?

Which Windows 10 client build and which Samba version did you use? My 
Win 10 was using most recent updates (as of August 23rd), and Samba 
version on my Thecus NAS was 4.6.5 (compiled by myself). (Don't know 
whether it matters, but please note that in this my preliminary testing 
as done on Aug 23rd, the test files still had non-zero length and random 
content with a length of between 1 and 2048 bytes.)

I am still fighting some health issues, but should hopefully be able to 
complete recording of all the test cases that I promised to do over this 
coming weekend. I will upload the pcapng traces to my home directory as 
above and then report back here...

Many thanks so far & best regards
Andreas