[linux-cifs-client] [RFC]

Kirill Kuvaldin kirill.kuvaldin at gmail.com
Sun Aug 19 20:59:10 GMT 2007


Hi,

I tried running oprofile to identify performance bottlenecks of the linux
cifs client. I was actually doing the large read from the samba server to
the client machine over the gigabit ethernet connection. The results
of running oprofile indicated that nearly 70% of samples were attributed
to the cifs_readpages() function:

of 0x00 (Unhalted core cycles) count 60000
warning: could not check that the binary file /cifs has not been
modified since the profile was taken. Results may be inaccurate.
samples  %        symbol name
17690    68.5951  cifs_readpages
1111      4.3080  cifs_demultiplex_thread
850       3.2960  cifs_writepages
768       2.9780  is_valid_oplock_break
747       2.8966  cifs_closedir
464       1.7992  SendReceive2
338       1.3106  sesInfoFree
255       0.9888  DeleteMidQEntry
255       0.9888  allocate_mid
237       0.9190  decode_negTokenInit
212       0.8221  SendReceive
212       0.8221  wait_for_response
184       0.7135  cifs_fsync
180       0.6980  header_assemble
168       0.6514  CIFSSMBRead
161       0.6243  cifs_close
139       0.5390  cifs_NTtimeToUnix
...

Looking further into "opannotate --assembly" results, I noticed that
virtually all sample hits were attributed to
the "rep movsl %ds:(%esi),%es:(%edi)" instruction

               :   129d9:       mov    0x40(%esp),%esi
 17187 66.6447 :   129dd:       rep movsl %ds:(%esi),%es:(%edi)
     4  0.0155 :   129df:       subl   $0x1000,0x44(%esp)


which corresponds to the "memcpy(target, data, PAGE_CACHE_SIZE);" line of
the cifs_copy_cache_pages() function, which was inlined by gcc.

What this seemed to mean was that we're doing memcpy most of the cifs
running time, copying the data from the temporary buffer, allocated from
cifs_demultiplex_thread, to the page cache.

My first thought was that if we managed to avoid doing this unnecessary
copy and read directly from the socket to the page cache, there would be a
performance boost on reads. I tried modifying the cifs source code to
eliminate that memcpy -- in fact, I only commented out the memcpy line of
cifs_copy_cache_pages() just to have an idea how big can be a performance
win if we would eliminate an unnecessary copy without updating actual
pages... Surprisingly enough, I haven't noticed big differences between
unmodified and modified versions (the latter was about 3-5% faster)
on copying a big file on the same setup.

Are the results of oprofile inaccurate in some way or am I missing
something important?
The only instruction (memcpy) taking the most of cifs running time seems
really odd...


        Thanks,
        Kirill


More information about the linux-cifs-client mailing list