[linux-cifs-client] Re: sequential read performance - read ahead ?

[Alexander Indenbaum]

Hello Jeff,

On Tue, Sep 16, 2008 at 8:44 PM, Jeff Layton <jlayton at redhat.com> wrote:
> On Tue, 16 Sep 2008 19:43:43 +0300
> "Alexander Indenbaum" <alexander.indenbaum at gmail.com> wrote:
>
>> Hello James,
>>
>> Thank you for your response! See my comments bellow.
>>
>> On Tue, Sep 16, 2008 at 2:55 AM, James Roper <u3205097 at anu.edu.au> wrote:
>> > Hi Alexander,
>> >
>> > The concept behind read ahead is that you can minimise the penalty caused by the in flight time as the data is traveling between the client and the server, by doing other things while you're waiting, eg by sending more requests. If it's the clients CPU speed that is slowing you down, then it's likely that the client isn't spending much time waiting in comparison to the amount of time it's spending processing, so the in flight time would not be that large a penalty.  So reading ahead probably won't help you that much. Additionally, it would be no trivial task to implement read ahead in the cifs client in a 2.4 kernel, you would have to do your own sequential read detection, caching of read ahead pages between vfs reads, and cleanup of unread pages.
>> >
>>
>> In my test environment I do not see that slow CPU is fully utilized
>> during CIFS transfers. Actually the CPU has a lot of spare cycles. I
>> suppose that we see degradation of sequential read performance on
>> slow CPU because there is no TCP like "sliding window" and
>> request<->response cycle takes longer due to slugish CPU. After all
>> one slow CPU tick takes as long as modern CPU 10 ticks :).  So this
>> way neither CPU nor network link are fully utilized. That's why I
>> still hope "read-ahead" could improve the read throughput. I have
>> almost no experience in Linux VM/VFS so I'm not really sure how to
>> implement file's memory page "pre-fetch". In case I get the
>> "pre-fetched" page from the server that application has not yet issued
>> cifs_readpage() for it. How can I allocate new page and associate with
>> the file, so VM/VFS will not try to fetch it over network again? Any
>> pointer to similar code will be appreciated.
>>
>
> It sounds like you're trying to throw a solution at a problem you don't
> yet fully understand. Nothing wrong with that -- sometimes you get
> lucky, but performance problems are usually tracked down through hard
> investigation. You might want to spend more time tracking down where
> the bottleneck is before you implement a readahead scheme. It's quite
> possible you'll end up with some great readahead code that doesn't help
> at all.

My understanding is that in case of sequential large file read
("streaming")  due to lack of prefetch/read-ahead/sliding window and
serial nature of Request-Reply algorithm  either CPU waits for network
or vice versa so neither of both resources are fully utilized.  In
this particular work load page pre-fetching could provide significant
throughput improvement. I agree with you though that this claims
should be tested by experiment.

>
>> > I'm not sure about other performance improvements, it's been a long time since I've looked at cifs vfs, but there may be some memcopys that can be eliminated, that has the potential to make a significant difference.
>>
>> Thank you for this idea. Please see the attached "very raw and
>> untested" patch against 1.20c. It should save memcpy of data on
>> receive path. Unfortunately, I did not notice any significant
>> performance improvement :)
>>
>
> Yikes! 1.20c is pretty ancient stuff (2004, I think?)
>
> Steve French has some intermittently updated tarballs of newer cifs
> code that is backported to older kernels. I think this is the latest
> one:
>
> ftp://ftp.samba.org/pub/samba/cifs-cvs/cifs-1.50c.tar.gz
>
> ...there are earlier ones in the same directory too.
>
> You may want to try building that for your kernel and seeing if it
> helps before you start trying to implement anything new. I've never
> done a backport of CIFS to a 2.4 kernel though -- it may take some
> significant work (though probably less than implementing your own
> readahead scheme).
>

kernel 2.4 is ancient! Unfortunately 1.20c is the latest snapshot that
compiles on my 2.4 kernel :(
I also tried svn://svnanon.samba.org/linux-cifs-client/branches/linux-2.4
which looks FOOBAR and does not compile.

Backporting is not trivial and in addition I'm not sure that current
kernel 2.6 cifs fs implementation includes page pre-fetch like
feature. My testes demonstrated that in case of streaming 2.4 cifs is
almost equivalent to latest Ubuntu from throughput point of view. So
maybe this feature could be integrated also in current cifs for
instance as special "streaming" mode. Just a thought.

> --
> Jeff Layton <jlayton at redhat.com>
>



-- 
Alexander Indenbaum