Zero-copy patch

[Jeremy Allison]

On Tue, Feb 08, 2011 at 01:53:14AM -0500, Jacky Lam wrote:
> Jeremy,
> 
>         After tracing the kernel code, I come up with a disappointing conclusion.
> 
>         The heavy memory copy are coming from two places in kernel:
> 
>         1. net/core/skbuff.c: linear_to_page()
>         2. fs/splice.c: pipe_to_file()
> 
>         The first one is actually introduced in 2.6.28.6/2.6.29 to fix a data corruption bug when splice from socket to socket. I quote the kernel change log at the end of mail. This one seems don't bother Samba, so I roll back the fix and get a 10% improvement.
> 
>         The second one do memcopy when
>                 *       Destination page already exists in the address space and there
>                 *       are users of it. For that case we have no other option that
>                 *       copying the data. Tough luck.
>         But I am curious who the users are (mistake?). So, I just comment out the memory copy and expect data corruption. I want to check what can I achieve if all these things are probably fixed.
> 
>         Finally, I get the throughput more or less the same as using read()/write(). It doesn't worthwhile to bother all those trouble at all. Maybe the data flow of current implementation of socket splice are inefficient enough indeed.
> 
>         If there is not any other idea, I think I would rather spend time on how to make the patches you send me to work.

Another OEM (who shall remain nameless) mentioned
that changing the kernel scheduler to the deadline
scheduler helps performance (you may already have
done this of course).

Let me know if you haven't and if it helps.

Jeremy.