[linux-cifs-client] CIFS slow on gigabit, doesn't support sockopt=TCP_NODELAY ?

[Timothy Normand Miller]

Here's my "patch" to fs/cifs/connect.c:

        /*
         * Eventually check for other socket options to change from
         *  the default. sock_setsockopt not used because it expects
         *  user space buffer
         */
        socket->sk->sk_rcvtimeo = 7 * HZ;
        socket->sk->sk_sndtimeo = 5 * HZ;

+        rc = kernel_setsockopt(socket, SOL_TCP, TCP_NODELAY, (char
__user *)&val, sizeof(val));
+        printk(KERN_WARNING "CIFS: TCP_NODELAY returns %d\n", rc);
+
+        val = IPTOS_LOWDELAY;
+        rc = kernel_setsockopt(socket, SOL_IP, IP_TOS, (char __user
*)&val, sizeof(val));
+        printk(KERN_WARNING "CIFS: IPTOS_LOWDELAY returns %d\n", rc);


Both calls return zero, possibly suggesting that they succeeded.  When
NODELAY alone, I didn't notice any improvement.  With both, I notice
roughly an 11% speedup, with improved _consistency_ in throughput as
measured by iozone.  This is still about 1/2 the speed I can get
accessing the same share from my Mac.

There are several possibilities that come to mind:

(1) [most probable] I'm doing it wrong.
(2) There's a bug or design defficiency in CIFS that's causing this
(3) The bottleneck(s) in CIFS are somewhere other than the networking
(4) The SMB protocol itself is deficient, and the Mac is using some
other protocol to access the share


The reason I suspected that TCP_NODELAY and IPTOS_LOWDELAY were
involved was that because I enabled those options on the Linux box's
SMB server, 10 megs/sec was the limit I could get between the Mac and
the Linux box over SMB.  When enabling those options, I then became
limited by the disks (typically about 50 megs/sec for writes, faster
for reads from the RAID1).  With the AEBS share, the speed was also
limited to about 10 megs/sec, which I didn't think was a coincidence.
Since the Mac can move more like 20 megs/sec to/from the AEBS (limited
by the fact that these are USB drives), my suspicion is that I simply
haven't done it right.

How can I be sure I'm setting these options correctly and on the right
socket and that it applies to both directions, etc?