[Samba] Client link utilization

[Stan Hoeppner]

Jeremy Allison put forth on 2/6/2010 11:07 PM:
> On Sat, Feb 06, 2010 at 09:26:32PM -0600, Stan Hoeppner wrote:
>> Bostjan Skufca put forth on 2/6/2010 6:14 PM:
>>> Hello everybody!
>>>
>>> This is probably going to be a classic question but I cannot find a
>>> decent answer on net.
>>>
>>> I have samba server set up and the following things work flawlessly:
>>> - iperf shows 92% link utilization
>>> - FTP/SCP/HTTP transfers work in 10MB/s range.
>>>
>>> However, when I mount samba share with linux client (mount.cifs) the
>>> link utilization cannot bypass cca 33%. Transfer speeds constantly
>>> stops around 3.8MB/s and will not rise above it no matter what socket
>>> and locking options I use.
>>>
>>> Do you have any ideas about why this is happening and/or FAQ websites
>>> to point me to?
>>
>> I've had a similar thread running for a few weeks without resolution.  In my
>> case I can max the wire (100FDX) at 92Mb/s to/from Win2K and WinXP clients using
>> FTP, and smbclient from the server to shares on the workstations maxes the wire
>> (at least GET from the workstations does).  I'm running Samba 3.2.5 on Debian
>> Lenny with custom kernel 2.6.31.1.
>>
>> The max smb performance I can get in a single stream to/from smbd is 65Mb/s, or
>> 8.5MB/s.  I've now tested Win2K, WinXP, and smbclient on SLED 10 (can't recall
>> version).  In all cases, no matter what performance settings I tweak in smb.conf
>> or on the workstations, I can't get wire speed with a single SMB stream---can't
>> get over 65Mb/s.
>>
>> Interestingly, two simultaneous SMB transfer streams (two Windows Explorer file
>> copy operations on the same workstation) will max the wire at 92Mb/s, or 11MB/s.
> 
> Hang on a minute, I haven't been paying attention to these emails
> as yet.


Thanks for jumping in Jeremy.

> Am I correct in saying:

Partially.  Before I forget, let me state I'm only speaking for my case, not the
other OP with a similar issue, whose thread this is.

> smbclient -> smbd maxes the wire.

This is not correct.  SLED 10's smbclient (2006'ish, not sure of version) gets
2/3 wire speed to/from the smbd server, Debian Lenny Samba 3.2.5.  In this case,
it's seeing exactly the same single stream performance as the Windows clients.

> smbclient -> WinXP maxes the wire.

Yes, smbclient 3.2.5 on the Debian Lenny smbd server can GET from WinXP and
Win2K at wire speed, however, PUT ops are half wire speed, 6MB/s.  I haven't
attempted to troubleshoot this half speed issue yet.

> But WinXP -> smbd gets 2/3 of the wire speed.

Correct.  Win2K and WinXP both achieve 2/3 wire speed to/from smbd.

> And WinXP+WinXP (two streams) -> smbd maxes the wire.

Correct, but mostly tested from a Win2K machine, not XP.  Just to be clear, this
is two streams from one Windows host.  Two concurrent file copy ops from two
shares on the smbd server, and this maxes the wire every time.

> If this is the case, it's the 64k per read/write
> limit plus only one outstanding packet per stream
> issue with the WinXP redirector that's the issue.

Hmm.  If this is the case, why do the Win hosts nearly max the wire talking smb
to one another, at ~10.5MB/s, and max the wire serving files to smbclient on the
Linux server host?

> smbclient sends up to maxmux outstanding packets
> on read/write and keeps the pipeline full. That's
> why it can max the wire.

Well, smbclient 3.2.5 maxes the wire GETing from the Windows hots' shares, but
only hits 1/2 wire speed on PUT ops to the Windows shares.  SLED 10 smbclient
only reaches 2/3 wire speed <-> smbd on the Debian Samba server, although I
didn't test SLED 10 smbclient against the Windows hosts.

> The WinXP redirector is just not very good I'm
> afraid.

That may be true.  But I'd be remiss if I didn't point out that XP is one of two
machines I have which can actually serve single stream smb at wire speed.  The
other, sadly, is Win2K.  Both can serve smb to smbclient running on the Debian
smbd server at wire speed, and just a shade under wire speed to one another.

The only piece of smb software I have in my normal environment that hasn't
demonstrated single stream wire speed capability is, sadly, smbd.

Thanks for taking interest Jeremy.  Hopefully you can point me in the right
direction so I can figure this out.  It's probably something simple or stupid,
but hidden or obscure.  I'm running a custom kernel, so I guess it's possible I
screwed something up in my kernel config.  I've got netfilter in the kernel but
no current iptables rules configured.  I just don't know where to look next.

-- 
Stan