[Samba] 1MB/s gigabit transfers on dell poweredge

[godwin at acp-online.co.uk]

John, thanks once again for the quick reply.

I am using openldap for authentication. The ldap server's on the same
system so theres no overhead added by authenticaion over a network link.

I am using debian etch r5 as the base os. I am not sure of the samba
version right now. I will have to  check that up onsite. But I think the
most probable cause is the samba version. I am also going to try
connecting the client and server directly without a switch in between.

Btw, in the last email I forgot to ask which network switch you are using.
I guess that would make quite a difference on gigabit ethernet. Have you
enabled jumbo frames on the NIC's. Any specific optimisations in the
network setup or in samba conf?

I am eager to understand how you are getting 50MB/s on samba transfers as
considering the overheads added by the samba protocol, you should be
getting 60-75 MB/s using scp/rsync/nfs.

I am also working on a COTS storage solution using lustre and Samba CTDB.
The aim is to provide 1000MB/s to clients (100MB/s to each client) so they
can edit video online. The solution also needs to be scalable in terms of
IO and storage capacity and built out of open source components and COTS
so there is no vendor lock in. Initial tests on lustre using standard dell
desktop hw are very good. However I need samba ctdb to communicate with
the clients as they are Apple macs. I havent reached samba ctdb
configuration yet. But the gigabit ethernet issue had me scared till I
received your reply. Now I see a lot of hope ;-).

Once again thanks for your help and do let me know if I can reciprocate
your kindness.

Cheers,
Godwin Monis


>> Now, if its not asking for too much, can you let me know
>> 1. the network chipsets used on your server and client
>>
> Main servers
> fileserv ~ # lspci | grep Giga
> 02:09.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704
> Gigabit Ethernet (rev 03)
> 02:09.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704
> Gigabit Ethernet (rev 03)
>
> dev6 ~ # lspci | grep Giga
> 02:09.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704
> Gigabit Ethernet (rev 03)
> 02:09.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704
> Gigabit Ethernet (rev 03)
>
> datastore0 ~ # lspci | grep net
> 00:08.0 Bridge: nVidia Corporation MCP55 Ethernet (rev a2)
> 00:09.0 Bridge: nVidia Corporation MCP55 Ethernet (rev a2)
>
> datastore1 ~ # lspci | grep net
> 00:08.0 Bridge: nVidia Corporation MCP55 Ethernet (rev a2)
> 00:09.0 Bridge: nVidia Corporation MCP55 Ethernet (rev a2)
>
> datastore2 ~ # lspci | grep net
> 00:08.0 Bridge: nVidia Corporation MCP55 Ethernet (rev a2)
> 00:09.0 Bridge: nVidia Corporation MCP55 Ethernet (rev a2)
>
> datastore3 ~ # lspci | grep net
> 02:00.0 Ethernet controller: Attansic Technology Corp. L1 Gigabit
> Ethernet Adapter (rev b0)
>
> Other servers:
> ASUS M2N
>
> clients (mostly XP SP3)
> mostly nforce chipset
>
> ASUS M2N / M3N
>
>> 2. which version of samba you are using
> 3.0.34 on all samba servers except 1 where I am testing 3.3.2. I am
> the only person who uses the samba share on that box so I can break it
> without consequences.
>
>> 3. What is your base server os?
> gentoo on all linux machines (30+) except for a few workstations that
> we had to use the vendor's choice.
>
> mostly running 2.6.26 openvz kernels and xfs filesystems for data with
> reiserfs for os. The systems all are using software raid5 or raid6
> with 6 to 10 sata drives in each array. The slowest (10 x 250GB sata1
> raid6 on fileserver) reads at 150MB/s while the fastest on datastore2
> reads at 300MB/s and writes at nearly 200MB/s.
>
>> 4. Is your samba installed from precompiled binaries for the system or
>> have you compiled from source?
> Everything is compiled from source with gentoo.
>
>> 5. You mention that your server is a few years old. However how old is
>> the
>> samba setup? (maybe I need to downgrade to lower version)
>>
> We started in 2000  with samba. I am not sure if that was samba 2 or 3
> using ldap from the start. The samba setup has been upgraded ever
> since. At first we were using redhat (6?) but we moved on to gentoo in
> 2004 and all has been well ever since.
>
>> Thanks a million for the reply.
>>
>
> What are you using for authentication? I have seen this play a role in
> performance specifically I have 3 ldap servers now. They run as guests
> under openvz on a few of our servers.
>
> John
>