[Samba] suggestions for a "fast" fileserver - 1G / 10G
samba at tlinx.org
Sat Mar 22 18:51:08 MDT 2014
Sabuj Pattanayek wrote:
>> E.g. from my Mac Pro I get smb r/w +- 40MB/s, with ftp I get 90MB/s on a
>> 1Gbit Server.
>> So I try to eliminate some bottle necks. But where are they?
>> I know there are some protocol overheads etc. comparing smb and e.g. ftp.
> What samba version are you running ? I'm using the latest (or close to
> latest) samba sernet 4.1.x . I think you're running into protocol
> overheads. I got the same max 40MB/s on reads and writes over a gig
> connection to our server which has 2x10gb LACP bond from a linux client
> running RHEL6/CENTOS6 the kernel for which only supports smb protocol ver=1
Win7 can get close to line speeds on 1Gb HW in benchmarking, but latency
seeking and small r/w sizes of files will bring that down to 10MB (+ or -)
Even with 10Gb, (which is really 8Gb, BTW -- the protocol uses only 8 bits
out of 10 like old modems). So max theoretical in MB is about 768MB/s.
The issue of packet sizes came up in a Tbird bug report...I did
a bench testing effects of TCP I/O size (underlying ethernet is up to
9000 bytes) in SMB ... was a quick & dirty test, but you can see the
trend really fast. I never did bench the 4k size as it would have taken
about near 15 minutes to run by itself....
Speeds are MB/s R, then W(rite) for a given packet size. Same amount is
each test... only variation is the packet size.
These are speeds on a 10Gb dedicated line, but background processes on
were still running:
size Read Write
128M 10.04s, 428 MB/s, 6.58s, 652 MB/s
64M 9.74s, 441 MB/s, 6.68s, 642 MB/s
32M 9.50s, 452 MB/s, 6.79s, 632 MB/s
16M 9.79s, 439 MB/s, 7.11s, 604 MB/s
8M 9.44s, 455 MB/s, 7.36s, 583 MB/s
4M 10.77s, 399 MB/s, 7.57s, 568 MB/s
2M 11.68s, 368 MB/s, 10.20s, 421 MB/s
1M 11.91s, 361 MB/s, 9.45s, 455 MB/s
128K 19.02s, 226 MB/s, 20.75s, 207 MB/s
64K 29.043, 148 MB/s, 33.01s, 130 MB/s
32K 56.46s, 76.1 MB/s, 60.91s, 70.5 MB/s
16K 109.92s, 39.1 MB/s, 110.15s, 39.0 MB/s
8K 198.86s, 21.6 MB/s, 234.77s, 18.3 MB/s
4K would have been maybe around 800 seconds or greater looking at the
Limiting factor is CPU -- smbd is usually peg'ed @ 100% during these tests.
Unfortunately with 1 TCP connection/server connect, the protocol is a
bit of a
bottleneck (not easily distributed to multiple cores).
More information about the samba