[Samba] Re: Win2k - samba-2.2.8a - can't get above 4MB/sec
akohlsmith-samba at benshaw.com
Thu Aug 7 15:53:10 GMT 2003
Sorry for the late reply. Do you have any idea why these messages aren't
getting into the mailing list?
> I can't imagine 64k TCP packets. You probably mean the
> window size.
Nope, I mean TCP packets -- TCP packets can be pretty damned large, they
just get fragmented to fit into the ethernet frames, which is exactly what
I'm trying to do. The window sizing can get much larger too but that's
only advantageous for "long thin" networks instead of fast LANs.
> # time tar cbf 64 - . | dd of=/dev/null obs=64k
> 4141517+0 records out # that's 271,418,458,112 bytes
> real 46m49.755s # that's 2,809.755 seconds
> only 96.6 megabytes per second.
That is impressive, but I find it hard to believe that ATA133 can sustain
this in a real test (multiple users accessing different parts of the
filesystem, most likely with many small file accesses). I agree though,
your setup blows mine out of the water for straight raw throughput!
> There's very little scope for tuning CIFS and NBT,
> other than having a faster server and disks. Using
But that's the issue! it's _not_ a disk bottleneck, and it isn't even a bus
> special programs to measure transfer rates by
> creating and then copying files with windows API
> calls, I do get about 9,5 MB/s from most clients but
> the actual real-life average is closer to 3-5 MB/s in
> my experience. One of my samba servers is backup
Sounds like my experience.
> server collecting data from a dozen Windows clients so
> I have lots of data showing that Windows clients are
> unable to exceed about 4 MB/s sending data to the
> backup server, most often going at 2.2-3.5 MB/s. All
> of them W2K with 2-3 GHz processors, new IDE disks and
> plenty of memory. You know what? I always start all
> the backups simultaneously to save time. It takes just
> a little longer than backing up any one of them alone.
If all they can do is 4MB/sec each then naturally doing the backups
concurrently will be advantageous, as the server is just sitting there idle
most of the time, even during network transfers.
> You can't have 64k NBT packets just the same as you
> can't have TCP packets (in which NetBEUI packets are
> encapsulated for TCP/IP transport) longer than about
> 1514 bytes at most. 1460 sounds good to me.
I'm trying to get win2k to fragment NBT packets by making them larger than
what a single ethernet frame can hold. I have to check again but I believe
that samba <--> samba transfers use large packet sizes and have
significantly higher throughputs. Samba's set up to allow up to 64k
packets, but win2k isn't letting it get that high.
I can simulate this with NFS: if I set rsize/wsize to 1460 bytes I get
almost identical throughput, but if I set the rsize/wsize up to 8k can
almost hit 10MB/sec, and tcpdump is showing that the packet size is
changing (and being fragmented to fit into ethernet frames). I'm trying to
do the same for NBT packets.
I'd also tried "raw tcp" -- port 445, no NetBIOS. Same speed, same packet
> You are mixing up the NFS mount options read/write
> block size with packets. These are different things.
If they are different things, then why is tcpdump showing the NFS packets to
be 8k packets, fragmented down to ethernet frame sizes? I understand that
no ethernet frame can be greater than 1500 bytes (in my network anyway) and
thus TCP is fragmenting the larger blocks in order to fit, but the
throughput is significantly higher.
> optimal to set the block size of NFS shares to 1500,
> so go figure.
My benchmarks disagree with this: my optimal throughput is with 8k
rsize/wsize, even though it's being fragmented.
> IMO the best way to speed up your smb traffic is to
> get rid of those lame scuzzies. Try a solid Opteron
> board with 64-bit/66 MHz PCI slots if you're so keen
> on speed. (I am too, but it's still a bit expensive:-)
Again, my SCSI system punks out at 40MB/sec, not 4MB/sec. Going to faster
disk I/O isn't going to help the situation since I'm nowhere near the limit
> On the other hand, try to think out of box. Perhaps
> reducing the packet size to something like 776 or 93
> bytes could do the trick. Who knows, perhaps there's a
> magic number somewhere in the range 41-1460. I've
> never tried but it's sure a lot easier to do than
> increasing the ethernet packet size to 64k.
I'm not trying to increase the ethernet frame size; I'm trying to get Samba
and Win2k to agree to transfer large packets (which get fragmented and
reassembled), just like I can do with NFS.
I will try the smaller NBT packet sizes but I don't think it'll yield much
more. I am under the impression that the bottleneck lies in the read()
call -- reading 1460 bytes at a time is not as effecient as reading 8k or
even 64k at a time and then spoon-feeding it to the ethernet card. I will
try it though and see.
Thanks for your reply. I'm not sure why I'm not seeing these posts in the
mailing list but at least someone got it. :-)
More information about the samba