[Performance] Samba 3 vs. Samba 4 performance in NetBench
marc_kaplan at adaptec.com
Wed Jun 22 22:10:49 GMT 2005
Looks like the NetBench hack did indeed bring the performance of Samba 4
up to near parity with Samba 3 (no oplocks). I am using the exact same
config as last time as far as the servers and clients go, the only
changes to the setup are that I used Samba4-SVN-7804 and I set
posix:allocationrounding = 0x100000 in smb.conf.
Here are the results:
Clients Samba3-nooplocks Samba4-normal Samba4-1M alloc
1 51.150 42.285 43.776
4 203.449 171.702 194.142
8 204.407 170.333 193.482
12 194.383 159.905 181.612
16 189.470 154.434 175.575
20 185.257 150.425 171.574
24 185.442 149.986 170.001
28 184.451 149.125 172.312
32 182.569 150.044 173.536
So right now Samba 4 with the alloc hack is consistently 10Mbit/s slower
than samba3. If you think that a network trace when samba has
posix:allocationrounding = 0x100000 would be insightful let me know and
I'll upload it to samba.org.
Also: I did the win2k3 single client NetBench sniff. You can find it in
/home/mkaplan/win2k3-netbench-1client.cap.rz. It's about 50 some Megs
compressed and 2.1GB uncompressed.
> -----Original Message-----
> From: Andrew Tridgell [mailto:tridge at osdl.org]
> Sent: Monday, June 20, 2005 9:45 PM
> To: Kaplan, Marc
> Cc: samba-technical at lists.samba.org
> Subject: RE: [Performance] Samba 3 vs. Samba 4 performance in NetBench
> Thank you for sending me the sniffs comparing Samba3 and Samba4 in
> netbench. They are certainly interesting!
> The differences I have noticed so far are:
> 1) samba4 negotiates a buffer size 12288 vs 16644 for Samba3
> 2) you have Samba3 setup in share level security, Samba4 in user level
> (probably not significant)
> 3) The test files already exist with samba3, not with samba4. Maybe
> you started the runs in different ways? This shouldn't matter, but
> I'm curious as to how it happened given the way netbench works.
> 4) samba3 reports 1 sector per unit in QUERY_FS_INFO, Samba3 reports 2
> 5) the read sizes between the two runs are about the same, but the
> write sizes are vastly different!
> The last point is probably the key factor. If we look at a histogram
> of write sizes for Samba3 and Samba4 we see:
> 1378 (3%) 4
> 1490 (3%) 4096
> 2239 (4%) 88
> 2452 (5%) 16384
> 2506 (5%) 65536
> 3422 (6%) 2
> 3490 (7%) 65534
> 3778 (7%) 1
> 8190 (15%) 2048
> 11634 (22%) 512
> 946 (1%) 4096
> 1050 (1%) 4
> 1728 (2%) 88
> 1858 (2%) 65536
> 1877 (2%) 16384
> 2666 (3%) 65534
> 2735 (3%) 2
> 6479 (7%) 2048
> 8867 (10%) 512
> 49509 (56%) 1
> I have omitted tail ends of the histograms.
> So this means that for some reason your client is doing a massive
> number (56%) of 1 byte write calls with Samba4, but only a moderate
> number (7%) for Samba3. The trick will be to work out why!
> My guess is this is caused by the extreme rounding of the allocation
> size in Samba3 QUERY_FILE_INFO calls. Samba3 rounds up to the nearest
> 1MB, whereas Samba4 rounds to the nearest 512 bytes. In the sniffs I
> see the client doing long series of 1 byte writes at gaps of about
> 2k. It doesn't write the data in between, it just does things like
> SMB Write AndX Request, FID: 0x0200, 1 byte at offset 4607
> SMB Write AndX Request, FID: 0x0200, 1 byte at offset 8703
> SMB Write AndX Request, FID: 0x0200, 1 byte at offset 12799
> SMB Write AndX Request, FID: 0x0200, 1 byte at offset 16895
> SMB Write AndX Request, FID: 0x0200, 1 byte at offset 20991
> SMB Write AndX Request, FID: 0x0200, 1 byte at offset 25087
> SMB Write AndX Request, FID: 0x0200, 1 byte at offset 29183
> SMB Write AndX Request, FID: 0x0200, 1 byte at offset 33279
> going on for hundreds of operations. I'm guessing the client is trying
> to force allocation of disk blocks. For Samba3 is skips these, as it
> thinks it only has to do one write every 1M to force allocation (in
> fact, Samba3 is lying and it does really need to do a write every 4k
> or so to force allocation, but the client doesn't know that).
> I've now added a tuning parameter in Samba4 for this. You can do:
> posix:allocationrounding = 0x100000
> and it will round to the nearest 1M (same as Samba3).
> If you are using xattrs in Samba4 then it instead uses the allocation
> information as provided by the client, but for the setup you are
> testing (not using xattrs for both Samba3 and Samba4) the above change
> should make things match better. I'll be interested to hear what
> difference it makes. Note you will need svn version 7793 or later to
> get the above option.
> To test its working, try putting a small file on a samba4 share, and
> using the allinfo command in smbclient to see what the allocation size
> is being rounded to.
> All this is really just a benchmark hack btw, but it will be
> interesting to see what effect it has.
> Cheers, Tridge
More information about the samba-technical