[REPORT] Performance Test Samba vs Windows

[William Brown]

As a comment, great work to get this test suite running against samba
and windows! It's awesome to see progress on this, and it will really
help in the future :) 

On Mon, 2018-05-14 at 15:16 +1200, Garming Sam via samba-technical
wrote:
> There's some interesting observations that you can make about the
> overall traffic, without looking at too much detail of the actual
> runs.
> 
> 1) In terms of maximum traffic (which in our examples has been quite
> LDAP based), Windows is at most 50% faster. This number likely
> shrinks
> depending on whether or not the service levels (e.g. 99% success) at
> such high load are acceptable or not.

Great! I think there are lots of ways to improve LDB as well to help
close this gap further. I'll contribute some ideas and patches later
for this. 

> 
> 2) Looking at median packet response times, we generally spend twice
> as
> long on processing a packet. However, the difference between our 95
> percentile values is often much smaller than this and some of the
> mean
> values are actually at parity.
> 
> 3) With small to moderate loads, Samba handles a similar amount of
> operations per second to Windows. For the values shown, it actually
> seems to get more throughput (although not incredibly so) for a good
> chunk of the table shown.

What about "large" loads? 

> 
> It's actually kind of surprising that Samba doesn't perform all that
> much worse than Windows does. It's certainly not off by an order of
> magnitude (or two), although this testing has been run in the prefork
> mode. Our previous analyses showed much smaller numbers and that's
> likely due to the limitations of being stuck with a single-process.

> Cheers,
> 
> Garming
> 
> On 14/05/18 14:45, joeg at catalyst.net.nz wrote:
> > 
> > Hi Team,
> > 
> > I've been trying to make the Samba performance test tool work
> > against
> > Windows.
> > 
> > After a lot of effort, Garming and I fixed all the errors for
> > different packets, and finally it's working(fixes are merged into
> > master).
> > 
> > Base on that, we can send traffic to Samba and Windows, get the
> > maximum load, and compare the performance.
> > 
> > An example of the command I was using:
> > 
> > 
> > script/traffic_replay -U Administrator%PASSWORD  --realm
> > krb.samba.site --workgroup KRB --fixed-password FIXED-PASSWORD -r 4
> > -S
> > 4 traffic-sample-1-model.txt dc1.krb.samba.site 2>&1 >
> > traffic_replay_stats/dc1_r4_S4.txt
> > 
> > 
> > By changing the combination of different -r and -S, we can get test
> > results and save to files, then parse files to get summary tables
> > as
> > below.
> > 
> > The number displayed is "*Successful operations per s**econd*", the
> > 0
> > ones are combinations we think not important and skipped.
> > 
> > 
> >       Test Result for Samba (master code on 2018.05.11):
> > 
> > 
> > 
> >       Test Result for Windows Server 2012r2:
> > 
> > 
> > 
> > 
> > I am running the test with 4 servers in Catalyst Cloud which is
> > Openstack:
> > 
> > 
> > 
> > All data are attached, hope this can help the team to understand
> > how
> > is Samba performing with compare to Windows.
> > -- 
> > Joe Guo
> > joeg at catalyst.net.nz
> > Catalyst IT
> 
>