[RFC] Performance improvements

[Swen Schillig]

Over the past few months I was creating a good few patches with the
goal to improve the Sambas performance in a clustered environment.

The patchset is modifying CTDB-code only but please keep reading...

The result of the changes is a huge reduction of the variation whether
it is execution time or maximum number of operations at a time and
an overall performance improvement of >10%.

Attached are a few graphs showing the results of tests performed on our
internal cluster with an LDX appliance.
The first three graphs are the results of one test stressing the
directory listing performance and the two SMB2_TP graphs were performed
in a separate test using a 4 cluster node with 3000 client connections
per node (so in total 12000).

The improvements are achieved by optimizing the memory management,
e.g. using memory pools or being a more precise in the memory
requirement prediction and using a less iterative approach for
calculation. In addition I tried to re-use allocated memory where
possible instead of free'ing and re-allocating the same.

Before I dump the code changes on the community I'd like to know
if there is any other test I need to perform and if the described
results are good enough for you to consider such non-functional
changes.

Thanks in advance for your comments and support.

Cheers Swen
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SMB2_Commands.png
Type: image/png
Size: 29576 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20180615/b3b1153d/SMB2_Commands-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SMB2_ResponseTime.png
Type: image/png
Size: 194663 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20180615/b3b1153d/SMB2_ResponseTime-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SMB2_Actions.png
Type: image/png
Size: 151063 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20180615/b3b1153d/SMB2_Actions-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SMB2_TP_Read.png
Type: image/png
Size: 144922 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20180615/b3b1153d/SMB2_TP_Read-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SMB2_TP_Create.png
Type: image/png
Size: 151223 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20180615/b3b1153d/SMB2_TP_Create-0001.png>