[RFC] Performance improvements

[Jeremy Allison]

On Fri, Jun 15, 2018 at 09:38:08AM +0200, Swen Schillig via samba-technical wrote:
> Over the past few months I was creating a good few patches with the
> goal to improve the Sambas performance in a clustered environment.
> 
> The patchset is modifying CTDB-code only but please keep reading...
> 
> The result of the changes is a huge reduction of the variation whether
> it is execution time or maximum number of operations at a time and
> an overall performance improvement of >10%.
> 
> Attached are a few graphs showing the results of tests performed on our
> internal cluster with an LDX appliance.
> The first three graphs are the results of one test stressing the
> directory listing performance and the two SMB2_TP graphs were performed
> in a separate test using a 4 cluster node with 3000 client connections
> per node (so in total 12000).
> 
> The improvements are achieved by optimizing the memory management,
> e.g. using memory pools or being a more precise in the memory
> requirement prediction and using a less iterative approach for
> calculation. In addition I tried to re-use allocated memory where
> possible instead of free'ing and re-allocating the same.
> 
> Before I dump the code changes on the community I'd like to know
> if there is any other test I need to perform and if the described
> results are good enough for you to consider such non-functional
> changes.
> 
> Thanks in advance for your comments and support.

Hi Swen,

These graphs look great ! I'd love to see the code please.

Depending on changes and intrusiveness we might want to
split it into micro-commits (in case you haven't already
done so).

Cheers,

	Jeremy.