ctdb: Adding memory pool for queue callback

[Swen Schillig]

On Wed, 2018-11-07 at 15:24 +0100, Volker Lendecke wrote:
> On Wed, Nov 07, 2018 at 03:16:14PM +0100, Swen Schillig wrote:
> > Sure, I can investigate.
> > But whatever the results may be, we would change the way
> > talloc_pool
> > works and not replace talloc_pool by malloc throwing the talloc API
> > over bord, right ?
> > 
> > Therefore, I think the suggested change is still valid.
> > 
> > Wouldn't you agree ?
> 
> No.
> 
> It adds one level of complexity that (according to the current
> measurement) gains probably nothing. In the pure torture test doing
> nothing but the talloc operations that you are optimizing, it gains
> roughly 2%. I would be extremely surprised if you could measure that
> in the real world test with ctdb and all the other stuff it's doing.
> It took me an hour to convince myself that the talloc_pool
> implementation does the right thing and go back to the pointer
> increment when it becomes empty. smbd's use of a talloc pool is
> different: In smbd's main loop we throw away the whole pool. In that
> pattern it's much simpler to understand that it does the pointer
> increment.
> 
> We don't have a specific test for the proper talloc_pool wipe when
> all
> chilred are gone (I did not find any), so this should be part of your
> further investigation of the talloc_pool misery I pointed out.
> 
> Volker
> 
In the area where the pool is introduced we almost always have the
situation that no memory from the pool is used (all mem available)
and we just want to make use of the fact that we re-use this memory.

I just wrote a quick program which is simulating this scenrario with
the 3 different options with which we could get such memory with the
following result

[swen at linux ~]$ ./a.out
It took 2.219697 seconds to execute 10 million talloc/free cycles.
It took 1.450206 seconds to execute 10 million talloc(pool)/free cycles.
It took 1.068344 seconds to execute 10 million malloc/free cycles.

...and yes we will not see those differences in reality, however we did
see enough improvement to make this small change.

So I'm afraid I don't fully understand your argument against this
patch. Do you question the talloc_pool "feature" in total or just the
way it was used by me in this patch ?

Cheers Swen