[PATCH] Remove all uses of talloc_autofree_context() from our code (except for test)

[Simo]

On Wed, 2017-07-26 at 14:26 +0200, Volker Lendecke wrote:
> On Wed, Jul 26, 2017 at 08:09:11AM -0400, Simo wrote:
> > On Tue, 2017-07-25 at 13:04 -0700, Jeremy Allison via samba-
> > technical
> > wrote:
> > > On Tue, Jul 25, 2017 at 09:58:06PM +0200, Volker Lendecke wrote:
> > > > On Tue, Jul 25, 2017 at 12:37:24PM -0700, Jeremy Allison wrote:
> > > > > > Does "smbcontrol smbd pool-usage" still work after this
> > > > > > change?
> > > > > 
> > > > > No. If we're not using the global null_context within
> > > > > talloc anymore, then pool usage doesn't have a central
> > > > > place to track all allocations off it.
> > > > 
> > > > That's tough. I definitely want pool-usage to be available
> > > > everywhere
> > > > and always. When you need it, it's too late to activate it.
> > > 
> > > Having it available "everywhere and always" is inconsistent
> > > with the design criteria behind this patchset. For that you
> > > need a completely MT-safe talloc - you know it and I know it :-).
> > 
> > Jeremy,
> > we can still have it by having a pool of counters instead.
> > 
> > In the main thread you have a linked list of structs reachable from
> > the
> > main thread. And only ever operated on with mutexes.
> > 
> > Each structs represents a thread and points to a thread bound
> > variable
> > that is ever only touched by the thread.
> > This way you can use it from threads without locking.
> > 
> > The tricky part is reading it for reporting without locks, I leave
> > that
> > as an exercise for the reader (use memory barriers).
> 
> smbcontrol pool-usage prints the whole hierarchy. This is important
> if
> you want to find out what kind of object you leak and if for example
> is was accidentially hung off a long-term context. It's not a matter
> of a simple counter. And walking the whole tree that belongs to
> another thread is more than just using memory barriers (not knowing
> how to do even that portably...).

You can hang a "context-per-thread" on the same holding structure.
The problem is how to stop the thread or ask the thread to walk it for
you.

I am afraid that if you want to walk it without races you would need
the thread to take a mutex every time it touches its own thread
context, so that when the thread executing pool-usage is run it can
freeze the thread's context safely.

I would still split contexts per-thread so that normally you have 0
contention on the thread-context mutex, and you pay penalty only when
trying to print the hierarchy.

Remains to see how much overhead acquiring an uncontended mutex on each
talloc operation will add.

Simo.