Samba4 and memory consumption - needing frequent kills

[Andrew Bartlett]

On Thu, 2012-08-09 at 08:44 +0100, Kev Latimer wrote:
> On 09/08/2012 08:08, Andrew Bartlett wrote:
> > On Wed, 2012-08-08 at 21:04 +0200, Arvid Requate wrote:
> >> Hello Kev,
> >>
> >> maybe you are facing the same issue here as reported in
> >> https://bugzilla.samba.org/show_bug.cgi?id=8827 ?
> > As I've just commented on the bug, I need a --leak-report-full rather
> > than just --leak-report.
> >
> > Also, if that does not show the cause, perhaps instead of shutting down
> > the server run with --leak-report-full, then attach to the large process
> > with:
> >
> > gdb -p <pid>
> >
> > get me a bt full (if the process is busy, just for interest)
> >
> > Then run
> >
> > p talloc_enable_null_tracking()
> > p talloc_report_full(0, stderr)
> >
> > and mail me the output or attach it to the bug.  (mail it to me
> > privately if it is not a test domain).
> >
> > Hopefully I can then figure out what is still attached to the
> > talloc_autofree_context() and figure out a way to ensure it is cleaned
> > up earlier.
> >
> > Thanks,
> >
> > Andrew Bartlett
> >
> 
> Hi Arvid/Andrew,
> 
> Thanks for pointing me in the direction of that one, I hadn't spotted 
> that bug report but yes, I think I have the same issue.  The growth of 
> two processes looks very similar to what I get.  I'll follow your 
> procedures and what Andrew has asked for later on today and hopefully 
> should have some useful info I can pass on over the next 24 hours.  Good 
> to know it's not just me :-)

Attached are two patches.  The first corrects the only memory handling
error I could find in this area, but doesn't fully explain what's going
on (and shown in the supplied logs) here.  The second is a further
attempt, that shouldn't help but if it does, please let me know. 

The first patch is (along with some other schema handling changes)
already in autobuild. 

I still have not found the root cause here, but in the log you gave me
there was 125 schemas loaded and attached to the autofree context as
'global' schema, which seems a little excessive...

Andrew Bartlett

-- 
Andrew Bartlett                                http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-s4-dsdb-Take-more-care-in-handling-of-global-schema-.patch
Type: text/x-patch
Size: 8865 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20120811/912de219/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-Pass-old-global-schema-to-dsdb_make_schema_global.patch
Type: text/x-patch
Size: 3376 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20120811/912de219/attachment-0001.bin>