Samba4 and memory consumption - needing frequent kills

[Kev Latimer]

On 15/08/2012 03:30, Ricky Nance wrote:
> Glad I could help.
> Thank you guys for all the hard work you have done on this!
>
> Ricky
>
> On Tue, Aug 14, 2012 at 9:22 PM, Andrew Bartlett <abartlet at samba.org> wrote:
>
>> On Sat, 2012-08-11 at 17:00 +1000, Andrew Bartlett wrote:
>>> On Thu, 2012-08-09 at 17:08 +1000, Andrew Bartlett wrote:
>>>> On Wed, 2012-08-08 at 21:04 +0200, Arvid Requate wrote:
>>>>> Hello Kev,
>>>>>
>>>>> maybe you are facing the same issue here as reported in
>>>>> https://bugzilla.samba.org/show_bug.cgi?id=8827 ?
>>>> As I've just commented on the bug, I need a --leak-report-full rather
>>>> than just --leak-report.
>>>>
>>>> Also, if that does not show the cause, perhaps instead of shutting down
>>>> the server run with --leak-report-full, then attach to the large
>> process
>>>> with:
>>> Given the issues with the memory dumps being repeated in overwealming
>>> detail, here is a new instruction:
>>>
>>> Run samba as:
>>>
>>> gdb --args samba -M single --leak-report
>>>
>>> Then:
>>>
>>> run
>>>
>>> When it becomes large, run
>>>
>>> p talloc_report_full(0, stderr)
>>>
>>> and mail me *only* the output of that command, not the earlier or later
>> output.
>>> The issue with the previous logs is that the same allocation (present in
>>> multiple forked children) was being logged multiple times, as each child
>>> exited.  This caused the massive log buildup, without giving me much
>>> more information.
>>>
>>> You may replace stderr with a file on disk with something like:
>>>
>>> p talloc_report_full(0, fopen("/tmp/leak.txt", "w")
>>>
>>> Hopefully this will give you and me the information we need.
>> Thanks to Ricky Nace who was able to provide me with exactly this (on a
>> standard process modal run it turns out), I've been able to fix this.
>>
>> For sites running older Samba betas, this patch series might be helpful.
>> Specifically, it should be tested and worked into things like the debian
>> packages, as they are stuck at beta2.
>>
>> Andrew Bartlett
>>
>> --
>> Andrew Bartlett                                http://samba.org/~abartlet/
>> Authentication Developer, Samba Team           http://samba.org
>>
>>
>
> --
Hi Andrew,

Just wanted to quickly apologise for not getting all the info you needed 
as quickly as either of us would have liked.  My availability was zero 
last week because of an internal project and I simply wasn't able to run 
my samba install in the way you needed as I wasn't there to keep any 
kind of eye on it.  I did find a few hours on Thursday night, after I'd 
seen this mail, to grab the latest master and I can confirm the issue 
has certainly been mitigated although I'm not sure it's gone 
completely.  I've been running this build since the 16th, so the fact it 
hasn't devoured all my swap yet is certainly progress, but I do have a 
few processes larger than the others still, just not massively larger: 
two at around 750MB, one tipping the scales at the wrong side of 1GB.  
I'm running 4.0.0beta7-GIT-56fc7bc.

Would you be able to confirm if your fixes are in the git I'm running 
and if that consumption is to be expected?  I'm more than happy to 
produce the reports you were after if they're still of any use if you 
think there's anything else worth looking at.

Ricky - are you able to report back on your results?

Thanks again for all your efforts.

Cheers,

Kev

-- 
Kev