Samba4 and memory consumption - needing frequent kills
Kev Latimer
klatimer at tolent.co.uk
Tue Aug 21 09:37:37 MDT 2012
Hi Ricky,
(Sorry all for the top/bottom posting shift!)
I hadn't seen ps_mem.py before. Having found, obtained and run, I very
much like it :-)
708.2 MiB + 4.6 MiB = 712.8 MiB named
1.3 GiB + 47.8 MiB = 1.4 GiB samba (15)
I'm running NTVFS rather than s3fs (these aren't really file servers,
just some very simple file sharing for antivirus updates and GPO's).
I'm also running BIND9_DLZ and have a similar size to you (~300 users,
~250 machines inc. servers and DC's), although I'm running 6 DC's due to
a geographic spread of users and have a site/subnet set up for each
server - don't know how much that differs from your setup. The report
above is for what I consider to be my "primary" DC, the one running
BIND9_DLZ and that I modify my GPO's on.
Here's a ps_mem.py report from another DC (just running BIND9 as a
secondary):
16.1 MiB + 254.0 KiB = 16.3 MiB named
1.1 GiB + 39.3 MiB = 1.1 GiB samba (15)
Now I think about it, I've another DC that actually is running s3fs
(simply because s3fs was the default when I built this one and I've not
gotten round to changing it to NTVFS to match it's partners):
1.6 MiB + 2.1 MiB = 3.7 MiB smbd (2)
19.0 MiB + 180.5 KiB = 19.2 MiB named
2.7 GiB + 31.1 MiB = 2.7 GiB samba (13)
Perhaps I jumped the gun and have too early a git to get the fixes so
I'll look to build a new one as soon as I can as see if that makes a
difference. They're all running 4.0.0beta7-GIT-56fc7bc (from Thursday
~1pm (GMT+1)) and I can't see anything in the git log that might have
addressed this bug (at least I don't think I have) so I'll get that
right up to date and take another look.
Cheers,
Kev
On 21/08/2012 13:23, Ricky Nance wrote:
> Kev, I think Andrew has patched it to where the leaks are either non
> existant, or just very very slow. Currently using ps_mem.py I have this:
>
> 51.1 MiB + 2.4 MiB = 53.5 MiB named
> 100.9 MiB + 50.0 MiB = 151.0 MiB samba (13)
> 233.7 MiB + 43.0 MiB = 276.8 MiB smbd (36)
>
> It should be noted that I am using s3fs and bind9 dlz in my setup and
> have around 350 users and about 250 computers. I will also say that
> the named process hasn't gained in size like it was. This setup has
> been running for about 3 days now, and memory usage seems fine. I am
> running Version 4.0.0beta7-GIT-4f4bb1f, as of the writing of this
> email its 7:20 on a the 8/21 and this was built around 21:00 on 8/17
> if that helps give you an estimate of time. How exactly are you
> checking memory consumption on your setup?
>
> Ricky
>
> On Tue, Aug 21, 2012 at 3:39 AM, Kev Latimer <klatimer at tolent.co.uk
> <mailto:klatimer at tolent.co.uk>> wrote:
>
> On 15/08/2012 03:30, Ricky Nance wrote:
>
> Glad I could help.
> Thank you guys for all the hard work you have done on this!
>
> Ricky
>
> On Tue, Aug 14, 2012 at 9:22 PM, Andrew Bartlett
> <abartlet at samba.org <mailto:abartlet at samba.org>> wrote:
>
> On Sat, 2012-08-11 at 17:00 +1000, Andrew Bartlett wrote:
>
> On Thu, 2012-08-09 at 17:08 +1000, Andrew Bartlett wrote:
>
> On Wed, 2012-08-08 at 21:04 +0200, Arvid Requate
> wrote:
>
> Hello Kev,
>
> maybe you are facing the same issue here as
> reported in
> https://bugzilla.samba.org/show_bug.cgi?id=8827 ?
>
> As I've just commented on the bug, I need a
> --leak-report-full rather
> than just --leak-report.
>
> Also, if that does not show the cause, perhaps
> instead of shutting down
> the server run with --leak-report-full, then
> attach to the large
>
> process
>
> with:
>
> Given the issues with the memory dumps being repeated
> in overwealming
> detail, here is a new instruction:
>
> Run samba as:
>
> gdb --args samba -M single --leak-report
>
> Then:
>
> run
>
> When it becomes large, run
>
> p talloc_report_full(0, stderr)
>
> and mail me *only* the output of that command, not the
> earlier or later
>
> output.
>
> The issue with the previous logs is that the same
> allocation (present in
> multiple forked children) was being logged multiple
> times, as each child
> exited. This caused the massive log buildup, without
> giving me much
> more information.
>
> You may replace stderr with a file on disk with
> something like:
>
> p talloc_report_full(0, fopen("/tmp/leak.txt", "w")
>
> Hopefully this will give you and me the information we
> need.
>
> Thanks to Ricky Nace who was able to provide me with
> exactly this (on a
> standard process modal run it turns out), I've been able
> to fix this.
>
> For sites running older Samba betas, this patch series
> might be helpful.
> Specifically, it should be tested and worked into things
> like the debian
> packages, as they are stuck at beta2.
>
> Andrew Bartlett
>
> --
> Andrew Bartlett http://samba.org/~abartlet/
> <http://samba.org/%7Eabartlet/>
> Authentication Developer, Samba Team http://samba.org
>
>
>
> --
>
> Hi Andrew,
>
> Just wanted to quickly apologise for not getting all the info you
> needed as quickly as either of us would have liked. My
> availability was zero last week because of an internal project and
> I simply wasn't able to run my samba install in the way you needed
> as I wasn't there to keep any kind of eye on it. I did find a few
> hours on Thursday night, after I'd seen this mail, to grab the
> latest master and I can confirm the issue has certainly been
> mitigated although I'm not sure it's gone completely. I've been
> running this build since the 16th, so the fact it hasn't devoured
> all my swap yet is certainly progress, but I do have a few
> processes larger than the others still, just not massively larger:
> two at around 750MB, one tipping the scales at the wrong side of
> 1GB. I'm running 4.0.0beta7-GIT-56fc7bc.
>
> Would you be able to confirm if your fixes are in the git I'm
> running and if that consumption is to be expected? I'm more than
> happy to produce the reports you were after if they're still of
> any use if you think there's anything else worth looking at.
>
> Ricky - are you able to report back on your results?
>
> Thanks again for all your efforts.
>
> Cheers,
>
> Kev
>
> --
> Kev
>
>
>
>
> --
>
>
--
Kev**
More information about the samba-technical
mailing list