[Samba] samba_spnupdate invoked oom-killer

Chan Min Wai dcmwai at gmail.com
Mon Sep 22 12:41:22 MDT 2014 

+addin the list back

Dear Louis,

It seem that the memory are building up again but I cannot see much issue
with log...

after the sysctl vm.swappiness=10

As my understanding once it reach to somepoint..
it will trigger the oom-killer.

So i think we have a bugs somewhere...

Where should we start?
DC2 are still on 9xx only



             total       used       free     shared    buffers     cached
Mem:          3.9G       2.7G       1.1G       1.2M       4.4M       781M
-/+ buffers/cache:       2.0G       1.9G
Swap:         1.9G         0B       1.9G


However PS seem to show something strange
a samba process with 1.6G memory. PID 4448

I left 3 process for reference, I cannot see the different...
So I think we have a memory leak somewhere under certain condition.

What should we do now?


root      4447  0.0  0.9 550556 37216 ?        S    Sep20   0:05  \_
/usr/sbin/samba
MANPATH=/usr/local/share/man:/usr/share/man:/usr/share/gcc-data/x86_64-pc-linux-gnu/4.7.3/man:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.23.2/man
CONSOLE=/dev/console SHELL=/bin/sh TERM=linux EINFO_LOG=/etc/init.d/samba
MULTIOSDIRS=../lib64:../lib32 CONFIG_PROTECT_MASK=/etc/gentoo-release
/etc/sandbox.d /etc/fonts/fonts.conf /etc/terminfo
/etc/ca-certificates.conf /etc/revdep-rebuild RC_SVCNAME=samba PWD=/
DISTCC_LOG= LANG=en_US.utf8 DISTCC_VERBOSE= DCCC_PATH=/usr/lib64/distcc/bin
SVCNAME=samba SHLVL=1 HOME=/ RC_SERVICE=/etc/init.d/samba LESS=-R -M
--shift 5 GCC_SPECS=
ROOTPATH=/opt/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/4.7.3 LESSOPEN=|lesspipe
%s
INFOPATH=/usr/share/info:/usr/share/gcc-data/x86_64-pc-linux-gnu/4.7.3/info:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.23.2/info
CONFIG_PROTECT=/var/bind _=/sbin/start-stop-daemon USER=root
PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
root      4448  4.1 42.5 2244104 1725108 ?     S    Sep20 124:49  \_
/usr/sbin/samba
MANPATH=/usr/local/share/man:/usr/share/man:/usr/share/gcc-data/x86_64-pc-linux-gnu/4.7.3/man:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.23.2/man
CONSOLE=/dev/console SHELL=/bin/sh TERM=linux EINFO_LOG=/etc/init.d/samba
MULTIOSDIRS=../lib64:../lib32 CONFIG_PROTECT_MASK=/etc/gentoo-release
/etc/sandbox.d /etc/fonts/fonts.conf /etc/terminfo
/etc/ca-certificates.conf /etc/revdep-rebuild RC_SVCNAME=samba PWD=/
DISTCC_LOG= LANG=en_US.utf8 DISTCC_VERBOSE= DCCC_PATH=/usr/lib64/distcc/bin
SVCNAME=samba SHLVL=1 HOME=/ RC_SERVICE=/etc/init.d/samba LESS=-R -M
--shift 5 GCC_SPECS=
ROOTPATH=/opt/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/4.7.3 LESSOPEN=|lesspipe
%s
INFOPATH=/usr/share/info:/usr/share/gcc-data/x86_64-pc-linux-gnu/4.7.3/info:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.23.2/info
CONFIG_PROTECT=/var/bind _=/sbin/start-stop-daemon USER=root
PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
root      4449  0.0  0.8 549804 34704 ?        S    Sep20   1:47  \_
/usr/sbin/samba
MANPATH=/usr/local/share/man:/usr/share/man:/usr/share/gcc-data/x86_64-pc-linux-gnu/4.7.3/man:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.23.2/man
CONSOLE=/dev/console SHELL=/bin/sh TERM=linux EINFO_LOG=/etc/init.d/samba
MULTIOSDIRS=../lib64:../lib32 CONFIG_PROTECT_MASK=/etc/gentoo-release
/etc/sandbox.d /etc/fonts/fonts.conf /etc/terminfo
/etc/ca-certificates.conf /etc/revdep-rebuild RC_SVCNAME=samba PWD=/
DISTCC_LOG= LANG=en_US.utf8 DISTCC_VERBOSE= DCCC_PATH=/usr/lib64/distcc/bin
SVCNAME=samba SHLVL=1 HOME=/ RC_SERVICE=/etc/init.d/samba LESS=-R -M
--shift 5 GCC_SPECS=
ROOTPATH=/opt/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/4.7.3 LESSOPEN=|lesspipe
%s
INFOPATH=/usr/share/info:/usr/share/gcc-data/x86_64-pc-linux-gnu/4.7.3/info:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.23.2/info
CONFIG_PROTECT=/var/bind _=/sbin/start-stop-daemon USER=root
PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin


On Fri, Sep 19, 2014 at 12:44 PM, Chan Min Wai <dcmwai at gmail.com> wrote:

> Strange enough oom killer had killed samba again.
>
> Even when I'm not using drbd and also ocfs2 on the sysvol partition (It
> have been move to other) just that the rsync and unision was not completed
> before today...
>
> But the oom killer happen yesterday.
>
> This vm have 8G RAM...  only have about 10~20 users normally...
> And I cannot think of a good reason on why it crash.
>
> Let me remove drbd and ocfs2 to see how...
>
>
>
> On Fri, Sep 19, 2014 at 12:21 PM, Chan Min Wai <dcmwai at gmail.com> wrote:
>
>> Dear Louis,
>>
>> Thank You for the correction.
>>
>> I'm sorry as I didn't notice the different :)
>>
>> Thank again
>>
>> On Tue, Sep 16, 2014 at 9:48 PM, L.P.H. van Belle <belle at bazuin.nl>
>> wrote:
>>
>>>  Hello Chan
>>>
>>> Ah, yes i see,
>>>
>>> rsync -XAavz --log-file /var/log/sysvol-sync.log --delete-after -f"+ */"
>>> -f"- *"  /var/lib/samba/sysvol root at server02:/var/lib/samba/  &&
>>> /usr/bin/unison
>>> This is the correct line.  ^^
>>>
>>> You did add sysvol in the destination of server02 and it only need its
>>> base path. ( aka /var/lib/samba )
>>> The unison config  does need /var/lib/samba
>>> like  :
>>> root = /var/lib/samba
>>> root = root at server02:/var/lib/samba
>>> AND
>>> path = sysvol
>>>
>>> which make it all work.
>>> Can you check and test above?
>>>
>>>
>>> Greetz,
>>>
>>> Louis
>>>
>>>
>>>  ------------------------------
>>> *Van:* Chan Min Wai [mailto:dcmwai at gmail.com]
>>> *Verzonden:* dinsdag 16 september 2014 15:31
>>> *Aan:* L.P.H. van Belle
>>>
>>> *CC:* samba at lists.samba.org
>>> *Onderwerp:* Re: [Samba] samba_spnupdate invoked oom-killer
>>>
>>>  Dear Louis,
>>>
>>> Finally I found my issue, Gentoo need to go eselest unison to choose the
>>> current running version so it will make a link on unison to that version.
>>>
>>> However, it seem to kept creating the sysvol dir....
>>>
>>> Rsync Command
>>> # rsync -XAavz --log-file /var/log/sysvol-sync.log --delete-after -f"+
>>> */" -f"- *"  /var/lib/samba/sysvol root at server02:/var/lib/samba/sysvol
>>> && /usr/bin/unison
>>>
>>>  sysvol/sysvol/lost+found/
>>>
>>> sent 55380 bytes  received 1475 bytes  37903.33 bytes/sec
>>> total size is 0  speedup is 0.00
>>> Contacting server...
>>> Connected [//server01//var/lib/samba -> //server02//var/lib/samba]
>>> Looking for changes
>>>   Waiting for changes from server
>>> Reconciling changes
>>>          <---- new dir    sysvol/sysvol/sysvol
>>> Propagating updates
>>> UNISON 2.40.102 started propagating changes at 21:24:47.36 on 16 Sep 2014
>>> [BGN] Copying sysvol/sysvol/sysvol from //server02//var/lib/samba to
>>> /var/lib/samba
>>> [END] Copying sysvol/sysvol/sysvol
>>> UNISON 2.40.102 finished propagating changes at 21:24:47.42 on 16 Sep
>>> 2014
>>> Saving synchronizer state
>>>
>>>
>>>
>>>
>>>
>>> default.prf
>>>
>>>  root = /var/lib/samba
>>> root = ssh://root@server02//var/lib/samba
>>>
>>> # Paths to synchronize
>>> path = sysvol
>>>
>>> #ignore = Path stats    ## ignores /var/www/stats
>>> auto=true
>>> batch=true
>>> perms=0
>>> rsync=true
>>> maxthreads=1
>>> retry=3
>>> confirmbigdeletes=false
>>> servercmd = /usr/bin/unison
>>> copythreshold=0
>>> copyprog = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --inplace --compress
>>> copyprogrest = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --partial
>>> --inplace --compress
>>> copyquoterem = true
>>> copymax = 1
>>> logfile = /var/log/sysvol-sync.log
>>>
>>>
>>> On Wed, Sep 10, 2014 at 6:59 PM, L.P.H. van Belle <belle at bazuin.nl>
>>> wrote:
>>>
>>>> Hai Chan,
>>>>
>>>> sorry for the late reaction, i've been ill .. still a bit..
>>>> you can safely disable : copythreshold in you config if needed.
>>>> this is only for speeding up copying of big files.
>>>> for example.
>>>> Unison internal file transfer is 5 MB/s
>>>> and the alternative copy method does : 20 MB/s
>>>>
>>>> these numbers are fictional, you need to mesure that for yourself.
>>>> but for sysvol, wel.. the internal is suffician.
>>>> I have that option enabled, because of some testing with automated
>>>> installations
>>>> and there are some big files in these shares.
>>>>
>>>> Greetz,
>>>>
>>>> Louis
>>>>
>>>>
>>>> >-----Oorspronkelijk bericht-----
>>>> >Van: dcmwai at gmail.com [mailto:samba-bounces at lists.samba.org]
>>>> >Namens Chan Min Wai
>>>> >Verzonden: maandag 8 september 2014 2:30
>>>> >Aan: steve
>>>> >CC: samba at lists.samba.org
>>>> >Onderwerp: Re: [Samba] samba_spnupdate invoked oom-killer
>>>>  >
>>>> >Hi Steve,
>>>> >
>>>> >Ya, can't get it work...
>>>> >with this error.
>>>> >I wonder Should I just comment the copythreshold line?
>>>> >
>>>> >sent 44886 bytes  received 1470 bytes  92712.00 bytes/sec
>>>> >total size is 0  speedup is 0.00 (DRY RUN)
>>>> >Fatal error: File "default", line 7: `copythreshold' is not a
>>>> >valid option
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >On Sun, Sep 7, 2014 at 3:41 PM, steve <steve at steve-ss.com> wrote:
>>>> >
>>>> >> On Sun, 2014-09-07 at 15:31 +0800, Chan Min Wai wrote:
>>>> >> > Dear Steve,
>>>> >> >
>>>> >> > Ah no more drbd.
>>>> >> > I move to Rsync +unison.
>>>> >>
>>>> >> Hi Chan
>>>> >> Ah, I see, but did you get it working? We couldn't.
>>>> >> Cheers,
>>>> >> Steve
>>>> >>
>>>> >>
>>>> >>
>>>>  >--
>>>> >To unsubscribe from this list go to the following URL and read the
>>>> >instructions:  https://lists.samba.org/mailman/options/samba
>>>> >
>>>> >
>>>>
>>>> --
>>>> To unsubscribe from this list go to the following URL and read the
>>>> instructions:  https://lists.samba.org/mailman/options/samba
>>>>
>>>
>>>
>>
>

More information about the samba mailing list