[Samba] samba_spnupdate invoked oom-killer ( samba BUG )

Tue Sep 23 00:45:26 MDT 2014

Hello Chan, 

I think you have hit the same but like : 
https://tracker.zentyal.org/issues/1098 

yes this is zental, but as i did read, this is exactly what happens with your DC. 

You DC gives oom killer, i noticed in the dmesg you send me this happend with update_dns called from samba
From this point i cant help you.. i suggest you report this at the samba devs. 

and your running an up2date samba ( 4.1.12) ? 

Best regards, 

Louis

Van: Chan Min Wai [mailto:dcmwai at gmail.com] 
Verzonden: maandag 22 september 2014 20:41
Aan: L.P.H. van Belle
CC: samba at lists.samba.org
Onderwerp: Re: [Samba] samba_spnupdate invoked oom-killer

+addin the list back

Dear Louis, 

It seem that the memory are building up again but I cannot see much issue with log...

after the sysctl vm.swappiness=10

As my understanding once it reach to somepoint..
it will trigger the oom-killer.

So i think we have a bugs somewhere...

Where should we start?
DC2 are still on 9xx only

             total       used       free     shared    buffers     cached
Mem:          3.9G       2.7G       1.1G       1.2M       4.4M       781M
-/+ buffers/cache:       2.0G       1.9G
Swap:         1.9G         0B       1.9G

However PS seem to show something strange
a samba process with 1.6G memory. PID 4448

I left 3 process for reference, I cannot see the different...
So I think we have a memory leak somewhere under certain condition.

What should we do now?

root      4447  0.0  0.9 550556 37216 ?        S    Sep20   0:05  \_ /usr/sbin/samba MANPATH=/usr/local/share/man:/usr/share/man:/usr/share/gcc-data/x86_64-pc-linux-gnu/4.7.3/man:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.23.2/man CONSOLE=/dev/console SHELL=/bin/sh TERM=linux EINFO_LOG=/etc/init.d/samba MULTIOSDIRS=../lib64:../lib32 CONFIG_PROTECT_MASK=/etc/gentoo-release /etc/sandbox.d /etc/fonts/fonts.conf /etc/terminfo /etc/ca-certificates.conf /etc/revdep-rebuild RC_SVCNAME=samba PWD=/ DISTCC_LOG= LANG=en_US.utf8 DISTCC_VERBOSE= DCCC_PATH=/usr/lib64/distcc/bin SVCNAME=samba SHLVL=1 HOME=/ RC_SERVICE=/etc/init.d/samba LESS=-R -M --shift 5 GCC_SPECS= ROOTPATH=/opt/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/4.7.3 LESSOPEN=|lesspipe %s INFOPATH=/usr/share/info:/usr/share/gcc-data/x86_64-pc-linux-gnu/4.7.3/info:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.23.2/info CONFIG_PROTECT=/var/bind _=/sbin/start-stop-daemon USER=root PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin

root      4448  4.1 42.5 2244104 1725108 ?     S    Sep20 124:49  \_ /usr/sbin/samba MANPATH=/usr/local/share/man:/usr/share/man:/usr/share/gcc-data/x86_64-pc-linux-gnu/4.7.3/man:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.23.2/man CONSOLE=/dev/console SHELL=/bin/sh TERM=linux EINFO_LOG=/etc/init.d/samba MULTIOSDIRS=../lib64:../lib32 CONFIG_PROTECT_MASK=/etc/gentoo-release /etc/sandbox.d /etc/fonts/fonts.conf /etc/terminfo /etc/ca-certificates.conf /etc/revdep-rebuild RC_SVCNAME=samba PWD=/ DISTCC_LOG= LANG=en_US.utf8 DISTCC_VERBOSE= DCCC_PATH=/usr/lib64/distcc/bin SVCNAME=samba SHLVL=1 HOME=/ RC_SERVICE=/etc/init.d/samba LESS=-R -M --shift 5 GCC_SPECS= ROOTPATH=/opt/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/4.7.3 LESSOPEN=|lesspipe %s INFOPATH=/usr/share/info:/usr/share/gcc-data/x86_64-pc-linux-gnu/4.7.3/info:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.23.2/info CONFIG_PROTECT=/var/bind _=/sbin/start-stop-daemon USER=root PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
root      4449  0.0  0.8 549804 34704 ?        S    Sep20   1:47  \_ /usr/sbin/samba MANPATH=/usr/local/share/man:/usr/share/man:/usr/share/gcc-data/x86_64-pc-linux-gnu/4.7.3/man:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.23.2/man CONSOLE=/dev/console SHELL=/bin/sh TERM=linux EINFO_LOG=/etc/init.d/samba MULTIOSDIRS=../lib64:../lib32 CONFIG_PROTECT_MASK=/etc/gentoo-release /etc/sandbox.d /etc/fonts/fonts.conf /etc/terminfo /etc/ca-certificates.conf /etc/revdep-rebuild RC_SVCNAME=samba PWD=/ DISTCC_LOG= LANG=en_US.utf8 DISTCC_VERBOSE= DCCC_PATH=/usr/lib64/distcc/bin SVCNAME=samba SHLVL=1 HOME=/ RC_SERVICE=/etc/init.d/samba LESS=-R -M --shift 5 GCC_SPECS= ROOTPATH=/opt/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/4.7.3 LESSOPEN=|lesspipe %s INFOPATH=/usr/share/info:/usr/share/gcc-data/x86_64-pc-linux-gnu/4.7.3/info:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.23.2/info CONFIG_PROTECT=/var/bind _=/sbin/start-stop-daemon USER=root PATH=/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin

On Fri, Sep 19, 2014 at 12:44 PM, Chan Min Wai <dcmwai at gmail.com> wrote:
Strange enough oom killer had killed samba again. 

Even when I'm not using drbd and also ocfs2 on the sysvol partition (It have been move to other) just that the rsync and unision was not completed before today...

But the oom killer happen yesterday.

This vm have 8G RAM...  only have about 10~20 users normally...
And I cannot think of a good reason on why it crash.

Let me remove drbd and ocfs2 to see how...

On Fri, Sep 19, 2014 at 12:21 PM, Chan Min Wai <dcmwai at gmail.com> wrote:
Dear Louis, 

Thank You for the correction.

I'm sorry as I didn't notice the different :)

Thank again

On Tue, Sep 16, 2014 at 9:48 PM, L.P.H. van Belle <belle at bazuin.nl> wrote:
Hello Chan 

Ah, yes i see, 

rsync -XAavz --log-file /var/log/sysvol-sync.log --delete-after -f"+ */" -f"- *"  /var/lib/samba/sysvol root at server02:/var/lib/samba/  && /usr/bin/unison

This is the correct line.  ^^ 

You did add sysvol in the destination of server02 and it only need its base path. ( aka /var/lib/samba ) 
The unison config  does need /var/lib/samba 
like  : 
root = /var/lib/samba
root = root at server02:/var/lib/samba 
AND 
path = sysvol 

which make it all work. 
Can you check and test above? 

Greetz, 

Louis

Van: Chan Min Wai [mailto:dcmwai at gmail.com] 
Verzonden: dinsdag 16 september 2014 15:31
Aan: L.P.H. van Belle 
CC: samba at lists.samba.org
Onderwerp: Re: [Samba] samba_spnupdate invoked oom-killer

Dear Louis, 

Finally I found my issue, Gentoo need to go eselest unison to choose the current running version so it will make a link on unison to that version.

However, it seem to kept creating the sysvol dir....

Rsync Command 
# rsync -XAavz --log-file /var/log/sysvol-sync.log --delete-after -f"+ */" -f"- *"  /var/lib/samba/sysvol root at server02:/var/lib/samba/sysvol && /usr/bin/unison

sysvol/sysvol/lost+found/

sent 55380 bytes  received 1475 bytes  37903.33 bytes/sec
total size is 0  speedup is 0.00
Contacting server...
Connected [//server01//var/lib/samba -> //server02//var/lib/samba]
Looking for changes
  Waiting for changes from server
Reconciling changes
         <---- new dir    sysvol/sysvol/sysvol
Propagating updates
UNISON 2.40.102 started propagating changes at 21:24:47.36 on 16 Sep 2014
[BGN] Copying sysvol/sysvol/sysvol from //server02//var/lib/samba to /var/lib/samba
[END] Copying sysvol/sysvol/sysvol
UNISON 2.40.102 finished propagating changes at 21:24:47.42 on 16 Sep 2014
Saving synchronizer state

default.prf

root = /var/lib/samba
root = ssh://root@server02//var/lib/samba

# Paths to synchronize
path = sysvol

#ignore = Path stats    ## ignores /var/www/stats
auto=true
batch=true
perms=0
rsync=true
maxthreads=1
retry=3
confirmbigdeletes=false
servercmd = /usr/bin/unison
copythreshold=0
copyprog = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --inplace --compress
copyprogrest = /usr/bin/rsync -XAavz --rsh='ssh -p 22' --partial --inplace --compress
copyquoterem = true
copymax = 1
logfile = /var/log/sysvol-sync.log

On Wed, Sep 10, 2014 at 6:59 PM, L.P.H. van Belle <belle at bazuin.nl> wrote:
Hai Chan,

sorry for the late reaction, i've been ill .. still a bit..
you can safely disable : copythreshold in you config if needed.
this is only for speeding up copying of big files.
for example.
Unison internal file transfer is 5 MB/s
and the alternative copy method does : 20 MB/s

these numbers are fictional, you need to mesure that for yourself.
but for sysvol, wel.. the internal is suffician.
I have that option enabled, because of some testing with automated installations
and there are some big files in these shares.

Greetz,

Louis

>-----Oorspronkelijk bericht-----
>Van: dcmwai at gmail.com [mailto:samba-bounces at lists.samba.org]
>Namens Chan Min Wai
>Verzonden: maandag 8 september 2014 2:30
>Aan: steve
>CC: samba at lists.samba.org
>Onderwerp: Re: [Samba] samba_spnupdate invoked oom-killer
>
>Hi Steve,
>
>Ya, can't get it work...
>with this error.
>I wonder Should I just comment the copythreshold line?
>
>sent 44886 bytes  received 1470 bytes  92712.00 bytes/sec
>total size is 0  speedup is 0.00 (DRY RUN)
>Fatal error: File "default", line 7: `copythreshold' is not a
>valid option
>
>
>
>
>On Sun, Sep 7, 2014 at 3:41 PM, steve <steve at steve-ss.com> wrote:
>
>> On Sun, 2014-09-07 at 15:31 +0800, Chan Min Wai wrote:
>> > Dear Steve,
>> >
>> > Ah no more drbd.
>> > I move to Rsync +unison.
>>
>> Hi Chan
>> Ah, I see, but did you get it working? We couldn't.
>> Cheers,
>> Steve
>>
>>
>>

>--
>To unsubscribe from this list go to the following URL and read the
>instructions:  https://lists.samba.org/mailman/options/samba
>
>

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba