[Samba] Samba 4.9.4 - high RAM usage - OOM killer

[L.P.H. van Belle]

Hai,

How is the swappiness set? 

cat /proc/sys/vm/swappiness 
Its probely set to 60, i suggest, lower it to 10 or 20.

That wil help keeping the samba processes out of swap. 
I've see more programs gettting OOM Killed due too out of swap. 

echo 10 > /proc/sys/vm/swappiness 
Stop and start samba


Greetz, 

Louis




> -----Oorspronkelijk bericht-----
> Van: samba [mailto:samba-bounces at lists.samba.org] Namens 
> Laurent CARON via samba
> Verzonden: maandag 28 januari 2019 11:46
> Aan: samba at lists.samba.org
> Onderwerp: [Samba] Samba 4.9.4 - high RAM usage - OOM killer
> 
> Hi,
> 
> 
> We upgraded a legacy (NT4) domain from 3.6 series to 4.8 and 
> then 4.9.4 
> samba version (using sernet subscription packages / debian stable)
> 
> The setup is composed of 4 DCs with each 2 CPU/16GB RAM.
> 
> We currently have ~700 user accounts / ~600 computers / ~150 groups
> 
> Our mail setup, SSO, ... query the 4 DCs constantly.
> 
> 
> Every 5 to 10 days the RAM consumption and CPU usage (due to 
> kswapd) are 
> peaking.
> 
> This leads to OOM killer killing samba processes
> 
> kernel: [765104.826327] samba invoked oom-killer: 
> gfp_mask=0x24201ca(GFP_HIGHUSER_MOVABLE|__GFP_COLD), nodemask=0, 
> order=0, oom_score_adj=0
> kernel: [765104.826355]  [<ffffffff8c3871ba>] ? 
> oom_kill_process+0x21a/0x3e0
> kernel: [765104.826357]  [<ffffffff8c386e3d>] ? oom_badness+0xed/0x170
> kernel: [765104.826455] [ pid ]   uid  tgid total_vm      rss nr_ptes 
> nr_pmds swapents oom_score_adj name
> 
> ...
> 
> kernel: [861216.518771] Out of memory: Kill process 603 
> (samba) score 3 
> or sacrifice child
> kernel: [861356.048484]  [<ffffffff8c387651>] ? 
> out_of_memory+0x111/0x470
> 
> samba[614]:   ../source4/dsdb/kcc/kcc_periodic.c:768: Failed 
> samba_kcc - 
> NT_STATUS_NO_MEMORY
> 
> Once this happens, the affected DC is unresponsive for all samba 
> authentication processes (including LDAP).
> 
> A reboot of the affected VM 'cures' the issue, but only for a short 
> amount of time (5 to 10 days).
> 
> Apart from either restarting samba processes on a daily basis, or 
> rebooting the DCs, is there a way to:
> 
> - pinpoint the root cause of the memory consumption (leak, 
> corrupted DB, 
> ...)
> 
> - have the DCs use a more 'normal' amount of RAM ?
> 
> 
> Thanks
> 
> 
> 
> Please note:
> 
> # samba-tool drs kcc
> 
> # samba-tool dbcheck --cross-ncs
> 
> are not showing any errors
> 
> 
> :/etc/nsswitch.conf:
> 
> passwd:         compat
> group:          compat
> shadow:         compat
> gshadow:        files
> 
> hosts:          files dns
> networks:       files
> 
> protocols:      db files
> services:       db files
> ethers:         db files
> rpc:            db files
> 
> 
> :/etc/samba/smb.conf:
> 
> [global]
>          netbios name = VS-DC-001
>          realm = CORP.MYDOMAIN
>          workgroup = SAMBA
> 
>          log file = /var/log/samba/samba.log.%m
>          log level = 1 auth_audit:3 auth_json_audit:3
>          max log size = 50000
>          debug timestamp = yes
>          dns forwarder = 192.168.0.20 192.168.100.20 192.168.0.21 
> 192.168.100.21
>          server role check:inhibit=yes
>          ldap server require strong auth = no
>          wins support = yes
>          server role = active directory domain controller
>          check password script = /usr/local/bin/crackcheck -c -d 
> /var/cache/cracklib/cracklib_dict
>          idmap_ldb:use rfc2307 = yes
>          server schannel = auto
> 
> [netlogon]
>          path = /var/lib/samba/sysvol/corp.lncsa.com/scripts
>          read only = No
> 
> [sysvol]
>          path = /var/lib/samba/sysvol
>          read only = No
> 
> # du -shxc sam.ldb*
> 4.1M    sam.ldb
> 132M    sam.ldb.d
> 136M    total
> 
> Samba packages:
> 
> ii  samba                            99:4.9.4-10  amd64 Glue 
> package for 
> sernet-samba.
> ii  samba-common                     99:4.9.4-10  all Glue 
> package for 
> sernet-samba-common.
> ii  samba-common-bin                 99:4.9.4-10  amd64 Glue 
> package for 
> sernet-samba-client.
> ii  sernet-samba                     99:4.9.4-10  amd64 
> SMB/CIFS file, 
> print, and login server for Unix
> ii  sernet-samba-ad                  99:4.9.4-10  amd64 Samba Active 
> Directory Domain Controller
> ii  sernet-samba-client              99:4.9.4-10  amd64        a 
> LanManager-like simple client for Unix
> ii  sernet-samba-common              99:4.9.4-10  all Samba 
> common files 
> used by both the server and the client
> ii  sernet-samba-libs:amd64          99:4.9.4-10  amd64 Samba common 
> library files used by both the server and the client
> ii  sernet-samba-libsmbclient0:amd64 99:4.9.4-10  amd64 
> Shared library 
> that allows applications to talk to SMB servers
> ii  sernet-samba-winbind             99:4.9.4-10  amd64 Samba 
> nameservice integration server
> 
> 
> 
> 
> -- 
> To unsubscribe from this list go to the following URL and read the
> instructions:  https://lists.samba.org/mailman/options/samba
>