[Samba] Samba 4.9.4 - high RAM usage - OOM killer

[Laurent CARON]

Hi,


We upgraded a legacy (NT4) domain from 3.6 series to 4.8 and then 4.9.4 
samba version (using sernet subscription packages / debian stable)

The setup is composed of 4 DCs with each 2 CPU/16GB RAM.

We currently have ~700 user accounts / ~600 computers / ~150 groups

Our mail setup, SSO, ... query the 4 DCs constantly.


Every 5 to 10 days the RAM consumption and CPU usage (due to kswapd) are 
peaking.

This leads to OOM killer killing samba processes

kernel: [765104.826327] samba invoked oom-killer: 
gfp_mask=0x24201ca(GFP_HIGHUSER_MOVABLE|__GFP_COLD), nodemask=0, 
order=0, oom_score_adj=0
kernel: [765104.826355]  [<ffffffff8c3871ba>] ? oom_kill_process+0x21a/0x3e0
kernel: [765104.826357]  [<ffffffff8c386e3d>] ? oom_badness+0xed/0x170
kernel: [765104.826455] [ pid ]   uid  tgid total_vm      rss nr_ptes 
nr_pmds swapents oom_score_adj name

...

kernel: [861216.518771] Out of memory: Kill process 603 (samba) score 3 
or sacrifice child
kernel: [861356.048484]  [<ffffffff8c387651>] ? out_of_memory+0x111/0x470

samba[614]:   ../source4/dsdb/kcc/kcc_periodic.c:768: Failed samba_kcc - 
NT_STATUS_NO_MEMORY

Once this happens, the affected DC is unresponsive for all samba 
authentication processes (including LDAP).

A reboot of the affected VM 'cures' the issue, but only for a short 
amount of time (5 to 10 days).

Apart from either restarting samba processes on a daily basis, or 
rebooting the DCs, is there a way to:

- pinpoint the root cause of the memory consumption (leak, corrupted DB, 
...)

- have the DCs use a more 'normal' amount of RAM ?


Thanks



Please note:

# samba-tool drs kcc

# samba-tool dbcheck --cross-ncs

are not showing any errors


:/etc/nsswitch.conf:

passwd:         compat
group:          compat
shadow:         compat
gshadow:        files

hosts:          files dns
networks:       files

protocols:      db files
services:       db files
ethers:         db files
rpc:            db files


:/etc/samba/smb.conf:

[global]
         netbios name = VS-DC-001
         realm = CORP.MYDOMAIN
         workgroup = SAMBA

         log file = /var/log/samba/samba.log.%m
         log level = 1 auth_audit:3 auth_json_audit:3
         max log size = 50000
         debug timestamp = yes
         dns forwarder = 192.168.0.20 192.168.100.20 192.168.0.21 
192.168.100.21
         server role check:inhibit=yes
         ldap server require strong auth = no
         wins support = yes
         server role = active directory domain controller
         check password script = /usr/local/bin/crackcheck -c -d 
/var/cache/cracklib/cracklib_dict
         idmap_ldb:use rfc2307 = yes
         server schannel = auto

[netlogon]
         path = /var/lib/samba/sysvol/corp.lncsa.com/scripts
         read only = No

[sysvol]
         path = /var/lib/samba/sysvol
         read only = No

# du -shxc sam.ldb*
4.1M    sam.ldb
132M    sam.ldb.d
136M    total

Samba packages:

ii  samba                            99:4.9.4-10  amd64 Glue package for 
sernet-samba.
ii  samba-common                     99:4.9.4-10  all Glue package for 
sernet-samba-common.
ii  samba-common-bin                 99:4.9.4-10  amd64 Glue package for 
sernet-samba-client.
ii  sernet-samba                     99:4.9.4-10  amd64 SMB/CIFS file, 
print, and login server for Unix
ii  sernet-samba-ad                  99:4.9.4-10  amd64 Samba Active 
Directory Domain Controller
ii  sernet-samba-client              99:4.9.4-10  amd64        a 
LanManager-like simple client for Unix
ii  sernet-samba-common              99:4.9.4-10  all Samba common files 
used by both the server and the client
ii  sernet-samba-libs:amd64          99:4.9.4-10  amd64 Samba common 
library files used by both the server and the client
ii  sernet-samba-libsmbclient0:amd64 99:4.9.4-10  amd64 Shared library 
that allows applications to talk to SMB servers
ii  sernet-samba-winbind             99:4.9.4-10  amd64 Samba 
nameservice integration server