[Samba] Notifyd process and "Too many open files"

[Rowland Penny]

On Wed, 01 Oct 2025 15:24:22 -0700
Devon Johnson via samba <samba at lists.samba.org> wrote:

> Hi Samba gurus,
>  
> Samba 4.19.9 on FreeBSD 14.3 with a ZFS pool for storage.
>  
> This is storage for roughly 30 million image files that our site
> displays via a number of Windows 2019 servers running IIS. There are
> a few other uses, however that is the primary usage. A few weeks back
> a software update was released for the web site running on those IIS
> servers and we noticed a marked increase in load averages after this.
> I do believe its related to these open file messages. We are seeing
> errors in our log about too many open files: Sep 30 11:13:11 XXXXX
> smbd[1111]: [2025/09/30 11:13:11.346275, 0]
> ../../lib/util/debug.c:1274(reopen_one_log) Sep 30 11:13:11 XXXXX
> smbd[1111]: reopen_one_log: Unable to open new log file
> '/var/log/samba4/log.smbd': Too many open files Sep 30 11:13:14 XXXXX
> smbd[1111]: [2025/09/30 11:13:14.603146, 0]
> ../../lib/util/debug.c:1274(reopen_one_log) Sep 30 11:13:14 XXXXX
> smbd[1111]: reopen_one_log: Unable to open new log file
> '/var/log/samba4/log.smbd': Too many open files The log file is being
> written to, however I don't know if process 1111 is writing to it.
> This is notifyd [XXXXXXX /var/log]# ps aux |grep 1111 XXXXXX 1111 0.0
> 0.5 285832 79212 - S Thu22 2:17.17 smbd: notifyd (smbd) XXXXXX 38195
> 0.0 0.0 13836 1920 0 S+ 11:16 0:00.00 grep 1111 If I run an fstat
> command to sum up the open files, I see it is hitting the limit set
> for the process which is 36000. [XXXXXXX /var/log]# fstat|cut -w
> -f1,2,3|sort |uniq -c|sort -rn | grep smbd 33603 XXXXXX smbd 1111
> Yesterday I increased this from 25600 to 33600 and immediately this
> process hit that limit.
> 
> The smbd processes handing the file shares usually stay in the range
> of 2000-3000 open files per process.
> 
> I am trying to figure out what is causing this. The FreeBSD server's
> load averages are higher than they historically have been. I know the
> code is C# that the IIS servers run and I do not have access to it. I
> understand that a new filestream operation was introduced that
> allowed the browser to pick a 'medium' sized file and these are
> properly disposed of in the code as I am being told.
> 
> I am looking for any suggestions or troubleshooting steps to help
> identify what is causing these file locks, or what I can do about it
> besides increasing the limit of open files… eventually that will just
> overload the system as we discovered when we did try to increase it.
> The system became unusable at roughly 520,000 open files which
> happened minutes after smbd was started.
> 
> Below is the sanitized smb4.conf * 
> [XXXXXXX /var/log]# cat /XXX/smb4.conf
> realm = DOMAIN.DOMAIN
> workgroup = WORKGROUP
> security = ADS
> guest account = User
> winbind separator = +
> winbind enum groups = Yes
> winbind enum users = Yes
> winbind nss info = rfc2307
> winbind offline logon = yes
> socket options = TCP_NODELAY IPTOS_LOWDELAY
> idmap config * : range = 1000-200000
> idmap config * : backend = tdb
> template shell = /xxx/xxx/xxx/xxx
> template homedir = /home/%D/%U
> ntlm auth = yes
> passdb backend samba_dsdb
> server role = member server
> username map script = /bin/echo
> log level = 6
> follow symlinks = yes
> unix extensions = no
> nt acl support = yes
> inherit acls = no
> map acl inherit = yes
> deadtime = 10
> 
> [share]
> path = /path/on/zfs
> guest ok = yes
> writeable = yes
> browsable = yes
> inherit permissions = yes
> inherit owner = yes
> locking = yes
> oplocks = yes
> kernel oplocks = no
> level2 oplocks = true
> strict locking = auto
> 
> Thank you in advance!

I am sorry, but there isn't really a polite way of saying this, but
your smb.conf file is rubbish.

Lets start with the pasdb backend, the one in your smb.conf is only
meant to be used on a Samba AD DC, your smb.conf isn't for a Samba AD
DC. However, it might be being ignored because there isn't an equals
sign, '=' , between 'passdb backend' and 'samba_dsdb'.

You haven't set the required idmap backend for 'WORKGROUP' (I take it
that there are AD DCs somewhere), this means that everything is going
into the default '*' domain.

Do you have a local user called 'User' ?
The real question is, why did you set this, you do not appear to be
using guest access. You think you are, you have 'guest ok = yes' set
in '[share]', but you do not have 'map to guest = bad user' set in
'[global]' and without it, there is no guest access.

By setting the 'enum' lines you are just slowing things down.

Why sanitise the template shell, it should be something like /bin/bash
(or the bsd equivalent) ?

Why are you using '/bin/echo' for the 'username map script' ?

Rowland