[Samba] Samba allocates too much memory

[Pluess, Tobias]

Hello SAMBA mailing list,

I experience on one SAMBA File Server the following problem.
I have several LXC Containers where I run SAMBA Fileservers inside. For
several years now this setup has already worked super reliably and
satisfying, the performance is good and the reliability exceptional. The
underlying storage is ZFS. On the largest of these SAMBA Servers I have
more than 50TiB of data and more than 50 million files. It works super nice
and with good performance.
Due to this good experiences, I recently set up a brand new LXC with SAMBA
File server inside for another use case, and here, I see occasional
crashes. I first thought it is some misconfiguration I made on my side, and
investigated, but it turns out that the OOM Killer from Linux was actually
killing my SAMBA processes. And why was this the case?

Well, I see that one particular user accessing the SAMBA Server lets the
SAMBA allocated heap grow and grow and grow beyond all limits. For example,
on my aforementioned 50 Million Files SAMBA Server, I see this:

root at tank:~# free -h
               total        used        free      shared  buff/cache
available
Mem:            16Gi       181Mi        14Gi        64Mi       998Mi
 15Gi
Swap:          8.0Gi       180Mi       7.8Gi

there are people actively working on these data sets, and still, the
consumed RAM is completely negligible. So in several years now that this
SAMBA Server is running, ther was never one single crash!

and now, on this new SAMBA Server, I see this:

root at files:~# free -h
               total        used        free      shared  buff/cache
available
Mem:            32Gi       7.1Gi        24Gi        31Mi       263Mi
 24Gi
Swap:          8.0Gi       2.3Gi       5.7Gi

so already 10GB consumed, and this is by just one single SAMBA process. To
confirm this, I used the below command:

watch -n 2 'ps -eo pid,user,rss,vsz,etime,cmd --sort=-rss | grep smbd'

Every 2.0s: ps -eo pid,user,rss,vsz,etime,cmd --sort=-rss | grep smbd
                                         files: Fri Sep 18 07:36:00 2026

1306309 <user> 7342972 9368128 2-17:30:10 smbd: client [ip.153]
1072879 <user> 45632 166164  4-01:15:54 smbd: client [ip..160]
1074414 root     45596 143756  4-01:05:48 smbd: client [ip.36]
1292641 root     25880 146108  2-19:21:31 smbd: client [ip..144]
1072881 <user> 25356 126988  4-01:15:54 smbd: client [ip..126]
1644218 root      4916 121460    23:59:21 smbd: scavenger
1072775 root      2672  92652  4-01:15:55 /usr/sbin/smbd --foreground
--no-process-group
1828082 root      2436   3544       00:02 watch -n 2 ps -eo
pid,user,rss,vsz,etime,cmd --sort=-rss | grep smbd
1072778 root      1448  89428  4-01:15:55 smbd: notifyd .
1072779 root      1436  89444  4-01:15:55 smbd: cleanupd
1828088 root      1416   2680       00:00 sh -c -- ps -eo
pid,user,rss,vsz,etime,cmd --sort=-rss | grep smbd
1828087 root      1324   3544       00:00 watch -n 2 ps -eo
pid,user,rss,vsz,etime,cmd --sort=-rss | grep smbd
1828090 root      1284   3508       00:00 grep smbd


So one particular user whose IP is ending with 153 is consuming a lot of
RAM. Interesting fact: the SAMBA Server is running since the last reboot
for 2 weeks now, and during these 2 weeks, until yesterday, the RAM
consumption was always in the 10-ish MB range, so very healthy. But
overnight it exploded to 10-ish GB. I try to look at this particular
process a bit closer, with this:

 watch -n 2 'grep -E "VmRSS|VmHWM|RssAnon|RssFile|RssShmem|VmSwap"
/proc/1306309/status'

and I get the below output. So we see that this process consumed even a
high watermark of more than 10GB, and he is even swapping also.

Every 2.0s: grep -E "VmRSS|VmHWM|RssAnon|RssFile|RssShmem|VmSwap"
/proc/1306309/status                               files: Fri Sep 18
07:38:52 2026

VmHWM:  10595420 kB
VmRSS:   7342972 kB
RssAnon:         7310488 kB
RssFile:            5412 kB
RssShmem:          27072 kB
VmSwap:  1901368 kB



What I would like to try to understand now is, what causes SAMBA to do
these massiva allocations, is it a misconfiguration (possible) or is it a
weird kind of access pattern, or does the problem occur when people are
opening large files? is there one configuration to tell samba that he does
not have to cache so much?

I also try to inspect the log file for that particular IP, but it is empty.
(/var/log/samba/log.<ip-addr>

My SAMBA config file is super basic, I have not tuned any tunables, I just
have these:

vfs objects = acl_xattr shadow_copy2
allocation roundup size = 4194304
strict allocate = Yes
kernel oplocks = no
kernel share modes = no


The latter 4 I have added due to some research I tried to do to fix this
problem. It looks to me that, until now, the problem is a bit mitigated,
but 10 GB of allocated memory for SAMBA is huge compared to my experience,
and I would like to understand what causes it to do so, how I could prevent
it or if it is a mistake from my side.

Any hints?

thanks!
Greetings Tobias.