[Samba] Apparent large memory leak with encryption + SMB3_00 or SMB3_02

Andrew Nicholson abhuman at gmail.com
Fri Jun 19 15:07:03 UTC 2020


I've recently set up Samba (4.12.3) on Arch Linux as the target for Time
Machine backups for a couple of Macs. Shortly thereafter I started seeing
OOMs whenever a backup would start. I stumbled upon disabling encryption on
the server (i.e. changing "smb encrypt" from "required" to "off") to
prevent this issue.

After further digging, I'm able to reproduce this issue using smbclient on
the server machine with either SMB3_00 or SMB3_02 as the max protocol with
the server configured to require encryption. Uploading a file increases the
RSS of the smbd process by roughly the size of the uploaded file.

My minimal smb.conf and relevant smbstatus output are below. Have I missed
something in the configuration? Or is this an issue with the AES-128-CCM
encryption? I noticed that SMB3_10 or SMB3_11 do not suffer from the memory
leak and use AES-128-GCM. Am happy to troubleshoot further as I would like
to re-enable encryption if possible.

Thank you,

   smb encrypt = required

   path = /mnt/data
   writable = yes

Samba version 4.12.3
PID     Username     Group        Machine
     Protocol Version  Encryption           Signing
837051  anichols     anichols (ipv4:
     SMB3_02           AES-128-CCM          AES-128-CMAC

Service      pid     Machine       Connected at
Encryption   Signing
data         837051    Fri Jun 19 03:58:16 PM 2020 BST

No locked files

More information about the samba mailing list