[Samba] Friendly Reminder: Huge number of small files performance regression from 3.5.16 to 4.6.5 with identical smb.conf

[awl1]

Hello Andrew,

many thanks for your thoughts and clarifications - here's my take on them:

Regarding 1), I had indeed started to look into simply adding "nt pipe 
support = no" to the original 3.5.6 Samba smb.conf first. But in order 
to do so, I would also need to create my own Samba module, replacing the 
Thecus built-in: Without deactivating the built-in Samba, it is not 
possible to add custom smb.conf entries (anything not settable through 
the admin GUI), and the admin GUI backs out/overwrites any custom 
changes to the default smb.conf from an SSH command line every time I 
execute any non-readonly operation from the admin GUI... :-(

So I ended up with not simply providing my own smb.conf, but also 
compiling my own Samba version, and I have indeed successfully 
cross-compiled Samba 4.6.5 including all dependencies without any 
issues: Besides the performance regression for huge numbers of small 
files, my self-compiled 4.6.5 works completely fine and as expected.

Would you indeed think that the performance issue (SMB2 3.1.1 notably 
slower than SMB 1.5 when copying a huge number of small files) will 
document itself in debug level log entries? As these log entries will 
also need to be written to the exact same NAS HDDs, they would also 
inevitably come with a throughput/performance overhead/bottleneck for 
this debug logging...!? (With Thecus' default log level settings, there 
are no error messages at all in any of the Samba logs when I execute the 
file copy tests...)

And regarding 3): Is there any particular reason why you seem to expect 
Louis' Debian modules for 4.6.5 to perform better than my self-compiled 
version? Could this indeed be due to the fact that my Samba 4.6.5 on the 
NAS is still running on Thecus kernel 2.6.33 (I'd rather question this, 
as I have compiled Samba itself as well as all its required dependencies 
in up-to-date versions with gcc-5.2 from the latest crosstools-ng that 
works on the NAS's kernel)?

This is my private NAS located in my Home Office LAN, and it also runs a 
custom (and more or less up-to-date) OpenSSH and Dovecot service 
compiled by myself. The NAS is mainly used as a backup target and for 
development purposes, so the huge number of small files scenario 
unfortunately is not that rare...


Base line:
 From my guts feeling, most probably comparing the Wireshark packet 
captures as proposed by Jeremy would be a good way to move forward and 
find out why the number of packets in the Samba 4.6.5/SMB2 3.1.1 
scenario seems to be higher than using Samba 3.6.15/SMB 1.5, but I will 
clearly need some help by you, SMB protocol/Wireshark experts, to know 
what to look for in those packet captures... ;-)

So I'd still prefer to stay with my self-compiled 4.6.x version and am 
looking forward for some help from the experts in how to read/analyze 
the Wireshark captures, trying to find out why SMB2/3.1.1 in Samba 4.6.5 
seems to be hitting a performance regression...

Thanks anyway & best regards
Andreas


Am 29.06.2017 um 22:14 schrieb Andrew Walker via samba:
> Andreas,
>
> A few thoughts regarding your system
> 1) If it's a home system and you're specifically concerned about mitigating
> CVE 2017-7494, (a) verify that your share isn't mounted 'noexec' - if it's
> mounted this way then you're safe (b) if not (a), then add the [global]
> parameter "nt pipe support = no". This will break functionality that relies
> on support for named pipes, but downloading / uploading files should still
> work normally.
>
> 2) If you need to use Samba 4.6.5, try starting with a minimal smb.conf
> with logging turned up. Then review your samba logs. Note that setting log
> level to "10' will probably be more verbose than you want. Choose an
> appropriate level. Here's one from on of my testing machines. :
>
> [global]
>     guest account = awalker
>     map to guest = Bad User
>     log level = 10
>
> [Donkey Vol]
>     path = "/mnt/Donkey/Vol1"
>     writeable = yes
>     vfs objects = zfs_space
>     guest ok = yes
>     guest only = yes
>
> 3) The last firmware update looks like it was from 2014. You're probably
> vulnerable to a lot more than just that single Samba CVE. If this is in a
> business environment, perhaps look into migrating to a new appliance /
> server that's not EOL.
>
> If it's a home environment, and you like to tinker with things look for
> guides on installing stock Debian on the Thecus (it looks the Thecus has
> x86 hardware and an IDE DOM), and then adding Louis's Samba repo /
> installing the package. I did this with an old WD MyCloud about a year or
> so ago, and was much happier with the system afterwards. It's hackish, but
> can be a fun side-project.
>
> Andrew