Fwd: Help Needed: Samba share slow to query not existing files

[Jônatas Hudler]

Thanks everyone for all the suggestions and sorry for taking so long to
feedback.

Jeremy's diagnosis was spot on - we applied the automatic case conversion
feature, following the parameters suggested (thanks!), dropped the
directory lookup lag drastically.
>From my measure, performance is now more than an order of magnitude better
(for a given operation, dropped 30s to 2s).

PS: There are some other questions that were raised back to me (Samba
version, why the others settings applied, ...) that are managed by my
client's IT support.
Since the issue is now solved, I'm not gonna bother you further with the
details, but I'll suggest my client to review/upgrade versions when
possible.

Thank you all again for the prompt support!

Jonatas Hudler


On Wed, 20 Jan 2021 at 15:34, Andrew Walker <awalker at ixsystems.com> wrote:

>
>
> On Wed, Jan 20, 2021 at 1:21 PM Jeremy Allison via samba-technical <
> samba-technical at lists.samba.org> wrote:
>
>> On Wed, Jan 20, 2021 at 02:37:45PM -0300, Jônatas Hudler via
>> samba-technical wrote:
>> >Hello everyone,
>> >
>> >*(First mail in this mailing list, so take it easy if I'm infringing any
>> >good manners here)*
>> >
>> >In a samba share we have a directory with lots of files - binaries and
>> >resources to run our ERP application (around 20K entries).
>> >
>> >The application is taking too long to load. The found reason is that the
>> >application tries to load several files in a trial/error fashion, looking
>> >for extensions in a particular order (e.g: file.dll, file.lbr, file.gnt,
>> >file.int). Many of these files are not required and doesn't even exist.
>> >
>> >Every "not found" hit take ~0.2 seconds. And there are lots of these
>> events.
>> >
>> >The thing is that apparently samba is not caching this result. My guess
>> is
>> >that every new not existing file open request causes a full directory's
>> >entries lookup.
>> >
>> >If I run this code (in a Windows Command Prompt) in this samba share, it
>> >takes considerable time, whereas in a native Windows share it is
>> >instantaneous.
>> >
>> >  for /L %i in (1,1,50) do @dir \\server\share\not_existing.txt >nul
>> >
>> >Anything that can be done?
>>
>> You are running into the classic "large directory performance"
>> issue, caused by Samba having to run on top of a POSIX case-sensitive
>> filesystem.
>>
>> Here's the brute force fix:
>>
>>
>> https://wiki.samba.org/index.php/Performance_Tuning#Directories_with_a_Large_Number_of_Files
>>
>> NOTE!!! from the text:
>>
>> "All files on the share must be converted to lowercase when using the
>> example.
>> Files using uppercase or both uppercase and lowercase are no longer
>> listed on the share."
>>
>> Or switch to a filesystem the supports case insensitive
>> lookups (I think xfs or zfs can do this).'
>>
>
> One quick note about the zfs side of this. "casesensitivity" is an
> immutable dataset property, and so it _must_ be set when the zfs dataset is
> first created.
>