The speed of copy is very very slow when much files in the folder

[Richard Sharpe]

On Fri, Apr 28, 2017 at 7:03 PM, Nico Kadel-Garcia via samba-technical
<samba-technical at lists.samba.org> wrote:
> On Fri, Apr 28, 2017 at 8:36 AM, Zhangxiaoxu via samba-technical
> <samba-technical at lists.samba.org> wrote:
>> Hi,
>>
>> 100,000 files in my share.
>> When I copy some little files to the folder, the speed is very very slow.
>
> In one directory, with no subdirectories? Maybe you should stop doing
> this? No matter how clever the network file system, getting the
> directory information sorted out and even sorting out permissions is
> complicated by so many files in one directory. It can be workable,
> especially if you're not trying to sort out a list of the files, but
> it's not free.
>
>> I add some logs in the smbd, the “get_real_filename_full_scan” function cast about 1s. It’s too slow.
>> I think the function is used for comparing the dest name to the exist files to check whether the dest file is exist.
>>
>> When I add “case sensitive = yes” to the configuration file, the speed is much quickly.
>
> Well, yes. Finding all the fiile names and sorting them is
> computationally expensive, especially to verify whether a mixed case
> file in that long list of files matches yours and there is a conflict.
>
>> So, I think whether we has some method to improve the performance?
>> Could you give us some suggestion about the case.
>
> You just said one. "case sensitive - yes". The other is "don't put
> that many files in one directory without splitting it out into
> subdirectories".

The problem with "case sensitive = yes" is that it is only for file
systems that are case insensitive. Yes, that sounds backwards, but
that is what it is for.

While it will solve the copying problem if you don't actually have a
case-insensitive files system, it introduces other big issues. One is
that it will actually let you create two files with the same name that
differ only by case, eg, File1.txt and file1.txt. Then you have a
problem, because which one you get will depend on the exact case used
when asking for the file. These (and worse) are the dragons I
mentioned.

To use "case sensitive = yes" you need a file system that supports
case-insensitive lookups, like XFS, ZFS (on Linux) and GPFS, it seems.

>> Environment:
>>        Ubuntu 14.04
>>        Samba 4.3.11
>>
>> Client:
>>        Windows 7
>>
>> Configuration:
>>        clustering = yes
>>        ctdbd socket = /var/run/ctdb/ctdbd.socket
>>    max protocol = SMB3
>>    large readwrite = yes
>>    idmap config *:range = 1000000-1999999
>>    log level = 2
>>    use sendfile = yes
>>    store dos attributes = yes
>>    acl_xattr:ignore system acls = yes
>>    aio read size = 1024
>>    deadtime = 10
>>    aio write behind = true
>>
>> Could you add my bugzilla account to the samba domain.
>> Email: zhang.xiaoxu at h3c.com<mailto:zhang.xiaoxu at h3c.com>
>>
>> Br,
>> Zhang Xiaoxu.
>>
>> -------------------------------------------------------------------------------------------------------------------------------------
>> 本邮件及其附件含有杭州华三通信技术有限公司的保密信息，仅限于发送给上面地址中列出
>> 的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、
>> 或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本
>> 邮件！
>> This e-mail and its attachments contain confidential information from H3C, which is
>> intended only for the person or entity whose address is listed above. Any use of the
>> information contained herein in any way (including, but not limited to, total or partial
>> disclosure, reproduction, or dissemination) by persons other than the intended
>> recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender
>> by phone or email immediately and delete it!
>



-- 
Regards,
Richard Sharpe
(何以解憂？唯有杜康。--曹操)